Lung tumor infiltration image recognition method and system based on deep learning

Abstract

The invention discloses a lung tumor infiltration image recognition method and system based on deep learning, and relates to the technical field of lung tumor infiltration recognition, and the method comprises the following steps: obtaining a medical image of a lung tumor, and collecting a well-labeled sample image; performing region segmentation on the lung tumor based on the labeling result to obtain a tumor region; extracting edge features, morphological features and density features of the tumor region; based on the edge features, the morphological features and the density features of the sample images, deep learning is carried out on the wettability identification of the lung tumor; the method is used for solving the problems that an existing lung tumor wettability recognition technology is too dependent on manual recognition and the specificity of extracted tumor features is insufficient, so that the lung tumor wettability recognition result is not accurate.

Description

Technical Field

[0001] The present invention relates to the technical field of lung tumor infiltration recognition, and specifically to a lung tumor infiltration image recognition method and system based on deep learning. Background Art

[0002] Lung tumor infiltration identification technology refers to a method that uses computer vision and medical image analysis to help doctors and researchers identify and evaluate the degree of infiltration of lung tumors. The main goal of this technology is to accurately determine the scope and degree of infiltration of lung tumors in lung tissue through automated or semi-automated methods.

[0003] The infiltration of lung tumors is an important indicator for assessing their malignancy, formulating treatment plans and predicting patient prognosis. Traditional image-based infiltration assessment relies on the experience of radiologists, and has problems such as strong subjectivity, low efficiency and high misdiagnosis rate. The existing lung tumor infiltration recognition technology is not intelligent enough for identifying the infiltration of lung tumors, and the specificity of the extracted tumor features is insufficient, which is prone to misjudgment when classifying the infiltration. For example, in the patent application with publication number CN110533667A, a 3D segmentation method of lung tumor CT images based on image pyramid fusion is disclosed. The scheme is used to segment images within different scale ranges, and finally the segmentation results of different scale ranges are fused. However, the feature fusion of the tumor features themselves is insufficient, and the accuracy of the recognition area is insufficient. The existing lung tumor infiltration recognition technology also has the problem of over-reliance on manual recognition and insufficient specificity of the extracted tumor features, resulting in inaccurate recognition results of the infiltration of lung tumors. Summary of the invention

[0004] The present invention aims to solve one of the technical problems in the prior art to at least a certain extent, by acquiring medical images of lung tumors and collecting well-labeled sample images, then performing regional segmentation of the lung tumor based on the labeling results to obtain the tumor region, then extracting edge features based on the tumor region of the sample image, and at the same time extracting morphological features based on the tumor region of the sample image, and then extracting density features based on the tumor region of the sample image, and finally performing deep learning on the infiltrative identification of the lung tumor based on the edge features, morphological features and density features of the sample image, so as to solve the problem that the existing lung tumor infiltrative identification technology still relies too much on manual identification and the extracted tumor features are not specific enough, resulting in inaccurate results of infiltrative identification of the lung tumor.

[0005] To achieve the above objectives, in a first aspect, the present application provides a method for recognizing lung tumor infiltrative images based on deep learning, comprising the following steps: Obtain medical images of lung tumors and collect well-labeled sample images; Perform regional segmentation on the lung tumor based on the annotation results to obtain the tumor area; Extract edge features, morphological features, and density features of the tumor area; Based on the edge features, morphological features, and density features of sample images, deep learning is performed on the invasiveness identification of lung tumors.

[0006] Furthermore, obtaining medical images of lung tumors and collecting well-labeled sample images includes the following sub-steps: The medical image is a CT image; A first number of medical images are obtained, and a doctor performs regional annotation on lung tumors in the medical images and performs infiltration annotation on the lung tumors to obtain sample images; The region annotation in the sample image is to circle the region where a lung tumor is located, and the invasiveness annotation includes non-invasiveness, micro-invasiveness and invasiveness.

[0007] Furthermore, based on the labeling results, the lung tumor is segmented to obtain the tumor region, including the following sub-steps: Mark the area circled by the area annotation in the sample image as the roughly segmented area; Perform edge extraction on the roughly segmented area, extract the contour of the lung tumor, and obtain the tumor contour; The area enclosed by the tumor outline is segmented and marked as the tumor area.

[0008] Furthermore, extracting edge features, morphological features, and density features of the tumor region includes the following sub-steps: Extract edge features of tumor regions based on sample images; Extract morphological features of tumor regions based on sample images; Density features are extracted from the tumor area based on the sample image.

[0009] Furthermore, extracting edge features of the tumor region based on the sample image includes the following sub-steps: Marking the non-tumor area in the sample image as a normal area; Marking the pixel points on the tumor contour as contour pixel points, and marking the grayscale values ​​of the contour pixel points as contour grayscale values; The pixels in the normal area are marked as normal pixels, and the grayscale values ​​of the normal pixels are marked as normal grayscale values; Obtain normal pixel points adjacent to the contour pixel points, mark them as adjacent pixel points, and mark the normal grayscale values ​​of the adjacent pixel points as adjacent grayscale values; For any contour pixel point, calculate the difference between the contour grayscale value and the corresponding adjacent grayscale value, mark it as the edge grayscale difference, calculate the edge grayscale difference of each contour pixel point, count the number of identical edge grayscale differences, mark it as the number of grayscale differences; A histogram is constructed with the edge grayscale difference as the X-axis and the number of grayscale differences as the Y-axis, marked as edge features, and the edge grayscale difference and the corresponding number of grayscale differences are entered into the edge features.

[0010] Furthermore, extracting morphological features of the tumor region based on the sample image includes the following sub-steps: Perform rectangular selection on the tumor area to obtain the tumor rectangular area; Mark the pixels in the tumor area of ​​the tumor rectangular area as tumor pixels; The rectangular area of ​​the tumor is the morphological feature.

[0011] Furthermore, extracting density features based on the tumor region of the sample image includes the following sub-steps: The gray value of the tumor pixel is marked as the tumor gray value, and the number of the same tumor gray value is marked as the tumor gray density; The gray density of tumors is sorted and numbered in ascending order, and the symbol H is used to represent the gray density of tumors. n represents, where n is a positive integer and n is the sequence number of H; A rectangular coordinate system is established with n as the horizontal axis and the tumor grayscale density as the vertical axis, named density feature map. The tumor grayscale density is calculated according to the corresponding H n Enter the density feature map; Perform linear regression on the density feature map and mark the slope of the regression function as the density feature.

[0012] Furthermore, based on the edge features, morphological features, and density features of the sample images, deep learning for lung tumor infiltration recognition includes the following sub-steps: Based on the infiltration labeling of the sample images, the sample images are divided into different infiltration groups, wherein the infiltration groups include a non-infiltration group, a micro-infiltration group and a strong infiltration group, and the strong infiltration group is a collection of sample images whose infiltration labeling is infiltration; For any infiltration group, the sample images are numbered and represented by the symbol P m Indicates, where m is a positive integer and m is the serial number of P, set natural numbers i and j, i is any number in m, and the value range of j is the same as that of m but j≠i; For any i, get P i With P j The edge features of P i The rectangular column in the edge feature of is marked as the first rectangular column, and Pj The bar in the edge feature of is marked as the second bar, and P i With P j The edge features of are merged to obtain the area of ​​the first rectangular column, the area of ​​the second rectangular column and the area of ​​the overlapping part of the first rectangular column and the second rectangular column, which are marked as S1, S2 and S3 respectively; Calculate (S3 / S1+S3 / S2) / 2, mark the result as edge similarity, and i With each P j Analyze and calculate max(m)-1 edge similarities, find the maximum value among them, and mark it as the maximum edge similarity, where max() is the maximum value operator. Analyze each value of i to obtain max(m) maximum edge similarities, find the minimum value among them, and mark it as the edge similarity limit; For any i, get P i With P j The morphological features with smaller resolution are enlarged until the resolution is the same as that of the morphological features with larger resolution. i With P j The tumor pixels in the morphological features are marked as the first pixel and the second pixel respectively, and the number of the first pixel, the second pixel, and the number of the pixel points where the first pixel and the second pixel overlap are counted, and marked as N1, N2, and N3 respectively; Calculate (N3 / N1+N3 / N2) / 2, mark the result as morphological similarity, and set P i With each P j Perform analysis and calculate max(m)-1 morphological similarities, find the maximum value among them, mark it as the maximum morphological similarity, analyze each value of i, obtain max(m) maximum morphological similarities, find the minimum value among them, mark it as the morphological similarity limit; For any i, get P i With P j The density features are marked as K1 and K2 respectively. If K1 ≥ K2, K2 / K1 is calculated. If K1 < K2, K1 / K2 is calculated. The calculation result is marked as density similarity. i With each P j Perform analysis and calculate max(m)-1 density similarities, find the maximum value among them, mark it as the maximum density similarity, analyze each value of i, obtain max(m) maximum density similarities, find the minimum value among them, mark it as the density similarity limit; Based on edge similarity limits, morphology similarity limits, and density similarity limits, deep learning is performed for the invasiveness recognition of lung tumors.

[0013] Furthermore, based on the edge similarity limit, the morphology similarity limit, and the density similarity limit, deep learning for lung tumor infiltration recognition includes the following sub-steps: The deep learning is to perform deep learning on the analysis process of edge similarity limit, morphology similarity limit and density similarity limit, so as to continuously update the sample image and update the edge similarity limit, morphology similarity limit and density similarity limit; The edge similarity limits include non-infiltration edge similarity limits, micro-infiltration edge similarity limits and strong infiltration edge similarity limits, which are represented by symbols EA, EB and EC respectively; the morphology similarity limits include non-infiltration morphology similarity limits, micro-infiltration morphology similarity limits and strong infiltration morphology similarity limits, which are represented by FA, ​​FB and FC respectively; the density similarity limits include non-infiltration density similarity limits, micro-infiltration density similarity limits and strong infiltration density similarity limits, which are represented by DA, DB and DC respectively; Obtain the patient's medical image, mark it as the image to be identified, extract the maximum edge similarity, maximum morphology similarity and maximum density similarity of the image to be identified in the non-infiltration group, and mark them as GAE, GAF and GAD respectively; extract the maximum edge similarity, maximum morphology similarity and maximum density similarity of the image to be identified in the micro-infiltration group, and mark them as GBE, GBF and GBD respectively; extract the maximum edge similarity, maximum morphology similarity and maximum density similarity of the image to be identified in the strong infiltration group, and mark them as GCE, GCF and GCD respectively; Calculate |GAE-EA|, |GAF-FA|, and |GAD-DA|, and mark the results as A1, A2, and A3, respectively; calculate |GBE-EB|, |GBF-FB|, and |GBD-DB|, and mark the results as B1, B2, and B3, respectively; calculate |GCE-EC|, |GCF-FC|, and |GCD-DC|, and mark the results as C1, C2, and C3, respectively; Calculate A1+A2+A3=QA, calculate B1+B2+B3=QB, calculate C1+C2+C3=QC, where QA is the non-invasion deviation, QB is the micro-invasion deviation, and QC is the strong invading deviation; Compare the sizes of QA, QB and QC, find the minimum value, mark it as the minimum deviation, and output the infiltrative label corresponding to the minimum deviation as the infiltrative label of the image to be identified.

[0014] In a second aspect, the present application provides a lung tumor infiltrative image recognition system based on deep learning, comprising an image collection module, a region segmentation module, a feature extraction module and a deep learning module; the image collection module, the region segmentation module and the deep learning module are respectively connected to the feature extraction module data; The image collection module is used to obtain medical images of lung tumors and collect well-labeled sample images; The region segmentation module is used to perform region segmentation on the lung tumor based on the labeling result to obtain the tumor region; The feature extraction module is used to extract edge features, morphological features and density features of the tumor area; The deep learning module is used to perform deep learning on the infiltrativeness identification of lung tumors based on the edge features, morphological features and density features of sample images.

[0015] Beneficial effects of the present invention: The present invention obtains medical images of lung tumors and collects well-labeled sample images, then performs regional segmentation of the lung tumor based on the labeling results to obtain the tumor region, then extracts edge features based on the tumor region of the sample image, and extracts morphological features based on the tumor region of the sample image, and then extracts density features based on the tumor region of the sample image. The advantage is that the tumor region is usually different from the normal region. The feature extraction of the tumor region through the three features of edge, morphology and density ensures sufficient specificity, thereby improving the accuracy and effectiveness of the identification of the infiltrativeness of the lung tumor. The present invention performs deep learning on the infiltrative identification of lung tumors based on the edge features, morphological features and density features of sample images, calculates the edge similarity limits, morphological similarity limits and density similarity limits of the edge features, morphological features and density features, and then matches the image to be identified with different infiltration groups to obtain the final infiltrative identification result. The advantage is that the matching of a single feature is not sufficient to classify and identify the infiltrativeness. Matching and analyzing the features in multiple dimensions can avoid the phenomenon of misjudgment when classifying the infiltrativeness. At the same time, deep learning is used to enable intelligent identification, eliminate the subjectivity of manual identification, and improve the efficiency and accuracy of lung tumor infiltrative identification. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 is a functional block diagram of the system of the present invention; Figure 2 is a schematic diagram of a sample image of the present invention; Figure 3 It is a schematic diagram of roughly segmenting the regions of the present invention; Figure 4 is a schematic diagram of a tumor region of the present invention; Figure 5 A schematic diagram of the present invention in which the tumor outline is rendered in gray; Figure 6 is the edge feature of the present invention; Figure 7 It is a schematic diagram of a rectangular tumor region of the present invention; Figure 8 is a density characteristic diagram of the present invention; Fig. 9 is the edge feature of P2 of the present invention; Fig.10 is a histogram obtained by merging the edge features of P1 and P2 of the present invention; Fig.11 Schematic diagram of the morphological features of P1 and P2 of the present invention; Fig.12 The figure is a flow chart of the steps of the method of the present invention. DETAILED DESCRIPTION

[0017] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0018] Example 1, please refer to Figure 1 As shown, the present application provides a lung tumor infiltrative image recognition system based on deep learning, including an image collection module, a region segmentation module, a feature extraction module and a deep learning module; the image collection module, the region segmentation module and the deep learning module are respectively connected to the feature extraction module data; The image collection module is used to obtain medical images of lung tumors and collect well-labeled sample images; The image collection module is configured with an image collection strategy, which includes: Medical images are CT images; See also Figure 2 As shown, a first number of medical images is obtained, where the first number is a specific value in the process of selecting medical images. In a specific implementation, the first number is set to 50 copies, and an experienced radiologist uses a segmentation tool to mark the region of the lung tumor in the medical image and marks the infiltration of the lung tumor to obtain a sample image; The region annotation in the sample image is to circle the area where a lung tumor is located, and the invasiveness annotation includes non-invasive, micro-invasive and invasive; In practical applications, the area marking is as follows Figure 2As shown in Figure 1, wettability annotation is to mark a label for the sample image. Figure 2 The sample image shown is microinvasive.

[0019] The regional segmentation module is used to segment the lung tumor based on the annotation results to obtain the tumor area; The region segmentation module is configured with a region segmentation strategy, which includes: See also Figure 3 to Figure 4 As shown, the area circled by the area annotation in the sample image is marked as a roughly segmented area; Perform edge extraction on the roughly segmented area, extract the contour of the lung tumor, and obtain the tumor contour; The area enclosed by the tumor outline is segmented and marked as the tumor area; In practical applications, roughly segmented regions such as Figure 3 As shown, the edge of the roughly segmented area is extracted to obtain the tumor area as shown in Figure 4 As shown, Figure 4 The white part in the image is the tumor area, and the boundary between the white and black parts is the tumor outline.

[0020] The feature extraction module is used to extract edge features, morphological features and density features of the tumor area; the feature extraction module includes an edge feature extraction unit, a morphological feature extraction unit and a density feature extraction unit; The edge feature extraction unit is used to extract edge features based on the tumor area of ​​the sample image; The edge feature extraction unit is configured with an edge feature extraction strategy, which includes: Marking the non-tumor area in the sample image as a normal area; Marking the pixel points on the tumor contour as contour pixel points, and marking the grayscale values ​​of the contour pixel points as contour grayscale values; The pixels in the normal area are marked as normal pixels, and the grayscale values ​​of the normal pixels are marked as normal grayscale values; See also Figure 5 As shown, normal pixel points adjacent to the contour pixel points are obtained, marked as adjacent pixel points, and the normal grayscale values ​​of the adjacent pixel points are marked as adjacent grayscale values; For any contour pixel point, calculate the difference between the contour grayscale value and the corresponding adjacent grayscale value, mark it as the edge grayscale difference, calculate the edge grayscale difference of each contour pixel point, count the number of identical edge grayscale differences, mark it as the number of grayscale differences; See also Figure 6 As shown, a histogram is constructed with the edge grayscale difference as the X-axis and the grayscale difference quantity as the Y-axis, marked as edge features, and the edge grayscale difference and the corresponding grayscale difference quantity are entered into the edge features; In practical applications, Figure 4 The black area in the figure is the normal area, and the white area is the tumor area. Figure 5 The tumor outline is rendered in gray for easy distinction. Figure 5 Taking the contour pixel in the upper left corner as an example, the adjacent pixels to the contour pixel are three dark gray normal pixels. Their edge grayscale differences are calculated respectively, and the edge grayscale differences are 12, 9, and 17. The edge features are constructed as follows Figure 6 As shown; The morphological feature extraction unit is used to extract morphological features of the tumor region based on the sample image; The morphological feature extraction unit is configured with a morphological feature extraction strategy, which includes: See also Figure 7 As shown, a rectangular frame is selected for the tumor area to obtain a tumor rectangular area; Mark the pixels in the tumor area of ​​the tumor rectangular area as tumor pixels; The rectangular area of ​​the tumor is the morphological feature; In practical applications, the tumor rectangular area is selected as follows Figure 7 As shown; The density feature extraction unit is used to extract density features based on the tumor area of ​​the sample image; The density feature extraction unit is configured with a density feature extraction strategy, which includes: The gray value of the tumor pixel is marked as the tumor gray value, and the number of the same tumor gray value is marked as the tumor gray density; The gray density of tumors is sorted and numbered in ascending order, and the symbol H is used to represent the gray density of tumors. n It represents, where n is a positive integer and n is the serial number of H; See also Figure 8 As shown in the figure, a rectangular coordinate system is established with n as the horizontal axis and the tumor gray density as the vertical axis, named density feature map, and the tumor gray density is calculated according to the corresponding H n Enter the density feature map; Perform linear regression on the density feature map and mark the slope of the regression function as the density feature; In practical applications, the size of the tumor grayscale value is not important. The density feature is only characterized by the change trend of the density of the tumor grayscale value. Therefore, this embodiment does not use H n The corresponding tumor grayscale value is specifically displayed, and the density feature map is constructed as follows Figure 8 As shown, the density feature obtained by linear regression analysis is 1.0345.

[0021] The deep learning module is used to perform deep learning on the infiltration identification of lung tumors based on the edge features, morphological features and density features of sample images; the deep learning module includes a similarity limit analysis unit and an infiltration identification unit; The similarity limit analysis unit is configured with a similarity limit analysis strategy, which includes: Based on the infiltration annotation of the sample images, the sample images are divided into different infiltration groups, including a non-infiltration group, a micro-infiltration group and a strong infiltration group. The strong infiltration group is a collection of sample images with infiltration annotation as infiltration; For any infiltration group, the sample images are numbered and represented by the symbol P m Indicates, where m is a positive integer and m is the serial number of P, set natural numbers i and j, i is any number in m, and the value range of j is the same as that of m but j≠i; See also Figures 9 and 10 As shown, for any i, get P i With P j The edge features of P i The rectangular column in the edge feature of is marked as the first rectangular column, and P j The bar in the edge feature of is marked as the second bar, and P i With P j The edge features of are merged to obtain the area of ​​the first rectangular column, the area of ​​the second rectangular column and the area of ​​the overlapping part of the first rectangular column and the second rectangular column, which are marked as S1, S2 and S3 respectively; Calculate (S3 / S1+S3 / S2) / 2, mark the result as edge similarity, and i With each P j Analyze and calculate max(m)-1 edge similarities, find the maximum value among them, and mark it as the maximum edge similarity, where max() is the maximum value operator. Analyze each value of i to obtain max(m) maximum edge similarities, find the minimum value among them, and mark it as the edge similarity limit; In practical applications, this embodiment takes the analysis and calculation process of the edge similarity limit, morphology similarity limit, and density similarity limit of the micro-invasion group as an example to illustrate the recognition of lung tumor infiltration; for the micro-invasion group, taking i=1, j=2 as an example, the edge features of P1 and P2 are obtained, where: Figure 6 is the edge feature of P1, Fig. 9 is the edge feature of P2, Fig.10 The histogram obtained by merging the edge features of P1 and P2. S1, S2 and S3 are 145, 124 and 116 respectively. The edge similarity is calculated to be 0.87. The calculation result is rounded to two decimal places. P1 is combined with all P jAnalyze and get max(m)-1 edge similarities. The maximum edge similarity of P1 is 0.89. m There is a maximum edge similarity, and the edge similarity limit is 0.82. See also Fig.11 As shown, for any i, get P i With P j The morphological features with smaller resolution are enlarged until the resolution is the same as that of the morphological features with larger resolution. i With P j The tumor pixels in the morphological features are marked as the first pixel and the second pixel respectively, and the number of the first pixel, the second pixel, and the number of the pixel points where the first pixel and the second pixel overlap are counted, and marked as N1, N2, and N3 respectively; Calculate (N3 / N1+N3 / N2) / 2, mark the result as morphological similarity, and set P i With each P j Perform analysis and calculate max(m)-1 morphological similarities, find the maximum value among them, mark it as the maximum morphological similarity, analyze each value of i, obtain max(m) maximum morphological similarities, find the minimum value among them, mark it as the morphological similarity limit; In practical applications, the morphological characteristics of P1 and P2 are as follows: Fig.11 As shown in the figure, the morphological feature of P1 is the image on the left with a resolution of 26×25, while the morphological feature of P2 is the image on the right with a resolution of 24×23. The morphological feature of P2 is smaller, so its resolution is changed to 26×25, and then they are overlapped. By statistics, N1, N2 and N3 are 534, 516 and 474 respectively. By calculation, the morphological similarity between P1 and P2 is 0.90. The calculation result is rounded to two decimal places. P1 is compared with each P j Analyze and find the maximum value, and get the maximum morphological similarity of 0.91. m Each of them has a maximum morphological similarity, and the morphological similarity limit is 0.81; For any i, get P i With P j The density features are marked as K1 and K2 respectively. If K1 ≥ K2, K2 / K1 is calculated. If K1 < K2, K1 / K2 is calculated. The calculation result is marked as density similarity. i With each P jPerform analysis and calculate max(m)-1 density similarities, find the maximum value among them, mark it as the maximum density similarity, analyze each value of i, obtain max(m) maximum density similarities, find the minimum value among them, mark it as the density similarity limit; In actual application, the density features K1 and K2 of P1 and P2 are 1.0345 and 1.1798 respectively. K1 is less than K2. Then K1 / K2 is calculated, and the density similarity is 0.88. The calculation result is rounded to two decimal places. P1 is compared with each P j Analyze and get max(m)-1 density similarities. Find and find that the maximum density similarity of P1 is 0.95. m There is a maximum density similarity for each of them, and the density similarity limit is 0.78; The infiltration identification unit is used to perform deep learning on the infiltration identification of lung tumors based on edge similarity limits, morphology similarity limits, and density similarity limits; The wettability identification unit is configured with an wettability identification strategy, which includes: Deep learning is to perform deep learning on the analysis process of edge similarity limit, morphology similarity limit and density similarity limit, so as to continuously update sample images and update edge similarity limit, morphology similarity limit and density similarity limit; The edge similarity limits include the non-infiltrating edge similarity limits, the micro-infiltrating edge similarity limits and the strong infiltrating edge similarity limits, which are represented by the symbols EA, EB and EC respectively; the morphology similarity limits include the non-infiltrating morphology similarity limits, the micro-infiltrating morphology similarity limits and the strong infiltrating morphology similarity limits, which are represented by FA, ​​FB and FC respectively; the density similarity limits include the non-infiltrating density similarity limits, the micro-infiltrating density similarity limits and the strong infiltrating density similarity limits, which are represented by DA, DB and DC respectively; Obtain the patient's medical image, mark it as the image to be identified, extract the maximum edge similarity, maximum morphology similarity and maximum density similarity of the image to be identified in the non-infiltration group, and mark them as GAE, GAF and GAD respectively; extract the maximum edge similarity, maximum morphology similarity and maximum density similarity of the image to be identified in the micro-infiltration group, and mark them as GBE, GBF and GBD respectively; extract the maximum edge similarity, maximum morphology similarity and maximum density similarity of the image to be identified in the strong infiltration group, and mark them as GCE, GCF and GCD respectively; Calculate |GAE-EA|, |GAF-FA|, and |GAD-DA|, and mark the results as A1, A2, and A3, respectively; calculate |GBE-EB|, |GBF-FB|, and |GBD-DB|, and mark the results as B1, B2, and B3, respectively; calculate |GCE-EC|, |GCF-FC|, and |GCD-DC|, and mark the results as C1, C2, and C3, respectively; Calculate A1+A2+A3=QA, calculate B1+B2+B3=QB, calculate C1+C2+C3=QC, where QA is the non-invasion deviation, QB is the micro-invasion deviation, and QC is the strong invading deviation; Compare the sizes of QA, QB and QC, find the minimum value among them, mark it as the minimum deviation, and output the infiltrative label corresponding to the minimum deviation as the infiltrative label of the image to be identified; In practical applications, the identification of the infiltrative nature of lung tumors has been described in detail in the infiltrative identification strategy, and the relationship between different data and the calculation process have been given. In this embodiment, no further explanation is given, only the final values ​​and results are given, and deep learning is further explained; through the calculation in the infiltrative identification strategy, QA, QB and QC are obtained to be 0.18, 0.33 and 0.25 respectively, and the minimum deviation is 0.18 through comparison, corresponding to the non-invasive group, that is, the patient's medical image is marked as non-invasive. At the same time, the patient's medical image is entered into the non-invasive group, and the edge similarity limit, morphological similarity limit and density similarity limit of the non-invasive group are re-analyzed and updated through the deep learning model.

[0022] Example 2, please refer to Fig.12 As shown, the present application provides a method for recognizing lung tumor infiltration images based on deep learning, comprising the following steps: Step S1, obtaining a medical image of a lung tumor and collecting well-labeled sample images; Step S1 includes the following sub-steps: Step S101, the medical image is a CT image; Step S102, obtaining a first number of medical images, and having an experienced radiologist use a segmentation tool to perform regional annotation on the lung tumors in the medical images, and also perform infiltration annotation on the lung tumors, to obtain sample images; Step S103, the region annotation in the sample image is to circle the region where a lung tumor is located, and the invasiveness annotation includes non-invasive, micro-invasive and invasive; Step S2, performing regional segmentation on the lung tumor based on the labeling result to obtain the tumor region; Step S2 includes the following sub-steps: Step S201, marking the area circled by the area annotation in the sample image as a roughly segmented area; Step S202, performing edge extraction on the roughly segmented area to extract the contour of the lung tumor and obtain the tumor contour; Step S203, segmenting the area enclosed by the tumor outline and marking it as the tumor area; Step S3, extracting edge features, morphological features and density features of the tumor area; Step S3 includes the following sub-steps: Step S301, extracting edge features based on the tumor region of the sample image; Step S301 includes the following sub-steps: Step S301.1, marking the non-tumor area in the sample image as a normal area; Step S301.2, marking the pixel points on the tumor contour as contour pixel points, and marking the grayscale values ​​of the contour pixel points as contour grayscale values; Step S301.3, marking the pixels in the normal area as normal pixels, and marking the grayscale values ​​of the normal pixels as normal grayscale values; Step S301.4, obtaining normal pixel points adjacent to the contour pixel points, marking them as adjacent pixel points, and marking the normal grayscale values ​​of the adjacent pixel points as adjacent grayscale values; Step S301.5, for any contour pixel, calculate the difference between the contour grayscale value and the corresponding adjacent grayscale value, mark it as the edge grayscale difference, calculate the edge grayscale difference of each contour pixel, count the number of identical edge grayscale differences, mark it as the number of grayscale differences; Step S301.6, constructing a histogram with the edge grayscale difference as the X-axis and the grayscale difference number as the Y-axis, marking it as an edge feature, and entering the edge grayscale difference and the corresponding grayscale difference number into the edge feature; Step S302, extracting morphological features based on the tumor region of the sample image; Step S302 includes the following sub-steps: Step S302.1, performing rectangular selection on the tumor area to obtain a tumor rectangular area; Step S302.2, marking the pixel points in the tumor area in the tumor rectangular area as tumor pixel points; Step S302.3, the rectangular area of ​​the tumor is the morphological feature; Step S303, extracting density features based on the tumor region of the sample image; Step S303 includes the following sub-steps: Step S303.1, marking the gray value of the tumor pixel as the tumor gray value, and marking the number of identical tumor gray values ​​as the tumor gray density; Step S303.2, sort and number the gray density of the tumors in ascending order, using the symbol H n represents, where n is a positive integer and n is the sequence number of H; Step S303.3, establish a plane rectangular coordinate system with n as the horizontal axis and tumor grayscale density as the vertical axis, named density feature map, and calculate the tumor grayscale density according to the corresponding H n Enter the density feature map; Step S303.4, performing linear regression on the density feature map, and marking the slope of the regression function as the density feature; Step S4, based on the edge features, morphological features and density features of the sample image, deep learning is performed on the infiltrativeness recognition of the lung tumor; Step S4 includes the following sub-steps: Step S401, based on the infiltration labeling of the sample image, the sample image is divided into different infiltration groups, the infiltration groups include a non-infiltration group, a slightly infiltration group and a strongly infiltration group, and the strongly infiltration group is a collection of sample images with infiltration labeling as infiltration; Step S402: for any infiltration group, number the sample images therein, using the symbol P m Indicates, where m is a positive integer and m is the serial number of P, set natural numbers i and j, i is any number in m, and the value range of j is the same as that of m but j≠i; Step S403, for any i, obtain P i With P j The edge features of P i The rectangular column in the edge feature of is marked as the first rectangular column, and P j The bar in the edge feature of is marked as the second bar, and P i With P j The edge features of are merged to obtain the area of ​​the first rectangular column, the area of ​​the second rectangular column and the area of ​​the overlapping part of the first rectangular column and the second rectangular column, which are marked as S1, S2 and S3 respectively; Step S404, calculate (S3 / S1+S3 / S2) / 2, mark the calculation result as edge similarity, and i With each P j Analyze and calculate max(m)-1 edge similarities, find the maximum value among them, and mark it as the maximum edge similarity, where max() is the maximum value operator. Analyze each value of i to obtain max(m) maximum edge similarities, find the minimum value among them, and mark it as the edge similarity limit; Step S405: for any i, obtain P i With P jThe morphological features with smaller resolution are enlarged until the resolution is the same as that of the morphological features with larger resolution. i With P j The tumor pixels in the morphological features are marked as the first pixel and the second pixel respectively, and the number of the first pixel, the second pixel, and the number of the pixel points where the first pixel and the second pixel overlap are counted, and marked as N1, N2, and N3 respectively; Step S406, calculate (N3 / N1+N3 / N2) / 2, mark the calculation result as morphological similarity, and i With each P j Perform analysis and calculate max(m)-1 morphological similarities, find the maximum value among them, mark it as the maximum morphological similarity, analyze each value of i, obtain max(m) maximum morphological similarities, find the minimum value among them, mark it as the morphological similarity limit; Step S407, for any i, obtain P i With P j The density features are marked as K1 and K2 respectively. If K1 ≥ K2, K2 / K1 is calculated. If K1 < K2, K1 / K2 is calculated. The calculation result is marked as density similarity. i With each P j Perform analysis and calculate max(m)-1 density similarities, find the maximum value among them, mark it as the maximum density similarity, analyze each value of i, obtain max(m) maximum density similarities, find the minimum value among them, mark it as the density similarity limit; Step S408, performing deep learning on the infiltrativeness recognition of the lung tumor based on the edge similarity limit, the morphology similarity limit, and the density similarity limit; Step S408 includes the following sub-steps: Step S408.1, deep learning is to perform deep learning on the analysis process of edge similarity limit, morphology similarity limit and density similarity limit, so as to continuously update the sample image and update the edge similarity limit, morphology similarity limit and density similarity limit; Step S408.2, the edge similarity limits include the non-infiltration edge similarity limits, the micro-infiltration edge similarity limits and the strong-infiltration edge similarity limits, which are represented by symbols EA, EB and EC respectively; the morphology similarity limits include the non-infiltration morphology similarity limits, the micro-infiltration morphology similarity limits and the strong-infiltration morphology similarity limits, which are represented by symbols FA, FB and FC respectively; the density similarity limits include the non-infiltration density similarity limits, the micro-infiltration density similarity limits and the strong-infiltration density similarity limits, which are represented by symbols DA, DB and DC respectively; Step S408.3, obtaining the medical image of the patient, marking it as the image to be identified, extracting the maximum edge similarity, maximum morphology similarity and maximum density similarity of the image to be identified in the non-infiltration group, marking them as GAE, GAF and GAD respectively; extracting the maximum edge similarity, maximum morphology similarity and maximum density similarity of the image to be identified in the micro-infiltration group, marking them as GBE, GBF and GBD respectively; extracting the maximum edge similarity, maximum morphology similarity and maximum density similarity of the image to be identified in the strong infiltration group, marking them as GCE, GCF and GCD respectively; Step S408.4, respectively calculate |GAE-EA|, |GAF-FA|, and |GAD-DA|, and mark the calculation results as A1, A2, and A3; respectively calculate |GBE-EB|, |GBF-FB|, and |GBD-DB|, and mark the calculation results as B1, B2, and B3; respectively calculate |GCE-EC|, |GCF-FC|, and |GCD-DC|, and mark the calculation results as C1, C2, and C3; Step S408.5, calculate A1+A2+A3=QA, calculate B1+B2+B3=QB, calculate C1+C2+C3=QC, where QA is the non-invasion deviation, QB is the micro-invasion deviation, and QC is the strong invading deviation; Step S408.6, compare the sizes of QA, QB and QC, find the minimum value among them, mark it as the minimum deviation, and use the infiltrative label corresponding to the minimum deviation as the infiltrative label of the image to be identified and output it.

[0023] Embodiment 3, the present application provides an electronic device, which may include: a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other through the communication bus. The memory stores computer-readable instructions, and the processor can call the instructions in the memory. When the computer-readable instructions are executed by the processor, the steps in the method for recognizing the infiltrative image of lung tumors based on deep learning are executed to achieve the following functions: obtain medical images of lung tumors, and collect well-labeled sample images at the same time; perform regional segmentation of lung tumors based on the labeling results to obtain tumor regions; extract edge features, morphological features and density features of the tumor region; perform deep learning on the infiltrative recognition of lung tumors based on the edge features, morphological features and density features of the sample images.

[0024] In addition, the logic instructions in the above-mentioned memory can be implemented in the form of software functional units and can be stored in a computer-readable storage medium when sold or used as an independent product. Based on this understanding, the technical solution of the present application can be essentially or partly embodied in the form of a software product that contributes to the prior art. The computer software product is stored in a storage medium, including several instructions to enable a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc. Various media that can store program codes.

[0025] Embodiment 4, the present application also provides a computer-readable storage medium, the present application provides a storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps in the above-mentioned deep learning-based lung tumor infiltrative image recognition method are executed to achieve the following functions: obtain medical images of lung tumors, and collect well-labeled sample images at the same time; perform regional segmentation of lung tumors based on the labeling results to obtain tumor areas; extract edge features, morphological features, and density features of the tumor area; and perform deep learning on the infiltrative recognition of lung tumors based on the edge features, morphological features, and density features of the sample images.

[0026] Through the description of the above implementation methods, the embodiments of the present invention can be provided as methods, systems or computer program products. Based on such an understanding, the above technical solutions can be essentially or partly contributed to the prior art in the form of software products, which can be stored in a computer-readable storage medium, such as ROM / RAM, a magnetic disk, an optical disk, etc., and include several instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0027] In the embodiments provided in the present application, it should be understood that the disclosed system or method can be implemented in other ways. The embodiments described above are merely schematic. For example, the division of modules or units is only a logical function division. There may be other division methods in actual implementation. For example, multiple modules or units can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some communication interfaces, and the indirect coupling or communication connection of systems, modules and units can be electrical, mechanical or other forms.

[0028] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit it. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A deep learning-based lung tumor infiltration image recognition method, characterized in that: The steps include: Obtain medical images of lung tumors and collect well-labeled sample images; Perform regional segmentation on the lung tumor based on the annotation results to obtain the tumor area; Extract edge features, morphological features, and density features of the tumor area; Based on the edge features, morphological features, and density features of sample images, deep learning is performed on the invasiveness identification of lung tumors.

2. The deep learning-based lung tumor infiltration image recognition method according to claim 1, characterized in that: Obtaining medical images of lung tumors and collecting well-labeled sample images includes the following sub-steps: The medical image is a CT image; A first number of medical images are obtained, and a doctor performs regional annotation on lung tumors in the medical images and performs infiltration annotation on the lung tumors to obtain sample images; The region annotation in the sample image is to circle the region where a lung tumor is located, and the invasiveness annotation includes non-invasiveness, micro-invasiveness and invasiveness.

3. The deep learning-based lung tumor infiltration image recognition method according to claim 2, characterized in that: Based on the annotation results, the lung tumor is segmented and the tumor region is obtained, which includes the following sub-steps: Mark the area circled by the area annotation in the sample image as the roughly segmented area; Perform edge extraction on the roughly segmented area, extract the contour of the lung tumor, and obtain the tumor contour; The area enclosed by the tumor outline is segmented and marked as the tumor area.

4. The deep learning-based lung tumor infiltration image recognition method according to claim 3, characterized in that: Extracting edge features, morphological features, and density features of the tumor area includes the following sub-steps: Extract edge features of tumor regions based on sample images; Extract morphological features of tumor regions based on sample images; Density features are extracted from the tumor area based on the sample image.

5. The deep learning-based lung tumor infiltration image recognition method according to claim 4, characterized in that: Extracting edge features of the tumor region based on the sample image includes the following sub-steps: Marking the non-tumor area in the sample image as a normal area; Marking the pixel points on the tumor contour as contour pixel points, and marking the grayscale values ​​of the contour pixel points as contour grayscale values; The pixels in the normal area are marked as normal pixels, and the grayscale values ​​of the normal pixels are marked as normal grayscale values; Obtain normal pixel points adjacent to the contour pixel points, mark them as adjacent pixel points, and mark the normal grayscale values ​​of the adjacent pixel points as adjacent grayscale values; For any contour pixel point, calculate the difference between the contour grayscale value and the corresponding adjacent grayscale value, mark it as the edge grayscale difference, calculate the edge grayscale difference of each contour pixel point, count the number of identical edge grayscale differences, mark it as the number of grayscale differences; A histogram is constructed with the edge grayscale difference as the X-axis and the number of grayscale differences as the Y-axis, marked as edge features, and the edge grayscale difference and the corresponding number of grayscale differences are entered into the edge features.

6. The deep learning-based lung tumor infiltration image recognition method according to claim 5, characterized in that: Extracting morphological features of tumor regions based on sample images includes the following sub-steps: Perform rectangular selection on the tumor area to obtain the tumor rectangular area; Mark the pixels in the tumor area of ​​the tumor rectangular area as tumor pixels; The rectangular area of ​​the tumor is the morphological feature.

7. The deep learning-based lung tumor infiltration image recognition method according to claim 6, characterized in that: Extracting density features of tumor regions based on sample images includes the following sub-steps: The gray value of the tumor pixel is marked as the tumor gray value, and the number of the same tumor gray value is marked as the tumor gray density; The gray density of tumors is sorted and numbered in ascending order, and the symbol H is used to represent the gray density of tumors. n It represents, where n is a positive integer and n is the serial number of H; A rectangular coordinate system is established with n as the horizontal axis and the tumor grayscale density as the vertical axis, named density feature map. The tumor grayscale density is calculated according to the corresponding H n Enter the density feature map; Perform linear regression on the density feature map and mark the slope of the regression function as the density feature.

8. The deep learning-based lung tumor infiltration image recognition method according to claim 7, characterized in that: Based on the edge features, morphological features, and density features of sample images, deep learning for lung tumor infiltration recognition includes the following sub-steps: Based on the infiltration labeling of the sample images, the sample images are divided into different infiltration groups, wherein the infiltration groups include a non-infiltration group, a micro-infiltration group and a strong infiltration group, and the strong infiltration group is a collection of sample images whose infiltration labeling is infiltration; For any infiltration group, the sample images are numbered and represented by the symbol P m Indicates, where m is a positive integer and m is the serial number of P, set natural numbers i and j, i is any number in m, and the value range of j is the same as that of m but j≠i; For any i, get P i With P j The edge features of P i The rectangular column in the edge feature of is marked as the first rectangular column, and P j The bar in the edge feature of is marked as the second bar, and P i With P j The edge features of are merged to obtain the area of ​​the first rectangular column, the area of ​​the second rectangular column and the area of ​​the overlapping part of the first rectangular column and the second rectangular column, which are marked as S1, S2 and S3 respectively; Calculate (S3 / S1+S3 / S2) / 2, mark the result as edge similarity, and i With each P j Analyze and calculate max(m)-1 edge similarities, find the maximum value among them, and mark it as the maximum edge similarity, where max() is the maximum value operator. Analyze each value of i to obtain max(m) maximum edge similarities, find the minimum value among them, and mark it as the edge similarity limit; For any i, get P i With P j The morphological features with smaller resolution are enlarged until the resolution is the same as that of the morphological features with larger resolution. i With P j The tumor pixels in the morphological features are marked as the first pixel and the second pixel respectively, and the number of the first pixel, the second pixel, and the number of the pixel points where the first pixel and the second pixel overlap are counted, and marked as N1, N2, and N3 respectively; Calculate (N3 / N1+N3 / N2) / 2, mark the result as morphological similarity, and set P i With each P j Perform analysis and calculate max(m)-1 morphological similarities, find the maximum value among them, mark it as the maximum morphological similarity, analyze each value of i, obtain max(m) maximum morphological similarities, find the minimum value among them, mark it as the morphological similarity limit; For any i, get P i With P j The density features are marked as K1 and K2 respectively. If K1 ≥ K2, K2 / K1 is calculated. If K1 < K2, K1 / K2 is calculated. The calculation result is marked as density similarity. i With each P j Perform analysis and calculate max(m)-1 density similarities, find the maximum value among them, mark it as the maximum density similarity, analyze each value of i, obtain max(m) maximum density similarities, find the minimum value among them, mark it as the density similarity limit; Based on edge similarity limits, morphology similarity limits, and density similarity limits, deep learning is performed for the invasiveness identification of lung tumors.

9. The deep learning-based lung tumor infiltration image recognition method according to claim 8, characterized in that: Based on edge similarity limits, morphology similarity limits, and density similarity limits, deep learning for lung tumor infiltration recognition includes the following sub-steps: The deep learning is to perform deep learning on the analysis process of edge similarity limit, morphology similarity limit and density similarity limit, so as to continuously update the sample image and update the edge similarity limit, morphology similarity limit and density similarity limit; The edge similarity limits include non-infiltration edge similarity limits, micro-infiltration edge similarity limits and strong infiltration edge similarity limits, which are represented by symbols EA, EB and EC respectively; the morphology similarity limits include non-infiltration morphology similarity limits, micro-infiltration morphology similarity limits and strong infiltration morphology similarity limits, which are represented by FA, ​​FB and FC respectively; the density similarity limits include non-infiltration density similarity limits, micro-infiltration density similarity limits and strong infiltration density similarity limits, which are represented by DA, DB and DC respectively; Obtain the patient's medical image, mark it as the image to be identified, extract the maximum edge similarity, maximum morphology similarity and maximum density similarity of the image to be identified in the non-infiltration group, and mark them as GAE, GAF and GAD respectively; extract the maximum edge similarity, maximum morphology similarity and maximum density similarity of the image to be identified in the micro-infiltration group, and mark them as GBE, GBF and GBD respectively; extract the maximum edge similarity, maximum morphology similarity and maximum density similarity of the image to be identified in the strong infiltration group, and mark them as GCE, GCF and GCD respectively; Calculate |GAE-EA|, |GAF-FA|, and |GAD-DA|, and mark the results as A1, A2, and A3, respectively; calculate |GBE-EB|, |GBF-FB|, and |GBD-DB|, and mark the results as B1, B2, and B3, respectively; calculate |GCE-EC|, |GCF-FC|, and |GCD-DC|, and mark the results as C1, C2, and C3, respectively; Calculate A1+A2+A3=QA, calculate B1+B2+B3=QB, calculate C1+C2+C3=QC, where QA is the non-invasion deviation, QB is the micro-invasion deviation, and QC is the strong invading deviation; Compare the sizes of QA, QB and QC, find the minimum value, mark it as the minimum deviation, and output the infiltrative label corresponding to the minimum deviation as the infiltrative label of the image to be identified.

10. A deep learning-based lung tumor infiltration image recognition system, used to implement the deep learning-based lung tumor infiltration image recognition method according to any one of claims 1 to 9, characterized in that: It includes an image collection module, a region segmentation module, a feature extraction module and a deep learning module; the image collection module, the region segmentation module and the deep learning module are respectively connected to the feature extraction module data; The image collection module is used to obtain medical images of lung tumors and collect well-labeled sample images; The region segmentation module is used to perform region segmentation on the lung tumor based on the labeling result to obtain the tumor region; The feature extraction module is used to extract edge features, morphological features and density features of the tumor area; The deep learning module is used to perform deep learning on the infiltrativeness identification of lung tumors based on the edge features, morphological features and density features of sample images.

Images