A method and system for detecting and analyzing lung nodules with adaptive layer thickness
By using an adaptive slice thickness deep learning method, a 3D or 2D network model is selected based on the slice thickness of the CT image. Combined with a classifier to remove false positives, the detection accuracy problem caused by slice thickness differences is solved, and accurate detection and segmentation of CT images with different slice thicknesses are achieved, providing rich nodule information.
Patent Information
- Application Number
- CN202011008757.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2020-09-23
- Publication Date
- 2024-11-01
- Estimated Expiration
- 2040-09-23
AI Technical Summary
Traditional methods and single deep learning models cannot simultaneously guarantee the detection accuracy of thin and thick CT images when there are large differences in the slice thickness of CT images, resulting in insufficient accuracy in lung nodule detection and segmentation.
An adaptive slice thickness analysis method is adopted, which selects a 3D or 2D deep learning network model according to the thickness of each slice of the CT image. By combining 3D and 2D deep learning classifiers, false positive lung nodules are removed through model training and validation, thus achieving accurate detection and segmentation.
It improves the detection accuracy of CT images with different slice thicknesses, and provides complete lung nodule detection, segmentation, anatomical location analysis and feature analysis, providing richer nodule information for clinical diagnosis.
Smart Images

Figure CN112241948B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of medical technology, and in particular to a method and system for detecting and analyzing lung nodules with adaptive layer thickness. Background Art
[0002] For the problem of lung nodule detection, traditional methods detect and segment lung nodules through morphological segmentation algorithms and manual feature extraction algorithms. However, due to the influence of different nodule shapes and densities, the detection and segmentation accuracy cannot meet the requirements of clinical use.
[0003] For example, patent CN102842132B proposes a CT lung nodule detection method, which first uses edge detection and spherical and current enhancement filters to enhance nodules and vascular areas, and then uses threshold segmentation to obtain the initial candidate nodule image. 3D and 2D features are calculated for the initial nodule image, such as shape, grayscale, curvature, eccentricity, Fourier descriptor, etc., and classified and screened according to the features to obtain the final nodule detection result.
[0004] In recent years, deep learning technology has been gradually applied to lung nodule detection. Most algorithms are based on a single deep learning detection model to select the nodule location. For example, patent CN108446730A proposes a deep learning-based CT lung nodule detection method. For each layer of the lung computed tomography (CT) image, the scheme selects the n layers of images before and after it and inputs them into the constructed 2n+1 channel multi-scale 2D convolutional deep learning network structure, and obtains the candidate box through the regional generation network (RPN), and then obtains the final nodule detection result through the classification module.
[0005] Since the thickness of the layers of CT images varies greatly, ranging from less than one millimeter to more than ten millimeters, the image features will also vary greatly. The essence of deep learning algorithms is to automatically find high-dimensional image features that can be used for detection or segmentation. Therefore, when the image features vary greatly, using a single nodule detection model cannot guarantee the detection and segmentation accuracy of all CT images. Therefore, when the thickness of CT images varies greatly, it is impossible to simultaneously guarantee relatively high detection accuracy for both thin-layer CT images and thick-layer CT images. Summary of the invention
[0006] In response to the above technical problems, an embodiment of the present invention provides an adaptive layer thickness analysis method and system to solve one or more problems in which the detection accuracy of traditional CT images cannot be guaranteed when the thickness difference is large.
[0007] The first aspect of an embodiment of the present invention provides an adaptive layer thickness lung nodule detection and analysis method, comprising the following steps: reading a CT image of the lungs, and determining a target deep learning network model based on the thickness of each layer in the CT image; using the CT image as input and the annotated lung nodule positions in the CT image as output, training the target deep learning network model to obtain a trained lung nodule segmentation and detection model; inputting a test CT image into the lung nodule segmentation and detection model to obtain an initial lung nodule detection result of the first stage; for the initial lung nodule detection result of the first stage, using a deep learning-based 3D deep learning classifier to remove false positive lung nodules to obtain a final lung nodule detection result of the second stage.
[0008] Optionally, determining the target deep learning network model according to the thickness of each layer in the CT image includes: determining the thickness of each layer in the CT image; judging whether the thickness of each layer is greater than a preset thickness, and if so, selecting a neural network model with a 3D first stage as the target deep learning model; and if so, selecting a neural network model with a 2D first stage as the target deep learning model.
[0009] Optionally, the use of a deep learning-based 3D deep learning classifier to remove false-positive lung nodules specifically includes: using the lung nodule segmentation and detection model to segment a nodule mask, and calculating a nodule diameter based on the nodule mask; and based on the nodule diameter, selecting different thresholds and probability selection strategies to remove the false-positive lung nodules.
[0010] Optionally, the method also includes: intercepting a sequence of three-dimensional image blocks with the three-dimensional center point of the lung nodule in the CT image as the center; taking the three-dimensional image block as input and the marked types of different lung nodules in the CT image as output, training a three-dimensional four-classification convolutional neural network model to obtain a trained nodule subclass classification model; inputting the nodule image output by the lung nodule segmentation and detection model into the nodule subclass classification model to obtain the predicted probability of different types of nodules; and selecting the category with the highest probability as the nodule subclass type of the lung nodule.
[0011] Optionally, the nodule subtypes include: ground glass, solid, subsolid and calcification.
[0012] Optionally, the method also includes: intercepting a two-dimensional image in the nodule center layer of the CT image with the nodule center point as the center; taking the two-dimensional image as input and the different benign and malignant grades of the lung nodules in the annotated CT image as output, training a two-dimensional four-classification convolutional neural network model to obtain a trained nodule benign and malignant classification model; inputting the nodule image output by the lung nodule segmentation and detection model into the nodule benign and malignant classification model to obtain the predicted probability of nodules of different benign and malignant grades; and selecting the grade with the highest probability as the benign and malignant grade of the nodule.
[0013] Optionally, the method also includes: taking the three-dimensional image block as input and the labeled lung lobes and lung segments in the three-dimensional image block as output, training a convolutional neural network model to obtain a trained nodule anatomical position judgment model; inputting the nodule image output by the lung nodule segmentation and detection model into the nodule anatomical position judgment model to obtain the predicted probabilities of different lung segments; and selecting the lung segment with the highest probability as the output result.
[0014] Optionally, the training of the nodule anatomical position judgment model includes the following two stages: the first stage: taking the three-dimensional image block as input, and the labeled lung lobes of various categories as output, to construct an initial network; the second stage: taking the output of the constructed initial network as input, and the labeled positions of various lung lobes and trachea of the lung segments as output, to obtain the final nodule anatomical position judgment model.
[0015] Optionally, when training the nodule anatomical position determination model, the loss function is obtained by weighting the lobe segmentation results and the interpulmonary fissure segmentation results.
[0016] The second aspect of an embodiment of the present invention provides an adaptive layer thickness lung nodule detection and analysis system, characterized in that it includes: a lung nodule segmentation and detection module, which is used to obtain the initial lung nodule detection result of the first stage after the CT image is input into the lung nodule segmentation and detection model; for the initial lung nodule detection result of the first stage, a 3D deep learning classifier based on deep learning is used to remove false positive lung nodules to obtain the final lung nodule detection result of the second stage; a nodule subclass classification module, which is used to perform subclass classification on the nodules in the CT image; a nodule benign and malignant classification module, which is used to classify the nodules in the CT image into benign and malignant; a nodule anatomical position judgment module, which is used to judge the anatomical position of the nodule in the CT image.
[0017] The deep learning-based nodule detection method and system provided by the embodiment of the present invention selects 3D and 2D two-stage deep learning neural network models according to the thickness characteristics of each layer in the CT image, which can better utilize the image features of CT images with different layer thicknesses, such as the continuity between different layers of thin layer images, and the more obvious nodule features of thick layer data on a single layer image, thereby achieving accurate detection and segmentation of various layer thickness data. At the same time, the embodiment of the present invention also provides a complete set of lung nodule detection, segmentation, anatomical position analysis and feature analysis methods on this basis, providing richer nodule information for clinical use. BRIEF DESCRIPTION OF THE DRAWINGS
[0018] One or more embodiments are exemplarily described by pictures in the corresponding drawings, and these exemplified descriptions do not constitute limitations on the embodiments. Elements with the same reference numerals in the drawings represent similar elements, and unless otherwise stated, the figures in the drawings do not constitute proportional limitations.
[0019] Figure 1 is a flowchart of a nodule detection method based on deep learning according to an embodiment of the present invention;
[0020] Figure 2 is a schematic diagram of building a network structure similar to Vnet provided by an embodiment of the present invention;
[0021] Figure 3 is a schematic diagram of building a network structure similar to Unet provided by an embodiment of the present invention;
[0022] Figure 4 It is a structural block diagram of a nodule detection system based on deep learning provided in an embodiment of the present invention. DETAILED DESCRIPTION
[0023] In order to facilitate the understanding of the present invention, the present invention is described in more detail below in conjunction with the accompanying drawings and specific embodiments. It should be noted that when an element is described as "fixed to" another element, it can be directly on another element or there can be one or more centered elements therebetween. When an element is described as "connected" to another element, it can be directly connected to another element or there can be one or more centered elements therebetween. The orientation or positional relationship indicated by the terms "upper", "lower", "inner", "outer", "bottom" etc. used in this specification is based on the orientation or positional relationship shown in the accompanying drawings, only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as a limitation of the present invention. In addition, the terms "first", "second", "third", etc. are only used for descriptive purposes and cannot be understood as indicating or implying relative importance.
[0024] Unless otherwise defined, all technical and scientific terms used in this specification have the same meaning as those commonly understood by those skilled in the art of the present invention. The terms used in this specification and in the specification of the present invention are only for the purpose of describing specific embodiments and are not intended to limit the present invention. The term "and / or" used in this specification includes any and all combinations of one or more of the related listed items.
[0025] The present invention first provides a method for detecting and analyzing lung nodules with adaptive layer thickness. Figure 1 , the method comprises the following steps,
[0026] Step 101: read a CT image of the lungs, and determine a target deep learning network model based on the thickness of each layer in the CT image.
[0027] When taking lung CT images, the thickness of the slices generally varies from less than one millimeter to more than ten millimeters. The thickness difference is large, so the image features extracted from thick and thin slice images will also be very different. For example, thin slice images have good continuity between different layers, while thick slice images have more obvious nodule features on a single layer. Therefore, when the image features vary greatly, the traditional single nodule detection model cannot guarantee the detection and segmentation accuracy of all CT images, that is, it cannot guarantee a relatively high detection accuracy for both thin and thick slice CT images.
[0028] In this step, after reading the CT image of the lungs, the threshold and morphological methods are first used to segment the lungs to obtain the image within the lung frame, and then the layer thickness of the image is determined. Different target deep learning network models are selected according to the different thicknesses of each layer in the CT image. Different target deep learning network models have different algorithm structures, which can better utilize the image features of CT images in different layer thicknesses.
[0029] The target deep learning network model is determined according to the thickness of each layer in the CT image, which specifically includes the following steps:
[0030] 1. Determine the thickness of each layer in the CT image;
[0031] 2. Determine whether the thickness of each layer is greater than the preset thickness. If it is less than the preset thickness, select the 3D neural network model as the target deep learning model; if it is greater than the preset thickness, select the 2D neural network model as the target deep learning model.
[0032] In the embodiment of the present invention, the preset thickness can be selected according to the actual requirements for image features. In the embodiment of the present invention, the preset thickness can be 3 mm, that is, when the layer thickness of the CT image is less than 3 mm, it is confirmed as a thin-layer CT image, and when the layer thickness of the CT image is greater than 3 mm, it is confirmed as a thick-layer CT image.
[0033] Step 102: Take the CT image as input and the annotated lung nodule position in the CT image as output, train the target deep learning network model, and thus obtain a trained lung nodule segmentation and detection model.
[0034] Before training the target deep learning network model, it is necessary to first construct a CT image containing the lungs. The data volume is about 3,000 CT data. The CT images in the CT images contain complete lungs. Most of them are chest CT images, and a small number can be chest, abdomen, head, chest or whole body CT images. The doctor also annotates the location and mask of the lung nodules in the CT images of the training set. The target deep learning network model is trained by taking the CT images in the training set as input and the annotated lung nodule locations in the CT images as output, thereby obtaining a trained lung nodule segmentation and detection model.
[0035] In the embodiment of the present invention, the 3D neural network model is selected as the target deep learning model for thin-layer CT images, and the 2D neural network model is selected as the target deep learning model for thick-layer CT images. The target deep learning network model in this step is used to achieve segmentation and detection of CT images.
[0036] For thin-layer CT images, the feature information on a single CT image is limited and easily confused with blood vessels, so three-dimensional image information is required. Therefore, a 3D convolutional neural network is used in the first stage to realize the detection and segmentation of lung nodules.
[0037] In the first stage, the embodiment of the present invention uses a 3D-Unet network model for training of thin-layer CT images. It uses the data of pulmonary nodule segmentation annotation to train a segmentation model. The network extracts the contour features of the image through three downsampling layers, and then restores the feature image to the size of the original image through three upsampling layers. The connection layer connects the output of each downsampling layer with the input of the upsampling layer for calculation. The last convolution layer outputs the possibility that each pixel is a pulmonary nodule. After being intercepted by a set threshold, the output matrix is subjected to connected domain analysis to obtain the suspicious area of the pulmonary nodule and the mask of each nodule.
[0038] For thick-layer CT images, the continuity of image information between different layers is insufficient. Such information jumps will interfere with detection. Therefore, a 2D convolutional neural network is used to realize the detection and segmentation of lung nodules.
[0039] In the first stage, the embodiment of the present invention specifically uses a 2D instance segmentation model Mask R-CNN for training thick-layer CT images. First, in the feature extraction part, the ResNet model is used to gradually downsample using a convolutional neural network, and then the RPN network is used to extract valid suspected areas, and then each suspected area is classified into two categories to determine whether it is a nodule area, and the nodules in the area are segmented.
[0040] By using different detection and segmentation deep learning models, good detection and segmentation accuracy can be achieved for both thick and thin layer CT images.
[0041] Step 103: input the tested CT image into the pulmonary nodule segmentation and detection model to obtain the initial pulmonary nodule detection result of the first stage.
[0042] The lung nodule segmentation and detection model trained only by the 3D neural network model or the 2D instance segmentation model will still have some false positives in the initial lung nodule detection results of the first stage during lung CT detection. Therefore, in the second stage, it is necessary to combine the 3D deep learning classifier to further reduce the false positives and thus improve the detection accuracy.
[0043] Step 104: For the initial lung nodule detection results of the first stage, a 3D deep learning classifier based on deep learning is used to remove false positive lung nodules to obtain the final lung nodule detection results of the second stage.
[0044] In an embodiment of the present invention, the second stage of this step uses a 3D-VGG model to train a binary classifier, and performs a negative input to suppress false positives in the pulmonary nodule segmentation and detection model. Secondary training is performed to remove the false positive area in the first stage. That is, by mining the image of interest to the false positive in the negative CT image, and then putting the false positive image and the training set of the aforementioned step 102 into the 3D-VGG model training binary classifier training at the same time, the final pulmonary nodule detection result of the second stage is obtained. In some embodiments, in the second stage of the false positive binary classification, the input training set can be amplified 8 times for prediction, and for candidate nodules with a diameter greater than 10 mm, the maximum prediction probability among the 8 prediction values is selected as the pulmonary nodule prediction probability, and for candidate nodules with a diameter less than 10 mm, the average probability of the 8 prediction values is selected as the pulmonary nodule prediction probability.
[0045] In the second-stage false positive removal algorithm, in order to better improve the detection of large nodules and suppress small blood vessel areas that are prone to cause false positives (small blood vessels are more common in areas close to the lung wall, the blood vessels are relatively thin, and appear as a dot on the CT image, and their characteristics are similar to those of small nodules), in some embodiments, the diameter of the nodule can be calculated based on the nodule mask obtained by segmentation, and different thresholds and probability selection strategies can be selected according to the size of the nodule diameter to filter out false positives.
[0046] The deep learning-based nodule detection method provided in an embodiment of the present invention uses a 3D model for thin-layer CT images and a 2D+3D model for thick-layer CT images for CT images with different layer thicknesses. Different algorithm structures are used to improve the detection accuracy of the algorithm for images with different layer thicknesses, making the algorithm performance more stable.
[0047] For the detection of lung nodules, in clinical applications, doctors not only need to find lung nodules, but also need to calculate and judge the shape, type, anatomical location and other information of the nodules. Therefore, it is not enough to only detect nodules. In some embodiments, on the basis of realizing detection, different deep learning network models can also be selected to realize the subclassification of lung nodules based on CT images, the judgment of benign and malignant and specific anatomical location, etc. During the training process, according to the different types and amounts of training data, technical personnel in this field can use random data enhancement methods to adjust the amount of data of a certain type according to actual needs, so as to balance the training data and achieve the best training effect. The above implementation methods are introduced one by one below.
[0048] The method for subclassifying different types of nodules includes the following steps:
[0049] 1. Take the 3D center point of the pulmonary nodule in the CT image as the center to intercept the 3D image blocks of the sequence;
[0050] 2. Take the 3D image block as input and the different types of lung nodules in the annotated CT image as output, train the 3D four-classification convolutional neural network model, and obtain the trained nodule subclass classification model;
[0051] 3. Input the nodule images output by the pulmonary nodule segmentation and detection model into the nodule subclassification model to obtain the predicted probabilities of different types of nodules;
[0052] 4. Select the category with the highest probability as the nodule subtype of the lung nodule.
[0053] In the constructed training set containing CT images of the lungs, doctors also labeled the four subclassifications of lung nodules in the CT images of the training set.
[0054] For subclassification of nodules, a three-dimensional image block is first captured with the three-dimensional center point of the lung nodule in the CT image as the center, and these three-dimensional image block data are used as input, and the types of different lung nodules in the labeled CT image are used as output. A three-dimensional four-classification convolutional neural network model is trained to realize the nodule subclassification model. The nodules detected are divided into four types: ground glass, solid, subsolid, and calcified to better assist doctors in diagnosis.
[0055] The method for classifying benign and malignant nodules includes the following steps:
[0056] 1. In the CT image, a two-dimensional image is captured with the center point of the nodule as the center in the central layer of the nodule;
[0057] 2. Take the two-dimensional image as input and the different benign and malignant grades of lung nodules in the annotated CT image as output, train the two-dimensional four-classification convolutional neural network model, and obtain the trained benign and malignant nodule classification model;
[0058] 3. Input the nodule image output by the lung nodule segmentation and detection model into the nodule benign and malignant classification model to obtain the predicted probability of nodules of different benign and malignant grades;
[0059] 4. Select the grade with the highest probability as the benign or malignant grade of the nodule.
[0060] In the constructed training set containing CT images of the lungs, doctors also labeled the four different grades of benign and malignant classifications of lung nodules in the CT images of the training set.
[0061] To classify nodules into benign and malignant, the 2D deep learning Densnet model is first used to classify nodules into four benign and malignant grades: benign, partially benign, partially malignant, and malignant. In the nodule center layer, a 2D image is intercepted with the nodule center point as the center, and input into the model for training. The model output is the probability value of the nodule belonging to the four benign and malignant grades. In the prediction, the nodule image blocks obtained by detection and segmentation are input into the network, and the grade with the highest probability is selected as the benign and malignant grade of the nodule.
[0062] The method for anatomical judgment of nodules includes the following steps:
[0063] 1. Take the 3D image block as input, and the labeled lobes and segments in the 3D image block as output, train the convolutional neural network model, and obtain a trained nodule anatomical location judgment model;
[0064] 2. Input the nodule image output by the nodule segmentation and detection model into the nodule anatomical position judgment model to obtain the predicted probability of different lung segments;
[0065] 3. Select the lung segment with the highest probability as the output result.
[0066] The nodule anatomical position judgment module requires anatomical segmentation of the lung area. Therefore, in the constructed training set containing CT images of the lungs, doctors are required to perform anatomical segmentation on the CT images of the training set, that is, to divide the entire lung into individual lobes and segments, and outline the contours of each anatomical region. The data volume is about 100 CT image data.
[0067] Therefore, the nodule anatomical position determination model of the present invention includes the following two stages during training:
[0068] Phase 1: Take the 3D image patch as input, label the lung lobes of each category as output, and build the initial network;
[0069] The second stage: the output of the constructed initial network is used as input, and the marked positions of each lung lobe and trachea of the lung segment are used as output to obtain the final nodule anatomical position judgment model.
[0070] In the first stage, the entire lung image is segmented to obtain a lung frame 3D image, and then the lung lobe is segmented through the lung frame 3D image to obtain the detection result of the lung lobe segmentation. Figure 2 As shown in the figure, in the first stage, a network structure similar to Vnet is first constructed. The input of the network is a three-dimensional CT image block. The network has two outputs, one of which is a score map that marks the segmentation of 5 different categories of lobes by comparing with the corresponding gold standard mask. That is, the probability map of the 5 lobes. The other is a score map that marks the pulmonary fissures, that is, the probability map of the segmented pulmonary fissures. By adding the supervision information brought by the segmentation of the pulmonary fissures, the model pays more attention to the boundary positions between different lobes, and can better segment the edge positions of each lobe, so as to achieve more accurate segmentation of the lobes.
[0071] During the training process, the loss function of the present invention is obtained by weighting the pulmonary lobe segmentation results and the pulmonary fissure segmentation results. Due to the limited GPU hardware resources, the 3D image of the lung frame to be detected can be cut into 128*128*64 image areas for input, and finally the pulmonary lobe segmentation results of multiple small areas are spliced into the segmentation result of the entire lung area.
[0072] The anatomical location of the nodule includes the information of the lung lobe and lung segment where the nodule is located. Therefore, after obtaining the results of the lung lobe segmentation, it is necessary to further divide the anatomical location of the CT image in detail in the second stage. The lungs are divided into 18 lung segments. In addition, since the location of the lung segment mainly depends on the segmentation results of the lung lobe and the location of the corresponding trachea, the trachea in the lung needs to be segmented.
[0073] The lung frame 3D image obtained after the first stage of lung segmentation is selected, and the lung image at a distance of more than 1 / 3 from the lung apex of the lung frame 3D image is selected. Then the lung image is scaled to 96*96*96 and input into the 3D-Unet network. After a multi-layer convolutional neural network and downsampling and upsampling, the sigmoid activation function is used to output the probability map of the trachea, and then the mask of the trachea segmentation is obtained according to the threshold.
[0074] After obtaining the lung lobe segmentation mask and trachea segmentation mask, Figure 3As shown in the figure, in the second stage, a 3D-Unet network is constructed, and the lung frame 3D image, lung lobe segmentation mask, and trachea segmentation mask are merged into a multi-channel image and input into the 3D-Unet network. After training, the 3D-Unet network can achieve probability maps of 18 different classifications of lung segment segmentation, and the lung segment with the highest probability for each pixel is taken as the final result of the pixel, and finally the segmentation results of all lung segments are obtained.
[0075] After the pulmonary nodules detected in the first stage are segmented to obtain the nodule mask, the overlap area between the nodule mask and each pulmonary segment mask is calculated, and the pulmonary segment with the largest overlap area is selected as the pulmonary lobe and segmentation result of the nodule. After obtaining the pulmonary lobe and segmentation results of the nodule, clinical parameters such as the nodule's long and short axes, volume, and maximum / minimum CT value can also be calculated based on the pulmonary nodule mask.
[0076] The deep learning-based nodule detection method provided by the embodiment of the present invention first selects 3D and 2D two-stage deep learning neural network models according to the characteristics of CT images with different layer thicknesses, which can better utilize the image features of CT images with different layer thicknesses, such as the continuity between different layers of thin layer images, and the more obvious nodule features of thick layer data on single layer images, etc., thereby achieving accurate detection and segmentation of various layer thickness data. At the same time, the embodiment of the present invention also provides a complete set of lung nodule detection, segmentation, anatomical position analysis and feature analysis methods on this basis, providing richer nodule information for clinical use.
[0077] The embodiment of the present invention also provides a pulmonary nodule detection and analysis system with adaptive layer thickness. Figure 4 As shown, the system 400 includes: a lung nodule segmentation and detection module 41, a nodule subclass classification module 42, a nodule benign or malignant classification module 43 and a nodule anatomical position determination module 44.
[0078] The pulmonary nodule segmentation and detection module 41 is used to obtain the initial pulmonary nodule detection result of the first stage after the CT image is input into the pulmonary nodule segmentation and detection model; for the initial pulmonary nodule detection result of the first stage, the 3D deep learning classifier based on deep learning is used to remove false positive pulmonary nodules to obtain the final pulmonary nodule detection result of the second stage;
[0079] A nodule subclassification module 42, configured to perform subclassification on the nodules in the CT image;
[0080] A nodule benign or malignant classification module 43, used for classifying nodules in the CT image into benign or malignant types;
[0081] The nodule anatomical position determination module 44 is used to determine the anatomical position of the nodule in the CT image.
[0082] The process of CT image detection based on this system is as follows:
[0083] After the CT image is input into the deep learning-based nodule detection system 400, the lung nodule segmentation and detection module 41 detects the CT image. First, the lung is segmented using thresholds and morphological methods to obtain a 3D image of the lung frame, and then the layer thickness of the image is determined. If the layer thickness is less than 3 mm, a thin layer 3D detection and segmentation model is used. If the layer thickness is greater than 3 mm, a thick layer 2D detection and segmentation model is used to obtain the nodule detection and segmentation results.
[0084] Then, the detected and segmented nodule results are subjected to subclassification, benign and malignant classification, and the lung anatomical structure position is obtained through the nodule benign and malignant classification module 42, the nodule benign and malignant classification module 43, and the nodule anatomical position judgment module 44 respectively.
[0085] After obtaining the pulmonary lobe and segmentation results of the nodule, the nodule anatomical position judgment module can also calculate clinical parameters such as the nodule's long and short axes, volume, maximum / minimum CT values, etc. based on the mask of the pulmonary nodule, providing richer nodule information for clinical use.
[0086] The deep learning-based nodule detection system provided by the embodiment of the present invention first selects 3D and 2D two-stage deep learning neural network models according to the characteristics of CT images with different layer thicknesses, which can better utilize the image features of CT images with different layer thicknesses, such as the continuity between different layers of thin layer images, and the more obvious nodule features of thick layer data on single layer images, etc., thereby achieving accurate detection and segmentation of various layer thickness data. At the same time, the embodiment of the present invention also provides a complete set of lung nodule detection, segmentation, anatomical position analysis and feature analysis methods on this basis, providing richer nodule information for clinical use.
[0087] It should be noted that the present invention can also be applied to the detection of other diseases in the medical field, such as pneumonia, rib fractures, etc. All detections of other diseases using the algorithm principles of the present invention fall within the protection scope of the present invention.
[0088] Those skilled in the art should further realize that the various steps of the exemplary dual-light image integration method described in conjunction with the embodiments disclosed herein can be implemented by electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the above description has generally described the composition and steps of each example according to function. Whether these functions are executed in hardware or software depends on the specific application and design constraints of the technical solution.
[0089] Those skilled in the art may use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention. The computer software may be stored in a computer-readable storage medium, and when the program is executed, it may include the process of the embodiments of the above methods. The storage medium may be a disk, an optical disk, a read-only storage memory, or a random access memory, etc.
[0090] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit them. Under the concept of the present invention, the technical features in the above embodiments or different embodiments may also be combined, the steps may be implemented in any order, and there are many other changes in different aspects of the present invention as described above, which are not provided in detail for the sake of simplicity. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they may still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the scope of the technical solutions of the embodiments of the present invention.
Claims
1. A method for detecting and analyzing lung nodules with adaptive layer thickness, characterized in that: The steps include: Reading a CT image of the lungs, and determining a target deep learning network model according to the thickness of each layer in the CT image; Taking the CT image as input and the annotated lung nodule position in the CT image as output, training the target deep learning network model, thereby obtaining a trained lung nodule segmentation and detection model; Inputting the tested CT image into the pulmonary nodule segmentation and detection model to obtain the initial pulmonary nodule detection result of the first stage; For the initial lung nodule detection results in the first stage, a 3D deep learning classifier based on deep learning is used to remove false positive lung nodules to obtain the final lung nodule detection results in the second stage; The step of determining a target deep learning network model according to the thickness of each layer in the CT image includes: Determining the thickness of each layer in the CT image; Determine whether the thickness of each layer is greater than a preset thickness. If it is less than the preset thickness, select the neural network model with a 3D first stage as the target deep learning model; if it is greater than the preset thickness, select the neural network model with a 2D first stage as the target deep learning model; The method of removing false positive pulmonary nodules by using a 3D deep learning classifier based on deep learning specifically includes: Using the pulmonary nodule segmentation and detection model, segmenting to obtain a nodule mask, and calculating the nodule diameter according to the nodule mask; According to the nodule diameter, different thresholds and probability selection strategies are selected to remove the false positive pulmonary nodules; Taking the three-dimensional center point of the lung nodule in the CT image as the center, intercepting a sequence of three-dimensional image blocks; Taking the three-dimensional image block as input and the marked types of different lung nodules in the CT image as output, training a three-dimensional four-classification convolutional neural network model to obtain a trained nodule subclass classification model; Inputting the nodule image output by the pulmonary nodule segmentation and detection model into the nodule subclassification model to obtain the predicted probabilities of different types of nodules; The category with the highest probability was selected as the nodule subtype of the pulmonary nodule; The nodule subtypes include: Ground glass, solid, subsolid and calcified; In the CT image, a two-dimensional image is intercepted at the nodule center layer with the nodule center point as the center; The two-dimensional image is used as input, and the different benign and malignant grades of lung nodules in the annotated CT image are used as output, and a two-dimensional four-classification convolutional neural network model is trained to obtain a trained benign and malignant nodule classification model; Inputting the nodule image output by the pulmonary nodule segmentation and detection model into the nodule benign and malignant classification model to obtain the predicted probability of nodules of different benign and malignant grades; The grade with the highest probability is selected as the benign or malignant grade of the nodule; Taking the three-dimensional image block as input and the labeled lung lobes and lung segments in the three-dimensional image block as output, training a convolutional neural network model to obtain a trained nodule anatomical position judgment model; The nodule image output by the pulmonary nodule segmentation and detection model is input into the nodule anatomical position determination model to obtain the prediction probability of different lung segments; The lung segment with the highest probability is selected as the output result.
2. The method according to claim 1, characterized in that: The training of the nodule anatomical position judgment model includes the following two stages: Stage 1: taking the three-dimensional image block as input and the labeled lung lobes of various categories as output, and constructing an initial network; The second stage: the output of the constructed initial network is used as input, and the marked positions of each lung lobe and trachea of the lung segment are used as output to obtain the final nodule anatomical position judgment model.
3. The method according to claim 2, characterized in that When training the nodule anatomical position determination model, the loss function is obtained by weighting the lung lobe segmentation results and the lung fissure segmentation results.
4. An adaptive slice thickness pulmonary nodule detection and analysis system, characterized in that: include: A pulmonary nodule segmentation and detection module, used to obtain the initial pulmonary nodule detection result of the first stage after the CT image is input into the pulmonary nodule segmentation and detection model; For the initial lung nodule detection results in the first stage, a 3D deep learning classifier based on deep learning is used to remove false positive lung nodules to obtain the final lung nodule detection results in the second stage; A nodule subclassification module, used for performing subclassification on nodules in the CT image; A nodule benign and malignant classification module, used for classifying nodules in the CT image into benign and malignant types; A nodule anatomical position determination module, used to determine the anatomical position of the nodule in the CT image; Determining a target deep learning network model according to the thickness of each layer in the CT image includes: Determining the thickness of each layer in the CT image; Determine whether the thickness of each layer is greater than a preset thickness. If it is less than the preset thickness, select the neural network model with a 3D first stage as the target deep learning model; if it is greater than the preset thickness, select the neural network model with a 2D first stage as the target deep learning model; The method of removing false positive pulmonary nodules by using a 3D deep learning classifier based on deep learning specifically includes: Using the pulmonary nodule segmentation and detection model, segmenting to obtain a nodule mask, and calculating the nodule diameter according to the nodule mask; According to the nodule diameter, different thresholds and probability selection strategies are selected to remove the false positive pulmonary nodules; Taking the three-dimensional center point of the lung nodule in the CT image as the center, intercepting a sequence of three-dimensional image blocks; Taking the three-dimensional image block as input and the marked types of different lung nodules in the CT image as output, training a three-dimensional four-classification convolutional neural network model to obtain a trained nodule subclass classification model; Inputting the nodule image output by the pulmonary nodule segmentation and detection model into the nodule subclassification model to obtain the predicted probabilities of different types of nodules; The category with the highest probability was selected as the nodule subtype of the pulmonary nodule; The nodule subtypes include: Ground glass, solid, subsolid and calcified; In the CT image, a two-dimensional image is intercepted at the nodule center layer with the nodule center point as the center; The two-dimensional image is used as input, and the different benign and malignant grades of lung nodules in the annotated CT image are used as output, and a two-dimensional four-classification convolutional neural network model is trained to obtain a trained benign and malignant nodule classification model; Inputting the nodule image output by the pulmonary nodule segmentation and detection model into the nodule benign and malignant classification model to obtain the predicted probability of nodules of different benign and malignant grades; The grade with the highest probability is selected as the benign or malignant grade of the nodule; Taking the three-dimensional image block as input and the labeled lung lobes and lung segments in the three-dimensional image block as output, training a convolutional neural network model to obtain a trained nodule anatomical position judgment model; The nodule image output by the pulmonary nodule segmentation and detection model is input into the nodule anatomical position determination model to obtain the prediction probability of different lung segments; The lung segment with the highest probability is selected as the output result.
Citation Information
Cited By
Method for re-screening detection results of pulmonary nodule screening model
CN122574525A
