A system for predicting the benignity or malignancy of a pulmonary nodule with a diameter of less than or equal to 1 cm based on deep learning technology

Abstract

The application provides a system for predicting the benignity or malignancy of lung nodules with a diameter of less than or equal to 1 cm based on a deep learning technology, and belongs to the technical field of disease diagnosis.The system comprises the following parts: a feature extraction module, which is used for inputting a preprocessed clinical chest CT image, performing down-sampling on the input image, and performing feature learning; a prediction module, which is used for obtaining the benignity or malignancy prediction probability of lung nodules with a diameter of less than or equal to 1 cm; and an output module, which is used for outputting the prediction result.The prediction system has high sensitivity, good specificity and high accuracy, and can solve the problem that it is difficult to judge the benignity or malignancy of lung nodules with a diameter of less than or equal to 1 cm, has important significance for early diagnosis of lung cancer, and has a good application prospect.

Description

Technical Field

[0001] This invention belongs to the field of disease diagnosis technology, specifically relating to a system for predicting the benign or malignant nature of lung nodules with a diameter ≤1cm based on deep learning technology. Background Technology

[0002] To achieve early detection, diagnosis, and treatment of lung cancer and improve patient prognosis, many countries have launched lung cancer screening trials. Previous studies, such as the National Lung Screening Trial (NLST) and the Nederlands–Leuvens Longkanker Screenings Onderzoek (NELSON) trial, have demonstrated that chest computed tomography (CT) screening can facilitate early intervention in lung cancer and reduce mortality. my country has also proposed strengthening early lung cancer screening and risk assessment, and is continuously refining relevant guidelines. Accurate risk assessment can reduce overtreatment, radiation exposure, and waste of medical resources, while also reducing patient anxiety. During screening, early lung cancer mainly manifests on chest CT as solitary pulmonary nodules, i.e., round or oval shadows ≤3cm in diameter, completely enclosed by the lung parenchyma, without lymphadenopathy, atelectasis, or pneumonia. However, pulmonary nodules can also be benign. With the development of lung cancer screening, the detection rate of pulmonary nodules is gradually increasing, making it crucial to identify potential lung cancer lesions from among numerous nodules. Differentiating sub-centimeter (≤1cm in diameter) pulmonary nodules is particularly challenging because these nodules account for a high percentage in clinical practice and are mostly benign, making misdiagnosis easy. Establishing a system to predict the benign or malignant nature of pulmonary nodules ≤1cm in diameter would be of great significance to lung cancer patients and could even save their lives.

[0003] In recent years, with advancements in computer hardware and machine learning algorithms, artificial intelligence technology has continuously developed in the medical field, leading to a qualitative leap in the imaging assessment of diseases. Unlike traditional image interpretation, deep learning technology can extract quantitative features related to clinical information from medical images, providing support for disease diagnosis, treatment, and monitoring. Patent CN106372390B discloses a self-service health cloud system for lung cancer prevention based on deep convolutional neural networks; however, this patent does not predict the benign or malignant nature of lung nodules ≤1cm in diameter. Currently, there are no inventions that use deep learning technology to establish risk prediction models to assist in the diagnosis of lung nodules ≤1cm in diameter. Summary of the Invention

[0004] To address the problems of existing technologies, this invention provides a system for predicting the benign or malignant nature of lung nodules with a diameter ≤1cm based on deep learning technology.

[0005] This invention provides a system for predicting the benign or malignant nature of lung nodules ≤1cm in diameter based on deep learning technology. The system includes the following components:

[0006] The feature extraction module takes a preprocessed clinical chest CT image as input and performs downsampling and feature learning on the input image.

[0007] The prediction module is used to obtain the benign or malignant prediction probability of lung nodules with a diameter ≤1cm;

[0008] The output module is used to output the prediction results;

[0009] In the feature extraction module, when the deep learning model is a baseline whole lung model, the input image is a whole lung image; when the deep learning model is a baseline nodule slice model, the input image is a nodule slice image; when the deep learning model is a baseline nodule model, the input image is a nodule image; and when the deep learning model is a follow-up nodule model, the input images are baseline nodule images and follow-up nodule images.

[0010] Furthermore, when the deep learning model is a baseline whole lung model, a baseline nodule slice model, or a baseline nodule model, the feature extraction module consists of one module, comprising one input module and four downsampling modules. The feature extraction module is used to extract features from the whole lung image, nodule slice image, or nodule image. The prediction module consists of one global average pooling layer, one fully connected layer, and one softmax activation function layer.

[0011] And / or, when the deep learning model is a follow-up nodule model, the number of feature extraction modules is 2, each feature extraction module consists of 1 input module and 4 downsampling modules, the feature extraction modules are used to extract features from the baseline nodule image and the follow-up nodule image respectively; the prediction module consists of 2 global average pooling layers, 1 fully connected layer and 1 softmax activation function layer, the data of the 2 global average pooling layers are cascaded through feature channels and input into the fully connected layer and the softmax activation function layer in sequence.

[0012] Furthermore, in the baseline whole-lung model, the network input includes the entire lung region, and the background intensity outside the lung is set to 0.

[0013] And / or, in the baseline nodule slice model, the network input includes several CT slices where the nodule is located, and the extrapulmonary background intensity is set to 0;

[0014] And / or, in the baseline nodule model, the network input contains two channel images, one channel being a grayscale image containing nodules; the other channel being a nodule mask image, wherein the nodule region is set to 1 and the background outside the nodule is set to 0;

[0015] And / or, in the follow-up nodule model, features are extracted from the baseline nodule image and the follow-up nodule image respectively. Both the baseline nodule input and the follow-up nodule input contain two-channel images, one of which is a grayscale image containing the nodule; the other channel is a nodule mask image, wherein the region of the nodule is set to 1 and the background outside the nodule is set to 0.

[0016] Furthermore, the preprocessing method for the clinical chest CT images is as follows: First, the images are pixel-normalized, and the grayscale value range of the CT images is normalized to a distribution with a mean of 0 and a variance of 1. After normalization, the grayscale value range of the input images is [-1, 1]. Second, the image size is normalized, and the image cropping sizes input to the baseline whole lung model, baseline nodule slice model, baseline nodule model, and follow-up nodule model are set to 352×240×48, 352×240×16, 32×32×32, and 64×64×64, respectively. The resolution of all images is uniformly set to 0.5×0.5×1mm. 3 .

[0017] Furthermore, the input module consists of a convolutional module, which comprises a convolutional layer, a batch normalization layer, and a ReLU activation function layer.

[0018] Preferably, the kernel size of the convolutional layer is set to 3×3×3, and the stride size is set to 1×1×1.

[0019] Furthermore, each downsampling module adopts a residual structure; each downsampling module consists of a different number of convolutional modules, with the number of convolutional modules in the four downsampling modules being 2, 3, 4 and 4 respectively.

[0020] Furthermore, in each downsampling module, the first convolutional module consists of a convolutional layer, a batch normalization layer, and a ReLU activation function layer, with the purpose of downsampling the input. The remaining convolutional modules consist of a convolutional layer, a batch normalization layer, and a ReLU activation function layer, with the purpose of feature learning.

[0021] Furthermore, in the first convolutional module, the kernel size and stride size are 2×2×2 and 2×2×2, respectively; and / or, in the remaining convolutional modules, the kernel size is set to 3×3×3 and the stride size is set to 1×1×1.

[0022] Furthermore, the output channels of the input module and the four downsampling modules are set to 16, 32, 64, 128 and 256, respectively;

[0023] And / or, the output channel of the softmax activation function layer is set to 2.

[0024] The present invention also provides a computer-readable storage medium having stored thereon a computer program for implementing the aforementioned system for predicting the benign or malignant nature of lung nodules with a diameter ≤1cm based on deep learning technology.

[0025] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0026] This invention establishes a system based on deep learning technology to predict the benign or malignant nature of lung nodules ≤1cm in diameter. This prediction system exhibits high sensitivity, specificity, and accuracy, solving the problem of difficulty in determining the benign or malignant nature of lung nodules ≤1cm in diameter. The system established in this invention is of great significance for the early diagnosis of lung cancer and has promising application prospects.

[0027] Obviously, based on the above description of the present invention, and according to common technical knowledge and conventional methods in the field, various other modifications, substitutions or alterations can be made without departing from the basic technical concept of the present invention.

[0028] The following detailed embodiments further illustrate the above-described content of the present invention. However, this should not be construed as limiting the scope of the present invention to the following examples. All technologies implemented based on the above-described content of the present invention fall within the scope of the present invention. Attached Figure Description

[0029] Figure 1 The diagram shows the framework of the deep learning models: A represents the overall framework of the four deep learning models; B represents the network structure of the baseline whole lung model, the baseline nodule slice model, and the baseline nodule model; and C represents the network structure of the follow-up nodule model.

[0030] Figure 2 The diagram shows the ROC curves and calibration curves for the deep learning models: Baseline_whole_lung model, Baseline_nodule_slice model, Baseline_nodule_box model, and Follow-up_nodule_box model. A represents the ROC curve for the training set; B represents the ROC curve for the test set; C represents the calibration curve for the training set; and D represents the calibration curve for the test set. Detailed Implementation

[0031] It should be noted that the algorithms for data acquisition, transmission, storage and processing steps not specifically described in the embodiments, as well as the hardware structures and circuit connections not specifically described, can all be implemented using content already disclosed in the prior art.

[0032] Example 1: Method for establishing the prediction system of the present invention

[0033] 1. Modeling Methods

[0034] (I) Patient Enrollment

[0035] Study participants were included in the 2009-2020 Sichuan University West China Hospital Lung Nodule Study Cohort. Inclusion criteria were as follows: (a) lung nodule with a maximum diameter ≤1 cm; (b) pathologically confirmed after a period of follow-up, or a solid nodule with no significant change after 2 years of follow-up. Exclusion criteria were as follows: (a) nodule with indistinct borders; (b) diffuse distribution of nodules or multiple primary lung cancers; (c) no lesions detected on baseline CT.

[0036] (II) Chest CT Image Extraction and Nodule Segmentation

[0037] We extracted and analyzed chest CT images of the subjects before the first discovery of pulmonary nodules and before treatment. The required DICOM images were exported from the PACS system of West China Hospital, Sichuan University, using the 3D reconstruction software IQQA-Chest (EDDA Technology, Princeton Junction, NJ) research platform.

[0038] The exported DICOM images are cleaned and processed to obtain the necessary sequences, and then lung nodule segmentation is performed based on an AI model. Finally, all data segmented by the AI ​​model is manually checked, and any incomplete segments are manually corrected.

[0039] Image preprocessing steps: First, the image is pixel-normalized, standardizing the grayscale value range of the CT image to a distribution with a mean of 0 and a variance of 1. After normalization, the grayscale value range of the input image is [-1, 1]. Second, the image size is normalized. The image cropping sizes for the baseline whole lung model, baseline nodule slice model, baseline nodule model, and follow-up nodule model are set to 352×240×48, 352×240×16, 32×32×32, and 64×64×64, respectively. The resolution of all images is uniformly set to 0.5×0.5×1mm. 3 .

[0040] (III) Model Establishment

[0041] (1) Modeling the dataset

[0042] This study ultimately included 2628 patients with pulmonary nodules ≤1cm in diameter, of which 1231 were benign and 1397 were malignant. All patients were randomly assigned to the training set and the validation set in a 7:3 ratio. The training set was used to train the model, and the validation set was used to validate the model's effectiveness (see Table 1 for details).

[0043] Table 1. Dataset for modeling patients with pulmonary nodules ≤1cm in diameter

[0044]

[0045] (2) Deep learning model establishment

[0046] This study established a total of four deep learning models. Figure 1 A summarizes the overall framework of the model. Figure 1 B and 1C illustrate the specific network structure.

[0047] like Figure 1 As shown in Figure A, the four deep learning models established in this study include three baseline models and one follow-up model. The three baseline models are constructed based on different regions of interest (ROIs) of the baseline images, including the whole lung region, nodule slice region, and nodule region. The follow-up model is constructed together with the nodule region from the follow-up images and the baseline lung images.

[0048] like Figure 1 As shown in Figure A, this invention constructs three convolutional neural network (CNN) models to perform end-to-end analysis on whole-lung images, nodule slice images, and nodule images, respectively: (1) baseline whole-lung model; (2) baseline nodule slice model; (3) baseline nodule model. In the baseline whole-lung model, the network input includes the entire lung region, and its extra-lung background intensity is set to 0. In the baseline nodule slice model, the network input includes several CT slices containing the nodule, and its extra-lung background intensity is set to 0. In the baseline nodule model, the network input includes two-channel images, one channel being a grayscale image containing the nodule, and the other channel being a nodule mask image, where the nodule region is set to 1, and the extra-nodule background is set to 0. In this study, the three baseline models use the same network structure, and the baseline images are used to classify nodules as benign or malignant.

[0049] like Figure 1 As shown in B, the baseline model architecture consists of one input module, four downsampling modules, one global average pooling (GAP) layer, one fully connected layer, and one softmax activation function layer. The input module and the four downsampling modules constitute the feature extraction module. The input module consists of a convolutional module, which comprises a convolutional layer (kernel size set to 3×3×3, stride set to 1×1×1), a batch normalization layer, and a ReLU activation function layer. In the network structure, the downsampling modules all adopt a residual structure, that is, the input and output of the downsampling module are added together and input into the next downsampling module. The downsampling modules consist of different numbers of convolutional modules. The specific number of convolutional modules in the four downsampling modules is set to 2, 3, 4, and 4, respectively. In each downsampling module, the first convolutional module ( Figure 1The green box in B consists of a convolutional layer (kernel size 2×2×2 and stride size 2×2×2), a batch normalization layer, and a ReLU activation function layer. Its purpose is to downsample the input and expand the network's field of view. The remaining convolutional modules ( Figure 1 The network (highlighted in orange in section B) consists of a convolutional layer (kernel size set to 3×3×3, stride set to 1×1×1), a batch normalization layer, and a ReLU activation function layer, designed for feature learning. The network's input module and the output channels of the four downsampling modules are set to 16, 32, 64, 128, and 256, respectively. The softmax activation function layer has two output channels, representing the predicted probabilities of benign and malignant effects, respectively.

[0050] like Figure 1 As shown in Figure A, this invention constructs a 3D CNN model that performs end-to-end analysis of baseline and follow-up nodule images. In the follow-up nodule model, as... Figure 1 As shown in Figure C, features are extracted from baseline and follow-up nodule images, respectively. Both baseline and follow-up nodule inputs contain two-channel images: one channel is a grayscale image containing the nodule, and the other channel is a nodule mask image, where the nodule region is set to 1, and the background outside the nodule is set to 0. In this study, a follow-up nodule network is proposed, such as... Figure 1 As shown in Figure C, this network structure consists of two feature extraction modules, two global average pooling (GAP) layers, one fully connected layer, and one softmax activation function layer. The detailed structure of the feature extraction module is shown below. Figure 1 As shown in B, the two feature extraction modules are used to extract deep feature maps of nodules from baseline and follow-up nodule images, respectively. A global average pooling layer compresses and reduces the dimensionality of the deep feature maps to obtain deep network features. The deep network features of baseline and follow-up nodules are concatenated by channel and then input into a fully connected layer and a softmax activation function layer to obtain the benign / malignant prediction probability and prediction results.

[0051] 2. Experimental Results

[0052] Table 2 describes the basic clinical characteristics of the included patients, with significant differences between the benign and lung cancer groups.

[0053] Table 2. Basic clinical characteristics of included patients

[0054]

[0055]

[0056] Table 3 summarizes the diagnostic performance of the established deep learning models. The follow-up nodule model has the best performance, with AUC of 0.969 and 0.942 on the training and validation sets, respectively, sensitivity of 95.9 and 93.0, and specificity of 89.8 and 87.7. Figure 2 The ROC curves and calibration curves of each model are further shown.

[0057] Table 3. Predictive performance of the established deep learning models

[0058]

[0059] In summary, this invention establishes a system based on deep learning technology to predict the benign or malignant nature of lung nodules ≤1cm in diameter. This prediction system exhibits high sensitivity, specificity, and accuracy, solving the problem of difficulty in determining the benign or malignant nature of lung nodules ≤1cm in diameter. The system established in this invention is of great significance for the early diagnosis of lung cancer and has promising application prospects.

Claims

1. A system for predicting the benignity or malignancy of a lung nodule with a diameter of ≤ 1 cm based on deep learning technology, characterized in that: The system comprises the following parts: a feature extraction module, which inputs the preprocessed clinical chest CT image, and performs down-sampling and feature learning on the input image; a prediction module, which is used to obtain the benign and malignant prediction probability of a lung nodule with a diameter of ≤1 cm; an output module, which is used to output the prediction result; in the feature extraction module, when the deep learning model is a baseline whole lung model, the input image is a whole lung image; when the deep learning model is a baseline nodule slice model, the input image is a nodule slice image; when the deep learning model is a baseline nodule model, the input image is a nodule image; and when the deep learning model is a follow-up nodule model, the input image is a baseline nodule image and a follow-up nodule image; when the deep learning model is a baseline whole lung model, a baseline nodule slice model or a baseline nodule model, the number of the feature extraction modules is 1, which is composed of 1 input module and 4 down-sampling modules, and the feature extraction module is used to extract the features of the whole lung image, the nodule slice image or the nodule image; the prediction module is composed of 1 global average pooling layer, 1 fully connected layer and 1 softmax activation function layer; when the deep learning model is a follow-up nodule model, the number of the feature extraction modules is 2, each of which is composed of 1 input module and 4 down-sampling modules, and the feature extraction modules are respectively used to extract the features of the baseline nodule image and the follow-up nodule image; the prediction module is composed of 2 global average pooling layers, 1 fully connected layer and 1 softmax activation function layer, and the data of the 2 global average pooling layers are sequentially input into the fully connected layer and the softmax activation function layer through feature channel cascading; in the baseline whole lung model, the network input contains a whole lung region, and the intensity of the lung external background is set to 0; in the baseline nodule slice model, the network input contains several CT layers where the nodule is located, and the intensity of the lung external background is set to 0; in the baseline nodule model, the network input contains two channel images, one of which is a grayscale image containing the nodule, and the other is a nodule mask image, in which the nodule region is set to 1 and the nodule external background is set to 0; in the follow-up nodule model, the features of the baseline nodule image and the follow-up nodule image are extracted respectively, and the baseline nodule input and the follow-up nodule input both contain two channel images, one of which is a grayscale image containing the nodule, and the other is a nodule mask image, in which the nodule region is set to 1 and the nodule external background is set to 0.

2. The system of claim 1, wherein: The preprocessing method of the clinical chest CT image is: first, the image is pixel standardized, the gray value range of the CT image is standardized to a distribution with an average value of 0 and a variance of 1, after normalization, the gray value range of the input image is [-1, 1]; second, the image size is standardized, and the image cropping size of the baseline whole lung model, the baseline nodule slice model, the baseline nodule model and the follow-up nodule model is set to 352*240*48, 352*240*16, 32*32*32 and 64*64*64 respectively; all image resolutions are unified to 0.5*0.5*1mm 3 .

3. The system of claim 1, wherein: The input module is composed of a convolution module, which is composed of a convolution layer, a batch normalization layer and a ReLU activation function layer.

4. The system of claim 3, wherein: The convolution layer has a kernel size of 3×3×3 and a step size of 1×1×1.

5. The system of claim 1, wherein: Each down-sampling module adopts a residual structure, and is composed of different numbers of convolution modules, and the number of convolution modules of the 4 down-sampling modules is 2, 3, 4 and 4 respectively.

6. The system of claim 5, wherein: In each down-sampling module, the first convolution module is composed of a convolution layer, a batch normalization layer and a ReLU activation function layer, aiming to down-sample the input, and the remaining convolution modules are composed of a convolution layer, a batch normalization layer and a ReLU activation function layer, aiming to feature learning.

7. The system of claim 6, wherein: In the first convolution module, the kernel size and the stride size of the convolution layer are 2x2x2 and 2x2x2 respectively; and / or, in the remaining convolution modules, the kernel size of the convolution layer is set to 3x3x3, and the stride size is set to 1x1x1.

8. The system of claim 1, wherein: The output channels of the input module and the four down-sampling modules are set to 16, 32, 64, 128 and 256 respectively. And / or, the output channel of the softmax activation function layer is set to 2.

9. A computer-readable storage medium, characterized in that: A computer program for implementing the system for predicting the benign or malignant of a pulmonary nodule with a diameter of ≤1cm based on deep learning technology according to any one of claims 1-8 is stored thereon.

Images