CT image pulmonary nodule intelligent detection method and system based on transfer learning

Abstract

The invention provides a transfer learning-based CT image pulmonary nodule intelligent detection method and system, and the method comprises the steps: carrying out the training and testing of a noise reduction network through employing a low-dose and full-dose CT data set, and building a pulmonary nodule noise reduction model; a lung CT data set is adopted, a U-shaped segmentation network is trained and tested, and a pulmonary nodule segmentation model is established through transfer learning; and based on the coding module structure of the pulmonary nodule segmentation model, establishing a pulmonary nodule classification model through transfer learning. According to the method, a transfer learning method is comprehensively used, part of structures and parameters of the segmentation model are obtained through transfer of the noise reduction model, and part of structures and parameters of the classification model are obtained through transfer of the segmentation model. According to the method, end-to-end auxiliary algorithm research is realized, hyper-parameters required by training are reduced by using task-to-task transfer learning, the fitting speed is accelerated, and a more accurate and reliable auxiliary detection tool is provided for clinicians.

Description

Technical Field

[0001] The present invention relates to the technical field of computer-aided detection of CT images, and in particular to a method and system for intelligent detection of lung nodules in CT images based on transfer learning, and also provides a corresponding computer terminal and a computer-readable storage medium. Background Art

[0002] In 2020, statistics from the International Agency for Research on Cancer (IARC) showed that lung cancer was the second most common cancer worldwide, accounting for 11.4% of all new cases. It was also the malignant tumor with the highest mortality rate, accounting for 18.0% of all cancer deaths. The reason is that lung cancer does not have obvious clinical symptoms in the early stages, so detection is easily delayed, resulting in poor treatment effects. In order to detect lung cancer early and develop effective treatment plans, doctors usually rely on lung screening to detect whether patients have lung nodules and assess their malignancy. This practice has greatly reduced the mortality rate of lung cancer.

[0003] In 2010, Temesguen Messay et al. developed a CAD algorithm for pulmonary nodule detection in CT images. The algorithm mainly used thresholds to extract preliminary lung regions and applied segmentation refinement to achieve lung parenchyma segmentation. Then, intensity thresholding was combined with morphological processing for segmentation. Finally, a sequential forward selection technique was used to select the best feature subset out of 245 features and classify based on these features (Messay, Temesguen, Russell C. Hardie, and Steven K. Rogers. "A new computationally efficient CAD system for pulmonary nodule detection in CT imagery." Medical image analys is 14.3 (2010): 390-406.). In 2014, Colin Jacobs et al. studied the automatic detection CAD algorithm for subsolid pulmonary nodules, which have a higher probability of malignancy than solid pulmonary nodules, and completed the segmentation and classification of subsolid pulmonary nodules. They artificially defined a set of 128 features such as intensity, shape, texture and newly defined context features, and used the method of double threshold density mask to select candidate target nodules, generate clustering regions, and then applied an effective combination of morphological operations to achieve accurate nodule segmentation. Finally, they used single-stage classification and two-stage classification schemes for effect comparison (Jacobs, Colin, et al. "Automatic detection of subsolid pulmonary nodules in thoracic computed tomography images." Medical image analysis 18.2 (2014): 374-384.). It can be seen that the traditional CAD algorithm for lung nodule detection mainly segments CT images through threshold method and morphological method, and generally transitions from coarse segmentation to fine segmentation. In terms of classification, the method mainly uses artificially defined features, and then selects the best features to use classifiers for classification. Traditional methods require researchers to have a high level of relevant professional knowledge, and the amount of manual calculations is large. They are not very versatile and require the feature set to be re-screened for similar tasks.

[0004] Since Alexnet was proposed in 2012, deep learning technology has achieved overwhelming advantages in the field of computer vision and realized end-to-end object detection and classification. The deep learning model can learn the features and patterns in lung images, and extract and analyze the shape, size, location and other information of nodules. It can provide important auxiliary information for doctors, help improve the accuracy of doctors' judgment of benign and malignant lung nodules, and thus help doctors formulate more appropriate medical plans for patients, while also improving doctors' examination efficiency. Using deep learning methods to replace traditional methods to achieve automated and intelligent detection has good development prospects and has received widespread attention.

[0005] In the field of deep learning applied to medical image processing, some classic cases have emerged that can significantly improve the efficiency of diagnosis and treatment. In terms of image denoising, Eunhee et al. proposed an algorithm using CNN in 2016, applied to the wavelet transform coefficients of low-dose CT images, and used a residual learning architecture in the network (Kang, Eunhee, Junhong Min, and Jong Chul Ye. "A deep convolutional neural network using directional wavelets for low-dose X-ray CT reconstruction." Medical physics 44.10 (2017): e360-e375.). In 2017, H. Chen et al. proposed a residual encoding module decoding module convolutional neural network RED-CNN, which combined the residual with the encoding and decoding module architecture and achieved good results in preserving the edge structure of the image (Chen, Hu, et al. "Low-dose CT with aresidual encoder-decoder convolutional neural network." IEEE transactions on medical imaging 36.12 (2017): 2524-2535.). In terms of medical segmentation, U-Net plays an important role in biomedical segmentation, and to this day, the U-net end-to-end network is still one of the most popular deep learning methods.

[0006] However, deep learning currently faces some challenges and limitations in lung nodule detection: deep learning network models usually have a large number of weight parameters, and single-task networks usually ignore the correlation between upstream and downstream tasks. Multi-target tasks in medicine are often coherent, and each part of the trained network will extract features with a lot of commonalities, so that the weight parameters of the trained network also have a lot of similarities. At this time, starting training from scratch will be time-consuming and wasteful. To solve this problem, transfer learning operations can be introduced. By migrating network weight parameters, new networks with similar targets can be trained on the basis of previous networks, thereby saving time and training costs and accelerating network convergence. In terms of transfer learning, in 2021, Tang Siyuan et al. used 3DCNN and adopted a weight-based transfer method to transfer the weights of the feature extraction layer pre-trained on the LUNA16 dataset to the network as the initial weights, and fine-tuned the training on the new dataset, obtaining better results than before (Tang Siyuan, Liu Yanru, and Yang Min. "Detection of pulmonary nodules based on transfer learning and three-dimensional convolutional neural network." Chinese Journal of Medical Imaging Technology 36.12 (2020): 1882-1886.). In 2020, Chen Daozheng et al. used VGG16, VGG19, and ResNet50 pre-trained models on the large image dataset ImageNet, froze the convolutional layer, fine-tuned some network layers and adjusted the hyperparameters, achieving a significant reduction in training time (Chen Daozheng, and Jiang Qian. "Detection of pulmonary nodules based on convolutional neural network and transfer learning." Computer Engineering and Design 42.1 (2021): 240-247.).

[0007] The above-mentioned transfer learning-related work also has certain problems. At present, there is still a lack of research on the full-process lung nodule diagnosis method and architecture, and there is an urgent need for a method that organically integrates the original intelligent processing steps. Summary of the invention

[0008] In view of the above-mentioned deficiencies in the prior art, the present invention provides a method and system for intelligent detection of lung nodules in CT images based on transfer learning, and also provides a corresponding computer terminal and computer-readable storage medium.

[0009] According to one aspect of the present invention, a method for intelligent detection of lung nodules in CT images based on transfer learning is provided, comprising:

[0010] Acquire a preprocessed CT data set A, wherein the CT data set A includes: low-dose CT data and full-dose CT data including segmentation annotations and benign and malignant classification labels;

[0011] Providing a denoising network, taking the low-dose CT data in the CT data set A as the input of the denoising network, taking the full-dose CT data in the CT data set A as the output of the denoising network, training and testing the denoising network to obtain a pulmonary nodule denoising model, which is used to obtain a denoising result of a pulmonary nodule image;

[0012] Acquire a lung CT data set B, and use the lung nodule denoising model to perform denoising on the lung CT data set B to obtain a model training data set;

[0013] Providing a U-shaped segmentation network, using the model training data set to train the U-shaped segmentation network, and migrating the trained parameters of the encoding module of the pulmonary nodule denoising model to the encoding module of the U-shaped segmentation network to obtain a pulmonary nodule segmentation model, which is used to obtain a pulmonary nodule tissue image result;

[0014] A CNN network with global-local feature fusion is provided, the model training data set is used to train the CNN network with global-local feature fusion, and the trained parameters of the encoding module of the lung nodule segmentation model are migrated to the feature extraction part of the CNN network to obtain a lung nodule classification model. The model is used to obtain the benign and malignant detection results of lung nodules in CT images and perform intelligent detection of lung nodules in CT images.

[0015] Preferably, the acquiring a preprocessed CT data set A comprises:

[0016] A CT data set A including low-dose CT data and full-dose CT data including segmentation annotations and benign and malignant classification labels is obtained and preprocessed; the preprocessing includes:

[0017] Reading image data and non-image data in the low-dose CT data and the full-dose CT data, and matching them;

[0018] The image data is subjected to image format conversion.

[0019] The image data is converted into a pixel storage unit, and invalid pixels are set to zero and converted into a background.

[0020] Preferably, the method further comprises: dividing the preprocessed CT data set A into a training set and a test set.

[0021] Preferably, a denoising network is provided, and the low-dose CT data and the full-dose CT data in the CT data set A are used as the input and output of the denoising network respectively, and the denoising network is trained and tested to obtain a pulmonary nodule denoising model, including:

[0022] A denoising network is provided, the denoising network comprising an encoding module and a decoding module; wherein the encoding module is mainly composed of five two-dimensional convolution modules, and the decoding module comprises a convolution layer, a deconvolution layer, and a residual layer connected therebetween;

[0023] The low-dose CT data in the CT data set A is used as the input of the denoising network in the form of a four-dimensional tensor, and the features of the noisy image are extracted from the low-dose CT data through the encoding module, and an intermediate feature representation is obtained and transmitted to the decoding part;

[0024] The decoding part reconstructs the intermediate feature representation, adds the feature map before the convolution layer and the feature map after the symmetrical deconvolution layer to generate a denoised image, and uses the full-dose CT data in the CT data set A as the gold standard of the denoising network;

[0025] The denoising network is iteratively trained, and a test set is used to perform a performance test on the trained denoising network to obtain a pulmonary nodule denoising model.

[0026] Preferably, the step of acquiring a lung CT data set B and performing noise reduction processing on the lung CT data set B using the lung nodule noise reduction model to obtain a model training data set includes:

[0027] Acquire a lung CT data set B, wherein the lung CT data set B includes: lung CT data and corresponding segmentation labels;

[0028] The lung CT data set B is divided into a nodule group and a nodule group, and a binary mask label image is generated according to the nodule annotation as a label value of the U-shaped segmentation network;

[0029] Performing image format conversion on the lung CT data set B, and performing noise reduction processing using the lung nodule noise reduction model;

[0030] Extracting lung parenchyma from the denoised lung CT data to obtain a lung parenchyma Mask, and then multiplying the lung parenchyma Mask with the original image to obtain a lung parenchyma image;

[0031] The lung parenchyma images are used to construct a model training data set.

[0032] Preferably, the lung CT dataset B is the same dataset as the CT dataset A or a different dataset.

[0033] Preferably, a U-shaped segmentation network is provided, the U-shaped segmentation network is trained using the model training data set, and the trained parameters of the encoding module of the pulmonary nodule denoising model are migrated to the encoding module of the U-shaped segmentation network to obtain a pulmonary nodule segmentation model, including:

[0034] A U-shaped segmentation network is provided, the U-shaped segmentation network comprising a symmetrical encoding module and a decoding module; wherein: the encoding module compresses an input image through a downsampling layer module, then extracts a feature map of the input image through a double convolution layer and inputs the feature map to the decoding module; the decoding module restores the feature map size through an upsampling layer module, then concatenates the downsampling feature map and the upsampling feature map of the same size, and obtains an output image from the output layer;

[0035] The U-shaped segmentation network is trained using the model training data set, and then the trained parameters of the encoding module of the pulmonary nodule denoising model are migrated to the encoding module of the U-shaped segmentation network. Finally, iterative training and performance testing are performed to obtain a pulmonary nodule segmentation model.

[0036] Preferably, the loss function of the pulmonary nodule segmentation model adopts a mixed loss function of BCE loss and Dice loss, wherein:

[0037] L BcE =-∑[yln(p)+(1-y)ln(1-p)]

[0038] L DICE =1–DICE

[0039] Loss = λL BCE +L DICE

[0040] Among them, L BCE is BCE loss, y is, p is, L DICE is Dice los, DICE is, and λ is.

[0041] Preferably, providing a CNN network with global-local feature fusion, using the model training data set to train the CNN network with global-local feature fusion, and migrating the trained parameters of the encoding module of the pulmonary nodule segmentation model to the feature extraction part of the CNN network to obtain a pulmonary nodule classification model, including:

[0042] Providing a global-local feature fusion CNN network, and using the model training data set to train the CNN network;

[0043] Migrating the encoding module of the pulmonary nodule segmentation model to the feature fusion module of the CNN network to obtain a global feature extraction module and a local feature extraction module of the CNN network;

[0044] The global feature extraction module and the local feature extraction module are connected in series and then placed into the fully connected layer and the Dropout layer of the CNN network to output the classification result;

[0045] The encoding module of the pulmonary nodule segmentation model is migrated to the global feature extraction module and the local feature extraction module for iterative training, and a performance test is performed to finally obtain a pulmonary nodule classification model.

[0046] According to another aspect of the present invention, a CT image pulmonary nodule intelligent detection system based on transfer learning is provided, comprising:

[0047] A data processing module, the module is used to obtain a pre-processed CT data set A, the CT data set A includes: low-dose CT data and full-dose CT data including segmentation annotations and benign and malignant classification labels; and is also used to obtain a lung CT data set B, and use a lung nodule denoising model to perform denoising on the lung CT data set B to obtain a model training data set;

[0048] A pulmonary nodule denoising model module, which is used to provide a denoising network, use the low-dose CT data in the CT data set A as the input of the denoising network, use the full-dose CT data in the CT data set A as the output of the denoising network, train and test the denoising network to obtain a pulmonary nodule denoising model, and the model is used to obtain a pulmonary nodule image denoising result;

[0049] A pulmonary nodule segmentation model module, which is used to provide a U-shaped segmentation network, use the model training data set to train the U-shaped segmentation network, and migrate the trained parameters of the encoding module of the pulmonary nodule denoising model to the encoding module of the U-shaped segmentation network to obtain a pulmonary nodule segmentation model, which is used to obtain pulmonary nodule tissue image results;

[0050] A lung nodule classification model module is used to provide a CNN network with global-local feature fusion, use the model training data set to train the CNN network with global-local feature fusion, and migrate the trained parameters of the encoding module of the lung nodule segmentation model to the feature extraction part of the CNN network to obtain a lung nodule classification model, which is used to obtain the benign and malignant detection results of lung nodules in CT images.

[0051] According to a third aspect of the present invention, there is provided a computer terminal comprising a memory, a processor and a computer program stored in the memory and executable on the processor, wherein the processor, when executing the computer program, can be used to execute any of the above-described methods of the present invention, or to run any of the above-described systems of the present invention.

[0052] According to a fourth aspect of the present invention, there is provided a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, can be used to execute any of the methods described above in the present invention, or to run any of the systems described above in the present invention.

[0053] Due to the adoption of the above technical solution, the present invention has at least one of the following beneficial effects compared with the prior art:

[0054] The intelligent detection method and system for lung nodules in CT images based on transfer learning provided by the present invention trains a RED-CNN denoising network to process CT image data sets, which can reduce existing noise in images and minimize feature loss in clean images. The dense structure and residual connection of RED-CNN can better pay attention to and retain the detailed information of the image, which is beneficial to the subsequent tasks of lung nodule segmentation and benign and malignant classification, thereby improving the overall detection performance of the system.

[0055] The present invention provides a transfer learning-based intelligent detection method and system for lung nodules in CT images, trains a U-shaped segmentation network to segment the lung nodule area in the CT image, adopts a U-shaped encoding module-decoding module architecture to extract features through multi-layer downsampling modules, and then restores the original image size through multi-layer upsampling modules. In addition, the feature jump connection adopted can better focus on the boundary and shape information of the target, thereby improving the segmentation performance.

[0056] The intelligent detection method and system for lung nodules in CT images based on transfer learning provided by the present invention are based on the encoding module of the optimized U-shaped segmentation network, combined with the feature fusion module, to construct and train a benign and malignant lung nodule classification network. Adding a Dropout layer in the network can effectively prevent overfitting in network training, while reducing the number of convolution kernels can reduce the depth of the feature map and the complexity of the model parameters. The feature fusion module enables the classification network to take into account both global features and local features. The obtained classification result is both a comprehensive result of the features of the entire original image and a concentrated feedback of the lesion site obtained after segmentation that requires more attention, thereby improving the accuracy and robustness of the benign and malignant lung nodule classification network.

[0057] The intelligent detection method and system for lung nodules in CT images based on transfer learning provided by the present invention adopt a transfer learning method, namely, the migration operation of the trained parameters of the denoising network to the segmentation network and the migration operation of the trained parameters of the segmentation network encoding module to the classification network, which effectively reduces the training cost and improves the training efficiency.

[0058] The intelligent detection method and system for lung nodules in CT images based on transfer learning provided by the present invention adopt a model-based transfer learning method to directly transfer some layers and parameters of the network model in the previously completed task, thereby eliminating the need to train the network from scratch, reducing training costs, and improving training efficiency. The initial Loss value can be reduced to increase the convergence speed of the network, thereby optimizing the training of the model.

[0059] The intelligent detection method and system for lung nodules in CT images based on transfer learning provided by the present invention use the features and parameters obtained in the previous task to guide the construction and training of subsequent networks, thereby reducing the demand for the amount of training data and reducing the training time and the total number of model parameters. BRIEF DESCRIPTION OF THE DRAWINGS

[0060] Other features, objects and advantages of the present invention will become more apparent from the detailed description of non-limiting embodiments made with reference to the following drawings:

[0061] Figure 1 The present invention is a flowchart of an intelligent method for detecting lung nodules in CT images based on transfer learning in one embodiment of the present invention.

[0062] Figure 2 The present invention is a flowchart of a method for intelligently detecting lung nodules in CT images based on transfer learning in a preferred embodiment of the present invention.

[0063] Figure 3 Schematic diagram of the operation of transfer learning in a preferred embodiment of the present invention.

[0064] Figure 4 This is a diagram showing the denoising effect of the trained denoising network on a test set in a preferred embodiment of the present invention.

[0065] Figure 5 Schematic diagram of the segmentation performance of the segmentation network after training in a preferred embodiment of the present invention.

[0066] Figure 6 This is a graph showing the loss reduction before and after the segmentation network migration in a preferred embodiment of the present invention.

[0067] Figure 7 This is a graph showing the loss reduction before and after the classification network migration in a preferred embodiment of the present invention.

[0068] Figure 8 This is a schematic diagram of the component modules of an intelligent detection system for lung nodules in CT images based on transfer learning in one embodiment of the present invention.

[0069] Fig. 9 This is a working schematic diagram of an intelligent detection system for lung nodules in CT images based on transfer learning in a preferred embodiment of the present invention. DETAILED DESCRIPTION

[0070] The following is a detailed description of the embodiments of the present invention: This embodiment is implemented on the premise of the technical solution of the present invention, and a detailed implementation method and a specific operation process are given. It should be pointed out that for ordinary technicians in this field, several variations and improvements can be made without departing from the concept of the present invention, which all belong to the protection scope of the present invention.

[0071] An embodiment of the present invention provides an intelligent detection method for lung nodules in CT images based on transfer learning. The method replaces the traditional method with a deep learning method to achieve automated and intelligent detection of lung nodules in CT images, thereby improving the accuracy of doctors' judgment of the benign and malignant nature of lung nodules, and helping doctors to develop more appropriate medical plans for patients and improve detection efficiency.

[0072] like Figure 1 As shown, this embodiment provides a method for intelligent detection of lung nodules in CT images based on transfer learning, and the method may include the following operations:

[0073] S100, obtaining a preprocessed CT data set A, wherein the CT data set A includes: low-dose CT data and full-dose CT data including segmentation annotations and benign and malignant classification labels;

[0074] S200, providing a denoising network, using low-dose CT data and full-dose CT data in CT data set A as input and output of the denoising network, respectively, training and testing the denoising network, and obtaining a pulmonary nodule denoising model, which is used to obtain a pulmonary nodule image denoising result;

[0075] S300, using the lung nodule denoising model to perform denoising on the CT data set to obtain a model training data set;

[0076] S400, providing a U-shaped segmentation network, using a model training data set to train the U-shaped segmentation network, and migrating the trained parameters of the encoding module of the lung nodule denoising model to the encoding module of the U-shaped segmentation network to obtain a lung nodule segmentation model, which is used to obtain a lung nodule tissue image result;

[0077] S500 provides a CNN network with global-local feature fusion, uses a model training data set to train the CNN network with global-local feature fusion, and migrates the trained parameters of the encoding module of the lung nodule segmentation model to the feature extraction part of the CNN network to obtain a lung nodule classification model. The model is used to obtain the benign and malignant detection results of lung nodules in CT images and perform intelligent detection of lung nodules in CT images.

[0078] In some preferred embodiments, the above S100, obtaining a pre-processed CT data set A, includes:

[0079] S101, obtaining a CT data set including low-dose CT data and full-dose CT data including segmentation annotations and benign and malignant classification labels, and performing preprocessing; the preprocessing includes:

[0080] S1011, reading image data and non-image data in low-dose CT data and full-dose CT data, and matching them;

[0081] S1012, converting the image format of the image data.

[0082] S1013, convert the pixel storage unit of the image data, and set invalid pixels to zero and convert them into background.

[0083] In some preferred embodiments, the above S100 further includes:

[0084] S102, dividing the preprocessed CT data set A into a training set and a test set.

[0085] In some preferred embodiments, the above S200 provides a denoising network, uses the low-dose CT data and the full-dose CT data in the CT data set A as the input and output of the denoising network, respectively, trains and tests the denoising network, and obtains a pulmonary nodule denoising model, including:

[0086] S201, providing a denoising network, the denoising network comprising an encoding module and a decoding module; wherein the encoding module is mainly composed of five two-dimensional convolution modules, and the decoding module comprises a convolution layer, a deconvolution layer, and a residual layer connected therebetween;

[0087] S202, using the low-dose CT data in the CT data set A in the form of a four-dimensional tensor as the input of the denoising network, extracting the features of the noisy image from the low-dose CT data through the encoding module, and obtaining an intermediate feature representation to be passed to the decoding part;

[0088] S203, the decoding part reconstructs the intermediate feature representation, adds the feature map before the convolution layer and the feature map after the symmetrical deconvolution layer, generates a denoised image, and uses the full-dose CT data in the CT data set as the gold standard of the denoising network;

[0089] S204, iteratively train the denoising network, and use the test set to perform a performance test on the trained denoising network to obtain a pulmonary nodule denoising model.

[0090] In some preferred embodiments, the above S300, obtaining a lung CT data set B, and using a lung nodule denoising model to perform denoising on the lung CT data set B to obtain a model training data set, includes:

[0091] S301, obtaining a lung CT data set B, where the lung CT data set B includes: lung CT data and corresponding segmentation labels;

[0092] S302, dividing the lung CT data set B into a group with nodules and a group without nodules, and generating a binary mask label image according to the nodule annotation as a label value of the U-shaped segmentation network;

[0093] S303, converting the image format of the lung CT data set B, and performing noise reduction processing using a lung nodule noise reduction model;

[0094] S304, extracting lung parenchyma from the denoised lung CT data to obtain a lung parenchyma Mask, and then multiplying the lung parenchyma Mask by the original image to obtain a lung parenchyma image;

[0095] S305, constructing a model training data set using the lung parenchyma image.

[0096] In some preferred embodiments, the lung CT dataset B may be the same dataset as the CT dataset A, or may be a different dataset.

[0097] In some preferred embodiments, the above S400 provides a U-shaped segmentation network, uses a model training data set to train the U-shaped segmentation network, and migrates the trained parameters of the encoding module of the pulmonary nodule denoising model to the encoding module of the U-shaped segmentation network to obtain a pulmonary nodule segmentation model, including:

[0098] S401, providing a U-shaped segmentation network, the U-shaped segmentation network comprising a symmetrical encoding module and a decoding module; wherein: the encoding module compresses the input image through a downsampling layer module, and then extracts a feature map of the input image through a double convolution layer and inputs it to the decoding module; the decoding module restores the feature map size through an upsampling layer module, and then splices the downsampling feature map and the upsampling feature map of the same size, and obtains an output image from the output layer;

[0099] S402, using the model training data set to train the U-shaped segmentation network, and then migrating the trained parameters of the encoding module of the pulmonary nodule denoising model to the encoding module of the U-shaped segmentation network, and finally performing iterative training and performance testing to obtain the pulmonary nodule segmentation model.

[0100] In some preferred embodiments, the loss function of the pulmonary nodule segmentation model adopts a mixed loss function of BCE loss and Dice loss, where:

[0101] L BCE =-∑[yln(p)+(1-y)ln(1-p)]

[0102]

[0103] Loss = λL BCE +L DICE

[0104] Among them, L BCE is the binary cross entropy loss, y∈{0,1} is the true label value, p∈{0,1} is the model prediction value, L DICE is the Dice loss, λ∈[0,1] is the weight value between loss functions, and in a preferred embodiment is 0.5.

[0105] In some preferred embodiments, the above S500 provides a global-local feature fusion CNN network, uses a model training data set to train the global-local feature fusion CNN network, and migrates the trained parameters of the encoding module of the pulmonary nodule segmentation model to the feature extraction part of the CNN network to obtain a pulmonary nodule classification model, including:

[0106] S501, providing a global-local feature fusion CNN network, and using a model training data set to train the CNN network;

[0107] S502, migrating the encoding module of the lung nodule segmentation model to the feature fusion module of the CNN network to obtain a global feature extraction module and a local feature extraction module of the CNN network;

[0108] S503, connecting the global feature extraction module and the local feature extraction module in series and putting them into the fully connected layer and the Dropout layer of the CNN network to output the classification result;

[0109] S504, the encoding module of the lung nodule segmentation model is migrated to the global feature extraction module and the local feature extraction module for iterative training, and a performance test is performed to finally obtain a lung nodule classification model.

[0110] The technical solution provided by the above embodiment of the present invention is further described in detail below in conjunction with a preferred embodiment.

[0111] like Figure 2 As shown, the preferred embodiment provides a CT image lung nodule intelligent detection method based on transfer learning, comprising the following steps:

[0112] Step 1: reading and preprocessing a CT data set, wherein the CT data set includes a CT data set of low-dose CT data and full-dose CT data and a lung CT data set; wherein the CT data set and the lung CT data set may use the same data set.

[0113] Step 2: Put the low-dose and full-dose lung CT data preprocessed in step 1 into the denoising network as input and output, respectively, to obtain a trained lung nodule denoising model and obtain the denoising result of the lung nodule image;

[0114] Step 3: Use the denoising network trained in step 2 to denoise the lung CT data set and train the U-shaped segmentation network. During the training, a transfer learning method is used to transfer the trained parameters of part of the encoding module structure in the denoising model in step 2 to part of the encoding module structure in the U-shaped segmentation network. Finally, a lung nodule segmentation test is performed.

[0115] Step 4: Use the same lung CT dataset as step 3 to train a CNN network with global-local feature fusion. During training, the segmentation network encoding module trained in step 3 is partially migrated to the feature extraction part of the classification network, and the global image and lung nodule tissue image are used to test the classification performance of benign and malignant lung nodules.

[0116] In a preferred embodiment, step 1 includes but is not limited to: reading low-dose CT data and full-dose CT data, performing image format conversion on the data, extracting task-related areas, dividing the data set and other preprocessing operations.

[0117] In a preferred embodiment, step 2 further comprises the following steps:

[0118] Step 2.1: Put the 4D tensor as input into the RED-CNN denoising network;

[0119] Step 2.2: The tensor is fed into an encoding module consisting of five layers of two-dimensional convolutional modules to extract features of the noisy image;

[0120] Step 2.3: Send the intermediate feature representation to the decoding module with a residual connection deconvolution layer to reconstruct the extracted features; full-dose CT data is used as the gold standard of the denoising network;

[0121] Step 2.4: The network is trained iteratively, and the performance of the trained denoising network is tested using the test set.

[0122] In a preferred embodiment, step 3 further comprises the following steps:

[0123] Step 3.1: Divide the lung CT data set into a nodule group and a non-nodule group, generate a mask based on the nodule annotation, perform format conversion, and finally extract the lung parenchyma;

[0124] Step 3.2: In the input link, the four-dimensional tensor of the same format is input into the U-shaped segmentation network. The main idea of ​​the encoding module and decoding module is also used. The features are extracted first and then the original image size is restored to obtain the lung nodule segmentation result.

[0125] Step 3.3: In the network coding module, the image is compressed by extracting features, normalizing, and performing nonlinear transformations through a four-layer downsampling module, and then a double convolutional layer is used to extract features of the input image.

[0126] Step 3.4: In the network decoding module, the original image size is restored through a four-layer upsampling module, from reducing the depth of the feature map, to splicing the feature maps of the same size and the upsampled feature map, and then to double convolution to compress the number of channels, and finally repeat four times to obtain the output image from the output layer;

[0127] Step 3.5: Migrate the trained parameters of the encoding module of the denoising network trained in step 2 to the encoding module of the segmentation network, followed by iterative training and performance testing.

[0128] In a preferred embodiment, the loss function used in step 3 adopts a mixed strategy of BCE loss and Dice loss to improve network evaluation performance. The BCEloss and Diceloss formulas are as follows:

[0129] L BCE =-∑[yln(p)+(1-y)ln(1-p)]

[0130]

[0131] Loss = λL BCE +L DICE In a preferred embodiment, step 4 further comprises the following steps:

[0132] Step 4.1: Using the data file obtained after preprocessing in step 3.1, use the mask to extract the lung nodule area, and extract the central image of the lung nodule without lung parenchyma in a unified 64*64 format;

[0133] Step 4.2: Use the encoding module of the U-shaped network and add a feature fusion module to enable the network to learn global features and local features of lung nodules at the same time. The global and local features are concatenated and put into the fully connected layer and the Dropout layer to output the classification results.

[0134] Step 4.3: Use the encoding module structure of the segmentation network trained in step 3 to migrate to the global and local encoding modules in the classification network for iterative training, and finally perform a performance test on the benign and malignant lung nodule classification network.

[0135] The technical solution provided by the above embodiment of the present invention is further described in detail below in conjunction with a specific application example.

[0136] In this specific application example, the public dataset 4D-lung is selected as CT dataset A, and low-dose and full-dose CT data are preprocessed for denoising network training. The dataset contains 5743 slices of 10 patients, and each patient has two images, one normal-dose CT and one low-dose CT, where the CT dose of the low-dose image is one-fourth of the normal dose. The image size of each slice is 512*512, and it is stored in the form of .IMA files.

[0137] In this specific application example, the public dataset LIDC-IDRI is selected as the lung CT dataset B, and the lung CT data is preprocessed for training the segmentation network and the classification network. The dataset contains 1018 lung CT scan images from 1010 patients. The images come from different medical centers, CT equipment and parameters, which can enhance the generalization ability of the network and well simulate the lung images collected by CT in different situations in real scenes. The annotations in the LIDC-IDRI dataset are jointly performed by multiple expert doctors, including the location, diameter, shape, and malignancy probability level of the nodules. The location and shape are given in the form of the coordinates of the outer contour of the nodule, and the malignancy probability score is divided into 1 to 5 levels, 1 is confirmed negative, 3 is uncertain, 5 is confirmed positive, and levels 2 and 4 are transitional levels.

[0138] like Figure 2 As shown, the intelligent detection method for lung nodules in CT images based on transfer learning adopted in this specific application example includes the following steps:

[0139] Step S1, performing preprocessing operations on low-dose and full-dose CT data sets and lung CT data;

[0140] Step S2, based on the RED-CNN architecture, build and train a denoising network, and then test it;

[0141] Step S3, using the encoding module-decoding module structure of the U-shaped network to build a pulmonary nodule segmentation network, and using the transfer learning method to transfer some encoding module parameters of the denoising network for network training and testing;

[0142] Step S4, using the optimized U-shaped network encoding module to build a lung nodule benign and malignant classification network with a feature fusion module, and using the transfer learning method to transfer the trained parameters in the encoding module of the segmentation network for network training and testing.

[0143] In step S1, the specific method for preprocessing low-dose and full-dose CT data is as follows: first, read the image data and non-image data, and match the two to facilitate indexing and finding relevant information. After format conversion, the image data is standardized and normalized to have similar data ranges and distributions. The pixel storage unit is converted according to the specific situation of the image data to better adapt to the training and processing requirements of the network model. Then, the invalid pixels in the image data are set to zero and converted to the background so that their influence can be ignored in subsequent processing. Finally, the data set is divided into: training set and test set.

[0144] In step S1, when preprocessing the lung CT data set, the patient data set is divided into a group with nodules and a group without nodules, and then a binary mask label image is generated using the nodule contour coordinates annotated by the doctor. At the same time, the format of the original data set is converted and denoised through a denoising network. The lung parenchyma is then extracted to avoid irrelevant factors interfering with network training. The lung parenchyma Mask is obtained by mainly using kmeans clustering, erosion expansion, connected region marking and other methods, which is then multiplied with the original image to obtain an image of the lung parenchyma.

[0145] In step S2, the denoising network built based on RED-CNN is divided into two parts: the encoding module and the decoding module. The input format of the network is a four-dimensional tensor of batch size, channels, width, and height. First, it passes through the encoding module, which is used to extract the features of the noisy image. The encoding module consists of five layers of two-dimensional convolution modules, in which the features extracted by the modules are more abstract and advanced. The convolution layer mainly uses a convolution kernel of size 3, and the sliding step size and padding size are set to 1 to keep the feature map size unchanged, so that the image retains more detailed features and reduces information loss. After the middle feature representation is obtained through the encoding module, it is passed through the decoding module, which is used to reconstruct the extracted features. The decoding module also adds residual connections between the symmetrical convolutional layer and the deconvolution layer, and adds the feature map before the convolution layer to the feature after the symmetrical deconvolution layer to prevent network degradation, solve the problem of the network being too deep, and transmit information at the same time, thereby improving the model's ability to restore details and textures.

[0146] Further, in step S2, a denoising network based on RED-CNN is constructed, which is divided into two key parts: the encoding module and the decoding module. The input of the network is in the form of a four-dimensional tensor, which includes batch size, number of channels, width and height. Since the input image used is a grayscale image, the number of channels is set to 1 and the image size is 512x512. First, the features of the noisy image are extracted from the input through the encoding module. The encoding module consists of five two-dimensional convolution modules, which gradually extract more abstract and advanced features. The convolution layer uses a 3x3 convolution kernel with a step size and a padding size of 1, which can keep the size of the feature map unchanged, thereby retaining more detailed features and reducing the risk of information loss. After passing through the encoding module, a most intermediate feature representation is obtained, and then these features are passed to the decoding module. The task of the decoding module is to reconstruct these features to generate a denoised image. In order to maintain the stability of the network and avoid the degradation problem of deep networks, residual connections are introduced between the convolution layer and the deconvolution layer of the decoding module. This connection method adds the feature map before the convolution layer to the feature map after the symmetrical deconvolution layer, effectively transferring information and preventing the network from degrading when it is too deep. This not only helps solve the depth problem of the network, but also improves the model's ability to restore image details and textures. Through this encoding module-decoding module structure and residual connection mechanism, the denoising network can extract key features from noisy images and reconstruct images in a more accurate manner, thereby effectively removing image noise and improving image quality. This network architecture not only helps capture important information in the image, but also provides smoother output results while retaining details.

[0147] In this specific application example, the loss function selected by the denoising network is the mean square error (MSE), and the formula is as follows, where x and y refer to two different samples:

[0148]

[0149] In this specific application example, the trained denoising network achieved good performance. During training, PSNR peak signal-to-noise ratio, SSIM structural similarity, and MSE mean square error were used as evaluation indicators for denoising. Full-dose CT images were used as the standard, and the converged model of 40 training cycles was used for testing. The average PNSR of the noisy image was 29.2489dB, and the average PNSR of the denoised image was 33.0498dB, an increase of 3.8009dB, an increase of about 13%; the average SSIM of the noisy image was 0.8759, and the SSIM of the denoised image was 0.9104, an increase of 0.0345, an increase of about 4%; the MSE of the noisy image was 14.2416, and the MSE of the denoised image was 9.0867, a decrease of 5.1549, an increase of 36.2%. The denoising effect of the trained denoising network on the test set is shown below. Figure 4 shown.

[0150] In step S3, the U-shaped segmentation network is also divided into a symmetrical structure of an encoding module and a decoding module. The U-shaped segmentation network as a whole extracts features through a four-layer downsampling module, and then restores the original image size through a four-layer upsampling module. The encoding module of the segmentation network is responsible for feature extraction, and includes four-layer downsampling modules and a double convolution layer. Each downsampling operation includes a double convolution and a maximum pooling operation. The two convolutions of the double convolution use the same number of convolution kernels, so the feature map of the same depth is output, and the work of compressing the image for downsampling by the encoding module is mainly completed by the maximum pooling layer. The decoding module part of the segmentation network includes a four-layer upsampling module and a final output module. It is worth noting that the unique feature jump connection and encoding and decoding module structure of the U-shaped segmentation network can better focus on the boundary and shape information of the target, and can adapt to the requirements of the segmentation task.

[0151] Specifically, the U-shaped segmentation network is also divided into a symmetrical structure of encoding module and decoding module. The input and denoising network are both in the form of four-dimensional tensors, where channels is also 1, that is, the input image is also a grayscale image, and the image size is also 512*512. The U-shaped segmentation network as a whole extracts features through a four-layer downsampling module, and then restores the original image size through a four-layer upsampling module. The encoding module of the segmentation network is responsible for feature extraction, including four layers of downsampling modules and a double convolution layer. Each downsampling operation includes a double convolution and a maximum pooling operation, and the pooling window size is set to 2, that is, the size is reduced by half each time. The two convolutions of the double convolution use the same number of convolution kernels, so the feature map of the same depth is output, and the work of compressing the image for downsampling in the encoding module is mainly completed by the maximum pooling layer. The decoding module of the segmentation network includes a four-layer upsampling module and a final output module. It is worth noting that the unique feature skip connection and encoding and decoding module structure of the U-shaped segmentation network can better focus on the boundary and shape information of the target, which can adapt to the requirements of the segmentation task.

[0152] Furthermore, in step S3, the double convolution module specifically includes the following operations: the double convolution module has a total of six layers, the feature map extracts features through the convolution layer in turn, enters the BN layer for normalization, enters the ReLU activation function layer to introduce nonlinear transformation, and repeats the above convolution-batch normalization-activation operation. Such six layers are the double convolution module.

[0153] Specifically, the double convolution module has a total of six layers. The feature map is extracted through the convolution layer in turn, enters the BN layer for normalization, enters the ReLU activation function layer to introduce nonlinear transformation, and repeats the above convolution-batch normalization-activation operation. These six layers are the double convolution module.

[0154] Furthermore, in step S3, the upsampling module specifically includes the following operations: each upsampling is first performed by restoring the size through interpolation or deconvolution, and at the same time reducing the depth of the feature map to make it the same as the depth of the feature map extracted by the symmetrical part of the encoding module, and then the feature map of the same size is concatenated with the upsampled feature map to obtain a feature map with double the number of channels, and then the number of channels is compressed through double convolution.

[0155] Specifically, each upsampling first restores the size through interpolation or deconvolution, and at the same time reduces the depth of the feature map to make it the same as the depth of the feature map extracted from the symmetrical part of the encoding module. Then, this part of the feature map of the same size is concatenated with the upsampled feature map to obtain a feature map with twice the number of channels, and then the number of channels is compressed through double convolution.

[0156] Furthermore, in step S3, the design of the network training loss function adopts a mixed strategy of BCE loss and Dice loss. BCE loss mainly considers the independent classification of each pixel. While Dice loss considers the overlap between the prediction result of the entire image and the true label. The combination of the two as loss can better judge the network performance.

[0157] Specifically, the design of the network training loss function adopts a mixed strategy of BCE loss and Dice loss. BCE loss mainly considers the independent classification of each pixel. Dice loss considers the overlap between the prediction result of the entire image and the true label. The combination of the two as loss can better judge the network performance.

[0158] BCE (Binary Cross Entropy) loss, that is, binary cross entropy, is calculated as follows:

[0159] BCEloss=-[yln(p)+(1-y)ln(1-p)]

[0160] Where y is the true label, which can only be 0 or 1, p is the probability value of the predicted output, ranging from 0 to 1, and ln represents the natural logarithm.

[0161] The Dice loss calculation formula is shown in formula (2):

[0162] DICEloss=1-Dice

[0163] The joint loss function obtained on the above basis is shown in formula (3):

[0164] Loss = 0.5.BCEloss+DICEloss

[0165] Furthermore, in step S3, the specific method of using transfer learning to process the U-shaped segmentation network is: migrating the partial coding module structure and trained parameters of the trained denoising network RED-CNN to the partial coding module structure of the U-shaped segmentation network. Since the convolutional layer weights and other parameters of the coding module have been learned in the denoising network in step 2, these parameters are also required in the downsampling module of the coding module in the segmentation network, so the weight migration can be directly performed to simplify the training operation.

[0166] In this specific application example, the U-shaped segmentation network model after 60,000 iterations was used for testing, and the average Dice coefficient was 0.5642, the average IOU score was 0.5019, and the average pixel classification accuracy was 0.9997. The centroid distance threshold was set to 100, and the confusion matrix of the predicted image and the real image is shown in Table 1. According to the confusion matrix, the accuracy rate is 0.639, the precision rate is 0.730, the recall rate is 0.743, and the F1 score is 0.7361.

[0167] Table 1 Split test confusion matrix

[0168]

[0169] In this specific application example, the segmentation performance diagram is as follows: Figure 5 As shown, Figure 5 The input of the segmentation network, the real mask image and the predicted mask image are shown respectively. It can be observed from the effect diagram that the network still has a certain detection ability for very small nodule networks.

[0170] In step S4, the specific method of building a benign and malignant lung nodule classification network is as follows: because there is a certain degree of overlap in the features required for segmentation and classification tasks, and the role of the decoding module is to restore the extracted features to the original image size, and the classification network does not need to output a complete image due to the nature of the task, the decoding module can be discarded, and a dual-channel CNN encoding module architecture is adopted. The global image and the segmented lung nodule lesion part are respectively input as local images. After the features generated by the dual channels are fused using the feature fusion module, a fully connected layer and a Dropout layer are added at the end of the network, and the classification results are output. This architecture avoids overfitting and meets the requirements of the classification task.

[0171] Furthermore, in step S4, the specific method of feature fusion of the classification network is: the original image and the segmented lesion part are respectively input into the classification network, and the global features of the original image and the local features of the lesion part are fused, thereby taking into account the global features of the whole image and the features of the lesion part that need more attention.

[0172] Specifically, since there is a certain degree of overlap in the features required for segmentation and classification tasks, and the role of the decoding module is to restore the extracted features to the original image size, and the classification network does not need to output a complete image due to the nature of the task, the decoding module can be discarded, and the encoding module architecture of the U-shaped network with a reduced number of convolution kernels is adopted to supplement the feature fusion module, and at the same time, a fully connected layer and a Dropout layer are added at the end of the network to avoid overfitting and meet the requirements of the classification task.

[0173] At the same time, the original image and the segmented lesion part are input into the classification network respectively, and the global features of the original image and the local features of the lesion part are fused, thereby taking into account the global features of the whole image and the features of the lesion part that requires more attention.

[0174] Furthermore, in step S4, the specific method of using the transfer learning method to process the pulmonary nodule classification network is: partially migrating the trained segmentation network encoding module to the dual-channel encoding module in the classification network.

[0175] In this specific application example, the classification network trained to convergence is used for testing, and the classification network confusion matrix is ​​shown in Table 2. The top of the table shows the true label value, and the left side shows the predicted value. It can be judged from the table that the classification network module rarely has large judgment errors and has good classification performance.

[0176] Table 2 Classification network test confusion matrix

[0177]

[0178] In this specific application example, the trained parameters of part of the encoding module structure of the trained denoising network RED-CNN are transferred to part of the encoding module structure of the U-shaped segmentation network, and the trained segmentation network encoding module is partially transferred to the classification network. Figure 3 As shown in the figure, since the encoding module belongs to the feature extraction part, the decoding module part has different output types (the output of the denoising network is the original image after denoising, the output of the segmentation network is the segmented Mask image, and the classification network can directly output the category through the fully connected layer from the features, and the decoding module part can be discarded), so only the encoding module part is migrated in steps S3 and S4. In addition, considering that the pooling layer, BN layer and ReLU function have no parameters available for migration, and the denoising and segmentation networks do not have a fully connected layer design, whether it is migrating from the trained denoising network to the segmentation network in step S3, or migrating from the trained segmentation network to the classification network in step S4, the convolutional layer with the same hyperparameter structure of the encoding module part is migrated.

[0179] In this specific application example, when performing transfer learning, the pre-trained model is first imported into the training script, and the parameters of the pre-trained model are loaded, so that the weights of the pre-trained model are used as the initial values ​​during the training process, and then the layers to be transferred are selected and their parameters are assigned to the corresponding target layers in turn. Considering that the features of the three tasks of denoising, segmentation, and classification have similarities and differences, it is chosen not to fix the network so that the performance of the model can be continuously optimized during the training process.

[0180] In this specific application example, the comparison of the two migration operations before and after migration is shown in Table 3. It can be seen from the table that the network convergence speed is accelerated after migration. After migration, the training of the segmentation network is accelerated by 42.8%, and the initial loss value is reduced by 36.3%. After migration, the training of the classification network is accelerated by 20%, and the initial loss value is reduced by 39.7%. Figure 6 and Figure 7 The Loss reduction curves of the segmentation network and the classification network before and after migration are shown respectively. It can be seen from the figure that, whether it is the segmentation network or the classification network, the initial Loss and the Loss of each iteration round after migration are lower than the training Loss when no migration operation is performed. Transfer learning can effectively help the segmentation network and the classification network to iterate quickly to convergence.

[0181] Table 3 Performance of segmentation and classification networks before and after migration

[0182]

[0183] An embodiment of the present invention provides a CT image pulmonary nodule intelligent detection system based on transfer learning, such as Figure 8 As shown, the system may include:

[0184] A data processing module, which is used to obtain a pre-processed CT data set A, wherein the CT data set A includes: low-dose CT data and full-dose CT data, wherein the full-dose CT data includes its segmentation annotation and benign and malignant classification labels; and is also used to obtain a lung CT data set B, and use a lung nodule denoising model to perform denoising on the lung CT data set B to obtain a model training data set;

[0185] A pulmonary nodule denoising model module is used to provide a denoising network, and the low-dose CT data and the full-dose CT data in the CT data set A are used as the input and output of the denoising network. The denoising network is trained and tested to obtain a pulmonary nodule denoising model, which is used to obtain a pulmonary nodule image denoising result.

[0186] A lung nodule segmentation model module is used to provide a U-shaped segmentation network, train the U-shaped segmentation network using a model training data set, and migrate the trained parameters of the encoding module of the lung nodule denoising model to the encoding module of the U-shaped segmentation network to obtain a lung nodule segmentation model, which is used to obtain lung nodule tissue image results;

[0187] A lung nodule classification model module is used to provide a global-local feature fusion CNN network. The model training data set is used to train the global-local feature fusion CNN network, and the trained parameters of the encoding module of the lung nodule segmentation model are migrated to the feature extraction part of the CNN network to obtain a lung nodule classification model. The model is used to obtain the benign and malignant detection results of lung nodules in CT images.

[0188] In the lung nodule segmentation model module and the lung nodule classification model module, the transfer learning operation is specifically described as follows: Since the encoding module is the feature extraction part, and the decoding module part has different output types (the output of the denoising network is the original image after denoising, the output of the segmentation network is the segmented Mask image, and the classification network directly outputs the category through the fully connected layer from the features, and the decoding module part can be discarded), only the encoding module part is migrated in the lung nodule segmentation model module and the lung nodule classification model module. In addition, considering that the pooling layer, BN layer and ReLU function have no parameters available for migration, and the denoising and segmentation networks do not have a fully connected layer design, whether it is migrating from the trained denoising network to the segmentation network in the lung nodule segmentation model module, or migrating from the trained segmentation network to the classification network in the lung nodule classification model module, the convolutional layer with the same hyperparameter structure of the encoding module is migrated. Fig. 9 As shown, it is a working schematic diagram of the intelligent detection system of lung nodules in CT images based on transfer learning; Fig. 9 Among them, module one is the data processing module, module two is the lung nodule denoising model module, module three is the lung nodule segmentation model module, and module four is the lung nodule classification model module.

[0189] In some preferred embodiments, the above-mentioned lung nodule denoising model module inputs the four-dimensional tensor into the RED-CNN denoising network; sends it to the encoding module composed of five layers of two-dimensional convolution modules, which is used to extract the features of the noisy image; sends the intermediate feature representation to the decoding module part with a residual connection deconvolution layer added, which is used to reconstruct the extracted features; the network is iteratively trained, and the performance of the trained denoising network is tested using a test set.

[0190] In some preferred embodiments, the data processing module divides the lung CT data set into a nodule group and a nodule group, generates a mask based on the nodule annotation, performs format conversion, and finally extracts the lung parenchyma;

[0191] In some preferred embodiments, the above-mentioned lung nodule segmentation model module inputs the four-dimensional tensor of the same format into the U-shaped segmentation network in the input link; extracts the input image features through a four-layer downsampling module in the network coding module link; restores the original image size through a four-layer upsampling module in the network decoding module link; and migrates the trained parameters of the encoding module part of the denoising network trained by the lung nodule denoising model module to the segmentation network, followed by iterative training and performance testing.

[0192] In some preferred embodiments, the loss function of the above-mentioned lung nodule segmentation model module is composed of a mixed strategy of BCE loss and Diceloss to improve the network evaluation performance.

[0193] In some preferred embodiments, the above-mentioned lung nodule classification model module uses the data file obtained after preprocessing by the lung nodule segmentation model module, uses a mask to extract the lung nodule area, and extracts the central image of the lung nodule without the lung parenchyma in a unified format, and fuses the local information containing the lung nodules with the original network through the feature fusion module; uses the encoding module part of the U-type network, reduces the number of convolution kernels at the same time, and finally adds a fully connected layer and a Dropout layer to output the classification result; uses the encoding module structure of the segmentation network trained by the lung nodule segmentation model module to migrate to the classification network for iterative training, and finally performs a performance test of the benign and malignant lung nodule classification network.

[0194] It should be noted that the steps in the method provided by the present invention can be implemented by using corresponding modules, devices, units, etc. in the system. Those skilled in the art can refer to the technical solution of the method to realize the composition of the system, that is, the embodiments in the method can be understood as preferred examples for constructing the system, which will not be elaborated here.

[0195] An embodiment of the present invention provides a computer terminal, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it can be used to execute any method of the above-mentioned embodiments of the present invention, or to run any system of the above-mentioned embodiments of the present invention.

[0196] Optionally, the memory is used to store programs; the memory may include volatile memory (English: volatile memory), such as random-access memory (English: random-access memory, abbreviated: RAM), such as static random-access memory (English: static random-access memory, abbreviated: SRAM), double data rate synchronous dynamic random access memory (English: Double Data Rate Synchronous Dynamic Random Access Memory, abbreviated: DDR SDRAM), etc.; the memory may also include non-volatile memory (English: non-volatile memory), such as flash memory (English: flash memory). The memory is used to store computer programs (such as applications, functional modules, etc. that implement the above method), computer instructions, etc., and the above computer programs, computer instructions, etc. can be partitioned and stored in one or more memories. And the above computer programs, computer instructions, data, etc. can be called by the processor.

[0197] The above-mentioned computer programs, computer instructions, etc. may be stored in one or more memories in partitions, and the above-mentioned computer programs, computer instructions, data, etc. may be called by a processor.

[0198] The processor is used to execute the computer program stored in the memory to implement the various steps of the method or various modules of the system involved in the above embodiments. For details, please refer to the relevant descriptions in the above method and system embodiments.

[0199] The processor and the memory may be independent structures or integrated structures. When the processor and the memory are independent structures, the memory and the processor may be coupled and connected via a bus.

[0200] An embodiment of the present invention further provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, it can be used to execute any method of the above embodiments of the present invention, or to run any system of the above embodiments of the present invention.

[0201] The above-mentioned embodiments of the present invention provide an intelligent detection method and system for lung nodules in CT images based on transfer learning. Based on transfer learning, the lung nodule detection algorithm for low-dose lung CT images sequentially completes noise reduction, segmentation, classification network training and testing, realizes end-to-end auxiliary algorithm research, and uses task-to-task transfer learning to reduce the hyperparameters required for training, thereby accelerating the fitting speed, aiming to provide clinicians with more accurate and reliable auxiliary detection tools.

[0202] Those skilled in the art know that, in addition to implementing the system and its various devices provided by the present invention in a purely computer-readable program code, the system and its various devices provided by the present invention can be made to implement the same functions in the form of logic gates, switches, application-specific integrated circuits, programmable logic controllers, and embedded microcontrollers by logically programming the method steps. Therefore, the system and its various devices provided by the present invention can be considered as a hardware component, and the devices for implementing various functions included therein can also be considered as structures within the hardware component; the devices for implementing various functions can also be considered as both software modules for implementing the method and structures within the hardware component.

[0203] All matters not covered in the above embodiments of the present invention are well known in the art.

[0204] The above describes the specific embodiments of the present invention. It should be understood that the present invention is not limited to the above specific embodiments, and those skilled in the art may make various modifications or variations within the scope of the claims, which do not affect the essence of the present invention.

Claims

1. An intelligent detection method for lung nodules in CT images based on transfer learning, It is characterized in that include: Acquire a preprocessed CT data set A, wherein the CT data set A includes: low-dose CT data and full-dose CT data including segmentation annotations and benign and malignant classification labels; Providing a denoising network, taking the low-dose CT data in the CT data set A as the input of the denoising network, taking the full-dose CT data in the CT data set A as the output of the denoising network, training and testing the denoising network to obtain a pulmonary nodule denoising model, which is used to obtain a denoising result of a pulmonary nodule image; Acquire a lung CT data set B, and use the lung nodule denoising model to perform denoising on the lung CT data set B to obtain a model training data set; Providing a U-shaped segmentation network, using the model training data set to train the U-shaped segmentation network, and migrating the trained parameters of the encoding module of the pulmonary nodule denoising model to the encoding module of the U-shaped segmentation network to obtain a pulmonary nodule segmentation model, which is used to obtain a pulmonary nodule tissue image result; A CNN network with global-local feature fusion is provided, the model training data set is used to train the CNN network with global-local feature fusion, and the trained parameters of the encoding module of the lung nodule segmentation model are migrated to the feature extraction part of the CNN network to obtain a lung nodule classification model, which is used to obtain the benign and malignant detection results of lung nodules in CT images.

2. The intelligent detection method for pulmonary nodules in CT images based on transfer learning according to claim 1, It is characterized in that The step of obtaining a preprocessed CT data set A comprises: A CT data set A including low-dose CT data and full-dose CT data including segmentation annotations and benign and malignant classification labels is obtained, and preprocessed; wherein the preprocessing includes: Reading image data and non-image data in the low-dose CT data and the full-dose CT data, and matching them; Performing image format conversion on the image data; Perform pixel storage unit conversion on the image data, and set invalid pixels to zero and convert them into background; Also includes: The preprocessed CT data set A is divided into a training set and a test set.

3. The intelligent detection method for lung nodules in CT images based on transfer learning according to claim 1, It is characterized in that The method provides a denoising network, uses the low-dose CT data and the full-dose CT data in the CT data set A as the input and output of the denoising network, respectively, trains and tests the denoising network, and obtains a pulmonary nodule denoising model, including: A denoising network is provided, the denoising network comprising an encoding module and a decoding module; wherein the encoding module is mainly composed of five two-dimensional convolution modules, and the decoding module comprises a convolution layer, a deconvolution layer and a residual layer connected therebetween; The low-dose CT data in the CT data set A is used as the input of the denoising network in the form of a four-dimensional tensor, and the features of the noisy image are extracted from the low-dose CT data through the encoding module, and an intermediate feature representation is obtained and transmitted to the decoding part; The decoding part reconstructs the intermediate feature representation, adds the feature map before the convolution layer and the feature map after the symmetrical deconvolution layer to generate a denoised image, and uses the full-dose CT data in the CT data set A as the gold standard of the denoising network; The denoising network is iteratively trained, and a test set is used to perform a performance test on the trained denoising network to obtain a pulmonary nodule denoising model.

4. The intelligent detection method for pulmonary nodules in CT images based on transfer learning according to claim 1, It is characterized in that The step of acquiring a lung CT data set B and performing noise reduction processing on the lung CT data set B using the lung nodule noise reduction model to obtain a model training data set includes: Acquire a lung CT data set B, wherein the lung CT data set B includes: lung CT data and corresponding segmentation labels; The lung CT data set B is divided into a nodule group and a nodule group, and a binary mask label image is generated according to the nodule annotation as a label value of the U-shaped segmentation network; Performing image format conversion on the lung CT data set B, and performing noise reduction processing using the lung nodule noise reduction model; Extracting lung parenchyma from the denoised lung CT data B to obtain a lung parenchyma Mask, and then multiplying the lung parenchyma Mask with the original image to obtain a lung parenchyma image; Using the lung parenchyma image to construct a model training data set; The lung CT dataset B may be the same dataset as the CT dataset A or a different dataset.

5. The intelligent detection method for pulmonary nodules in CT images based on transfer learning according to claim 1, It is characterized in that A U-shaped segmentation network is provided, the U-shaped segmentation network is trained using the model training data set, and the trained parameters of the encoding module of the pulmonary nodule denoising model are migrated to the encoding module of the U-shaped segmentation network to obtain a pulmonary nodule segmentation model, including: A U-shaped segmentation network is provided, the U-shaped segmentation network comprising a symmetrical encoding module and a decoding module; wherein: the encoding module compresses an input image through a downsampling layer module, then extracts a feature map of the input image through a double convolution layer and inputs the feature map to the decoding module; the decoding module restores the feature map size through an upsampling layer module, then concatenates the downsampling feature map and the upsampling feature map of the same size, and obtains an output image from the output layer; The U-shaped segmentation network is trained using the model training data set, and then the trained parameters of the encoding module of the pulmonary nodule denoising model are migrated to the encoding module of the U-shaped segmentation network. Finally, iterative training and performance testing are performed to obtain a pulmonary nodule segmentation model.

6. The intelligent detection method for lung nodules in CT images based on transfer learning according to claim 5, It is characterized in that The loss function of the pulmonary nodule segmentation model adopts a mixed loss function of BCE loss and Dice loss, where: L BCE =-∑[y ln(p)+(1-y)ln(1-p)] Loss=λL BCE +L DICE Among them, L BCE is the binary cross entropy loss, y∈{0,1} is the true label value, p∈{0,1} is the model prediction value, L DICE is the Dice loss, and λ∈[0,1] is the weight value between loss functions.

7. The intelligent detection method for lung nodules in CT images based on transfer learning according to claim 1, It is characterized in that The method provides a CNN network with global-local feature fusion, uses the model training data set to train the CNN network with global-local feature fusion, and migrates the trained parameters of the encoding module of the pulmonary nodule segmentation model to the feature extraction part of the CNN network to obtain a pulmonary nodule classification model, including: Providing a global-local feature fusion CNN network, and using the model training data set to train the CNN network; Migrating the encoding module of the pulmonary nodule segmentation model to the feature fusion module of the CNN network to obtain a global feature extraction module and a local feature extraction module of the CNN network; The global feature extraction module and the local feature extraction module are connected in series and then placed into the fully connected layer and the Dropout layer of the CNN network to output the classification result; The encoding module of the pulmonary nodule segmentation model is migrated to the global feature extraction module and the local feature extraction module for iterative training, and a performance test is performed to finally obtain a pulmonary nodule classification model.

8. An intelligent detection system for lung nodules in CT images based on transfer learning. It is characterized in that include: A data processing module, the module is used to obtain a pre-processed CT data set A, the CT data set A includes: low-dose CT data and full-dose CT data including segmentation annotations and benign and malignant classification labels; and is also used to obtain a lung CT data set B, and use a lung nodule denoising model to perform denoising on the lung CT data set B to obtain a model training data set; A pulmonary nodule denoising model module, which is used to provide a denoising network, use the low-dose CT data in the CT data set A as the input of the denoising network, use the full-dose CT data as the output of the denoising network, train and test the denoising network, and obtain a pulmonary nodule denoising model, which is used to obtain a pulmonary nodule image denoising result; A pulmonary nodule segmentation model module, which is used to provide a U-shaped segmentation network, use the model training data set to train the U-shaped segmentation network, and migrate the trained parameters of the encoding module of the pulmonary nodule denoising model to the encoding module of the U-shaped segmentation network to obtain a pulmonary nodule segmentation model, which is used to obtain pulmonary nodule tissue image results; A lung nodule classification model module is used to provide a CNN network with global-local feature fusion, use the model training data set to train the CNN network with global-local feature fusion, and migrate the trained parameters of the encoding module of the lung nodule segmentation model to the feature extraction part of the CNN network to obtain a lung nodule classification model, which is used to obtain the benign and malignant detection results of lung nodules in CT images.

9. A computer terminal comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, It is characterized in that When the processor executes the computer program, it can be used to perform the method described in any one of claims 1 to 7, or to run the system described in claim 8.

10. A computer-readable storage medium having a computer program stored thereon, It is characterized in that When the computer program is executed by a processor, it can be used to perform the method described in any one of claims 1 to 7, or to run the system described in claim 8.

Images