Intelligent detection method and system for lung nodules in CT images based on transfer learning

Abstract

This invention provides an intelligent method and system for detecting lung nodules in CT images based on transfer learning. It trains and tests a denoising network using low-dose and full-dose CT datasets to establish a lung nodule denoising model. Using a lung CT dataset, a U-shaped segmentation network is trained and tested, and a lung nodule segmentation model is established through transfer learning. Based on the coding module structure of the lung nodule segmentation model, a lung nodule classification model is established through transfer learning. This invention comprehensively utilizes transfer learning methods, where some structures and parameters of the segmentation model are transferred from the denoising model, and some structures and parameters of the classification model are transferred from the segmentation model. This invention achieves end-to-end auxiliary algorithm research and uses task-to-task transfer learning to reduce the hyperparameters required for training, thereby accelerating the fitting speed. It aims to provide clinicians with a more accurate and reliable auxiliary detection tool.

Description

Technical Field

[0001] This invention relates to the field of computer-aided detection technology for CT images, specifically to an intelligent detection method and system for lung nodules in CT images based on transfer learning, and also provides a corresponding computer terminal and computer-readable storage medium. Background Technology

[0002] In 2020, the International Agency for Research on Cancer (IARC) reported that lung cancer is the second most common cancer worldwide, accounting for 11.4% of all new cases, and also the leading cause of cancer death, accounting for 18.0% of all cancer deaths. This is largely because early-stage lung cancer often presents with no obvious clinical symptoms, leading to delayed detection and poor treatment outcomes. To detect lung cancer early and develop effective treatment plans, doctors typically rely on lung screening to detect the presence of lung nodules and assess their malignancy. This practice significantly reduces the mortality rate of lung cancer.

[0003] In 2010, Temesguen Messay et al. developed a CAD algorithm for detecting pulmonary nodules in CT images. This algorithm primarily uses thresholding to extract preliminary lung regions and applies segmentation refinement to achieve lung parenchyma segmentation. Then, intensity thresholding is combined with morphological processing for segmentation. Finally, a sequential forward selection technique is used to select the optimal feature subset from 245 features and classify based on these features (Messay, Temesguen, Russell C. Hardie, and Steven K. Rogers. "A new computationally efficient CAD system for pulmonary nodule detection in CT imagery." Medical image analysis is 14.3(2010):390-406.). In 2014, Colin Jacobs et al. studied an automatic detection CAD algorithm for subsolid pulmonary nodules, which have a higher probability of malignancy than solid pulmonary nodules. They segmented and classified subsolid pulmonary nodules by manually defining a set of 128 features, such as intensity, shape, texture, and newly defined contextual features. They used a double-threshold density mask method to select target nodules, generating clustered regions. Then, they applied an effective combination of morphological operations to achieve accurate nodule segmentation. Finally, they compared the performance using single-stage and two-stage classification schemes (Jacobs, Colin, et al. "Automatic detection of subsolid pulmonary nodules in thoracic computed tomography images." Medical image analysis 18.2(2014):374-384.). This shows that traditional CAD algorithms for pulmonary nodule detection mainly segment CT images using thresholding and morphological methods, typically transitioning from coarse to fine segmentation. For classification, they primarily use manually defined features, then select the best features for classification using a classifier. Traditional methods require researchers to have a high level of relevant professional knowledge, involve a large amount of manual computation, have low versatility, and require re-selection of feature sets for similar tasks.

[0004] Since the introduction of AlexNet in 2012, deep learning technology has achieved overwhelming dominance in the field of computer vision, enabling end-to-end object detection and classification. Deep learning models can learn features and patterns in lung images and extract and analyze information such as the shape, size, and location of nodules. This provides crucial auxiliary information for doctors, helping to improve the accuracy of their assessment of the benign or malignant nature of lung nodules. This, in turn, helps doctors develop more appropriate treatment plans for patients and also improves the efficiency of their examinations. Using deep learning methods to replace traditional methods for automated and intelligent detection has promising development prospects and has already attracted widespread attention.

[0005] In the field of deep learning applied to medical image processing, there have been some classic cases that have significantly improved diagnostic and treatment efficiency. In terms of image denoising, Eunhee et al. proposed an algorithm using CNN in 2016, applied to the wavelet transform coefficients of low-dose CT images, and used a residual learning architecture in the network (Kang, Eunhee, Junhong Min, and Jong Chul Ye. A deep convolutional neural network using directional wavelets for low-dose X-ray CT reconstruction. Medical physics 44.10(2017):e360-e375.). In 2017, H. Chen et al. proposed a residual encoder-decoder convolutional neural network (RED-CNN), which combines residuals with an encoder-decoder architecture, achieving good results in preserving image edge structures (Chen, Hu, et al. "Low-dose CT with aresidual encoder-decoder convolutional neural network." IEEE Transactions on Medical Imaging 36.12(2017):2524-2535.). In medical segmentation, U-Net plays a crucial role in biomedical segmentation, and to this day, the U-shaped end-to-end network remains one of the most popular deep learning methods.

[0006] However, deep learning currently faces some challenges and limitations in lung nodule detection: deep learning network models typically have a large number of weight parameters, and single-task networks often neglect the correlation between upstream and downstream tasks. Multi-objective tasks in medicine often have coherence, with each part of the trained network extracting numerous common features, resulting in many similar weight parameters in the trained networks. Retraining from scratch in this case would be time-consuming and wasteful. To address this issue, transfer learning can be introduced. By transferring network weight parameters, new networks with similar objectives can be trained on top of previous networks, thus saving time and training costs and accelerating network convergence. In terms of transfer learning, in 2021, Tang Siyuan et al. used 3DCNN and a weight-based transfer learning method to transfer the weights of the feature extraction layer pre-trained on the LUNA16 dataset to the network as initial weights, and then fine-tuned the training on the new dataset, obtaining better results than before (Tang Siyuan, Liu Yanru, and Yang Min. Detection of lung nodules based on transfer learning and 3D convolutional neural network. Chinese Journal of Medical Imaging Technology 36.12(2020):1882-1886.). In 2020, Chen Daozheng et al. used pre-trained models such as VGG16, VGG19, and ResNet50 on the large image dataset ImageNet, froze the convolutional layers, fine-tuned some network layers and adjusted hyperparameters, achieving a significant reduction in training time (Chen Daozheng, and Jiang Qian. Detection of lung nodules based on convolutional neural network and transfer learning. Computer Engineering and Design 42.1(2021):240-247.).

[0007] The aforementioned work on transfer learning also has certain limitations. Currently, there is a lack of research on a comprehensive diagnostic method and architecture for lung nodules, and there is an urgent need for a method that organically integrates existing intelligent processing steps. Summary of the Invention

[0008] To address the aforementioned shortcomings in the prior art, this invention provides an intelligent detection method and system for lung nodules in CT images based on transfer learning, along with a corresponding computer terminal and computer-readable storage medium.

[0009] According to one aspect of the present invention, a method for intelligent detection of lung nodules in CT images based on transfer learning is provided, comprising:

[0010] Obtain a preprocessed CT dataset A, which includes: low-dose CT data and full-dose CT data containing segmentation annotations and benign / malignant classification labels;

[0011] A denoising network is provided, which takes low-dose CT data from the CT dataset A as input and full-dose CT data from the CT dataset A as output. The denoising network is trained and tested to obtain a lung nodule denoising model, which is used to obtain lung nodule image denoising results.

[0012] Obtain a lung CT dataset B, and use the lung nodule denoising model to denoise the lung CT dataset B to obtain the model training dataset;

[0013] A U-shaped segmentation network is provided. The U-shaped segmentation network is trained using the model training dataset. The trained parameters of the encoding module of the lung nodule denoising model are transferred to the encoding module of the U-shaped segmentation network to obtain a lung nodule segmentation model. This model is used to obtain lung nodule tissue image results.

[0014] A global-local feature fusion CNN network is provided. The global-local feature fusion CNN network is trained using the model training dataset. The trained parameters of the encoding module of the lung nodule segmentation model are transferred to the feature extraction part of the CNN network to obtain a lung nodule classification model. This model is used to obtain the benign and malignant detection results of lung nodules in CT images and to perform intelligent detection of lung nodules in CT images.

[0015] Preferably, obtaining a preprocessed CT dataset A includes:

[0016] A CT dataset A is acquired, comprising low-dose CT data and full-dose CT data including segmentation annotations and benign / malignant classification labels, and preprocessed; the preprocessing includes:

[0017] The image and non-image data from the low-dose CT data and the full-dose CT data are read and matched.

[0018] The image data is converted to an image format.

[0019] The image data is converted to pixel storage units, and invalid pixels are set to zero and converted to background.

[0020] Preferably, the method further includes: dividing the preprocessed CT dataset A into a training set and a test set.

[0021] Preferably, the step of providing a noise reduction network involves using low-dose CT data and full-dose CT data from the CT dataset A as the input and output of the noise reduction network, respectively, and training and testing the noise reduction network to obtain a lung nodule noise reduction model, including:

[0022] A noise reduction network is provided, the noise reduction network including an encoding module and a decoding module; wherein, the encoding module mainly consists of five two-dimensional convolutional modules, and the decoding module includes a convolutional layer, a deconvolutional layer and a residual layer connected between the two;

[0023] The low-dose CT data in the CT dataset A is used as the input of the denoising network in the form of a four-dimensional tensor. The encoding module extracts the features of the noisy image from the low-dose CT data and obtains an intermediate feature representation, which is then passed to the decoding part.

[0024] The decoding part reconstructs the intermediate feature representation by adding the feature map before the convolutional layer to the feature map after the symmetrical deconvolutional layer to generate a denoised image, and uses the full-dose CT data in the CT dataset A as the gold standard of the denoising network.

[0025] The denoising network is iteratively trained, and the performance of the trained denoising network is tested using a test set to obtain a lung nodule denoising model.

[0026] Preferably, the step of acquiring a lung CT dataset B, and then using the lung nodule denoising model to denoise the lung CT dataset B to obtain a model training dataset includes:

[0027] Obtain a lung CT dataset B, which includes: lung CT data and corresponding segmentation labels;

[0028] The lung CT dataset B is divided into a nodule group and a nodule-free group, and a binary mask label image is generated based on the nodule annotation, which serves as the label value of the U-shaped segmentation network.

[0029] The lung CT dataset B is converted to an image format and then denoised using the lung nodule denoising model.

[0030] Lung parenchyma is extracted from the noise-reduced lung CT data to obtain a lung parenchyma mask. Then, the lung parenchyma mask is multiplied with the original image to obtain a lung parenchyma image.

[0031] The lung parenchyma images were used to construct the model training dataset.

[0032] Preferably, the lung CT dataset B uses the same dataset as the CT dataset A or a different dataset.

[0033] Preferably, a U-shaped segmentation network is provided, which is trained using the model training dataset, and the trained parameters of the encoding module of the lung nodule denoising model are transferred to the encoding module of the U-shaped segmentation network to obtain the lung nodule segmentation model, including:

[0034] A U-shaped segmentation network is provided, comprising a symmetrical encoding module and a decoding module; wherein: the encoding module compresses the input image through a downsampling layer module, then extracts the feature map of the input image through a double convolutional layer and inputs it to the decoding module; the decoding module restores the feature map size through an upsampling layer module, then concatenates the downsampling feature map and the upsampling feature map of the same size to obtain the output image from the output layer;

[0035] The U-shaped segmentation network is trained using the model training dataset. Then, the trained parameters of the encoding module of the lung nodule denoising model are transferred to the encoding module of the U-shaped segmentation network. Finally, iterative training and performance testing are performed to obtain the lung nodule segmentation model.

[0036] Preferably, the loss function of the lung nodule segmentation model adopts a hybrid loss function of BCE loss and Dice loss, wherein:

[0037] L BcE =-∑[yln(p)+(1-y)ln(1-p)]

[0038] L DICE =1–DICE

[0039] Loss=λL BCE +L DICE

[0040] Among them, L BCE Let y be the BCE loss, p be the value of L. DICE Let Dice los be DICE, DICE be λ be λ.

[0041] Preferably, the provision of a global-local feature fusion CNN network, training the global-local feature fusion CNN network using the model training dataset, and transferring the trained parameters of the encoding module of the lung nodule segmentation model to the feature extraction part of the CNN network to obtain a lung nodule classification model, includes:

[0042] A global-local feature fusion CNN network is provided, and the CNN network is trained using the model training dataset.

[0043] The encoding module of the lung nodule segmentation model is transferred to the feature fusion module of the CNN network to obtain the global feature extraction module and the local feature extraction module of the CNN network.

[0044] The global feature extraction module and the local feature extraction module are concatenated and then fed into the fully connected layer and Dropout layer of the CNN network to output the classification result;

[0045] The encoding module of the lung nodule segmentation model is migrated to the global feature extraction module and the local feature extraction module for iterative training, and performance testing is performed to finally obtain the lung nodule classification model.

[0046] According to another aspect of the present invention, a CT image lung nodule intelligent detection system based on transfer learning is provided, comprising:

[0047] The data processing module is used to acquire a preprocessed CT dataset A, which includes low-dose CT data and full-dose CT data containing segmentation annotations and benign / malignant classification labels; it is also used to acquire a lung CT dataset B, and to perform noise reduction processing on the lung CT dataset B using a lung nodule noise reduction model to obtain a model training dataset.

[0048] A lung nodule denoising model module is provided. This module is used to provide a denoising network. The low-dose CT data in the CT dataset A is used as the input of the denoising network, and the full-dose CT data in the CT dataset A is used as the output of the denoising network. The denoising network is trained and tested to obtain a lung nodule denoising model. This model is used to obtain the denoising results of lung nodule images.

[0049] A lung nodule segmentation model module is provided, which is used to provide a U-shaped segmentation network. The U-shaped segmentation network is trained using the model training dataset, and the trained parameters of the encoding module of the lung nodule denoising model are transferred to the encoding module of the U-shaped segmentation network to obtain the lung nodule segmentation model. This model is used to obtain lung nodule tissue image results.

[0050] A lung nodule classification model module is provided, which is used to provide a global-local feature fusion CNN network. The global-local feature fusion CNN network is trained using the model training dataset, and the trained parameters of the encoding module of the lung nodule segmentation model are transferred to the feature extraction part of the CNN network to obtain the lung nodule classification model. This model is used to obtain the benign and malignant detection results of lung nodules in CT images.

[0051] According to a third aspect of the present invention, a computer terminal is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it can be used to perform the method described in any one of the above-described inventions, or to run the system described in any one of the above-described inventions.

[0052] According to a fourth aspect of the present invention, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, can be used to perform the method described in any one of the preceding claims of the present invention, or to run the system described in any one of the preceding claims of the present invention.

[0053] By adopting the above technical solution, the present invention has at least one of the following beneficial effects compared with the prior art:

[0054] The present invention provides an intelligent detection method and system for lung nodules in CT images based on transfer learning. The RED-CNN noise reduction network is trained to process CT image datasets, which can reduce existing noise in the images and minimize feature loss in clean images. The dense structure and residual connections of RED-CNN can better pay attention to and preserve the detailed information of the images, which is beneficial for subsequent lung nodule segmentation and benign / malignant classification tasks, thereby improving the overall detection performance of the system.

[0055] The present invention provides an intelligent detection method and system for lung nodules in CT images based on transfer learning. The method trains a U-shaped segmentation network to segment lung nodule regions in CT images. The U-shaped encoding-decoding module architecture can extract features through a multi-layer downsampling module and then restore the original image size through a multi-layer upsampling module. In addition, the feature skip connections used can better focus on the boundary and shape information of the target, thereby improving the segmentation performance.

[0056] The present invention provides an intelligent detection method and system for lung nodules in CT images based on transfer learning. It constructs and trains a classification network for benign and malignant lung nodules using an optimized U-shaped segmentation network encoding module combined with a feature fusion module. Adding a Dropout layer effectively prevents overfitting during network training, while reducing the number of convolutional kernels decreases the depth of feature maps and the complexity of model parameters. The feature fusion module allows the classification network to simultaneously consider both global and local features. The resulting classification is a comprehensive result of the original image's full-image features, as well as a concentrated feedback of the lesion sites obtained after segmentation, thus improving the accuracy and robustness of the lung nodule benign and malignant classification network.

[0057] The intelligent detection method and system for lung nodules in CT images based on transfer learning provided by this invention employs a transfer learning method, namely, the transfer operation from the trained parameters of the denoising network to the segmentation network and the transfer operation from the trained parameters of the segmentation network encoding module to the classification network, which effectively reduces training costs and improves training efficiency.

[0058] The present invention provides a method and system for intelligent detection of lung nodules in CT images based on transfer learning. The method adopts a model-based transfer learning approach, which directly transfers some layers and parameters of the network model in the previously completed task. This eliminates the need to train the network from scratch, reduces training costs, improves training efficiency, reduces the initial loss value, thereby improving the convergence speed of the network and optimizing model training.

[0059] The present invention provides a method and system for intelligent detection of lung nodules in CT images based on transfer learning. It uses the features and parameters obtained in the previous task to guide the construction and training of the subsequent network, thereby reducing the need for training data and reducing the training time and the total number of model parameters. Attached Figure Description

[0060] Other features, objects, and advantages of the present invention will become more apparent from the following detailed description of non-limiting embodiments with reference to the accompanying drawings:

[0061] Figure 1 This is a flowchart illustrating the workflow of an intelligent lung nodule detection method based on transfer learning in CT images according to an embodiment of the present invention.

[0062] Figure 2 This is a flowchart illustrating the workflow of a preferred embodiment of the intelligent detection method for lung nodules in CT images based on transfer learning.

[0063] Figure 3 This is a schematic diagram illustrating the operation of transfer learning in a preferred embodiment of the present invention.

[0064] Figure 4 This is a diagram showing the noise reduction effect of the trained noise reduction network on the test set in a preferred embodiment of the present invention.

[0065] Figure 5 This is a schematic diagram of the segmentation performance of the segmentation network after training in a preferred embodiment of the present invention.

[0066] Figure 6 This is a graph showing the decrease in loss before and after the segmentation network migration in a preferred embodiment of the present invention.

[0067] Figure 7 This is a graph showing the decrease in loss before and after the classification network migration in a preferred embodiment of the present invention.

[0068] Figure 8 This is a schematic diagram of the constituent modules of an intelligent lung nodule detection system based on transfer learning in CT images according to an embodiment of the present invention.

[0069] Figure 9 This is a schematic diagram of the operation of the intelligent lung nodule detection system based on transfer learning in CT images according to a preferred embodiment of the present invention. Detailed Implementation

[0070] The embodiments of the present invention are described in detail below: These embodiments are implemented based on the technical solution of the present invention, and provide detailed implementation methods and specific operation processes. It should be noted that those skilled in the art can make several modifications and improvements without departing from the concept of the present invention, and these all fall within the protection scope of the present invention.

[0071] One embodiment of the present invention provides an intelligent detection method for lung nodules in CT images based on transfer learning. This method replaces the traditional method with a deep learning method to achieve automated and intelligent detection of lung nodules in CT images, thereby improving the accuracy of doctors in judging the benign or malignant nature of lung nodules and helping doctors to develop more appropriate medical plans for patients, thus improving detection efficiency.

[0072] like Figure 1 As shown, this embodiment provides an intelligent detection method for lung nodules in CT images based on transfer learning. This method may include the following operations:

[0073] S100, Obtain a preprocessed CT dataset A, which includes: low-dose CT data and full-dose CT data containing segmentation annotations and benign / malignant classification labels;

[0074] S200 provides a denoising network. Low-dose CT data and full-dose CT data in CT dataset A are used as the input and output of the denoising network, respectively. The denoising network is trained and tested to obtain a lung nodule denoising model. This model is used to obtain the denoising results of lung nodule images.

[0075] S300 uses a lung nodule denoising model to denoise the CT dataset to obtain the model training dataset;

[0076] S400 provides a U-shaped segmentation network. The U-shaped segmentation network is trained using a model training dataset. The trained parameters of the encoding module of the lung nodule denoising model are transferred to the encoding module of the U-shaped segmentation network to obtain the lung nodule segmentation model. This model is used to obtain lung nodule tissue image results.

[0077] The S500 provides a global-local feature fusion CNN network. The global-local feature fusion CNN network is trained using the model training dataset, and the trained parameters of the encoding module of the lung nodule segmentation model are transferred to the feature extraction part of the CNN network to obtain a lung nodule classification model. This model is used to obtain the benign and malignant detection results of lung nodules in CT images and to perform intelligent detection of lung nodules in CT images.

[0078] In some preferred embodiments, S100 above, obtaining a preprocessed CT dataset A, includes:

[0079] S101, acquire a CT dataset including low-dose CT data and full-dose CT data containing segmentation annotations and benign / malignant classification labels, and perform preprocessing; preprocessing includes:

[0080] S1011, Read the image data and non-image data from low-dose CT data and full-dose CT data, and match them;

[0081] S1012 performs image format conversion on the image data.

[0082] S1013 performs pixel storage unit conversion on the image data and sets invalid pixels to zero to convert them into background.

[0083] In some preferred embodiments, the above-mentioned S100 further includes:

[0084] S102, the preprocessed CT dataset A is divided into training set and test set.

[0085] In some preferred embodiments, S200 above provides a denoising network, using low-dose CT data and full-dose CT data from CT dataset A as the input and output of the denoising network, respectively, to train and test the denoising network, thereby obtaining a lung nodule denoising model, including:

[0086] S201 provides a noise reduction network, which includes an encoding module and a decoding module; the encoding module mainly consists of five two-dimensional convolutional modules, and the decoding module includes convolutional layers, deconvolutional layers, and residual layers connected between the two.

[0087] S202, the low-dose CT data in CT dataset A is used as the input of the denoising network in the form of a four-dimensional tensor. Through the encoding module, the features of the noisy image are extracted from the low-dose CT data, and an intermediate feature representation is obtained and passed to the decoding part.

[0088] S203, the decoding part reconstructs the intermediate feature representation, adds the feature map before the convolutional layer to the feature map after the symmetrical deconvolutional layer to generate the denoised image, and uses full-dose CT data in the CT dataset as the gold standard for the denoising network.

[0089] S204. The noise reduction network is iteratively trained, and the performance of the trained noise reduction network is tested using a test set to obtain the lung nodule noise reduction model.

[0090] In some preferred embodiments, in step S300 above, a lung CT dataset B is acquired, and a lung nodule denoising model is used to denoise the lung CT dataset B to obtain a model training dataset, including:

[0091] S301, Obtain a lung CT dataset B, which includes: lung CT data and corresponding segmentation labels;

[0092] S302, divide the lung CT dataset B into nodule group and nodule-free group, and generate a binary mask label image based on the nodule annotation, which serves as the label value for the U-shaped segmentation network;

[0093] S303 performs image format conversion on lung CT dataset B and then performs noise reduction processing using a lung nodule noise reduction model.

[0094] S304 extracts lung parenchyma from the denoised lung CT data to obtain the lung parenchyma Mask, and then multiplies the lung parenchyma Mask with the original image to obtain the lung parenchyma image.

[0095] S305 uses lung parenchyma images to construct the model training dataset.

[0096] In some preferred embodiments, the lung CT dataset B can be the same dataset as CT dataset A, or it can be a different dataset.

[0097] In some preferred embodiments, S400 above provides a U-shaped segmentation network, which is trained using a model training dataset. The trained parameters of the encoding module of the lung nodule denoising model are then transferred to the encoding module of the U-shaped segmentation network to obtain the lung nodule segmentation model, including:

[0098] S401 provides a U-shaped segmentation network, which includes a symmetrical encoding module and a decoding module; wherein: the encoding module compresses the input image through a downsampling layer module, then extracts the feature map of the input image through a double convolutional layer and inputs it into the decoding module; the decoding module restores the feature map size through an upsampling layer module, and then concatenates the downsampling feature map and the upsampling feature map of the same size to obtain the output image from the output layer;

[0099] S402, the U-shaped segmentation network is trained using the model training dataset, and then the trained parameters of the encoding module of the lung nodule denoising model are transferred to the encoding module of the U-shaped segmentation network. Finally, iterative training and performance testing are performed to obtain the lung nodule segmentation model.

[0100] In some preferred embodiments, the loss function of the lung nodule segmentation model adopts a hybrid loss function of BCE loss and Dice loss, wherein:

[0101] L BCE =-∑[yln(p)+(1-y)ln(1-p)]

[0102]

[0103] Loss=λL BCE +L DICE

[0104] Among them, L BCE The loss is a binary cross-entropy loss, where y∈{0,1} is the true label value, p∈{0,1} is the model prediction value, and L DICE The loss is Dice loss, and λ∈[0,1] is the weight value that measures the loss function. In a preferred embodiment, it is taken as 0.5.

[0105] In a preferred embodiment, S500 above provides a global-local feature fusion CNN network, which is trained using a model training dataset. The trained parameters of the encoding module of the lung nodule segmentation model are then transferred to the feature extraction part of the CNN network to obtain a lung nodule classification model, including:

[0106] S501 provides a global-local feature fusion CNN network and trains the CNN network using a model training dataset;

[0107] S502, the encoding module of the lung nodule segmentation model is transferred to the feature fusion module of the CNN network to obtain the global feature extraction module and local feature extraction module of the CNN network;

[0108] S503 connects the global feature extraction module and the local feature extraction module in series and puts them into the fully connected layer and Dropout layer of the CNN network to output the classification results;

[0109] S504, the encoding module of the lung nodule segmentation model is migrated to the global feature extraction module and the local feature extraction module for iterative training, and performance testing is performed to finally obtain the lung nodule classification model.

[0110] The technical solution provided by the above embodiments of the present invention will be further described in detail below with reference to a preferred embodiment.

[0111] like Figure 2 As shown, the preferred embodiment of the intelligent detection method for lung nodules in CT images based on transfer learning includes the following steps:

[0112] Step 1: CT dataset reading and preprocessing, where the CT dataset includes low-dose CT data, full-dose CT data, and lung CT dataset; the CT dataset and lung CT dataset can use the same dataset.

[0113] Step 2: The low-dose and full-dose lung CT data preprocessed in Step 1 are fed into the denoising network as input and output respectively to obtain the trained lung nodule denoising model and the lung nodule image denoising result.

[0114] Step 3: Use the denoising network trained in Step 2 to denoise the lung CT dataset, train the U-shaped segmentation network, and adopt the transfer learning method of transferring the trained parameters of some coding module structures in the denoising model in Step 2 to some coding module structures in the U-shaped segmentation network during training. Finally, perform lung nodule segmentation test.

[0115] Step 4: Using the same lung CT dataset as in Step 3, train a global-local feature fusion CNN network. During training, the segmentation network encoding module trained in Step 3 is partially transferred to the feature extraction part of the classification network. The classification performance of benign and malignant lung nodules is tested using global images and lung nodule tissue images.

[0116] In a preferred embodiment, step 1 includes, but is not limited to, preprocessing operations such as reading low-dose CT data and full-dose CT data, converting the data into image formats, extracting task-related regions, and dividing the dataset.

[0117] In a preferred embodiment, step 2 further includes the following steps:

[0118] Step 2.1: Input the four-dimensional tensor into the RED-CNN denoising network;

[0119] Step 2.2: Feed the tensor into the encoding module consisting of five layers of two-dimensional convolutional modules to extract features from the noisy image;

[0120] Step 2.3: The intermediate feature representation is fed into the decoding module with added residual connection deconvolution layers to reconstruct the extracted features; full-dose CT data is used as the gold standard for the noise reduction network.

[0121] Step 2.4: Iteratively train the network and use the test set to test the performance of the trained denoising network.

[0122] In a preferred embodiment, step 3 further includes the following steps:

[0123] Step 3.1: Divide the lung CT dataset into nodule group and nodule-free group, generate a mask based on nodule annotation, perform format conversion, and finally extract lung parenchyma;

[0124] Step 3.2: In the input stage, a four-dimensional tensor of the same format is input into the U-shaped segmentation network. The same main idea of ​​the encoding module and decoding module is used. First, the features are extracted and then the original image size is restored to obtain the lung nodule segmentation result.

[0125] Step 3.3: In the network coding module stage, the image is compressed by performing operations such as feature extraction, normalization, and nonlinear transformation through a four-layer downsampling module, and then the features of the input image are extracted through a double convolutional layer.

[0126] Step 3.4: In the network decoding module stage, the original image size is restored through a four-layer upsampling module. This process involves reducing the depth of the feature map, stitching together feature maps of the same size and upsampled feature maps, compressing the number of channels using double convolution, and finally repeating this process four times to obtain the output image from the output layer.

[0127] Step 3.5: Transfer the trained parameters of the encoding module of the denoising network trained in Step 2 to the encoding module of the segmentation network, and then perform iterative training and performance testing.

[0128] In a preferred embodiment, the loss function used in step 3 employs a hybrid strategy of BCE loss and Dice loss to improve network evaluation performance. The formulas for BCE loss and Dice loss are shown below:

[0129] L BCE =-∑[yln(p)+(1-y)ln(1-p)]

[0130]

[0131] Loss=λL BCE +L DICE In a preferred embodiment, step 4 further includes the following steps:

[0132] Step 4.1: Using the data file obtained after preprocessing in Step 3.1, use a mask to extract the lung nodule region and extract the central image of the lung nodule with lung parenchyma removed in a uniform 64*64 format;

[0133] Step 4.2: Using the encoding module of the U-shaped network, add a feature fusion module so that the network can learn global features and local features of lung nodules at the same time. The obtained global and local features are concatenated and put into a fully connected layer and a Dropout layer to output the classification result.

[0134] Step 4.3: Use the encoding module structure of the segmentation network trained in Step 3 to transfer to the global and local encoding modules in the classification network for iterative training, and finally perform performance testing on the lung nodule benign and malignant classification network.

[0135] The technical solution provided by the above embodiments of the present invention will be further described in detail below with reference to a specific application example.

[0136] In this specific application example, the publicly available dataset 4D-lung was selected as CT dataset A. Preprocessing operations were performed on low-dose and full-dose CT data for training the denoising network. This dataset contains 5743 slices from 10 patients. Each patient's image consists of two copies: a normal-dose CT image and a low-dose CT image, with the low-dose image containing one-quarter of the normal-dose CT dose. Each slice image is 512*512 pixels and stored as an .IMA file.

[0137] In this specific application example, the publicly available dataset LIDC-IDRI was selected as the lung CT dataset B. Preprocessing of the lung CT data was performed on the data for training the segmentation and classification networks. This dataset contains 1018 lung CT scan images from 1010 patients. The images originate from different medical centers, CT equipment, and parameters, enhancing the network's generalization ability and effectively simulating lung images acquired by CT under different real-world conditions. The annotations in the LIDC-IDRI dataset were collaboratively performed by multiple expert physicians, including information such as nodule location, diameter, shape, and malignancy probability level. Location and shape are provided by giving the coordinates of the nodule's outer contour. The malignancy probability score is divided into 1 to 5 levels, with 1 being definitively negative, 3 being uncertain, and 5 being definitively positive. Levels 2 and 4 are considered transitional levels.

[0138] like Figure 2 As shown, the intelligent detection method for lung nodules in CT images based on transfer learning used in this specific application example includes the following steps:

[0139] Step S1: Preprocess the low-dose and full-dose CT datasets and lung CT data;

[0140] Step S2: Based on the RED-CNN architecture, construct and train the denoising network, and then test it;

[0141] Step S3: Construct a lung nodule segmentation network using the encoder-decoder structure of a U-shaped network, and use transfer learning to transfer some encoder module parameters of the noise reduction network for network training and testing.

[0142] Step S4: Use the optimized U-shaped network's encoding module to build a lung nodule benign / malignant classification network with a feature fusion module, and use transfer learning to transfer the trained parameters in the encoding module of the segmentation network for network training and testing.

[0143] In step S1, the specific method for preprocessing low-dose and full-dose CT data is as follows: First, image data and non-image data are read and matched to facilitate indexing and finding relevant information. After format conversion, the image data is standardized and normalized to ensure similar data range and distribution. Pixel storage units are converted according to the specific characteristics of the image data to better suit the training and processing needs of the network model. Invalid pixels in the image data are then zeroed out and converted to background so that their influence can be ignored in subsequent processing. Finally, the dataset is divided into a training set and a test set.

[0144] In step S1, when preprocessing the lung CT dataset, the patient dataset is divided into groups with nodules and groups without nodules. Then, a binary mask label image is generated using the nodule contour coordinates annotated by the doctor. At the same time, the format of the original dataset is converted and denoised by a denoising network. Then, lung parenchyma is extracted to avoid irrelevant factors interfering with network training. The lung parenchyma mask is obtained mainly by using methods such as k-means clustering, erosion dilation, and connected component labeling. The mask is then multiplied with the original image to obtain the lung parenchyma image.

[0145] In step S2, the denoising network built based on RED-CNN is divided into two parts: an encoding module and a decoding module. The network input format is a four-dimensional tensor with batch size, channels, width, and height. First, it passes through the encoding module, which extracts features from the noisy image. The encoding module consists of five two-dimensional convolutional layers, where the features extracted by later layers are increasingly abstract and advanced. The convolutional layers primarily use kernels of size 3, with a stride and padding size both set to 1 to maintain the feature map size, thus preserving more detailed features and reducing information loss. After the encoding module obtains the most central feature representation, it passes through the decoding module, which reconstructs the extracted features. The decoding module also adds residual connections between the symmetrical convolutional and deconvolutional layers, adding the feature map before the convolutional layer to the feature map after the symmetrical deconvolutional layer. This prevents network degradation, addresses the problem of excessive network depth, and simultaneously transmits information, thereby improving the model's ability to recover details and textures.

[0146] Further, in step S2, a denoising network based on RED-CNN is constructed, divided into two key parts: an encoding module and a decoding module. The input of this network is in the form of a four-dimensional tensor, including batch size, number of channels, width, and height. Since the input image used is a grayscale image, the number of channels is set to 1, and the image size is 512x512. First, the encoding module extracts features of the noisy image from the input. The encoding module consists of five two-dimensional convolutional modules, which progressively extract more abstract and high-level features. The convolutional layers use 3x3 kernels with a stride and padding size of 1, which keeps the feature map size constant, thus preserving more detailed features and reducing the risk of information loss. After passing through the encoding module, an intermediate feature representation is obtained, and these features are then passed to the decoding module. The task of the decoding module is to reconstruct these features to generate the denoised image. To maintain the stability of the network and avoid the degradation problem of deep networks, residual connections are introduced between the convolutional and deconvolutional layers of the decoding module. This connection method adds the feature map before the convolutional layer to the feature map after the symmetrical deconvolutional layer, effectively transferring information and preventing network degradation when the network is too deep. This not only helps solve the problem of network depth but also improves the model's ability to recover image details and textures. Through this encoder-decoder module structure and residual connection mechanism, the denoising network can extract key features from noisy images and reconstruct the image more accurately, thereby achieving effective removal of image noise and improvement of image quality. This network architecture not only helps capture important information in the image but also provides smoother output results while preserving details.

[0147] In this specific application example, the loss function selected by the denoising network is the mean squared error (MSE), as shown in the following formula, where x and y represent two different samples:

[0148]

[0149] In this specific application example, the trained denoising network achieved good performance. During training, three metrics—PSNR (Peak Signal-to-Noise Ratio), SSIM (Structural Similarity to Image) and MSE (Mean Squared Error)—were used simultaneously as evaluation metrics for denoising. Using full-dose CT images as the standard, a converged model was tested after 40 training epochs. The average PSNR for noisy images was 29.2489 dB, while the average PSNR for denoised images was 33.0498 dB, an increase of 3.8009 dB, or approximately 13%. The average SSIM for noisy images was 0.8759, while the SSIM for denoised images was 0.9104, an increase of 0.0345, or approximately 4%. The MSE for noisy images was 14.2416, while the MSE for denoised images was 9.0867, a decrease of 5.1549, or 36.2%. The denoising performance of the trained denoising network on the test set is shown below. Figure 4 As shown.

[0150] In step S3, the U-shaped segmentation network is also divided into a symmetrical structure of an encoding module and a decoding module. The U-shaped segmentation network extracts features through a four-layer downsampling module and then restores the original image size through a four-layer upsampling module. The encoding module of the segmentation network is responsible for feature extraction and includes a four-layer downsampling module and a double convolutional layer. Each downsampling operation includes a double convolution and a max pooling operation. The two convolutions in the double convolution use the same number of kernels, thus outputting feature maps of the same depth. The downsampling work of compressing the image in the encoding module is mainly done by the max pooling layer. The decoding module of the segmentation network includes a four-layer upsampling module and the final output module. It is worth noting that the unique feature skip connections and encoding / decoding module structure of the U-shaped segmentation network can better focus on the boundary and shape information of the target, adapting to the requirements of segmentation tasks.

[0151] Specifically, the U-shaped segmentation network is also divided into a symmetrical structure of encoding and decoding modules. The input and denoising networks are both in the form of four-dimensional tensors, with one channel, meaning the input image is also a grayscale image with a size of 512*512. The U-shaped segmentation network extracts features through four downsampling layers and then restores the original image size through four upsampling layers. The encoding module of the segmentation network is responsible for feature extraction, containing four downsampling layers and a double convolutional layer. Each downsampling operation includes one double convolution and one max pooling operation, with the pooling window size set to 2, meaning the size is halved each time. The two convolutions in the double convolution use the same number of kernels, thus outputting feature maps of the same depth. The downsampling work of the encoding module is mainly done by the max pooling layer. The decoding module of the segmentation network includes four upsampling layers and the final output module. It is worth noting that the unique feature skip connections and encoding / decoding module structure of the U-shaped segmentation network can better focus on the boundary and shape information of the target, adapting to the requirements of segmentation tasks.

[0152] Furthermore, in step S3, the dual convolution module specifically includes the following operations: the dual convolution module has six layers. The feature map sequentially passes through the convolutional layer to extract features, enters the BN layer for normalization, enters the ReLU activation function layer to introduce nonlinear transformation, and repeats the above convolution-batch normalization-activation operation. Such six layers constitute the dual convolution module.

[0153] Specifically, the double convolution module has six layers. The feature map is sequentially passed through the convolutional layer to extract features, then enters the BN layer for normalization, and then enters the ReLU activation function layer to introduce non-linear transformation. The above convolution-batch normalization-activation operation is repeated. These six layers constitute the double convolution module.

[0154] Further, in step S3, the upsampling module specifically includes the following operations: each upsampling first restores the size through interpolation or deconvolution, while reducing the depth of the feature map to be the same as the depth of the feature map extracted from the symmetrical part of the encoding module. Then, this feature map of the same size is concatenated with the upsampled feature map to obtain a feature map with double the number of channels. Then, the number of channels is compressed through double convolution.

[0155] Specifically, each upsampling first restores the size through interpolation or deconvolution, while reducing the depth of the feature map to be the same as the depth of the feature map extracted from the symmetrical part of the encoding module. Then, this feature map of the same size is concatenated with the upsampled feature map to obtain a feature map with double the number of channels. Finally, the number of channels is compressed through double convolution.

[0156] Furthermore, in step S3, the network training loss function is designed using a hybrid strategy of BCE loss and Dice loss. BCE loss primarily considers the independent classification of each pixel. Dice loss, on the other hand, considers the degree of overlap between the predicted result and the true label of the entire image. Combining the two as the loss function can better evaluate the network performance.

[0157] Specifically, the network training loss function adopts a hybrid strategy of BCE loss and Dice loss. BCE loss mainly considers the independent classification of each pixel. Dice loss, on the other hand, considers the degree of overlap between the predicted result and the true label of the entire image. Combining the two as the loss function can better evaluate the network performance.

[0158] BCE (Binary Cross Entropy) loss, also known as binary cross entropy, is calculated using the following formula:

[0159] BCEloss=-[yln(p)+(1-y)ln(1-p)]

[0160] Where y is the true label, which can only be 0 or 1, p is the probability value of the predicted output, which ranges from 0 to 1, and ln represents the natural logarithm.

[0161] The formula for calculating Dice loss is shown in formula (2):

[0162] DICEloss = 1 - Dice

[0163] Based on the above, the joint loss function is shown in Equation (3):

[0164] Los = 0.5.BCEloss + DICEloss

[0165] Furthermore, in step S3, the specific method for using transfer learning to process the U-shaped segmentation network is as follows: the partial encoding module structure and trained parameters of the trained denoising network RED-CNN are transferred to the partial encoding module structure of the U-shaped segmentation network. Since the parameters such as the convolutional layer weights of the encoding module have been learned in the denoising network in step two, and these parameters are also required by the downsampling module of the encoding module in the segmentation network, the weight transfer can be performed directly to simplify the training operation.

[0166] In this specific application example, a U-shaped segmentation network model with 60,000 iterations was used for testing. The average Dice coefficient was 0.5642, the average IOU score was 0.5019, and the average pixel classification accuracy was 0.9997. With a centroid distance threshold of 100, the confusion matrix between the predicted and ground images is shown in Table 1. Based on the confusion matrix, the accuracy was 0.639, the precision was 0.730, the recall was 0.743, and the F1 score was 0.7361.

[0167] Table 1. Segmentation Test Confusion Matrix

[0168]

[0169] In this specific application example, the segmentation performance diagram is as follows: Figure 5 As shown, Figure 5 The input to the segmentation network, the real mask image, and the predicted mask image are shown respectively. As can be seen from the results, the network still has a certain detection capability for very small nodules.

[0170] In step S4, the specific method for constructing the lung nodule benign / malignant classification network is as follows: Since the features extracted for segmentation and classification tasks overlap to some extent, and the decoding module restores the extracted features to the original image size, while the classification network, due to the nature of the task, does not need to output a complete image, the decoding module can be discarded. A dual-channel CNN encoding module architecture is adopted, with the global image and the segmented lung nodule lesion portion as local images, respectively. After fusing the features generated by the two channels using a feature fusion module, a fully connected layer and a Dropout layer are added at the end of the network, and the classification result is output. This architecture avoids overfitting and meets the requirements of the classification task.

[0171] Furthermore, in step S4, the specific method for feature fusion of the classification network is as follows: the original image and the segmented lesion parts are input into the classification network at the same time, and the global features of the original image and the local features of the lesion parts are fused together, thereby taking into account both the global features of the whole image and the features of the lesion parts that require more attention.

[0172] Specifically, since the features extracted for segmentation and classification tasks overlap to some extent, and the role of the decoding module is to restore the extracted features to the original image size, while the classification network does not need to output a complete image due to the nature of the task, the decoding module can be discarded. Instead, a U-shaped network encoding module architecture with a reduced number of convolutional kernels can be adopted, supplemented by a feature fusion module. At the same time, fully connected layers and Dropout layers are added at the end of the network to avoid overfitting and meet the requirements of the classification task.

[0173] Simultaneously, the original image and the segmented lesion parts are input into the classification network respectively, and the global features of the original image and the local features of the lesion parts are fused together, thus taking into account both the global features of the whole image and the features of the lesion parts that require more attention.

[0174] Furthermore, in step S4, the specific method for processing the lung nodule classification network using transfer learning is as follows: the trained segmentation network encoding module is partially transferred to the dual-channel encoding module in the classification network.

[0175] In this specific application example, a classification network trained to convergence was used for testing, and the confusion matrix of the classification network is shown in Table 2. The top of the table represents the true label values, and the left side represents the predicted values. The table shows that the classification network module rarely makes large judgment errors, demonstrating good classification performance.

[0176] Table 2. Categorized Network Testing Obfuscation Matrix

[0177]

[0178] In this specific application example, the trained parameters of some encoding module structures of the trained RED-CNN denoising network are transferred to some encoding module structures of the U-shaped segmentation network, and the trained segmentation network encoding modules are transferred to the classification network. The transfer learning architecture used is as follows: Figure 3 As shown, since the encoding module is part of feature extraction, and the decoding module outputs different types (the denoising network outputs the original denoised image, the segmentation network outputs the segmented mask image, and the classification network directly outputs the category through fully connected layers from features, the decoding module can be discarded), only the encoding module is transferred in steps S3 and S4. Furthermore, considering that pooling layers, BN layers, and the ReLU function have no parameters available for transfer, and that neither the denoising nor segmentation networks have fully connected layers, whether transferring from the trained denoising network to the segmentation network in step S3, or from the trained segmentation network to the classification network in step S4, the transfer operation is performed on the convolutional layers with the same hyperparameter structure in the encoding module.

[0179] In this specific application example, before performing transfer learning, the pre-trained model is first imported into the training script, and its parameters are loaded. During training, the weights of the pre-trained model are used as initial values. Then, the layers to be transferred are selected, and their parameters are assigned to the corresponding target layers. Considering that the features targeted by denoising, segmentation, and classification tasks share some commonalities but also have differences, a non-fixed network is chosen to allow for continued optimization of the model's performance during training.

[0180] In this specific application example, the comparison before and after the two transfer operations is shown in Table 3. As can be seen from the table, the convergence speed of the network is faster after the transfer. The training of the segmentation network is faster by 42.8% and the initial loss value is reduced by 36.3% after the transfer. The training of the classification network is faster by 20% and the initial loss value is reduced by 39.7% after the transfer. Figure 6 and Figure 7 The figures show the loss decrease curves before and after the transfer for the segmentation network and the classification network, respectively. As can be seen from the figures, for both the segmentation network and the classification network, the initial loss after the transfer and the loss in each iteration are lower than the training loss without the transfer operation. Transfer learning can effectively help the segmentation network and the classification network to iterate to converge quickly.

[0181] Table 3. Performance of segmentation and classification networks before and after migration.

[0182]

[0183] One embodiment of the present invention provides an intelligent detection system for lung nodules in CT images based on transfer learning, such as... Figure 8 As shown, the system may include:

[0184] The data processing module is used to acquire a preprocessed CT dataset A, which includes low-dose CT data and full-dose CT data, wherein the full-dose CT data includes its segmentation annotation and benign / malignant classification labels; it is also used to acquire a lung CT dataset B, and to perform noise reduction processing on the lung CT dataset B using a lung nodule noise reduction model to obtain the model training dataset.

[0185] The lung nodule denoising model module provides a denoising network. Low-dose CT data and full-dose CT data in CT dataset A are used as the input and output of the denoising network. The denoising network is trained and tested to obtain a lung nodule denoising model. This model is used to obtain the denoising results of lung nodule images.

[0186] The lung nodule segmentation model module provides a U-shaped segmentation network. The U-shaped segmentation network is trained using the model training dataset, and the trained parameters of the encoding module of the lung nodule denoising model are transferred to the encoding module of the U-shaped segmentation network to obtain the lung nodule segmentation model. This model is used to obtain lung nodule tissue image results.

[0187] The lung nodule classification model module provides a global-local feature fusion CNN network. The global-local feature fusion CNN network is trained using the model training dataset, and the trained parameters of the encoding module of the lung nodule segmentation model are transferred to the feature extraction part of the CNN network to obtain the lung nodule classification model. This model is used to obtain the benign and malignant detection results of lung nodules in CT images.

[0188] In the lung nodule segmentation and classification model modules, the transfer learning operation is specifically explained as follows: Since the encoding module is the feature extraction part, and the decoding module outputs different types (the denoising network outputs the original denoised image, the segmentation network outputs a segmented mask image, and the classification network directly outputs the category from features through fully connected layers, so the decoding module can be discarded), only the encoding module is transferred in both the lung nodule segmentation and classification model modules. Furthermore, considering that pooling layers, BN layers, and the ReLU function have no parameters available for transfer, and that the denoising and segmentation networks lack fully connected layers, whether transferring from the trained denoising network to the segmentation network in the lung nodule segmentation model module, or from the trained segmentation network to the classification network in the lung nodule classification model module, the transfer operation is performed on the convolutional layers with the same hyperparameter structure in the encoding module. For example... Figure 9 The figure shown is a schematic diagram of the working process of the intelligent lung nodule detection system based on transfer learning in CT images. Figure 9 In this module, module one is the data processing module, module two is the lung nodule noise reduction model module, module three is the lung nodule segmentation model module, and module four is the lung nodule classification model module.

[0189] In some preferred embodiments, the lung nodule denoising model module above inputs a four-dimensional tensor into the RED-CNN denoising network; feeds it into an encoding module consisting of five layers of two-dimensional convolutional modules to extract features from the noisy image; feeds the intermediate feature representation into a decoding module with added residual connections to deconvolutional layers to reconstruct the extracted features; trains the network iteratively, and uses a test set to test the performance of the trained denoising network.

[0190] In some preferred embodiments, the above-mentioned data processing module divides the lung CT dataset into a group with nodules and a group without nodules, generates a mask based on the nodule annotations, performs format conversion, and finally extracts the lung parenchyma.

[0191] In some preferred embodiments, the lung nodule segmentation model module inputs a four-dimensional tensor of the same format into a U-shaped segmentation network at the input stage; extracts input image features through a four-layer downsampling module at the network encoding stage; restores the original image size through a four-layer upsampling module at the network decoding stage; transfers the trained parameters of the encoding module of the denoising network trained by the lung nodule denoising model module to the segmentation network, and then performs iterative training and performance testing.

[0192] In some preferred embodiments, the above-mentioned lung nodule segmentation model module uses a loss function composed of a combination of BCE loss and Diceloss to improve network evaluation performance.

[0193] In some preferred embodiments, the lung nodule classification model module uses the data file preprocessed by the lung nodule segmentation model module, extracts the lung nodule region using a mask, and extracts the lung nodule center image with lung parenchyma removed according to a unified format. The local information containing lung nodules is then fused with the original network through a feature fusion module. The encoding module of the U-shaped network is used, while reducing the number of convolutional kernels and adding fully connected layers and Dropout layers at the end to output the classification result. The encoding module structure of the segmentation network trained by the lung nodule segmentation model module is transferred to the classification network for iterative training. Finally, the performance of the lung nodule benign and malignant classification network is tested.

[0194] It should be noted that the steps in the method provided by the present invention can be implemented using corresponding modules, devices, units, etc. in the system. Those skilled in the art can refer to the technical solution of the method to realize the composition of the system. That is, the embodiments in the method can be understood as preferred examples for building the system, and will not be elaborated here.

[0195] An embodiment of the present invention provides a computer terminal, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it can be used to perform any of the methods in the above embodiments of the present invention, or to run any of the systems in the above embodiments of the present invention.

[0196] Optionally, the memory is used to store programs; the memory may include volatile memory, such as random-access memory (RAM), such as static random-access memory (SRAM), double data rate synchronous dynamic random-access memory (DDR SDRAM), etc.; the memory may also include non-volatile memory, such as flash memory. The memory is used to store computer programs (such as application programs, functional modules, etc. that implement the above methods), computer instructions, etc., and the aforementioned computer programs, computer instructions, etc., can be partitioned and stored in one or more memories. Furthermore, the aforementioned computer programs, computer instructions, data, etc., can be accessed by the processor.

[0197] The aforementioned computer programs, computer instructions, etc., can be stored in partitions within one or more memory locations. Furthermore, the aforementioned computer programs, computer instructions, data, etc., can be accessed by a processor.

[0198] A processor is used to execute computer programs stored in memory to implement the various steps of the methods or various modules of the systems involved in the above embodiments. For details, please refer to the relevant descriptions in the preceding method and system embodiments.

[0199] The processor and memory can be separate structures or integrated structures. When the processor and memory are separate structures, they can be coupled together via a bus.

[0200] An embodiment of the present invention also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, can be used to perform the method of any of the above embodiments of the present invention, or to run the system of any of the above embodiments of the present invention.

[0201] The intelligent detection method and system for lung nodules in CT images based on transfer learning provided in the above embodiments of the present invention, based on transfer learning, for the lung nodule detection algorithm in low-dose lung CT images, sequentially completes the noise reduction, segmentation, classification network training and testing, realizing end-to-end auxiliary algorithm research, and uses task-to-task transfer learning to reduce the hyperparameters required for training, thereby accelerating the fitting speed, aiming to provide clinicians with a more accurate and reliable auxiliary detection tool.

[0202] Those skilled in the art will understand that, in addition to implementing the system and its various devices provided by this invention in the form of purely computer-readable program code, the same functions can be achieved entirely through logical programming of the method steps, making the system and its various devices of this invention function as logic gates, switches, application-specific integrated circuits, programmable logic controllers, and embedded microcontrollers. Therefore, the system and its various devices provided by this invention can be considered as a hardware component, and the devices included therein for implementing various functions can also be considered as structures within the hardware component; alternatively, the devices for implementing various functions can be considered as both software modules implementing the method and structures within the hardware component.

[0203] Any matters not covered in the above embodiments of the present invention are well-known in the art.

[0204] The specific embodiments of the present invention have been described above. It should be understood that the present invention is not limited to the specific embodiments described above, and those skilled in the art can make various modifications or variations within the scope of the claims, which do not affect the essence of the present invention.

Claims

1. A method for intelligent detection of lung nodules in CT images based on transfer learning, characterized in that, include: Obtain a preprocessed CT dataset A, which includes: low-dose CT data and full-dose CT data containing segmentation annotations and benign / malignant classification labels; A denoising network is provided, which takes low-dose CT data from the CT dataset A as input and full-dose CT data from the CT dataset A as output. The denoising network is trained and tested to obtain a lung nodule denoising model, which is used to obtain lung nodule image denoising results. Obtain a lung CT dataset B, and use the lung nodule denoising model to denoise the lung CT dataset B to obtain the model training dataset; A U-shaped segmentation network is provided. The U-shaped segmentation network is trained using the model training dataset. The trained parameters of the encoding module of the lung nodule denoising model are transferred to the encoding module of the U-shaped segmentation network to obtain a lung nodule segmentation model. This model is used to obtain lung nodule tissue image results. A global-local feature fusion CNN network is provided. The global-local feature fusion CNN network is trained using the model training dataset. The trained parameters of the encoding module of the lung nodule segmentation model are transferred to the feature extraction part of the CNN network to obtain a lung nodule classification model. This model is used to obtain the benign and malignant detection results of lung nodules in CT images. The method provides a noise reduction network, using low-dose CT data and full-dose CT data from the CT dataset A as the input and output of the noise reduction network, respectively. The noise reduction network is trained and tested to obtain a lung nodule noise reduction model, including: A noise reduction network is provided, the noise reduction network including an encoding module and a decoding module; wherein, the encoding module consists of five two-dimensional convolutional modules, and the decoding module includes a convolutional layer, a deconvolutional layer and a residual layer connected between the two; The low-dose CT data in the CT dataset A is used as the input of the denoising network in the form of a four-dimensional tensor. The encoding module extracts the features of the noisy image from the low-dose CT data and obtains an intermediate feature representation, which is then passed to the decoding part. The decoding part reconstructs the intermediate feature representation by adding the feature map before the convolutional layer to the feature map after the symmetrical deconvolutional layer to generate a denoised image, and uses the full-dose CT data in the CT dataset A as the gold standard of the denoising network. The denoising network is iteratively trained, and the performance of the trained denoising network is tested using a test set to obtain a lung nodule denoising model.

2. The intelligent detection method for lung nodules in CT images based on transfer learning according to claim 1, characterized in that, The process of obtaining a preprocessed CT dataset A includes: A CT dataset A is obtained, comprising low-dose CT data and full-dose CT data including segmentation annotations and benign / malignant classification labels, and preprocessed; wherein the preprocessing includes: The image and non-image data from the low-dose CT data and the full-dose CT data are read and matched. Perform image format conversion on the image data; The image data is converted to pixel storage units, and invalid pixels are set to zero and converted to background. Also includes: The preprocessed CT dataset A is divided into a training set and a test set.

3. The intelligent detection method for lung nodules in CT images based on transfer learning according to claim 1, characterized in that, The process involves acquiring a lung CT dataset B, and then using the lung nodule denoising model to denoise the lung CT dataset B to obtain a model training dataset, including: Obtain a lung CT dataset B, which includes: lung CT data and corresponding segmentation labels; The lung CT dataset B is divided into a nodule group and a nodule-free group, and a binary mask label image is generated based on the nodule annotation, which serves as the label value of the U-shaped segmentation network. The lung CT dataset B is converted to an image format and then denoised using the lung nodule denoising model. Lung parenchyma is extracted from the noise-reduced lung CT data B to obtain the lung parenchyma Mask. Then, the lung parenchyma Mask is multiplied with the original image to obtain the lung parenchyma image. The model training dataset was constructed using the lung parenchyma images; The lung CT dataset B may be the same dataset as or a different dataset from the CT dataset A.

4. The intelligent detection method for lung nodules in CT images based on transfer learning according to claim 1, characterized in that, A U-shaped segmentation network is provided. The U-shaped segmentation network is trained using the model training dataset, and the trained parameters of the encoding module of the lung nodule denoising model are transferred to the encoding module of the U-shaped segmentation network to obtain a lung nodule segmentation model, including: A U-shaped segmentation network is provided, comprising a symmetrical encoding module and a decoding module; wherein: the encoding module compresses the input image through a downsampling layer module, then extracts the feature map of the input image through a double convolutional layer and inputs it to the decoding module; the decoding module restores the feature map size through an upsampling layer module, then concatenates the downsampling feature map and the upsampling feature map of the same size to obtain the output image from the output layer; The U-shaped segmentation network is trained using the model training dataset. Then, the trained parameters of the encoding module of the lung nodule denoising model are transferred to the encoding module of the U-shaped segmentation network. Finally, iterative training and performance testing are performed to obtain the lung nodule segmentation model.

5. The intelligent detection method for lung nodules in CT images based on transfer learning according to claim 4, characterized in that, The loss function Loss of the lung nodule segmentation model adopts a hybrid loss function of BCEloss and Diceloss, where: L BCE =-∑[y ln(p)+(1-y)ln(1-p)] Loss=λL BCE +L DICE Among them, L BCE The loss is a binary cross-entropy loss, where y∈{0,1} is the true label value, p∈{0,1} is the model prediction value, and L DICE Let λ be the Dice loss, and λ∈[0,1] be the weight values ​​that measure the loss functions.

6. The intelligent detection method for lung nodules in CT images based on transfer learning according to claim 1, characterized in that, The method provides a global-local feature fusion CNN network, trains the global-local feature fusion CNN network using the model training dataset, and transfers the trained parameters of the encoding module of the lung nodule segmentation model to the feature extraction part of the CNN network to obtain a lung nodule classification model, including: A global-local feature fusion CNN network is provided, and the CNN network is trained using the model training dataset. The encoding module of the lung nodule segmentation model is transferred to the feature fusion module of the CNN network to obtain the global feature extraction module and the local feature extraction module of the CNN network. The global feature extraction module and the local feature extraction module are concatenated and then fed into the fully connected layer and Dropout layer of the CNN network to output the classification result; The encoding module of the lung nodule segmentation model is migrated to the global feature extraction module and the local feature extraction module for iterative training, and performance testing is performed to finally obtain the lung nodule classification model.

7. A smart detection system for lung nodules in CT images based on transfer learning, characterized in that, include: The data processing module is used to acquire a preprocessed CT dataset A, which includes low-dose CT data and full-dose CT data containing segmentation annotations and benign / malignant classification labels; it is also used to acquire a lung CT dataset B, and to perform noise reduction processing on the lung CT dataset B using a lung nodule noise reduction model to obtain a model training dataset. A lung nodule denoising model module is provided. This module is used to provide a denoising network. The low-dose CT data in the CT dataset A is used as the input of the denoising network, and the full-dose CT data is used as the output of the denoising network. The denoising network is trained and tested to obtain a lung nodule denoising model. This model is used to obtain the denoising results of lung nodule images. A lung nodule segmentation model module is provided, which is used to provide a U-shaped segmentation network. The U-shaped segmentation network is trained using the model training dataset, and the trained parameters of the encoding module of the lung nodule denoising model are transferred to the encoding module of the U-shaped segmentation network to obtain the lung nodule segmentation model. This model is used to obtain lung nodule tissue image results. A lung nodule classification model module is provided, which is used to provide a global-local feature fusion CNN network. The global-local feature fusion CNN network is trained using the model training dataset, and the trained parameters of the encoding module of the lung nodule segmentation model are transferred to the feature extraction part of the CNN network to obtain the lung nodule classification model. This model is used to obtain the benign and malignant detection results of lung nodules in CT images. The method provides a noise reduction network, using low-dose CT data and full-dose CT data from the CT dataset A as the input and output of the noise reduction network, respectively. The noise reduction network is trained and tested to obtain a lung nodule noise reduction model, including: A noise reduction network is provided, the noise reduction network including an encoding module and a decoding module; wherein, the encoding module consists of five two-dimensional convolutional modules, and the decoding module includes a convolutional layer, a deconvolutional layer and a residual layer connected between the two; The low-dose CT data in the CT dataset A is used as the input of the denoising network in the form of a four-dimensional tensor. The encoding module extracts the features of the noisy image from the low-dose CT data and obtains an intermediate feature representation, which is then passed to the decoding part. The decoding part reconstructs the intermediate feature representation by adding the feature map before the convolutional layer to the feature map after the symmetrical deconvolutional layer to generate a denoised image, and uses the full-dose CT data in the CT dataset A as the gold standard of the denoising network. The denoising network is iteratively trained, and the performance of the trained denoising network is tested using a test set to obtain a lung nodule denoising model.

8. A computer terminal, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it can be used to perform the method of any one of claims 1-6, or to run the system of claim 7.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When executed by a processor, the computer program can be used to perform the method of any one of claims 1-6, or to run the system of claim 7.

Images