Lung nodule detection method and system based on supervisor improved U-Net segmentation network

Abstract

The application discloses a lung nodule detection method and system based on a supervisor improved U-Net segmentation network. According to the basic idea of combining an existing segmentation model with a generative adversarial network, in the traditional training network, only the difference between the output of the comparison network and the real label is used to adjust the network parameters in the reverse propagation, and the application provides another way of reverse propagation for the segmentation network by combining the discriminant network, uses higher-dimensional information, discriminates the matching degree of input and output, and provides more diversified ways for parameter optimization of the segmentation model. The application adds a supervisor on the basis of the original U-Net segmentation model, so that the segmentation graph output by the segmentation model is more reasonable, the segmentation result has higher matching degree with the CT sample input, the segmentation accuracy is higher, and the false positive rate is lower.

Description

Technical Field

[0001] This invention relates to a method and system for detecting lung nodules based on a supervisor-improved U-Net segmentation network. By designing a novel method for training the segmentation network, it enables the detection of lung nodules in medical images of lung CT samples, belonging to the field of medical image detection technology. Background Technology

[0002] Lung cancer is considered the world's deadliest cancer, and many countries have developed strategies for its early diagnosis. The NLST trial demonstrated that using low-dose chest computed tomography (CT) for lung cancer screening can significantly reduce lung cancer mortality. These measures mean that a large number of CT scans must be examined by radiologists. With the increase in the number of CT slices, even experienced physicians may struggle to detect some nodules, further increasing the workload for radiologists. Therefore, researching high-performance computer-aided detection systems is of paramount importance to improving diagnostic accuracy and efficiency.

[0003] With the tremendous success of deep learning methods in image processing, CAD computer-aided detection systems based on deep learning have achieved extremely high accuracy in medical image identification. Lung nodule detection systems generally consist of two parts: candidate nodule detection and false positive removal. Candidate nodule detection filters out several potential nodule regions from lung CT images. False positive removal aims to eliminate false positives from the candidate nodule pool to improve the accuracy of positive nodule identification.

[0004] Compared to general computer vision tasks, medical image processing, especially lung nodule detection, presents significant challenges. This is due to the complexity of lung structure, the diverse sizes and shapes of lung nodules, the extremely small diameter of most nodules (occupying a minimal proportion of the CT image area), and their high similarity to other tissues. Consequently, missed detections are common. Another challenge lies in the limited datasets. Deep learning is inextricably linked to large datasets; the high accuracy and generalization ability of deep learning algorithms are largely attributed to the massive increase in training data and computational power. While typical computer vision detection tasks involve tens of thousands of data points, medical image processing datasets are typically in the thousands due to the difficulty of data acquisition and annotation. Furthermore, the unique characteristics of medical images, particularly lung CT data, exhibit uneven distribution, with a significant imbalance between positive and negative samples, further complicating lung nodule detection research. Therefore, training an effective model under conditions of severe data shortage and uneven distribution is of great value. Summary of the Invention

[0005] To address the shortcomings of existing technologies, this application proposes a lung nodule detection method and system based on a supervisor-improved U-Net segmentation network, which trains the network model in a more efficient and integrated manner.

[0006] This application combines existing segmentation models with the basic ideas of generative adversarial networks. In traditional training networks, only the differences between the network output and the real labels are compared to adjust the network parameters through backpropagation. This application combines a discriminative network to provide another way for backpropagation of the segmentation network, using higher-dimensional information to identify the degree of matching between input and output, and providing more diverse ways to optimize the parameters of the segmentation model.

[0007] A first aspect of this application provides a lung nodule detection method based on a supervisor-improved U-Net segmentation network, the method comprising the following steps:

[0008] S1: Extract lung parenchyma from all CT sample data; divide all CT samples and annotations into training dataset and test dataset, and perform data augmentation on the training data;

[0009] S2: Construct the U-Net segmentation network and build the supervisor;

[0010] S3: Define the loss function, train the U-Net segmentation network and supervisor built in S2 using the training dataset in S1, and verify whether the performance of the trained U-Net segmentation network meets the requirements using the test dataset;

[0011] S4: Construct a false positive removal network, build a training dataset and a test dataset for the false positive network, use the training dataset to train the false positive removal network and optimize its parameters, and use the test dataset to verify whether the performance of the trained false positive removal network meets the requirements.

[0012] S5: Preprocess the CT sample to be detected. Use the U-Net segmentation network trained in S3 to segment the CT sample and generate a segmentation map. Use the false positive removal network trained in S4 to remove the false positive part in the segmentation map to achieve complete lung nodule detection.

[0013] Furthermore, the lung parenchyma extraction from the CT sample data mentioned in step S1 specifically involves:

[0014] All CT samples were binarized to generate binary images;

[0015] Use the opening operation to remove impurities and noise;

[0016] Invert the values ​​and set all pixels except those in the lung region to zero;

[0017] Dilation is applied to the binarized image;

[0018] Lung parenchyma extraction is achieved by multiplying the binarized image with the original CT sample bitwise.

[0019] Furthermore, the step S1 of dividing the CT samples and annotations into training and testing datasets specifically means:

[0020] Use the nodule location information in the annotations to generate a segmentation map;

[0021] Treat the CT sample and its corresponding segmentation map as a data pair;

[0022] Divide all data pairs into ten sets, use nine sets to build the training dataset, and use the remaining set to build the test dataset.

[0023] Furthermore, the specific structure of the U-Net segmentation network is as follows:

[0024] The U-Net segmentation network consists of two downsampling branches, one intermediate branch, two upsampling branches, and one convolutional output layer.

[0025] The downsampling branch consists of a ConvBlock layer and an average pooling layer AvgPooling connected in series.

[0026] The intermediate branch contains only one ConvBlock layer;

[0027] The upsampling branch consists of a ConvTranspose inverse convolutional upsampling layer, a CenterCrop layer, a Cat layer, and a ConvBlock layer.

[0028] The CenterCrop layer is responsible for cropping the output feature map of the downsampling branch, and the Cat splicing layer is responsible for splicing the cropped downsampling features and the upsampling extracted features.

[0029] Furthermore, the supervisor comprises two convolutional branches and one output convolutional layer, wherein:

[0030] The first convolutional branch is responsible for extracting the data features after CT sample cutting;

[0031] The second convolutional branch is responsible for extracting features from the labeled segmentation map or the output of the U-Net segmentation network.

[0032] The features extracted by each convolutional branch 2 after passing through a convolutional part are fused with the features extracted by the corresponding convolutional part in convolutional branch 1.

[0033] The output convolutional layer convolves the final fused features and outputs the result.

[0034] Furthermore, the loss function consists of two parts:

[0035] Part 1: Calculate the segmentation loss by comparing the segmentation map generated by the U-Net segmentation network with the label segmentation map;

[0036] Part Two: The classification loss calculated by the supervisor based on the preset classification labels.

[0037] A second aspect of this application provides a lung nodule detection system based on a supervisor-improved U-Net segmentation network, comprising:

[0038] Data preprocessing unit: used to extract lung parenchyma from all CT sample data; divide all CT samples and annotations into training datasets and test datasets, and perform data augmentation on the training data;

[0039] Model building unit: used to build the U-Net segmentation network and the supervisor;

[0040] U-Net segmentation network training unit: used to define the loss function, train the U-Net segmentation network and supervisor built in S2 using the training dataset, and verify whether the performance of the trained U-Net segmentation network meets the requirements using the test dataset;

[0041] False positive removal network building blocks: training and test datasets used to build the false positive removal network. The training dataset is used to train the false positive removal network and optimize its parameters. The test dataset is used to verify whether the performance of the trained false positive removal network meets the requirements.

[0042] The lung nodule detection unit is used to preprocess the CT samples to be detected. It uses a trained U-Net segmentation network to segment the CT samples and generate segmentation maps. It also uses a trained false positive removal network to remove false positive parts in the segmentation maps, thus achieving complete lung nodule detection.

[0043] A third aspect of this application provides a lung nodule detection device based on a supervisor-improved U-Net segmentation network, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the aforementioned lung nodule detection method based on a supervisor-improved U-Net segmentation network.

[0044] A fourth aspect of this application provides a computer-readable storage medium, characterized in that the storage medium stores a computer program for executing the above-described lung nodule detection method based on a supervisor-improved U-Net segmentation network.

[0045] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0046] This invention adds a supervisor to the original U-Net segmentation model, thereby making the segmentation map output by the segmentation model more reasonable, with a higher matching degree between the segmentation results and the CT sample input, higher segmentation accuracy and lower false positive rate.

[0047] Compared to training with only a segmentation model, adding a supervisor to the segmentation model can capture high-dimensional and global features of the segmentation map, training the network model in a more efficient and integrated way. This effectively avoids gradient vanishing or overfitting that may occur during training, and improves the segmentation accuracy of the segmentation network.

[0048] In particular, this invention introduces a loss weight β into the loss function. The loss weight β can be dynamically adjusted during training. The dynamic adjustment of the weight ratio of the two parts of the loss can adapt to different training stages, control the training process, and divide the training into several more refined stages. This allows the segmentation task focus of the segmentation model to be changed at different training stages to achieve better segmentation results, effectively improve the detection rate of lung nodules, and effectively reduce the false positive rate. Attached Figure Description

[0049] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the accompanying drawings used in the description of the embodiments or the prior art will be briefly introduced below. The structural features and principles of the present invention can be more clearly understood through the accompanying drawings. The drawings are schematic and should not be construed as any limitation on the present invention. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort. Wherein:

[0050] Figure 1 A schematic diagram of the U-Net segmentation network structure;

[0051] Figure 2 This is a schematic diagram of the supervisor structure;

[0052] Figure 3 This is a schematic diagram of the segmentation network training of the present invention;

[0053] Figure 4 This is a schematic diagram of the segmentation process of the present invention;

[0054] Figure 5 This is a structural diagram of a lung nodule detection device based on a supervisor-improved U-Net segmentation network. Detailed Implementation

[0055] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0056] One embodiment of this application provides a method for lung nodule detection based on a supervisor-improved U-Net segmentation network, which includes the following steps:

[0057] S1: In the data preprocessing stage, lung parenchyma is extracted from all CT sample data. All CT samples and annotations are divided into training datasets and test datasets. Data augmentation is performed on the training data.

[0058] S2: Construct the U-Net segmentation network and build the supervisor.

[0059] S3: Define the loss function, train the U-Net segmentation network and supervisor built in S2 using the training dataset in S1, and verify whether the performance of the trained U-Net segmentation network meets the requirements using the test dataset.

[0060] S4: Construct a false positive removal network, build training and test datasets for the false positive network, use the training dataset to train the false positive removal network and optimize its parameters, and use the test dataset to verify whether the performance of the trained false positive removal network meets the requirements.

[0061] S5: Preprocess the CT samples to be detected. Use the U-Net segmentation network trained in S3 to segment the CT samples and generate segmentation maps. Use the false positive removal network trained in S4 to remove false positive parts in the segmentation maps, thus realizing the complete lung nodule detection process.

[0062] Furthermore, the data preprocessing stage in step S1 is specifically as follows:

[0063] In the data preprocessing stage, lung parenchyma was extracted from all CT sample data, and all CT samples and annotations were divided into training datasets and test datasets. Data augmentation was performed on the training data.

[0064] In a preferred embodiment, the steps for extracting lung parenchyma from CT sample data include: generating binarized images from all CT samples using a binarization method; removing impurities and noise using an opening operation; inverting pixels and setting all pixels except those in the lung region to zero; dilating the binarized image; and multiplying the binarized image bitwise with the original CT sample to complete lung parenchyma extraction. Specifically:

[0065] A CT sample consists of multiple CT images, which contain not only lung information but also information from other organs and the trunk, as well as noise that could interfere with the segmentation results. All CT images are binarized using appropriate thresholds to generate binarized images, while the original CT images are retained. The generated binarized images are then opened using an 8×8 convolution kernel to remove small impurities and noise in the lung region. The binarized images, after removing impurities and noise, are then inverted, and only the lung parenchyma pixels are set to 1. The binarized images are then dilated using an 8×8 kernel to preserve the lung wall, preventing lung nodules adhered to the lung wall from being removed during lung parenchyma extraction and losing important information. The original CT images and the binarized images are then multiplied bitwise to extract the lung parenchyma. This same operation is performed on all CT samples to complete the lung parenchyma extraction task.

[0066] In a preferred embodiment, the step of dividing CT samples and annotations into training and testing datasets includes: generating segmentation maps using nodule location information from the annotations. Each CT sample and its corresponding segmentation map is considered a data pair; all data pairs are divided into ten sets; nine sets are used to construct the training dataset, and the remaining set is used to construct the testing dataset. Specifically:

[0067] Using labeled data containing information about the location of lung nodules in each CT sample, a labeled segmentation map is generated based on the location of the lung nodules in the CT sample. This segmentation map is exactly the same size as the corresponding CT sample. Pixels belonging to lung nodules in the segmentation map have a value of 1, and pixels not belonging to lung nodules have a value of 0. A corresponding labeled segmentation map is generated for all CT samples. Each CT sample and its corresponding labeled segmentation map are treated as an independent sample. These samples are divided into ten equal parts. Nine of these parts are used to generate the training dataset, and the remaining part is used to generate the test dataset.

[0068] Since lung nodules constitute a very small percentage of pixels in a single CT sample, 64 pixels were chosen as the height and width of the cropped data to ensure that the proportion of lung nodule pixels in each dataset is not too low. Because lung nodules are difficult to identify from a single CT image alone, the network is provided with seven adjacent CT images centered on the image to be segmented as supplementary feature extraction. For these reasons, the size of the cropped data block was chosen to be 7×64×64.

[0069] In a preferred embodiment, the training dataset construction step includes: cutting all CT samples from nine sets and the corresponding lung nodule images in the labeled segmentation maps to appropriate sizes based on the lung nodule location information in the labeled data. Data augmentation is then performed on the cut data using methods such as random rotation, flipping, cropping, and adding noise. The set of augmented data constitutes the training dataset. Specifically:

[0070] Take a CT sample and its corresponding labeled segmentation image. Based on the location information of the lung nodules in the labeled data, crop a pair of 7×96×96 data blocks from the CT sample and the labeled segmentation image, centered on the location information of the lung nodules. The two 96s in the cropping size are reserved for subsequent random cropping, ensuring that the lungs in the randomly cropped data contain the nodules. Perform the above operation on all lung nodule labeled information in the nine sample sets used to generate the training dataset. Due to the scarcity of lung nodule samples and the large amount of data required for training the network, data augmentation is performed on the training samples to improve the model's generalization ability. All generated pairs of 7×96×96 data blocks are augmented and their size increased by random rotation, flipping, cropping, and adding noise. The cropped pixel size is 7×64×64. Except for adding noise to the cropped portion of the CT sample, which does not require noise to the segmentation image, the other operations on the CT sample cropped image and the corresponding segmentation image are completely identical. The dataset expanded by 40 times after the above augmentation is the training dataset.

[0071] In a preferred embodiment, the test dataset construction step includes: dividing all CT samples in the remaining set and the corresponding lung nodule images in the labeled segmentation maps into appropriate sizes based on the lung nodule location information in the labeled data; the data set composed of the divided images is the test dataset. Specifically:

[0072] The steps for generating the test dataset include: taking a CT sample and its corresponding annotated segmentation map; based on the location information of the lung nodules in the CT sample as indicated in the annotation data, cropping a pair of 7×64×64 data blocks from the CT sample and the annotated segmentation map, centered on the location information of the lung nodules; performing the above operation on all lung nodule annotation information in a sample set used to generate the test dataset; the resulting dataset is the required test data.

[0073] Let the training dataset be CT. t ={(ct t1 m t1 ), (ct t2 m t2 ), ..., (ct tn m tn )}, where ct ti Let i∈(1,...,n) represent the data after CT sample cutting, and m ti , i∈(1,...,n) means that with ct ti The corresponding labeled segmentation image.

[0074] Let the test dataset be CT. v ={(ct v1m v1 ), (ct v2 m v2 ), ..., (ct vm m vm )}, where ct vi Let i∈(1,...,m) represent the data after CT sample cutting, and m vi , i∈(1,...,m) represents the segmentation map corresponding to the CT sample.

[0075] Furthermore, step S2 involves constructing the U-Net segmentation network and supervisor, as detailed below:

[0076] like Figure 1 As shown, due to the network input data ct ti The height and width are 64×64. The U-Net segmentation network can only contain two downsampling branches, one intermediate branch, two upsampling branches, and one convolutional output layer. The downsampling branch consists of a ConvBlock layer and an AvgPooling layer connected in series. The intermediate branch contains only one ConvBlock layer, which consists of two convolutional layers, two ReLU activation layers, and two Batch Normalization (BN) layers connected in series. The upsampling branch consists of one UpBlock layer, which consists of a ConvTranspose inverse convolutional upsampling layer, a CenterCrop layer, a Cat concatenation layer, and another ConvBlock layer.

[0077] The CenterCrop layer is responsible for cropping the output feature map of the downsampling branch, and the Cat concatenation layer is responsible for concatenating the cropped downsampling features and the upsampling extracted features. Finally, the convolutional output layer outputs the segmentation map. The input of the U-Net segmentation network is defined as m. ti When its output segmentation image is m tvi .

[0078] like Figure 2 As shown, the supervisor input consists of two parts: one is the data ct after CT sample cutting. ti Secondly, CT scan. ti The corresponding segmentation image, ct ti The features of the input segmentation map differ significantly from those of the labeled segmentation map. Therefore, two feature extraction branches are used to extract the feature map, where the input segmentation map represents the labeled segmentation map m. ti Or the output of U-Net and ct tiThe corresponding segmentation map is generated. Furthermore, an overly complex supervisor design would make it nearly impossible for U-Net to receive updates from the supervisor during backpropagation during training; therefore, the supervisor structure should be relatively simple. For these reasons, the constructed supervisor contains two convolutional branches and one output convolutional layer, where convolutional branch one is responsible for extracting the ct. ti The features are extracted by the second convolutional branch, which is responsible for extracting m. ti Or the m output of the aforementioned U-Net segmentation network tvi Its characteristics.

[0079] Convolutional branch one consists of an initial convolutional part and three concatenated convolutional parts. The initial convolutional part consists of a concatenated convolutional layer and a Leaky ReLU activation layer. Each convolutional part consists of a convolutional layer, a batch normalization (BN) layer, and a Leaky ReLU activation layer. Convolutional branch two has the same structure as convolutional branch one. To ensure the final convolutional layer fully integrates the features from both inputs, the features extracted from each convolutional part of convolutional branch two are added bit-by-bit to the features extracted from the corresponding layer in convolutional branch one, serving as the input features for the next convolutional part of convolutional branch one. Finally, the output convolutional layer outputs the fused feature convolution result.

[0080] Furthermore, step S3 defines the loss function and trains and validates the network, as follows:

[0081] To enable the U-Net segmentation network to capture high-dimensional and global features of the segmentation map during training, its loss function is defined to consist of two parts: one part is the segmentation map m generated by U-Net. tvi With label segmentation image m ti The segmentation loss is calculated after comparison. Another part is the classification loss calculated by the supervisor based on the preset classification labels.

[0082] The segmentation loss uses Dice loss, a common loss mechanism for segmentation tasks. Dice loss is calculated by comparing the generated segmentation map with the labeled segmentation map. Its definition is as follows:

[0083]

[0084] Where `pred` represents the generated segmentation map, which is a 64×64 graph consisting of constants between 0 and 1. `label` represents the label segmentation map, which is a 64×64 graph consisting of either 0 or 1. `pred∩label` represents the sum of the intersections of `pred` and `label`, i.e., multiplying `pred` bitwise by `|label` and summing the results. `pred∪label` represents the sum of the unions of `pred` and `label`, i.e., the sum of all pixel values ​​in `pred` and `label`. If the predicted segmentation map is very similar to the label segmentation map, `DiceLoss` will be very close to 0; otherwise, it will be close to 1. The U-Net model optimizes its parameters to make `DiceLoss` close to 0.

[0085] To avoid the gradient vanishing problem during U-Net training, the mean squared error loss (MSELoss), commonly used in binary classification tasks, is adopted as the classification loss. MSELoss generates the loss by comparing the difference between the classification result and the actual prediction result, and its definition is as follows:

[0086]

[0087] Where y represents the label value 0 or 1, This represents the probability value output by the supervisor. MSELoss increases when the output probability value differs significantly from the label value, and decreases vice versa.

[0088] The actual loss (Loss) is the sum of the segmentation loss and the supervision loss, and its formula is as follows:

[0089] Loss=β*(1-y)*DiceLoss+(1-β)*MSELoss

[0090] In the formula, β represents the weights of the two loss components, a manually selected hyperparameter that indicates the proportion of each component in the total loss. During training, the importance of the two loss components can be controlled by pre-setting a function to change the β value or by pausing training to adjust β, thereby controlling the training process and differentiating the effects of different training phases.

[0091] When training the discriminator, its loss function is defined as MSELoss, which is commonly used in classification networks, and its formula is the same as the MSELoss formula mentioned above.

[0092] The input data for the U-Net segmentation network and the discriminator established above are not the same. The input data for the U-Net segmentation network is the ct data from the training dataset. ti The size of a single data set is 7×64×64, and the batch size is defined as 64 during training. The supervisor's input is the ct data set from the training set. ti And the corresponding segmentation image, m tiOr the corresponding generated segmentation map m tvi The sizes of individual data sets are 7×64×64 and 1×64×64, respectively. During training, the batch size of a single input batch is defined as 64.

[0093] As described above, input the training dataset into both networks. Optimize the parameters of both networks using backpropagation based on the predefined loss function until the DlceLoss loss calculated by the U-Net segmentation network when the input is the training dataset meets the requirements. Verify whether the segmentation accuracy calculated by the U-Net segmentation network when the input is the test dataset meets the expected target. If it does, training is complete. Specifically:

[0094] First, set the β value of the loss function Loss defined above to 0.95. Train the classifier: Take 64 data points from the training set and input them into U-Net as described above to generate 64 segmentation maps. Calculate the segmentation loss using the generated segmentation maps and the labeled segmentation maps. Input the training data and the generated segmentation maps into the supervisor, and output probability values. Since it is expected that the segmentation maps generated by the U-Net segmentation network have a high matching degree with the corresponding training data, the label y is set to 1 when calculating MSELoss. Calculate the Loss according to the formula and backpropagate to optimize the U-Net segmentation network parameters; the supervisor parameters are not optimized here. Train the supervisor: Input the training data and the corresponding generated segmentation maps into the supervisor, and output probability values. Here, it is expected that the supervisor can distinguish between mismatches between the training data and the generated segmentation maps; therefore, the label y is set to 0 when calculating MSELoss. Optimize the supervisor parameters based on the calculated MSELoss through backpropagation; the U-Net segmentation network parameters are not optimized here. Input the training data and the corresponding labeled segmentation maps into the supervisor again, and output probability values. Here, we aim for the supervisor to distinguish between matching training data and labeled segmentation maps; therefore, the label y is set to 1 when calculating MSELoss. The supervisor parameters are optimized via backpropagation based on the calculated MSELoss; the U-Net segmentation network parameters are not optimized here. The above steps constitute a batch-size training process. Different batches of data from the training dataset are input into the network, and this process is repeated until all data in the training dataset has been used. This is called a training epoch. After each training epoch, the value of β decays to 95% of the previous epoch's value until DiscLoss is reached, indicating training is complete. The connection method and backpropagation path during training are as follows: Figure 3 As shown.

[0095] Furthermore, step S4 involves constructing a false positive removal network, building relevant training datasets and training and validation datasets, and training and validating the network, as detailed below:

[0096] The false positive removal network consists of a batch normalization (BN) layer, four block units, a fully connected layer, and a softmax normalized exponential function concatenated together. Each block unit comprises two convolutional layers, two ReLU activation layers, and a max-pooling layer.

[0097] In a preferred embodiment, the specific steps for constructing the relevant training dataset and test dataset are as follows:

[0098] In step 1.2, each CT sample and its corresponding labeled segmentation map are treated as an independent sample. These samples are divided into ten equal parts, nine of which are used to generate the training dataset, and the remaining part is used to generate the test dataset. When generating a new training dataset, the nine parts from step 1.2 are used to generate the new training dataset, and the remaining part is used to generate the new test dataset. The labeled segmentation map is not used, and the data is not re-divided or changed to avoid data leakage.

[0099] The training dataset generation steps include: taking a CT sample and, based on the location information of lung nodules labeled in the annotation data, cropping a 32×64×64 data block from the CT sample, centered on the location information of the lung nodules. Performing the same operation on all lung nodule annotations in the nine sample sets used to generate the training data. Augmenting and expanding the data size by randomly rotating, flipping, cropping, and adding noise to all generated 32×64×64 data blocks, where the cropped pixel size is 32×48×48. Based on the location information of suspected lung nodules labeled in the annotation data within the CT sample, cropping a 32×48×48 data block from the CT sample, centered on the location information of the suspected lung nodules (which are not actually lung nodules). Labeling the cropped lung nodule data as 1 and the cropped suspected nodules as 0. The set formed by performing the above operations on all lung nodule and suspected lung nodule annotations in the nine sample sets used to generate the training data constitutes the training dataset.

[0100] The steps for generating the new test dataset are basically the same as those for generating the new test data. The difference is that the lung nodule data in the new test dataset is directly cropped to a size of 32×48×48, and no data augmentation operations are performed. The other steps remain unchanged.

[0101] Let the new training dataset be CT. Nt ={(ct Nt1 y Nt1 ), ..., (ct Ntn y Ntn )}, where ct Nti Let i∈(1,…,n) represent the data after CT cutting, and y Nti, i∈(1,…,n) means that ct Nti The corresponding label is 1 or 0.

[0102] Let the new training dataset be CT. Nv ={(ct Nv1 y Nv1 ), ..., (ct Nvn y Nvn )}, where ct Nvi Let i∈(1,...,n) represent the data after CT cutting, and y Nvi , i∈(1,…,n) means that ct Nti The corresponding label is 1 or 0.

[0103] The loss value for the false positive removal network is defined as CrossEmptroyLoss. The size of a single input data set for the false positive removal network is 1×32×48×48, and the batch size for a single input data set during training is defined as 64. The false positive removal network is configured to process the ct data in the training dataset according to the above requirements. Nvi Inputting data into the false positive removal network in predetermined batches, the network parameters are optimized using backpropagation based on the predefined loss function until the loss value calculated by the false positive removal network when the input is a new training dataset meets the requirements. The classification accuracy calculated by the false positive removal network when the input is a test dataset is verified to meet the expected target; if it does, training is complete.

[0104] Furthermore, step S5 completes the entire lung nodule detection process, and the specific operations are as follows:

[0105] The U-Net segmentation network trained in S3 is combined with the false positive removal network trained in S4 to complete a complete method and system for lung nodule detection based on a supervisor-improved U-Net segmentation network. The specific process is as follows:

[0106] like Figure 4 As shown, a CT sample is obtained, and the lung parenchyma corresponding to S1 is extracted from the CT image. The CT sample after lung parenchyma extraction is grouped and integrated into multiple data according to the input requirements. Each data is 7×512×512 in size. The integrated multiple data are input into the U-Net network in batches. Each batch generates a number of generated segmentation maps. These segmentation maps are stacked into a complete CT segmentation map in a certain order.

[0107] Obtain the center point location of the suspected lung nodule portion in the segmentation image generated in the previous step. Based on the center point location, crop the input data required by the false positive removal network and input the data into the false positive removal network. Filter out the false positive portions of the suspected lung nodule based on the output of the false positive removal network, remove the false positive portions from the complete CT segmentation image, and the remaining portion is the final lung nodule segmentation result.

[0108] Another embodiment of this application illustrates a lung nodule detection system based on a supervisor-improved U-Net segmentation network; the system includes:

[0109] Data preprocessing unit: used to extract lung parenchyma from all CT sample data; divide all CT samples and annotations into training datasets and test datasets, and perform data augmentation on the training data;

[0110] Model building unit: used to build the U-Net segmentation network and the supervisor;

[0111] U-Net segmentation network training unit: used to define the loss function, train the U-Net segmentation network and supervisor built in S2 using the training dataset, and verify whether the performance of the trained U-Net segmentation network meets the requirements using the test dataset;

[0112] False positive removal network building blocks: training and test datasets used to build the false positive removal network. The training dataset is used to train the false positive removal network and optimize its parameters. The test dataset is used to verify whether the performance of the trained false positive removal network meets the requirements.

[0113] The lung nodule detection unit is used to preprocess the CT samples to be detected. It uses a trained U-Net segmentation network to segment the CT samples and generate segmentation maps. It also uses a trained false positive removal network to remove false positive parts in the segmentation maps, thus achieving complete lung nodule detection.

[0114] Embodiments of the system of the present invention can be applied to network devices. The system embodiments can be implemented through software, hardware, or a combination of both. Taking software implementation as an example, as a logical system, it is formed by the processor of the device loading corresponding computer program instructions from non-volatile memory into memory and executing them. The computer program is used to execute a lung nodule detection method based on a supervisor-improved U-Net segmentation network. From a hardware perspective, such as... Figure 5 The diagram shown is a hardware structure diagram of the lung nodule detection device based on the supervisor-improved U-Net segmentation network of the present invention. Except for... Figure 5In addition to the processor, network interface, memory, and non-volatile memory shown, the device may also include other hardware for hardware-level expansion. On the other hand, this application also provides a computer-readable storage medium storing a computer program that executes a lung nodule detection method based on a supervisor-improved U-Net segmentation network.

[0115] For the system embodiments, since they basically correspond to the method embodiments, the relevant parts can be referred to in the description of the method embodiments. The system embodiments described above are merely illustrative, and those skilled in the art can understand and implement them without creative effort.

[0116] Other embodiments of this application will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of this application that follow the general principles of this application and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only.

[0117] It should also be noted that the terms “comprising,” “including,” or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus.

[0118] The above description is merely a preferred embodiment of this application and is not intended to limit this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of protection of this application.

Claims

1. A lung nodule detection method based on a supervisor-improved U-Net segmentation network, characterized in that... The method includes the following steps: S1: Extract lung parenchyma from all CT sample data; divide all CT samples and annotations into training dataset and test dataset, and perform data augmentation on the training data; S2: Construct the U-Net segmentation network and build the supervisor; S3: Define the loss function, train the U-Net segmentation network and supervisor built in S2 using the training dataset in S1, and verify whether the performance of the trained U-Net segmentation network meets the requirements using the test dataset; S4: Construct a false positive removal network, build a training dataset and a test dataset for the false positive network, use the training dataset to train the false positive removal network and optimize its parameters, and use the test dataset to verify whether the performance of the trained false positive removal network meets the requirements. S5: Preprocess the CT sample to be detected. Use the U-Net segmentation network trained in S3 to segment the CT sample and generate a segmentation map. Use the false positive removal network trained in S4 to remove the false positive parts in the segmentation map to achieve complete lung nodule detection. The supervisor contains two convolutional branches and one output convolutional layer, wherein: The first convolutional branch is responsible for extracting the data features after CT sample cutting; The second convolutional branch is responsible for extracting features from the labeled segmentation map or the output of the U-Net segmentation network. The features extracted by each convolutional branch 2 after passing through a convolutional part are fused with the features extracted by the corresponding convolutional part in convolutional branch 1. The output convolutional layer convolves the final fused features and outputs the result. The loss function consists of two parts: Part 1: Calculating the segmentation loss after comparing the segmentation map generated by the U-Net segmentation network with the label segmentation map. ; Part 2: The classification loss MSELoss calculated by the supervisor based on the preset classification labels; The loss function is defined as follows: In the formula, β represents the weights of the two parts of the loss, and y represents the label value; Based on the aforementioned loss function, backpropagation is used to optimize the parameters of the two networks until the segmentation loss value calculated by the U-Net segmentation network when the input is the training dataset reaches the required level. Then, the segmentation accuracy calculated by the U-Net segmentation network when the input is the test dataset is verified to meet the expected target. If the target is met, training is complete. Specifically: Step A: Input the training data into the U-Net segmentation network to generate 64 segmentation maps, and calculate the segmentation loss using the generated segmentation maps and the labeled segmentation maps. The training data and the generated segmentation map are input into the supervisor to obtain the output probability value. When calculating MSELoss, the label y is recorded as 1. The Loss is calculated according to the formula and backpropagation is used to optimize the parameters of the U-Net segmentation network. The supervisor parameters are not optimized. Step B: Input the training data and the corresponding generated segmentation map into the supervisor, output the probability value, and record the label y as 0 when calculating MSELoss. Optimize the supervisor parameters through backpropagation based on the calculated MSELoss. The U-Net segmentation network parameters are not optimized here. Step C: Input the training data and corresponding labeled segmentation map back into the supervisor, output the probability value, and record the label y as 1 when calculating MSELoss. Optimize the supervisor parameters through backpropagation based on the calculated MSELoss, without optimizing the U-Net segmentation network parameters. Steps A to C constitute a BatchSize training process, in which different batches of data are taken from the training dataset and input into the network. The above steps are repeated until all data in the training dataset has been used, which is called a training cycle. After each training cycle, the value of β will decay to 95% of the previous cycle, until DiscLoss reaches the required level and training is completed, where the value of β is set to 0.

95.

2. The method for detecting lung nodules according to claim 1, characterized in that: The lung parenchyma extraction from the CT sample data mentioned in step S1 specifically involves: All CT samples were binarized to generate binary images; Use the opening operation to remove impurities and noise; Invert the values ​​and set all pixels except those in the lung region to zero; Dilation is applied to the binarized image; Lung parenchyma extraction is achieved by multiplying the binarized image with the original CT sample bitwise.

3. The method for detecting lung nodules according to claim 1, characterized in that: The step S1, which involves dividing all CT samples and annotations into training and testing datasets, specifically means: Use the nodule location information in the annotations to generate a segmentation map; Treat the CT sample and its corresponding segmentation map as a data pair; Divide all data pairs into ten sets, use nine sets to build the training dataset, and use the remaining set to build the test dataset.

4. The method for detecting lung nodules according to claim 1, characterized in that: The specific structure of the U-Net segmentation network is as follows: The U-Net segmentation network consists of two downsampling branches, one intermediate branch, two upsampling branches, and one convolutional output layer. The downsampling branch consists of a ConvBlock layer and an average pooling layer AvgPooling connected in series. The intermediate branch contains only one ConvBlock layer; The upsampling branch consists of a ConvTranspose inverse convolutional upsampling layer, a CenterCrop layer, a Cat layer, and a ConvBlock layer. The CenterCrop layer is responsible for cropping the output feature map of the downsampling branch, and the Cat splicing layer is responsible for splicing the cropped downsampling features and the upsampling extracted features.

5. A lung nodule detection system based on a supervisor-improved U-Net segmentation network, characterized in that, include: Data preprocessing unit: used to extract lung parenchyma from all CT sample data; All CT samples and annotations were divided into training datasets and test datasets, and data augmentation was performed on the training data. Model building unit: used to build the U-Net segmentation network and the supervisor; U-Net segmentation network training unit: used to define the loss function, train the U-Net segmentation network and supervisor built in S2 using the training dataset, and verify whether the performance of the trained U-Net segmentation network meets the requirements using the test dataset; False positive removal network building blocks: training and test datasets used to build the false positive removal network. The training dataset is used to train the false positive removal network and optimize its parameters. The test dataset is used to verify whether the performance of the trained false positive removal network meets the requirements. The lung nodule detection unit is used to preprocess the CT sample to be detected. It uses a trained U-Net segmentation network to segment the CT sample and generate a segmentation map. It uses a trained false positive removal network to remove false positive parts in the segmentation map, thus achieving complete lung nodule detection. The supervisor contains two convolutional branches and one output convolutional layer, wherein: The first convolutional branch is responsible for extracting the data features after CT sample cutting; The second convolutional branch is responsible for extracting features from the labeled segmentation map or the output of the U-Net segmentation network. The features extracted by each convolutional branch 2 after passing through a convolutional part are fused with the features extracted by the corresponding convolutional part in convolutional branch 1. The output convolutional layer convolves the final fused features and outputs the result. The loss function consists of two parts: Part 1: Calculating the segmentation loss after comparing the segmentation map generated by the U-Net segmentation network with the label segmentation map. ; Part Two: The classification loss MSELos calculated by the supervisor based on the preset classification labels; The loss function is defined as follows: In the formula, β represents the weights of the two parts of the loss, and y represents the label value; Based on the aforementioned loss function, backpropagation is used to optimize the parameters of the two networks until the segmentation loss value calculated by the U-Net segmentation network when the input is the training dataset reaches the required level. Then, the segmentation accuracy calculated by the U-Net segmentation network when the input is the test dataset is verified to meet the expected target. If the target is met, training is complete. Specifically: Step A: Input the training data into the U-Net segmentation network to generate 64 segmentation maps, and calculate the segmentation loss using the generated segmentation maps and the labeled segmentation maps. The training data and the generated segmentation map are input into the supervisor to obtain the output probability value. When calculating MSELoss, the label y is recorded as 1. The Loss is calculated according to the formula and backpropagation is used to optimize the parameters of the U-Net segmentation network. The supervisor parameters are not optimized. Step B: Input the training data and the corresponding generated segmentation map into the supervisor, output the probability value, and record the label y as 0 when calculating MSELoss. Optimize the supervisor parameters through backpropagation based on the calculated MSELoss. The U-Net segmentation network parameters are not optimized here. Step C: Input the training data and corresponding labeled segmentation map back into the supervisor, output the probability value, and record the label y as 1 when calculating MSELoss. Optimize the supervisor parameters through backpropagation based on the calculated MSELoss, without optimizing the U-Net segmentation network parameters. Steps A to C constitute a BatchSize training process, in which different batches of data are taken from the training dataset and input into the network. The above steps are repeated until all data in the training dataset has been used, which is called a training cycle. After each training cycle, the value of β will decay to 95% of the previous cycle, until DiscLoss reaches the required level and training is completed, where the value of β is set to 0.

95.

6. A lung nodule detection device based on a supervisor-improved U-Net segmentation network, characterized in that, include: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor, when executing the program, implements the lung nodule detection method based on a supervisor-improved U-Net segmentation network as described in any one of claims 1-4.

7. A computer-readable storage medium, characterized in that, The storage medium stores a computer program for executing the lung nodule detection method based on the supervisor-improved U-Net segmentation network as described in any one of claims 1-4.

Images