Lung nodule detection method based on progressive multi-scale network model

Abstract

The application discloses a lung nodule detection method based on a progressive multi-scale network model, and comprises the following steps: constructing a progressive multi-scale network model PMSN-YOLOX with YOLOX as a backbone network; using an existing public data set to perform self-supervised pre-training on the PMSN-YOLOX, taking the parameters obtained after the self-supervised pre-training as the initialization parameters of the backbone network, and then using a training set with labels to perform supervised training on the PMSN-YOLOX, wherein the training set contains a plurality of labeled lung nodule medical image pictures; in the supervised training process, the optimal PMSN-YOLOX is obtained when the loss function no longer changes. A new lung medical image picture is input into the optimal PMSN-YOLOX, and the output is a probability value of whether suffering from lung nodule and a surrounding box coordinate of the lung nodule. The method has more excellent ability in feature extraction, which enables the model to exhibit higher sensitivity to lung nodules of different sizes.

Description

Technical Field

[0001] This invention relates to the field of data processing technology, and more particularly to the field of medical image data processing technology, specifically to a method for detecting lung nodules based on a progressive multiscale network model. Background Technology

[0002] In computer vision, particularly in object detection tasks, deep learning models rely on powerful backbone feature extraction networks to capture rich visual information. Traditional deep learning models primarily use ResNet and its variants as backbone networks. These networks effectively mitigate the vanishing gradient problem by introducing the concept of residual learning, thus supporting the deepening of the network. Furthermore, Transformer networks with global dependencies are also increasingly being introduced into vision tasks due to their excellent feature extraction capabilities.

[0003] While existing network architectures such as VGG, ResNet, and DenseNet perform well in vision tasks, they still face several challenges. For example, while VGG networks improve learning capabilities by using small-sized convolutional kernels, their depth and large number of parameters result in high computational costs, especially when GPU resources are limited. ResNet addresses the vanishing gradient problem through residual connections, but the marginal effect of its performance improvement diminishes when dealing with very deep network structures. DenseNet optimizes feature transfer and reuse through improved connection methods, but the increased number of parameters and computational cost limits its application in resource-constrained environments.

[0004] Designing novel network structures for lung nodule auxiliary diagnosis, focusing on lung nodule detection, segmentation, classification, and attribute feature learning, has become a key development area in lung nodule screening technology. This has led to the emergence of new neural network structures such as U-Net and V-Net, suitable for medical image analysis. These networks acquire image information by stacking deep learning model structures to achieve different medical image analysis objectives. However, existing auxiliary diagnostic systems have relatively limited detection capabilities in terms of scale, morphology, structure, and texture. While they provide stable detection results for larger nodules with a single morphology, fixed structure, and similar texture, their ability to detect nodules of different scales, diverse morphologies, complex structures, and different textures is weak. Relying solely on common object detection models for lung nodule detection is insufficient to solve many of the problems encountered. In other words, when dealing with medical images, the accuracy of standalone object detection models is insufficient, resulting in a large number of missed and false positives.

[0005] The key to lung nodule-assisted detection lies in the ability to perform multi-scale optimized perception of lung nodules. However, designing a network structure that can effectively integrate multi-scale information while controlling model complexity is a critical challenge. Furthermore, existing multi-scale fusion strategies often focus on improving information flow, while lacking sufficient optimization for direct feature-level fusion processing, which is particularly important when dealing with targets with complex spatial structures. Summary of the Invention

[0006] In view of the above-mentioned problems in the existing technology, the technical problem to be solved by the present invention is: how to design a network structure that can effectively integrate multi-scale information and control the model complexity to accurately detect lung nodules.

[0007] To address the above problems, this invention adopts the following technical solution: a lung nodule detection method based on a progressive multi-scale network model, comprising the following steps:

[0008] S1: Construct a progressive multi-scale network model PMSN-YOLOX, which uses YOLOX as the basic network and replaces CSPrayer in YOLOX with PMSNet.

[0009] S2: Self-supervised pre-training of PMSN-YOLOX is performed using an existing public dataset. The parameters obtained after self-supervised pre-training are used as the initialization parameters of the backbone network. Then, supervised training of PMSN-YOLOX is performed using a labeled training set, which contains several labeled medical images of lung nodules.

[0010] In supervised training, the optimal PMSN-YOLOX is obtained when the loss function no longer changes.

[0011] S3: Input a new medical image of the lungs into the optimal PMSN-YOLOX, and output the probability value of whether the patient has lung nodules and the coordinates of the bounding box of the lung nodules.

[0012] Furthermore, the convolutional block in S1 comprises a 1x1 convolutional kernel layer for preprocessing the input image features and a structure block connected in sequence. The structure block includes four branches, each branch including 3x3 convolutions and CBLs of increasing number. The output of the CBL on the first branch is passed to the input of the second CBL on the second branch, the output of the CBL on the second branch is passed to the input of the third CBL on the third branch, and the output of the CBL on the third branch is passed to the input of the fourth CBL on the fourth branch. At the same time, the output of the last CBL of each branch is concatenated and then subjected to a 1x1 convolution for dimensionality reduction.

[0013] Furthermore, the self-supervised pre-training process for PMSN-YOLOX in S2 is as follows:

[0014] During self-supervised pre-training of PMSN-YOLOX, the encoder is a PMSNet network. First, a random mask array is generated. This array is then multiplied by the input image to ensure that more than 75% of the image is randomly masked. In the generated random mask array, each number represents a 32x32 pixel value in the image; 0 indicates masked, and 1 indicates preserved. By replacing the 2D convolutions in the encoder with sparse convolutions, feature maps can be obtained by extracting features only from the unmasked regions of the image.

[0015] The features of each layer of the encoder are fused with the features upsampled by the decoder, and prediction is performed at the output of the decoder to predict the pixel value of each pixel in the masked region.

[0016] Then, the similarity between the predicted image and the original image before the mask is calculated using the following formula (8). This similarity is the loss function L2Loss used to update the weights during PMSN-YOLOX self-supervised pre-training.

[0017]

[0018] In this loss function, L2Loss represents L2 loss; N represents the number of samples; D represents the feature dimension; x ij This represents the j-th feature value of the i-th sample in the original input; This represents the j-th feature value of the i-th sample output by PMSN-YOLOX.

[0019] When the value of L2Loss no longer changes, it means that pre-training is complete.

[0020] Furthermore, in S2, when performing supervised training of PMSN-YOLOX using a labeled training set, the images in the training set are first subjected to lung parenchyma segmentation data augmentation processing, as follows:

[0021] 1) Use the average pixel value of the image to binarize the image, and then retain the largest connected component in the entire image to remove all interfering information outside the lung region.

[0022] 2) After finding the lung region through the largest connected component, a circular cell with radius r is used to perform a closing operation to avoid information loss.

[0023] 3) Next, paint the background area in the image white, and the remaining black area is the area where the lung parenchyma is located.

[0024] 4) Obtain a segmentation mask image based on the black region mask where the lung parenchyma is located. Overlay the segmentation mask image with the original image according to the degree of random parameter a to segment out the lung parenchyma and obtain the lung parenchyma segmentation enhancement image.

[0025] Compared with the prior art, the present invention has at least the following beneficial effects:

[0026] 1. By replacing the YOLOX network with PMSN-YOLOX, a comprehensive improvement in model performance was observed across various average false positive levels, indicating that PMSN-YOLOX has superior capabilities in feature extraction. This enables the model to exhibit higher sensitivity to lung nodules of different sizes.

[0027] 2. Introducing self-supervised pre-training weights further improved the model's sensitivity, highlighting the importance of prior knowledge in model training. It helps the model more accurately steer in the correct learning direction while significantly reducing the average number of candidate scans (ACPS), thus dramatically improving the method's efficiency in nodule detection.

[0028] 3. By enhancing lung parenchyma segmentation, the model's sensitivity was significantly improved at low false positive rates, meaning a substantial reduction in both false negatives and false negatives of lung nodules. This data augmentation method effectively conveyed the relative irrelevance of the background region to the model while adequately expanding the data, reducing the likelihood of the model misidentifying background areas as candidate nodules at low confidence levels, thus further reducing the number of false positives. Attached Figure Description

[0029] Figure 1 Here is the structure diagram of PMSN(e).

[0030] Figure 2 This describes the structure of an image recognition network.

[0031] Figure 3 It is a progressive multiscale fusion network (PMSN(a)).

[0032] Figure 4 It is a PMSN(b) structure.

[0033] Figure 5 It is a PMSN(c) structure.

[0034] Figure 6 It is a PMSN(d) structure.

[0035] Figure 7 This is a self-supervised dataset instance.

[0036] Figure 8 A random mask for the encoder.

[0037] Figure 9 It is a self-supervised network structure.

[0038] Figure 10 This is a prediction graph for a self-supervised network.

[0039] Figure 11 This is a segmentation diagram of the lung parenchyma.

[0040] Figure 12 Image showing missing lung nodules.

[0041] Figure 13 For comparison of different segmentation intensities.

[0042] Figure 14 YOLOX predicted incorrect results. Detailed Implementation

[0043] The present invention will now be described in detail with reference to the accompanying drawings.

[0044] 1. Performance of convolutional block PMSN(e) in PMSN-YOLOX

[0045] (1) PMSN-YOLOX uses YOLOX as the backbone network, replacing the CSPrayer in YOLOX with the convolutional block PMSN(e). See details. Figure 1 The input image features are preprocessed using a 1x1 convolutional kernel. Specifically, this process operates on the entire input feature map using a 1x1 convolutional kernel. Through this convolutional layer, the original image's feature map is divided into four independent sub-feature blocks, with the number of channels in each sub-block halved compared to the original input feature map.

[0046] Then, each feature is processed through a multi-scale feature extraction structure consisting of varying numbers of 3x3 convolutions, normalization, and activation functions, resulting in four features of size H*W*2C. The output features of each branch are summed to the input features of adjacent branches at the same depth. The four features are then concatenated along the channel dimension to obtain a feature of size H*W*8C. Finally, a 1x1 convolution is performed for dimensionality reduction, restoring the feature to its original size of H*W*4C.

[0047] (2) Experiment

[0048] The experiments used the CIFAR-10 dataset, and the parameter settings for each experiment were the same: the image input size was 32*32, the batch size was 128, the training epoch was 100, the optimizer used stochastic gradient descent (SGD) with a momentum of 0.9, the learning rate adjustment strategy was cosine annealing, and the initial learning rate was 0.01.

[0049] Experimental model structure as follows Figure 2 As shown, a total of seven sets of experiments were designed, namely, replacing Block in the structure shown in the figure below with CSPDarknet, Res2Net, PMSN(a), PMSN(b), PMSN(c), PMSN(d), and PMSN(e).

[0050] (3) Experimental evaluation indicators

[0051] Evaluation metrics used in image recognition tasks include Top-1 accuracy, Top-5 accuracy, precision, and recall. Evaluation metrics used in object detection tasks include precision, recall, F1 score, and mAP.

[0052] Accuracy refers to the proportion of samples correctly predicted as positive out of the total number of positive samples. Top-1 accuracy refers to the proportion of a single prediction given by the model for each input image that is correct. Top-5 accuracy refers to the proportion of the model's top five most likely predictions for each input image that include the correct class.

[0053]

[0054] Recall: refers to the proportion of samples correctly predicted as positive out of the total number of samples that are actually positive.

[0055]

[0056] F1 score: The harmonic mean of precision and recall, used to measure the impact of imbalanced data on precision and recall.

[0057]

[0058] mAP (mean Average Precision): For each class, AP is the average precision of that class at different recall levels. mAP is the average of the AP values ​​of all classes, reflecting the overall performance of the model across all classes.

[0059] The experimental results are shown in Table 1 below. As can be seen from the experimental results, the PMSN(e) network has the best performance and the fewest parameters.

[0060] Table 1. Results of CIFAR-10 Experiment

[0061]

[0062] (4) Target detection experimental results

[0063] This section's experiments use the entire dataset from VOC2007 and VOC2012 combined. The parameter settings for each experiment are consistent: the image input size is 320*320, the batch size is 128, the training epoch is 600, the optimizer is stochastic gradient descent (SGD) with a momentum of 0.9, the learning rate adjustment strategy is cosine annealing, and the initial learning rate is 0.01.

[0064] The experimental model structure was based on the YOLOX model, with a total of seven sets of experiments. The first set used YOLOX, and the remaining six sets replaced the CSPrayer of YOLOX with Res2Net, PMSN(a), PMSN(b), PMSN(c), PMSN(d), and PMSN(e), respectively. The experimental results are shown in Table 2. Table 2 shows that the PMSN(e) network performed best, and its parameter count was significantly reduced compared to CSPrayer.

[0065] Table 2 VOC Experiment Results

[0066]

[0067]

[0068] (5) Analysis of experimental results

[0069] The experimental results above show that the improved network outperforms the original YOLOX backbone network CSPDarknet. PMSN(e), which follows the principle of adding features at the same scale and concatenating features at different scales, is significantly better than Res2Net and has fewer parameters. This is because, although the number of channels was increased in the horizontal width and four 1*1 convolutions were introduced when the input of each structure was split into four branches, the number of channels in each branch was half of the original. Furthermore, there were no additional 1*1 convolutions in the middle of the consecutive 3*3 convolutional blocks to expand or shrink the channels, which actually reduced the number of parameters compared to the original CSPDarknet.

[0070] The comparison between PMSN(a) and PMSN(b) shows that concatenation is more advantageous than direct addition when fusing features at different scales. This is because the semantic information is more completely preserved after concatenation. Although the subsequent 1*1 convolution has a larger number of parameters, the complete and clear semantic information helps the model converge in the right direction.

[0071] The comparison between PMSN(c) and PMSN(d) structures reveals the critical importance of the placement of lateral connections; incorrect placement can even degrade network performance. PMSN(c) laterally connects the output of each branch to the input of the next. While this is intended for multi-scale feature fusion, the convolutional layers traversed by the two added features are too large, resulting in blurred semantic information after addition. Consequently, the overall network performance is inferior to PMSN(a) which does not employ lateral connections. PMSN(d), on the other hand, adds features from branches at the same depth and then concatenates features from branches at different depths within a concat function. This approach aligns with multi-scale feature fusion principles and outperforms PMSN(a).

[0072] Finally, residual connections were added, and PMSN(e) that simultaneously satisfies the optimal performance of concatenating features at different scales and adding features at the same scale proves the correctness of the previous analysis of mainstream backbone networks and the effectiveness of the multi-scale fusion approach.

[0073] (6) Introduction to the structures of PMSN(a), PMSN(b), PMSN(c), and PMSN(d)

[0074] PMSN(a): A staged convolutional structure, Progressive Multi-Scale Network (PMSN), which does not target standard bottleneck structures or connection paths between basic structures, but rather the entire extraction process of features of the same size. The specific design concept is as follows: Figure 3 As shown, the input feature size is H*W*4C. After four 1*1 convolutions, the input feature is divided into four blocks, each with a size of H*W*2C. Then, each feature is subjected to a multi-scale feature extraction through a structure of different numbers of 3*3 convolutions, normalization, and activation functions, resulting in four features with a size of H*W*2C. The four features are then concatenated along the channel dimension to obtain a feature with a size of H*W*8C. Finally, a 1*1 convolution is performed to reduce the dimensionality, restoring the feature to its original size of H*W*4C.

[0075] The expression for the entire module is as follows, where x i Let y represent the input feature of the i-th branch. i f represents the output characteristic of the i-th branch. i_jLet OUT represent the j-th convolution of the i-th branch, and OUT represent the final output of the module. The four outputs of the entire network structure undergo different numbers of convolutions, thereby obtaining feature information at different scales. These feature information are not directly added together, because direct addition makes it difficult to further adjust the features through subsequent convolutions. Instead, the features are concatenated and passed through 1*1 convolutions, and each feature value is adjusted using a convolution with learnable parameters, ultimately changing their number of channels.

[0076]

[0077] PMSN(b): At the end of PMSN(a), concat was used to implement the design of concatenating features of different scales on each branch as follows: Figure 4 As shown, the outputs of each branch are summed, and then the channel dimension is expanded.

[0078] The expression for the entire module is as follows:

[0079]

[0080] PMSN(c): Based on the Res2Net network structure, the network model is constructed by adding the output features to the input features of adjacent branches, as follows. Figure 5 As shown, it will be abbreviated as PMSN(c) below. Its expression is as follows:

[0081]

[0082] PMSN(d): The network model is constructed by adding features of the same scale. The output features of each branch are added to the input features of adjacent branches at the same depth, as follows: Figure 6 As shown below, PMSN(d) is expressed as follows:

[0083]

[0084] 2. Self-supervised pre-training using PMSN-YOLOX

[0085] (1) Obtaining pre-trained weights

[0086] In the process of obtaining pre-training weights for lung nodule detection, a large number of publicly available lung CT datasets were first searched on the Kaggle website. These publicly available datasets may not necessarily show lung nodules or have target detection labels, and their image quality varies because they were acquired by different MRI scanners, as follows: Figure 7 As shown, this conforms to the principle of randomness and diversity in self-supervised datasets. A total of 20,000 lung nodule images were collected in this phase.

[0087] During self-supervised pre-training of PMSN-YOLOX, the encoder is a PMSNet network. First, a random mask array is generated, ensuring that more than 75% of the image area is randomly masked. The generated random mask array is as follows: Figure 8 As shown, each number in the random mask array represents the pixel value of a 32*32 region on the image; 0 indicates that the image is masked, and 1 indicates that it is preserved. The random mask can be used to obtain... Figure 9 The input image is shown. Directly inputting the masked image into the encoder at this point will not allow for feature extraction because 75% of the encoded region alters the pixel value distribution in the image. Replacing the 2D convolution in the encoder with sparse convolution allows feature extraction only from the unmasked regions of the image, thus obtaining the feature map.

[0088] The structure and method of the encoder generating feature maps are exactly the same as those of the semantic segmentation model UNet, and its network structure is as follows: Figure 9 As shown, the features of each layer of the encoder are fused with the features upsampled by the decoder, and prediction is performed at the output of the decoder. The prediction is performed on each pixel value of the masked region in an attempt to restore the original appearance of the image.

[0089] Then, using the following formula (8), the similarity between the predicted image and the original image before masking is calculated. The square of the difference in each dimension is calculated, and then the average is applied across all dimensions. This similarity is the loss function L2Loss used to update the weights during PMSN-YOLOX self-supervised pre-training. This process is repeated continuously, making the generated image increasingly closer to the image before masking.

[0090]

[0091] In this loss function, L2Loss represents L2 loss; N represents the number of samples; D represents the feature dimension; x ij This represents the j-th feature value of the i-th sample in the original input; This represents the j-th feature value of the i-th sample output by PMSN-YOLOX;

[0092] When the value of L2Loss no longer changes, it means that pre-training is complete.

[0093] A self-supervised generative training method is employed, focusing on building an efficient encoder-decoder network for lung nodule detection. First, a self-supervised dataset specifically designed for lung CT image analysis is constructed. Using this dataset, the encoder-decoder structure undergoes in-depth self-supervised generative training. This training method enables the model to learn key features of lung nodules and capture prior knowledge from lung CT images without explicit annotation.

[0094] During training, the model weights were meticulously tuned and optimized, and these trained weights were successfully preserved. This step was crucial to retain the knowledge the model acquired in the self-supervised learning environment. Subsequently, a key strategy adjustment was made: only the encoder part of the network was retained, and the original sparse convolutional layers were replaced with two-dimensional convolutional kernels. This improvement simplified the model structure, making the encoder more suitable for processing two-dimensional image data.

[0095] Through comprehensive training of the generative network, the encoder has acquired crucial prior knowledge about lung nodule data, providing a solid foundation for identifying and analyzing lung CT images. Furthermore, this pre-trained encoder strongly supports subsequent training on the more challenging LUNA16 dataset. This method combines self-supervised generative training with an advanced encoder architecture, and is expected to significantly improve the model's performance and accuracy in lung nodule detection. The final prediction results of the self-supervised model are as follows: Figure 10 As shown.

[0096] (2) Results and Analysis of Self-Supervised Pre-training Experiment

[0097] The dataset used in the experiment was LUNA16, and the specific experimental results are shown in Table 3 below:

[0098] Table 3 Results of the Self-Supervised Experiment

[0099]

[0100] The experimental results above show that PMSN-YOLO performs better with pre-trained weights, and the prior knowledge acquired by the model from the weights plays a crucial role. Furthermore, this prior knowledge only requires unlabeled lung nodule data, which can be utilized with existing massive datasets.

[0101] 3. Supervised training using PMSN-YOLOX.

[0102] During supervised training of PMSN-YOLOX, lung parenchyma segmentation and lung parenchyma segmentation data augmentation are first performed on CT images in the labeled training set.

[0103] (1) Substantive division

[0104] Background removal of lung nodules is performed using lung parenchyma segmentation methods, since lung nodule regions only exist in the left and right lungs, and excess image information is unnecessary noise interference for the network model. Common lung parenchyma segmentation methods include thresholding segmentation, region growing segmentation, morphological operations, active contour models, graph-based methods, machine learning, and deep learning. This section only describes and improves thresholding segmentation and morphological operations.

[0105] Lung parenchyma segmentation mainly includes the following steps:

[0106] 1. In lung CT images, there are mainly two targets: the lung region and the lung bed. First, the image is binarized using the average pixel value. Then, the largest connected component in the entire image is retained to remove all interfering information outside the lung region, as follows: Figure 11 As shown in the first image on the left.

[0107] 2. After locating the lung region through the maximum connected component, some information loss may occur due to spikes and cavities near the edges. Therefore, a circular cell with a radius of 10 is used to perform a closing operation to avoid this information loss, as follows: Figure 11 As shown in the second image from the left.

[0108] 3. Then, paint over the background area of ​​the image with white. The remaining black area will then be the region where the lung parenchyma is located, as shown below. Figure 11 As shown in the third image from the left.

[0109] 4. Based on the obtained black region mask of the lung parenchyma, the lung parenchyma in the original image can be segmented by overlaying it with the original image, as shown below. Figure 11 As shown in the fourth image from the left.

[0110] After a series of thresholding and image morphological operations, lung parenchyma segmentation is complete. Lung parenchyma segmentation can remove the bed lamina and irrelevant tissue. However, this rapid lung parenchyma segmentation method is not perfect. Common-sized lung nodules in the central region of the lung parenchyma can be preserved, but some larger lung nodules at the lung parenchyma margins or in the central region may be classified as irrelevant tissue and segmented out during the thresholding step, as shown below. Figure 12 As shown, directly using the segmented lung parenchyma data as the training set of the model has a certain impact on the model's accuracy. Furthermore, subsequent prediction input images need to undergo lung parenchyma segmentation before lung nodule target detection, which increases the unreliability of the entire lung nodule detection system.

[0111] (2) Enhanced lung parenchyma segmentation data

[0112] Using the segmented lung parenchyma dataset directly significantly improves the detection accuracy for most lung nodules, but it can lead to missed detections for certain types of nodules. To avoid this problem, lung parenchyma segmentation is incorporated as a data augmentation method with random intensity during model training. In the fourth step of lung nodule segmentation, the segmentation mask is overlaid on the original image to obtain the segmented lung parenchyma region. The degree of overlay is used as a random parameter 'a'. The higher the value of 'a', the clearer the remaining areas are. When 'a=1', lung parenchyma segmentation is not performed, as shown below. Figure 13 As shown.

[0113] This lung parenchyma data augmentation method effectively avoids segmenting lung nodules during the lung parenchyma segmentation stage. Furthermore, the varying intensity values ​​of the background allow the network to fully recognize that background regions are unimportant to lung nodules, thus significantly expanding the lung nodule dataset. Simultaneously, for lung nodules that were segmented as background, corresponding features can still be learned even when the intensity value is not zero. To enable the model to learn the data distribution during prediction, this data augmentation is disabled at 70% of the total training epochs, allowing the model to learn the normal distribution of lung CT images.

[0114] (3) Experimental Results and Analysis

[0115] After augmenting the lung parenchyma segmentation data, YOLOX training based on the PMSN method was performed. During training, self-supervised pre-trained weights were used for model initialization. To ensure that the model could learn the correct data distribution, lung parenchyma segmentation data augmentation was turned off at 70% of the total rounds, allowing the model to learn the normal data distribution. The training results are shown in Table 4 below.

[0116] Table 4 PMSN-YOLO Training Results

[0117]

[0118] Analysis of the training results of the PMSN-YOLO model reveals that complete lung parenchyma segmentation leads to a decrease in the model's recall. This is because some nodules are segmented as background after lung parenchyma segmentation, resulting in some labeled locations lacking targets during training. Random-intensity lung parenchyma segmentation, on the other hand, helps improve the model's recall, reduces false negatives, and enhances overall model performance.

[0119] 4. PMSN-YOLOX Performance Prediction

[0120] (1) Under the same training conditions, a comparative experiment on lung nodule detection between YOLOX and YOLOX based on progressive multiscale fusion network (PMSN-YOLOX) was conducted. It was found that the performance indicators of YOLOX were lower than those of PMSN-YOLOX. The specific training results are shown in Table 5 below.

[0121] Table 5. Training Results of PMSN-YOLOX

[0122]

[0123] as follows Figure 14As shown, directly using YOLOX for lung nodule detection under the same training conditions leads to several problems. The prediction indicated by the arrow clearly does not belong to a lung nodule. This is because CT images provide limited information, and lung nodules are small, landmark objects; the YOLOX model performs poorly in feature extraction for such objects. In contrast, PMSN-YOLOX incorporates a multi-scale fusion approach in feature extraction, making it more effective for feature extraction of small, landmark objects like lung nodules, which occupy less than 1% of the image area.

[0124] (2) Experimental Results and Analysis of Self-Supervised Pre-training Weights

[0125] A comparative experiment was conducted to compare the use of self-supervised pre-trained weights with and without them. The Self+PMSN-YOLOX network uses self-supervised pre-trained weights. The specific experimental results are shown in Table 6 below.

[0126] Table 6 Results of the Self-Supervised Experiment

[0127]

[0128] The experimental results above show that PMSN-YOLOX performs better with pre-trained weights. The prior knowledge acquired by the model from the weights plays a crucial role, improving the model's recall by 1.57%. Furthermore, this acquisition of prior knowledge only requires unlabeled lung CT data, which can reduce the amount of data required when dealing with more complex models.

[0129] (3) Results and analysis of lung parenchyma segmentation data enhancement experiments

[0130] PMSN-YOLOX training was performed after lung parenchyma segmentation data augmentation. Self-supervised pre-trained weights were used for model initialization during training. To ensure the model learned the correct data distribution, lung parenchyma segmentation data augmentation was turned off at 70% of the total epochs, allowing the model to learn the normal data distribution. Lung_seg represents the lung parenchyma segmentation data augmentation used in this section, where Lung_seg(0) means only lung parenchyma is retained, and Lung_seg(random) means the background intensity is set randomly. The training results are shown in Table 7 below.

[0131] Table 7 Results of Augmentation Training of Lung Parenchyma Segmentation Data

[0132]

[0133] After improvements to the self-supervised and PMSN methods, false positives can be significantly reduced, and the overall recall and accuracy in the lung nodule detection process can be increased. However, due to limitations in the dataset, there will still be a small number of false positives. Lung parenchyma segmentation can efficiently and quickly avoid such false positives.

[0134] Analysis of the training results of the Self+PMSN-YOLOX model reveals that complete lung parenchyma segmentation leads to a decrease in the model's recall. This is because some nodules are segmented as background after lung parenchyma segmentation, resulting in some labeled locations being unlabeled during training. Random-intensity lung parenchyma segmentation, on the other hand, helps improve the model's recall and reduce false negatives, while also significantly expanding the dataset.

[0135] (4) Results and Analysis of Ablation Experiments for Lung Nodules

[0136] The entire ablation experiment was based on a lung nodule detection method that only included the YOLOX network and a false positive classification network, and was improved upon. Here, YOLOX represents the basic detection method, PMSN-YOLOX represents replacing the backbone network of the basic detection method's YOLOX with PMSNet, Self represents using self-supervised pre-trained weights, and Lung_seg represents using lung parenchyma segmentation data augmentation. To achieve high sensitivity for lung nodule detection and reduce the number of missed nodules, the confidence level of the target detection network's prediction results was adjusted to 0.25. Although lowering the confidence level introduces more false positives, these are eliminated during post-processing screening and false positive detection. The overall experimental results are shown in Table 8 below.

[0137] Table 8. Experimental Performance of Lung Nodule Detection Algorithm in Ablation

[0138]

[0139] Based on the ablation experiment results above, it is clear that each improvement has a positive impact on the overall performance of lung nodule detection. Firstly, by replacing the YOLOX network with PMSN-YOLOX, a comprehensive improvement in model performance was observed across various average false positive levels, indicating that PMSN-YOLOX has superior feature extraction capabilities. This enables the model to exhibit higher sensitivity for lung nodules of different sizes.

[0140] Introducing self-supervised pre-trained weights further improved the model's sensitivity, highlighting the importance of prior knowledge in model training. It helps the model more accurately learn in the correct direction while significantly reducing the average number of candidate scans (ACPS), thus dramatically improving the method's efficiency in nodule detection.

[0141] Furthermore, by enhancing lung parenchyma segmentation, the model's sensitivity was significantly improved at low false positive rates, meaning a substantial reduction in both false negatives and false negatives of lung nodules. This data augmentation method effectively conveyed the relative irrelevance of the background region to the model, reducing the likelihood of the model misidentifying background areas as candidate nodules at low confidence levels, thereby further reducing the number of false positives.

[0142] In summary, these improvements significantly enhance the accuracy and efficiency of the model, ensuring the reliability and stability of the method when faced with complex medical image analysis tasks.

Claims

1. A lung nodule detection method based on a progressive multi-scale network model, characterized in that, Includes the following steps: S1: Construct a progressive multi-scale network model PMSN-YOLOX, which uses YOLOX as the basic network and replaces CSPrayer in YOLOX with convolutional block PMSNet; The convolutional block PMSNet in S1 consists of a 1x1 convolutional kernel layer that preprocesses the input image features, connected sequentially, and a structural block. The structural block includes four branches, each branch comprising... The convolutional layer, with an increasing number of convolutional layers, is used. The output of the first CBL in the first branch is fed into the input of the second CBL in the second branch, the output of the second CBL in the second branch is fed into the input of the third CBL in the third branch, and the output of the third CBL in the third branch is fed into the input of the fourth CBL in the fourth branch. Simultaneously, the outputs of the last CBL in each branch are concatenated and then processed by a... Convolution is used for dimensionality reduction. S2: Self-supervised pre-training of PMSN-YOLOX is performed using existing public datasets. The parameters obtained after self-supervised pre-training are used as the initialization parameters of the backbone network. Then, supervised training of PMSN-YOLOX is performed using a labeled training set, which contains several labeled medical images of lung nodules. In supervised training, the optimal PMSN-YOLOX is obtained when the loss function no longer changes. S3: Input a new medical image of the lungs into the optimal PMSN-YOLOX, and output the probability value of whether the patient has lung nodules and the coordinates of the bounding box of the lung nodules.

2. The lung nodule detection method based on a progressive multi-scale network model as described in claim 1, characterized in that, The process of self-supervised pre-training of PMSN-YOLOX in S2 is as follows: During self-supervised pre-training of PMSN-YOLOX, the encoder is a PMSNet network. First, a random mask array is generated. This random mask array is then multiplied by the input image to determine if more than 75% of the image is randomly masked. Each number in the generated random mask array represents a region in the image... The region pixel values, where 0 indicates masking and 1 indicates preservation; by replacing the 2D convolution in the encoder with sparse convolution, feature maps can be obtained by extracting features only from the unmasked regions of the image; The features of each layer of the encoder are fused with the features upsampled by the decoder, and prediction is performed at the output of the decoder to predict the pixel value of each pixel in the masked region. Then, using the following formula (8), the similarity between the predicted image and the original image before masking is calculated. This similarity is the loss function for updating weights during PMSN-YOLOX self-supervised pre-training. ; （8） In this loss function, express loss; Indicates the number of samples; Indicates the feature dimension; Represents the first of the original inputs. The first sample One eigenvalue; This indicates the output of PMSN-YOLOX. The first sample One eigenvalue; when If the value no longer changes, it means that pre-training is complete. After pre-training, the encoder part of the network is retained, and the original sparse convolutional layer is replaced with a two-dimensional convolutional kernel.

3. The lung nodule detection method based on a progressive multi-scale network model as described in claim 1, characterized in that, When performing supervised training of PMSN-YOLOX using a labeled training set in S2, lung parenchyma segmentation data augmentation processing is first performed on the images in the training set, as follows: 1) Use the average pixel value of the image to binarize the image, and then retain the largest connected component in the entire image to remove all interfering information outside the lung region; 2) After finding the lung region through the maximum connected component, a circular cell with radius r is used to perform a closing operation to avoid information loss; 3) Next, paint the background area in the image white, and the remaining black area is the area where the lung parenchyma is located; 4) Obtain a segmentation mask image based on the black region mask where the lung parenchyma is located. Overlap the segmentation mask image with the original image according to the degree of random parameter a to segment out the lung parenchyma and obtain the lung parenchyma segmentation enhancement image.

Images