False detection and missing detection method for pulmonary nodule detection
Through the combination of the AFPN-YOLO network and Focal CIoU loss function, the data labeling difficulties, diversity differences and sample imbalance in lung nodule detection are solved, and the accuracy and robustness of lung nodule detection are improved, especially the detection effect of small targets.
Patent Information
- Application Number
- CN202510478886.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-04-16
- Publication Date
- 2025-08-12
AI Technical Summary
The prior art has problems of misdetection and missed detection due to the lack of large-scale labeling data, data diversity differences, training samples imbalance, and small size of lung nodules.
AFPN-YOLO pulmonary nodule detection network based on a progressive feature fusion structure is adopted, combining an efficient multi-scale attention module and an improved loss function Focal CIoU, through adaptive spatial fusion mechanism and multi-scale feature extraction, the importance of key levels is enhanced, the semantic gap is reduced, and the target detection performance is improved.
It significantly improves the recall and accuracy of lung nodule detection, improves the detection ability of small targets and occluded targets, and reduces the rate of false detection and missed detection.
Smart Images
Figure CN120472272A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the fields of deep learning target detection and computer-aided diagnosis of medical cancer, and specifically to a deep learning-based computer-aided system for lung nodule detection, which enables early symptom detection and treatment of lung cancer, helping to improve patients' survival rate and quality of life. Background Art
[0002] Deep learning-based lung nodule detection is a research direction in the field of medical imaging. It aims to automatically or assist doctors in detecting nodules in patients' lung CT images through deep learning algorithms to help with early detection and treatment of lung cancer.
[0003] The main difficulties in deep learning-based lung nodule detection are:
[0004] The first is the lack of large-scale annotated data. Data annotation is difficult. Obtaining a large-scale annotated lung nodule dataset is a difficult task because the annotation of medical imaging data requires professional knowledge and experience.
[0005] The second is that differences in scanning equipment and parameters cause data diversity. CT scan images produced by different hospitals and equipment may vary greatly, which challenges the model's generalization ability in different environments.
[0006] The third is the imbalance of training samples, the imbalance of positive and negative samples. Since lung nodules are usually rare, the imbalance of positive and negative samples may cause the model to tend to predict more negative samples.
[0007] Finally, lung nodules are small in size and contain little information in the overall lung image, which can easily lead to misdetection or missed detection. Summary of the Invention
[0008] The technical purpose of this invention is to propose a lung nodule detection method based on deep learning neural network, which mainly solves the following problems:
[0009] 1. An AFPN-YOLO lung nodule detection network based on a progressive feature fusion structure is proposed. To address the problems of false detection and missed detection of lung nodules, the AFPN structure is used in the feature fusion stage, and an adaptive spatial fusion mechanism is introduced. Varying spatial weights are introduced in the multi-level feature fusion process to emphasize the importance of key levels. The idea of progressive fusion helps to reduce the semantic gap between features at different levels, improve feature fusion effects, and enhance target detection performance.
[0010] 2. To address the problem of complex background and small target size in lung nodule detection, an efficient multi-scale attention (EMA) module is introduced into the backbone network to reduce computational overhead while retaining the key information of each channel and enhancing the feature representation capability.
[0011] 3. Add a small target detection layer to improve the detection capability of small targets
[0012] 4. Improve the loss function. The loss function is used to calculate the regression loss of the bounding box. Optimizing the loss function can effectively improve the bounding box positioning accuracy of the model in target detection tasks, ensuring that the detection box can better overlap with the real target. This invention uses Focal CIOU, combining Focal Loss and CIOU loss, to not only address the problem of class imbalance but also optimize the accuracy of the bounding box, especially for the detection of small and occluded targets.
[0013] The technical solution employed in the present invention is a method for preventing false detection and missed detection of lung nodules. This method is based on a false detection and missed detection model for lung nodule detection, which comprises four modules: an image preprocessing module, a feature extraction module, a feature fusion module, and a detection and recognition module. The collected raw lung images are preprocessed by the image preprocessing module. The preprocessed lung images are then fed into the feature extraction module, which extracts low-level and high-level features from the lung images. The extracted lung image feature maps at different scales are fed into the feature fusion module to fuse the features for target detection. The fused lung image feature maps are then fed into the detection and recognition module, which generates the final prediction results. The feature extraction network typically consists of multiple layers, with the features extracted at each layer becoming increasingly abstract. Low-level features are typically simple features extracted directly from the raw data, such as edges and textures in lung images. These features have low semantic information and are primarily used to describe the basic components of the image. High-level features, on the other hand, are more complex features extracted by combining multiple low-level features and are capable of capturing more semantically relevant content. These features have a higher level of abstraction and can represent the overall characteristics of the shape, category, or scene of the lung image detection target. Through this layer-by-layer extraction, the network can effectively transform raw data into useful high-level features to support subsequent classification tasks.
[0014] Furthermore, the image preprocessing module performs lung parenchyma segmentation on the input lung image. In lung CT images, the trunk and bed are common anatomical structures or equipment elements that typically appear as background in lung images. Lung parenchyma segmentation removes these background elements, leaving only the lung tissue required for subsequent detection and reducing noise interference during the detection process. Lung parenchyma segmentation uses a threshold to separate the lung parenchyma from other thoracic structures and, based on a region growing method, obtains connected left and right lung images. Images of the main trachea and main bronchi extending into the lungs are removed, and holes formed by blood vessels, nodules, and other structures in the parenchyma are filled to obtain a complete lung tissue image.
[0015] Furthermore, the feature extraction module uses a convolutional neural network to extract lung image features, extracting features at different levels within lung images through a multi-layer convolutional neural network structure. The feature extraction module includes convolutional layers, a C2f module, and a spatial pyramid pooling module. The convolutional layers extract preliminary features, the C2f module fuses and enhances these features, and the SPPF module generates the final feature representation through multi-scale pooling. Deeper convolution layers produce more abstract features, but also lose a lot of detailed information, making small-scale object detection difficult. Therefore, an efficient multi-scale attention mechanism is introduced at the bottom of the backbone network. Small and large objects are represented differently in feature maps. The EMA module uses a multi-scale feature extractor to extract information from feature maps of different scales, capturing the characteristics of objects of different scales in the image.
[0016] After extracting multi-scale features, the EMA module introduces an attention mechanism to emphasize important features and suppress irrelevant or redundant features. It then aggregates attention-weighted features from different scales, fusing multi-scale information and making the model effective for detecting both small and large objects.
[0017] By dynamically adjusting the weights of feature maps at different scales and combining spatial and channel attention, the model's ability to utilize multi-scale features is significantly improved. This not only effectively captures information about objects of varying sizes, but also optimizes computational processes for efficient resource utilization, making the model more accurate and efficient in visual tasks. This goal is achieved by dividing channels into subgroups and balancing spatial semantic features within feature groups, while maintaining information validity and reducing computational costs.
[0018] Spatial semantic features are an efficient feature representation that combines the spatial information of objects in an image (such as the location and shape of nodules) with semantic information (including category, meaning, and context). The key to spatial semantic features is the organic integration of spatial and semantic information, enabling the model to more comprehensively understand the image content.
[0019] Furthermore, the feature fusion module introduces a progressive feature fusion strategy at the neck layer, gradually integrating features from the bottom, top, and bottom layers into the object detection process. This helps reduce the semantic gap between features at different levels, improves feature fusion, and enables the false detection and missed detection model for lung nodule detection to better adapt to semantic information at different levels. The lung image feature maps at different scales output by the feature extraction module are gradually integrated into the feature fusion module. A progressive feature pyramid network is incorporated into the fusion module to gradually fuse lung image feature maps from different scales. Feature fusion is not a one-time process, but rather a layer-by-layer process of progressively incorporating high-level features into lower-level features. Starting with low-level features, high-level semantic features are gradually incorporated into the fusion process. This preserves low-level detail information at each fusion step while gradually enhancing the semantic expressiveness of the features. An adaptive fusion operation is employed to fuse different features. Based on the contribution of features to different tasks, the adaptive mechanism assigns appropriate weights to features at different levels. The varying spatial weights better preserve key features, enabling better detection of lung nodules of varying sizes. The module extracts the final layer of features at different scales from each feature layer of the backbone, denoted as {S2, S3, S4, S5}. It first fuses the bottom-level features of S2 and S3 at different resolutions, then adds S4, and finally S5. Through this gradual feature fusion, a set of multi-scale features {C2, C3, C4, C5} is obtained. AFPN fuses two bottom-level features in the initial stage. Subsequent stages fuse deeper features, and the final stage incorporates top-level features in the feature fusion process.
[0020] Furthermore, the detection layer of the detection and recognition module has been expanded to include a target detection head, increasing the number from three to four. This head detects the fused shallow feature layer, improving the accuracy of small target detection and reducing missed and false detection rates. This additional target detection head can detect and identify shallow feature maps of the input lung image, improving the detection capability of small nodules and reducing the false detection rate.
[0021] Furthermore, the loss function is replaced with Focal CIoU. Focal CIoU combines the idea of Focal Loss to solve the class imbalance problem by giving greater weight to difficult-to-classify samples with CIoU loss, focusing on improving the false detection and missed detection model of lung nodule detection's attention to and detection ability of difficult-to-detect targets. The Focal CIoU formula is:
[0022]
[0023] α t is a balancing factor used to adjust the weights of positive and negative samples;
[0024] p tis the predicted probability value;
[0025] γ is a regulating factor that controls the weight of difficult-to-classify samples.
[0026] IoU represents the intersection over union (IoU) of the predicted box and the true box.
[0027] ρ(b,b gt ) represents the Euclidean distance between the center point of the predicted box and the center point of the true box.
[0028] c is the diagonal length of the bounding box between the predicted box and the ground-truth box.
[0029] v is a term related to the aspect ratio of the predicted box and the true box.
[0030] α is a coefficient used to balance the aspect ratio term.
[0031] The input lung image first needs to be preprocessed to remove lung tissue irrelevant to the detection, reduce background noise, and lower the difficulty of detection. The processed image will be subjected to feature extraction to obtain feature maps of different scales. Because the increase in the number of convolutions will lead to the loss of detailed information, some small targets with fewer pixels cannot be detected. In this module, four different levels of feature maps are extracted. These four feature maps come from different levels of the feature extraction network, with resolutions of 1 / 4, 1 / 8, 1 / 16, and 1 / 32 of the input image, respectively. They contain information of different scales and semantics, which can better detect and recognize targets of different sizes. A shallow feature is introduced on the basis of the original, which is conducive to the detection of small nodules. Then, the feature maps of different scales are fused through feature fusion. The progressive fusion strategy and adaptive fusion operation can reduce the fusion error caused by the semantic gap between features at different levels, so that the model can detect targets of different sizes. The fused feature map is input into the detection and recognition module for target detection. Adding a detection head to the original three basic detection heads helps to solve the problem of missed detection of lung nodules.
[0032] Compared with existing technologies, the improved model of this invention achieved superior results on a lung nodule dataset. The improved YOLOv8 algorithm achieved a 1.1% increase in recall rate, reaching 0.901, compared to the original algorithm. This indicates that the improved algorithm has improved the missed detection problem of lung nodules. The training accuracy also reached 0.938, a significant improvement. The mAP50 increased by 0.5% to 0.944, and the mAP50-95 also increased by 0.8% to 0.686. This shows that the YOLOv8 algorithm, which improves the backbone network and introduces a progressive feature fusion module and a small target detection layer, is very effective in detecting lung nodules with few pixels in lung CT images. BRIEF DESCRIPTION OF THE DRAWINGS
[0033] Figure 1 It is a progressive feature pyramid structure diagram.
[0034] Figure 2 This is a working diagram of the lung nodule detection network structure.
[0035] Figure 3 It is a detection flow chart.
[0036] Figure 4 This is the structure diagram of the multi-scale attention module.
[0037] Figure 5 It is the loss function, accuracy, recall rate, and map data graph of the original algorithm.
[0038] Figure 6 It is the map data graph of the original algorithm.
[0039] Figure 7 This is the F1 data change graph of the original algorithm.
[0040] Figure 8 It is the loss function, accuracy, recall rate, and map data graph of this algorithm.
[0041] Figure 9 It is the map data graph of this algorithm.
[0042] Figure 10 This is the F1 data change graph of this algorithm. DETAILED DESCRIPTION
[0043] The present invention is described in detail below with reference to the accompanying drawings and embodiments.
[0044] Through the CT image processed by lung parenchyma segmentation, the target detection network can be used to accurately locate the position of lung nodules. The invention uses the YOLOv8 network as the basic skeleton and makes a series of reasonable and effective modifications to it to obtain a new network, making it more suitable for target detection in this scenario than the original version.
[0045] The present invention comprises four modules in total: an image preprocessing module, a feature extraction module, a feature combination module, and a detection and recognition module.
[0046] 1. Image preprocessing module
[0047] When detecting lung nodules, the torso and bed board in the lung CT image are not used. Segmenting the lung parenchyma in the image preprocessing stage can greatly reduce the computational complexity of subsequent operations. Lung parenchyma segmentation uses a threshold to separate the lung parenchyma from other structures in the chest cavity, and obtains the connected left and right lungs based on the region growing method. The main trachea and the main bronchi extending into the lungs are removed, and the holes formed by blood vessels, nodules, etc. in the parenchyma are filled to obtain complete lung tissue. The pseudo code for its implementation process is shown below.
[0048]
[0049]
[0050] 2. Feature extraction module
[0051] This invention improves the original backbone feature extraction network of yolov8 and introduces a high-efficiency multi-scale attention module (EMA) to reduce computational overhead while retaining key information in each channel. Small and large objects are represented differently on feature maps. The EMA module uses a multi-scale feature extractor to extract information from feature maps of different scales, capturing the characteristics of objects of different scales in an image.
[0052] After extracting multi-scale features, the EMA module introduces an attention mechanism to emphasize important features and suppress irrelevant or redundant ones. It then aggregates attention-weighted features from different scales, fusing information from multiple scales. This makes the model effective for detecting both small and large objects. Furthermore, the EMA module is typically designed with a lightweight structure to reduce computational effort while maintaining high model efficiency.
[0053] 3. Feature fusion module
[0054] In the unmodified YOLOv8 model, the feature fusion module for object detection uses the FPN-PAN architecture to achieve multi-scale feature fusion. This original feature fusion architecture fails to meet the model's accuracy requirements, prone to missed detections and false detections, resulting in insufficient accuracy. Furthermore, in classic multi-scale feature fusion, high-order features propagate and interact with underlying low-order features through multiple intermediate scales, which can easily lead to feature information loss or degradation, reducing the effectiveness of feature fusion at non-adjacent levels.
[0055] The present invention uses a progressive feature pyramid to replace the original neck structure of the YOLOv8 model, and adaptively spatially fuses the features extracted from the backbone from a low level to a high level.
[0056] 4. Improvement of loss function
[0057] The original YOLOv8 model uses a combination of CIoU and DFL loss to calculate the bounding box regression loss. This loss function strategy does not consider the balance between easy and difficult samples, and has a low contribution to small objects, causing the model to easily ignore these objects.
[0058] This paper replaces the loss function with Focal CIoU. Focal CIoU combines the idea of Focal Loss to solve the class imbalance problem by giving greater weight to difficult-to-classify samples (i.e., positive samples or difficult-to-distinguish negative samples) with CIoU loss, focusing on improving the model's attention and detection ability for difficult-to-detect targets. The Focal CIoU formula is:
[0059]
[0060] α t Is the balance factor, used to adjust the weight of positive and negative samples
[0061] p t is the predicted probability value
[0062] γ is a regulating factor that controls the weight of difficult-to-classify samples (usually set to 2). IoU represents the intersection over union (IoU) of the predicted box and the true box.
[0063] ρ(b,b gt ) represents the Euclidean distance between the center point of the predicted box and the center point of the ground-truth box. c is the diagonal length of the bounding box of the predicted box and the ground-truth box.
[0064] v is a term related to the aspect ratio of the predicted box and the true box.
[0065] α is a coefficient used to balance the aspect ratio term.
[0066] Table 1 Performance comparison between the original algorithm and the improved algorithm
[0067]
[0068] Figure 5 It is the loss function, accuracy, recall rate, and map data graph of the original algorithm.
[0069] Figure 6 It is the map data graph of the original algorithm.
[0070] Figure 7 This is the F1 data change graph of the original algorithm.
[0071] Figure 8 It is the loss function, accuracy, recall rate, and map data graph of this improved algorithm.
[0072] Figure 9 It is the map data graph of this improved algorithm.
[0073] Figure 10 This is the F1 data change diagram of the improved algorithm.
Claims
1. A method for detecting false or missed lung nodules, characterized in that: The method is implemented based on a false detection and missed detection model for lung nodule detection. The false detection and missed detection model for lung nodule detection includes four modules: an image preprocessing module, a feature extraction module, a feature fusion module, and a detection and recognition module. The collected original lung image is preprocessed by the image preprocessing module. The preprocessed lung image is input into the feature extraction module, which extracts low-level and high-level features of the lung image. The extracted lung image feature maps of different scales are input into the feature fusion module to fuse the features of different scales for target detection. The fused lung image feature maps are input into the detection and recognition module, which generates the final prediction result. The feature extraction network consists of multiple layers, and the features extracted at each layer are gradually abstracted. High-level features are more complex features extracted by combining multiple low-level features. They can capture more semantic content and represent the overall characteristics of the shape, category or scene of the lung image detection target. Through layer-by-layer extraction, the raw data is converted into useful high-level features to support subsequent classification tasks.
2. The method for detecting false or missed pulmonary nodules according to claim 1, wherein: The image preprocessing module performs lung parenchyma segmentation on the input lung image. In lung CT images, the trunk and bed are common anatomical structures or equipment elements, which serve as the background in the lung image. Lung parenchyma segmentation is to remove these background elements such as the bed and trunk from the lung image, leaving only the lung tissue required for subsequent detection, thereby reducing noise interference during the detection process.
3. The method for detecting false detection and missed detection of pulmonary nodules according to claim 2, characterized in that: The lung parenchyma segmentation uses a threshold to separate the lung parenchyma from other structures of the thoracic cavity, and obtains connected left and right lung images based on the region growing method, removes the main trachea and the main bronchi extending into the lungs, and then fills the holes formed by blood vessels and nodules in the parenchyma to obtain a complete lung tissue image.
4. The method for detecting false or missed lung nodules according to claim 1, wherein: The feature extraction module uses a convolutional neural network to extract lung image features, extracting features at different levels in lung images through a multi-layer convolutional neural network structure. The feature extraction module includes a convolutional layer, a C2f module, and a spatial pyramid pooling module. The convolutional layer extracts preliminary features, the C2f module fuses and enhances these features, and the SPPF module generates the final feature representation through multi-scale pooling. An efficient multi-scale attention mechanism is introduced at the bottom of the backbone network. Small and large objects are represented differently on feature maps. The EMA module uses a multi-scale feature extractor to extract information from feature maps of different scales, capturing the characteristics of objects of different scales in the image. After extracting multi-scale features, the EMA module introduces an attention mechanism to emphasize important features and suppress irrelevant or redundant features; Then, attention-weighted features from different scales are aggregated to fuse multi-scale information, making the model effective in detecting both small and large objects. By dynamically adjusting the weights of feature maps at different scales and combining spatial and channel attention, the ability to utilize multi-scale features is improved; by dividing channels into subgroups and balancing the spatial semantic features within feature groups, the goal of maintaining information validity and reducing computational costs is achieved; Spatial semantic features are an efficient feature representation that combines the spatial information and semantic information of objects in an image; spatial The key to semantic features is to organically combine spatial information and semantic information, allowing the model to understand the image content more comprehensively.
5. The method for detecting false or missed lung nodules according to claim 1, wherein: The feature fusion module introduces a progressive feature fusion strategy at the neck layer, gradually integrating the features of the bottom, top and top layers into the target detection process, reducing the semantic gap between features at different levels, improving the feature fusion effect, and making the false detection and missed detection model of lung nodule detection adapt to semantic information at different levels; the lung image feature maps of different scales output by the feature extraction module are input into the feature fusion module for gradual integration, and a progressive feature pyramid network is introduced into the fusion module to gradually fuse the lung image feature maps from different scales. Feature fusion progressively introduces high-level features into low-level features layer by layer; starting from low-level features, high-level semantic features are gradually introduced into the fusion process, retaining the low-level detail information in each step of the fusion, while gradually enhancing the semantic representation of the features. The adaptive fusion operation is used when fusing different features. According to the contribution of the features in different tasks, the adaptive mechanism assigns appropriate weights to features at different levels. The varying spatial weights retain the key levels for the detection of lung nodules of different sizes. The feature fusion module extracts the last layer of features of different scales from each feature layer of the backbone, denoted as {S2, S3, S4, S5}, first fuses the bottom-level features of S2 and S3 with different resolutions, then adds S4, and finally adds S5. After gradual feature fusion, a set of multi-scale features {C2, C3, C4, C5} is obtained. AFPN fuses two bottom-level features in the initial stage; the subsequent stages fuse deep features, and the final stage adds top-level features in the feature fusion process.
6. The method for detecting false or missed lung nodules according to claim 1, wherein: An additional target detection head is added to the detection layer stage of the detection and recognition module, increasing the number of detection heads from three to four, to detect the shallow feature layer obtained by fusion, improve the accuracy of small target detection, and reduce the incidence of missed and false detection of target detection; the added target detection head detects and recognizes the shallow feature map of the input lung image, improves the detection capability of small-sized nodules, and reduces the false detection rate of nodules.
7. The method for detecting false or missed lung nodules according to claim 1, wherein: Replace the loss function with Focal CIoU. Focal CIoU gives Focal Loss a greater weight to difficult-to-classify samples. The Focal CIoU formula is: α t is a balancing factor used to adjust the weights of positive and negative samples; p t is the predicted probability value; γ is a regulating factor that controls the weight of difficult-to-classify samples; IoU represents the intersection-over-union ratio between the predicted box and the true box; ρ(b,b gt ) represents the Euclidean distance between the center point of the predicted box and the center point of the real box; c is the diagonal length of the bounding box of the predicted box and the true box; v is a term related to the aspect ratio of the predicted box and the true box; α is a coefficient used to balance the aspect ratio term.