CT image pulmonary nodule detection and identification method based on improved YOLOv8 model

Abstract

The invention discloses a CT image pulmonary nodule detection and identification method based on an improved YOLOv8 model, and belongs to the technical field of target detection, and the method comprises the steps: obtaining a lung CT image from an original mhd file in an LUNA16 data set, and generating a lung CT image original data set; s2, lung parenchyma segmentation is carried out on the lung CT image original data set generated in the step S1, the lung CT image original data set is optimized, and a region of interest is reserved; the YOLOv8 model is improved; and S3, extracting image features of the training set of the original data set of the lung CT image by using the improved YOLOv8 model obtained in the step S3 to obtain a trained CT image pulmonary nodule detection and recognition model, and testing on a test set. According to the CT image pulmonary nodule detection and identification method based on the improved YOLOv8 model, the generalization ability of the detection model and the CT image pulmonary nodule identification precision are effectively improved, and the size and the position of the pulmonary nodule in the CT image can be detected more accurately.

Description

Technical Field

[0001] The present invention relates to the technical field of target detection, and in particular to a method for detecting and identifying lung nodules in CT images based on an improved YOLOv8 model. Background Art

[0002] Lung cancer is one of the leading causes of death worldwide, and early screening and diagnosis are crucial to improving patient survival rates. Computed tomography (CT) images have become a core tool for lung cancer diagnosis due to their high resolution and sensitivity. CT images can clearly display the structure of lung tissue and support three-dimensional reconstruction, which helps to assess the size, location and spread of tumors, and thus accurately stage them. However, in the face of a large amount of image data, traditional manual reading methods have problems such as long time consumption and large subjective differences, which limits the efficiency and effectiveness of CT technology in clinical application.

[0003] In recent years, deep learning technology has been widely used in the field of medical image analysis, especially FasterR-CNN, YOLO, and SSD algorithms, which can significantly improve the efficiency and accuracy of lung nodule detection. YOLO is a real-time object detection algorithm that can predict multiple bounding boxes and their categories in a single forward propagation process. Through training with a large amount of labeled data, the YOLO model can automatically identify and classify lung nodules in CT images, extract complex features such as shape, boundaries, and density, and thus achieve efficient and accurate detection results. Deep learning models can not only improve the accuracy of lung nodule detection, but also reduce the occurrence of misdiagnosis and missed diagnosis. Studies have shown that the performance of the trained YOLO model on some tasks is comparable to or even exceeds that of radiologists. Especially in large-scale lung cancer screening projects for high-risk populations, YOLO's efficient computing power makes real-time and large-scale screening possible, and can extract quantitative features from medical images in a non-invasive way, significantly reducing the burden on the medical system.

[0004] However, medical image analysis technology based on deep learning still faces many problems and challenges. For example, there are interferences such as pulmonary arteries, pleura, heart, and bones in abdominal CT images. By segmenting the lung parenchyma from CT scan images, the speed and accuracy of lung nodule detection can be improved; in addition, lung nodules are often very small (especially in early detection), making it difficult for deep learning models to focus and extract enough features. The shapes and densities of lung nodules are diverse, which increases the complexity of the model. Therefore, how to improve its modeling and algorithm to improve its accuracy and robustness is an urgent problem to be solved. Summary of the invention

[0005] The purpose of the present invention is to provide a CT image lung nodule detection and recognition method based on an improved YOLOv8 model to solve the problems existing in the above-mentioned background technology.

[0006] To achieve the above object, the present invention provides a method for detecting and identifying lung nodules in CT images based on an improved YOLOv8 model, comprising the following steps:

[0007] S1. Obtain lung CT images from the original mhd file in the LUNA16 dataset to generate an original lung CT image dataset;

[0008] S2, performing lung parenchyma segmentation on the original lung CT image data set generated in step S1, optimizing the original lung CT image data set, and retaining the region of interest;

[0009] S3. Improved YOLOv8 model;

[0010] S4. Use the improved YOLOv8 model obtained in step S3 to extract image features of the training set (subset1 to subset9) of the original lung CT image data set to obtain a trained CT image lung nodule detection and recognition model, and test it on the test set (subset0).

[0011] Preferably, in step S1, specifically: generate a 640×640×3 image from the original mhd file in the LUNA16 data set, read the lung nodule location information, and convert it into a VOC format that is easy to process.

[0012] Preferably, in step S2, lung parenchyma segmentation is performed on the original data set of lung CT images generated in step S1, specifically including: using a lung parenchyma segmentation algorithm to process the original data set of lung CT images, first standardizing the CT image, then performing K-means clustering and morphological processing on it to generate a lung CT mask image, and then using the mask image to segment the original image to remove unnecessary areas outside the lungs.

[0013] Preferably, step S3 specifically comprises:

[0014] Backbone part of the YOLOv8 model: The convolution module in the original Backbone part is replaced by a more lightweight ADown downsampling module to reduce the number of parameters while maintaining the accuracy of lung nodule detection;

[0015] Neck part of the YOLOv8 model: A lightweight SimAM self-attention mechanism is introduced after the C2f convolution module, and a Slim-Neck structure is adopted to improve the feature fusion and processing effects of the YOLOv8 model. Lightweight GSConv and VoVGSCSP are used to replace the standard convolution and C2f convolution modules.

[0016] Head part of the YOLOv8 model: Introducing the SelfAttention mechanism.

[0017] The YOLOv8 model (YOLOv8m) network enhanced by the attention mechanism includes:

[0018] Backbone:

[0019] Network input: 3 channels, size 640×640;

[0020] Layer 0: CBS structure (Conv-BN-SiLU), convolution kernel size is 3×3, stride is 2, number of input channels is 3, number of output layer channels is 48. After convolution operation on the input feature map, the number of channels becomes 48, and the feature map size is reduced to 320×320;

[0021] Layer 1: ADown structure, Average Pooling is performed on the input feature map, and then the feature map is divided into two parts. The first part is convolved, and the second part is convolved after Max Pooling. Then the two parts are concat. After the ADown operation is performed on the input feature map, the number of channels becomes 96, and the feature map size is reduced to 160×160;

[0022] Layer 2: C2f structure, the number of input channels is 96, the input feature map is split, convolved, residual connected and other operations are performed, the number of output channels becomes 96, and the feature map size is 160×160;

[0023] Layer 3: ADown structure, Average Pooling is performed on the input feature map, and then the feature map is divided into two parts. The first part is convolved, and the second part is convolved after Max Pooling. Then the two parts are concat. After the ADown operation is performed on the input feature map, the number of channels becomes 192, and the feature map size is reduced to 80×80;

[0024] Layer 4: C2f structure, the number of input channels is 192, the input feature map is split, convolved, residual connected and other operations are performed, the number of output channels becomes 192, and the feature map size is 80×80;

[0025] Layer 5: ADown structure, Average Pooling is performed on the input feature map, and then the feature map is divided into two parts. The first part is convolved, and the second part is convolved after Max Pooling. Then the two parts are concat. After the ADown operation is performed on the input feature map, the number of channels becomes 384, and the feature map size is reduced to 40×40;

[0026] Layer 6: C2f structure, the number of input channels is 384, the input feature map is split, convolved, residual connected and other operations are performed, the number of output channels becomes 384, and the feature map size is 40×40;

[0027] Layer 7: ADown structure, Average Pooling is performed on the input feature map, and then the feature map is divided into two parts. The first part is convolved, and the second part is convolved after Max Pooling. Then the two parts are concat. After the ADown operation is performed on the input feature map, the number of channels becomes 576, and the feature map size is reduced to 20×20;

[0028] Layer 8: C2f structure, the number of input channels is 576, the input feature map is split, convolved, residual connected and other operations are performed, the number of output channels becomes 576, and the feature map size is 20×20;

[0029] Layer 9: SPPF structure, the number of input channels is 576, the pooling kernel size is 5×5, it extracts features of different scales in the feature map and performs feature fusion;

[0030] Neck:

[0031] Layer 10: Upsample structure, the number of input channels is 576, the upsampling algorithm uses nearest, after upsampling the input feature map, the number of channels becomes 576, and the feature map size is expanded to 40×40;

[0032] Layer 11: Concat structure, which concatenates the feature map output by the 6th layer C2f structure of Backbone with the feature map output by the 10th layer Upsample structure. The number of channels becomes 960 and the size is 40×40.

[0033] Layer 12: VoVGSCSP structure, the number of input channels is 960, the input feature map is convolved, and the nonlinear expression of features and the utilization of feature information are further enhanced through the GSBottleneck operation (combined with the GSConv module and residual connection), the output information is concat and convolved, the number of output channels becomes 384, and the feature map size is 40×40;

[0034] Layer 13: SimAM structure. SimAM is a lightweight, efficient and parameter-free convolutional neural network (CNN) attention mechanism. The local correlation of the feature map is obtained through Euclidean distance, and the attention weight is generated. The generated attention weight is applied to the input feature map to generate a new weighted feature map. The number of output channels becomes 384, and the feature map size is 40×40;

[0035] Layer 14: Upsample structure, the number of input channels is 384, the upsampling algorithm uses nearest, after upsampling the input feature map, the number of channels becomes 384, and the feature map size is expanded to 80×80;

[0036] Layer 15: Concat structure, which concatenates the feature map output by the 4th layer C2f structure of Backbone with the feature map output by the 14th layer Upsample structure. The number of channels becomes 576 and the size is 80×80.

[0037] Layer 16: VoVGSCSP structure, the number of input channels is 576, the input feature map is convolved, and the nonlinear expression of features and the utilization of feature information are further enhanced through the GSBottleneck operation. The output information is concat and convolved, the number of output channels becomes 192, and the feature map size is 80×80;

[0038] Layer 17: SimAM structure. The number of output channels becomes 192, and the feature map size is 80×80;

[0039] Layer 18: GSConv structure, first convolution is performed on the input feature map (the convolution kernel size is 3×3 and the step size is 2), then depth-wise separable convolution is performed on the output feature map, and finally the two feature maps are concat and shuffled. The number of channels of the output feature map is 192 and the size is reduced to 40×40;

[0040] Layer 19: Concat structure, which performs a concatenation operation on the output feature map of the 12th layer C2f structure of Neck and the output feature map of the 18th layer CBS structure. The number of channels becomes 576 and the size is 40×40.

[0041] Layer 20: VoVGSCSP structure, the number of input channels is 576, the input feature map is convolved, and the nonlinear expression of features and the utilization of feature information are further enhanced through the GSBottleneck operation. The output information is concat and convolved, the number of output channels becomes 384, and the feature map size is 40×40;

[0042] Layer 21: SimAM structure. The number of output channels becomes 384, and the feature map size is 40×40;

[0043] Layer 22: GSConv structure, first convolution is performed on the input feature map (the convolution kernel size is 3×3 and the step size is 2), then depth-wise separable convolution is performed on the output feature map, and finally the two parts of the feature map are concat and shuffled. The number of channels of the output feature map is 384 and the size is reduced to 20×20;

[0044] Layer 23: Concat structure, which concatenates the feature map output by the 9th layer SPPF structure of Backbone with the feature map output by the 22nd layer standard convolution GSConv of Neck. The number of channels becomes 960 and the size is 20×20.

[0045] Layer 24: VoVGSCSP structure, the number of input channels is 960, the input feature map is convolved, and the nonlinear expression of features and the utilization of feature information are further enhanced through the GSBottleneck operation. The output information is concat and convolved, the number of output channels becomes 576, and the feature map size is 20×20;

[0046] Layer 25: SimAM structure. The number of output channels becomes 576, and the feature map size is 20×20;

[0047] Layer 26: Detect-SelfAttention structure, used for target detection and generating the final output.

[0048] Preferably, the SimAM self-attention mechanism is calculated as follows:

[0049]

[0050] in, represents the energy function, Represents the other neurons x except neuron t i The variance and mean of M represents the number of energy functions for each channel, X is the input feature map, is the enhanced feature map, E represents the weight tensor of the input feature map, sigmoid is an activation function, and λ represents an adjustable parameter, which is 0.0001.

[0051] Preferably, the SelfAttention mechanism is a combination of a self-attention module and a convolution module, which effectively reduces the amount of computation and can efficiently extract sample features. The self-attention module first uses a 1×1 convolution kernel to reduce the number of channels, uses a multi-head self-attention module to capture the dependencies between features, and then uses a 1×1 convolution kernel to restore the number of channels, performs BatchNormal, and then uses a residual operation to add the batch-normalized feature map to the input feature map, and then passes through a ReLU activation function; the convolution module reduces the amount of computation and maintains feature extraction efficiency through downsampling, upsampling, and residual connection operations.

[0052] Preferably, in step S4, the Mosaic data enhancement algorithm is used to randomly scale, crop, and arrange any four original images and then splice them to generate more diverse training samples of 640px×640px size, which helps to improve the small target detection ability and generalization ability of YOLOv8 in this scenario.

[0053] Therefore, the present invention adopts the above-mentioned CT image lung nodule detection and recognition method based on the improved YOLOv8 model, which has the following beneficial effects:

[0054] (1) Reducing the number of model parameters: By adopting lightweight design and self-attention mechanism, the number of model parameters is significantly reduced. The number of parameters of the YOLOv8 model is 25.9 million, and the number of parameters of the improved YOLOv8 model is 19.16 million, which is about 73.98% of the original model, thereby reducing the demand for computing resources.

[0055] (2) Improving the model’s attention to key features: The introduction of the self-attention mechanism helps the model better focus on and extract important features, thereby improving the detection accuracy of lung nodules. The mAP@0.5 of the YOLOv8 model on the test set is 82.0%, and the mAP@0.5 of the improved YOLOv8 model on the test set is 85.5%.

[0056] (3) Improving the generalization ability of the model: Through structural optimization and the role of the attention mechanism, the generalization ability of the model is enhanced, and it performs better in a variety of practical application scenarios.

[0057] The technical solution of the present invention is further described in detail below through the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] Figure 1 This is a flow chart of a method for detecting and identifying lung nodules in CT images based on an improved YOLOv8 model of the present invention;

[0059] Figure 2 This is a diagram of the YOLOv8 network structure after the SimAM mechanism is introduced into the present invention;

[0060] Figure 3 This is a comparison chart of the improved YOLOv8 training results of the present invention;

[0061] Figure 4 It is a comparison diagram of the prediction results of an embodiment of the present invention, where (a) is the true annotation result diagram of the test set; (b) is the improved YOLOv8 prediction result diagram; and (c) is the YOLOv8 prediction result diagram. DETAILED DESCRIPTION

[0062] Example

[0063] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Generally, the components of the embodiments of the present invention described and shown in the drawings here can be arranged and designed in various different configurations.

[0064] like Figure 1 As shown, a method for detecting and identifying lung nodules in CT images based on an improved YOLOv8 model includes the following steps:

[0065] S1. Generate a 640×640×3 image from the original mhd file in the LUNA16 dataset, read the location information of the lung nodules, and convert it into a VOC format that is easy to process, generating a raw dataset of lung CT images;

[0066] S2, performing lung parenchyma segmentation on the original lung CT image data set generated in step S1, optimizing the original lung CT image data set, retaining the region of interest, and processing the original lung CT image data set using a lung parenchyma segmentation algorithm. First, the CT image is standardized, and then K-means clustering and morphological processing are performed on it to generate a lung CT mask image, and then the mask image is used to segment the original image to remove unnecessary areas outside the lung;

[0067] S3, improve the YOLOv8 model, such as Figure 2 As shown, including:

[0068] Backbone part of the YOLOv8 model: The convolution module in the original Backbone part is replaced by a more lightweight ADown downsampling module to reduce the number of parameters while maintaining the accuracy of lung nodule detection. The downsampling module is used to reduce the size of the feature map while maintaining feature information, thereby reducing the computational complexity. The lightweight design can reduce the number of model parameters while maintaining or improving the feature extraction effect;

[0069] Neck part of the YOLOv8 model: A lightweight SimAM self-attention mechanism is introduced after the C2f convolution module. The SimAM self-attention mechanism enhances the model's ability to focus on important features, allowing the model to learn autonomously and focus on key areas, improving the quality of feature representation. The SimAM self-attention mechanism is calculated in the following way:

[0070]

[0071]

[0072] in, represents the energy function, Represents the other neurons x except neuron t i The variance and mean of M represents the number of energy functions for each channel, X is the input feature map, is the enhanced feature map, E represents the weight tensor of the input feature map, sigmoid is an activation function, and λ represents an adjustable parameter, which is 0.0001;

[0073] The Slim-Neck structure is adopted. The Slim-Neck structure further reduces the number of model parameters and computational overhead. The lightweight design helps reduce the computational cost without significantly sacrificing model performance. The feature fusion and processing effects of the YOLOv8 model are improved, and the standard convolution and C2f convolution modules are replaced with lightweight GSConv and VoVGSCSP.

[0074] Head part of the YOLOv8 model: The SelfAttention mechanism is introduced to improve the model's attention to key features in the Head part, so as to more accurately identify and locate lung nodules, enable the model to better integrate overall context information, and improve detection accuracy; the SelfAttention mechanism is a combination of the self-attention module and the convolution module, which effectively reduces the amount of calculation and can efficiently extract sample features. The self-attention module first uses a 1×1 convolution kernel to reduce the number of channels, uses a multi-head self-attention module to capture the dependency between features, and then uses a 1×1 convolution kernel to restore the number of channels, and performs BatchNormal, and then uses the residual operation to add the batch normalized feature map to the input feature map, and then passes through the ReLU activation function; the convolution module reduces the amount of calculation and maintains the feature extraction efficiency through downsampling, upsampling, and residual connection operations, as shown in Table 1:

[0075] Table 1 Comparison of parameters between the original model and the model of the present invention

[0076]

[0077] From the above, we can see that by adopting lightweight design and self-attention mechanism, the number of model parameters is significantly reduced. The number of parameters of the YOLOv8 model is 25.9 million, and the number of parameters of the improved YOLOv8 model is 19.16 million, which is about 73.98% of the original model, thereby reducing the demand for computing resources.

[0078] S4. Use the improved YOLOv8 model obtained in step S3 to extract the image features of the training set (subset1 to subset9) of the original lung CT image data set to obtain a trained CT image lung nodule detection and recognition model, and test it on the test set (subset0). Using the Mosaic data enhancement algorithm, any four original images are randomly scaled, cropped, and arranged, and then spliced ​​to generate more diverse 640px×640px training samples, which helps to improve the small target detection and generalization capabilities of YOLOv8 in this scenario.

[0079] like Figure 3 As shown in the figure, the introduction of the self-attention mechanism helps the model to better focus on and extract important features, thereby improving the detection accuracy of lung nodules. The mAP@0.5 of the YOLOv8 model in the test set is 82.0%, and the mAP@0.5 of the improved YOLOv8 model in the test set is 85.5%.

[0080] Through structural optimization and the role of attention mechanism, the generalization ability of the model is enhanced, and it performs better in a variety of practical application scenarios. Figure 4 .

[0081] Therefore, the present invention adopts the above-mentioned CT image lung nodule detection and recognition method based on the improved YOLOv8 model, and the above-mentioned improvement is mainly applied to medical image analysis, especially the detection of lung nodules. In practical applications, this method can achieve higher detection accuracy with fewer resources, which is of great significance for clinical diagnosis and early detection of lung diseases.

[0082] Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that they can still modify or replace the technical solution of the present invention with equivalents, and these modifications or equivalent replacements cannot cause the modified technical solution to deviate from the spirit and scope of the technical solution of the present invention.

Claims

1. A method for detecting and identifying lung nodules in CT images based on an improved YOLOv8 model, characterized in that: The following steps are involved: S1. Obtain lung CT images from the original mhd file in the LUNA16 dataset to generate an original lung CT image dataset; S2, performing lung parenchyma segmentation on the original lung CT image data set generated in step S1, optimizing the original lung CT image data set, and retaining the region of interest; S3. Improved YOLOv8 model; S4. Use the improved YOLOv8 model obtained in step S3 to extract image features of the training set of the original lung CT image data set to obtain a trained CT image lung nodule detection and recognition model, and test it on the test set.

2. According to a method for detecting and identifying lung nodules in CT images based on an improved YOLOv8 model according to claim 1, it is characterized in that: In step S1, specifically: generate a 640×640×3 image from the original mhd file in the LUNA16 dataset, read the lung nodule location information, and convert it into the VOC format.

3. The method for detecting and identifying lung nodules in CT images based on an improved YOLOv8 model according to claim 1, characterized in that: In step S2, the original data set of lung CT images generated in step S1 is subjected to lung parenchyma segmentation, specifically comprising: using a lung parenchyma segmentation algorithm to process the original data set of lung CT images, firstly standardizing the CT images, then performing K-means clustering and morphological processing on them to generate a lung CT mask image, and then using the mask image to segment the original image to remove unnecessary areas outside the lungs.

4. The method for detecting and identifying lung nodules in CT images based on an improved YOLOv8 model according to claim 1, characterized in that: Step S3 is specifically as follows: Backbone part of the YOLOv8 model: Use the lightweight ADown downsampling module to replace the convolution module in the original Backbone part; Neck part of the YOLOv8 model: introduce the Slim-Neck structure and the lightweight SimAM self-attention mechanism after the C2f convolution module, and use lightweight GSConv and VoVGSCSP to replace the standard convolution and C2f convolution modules; Head part of the YOLOv8 model: Introducing the SelfAttention mechanism.

5. The method for detecting and identifying lung nodules in CT images based on the improved YOLOv8 model according to claim 4, characterized in that: The SimAM self-attention mechanism is calculated in the following way: in, represents the energy function, Represents the other neurons x except neuron t i The variance and mean of M represents the number of energy functions for each channel, X is the input feature map, is the enhanced feature map, E represents the weight tensor of the input feature map, sigmoid is an activation function, and λ represents an adjustable parameter, which is 0.0001.

6. The method for detecting and identifying lung nodules in CT images based on the improved YOLOv8 model according to claim 4, characterized in that: The SelfAttention mechanism is a combination of the self-attention module and the convolution module. The self-attention module first uses a 1×1 convolution kernel to reduce the number of channels, uses a multi-head self-attention module to capture the dependencies between features, and then uses a 1×1 convolution kernel to restore the number of channels and perform BatchNormal. The residual operation is then used to add the batch-normalized feature map to the input feature map, and then passes through the ReLU activation function.

7. The method for detecting and identifying lung nodules in CT images based on the improved YOLOv8 model according to claim 1, characterized in that: In step S4, the Mosaic data enhancement algorithm is used to randomly scale, crop, and arrange any four original images and then splice them to generate more diverse training samples of 640px×640px size.

Images