Improved mask-r-cnn lung nodule auxiliary detection method fusing dual-path channel attention and cavity space attention

Abstract

The application relates to an improved Mask-R-CNN lung nodule auxiliary detection method fusing a double-path channel attention and a hollow space attention, and belongs to the image processing field, and comprises the following steps: S1, data set pretreatment; S2, lung parenchyma segmentation; S3, constructing an improved candidate nodule detection and segmentation model; S4, modifying RPN according to lung nodule features; S5, improving a loss function according to data imbalance; S6, constructing a three-dimensional ResNet false positive elimination model to eliminate false positives; S7, training the improved lung nodule detection model by using selected data sets, loading the best weight file after training into the model to perform feature extraction, generating a series of candidate regions, then labeling the candidate frame according to the position relationship between the candidate region and the object real frame on the picture, generating a lung nodule prediction frame and lung nodule prediction confidence, and achieving the expected effect.

Description

TECHNICAL FIELD

[0001] The application belongs to the field of image processing, and relates to an improved Mask-R-CNN lung nodule auxiliary detection method fusing double-path channel attention and hollow space attention. BACKGROUND

[0002] The lung is the respiratory organ of the human body. Once there is a problem in the lung, the human body cannot normally exchange gas with the outside world, thereby causing hypoxia and even endangering life. At present, lung diseases pose a great threat to people's life and health. Among them, lung cancer is one of the lung diseases that are concerned, and is also the cancer with the highest incidence and mortality in the world. The early manifestation of lung cancer is the generation of lung nodules in the lung, but detecting lung nodules does not necessarily mean having cancer, and further accurate analysis of each suspicious nodule is needed. If lung cancer patients can be detected as early as possible and actively treated in the first time, the survival rate can be greatly improved.

[0003] A lung nodule refers to a round or irregular shaped lesion area proliferated in the lung. On a lung CT image, it generally appears as a region with clear or unclear boundary and increased density shadow. According to research, early diagnosis and treatment of lung cancer can significantly improve the survival rate of patients. According to the spatial position of the lung nodule, the lung nodule can be divided into isolated lung nodules, lung nodules close to the inner wall of the chest cavity, and lung nodules close to blood vessels. According to the density of the nodule, the lung nodule can be divided into three categories: solid nodules, sub-solid nodules and ground-glass nodules. Among them, the ground-glass lung nodules have the highest possibility of developing into malignant tumors, i.e. lung cancer. If lung nodules are detected early and treated in time, the five-year survival rate can be improved to 55%. How to quickly and accurately diagnose lung nodules through lung CT images plays a crucial role in the treatment of lung cancer.

[0004] Generally, in the traditional manual diagnosis method, a radiologist needs to observe the chest CT images of a patient one by one and judge whether there is a lung nodule lesion. However, for experienced doctors, some nodules are also difficult to be found, and such a method not only has low detection efficiency, but also brings fatigue to the doctors and affects the judgment, thereby causing misdiagnosis or missed diagnosis. In order to effectively deal with this situation, an auxiliary detection system with high detection accuracy is needed to provide valuable "second opinion" to the doctors, so as to reduce the workload of the doctors and improve the accuracy of lung disease diagnosis. SUMMARY

[0005] Therefore, the purpose of the present application is to provide an improved Mask-R-CNN lung nodule auxiliary detection method fusing double-path channel attention and hollow space attention.

[0006] To achieve the above purpose, the present application provides the following technical scheme:

[0007] An improved Mask-R-CNN lung nodule auxiliary detection method fusing dual-path channel attention and cavity space attention, comprising the following steps:

[0008] S1: data set preprocessing;

[0009] S2: lung parenchyma segmentation;

[0010] S3: fusion of dual-path channel attention and cavity space attention, construction of improved candidate nodule detection and segmentation model based on improved Mask-R-CNN;

[0011] S4: modifying RPN for lung nodule features;

[0012] S5: improving the loss function for data imbalance;

[0013] S6: constructing a three-dimensional ResNet false positive model to remove false positives;

[0014] S7: using the selected data set to train the improved lung nodule detection model, loading the best weight file after training into the improved Mask-R-CNN to construct the improved candidate nodule detection and segmentation model, performing feature extraction, generating a series of candidate regions, then labeling the candidate boxes according to the position relationship between the candidate regions and the real boxes on the picture, generating lung nodule prediction boxes and lung nodule prediction confidence, and achieving the expected effect.

[0015] Further, in step S1, the Luna16 data set is used, first, the CT image of the data set is preprocessed, including image flipping, coordinate conversion, pixel value truncation processing, then converted into a VOC format data set, and divided into a training set and a validation set.

[0016] Further, in step S2, the lung parenchyma segmentation, specifically comprising the following steps:

[0017] S21: standardizing the pixel value of the CT slice image by threshold method;

[0018] S22: visualizing the brightness distribution histogram, finding the average pixel value near the lung, and re-normalizing the washed image;

[0019] S23: using K-means clustering algorithm to separate foreground and background, after getting binary pixel, using dilation operation to expand lung parenchyma area, and filling inside the area, generating a connected area, then using corrosion to narrow the lung parenchyma area, finally the lung parenchyma part is white and the rest is black, thereby generating the corresponding lung parenchyma binary mask;

[0020] S24: Overlay the obtained lung parenchyma mask with the original image to obtain a lung parenchyma image.

[0021] Furthermore, step S3 describes constructing an improved candidate nodule detection and segmentation model by adding a dual-path channel attention module (DPCA) after each stage of the Mask-R-CNN backbone network; and introducing a dilated spatial attention module (ESA) fused from dilated convolutions with different dilation rates and a spatial attention mechanism (CAM) in the lateral connection stage of the feature fusion network; including the following steps:

[0022] S31: Squeeze (F) sq (·) Perform global average pooling on the feature map to generate a 1×1×C vector, and represent each channel with a numerical value:

[0023]

[0024] H and W represent the height and width of the feature map, respectively. c This represents the feature map after convolution;

[0025] S32: Excitation (F) ex (·,)) is accomplished through two fully connected layers, generating the weight information we need through weights:

[0026] s = F ex (z,W)=σ(g(z,W))=σ(W2δ(W1)) (2)

[0027] W1 and W2 represent the first and second fully connected layers, respectively; δ represents the ReLU activation function; and σ represents the sigmoid activation function.

[0028] S33: Scale(F) scale The process involves assigning weights to the feature map using the weight vector generated in step S32, resulting in the feature map:

[0029]

[0030] u c s represents the feature map after convolution. c This represents the feature map obtained in the second step;

[0031] S34: The different feature maps obtained from the two paths are fused; the dilated spatial attention module ESA is formed by fusing dilated convolutions with different dilation rates and the spatial attention mechanism CAM; the calculation method of the convolution kernel of dilated convolution is as follows:

[0032] K = k + (k-1) × (r-1) (4)

[0033] Where K is the actual kernel size of dilated convolution, k is the original kernel size, and r is the dilation parameter of dilated convolution—the number of kernel intervals. For standard convolution, r = 1.

[0034] The receptive field of dilated convolution is calculated as follows:

[0035]

[0036] Where, r n Let r be the receptive field of each point in the nth layer. n-1 Let k be the receptive field of each point in the (n-1)th layer. n S is the kernel size of the nth convolutional layer. i It is the stride of the i-th convolutional layer;

[0037] Spatial Attention Module (CAM): Maintaining the same spatial dimension while compressing the channel dimension, the spatial attention module focuses on the target's positional information. The spatial attention formula is as follows:

[0038]

[0039] in, and , respectively, perform global average pooling and max pooling operations on the spatial features, f is a 1×1 convolution operation on the two concatenated and fused features, and σ is the sigmoid activation function.

[0040] Furthermore, in step S4, the aspect ratio and size of the anchor frame generated by the improved candidate nodule detection and segmentation model are changed.

[0041] Furthermore, in step S5, focal loss is introduced as the loss function of the improved candidate nodule detection and segmentation model. The definition of focal loss is as follows:

[0042]

[0043]

[0044] FL(P t )=-α(1-P t ) γ log(P t (9)

[0045] In the formula, α and γ are parameters indicating their sensitivity to the classified samples. α is used to adjust the imbalance between positive and negative samples, and γ is used to reduce the loss in easily classified samples; P is the predicted probability of the sample; and y is the sample label.

[0046] Further, in step S6, the detected candidate nodules are cropped according to their detection locations and fed into a 3D ResNet convolutional neural network. First, they pass through a 5×5×5 Conv, and then a residual block of the first type of Bottleneck structure is used, which has a 1×1×1 Conv to adjust the channels and resolution. The residual block of the second type of Bottleneck structure does not have the 1×1×1 Conv. The two Bottleneck residual blocks are stacked sequentially three times, followed by a global flat pooling and a fully connected layer. Finally, Softmax binary classification is used to determine whether it is a nodule and to screen out false positives.

[0047] The beneficial effects of this invention are as follows:

[0048] I. The introduced Dual-Path Channel Attention Module (DPCA) and Hollow Spatial Attention Module (ESA) enhance the network's ability to acquire multi-scale context and feature representation, as well as its detection performance for non-salient targets such as small nodules and vascular nodules. Simultaneously, they make the model focus more on effective key features, reducing or ignoring some invalid feature information, thus improving the accuracy of lung nodule detection.

[0049] Second, the anchor box generation mechanism of the RPN network in the model was modified to make the detection boxes fit the size of the lung nodules better.

[0050] Third, focal loss is introduced as an improved loss function to address the imbalance of training samples.

[0051] Fourth, a three-dimensional ResNet convolutional neural network is used as a false positive removal algorithm to screen out false positives of detected candidate nodules, thereby improving the overall accuracy of lung nodule detection in the system.

[0052] Other advantages, objectives, and features of the invention will be set forth in part in the description which follows, and in part will be apparent to those skilled in the art from the following examination, or may be learned from practice of the invention. The objectives and other advantages of the invention can be realized and obtained through the following description. Attached Figure Description

[0053] To make the objectives, technical solutions, and advantages of the present invention clearer, the preferred embodiments of the present invention will be described in detail below with reference to the accompanying drawings, wherein:

[0054] Figure 1 This is a flowchart of the lung nodule auxiliary detection system provided by the present invention.

[0055] Figure 2 This is a schematic diagram of the improved Mask-R-CNN network structure for lung nodule detection and segmentation.

[0056] Figure 3 This is a schematic diagram of the Dual Path Channel Attention (DPCA) module.

[0057] Figure 4 This is a schematic diagram of the Empty Space Attention Module (ESA).

[0058] Figure 5 This is a schematic diagram of the 3D ResNet algorithm for removing false positives. Detailed Implementation

[0059] The following specific examples illustrate the implementation of the present invention. Those skilled in the art can easily understand other advantages and effects of the present invention from the content disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and various details in this specification can be modified or changed based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that the illustrations provided in the following embodiments are only schematic representations of the basic concept of the present invention. Unless otherwise specified, the following embodiments and features can be combined with each other.

[0060] The accompanying drawings are for illustrative purposes only and are schematic diagrams, not actual pictures. They should not be construed as limiting the invention. To better illustrate the embodiments of the invention, some parts in the drawings may be omitted, enlarged, or reduced, and do not represent the actual product dimensions. It is understandable to those skilled in the art that some well-known structures and their descriptions may be omitted in the drawings.

[0061] In the accompanying drawings of the embodiments of the present invention, the same or similar reference numerals correspond to the same or similar components. In the description of the present invention, it should be understood that if terms such as "upper," "lower," "left," "right," "front," and "rear" indicate the orientation or positional relationship based on the orientation or positional relationship shown in the drawings, they are only for the convenience of describing the present invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, the terms used to describe positional relationships in the drawings are only for illustrative purposes and should not be construed as limiting the present invention. For those skilled in the art, the specific meaning of the above terms can be understood according to the specific circumstances.

[0062] This includes small lung nodules that occupy less than 8×8 pixels in the image. The main process for lung nodule detection is as follows: Figure 1 As shown. Regarding this process, the main research content of this design includes:

[0063] (1) Dataset preprocessing research: First, the CT images in the dataset are preprocessed and converted into VOC format dataset, and divided into training set and validation set.

[0064] (2) Lung parenchyma segmentation study: Pixel values ​​of CT slice images converted to VOC format were standardized using a thresholding method, and brightness distribution histograms were visualized. The average pixel values ​​near the lungs were identified, and the washed-out images were re-normalized. K-means clustering was used to separate the foreground and background, resulting in binary pixels. Morphological dilation was then used to expand the lung parenchyma region, shrink the pores, and fill the interior of the region to generate a connected area. Erosion was then used to narrow the lung parenchyma region, making the lung parenchyma image more accurate. The lung parenchyma portion was retained in white, while the remaining areas were filled with black, thus generating a corresponding binary mask for each image. The obtained lung parenchyma mask was overlaid on the original image to obtain the lung parenchyma image.

[0065] (3) Research on improved candidate nodule detection and segmentation algorithms:

[0066] While some small nodules in CT images may be benign, others are malignant. If left undetected, they can develop into large nodules and lead to lung cancer. To address the difficulty of detecting small targets, this invention adds a dual-path channel attention module (DPCA) after each stage of its backbone network. In the lateral connection stage of the feature fusion network, a dilated spatial attention module (ESA) is introduced, which is a fusion of dilated convolutions with different dilation rates and a spatial attention mechanism (CAM). This enhances the network's ability to obtain multi-scale context and feature representation, and improves its detection performance for non-salient targets such as small nodules and nodules near blood vessels. Simultaneously, it makes the model focus more on effective key features, reducing or ignoring some invalid feature information, thus improving the accuracy of lung nodule detection.

[0067] (4) Modifying the RPN study based on lung nodule characteristics:

[0068] In the original Mask-R-CNN, the RPN network generates anchor boxes with aspect ratios of 2:1, 1:1, and 1:2, and sizes of 128*128, 256*256, and 512*512, totaling 9 anchor boxes. The anchor box size is too large and does not match the actual size of the detected target. Based on the morphology and characteristics of lung nodules, the anchor box mechanism in the original model is improved.

[0069] (5) Research on improving the loss function to address data imbalance:

[0070] During the training of lung nodule detection models, a severe imbalance in the ratio of positive to negative samples occurs, meaning the number of samples in each category of the dataset is extremely uneven. This leads to a significant bias in the classification model, causing it to fail to learn the features of fewer samples and ultimately resulting in prediction errors. This invention introduces focal loss as an improved loss function to address the imbalance of training samples.

[0071] (6) Research on 3D ResNet False Positive Removal Algorithm:

[0072] The detected candidate nodules are cropped according to their detection location and fed into a 3D ResNet convolutional neural network to screen out false positives and improve the overall lung nodule detection accuracy of the system.

[0073] (7) Model training research:

[0074] The improved lung nodule detection model was trained using the selected dataset, the weight file was saved, predictions were made, and lung nodule prediction boxes and lung nodule prediction confidence scores were generated.

[0075] Example: The solution in this example is as follows:

[0076] See Figures 1 to 5 An improved Mask-R-CNN-assisted lung nodule detection method is proposed, which integrates dual-path channel attention and hollow spatial attention. Its features include the following steps: Step 1: Dataset preprocessing; Step 2: Lung parenchyma segmentation; Step 3: Improved candidate nodule detection and segmentation algorithm; Step 4: Modified RPN based on lung nodule features; Step 5: Improved loss function to address data imbalance; Step 6: 3D ResNet false positive removal algorithm; Step 7: Model training.

[0077] In step one, the Luna16 dataset used in this invention is a subset of the largest common lung nodule dataset LIDC-IDRI, comprising 888 low-dose lung CT images (mhd format), each image containing a series of multiple axial slices of the thoracic cavity. The CT images in the dataset are first preprocessed, including image flipping, coordinate transformation, and pixel value truncation, then converted to VOC format and divided into training and validation sets.

[0078] In step two, the CT images read in this dataset are whole lung images, mainly including the lungs and surrounding tissues and organs. Tissues such as blood vessels, bronchi, and bones may affect subsequent lung nodule detection; therefore, it is necessary to segment the lung parenchyma and remove other tissues and organs outside the lungs. First, the pixel values ​​of the CT slice images converted to VOC format are standardized using a thresholding method, and the brightness distribution histogram is visualized. The average pixel value near the lungs is found, and the washed-out image is re-normalized. The K-means clustering algorithm is used to separate the foreground (opaque tissue) and background (transparent tissue, i.e., the lungs), where K=2. After obtaining binary pixels, the lung parenchyma region is extracted using morphological methods of erosion and dilation. Dilation expands the lung parenchyma region, reduces the internal pores, and fills the interior of the region, preventing the loss of lung parenchyma information due to external factors, thus generating a connected region. Erosion then narrows the lung parenchyma region, making the lung parenchyma image more accurate. Finally, the lung parenchyma region is selected according to the set image size. The lung parenchyma is left white, while the remaining areas are filled with black, thus generating a corresponding binary mask for the lung parenchyma for each image. The resulting lung parenchyma mask is then overlaid on the original image, with the black areas covering the original image and the white areas displaying the original image, resulting in a lung parenchyma image.

[0079] In step three, since the original Mask-R-CNN cannot extract enough channel and spatial features for lung nodule detection for small nodules, this invention adds a dual-path channel attention module (DPCA) after each stage of its backbone network, such as... Figure 3 As shown, in the lateral connection stage of the feature fusion network, a dilated spatial attention module (ESA) is introduced, which is a fusion of dilated convolutions with different dilation rates and a spatial attention mechanism (CAM), such as... Figure 4 As shown.

[0080] The Dual Path Channel Attention (DPCA) module first convolves the feature map with two different kernel sizes of 3 and 5, respectively, to extract features along two paths. The operation steps for the two paths are the same.

[0081] Step 1, Squeeze (F) sq (·) Perform global average pooling on the feature map to generate a 1×1×C vector, and represent each channel with a numerical value.

[0082]

[0083] H and W represent the height and width of the feature map, respectively. c This represents the feature map after convolution.

[0084] Step 2: Encourage Excitation (F) ex The process (·,) is completed through two fully connected layers. Weights W are used to generate the desired weight information, where W is learned and used to explicitly model the feature relevance we need. The vector z obtained in the previous step is processed through two fully connected layers W1 and W2 to obtain the desired channel weight values ​​s. After passing through two fully connected layers, different values ​​in s represent the weight information of different channels, assigning different weights to each channel. There is a hyperparameter R between the two fully connected layers. The dimension of the vector z (1×1×C) changes from (1×1×C) to (1×1×C / R) after passing through the first fully connected layer, and then changes from (1×1×C / R) to (1×1×C) after passing through the second fully connected layer. The activation function of the first fully connected layer is ReLU, and the activation function of the second fully connected layer is Sigmoid.

[0085] s = F ex (z,W)=δ(g(z,W))=δ(W2δ(W1)) (2)

[0086] W1 and W2 represent the first and second fully connected layers, respectively, δ represents the ReLU activation function, and σ represents the sigmoid activation function.

[0087] Step 3, Scale(F) scale The second step involves generating a weight vector s and assigning weights to the feature map to obtain the desired feature map. Its size is exactly the same as the feature map, and the SE module does not change the size of the feature map. The generated feature vector s (1×1×C) is multiplied by the corresponding channel of the feature map (H×W×C), that is, the H×W values ​​of each channel in the feature map are multiplied by the weight of the corresponding channel in s.

[0088]

[0089] u c s represents the feature map after convolution. c This represents the feature map obtained in the second step.

[0090] The fourth step is to fuse the different feature maps obtained from the two paths.

[0091] The dilated spatial attention module (ESA) is composed of dilated convolutions with different dilation rates and a spatial attention mechanism (CAM).

[0092] The method for calculating the convolution kernel of dilated convolution is as follows:

[0093] K = k + (k-1) × (r-1) (4)

[0094] Where K is the actual kernel size of the dilated convolution, k is the original kernel size, and r is the dilation parameter of the dilated convolution—the number of intervals in the kernel. For standard convolution, r = 1. When r = 2, the size of the 3×3 convolution kernel expands to 3 + (3-1)*(2-1) = 5, i.e., 5×5; when r = 4, it expands to 9×9.

[0095] The receptive field of dilated convolution is calculated as follows:

[0096]

[0097] Where, r n Let r be the receptive field of each point in the nth layer. n-1 Let k be the receptive field of each point in the (n-1)th layer. n S is the kernel size of the nth convolutional layer. i It is the stride of the i-th convolutional layer.

[0098] Spatial Attention Module (CAM): This module maintains the spatial dimension while compressing the channel dimension. It focuses on the target's positional information. The output of the channel attention module is processed through max pooling and average pooling to obtain two 1*H*W feature maps. These two feature maps are then concatenated, and a 7*7 convolution is applied to transform them into a 1-channel feature map. A sigmoid function is then applied to obtain the spatial attention feature map. Finally, the output is multiplied by the original image to revert to C*H*W size.

[0099] The formula for spatial attention is as follows:

[0100]

[0101] in, and , respectively, perform global average pooling and max pooling operations on the spatial features, f is a 1×1 convolution operation on the two concatenated and fused features, and σ is the sigmoid activation function.

[0102] In step four, the anchor boxes generated by the RPN network in the original Mask-R-CNN have aspect ratios of 2:1, 1:1, and 1:2, with sizes of 128*128, 256*256, and 512*512, totaling nine anchor boxes. These anchor box sizes are too large and do not match the actual size of the detected targets. Based on the morphology and characteristics of lung nodules, the anchor box mechanism in the original model is improved. Since lung nodules are relatively small and are all spherical entities, the three different aspect ratios in the original model are all changed to 1:1. Furthermore, based on the pixel size of the lung nodules, eight anchor boxes with sizes of 8, 16, 24, 32, 40, 48, 56, and 64 are designed for region proposal generation.

[0103] In step five, a severe imbalance in the ratio of positive to negative samples occurs during the training of the lung nodule detection model. This means the number of samples in each category of the dataset is extremely uneven, leading to a severe bias in the classification model. Consequently, the model fails to learn the features of fewer samples, ultimately resulting in prediction errors. To address this problem, this invention introduces focal loss as an improved loss function to resolve the imbalance of training samples. The definition of focal loss is as follows:

[0104]

[0105]

[0106] FL(P t )=-α(1-P t ) γ log(P t (9)

[0107] In the formula: α and γ are parameters indicating their sensitivity to the classified samples. α is used to adjust the imbalance between positive and negative samples, and γ is used to reduce the loss in easily classified samples; P is the predicted probability of the sample; y is the sample label.

[0108] In step six, learning complex lung nodule features typically requires deep networks. ResNet aims to overcome the accuracy decrease with increasing network depth. Detected candidate nodules are cropped according to their detection location and fed into a 3D ResNet convolutional neural network. First, they pass through a 5×5×5 Conv. Next, a residual block with a first type of Bottleneck structure is used, which includes a 1×1×1 Conv to adjust channels and resolution. A second type of Bottleneck residual block does not have this 1×1×1 Conv. These two Bottleneck residual blocks are stacked sequentially three times, followed by global flat pooling and a fully connected layer. Finally, Softmax binary classification is used to determine whether a nodule is detected, filtering out false positives and improving the overall lung nodule detection accuracy of the system.

[0109] In step seven, the improved lung nodule detection model is trained using the selected dataset. The best weight file after training is loaded into the model of the present invention for feature extraction, generating a series of candidate regions. Then, the candidate regions are labeled according to the positional relationship between the candidate regions and the ground truth bounding boxes of objects in the image, generating lung nodule prediction boxes and lung nodule prediction confidence, achieving the expected effect.

[0110] This invention provides an improved Mask-R-CNN method for lung nodule auxiliary detection that integrates dual-path channel attention and cavity space attention. The aim is to accurately and quickly detect lung nodules from real clinical CT cases, saving radiologists a lot of time reading images and providing doctors with valuable "second opinions" to reduce their workload and thus improve the accuracy and efficiency of lung disease diagnosis.

[0111] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

Claims

1. An improved Mask-R-CNN lung nodule auxiliary detection method fusing dual-path channel attention and cavity space attention, characterized in that: Comprise the following steps: S1: data set pretreatment; S2: lung parenchyma segmentation; S3: fuse double-path channel attention and cavity space attention, and construct an improved candidate nodule detection and segmentation model based on the improved Mask-R-CNN; the step S3 constructs the improved candidate nodule detection and segmentation model by adding a double-path channel attention module DPCA after each stage of the backbone network of the Mask-R-CNN network; In the transverse connection stage of the feature fusion network, a cavity space attention module ESA formed by fusing a cavity convolution with different hole rates and a spatial attention mechanism CAM is introduced; comprising the following steps: S31: Squeeze )Global average pooling is performed on the feature map to generate a 1x1xC vector, and each channel is represented by a numerical value: H and W represent the height and width of the feature map, respectively, denotes the feature map after convolution; S32: Excitation ) is completed by two full connection layers, and the desired weight information is generated by weights. and denote the first and second fully connected layers, respectively, denotes a ReLU activation function, denotes a sigmoid activation function; S33: Scale( ) is to assign weights to the feature map generated in step S32 to obtain a feature map: denotes the feature map after convolution, denotes the feature map obtained in the second step; S34: fuse the different feature maps obtained by the two paths; the cavity space attention module ESA is formed by fusing a cavity convolution with different hole rates and a spatial attention mechanism CAM; the calculation method of the convolution kernel of the cavity convolution is: wherein, is the actual kernel size of the dilated convolution, is the original kernel size, is the dilation parameter of the dilated convolution - the number of intervals of the kernel, standard convolution = 1 ; The calculation method of the receptive field of the cavity convolution is: in, For the first The receptive field of each point in the layer For the first The receptive field of each point in the layer It is the first The kernel size of a convolutional layer. It is the first The stride of a convolutional layer; The spatial attention module CAM: the spatial dimension is unchanged, and the channel dimension is compressed; the spatial attention module focuses on the position information of the target, and the spatial attention formula is as follows: wherein, and are global average pooling and max pooling operation on features in space, respectively, is using 1x1 convolution operation on two spliced fused features, is sigmoid activation function; S4: modify RPN for lung nodule features; S5: improve the loss function for data imbalance; S6: construct a three-dimensional ResNet false positive model to remove false positives; in the step S6, the detected candidate nodule is cropped according to the detection position and sent into a three-dimensional ResNet convolutional neural network; first, a 5*5*5 Conv is used, then a residual block with the first bottleneck structure is used, which has a 1*1*1 Conv for adjusting the channel and resolution; a residual block with the second bottleneck structure does not have the 1*1*1 Conv; the two bottleneck structure residual blocks are repeatedly stacked three times in order, connected with a global average pooling and a full connection, and finally a Softmax binary classification is used to determine whether it is a nodule to remove false positives; S7: use the selected data set to train the improved lung nodule detection model, load the best weight file after training into the improved Mask-R-CNN to construct an improved candidate nodule detection and segmentation model for feature extraction, generate a series of candidate regions, then label the candidate regions according to the position relationship between the candidate regions and the real box of the object on the picture, generate lung nodule prediction box and lung nodule prediction confidence, and achieve the expected effect.

2. The improved Mask-R-CNN lung nodule auxiliary detection method fusing dual-path channel attention and cavity space attention according to claim 1, characterized in that: In the step S1, the Luna16 data set is used; first, the CT image of the data set is preprocessed, including image flipping, coordinate conversion, and pixel value truncation processing; then, it is converted into a VOC format data set, and divided into a training set and a validation set.

3. The improved Mask-R-CNN lung nodule auxiliary detection method fusing dual-path channel attention and cavity space attention according to claim 1, characterized in that: The lung parenchyma segmentation in the step S2 specifically comprises the following steps: S21: standardize the pixel value of the CT slice image by threshold method; S22: visualize the brightness distribution histogram, find the average pixel value near the lung, and re-normalize the washed image; S23: After separating foreground and background using K-means clustering algorithm, the lung parenchyma region is expanded using dilation operation, and the region is filled to generate a connected region, then the lung parenchyma region is narrowed by erosion, finally the lung parenchyma part is white and the remaining region is black, thereby generating a corresponding lung parenchyma binary mask; S24: The obtained lung parenchyma mask is superimposed with the original image to obtain a lung parenchyma image.

4. The improved Mask-R-CNN lung nodule auxiliary detection method fusing dual-path channel attention and cavity space attention according to claim 1, characterized in that: In step S4, the anchor box generation aspect ratio and size of the improved candidate nodule detection and segmentation model are changed.

5. The improved Mask-R-CNN lung nodule auxiliary detection method of fusing dual-path channel attention and hollow space attention according to claim 1, characterized in that: In step S5, focal loss is introduced as the loss function of the improved candidate nodule detection and segmentation model, and the definition of focal loss is as follows: In the formula, α and γ are parameters indicating the sensitivity to classification samples, α is used to adjust the imbalance of positive and negative samples, and γ is used to reduce the loss of easy classification samples; P is the prediction probability of the sample; y is the sample label.

Images