Pulmonary nodule detection method based on double attention mechanisms

Abstract

The invention discloses a pulmonary nodule detection method based on double attention mechanisms, which mainly introduces two attention mechanisms, namely GCTA and PEA, into a U-shaped encoder and a decoder. The two attention mechanisms can effectively capture context-aware features, spatial relationships and global context information. In the model, global and fine-grained context information in the image is effectively captured by using a DETR model, and the detection performance of the small nodules is improved through a multi-scale feature aggregation module. In addition, the system also comprises an FPR module. Experimental verification is carried out on the model in a public data set LUNA16, and the model has outstanding performance on the sensitivity of nodule detection. Compared with an existing model, the model has better performance in the aspect of pulmonary nodule detection based on the CT image. The method has great potential and advantages in early-stage lung cancer detection in the future, and can be further applied to clinical auxiliary screening of early-stage lung cancer.

Description

Technical Field

[0001] The present invention belongs to the technical field of image detection, and in particular relates to a lung nodule detection method based on a dual attention mechanism. Background Art

[0002] CAD systems for lung nodules usually consist of two steps: nodule candidate generation and false positive reduction. In the first step, the system scans the CT image and generates candidate boxes for suspicious nodules. Typically, CAD systems try to detect all nodules for accurate diagnosis, resulting in many false positive candidates. In the second step, all candidate boxes are re-evaluated to remove non-nodules, as false positives may interfere with accurate diagnosis and lead to unnecessary treatment. Therefore, how to ensure high sensitivity and low false positive rate is a challenging problem.

[0003] Most traditional lung nodule detection methods rely on manually designed features or descriptors, which require domain expertise and are difficult to detect nodules in complex areas, resulting in very limited detection accuracy. With the development of deep learning, CNN-based methods have been introduced into the field of target detection, and a large amount of research has proven its effectiveness in nodule detection.

[0004] The researchers further improved the performance of the nodule detection network by adding an attention mechanism to the convolutional network. H. Xiao et al. used the multi-scale spatial channel attention module MSAM to retain spatial features during the channel feature fusion process to capture high-order statistical information to enhance the perception of small lung lesions. The traditional SE attention mechanism implicitly learns channel relationships through the fully connected layer, which makes it difficult to explain the specific interactions between channels. It also leads to an increase in the number of parameters and computational complexity, and may cause the problem of gradient disappearance, thus affecting the stability and effect of model training. In order to overcome the problems of spatial information loss and high computational complexity in the SE module, A.-M. Rickmann proposed that the module retains more spatial information by performing projection operations along the three axes of height, width and depth, thereby improving the ability to capture spatial details. The module proposed by Z. Yang was also improved on the basis of SEA. By introducing the channel normalization layer and the gated activation function, the performance of model training was effectively improved.

[0005] Deep learning methods have shown promise in the detection of lung nodules, and their combination with attention mechanisms will further improve the accuracy of detection. However, most current studies choose to introduce traditional attention mechanisms such as SE, CBAM, and ECA, which have problems such as high computational complexity, limited applicability, and insufficient interpretability. In addition, traditional attention mechanisms were originally designed for 2D convolutional neural networks, which may lead to the loss of spatial information during application, so the effect of direct application in 3D convolutional neural networks is limited. Summary of the invention

[0006] The purpose of the method of the present invention is to solve the problems of high complexity, limited applicability and loss of spatial information of traditional attention mechanism.

[0007] To achieve the above object, the present invention provides a lung nodule detection method based on a dual attention mechanism, which comprises the following specific steps:

[0008] S1. Obtain data and preprocess it

[0009] Convert the data LUNA16 to Hounsfield unit intensity, clip the data to [-1200, 600] to extract the lung parenchyma, perform intensity standardization, and linearly map the data range to [0, 255]. Extract a 3D patch from the data set, clip the 3D patch, calculate the starting point of the clipping area based on the coordinate information of the target nodule, adjust the clipping area according to the nodule size through the scaling function, and then fill the clipping area. Finally, return the clipped 3D patch and the adjusted nodule position and coordinate information.

[0010] S2. Build DANet network and output lung nodule detection results

[0011] S201, input a 3D CT patch of size H×W×D, after being processed by two stacked convolutional layers, and then sequentially subjected to 3D batch normalization and ReLU activation function processing;

[0012] S202, obtain a feature map through four downsampling modules, and use a maximum pooling operation to halve the size of the feature map;

[0013] S203: A portion of the feature graphs in step S202 enter the bottleneck layer, and the DETR module uses the self-attention mechanism to perform global feature modeling to capture long-distance dependencies and global context information;

[0014] S204, passing the feature map processed in step S203 through three upsampling modules, and gradually restoring its spatial size by using 3D transposed convolution, and finally obtaining a feature map with a size of H / 4×W / 4×D / 4;

[0015] S205, the jump connection part, takes another part of the feature map in step S202 as input, processes it through the multi-scale feature aggregation module and the PEA module, and then splices it with the feature map obtained in step S204 through the channel dimension, so as to integrate features of different scales;

[0016] S206, the double nodule detection module includes an NCD head and an FPR head, and the NCD head is used to generate the position deltas of the nodule prediction box and the predicted probability logits of whether it is a nodule, which are used to calculate the loss function of the NCD;

[0017] S207, after 65 rounds of training, the FPR head training is introduced; the FPR head takes the region of interest ROI of the feature map of the position map of the candidate box generated by the NCD head as input, and then adjusts the feature map to a uniform size through adaptive maximum pooling, and passes through three linear layers to finally obtain the re-scored nodule prediction position deltas and the probability logits of whether it is a nodule, which are used to calculate the loss function of the FPR head; the total loss is calculated based on the loss of the NCD head in step S206 and the loss of the FPR, and finally the model is calibrated using back propagation based on the total loss;

[0018] S208, judging whether the lung nodule is positive by using the position deltas of the prediction box of the nodule obtained in step S206 and the prediction probability logits of whether it is a nodule;

[0019] From the classification point of view, when the probability logits of a nodule reaches above 0.5, the tissue at the corresponding position in the image is a true positive lung nodule;

[0020] From a spatial perspective, when the 3D intersection-union ratio between the predicted box and the true box is greater than the non-maximum suppression score, the candidate nodule is considered a true positive, otherwise it is a false positive;

[0021] Evaluation indicators of S3 and DANet networks

[0022] The sensitivity in the FROC curve is used to evaluate the DANet network. The higher the sensitivity, the stronger the ability to detect true positive lung nodules. The sensitivity calculation formula is:

[0023]

[0024] Where TP is the number of true positive nodules detected by the model, and FN is the number of true positive nodules not detected by the model.

[0025] Beneficial effects:

[0026] This application mainly introduces two attention mechanisms, namely GCTA and PEA, in the U-shaped encoder and decoder. These two attention mechanisms can effectively capture context-aware features, spatial relationships, and global context information. In the model, the DETR model is also used to effectively capture the global and fine-grained context information in the image and improve the detection performance of small nodules through the multi-scale feature aggregation module. In this application, the GCTA module adaptively calibrates the features of each channel, so that the network can dynamically adjust the importance of the features, thereby better capturing key feature information. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] Figure 1 is a flow chart of a lung nodule detection method based on a dual attention mechanism provided by the present invention;

[0028] Figure 2 It is a structural framework diagram of the DANet model provided by the present invention;

[0029] Figure 3 This is a structural diagram of the GCTA module provided by the present invention;

[0030] Figure 4 It is a structural diagram of a PEA module provided by the present invention;

[0031] Figure 5 It is a structural diagram of the MSAI module provided by the present invention;

[0032] Figure 6 It is a structural diagram of the DERT module provided by the present invention;

[0033] Figure 7 This is a comparison chart of the actual number and the detected number of nodules provided by the present invention;

[0034] Figure 8 This is a visualization detection result diagram of DANet in LUNA16 data provided by the present invention; the red box is the real nodule, the blue box is the nodule detected by the model, and the number on the blue box represents the probability of the nodule;

[0035] Fig. 9 This is a graph of ablation experiment results of DANet on the LUNA16 dataset provided by the present invention;

[0036] Fig.10 This is a comparative experimental result diagram of DANet provided by the present invention and other models on LUNA16. DETAILED DESCRIPTION

[0037] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0038] The application principle of the present invention is further described below in conjunction with the accompanying drawings and specific embodiments.

[0039] Embodiment 1:

[0040] like Figure 1 As shown in the figure, a lung nodule detection method based on dual attention mechanism is as follows:

[0041] S1. Obtain data and preprocess it

[0042] The LUNA16 (Lung Nodule Analysis 2016) dataset used in this embodiment is an important public dataset for lung nodule detection and classification research, which is derived from the LIDC-IDRI database. The dataset contains 888 chest CT scan images and 1186 nodules (diameter greater than 3mm) independently annotated by four radiologists, including the centroid and diameter of the nodules. The LUNA16 dataset includes 10 subsets, and a 10-fold cross-validation experiment was performed using these 10 subsets.

[0043] All raw data are converted into the intensity of Hounsfield units (HU, a standard quantitative value used to describe radiodensity). Then, the data is clipped to [-1200, 600] to extract the lung parenchyma. Finally, the intensity is normalized, that is, the data range is linearly mapped to [0, 255]. Then the data is clipped, and a 3D patch is extracted from the data set. The clipping size of the 3D patch is set to 128×128×128×1 (height×length×width×channel). Combined with the coordinate information of the target nodule, the starting point of the clipping area is calculated to ensure that the nodule is in the center of the clipping block. Then, the clipping area is adjusted according to the size of the nodule through the scaling function, and the clipped area is filled to ensure data integrity. Finally, the clipped 3D patch and the adjusted nodule position and coordinate information are returned for subsequent lung nodule detection and analysis.

[0044] During the training phase of the 3D CNN, the entire CT image cannot be used as input due to GPU memory limitations. Therefore, small 3D patches are extracted from the CT image and fed into the network individually. The size of the input 3D patch is 128×128×128×1 (depth×height×width×channels). During the testing phase, the entire CT scan image is used as input without cropping the 3D patch individually.

[0045] S2. Build DANet network and output lung nodule detection results

[0046] like Figure 2As shown in the figure, the input 3D patch of size H×W×D is first processed by two stacked convolutional layers with a convolution kernel size of 3×3×3, followed by 3D batch normalization and ReLU activation function processing. Next, it will pass through four downsampling modules (encoders) to obtain feature maps, each of which consists of a 3D residual block and a GCTA module, and then the maximum pooling operation is used to halve the size of the feature map. After downsampling, a part of the feature map enters the bottleneck layer, at which time it will pass through the DETR module, which uses the self-attention mechanism to model global features and capture long-range dependencies and global context information. Subsequently, the feature map passes through three upsampling modules (decoders) and uses 3D transposed convolution to gradually restore its spatial size, and finally obtains a feature map of size H / 4×W / 4×D / 4. In the skip connection part, the other part of the feature map obtained by the downsampling module is used as input and processed by the multi-scale feature aggregation module (MSAI) and the PEA module, and then spliced ​​with the feature map in the upsampling process from the channel dimension to integrate features of different scales. After feature fusion, it enters the dual nodule detection module and passes through the nodule candidate detection head (NCD Head) to generate the position deltas of the prediction box of the candidate nodule and the predicted probability logits of whether it is a nodule. Deltas is a six-dimensional vector, including the z, x, y coordinates of the nodule center and the regression values ​​of the depth, height and width of the bounding box. Finally, the generated logits and deltas of the candidate nodules are used to calculate the loss function of the NCD head. The FPR head is introduced into the training after the number of training rounds reaches 65, and is used to re-score the candidate boxes generated by the NCD head. At this time, the NCD head and the FPR head are trained in parallel, and the total loss is calculated as the sum of these two parts of the loss. The FPR head takes the region of interest (ROI) of the feature map of the candidate box map generated by the NCD head as input, and then goes through adaptive maximum pooling to adjust the feature map to a uniform size. It then goes through three linear layers, and finally obtains the position deltas of the nodule prediction box after rescoring and the probability logits of whether it is a nodule, which are used to calculate the loss function of the FPR head. Finally, the total loss is calculated, that is, the loss function of the FPR head and the loss function of the NCD head are added, and the model is calibrated using back propagation. The NCD head obtains the position deltas of the nodule prediction box and the predicted probability logits of whether it is a nodule to determine whether the lung nodule is positive; from the classification criterion, when the calculated probability logits of the nodule reaches 0.5 or above, the tissue at the corresponding position in the image is a true positive lung nodule; from the spatial criterion, when the 3D intersection of union (IOU) between the predicted bounding box and the true box is greater than the non-maximum suppression score, the candidate nodule is considered to be a true positive, otherwise it is a false positive.

[0047] like Figure 3As shown in the figure, the GCTA module obtains the embedding information by calculating the L2 norm of the input feature map, then uses the tanh activation function to calculate the gate value, and finally multiplies the input feature map by the gate value element by element. By adaptively calibrating the features of each channel, the network can dynamically adjust the importance of the features to better capture key feature information. The details are as follows:

[0048] is the input feature map, where C, H, W, and D represent the number of channels, height, width, and depth, respectively. GCTA is calculated as follows:

[0049] (1) Global context position embedding

[0050] For each channel c of the input feature map, calculate the L2 norm of all its spatial positions (i, j, k) to obtain the embedding value S c , the calculation formula is as follows:

[0051]

[0052] Among them, u c,i,j,k Represents the value of the input feature map at channel c, height i, width j, and depth k. ε is represented as a small constant used to prevent zero division errors and enhance numerical stability. α is a learnable parameter used to scale the calculated norm.

[0053] (2) Channel normalization

[0054] Calculate the normalized value of the L2 norm The embedding value S for each channel C c The squares are summed and averaged along the channel dimension, and a constant ε is added to prevent division by zero errors and enhance numerical stability. The square root is taken to complete the normalization calculation to obtain The formula is as follows:

[0055]

[0056] Among them, l is the channel.

[0057] (3) Gating Adaptation

[0058] Calculate the gating value g for each channel c , embed the normalized channel into the value S c After multiplying with the learnable parameter γ, add the learnable parameter β for offset. Use the tanh activation function to ensure that the output value is between [-1, 1]. Add 1 to adjust the gate value range to between [0, 2]:

[0059]

[0060] (4) Output

[0061] The input feature map U and the gate value g of each channel c Corresponding multiplication, get the output feature map

[0062]

[0063] GCTA achieves adaptive calibration of channels by calculating the L2 norm of feature maps and using a gating mechanism. Its design is simple and efficient, and it can dynamically adjust the importance of features, enhance the expressiveness of key features, and suppress irrelevant features. Compared with other complex attention mechanisms (such as self-attention and channel attention), GCTA is simple and efficient in design and has lower computational overhead.

[0064] like Figure 4 As shown in the figure, the PEA module is added after the multi-scale feature aggregation module to further extract and enhance local features. First, average pooling is performed along the height, width and depth of the input tensor to generate projection vectors to retain more spatial information. Then, the dependency between these projection vectors is learned through two convolutional layers, and they are multiplied element by element with the original feature map to finally achieve recalibration of the feature map.

[0065] The details are as follows:

[0066] is the input feature map, where C, H, W, and D represent the number of channels, height, width, and depth, respectively. PEA is calculated as follows:

[0067] (1) Projection operation: The PEA module generates three projection vectors by performing average pooling along the three main axes (height, width, depth) of each channel of the tensor. The specific formula is as follows:

[0068]

[0069] in, are projection vectors, u c,i,j,k Represents the value of the input feature map at channel C, height i, width j, and depth k.

[0070] (2) Excitation operation: The dependencies between these projection vectors are learned through two convolutional layers with a convolution kernel size of 1×1×1, which reduces and restores the number of channels respectively. The final output passes through the sigmoid activation function and is element-wise multiplied with the original feature map to obtain the recalibrated feature map. The specific formula is as follows:

[0071] Z=Z h +Z w +Z d (8)

[0072]

[0073] Among them, Z represents the projection vector, * represents the convolution operation, ⊙ represents the element-by-element multiplication, σ represents the sigmoid activation function, δ represents the ReLU activation function, V1 and V2 represent the convolution layer function, where U is the input feature map.

[0074] The PEA module can significantly improve the accuracy of lung nodule detection with a small increase in model complexity by retaining spatial information and recalibrating feature maps.

[0075] like Figure 5 As shown in the figure, the multi-scale feature aggregation module (MSAI) is used in the jump connection part of the U-shaped encoder and decoder. By aggregating and transmitting multi-scale information, the feature representation ability of the feature map at different resolutions is enhanced, thereby improving the feature description of the nodules. In this module, the number of channels of the feature map is first normalized using convolution to make it consistent with the number of channels in the middle layer. The feature map is converted to the same scale as the middle layer through upsampling and downsampling, and then 3D convolution is used for interleaved fusion. The final output feature map is generated, and its number of channels and size are the same as the feature map of the middle layer.

[0076] like Figure 6 As shown in the figure, the DETR module used at the bottleneck of the U-shaped encoder and decoder integrates global context information to solve the limitations of traditional convolutional networks in dealing with long-range dependencies and target detection tasks, thereby improving the efficiency and accuracy of lung nodule detection. Specifically, the downsampled feature map is first positionally encoded, the feature map is flattened, and sinusoidal position encoding is added to retain spatial position information. The encoded feature map is then input into the DETR encoder for long-range dependency modeling. The DETR decoder interacts the encoded global features with the predefined query embedding to generate the final output result.

[0077] The dual-head nodule detection module includes a nodule candidate detection head (NCD) and a false positive reduction head (FPR), which aims to improve the accuracy of nodule detection. The NCD head generates a nodule candidate layer containing position and classification information, and outputs the prediction probability and prediction box position through two 1×1×1 convolutional layers. The initial prediction box is set as a cube of different sizes to accommodate different nodule sizes. The FPR head re-scores the candidate boxes generated by the NCD head, selects the previous feature maps that retain more image details, and extracts the region of interest (ROI) from these feature maps through three linear layers. Combining the detection of the NCD head and the re-scoring function of the FPR head, the accuracy of nodule detection is improved and the false positive rate is effectively reduced.

[0078] The loss function of the NCD head is the sum of the classification loss and the regression loss. The classification loss function is used to evaluate whether the anchor box contains nodules, and the regression loss function is used to correct the position of the predicted box. The loss function formula of the NCD head is as follows:

[0079]

[0080] Among them, n is the index of the anchor box, N cls and N reg are the number of anchor boxes for classification and regression, respectively, and λ is set to balance the weights of the two losses. n is the probability prediction value that the nth anchor box contains a nodule, is the label of the nodule (1 for nodule, 0 for others). n and are the offset vectors of the predicted box and the real box respectively. cls is the weighted binary cross entropy loss, i.e., classification loss; L reg for Loss, that is, regression loss. Offset vector t n and is defined as:

[0081]

[0082] Among them, x n ,y n ,z n ,d n ,h n ,w n They are the center coordinates and size of the prediction box, are the center coordinates and size of the real box respectively. a ,y a ,z a ,d a ,h a ,w a are the parameters of the anchor box.

[0083] The loss function of the FPR head is the classification loss L cls. During the training process, online hard sample mining (OHEM) is also used to select some hard samples as training samples to balance positive samples and negative samples. The loss functions of the NCD head and the FPR head are calculated in parallel and back-propagated respectively. Among them, FPR does not directly update the output of NCD, but obtains the feature map of the corresponding coordinates in the feature map during the downsampling process according to the output (coordinate information) of NCD, which is similar to setting some coordinate information in a picture, and then cutting out part of the image within these coordinate ranges. According to the features of this part of the image, the maximum pooling, linear layer, etc. are used to obtain new logits and deltas. That is, FPR uses the region of interest (ROI) of the feature map of the candidate box map generated by the NCD head as input, and then adjusts the feature map to a uniform size through adaptive maximum pooling, and then passes through three linear layers, and finally obtains the re-scored nodule prediction position deltas and the probability logits of whether it is a nodule.

[0084] The total loss function of the network is the sum of the loss function of the NCD head and the loss function of the FPR. The total loss function uses back propagation to calibrate the model according to the loss value.

[0085] Intersection over Union (IoU): It is used to measure the degree of overlap between the predicted bounding box and the true target detection box. It is a spatial metric that determines whether the detection box accurately covers the real lung nodules.

[0086] Prediction probability logits: measures the model's confidence in whether there is a nodule in the detection box, which is a measurement standard for classification.

[0087] The combination of the two (specifically, the classification and regression losses are included in the calculation of the total loss) jointly determines the reliability and accuracy of the test results.

[0088] Implementation of Intersection over Union (IoU): By comparing the boundaries of the predicted bounding box and the true target detection box, the width, height, and depth of their overlapping parts are calculated. Then the volume of the overlapping part of the two bounding boxes is calculated, that is, the intersection volume. The total coverage volume of the two bounding boxes, that is, the union volume, is equal to the sum of the volumes of the two bounding boxes minus the volume of the overlapping part. Divide the intersection volume by the union volume to get the intersection over union ratio. The calculation formula is as follows:

[0089]

[0090] Inter section Volume refers to the intersection volume, and Union Volume refers to the union volume.

[0091] Non-Maximum Suppression (NMS) is a technique for suppressing overlapping candidate boxes. Typically, the detection model generates a large number of proposals and assigns a score to each proposal (indicating the possibility of an object in the box). The NMS algorithm filters these proposals according to the following steps: Keep the proposal with the highest score. Calculate the degree of overlap between the remaining proposals and the proposal (usually measured by the intersection over union (IoU)). If the degree of overlap is greater than a preset threshold, the proposal is removed. Continue iterating until all proposals are processed.

[0092] Evaluation indicators of S3 and DANet networks

[0093] In lung nodule detection, sensitivity is an important indicator for evaluating model performance, which indicates the ability of the model to successfully detect all true lung nodules (true positives), that is, the proportion of lung nodules that the model can successfully identify. The higher the sensitivity, the stronger the ability to detect true positive lung nodules. The FROC (Free-response Receiver Operating Characteristic) curve is an indicator used to evaluate the performance of target detection algorithms and is often used in medical image analysis. The horizontal axis of the FROC curve represents the number of acceptable false positive nodules in each image, and the vertical axis represents the proportion of true positive nodules detected under a given number of false positives, that is, sensitivity. The average sensitivity under a specific set of false positive numbers can reflect the overall performance of the model under different false positive numbers. The sensitivity of the adopted FROC curve under 0.125, 0.25, 0.5, 1, 2, 4 and 8 false positives per scan (FPs / Scan) is shown. The performance indicator (Competition Performance Metric, CPM) is the average sensitivity under these seven predefined false positive numbers.

[0094]

[0095] Where TP is the number of true positive nodules detected by the model, and FN is the number of true positive nodules not detected by the model.

[0096] Embodiment 2:

[0097] We used 10-fold cross validation for training and validation on 10 subsets of the LUNA16 dataset. During training, we used the stochastic gradient descent (SGD) optimizer and set the weight decay coefficient to 1e -4DANet is trained for 120 epochs with a batch size of 4. The first 65 epochs are for training the NCD head only, and the last 55 epochs introduce the FPR head. The initial learning rate is set to 1e -2 , which drops to 1e at the 60th epoch -3 , and further dropped to 1e at the 96th epoch -4 All methods are implemented based on the PyTorch framework, and the experimental environment is a computer equipped with an NVIDIA GeForce RTX 4090D with 24G memory.

[0098] In the training of 3D Convolutional Neural Network (CNN), the entire CT image cannot be used as input due to GPU memory limitations. Therefore, small 3D patches are extracted from the CT image and fed into the network separately. The dimensions of the input 3D patch are 128×128×128×1 (depth×height×width×channels). If the patch is beyond the range of the CT image, it is padded with a value of 170, which represents the brightness of common tissues and can be distinguished from lung nodules. During the testing phase, we use the entire CT scan image as input without cropping the 3D patch. In order to avoid odd dimensions of the entire 3D image, it is also padded with a value of 170 before entering the model.

[0099] Table 1 shows the results of nodule detection by the DANet model on ten subsets of LUNA16. The total number of nodules detected from each fold of data is counted, and then the number of all detected nodules is counted according to the size of the diameter, and the sensitivity of the model for each fold of data is calculated. As can be seen from Table 1, the nodule sensitivity of the DANet model for each fold of data is basically above 95%, especially for nodules with a diameter of more than 10mm, almost all of them are detected, such as Figure 7 As shown in Figure 2. For all data, the sensitivity of the DANet model to nodules reached 97.13%. Therefore, it can be concluded that the DANet model performs very well in the detection of positive nodules. The model's pulmonary nodule detection visualization results are shown in Figure 2. Figure 8 shown.

[0100] Table 1. Quantitative measurement results of nodules by DANet based on LUNA16 ten-fold cross validation

[0101]

[0102] Embodiment 3:

[0103] A series of experiments were conducted in this application to improve the model by adding modules and evaluating the impact of different modules on the overall model. These experiments include combining the basic residual U-shaped encoder-decoder with the basic modules (MSAI, DETR, NCD and FPR) as the initial benchmark, represented by "Base". GCTA is added after each upsampling and downsampling module of the benchmark model, represented by "Base+GCTA". PEA is applied to the jump connection of the benchmark model, represented by "Base+PEA". GCTA and PEA are applied to the benchmark model together, represented by "Base+GCTA+PEA". Each model is trained with ten-fold cross validation under the same experimental environment. The results in Table 2 record the sensitivity at 0.125, 0.25, 0.5, 1, 2, 4 and 8 FPs / scan and CPM, showing the performance of each module.

[0104] like Fig. 9 As shown in the figure, it can be clearly seen that the addition of GCTA and PEA has improved the sensitivity of the baseline model to a certain extent. The addition of the GCTA model and the PEA module respectively has increased the CPM to a certain extent compared with the Base model. The combined effect of the two further improves the sensitivity of the model, and the CPM has increased compared with the baseline model.

[0105] Table 2. Ablation experiment results of DANet

[0106]

[0107] Implementation 4:

[0108] The DANet model of this application is compared with some other advanced lung nodule detection models on the LUNA16 dataset, including DeepLung, DeepSeed, NoduleNet, SANet, and TiCNet. The qualitative and quantitative comparison results are shown in Tables 3 and Fig.10As shown. Compared with DeepLung, the DANet model performs multi-scale feature aggregation at the jump connection and connects the PEA module to enhance the feature extraction capability. The SE attention mechanism introduced in DeepSeed uses the dependency between channels to enhance the effect of feature extraction, and the GCTA and PEA modules used in this application have made certain improvements to SE, so that the model has better spatial and channel information capture capabilities. The CPM obtained by the method of this application is more than 5% higher than that of DeepSeed. Different from the multi-task learning method adopted by NoduleNet, the DANet model trains nodule detection and false positive reduction in stages. In this method, by adding the DETR module, the pixel-level long-distance dependency problem in SANet and the capture of global contextual information can be further optimized. Compared with TiCNet, this application adds GCTA after the U-shaped encoder and decoder and adds the PEA module at the jump connection. The addition of these two modules can enable the model to further improve the feature extraction capability of the lung nodule detection network and the detection accuracy of nodules. Experimental results show that in [0.125, 0.25, 0.5, 1, 2, 4, 8] FPs / scan, the sensitivity of the DANet model is higher than that of other detection models, and the CPM is the highest. Therefore, the detection performance of this method is significantly better than other detection methods. Even when the false positive rate of each scan is low, the sensitivity of the model is close to 80%, which has good detection capabilities.

[0109] Table 3 Comparative experimental results of DANet and other models

[0110]

[0111] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A lung nodule detection method based on a dual attention mechanism, characterized in that: Specific steps: S1. Obtain data and preprocess it Convert the data LUNA16 to Hounsfield unit intensity, clip the data to [-1200, 600] to extract the lung parenchyma, perform intensity standardization, and linearly map the data range to [0, 255]. Extract a 3D patch from the data set, clip the 3D patch, calculate the starting point of the clipping area based on the coordinate information of the target nodule, adjust the clipping area according to the nodule size through the scaling function, and then fill the clipping area. Finally, return the clipped 3D patch and the adjusted nodule position and coordinate information. S2. Build DANet network and output lung nodule detection results S201, input a 3D CT patch of size H×W×D, after being processed by two stacked convolutional layers, and then sequentially subjected to 3D batch normalization and ReLU activation function processing; S202, obtain a feature map through four downsampling modules, and use a maximum pooling operation to halve the size of the feature map; S203: A portion of the feature graphs in step S202 enter the bottleneck layer, and the DETR module uses the self-attention mechanism to perform global feature modeling to capture long-distance dependencies and global context information; S204, passing the feature map processed in step S203 through three upsampling modules, and gradually restoring its spatial size by using 3D transposed convolution, and finally obtaining a feature map with a size of H / 4×W / 4×D / 4; S205, the jump connection part, takes another part of the feature map in step S202 as input, processes it through the multi-scale feature aggregation module and the PEA module, and then splices it with the feature map obtained in step S204 through the channel dimension, so as to integrate features of different scales; S206, the double nodule detection module includes an NCD head and an FPR head, which generates the position deltas of the nodule prediction box and the predicted probability logits of whether it is a nodule through the NCD head, and is used for calculating the loss function of the NCD; S207, after 65 rounds of training, the FPR head training is introduced; the FPR head takes the region of interest ROI of the feature map of the position map of the candidate box generated by the NCD head as input, and then adjusts the feature map to a uniform size through adaptive maximum pooling, and passes through three linear layers to finally obtain the re-scored nodule prediction position deltas and the probability logits of whether it is a nodule, which are used to calculate the loss function of the FPR head; the total loss is calculated based on the loss of the NCD head in step S206 and the loss of the FPR, and finally the model is calibrated using back propagation based on the total loss; S208, judging whether the lung nodule is positive by using the position deltas of the prediction box of the nodule obtained in step S206 and the prediction probability logits of whether it is a nodule; From the classification point of view, when the probability logits of a nodule reaches above 0.5, the tissue at the corresponding position in the image is a true positive lung nodule; From a spatial perspective, when the 3D intersection-union ratio between the predicted box and the true box is greater than the non-maximum suppression score, the candidate nodule is considered a true positive, otherwise it is a false positive; Evaluation indicators of S3 and DANet networks The sensitivity in the FROC curve is used to evaluate the DANet network. The higher the sensitivity, the stronger the ability to detect true positive lung nodules. The sensitivity calculation formula is: Where TP is the number of true positive nodules detected by the model, and FN is the number of true positive nodules not detected by the model.

2. The lung nodule detection method based on dual attention mechanism according to claim 1, characterized in that: In step S202, the downsampling module is composed of a 3D residual module and a GCTA module.

3. The lung nodule detection method based on dual attention mechanism according to claim 2, characterized in that: The GCTA modules are as follows: (1) Global context position embedding For the input feature map For each channel c, calculate the L2 norm of all its spatial positions (i, j, k) to get the embedding value S c , the calculation formula is as follows: Among them, u c,i,j,k represents the value of the input feature map at channel c, height i, width j and depth k; C, H, W and D represent the number of channels, height, width and depth respectively; ε is represented as a constant; α is a learnable parameter used to scale the calculated norm; (2) Channel normalization Calculate the normalized value of the L2 norm The embedding value S for each channel C c The squares are summed and averaged along the channel dimension, and a constant ε is added and the square root is taken to complete the normalization calculation. The formula is as follows: Among them, l is the channel; (3) Gating Adaptation Calculate the gating value g for each channel c , embed the normalized channel into the value S c After multiplying with the learnable parameter γ, add the learnable parameter β for offset; use the tanh activation function to ensure that the output value is between [-1, 1]; add 1 to adjust the gate value range to between [0, 2]: (4) Output The input feature map U and the gate value g of each channel c Corresponding multiplication, get the output feature map 4. The lung nodule detection method based on the dual attention mechanism according to claim 1, wherein: The DETR module in step S203 is as follows: S1, position encoding the downsampled feature map, flattening the feature map, and adding sinusoidal position encoding; S2, input the encoded feature map into the DETR encoder for long-range dependency modeling; S3,DETR’s decoder interacts the encoded global features with the predefined query embedding to generate the final output result.

5. The lung nodule detection method based on dual attention mechanism according to claim 1, characterized in that: The PEA modules are as follows: (1) Projection operation: The PEA module generates three projection vectors by performing average pooling along the three principal axes of each channel of the tensor; the formula is as follows: in, are projection vectors, u c,i,j,k Represents the value of the input feature map in channel C, height i, width j and depth k; C, H, W and D represent the number of channels, height, width and depth respectively; (2) Excitation operation: The dependency between the projection vectors is learned through two convolutional layers. The convolution kernel size is 1×1×1, which reduces and restores the number of channels respectively. The final output passes through the sigmoid activation function and is element-wise multiplied with the original feature map to obtain a recalibrated feature map. The formula is as follows: Z=Z h +Z w +Z d Among them, Z represents the projection vector, * represents the convolution operation, ⊙ represents the element-by-element multiplication, σ represents the sigmoid activation function, δ represents the ReLU activation function, V1 and V2 represent the convolution layer function, where U is the input feature map.

6. The lung nodule detection method based on dual attention mechanism according to claim 1, characterized in that: In the multi-scale feature aggregation module, convolution is used to normalize the number of channels of the feature map to make it consistent with the number of channels of the middle layer; the feature map is converted to the same scale as the middle layer through upsampling and downsampling, and then 3D convolution is used for interleaved fusion; the final output feature map is generated, whose number of channels and size are the same as the feature map of the middle layer.

7. The lung nodule detection method based on dual attention mechanism according to claim 1, characterized in that: In step S206, the NCD head generates a nodule candidate layer containing position and classification information, and outputs the predicted probability and bounding box position through two 1×1×1 convolutional layers. The initial bounding box is set as a cube of different sizes; The loss function of the NCD head is the sum of the classification loss and the regression loss. The classification loss function is used to evaluate whether the anchor box contains nodules, and the regression loss function is used to correct the position of the predicted box. The formula of the loss function is as follows: Among them, n is the index of the anchor box, N cls and N reg are the number of anchor boxes for classification and regression, respectively, and λ is set to balance the weights of the two losses; p n is the probability prediction value that the nth anchor box contains a nodule, is the label of the nodule, 1 indicates a nodule, 0 indicates others; t n and are the offset vectors of the predicted box and the real box respectively; L cls is the weighted binary cross entropy loss, L reg for Loss; offset vector t n and is defined as: Among them, x n ,y n ,z n ,d n ,h n ,w n They are the center coordinates and size of the prediction box, are the center coordinates and size of the real frame respectively; x a ,y a ,z a ,d a ,h a ,w a are the parameters of the anchor box.

8. The lung nodule detection method based on dual attention mechanism according to claim 1, characterized in that: The implementation of the intersection-and-union ratio in step S208 is as follows: by comparing the boundaries of the predicted bounding box and the true target detection box, the width, height and depth of their overlapping parts are calculated; then the volume of the overlapping parts of the two bounding boxes, that is, the intersection volume, is calculated; the total coverage volume of the two bounding boxes, that is, the union volume, is equal to the sum of the volumes of the two bounding boxes minus the volume of the overlapping parts, and the intersection-and-union ratio is obtained by dividing the intersection volume by the union volume. The calculation formula is as follows.

Images