Neural network-based three-dimensional pulmonary nodule target detection method and system

Abstract

The invention discloses a three-dimensional pulmonary nodule target detection method and system based on a neural network. The method comprises the steps of obtaining a pulmonary nodule three-dimensional CT image and preprocessing the image to obtain a training set and a test set; a target detection model eFATE-Net is constructed, and training and evaluation are carried out; and preprocessing a to-be-detected three-dimensional CT image, and inputting the preprocessed to-be-detected three-dimensional CT image into the trained eFATE-Net to realize target detection. A false positive reduction module is introduced into the eFATE-Net, so that false positive misinformation of pulmonary nodules is reduced on the premise of keeping high detection sensitivity. On the basis of the fine-grained spatial features and the coarse-grained semantic features of the pulmonary nodules, global dependency modeling is carried out through a self-attention mechanism to reconstruct and enhance the deep semantic features of the pulmonary nodules so as to further reduce false positive and false alarm of the pulmonary nodules. A gating attention mechanism is introduced to improve key feature selectivity, the response of the model to key features of the pulmonary nodule is enhanced, and the detection performance of the pulmonary nodule is improved.

Description

Technical Field

[0001] This application relates to the field of image processing technology, specifically to a method and system for detecting three-dimensional lung nodules based on neural networks. Background Technology

[0002] Lung cancer is one of the leading causes of cancer-related morbidity and mortality worldwide. Its high mortality rate is primarily due to delayed diagnosis, making timely detection and intervention crucial for improving patient survival. Early-stage lung cancer typically presents as round or irregularly shaped lung nodules. Computed tomography (CT) scans are an effective method for detecting lung nodules and screening for lung cancer; however, analyzing the CT results of a single patient often requires experienced physicians to spend considerable time processing hundreds of images. The presence of small or irregularly shaped nodules further complicates detection. Computer-aided detection systems can be used to quickly and automatically label candidate nodules, assisting physicians in improving the efficiency and detection rate of lung nodule screening.

[0003] With the development of deep learning, object detection algorithms based on convolutional neural networks (CNNs) have been effectively applied to the task of detecting lung nodules. In recent years, basic models trained through self-supervised learning on large amounts of unlabeled data have achieved great success in natural language processing and computer vision, and can be effectively applied to various downstream tasks. In the task of lung nodule detection, traditional object detection algorithms typically use neural networks based on two-dimensional convolutional structures. They process slices of the original CT image in layers, and finally fuse or stack the features or detection results to generate a three-dimensional object detection output. The mainstream algorithm frameworks can be divided into two-stage object detection models represented by the R-CNN series and one-stage object detection models represented by the YOLO series.

[0004] Most current lung nodule detection methods still rely on CNN architectures trained from scratch, lacking solutions to fully utilize the deep semantic information and general image features inherent in the base model. This limitation makes it difficult to obtain sufficiently rich contextual priors for subsequent false positive nodule screening. Therefore, exploring how to effectively introduce the feature mining capabilities of basic medical image models into the lung nodule detection system has become a key challenge to improve the overall system performance and reduce the false alarm rate. Summary of the Invention

[0005] In order to solve the above-mentioned technical problems, this application proposes the following technical solution: In a first aspect, embodiments of this application provide a three-dimensional lung nodule target detection method based on a neural network, including: Acquire 3D CT images of lung nodules and preprocess the images to obtain training and test sets; A target detection model eFATE-Net based on a hybrid architecture of 3D-CNN and Transformer is constructed, and it is trained and evaluated using the training set and test set. The preprocessed 3D CT image to be detected is input into the trained eFATE-Net to achieve target detection.

[0006] In one possible implementation, acquiring three-dimensional CT images of lung nodules and preprocessing the images to obtain training and testing sets includes: Obtain 3D CT images of lung nodules from public datasets; CT images with nodule diameters smaller than a preset value are removed based on the slice sequence and slice spacing. The remaining CT images are labeled, and then the labeled dataset is divided into multiple non-overlapping subsets, which are divided into training set and test set according to a preset ratio.

[0007] In one possible implementation, the eFATE-Net includes a feature extraction module and a false positive reduction module. After a 3D CT image is input into the feature extraction module, the feature extraction module extracts the main features of the 3D CT image and outputs the preliminary predicted probability and location of candidate nodules. The false positive reduction module is used to receive the output of the feature extraction module and re-score the candidate nodules to distinguish between true positive and false positive nodules.

[0008] In one possible implementation, the feature extraction module uses a U-shaped encoder-decoder framework and recovers the spatial information loss caused by downsampling through multi-scale skip connections. The encoder receives the preprocessed 3D CT image and first enters the convolution operation module composed of two sets of basic convolution units to perform preliminary feature extraction. It then completes the first dimensionality reduction by controlling the convolution stride. Next, it performs feature extraction and downsampling through four repeated basic residual convolution modules. The basic residual convolution module is composed of two sets of basic convolution units that have established residual connections. The encoder outputs the feature map after downsampling at the bottleneck layer. In the decoding stage, two layers of 3D transposed convolution are used to upsample the features and fuse them with the corresponding encoder layer features to form a multi-scale representation. The encoder and decoder introduce gated attention modules between each layer of feature maps to dynamically adjust the channel weights, and introduce multi-head attention modules between the encoder and decoder to establish spatial attention relationships between features in the high semantic space. Finally, the encoder's output is input to the region proposal head, which outputs the preliminary predicted probability and location of candidate nodules based on the fused multi-scale features.

[0009] In one possible implementation, the gated attention module includes a global channel content embedding module, a channel normalization module, and gated adaptive activation; The global channel content embedding module receives feature maps. Calculate for each channel Norms are used to extract global information from the feature maps of each channel, and training parameters are used. Used to control the weights of each channel, the feature vector after global channel content embedding is defined as: Where: C, D, H, and W represent the number of channels, depth, height, and width of the feature map, respectively, and c represents the channel index. For the first Channel feature map, Indicates the first The feature value of the channel feature map at position (i, j, k). To prevent numerically unstable minimal constants, express Norm; The channel normalization module then performs channel normalization on the feature vector after global content embedding: Where: global feature vector before normalization , Indicates the first The eigenvalues ​​corresponding to the channel For a minimal constant, a scalar Used for normalization The scale should be adjusted to avoid situations with a large number of channels. The scale is too small; Finally, gated adaptive activation selectively adjusts the activation intensity of feature maps from different channels. Through competition and cooperation among different features during training, attention weights for different channel features are obtained through learnable gating operators. in: Indicates the first The output feature map of the channel after attention modulation. For hyperbolic tangent functions, when the gate weights When a channel is positively activated, its feature map importance is amplified; conversely, it is suppressed.

[0010] In one possible implementation, a multi-scale skip connection module is introduced between the encoder and the decoder to enrich the multi-scale information of the stitched feature map. The multi-scale skip connection module applies 3D convolution, upsampling and downsampling operations to feature maps of different scales respectively, so that the context information from adjacent scales can be aligned and fused, thereby generating a more expressive multi-scale fused feature map.

[0011] In one possible implementation, the false positive reduction module takes the encoder's 4x downsampled feature map as input and extracts fine-grained spatial features and deep semantic features along two parallel branches, respectively. The first branch uses 3D ResNet50 as the basic model skeleton to perform rotation, horizontal flipping, histogram intensity shift transformation and data augmentation operations on the 4x downsampled feature map to construct positive sample pairs and construct negative sample pairs by random sampling. Self-supervised training is completed by maximizing and minimizing the feature similarity between positive and negative sample pairs. The second branch directly uses the 4x downsampled feature map to preserve the texture and spatial information of the nodules. The multi-scale features output from the two branches are further integrated by the feature fusion module, and the confidence of candidate nodules is re-scored by the re-scoring head to more accurately identify false positive nodules.

[0012] In one possible implementation, the feature fusion module includes two processing branches, the first processing branch receiving the deep semantic features output by the first branch. The second processing branch receives the shallow, fine-grained features output from the second branch. ; and After being reduced to 256 dimensions, learnable position embeddings are obtained. and The semantic features of lung nodule images are reconstructed by calculating the global dependencies between features through self-attention. The calculation formula is as follows: Where: Q, K, V are features and The three attention matrices generated by combination are used, where d is the vector dimension and is used as a scaling factor to smooth the softmax output.

[0013] In one possible implementation, the region proposal head consists of multiple 3D convolutional layers with a kernel size of 1×1×1, used to predict the confidence score and location regression parameters of candidate nodules; the rescoring head consists of two parallel fully connected layers, which take the fused candidate nodule features as input to rescore the candidate nodules to reduce false positives. Different loss functions are used for training the region proposal header and the re-scoring header: Loss function of the region proposal header Including classification loss Regression loss The loss function is as follows: in: For the anchor box index, and These represent the total number of anchor boxes for classification and regression, respectively. To address the imbalance between positive and negative samples, a weighted cross-entropy loss is used. for Loss, used to measure the geometric error between the predicted bounding box and the ground truth bounding box, parameters It is used to balance the contributions of classification loss and regression loss. The representative model predicted the first The probability of the existence of candidate nodes in each anchor frame. For the corresponding truth value label, and These represent the relative offsets of the predicted bounding box and the ground truth bounding box relative to the i-th anchor box, respectively, and are specifically defined as: in: These represent the spatial location and border length of the nodule prediction box, respectively. These represent the spatial position and border length of the nodule truth box, respectively. These are the position and size parameters of the anchor frame, respectively. Loss function of re-scoring head Defined as: Where: j is the index of the candidate nodule. and The number of candidate nodules. The representative model predicted the first The probability of a candidate node existing in each suggestion box. For the corresponding truth value label, and These represent the predicted bounding box and the true bounding box relative to the first... The relative offset of each suggestion box; During the training of the region proposal head, difficult examples are selected as training samples to guide the model to focus on training samples with greater discriminative difficulty. During the training of the re-scoring head, a partial random sample training strategy is introduced to enrich the diversity of training samples for the false positive reduction module, thereby increasing the generalization ability of the re-scoring head under different sample distributions.

[0014] Secondly, embodiments of this application provide a three-dimensional lung nodule target detection system based on a neural network, comprising: The acquisition module is used to acquire 3D CT images of lung nodules and preprocess the images to obtain training and test sets; The model building module is used to build an object detection model eFATE-Net based on a hybrid architecture of 3D-CNN and Transformer, and to train and evaluate it using the training set and test set. The target detection module is used to input the preprocessed 3D CT image to be detected into the trained eFATE-Net to achieve target detection.

[0015] In this embodiment, a false positive reduction module is introduced into the target detection model eFATE-Net to fully utilize its learned general image representation and deep semantic feature extraction capabilities, reducing false positives for lung nodules while maintaining high detection sensitivity. A self-attention-based multi-scale feature fusion strategy is designed, using fine-grained spatial features and coarse-grained semantic features of lung nodules to perform global dependency modeling through a self-attention mechanism. This reconstructs and enhances the deep semantic features of lung nodules, aiming to further reduce false positives. A gated attention mechanism is introduced to improve the selectivity of key features, enhancing the model's response to key lung nodule features and improving lung nodule detection performance. Attached Figure Description

[0016] Figure 1 A flowchart illustrating a three-dimensional lung nodule target detection method based on a neural network, provided in an embodiment of this application; Figure 2 This is a schematic diagram of the network framework of eFATE-Net provided in the embodiments of this application; Figure 3 This is a schematic diagram of the gating attention module structure provided in an embodiment of this application; Figure 4 This is a schematic diagram of the multi-scale jump connection module structure provided in an embodiment of this application; Figure 5 This is a schematic diagram of the structure of the Transformer attention module provided in an embodiment of this application; Figure 6 This is a schematic diagram of the structure of the false positive reduction module provided in the embodiments of this application; Figure 7 This is a schematic diagram of the feature fusion module provided in an embodiment of this application; Figure 8 The FROC curve of the ablation experiment provided in the embodiments of this application; Figure 9A comparison chart of FROC curves for different detection methods provided in the embodiments of this application; Figure 10 This is a schematic diagram of a three-dimensional lung nodule target detection system based on a neural network, provided in an embodiment of this application. Detailed Implementation

[0017] The present solution will now be described in conjunction with the accompanying drawings and specific embodiments.

[0018] See Figure 1 The three-dimensional lung nodule target detection method based on neural networks provided in this embodiment includes: S101. Acquire 3D CT images of lung nodules and preprocess the images to obtain training and test sets.

[0019] In this embodiment, 3D CT images of lung nodules were obtained from the LIDC-IDRI database. These images were then filtered, with samples containing slices thicker than 2.5 mm, inconsistent slice spacing, or nodule diameters less than 3 mm being removed. This resulted in a high-quality subset comprising 888 low-dose chest CT scans and 1168 labeled nodules. All nodule annotations were performed by experienced radiologists using a two-stage blind reading and deblinding process, ensuring the reliability of the dataset.

[0020] S102, Construct the object detection model eFATE-Net based on a hybrid architecture of 3D-CNN and Transformer, and train and evaluate it using the training set and test set.

[0021] In this embodiment, eFATE-Net is an end-to-end neural network based on a hybrid architecture of 3D CNN and Transformer. The network framework structure is as follows: Figure 2 As shown, eFATE-Net mainly consists of two parts: a feature extraction module for main feature extraction and candidate nodule proposal, and a false positive reduction module that integrates the base model. After the 3D CT image is input into the feature extraction module, the module extracts the main features of the 3D CT image and outputs the preliminary predicted probability and location of candidate nodules. The false positive reduction module receives the output of the feature extraction module and re-scores the candidate nodules to distinguish between true positives and false positives.

[0022] The feature extraction module of eFATE-Net uses a U-shaped encoder-decoder framework and recovers the spatial information loss caused by downsampling through multi-scale skip connections. The model input is a preprocessed 3D CT image with dimensions D×H×W. It first enters the StemConv module, consisting of two sets of basic convolutional units, for initial feature extraction, and performs the first dimensionality reduction by controlling the convolution stride. The basic convolutional unit consists of a 3×3×3 convolution, 3D batch normalization, and a ReLU activation layer in sequence. Then, feature extraction and downsampling are performed through four repeated BasicRes modules, which consist of two sets of basic convolutional units with residual connections, until the bottleneck layer outputs a feature map with dimensions D / 16×H / 16×W / 16.

[0023] In the decoding stage, two layers of 3D transposed convolutions are used to upsample the features to D / 4×H / 4×W / 4, and then fused with the features from the corresponding encoder layer to form a multi-scale representation. To enhance feature representation, the module introduces a gated attention mechanism between feature maps of each layer to dynamically adjust channel weights, and introduces a Transformer architecture between bottleneck feature layers to establish spatial attention relationships between features in a high semantic space. Finally, the RPN head outputs the preliminary predicted probability and location of candidate nodules based on the fused multi-scale features.

[0024] The gated attention module differs from the channel attention mechanisms such as SE and ECA used in existing DeepSEED and MSANet. These methods model channel relationships based on fully connected layers or convolutional operations, which are essentially implicit modeling processes. In contrast, this module models channel importance through explicit, learnable gating operators, adaptively adjusting the weights of different channel feature maps to guide the model to focus on key features of lung nodules. This not only improves detection sensitivity but also enhances the model's interpretability.

[0025] like Figure 3 As shown, the gating attention module is mainly divided into three parts: global channel content embedding module, channel normalization module, and gating adaptive activation.

[0026] Assume the shape of the input feature map is Where C, D, H, and W represent the number of channels, depth, height, and width of the feature map, respectively. First, the module calculates for each channel... Norms are used to extract global information from the feature maps of each channel, and trainable parameters are available. Used to control the weights of each channel, the feature vector after global channel content embedding is defined as: in: Indicates the index of the channel. For the first Channel feature map, Indicates the first The feature value of the channel feature map at position (i, j, k). To prevent numerically unstable minimal constants, express Norm.

[0027] Then, channel normalization is performed on the feature vector after global content embedding. The calculation method is as follows: Where: global feature vector before normalization , Indicates the first The eigenvalues ​​corresponding to the channels, scalars Used for normalization The scale should be adjusted to avoid situations with a large number of channels. The scale is too small.

[0028] Gated adaptive activation is used to selectively adjust the activation intensity of feature maps in different channels. Through the competition and cooperation of different features during training, the model can obtain the attention weights of different channel features through learnable gating operators.

[0029] Assuming gating weights Gating deviation The gated adaptive activation function is shown below: in: Indicates the first The output feature map of the channel after attention modulation. It is a hyperbolic tangent function. When the gate weight... When a channel is positively activated, its feature map importance is amplified; conversely, it is suppressed.

[0030] To enrich the multi-scale information of the stitched feature map, this embodiment introduces a multi-scale skip connection module between the encoder and decoder. This module applies 3D convolution, upsampling, and downsampling operations to feature maps at different scales, aligning and fusing contextual information from adjacent scales to generate a more expressive multi-scale fused feature map. Finally, this fused feature map is fused with feature maps at various levels of the decoder through skip connections, enhancing the linkage between shallow spatial details and deep semantic information, and helping to improve the detection sensitivity of small lung nodules. The structure of this module is as follows: Figure 4 As shown.

[0031] Furthermore, a Transformer attention module is introduced at the bottleneck layer of the U-shaped encoder-decoder, aiming to compensate for the limitations of the local receptive field of the convolutional layer by leveraging its ability to capture long-range dependencies. The structure of this module is as follows: Figure 5 As shown.

[0032] Specifically, the deep semantic feature map is first flattened into a sequence representation, and interleaved sine and cosine positional encodings are added to preserve the three-dimensional spatial position of the feature map. Assuming the input feature map size is (C, D, H, W), the feature sequence after positional embedding has a shape of (D×H×W, C). Subsequently, the feature sequence is fed into a standard Transformer architecture consisting of a stacked 6-layer encoder and a 6-layer decoder. In the decoding stage, a fixed-length learnable vector is introduced as an object query, and the feature sequence output by the encoder serves as the key and value for cross-attention computation. Through adaptive learning during training, these queries can effectively locate key semantic regions in the image, thereby achieving focused extraction of key nodal features. The output of the Transformer is then restored to a shape matching the original feature map and fed into subsequent networks for further processing.

[0033] In this embodiment, the false positive reduction module is used to re-evaluate the candidate nodules output by the RPN header to further distinguish between true positive and false positive nodules. Figure 6 As shown, the false positive reduction module takes a 4x downsampled feature map of the candidate nodules as input and extracts fine-grained spatial features and deep semantic features along two parallel branches.

[0034] The first branch uses a 3D ResNet50 as the basic model skeleton to perform data augmentation operations such as rotation, horizontal flipping, and histogram intensity shift transformation on the 4x downsampled feature map patches to construct positive sample pairs. Negative sample pairs are constructed using random sampling, and self-supervised training is completed by maximizing and minimizing the feature similarity between positive and negative sample pairs. The second branch directly uses the 4x downsampled feature map to preserve the texture and spatial information of the nodules. The multi-scale features output from both branches are further integrated by the feature fusion module, and the confidence of candidate nodules is re-scored by the re-scoring head to more accurately identify false positive nodules.

[0035] In this embodiment, before inputting the 4x downsampled feature map into the first and second branches, a candidate box is used for Region of Interest (ROI) truncation. The regression output of the region proposal head provides the model's preliminary localization information for suspected lung nodules. For each output candidate box, its parameters are (x, y, z, d, h, w), representing the relative position and relative side length of the region proposal box on the 3D coordinate axes, respectively. After the model's region proposal head obtains the coordinates of the proposal boxes, the false positive reduction module truncates the feature map at the corresponding position on the 4x downsampled feature map obtained in the feature extractor based on the coordinates of the proposal boxes, and uses a 3D adaptive max pooling method to unify the feature map size to 7×7×7. This 7×7×7 feature image block is the input image for the false positive reduction module.

[0036] In the process of feature extraction by neural network models, shallow features often have higher resolution and contain more fine-grained spatial information, but their semantics are lower and they contain more noise. Deep features have deeper semantic information, but their resolution is lower and their ability to perceive details is poor. In this embodiment, several different feature-level fusion strategies are compared to fully explore a fusion method that takes into account both the high-resolution details of shallow features and the rich semantics of deep features. Ultimately, this application determines the following... Figure 7 The feature fusion module shown.

[0037] See Figure 7 The feature fusion module includes two processing branches. The first processing branch receives the deep semantic features output by the first branch. The second processing branch receives the shallow, fine-grained features output from the second branch. .

[0038] and After reducing the dimensionality of each feature to 256, the feature is obtained through learnable position embedding. and Self-attention is used to calculate global dependencies between features to reconstruct the semantic features of lung nodule images. The calculation formula is as follows: Where Q, K, and V are features and The three attention matrices generated by combination are used, where d is the vector dimension and is used as a scaling factor to smooth the softmax output.

[0039] In this embodiment, eFATE-Net includes two key detection heads: a region proposal head for candidate nodule detection and a re-scoring head for candidate nodule re-scoring. The region proposal head consists of multiple 3D convolutional layers with a kernel size of 1×1×1, used to predict the confidence score and location regression parameters of candidate nodules; the re-scoring head consists of two parallel fully connected layers, which take the fused candidate nodule features as input to re-score the candidate nodules to reduce false positives. Different loss functions are used for training the region proposal header and the re-scoring header. The loss function for the region proposal header is as follows: Including classification loss Regression loss The loss function is as follows: in: For the anchor box index, and These represent the total number of anchor boxes for classification and regression, respectively. To address the imbalance between positive and negative samples, a weighted cross-entropy loss is used. for Loss, used to measure the geometric error between the predicted bounding box and the ground truth bounding box, parameters It is used to balance the contributions of classification loss and regression loss. The representative model predicted the first The probability of the existence of candidate nodes in each anchor frame. For the corresponding truth value label, and These represent the relative offsets of the predicted bounding box and the ground truth bounding box relative to the i-th anchor box, respectively, and are specifically defined as: in: These represent the spatial location and border length of the nodule prediction box, respectively. These represent the spatial position and border length of the nodule truth box, respectively. These are the position and size parameters of the anchor frame, respectively. Loss function of re-scoring head Defined as: in: Index of candidate nodules, and The number of candidate nodules. The representative model predicted the first The probability of a candidate node existing in each suggestion box. For the corresponding truth value label, and These represent the predicted bounding box and the true bounding box relative to the first... The relative offset of each suggestion box.

[0040] During the training of the region proposal head, difficult examples are selected as training samples to guide the model to focus on training samples with greater discriminative difficulty. During the training of the re-scoring head, a partial random sample training strategy is introduced to enrich the diversity of training samples for the false positive reduction module, thereby increasing the generalization ability of the re-scoring head under different sample distributions.

[0041] S103, the preprocessed 3D CT image to be detected is input into the trained eFATE-Net to achieve target detection.

[0042] When performing target detection tasks, the first step is to perform standardized preprocessing operations on the 3D CT images to be detected. This includes key steps such as image grayscale normalization, noise filtering, region of interest cropping, and voxel resampling. This eliminates interference caused by differences in equipment and scanning parameters, improving the quality and consistency of the image data. Subsequently, the preprocessed 3D CT image data is input into the trained eFATE-Net model according to a preset input format. Leveraging the model's deep feature extraction and accurate modeling capabilities, the target region can be efficiently located, identified, and detected, providing reliable data support for subsequent clinical diagnosis or industrial inspection applications.

[0043] To verify the neural network-based three-dimensional lung nodule target detection method proposed in this embodiment, the following experiments were also conducted.

[0044] This embodiment first validates the effectiveness of the proposed lung nodule detection method and its sub-modules on the LUNA16 public dataset. This dataset originates from the LIDC-IDRI database. After removing scan samples with slice thickness greater than 2.5 mm, inconsistent slice spacing, and nodule diameter less than 3 mm, a high-quality subset containing 888 low-dose chest CT scans and 1168 labeled nodules was obtained. All nodule annotations were completed by four experienced radiologists using a two-stage blind and deblinded reading process, ensuring the reliability of the annotation quality. To fully evaluate model performance, we employ a standard 10-fold cross-validation experiment, dividing the entire dataset into 10 non-overlapping subsets. Nine subsets are used for training in each round, with the remaining subset used for testing. The final average performance metrics are reported.

[0045] For evaluation metrics, the FROC curve was used as the primary indicator. The FROC curve, with the number of false positives per scan (FPs / scan) on the horizontal axis and detection sensitivity on the vertical axis, reflects the target detection model's ability to detect targets at different false alarm levels. Specifically, we evaluated the model's sensitivity at seven predefined thresholds (0.125, 0.25, 0.5, 1, 2, 4, 8 FPs / scan). We also calculated the average detection precision (CPM) to evaluate the system's overall performance in multi-target detection, considering both detection sensitivity and false alarm rate. CPM was defined as the arithmetic mean of the sensitivity at the seven false positive thresholds.

[0046] In the data preprocessing stage, the original CT images were first cropped in intensity between -1200 and 600 HU to remove interference from dense bone structures. Then, the image intensity was normalized to the range of [0, 255] and resampled to isotropic voxels with a spacing of 1 mm to unify the spatial resolution. Finally, a mask was added to remove irrelevant areas.

[0047] During model training, data augmentation operations such as random flipping, random rotation, and random scaling were used to enhance the diversity of training samples. The model underwent a total of 150 training epochs, with only the feature encoder and RPN head participating in the training for the first 64 epochs. Stochastic gradient descent (SGD) was chosen as the optimizer for the neural network, with an initial learning rate of 0.01, which began to decay in stages to 0.0001 after 120 epochs. The optimizer's momentum and weight decay were set to 0.9 and 0.0001, respectively, and the batch size was 4. All experiments were performed on a server equipped with two Nvidia GeForce GTX 4090 GPUs. The model and experimental code were implemented using Python 3.9 and the PyTorch framework.

[0048] A standard 10-fold cross-validation experiment was conducted on the LUNA16 dataset, and the model performance was evaluated using FROC curves, CPM, and sensitivity metrics. Table 1 shows the detection sensitivity of the proposed eFATE-Net at various folds and different FPs / scan settings. The CPM reached 91.22%, and under the low false positive condition of 0.125 FPs / scan, the model achieved a high sensitivity of 81.23%, indicating that this method can greatly control the number of false positives while maintaining a high lung nodule detection capability.

[0049] Table 1. Quantitative detection results of 10-fold cross-validation on the LUNA16 dataset. In addition to overall detection performance, this experiment further analyzed the model's ability to detect lung nodules of different sizes, and the results are shown in Table 2. Nodules with diameters of 3-10 mm, 10-20 mm, and >20 mm were defined as small nodules, medium nodules, and large nodules, respectively. It can be found that eFATE-Net achieved a detection sensitivity of 96.08% for all nodules and a sensitivity of 95.91% for the more difficult-to-detect small nodules.

[0050] Table 2. 10-fold cross-validation results of detection sensitivity for lung nodules of different sizes. Under the same training configuration and hyperparameter settings, three key components of the eFATE-Net framework—the base model branch, the false positive reduction module, and the gated attention module—were removed to explore their contributions to the model's nodule detection performance and false positive suppression ability. All ablation experiments were conducted on the LUNA16 dataset using 10x cross-validation, and the overall detection performance was evaluated using FROC curves and CPM metrics. Furthermore, to more intuitively compare the effects of different components on reducing false positives, we defined a confidence threshold t for lung nodules; targets with a predicted probability higher than t were identified as nodules, and the false positive rate (FPs / scan) was calculated at t=0.3, 0.5, and 0.9.

[0051] The results of the ablation experiments are shown in Table 3. The CPM of the complete model was 91.22%. After removing the basic model branch, the false positive reduction module, and the gated attention module, the CPM decreased to 90.14%, 89.55%, and 89.77%, respectively. The FROC curves of the ablation experiments are shown in Table 3. Figure 8 As shown, this further demonstrates the crucial role of these three components in improving model performance.

[0052] Table 3 Ablation Experiment Results Specifically, removing the gated attention module had the greatest impact on sensitivity under high false positive (8 FPs / scan) conditions, decreasing it by 1.03%. However, at a low false positive level of 0.125 FPs / scan, the decrease in sensitivity was far less than the 3.68% decrease after removing the false positive reduction module. This indicates that the gated attention module, through selective amplification of key model features, has a positive impact on both improving detection sensitivity and suppressing false positive rates, but its main effect is to improve the overall sensitivity of nodule detection. At a low false positive level of 0.125 FPs / scan, removing the base model branch and the false positive reduction module reduced the sensitivity of the complete model from 81.23% to 78.80% and 77.55%, respectively. To more intuitively evaluate the contribution of each component to false positive suppression, using a confidence threshold of t=0.5 as an example, the model without the false positive reduction module had 7.18 false positives per scan, while the complete model using the false positive reduction module reduced the number of false positives per scan to 2.71. The base model is an important branch of the false positive reduction module. If only this branch is removed, the number of false positives increases to 4.80, indicating that the base model plays an important role in the identification of true and false nodules and can effectively capture the deep semantic information of lung nodules.

[0053] In summary, the base model, as an important branch of the false positive reduction module, provides key deep semantic information for the identification of false positive nodules, assisting the false positive reduction module in playing a core role in filtering false positive nodules. Meanwhile, the attention mechanism of the gating attention module selects key nodule features in the channel dimension, improving the overall sensitivity of the model detection. The three work together to drive the optimal performance of this method on the lung nodule detection task.

[0054] To explore the optimal strategy for feature fusion in the false positive reduction module, we replaced the feature fusion module in the backbone model while keeping the rest of the network structure and hyperparameters unchanged, and evaluated its performance on the LUNA16 dataset. The experimental results are shown in Table 4.

[0055] Table 4 Comparison of detection performance under different feature fusion strategies Fusion strategy (d) outperforms the other three strategies at all FPs / scan levels, improving CPM by 0.47% compared to fusion strategy (c) and by 1.13% compared to the worst-performing fusion strategy (a). This strategy calculates the global dependencies between features through a self-attention mechanism, adaptively assigning importance weights to features at each scale, and compensating for the lack of detail in deep features by using shallow features, achieving alignment and complementarity between fine-grained textures and deep semantic features. In contrast, strategies (a), (b), and (c) mainly rely on feature concatenation or layer-by-layer dimensionality reduction and concatenation, lacking the ability to model long-range dependencies between features, resulting in insufficient information fusion and limited performance improvement potential. On the other hand, although strategy (a) introduces three multi-scale feature branches in its structure, which theoretically helps with multi-level information complementarity, its training process suffers from instability issues such as gradient explosion, ultimately leading to poor convergence. This instability may stem from the fact that the parallel features of the three branches introduce too many noise channels, and the gradient is propagated and amplified between the three branches, exacerbating numerical instability. Therefore, introducing a feature fusion strategy (d) with learnable location embedding and self-attention mechanism can more effectively reconstruct and enhance the semantic features of lung nodules and improve the overall detection performance.

[0056] To further verify the effectiveness of the gated attention module in lung nodule detection, while maintaining consistency in other network structures and training hyperparameters, we replaced the gated attention module in the target detection model with three classic attention mechanisms: SE, ECA, and CBAM, and compared it with the baseline model without an attention module in the ablation study. The experimental results are shown in Table 5. The gated attention module achieved a sensitivity of 81.23% and a CPM of 91.22% at 0.125 FPs / scan, representing improvements of 1.45, 0.79, and 0.61 percentage points compared to the model without an attention module, SE, and ECA, respectively. Compared to the SE and ECA channel attention mechanisms, the gated attention module explicitly models the importance of each channel through learnable gating weights and biases, allowing the network to "switch" the response of each channel. This avoids the information redundancy and uniform weighting problems introduced in channel modeling using ordinary fully connected (SE) or fixed convolutional kernels (ECA), thus accurately amplifying nodule-related channel features and suppressing redundant noise. It is worth noting that when the model adopts the CBAM spatial channel attention mechanism, the detection performance degrades significantly, with CPM decreasing from 89.77% to 87.74%. This phenomenon may be attributed to the fact that the spatial attention of the CBAM module mainly relies on fixed convolutional kernels. As the downsampling layers of the network increase, its receptive field expands rapidly, causing excessive smoothing of deep feature maps. This leads to a decrease in the model's ability to perceive small lesion areas and an inability to effectively focus on local details such as small lung nodules, thus resulting in a decrease in detection sensitivity.

[0057] Table 5 Comparison of detection performance under different attention mechanisms To comprehensively evaluate the advantages of the method proposed in this embodiment, several state-of-the-art lung nodule target detection methods were compared with the method provided in this embodiment on the LUNA16 dataset, including 3D Faster RCNN

[10] , Deepplung

[10] , DeepSeed[9], SANet

[42] , OMS-CNN

[43] and TicNet

[30] . The FROC curves and CPM results of each method are shown below. Figure 9 As shown in Table 6.

[0058] Table 6 Comparative Study Results of Different Methods Compared to 3D Faster RCNN, which directly uses 3D ResNet18 as the backbone network, the method provided in this embodiment designs a U-shaped encoder-decoder backbone network framework to recover spatial resolution layer by layer, thereby better locating small targets. Compared to Deeplung, which uses 3D dual-path blocks and traditional skip connections, the method provided in this embodiment uses a multi-scale skip connection module to enrich the contextual information of nodules by aggregating feature maps at different scales.

[0059] DeepSeed uses a channel attention module (SE) to enhance the model's feature extraction capabilities, implicitly capturing channel relationships through channel compression and complex fully connected layers. In contrast, the method presented in this embodiment uses a gated attention mechanism. Through learnable gating operators, it can explicitly establish competition and cooperation relationships between different channel feature maps without compressing channel features, adaptively adjusting feature map weights to guide the model to focus on key nodule features, achieving a CPM improvement of over 5% compared to DeepSeed.

[0060] Unlike the complex slice grouping non-local module used by SANet, the method provided in this embodiment uses the Transformer architecture to better capture long-distance dependencies between features, and reconstructs the semantic features of lung nodules by feature fusion through the global dependencies between shallow and deep features, which are used to improve the sensitivity of nodule detection and identify false positive nodules, respectively.

[0061] Compared with the 2D-3D multi-scale hybrid CNN architecture used in OMS-CNN, the method provided in this embodiment uses a 3D convolutional architecture to extract the 3D spatial features of the original image, avoiding the loss of spatial information in the slice dimension.

[0062] Compared to TicNet, the method presented in this embodiment introduces a self-supervised learning-based cancer image model to further extract deep semantic features of lung nodules. It also incorporates a Transformer-based feature-level fusion strategy to reduce false positives in nodule detection, improving sensitivity by 3.6% at 0.125 FPs / scan. Experimental results demonstrate that the model in this embodiment achieves the highest CPM and outperforms other detection methods in sensitivity across all FPs / scan levels, particularly excelling in nodule detection at lower false positive rates.

[0063] Corresponding to the three-dimensional lung nodule target detection method based on neural networks provided in the above embodiments, this embodiment also provides an embodiment of a three-dimensional lung nodule target detection system based on neural networks.

[0064] See Figure 10 The neural network-based three-dimensional lung nodule target detection system 20 in this embodiment includes: The acquisition module 201 is used to acquire three-dimensional CT images of lung nodules and preprocess the images to obtain training and test sets.

[0065] The model building module 202 is used to build an object detection model eFATE-Net based on a hybrid architecture of 3D-CNN and Transformer, and to train and evaluate it using the training set and test set.

[0066] The target detection module 203 is used to input the preprocessed three-dimensional CT image to be detected into the trained eFATE-Net to achieve target detection.

[0067] In this application embodiment, "at least one" refers to one or more, and "more than one" refers to two or more. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent the existence of A alone, the simultaneous existence of A and B, or the existence of B alone. A and B can be singular or plural. The character " / " generally indicates that the preceding and following related objects have an "or" relationship. "At least one of the following" and similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one of a, b, and c can represent: a, b, c, ab, ac, bc, or abc, where a, b, and c can be single or multiple.

[0068] The above description is merely a specific embodiment of this application. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the protection scope of this application. The protection scope of this application should be determined by the protection scope of the claims.

Claims

1. A method for detecting three-dimensional lung nodules based on neural networks, characterized in that, include: Acquire 3D CT images of lung nodules and preprocess the images to obtain training and test sets; A target detection model eFATE-Net based on a hybrid architecture of 3D-CNN and Transformer is constructed, and it is trained and evaluated using the training set and test set. The preprocessed 3D CT image to be detected is input into the trained eFATE-Net to achieve target detection.

2. The method for detecting three-dimensional lung nodules based on neural networks according to claim 1, characterized in that, The process of acquiring 3D CT images of lung nodules and preprocessing the images to obtain training and testing sets includes: Obtain 3D CT images of lung nodules from public datasets; CT images with nodule diameters smaller than a preset value are removed based on the slice sequence and slice spacing. The remaining CT images are labeled, and then the labeled dataset is divided into multiple non-overlapping subsets, which are divided into training set and test set according to a preset ratio.

3. The method for detecting three-dimensional lung nodules based on neural networks according to claim 1, characterized in that, The eFATE-Net includes a feature extraction module and a false positive reduction module. After a 3D CT image is input into the feature extraction module, the feature extraction module extracts the main features of the 3D CT image and outputs the preliminary predicted probability and location of candidate nodules. The false positive reduction module is used to receive the output of the feature extraction module and re-score the candidate nodules to distinguish between true positive and false positive nodules.

4. The method for detecting three-dimensional lung nodules based on neural networks according to claim 3, characterized in that, The feature extraction module uses a U-shaped encoder-decoder as its basic framework and recovers the spatial information loss caused by downsampling through multi-scale skip connections. The encoder receives the preprocessed 3D CT image and first enters the convolution operation module composed of two sets of basic convolution units to perform preliminary feature extraction. It then completes the first dimensionality reduction by controlling the convolution stride. Next, it performs feature extraction and downsampling through four repeated basic residual convolution modules. The basic residual convolution module is composed of two sets of basic convolution units that have established residual connections. The encoder outputs the feature map after downsampling at the bottleneck layer. In the decoding stage, two layers of 3D transposed convolution are used to upsample the features and fuse them with the corresponding encoder layer features to form a multi-scale representation. The encoder and decoder introduce gated attention modules between each layer of feature maps to dynamically adjust the channel weights, and introduce multi-head attention modules between the encoder and decoder to establish spatial attention relationships between features in the high semantic space. Finally, the encoder's output is input to the region proposal head, which outputs the preliminary predicted probability and location of candidate nodules based on the fused multi-scale features.

5. The method for detecting three-dimensional lung nodules based on neural networks according to claim 4, characterized in that, The gated attention module includes a global channel content embedding module, a channel normalization module, and gated adaptive activation; The global channel content embedding module receives feature maps. Calculate for each channel Norms are used to extract global information from the feature maps of each channel, and training parameters are used. Used to control the weights of each channel, the feature vector after global channel content embedding is defined as: Where: C, D, H, and W represent the number of channels, depth, height, and width of the feature map, respectively, and c represents the channel index. For the first Channel feature map, Indicates the first The feature value of the channel feature map at position (i, j, k). To prevent numerically unstable minimal constants, express Norm; The channel normalization module then performs channel normalization on the feature vector after global content embedding: Where: global feature vector before normalization , Indicates the first The eigenvalues ​​corresponding to the channels, scalars Used for normalization The scale should be adjusted to avoid situations with a large number of channels. The scale is too small; Finally, gated adaptive activation selectively adjusts the activation intensity of feature maps from different channels. Through competition and cooperation among different features during training, attention weights for different channel features are obtained through learnable gating operators. in: Indicates the first The output feature map of the channel after attention modulation. For hyperbolic tangent functions, when the gate weights When a channel is positively activated, its feature map importance is amplified; conversely, it is suppressed.

6. The method for detecting three-dimensional lung nodules based on neural networks according to claim 4 or 5, characterized in that, A multi-scale skip connection module is introduced between the encoder and the decoder to enrich the multi-scale information of the stitched feature map. The multi-scale skip connection module applies 3D convolution, upsampling and downsampling operations to feature maps of different scales respectively, so that the context information from adjacent scales can be aligned and fused, thereby generating a more expressive multi-scale fused feature map.

7. The method for detecting three-dimensional lung nodules based on neural networks according to claim 4, characterized in that, The false positive reduction module takes the encoder's 4x downsampled feature map as input and extracts fine-grained spatial features and deep semantic features along two parallel branches respectively. The first branch uses 3D ResNet50 as the basic model skeleton to perform rotation, horizontal flipping, histogram intensity shift transformation and data augmentation operations on the 4x downsampled feature map to construct positive sample pairs and construct negative sample pairs by random sampling. Self-supervised training is completed by maximizing and minimizing the feature similarity between positive and negative sample pairs. The second branch directly uses the 4x downsampled feature map to preserve the texture and spatial information of the nodules. The multi-scale features output from the two branches are further integrated by the feature fusion module, and the confidence of candidate nodules is re-scored by the re-scoring head to more accurately identify false positive nodules.

8. The method for detecting three-dimensional lung nodules based on neural networks according to claim 7, characterized in that, The feature fusion module includes two processing branches. The first processing branch receives the deep semantic features output by the first branch. The second processing branch receives the shallow, fine-grained features output from the second branch. ; and After being reduced to 256 dimensions, learnable position embeddings are obtained. and The semantic features of lung nodule images are reconstructed by calculating the global dependencies between features through self-attention. The calculation formula is as follows: Where: Q, K, V are features and The three attention matrices generated by combination are used, where d is the vector dimension and is used as a scaling factor to smooth the softmax output.

9. The method for detecting three-dimensional lung nodules based on neural networks according to claim 7, characterized in that, The region proposal head consists of multiple 3D convolutional layers with a kernel size of 1×1×1, used to predict the confidence score and location regression parameters of candidate nodules; the rescoring head consists of two parallel fully connected layers, which take the fused candidate nodule features as input to rescore the candidate nodules to reduce false positives. Different loss functions are used for training the region proposal header and the re-scoring header: Loss function of the region proposal header Including classification loss Regression loss The loss function is as follows: in: For the anchor box index, and These represent the total number of anchor boxes for classification and regression, respectively. To address the imbalance between positive and negative samples, a weighted cross-entropy loss is used. for Loss, used to measure the geometric error between the predicted bounding box and the ground truth bounding box, parameters It is used to balance the contributions of classification loss and regression loss. The representative model predicted the first The probability of the existence of candidate nodes in each anchor frame. For the corresponding truth value label, and These represent the relative offsets of the predicted bounding box and the ground truth bounding box relative to the i-th anchor box, respectively, and are specifically defined as: in: These represent the spatial location and border length of the nodule prediction box, respectively. These represent the spatial position and border length of the nodule truth box, respectively. These are the position and size parameters of the anchor frame, respectively. Loss function of re-scoring head Defined as: in: Index of candidate nodules, and The number of candidate nodules. The representative model predicted the first The probability of a candidate node existing in each suggestion box. For the corresponding truth value label, and These represent the predicted bounding box and the true bounding box relative to the first... The relative offset of each suggestion box; During the training of the region proposal head, difficult examples are selected as training samples to guide the model to focus on training samples with greater discriminative difficulty. During the training of the re-scoring head, a partial random sample training strategy is introduced to enrich the diversity of training samples for the false positive reduction module, thereby increasing the generalization ability of the re-scoring head under different sample distributions.

10. A three-dimensional lung nodule target detection system based on neural networks, characterized in that, include: The acquisition module is used to acquire 3D CT images of lung nodules and preprocess the images to obtain training and test sets; The model building module is used to build an object detection model eFATE-Net based on a hybrid architecture of 3D-CNN and Transformer, and to train and evaluate it using the training set and test set. The target detection module is used to input the preprocessed 3D CT image to be detected into the trained eFATE-Net to achieve target detection.

Images