A distributed small-scale lung nodule detection method based on federated learning
By employing a distributed training method within a federated learning framework, combined with the AdaGrad optimizer and feature pyramid network, the accuracy and speed issues of small-scale lung nodule detection are addressed. This approach is suitable for lung lesion detection and large-scale lung cancer screening, achieving efficient and accurate lung nodule detection while protecting data privacy.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- DALIAN UNIV
- Filing Date
- 2023-10-31
- Publication Date
- 2026-05-12
AI Technical Summary
Existing technologies struggle to efficiently and accurately detect small-scale lung nodules in medical images, and the difficulty in acquiring data poses a challenge to traditional centralized data-based training models.
A distributed small-scale lung nodule detection method based on federated learning is adopted. By averaging the training time of models from multiple local clients, and combining the AdaGrad optimizer, CBAM module and FPN network, the detection of small-scale lesions of lung nodules is achieved.
It improves the accuracy and speed of detecting small-scale lung nodules, and is suitable for lung lesion detection, lung cancer screening, and medical research data analysis, while ensuring data privacy and model generalization ability.
Smart Images
Figure CN117237332B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of target detection technology, specifically to a distributed small-scale lung nodule detection method based on federated learning. Background Technology
[0002] In recent years, driven by the rapid development of deep learning (DL), object detection algorithms have been applied to multiple fields. In the field of smart healthcare, deep learning in image recognition, particularly visual algorithms implemented through convolutional neural networks, surpassed human capabilities in both error rate and efficiency in 2015. The goal of small-scale lung nodule detection is to accurately detect small-scale lesions with limited visual features in medical images. Deep learning still faces significant challenges in this area. Compared to targets of normal size, small-scale lesions contain very limited appearance information, making them difficult to separate from complex backgrounds or adjacent targets.
[0003] The success of deep learning is built upon training on massive amounts of data, but obtaining sufficient high-quality medical data is extremely difficult. With increasingly stringent laws and regulations, regulatory bodies are imposing stricter approval processes for data use, and the era of easily acquiring large amounts of data is over. Data is the fuel of the artificial intelligence era, and as the difficulty of obtaining high-quality training data continues to rise, traditional centralized data-driven model training methods are facing significant challenges.
[0004] Federated Learning (FL), proposed in 2016 by H. Brendan McMahan and others from Google, is a privacy-focused distributed machine learning technique. It allows deep learning algorithms to be deployed on various clients, training models using local data. After local training, a central server aggregates the models and redistributes them to the clients for further training. This ensures that data remains locally while participating parties collaboratively train the model, overcoming data privacy constraints. Furthermore, Federated Learning is a distributed computing framework that enables distributed computing on edge devices with limited computing power. For example, Google Gboard deploys federated learning models on multiple phones with the Gboard virtual keyboard installed, training the model through a distributed federated learning framework. This achieves collaborative optimization of the overall model without transferring user data, further enhancing the user experience. By introducing federated learning, the problem of data silos can be effectively solved. Moreover, federated learning achieves convergence in a shorter time compared to centralized deep learning during distributed training. In federated training, the participating clients are randomly selected for each round of training, and the training data varies. Compared to centralized deep learning training, it can further learn the global distribution characteristics of the data, resulting in better model generalization ability. Therefore, researching small-scale object detection within a distributed federated learning framework is of great significance. Summary of the Invention
[0005] The purpose of this invention is to propose a distributed small-scale lung nodule detection method based on federated learning. This method can not only detect small-scale lesions more accurately, but also provide more information about the characteristics of the lesions, providing doctors with a more comprehensive and accurate basis for diagnosis.
[0006] To achieve the above objectives, this application proposes a distributed small-scale lung nodule detection method based on federated learning, comprising:
[0007] Acquire and preprocess the patient's actual lung CT images;
[0008] The preprocessed CT images were input into the small-scale lesion detection model for lung nodules for training. The training time was averaged by using multiple local clients.
[0009] The AdaGrad optimizer is used to achieve convergence of the small-scale lesion detection model for lung nodules by utilizing sparse gradient information under discrete local clients.
[0010] The converged result is input into the CBAM module to obtain a feature map containing spatial and channel attention features;
[0011] The feature map containing spatial and channel attention features is input into the FPN multi-scale prediction network to obtain a multi-scale feature map, and a new feature pyramid is obtained based on the multi-scale feature map.
[0012] The new feature pyramid is input into the RPN network, which generates candidate boxes. These boxes are then fed into the RoIPooling layer to obtain four feature vectors of the same size. Finally, the feature vectors are fused into a new feature vector in a specified dimension and placed in a fully connected layer to complete the recognition task.
[0013] Furthermore, the average training time of the model across multiple local clients during training is as follows:
[0014]
[0015] Where Tavg is the average training time of the model in the first s rounds.
[0016] Furthermore, the AdaGrad optimizer is implemented as follows:
[0017]
[0018]
[0019] In the formula, J is the loss function; g t,i θ represents time t i Gradient; G t,ii Represents the parameter θ at time t i The sum of squared gradients.
[0020] Furthermore, the CBAM module includes a channel attention module and a spatial attention module. The channel attention module performs global max pooling and global average pooling based on width and height on the input feature map F(H×W×C) to obtain different spatial semantic descriptors, which are then fed into a shared two-layer neural network (MLP) with ReLU activation function. The output feature maps of the shared two-layer neural network are then summed and activated by sigmoid to generate a channel attention feature, Mc(F). Finally, Mc(F) is multiplied by the input feature map F to obtain a feature map F' containing channel attention, as shown in the following formula:
[0021]
[0022] Note: In the formula, σ is the sigmoid function, W0 = W1= ;
[0023]
[0024] The spatial attention module takes the output feature map F' as input and performs global max pooling and global average pooling operations along the channel dimension to obtain two H×W×1 feature maps respectively. Then, a concat operation is performed based on the channels, followed by a convolution operation using a 7×7 convolution kernel to reduce the dimensionality to one channel H×W×1. After passing through a sigmoid function, the spatial attention feature, Ms(F), is generated. Finally, Ms(F) is multiplied by the input feature map F' of the module to obtain the feature map F' containing channel attention and spatial attention. The formula is as follows:
[0025]
[0026] Note: In the formula, σ represents the sigmoid function, and f 7×7 This represents a convolution operation with a filter size of 7×7.
[0027] Furthermore, the FPN multi-scale prediction network upsamples the extracted top-level features to obtain a feature map in a top-down manner, and then adds the bottom-level features and the upsampled feature map element by element to obtain a multi-scale feature map.
[0028] Furthermore, the feature pyramids are combined with 3*3 convolution kernels to eliminate aliasing effects for each fusion result, generating new feature pyramid layers P2, P3, P4, and P5, which are consistent with the convolution results of feature maps C2, C3, C4, and C5 at different scales.
[0029] Furthermore, the RPN network generates a set of candidate boxes at each layer. These candidate boxes correspond to target regions of different scales and locations on the original CT image, thus obtaining corresponding candidate feature maps on different feature layers.
[0030] Compared with the prior art, the above technical solution adopted in this invention has the following advantages: (1) It is applicable to the situation where "isolated" lesions of pulmonary nodules are detected by model using patient medical record images.
[0031] This invention uses patient CT images as input, extracts global visual features at different scales and local visual features from the model, and adds and fuses the two to obtain a feature pyramid, which has dual visual feature expression of local and global. It can detect lesions that are hidden and is suitable for capturing small-scale lesions that are not easy to find in CT images and exist in isolation.
[0032] (2) Application scenarios suitable for lung lesion detection and treatment
[0033] This invention enables routine monitoring of diagnosed lung lesions. After a period of treatment, lesions should be monitored, and this method can be used to analyze images, detect, and locate any new lung nodules or lesion changes. These lesions can be automatically identified and measured, including their size, location, and shape. Furthermore, it can be used to monitor treatment effectiveness and for regular lung CT scans to determine if recurrence has occurred.
[0034] (3) Applicable to large-scale lung cancer screening tasks
[0035] The speed and accuracy of the lung lesion detection task in this invention can meet the needs of lung cancer screening for a large population in a certain area, help to automatically screen a large number of lung CT scan images, improve screening efficiency, and ensure screening accuracy.
[0036] (4) Suitable for statistical scenarios involving data by medical researchers
[0037] This invention can be used to analyze statistical data of pulmonary nodules to study the pathogenesis, risk factors and prediction of lung cancer. Based on its own model structure design, the main function is to analyze pulmonary nodule case data quickly, efficiently and accurately. At the same time, this method can also be used in clinical trials to evaluate the effectiveness of new drugs or innovative treatments. Attached Figure Description
[0038] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0039] Figure 1 A diagram illustrating multi-party participation in federal learning;
[0040] Figure 2 A flowchart illustrating the training process structure of the Federated Learning Framework;
[0041] Figure 3 Here is a diagram of the CBAM (Convolutional Attention Module) structure.
[0042] Figure 4 This is a schematic diagram of prediction before (left) and after (right) FPN fusion;
[0043] Figure 5 A schematic diagram of the FL-Resnet-CBAM model;
[0044] Figure 6 Diagram of the improved ResNet50 network structure;
[0045] Figure 7 This is a schematic diagram of the ResNet50 fused FPN structure.
[0046] Figure 8 To visualize the experimental test results of the model;
[0047] Figure 9 This is a statistical graph showing the accuracy of the method during the experiment. Specific implementation methods
[0048] The principles of this disclosure will now be described with reference to several exemplary embodiments illustrated in the accompanying drawings. While preferred embodiments of this disclosure are shown in the drawings, it should be understood that these embodiments are described only to enable those skilled in the art to better understand and implement this disclosure, and are not intended to limit the scope of this disclosure in any way.
[0049] This embodiment provides a distributed small-scale lung nodule detection method based on federated learning. For example... Figure 4-8 As shown, the specific implementation steps are as follows:
[0050] Step 1: Obtain and preprocess the patient's actual lung CT images;
[0051] Specifically, medical image case datasets containing small-scale pulmonary nodule lesions were obtained from multiple medical institutions or data providers to ensure sufficient diversity in the dataset so that the model could generalize to different types of lesions. The collected case image data were preprocessed, including standardization, cropping, and scaling, before being input into the improved ResNet50 network.
[0052] Step 2: Input the preprocessed CT images into the small-scale lesion detection model for lung nodules for training. The training time is averaged by using multiple local clients.
[0053] Specifically, in the ResNet50 network, feature maps of different sizes are obtained through feature extraction at different scales. The focus is on extracting high-level features in the later stages of the network, resulting in smaller but deeper feature maps. These feature maps are used to detect small-scale lesions or complex textures in lesion images. Since a centralized global model is used, multiple clients communicate with the central server simultaneously. Considering that some clients may experience uneven data distribution or communication difficulties, causing their local models to fail to converge for extended periods and thus affecting global convergence, this invention introduces the local model's average training time, Tavg. In the s-th training round, the convergence time for the local client is Ts, and Tavg is initialized to a fixed value measured in the experiment during the first training round.
[0054] Step 3: Using the AdaGrad optimizer, the sparse gradient information is utilized in a discrete local client to achieve convergence of the small-scale lesion detection model for lung nodules.
[0055] Specifically, in federated learning frameworks, clients are discretely distributed. Traditional gradient descent algorithms have limited learning rate adjustment strategies and are prone to getting trapped in numerous local suboptimal solutions. Therefore, the AdaGrad (Adaptive Gradient) optimization algorithm was invented. This optimizer can adjust different learning rates according to different parameters, using smaller step sizes to update rapidly changing parameters and larger step sizes to update sparse parameters. The square root of the denominator is equivalent to automatically adjusting the learning rate (lr) and multiplying it with the current gradient. When the data distribution is sparse, it can better utilize sparse gradient information and converge more efficiently than general gradient descent algorithms.
[0056] Step 4: Input the converged result into the CBAM module to obtain a feature map containing spatial and channel attention features;
[0057] Specifically, the CBAM mechanism refers to the simultaneous comprehensive evaluation of both channel attention modules and spatial attention modules, exemplified by the Convolutional Block Attention Module (CBAM). Figure 3 .
[0058] Step 5: Input the feature map containing spatial and channel attention features into the FPN multi-scale prediction network to obtain multi-scale feature maps, and obtain a new feature pyramid based on the multi-scale feature maps;
[0059] Specifically, feature maps have multiple scales. Here, multi-scale feature maps are constructed, and predictions are made for each layer separately to improve accuracy. These predictions include the target's class probability and location information, replacing the original method of only predicting the last layer of the feature map. A Feature Pyramid Network (FPN) is used to fuse multi-layer network feature maps, achieving more detailed multi-scale predictions for small-scale targets. Based on the improved federated learning framework, the improved feature extraction network ResNet50 is used as the lung nodule detection model. Since the feature map output by the Conv5_x convolutional layer in ResNet50 is small in size, it is not conducive to accurate location detection of small targets. Multi-scale feature fusion is to construct a feature pyramid structure using the feature maps of four different scales (C2, C3, C4, and C5) generated by the convolutional residual modules of Conv2_x, Conv3_x, Conv4_x, and Conv5_x. The top-level features are fused with the low-level features through upsampling, and then the aliasing effect is eliminated for each fusion result by combining 3*3 convolutional kernels. The resulting new feature pyramid layers P2, P3, P4, and P5 are consistent with the convolution results of C2, C3, C4, and C5.
[0060] Step 6: Input the new feature pyramid into the RPN network. The RPN network generates candidate boxes, which are then fed into the RoIPooling layer to obtain four feature vectors of the same size. Finally, the feature vectors are fused into a new feature vector in the specified dimension and placed in a fully connected layer to complete the recognition task.
[0061] Specifically, the weights are scaled to between 0 and 1 using the sigmoid activation function. Embedding a CBAM module and introducing a residual structure in the final convolutional layer of stage 5 effectively suppresses gradient explosion and network degradation. The RPN network generates a set of candidate boxes at each layer, corresponding to target regions at different scales and locations in the original image. These candidate box regions obtain corresponding candidate feature maps on different feature layers, which are then fed into the corresponding RoI Pooling layers to obtain four feature vectors of the same size. These feature vectors capture target information at different levels and scales. Finally, the feature vectors are fused into a new feature vector in a specified dimension. This new vector contains information from different scales and locations.
[0062] The lung CT images that have been tested by the model are collected by the hospital and the doctors observe and analyze the parts marked by the candidate boxes of the model to determine whether the model identification is correct, the condition of the lung nodules, and whether there are smaller and more hidden "isolated" lung nodule lesions near the lung nodules marked by the candidate boxes.
[0063] In the experiments of the distributed small-scale target detection model based on federated learning proposed in this invention, some experimental visualization results for lung nodule lesion detection are as follows: Figure 8 As shown in the figure, the model marks the identified lesions with bounding boxes, and the lesion type and prediction accuracy are displayed in the upper left corner of the bounding box. Annotating the location and accuracy of the lung nodules detected by the model on the CT image with rectangles and text allows doctors to quickly identify the location and number of nodules, providing a more intuitive demonstration of the proposed model's detection performance.
[0064] In summary, this invention proposes a distributed training model for detecting small-scale lesions of lung nodules within a federated learning framework; by optimizing the federated learning framework and introducing the average training time of local models during the training process, the problem of global model non-convergence caused by the long-term non-convergence of local models can be prevented.
[0065] The AdaGrad optimizer is employed to leverage sparse gradient information under discrete clients, enabling more efficient model convergence. Faster R-CNN is improved by introducing a hybrid attention mechanism to help the network focus on more important features. FPN is integrated to achieve multi-scale prediction, further improving the accuracy of small-scale object detection. An improved ResNet feature extraction network with stronger representation capabilities is used, and the hybrid attention mechanism is incorporated into the ResNet network. Feature Pyramid Network (FPN) is used to fuse feature maps from multiple layers, achieving more detailed multi-scale predictions for small-scale objects.
[0066] Example 1: Detection of small-scale pulmonary nodules in medical institutions
[0067] This example is applied to the task of detecting and marking pulmonary nodules in lung CT images. It demonstrates enhanced detection capabilities for small-scale lesions and displays them with bounding boxes, thereby improving the detection rate. Small-scale lesions in lung nodule CT images are shown below. Figure 8 As shown.
[0068] Example 2: Medical Data Analysis Research Scenario
[0069] This example is applied to a professional medical data analysis institution. While speed is often a crucial factor in analyzing large volumes of lung CT images, accuracy is frequently considered the primary consideration for lesion detection. Compared to three other currently popular detection algorithms, this example demonstrates a significant improvement in accuracy in experimental results, ensuring reliable identification in practical applications. Accuracy comparison: Figure 9 As shown.
[0070] Example 3: Analysis of Medical Data Privacy Protection
[0071] This example can be applied to medical data analysis in specific settings (requiring data protection). When medical resources are limited and the number of patients is large, this example can lower the technical barrier for analysts of CT lesion images. Simultaneously, it ensures data privacy and enables local data analysis.
[0072] The above description is merely an optional embodiment of this disclosure and is not intended to limit this disclosure. Various modifications and variations can be made to this disclosure by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this disclosure should be included within the scope of protection of this disclosure.
[0073] Although the claims in this application have been formulated for specific combinations of features, it should be understood that the scope of this disclosure also includes any novel feature or any novel combination of features, whether express or implied or generalized herein, whether or not it relates to the same scheme in any of the claims currently claimed.
Claims
1. A distributed small-scale lung nodule detection method based on federated learning, characterized in that, include: Acquire and preprocess the patient's actual lung CT images; The preprocessed CT images were input into the small-scale lesion detection model for lung nodules for training. The training time was averaged by using multiple local clients. The AdaGrad optimizer was used to achieve convergence of the small-scale lesion detection model for lung nodules by utilizing sparse gradient information under discrete local clients. The converged result is input into the CBAM module to obtain a feature map containing spatial and channel attention features; The feature map containing spatial and channel attention features is input into the FPN multi-scale prediction network to obtain a multi-scale feature map, and a new feature pyramid is obtained based on the multi-scale feature map. The new feature pyramid is input into the RPN network, which generates candidate boxes. These boxes are then fed into the RoI Pooling layer to obtain four feature vectors of the same size. Finally, the feature vectors are fused into a new feature vector in a specified dimension and placed in a fully connected layer to complete the recognition task.
2. The distributed small-scale lung nodule detection method based on federated learning according to claim 1, characterized in that, The average training time of the model across multiple local clients during training is as follows: Where, Tavg is the average training time of the model in the first s rounds. This is the time taken for the local client model to converge during the s-th round of training.
3. The distributed small-scale lung nodule detection method based on federated learning according to claim 1, characterized in that, The AdaGrad optimizer is implemented as follows: In the formula Represents time t gradient; Represents the parameters at time t The sum of squared gradients, The learning rate is used for training the model.
4. The distributed small-scale lung nodule detection method based on federated learning according to claim 1, characterized in that, The CBAM module includes a channel attention module and a spatial attention module; The channel attention module performs global max pooling and global average pooling based on width and height on the input feature map F to obtain different spatial semantic descriptors. These operators are then fed into a shared two-layer neural network with ReLU activation. The output feature maps of the two shared neural networks are then summed and activated by sigmoid to generate the channel attention feature, Mc(F). Finally, Mc(F) is multiplied by the input feature map F to obtain the feature map F' containing channel attention, as shown in the following formula: Note: in the formula For the sigmoid function, W0 = W1= ; The spatial attention module takes the output feature map F' as input and performs global max pooling and global average pooling operations along the channel dimension to obtain two H×W×1 feature maps respectively. Then, a concat operation is performed based on the channel, followed by a convolution operation using a 7×7 convolution kernel to reduce the dimensionality to one channel. After passing through a sigmoid function, the spatial attention feature, Ms(F), is generated. Finally, Ms(F) is multiplied by the input feature map F' of the module to obtain the feature map F' containing channel attention and spatial attention. The formula is as follows: Note: in the formula f represents the sigmoid function. 7×7 This represents a convolution operation with a filter size of 7×7.
5. The distributed small-scale lung nodule detection method based on federated learning according to claim 1, characterized in that, The FPN multi-scale prediction network upsamples the extracted top-level features to obtain a feature map in a top-down manner, and then adds the bottom-level features and the upsampled feature map element by element to obtain a multi-scale feature map.
6. The distributed small-scale lung nodule detection method based on federated learning according to claim 1, characterized in that, The feature pyramid, combined with a 3*3 convolution kernel, eliminates aliasing for each fusion result, generating new feature pyramid layers P2, P3, P4, and P5, which are consistent with the convolution results of feature maps C2, C3, C4, and C5 at different scales.
7. The distributed small-scale lung nodule detection method based on federated learning according to claim 1, characterized in that, The RPN network generates a set of candidate boxes at each layer. These candidate boxes correspond to target regions of different scales and locations on the original CT image, thus obtaining corresponding candidate feature maps on different feature layers.