Lung image analysis method based on adaptive cascade deep learning network

Abstract

The invention discloses a lung image analysis method based on an adaptive cascade deep learning network, and relates to the technical field of medical images.The lung image analysis method comprises the steps that firstly, a medical image is preprocessed, including desensitization and format conversion; and constructing an automatic identification model and a segmentation model. Training constants are set through contrast adjustment, rotation and other enhanced data; a three-layer server system is built, a model is locally trained, parameters are encrypted and uploaded, an edge server detects transmission abnormity based on mahalanobis distance, and a central server aggregates the parameters through a federated average algorithm and issues and updates the parameters. During training, the recognition model is combined with a dual attention mechanism, and the segmentation model adopts a Dice loss function; positive and negative propagation speeds of a local server are calculated through test data, a comprehensive calculation capability index is generated, a learning rate is dynamically adjusted to adapt to equipment performance, and training efficiency is improved.

Description

Technical Field

[0001] The present invention relates to the field of medical imaging technology, and in particular to a lung image analysis method based on an adaptive cascade deep learning network. Background Art

[0002] In recent years, the application of computer-aided diagnosis (CAD) technology in the field of medical imaging has significantly reduced the burden on doctors and improved diagnostic accuracy. Deep learning, as a powerful data modeling method, can extract high-level semantic information from complex data and has shown great potential in medical image processing. Image classification and automatic segmentation are the two main application directions of deep learning in the medical field. The former can classify images as benign or malignant, while the latter can provide accurate positioning of diseased tissues. However, in existing technologies, the recognition and segmentation tasks of lung images are usually separated, lacking an integrated automated system, and there is still room for improvement in recognition accuracy and segmentation accuracy. In particular, the accuracy of the segmentation task directly affects the effectiveness of surgical resection and radiotherapy, and inaccurate positioning makes it difficult to meet clinical needs. Therefore, there is an urgent need for an efficient and accurate lung image analysis method to achieve automatic recognition and segmentation of tissues. Summary of the Invention

[0003] The purpose of the present invention is to provide a lung image analysis method based on an adaptive cascade deep learning network to solve the problems raised in the prior art.

[0004] To achieve the above object, the present invention provides the following technical solutions:

[0005] A lung image analysis method based on an adaptive cascade deep learning network comprises the following steps:

[0006] Step S1: pre-processing the medical images, wherein the pre-processing includes desensitization and data format conversion; the desensitization includes removing personal identification marks; the data format conversion includes classifying and arranging the medical image data set, and converting the original format medical images into the required format using a script;

[0007] Step S2: constructing an automatic recognition model and an automatic segmentation model; the automatic recognition model is used to recognize the medical image; the automatic segmentation model is used to delineate and segment the medical image;

[0008] Step S3, enhancing the image data of the medical image and setting constants for the training of the automatic recognition model and the automatic segmentation model; the constants include the size of the three-dimensional data block input to the model, the number of samples fed into the model in each iteration, and the total number of training rounds;

[0009] Step S4: Build a three-tier system of local servers, edge servers, and central servers; the local servers are used to train local models and encrypt uploaded parameters; the edge servers are used to detect transmission anomalies during data transmission; and the central server is used to aggregate the encrypted transmission parameters and then update the local model.

[0010] Step S5: training the automatic recognition model and the automatic segmentation model, and adjusting the learning rate of the model training according to the model training speed of the local server.

[0011] Furthermore, step S1 includes the following specific steps:

[0012] Step S1-1, removing personal identifiers from medical image data; classifying and arranging the medical image data according to body parts to obtain a medical image data set, wherein the medical image data is a medical image;

[0013] Step S1-2: for medical image datasets of different parts and the same part with the same diagnostic type, batch-processing scripts are used to convert the original DICOM format medical image datasets into PNG format;

[0014] Step S1-3: For medical imaging datasets of the same part but different diagnostic types, use a batch processing script to convert the DICOM format image data into NIFTI format, annotate and save it to generate a nii.gz image, and then use a batch processing script to convert it into png format.

[0015] Furthermore, step S2 includes the following specific steps:

[0016] A deep residual network is used as the basic architecture of the automatic recognition model, and the automatic recognition network is constructed through a dual attention network attention mechanism and pre-trained weights; the dual attention network attention mechanism includes spatial attention and channel attention;

[0017] An automatic segmentation model is constructed using a Unet-based encoder-decoder network architecture. The network architecture uses residual blocks and fully convolutional layers and consists of an encoder, a decoder, and a bridge connection. The bridge connection consists of a residual block and a receptive field block, which is a module in the model used to simulate the receptive field characteristics.

[0018] The K-fold cross-validation method is used to divide the medical imaging dataset of the automatic recognition model into training set and test set; the medical imaging dataset of the automatic segmentation model is divided by setting random seeds.

[0019] Furthermore, step S3 includes the following specific steps:

[0020] The medical image data is enhanced by contrast adjustment, rotation, flipping, scaling and noise addition methods, and the enhanced medical image data is input into the automatic recognition model and the automatic segmentation model; and constants are set during model training.

[0021] Furthermore, step S4 includes the following specific steps:

[0022] Build a three-tier system of local servers, edge servers, and central servers; and complete the data interaction process between local servers, edge servers, and central servers;

[0023] The data interaction process includes:

[0024] The local server trains the automatic recognition model and the automatic segmentation model locally, encrypts the model parameters using an encryption algorithm after the training is completed, and forms an encrypted parameter data packet; and sends the encrypted parameter data packet to the central server;

[0025] The edge server collects the encrypted parameter data packets uploaded by the local server in real time, performs anomaly detection on the encrypted parameter data packets based on a preset anomaly detection model, and blocks the communication link between the local server and the central server when an anomaly is detected until the anomaly is resolved;

[0026] The central server performs a model aggregation operation on the received encrypted parameter data packet in a ciphertext state according to homomorphic encryption technology; the model aggregation operation includes using a federated averaging algorithm to perform weighted averaging on the model parameters after encryption processing on each local server to generate globally updated model parameters; and the encrypted parameter data packet after the model aggregation operation is sent to each local server for model update.

[0027] The anomaly detection method includes: after receiving the encrypted parameter data packets uploaded by each local server, the edge server first calculates the statistical characteristics of the global gradient distribution in the encrypted parameter data packets, including the mean vector, covariance matrix and standard deviation; and calculates the degree of deviation between the gradient vector of each local server and the global mean vector based on the Mahalanobis distance. When the Mahalanobis distance exceeds a preset critical value, it is determined to be an anomaly.

[0028] Methods for adjusting the learning rate for model training include:

[0029] S9-1. Input test data into the local server before model training, where the test data is a random tensor of fixed size; calculate the time it takes for the test data to be forward propagated and backward propagated in the local server, where forward propagation is a unidirectional flow of data from the input layer to the output layer of the neural network, and backward propagation is a unidirectional flow of data from the output layer to the input layer of the neural network; perform N forward propagations and N backward propagations, and record the average time of forward propagation and backward propagation respectively. The result of dividing the amount of forward propagation and backward propagation calculations by their average time is used as the forward propagation speed and the backward propagation speed;

[0030] S9-2. Preset the computing speed of the reference device and calculate the comprehensive computing capability index of the local server based on the forward propagation speed and the reverse propagation speed:

[0031]

[0032] Among them S ref is the calculation speed of the preset reference device, S f is the forward propagation speed, S b is the back propagation speed, C is the comprehensive computing power index of the local server; where S ref Used to normalize the comprehensive computing capability index of the local server;

[0033] S9-3. Dynamically adjust the learning rate of the local server based on the weighted average comprehensive computing power of all local servers participating in the training:

[0034]

[0035] Where η is the learning rate of the local server, η0 is the initial learning rate, λ is the sensitivity coefficient, and C is the comprehensive computing capability index of the local server. It is the weighted average comprehensive computing power of all local servers participating in the training.

[0036] Compared with the prior art, the present invention has the following beneficial effects:

[0037] 1. This invention improves the quality and diversity of input data by desensitizing, converting data formats, and enhancing images of medical images. At the same time, it uses a deep residual network, a dual attention network attention mechanism, and a Unet-based encoder-decoder network architecture to build an automatic recognition and segmentation model, effectively improving the recognition accuracy and segmentation precision of lung medical images.

[0038] 2. The present invention realizes distributed model training while ensuring data privacy and security by building a three-layer federated framework of local servers, edge servers and central servers. The anomaly detection mechanism of the edge server ensures the reliability of data transmission, and the model aggregation operation of the central server improves the generalization ability and training efficiency of the model.

[0039] 3. The present invention optimizes training parameters in real time according to the computing power of the local server through a mechanism of dynamically adjusting the learning rate, thereby solving the collaborative efficiency problem caused by differences in training speeds of different devices and further improving the overall performance and convergence speed of model training. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] Figure 1 It is a schematic diagram of the process of the present invention.

[0041] Figure 2 The technical route for data preprocessing.

[0042] Figure 3 Confusion matrix for models using image-level and subject-level allocators.

[0043] Figure 4 Schematic diagram of the evaluation index of the present invention. DETAILED DESCRIPTION

[0044] The present invention will be further described below with reference to the accompanying drawings.

[0045] like Figures 1 to 4 As shown, the present invention provides a technical solution, a lung image analysis method based on an adaptive cascade deep learning network, and the lung image analysis method includes the following steps:

[0046] Step S1: pre-processing the medical images, wherein the pre-processing includes desensitization and data format conversion; the desensitization includes removing personal identification marks; the data format conversion includes classifying and arranging the medical image data set, and converting the original format medical images into the required format using a script;

[0047] Step S2: constructing an automatic recognition model and an automatic segmentation model; the automatic recognition model is used to recognize the medical image; the automatic segmentation model is used to delineate and segment the medical image;

[0048] Step S3, enhancing the image data of the medical image and setting constants for the training of the automatic recognition model and the automatic segmentation model; the constants include the size of the three-dimensional data block input to the model, the number of samples fed into the model in each iteration, and the total number of training rounds;

[0049] Step S4: Build a three-tier system of local servers, edge servers, and central servers; the local servers are used to train local models and encrypt uploaded parameters; the edge servers are used to detect transmission anomalies during data transmission; and the central server is used to aggregate the encrypted transmission parameters and then update the local model.

[0050] Step S5: training the automatic recognition model and the automatic segmentation model, and adjusting the learning rate of the model training according to the model training speed of the local server.

[0051] Furthermore, step S1 includes the following specific steps:

[0052] Step S1-1, removing personal identifiers from medical image data; classifying and arranging the medical image data according to body parts to obtain a medical image data set, wherein the medical image data is a medical image;

[0053] Step S1-2: for medical image datasets of different parts and the same part with the same diagnostic type, batch-processing scripts are used to convert the original DICOM format medical image datasets into PNG format;

[0054] Step S1-3: For medical imaging datasets of the same part but different diagnostic types, use a batch processing script to convert the DICOM format image data into NIFTI format, annotate and save it to generate a nii.gz image, and then use a batch processing script to convert it into png format.

[0055] Furthermore, step S2 includes the following specific steps:

[0056] A deep residual network is used as the basic architecture of the automatic recognition model, and the automatic recognition network is constructed through a dual attention network attention mechanism and pre-trained weights; the dual attention network attention mechanism includes spatial attention and channel attention;

[0057] An automatic segmentation model is constructed using a Unet-based encoder-decoder network architecture. The network architecture uses residual blocks and fully convolutional layers and consists of an encoder, a decoder, and a bridge connection. The bridge connection consists of a residual block and a receptive field block, which is a module in the model used to simulate the receptive field characteristics.

[0058] The K-fold cross-validation method is used to divide the medical imaging dataset of the automatic recognition model into training set and test set; the medical imaging dataset of the automatic segmentation model is divided by setting random seeds.

[0059] Furthermore, step S3 includes the following specific steps:

[0060] The medical image data is enhanced by contrast adjustment, rotation, flipping, scaling and noise addition methods, and the enhanced medical image data is input into the automatic recognition model and the automatic segmentation model; and constants are set during model training.

[0061] Furthermore, step S4 includes the following specific steps:

[0062] Build a three-tier system of local servers, edge servers, and central servers; and complete the data interaction process between local servers, edge servers, and central servers;

[0063] The data interaction process includes:

[0064] The local server trains the automatic recognition model and the automatic segmentation model locally, encrypts the model parameters using an encryption algorithm after the training is completed, and forms an encrypted parameter data packet; and sends the encrypted parameter data packet to the central server;

[0065] The edge server collects the encrypted parameter data packets uploaded by the local server in real time, performs anomaly detection on the encrypted parameter data packets based on a preset anomaly detection model, and blocks the communication link between the local server and the central server when an anomaly is detected until the anomaly is resolved;

[0066] The central server performs a model aggregation operation on the received encrypted parameter data packet in a ciphertext state according to homomorphic encryption technology; the model aggregation operation includes using a federated averaging algorithm to perform weighted averaging on the model parameters after encryption processing on each local server to generate globally updated model parameters; and the encrypted parameter data packet after the model aggregation operation is sent to each local server for model update.

[0067] The anomaly detection method includes: after receiving the encrypted parameter data packets uploaded by each local server, the edge server first calculates the statistical characteristics of the global gradient distribution in the encrypted parameter data packets, including the mean vector, covariance matrix and standard deviation; and calculates the degree of deviation between the gradient vector of each local server and the global mean vector based on the Mahalanobis distance. When the Mahalanobis distance exceeds a preset critical value, it is determined to be an anomaly.

[0068] Methods for adjusting the learning rate for model training include:

[0069] S9-1. Input test data into the local server before model training, where the test data is a random tensor of fixed size; calculate the time it takes for the test data to be forward propagated and backward propagated in the local server, where forward propagation is a unidirectional flow of data from the input layer to the output layer of the neural network, and backward propagation is a unidirectional flow of data from the output layer to the input layer of the neural network; perform N forward propagations and N backward propagations, and record the average time of forward propagation and backward propagation respectively. The result of dividing the amount of forward propagation and backward propagation calculations by their average time is used as the forward propagation speed and the backward propagation speed;

[0070] S9-2. Preset the computing speed of the reference device and calculate the comprehensive computing capability index of the local server based on the forward propagation speed and the reverse propagation speed:

[0071]

[0072] Among them S ref is the calculation speed of the preset reference device, S f is the forward propagation speed, S b is the back propagation speed, C is the comprehensive computing power index of the local server; where S ref Used to normalize the comprehensive computing capability index of the local server;

[0073] S9-3. Dynamically adjust the learning rate of the local server based on the weighted average comprehensive computing power of all local servers participating in the training:

[0074]

[0075] Where η is the learning rate of the local server, η0 is the initial learning rate, λ is the sensitivity coefficient, and C is the comprehensive computing capability index of the local server. It is the weighted average comprehensive computing power of all local servers participating in the training.

[0076] System performance verification:

[0077] In this example, the network model is evaluated based on its output results. Multiple evaluation metrics and learning curves are used to assess and test its robustness and generalization capabilities. In the classification model's predictions, the confusion matrix is used to evaluate the model's ability to distinguish images, marking abnormalities as 0 and normal people as 1. Figure 3 The first row uses the image-level allocator, and the second row uses the subject-level allocator. By comparing the confusion matrices in the figure, we can find that the classification probability of the image-level allocator is higher than that of the subject-level allocator.

[0078] like Figure 4As shown in the figure, the prediction results of the automatic segmentation network model show that the prediction accuracy of the training and validation sets is above 90%, with the numerical value ranging from 0 to 1. The closer the value is to 1, the higher the prediction accuracy and the better the performance of the model. The prediction loss of the training and validation sets is less than 10%. The closer the value is to 0, the smaller the loss and the better the performance of the model.

[0079] Federal framework construction:

[0080] For example:

[0081] Local server (3 hospitals):

[0082] Hospital A: 5,000 training data cases, GPU model RTX 3090;

[0083] Hospital B: 3,000 training data cases, GPU model RTX 2080;

[0084] Hospital C: Training data volume: 2000 cases, CPU training (no independent GPU).

[0085] Edge server: Deploy an anomaly detection model, monitor parameter transmission in real time, and preset the Mahalanobis distance critical value D = 2.5.

[0086] Central server: Aggregate the global model, use the federated averaging algorithm, and distribute weights based on data volume to hospitals A:B:C = 5:3:2.

[0087] Normal process:

[0088] After training, Hospital A uploads encrypted parameters, such as the weight matrix of the last layer of the ResNet-50 recognition model and the encryption algorithm RSA.

[0089] The edge server calculates the global gradient mean vector μ = [0.3, 0.4, 0.2], the covariance matrix Σ, the gradient vector g of hospital A = [0.35, 0.42, 0.21], and the Mahalanobis distance D = 1.2 < 2.5. It is determined to be normal and transmission is allowed.

[0090] Abnormal scenario:

[0091] Hospital C has a slow CPU training speed, an abnormal gradient vector, g = [0.1, 0.6, 0.8], and a Mahalanobis distance D = 3.0 > 2.5. The edge server blocks communication and triggers the retry mechanism.

[0092] When dynamically adjusting the learning rate of a local server, pre-enter a fixed-size random tensor as test data, such as a 1024x1024 floating-point matrix, to simulate batch training data. N is set to 10 times for forward propagation and backward propagation. At this time:

[0093] S f =1.5×10 12 FLOP / S

[0094] S b =1.3×10 12 FLOP / S

[0095] Pick

[0096] S ref =10 12 FLOP / S

[0097] but

[0098] C=1.4

[0099] like

[0100]

[0101] λ=0.5

[0102] The learning rate is adjusted to

[0103]

[0104] That is, the local server's performance is higher than the average 40% of devices, and the learning rate is increased by 20%, balancing its faster computing speed and avoiding gradient update lag.

[0105] Finally, it should be noted that the above descriptions are merely preferred embodiments of the present invention and are not intended to limit the present invention. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art will be able to modify the technical solutions described in the aforementioned embodiments or substitute equivalents for some of the technical features. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention shall be included within the scope of protection of the present invention.

Claims

1. A lung image analysis method based on an adaptive cascade deep learning network, characterized by: The method comprises the following steps: Step S1: pre-processing the medical images, wherein the pre-processing includes desensitization and data format conversion; the desensitization includes removing personal identification marks; the data format conversion includes classifying and arranging the medical image data set, and converting the original format medical images into the required format using a script; Step S2: constructing an automatic recognition model and an automatic segmentation model; the automatic recognition model is used to recognize the medical image; the automatic segmentation model is used to delineate and segment the medical image; Step S3, enhancing the image data of the medical image and setting constants for the training of the automatic recognition model and the automatic segmentation model; the constants include the size of the three-dimensional data block input to the model, the number of samples fed into the model in each iteration, and the total number of training rounds; Step S4: Build a three-tier system of local servers, edge servers, and central servers; the local servers are used to train local models and encrypt uploaded parameters; the edge servers are used to detect transmission anomalies during data transmission; and the central server is used to aggregate the encrypted transmission parameters and then update the local model. Step S5: training the automatic recognition model and the automatic segmentation model, and adjusting the learning rate of the model training according to the model training speed of the local server.

2. The lung image analysis method based on the adaptive cascade deep learning network according to claim 1, characterized in that: The step S1 comprises: Step S1-1, removing personal identifiers from medical image data; classifying and arranging the medical image data according to body parts to obtain a medical image data set, wherein the medical image data is a medical image; Step S1-2: for medical image datasets of different parts and the same part with the same diagnostic type, batch-processing scripts are used to convert the original DICOM format medical image datasets into PNG format; Step S1-3: For medical imaging datasets of the same part but different diagnostic types, use a batch processing script to convert the DICOM format image data into NIFTI format, annotate and save it to generate a nii.gz image, and then use a batch processing script to convert it into png format.

3. The lung image analysis method based on the adaptive cascade deep learning network according to claim 1, characterized in that: The step S2 further includes: A deep residual network is used as the basic architecture of the automatic recognition model, and the automatic recognition network is constructed through a dual attention network attention mechanism and pre-trained weights; the dual attention network attention mechanism includes spatial attention and channel attention; An automatic segmentation model is constructed using a Unet-based encoder-decoder network architecture. The network architecture uses residual blocks and fully convolutional layers and consists of an encoder, a decoder, and a bridge connection. The bridge connection consists of a residual block and a receptive field block, which is a module in the model used to simulate the receptive field characteristics.

4. The lung image analysis method based on the adaptive cascade deep learning network according to claim 3, characterized in that: The K-fold cross-validation method is used to divide the medical imaging dataset of the automatic recognition model into training set and test set; the medical imaging dataset of the automatic segmentation model is divided by setting random seeds.

5. The lung image analysis method based on the adaptive cascade deep learning network according to claim 1, characterized in that: The step S3 further comprises: The medical image data is enhanced by contrast adjustment, rotation, flipping, scaling and noise addition methods, and the enhanced medical image data is input into the automatic recognition model and the automatic segmentation model; and constants are set during model training.

6. The lung image analysis method based on an adaptive cascade deep learning network according to claim 1, characterized in that: The step S4 further includes: Build a three-tier system of local servers, edge servers, and central servers; and complete the data interaction process between local servers, edge servers, and central servers; The data interaction process includes: The local server trains the automatic recognition model and the automatic segmentation model locally, encrypts the model parameters using an encryption algorithm after the training is completed, and forms an encrypted parameter data packet; and sends the encrypted parameter data packet to the central server; The edge server collects the encrypted parameter data packets uploaded by the local server in real time, performs anomaly detection on the encrypted parameter data packets based on a preset anomaly detection model, and blocks the communication link between the local server and the central server when an anomaly is detected until the anomaly is resolved; The central server performs a model aggregation operation on the received encrypted parameter data packet in a ciphertext state according to homomorphic encryption technology; the model aggregation operation includes using a federated averaging algorithm to perform weighted averaging on the model parameters after encryption processing on each local server to generate globally updated model parameters; and the encrypted parameter data packet after the model aggregation operation is sent to each local server for model update.

7. The lung image analysis method based on an adaptive cascade deep learning network according to claim 6, characterized in that: The anomaly detection method includes: after receiving the encrypted parameter data packets uploaded by each local server, the edge server first calculates the statistical characteristics of the global gradient distribution in the encrypted parameter data packets, including the mean vector, covariance matrix and standard deviation; and calculates the degree of deviation between the gradient vector of each local server and the global mean vector based on the Mahalanobis distance. When the Mahalanobis distance exceeds a preset critical value, it is determined to be an anomaly.

8. The lung image analysis method based on an adaptive cascade deep learning network according to claim 1, characterized in that: The step S5 further includes: The automatic recognition model and the automatic segmentation model are trained; when training the automatic recognition model, image-level and subject-level dataset allocators are used, and the automatic recognition model is trained in combination with a dual attention network attention mechanism; when training the automatic segmentation model, a Dice coefficient loss function is adopted; and the learning rate of the model training is adjusted according to the model training speed of the local server.

9. The lung image analysis method based on the adaptive cascade deep learning network according to claim 8, characterized in that: Methods for adjusting the learning rate for model training include: S9-1. Input test data into the local server before model training, where the test data is a random tensor of fixed size; calculate the time it takes for the test data to be forward propagated and backward propagated in the local server, where forward propagation is a unidirectional flow of data from the input layer to the output layer of the neural network, and backward propagation is a unidirectional flow of data from the output layer to the input layer of the neural network; perform N forward propagations and N backward propagations, and record the average time of forward propagation and backward propagation respectively. The result of dividing the amount of forward propagation and backward propagation calculations by their average time is used as the forward propagation speed and the backward propagation speed; S9-2. Preset the computing speed of the reference device and calculate the comprehensive computing capability index of the local server based on the forward propagation speed and the reverse propagation speed: Among them S ref is the calculation speed of the preset reference device, S f is the forward propagation speed, S b is the back propagation speed, C is the comprehensive computing power index of the local server; where S ref Used to normalize the comprehensive computing capability index of the local server; S9-3. Dynamically adjust the learning rate of the local server based on the weighted average comprehensive computing power of all local servers participating in the training: Where η is the learning rate of the local server, η0 is the initial learning rate, λ is the sensitivity coefficient, and C is the comprehensive computing capability index of the local server. It is the weighted average comprehensive computing power of all local servers participating in the training.

Images