A Method and System for Protecting the Privacy of Medical Image Data in the Federated Learning Mode

Abstract

The present invention discloses a method and system for protecting the privacy of medical image data in a federated learning mode, including the following steps: Each client uses the gradient descent method to train the original federated learning model based on local medical image data as the local model, calculates the training gradient information of the local model and the statistics of the local medical image data, and uploads the statistics of the local medical image data and the training gradient information of the local model to the server side; The server side receives the statistics and information uploaded by each client to generate a perturbation model; Each client receives the perturbation model and updates the local model parameters; The attacker trains a surrogate model based on the generalization error-guided adversarial optimization mechanism, and the server side updates the generator parameters according to the attack situation and defense feedback of the surrogate model to optimize the defense; After the attacker trains the surrogate model in multiple rounds and the server side optimizes the defense, evaluate the defense effect of the federated learning model against the attack of the surrogate model. If the expected effect is not achieved, continue training.

Description

Technical Field

[0001] The present invention is applicable to the field of artificial intelligence system security, and specifically relates to a method and system for protecting the privacy of medical image data in the federated learning mode. Background Art

[0002] In the field of medical images, the federated learning technology has been booming. Its core goal is to achieve local data storage while jointly optimizing the model through multi-party collaboration, so as to improve the efficiency of medical image analysis. This technical mode can theoretically integrate the data resources of all parties, provide strong support for the accurate interpretation of medical images, and thus push the medical diagnosis technology to a new level.

[0003] However, the existing technologies in this field currently expose extremely significant shortcomings in data privacy protection. Many medical image algorithms based on federated learning ignore a key link - the data privacy security protection in the federated learning process - while pursuing local data preservation and improving the efficiency of joint optimization modeling.

[0004] Existing federated learning methods (such as Chinese Patent Document CN202310970584.4) initialize the global model parameters by the server first and send these parameters to the medical client. After receiving the parameters, the client uses the private dataset stored locally to train the local model. After the training is completed, the local model parameters are sent back to the server, and the server prepares for the next round of training by adaptively aggregating these parameters. This method has achieved something in solving the class imbalance problem, but it completely ignores the consideration of data privacy protection in the federated learning process. In the actual application scenario, this means that malicious attackers are very likely to take advantage of the opportunity of information transmission between the server and the client by the model to steal the original medical image data, posing a serious threat to patient privacy. Another example is that Chinese Patent Document CN202011393242.3 discloses a CT image detection method and related device based on federated learning, which focuses on the optimization of the CT image detection process. The first device trains the model based on its own data, then sends the parameters to the second device, and receives the relevant average values to update its own parameters, and then retrains the model to mark the abnormal areas in the CT image. Although this method is carefully designed in the CT image detection process, it does not mention anything about preventing data privacy leakage in the federated learning process. This undoubtedly leaves potential security risks in key links such as model parameter transmission, making it possible for malicious attackers to take advantage of it, which may lead to data privacy leakage incidents. Chinese Patent Document CN202211331593.0 discloses an online cervical cell TCT slice detection method and system based on federated learning. The implementation method is that the cloud server initializes the global auxiliary diagnosis model, and the local TCT reader initializes the local model and receives the global model as an intermediate model. After the local reader discriminates the image, it is reviewed by a doctor. When the federated learning mechanism is not triggered, the model is retrained using local private data. When the federated learning conditions are met, the local uploads the gradient parameters of the intermediate model to the cloud server. After the cloud server completes the aggregation update, it then sends down the new global model to update the local intermediate model. While this method improves the online cervical cell TCT slice detection process, it seriously ignores the importance of data privacy protection. In the current complex network environment, malicious attackers are completely likely to use the model stealing technology based on the generative adversarial network architecture to reverse recover the original medical image data of the client through the model information transmitted by the system, leading to extremely serious data privacy leakage risks.

[0005] In summary, for existing medical image algorithms based on federated learning, although many efforts have been made in aspects such as local data storage and improvement of joint optimization modeling efficiency, during the federated learning process, there is generally a lack of security protection against data privacy leakage. Potential malicious attackers can use model stealing techniques to easily recover the original medical image data by means of the model information transmitted by the system. This not only seriously threatens the privacy security of patients but also hinders the further popularization and application of federated learning technology in the field of medical images. Summary of the Invention

[0006] The present invention aims to propose a method and system for protecting the privacy of medical image data in a federated learning mode, including a complex mask generation mechanism, a dynamic defense mechanism, and an adversarial optimization mechanism, etc., for the security protection of model training in a distributed artificial intelligence system, effectively coping with the security threats in artificial intelligence model training faced by the distributed artificial intelligence system, solving the deficiencies of existing defense technologies, and providing practical and effective protection for the privacy security of medical image data during the federated learning process.

[0007] The object of the present invention is achieved by at least one of the following technical solutions.

[0008] A method for protecting the privacy of medical image data in a federated learning mode includes the following steps:

[0009] S1. The server initializes a federated learning model for disease prediction. Each client trains the original federated learning model as a local model based on local medical image data using the gradient descent method, calculates the local model training gradient information and the statistics of the local medical image data, and uploads the statistics of the local medical image data and the local model training gradient information to the server.

[0010] S2. The server receives the statistics of the local medical image data and the local model training gradient information uploaded by each client to generate a perturbation model, and distributes the perturbation model to the client.

[0011] S3. Each client receives the perturbation model and updates the local model parameters.

[0012] S4. The attacker trains a surrogate model based on the adversarial optimization mechanism guided by the generalization error. The server updates the global generator parameters using the alternating direction method of multipliers and the proximal policy optimization algorithm to optimize the defense. After the attacker trains the surrogate model for multiple rounds and the server optimizes the defense, the defense effect of the federated learning model against the surrogate model attack is evaluated. If the expected effect is not achieved, continue training.

[0013] Further, in step S2, the server receives the statistics of the local medical image data uploaded by each client and the local model training gradient information to generate a perturbation model, and sends the perturbation model to the client, including the following steps:

[0014] S21. Initially aggregate the gradient information using the federated averaging algorithm to obtain the global gradient trend information, calculate the Wasserstein distance between clients based on the global gradient trend information, and use the K-means clustering algorithm to divide the clients;

[0015] S22. Assign an independent generator to each client, initialize it as a copy of the global generator, incorporate the global gradient trend information into the copy of the global generator, adjust the client generator to generate a perturbation mask through the federated averaging algorithm, and apply a two-dimensional discrete cosine transform to the generated perturbation mask to convert it to the frequency domain;

[0016] S23. Calculate the client adaptation score using a multi-layer perceptron and fuse the perturbation mask using the attention mechanism;

[0017] S24. Aggregate the client generator parameters using the federated averaging algorithm to obtain the global generator, combine the global generator with the original federated learning model, generate a perturbation model based on the global gradient trend information and the fused perturbation mask, and send the perturbation model to the client.

[0018] Further, in step S23, calculating the client adaptation score using a multi-layer perceptron specifically is: for each client, first calculate its adaptation score. Calculating its adaptation score specifically is to combine the mean and covariance matrix of the local medical image data of the client into a vector, and then input the combined vector into the multi-layer perceptron for calculation, and finally output the client adaptation score;

[0019] Fusing the perturbation mask using the attention mechanism is as follows: first divide the adaptation score of the client by the correlation coefficient, then input the obtained result into the normalization exponential function for processing. The processing process of the normalization exponential function will make the sum of the values corresponding to all clients equal to 1 after normalization, that is, weight each client. The client with a higher adaptation score will have a greater weight. Then, multiply the value obtained after processing by the normalization exponential function by the output value of the client generator. Finally, add up the results of multiplying the values obtained after processing by the normalization exponential function by the output value of the client generator for all clients, and the sum obtained is the finally fused perturbation mask; the output value of the client generator is the value obtained by inputting the medical image data sample into the client generator.

[0020] Further, the generalization error-guided adversarial optimization mechanism includes the optimization of the outer objective and the inner objective;

[0021] The outer layer aims to maximize the generalization error of the attack model. The surrogate model takes the output of the perturbation model as input to reconstruct the original medical image data to generate an image close to the original medical image data. By generalizing the error of the outer layer's maximization attack model on the attack data distribution, the impact on the federated learning model is minimized, making it difficult for attackers to accurately reconstruct the data.

[0022] The inner layer aims to ensure that while defending against attacks, the performance of the federated learning model is not overly reduced by minimizing the impact on the federated learning model.

[0023] Furthermore, the surrogate model reconstructing the original medical image data with the output of the perturbation model as input includes: The surrogate model is a surrogate model similar to the original federated learning model structure trained with a generative adversarial network; the training process of the surrogate model is to continuously adjust the parameters of the surrogate model through the backpropagation algorithm using the data output by the perturbation model, enabling the surrogate model to mimic the behavior of the federated learning model.

[0024] Furthermore, in step S4, the server side uses the alternating direction method of multipliers to decouple the optimization of the inner layer objective. In each iteration, first fix the server global generator parameters, update the attack model, then fix the updated attack model, and then update the server global generator parameters through the proximal policy optimization algorithm gradient.

[0025] Furthermore, update the attack model The formula is:

[0026] ;

[0027] where is the attack model after the th update, is the attack model to be optimized, represents the expectation of the input medical image data sample , is the cross-entropy loss function, is the output of the perturbation model, is the output of the federated learning model, represents the cross-entropy loss function.

[0028] Furthermore, updating the server global generator parameters through the proximal policy optimization algorithm gradient is: The server-side generator parameters after the th update are equal to the The server - side generator parameters after the next update are added with an adjustment amount, where the adjustment amount is the product of the learning rate and a gradient. The learning rate controls the step - size of each update, and the gradient is the attack loss function minus the product of the trade - off factor and the total - variation regularization term. The attack loss function measures the loss of the attack model during the attack process; the trade - off factor is used to balance the importance of the attack loss function and the total - variation regularization term; the total - variation regularization term is obtained based on the perturbation mask, and the perturbation mask is used to perturb the server - side generator parameters to protect data privacy.

[0029] A system for implementing the method for protecting medical image data privacy in a federated learning mode as described above, includes:

[0030] A client - side data processing and interaction module, which trains the original federated learning model through stochastic gradient descent, calculates the gradient of the federated learning model and the statistics of the mean and covariance matrix of the local medical image data; uploads the corresponding data to the server - side through a network interface and receives the perturbation model sent by the server - side.

[0031] A server - side gradient aggregation and client - side partitioning module: uses the federated averaging algorithm to aggregate the client - side gradients to obtain the global gradient trend, calculates the Wasserstein distance between clients, and performs K - means clustering to partition the clients.

[0032] A server - side mask generation and model distribution module: allocates and adjusts the independent generator parameters for the clients through the federated averaging algorithm, generates an adapted perturbation mask, fuses the mask through an attention mechanism to obtain a global generator, combines the global generator with the original federated learning model to generate a perturbation model and distributes it to the clients; uses the alternating direction multiplier method and the proximal policy optimization algorithm to calculate the gradient and dynamically update the generator parameters to improve the defense effect, and at the same time stores the data uploaded by the clients.

[0033] A computer device of the present invention includes: a memory, a processor, and a computer program stored on the memory. When the computer program is executed on the processor, it implements the method for protecting medical image data privacy in a federated learning mode as described above.

[0034] Compared with the existing technologies, the beneficial effects of the present invention are:

[0035] Through mechanisms such as latent space dynamic obfuscation, generalization error - oriented adversarial optimization, and federated clustering - driven adaptive defense, the method of the present invention, with the same privacy budget, makes the evaluation quality of the images reconstructed by the attacker poor, effectively disrupts the data reconstruction process of the attacker, and greatly enhances the protection ability for the federated learning model. Ensure model performance: The decline in the test accuracy of the federated global model is low. While defending against attacks, the performance of the model is maintained to the greatest extent, ensuring the effectiveness and reliability of federated learning in practical applications. The dynamic defense has significant advantages: The dynamic defense mechanism shows significant advantages in continuous confrontation, fully proving that the algorithm can adapt to changes in the attacker's strategy in real - time and continuously maintain high - efficiency defense capabilities. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] Figure 1 It is a flowchart of a method and system for protecting the privacy of medical image data in a federated learning mode in an embodiment of the present invention.

[0037] Figure 2 It is a flowchart of the latent space dynamic obfuscation mechanism in an embodiment of the present invention.

[0038] Figure 3 It is a flowchart of the generalization error - oriented adversarial optimization mechanism in an embodiment of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0039] To make the objectives, technical solutions, and advantages of the present invention clearer and more understandable, the following examples are given in conjunction with the accompanying drawings to detail the specific implementation of the present invention.

[0040] In this embodiment, taking multiple medical institutions as an example, each medical institution has a large amount of medical imaging data of patients. These data contain patients' privacy information and thus cannot be directly shared. At the same time, there are malicious attackers attempting to steal the original medical imaging data through the output of the federated learning model. To prevent theft, a federated learning network composed of multiple medical institutions aims to jointly train a deep - learning model for disease prediction and resist various adversarial attacks.

[0041] As Figures 1 to 3 shown, a method for protecting the privacy of medical image data in a federated learning mode in this embodiment includes the following steps:

[0042] S1. Initialize the parameters of the federated learning model, global generator, and discriminator for disease prediction, and set various algorithm hyperparameters such as the perturbation intensity threshold, trade - off factor, learning rate, and correlation coefficient; each client trains the original federated learning model as a local model based on local medical imaging data using stochastic gradient descent, calculates the statistical quantities of the training gradient information of the local model and the mean and covariance matrix of the local medical imaging data, and uploads the statistical quantities of the local medical imaging data and the training gradient information of the local model to the server - side.

[0043] Each client uses the local medical image data to train the original federated learning model using stochastic gradient descent, calculates the gradients, and is used for medical image feature extraction and classification diagnosis. At the same time, each client of each medical institution uploads the statistics of the local medical image data to the server side, denotes the mean of the local medical image data of the -th client, reflecting the central tendency of the data; denotes the covariance matrix of the local medical image data of the -th client, describing the correlation and dispersion degree between the dimensions of the data.

[0044] As an embodiment, this embodiment selects the convolutional neural network structure VGG16 as the basic model to initialize the original federated learning model , which is used for feature extraction and classification prediction of medical images. The original federated learning model includes an input layer, a hidden layer, and an output layer. The input layer of the original federated learning model is adapted to the medical image format, and the hidden layer extracts and transforms features through convolution, pooling, and fully connected operations. The output layer outputs the prediction result according to the disease prediction task. The original federated learning model is:

[0045] ;

[0046] wherein, is the original federated learning model with parameters , is the set of input medical image data, and these image data cover multiple modalities, such as X-ray, CT, MRI, etc.; is the set of prediction results output by the original federated learning model, such as disease category, disease severity score.

[0047] Initialize the global generator and the discriminator , set the initial generator parameters and the initial discriminator parameters . At the same time, set the algorithm hyperparameters, such as the perturbation intensity threshold , the trade-off factor , the learning rate , the correlation coefficient , etc.

[0048] The clients on each medical institution perform federated learning model training based on the local medical image data, use the automatic differentiation technology to calculate the gradients of the model (such as in PyTorch or TensorFlow, by constructing a computational graph, automatically tracking the operations during the calculation process, and automatically calculating the gradients during backpropagation), and combine the gradient descent method to calculate the gradients of the input images to obtain the gradient matrix , is the input medical image data sample; represents the gradient operator for the input medical image data sample ; is the loss function, which is used to measure the difference between the prediction result of the federated learning model and the true label . Common loss functions such as the cross - entropy loss function; is the federated learning model 's prediction output for the input medical image data sample ; is the corresponding true diagnosis label. By analyzing the amplitude distribution of the gradient matrix, the sensitivity of the federated learning model to different input features is mined, providing a key basis for subsequent mask generation. For example, for a chest X - ray image, the attention degree of the federated learning model to the features of different regions of the lungs can be determined by calculating the gradient.

[0049] S2. The server - side receives the statistics of the local medical image data uploaded by each client and the local model training gradient information, generates a perturbation mask based on the latent space dynamic confusion mechanism, combines the perturbation mask with the learnable spatial attention weight and injects it into the perturbation model. The spatial attention weight can dynamically adjust the action intensity of the mask according to the spatial features of the input data, realizing refined perturbation of the output of the perturbation model, as Figure 2 shown, specifically including the following steps:

[0050] S21. Use the federated average algorithm (FedAvg) to initially aggregate the gradient information uploaded by each client to obtain the global gradient trend information; calculate the Wasserstein distance between clients according to the global gradient trend information, and use the K - means clustering algorithm to divide the clients.

[0051] The calculation of the Wasserstein distance between clients is:

[0052] ;

[0053] where is the Wasserstein distance between client and client , is the set of joint distributions between client and client , represents the expectation operation on the data sample pair under the joint distribution , and performs weighted average according to the probability given by the joint distribution ; are respectively from the client Medical image data samples.

[0054] As an example, in this embodiment, the clients of all medical institutions are divided into clients. . For example, according to the characteristic distribution of the image data, the clients on the medical institutions dealing with lung disease images are divided into one client, and those dealing with brain disease images are divided into another client, etc.

[0055] S22: Assign an independent generator to each client, initialize it as a copy of the global generator, incorporate the global gradient trend information into the copy of the global generator, and further adjust the client generator parameters through the federated averaging algorithm to generate a perturbation mask.

[0056] Specifically, assign an independent generator to each client according to the gradient magnitude analysis , denotes the mask, initialize the client generator as a copy of the global generator , and then update the client generator parameters through federated averaging (FedAvg) , and adaptively adjust the client generator parameters according to the characteristics of the data within the client to generate a perturbation mask that is more suitable for the distribution of the client's local medical image data:

[0057] ;

[0058] ;

[0059] In the formula is the client generator parameter after the th update; is the set of clients within the th client; is the generator parameter of a certain client within the client at the th update; is the gradient operator of the client generator parameter; is the client clustering loss function, which is used to measure the difference between the generated mask and the generation result of the client generator according to the gradient ; is the input medical image data sample gradient operator; is the loss function; is the client generator according to the gradient The generated result; is the update step size; is for the input medical image data sample to generate a perturbation mask that satisfies the norm constraint , , is the perturbation intensity threshold, is the dimension of the perturbation mask. The perturbation mask can perturbe the output of the perturbation model according to the gradient information, effectively confusing the attacker's inference of the original data. Apply the two-dimensional discrete cosine transform (DCT) to the generated perturbation mask to convert it to the frequency domain. Randomly discard p (e.g., p = 30%) low-frequency components in the frequency domain to generate a high-frequency dominant perturbation mask. This high-frequency perturbation pattern can increase the randomness and complexity of the perturbation without affecting the expression of the main features of the perturbation model, further improving the defense effect.

[0060] S23. Calculate the client adaptation score by means of a multi-layer perceptron (MLP), and fuse the perturbation mask using the attention mechanism. The corresponding formula is:

[0061] ;

[0062] ;

[0063] where, represents the finally fused perturbation mask, is the number of clients, represents the normalization exponential function, represents the th output of the client generator for the input medical image data sample , is the mean of the th client's local medical image data, represents the th covariance matrix of the client's local medical image data, is the adaptation score, is the correlation coefficient, represents the multi-layer perceptron.

[0064] S24. Apply the federated averaging algorithm again to aggregate the client generator parameters to obtain the global generator. Combine the global generator with the original federated learning model, and generate a perturbation model based on the global gradient trend information and the fused perturbation mask. Send the perturbation model to the client. The perturbation model is as follows:

[0065] ;

[0066] where, is the output of the perturbation model; is the original federated learning model for the input medical image data sample output; is the client the generator of which generates a mask according to the gradient and generator parameters generated mask, is the gradient operator of the input medical image data sample ; is the loss function; is the Hadamard product, is the Sigmoid function, is the attention weight for the input medical image data sample The attention weight can dynamically adjust the action intensity of the mask according to the spatial features of the input data, so as to achieve refined perturbation of the model output. By continuously adjusting the attention weight dynamically adjusts the action intensity of the mask according to the features of the image, so as to achieve refined perturbation of the model output.

[0067] S3. The client receives the perturbed model and updates the local model parameters.

[0068] S4. The attacker trains a surrogate model similar to the structure of the original federated learning model based on the generalization error-guided adversarial optimization mechanism. The server updates the generator parameters according to the attack situation and defense feedback to optimize the defense. After multiple rounds of training the surrogate model and the server optimizing the defense, the defense effect of the federated learning model against the surrogate model attack is evaluated.

[0069] In this embodiment, the attacker is assumed to be some malicious clients, and a surrogate model similar to the structure of the original federated learning model is trained using a generative adversarial network to break through privacy protection, steal data or interfere with training. The surrogate model similar to the structure of the original federated learning model is a neural network, and the neural network structure includes several convolutional layers, pooling layers and fully connected layers. The surrogate model training process is to continuously adjust the surrogate model parameters through the backpropagation algorithm using the data output by the perturbed model, so that it can simulate the behavior of the federated learning model as much as possible.

[0070] The generalization error-guided adversarial optimization mechanism is the key defense part of this method. The generalization error-guided adversarial optimization mechanism includes the optimization of a two-layer optimization problem, namely the outer layer objective and the inner layer objective.

[0071] The optimization of the outer-layer objective is the generalization error of the maximum attack model for the outer layer: the surrogate model reconstructs the original medical image data using the output of the perturbation model (the surrogate model is trained using a generative adversarial network (GAN), with the output of the perturbation model as the input to generate an image close to the original medical image data), and through the generalization of the error of the maximum attack model for the outer layer on the attack data distribution it makes it difficult for the attacker to accurately reconstruct the data. The optimization of the inner-layer objective is to ensure that while defending against the attack, the performance of the model is not overly reduced by minimizing the impact on the federated learning model.

[0072] The maximum attack model for the outer layer The generalization of the error on the attack data distribution is as follows:

[0073] ;

[0074] where is the attack data distribution; represents the expectation of the input medical image data sample under the attack data distribution ; is the loss function; is the output of the federated learning model; is the generator of the generative adversarial network for the input medical image data sample ; is the attack model.

[0075] The inner-layer objective is then to minimize the impact on the federated learning model:

[0076] ;

[0077] where is the normal data distribution, represents the minimum expectation of the generator of the generative adversarial network under the normal data distribution ; is the output of the perturbation model; represents the input medical image data sample, is the output of the federated learning model; is the trade-off factor, is the total variation regularization term, is the perturbation mask. On the normal data distribution by minimizing the inner-layer objective, it is ensured that while defending against the attack, the accuracy of the model's disease prediction is not overly reduced.

[0078] The server - side uses the Alternating Direction Method of Multipliers (ADMM) and the Proximal Policy Optimization (PPO) algorithm to update the generator parameters based on the attack situation and defense feedback.

[0079] Specifically, the ADMM is used to decouple the optimization of the inner - layer objective. In each iteration, first fix the global server generator parameters and update the attack model , and then fix the updated attack model and update the global server generator parameters through the gradient of the Proximal Policy Optimization (PPO) algorithm. This optimization method can effectively balance the generalization error of the attack model and its impact on the federated learning model, and improve the performance of the defense algorithm. The formula for updating the attack model is:

[0080] ;

[0081] where is the attack model after the -th update, is the attack model to be optimized, represents the expectation of the input medical image data sample , is the cross - entropy loss function, is the output of the perturbation model, is the output of the federated learning model, represents the cross - entropy loss function, and is optimized through the cross - entropy loss function .

[0082] The formula for updating the server generator parameters through the Proximal Policy Optimization (PPO) gradient is:

[0083] ;

[0084] where is the server - side generator parameter after the -th update; is the attack loss function; is the gradient operator of the server generator parameter ; is the trade - off factor; is the total variation regularization term; is the perturbation mask; is the learning rate.

[0085] Evaluate the defense effect and determine whether the termination condition is met (the termination condition is that the accuracy of the attacker's reconstructed data is lower than the expected threshold, such as , or the performance degradation of the federated learning model is within an acceptable range, such as the accuracy degradation does not exceed If not, continue the iteration, which includes the attacker training a surrogate model based on the new perturbation output and the server updating the generator parameters according to the attack situation and defense feedback, etc.

[0086] After multiple rounds of training (including training the federated learning model, the attacker training the surrogate model, and the server updating the generator parameters, etc.) and defense optimization, the proposed solution in this embodiment evaluates the defense effect and performance of the federated learning model. With the same privacy budget the SSIM value of the attacker's reconstructed medical image drops to 0.21, while the baseline solution is 0.53, effectively disrupting the attacker's data reconstruction process. The test accuracy of the federated global model only drops by 1.8%, which is significantly lower than the 9.7% of the traditional encryption solution, ensuring the effectiveness and reliability of the model in disease prediction. In the continuous 10-round attack and defense game, the attack success rate drops from 31% in the first round to 7% in the 10th round, while the attack success rate of the static solution (such as gradient clipping) rises from 28% to 43%, fully demonstrating that the dynamic defense mechanism of the present invention can adapt to the changes in the attacker's strategy in real time and continuously maintain high defense capabilities.

[0087] The preferred embodiments of the present invention disclosed above are only used to help illustrate the present invention. The preferred embodiments do not describe all the details in detail, nor do they limit the invention to the specific embodiments described. Obviously, many modifications and variations can be made according to the content of this specification. These embodiments are selected and specifically described to better explain the principles and practical applications of the present invention, enabling those skilled in the relevant technical fields to understand and utilize the present invention well.

Claims

1. A method for protecting medical image data privacy in a federated learning model, characterized in that: The following steps are involved: S1. The server initializes the federated learning model for disease prediction. Each client uses the gradient descent method to train the original federated learning model as a local model based on the local medical imaging data, calculates the local model training gradient information and the statistics of the local medical imaging data, and uploads the statistics of the local medical imaging data and the local model training gradient information to the server. S2, the server receives the statistics of local medical imaging data uploaded by each client and the local model training gradient information to generate a perturbation model, and sends the perturbation model to the client; The specific steps include: S21, using the federated average algorithm to initially aggregate gradient information to obtain global gradient trend information, calculating the Wasserstein distance between clients based on the global gradient trend information, and using the K-means clustering algorithm to divide the clients; S22, assigning an independent generator to each client, initializing it as a global generator copy, integrating the global gradient trend information into the global generator copy, adjusting the client generator to generate a perturbation mask through a federated average algorithm, applying a two-dimensional discrete cosine transform to the generated perturbation mask, and converting it to the frequency domain; S23, calculate the client adaptation score with the help of multi-layer perceptron, and fuse the perturbation mask using the attention mechanism; S24. Use the federated average algorithm to aggregate the client generator parameters to obtain a global generator, combine the global generator with the original federated learning model, generate a perturbation model based on the global gradient trend information and the fused perturbation mask, and send the perturbation model to the client; S3, each client receives the disturbance model and updates the local model parameters; S4. The attacker trains the alternative model based on the generalization error-guided adversarial optimization mechanism, and the server uses the alternating direction multiplier method and the proximal strategy optimization algorithm to update the global generator parameters to optimize the defense. After multiple rounds of training of the alternative model by the attacker and the server-side defense optimization, the defense effect of the federated learning model against the alternative model attack is evaluated. If the expected effect is not achieved, the training continues.

2. According to the method for protecting medical image data privacy in a federated learning mode in claim 1, it is characterized in that: In step S23, the calculation of the client adaptation score by means of the multilayer perceptron specifically includes: for each client, first calculating its adaptation score, calculating its adaptation score specifically by combining the mean and covariance matrix of the local medical image data into a vector, then inputting the synthesized vector into the multilayer perceptron for calculation, and finally outputting the adaptation score of the client; The method of fusing the perturbation mask using the attention mechanism specifically includes: first dividing the client's adaptation score by the correlation coefficient, and then inputting the result into a normalized exponential function for processing. The normalized exponential function processing process will make the sum of the corresponding values ​​of all clients normalized to 1, that is, each client is weighted, and the weight of the client with a high adaptation score will be greater. Then, the value obtained after processing with the normalized exponential function is multiplied by the output value of the client generator. Finally, the value of all clients processed by the normalized exponential function is multiplied by the output value of the client generator. The sum obtained is the final fused perturbation mask; the output value of the client generator is the value obtained by inputting the medical imaging data sample into the client generator.

3. According to the method for protecting medical image data privacy in a federated learning mode in claim 1, it is characterized in that: The generalization error-guided adversarial optimization mechanism includes optimization of outer objectives and inner objectives; The outer layer goal is to maximize the generalization error of the attack model. The substitution model uses the output of the perturbation model as input to reconstruct the original medical image data to generate images close to the original medical image data. By maximizing the error generalization of the attack model on the attack data distribution in the outer layer and minimizing the error on the federated learning model, it is difficult for the attacker to accurately reconstruct the data. The inner goal is to minimize the impact on the federated learning model and ensure that the performance of the federated learning model is not excessively degraded while defending against attacks.

4. According to the method for protecting medical image data privacy in a federated learning mode according to claim 3, it is characterized in that: The alternative model reconstructs the original medical imaging data using the output of the perturbation model as input, specifically including: the alternative model is a substitute model with a similar structure to the original federated learning model trained using a generative adversarial network; the alternative model training process is to use the data output by the perturbation model to continuously adjust the parameters of the alternative model through a back propagation algorithm, so that the alternative model simulates the behavior of the federated learning model.

5. According to the method for protecting medical image data privacy in a federated learning mode in claim 3, it is characterized in that: In step S4, the server side uses the alternating direction multiplier method to decouple the optimization of the inner layer target. In each iteration, the server global generator parameters are first fixed, the attack model is updated, and then the updated attack model is fixed, and then the server global generator parameters are updated through the proximal policy optimization algorithm gradient.

6. According to the method for protecting medical image data privacy in a federated learning mode according to claim 5, it is characterized in that: The updated attack model The formula used is: ； in It is After the updated attack model, is the attack model to be optimized. Represents the input medical image data sample expectations, is the cross entropy loss function, is the output of the perturbation model, is the output of the federated learning model.

7. According to the method for protecting medical image data privacy in a federated learning mode in claim 5, it is characterized in that: The server global generator parameters are updated through the proximal policy optimization algorithm gradient as follows: The updated server-side generator parameters are equal to The updated server-side generator parameters are added with an adjustment amount, which is the product of the learning rate and a gradient, where the learning rate controls the step size of each update, and the gradient is the attack loss function minus the product of the trade-off factor and the total variation regularization term. The attack loss function measures the loss of the attack model during the attack process; the trade-off factor is used to balance the importance of the attack loss function and the total variation regularization term; the total variation regularization term is obtained based on the perturbation mask, and the perturbation mask is used to perturb the server-side generator parameters to protect data privacy.

8. A system for implementing the medical image data privacy protection method in a federated learning mode as described in claim 1, characterized in that: include: The client data processing and interaction module is used to train the original federated learning model through stochastic gradient descent, calculate the gradient of the federated learning model and the statistics of the mean and covariance matrix of the local medical imaging data; Use the network interface to upload the corresponding data to the server and receive the disturbance model sent by the server; The server gradient aggregation and client partitioning module is used to aggregate client gradients through the federated average algorithm to obtain the global gradient trend, calculate the Wasserstein distance between clients, and perform K-means clustering to partition clients; The server mask generation and model delivery module is used to allocate and adjust independent generator parameters for the client through the federated averaging algorithm to generate an adaptive perturbation mask, fuse the mask through the attention mechanism to obtain the global generator, and combine the global generator with the original federated learning model to generate a perturbation model and deliver it to the client; the alternating direction multiplier method and the proximal strategy optimization algorithm gradient are used to update the generator parameters to improve the defense effect, while storing the data uploaded by the client.

9. A computer device, characterized in that: include: A memory and a processor and a computer program stored in the memory, when the computer program is executed on the processor, implements a medical image data privacy protection method under a federated learning mode as described in any one of claims 1 to 7.

Images