Medical image data privacy protection method and system in federated learning mode

Abstract

The invention discloses a medical image data privacy protection method and system in a federated learning mode, and the method comprises the following steps: training an original federated learning model through a gradient descent method based on local medical image data by each client as a local model, calculating the training gradient information of the local model and the statistical magnitude of the local medical image data, uploading the statistics of the local medical image data and the local model training gradient information to a server side; the server side receives the statistics and the information uploaded by each client side to generate a disturbance model; each client receives the disturbance model and updates local model parameters; an attacker trains the substitution model based on a generalization error-oriented confrontation optimization mechanism, and the server side updates generator parameters according to the attack situation and defense feedback of the substitution model to optimize defense; and through multi-round training of the substitution model by an attacker and optimization defense of the server side, the defense effect of the federated learning model for resisting the attack of the substitution model is evaluated, and if an expected effect is not achieved, training continues.

Description

Technical Field

[0001] The present invention is applicable to the field of artificial intelligence system security, and specifically provides a medical image data privacy protection method and system under a federated learning model. Background Art

[0002] In the field of medical imaging, federated learning technology is booming. Its core goal is to achieve localized data storage while jointly optimizing the model through multi-party collaboration to improve the efficiency of medical image analysis. This technical model can theoretically integrate data resources from all parties to provide strong support for the accurate interpretation of medical images, thereby pushing medical diagnostic technology to a new level.

[0003] However, the existing technologies in this field have exposed significant shortcomings in data privacy protection. Many medical image algorithms based on federated learning have overlooked a key link in the process of pursuing data localization and improving the efficiency of joint optimization modeling: data privacy and security protection during the federated learning process.

[0004] Existing federated learning methods (such as Chinese patent document CN202310970584.4) are for the server to first initialize global model parameters and send these parameters to the medical user end. After receiving the parameters, the user end uses the private data set stored locally to train the local model. After the training is completed, the local model parameters are transmitted back to the server end. The server end prepares for the next round of training by adaptively aggregating these parameters. This method has made some achievements in solving the problem of class imbalance, but it does not involve any consideration of data privacy protection in the federated learning process. In actual application scenarios, this means that malicious attackers are very likely to take advantage of the opportunity of the model to transmit information between the server and the user end to steal the original medical image data, posing a serious threat to patient privacy. For example, Chinese patent document CN202011393242.3 discloses a CT image detection method based on federated learning and related devices focusing on the optimization of the CT image detection process. The first device trains the model based on its own data, then sends the parameters to the second device, and receives the relevant average value for updating its own parameters, and then retrains the model to mark abnormal areas in the CT image. Although this method is carefully designed in the CT image detection process, there is no mention of the prevention of data privacy leakage in the federated learning process. This undoubtedly leaves security risks in key links such as model parameter transmission, which allows malicious attackers to take advantage of it and may lead to data privacy leakage incidents. Chinese patent document CN202211331593.0 discloses an online cervical cell TCT slice detection method and system based on federated learning, which is implemented in that the cloud server initializes the global auxiliary diagnosis model, and the local TCT reader initializes the local model and receives the global model as an intermediate model. After the local reader distinguishes the image, it is reviewed by the doctor. When the federated learning mechanism is not triggered, the model is retrained using local private data. When the federated learning conditions are met, the local uploads the gradient parameters of the intermediate model to the cloud server. After the cloud server completes the aggregation update, the new global model is sent down to update the local intermediate model. While this method improves the online cervical cell TCT slice detection process, it seriously ignores the importance of data privacy protection. In the current complex network environment, it is entirely possible for malicious attackers to use model stealing technology based on the generative adversarial network architecture to reversely restore the client's original medical image data through the model information transmitted by the system, causing extremely serious risks of data privacy leakage.

[0005] In summary, although existing medical image algorithms based on federated learning have made many efforts in local data storage and joint optimization modeling efficiency improvement, there is a general lack of security protection for data privacy leakage in the federated learning process. Potential malicious attackers can use model stealing technology and model information transmitted by the system to easily restore the original medical image data, which not only seriously threatens the privacy security of patients, but also hinders the further promotion and application of federated learning technology in the field of medical imaging. Summary of the invention

[0006] The present invention aims to propose a medical image data privacy protection method and system under the federated learning mode, including a complex mask generation mechanism, a dynamic defense mechanism, and an adversarial optimization mechanism, etc., for the model training security protection of distributed artificial intelligence systems, effectively responding to the security threats faced by distributed artificial intelligence systems in artificial intelligence model training, solving the shortcomings of existing defense technologies, and providing practical and effective protection for the privacy security of medical image data in the federated learning process.

[0007] The purpose of the present invention is achieved by at least one of the following technical solutions.

[0008] A method for protecting privacy of medical image data in a federated learning mode includes the following steps: S1. The server initializes the federated learning model for disease prediction. Each client uses the gradient descent method to train the original federated learning model as a local model based on the local medical imaging data, calculates the local model training gradient information and the statistics of the local medical imaging data, and uploads the statistics of the local medical imaging data and the local model training gradient information to the server. S2, the server receives the statistics of local medical imaging data uploaded by each client and the local model training gradient information to generate a perturbation model, and sends the perturbation model to the client; S3, each client receives the disturbance model and updates the local model parameters; S4. The attacker trains the alternative model based on the generalization error-guided adversarial optimization mechanism, and the server uses the alternating direction multiplier method and the proximal strategy optimization algorithm to update the global generator parameters to optimize the defense. After multiple rounds of training of the alternative model by the attacker and the server-side defense optimization, the defense effect of the federated learning model against the alternative model attack is evaluated. If the expected effect is not achieved, the training continues.

[0009] Furthermore, in step S2, the server receives the statistics of the local medical imaging data uploaded by each client and the local model training gradient information to generate a perturbation model, and sends the perturbation model to the client, including the following steps: S21, using the federated average algorithm to initially aggregate gradient information to obtain global gradient trend information, calculating the Wasserstein distance between clients based on the global gradient trend information, and using the K-means clustering algorithm to divide the clients; S22, assigning an independent generator to each client, initializing it as a global generator copy, integrating the global gradient trend information into the global generator copy, adjusting the client generator to generate a perturbation mask through a federated average algorithm, applying a two-dimensional discrete cosine transform to the generated perturbation mask, and converting it to the frequency domain; S23, calculate the client adaptation score with the help of multi-layer perceptron, and fuse the perturbation mask using the attention mechanism; S24. Use the federated averaging algorithm to aggregate the client generator parameters to obtain the global generator, combine the global generator with the original federated learning model, generate a perturbation model based on the global gradient trend information and the fused perturbation mask, and send the perturbation model to the client.

[0010] Further, in step S23, the client adaptation score is calculated by means of a multi-layer perceptron. Specifically, for each client, its adaptation score is first calculated. Specifically, the adaptation score is calculated by combining the mean and covariance matrix of the local medical imaging data of the client into a vector, and then the synthesized vector is input into the multi-layer perceptron for calculation, and finally the adaptation score of the client is output; The perturbation mask is fused using the attention mechanism as follows: first, the adaptation score of the client is divided by the correlation coefficient, and then the result is input into the normalized exponential function for processing. The normalized exponential function processing process will make the sum of the corresponding values ​​of all clients normalized to 1, that is, each client is weighted, and the weight of the client with a high adaptation score will be greater. Then, the value obtained after processing by the normalized exponential function is multiplied by the output value of the client generator. Finally, the value of all clients processed by the normalized exponential function is multiplied by the output value of the client generator. The sum is the final fused perturbation mask; the output value of the client generator is the value obtained by inputting the medical imaging data sample into the client generator.

[0011] Furthermore, the generalization error-guided adversarial optimization mechanism includes optimization of outer objectives and inner objectives; The outer layer goal is to maximize the generalization error of the attack model. The substitution model uses the output of the perturbation model as input to reconstruct the original medical image data to generate images close to the original medical image data. By maximizing the error generalization of the attack model on the attack data distribution in the outer layer and minimizing the error on the federated learning model, it is difficult for the attacker to accurately reconstruct the data. The inner goal is to minimize the impact on the federated learning model and ensure that the performance of the federated learning model is not excessively degraded while defending against attacks.

[0012] Furthermore, the alternative model reconstructs the original medical imaging data using the output of the perturbation model as input, including: the alternative model is a substitute model with a similar structure to the original federated learning model trained using a generative adversarial network; the alternative model training process is to use the data output by the perturbation model to continuously adjust the parameters of the alternative model through a back propagation algorithm, so that the alternative model can simulate the behavior of the federated learning model.

[0013] Furthermore, in step S4, the server side uses the alternating direction multiplier method to decouple the optimization of the inner layer target. In each iteration, the server global generator parameters are first fixed, the attack model is updated, and then the updated attack model is fixed, and then the server global generator parameters are updated through the proximal strategy optimization algorithm gradient.

[0014] Further, update the attack model The formula is: ; in It is After the updated attack model, is the attack model to be optimized. Represents the input medical image data sample expectations, is the cross entropy loss function, is the output of the perturbation model, is the output of the federated learning model, represents the cross entropy loss function.

[0015] Furthermore, the server global generator parameters are updated through the proximal policy optimization algorithm gradient as follows: The updated server-side generator parameters are equal to The updated server-side generator parameters are added with an adjustment amount, which is the product of the learning rate and a gradient, where the learning rate controls the step size of each update, and the gradient is the attack loss function minus the product of the trade-off factor and the total variation regularization term. The attack loss function measures the loss of the attack model during the attack process; the trade-off factor is used to balance the importance of the attack loss function and the total variation regularization term; the total variation regularization term is obtained based on the perturbation mask, and the perturbation mask is used to perturb the server-side generator parameters to protect data privacy.

[0016] A system for implementing the medical image data privacy protection method in a federated learning mode includes: The client data processing and interaction module trains the original federated learning model through stochastic gradient descent, calculates the gradient of the federated learning model and the mean and covariance matrix statistics of the local medical imaging data; uses the network interface to upload the corresponding data to the server and receive the perturbation model sent by the server; Server gradient aggregation and client partitioning module: uses the federated average algorithm to aggregate client gradients to obtain the global gradient trend, calculates the Wasserstein distance between clients, and performs K-means clustering to partition clients; Server mask generation and model delivery module: allocates and adjusts independent generator parameters for the client through the federated averaging algorithm to generate an adaptive perturbation mask, fuses the mask through the attention mechanism to obtain the global generator, and combines the global generator with the original federated learning model to generate a perturbation model and deliver it to the client; uses the alternating direction multiplier method and proximal strategy to optimize the algorithm gradient and dynamically update the generator parameters to improve the defense effect, while storing the data uploaded by the client.

[0017] A computer device of the present invention comprises: a memory and a processor and a computer program stored in the memory. When the computer program is executed on the processor, the medical image data privacy protection method in a federated learning mode is implemented.

[0018] Compared with the prior art, the present invention has the following beneficial effects: The method of the present invention uses a latent space dynamic obfuscation mechanism, generalization error-guided adversarial optimization, and federated clustering-driven adaptive defense to make the evaluation quality of the attacker's reconstructed image poor under the same privacy budget, effectively destroy the attacker's data reconstruction process, and greatly enhance the protection ability of the federated learning model. Guarantee model performance: The test accuracy of the federated global model decreases at a low rate. While defending against attacks, the performance of the model is maintained to the greatest extent, ensuring the effectiveness and reliability of federated learning in practical applications. Significant advantages of dynamic defense: The dynamic defense mechanism shows significant advantages in continuous confrontation, which fully proves that this algorithm can adapt to changes in attacker strategies in real time and continue to maintain efficient defense capabilities. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 This is a flowchart of a medical image data privacy protection method and system under a federated learning mode in an embodiment of the present invention.

[0020] Figure 2 This is a flow chart of the latent space dynamic obfuscation mechanism in an embodiment of the present invention.

[0021] Figure 3 This is a flow chart of the generalized error-guided adversarial optimization mechanism in an embodiment of the present invention. DETAILED DESCRIPTION

[0022] In order to make the objectives, technical solutions and advantages of the present invention more clearly understood, the specific implementation of the present invention is described in detail below with reference to the accompanying drawings and examples.

[0023] This embodiment takes multiple medical institutions as examples. Each medical institution has a large amount of medical imaging data of patients. These data contain the privacy information of patients and therefore cannot be shared directly. At the same time, there are malicious attackers who attempt to steal the original medical imaging data through the output of the federated learning model. To prevent theft, a federated learning network composed of multiple medical institutions aims to jointly train a deep learning model for disease prediction to resist various adversarial attacks.

[0024] like Figure 1~Figure 3 As shown, a medical image data privacy protection method in a federated learning mode of this embodiment includes the following steps: S1. Initialize the parameters of the federated learning model, global generator and discriminator for disease prediction, set the perturbation intensity threshold, trade-off factor, learning rate, correlation coefficient and other algorithm hyperparameters; each client uses stochastic gradient descent to train the original federated learning model as the local model based on the local medical imaging data, calculates the local model training gradient information and the mean and covariance matrix statistics of the local medical imaging data, and uploads the statistics of the local medical imaging data and the local model training gradient information to the server.

[0025] Each client uses stochastic gradient descent to train the original federated learning model based on local medical imaging data, calculates gradients, and uses them for medical imaging feature extraction and classification diagnosis. At the same time, each client of each medical institution uploads statistics of local medical imaging data. To the server, Indicates The mean of local medical imaging data of each client, reflecting the central trend of the data; Indicates The covariance matrix of the local medical imaging data of each client describes the correlation and discreteness between the dimensions of the data.

[0026] As an embodiment, this embodiment selects the convolutional neural network structure VGG16 as the basic model to initialize the original federated learning model , used for feature extraction and classification prediction of medical images. The original federated learning model includes an input layer, a hidden layer, and an output layer. The input layer of the original federated learning model is adapted to the medical image format, the hidden layer extracts and transforms features through convolution, pooling, and full connection operations, and the output layer outputs the prediction results according to the disease prediction task. The original federated learning model is: ; in, Is with parameters The original federated learning model, The input medical image data set covers multiple modalities, such as X-ray, CT, MRI, etc. It is a collection of prediction results output by the original federated learning model, such as disease category and disease severity score.

[0027] Initialize the global generator and the discriminator , set the initial generator parameters and the initial discriminator parameters At the same time, set the algorithm hyperparameters, such as the perturbation intensity threshold , the trade-off factor , learning rate , correlation coefficient wait.

[0028] The clients of each medical institution train the federated learning model based on local medical imaging data, and use the automatic differentiation technology to calculate the model (for example, in PyTorch or TensorFlow, by building a calculation graph, automatically tracking the operations in the calculation process, and automatically calculating the gradient during back propagation) and combine the gradient descent method to calculate the gradient of the input image to obtain the gradient matrix , is the input medical imaging data sample; Represents the input medical image data sample The gradient operator of is the loss function used to measure the prediction results of the federated learning model With the true label The difference between them, common loss functions such as cross entropy loss function; It is a federated learning model For the input medical image data sample The predicted output of is the corresponding true diagnostic label. By analyzing the amplitude distribution of the gradient matrix, we can explore the sensitivity of the federated learning model to different input features and provide a key basis for subsequent mask generation. For example, for a lung X-ray image, by calculating the gradient, we can determine the degree of attention paid by the federated learning model to the features of different lung regions.

[0029] S2. The server receives the statistics of local medical imaging data uploaded by each client and the local model training gradient information, generates a perturbation mask based on the latent space dynamic confusion mechanism, and injects the perturbation mask into the perturbation model in combination with the learnable spatial attention weight. The spatial attention weight can dynamically adjust the intensity of the mask according to the spatial characteristics of the input data, thereby achieving refined perturbation of the perturbation model output, such as Figure 2 As shown, the specific steps include: S21. Use the FedAvg algorithm to preliminarily aggregate the gradient information uploaded by each client to obtain the global gradient trend information; calculate the Wasserstein distance between clients based on the global gradient trend information, and use the K-means clustering algorithm to divide the clients.

[0030] The Wasserstein distance between the calculated clients is: ; in For Clients and client The Wasserstein distance between For Clients and client The joint distribution set between In the joint distribution The following data sample pairs of Expected operation, according to the joint distribution The assigned probabilities are weighted averaged; From the client Sample medical imaging data.

[0031] As an embodiment, this embodiment divides the clients of all medical institutions into Clients, For example, based on the characteristic distribution of image data, clients on a medical institution that processes lung disease images are divided into one client, and clients that process brain disease images are divided into another client, and so on.

[0032] S22. Assign an independent generator to each client, initialize it as a global generator copy, integrate the global gradient trend information into the global generator copy, and further adjust the client generator parameters through the federated averaging algorithm to generate a perturbation mask.

[0033] Specifically, an independent generator is assigned to each client based on the gradient amplitude analysis. , Represents the mask and initializes the client generator For global generator The client generator parameters are then updated through the federated average (FedAvg) , according to the characteristics of the data in the client, the client generator parameters are adaptively adjusted to generate a perturbation mask that is more suitable for the distribution of local medical imaging data on the client: ; ; In the formula It is Client after update Generator parameters; It is A client collection within a client; It is Client Update A client Generator parameters; is the gradient operator of the client generator parameters; is the client-side clustering loss function, which is used to measure the generated mask With the client builder According to the gradient Generate differences between results; is the input medical imaging data sample The gradient operator of is the loss function; Is the client The generator is based on the gradient The generated results; is the update step size; For the input medical imaging data sample The generated norm constraint is satisfied The perturbation mask of , is the disturbance intensity threshold, is the dimension of the perturbation mask. The perturbation mask can perturb the perturbation model output in a targeted manner based on the gradient information, effectively confusing the attacker's inference of the original data. Apply a two-dimensional discrete cosine transform (DCT) to the generated perturbation mask and convert it to the frequency domain. In the frequency domain, randomly discard p (e.g., p = 30%) low-frequency components to generate a high-frequency-dominated perturbation mask. This high-frequency perturbation pattern can increase the randomness and complexity of the perturbation without affecting the expression of the main features of the perturbation model, further improving the defense effect.

[0034] S23. Calculate the client adaptation score with the help of multi-layer perceptron (MLP), and use the attention mechanism to fuse the perturbation mask. The corresponding formula is: ; ; in, represents the final fused perturbation mask, is the number of clients, represents the normalized exponential function, Representative The client generator takes the input medical image data sample The output, It is The average value of local medical imaging data of clients, Indicates The covariance matrix of the local medical imaging data of each client, is the adaptation score, is the correlation coefficient, Represents a multilayer perceptron.

[0035] S24, again use the federated average algorithm to aggregate the client generator parameters to obtain a global generator, combine the global generator with the original federated learning model, generate a perturbation model based on the global gradient trend information and the fused perturbation mask, and send the perturbation model to the client. The perturbation model is as follows: ; in, is the output of the perturbation model; It is the original federated learning model For the input medical image data sample Output: Is the client The generator of and generator parameters The generated mask, is the input medical imaging data sample The gradient operator of is the loss function; is the Hadamard product, is the Sigmoid function, For the input medical imaging data sample The attention weight can dynamically adjust the strength of the mask according to the spatial characteristics of the input data, thus achieving a refined perturbation of the model output. , dynamically adjust the intensity of the mask according to the characteristics of the image to achieve refined perturbations on the model output.

[0036] S3. The client receives the perturbation model and updates the local model parameters.

[0037] S4. The attacker trains a substitute model with a similar structure to the original federated learning model based on the generalization error-guided adversarial optimization mechanism. The server updates the generator parameters to optimize the defense based on the attack situation and defense feedback. After multiple rounds of training the substitute model and optimizing the defense on the server side, the defense effect of the federated learning model against the substitute model attack is evaluated.

[0038] The attackers in this embodiment are assumed to be some malicious clients, and use generative adversarial networks to train alternative models with similar structures to the original federated learning model to break through privacy protection, steal data or interfere with training. The alternative model with a similar structure to the original federated learning model is a neural network, and the neural network structure includes several convolutional layers, pooling layers and fully connected layers. The alternative model training process uses the data output by the perturbation model to continuously adjust the alternative model parameters through the back propagation algorithm to simulate the behavior of the federated learning model as much as possible.

[0039] The generalization error-guided adversarial optimization mechanism is the key defense part of the method. The generalization error-guided adversarial optimization mechanism includes a two-level optimization problem, namely the optimization of the outer layer objective and the inner layer objective.

[0040] The optimization of the outer layer objective is to maximize the generalization error of the outer layer attack model: the substitute model uses the output of the perturbation model to reconstruct the original medical image data (the substitute model is trained using a generative adversarial network (GAN), and the output of the perturbation model is used as input to generate an image close to the original medical image data). The error generalization on the network makes it difficult for attackers to accurately reconstruct the data. The optimization of the inner layer objective is to minimize the impact on the federated learning model to ensure that the performance of the model is not excessively degraded while defending against attacks.

[0041] Outer Maximization Attack Model Attack data distribution The error on is generalized to: ; in To attack data distribution; Indicates the attack data distribution The following is a sample of the input medical image data expectations; is the loss function; is the output of the federated learning model; is the generator of the generative adversarial network For the input medical image data sample The output, For attack model.

[0042] The inner goal is to minimize the impact on the federated learning model: ; in For normal data distribution, Indicates that in normal data distribution The generator of the generative adversarial network Find the minimum expectation, is the output of the perturbation model; represents the input medical image data sample, is the output of the federated learning model; is the trade-off factor, is the total variation regularization term, is the perturbation mask. In normal data distribution By minimizing the inner target, we ensure that the accuracy of the model's disease prediction will not be excessively reduced while defending against attacks.

[0043] The server side uses the alternating direction multiplier method and proximal strategy optimization algorithm to update the generator parameters according to the attack situation and defense feedback.

[0044] Specifically, the alternating direction multiplier method is used to decouple the optimization of the inner layer objectives. In each iteration, the server global generator parameters are fixed first, and the attack model is updated. , then fix the updated attack model and update the server global generator parameters through the proximal policy optimization algorithm (PPO) gradient. This optimization method can effectively balance the generalization error of the attack model and the impact on the federated learning model, and improve the performance of the defense algorithm. The formula is: ; in It is After the updated attack model, is the attack model to be optimized. Represents the input medical image data sample expectations, is the cross entropy loss function, is the output of the perturbation model, is the output of the federated learning model, Represents the cross entropy loss function, through the cross entropy loss function Make optimizations.

[0045] The formula for updating the server generator parameters through the proximal policy optimization algorithm (PPO gradient is: ; in It is Updated server-side generator parameters; is the attack loss function; is a parameter to the server generator The gradient operator of is a weighing factor; is the total variation regularization term; is the perturbation mask; is the learning rate.

[0046] Evaluate the defense effect and determine whether the termination condition is met (the termination condition is that the accuracy of the attacker's reconstructed data is lower than the expected threshold, such as , or the performance of the federated learning model is within an acceptable range, such as a decrease in accuracy of no more than If it is not reached, the iteration continues. The iteration includes the attacker training the replacement model based on the new perturbation output, and the server updating the generator parameters according to the attack situation and defense feedback.

[0047] This embodiment provides a solution that, after multiple rounds of training (including training the federated learning model, the attacker training the alternative model, and the server-side updating the generator parameters and other related operations) and defense optimization, the federated learning model The defense effect and performance of the same privacy budget are evaluated. , the SSIM value of the attacker's reconstructed medical images dropped to 0.21, while the baseline solution was 0.53, effectively destroying the attacker's data reconstruction process. The test accuracy of the federated global model only dropped by 1.8%, which is significantly lower than the 9.7% of the traditional encryption scheme, ensuring the effectiveness and reliability of the model in disease prediction. In the 10-round attack and defense game, the attack success rate dropped from 31% in the first round to 7% in the tenth round, while the attack success rate of static schemes (such as gradient clipping) increased from 28% to 43%, which fully proves that the dynamic defense mechanism of the present invention can adapt to changes in attacker strategies in real time and continue to maintain efficient defense capabilities.

[0048] The preferred embodiments of the present invention disclosed above are only used to help explain the present invention. The preferred embodiments do not describe all the details in detail, nor do they limit the invention to the specific implementation methods described. Obviously, many modifications and changes can be made according to the content of this specification. This specification selects and specifically describes these embodiments in order to better explain the principles and practical applications of the present invention, so that those skilled in the art can understand and use the present invention well.

Claims

1. A method for protecting medical image data privacy in a federated learning model, characterized in that: The following steps are involved: S1. The server initializes the federated learning model for disease prediction. Each client uses the gradient descent method to train the original federated learning model as a local model based on the local medical imaging data, calculates the local model training gradient information and the statistics of the local medical imaging data, and uploads the statistics of the local medical imaging data and the local model training gradient information to the server. S2, the server receives the statistics of local medical imaging data uploaded by each client and the local model training gradient information to generate a perturbation model, and sends the perturbation model to the client; S3, each client receives the disturbance model and updates the local model parameters; S4, the attacker trains the alternative model based on the generalization error-guided adversarial optimization mechanism, and the server uses the alternating direction multiplier method and the proximal strategy optimization algorithm to update the global generator parameters to optimize the defense; After multiple rounds of training of the attacker’s substitution model and server-side optimization defense, the defense effectiveness of the federated learning model against the substitution model attack is evaluated. If the expected effect is not achieved, training will continue.

2. According to the method for protecting medical image data privacy in a federated learning mode in claim 1, it is characterized in that: Step S2 specifically includes the following steps: S21, using the federated average algorithm to initially aggregate gradient information to obtain global gradient trend information, calculating the Wasserstein distance between clients based on the global gradient trend information, and using the K-means clustering algorithm to divide the clients; S22, assigning an independent generator to each client, initializing it as a global generator copy, integrating the global gradient trend information into the global generator copy, adjusting the client generator to generate a perturbation mask through a federated average algorithm, applying a two-dimensional discrete cosine transform to the generated perturbation mask, and converting it to the frequency domain; S23, calculate the client adaptation score with the help of multi-layer perceptron, and fuse the perturbation mask using the attention mechanism; S24. Use the federated averaging algorithm to aggregate the client generator parameters to obtain the global generator, combine the global generator with the original federated learning model, generate a perturbation model based on the global gradient trend information and the fused perturbation mask, and send the perturbation model to the client.

3. According to the method for protecting medical image data privacy in a federated learning mode in claim 2, it is characterized in that: In step S23, the calculation of the client adaptation score by means of the multilayer perceptron specifically includes: for each client, first calculating its adaptation score, specifically calculating its adaptation score by combining the mean and covariance matrix of the local medical imaging data of the client into a vector, then inputting the synthesized vector into the multilayer perceptron for calculation, and finally outputting the adaptation score of the client; The method of fusing the perturbation mask using the attention mechanism specifically includes: first dividing the client's adaptation score by the correlation coefficient, and then inputting the result into a normalized exponential function for processing. The normalized exponential function processing process will make the sum of the corresponding values ​​of all clients normalized to 1, that is, each client is weighted, and the weight of the client with a high adaptation score will be greater. Then, the value obtained after processing with the normalized exponential function is multiplied by the output value of the client generator. Finally, the value of all clients processed by the normalized exponential function is multiplied by the output value of the client generator. The sum obtained is the final fused perturbation mask; the output value of the client generator is the value obtained by inputting the medical imaging data sample into the client generator.

4. According to the method for protecting medical image data privacy in a federated learning mode in claim 2, it is characterized in that: The generalization error-guided adversarial optimization mechanism includes optimization of outer objectives and inner objectives; The outer layer goal is to maximize the generalization error of the attack model. The substitution model uses the output of the perturbation model as input to reconstruct the original medical image data to generate images close to the original medical image data. By maximizing the error generalization of the attack model on the attack data distribution in the outer layer and minimizing the error on the federated learning model, it is difficult for the attacker to accurately reconstruct the data. The inner goal is to minimize the impact on the federated learning model and ensure that the performance of the federated learning model is not excessively degraded while defending against attacks.

5. According to the method for protecting medical image data privacy in a federated learning mode as described in claim 4, it is characterized in that: The alternative model reconstructs the original medical imaging data using the output of the perturbation model as input, specifically including: the alternative model is a substitute model with a similar structure to the original federated learning model trained using a generative adversarial network; the alternative model training process is to use the data output by the perturbation model to continuously adjust the parameters of the alternative model through a back propagation algorithm, so that the alternative model can simulate the behavior of the federated learning model.

6. According to the method for protecting medical image data privacy in a federated learning mode as described in claim 4, it is characterized in that: In step S4, the server side uses the alternating direction multiplier method to decouple the optimization of the inner layer target. In each iteration, the server global generator parameters are first fixed, the attack model is updated, and then the updated attack model is fixed, and then the server global generator parameters are updated through the proximal policy optimization algorithm gradient.

7. According to the method for protecting medical image data privacy in a federated learning mode of claim 6, it is characterized in that: The updated attack model The formula used is: ； in It is After the updated attack model, is the attack model to be optimized. Represents the input medical image data sample expectations, is the cross entropy loss function, is the output of the perturbation model, is the output of the federated learning model, represents the cross entropy loss function.

8. According to the method for protecting medical image data privacy in a federated learning mode as described in claim 6, it is characterized in that: The server global generator parameters are updated through the proximal policy optimization algorithm gradient as follows: The updated server-side generator parameters are equal to The updated server-side generator parameters are added with an adjustment amount, which is the product of the learning rate and a gradient, where the learning rate controls the step size of each update, and the gradient is the attack loss function minus the product of the trade-off factor and the total variation regularization term. The attack loss function measures the loss of the attack model during the attack process; the trade-off factor is used to balance the importance of the attack loss function and the total variation regularization term; the total variation regularization term is obtained based on the perturbation mask, and the perturbation mask is used to perturb the server-side generator parameters to protect data privacy.

9. A system for implementing the medical image data privacy protection method in a federated learning mode as described in claim 1, characterized in that: include: The client data processing and interaction module is used to train the original federated learning model through stochastic gradient descent, calculate the gradient of the federated learning model and the statistics of the mean and covariance matrix of the local medical imaging data; Use the network interface to upload the corresponding data to the server and receive the disturbance model sent by the server; The server gradient aggregation and client partitioning module is used to aggregate client gradients through the federated average algorithm to obtain the global gradient trend, calculate the Wasserstein distance between clients, and perform K-means clustering to partition clients; The server mask generation and model delivery module is used to allocate and adjust independent generator parameters for the client through the federated averaging algorithm to generate an adaptive perturbation mask, fuse the mask through the attention mechanism to obtain the global generator, and combine the global generator with the original federated learning model to generate a perturbation model and deliver it to the client; the alternating direction multiplier method and the proximal strategy optimization algorithm gradient are used to update the generator parameters to improve the defense effect, while storing the data uploaded by the client.

10. A computer device, characterized in that: include: A memory and a processor and a computer program stored in the memory, when the computer program is executed on the processor, implements a medical image data privacy protection method under a federated learning mode as described in any one of claims 1 to 8.

Images