A federated learning privacy protection method based on gradient-guided key region disturbance

Abstract

The application discloses a federated learning privacy protection method based on gradient guidance key region disturbance, and belongs to the field of federated learning. The method considers the characteristics of sparse characteristics of medical images, positions high information density areas through dynamic mask, and combines projection gradient descent to accurately inject noise into the most critical data area of the medical image task to prevent gradient leakage attacks in the model training process. The method can effectively protect the privacy of local medical image data while maximizing the utility and accuracy of the global model. Thus, differentiated defense is realized according to the spatial sparsity of medical image data, and fine security protection is provided for the key sensitive areas without significantly damaging the model accuracy.

Description

Technical Field

[0001] This invention belongs to the field of federated learning, and more specifically, relates to a privacy protection method for federated learning based on gradient-guided perturbation of key regions. Background Technology

[0002] Federated Learning (FL), a revolutionary distributed machine learning paradigm, allows for collaborative training of global models without sharing local raw data. This architecture holds immense potential in fields with extremely high data privacy requirements, such as healthcare, effectively addressing the "data silo" problem caused by stringent data regulations like HIPAA and GDPR. However, despite avoiding direct exposure of raw data, recent research has revealed serious security vulnerabilities in its distributed architecture. Attackers can exploit shared gradient information to launch gradient leakage attacks (GLAs, such as deep gradient leakage (DLGs), reconstructing sensitive training samples from the client with extremely high fidelity.

[0003] Existing mechanisms designed to mitigate the aforementioned privacy breach risks all have limitations that are difficult to overcome in practical applications: 1. Cryptographic methods face practical limitations: While existing technologies such as homomorphic encryption (HE) and secure multi-party computation (SMC) can theoretically provide privacy guarantees during communication and aggregation, these technologies introduce extremely high computational complexity and communication overhead, making them difficult to deploy effectively in large-scale federated learning systems in the real world.

[0004] 2. Traditional differential privacy (DP) faces a privacy-utility imbalance: Differential privacy is currently the most widely adopted defense mechanism, which resists attacks by injecting calibration noise into model updates or gradients (such as Local Differential Privacy (LDP) or Centralized Differential Privacy (CDP). However, this approach has drawbacks: stronger privacy protection often comes at the cost of severely sacrificing model utility (such as classification accuracy).

[0005] 3. “Global noise injection” ignores the spatial heterogeneity of data (core flaw): Existing perturbation-based privacy protection methods (including some methods that inject structured perturbations into the data space) all have a key limitation: they do not consider the spatial heterogeneity of information density in different regions of the image when applying noise. This one-size-fits-all global noise injection strategy is particularly fatal when processing sparse data such as medical images. In medical images, diagnostically valuable sensitive features are usually highly concentrated in specific anatomical or lesion areas, while large areas of background tissue are often homogeneous and lack discriminative value. Indiscriminately injecting global noise into the entire image leads to two extremely negative consequences: on the one hand, excessive perturbation of non-critical background areas introduces unnecessary utility degradation; on the other hand, the spread-out noise may not provide sufficient protection for high information density areas (sensitive areas), making them vulnerable to complex reconstruction attacks. Summary of the Invention

[0006] To address the aforementioned deficiencies or improvement needs of existing technologies, this invention provides a federated learning privacy protection method based on gradient-guided perturbation of key regions. This method takes into account the sparse characteristics of medical images and injects defensive noise precisely into the data regions most critical to the task, thereby effectively resisting gradient leakage attacks while maximizing the utility of the global model.

[0007] To achieve the above objectives, according to a first aspect of the present invention, a privacy-preserving method for federated learning based on gradient-guided perturbation of key regions is provided, comprising: S1, The central server initializes the global model parameters and sends them to the clients participating in the current round of training; S2, the client initializes the local model parameters to the global model parameters issued by the central server; S3, the client calculates the gradient matrix of the loss function with respect to the input sample superimposed with the current perturbation, and generates a binary mask matrix corresponding to the gradient matrix through the corresponding mask threshold. The gradient matrix and the binary mask matrix are multiplied element by element to obtain the gradient information that is masked, and the perturbation is updated through the gradient information; wherein, the input sample is a medical image, and the initial value of the perturbation is 0. S4, take the updated perturbation as the current perturbation, return to S3, until the maximum number of iterations is reached, take the latest perturbation as the optimal perturbation, and enter S5; S5, the client superimposes the optimal perturbation with the input sample to obtain protected data, and uses the protected data for forward and backward propagation to calculate the parameter gradient of the local model and upload it to the central server. S6, the central server performs a weighted summation of the parameter gradients uploaded by each client to obtain the global gradient, updates the global model in conjunction with the global learning rate, distributes the updated global model to each client participating in the next round of training, and returns to S2, until the maximum number of communication rounds is reached.

[0008] According to a second aspect of the present invention, a federated learning system is provided, comprising: a central server and a client, the central server and the client being used to execute a federated learning privacy-preserving method based on gradient-guided key region perturbation as described in the first aspect.

[0009] According to a third aspect of the present invention, a medical image classification method is provided for a client, comprising: classifying a medical image to be classified using a trained local model; The local model is trained using the federated learning privacy-preserving method based on gradient-guided perturbation of key regions as described in the first aspect.

[0010] According to a fourth aspect of the present invention, a client is provided, comprising: a memory and a processor, the memory storing a computer program, the processor executing the computer program to perform the medical image classification method of the third aspect.

[0011] According to a fifth aspect of the present invention, a computer-readable storage medium is provided, characterized in that the computer-readable storage medium stores computer instructions for causing a processor to execute either the gradient-guided key region perturbation-based federated learning privacy protection method as described in the first aspect or the medical image classification method as described in the third aspect.

[0012] In summary, compared with the prior art, the above-described technical solutions conceived by this invention can achieve the following beneficial effects: 1. Achieving fine-grained protection, breaking through the traditional bottleneck of the "privacy-utility" trade-off: Traditional global noise-adding mechanisms such as differential privacy (DP) blindly destroy the entire data space. This invention innovatively adopts gradient sensitivity analysis to strictly limit perturbations to high-information-density task-critical regions. This not only provides stronger noise protection for vulnerable sensitive areas but also avoids unnecessary damage to non-critical background regions, achieving extremely high model utility preservation.

[0013] 2. Highly compatible with the inherent characteristics of sparse data such as medical images: Medical image data exhibits strong spatial sparsity, meaning diagnostic features are often concentrated in small lesion areas. The dynamic masking mechanism of this invention can adaptively locate these anatomically or pathologically sensitive areas, perfectly aligning privacy protection strategies with the inherent sparse distribution of the data.

[0014] 3. Superior robustness against advanced reconstruction attacks: The non-uniform, optimized mask noise generated in this invention improves the non-convexity of the loss plane. Compared to existing FedEM or DP algorithms, this invention significantly increases the difficulty for attackers to reverse-engineer the original pixels when dealing with advanced attacks such as depth gradient leakage, demonstrating superior protection levels in privacy evaluation metrics such as Feature MSE and PSNR.

[0015] In summary, the method provided by this invention locates high-information-density regions through dynamic masking and combines projective gradient descent to accurately inject noise into local regions to defend against gradient leakage attacks during model training. This method can effectively protect the privacy of local training data (especially sparse data such as medical images) while maximizing the utility and accuracy of the global model. Attached Figure Description

[0016] Figure 1 A flowchart of a federated learning privacy protection method based on gradient-guided perturbation of key regions provided in an embodiment of the present invention. Detailed Implementation

[0017] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention. Furthermore, the technical features involved in the various embodiments of this invention described below can be combined with each other as long as they do not conflict with each other.

[0018] Existing federated learning privacy protection techniques fail to incorporate the spatial sparsity of data for differentiated defense, and cannot provide fine-grained security protection for key sensitive regions without significantly compromising model accuracy. Therefore, this invention provides a federated learning privacy protection method based on gradient-guided perturbation of key regions, comprising: S1, The central server initializes the global model parameters and sends them to the clients participating in the current round of training; S2, the client initializes the local model parameters to the global model parameters issued by the central server; S3, the client calculates the gradient matrix of the loss function with respect to the input sample superimposed with the current perturbation, and generates a binary mask matrix corresponding to the gradient matrix through the corresponding mask threshold. The gradient matrix and the binary mask matrix are multiplied element by element to obtain the gradient information that is masked, and the perturbation is updated through the gradient information; wherein, the input sample is a medical image, and the initial value of the perturbation is 0. S4, take the updated perturbation as the current perturbation, return to S3, until the maximum number of iterations is reached, take the latest perturbation as the optimal perturbation, and enter S5; S5, the client superimposes the optimal perturbation with the input sample to obtain protected data, and uses the protected data for forward and backward propagation to calculate the parameter gradient of the local model and upload it to the central server. S6, the central server performs a weighted summation of the parameter gradients uploaded by each client to obtain the global gradient, updates the global model in conjunction with the global learning rate, distributes the updated global model to each client participating in the next round of training, and returns to S2, until the maximum number of communication rounds, i.e. the maximum number of training rounds, is reached.

[0019] Specifically, such as Figure 1 As shown, the method provided in this embodiment of the invention is implemented based on a standard federated learning system architecture, which includes a central server and A local private dataset The heterogeneous clients. The core of the method provided in this embodiment of the invention lies in the "gradient-guided masking and local perturbation" process on the client's local machine. The entire single-round communication iteration process of secure training includes the following specific steps: (1) Model initialization and distribution In the first federal study Wheel (of which) , During the communication (total number of global communication rounds), the central server maintains the current global model parameters. The server selects a subset of clients. Participate in this round of training and set global model parameters The broadcast is sent to the selected clients. Upon receiving the broadcast, the clients will set their local model parameters. Initialize to .

[0020] Specifically, the first round of communication (i.e. When the central server initializes the global model parameters, it sends them to the selected clients participating in the current training round.

[0021] (2) Local data sampling and disturbance initialization In any of the selected clients ( The local training phase of the dataset is performed using a local private dataset. A small batch of training samples was sampled from (i.e., medical image datasets). Regarding this input data Initialize a defensive perturbation quantity Its dimensions are consistent with the input data.

[0022] (3) Generation of key region perturbations based on projection gradient descent (inner loop) In order to precisely inject noise into the critical areas with the highest information density, the client executes... The PGD iteration seeks optimization. In each internal iteration, the following sub-steps are executed: 1) Input gradient calculation: For the th In the next iteration, calculate the current loss function with respect to the input data (carrying the current perturbation). The gradient matrix of ) ,Right now . For gradient operators, For loss function, For the first The local model of each client, that is, the local model participating in federated learning, is usually a deep neural network model (such as ResNet-18).

[0023] 2) Dynamic mask generation: Based on preset sparsity ratio parameters (For example ), calculate the absolute value of the gradient matrix Middle position in front Percentile threshold A binary mask matrix is ​​generated based on this threshold. For matrices whose absolute values ​​are greater than or equal to For each element, set its corresponding mask position to 1 (i.e., the critical region); otherwise, set it to 0.

[0024] 3) Mask filtering: This involves filtering the binary mask matrix. With gradient matrix Perform element-wise multiplication to extract the filtering gradient containing only key region information. .

[0025] 4) Local perturbation update and projection: The extracted filter gradient indicates the perturbation update direction, and the update formula is: ,in, For the perturbation step size, For symbolic functions, This indicates a projection operation, ensuring the updated perturbation. The norm is strictly limited to the boundary between security and utility. Inside.

[0026] In step (3), the binary mask matrix is... With gradient plot Element-wise multiplication yields the gradient information masked by the mask. Then, the projected gradient descent (PGD) algorithm is used, according to a preset perturbation step size. The perturbation is updated iteratively in multiple steps. After each update, a projection operation is performed. This ensures that the updated perturbation is strictly limited within the preset safety boundary. Within, the process involves repeated steps to generate the final locally optimized noise.

[0027] (4) Local noise injection and local model update go through After several iterations, the final optimized perturbation (i.e., the optimal perturbation) for the key region is obtained. The client then overlays this data with the original input to generate the final protected data. Subsequently, the client uses this protected data to perform standard forward and backward propagation, calculating the loss function with respect to the model parameters. gradient .

[0028] (5) Local parameter uploading and global security aggregation Each selected client will contain parameter gradients that include locally protected information. Uploaded to the central server. The server does not need to decrypt or access any raw image data; it directly adjusts the upload size based on the local dataset size of each client. Calculate aggregate weights . Let be the size of the local dataset for the j-th client. Then, the server calculates the gradients of the collected parameters. We perform a weighted summation to obtain the global gradient. and with global learning rate Update the global model: .

[0029] Repeat steps one through five continuously until the process is complete. The global model converges after round-robin communication.

[0030] This invention provides a federated learning system, including a central server and a client, which are used to execute a federated learning privacy protection method based on gradient-guided key region perturbation as described in any of the above embodiments.

[0031] This invention provides a medical image classification method for a client, comprising: classifying the medical image to be classified using a trained local model; The local model is trained using a federated learning privacy-preserving method based on gradient-guided perturbation of key regions, as described in any of the above embodiments.

[0032] This invention provides a client, including a memory and a processor. The memory stores a computer program, and the processor executes the computer program to perform the medical image classification method as described in any of the above embodiments.

[0033] This invention provides a computer-readable storage medium, characterized in that the computer-readable storage medium stores computer instructions, which are used to cause a processor to execute either the gradient-guided key region perturbation-based federated learning privacy protection method as described in any of the above embodiments or the medical image classification method as described in any of the above embodiments.

[0034] Those skilled in the art will readily understand that the above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A privacy-preserving method for federated learning based on gradient-guided perturbation of key regions, characterized in that, include: S1, The central server initializes the global model parameters and sends them to the clients participating in the current round of training; S2, the client initializes the local model parameters to the global model parameters issued by the central server; S3, the client calculates the gradient matrix of the loss function with respect to the input sample superimposed with the current perturbation, and generates a binary mask matrix corresponding to the gradient matrix through the corresponding mask threshold. The gradient matrix and the binary mask matrix are multiplied element by element to obtain the gradient information that is masked, and the perturbation is updated through the gradient information; wherein, the input sample is a medical image, and the initial value of the perturbation is 0. S4, take the updated perturbation as the current perturbation, return to S3, until the maximum number of iterations is reached, take the latest perturbation as the optimal perturbation, and enter S5; S5, the client superimposes the optimal perturbation with the input sample to obtain protected data, and uses the protected data for forward and backward propagation to calculate the parameter gradient of the local model and upload it to the central server. S6, the central server performs a weighted summation of the parameter gradients uploaded by each client to obtain the global gradient, updates the global model in conjunction with the global learning rate, distributes the updated global model to each client participating in the next round of training, and returns to S2, until the maximum number of communication rounds is reached.

2. The method as described in claim 1, characterized in that, No. The first client In the next iteration, the formula for calculating the gradient matrix of the loss function with respect to the input sample superimposed with the current perturbation is: in, For the first The first client The gradient matrix at the next iteration For the current disturbance, i.e. the th The first client Perturbation during the next iteration For the first Input samples from one client, for Category tags, This represents taking the partial derivative with respect to the input image x. For loss function, For the first The local model of a client predicts the output of a noisy input.

3. The method as described in claim 2, characterized in that, No. The first client In the next iteration, the update formula for the perturbation is: in, For the first The first client The perturbation after the next iteration update. The preset perturbation step size, For symbolic functions, This indicates a projection operation. They are respectively The upper and lower limits of the safety boundary.

4. The method as described in claim 1, characterized in that, No. The first client In the next iteration, the corresponding mask threshold is obtained as follows: Based on the preset sparsity ratio parameters calculate Middle position in front Percentile threshold , and use it as the corresponding mask threshold; in, For the first The first client The gradient matrix at the next iteration The absolute value of.

5. The method as described in claim 1, characterized in that, No. During round-robin communication, the formula for calculating the global gradient is: in, For the first Global gradient during round-robin communication; For the first The aggregate weight of each client, , For the first Local datasets for each client Size, For the first Local datasets for each client Size; For the first The first client The gradient of parameters uploaded during round-robin communication; To participate in the A collection of clients for round-robin communication.

6. A federated learning system, characterized in that, include: A central server and a client are used to execute the federated learning privacy protection method based on gradient-guided key region perturbation as described in any one of claims 1-5.

7. A medical image classification method, characterized in that, For client-side applications, this includes classifying medical images to be classified using a pre-trained local model. The local model is trained using the federated learning privacy-preserving method based on gradient-guided key region perturbation as described in any one of claims 1-5.

8. A client application, characterized in that, include: A memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to perform the medical image classification method of claim 7.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions for causing a processor to execute the federated learning privacy protection method based on gradient-guided key region perturbation as described in any one of claims 1-6 or the medical image classification method as described in claim 7.

Images