Federated learning gradient attack defense method, system, device and medium
By using gradient masking matrices and noise reduction methods in federated learning, the gradient inversion problem is solved, improving model accuracy and achieving effective privacy protection.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- XI AN JIAOTONG UNIV
- Filing Date
- 2022-05-24
- Publication Date
- 2026-07-31
AI Technical Summary
Even after gradient compression, existing federated learning algorithms can still be exploited by malicious attackers to back-calculate the training sets of other participants through gradients, and existing methods are prone to reducing model training accuracy.
The local gradient is compressed using a gradient mask matrix, and noise is added to the updated gradient mask matrix. Combined with differential privacy technology, the aggregation algorithm is performed using the noisy gradients to control the amount of noise and improve model accuracy.
It effectively prevents gradient attacks, improves the training accuracy of the model, and achieves effective gradient compression and privacy protection.
Smart Images
Figure CN115222057B_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of artificial intelligence technology, and specifically relates to a federated learning gradient attack defense method, system, device and medium. Background Technology
[0002] In recent years, deep learning has developed rapidly and has been widely applied in fields such as speech recognition, computer vision, and recommendation systems. With the development of the Internet of Things and the continuous improvement of the performance of various terminal devices, massive amounts of data are generated in daily life. While solving the data shortage problem, data privacy and security issues have also arisen. Against this backdrop, federated learning has emerged. Federated learning is essentially a distributed machine learning framework that allows participating parties to jointly model and improve model performance without sharing training set data. To reduce the impact of bandwidth on the federated learning training process, participants typically compress gradients before uploading their local gradients. Gradient compression reduces bandwidth while also providing a certain degree of privacy protection.
[0003] Even after gradient compression, current federated learning algorithms can still be exploited by malicious attackers to back-engineer the training sets of other participants. Currently, most methods address gradient attack issues by adding noise to the uploaded gradients. For example: 1. Adding noise to the gradient before compression. The problem with this is that the gradient value changes after adding noise, making it easy to compress important gradients during compression; 2. Compressing the gradient before adding noise. The problem with this is that adding noise also adds noise to the compressed locations, rendering the gradient compression ineffective. Therefore, all of these methods easily lead to reduced model training accuracy. Summary of the Invention
[0004] In view of the technical problems existing in the prior art, the present invention provides a federated learning gradient attack defense method, system, device and medium to solve the technical problem that the existing method of adding noise to the training gradient cannot effectively prevent attackers from back-deriving training data from the training gradient, and easily causes a decrease in model training accuracy.
[0005] To achieve the above objectives, the technical solution adopted by the present invention is as follows:
[0006] This invention discloses a federated learning gradient attack defense method, the method comprising:
[0007] The model is trained using local data to obtain a local model, and the local gradient is calculated.
[0008] The compression threshold of federated learning is obtained, and the gradient mask matrix is calculated.
[0009] Based on the gradient mask matrix, the local gradient is compressed to obtain the compressed gradient;
[0010] The gradient mask matrix is updated to obtain the updated gradient mask matrix;
[0011] Noise is added to the updated gradient mask matrix to obtain a noisy gradient mask matrix.
[0012] Based on the compressed gradient and the gradient mask matrix with added noise, the gradient with added noise is obtained;
[0013] For the gradient with added noise, an aggregation algorithm is performed to obtain the global gradient;
[0014] The local model is updated based on the global gradient, and the next round of model training begins.
[0015] Furthermore, the process of training the model using local data to obtain a local model is as follows:
[0016] The number of clients participating in federated learning is determined by the central server involved in federated learning, and the hyperparameters for federated learning and local client training are set.
[0017] The model is initialized by the central server participating in federated learning, the model is sent to all clients participating in federated learning, and the clients participating in the current training round are determined.
[0018] By selecting the client to participate in the current training round, the model is trained using local data to obtain a local model.
[0019] Furthermore, the process of obtaining the compression threshold of federated learning and calculating the gradient mask matrix is as follows:
[0020] Using the client participating in federated learning, calculate the compression threshold for the current training round of federated learning;
[0021] The gradient mask matrix for the current training round is calculated based on the compression threshold of the current round of federated learning.
[0022] The process of compressing the local gradient based on the gradient mask matrix to obtain the compressed gradient is as follows:
[0023] The compressed gradient is obtained by performing a dot product operation on the gradient mask matrix and the local gradient.
[0024] Furthermore, the process of updating the gradient mask matrix to obtain the updated gradient mask matrix is as follows:
[0025] Using the central server participating in federated learning, the gradient mask matrices uploaded by all clients participating in federated learning are summed, and a global mask matrix is calculated based on a preset global threshold for the gradient mask matrix.
[0026] The gradient mask matrix is updated based on the global mask matrix to obtain the updated gradient mask matrix.
[0027] Furthermore, in the process of adding noise to the updated gradient mask matrix to obtain a noisy gradient mask matrix, the noise is Laplace noise or Gaussian noise.
[0028] Furthermore, the global gradient is:
[0029]
[0030] in, For global gradient; Add noisy gradients to the data uploaded by the i-th client; N is the number of clients participating in federated learning.
[0031] Furthermore, the process of updating the local model based on the global gradient is as follows:
[0032]
[0033] Among them, W t+1 For the updated local model; W t η represents the local model in the current round; η is the learning rate; and t is the current round of federated learning.
[0034] This invention also provides a federated learning gradient attack defense system, comprising:
[0035] The local training module is used to train the model using local data, obtain a local model, and calculate the local gradient.
[0036] The gradient mask matrix module is used to obtain the compression threshold of federated learning and calculate the gradient mask matrix.
[0037] The gradient compression module is used to compress the local gradient according to the gradient mask matrix to obtain the compressed gradient.
[0038] The gradient mask matrix update module is used to update the gradient mask matrix to obtain the updated gradient mask matrix.
[0039] A noise module is used to add noise to the updated gradient mask matrix to obtain a noisy gradient mask matrix; and to obtain a noisy gradient based on the compressed gradient and the noisy gradient mask matrix.
[0040] The gradient aggregation module is used to perform an aggregation algorithm on the added noisy gradients to obtain the global gradient;
[0041] The local update module is used to update the local model based on the global gradient and start the next round of model training.
[0042] This invention also provides a federated learning gradient attack defense device, comprising:
[0043] Memory, used to store computer programs;
[0044] A processor is used to implement the steps of the federated learning gradient attack defense method when executing the computer program.
[0045] The present invention also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps of the federated learning gradient attack defense method.
[0046] Compared with the prior art, the beneficial effects of the present invention are as follows:
[0047] This invention provides a federated learning gradient attack defense method and system. By adding noise to the updated gradient mask matrix and combining it with differential privacy technology, the added noisy gradients can effectively prevent gradient attacks. By performing an aggregation algorithm on the added noisy gradients, a global matrix is calculated based on a preset global threshold of the gradient mask matrix, thereby controlling the amount of added noise and effectively improving the accuracy of the model. Attached Figure Description
[0048] Figure 1 This is a flowchart of the federated learning gradient attack defense method described in the embodiment;
[0049] Figure 2 This is a structural diagram of the federated learning framework described in the embodiment. Detailed Implementation
[0050] To make the technical problems solved by the present invention, the technical solutions, and the beneficial effects clearer, the following specific embodiments provide a further detailed description of the present invention. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.
[0051] This invention provides a method for defending against federated learning gradient attacks, comprising the following steps:
[0052] Step 1: Train the model using local data to obtain a local model and calculate the local gradient; the specific process is as follows:
[0053] Step 11: Determine the number of clients participating in federated learning and set the hyperparameters for federated learning and local client training through the central server participating in federated learning.
[0054] Step 12: Initialize the model through the central server participating in federated learning, send the model to all clients participating in federated learning, and determine the clients participating in the current training round.
[0055] Step 13: Using the selected client to participate in the current training round, train the model with local data to obtain the local model and calculate the local gradient.
[0056] Step 2: Obtain the compression threshold for federated learning and calculate the gradient mask matrix; compress the local gradient according to the gradient mask matrix to obtain the compressed gradient;
[0057] The specific process is as follows:
[0058] Using the client participating in federated learning, calculate the compression threshold for the current round of federated learning; based on the compression threshold for the current round of federated learning, calculate the gradient mask matrix for the current training round.
[0059] The compressed gradient is obtained by performing a dot product operation on the gradient mask matrix and the local gradient.
[0060] In this invention, a matrix multiplication operation is performed between the gradient mask matrix and the local gradient to compress the local gradient, effectively improving the compression efficiency of the local gradient; in addition, the compressed gradient is used for global gradient updates.
[0061] Step 3: Update the gradient mask matrix to obtain the updated gradient mask matrix.
[0062] The update process is as follows:
[0063] Using the central server participating in federated learning, the gradient mask matrices uploaded by all participating clients are summed, and a global mask matrix is calculated based on a preset global threshold for the gradient mask matrix. The gradient mask matrix is then updated based on the global mask matrix to obtain the updated gradient mask matrix.
[0064] Step 4: Add noise to the updated gradient mask matrix to obtain a gradient mask matrix with added noise; preferably, the noise is Laplace noise or Gaussian noise.
[0065] Step 5: Obtain the noisy gradient based on the compressed gradient and the noisy gradient mask matrix; wherein the noisy gradient is obtained by adding the compressed gradient and the noisy gradient mask matrix.
[0066] Step 6: Perform an aggregation algorithm on the added noisy gradient to obtain the global gradient; wherein the global gradient is:
[0067]
[0068] in, For global gradient; Add noisy gradients to the data uploaded by the i-th client; N is the number of clients participating in federated learning.
[0069] Step 6: Update the local model based on the global gradient and begin the next round of model training; wherein, the updated local model is:
[0070]
[0071] Among them, W t+1 For the updated local model; W t η represents the local model in the current round; η is the learning rate; and t is the current round of federated learning.
[0072] This invention also provides a federated learning gradient attack defense system, including a local training module, a gradient mask matrix module, a gradient compression module, a gradient mask matrix update module, a noise module, a gradient aggregation module, and a local update module. The local training module trains the model using local data to obtain a local model and calculates the local gradient. The gradient mask matrix module obtains a compression threshold for federated learning and calculates the gradient mask matrix. The gradient compression module compresses the local gradient based on the gradient mask matrix to obtain a compressed gradient. The gradient mask matrix update module updates the gradient mask matrix to obtain an updated gradient mask matrix. The noise module adds noise to the updated gradient mask matrix to obtain a noisy gradient mask matrix; and obtains a noisy gradient based on the compressed gradient and the noisy gradient mask matrix. The gradient aggregation module performs an aggregation algorithm on the noisy gradient to obtain the global gradient. The local update module updates the local model based on the global gradient and starts the next round of model training.
[0073] The present invention also provides a federated learning gradient attack defense device, comprising: a memory for storing a computer program; and a processor for executing the computer program to implement the steps of the federated learning gradient attack defense method.
[0074] When the processor executes the computer program, it implements the steps of the federated learning gradient attack defense method described above, for example: training the model using local data to obtain a local model and calculating the local gradient; obtaining the compression threshold of federated learning and calculating the gradient mask matrix; compressing the local gradient according to the gradient mask matrix to obtain the compressed gradient; updating the gradient mask matrix to obtain the updated gradient mask matrix; adding noise to the updated gradient mask matrix to obtain a noisy gradient mask matrix; obtaining a noisy gradient according to the compressed gradient and the noisy gradient mask matrix; performing an aggregation algorithm on the noisy gradient to obtain the global gradient; updating the local model according to the global gradient and starting the next round of model training.
[0075] Alternatively, when the processor executes the computer program, it implements the functions of each module in the above system, such as: a local training module for training the model using local data to obtain a local model and calculating the local gradient; a gradient mask matrix module for obtaining the compression threshold of federated learning and calculating the gradient mask matrix; a gradient compression module for compressing the local gradient according to the gradient mask matrix to obtain the compressed gradient; a gradient mask matrix update module for updating the gradient mask matrix to obtain the updated gradient mask matrix; a noise module for adding noise to the updated gradient mask matrix to obtain a noisy gradient mask matrix; and obtaining a noisy gradient based on the compressed gradient and the noisy gradient mask matrix; a gradient aggregation module for performing an aggregation algorithm on the noisy gradient to obtain the global gradient; and a local update module for updating the local model according to the global gradient and starting the next round of model training.
[0076] For example, the computer program can be divided into one or more modules / units, which are stored in the memory and executed by the processor to complete the present invention. The one or more modules / units can be a series of computer program instruction segments capable of performing preset functions, wherein the instruction segments describe the execution process of the computer program in the federated learning gradient attack defense device. For example, the computer program can be divided into: a local training module, a gradient mask matrix module, a gradient compression module, a gradient mask matrix update module, a noise module, a gradient aggregation module, and a local update module. The specific functions of each module are as follows: The local training module trains the model using local data to obtain a local model and calculates the local gradient; the gradient mask matrix module obtains the compression threshold for federated learning and calculates the gradient mask matrix; the gradient compression module compresses the local gradient based on the gradient mask matrix to obtain a compressed gradient; the gradient mask matrix update module updates the gradient mask matrix to obtain an updated gradient mask matrix; the noise module adds noise to the updated gradient mask matrix to obtain a noisy gradient mask matrix; and obtains a noisy gradient based on the compressed gradient and the noisy gradient mask matrix; the gradient aggregation module performs an aggregation algorithm on the noisy gradient to obtain the global gradient; and the local update module updates the local model based on the global gradient and starts the next round of model training.
[0077] The federated learning gradient attack defense device can be a computing device such as a desktop computer, laptop, handheld computer, or cloud server. The federated learning gradient attack defense device may include, but is not limited to, processors and memory. Those skilled in the art will understand that the above are merely examples of federated learning gradient attack defense devices and do not constitute a limitation on the device. It may include more components than described above, or combine certain components, or use different components. For example, the federated learning gradient attack defense device may also include input / output devices, network access devices, buses, etc.
[0078] The processor referred to can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor, or any conventional processor. This processor is the control center of the federated learning gradient attack defense device, connecting all parts of the device via various interfaces and lines.
[0079] The memory can be used to store the computer program and / or module. The processor implements various functions of the federated learning gradient attack defense device by running or executing the computer program and / or module stored in the memory and calling the data stored in the memory.
[0080] The memory may primarily include a program storage area and a data storage area. The program storage area may store the operating system and at least one application program required for a given function (such as sound playback or image playback). The data storage area may store data created based on the use of the phone (such as audio data or a phonebook). Furthermore, the memory may include high-speed random access memory (RAM) and non-volatile memory, such as hard disks, RAM, plug-in hard disks, SmartMediaCards (SMC), Secure Digital (SD) cards, flash cards, at least one disk storage device, flash memory device, or other volatile solid-state storage devices.
[0081] The present invention also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps of the federated learning gradient attack defense method.
[0082] If the modules / units integrated in the federated learning gradient attack defense system are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium.
[0083] Based on this understanding, the present invention can implement all or part of the processes in the above-described federated learning gradient attack defense method, or it can be accomplished by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the above-described federated learning gradient attack defense method. The computer program includes computer program code, which can be in the form of source code, object code, executable file, or a preset intermediate form, etc.
[0084] The computer-readable storage medium may include: any entity or device capable of carrying the computer program code, recording media, USB flash drive, portable hard drive, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signal, telecommunication signal, and software distribution medium, etc.
[0085] It should be noted that the content contained in the computer-readable storage medium may be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, the computer-readable storage medium does not include electrical carrier signals and telecommunication signals.
[0086] Example
[0087] As attached Figure 1-2 As shown, this embodiment provides a federated learning gradient attack defense method, which specifically includes the following steps:
[0088] Step 101: Using the central server participating in federated learning, determine the number of clients participating in federated learning and set the hyperparameters for federated learning and local client training.
[0089] In this embodiment, the number of clients participating in federated learning is N; the hyperparameters of federated learning include: the maximum number of learning epochs T and the proportion of clients participating in training in each training epoch frac; the hyperparameters of client-side local training include the number of local training epochs local_epoch, the local training batch size local_batchsize, the gradient compression ratio cr, the learning rate η, and the privacy budget ε.
[0090] Step 102: Initialize the model through the central server participating in federated learning and send the model to all clients participating in federated learning; wherein, the model can be any machine learning model, such as a ResNet model, a VGG model, a GBDT model, or a logistic regression model.
[0091] Step 103: Based on the proportion of clients participating in training in each training round (frac), use the central server participating in federated learning to determine the clients randomly selected to participate in the current training round; wherein, the clients participating in the current training round are used for local training in the current training round.
[0092] It should be noted that in this embodiment, a uniform distribution is used to select clients participating in the current training round, so that all clients participating in federated learning have the same probability of being selected; and clients that are not selected do not participate in the training of the current training round.
[0093] Step 104: Using the selected client to participate in the current training round, train the model with local data to obtain the local model, and calculate the local gradient of the current training round.
[0094] Since the selected client participating in the current training epoch often has a local training epoch number (local_epoch) greater than 1 when training the model using local data, and existing deep learning frameworks do not support outputting gradients after training for multiple local training epochs (local_epoch), this embodiment records the model parameters before training and subtracts the model parameters before training from the model parameters after local training to obtain the local gradient of the current training epoch.
[0095] The local gradient of the current training round is:
[0096]
[0097] in, This is the local gradient for the nth client participating in the current training round.
[0098] Step 105: Obtain the compression threshold for federated learning and calculate the gradient mask matrix; compress the local gradient matrix according to the gradient mask matrix to obtain the compressed gradient; wherein, the gradient mask matrix is a Boolean matrix indicating the distribution of the compressed gradient positions; the specific process is as follows:
[0099] Using the clients participating in federated learning, calculate the compression threshold for the current training epoch of federated learning; wherein, the compression threshold for the current training epoch of federated learning is:
[0100]
[0101] Where thr is the compression threshold for the current training epoch of federated learning; cr is the gradient compression rate; For the local gradient of the i-th client participating in the current training round; This is the gradient value at the j-th position after flattening the local gradient of the i-th client participating in the current training round.
[0102] Based on the compression threshold thr of the current round of federated learning, the gradient mask matrix M of the i-th client in the current training round is calculated. i Wherein, the gradient mask matrix M of the i-th client participating in the current training round i for:
[0103]
[0104] Among them, M i This is the gradient mask matrix for the i-th client participating in the current training round.
[0105] In this embodiment, when the j-th gradient of the i-th client participating in the current training round is greater than the compression threshold thr of the current round of federated learning, M i [j] is set to 1; if the j-th gradient of the i-th client participating in the current training round is less than the compression threshold thr of the current round of federated learning, M i [j] is set to 0; where M i [j] represents the j-th gradient of the i-th client participating in the current training round.
[0106] The local gradient is compressed based on the gradient mask matrix to obtain the compressed gradient.
[0107] The compressed gradient is:
[0108]
[0109] in, Let be the compressed gradient for the i-th client. M is the local gradient of the i-th client; i Let be the gradient mask matrix of the i-th client participating in the current training round; ⊙ represents the dot product operation of the matrix.
[0110] S106. Using the central server participating in federated learning, collect the gradient mask matrices uploaded by all clients participating in federated learning, add the gradient mask matrices uploaded by all clients participating in federated learning, and calculate the global mask matrix according to the preset global threshold of the gradient mask matrix; update the gradient mask matrix according to the global mask matrix to obtain the updated gradient mask matrix.
[0111] The specific process is as follows:
[0112] All clients participating in federated learning upload the gradient mask matrix {M1,M2,...,M...} n}; where M n Let be the gradient mask matrix of the nth client participating in federated learning;
[0113] Using the central server participating in federated learning, the gradient mask matrices uploaded by all participating clients are summed, and then calculated according to the preset gradient M. G ←M G The global threshold of the [j]>s mask matrix is used to calculate the global mask matrix;
[0114] The global mask matrix is as follows:
[0115]
[0116] M G ←M G [j]>s
[0117] Among them, M G M is the global mask matrix; i Let M be the local gradient mask matrix of the i-th client; n is the number of clients participating in this round of training; M G [j] represents the value at the j-th position after the global mask matrix is flattened.
[0118] The global mask matrix is sent back to the client. The client updates the gradient mask matrix based on the global mask matrix to obtain the updated gradient mask matrix; wherein, the updated gradient mask matrix is:
[0119] M i ′=M i ∩M G
[0120] Among them, M i M is the local gradient mask matrix for the i-th client; i ′ is the updated local gradient mask matrix for the i-th client; M G This is the global mask matrix.
[0121] In this embodiment, M[j] is not zero only when more than s participants have non-zero elements at position M[j]. When s is larger, the model accuracy is higher, but the protection effect is worse. When s is smaller, the model accuracy will decrease, but the data protection effect will improve.
[0122] S107. Add noise to the updated gradient mask matrix to obtain a noisy gradient mask matrix; obtain a noisy gradient based on the compressed gradient and the noisy gradient mask matrix.
[0123] The specific process is as follows:
[0124] First, noise is added to the updated gradient mask matrix according to the following formula:
[0125] M i ′[j]=M i [j]+Y,M i [j]≠0
[0126]
[0127] Where Δf is the sensitivity; ε is the privacy budget, the smaller ε is, the greater the noise, the worse the usability of the result, and the better the privacy protection effect; L(·) is the Laplace distribution, when At that time, the noise satisfies (ε,0)-dp; N(·) is a Gaussian distribution, for any The noise satisfies (ε,δ)-dp.
[0128] Then, the compressed gradient is added to the gradient mask matrix with added noise to obtain the gradient with added noise; and the gradient with added noise is uploaded to the central server participating in federated learning.
[0129] The gradient with added noise is:
[0130]
[0131] in, Add a noisy gradient to the i-th client; M represents the compressed gradient for the i-th client; i Let ' be the updated local gradient mask matrix for the i-th client;
[0132] S108. For the gradient with added noise, perform an aggregation algorithm to obtain the global gradient.
[0133] The global gradient is:
[0134]
[0135] in, For global gradient; Add noisy gradients to the data uploaded by the i-th client; N is the number of clients participating in federated learning.
[0136] The specific process is as follows:
[0137] The client uploads local gradients as follows in, The i-th client uploads a noisy gradient; after receiving the noisy gradients uploaded by all clients, the central server participating in federated learning calculates the global gradient using the following formula. Others are as follows:
[0138]
[0139] in, For global gradient; Add noisy gradients to the data uploaded by the i-th client; N is the number of clients participating in federated learning.
[0140] S109. Use global gradients to update the model locally on the client side.
[0141] The client updates the model locally, and the specific process is as follows:
[0142] The central server will implement global gradients. The data is then sent back to all clients, who update the model according to the following formula:
[0143]
[0144] Among them, W t+1 For the updated local model; W t η represents the local model in the current round; η is the learning rate; and t is the current round of federated learning.
[0145] S110. Determine whether the model has converged or reached the maximum number of training rounds.
[0146] S111. If the model has converged or reached the maximum learning rounds of federated learning, output the final trained model; if the model has not converged or has not reached the maximum learning rounds of federated learning, return to S103 to continue training.
[0147] The federated learning gradient attack defense method described in this embodiment is applicable to multiple federated learning fields such as computer vision and the Internet of Things. It solves the problem that existing federated learning frameworks cannot prevent attackers from back-deriving training data from training gradients. The method can effectively improve the accuracy of the model while ensuring that the original data is not attacked by inference. Compared with other federated learning algorithms, it can effectively improve the accuracy of the model while preventing gradient attacks.
[0148] The description of relevant parts of the federated learning gradient attack defense system, device and computer-readable storage medium provided in this embodiment can be found in the detailed description of the corresponding parts of the federated learning gradient attack defense method described in this embodiment, and will not be repeated here.
[0149] The above embodiments are merely one of the implementation methods for achieving the technical solution of the present invention. The scope of protection claimed by the present invention is not limited to this embodiment, but also includes any variations, substitutions and other implementation methods that can be easily conceived by those skilled in the art within the scope of the technology disclosed in the present invention.
Claims
1. A federated learning gradient attack defense method, characterized in that, The methods include: The model is trained using local data to obtain a local model, and the local gradient is calculated. The compression threshold of federated learning is obtained, and the gradient mask matrix is calculated. Based on the gradient mask matrix, the local gradient is compressed to obtain the compressed gradient; The gradient mask matrix is updated to obtain the updated gradient mask matrix; Noise is added to the updated gradient mask matrix to obtain a gradient mask matrix with added noise. Based on the compressed gradient and the gradient mask matrix with added noise, the gradient with added noise is obtained; For the added noisy gradient, an aggregation algorithm is performed to obtain the global gradient; Based on the global gradient, the local model is updated, and the next round of model training begins; The process of obtaining the compression threshold for federated learning and calculating the gradient mask matrix is as follows: Using the client participating in federated learning, calculate the compression threshold for the current training round of federated learning; The gradient mask matrix for the current training round is calculated based on the compression threshold of the current round of federated learning. The process of compressing the local gradient based on the gradient mask matrix to obtain the compressed gradient is as follows: The compressed gradient is obtained by performing a dot product operation on the gradient mask matrix and the local gradient. The process of updating the gradient mask matrix to obtain the updated gradient mask matrix is as follows: Using the central server participating in federated learning, the gradient mask matrices uploaded by all clients participating in federated learning are summed, and a global mask matrix is calculated based on a preset global threshold for the gradient mask matrix. The gradient mask matrix is updated based on the global mask matrix to obtain the updated gradient mask matrix.
2. The federated learning gradient attack defense method of claim 1, wherein, The process of training the model using local data to obtain a local model is as follows: The number of clients participating in federated learning is determined by the central server involved in federated learning, and the hyperparameters for federated learning and local client training are set. The model is initialized by the central server participating in federated learning, the model is sent to all clients participating in federated learning, and the clients participating in the current training round are determined. By selecting clients to participate in the current training round, the model is trained using local data to obtain a local model.
3. The federated learning gradient attack defense method according to claim 1, characterized in that, In the process of adding noise to the updated gradient mask matrix to obtain a noisy gradient mask matrix, the noise is Laplace noise or Gaussian noise.
4. The federated learning gradient attack defense method according to claim 1, characterized in that, The global gradient is: in, For global gradient; For the first Noisy gradients uploaded by each client; The number of clients participating in federated learning.
5. The federated learning gradient attack defense method according to claim 1, characterized in that, The process of updating the local model based on the global gradient is as follows: in, For the updated local model; This is the local model for the current round; The learning rate; The current round of federated learning.
6. A federated learning gradient attack defense system, characterized in that, To implement the federated learning gradient attack defense method as described in any one of claims 1-5, comprising: The local training module is used to train the model using local data, obtain a local model, and calculate the local gradient. The gradient mask matrix module is used to obtain the compression threshold of federated learning and calculate the gradient mask matrix. The gradient compression module is used to compress the local gradient according to the gradient mask matrix to obtain the compressed gradient. The gradient mask matrix update module is used to update the gradient mask matrix to obtain the updated gradient mask matrix. A noise module is used to add noise to the updated gradient mask matrix to obtain a noisy gradient mask matrix; and to obtain a noisy gradient based on the compressed gradient and the noisy gradient mask matrix. The gradient aggregation module is used to perform an aggregation algorithm on the added noisy gradients to obtain the global gradient; The local update module is used to update the local model based on the global gradient and start the next round of model training.
7. A federated learning gradient attack defense device, characterized in that, include: Memory, used to store computer programs; A processor, configured to implement the steps of the federated learning gradient attack defense method as described in any one of claims 1-5 when executing the computer program.
8. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the steps of the federated learning gradient attack defense method as described in any one of claims 1-5.