Privacy protection method based on data enhancement in federated learning

Abstract

The invention discloses a privacy protection method based on data enhancement in federated learning, and belongs to the field of federated learning privacy protection, and the method comprises the following steps: firstly, a client participating in federated learning training processes a data set by using a selected data enhancement method before local training; then the central server initializes all parameters and issues the parameters to each client to start training; the clients participating in training in each round acquire global model parameters, calculate gradients after training each batch, update the parameters, cut the parameters and add differential disturbance noise; and after each round is finished, the server aggregates, updates global parameters, broadcasts the global parameters to the client, and circulates until the model converges. According to the method, an existing differential privacy-based privacy protection method of federal learning is mainly improved, and a data enhancement method is added to better balance model availability and privacy protection capability. The data enhancement method comprises the steps of SaliencyMix, a basic data enhancement strategy, combination of the SaliencyMix and the basic data enhancement strategy and the like.

Description

Technical Field

[0001] The present invention relates to the technical field of privacy protection based on data enhancement in federated learning, and more particularly to a privacy protection method based on the combination of data enhancement and differential privacy. Background Art

[0002] Google first proposed the concept of a federated learning framework in 2016. Federated learning is a distributed machine learning framework that allows participants to download the latest model parameters from the cloud, update the model parameters using local data, and send the obtained parameters back to the server. When the model converges, the server aggregates the updated model parameters and sends them to the participants for training, and so on. This can maximize the use of the storage space and computing power of the participants on the one hand; on the other hand, since the information of the participants is stored locally, only the model parameters need to be shared, ensuring the privacy of the participants.

[0003] However, although the federated learning model training enables multiple users to jointly train high-quality models through gradients as a medium while ensuring the privacy of their own data, the federated learning model must continuously optimize the model parameters iteratively during the training process to achieve the expected effect. In this process, the gradient is indispensable for backpropagation of parameters to adjust and optimize the parameters, which indicates that the gradient carries a large amount of information about the original training data. Therefore, some attack methods that infer training data information through gradients have been proposed, which causes the leakage of user training data and threatens the security of training models such as federated learning. Common privacy protection methods still have limitations: differential privacy methods cannot balance model accuracy and security; secure multi-party computing requires collaboration between participants for calculations and operations; homomorphic encryption requires a trusted third party to complete encryption / decryption operations, which requires a large amount of computing resources and has high storage overhead; methods other than cryptography, such as using GAN and other technologies, will have more additional communication overhead. Relatively speaking, some solutions based on differential privacy require fewer resources to implement, have lower computational complexity, and are better, but the problem that it is difficult to balance privacy protection strength and model availability needs to be improved. Summary of the invention

[0004] The present invention is used to solve the problem that the protection method based on differential privacy in federated learning is difficult to take into account both the privacy protection strength and the model availability. Specifically, a federated learning privacy protection method based on data enhancement is proposed, which performs data enhancement processing on the local data of the client participating in the federated learning training, and uses differential privacy to add noise to the model parameters during the process of uploading the model on the client, so as to ensure the defense capability in the federated learning training process, prevent attackers from obtaining training data through gradient reconstruction, and improve the model performance.

[0005] To achieve the above purpose, the technical solution adopted by the present invention is based on the combination of data enhancement and differential privacy. Its main process is shown in the figure. The implementation steps of the method are as follows:

[0006] Step (1) Each client participating in federated learning first locally performs data enhancement on the data to be used for federated learning training, and may use a method selected from the basic data enhancement strategy, the SaliencyMix method, or a method combining the basic data enhancement and the SaliencyMix method to perform the processing;

[0007] Step (2) The central server initializes the global model and randomly initializes the model parameter w0, specifies the privacy budget ∈ and relaxation factor δ required for differential privacy, the proportion of clients participating in local training in each round is C, the number of rounds that the participating clients need to train the local data set in each round of local training is E, the sample size of each round of training is batch, the learning rate is η, and the number of clients participating in local training in each round is N = C·K. These N clients can be indexed by i, that is, i∈{1,2,…,N};

[0008] Step (3) Prepare to start the tth round of training. The central server randomly selects N clients to participate in this round of training. If this round is the first round, that is, t = 1, the initialized global model obtained in step (2) and the randomly initialized model parameter w0 are distributed to the clients participating in the training; if this round is not the first round, that is, t ≠ 1, the global model parameter w0 obtained in the previous round is t Distribute to clients participating in training;

[0009] Step (4) N clients participating in the training receive the global model and model parameters w0 or w0 sent by the central server. t , that is, the model parameters of client i are or And use the data after data enhancement in step (1) to perform local model training;

[0010] Step (5) The i-th client divides the data set into n batches according to the batch size. i / batch batches, each completes n i / batch training is an epoch, and a total of E rounds are completed locally, that is, training The i-th client participating in this round of training will use the model parameters obtained from this E round of training Upload to the server;

[0011] Step (6) The central server receives the model updates of all local devices and calculates the model updates according to the formula Perform model aggregation, where n irepresents the amount of data contained in client i participating in the t-th round of local training, and n represents the total amount of data contained in all clients participating in the t-th round of local training;

[0012] Step (7) The server broadcasts the updated model parameters w t to the clients, and the clients test the performance of the updated model;

[0013] Step (8) Loop steps (3) to (7) until the global model reaches a certain performance and stops after T rounds;

[0014] Furthermore, step (1) also includes: The specific processes of the three methods for data augmentation of data in step (1), namely the basic data augmentation strategy, the SaliencyMix method, and the method combining basic data augmentation and SaliencyMix, are as follows: The basic data augmentation strategy is composed of pairwise combinations of operations such as scaling, rotation, cropping, translation, and color transformation. In each method, the implementation probabilities p1, p2 and intensities m1, m2 of two operations option1 and option2 are respectively set. When adopting a certain method, first randomly generate two random floating-point numbers r1, r2 between 0 and 1. If r1 < p1, then perform option1. If r2 < p2, then perform option2; The SaliencyMix method was proposed by A.F.M. Shahab Uddin et al. The process of this method is: First, select patches, randomly select some data from the training dataset to generate a patch image I s and label y s , and use the saliency detection model f(·) on it to obtain the visual saliency map I vs = f(I s ); Then, search for a pixel with the maximum intensity value where i, j represent the horizontal and vertical coordinates of the salient pixel, i, j = argmax(I vs ); Next, select a small area around the pixel or keep the pixel as a patch M, that is, a binary mask. The size of this patch is determined by the ratio λ, and its value is uniformly distributed between (0, 1); Secondly, perform fusion. The original image I t to be data-augmented is fused with this patch image I s to obtain I a , I a = M ⊙ I s + M' ⊙ I t , where the corresponding positions of M are set to 1 and other positions are set to 0. M' is complementary to M. ⊙ represents pixel-level multiplication. At the same time, for the labels y t and ys Fusion to get y a =λy t +(1-λ)y s ; The final fused image I a and label y a Retained locally on the client for subsequent training; The method combining basic data enhancement and SaliencyMix is ​​obtained by combining the above two methods. First, the basic data enhancement method is used to obtain a new data set, and then the SaliencyMix method is used to process it to obtain the final data set used for training and keep it locally on the client.

[0015] Furthermore, step (5) specifically includes the following sub-steps:

[0016] Step (5-1): Calculate the gradient for each batch and get Where L(·) is the loss function;

[0017] Step (5-2): Each i-th client participating in this round of training uses the formula Update parameters;

[0018] Step (5-3): Client i participating in this round of training trims the updated model parameters to obtain This ensures that the model parameters like Then the parameter value will be discarded and replaced with C, which can reduce unnecessary parameters, speed up training, improve generalization ability, and reduce the risk of overfitting;

[0019] Step (5-4): The i-th client participating in this round of training sends the trimmed model parameters Add noise to get And upload to the server, where is the noise added by the i-th client to the model parameters in this round. This noise needs to satisfy (∈,δ)-differential privacy, which is defined as: given a random mechanism M:X→R, for all measurable sets And any two adjacent data sets D i ,D' i ∈X, if the random mechanism M in D i and D' i The above meets the following conditions: P r (M(D i )∈S]≤e ∈ P r [M(D′ i )∈S]+δ Then the random mechanism M is said to satisfy (∈,δ)-differential privacy on the dataset X, where ∈ represents the privacy budget, δ represents the relaxation factor, and P r Represents probability. According to the above definition, in this method, the random mechanism M is to add noise to the model parameters. in represents a Gaussian distribution, and the noise scale σ ≥ cΔs / ∈ is selected, where ∈∈(0,1), Δs is the sensitivity of function s, s is a real-valued query function, and its calculation formula is: During the training process, each calculated gradient can be regarded as a query function s. Beneficial Effects This method performs data enhancement processing on the data before local training on the client, and adds differential perturbation noise to the parameters before uploading the model parameters, which can effectively improve the model performance while maintaining the model's ability to resist attacks. Attached pictures recommended

[0020] Figure 1 This is a diagram of the federated learning framework based on data enhancement and differential privacy. Specific implementation methods

[0021] The purpose of this invention is to propose a protection method based on the combination of data enhancement and differential privacy, and to reduce the problem of model performance degradation caused by adding noise to improve defense capabilities by adding data enhancement technology.

[0022] The technical solutions provided by the present invention will be described in detail below in conjunction with specific embodiments. It should be understood that the following specific embodiments are only used to illustrate the present invention and are not used to limit the scope of the present invention.

[0023] During the federated learning training process, attackers may use gradient inversion attacks and other means to infer participants' sensitive data, thereby exposing participants' data to security risks. Using differential privacy to add perturbations to parameters can effectively address the privacy leakage problem caused by the above attacks. However, the added noise intensity has a large negative impact on the accuracy of the model, and it is difficult to balance the contradiction between privacy protection strength and model availability. The present invention adds a data enhancement method to federated learning based on differential privacy. Federated learning participants use data enhancement methods to process the data set before local training to increase sample diversity and improve the generalization ability of the model. In this way, the negative impact caused by adding differential perturbations to the uploaded parameter information during training is reduced.

[0024] Specifically, step (1) first determines a server as the central server in federated learning, determines the training task of federated learning, designs and uses a deep learning model as the training model according to the task, such as using a CNN network architecture for image classification tasks, sets the data enhancement method used by the client and processes the data set used for training. The data enhancement method can be one of the basic data enhancement strategies, the SaliencyMix method, or a combination of the basic data enhancement and the SaliencyMix method. In this example, the SaliencyMix method is selected as the data enhancement method for each client. Each client first selects a patch locally, randomly selects some images from the training data set to generate the patch image I s and label y s , and use the saliency detection model f(·) to obtain the visual saliency map I vs =f(I s ), then search for a pixel with the largest intensity value Where i,j represents the horizontal and vertical coordinates of the salient pixels, i,j = argmax(I vs ); Next, select a surrounding pixel Or keep the pixels The small area is taken as the patch M, that is, the binary mask. The size of the patch is determined by the ratio λ, and its value is evenly distributed between (0,1); secondly, the fusion is performed to make the original image I t With the patch image I s Fusion to obtain I a , I a =M⊙I s +M'⊙I t , where the corresponding position of M is set to 1, and the other positions are set to 0, M' is complementary to M, ⊙ represents pixel-level multiplication, and the labels y of the two images are t and s Fusion to get y a =λy t +(1-λ)y s ; The final fused image I a and label y a Stored locally on the client for subsequent training.

[0025] Step (2) The central server initializes the global model and randomly initializes the model parameters w0, specifies the privacy budget ∈ and relaxation factor δ required for differential privacy, the proportion of clients participating in local training in each round is C, the number of rounds that the participating clients need to train the local data set in each round of local training is E, the sample size of each round of training is batch, the learning rate is η, and the number of clients participating in local training in each round is N = C·K. These N clients can be indexed by i, that is, i∈{1,2,…,N}.

[0026] Step (3) Prepare to start the tth round of training. The central server randomly selects N clients to participate in this round of training. If this round is the first round, that is, t = 1, the initialized global model obtained in step (2) and the randomly initialized model parameter w0 are distributed to the clients participating in the training; if this round is not the first round, that is, t ≠ 1, the global model parameter w0 obtained in the previous round is t Distribute to clients participating in training.

[0027] Step (4) N clients participating in the training receive the global model and model parameters w0 or w0 sent by the central server. t , that is, the model parameters of client i are or And use the data after data enhancement in step (1) to train the local model.

[0028] Step (5) The i-th client divides the data set into n batches according to the batch size. i / batch batches, each completes n i / batch training is an epoch, and a total of E rounds are completed locally, that is, training The i-th client participating in this round of training will use the model parameters obtained from this E round of training Upload to the server. Specifically:

[0029] Step (5-1): Calculate the gradient for each batch and get Where L(·) is the loss function;

[0030] Step (5-2): Each i-th client participating in this round of training uses the formula Update parameters;

[0031] Step (5-3): Client i participating in this round of training trims the updated model parameters to obtain This ensures that the model parameters like The parameter value will be discarded and replaced by C. This can reduce unnecessary parameters, speed up training, improve generalization ability, and reduce the risk of overfitting.

[0032] Step (5-4): The i-th client participating in this round of training sends the trimmed model parameters Add noise to get And upload to the server, where For the i-th client, add noise that satisfies (∈,δ)-differential privacy to the model parameters in this round.

[0033] Step (6) The central server receives the model updates from all clients and updates them according to the formula Perform model aggregation, where n i represents the amount of data contained in client i participating in the tth round of local training, and n represents the total amount of data contained in all clients participating in the tth round of local training.

[0034] Step (7) The server updates the model parameter w t Broadcast to the client, and the client tests the performance of the updated model.

[0035] Step (8) loops through steps (3) to (7) until the global model reaches a certain performance and stops after T rounds. The overall framework is as follows Figure 1 shown.

[0036] Step (9) The central server generates the global model w generated by the last round of federated training. t Saving the final training result for image classification can ensure model performance while maintaining the model's ability to resist attacks.

[0037] In the embodiments provided in this application, it should be understood that the disclosed method can be implemented in other ways without exceeding the spirit and scope of this application. The current embodiment is only an illustrative example and should not be used as a limitation. The specific content given should not limit the purpose of this application.

[0038] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art can easily think of changes or substitutions within the technical scope disclosed by the present invention, which should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.

Claims

1. A privacy protection method based on data enhancement in federated learning, characterized by: Including the following steps: Step (1), each local device or client participating in federated learning first performs data augmentation on the data to be used for federated learning training locally; Step (2), the central server initializes the global model and randomly initializes the model parameter w0, specifies the privacy budget ∈ and relaxation factor δ required for differential privacy, the proportion of clients participating in local training in each round is k, the number of rounds for the clients participating in training to train on the local dataset in each round is E, the sample size per round of training is batch, the learning rate is η, the clipping threshold for differential privacy in each round is C. Suppose there are a total of K clients participating in federated learning training, then the number of clients participating in local training in each round is N = k·K. These N clients are indexed by i, that is, i ∈ {1, 2, …, N}; Step (3), prepare to start the tth round of training, the central server randomly selects N clients to participate in this round of training. If this round is the first round, that is, t = 1, the initialized global model obtained in step (2) and the randomly initialized model parameter w0 are distributed to the local devices or clients participating in the training; if this round is not the first round, that is, t ≠ 1, the global model parameter w0 obtained in the previous round is t Distribute to local devices or clients participating in training; Step (4), N local devices or clients participating in the training receive the global model and model parameters w0 or w0 sent by the central server. t , using ← to represent the assignment process of a variable, the variable on the left side of the symbol is assigned the value on the right side of the symbol, then the model parameters of client i are or And use the data after data enhancement in step (1) to perform local model training; Step (5), the i-th client divides the data set into n batches according to the batch size i / batch batches, where n i Represents the amount of data contained in client i participating in the tth round of local training. i / batch training is an epoch, and a total of E rounds are completed locally, that is, training batches, the i-th client participating in this round of training will use the model parameters obtained from this E round of training Upload to the server; Step (6), the central server receives the model updates from all local devices; According to the formula Perform model aggregation, where n i represents the amount of data contained in client i participating in the tth round of local training, n represents the total amount of data contained in all clients participating in the tth round of local training, Represents the new model parameters after the i-th client adds noise to the model parameters in the t-th round; Step (7), the server updates the model parameters w t Broadcast to the client, and the client tests the performance of the updated model; Step (8), loop steps (3) to (7) until the global model reaches a certain performance and stops after T rounds; 2. The privacy protection method based on data enhancement in federated learning according to claim 1, characterized in that: This method is applied to image data. The specific processes of the three methods for data augmentation in step (1), namely the basic data augmentation strategy, the SaliencyMix method, and the method combining basic data augmentation and SaliencyMix, are as follows: The basic data augmentation strategy is composed of pairwise combinations of operations such as scaling, rotation, cropping, translation, and color transformation. The implementation probabilities p1, p2 and intensities m1, m2 of two operations option1 and option2 are set respectively in each method. When adopting a certain method, first randomly generate two random floating-point numbers r1, r2 between 0 and 1. If r1 < p1, then perform option1. If r2 < p2, then perform option2; The implementation process of the SaliencyMix method is as follows: First, select patches and randomly select some data from the training dataset to generate patch images I s and label y s , and use the saliency detection model f(·) to obtain the visual saliency map I vs =f(I S ), then search for a pixel with the largest intensity value Where i,j represents the horizontal and vertical coordinates of the salient pixels, i,j = argmax(I vs ); Next, select a surrounding pixel Or keep the pixels The small area is taken as the patch M, that is, the binary mask. The size of the patch is determined by the ratio λ, and its value is evenly distributed between (0,1); secondly, the fusion is performed to make the original image I t With the patch image I s Fusion I a , I a =M⊙I s +M'⊙I t , where the corresponding position of M is set to 1, and the other positions are set to 0, M' is complementary to M, ⊙ represents pixel-level multiplication, and the labels y of the two images are t and s Fusion to get y a =λ yt +(1-λ)ys; the final fused image I a and label y a Retained locally on the client for subsequent training; The method combining basic data augmentation and SaliencyMix is obtained by combining the above two methods. First, use the basic data augmentation method to obtain a new dataset, and then use the SaliencyMix method for processing to obtain the final dataset for training and leave it locally on the client; 3. The privacy protection method based on data enhancement in federated learning according to claim 1, characterized in that: In step (5), the client i participating in the local training sets the model parameters The specific process of updating includes the following sub-steps: Step (5-1): Calculate the gradient for each batch and get Where L(·) is the loss function, is the gradient function; Step (5-2): Each i-th client participating in this round of training uses the formula Update the parameters, where η is the learning rate obtained in step (2); Step (5-3): The local device or client i participating in this round of training trims the updated model parameters to obtain Where C is the pruning threshold of each round of differential privacy obtained in step (2), which can ensure that the model parameters like The parameter value will be discarded and changed to C. This can reduce unnecessary parameters, speed up training, improve generalization ability, and reduce the risk of overfitting. Step (5-4): The i-th client participating in this round of training sends the trimmed model parameters Add noise to get And upload to the server, where is the noise added by the i-th client to the model parameters in this round. This noise needs to satisfy (∈,δ)-differential privacy, which is defined as: given a random mechanism M:X→R, for all measurable sets And any two adjacent data sets D i ,D' i ∈X, if the random mechanism M in D i and D' i The following conditions are met: P r [M(D i )∈S]≤e ∈ P r [M(D′ i )∈S]+δ Then the random mechanism M is said to satisfy (∈,δ)-differential privacy on the dataset X, where ∈ represents the privacy budget, δ represents the relaxation factor, and P r Represents probability. According to the above definition, the random mechanism M is to add noise to the model parameters. in represents a Gaussian distribution, and the noise scale σ ≥ cΔs / ∈ is selected, where Δs is the sensitivity of function s, which is a real-valued query function and is calculated as: In the training process, each calculation of the gradient is regarded as a query function s.

Images