Privacy protection method based on data augmentation in federated learning

Abstract

The application discloses a privacy protection method based on data enhancement in federated learning and belongs to the field of privacy protection of federated learning. The steps of the method are as follows: firstly, a client participating in federated learning training processes a data set by using a selected data enhancement method before local training; then, a central server initializes all parameters and distributes them to each client to start training; in each round, a client participating in training obtains global model parameters, calculates gradients after training each batch, updates parameters, clips parameters and adds differential perturbation noise; after each round, the server aggregates and updates global parameters, and broadcasts them to the client, and the cycle continues until the model converges. The method mainly improves the existing privacy protection method based on differential privacy in federated learning, and adds a data enhancement method to better balance the model availability and privacy protection capability. The data enhancement method includes SaliencyMix, a basic data enhancement strategy, a combination of SaliencyMix and the basic data enhancement strategy, etc.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of privacy protection based on data enhancement in federated learning. In particular, it relates to a privacy protection method based on the combination of data enhancement and differential privacy. BACKGROUND

[0002] In 2016, Google first proposed the concept of a federated learning framework. Federated learning is a distributed machine learning framework that allows participants to download the latest model parameters from the cloud, update the model parameters using local data, and send the obtained parameters back to the server. When the model converges, the server aggregates the model update parameters and sends them to the participants for training. This iteration continues. On the one hand, this maximizes the use of participants' storage space and computing power. On the other hand, since participants' information is stored locally, they only need to share model parameters, ensuring their privacy.

[0003] However, while federated learning model training allows multiple users to train high-quality models together through gradients as a medium while ensuring their own data privacy, federated learning must continuously iterate and optimize model parameters during training to achieve the desired results. In this process, gradients are used as adjustment media for parameter backpropagation to adjust and optimize parameters. This indicates that gradients carry a large amount of information about the original training data. Therefore, some attack methods that infer training data information through gradients have been proposed, which can leak user training data and threaten the security of training models such as federated learning. Common privacy protection methods still have limitations: differential privacy methods cannot balance model accuracy and security; secure multi-party computation requires collaboration between participants for computation and operation; homomorphic encryption requires a trusted third party to complete encryption / decryption operations, requiring a large amount of computing resources and high storage overhead; methods other than cryptography, such as using GAN technology, will have more additional communication overhead. Compared to other methods, some differential privacy-based solutions require fewer resources and have lower computational complexity, making them more optimal. However, they still need to improve in terms of balancing privacy protection strength and model usability. SUMMARY

[0004] The present application is used to solve the problem of balancing privacy protection strength and model usability in federated learning based on differential privacy protection methods. Specifically, it proposes a federated learning privacy protection method based on data enhancement, which processes data enhancement on the local data of clients participating in federated learning training. During the upload of the model on the client side, differential privacy is used to add noise to the model parameters, ensuring that the federated learning training process has defense capabilities and can prevent attackers from obtaining training data through gradient reconstruction while improving model performance.

[0005] To achieve the above object, the technical scheme adopted by the present application is based on the combination of data enhancement and differential privacy, the main flow of which is shown in the figure, and the implementation steps of the method are as follows:

[0006] Step (1) Each client participating in federated learning first performs data enhancement on the data to be used for federated learning training locally, which can be processed by one of the basic data enhancement strategy, SaliencyMix method or the combination of basic data enhancement and SaliencyMix method;

[0007] Step (2) The central server initializes the global model and randomly initializes the model parameters w0, specifies the privacy budget ε and the relaxation factor δ required by differential privacy, the proportion of clients participating in local training in each round is C, the number of rounds in which the client participating in training needs to train the local data set in each round of local training is E, the sample size of each round of training is batch, the learning rate is η, and the number of clients participating in local training in each round is N=C·K, and the N clients can be indexed by i, i.e. i∈{1,2,…,N};

[0008] Step (3) Prepare to start the tth round of training, the central server randomly selects N clients to participate in this round of training, if this round is the first round, i.e. t=1, the initialized global model obtained from step (2) and the randomly initialized model parameters w0 are distributed to the clients participating in training; if this round is not the first round, i.e. t≠1, the global model parameters w t are distributed to the clients participating in training;

[0009] Step (4) The N clients participating in training receive the global model and model parameters w0 or w t distributed by the central server, i.e. the model parameters of the i-th client are or and use the data enhanced in step (1) to perform local model training;

[0010] Step (5) The i-th client divides the data set into n i / batch batches according to the batch size batch, and completes n i / batch batches of training as an epoch, a total of E rounds of local training, i.e. training / batch batches. The i-th client participating in this round of training uploads the model parameters obtained by E rounds of training to the server;

[0011] Step (6) The central server receives the model updates of all local devices, and performs model aggregation according to the formula , wherein n iThe data amount contained in the client i participating in the tth round of local training, n represents the total data amount contained in all clients participating in the tth round of local training;

[0012] Step (7) The server updates the model parameters w t to the clients, and the clients test the performance of the updated model;

[0013] Step (8) Steps (3) to (7) are repeated until the global model reaches a certain performance, and the process stops after T rounds;

[0014] Further, the three methods of data augmentation in step (1) are as follows:

[0015] The basic data augmentation strategy is composed of scaling, rotation, cropping, translation, color transformation and other operations, and each method sets the implementation probability p1, p2 and intensity m1, m2 of two operations option1 and option2. When a method is used, two random floating-point numbers r1 and r2 between 0 and 1 are first randomly generated. If r1 < p1, option1 is performed, and if r2 < p2, option2 is performed; The SaliencyMix method is proposed by A.F.M. Shahab Uddin et al. The process of this method is as follows: first, select a patch, randomly select some data from the training data set to generate a patch image I s and label y s , use the saliency detection model f(·) to obtain the visual saliency map I vs = f(I s ); then, search for a pixel with the maximum intensity value in it where i, j represent the horizontal and vertical coordinates of the salient pixel, i, j = argmax(I vs ); then, select a small area around the pixel or the pixel as a patch M, that is, a binary mask, and the size of the patch is determined by the ratio λ, which is uniformly distributed between 0 and 1; secondly, fusion is performed to obtain I t , I s = M' ⊙ I a , I a = M ⊙ I s + M' ⊙ I t , where M is set to 1 at the corresponding position and 0 at other positions, M' is complementary to M, and represents pixel-level multiplication.t and y s fusion to obtain y a = λy t +(1-λ)y s ; finally, the fused image I a and the label y a is left in the local client for subsequent training;

[0016] The method combining basic data augmentation with SaliencyMix is obtained by combining the above two methods, first using the basic data augmentation method to obtain a new data set, and then using the SaliencyMix method to process to obtain the final data set used for training left in the local client;

[0017] Further, step (5) specifically includes the following sub-steps:

[0018] Step (5-1): Calculate the gradient for each batch to obtain where L(·) is the loss function;

[0019] Step (5-2): The i-th client participating in this round of training updates the parameters according to the formula

[0020] Step (5-3): The i-th client participating in this round of training clips the updated model parameters to obtain In this way, the model parameters If , the parameter value will be discarded and C will be used instead. This can reduce unnecessary parameters, speed up training, improve generalization ability, and reduce the risk of overfitting.

[0021] Step (5-4): The i-th client participating in this round of training adds noise to the clipped model parameters to obtain and uploads it to the server, where is the noise added to the model parameters by the i-th client in this round. This noise needs to satisfy (ε, δ)-differential privacy, which is defined as follows: Given a random mechanism M: X→R, for all measurable sets and any two adjacent data sets D i ,D' i ∈X, if the random mechanism M satisfies the following conditions on D i and D' i :

[0022] P r (M(D i )∈S]≤e ∈ P r [M(D′​i )∈S]+δ

[0023] Then the random mechanism M satisfies (e, d)-differential privacy on the data set X, where e represents the privacy budget, d represents the relaxation factor, P r represents the probability, according to the above definition, in the method, the random mechanism M is to add noise to the model parameters, the noise Wherein represents a Gaussian distribution, the noise scale s is selected to be greater than or equal to cDs / e, wherein e e (0, 1), and Ds is the sensitivity of the function s, s is a real-valued query function, and the calculation formula is:

[0024]

[0025] In the training process, each calculation of the gradient can be regarded as a query function s.

[0026] Advantages

[0027] The method can perform data enhancement processing on the data before local training on the client side, and add differential disturbance noise to the parameters before uploading the model parameters, so that the model performance can be effectively improved while maintaining the attack resistance of the model.

[0028] Figure recommendation

[0029] Figure 1 A federated learning framework based on the combination of data enhancement and differential privacy Specific implementation method

[0030] The purpose of the present application is to propose a protection method based on the combination of data enhancement and differential privacy, which reduces the problem of model performance decline caused by adding noise to improve defense capability by adding data enhancement technology.

[0031] The technical solutions provided by the present application will be described in detail below in combination with specific embodiments, and it should be understood that the following specific embodiments are only used to illustrate the present application and not to limit the scope of the present application.

[0032] During federated learning training, attackers may exploit gradient inversion attacks to infer sensitive participant data, thus jeopardizing participant data security. Differential privacy, by adding perturbations to parameters, can effectively address the privacy leaks caused by such attacks. However, the intensity of the added noise significantly negatively impacts model accuracy, making it difficult to balance the conflict between privacy protection and model usability. This invention incorporates data augmentation methods into differential privacy-based federated learning. Before local training, federated learning participants process the dataset using data augmentation methods to increase sample diversity and improve model generalization ability. This approach reduces the negative impact of adding differential perturbations to uploaded parameter information during training. Specifically, step (1) first requires identifying a server as the central server in the federated learning process and determining the training task. Based on the task, a deep learning model is designed and used as the training model, such as using a CNN network architecture for image classification. The data augmentation method used by the clients is then set up, and the training dataset is processed. The data augmentation method can be one of the following: basic data augmentation strategies, the SaliencyMix method, or a combination of basic data augmentation and SaliencyMix. In this example, the SaliencyMix method is chosen as the data augmentation method for each client. Each client first selects a patch locally, randomly choosing some images from the training dataset to generate patch images I. s and tag y s The visual saliency map I is obtained by using the saliency detection model f(·). vs =f(I s Then, search for the pixel with the highest intensity value among them. Where i,j represent the x and y coordinates of significant pixels, i,j = argmax(I vs Next, select a pixel. Or preserve pixels A small region is used as a patch M, which is a binary mask. The size of this patch is determined by the ratio λ, and its value is uniformly distributed between (0,1). Next, fusion is performed on the original image I to be data augmented. t With the patch image I s Fusion to obtain I a I a =M⊙I s +M'⊙I t In this context, M is set to 1 at the corresponding position and 0 at other positions. M' is complementary to M. ⊙ represents pixel-level multiplication, and the labels y of the two images are also multiplied. t and y s By fusing, we can obtain y a =λy t+ (1 - λ) y s ; the final fused image I a and the label y a is left in the client locally for subsequent training.

[0033] Step (2) The central server initializes the global model and randomly initializes the model parameters w0, specifies the privacy budget ε and the relaxation factor δ required by differential privacy, the proportion of participating local training clients in each round is C, the number of rounds of local data set training required by the participating training clients in each round of local training is E, the sample size of each round of training is batch, the learning rate is η, the number of participating local training clients in each round is N = C·K, and the N clients can be indexed by i, i.e. i ∈ {1, 2, …, N}.

[0034] Step (3) Prepare to start the t-th round of training, the central server randomly selects N clients to participate in this round of training, if this round is the first round, i.e. t = 1, the initialized global model obtained from step (2) and the randomly initialized model parameters w0are distributed to the participating training clients; if this round is not the first round, i.e. t ≠ 1, the global model parameters w t are distributed to the participating training clients.

[0035] Step (4) The N participating training clients receive the global model and model parameters w0or w t sent by the central server, i.e. the model parameters of the client i are or and use the data enhanced in step (1) for local model training.

[0036] Step (5) The i-th client divides the data set into n i / batch batches according to the batch size batch, and completes n i / batch batches of training for an epoch, a total of E rounds of local training, i.e. training / batch batches. The i-th client participating in this round of training uploads the model parameters obtained by the E rounds of training to the server. Specifically:

[0037] Step (5-1): Calculate the gradient for each batch to obtain where L(·) is the loss function;

[0038] Step (5-2): Each i-th client participating in this round of training updates the parameters according to the formula ;

[0039] Step (5-3): The i-th client participating in this round of training clips the updated model parameters to obtain Thus, the model parameters If The parameter value will be discarded and C will be used instead. This can reduce unnecessary parameters, speed up training, improve generalization ability, and reduce the risk of overfitting.

[0040] Step (5-4): The i-th client participating in this round of training adds noise to the pruned model parameters to obtain and uploads it to the server, where is the noise added to the model parameters by the i-th client in this round to satisfy (e, d)-differential privacy.

[0041] Step (6) The central server receives all client model updates and performs model aggregation according to the formula , where n i represents the amount of data contained in the i-th client participating in the t-th round of local training, and n represents the total amount of data contained in all clients participating in the t-th round of local training.

[0042] Step (7) The server broadcasts the updated model parameters w t to the clients, and the clients test the performance of the updated model.

[0043] Step (8) Repeat steps (3) to (7) until the global model reaches a certain performance T rounds, and then stop. The overall framework is shown in Figure 1 .

[0044] Step (9) The central server saves the global model w t generated by the last round of federated training as the final training result, which can be used for image classification while maintaining the model's ability to resist attacks and ensuring model performance.

[0045] In the embodiments provided in the present application, it should be understood that the disclosed method can be implemented in other ways without exceeding the spirit and scope of the present application. The current embodiments are only exemplary and should not be considered as limiting. The specific content given should not limit the purpose of the present application.

[0046] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any skilled person in the art can easily think of changes or replacements within the technical range disclosed by the present application, which should be covered within the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. A data-augmented privacy protection method in federated learning, characterized by: It includes the following steps: Step (1), each local device or client participating in federated learning first performs data augmentation on the data to be used for federated learning training locally. Step (2), the central server initializes the global model and randomly initializes the model parameters w0, specifies the privacy budget ∈ and relaxation factor δ required for differential privacy, the proportion k of clients participating in local training in each round, the number of rounds E for which the clients participating in training need to train on the local dataset in each round of local training, the sample size batch in each round of training, the learning rate η, the clipping threshold C for differential privacy in each round. Suppose there are a total of K clients participating in federated learning training, then the number of clients participating in local training in each round is N = k·K, and these N clients are indexed by i, that is, i ∈ {1, 2,..., N}. Step (3): Prepare to start the t-th round of training. The central server randomly selects N clients to participate in this round of training. If this round is the first round, i.e., t=1, the initialized global model obtained from step (2) and the randomly initialized model parameters w0 are distributed to the local devices or clients participating in the training. If this round is not the first round, i.e., t≠1, the global model parameters w0 obtained from the previous round are distributed to the local devices or clients participating in the training. t Distribute to local devices or clients participating in the training; Step (4): N local devices or clients participating in the training receive the global model and model parameters w0 or w0 sent by the central server. t Using the left arrow (←) to represent the assignment of a variable, the variable on the left of the symbol is assigned the value on the right. Therefore, the model parameters for client i are... or And use the data augmented in step (1) to train the local model; Step (5): The i-th client divides the dataset into n parts according to the batch size. i / batch of batches, where n i This represents the amount of data contained in client i participating in the t-th round of local training, and the amount of data after completing n rounds. i Training in batches ( / batch) constitutes one epoch, with a total of E rounds completed locally, i.e., training... In each batch, the i-th client participating in this round of training will receive the model parameters obtained from these E rounds of training. Uploaded to the server; Step (6), the central server receives the model updates from all local devices. According to the formula Perform model aggregation, where n i Let represent the amount of data contained in client i participating in the t-th round of local training, and let n represent the total amount of data contained in all clients participating in the t-th round of local training. This represents the new model parameters after the i-th client adds noise to the model parameters in round t; Step (7), the server will update the model parameters w t The update is broadcast to the client, who then tests the performance of the updated model. Step (8), loop through steps (3) to (7) until the global model reaches a certain performance and stops after T rounds.

2. The data-augmented privacy protection method in federated learning according to claim 1, characterized in that: This method is applied to image data. The specific processes of the three methods for data augmentation in step (1), namely the basic data augmentation strategy, the SaliencyMix method, and the method combining basic data augmentation and SaliencyMix, are as follows: The basic data augmentation strategy is composed of pairwise combinations of operations such as scaling, rotation, cropping, translation, and color transformation. In each method, the implementation probabilities p1, p2 and intensities m1, m2 of two operations option1 and option2 are set respectively. When adopting a certain method, first randomly generate two random floating-point numbers r1, r2 between 0 and 1. If r1 < p1, then perform option1. If r2 < p2, then perform option2. The implementation process of the SaliencyMix method is as follows: First, select patches by randomly choosing some data from the training dataset to generate patch images I. s and tag y s The visual saliency map I is obtained by using the saliency detection model f(·). vs =f(I S Then, search for the pixel with the highest intensity value among them. Where i,j represent the x and y coordinates of significant pixels, i,j = argmax(I vs Next, select a pixel. Or preserve pixels A small region is used as a patch M, which is a binary mask. The size of this patch is determined by the ratio λ, and its value is uniformly distributed between (0,1). Next, fusion is performed on the original image I to be data augmented. t With the patch image I s Fusion to obtain I a I a =M⊙I s +M'⊙I t In this context, M is set to 1 at the corresponding position and 0 at other positions. M' is complementary to M. ⊙ represents pixel-level multiplication, and the labels y of the two images are also multiplied. t and y s By fusing, we can obtain y a =λ yt +(1-λ)ys; the final fused image I a and tag y a It is stored locally on the client for subsequent training; The method combining basic data augmentation and SaliencyMix is obtained by combining the above two methods. First, use the basic data augmentation method to obtain a new dataset, and then use the SaliencyMix method for processing to obtain the final dataset for training and leave it locally on the client.

3. The data-augmented privacy protection method in federated learning according to claim 1, characterized in that: In step (5), the client i participating in local training will transfer the model parameters. The update process includes the following sub-steps: Step (5-1): Calculate the gradient for each batch, and obtain... Where L(·) is the loss function, • is the gradient function; Step (5-2): Each i-th client participating in this round of training follows the formula... Update the parameters, where η is the learning rate obtained in step (2); Step (5-3): The local device or client i participating in this round of training prunes the updated model parameters to obtain... Where C is the pruning threshold for differential privacy obtained in step (2) for each round, which ensures the model parameters like The parameter value will then be discarded and replaced with C. This reduces unnecessary parameters, speeds up training, improves generalization ability, and reduces the risk of overfitting. Step (5-4): The i-th client participating in this round of training sends the clipped model parameters. Adding noise, we get And uploaded to the server, where This is the noise added by the i-th client to the model parameters in this round. This noise needs to satisfy (∈,δ)-differential privacy, which is defined as: given a random mechanism M:X→R, for all measurable sets and any two adjacent datasets D i ,D' i ∈X, if the random mechanism M is in D i and D' i The above must meet the following conditions: P r [M(D i )∈S]≤e ∈ P r [M(D′ i )∈S]+δ The random mechanism M is said to satisfy (∈,δ)-differential privacy on the dataset X, where ∈ represents the privacy budget, δ represents the relaxation factor, and P r Representing probability, according to the above definition, the random mechanism M is equivalent to adding noise to the model parameters; noise in Representing a Gaussian distribution, we choose a noise scale σ ≥ cΔs / ∈, where Δs is the sensitivity of function s, where s is a real-valued lookup function, and its calculation formula is: In the training process, each calculation of the gradient is regarded as a query function s.

Images