A federated learning method against backdoor attacks

Abstract

The application discloses a backdoor attack resistant federated learning method, first, a few benign model updates with high confidence are selected through clustering-based voting, then the selected model updates are regarded as benign patterns of model updates, and a variational autoencoder-based anomaly detection method is used to gradually select more benign model updates from the remaining candidate model updates, so that the size of the selected benign model update group is continuously expanded. The application does not depend on differential privacy, weight clipping and learning rate adjustment, only makes slight changes to the original federated learning protocol, has little influence on the accuracy of the global model, and is convenient for integration with the existing federated learning system. Compared with the existing federated learning methods resisting backdoor attacks, the application has stronger adaptability.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of artificial intelligence, and particularly relates to a federated learning method against backdoor attacks. BACKGROUND

[0002] Artificial intelligence has become one of the important technologies driving social and economic development and is deeply integrated into every corner of people's lives. With the continuous breakthroughs in core artificial intelligence technologies represented by deep learning, artificial intelligence technology gradually relies on a large amount of data for model training, but this has brought the problem of excessive collection and use of personal privacy data, resulting in the growing awareness and concern of people about data privacy. The introduction of data regulation policies and the emergence of related regulation technologies have promoted the development of privacy-protecting artificial intelligence technologies and promoted the progress of federated learning (FL), a computing paradigm for training machine learning models in collaboration with multiple participants under the premise of protecting data privacy.

[0003] Due to the inaccessibility of distributed data, FL is vulnerable to attacks by malicious clients, especially backdoor attacks, which neither significantly change the statistical properties of the model like Gaussian noise attacks nor cause obvious modifications to the training data like label flipping attacks, and are therefore more covert to existing defense methods based on model statistics and spectral analysis.

[0004] Existing works defend against target attacks in FL by the following ways: (1) Byzantine robust aggregation [Peva Blanchard, El Mahdi El Mhamdi, Rachid Guerraoui, and Julien Stainer. Machine learning with adversaries: Byzantine tolerant gradient descent. Advances in neural information processing systems, 30, 2017]; (2) robust learning rate [Mustafa Safa Ozdayi, Murat Kantarcioglu, and Yulia R Gel. Defending against backdoors in federated learning with robust learning rate. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 35, pages 9268-9276, 2021]; (3) a combination of weight clipping, noise overlay, and cluster option [Eugene Bagdasaryan, Andreas Veit, Yiqing Hua, Deborah Estrin, and Vitaly Shmatikov. How to backdoor federated learning. In International Conference on Artificial Intelligence and Statistics, pages 2938-2948. PMLR, 2020]; (4) provably secure FL relying on model ensemble [Chulin Xie, Minghao Chen, Pin-Yu Chen, and Bo Li. CRFL: Certifiably robust federated learning against backdoor attacks. In International Conference on Machine Learning, pages 11372-11382, 2021]; (5) anomaly detection [Suyi Li, Yong Cheng, Wei Wang, Yang Liu, and Tianjian Chen.Learning to detect malicious clients for robust federated learning. arXiv preprint arXiv:2002.00211, 2020. However, provably secure methods rely on certain assumptions to provide theoretical guarantees, and their performance can be weaker than empirical defenses in reality because the assumptions can not always be satisfied. Moreover, they rely on a large set of noise models, which reduces the inference efficiency and cannot obtain a single model that can be deployed to clients, the rest of which requires an explicit boundary between benign and malicious model updates with a global perspective, which usually only occurs when: ① the statistical heterogeneity of data is not complex, that is, independent and identically distributed or simple pathological non-independent and identically distributed, so that model updates are easily formed into different clusters; ② the poison data rate (PDR) is relatively high, so that malicious model updates are significantly different from benign model updates, for example, PDR is not less than 50%. In addition, many defenses are only evaluated with a relatively small number of malicious clients, for example, no more than 10% are malicious, in which case the impact of the attack will not be great even if the malicious model updates escape detection. In general, it can be more appropriate to design a federated learning method against backdoor attacks that has little impact on the global model accuracy based on anomaly detection techniques, because no noise is introduced and the aggregation of the original federated learning does not change significantly, keeping a consistent optimization goal. However, there are two main challenges in directly adopting anomaly detection techniques:

[0005] Challenge 1 (lack of benign patterns): Due to the unpredictable distribution and trajectory shift of model updates, each round of benign model updates lacks benign patterns.

[0006] Challenge 2 (boundary ambiguity): Due to the minimal impact of backdoor attacks on model parameters, combined with the data heterogeneity problem faced by federated learning, the boundary between benign and malicious model updates is usually unclear. SUMMARY

[0007] In view of the above, the present application provides a federated learning method against backdoor attacks, which modifies the original federated aggregation algorithm, no longer directly aggregating all received model updates, but through two key steps of "clustering-based voting" and "progressive selection", selects a subset from the received model update set for aggregation, thereby excluding malicious model updates.

[0008] A backdoor attack resistant federated learning method, in a round of federated learning process, the participants will first download the global model from the center server, and then use their own data set to train and update the parameters of the local global model, the difference between the updated local global model and the parameters of the global model downloaded at the beginning of this round is the model update, and the participants further upload the model update to the center server;

[0009] The center server receives N model updates and selects benign model updates from a decentralized perspective in the case of a majority of benign model updates through clustering-based voting, N being the number of participants in federated learning;

[0010] A difference training set is constructed according to the selected benign model updates, and a variational autoencoder is trained using the set;

[0011] A difference verification set is constructed, and the data in the set is reconstructed using the variational autoencoder, and the population of selected benign model updates is gradually expanded according to the reconstruction error;

[0012] The center server performs federated averaging algorithm on all selected benign model updates to obtain a global model without backdoor and distributes it to the participants for the next round of federated learning.

[0013] Further, the benign model update is the model update obtained by the participant using the data set without injecting triggers, and vice versa, the model update obtained by the participant using the data set based on injecting triggers is the malignant model update.

[0014] Further, the specific implementation of the center server mining benign model updates is: first, the set of model updates received by the center server is denoted as where Δw i is the model update uploaded by the i-th participant, i is a natural number and 1≤i≤N; let the global model be a neural network with L layers, for the m-th layer parameter Δw i in the model update Δw i,m , select the zero vector and the K-1 model updates farthest from Δw i,m as the initial points of the K-means algorithm, after clustering, Δw i,m vote for all model updates in the cluster where it is located, m is a natural number and 1≤m≤L; in this way, each model update will cast L votes, and each vote has a weight of 1; finally, the model updates with the highest votes form a benign model update set with high confidence, denoted as

[0015] Further, after obtaining the set , the center server first calculates it according to the Cartesian product The difference between the model updates, thereby forming a difference training set, and any difference data in the difference training set is Δw a -Δw b , Δw a and Δw b is a set Any two different model updates in the set; then use the difference training set to train a variational autoencoder, the input of which is any difference data in the difference training set, and the output is a vector of the same dimension, and the variational autoencoder can generate an output that is as similar as possible to the input.

[0016] Further, take the complement of the set , that is Any difference data in the difference validation set is Δw c -Δw d , wherein Δw c is any model update in the complement, and Δw d is any model update in the set , use the variational autoencoder to reconstruct each difference data in the difference validation set, select a number of difference data with the smallest average reconstruction error, and add Δw c in these difference data to the set .

[0017] Further, the reconstruction error is measured by the mean square error between the input data and the reconstructed data of the variational autoencoder.

[0018] Further, when the population of benign model updates is expanded, update the difference training set with the latest set and fine-tune the variational autoencoder, and then determine: if the number of sets exceeds a certain threshold, perform a federated averaging algorithm; otherwise, update the difference validation set with the latest set , and then expand the benign model updates again by reconstruction, until the number of sets exceeds a certain threshold.

[0019] Based on the above technical solution, the present application has the following beneficial technical effects:

[0020] 1. The present application enables a backdoor attack-resistant federated learning method, which does not rely on differential privacy, weight clipping and learning rate adjustment, only makes minor changes to the original federated learning protocol, has little effect on the accuracy of the global model, and is easy to integrate with existing federated learning systems.

[0021] 2.The method can steadily select a set of benign model updates for aggregation in the case that benign and malicious model updates do not obviously gather in different positions, and has stronger adaptability compared with existing federated learning methods resisting backdoor attacks. BRIEF DESCRIPTION OF DRAWINGS

[0022] Figure 1 The figure is a schematic diagram of the federated learning system architecture resisting backdoor attacks of the present application.

[0023] Figure 2 The figure is a schematic diagram of the process of the federated learning method resisting backdoor attacks of the present application in a round of federated aggregation. DETAILED DESCRIPTION

[0024] In order to more specifically describe the present application, the technical solutions of the present application are described in detail below in combination with the drawings and specific embodiments.

[0025] The system architecture in which the federated learning method resisting backdoor attacks of the present application is run is shown in Figure 1 The system mainly includes a center server and participants, the center server is responsible for coordinating each participant to run the federated learning method, and the participant is responsible for local training, submitting model updates, and receiving aggregation results from the center server.

[0026] In order to benefit from the federated learning system (obtain a model with higher accuracy) and participate in the federated learning system, the terminal device of the participant uploads model parameters to the center server and downloads aggregated model parameters from the center server. For a participant of federated learning, in a round of federated learning, the participant first downloads a global model, forms a copy of the global model, then updates the parameters of the copy of the global model using its own data set, and the difference between the updated global model and the parameters of the global model downloaded at the beginning of the round is called model update.

[0027] The malicious participant (attacker) will inject a trigger into its own data set, and update the global model based on the training data injected with the trigger, so that the model update submitted by the malicious participant will enable the global model to recognize the trigger and make a specific erroneous output. The model update submitted by the malicious participant is called malicious model update; and the model update obtained by the normal participant using the data set without injecting the trigger is called benign model update.

[0028] The federated learning method resisting backdoor attacks of the present application is executed in each round of federated learning, and is used to replace the original federated averaging algorithm. As shown in Figure 2 The federated learning method of the present application starts with the center server receiving N model updates in a round of federated learning (denoted as ), and includes the following steps:

[0029] 1. Considering that for a specific model update, the model updates closer to it are most likely to have the same stance as it. That is, if the model update is benign, the few model updates closest to it are often also benign, and vice versa. Therefore, through voting, benign model updates can be mined from a decentralized perspective when benign model updates account for the majority among the N model updates received by the central server in this round.

[0030] Assume that the model parameters are of an L-layer neural network. For the parameters Δw of the m-th layer of the i-th model update in i,m , regard it as an agent with voting rights, and it casts its vote in the following way: Select the 0 vector and the K - 1 model updates farthest from Δw i,m as the initial points of the K-means algorithm (K < N). After dividing the N received model updates into K clusters, Δw i,m votes for all the model updates in its cluster. In this way, each model update Δw i casts a total of L votes, and in this embodiment, the weight of each vote is uniformly set to be 1; the entire voting process is carried out on the central server, which is actually equivalent to having the central server select benign model updates from the perspectives of each model update respectively.

[0031] 2. The model updates with the highest votes are regarded as benign model updates with high confidence, denoted as The "clustering-based voting" step ends here, where

[0032] 3. Using the set of benign model updates obtained in the previous step as the benign pattern of model updates, referring to the form of the Cartesian product, obtain the differences between pairwise model updates in The differences between pairwise model updates in form a set i where Δw j and Δw are any different model updates in

[0033] 4. Using the data in , train a Variational Auto-Encoder (VAE). The goal of the training process is to enable this VAE to have the following function: Given a data in, this VAE can generate an output as similar as possible to the original input.

[0034] 5. Based on the latest Update as described in step 3. Subsequently based on the updated Fine-tune VAE.

[0035] 6. For sets Each model updates Δw i as well as Each model updates Δw j , build The model difference Δw i -Δw j Using these model differences as input to the VAE, the VAE attempts to reconstruct these Δw values. i -Δw j Select The model update Δw with the smallest average reconstruction error i Add to In this context, the reconstruction error can be measured by the mean square error (MSE) between the original input data and the reconstructed data. For each model update Δw i In this context, the average reconstruction error refers to the reconstruction error constructed in this step. Among the differences in the models, Δw is involved. i of The average reconstruction error of each model difference.

[0036] 7. If If the number of model updates exceeds M, proceed to step 8; otherwise, proceed to step 5.

[0037] 8. Return

[0038] After the method is completed, The model updates included in the process execute a federated averaging algorithm to obtain a global model that does not contain backdoors, which is then distributed to the participants for the next round of federated learning.

[0039] Most of the existing methods to defend against backdoor attacks in federated learning judge which model updates are malicious from a centralized global perspective. Due to the backdoor attack, neither significantly change the statistical properties of the model like Gaussian noise attack, nor cause obvious modification to the training data like label flipping attack, the influence on the distribution of model parameters is smaller, so it often appears to be randomly scattered in space, making it difficult to distinguish between benign model updates and malicious model updates from a global perspective. If the model updates are examined from the perspective of a certain model update, the model updates closest to it may have the same purpose (i.e. if the model update is benign, the several model updates closest to it are more likely to be benign), assuming that both benign model updates and infected model updates hope to exclude each other from aggregation, so if each model update is assumed to be an intelligent agent with voting rights and votes for the model update closest to it, when benign model updates are in the majority, benign model updates may get more votes. In addition, if the selected benign model updates are used as benign patterns, the benign model updates are gradually supplemented, more benign model updates can be more safely included in the global model, because this gradual expansion can make more and more benign patterns available when screening benign model updates from the remaining candidate model updates.

[0040] The above description of the embodiments is to facilitate those skilled in the art to understand and apply the present application, and those skilled in the art can easily make various modifications to the above embodiments, and apply the general principles described herein to other embodiments without creative labor. Therefore, the present application is not limited to the above embodiments, and the improvements and modifications made to the present application by those skilled in the art according to the disclosure of the present application should be within the scope of protection of the present application.

Claims

1. A federated learning method resistant to backdoor attacks, wherein in a round of federated learning, participants first download a global model from a central server, then use their own datasets to train and update the parameters of their local global model. The difference between the parameters of the updated local global model and the initially downloaded global model in this round constitutes the model update. Participants then upload the model update to the central server; characterized in that: The central server received N After each model is updated, a cluster-based voting method is used to select the best model for update from a decentralized perspective, provided that a majority of the updates are benign. N The number of participants in federated learning; The differential training set is updated based on the selected benign model, and a variational autoencoder is trained using this training set. Construct a difference validation set, reconstruct the data in the set using a variational autoencoder, and progressively expand the selected benign model update population based on the reconstruction error. The central server updates all selected benign models and performs a federated averaging algorithm to obtain a global model that does not contain backdoors, which is then distributed to the participants for the next round of federated learning. The specific implementation method of the central server mining benign model updates is as follows: First, the set of model updates received by the central server is denoted as... ,in For the first i Model updates uploaded by each participant i It is a natural number and 1≤ i ≤ N Let the global model be a model with L Layered neural networks for model updates The first in m Layer parameters Choose the zero vector and distance The farthest K -1 model update is used as the initial point for the K-means algorithm. After clustering, The model will be updated by voting for all models in its cluster. m It is a natural number and 1 ≤ m ≤ L ; In this way, each model update will be projected. L Each vote has a weight of 1; finally, the models with the highest number of votes are combined into a benign model update set with high confidence, denoted as . ; In obtaining the set Then, the central server first calculates the result based on the form of a Cartesian product. The differences in model updates constitute the difference training set, where any difference data in the difference training set is... , and For set The algorithm updates any two different models; then it uses the difference training set to train a variational autoencoder, whose input is any difference data in the difference training set and whose output is a vector of the same dimension. The variational autoencoder can generate an output that is as similar as possible to the input. Take set The complement is Any difference data in the difference validation set is ,in Update any model in the complement set. For set For any model update in the dataset, reconstruct each difference data point in the difference validation set using a variational autoencoder, select several difference data points with the smallest average reconstruction error, and then select these difference data points... Add to collection middle.

2. The federated learning method according to claim 1, characterized in that: A benign model update is a model update obtained by participants training on a dataset without injected triggers, while a malicious model update is a model update obtained by participants training on a dataset based on injected triggers.

3. The federated learning method according to claim 1, characterized in that: The reconstruction error is measured by the mean square error between the input data and the reconstructed data of the variational autoencoder.

4. The federated learning method according to claim 1, characterized in that: When the benign model updates the population and expands, the latest set is used. Update the differential training set and fine-tune the variational autoencoder, then determine: if the set If the number exceeds a set threshold, a federated average algorithm is applied; otherwise, the latest set is used. Update the difference validation set, and then expand the benign model again through reconstruction until the set is updated. The number exceeds the set threshold.

Images