A method and device for detecting poisoning in federated learning based on feature adversarial

Abstract

The present invention discloses a method and device for detecting poisoning in federated learning based on feature confrontation, comprising: dividing all clients in each round of parameter training into benign clients and defense clients, and configuring a defense patch data set for the defense client; in each round of training, the benign client optimizes the benign model using the local data set, and the defense client optimizes the defense model using the defense patch data set and local data, and the server aggregates all the benign models and defense models to obtain a federated learning model; after multiple rounds of training, the federated learning model of the last round is used for detecting poisoned samples, and during the detection, judging whether the test sample is poisoned is based on the prediction result of the target label of the test sample in the federated learning model and whether the prediction result of the defense target label in the federated learning model after the best defense patch data is added to the test sample satisfies the label mapping relationship, that is, realizing federated learning poisoning detection.

Description

Technical Field

[0001] The present invention belongs to the field of federated learning, and specifically relates to a method and device for detecting federated learning poisoning based on feature adversarial. Background Art

[0002] With the rapid development of data-driven intelligent applications, the machine learning paradigm is also facing new difficulties and challenges. On the one hand, the machine learning paradigm hopes to provide a robust and efficient functional service for all users. On the other hand, data, as the nutrition of learning algorithms, is difficult to fully share.

[0003] In order to solve this problem, federated learning has emerged as a highly promising solution. Its main innovation is that it provides a distributed machine learning framework with privacy protection characteristics, and can coordinate thousands of participants in a distributed manner to perform iterative training on a specific machine learning model. Since the training data is still stored locally by the participants during the federated learning process, this mechanism can not only share the training data of each participant, but also ensure the protection of the privacy of each participant.

[0004] The basic workflow of federated learning mainly includes: (1) Participants download the initialized global model from the cloud server, train the model using local data sets, and generate the latest local model updates (i.e., model parameters). (2) The cloud server collects each local update parameter and updates the global model through the model averaging algorithm. Due to the unique advantages of federated learning, it can train a unified machine learning model with the local data of multiple participants while protecting data privacy. Therefore, federated learning has shown excellent application prospects in privacy-sensitive scenarios (including finance, industry, and many other data-aware scenarios).

[0005] Although federated learning can aggregate decentralized information provided by different parties to train better models, its distributed learning approach and the inherent heterogeneous data distribution among different parties may inadvertently provide a new attack surface. In particular, the fact that access to a single party’s data is limited due to privacy issues or regulatory restrictions may facilitate label reversal attacks and backdoor attacks on shared models trained using federated learning. Among them, label reversal attacks refer to malicious users embedding pre-prepared attack points into training data by flipping sample labels, so that the trained model deviates from the established prediction boundary. Backdoor attacks are a type of data poisoning attack that aims to manipulate a subset of training data so that a machine learning model trained on a tampered dataset will be vulnerable to attacks from a test set embedded with similar triggers. The failure of model training or the implantation of backdoors will directly or indirectly cause huge economic losses to AI service platforms and ordinary users.

[0006] In short, since the privacy mechanism of federated learning makes local data unobservable, malicious data nodes are difficult to detect. At the same time, the poisoning attack achieved by training-optimizing the poisoned model is too covert, which greatly increases the cost of training the model. Therefore, it is very necessary to perform poisoning detection on the federated learning model. Summary of the invention

[0007] In view of the above, the object of the present invention is to provide a method and device for detecting poisoning in federated learning based on feature adversarial, so as to realize poisoning detection of the federated learning model and improve the robustness of the federated learning model.

[0008] In order to achieve the above-mentioned invention object, the present invention provides the following technical solutions:

[0009] In the first aspect, a method for detecting poisoning in federated learning based on feature adversarial methods comprises the following steps:

[0010] Divide all clients in each round of parameter training into benign clients and defense clients, and configure a defense patch dataset for each defense client;

[0011] In each round of training, the benign client uses the local data set to optimize the benign model. The defensive client selects the best defense patch data set that maximizes the information entropy content of the model from the defense patch data set, and then uses the best defense patch data set and the local data set to jointly optimize the defense model. The server aggregates all benign models and defense models to obtain a federated learning model.

[0012] After multiple rounds of training, the last round of federated learning model is used to detect poisoned samples. During the detection, the prediction result of the target label of the test sample in the federated learning model is used to determine whether the test sample is poisoned based on whether the prediction result of the defense target label in the federated learning model after the best defense patch data is added to the test sample satisfies the label mapping relationship, that is, federated learning poisoning detection is realized.

[0013] In the second aspect, a federated learning poisoning detection device based on feature adversarial is provided, comprising a computer memory, a computer processor, and a computer program stored in the computer memory and executable on the computer processor, wherein the computer processor implements the above-mentioned federated learning poisoning detection method based on feature adversarial when executing the computer program.

[0014] Compared with the prior art, the method and device for detecting poisoning in federated learning based on feature confrontation provided by the present invention have at least the following beneficial effects:

[0015] By dividing local clients into benign clients and defensive clients, using local data sets and defense patch data sets to optimize benign models and defense models respectively, and then aggregating all benign models and defense models to obtain a federated learning model, federated learning can converge faster and will not lose performance due to the introduction of adversarial clients. At the same time, it can explain the working mechanism of poisoning attacks on the federated learning model, realize the feasibility analysis of detection, and improve the robustness of the model. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0017] Figure 1 is a flow chart of a method for detecting poisoning in federated learning based on feature confrontation provided by an embodiment of the present invention;

[0018] Figure 2 It is a schematic diagram of the training phase in the feature adversarial-based federated learning poisoning detection method provided in an embodiment of the present invention;

[0019] Figure 3 It is a schematic diagram of the detection phase in the feature adversarial based federated learning poisoning detection method provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0020] To make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific implementation methods described herein are only used to explain the present invention and do not limit the scope of protection of the present invention.

[0021] In view of the difficulty in detecting poisoning attacks in existing models, the embodiments of the present invention provide a method and device for detecting poisoning in federated learning based on feature confrontation, the main technical concepts of which are:

[0022] The poisoning attack mechanism is explained by using feature space. The essence of poisoning attack is feature embedding. In principle, the embedded defense patch can be confronted with the features of the poisoned patch and detected according to this confrontation form. Because the privacy mechanism of federated learning makes local data unobservable, malicious data nodes are difficult to detect. At the same time, the backdoor implantation of the model by training-optimizing the poisoning model makes the poisoning attack too hidden, which greatly increases the cost of training the model. Based on this situation, the embodiment proposes a federated learning poisoning detection method and device based on feature confrontation, which specifically includes two stages: in the training stage, a local defense client is used to create a defense patch data set and train a defense model, and an index table representing the label mapping relationship is constructed. In the testing stage, the output relationship before and after the defense backdoor is added to the test sample is compared with the index table for verification. First, after the defense backdoor is added to the benign sample, the feature transformation is known, and the two recognition results should meet the index table rules. Secondly, after the defense backdoor is added to the poisoned sample, the feature transformations of the defense backdoor and the poisoned backdoor are inconsistent, and the two recognition results do not meet the index table rules.

[0023] Figure 1 : is a flow chart of a method for detecting poisoning in federated learning based on feature confrontation provided by an embodiment of the present invention. Figure 1 As shown, the method for detecting poisoning in federated learning based on feature confrontation provided in the embodiment includes the following steps:

[0024] Step 1: Divide all clients in each round of parameter training into benign clients and defense clients, and configure a defense patch dataset for each defense client.

[0025] Before federated learning, model initialization is required, which includes setting the total number of training rounds E, local data sets, and the total number of devices participating in federated learning M. The total number of clients participating in each round of training is M. t , the number of benign clients participating in the training K (K≤M t ), the number of defense clients participating in each round of training DE (DE≤M t ,DE+K=M t ), configure the defense patch dataset DP for the defense client.

[0026] Step 2, such as Figure 2 As shown, in each round of training, the benign client uses the local dataset to optimize the benign model, the defensive client uses the defensive patch dataset and local data to optimize the defensive model, and the server aggregates all benign models and defensive models to obtain the federated learning model.

[0027] For all benign clients participating in the training, each benign client uses formula (1) to optimize the benign model using the local dataset:

[0028]

[0029] Among them, (x, y) are the local datasets D k A sample data and its label, represents the prediction confidence of the benign model of the kth benign client in the tth round of training, L(·) represents the cross entropy loss function between the prediction confidence and the true label y, Represents a benign model The model parameters, represents the benign model of the kth benign client in the tth round of training Formula (1) is understood as the benign client training the model based on the local benign data set.

[0030] For all defense clients participating in the training, each defense client selects the best defense patch dataset that maximizes the information entropy content of the model from the defense patch dataset, and then uses the best defense patch dataset and the local dataset to jointly optimize the defense model.

[0031] When each defense client selects the best defense patch dataset from the defense patch dataset, it uses the active learning strategy to select the candidate defense patches so that the information entropy content of the model is maximized. The greater the information entropy content, the higher the success rate of the poisoning of the patch used for feature embedding. Specifically, formula (2) is used to select the best defense patch dataset DP that maximizes the information entropy content of the model from the defense patch dataset. log :

[0032] DP log = arg max N (-L2[att,len(dp)]log L2[att,len(dp)]) (2)

[0033] Where dp represents the defense patch data selected from the defense patch dataset DP, len(dp) represents the patch size of the defense patch data dp, att represents the attack success rate of the defense patch data dp for other target classes, and N represents the number of selected optimal defense patch data, that is, the optimal defense patch dataset DP. logContains N best defense complement data, L2[att,len(dp)] represents the relationship between the attack success rate att and the patch size len(dp), that is, L2[att,len(dp)]=att*[max(DP)-len(dp)], max(DP) represents the patch size of the largest defense patch data in the defense patch data set DP. Formula (2) is understood as selecting different defense patch data to make the patch trigger success rate of feature embedding the highest and the patch smaller, and multiple cycles are performed to select N defense patch data as the best defense patch data to form the best defense complement data set DP. log .

[0034] In obtaining the best defense complement dataset DP log Finally, the defense client uses formula (3) to jointly optimize the defense model using the best defense patch dataset and the local dataset:

[0035]

[0036] Among them, (x, y) are the local datasets D de A sample data and its label in Represents the defense model of the de-th benign client in the t-th round of training , τ represents the label of the defense target, L(·) represents the cross entropy loss function between the prediction confidence and the label τ of the defense target, dp log Denotes the best defense patch dataset DP log The best defense patch data selected from R(x,dp log ) indicates that the best defense patch data dp is superimposed by the defense patch method R log The data obtained by superimposing it on the sample data x is and Denotes the defense model and and the model parameters.

[0037] In the process of optimizing the defense model, the defense target label τ and the original target label y will form a label mapping relationship and be saved in the index table Hashmap for subsequent poisoning detection.

[0038] The benign client and the defensive client upload the locally optimized benign model and defensive model to the server. The server uses formula (4) to aggregate all benign models and defensive models to obtain the federated learning model:

[0039]

[0040] Among them, G t and G t+1They represent the federated learning models of the tth round and the t+1th round respectively, and They represent the benign model and defense model in the t+1th round respectively, K and DE represent the total number of benign models and defense models respectively, 1 / K is the weight scaling of the benign model, and 1 / DE is the weight scaling of the defense model.

[0041] The server aggregates all benign models and defense models to obtain the federated learning model, and then sends the federated learning model to the benign clients and defense clients as the basis for the next round of training of benign models and defense models.

[0042] Step 3: Repeat step 2 until the total number of rounds E is reached, and update the structural parameters of the edge and server to obtain the last round of optimized federated learning model G. best Used for the detection of poisoned samples.

[0043] Step 4. During detection, the prediction result of the target label of the test sample in the federated learning model is used to determine whether the test sample is poisoned, and whether the prediction result of the defense target label in the federated learning model after the best defense patch data is added to the test sample satisfies the label mapping relationship (that is, the corresponding relationship of feature conflicts), so as to realize federated learning poisoning detection.

[0044] Specifically, Figure 3 As shown in the figure, the method of judging whether the test sample is poisoned based on the label mapping relationship is:

[0045] Input the test sample S into the federated learning model G best , get the prediction result G of the target label best (S);

[0046] The best defense patch dataset DP log The best defense patch data selected in dp log Add to the test sample S to get the data S+dp log Input to the federated learning model G best , and obtain the predicted result G of the defense target label best (S+dp log ), where the test sample S and data S+dp log Form a pair of samples to be tested;

[0047] When G is satisfied best (S) = Hashmap (G best (S+dp log )), then the test sample S is a benign sample;

[0048] When G is satisfied best (S)≠Hashmap(G best(S+dp log )), then the test sample S is a poisoned sample;

[0049] Among them, Hashmap (G best (S+dp log )) represents the prediction result G of the defense target label best (S+dp log ) corresponds to the target label in the label mapping relationship.

[0050] An embodiment also provides a federated learning poisoning detection device based on feature adversarial science, comprising a computer memory, a computer processor, and a computer program stored in the computer memory and executable on the computer processor, wherein the computer processor implements the above-mentioned federated learning poisoning detection method based on feature adversarial science when executing the computer program.

[0051] In practical applications, the processor can be implemented by a central processing unit (CPU), a microprocessor (MPU), a digital signal processor (DSP), or a field programmable gate array (FPGA) of a base station server.

[0052] The above-mentioned feature-based adversarial federated learning poisoning detection and device address the problem of lack of explanation for federated learning poisoning attacks, realize the explanation of the working mechanism of federated learning model poisoning attacks, realize the feasibility analysis of detection, and improve the robustness of the model. At the same time, before the defense patch is uploaded to the server, the semi-supervised learning method is used to select the defense patch that makes the model training fastest or helps the defense patch embedding the most, which can make federated learning converge faster and will not lose performance due to the introduction of adversarial clients. In addition, there is no need to modify the model structure, and both institutions and companies can use it, reducing the cost of use and the complexity of operation.

[0053] The specific implementation methods described above provide a detailed description of the technical solutions and beneficial effects of the present invention. It should be understood that the above is only the most preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, supplements and equivalent substitutions made within the scope of the principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A method for detecting poisoning in federated learning based on feature adversarial, characterized in that: The following steps are involved: Divide all clients in each round of parameter training into benign clients and defense clients, and configure a defense patch dataset for each defense client; In each round of training, the benign client uses the local data set to optimize the benign model. The defensive client selects the best defense patch data set that maximizes the information entropy content of the model from the defense patch data set, and then uses the best defense patch data set and the local data set to jointly optimize the defense model. The server aggregates all benign models and defense models to obtain a federated learning model. After multiple rounds of training, the last round of federated learning model is used to detect poisoned samples. During the detection, the prediction result of the target label of the test sample in the federated learning model and the prediction result of the defense target label in the federated learning model after the best defense patch data is added to the test sample satisfy the label mapping relationship to determine whether the test sample is poisoned, that is, to achieve federated learning poisoning detection.

2. The method for detecting poisoning in federated learning based on feature confrontation as claimed in claim 1, characterized in that: The benign client uses formula (1) to optimize the benign model using the local dataset: Among them, (x, y) are the local datasets D k A sample data and its label, represents the prediction confidence of the benign model of the kth benign client in the tth round of training, L(·) represents the cross entropy loss function between the prediction confidence and the true label y, Represents a benign model The model parameters, represents the benign model of the kth benign client in the tth round of training The model parameters.

3. The method for detecting poisoning in federated learning based on feature confrontation as claimed in claim 1, characterized in that: The defense client uses active learning strategy and adopts formula (2) to select the best defense patch dataset DP that maximizes the information entropy content of the model from the defense patch dataset. log : DP log =angry max N (-L2[att,len(dp)]logL2[att,len(dp)]) (2) Where dp represents the defense patch data selected from the defense patch dataset DP, len(dp) represents the patch size of the defense patch data dp, att represents the attack success rate of the defense patch data dp for other target classes, and N represents the number of selected optimal defense patch data, that is, the optimal defense patch dataset DP. log Contains N optimal defense patch data, L2[att,len(dp)] represents the relationship between the attack success rate att and the patch size len(dp), that is, L2[att,len(dp)]=att*[max(DP)-len(dp)], max(DP) represents the patch size of the maximum defense patch data in the defense patch dataset DP.

4. The method for detecting poisoning in federated learning based on feature confrontation as claimed in claim 1 or 3, characterized in that: The defense client uses formula (3) to jointly optimize the defense model using the best defense patch dataset and the local dataset: Among them, (x, y) are the local datasets D de A sample data and its label in Represents the defense model of the de-th benign client in the t-th round of training , τ represents the label of the defense target, L(·) represents the cross entropy loss function between the prediction confidence and the label τ of the defense target, dp log Denotes the best defense patch dataset DP log The best defense patch data selected from R(x,dp log ) indicates that the best defense patch data dp is superimposed by the defense patch method R log The data obtained by superimposing it on the sample data x is and Denotes the defense model and The model parameters.

5. The method for detecting poisoning in federated learning based on feature confrontation as claimed in claim 1, characterized in that: The server uses formula (4) to aggregate all benign models and defense models to obtain the federated learning model: Among them, G t and G t+1 They represent the federated learning models of the tth round and the t+1th round respectively, and They represent the benign model and defense model in the t+1th round respectively, and K and DE represent the total number of benign models and defense models respectively.

6. The method for detecting poisoning in federated learning based on feature confrontation as claimed in claim 1, characterized in that: The method of judging whether the test sample is poisoned based on the label mapping relationship is: Input the test sample S into the federated learning model G best , get the prediction result G of the target label best (S); The best defense patch dataset DP log The best defense patch data selected in dp log Add to the test sample S to get the data S+dp log Input to the federated learning model G best , and obtain the predicted result G of the defense target label best (S+dp log ); When G is satisfied best (S) = Hashmap (G best (S+dp log )), then the test sample S is a benign sample; When G is satisfied best (S)≠Hashmap(G best (S+dp log )), then the test sample S is a poisoned sample; Among them, Hashmap (G best (S+dp log )) represents the prediction result G of the defense target label best (S+dp log ) corresponds to the target label in the label mapping relationship.

7. The method for detecting poisoning in federated learning based on feature confrontation as claimed in claim 1, characterized in that: During each round of training, the benign client and the defensive client upload the locally optimized benign model and the defensive model to the server. The server aggregates all the benign models and the defensive models to obtain the federated learning model, and then sends the federated learning model to the benign client and the defensive client as the basis for the benign model and the defensive model for the next round of training.

8. A federated learning poisoning detection device based on feature confrontation, comprising a computer memory, a computer processor, and a computer program stored in the computer memory and executable on the computer processor, characterized in that: When the computer processor executes the computer program, the method for detecting poisoning in federated learning based on feature adversarial communication described in any one of claims 1 to 7 is implemented.

Images