Federal learning poisoning attack detection method based on gradient credibility

Abstract

The invention discloses a federal learning poisoning attack detection method based on gradient credibility. The method comprises the following steps: a client performs model training on an image data set under a federated learning framework, and sends a training result to a verification party and a server after training is completed; the verification party verifies the local model, summarizes verification results and sends the verification results to the server, and the server selects the optimal gradient and the most harmful gradient in the current round according to the verification results and stores the optimal gradient and the most harmful gradient in the server; meanwhile, the server calculates the credibility of the local gradient, malicious clients are filtered based on the credibility, the credibility is processed to serve as an aggregation weight during aggregation, and a credible global model is obtained through aggregation. The method can effectively identify malicious client poisoning attacks, and solves the problems that the detection effect is reduced when the number of malicious clients is greater than 50% in a traditional detection method, and the detection effect is unstable under different data sets and attack methods.

Description

Technical Field

[0001] The present invention relates to the technical field of federated learning, and in particular to a federated learning poisoning attack detection method based on gradient credibility. Background Art

[0002] Federated learning is an emerging distributed machine learning approach that allows models to be trained across multiple devices or institutions without centralizing data. This approach is particularly suitable for scenarios that require privacy protection or compliance with data protection regulations, such as healthcare, finance, and the Internet of Things. In addition, federated learning can improve the generalization ability of the model because it is able to leverage diverse data scattered across different devices. However, this architecture also makes it vulnerable to poisoning attacks, where malicious actors deliberately submit incorrect or harmful updates to affect the performance of the final model.

[0003] In federated learning, each client trains the model locally and sends its updates to a central server. An attacker can manipulate the model's training process by submitting false or inappropriate updates, causing the model to favor the attacker's goals. Although federated learning has significant advantages in protecting privacy and decentralizing computing, its security still needs to be further strengthened. In existing technologies, most detection methods for poisoning attacks consider the situation where the number of malicious clients is less than 50%, and pay less attention to the situation where the number of malicious clients is large. Therefore, the detection effect will be significantly reduced when the number of malicious clients is greater than 50%. At the same time, when training on different training data sets to deal with different poisoning attacks, the robustness of the detection method cannot be guaranteed to prevent the potential threat of poisoning attacks to model performance. Summary of the invention

[0004] The purpose of the present invention is to provide a federated learning poisoning attack detection method based on gradient credibility; the method of the present invention can effectively identify malicious client poisoning attacks, solves the problem that the detection effect of traditional detection methods decreases when the number of malicious clients is greater than 50%, and the detection effect is unstable under different data sets and attack methods.

[0005] In order to achieve the above object, the present invention provides a new method for detecting a malicious client poisoning attack, comprising the following steps:

[0006] Step 1: The client performs model training on the image dataset that has been classified under the federated learning framework, and the number of categories is set to k. The client performs model training on the image dataset that has been classified under the federated learning framework, and the number of categories is set;

[0007] Step 2: The verifier verifies the local models of all clients on different categories. The verification set selects a part of the image data set to calculate the difference between the loss value of the current local model and the loss value of the previous round of global model. After the verifier completes the verification of all local models, it summarizes the verification results of this round and uploads them to the central server;

[0008] Step 3: The central server preprocesses the verification results sent by the verifier in this round, selects the best gradient and the worst gradient in this round of training by analyzing the preprocessing results, and stores them in the server for the next round of credibility calculation;

[0009] Step 4: Perform credibility calculation. The central server obtains credibility by calculating the difference between the local gradient in this round of training and the cosine similarity of the best gradient and the worst gradient in the previous round of training.

[0010] Step 5: The central server identifies malicious clients in this round of training through credibility and filters them. The aggregation weight of the filtered clients is set to 0 when aggregating the global model in this round. The aggregation weight of the remaining clients is calculated through credibility and participates in the aggregation of the global model in this round.

[0011] Step 6: The central server distributes the global model of this round to all clients. Each client starts the next round of training on the global model and repeats steps 1 to 5 to filter malicious clients during the model training process until the model converges.

[0012] In the present invention, in step 1, all clients send the training results to the verifier for verification, and send them to the central server for gradient credibility calculation.

[0013] In the present invention, in step 2, the verifier selects a part from the image data set to serve as a verification set, selects k samples of different categories, where k is the number of categories under the current classification task, and divides the verification set by category. The local model is verified based on the difference in class loss values. Under the same class, the loss value of the previous round of the global model and the loss value of the current local model are calculated respectively, and the difference between the two is calculated. This operation is repeated under all classes to complete the verification of the current local model; the verification results of all local models are summarized into an array R, and the verification result R is sent to the central server.

[0014] In the present invention, the verification result R is a two-dimensional array, expressed as Where n is the number of clients participating in the training, k is the number of categories, Represents the local model w p , in category c qThe difference between the loss value of the global model and the loss value expression is L(w,y)=-log(p y ), where y represents the classification label, p y represents the predicted probability of category y under the local model w.

[0015] In the present invention, in step 3, the central server processes the verification result R twice, wherein the first processing calculates the mean of the difference between the loss value of the global model in the previous round and the local model in all categories as the mean excellence value of the current local model. The expression is: The second process calculates the minimum difference between the loss value of the global model in the previous round and the local model in all classes as the minimum excellence value ψ of the current local model, which is expressed as: The gradient with the highest average excellence score among all local models is taken as the optimal gradient of this round, and the gradient with the lowest score among the minimum excellence scores is taken as the worst gradient of this round.

[0016] In the present invention, in step 4, the credibility calculation formula is:

[0017] S(g i ) = g i ·g b -g i ·g m ;

[0018] Where: g b represents the optimal gradient in the previous round of training, g m represents the worst gradient in the last round of training, g i represents the local gradient of the i-th client, g x ·g y Represents the cosine similarity between two gradients.

[0019] In the present invention, in step 5, it is determined whether the credibility of the current client is greater than zero. If the credibility is a positive value, it indicates that the current gradient is a benign gradient, which will participate in model aggregation and use the credibility as the aggregation weight; if the credibility is a negative value, it indicates that the current gradient is a malicious gradient, and the aggregation weight is set to 0 during the current round of global model aggregation to filter the current client. Finally, the central server aggregates the weight value and the pruned local model to obtain the global model of this round.

[0020] In the present invention, the local model can be an image classification model such as CNN, Resnet, etc.

[0021] Compared with the prior art, the present invention has the following beneficial effects:

[0022] The present invention selects the optimal gradient and the worst gradient by the difference between the loss value of the client's local model on different classes and the loss value of the previous round of global model, and considers the difference in loss values ​​on multiple different classes at the same time to more comprehensively evaluate the local model; the present invention also uses the optimal gradient and the worst gradient to evaluate the client's local gradient, and compares the similarity of the local gradient with the benign gradient and the malicious gradient, which can more effectively reflect whether the local gradient is a malicious gradient; the present invention filters the malicious gradient uploaded by the malicious client based on the credibility, processes the credibility, and uses it to aggregate to obtain the global model. This method can effectively identify malicious poisoning clients in the model training process, and when the number of malicious clients is greater than 50%, it can still ensure a certain stability to resist various poisoning attacks, thereby ensuring the credibility and security of federated learning. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] Figure 1 A flowchart of a gradient credibility-based federated learning poisoning attack detection method provided by the present invention.

[0024] Figure 2 A diagram of the system architecture for resisting poisoning attacks during the federated learning training phase provided by the present invention.

[0025] Figure 3 This is the model accuracy result of the method provided by the present invention when resisting label reversal attacks on the Fmnist dataset.

[0026] Figure 4 This is the model recall rate result of the method provided by the present invention when defending against label reversal attacks on the Fmnist dataset.

[0027] Figure 5 This is the model accuracy result of the method provided by the present invention when defending against backdoor attacks on the Fmnist dataset.

[0028] Figure 6 This is the backdoor accuracy result of the method provided by the present invention when defending against backdoor attacks on the Fmnist dataset. DETAILED DESCRIPTION

[0029] The present invention will be further described below by way of the accompanying drawings and examples.

[0030] This embodiment provides a method for detecting federated learning poisoning attacks based on gradient credibility. Figure 2 As shown, it shows that the system architecture provided by the present invention consists of three parts: client, verifier and central server; wherein:

[0031] The client performs model training in the process of federated learning;

[0032] The verifier verifies the local model trained by the client;

[0033] The central server is responsible for processing the verification results, selecting the optimal gradient and the worst gradient, calculating the credibility, and obtaining the global model based on the credibility aggregation.

[0034] Accordingly, a gradient credibility-based federated learning poisoning attack detection method includes:

[0035] The client trains the model on the image dataset under the federated learning framework and sends the training results to the verifier and the server after the training is completed;

[0036] The verifier verifies the local model and sends the verification results to the server.

[0037] The server selects the best and worst gradients in this round through verification results and stores them in the server;

[0038] At the same time, the server will calculate the credibility of the local gradient and filter malicious clients based on the credibility to prevent them from affecting the global model during the aggregation process. During aggregation, the credibility will be processed and used as the aggregation weight to obtain a credible global model.

[0039] The following are specific embodiments.

[0040] Example 1

[0041] A gradient credibility-based federated learning poisoning attack detection method, such as Figure 1 As shown, it includes the following steps:

[0042] Step 1: The client performs model training on the classified image dataset under the federated learning framework. The number of categories is set to k. There are a certain number of malicious clients among the clients. Malicious clients will poison the network during the training process. After all clients have completed training, the local model and local gradient obtained from the training are sent to the verifier and the central server.

[0043] Step 2: The verifier verifies the local models of all clients on different categories. The verification set selects a part of the image data set to calculate the difference between the loss value of the current local model and the loss value of the previous round of global model. After the verifier completes the verification of all local models, it summarizes the verification results of this round and uploads them to the central server;

[0044] Step 3: The central server preprocesses the verification results sent by the verifier in this round, selects the best gradient and the worst gradient in this round of training by analyzing the preprocessing results, and stores them in the server for the next round of credibility calculation;

[0045] Step 4: Perform credibility calculation. The central server obtains credibility by calculating the difference between the local gradient in this round of training and the cosine similarity of the best gradient and the worst gradient in the previous round of training.

[0046] Step 5: The central server identifies malicious clients in this round of training through credibility and filters them. The aggregation weight of the filtered clients is set to 0 when aggregating the global model in this round. The aggregation weight of the remaining clients is calculated through credibility and participates in the aggregation of the global model in this round.

[0047] Step 6: The central server distributes the global model of this round to all clients. Each client starts the next round of training on the global model and repeats steps 1 to 5 to filter malicious clients during the model training process until the model converges.

[0048] More specific steps are:

[0049] Step 1 is specifically as follows: in this embodiment, the client performs model training on three image datasets, FMNIST, EMNIST and CIFAR-10. When training on FMNIST or EMNIST datasets, a standard convolutional neural network (CNN) is used. When training on CIFAR-10 datasets, a lightweight Resnet-18 model is used. In each iteration of model training, the number of clients is set to 25 clients, of which malicious clients account for 60%, the learning rate is set to 0.1, and the local batch size is set to 64. Malicious clients can poison the training dataset or training results by tampering with them during training. The poisoning attack algorithms used include distributed backdoor attacks, inner product manipulation attacks, and label reversal attacks. The rounds of poisoning start from the 35th round on FMNIST and EMNIST datasets and from the 200th round on CIFAR-10 datasets. After all clients complete training, all clients send local models to the verifier for verification and send local gradients to the central server for gradient credibility calculation. In this embodiment, the verifier is initially randomly selected from the client. In subsequent training rounds, it will be selected based on credibility. The client with high credibility in this round will serve as the verifier in the next round.

[0050] Specifically, step 2 is that the verifier verifies the local model based on the difference in class loss values. In each round of training, k samples of different categories are selected. The value of k is determined by the number of categories of the classification task. In this embodiment, k=10 is selected. The verifier selects part of the training data set as the verification set, and divides the verification set by category. Under the same category, the loss value of the global model in the previous round and the loss value of the current local model are calculated respectively, and the difference between the two is calculated. This operation is repeated under all classes to complete the verification of the current local model. The difference in the loss values ​​of all local models under different classes is summarized to obtain the verification result R, and the verification result R is sent to the central server.

[0051] Among them, the verification result R is a two-dimensional array, the expression is Where n is the number of clients participating in the training, k is the number of categories, where Represents the local model w p , in category c q The loss value difference between the global model and the local model is expressed as L(w,y)=-log(p y ), where y represents the classification label, p y represents the predicted probability of category y under the local model w.

[0052] Specifically, in step 3, the central server processes the verification result R of this round twice. The first processing calculates the average difference between the loss value of the global model in the previous round and the local model in all categories as the average excellence of the current local model. The average excellence value under all classes can better reflect the average performance of the current model under all classes and better measure the stability of the current local model. The second processing will calculate the minimum difference between the loss value of the global model in the previous round and the local model under all classes as the minimum excellence value ψ of the current local model. The extreme performance of the local model in individual classes is more in line with the idea of ​​poisoning specific categories in actual attacks. The gradient obtained by training the client with the highest average excellence value among all local models is used as the optimal gradient of this round, and the gradient obtained by training the client with the lowest minimum excellence value is used as the worst gradient of this round. The selected optimal and worst gradients will be stored in the central server and will be used for the credibility calculation of local gradients in the next round of training.

[0053] in, represents the mean excellence of the local model w, expressed as: ψ(w) represents the minimum excellence of the local model w, and its expression is:

[0054] Specifically, step 4 is that the central server will calculate the credibility of all local gradients in this round. The server will also select the best gradient and the worst gradient according to the method of step 3 after receiving the verification result in the previous round of training, and store them in the server for the credibility calculation of this round. Malicious clients are identified based on the credibility results and filtered out. The credibility S(g i ) = g i ·g b -g i ·g m , g b represents the optimal gradient in the previous round of training, g m represents the worst gradient in the last round of training, g i It represents the local gradient of the i-th client, g x ·g y represents the cosine similarity between the gradients. If the local gradient is benign, it will be similar to g b The similarity is greater than that of g m The smaller the similarity, the larger the difference, and the higher the credibility score of the current gradient, and vice versa. If the credibility of all clients in the current round of training is shown as malicious clients, it means that the optimal gradient and the worst gradient selected in the previous round of training may not be good enough. In this case, g will be reselected from the historical optimal gradient and the worst gradient of the server. b and g m , and recalculate the credibility of the current round of gradients. It should be noted that during the first round of training, the server does not store the best and worst gradients of the previous round. When aggregating, the clients with the top 50% credibility are selected by default to participate in model aggregation.

[0055] Specifically, step 5 is that the central server will filter out malicious clients through gradient credibility and determine whether the current credibility is greater than zero. If the credibility is greater than zero, it means that the current gradient is a benign gradient and will participate in model aggregation, and the credibility will be used as the aggregation weight. If the current credibility is a negative value, it means that the current gradient is a malicious gradient, and the current client will be filtered during this round of model aggregation, and the aggregation weight will be set to zero. Then the local model is pruned. In this embodiment, the pruning method in FLAME is used to prune the local model. This method is a commonly used adaptive pruning method, and the expression is w * =G t-1 +(wG t-1 )·min(1,γ), where w * Represents the model update after this adaptive cropping, G t-1represents the global model of the previous round, w represents the local model update of the current client, S is the threshold of adaptive clipping, e is the L2 norm of the current model update, and finally the global model of this round is obtained by weighted average aggregation of credibility and the clipped local model. The aggregation expression is where α n represents the aggregation weight, and N represents the number of clients participating in the aggregation.

[0056] Specifically, step 6 is that the central server distributes the aggregated global model to all clients. Each client starts the next round of training on the global model and filters malicious clients during the training process. The filtered clients set the aggregation weight to 0 when aggregating the global model in this round to prevent them from affecting the global model. Repeat steps 1 to 5 to train the model until the model converges.

[0057] In the present invention, the filtered malicious clients do not participate in the aggregation of the global model in this round, but will participate in the federated learning training normally in the next round; the purpose of the present invention is to filter out the bad results in this round of training and aggregate to obtain the best possible global model.

[0058] Table 1 compares the proposed method with two extreme methods, namely the poisoned model trained with the method of no detection of poisoning attacks and the perfect model trained with the method of assuming perfect filtering of malicious clients in each round of detection. The final model has the accuracy, recall rate, and backdoor accuracy rate under label reversal attacks and distributed backdoor attacks.

[0059] Table 1

[0060]

[0061] Table 1 shows the effects of the final model obtained by the proposed method under different data sets and different attacks. As can be seen from the table, this method can ensure a certain degree of robustness under different attacks on various data sets, can successfully resist label reversal attacks and distributed backdoor attacks, and its accuracy MA and recall RR are close to the final model obtained by perfect resistance to poisoning attacks. In some cases, the model obtained by aggregation through this method is even better than the perfect model obtained by completely filtering malicious clients. This is because the aggregation algorithm in the proposed method will further affect the global model obtained by aggregation based on credibility as a weight. For example, on the Cifar-10 dataset, the recall rate RR of the final model is higher than the recall rate in the perfect detection scheme, up to 84.200%; its backdoor accuracy BA is also lower, only 1.567%.

[0062] exist Figure 3 , Figure 4In the figure, the accuracy and recall comparison of this method in dealing with label reversal attacks on the FMNIST dataset is shown, compared with the model obtained by perfect detection under ideal conditions and the model obtained by complete poisoning without any defense measures. Figure 5 , Figure 6 The comparison chart of the accuracy of this method in dealing with distributed backdoor attacks and backdoor accuracy under the FMNIST dataset is shown. From the comparison chart, it can be seen that the performance of the model obtained by this method under different attacks is close to the model obtained by perfectly detecting malicious clients.

[0063] In summary, the present invention is a detection method for poisoning attacks by malicious clients in image classification scenarios in federated learning neighborhoods. The optimal gradient and the worst gradient are selected by verifying the user's local model on different classes, and the gradient credibility is calculated by the similarity difference between the local gradient and the optimal gradient and the worst gradient. The client of the malicious poisoning attack is identified based on the credibility, and the credibility is processed and used for aggregation to obtain a global model. This method can achieve a good defense effect against different poisoning attacks under various data sets. At the same time, when the number of malicious clients is greater than 50%, it can also identify the poisoner well, and has a certain robustness.

[0064] The above is only a preferred embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any simple modification, equivalent substitution and improvement made by any technical personnel in this field without departing from the technical scope disclosed in the present invention shall still fall within the scope of the technical solution of the present invention.

Claims

1. A method for detecting poisoning attacks in federated learning based on gradient credibility, characterized in that: The steps include: Step 1: The client performs model training on the classified image dataset under the federated learning framework. The number of categories is set to k. There are a certain number of malicious clients among the clients. Malicious clients will poison the network during the training process. After all clients have completed training, the local model and local gradient obtained from the training are sent to the verifier and the central server. Step 2: The verifier verifies the local models of all clients on different categories. The verification set selects a part of the image data set to calculate the difference between the loss value of the current local model and the loss value of the previous round of global model. After the verifier completes the verification of all local models, it summarizes the verification results of this round and uploads them to the central server; Step 3: The central server preprocesses the verification results sent by the verifier in this round, selects the best gradient and the worst gradient in this round of training by analyzing the preprocessing results, and stores them in the server for the next round of credibility calculation; Step 4: Perform credibility calculation. The central server obtains credibility by calculating the difference between the local gradient in this round of training and the cosine similarity of the best gradient and the worst gradient in the previous round of training. Step 5: The central server identifies malicious clients in this round of training through credibility and filters them. The aggregation weight of the filtered clients is set to 0 when aggregating the global model in this round. The aggregation weight of the remaining clients is calculated through credibility and participates in the aggregation of the global model in this round. Step 6: The central server distributes the global model of this round to all clients. Each client starts the next round of training on the global model and repeats steps 1 to 5 to filter malicious clients during the model training process until the model converges.

2. The gradient credibility-based federated learning poisoning attack detection method according to claim 1 is characterized in that: In step 1, all clients send the training results to the verifier for verification and to the central server for gradient credibility calculation.

3. The gradient credibility-based federated learning poisoning attack detection method according to claim 1, characterized in that: In step 2, the verifier selects part of the image dataset as the verification set, selects k samples of different categories, where k is the number of categories under the current classification task, and divides the verification set by category. The local model is verified based on the difference in class loss values. Under the same category, the loss value of the previous round of the global model and the loss value of the current local model are calculated respectively, and the difference between the two is calculated. This operation is repeated under all classes to complete the verification of the current local model. The verification results of all local models are summarized into an array R, and the verification result R is sent to the central server.

4. The gradient credibility-based federated learning poisoning attack detection method according to claim 3 is characterized in that: The verification result R is a two-dimensional array, expressed as Where n is the number of clients participating in the training, k is the number of categories, Represents the local model w p , in category c q The difference between the loss value of the global model and the loss value expression is L(w,y)=-log(p y ), where y represents the classification label, p y represents the predicted probability of category y under the local model w.

5. The gradient credibility-based federated learning poisoning attack detection method according to claim 1, characterized in that: In step 3, the central server processes the verification result R twice. The first process calculates the average of the difference between the loss value of the global model in the previous round and the local model in all categories as the average excellence of the current local model. The expression is: The second process calculates the minimum difference between the loss value of the global model in the previous round and the local model in all classes as the minimum excellence value ψ of the current local model, which is expressed as: The gradient with the highest average excellence score among all local models is taken as the optimal gradient of this round, and the gradient with the lowest score among the minimum excellence scores is taken as the worst gradient of this round.

6. The gradient credibility-based federated learning poisoning attack detection method according to claim 1, characterized in that: In step 4, the credibility calculation formula is: S(g i )=g i ·g b -g i ·g m ; Where: g b represents the optimal gradient in the previous round of training, g m represents the worst gradient in the last round of training, g i represents the local gradient of the i-th client, g x ·g y Represents the cosine similarity between two gradients.

7. The gradient credibility-based federated learning poisoning attack detection method according to claim 1, characterized in that: In step 5, determine whether the credibility of the current client is greater than zero. If the credibility is positive, it means that the current gradient is a benign gradient, which will participate in model aggregation and use the credibility as the aggregation weight. If the credibility is negative, it means that the current gradient is a malicious gradient. In this round of global model aggregation, the aggregation weight is set to 0 to filter the current client. Finally, the central server aggregates the weight value and the pruned local model to obtain the global model of this round.

Images