Gradient confidence level-based poisoning attack detection method for federated learning

Abstract

Disclosed is a gradient confidence level-based poisoning attack detection method for federated learning. The method comprises: clients perform model training for an image dataset under a federated learning framework, and send training results to a verifier and a server when the training is complete; the verifier verifies local modes, summarizes verification results and sends them to the server; the server selects the best gradient and worst gradient from the current round, and stores them in the server; and in addition, the server calculates confidence levels of local gradients, filters a malicious client on the basis of the confidence levels, and uses processed confidence levels as aggregation weights to perform aggregation so as to obtain a trusted global model. The method can effectively identify poisoning attacks from malicious clients, and solves the problem that conventional detection methods exhibit a reduced detection effect when the proportion of malicious clients exceeds 50%, and an unstable detection effect across different datasets and attack methods.

Description

A Federated Learning-Based Poisoning Attack Detection Method Based on Gradient Reliability Technical Field

[0001] This invention relates to the field of federated learning technology, and more specifically, to a federated learning poisoning attack detection method based on gradient credibility. Background Technology

[0002] Federated learning is an emerging distributed machine learning approach that allows models to be trained across multiple devices or institutions without centralized data. This approach is particularly suitable for scenarios requiring privacy protection or compliance with data protection regulations, such as healthcare, finance, and the Internet of Things (IoT). Furthermore, federated learning can improve a model's generalization ability because it can leverage diverse data distributed across different devices. However, this architecture also makes it vulnerable to poisoning attacks, where malicious actors intentionally submit erroneous or harmful updates to negatively impact the performance of the final model.

[0003] In federated learning, each client trains the model locally and sends its updates to a central server. Attackers can manipulate the model's training process by submitting false or improper updates, causing the model to be biased towards the attacker's target. While federated learning offers significant advantages in protecting privacy and distributing computation, its security still needs further improvement. Existing poisoning attack detection methods mostly consider scenarios where the number of malicious clients is less than 50%, paying less attention to situations with a large number of malicious clients. Therefore, detection effectiveness drops significantly when dealing with more than 50% of malicious clients. Furthermore, when training on different datasets to handle different poisoning attacks, the robustness of the detection method cannot be guaranteed, failing to prevent potential threats to model performance from poisoning attacks. Summary of the Invention

[0004] The purpose of this invention is to provide a federated learning poisoning attack detection method based on gradient credibility. This invention can effectively identify malicious client poisoning attacks and solves the problems of decreased detection performance of traditional detection methods when the number of malicious clients exceeds 50%, as well as unstable detection performance under different datasets and attack methods.

[0005] To achieve the above objectives, this invention provides a novel method for detecting malicious client-side poisoning attacks, comprising the following steps:

[0006] Step 1: The client trains a model on the pre-classified image dataset under the federated learning framework, with the number of classes set to [number to be filled in]. The client trains a model on a pre-classified image dataset within the federated learning framework, with the number of classes set.

[0007] Step 2: The validator validates the local models of all clients on different categories. The validation set is selected from a portion of the image dataset. The difference between the loss value of the current local model and the previous round of global model is calculated. After the validator has validated all local models, the validation results of this round are summarized and uploaded to the central server.

[0008] Step 3: The central server preprocesses the verification results sent by the verification party in this round. By analyzing the preprocessing results, the optimal gradient and the worst gradient in this round of training are selected and stored in the server for the credibility calculation in the next round.

[0009] Step 4: Perform credibility calculation. The central server obtains the credibility by calculating the difference between the cosine similarity between the local gradient in this round of training and the best and worst gradient in the previous round of training.

[0010] Step 5: The central server identifies malicious clients in this round of training based on credibility and filters them. The aggregate weight of the filtered clients is set to 0 when aggregating the global model in this round. The aggregate weight of the remaining clients is calculated based on credibility and participates in the aggregation of the global model in this round.

[0011] Step 6: The central server distributes the global model for this round to all clients. Each client starts the next round of training on the global model and repeats steps 1 to 5 to filter malicious clients during the model training process until the model converges.

[0012] In this invention, in step 1, all clients send the training results to the verification party for verification, and send them to the central server for gradient confidence calculation.

[0013] In this invention, in step 2, the verifier selects a portion of the image dataset to serve as the verification set. Samples of different categories, Given the number of classes in the current classification task, the validation set is divided by class. The local model is validated based on the difference in class loss values. Within the same class, the loss value of the previous global model and the loss value of the current local model are calculated, and their difference is calculated. This process is repeated for all classes to complete the validation of the current local model. The validation results of all local models are then summarized into an array. In, and verify the results Send to the central server.

[0014] In this invention, the verification result is... It is a two-dimensional array, expressed as: ,in The number of clients participating in the training. For the number of categories, Represents the local model In category The difference between the loss value of the lower and global models is expressed as follows: ,in Indicates category labels, Indicates local model Next category The predicted probability.

[0015] In this invention, in step 3, the central server verifies the results. The process involves two steps. The first step calculates the average difference between the global model's loss and the local model's loss across all categories in the previous round, which is then used as the average excellence score of the current local model. The expression is: The second processing step calculates the minimum difference between the global model's loss and the local model's loss across all classes from the previous round, which is then used as the minimum excellence score of the current local model. The expression is: The gradient with the highest average score among all local models is taken as the best gradient in this round, and the gradient with the lowest score among the minimum values ​​of excellence is taken as the worst gradient in this round.

[0016] In this invention, the credibility calculation formula in step 4 is:

[0017] ;

[0018] in: This represents the optimal gradient in the previous training round. This represents the worst gradient in the previous training round. Indicates the first Local gradients for each client, This represents the cosine similarity between two gradients.

[0019] In this invention, in step 5, it is determined whether the credibility of the current client is greater than zero. If the credibility is positive, it indicates that the current gradient is a benign gradient and will participate in model aggregation, with credibility as the aggregation weight. If the credibility is negative, it indicates that the current gradient is a malicious gradient, and the aggregation weight is set to 0 in this round of global model aggregation to filter the current client. Finally, the central server aggregates the weight values ​​and the pruned local model to obtain the global model for this round.

[0020] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0021] This invention selects the optimal and worst gradients by comparing the loss values ​​of the client's local model (which can be any image classification model, such as VGGNet, InceptionNet, etc.) across different classes with the loss value of the previous global model. Considering the differences in loss values ​​across multiple classes allows for a more comprehensive evaluation of the local model. Furthermore, this invention uses the optimal and worst gradients to evaluate the client's local gradients, comparing the similarity between the local gradients and benign and malicious gradients to more effectively reflect whether a local gradient is malicious. This invention filters malicious gradients uploaded by malicious clients based on credibility, and the processed credibility is used to aggregate and obtain the global model. This method can effectively identify malicious clients during model training. Even when the number of malicious clients exceeds 50%, it can still ensure a certain level of stability to resist various poisoning attacks, guaranteeing the credibility and security of federated learning. Attached Figure Description

[0022] Figure 1 is a flowchart of a federated learning poisoning attack detection method based on gradient credibility provided by the present invention.

[0023] Figure 2 is a system architecture diagram of the federated learning training phase to resist poisoning attacks provided by the present invention.

[0024] Figure 3 shows the model accuracy results of the method provided in this invention on the Fmnist dataset when resisting label inversion attacks.

[0025] Figure 4 shows the model recall results of the method provided in this invention on the Fmnist dataset when resisting label inversion attacks.

[0026] Figure 5 shows the model accuracy results of the method provided in this invention on the Fmnist dataset when resisting backdoor attacks.

[0027] Figure 6 shows the backdoor accuracy results of the method provided in this invention on the Fmnist dataset when resisting backdoor attacks. Detailed Implementation

[0028] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0029] This embodiment provides a federated learning poisoning attack detection method based on gradient credibility, as shown in Figure 2. It illustrates that the system architecture provided by this invention consists of three parts: a client, a verifier, and a central server; wherein:

[0030] The client performs model training during the federated learning process;

[0031] The verification party verifies the local model trained on the client side;

[0032] The central server is responsible for processing the verification results, selecting the optimal and worst gradients, calculating the confidence level, and aggregating the results based on the confidence level to obtain the global model.

[0033] Accordingly, a federated learning poisoning attack detection method based on gradient credibility includes:

[0034] The client trains a model on the image dataset under the federated learning framework, and after training is completed, sends the training results to the verifier and the server.

[0035] The verifier verifies the local model and sends the verification results to the server.

[0036] The server selects the best and worst gradients in this round based on the verification results and stores them on the server.

[0037] At the same time, the server calculates the credibility of the local gradient and filters malicious clients based on the credibility to prevent them from affecting the global model during the aggregation process. The credibility is processed and used as the aggregation weight during aggregation to obtain a credible global model.

[0038] The following are specific examples.

[0039] Example 1

[0040] A federated learning poisoning attack detection method based on gradient credibility, as shown in Figure 1, includes the following steps:

[0041] Step 1: The client trains a model on the pre-classified image dataset under the federated learning framework, with the number of classes set to [number to be filled in]. There are a certain number of malicious clients in the client. These malicious clients will poison the training process. After all clients have completed training, they will send the trained local model and local gradient to the verification party and the central server.

[0042] Step 2: The validator validates the local models of all clients on different categories. The validation set is selected from a portion of the image dataset. The difference between the loss value of the current local model and the previous round of global model is calculated. After the validator has validated all local models, the validation results of this round are summarized and uploaded to the central server.

[0043] Step 3: The central server preprocesses the verification results sent by the verification party in this round. By analyzing the preprocessing results, the optimal gradient and the worst gradient in this round of training are selected and stored in the server for the credibility calculation in the next round.

[0044] Step 4: Perform credibility calculation. The central server obtains the credibility by calculating the difference between the cosine similarity between the local gradient in this round of training and the best and worst gradient in the previous round of training.

[0045] Step 5: The central server identifies malicious clients in this round of training based on credibility and filters them. The aggregate weight of the filtered clients is set to 0 when aggregating the global model in this round. The aggregate weight of the remaining clients is calculated based on credibility and participates in the aggregation of the global model in this round.

[0046] Step 6: The central server distributes the global model for this round to all clients. Each client starts the next round of training on the global model and repeats steps 1 to 5 to filter malicious clients during the model training process until the model converges.

[0047] The more specific steps are as follows:

[0048] Step 1 specifically involves the following steps: In this embodiment, the client trains the model on three image datasets: FMNIST, EMNIST, and CIFAR-10. For training on the FMNIST or EMNIST datasets, a standard convolutional neural network (CNN) is used; for training on the CIFAR-10 dataset, a lightweight ResNet-18 model is used. In each training iteration, the number of clients is set to 25, with malicious clients accounting for 60%. The learning rate is set to 0.1, and the local batch size is set to 64. Malicious clients can poison the training dataset or training results by tampering with them. The poisoning attack algorithms used include distributed backdoor attacks, inner product manipulation attacks, and label inversion attacks. The poisoning rounds begin from round 35 on the FMNIST and EMNIST datasets and from round 200 on the CIFAR-10 dataset. After training, all clients send their local models to the verification team for validation and send their local gradients to the central server for gradient confidence calculation. In this embodiment, the verifier is initially randomly selected from the clients. In subsequent training rounds, the verifier will be selected based on credibility. Clients with high credibility in the current round will become the verifiers in the next round.

[0049] Step 2 specifically involves the validator validating the local model based on the difference in class loss values. In each training round, a selection will be made... Samples of different categories, The value of is determined by the number of categories in the classification task; in this embodiment, it is selected as . =10. The validator selects a portion of the training dataset as the validation set, and divides the validation set according to categories. Within the same category, it calculates the loss value of the previous global model and the loss value of the current local model, and calculates the difference between the two. This operation is repeated for all classes to complete the validation of the current local model. The validation result is obtained by summing the differences of the loss values ​​of all local models in different classes. and the verification results Send to the central server.

[0050] Among them, the verification results It is a two-dimensional array, expressed as: ,in The number of clients participating in the training. Let be the number of categories, where Represents the local model In category The difference between the loss values ​​of the global model and the local model is given by the loss value expression as follows: ,in Indicates category labels, Indicates local model Next category The predicted probability.

[0051] Step 3 specifically involves the central server processing the verification results of this round. The process involves two steps. The first step calculates the average difference between the global model's loss and the local model's loss across all categories, which is then used as the average excellence score of the current local model. The first step, using the average excellence score across all classes, better reflects the current model's average performance across all classes and provides a better measure of the local model's stability. The second step calculates the minimum difference between the previous round's global model loss value and the local model loss value across all classes, which is taken as the minimum excellence score of the current local model. By observing the extreme performance of local models on individual classes, this approach better aligns with the attacker's strategy of poisoning specific classes in actual attacks. The gradient obtained by the client with the highest average score of excellence among all local models is selected as the optimal gradient for this round, and the gradient obtained by the client with the lowest score of excellence is selected as the worst gradient for this round. The selected optimal and worst gradients are stored in a central server and used to calculate the credibility of local gradients in the next round of training.

[0052] in, Represents the local model The mean of excellence is expressed as: , Represents the local model The minimum value of excellence is expressed as: .

[0053] Step 4 specifically involves the central server calculating the credibility of all local gradients in this round. The server, having received validation results from the previous training round, will also select the optimal and worst gradients using the method in Step 3 and store them for credibility calculation in this round. Based on the credibility results, malicious clients are identified and filtered out. The credibility... , This represents the optimal gradient in the previous training round. This represents the worst gradient in the previous training round. Then it means the first Local gradients for each client, This represents the cosine similarity between these gradients. If the local gradient is benign, it will be similar to... Greater similarity, and with A smaller similarity results in a larger difference, leading to a higher confidence score for the current gradient, and vice versa. If all clients exhibit malicious client confidence in the current training round, it indicates that the optimal and worst gradients selected in the previous training round may not be good enough. In this case, a new gradient will be selected from the server's historical optimal and worst gradients. and Then, the confidence level of the gradient in this round is recalculated. It should be noted that the server does not store the best and worst gradients from the previous round during the first round of training. During aggregation, the top 50% of clients with the highest confidence levels are selected by default to participate in model aggregation.

[0054] Step 5 specifically involves the central server filtering out malicious clients using gradient confidence. It checks if the current confidence level is greater than zero. If the confidence level is greater than zero, it indicates a benign gradient, which will participate in model aggregation, with confidence level used as the aggregation weight. If the current confidence level is negative, it indicates a malicious gradient, and the client will be filtered out during this round of model aggregation, with the aggregation weight set to zero. Then, the local model is pruned. In this embodiment, the pruning method in FLAME is used to prune the local model. This method is a commonly used adaptive pruning method, and its expression is: ,in This indicates the model update after this adaptive pruning. This represents the global model from the previous round. This indicates that the local model on the current client has been updated. It is the threshold for adaptive cropping. This is the L2 norm of the current model update. Finally, the global model for this round is obtained by weighted averaging of the confidence level and the pruned local model. The aggregation expression is: ,in Indicates aggregate weight, This indicates the number of clients participating in the aggregation.

[0055] Step 6 specifically involves the central server distributing the aggregated global model to all clients. Each client then begins the next round of training on the global model. Malicious clients are filtered out during the training process. Filtered clients will have their aggregation weights set to 0 when aggregating the global model in this round, thus preventing them from affecting the global model. Steps 1 to 5 are repeated to train the model until it converges.

[0056] In this invention, malicious clients that are filtered out do not participate in the aggregation of the global model in this round, but will participate in federated learning training normally in the next round. The purpose of this invention is to filter out bad results in this round of training and aggregate to obtain the best possible global model.

[0057] Table 1 compares the proposed method with two extreme methods: a poisoned model trained using a method without detection of poisoning attacks, and a perfect model trained using a method assuming perfect filtering of malicious clients in each round of detection. The final model's accuracy, recall, and backdoor accuracy are also presented under label inversion attacks and distributed backdoor attacks.

[0058] Table 1:

[0059]

[0060] Table 1 presents the performance of the proposed method on different datasets and under different attacks. As can be seen from the table, this method maintains a certain level of robustness under various attacks on different datasets, successfully resisting label inversion attacks and distributed backdoor attacks. Furthermore, its accuracy (MA) and recall (RR) are close to the final model obtained by perfectly resisting poisoning attacks. In some cases, the model obtained by this method even outperforms the perfect model obtained by completely filtering malicious clients. This is because the aggregation algorithm in the proposed method further influences the global model obtained by using credibility as a weight. For example, on the Cifar-10 dataset, the final model's recall (RR) is higher than that of the perfect detection scheme, reaching 84.200%; its backdoor accuracy (BA) is also lower, at only 1.567%.

[0061] Figures 3 and 4 show the accuracy and recall of the proposed method when dealing with label inversion attacks on the FMNIST dataset, compared to the model obtained under ideal perfect detection and the model obtained under complete poisoning without any defenses. Figures 5 and 6 show the accuracy and backdoor accuracy of the proposed method when dealing with distributed backdoor attacks on the FMNIST dataset. The comparisons demonstrate that the performance of the model obtained by the proposed method under different attacks is close to that of the model obtained under perfect detection of malicious clients.

[0062] In summary, this invention presents a detection method for malicious client poisoning attacks in image classification scenarios within a federated learning neighborhood. It selects the optimal and worst gradients by validating the user's local model across different classes. The gradient credibility is calculated by the similarity difference between the local gradient and the optimal and worst gradients. Based on this credibility, malicious poisoning clients are identified. The credibility scores are then processed and aggregated to obtain a global model. This method achieves good defense against various poisoning attacks across different datasets. Furthermore, it demonstrates robustness even when the number of malicious clients exceeds 50%, effectively identifying the poisoner.

[0063] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any simple modifications, equivalent substitutions and improvements made by any person skilled in the art without departing from the technical scope disclosed in the present invention shall still fall within the scope of the technical solution of the present invention.

Claims

1. A method for detecting a poisoning attack of federated learning based on gradient credibility, characterized in that, Includes the following steps: Step 1: The client trains a model on a dataset of images that have already been classified, with the number of classes set to There are a certain number of malicious clients in the client. These malicious clients will poison the training process. After all clients have completed training, they will send the trained local model and local gradient to the verification party and the central server. Step 2: The validator validates the local models of all clients on different categories. The validation set is selected from a portion of the image dataset. The difference between the loss value of the current local model and the previous round of global model is calculated. After the validator has validated all local models, the validation results of this round are summarized and uploaded to the central server. Step 3: The central server preprocesses the verification results sent by the verification party in this round. By analyzing the preprocessing results, the optimal gradient and the worst gradient in this round of training are selected and stored in the server for the credibility calculation in the next round. Step 4: Perform credibility calculation. The central server obtains the credibility by calculating the difference between the cosine similarity between the local gradient in this round of training and the best and worst gradient in the previous round of training. Step 5: The central server identifies malicious clients in this round of training based on credibility and filters them. The aggregate weight of the filtered clients is set to 0 when aggregating the global model in this round. The aggregate weight of the remaining clients is calculated based on credibility and participates in the aggregation of the global model in this round. Step 6: The central server distributes the global model for this round to all clients. Each client starts the next round of training on the global model and repeats steps 1 to 5 to filter malicious clients during the model training process until the model converges.

2. The gradient confidence-based federated learning poisoning attack detection method according to claim 1, characterized in that, In step 1, all clients send the training results to the verifier for verification and to the central server for gradient confidence calculation.

3. The gradient confidence-based federated learning poisoning attack detection method of claim 1, wherein, In step 2, the validator selects a portion of the image dataset to serve as the validation set. Samples of different categories, Given the number of classes in the current classification task, the validation set is divided by class. The local model is validated based on the difference in class loss values. Within the same class, the loss value of the previous global model and the loss value of the current local model are calculated, and their difference is calculated. This process is repeated for all classes to complete the validation of the current local model. The validation results of all local models are then summarized into an array. In, and verify the results Send to the central server.

4. The gradient confidence-based federated learning poisoning attack detection method of claim 3, wherein, The verification results It is a two-dimensional array, expressed as: ,in The number of clients participating in the training. For the number of categories, Represents the local model In category The difference between the loss value of the lower and global models is expressed as follows: ,in Indicates category labels, Indicates local model Next category The predicted probability.

5. The gradient confidence-based federated learning poisoning attack detection method according to claim 1, characterized in that, In step 3, the central server will verify the results. The process involves two steps. The first step calculates the average difference between the global model's loss and the local model's loss across all categories, which is then used as the average excellence score of the current local model. The expression is: The second processing step calculates the minimum difference between the global model's loss and the local model's loss across all classes from the previous round, which is then used as the minimum excellence score of the current local model. The expression is: The gradient with the highest average score among all local models is taken as the best gradient in this round, and the gradient with the lowest score among the minimum values ​​of excellence is taken as the worst gradient in this round.

6. The gradient confidence-based federated learning poisoning attack detection method according to claim 1, characterized in that, In step 4, the credibility calculation formula is: ； wherein: denotes the optimal gradient in the previous round of training, denotes the worst gradient in the previous training round, represents the 1 local gradients of the respective clients, This represents the cosine similarity between two gradients.

7. The gradient confidence-based federated learning poisoning attack detection method according to claim 1, characterized in that, In step 5, it is determined whether the credibility of the current client is greater than zero. If the credibility is positive, it indicates that the current gradient is a benign gradient and will participate in model aggregation, with credibility as the aggregation weight. If the credibility is negative, it indicates that the current gradient is a malicious gradient, and the aggregation weight is set to 0 in this round of global model aggregation to filter the current client. Finally, the central server aggregates the weight values ​​and the pruned local model to obtain the global model for this round.

Images