Dual defense method and system for federal learning poisoning attack

Abstract

The invention discloses a federated learning poisoning attack dual defense method and system, and the method is based on a federated learning framework, and comprises the steps: S100, a full knowledge distillation defense stage: transferring the knowledge of a local model of a terminal node to a student model on an edge server through employing a full knowledge distillation method, testing the learning rate of the student model, and determining the learning rate of the student model; judging whether the terminal node is a malicious node or not according to a set threshold value, and if yes, preventing the terminal node from participating in the federated learning framework; and S200, a prototype discrimination defense stage: maintaining a prototype discrimination model on the edge server for discriminating whether the prototype uploaded by the terminal node is correct or not, if not, refusing the prototype to participate in aggregation, and if yes, allowing the prototype to participate in aggregation and utilizing the prototype to continuously train the prototype discrimination model. According to the method, wrong prototypes are eliminated through total knowledge distillation and prototype discrimination, and prototype poisoning attacks of federal prototype learning can be effectively defended.

Description

Technical Field

[0001] The present invention relates to the field of federated learning technology, and in particular to a dual defense method and system for federated learning poisoning attacks. Background Art

[0002] Federated Learning has been widely welcomed and applied due to its advantages in data privacy protection. Unlike traditional centralized machine learning, Federated Learning does not need to transfer data from local terminal nodes to other untrusted third-party processing centers. Instead, it transfers the machine learning model to a third-party processing center for analysis and processing, and the processing center cannot obtain the original data through the model.

[0003] Although federated learning has natural advantages in learning efficiency and privacy protection, it still faces two major challenges in the model optimization process: data heterogeneity and model heterogeneity. Many studies have tried to mitigate the threats brought by these two challenges. In response to the challenge of data heterogeneity, adding additional regularization terms to the loss function in the local model optimization process is a common solution. It effectively reduces parameter "drift" by constraining the parameter update of the local model, so that the local model and the global model are close to the optimal point, and finally achieves the effect of improving the model accuracy. Another strategy is to generate a personalized model for each terminal through the global model and local personalized data. For model heterogeneity, the usual solution is to use knowledge distillation (KD). Federated learning based on knowledge distillation uses public data sets and local private data sets of terminal nodes to transfer the knowledge of teacher models with different model structures (pre-trained models that generate soft labels for training data for student models to learn) to a student model (the model is trained using training data and soft labels provided by the teacher model and real labels), so that the student model has different teacher knowledge, and the requirement of knowledge aggregation in federated learning is realized. It is an effective method commonly used to solve model heterogeneity.

[0004] The above studies have improved the data heterogeneity and model heterogeneity problems in federated learning to varying degrees, but these improvements usually only target one of them. In response to this, Tan et al. proposed the Federated Prototype Learning Method (FedProto) in 2022, which transforms the aggregation of gradients in traditional federated learning into the aggregation of prototypes, eliminates the gradient misalignment caused by differences in data distribution and model structure between terminal nodes, makes model aggregation independent of the local model structure and data distribution of the terminal nodes, optimizes the convergence speed and generalization ability of the model, and improves the tolerance of federated learning to data heterogeneity and model heterogeneity. In essence, FedProto transforms model gradient aggregation into prototype aggregation, supports local model optimization of terminal nodes, and is affected by both local data classification tasks and prototype fitting. Since the local data classification task of the terminal node occupies the main part of the optimization target, the terminal node can learn and generate a more efficient model through prototype fitting while ensuring the normal convergence of the local model. Due to FedProto's tolerance for data and model heterogeneity and its high learning efficiency, it has been applied to clinical medicine, metaverse, autonomous driving and other fields.

[0005] The inventors found that FedProto did not consider the correctness of the prototype, which plays a key role. That is, the terminal node may be controlled by the attacker and transmit the wrong prototype, which faces the threat of data poisoning attack. Data poisoning attack refers to the attacker consciously inputting incorrect or biased data in the data preparation stage to reduce the availability of the trained model. Assuming that some terminal nodes participating in federated learning are malicious, these terminal nodes first construct poisoned data locally, and then train local models based on the poisoned data. The edge server aggregates the local model parameters of all terminal nodes to obtain the global model. Since the local models and parameters of the malicious terminal nodes are defective, the global model is also defective. After several rounds of iterations, a defective reasoning model will eventually be trained. It can be seen that when the existing federated learning model is attacked by prototype poisoning, the wrong prototype uploaded to the edge server will destroy the correctness of the global prototype, causing the local model to be updated in the wrong direction, resulting in a decrease in the prediction accuracy of all local models on the entire data set; and experiments have also proved that the effect of poisoning attacks using wrong prototypes will become significant as the number of poisoned prototypes increases. How to effectively defend against this prototype poisoning attack is an important issue facing federated learning methods at this stage. Summary of the invention

[0006] In view of the problem in the prior art that prototype poisoning attacks reduce the accuracy of federated learning methods, the present invention provides a dual defense method for federated learning poisoning attacks.

[0007] In order to achieve the above object, the technical solution adopted by the present invention is as follows:

[0008] A dual defense method for federated learning poisoning attacks, based on the federated learning framework, includes the following steps:

[0009] S100, full knowledge distillation defense stage: Use the full knowledge distillation method to transfer the knowledge of the local model of the terminal node to the student model on the edge server, then test the learning rate of the student model, and determine whether the terminal node is a malicious node based on the set threshold. If so, prevent the terminal node from participating in the federated learning framework;

[0010] S200, prototype identification defense stage: A prototype identification model is maintained on the edge server to identify whether the prototype uploaded by the terminal node is correct. If it is incorrect, the prototype is rejected from participating in the aggregation. If it is correct, the prototype is allowed to participate in the aggregation and the prototype identification model is continuously trained using the prototype.

[0011] Specifically, the method for full knowledge distillation in step S100 includes:

[0012] The local model of the terminal node is used as a soft label of the probability distribution of the teacher model output to help the student model learn;

[0013] Introducing a temperature parameter in the generation of soft labels to improve knowledge extraction from the teacher model;

[0014] The soft labels output by the teacher model are used to supervise the training of the student model so that the student model can learn the knowledge of the teacher model.

[0015] Specifically, the soft label in step S100 is generated by a softmax function, which is expressed as:

[0016]

[0017] In the formula, q i represents the soft label generated by terminal node i, z i Represents the original output value of the teacher model, j is the label category;

[0018] The softmax function after introducing the temperature parameter T is expressed as:

[0019]

[0020] The loss function F of the student model full_kd It is expressed as:

[0021]

[0022] Where KL represents the KL divergence function, soft_label represents the soft label generated by the teacher model, and softmax(logits / T) represents the predicted label of the student model.

[0023] Specifically, the edge server in step S100 verifies the accuracy of the student model using the test data set, and calculates the learning rate of the student model = student model accuracy / teacher model accuracy;

[0024] Determine whether the learning rate of the student model is lower than the pre-configured learning rate threshold. If so, the terminal node is considered to be a malicious node. Otherwise, the terminal node is considered to be an honest terminal node, and the honest terminal node is allowed to upload the trained prototype to the edge server.

[0025] Specifically, the prototype identification model in step S200 includes three convolutional layers and two fully connected layers arranged in sequence, the input is the prototype data, and the output is the label corresponding to the prototype;

[0026] First, the prototype identification model is pre-trained using the public data set on the edge server. During the federated prototype learning process, the prototype identification model is continuously trained using the prototypes uploaded by the correct terminal nodes.

[0027] Specifically, the process of the prototype identification model in step S200 identifying whether the prototype uploaded by the terminal node is correct is as follows:

[0028] Input the prototype uploaded by the terminal node into the trained prototype identification model to obtain the predicted label of the prototype of the terminal node;

[0029] Compare whether the predicted label of the prototype of the terminal node is consistent with the true label. If not, reject the prototype from participating in the subsequent global prototype aggregation. If consistent, accept the prototype from participating in the subsequent global prototype aggregation.

[0030] Based on the above dual defense method for federated learning poisoning attacks, the present invention also provides a dual defense system for federated learning prototype poisoning attacks, including:

[0031] Full knowledge distillation defense module: Use the full knowledge distillation method to transfer the knowledge of the local model of the terminal node to the student model on the edge server, then test the learning rate of the student model, determine whether the terminal node is a malicious node based on the set threshold, and prevent malicious terminal nodes from participating in the federated learning framework;

[0032] Prototype identification defense module: A prototype identification model is maintained on the edge server to identify whether the prototype uploaded by the terminal node is correct, and to reject incorrect prototypes from participating in aggregation, while allowing correct prototypes to participate in aggregation and using the correct prototypes to continuously train the prototype identification model.

[0033] Compared with the prior art, the present invention has the following beneficial effects:

[0034] (1) The present invention designs a dual defense mechanism against prototype poisoning attacks through full knowledge distillation technology and prototype identification technology. First, the full knowledge distillation technology is used to quickly screen out malicious nodes and prevent them from uploading prototypes. Then, the prototype identification technology is used to determine whether the prototype uploaded by the terminal node to the edge server is correct, and the incorrect prototype is discarded, thereby effectively avoiding the influence of the poisoned prototype on the global prototype calculation, thereby affecting the accuracy of the local model of the correct terminal node. Experiments on real data sets have shown that the present invention can increase the inference accuracy of the attacked model by 1-5 times, and only increase the system running time by 2%.

[0035] (2) The present invention first uses the full knowledge distillation defense stage to perform a round of pre-screening before the terminal node joins the federated learning to prevent malicious terminal nodes that use poisoned data to train the model from joining, thereby reducing the additional running time brought by the prototype identification model when inferring the prototype. At the same time, it can also directly prevent malicious terminal nodes from uploading poisoned prototypes to improve the model accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] Figure 1 It is a schematic diagram of the overall process framework of an embodiment of the present invention.

[0037] Figure 2 A schematic diagram of the process framework of the prototype identification defense of the embodiment of the present invention.

[0038] Figure 3 This is a comparison chart of the model accuracy when the MNIST poisoning frequency is 1 / 5 and the prototype identification defense is not enabled in the embodiment.

[0039] Figure 4 This is a comparison chart of the model accuracy when the MNIST poisoning frequency is 1 / 3 and the prototype identification defense is not enabled in the embodiment.

[0040] Figure 5 This is a comparison chart of the model accuracy when the MNIST poisoning frequency is 1 and the prototype identification defense is not enabled in the embodiment.

[0041] Figure 6 This is a comparison chart of the model accuracy when the MNIST poisoning frequency is 1 / 5 and the prototype identification defense is turned on in the embodiment.

[0042] Figure 7 This is a comparison chart of the model accuracy when the MNIST poisoning frequency is 1 / 3 and the prototype identification defense is turned on in the embodiment.

[0043] Figure 8 This is a comparison chart of the model accuracy when the MNIST poisoning frequency is 1 and the prototype identification defense is turned on in the embodiment.

[0044] Fig. 9This is a comparison chart of the model accuracy when the CIFAR10 poisoning frequency is 1 / 5 and the prototype identification defense is not enabled in the embodiment.

[0045] Fig.10 This is a comparison chart of the model accuracy when the CIFAR10 poisoning frequency is 1 / 3 and the prototype identification defense is not enabled in the embodiment.

[0046] Fig.11 This is a comparison chart of the model accuracy when the CIFAR10 poisoning frequency is 1 and the prototype identification defense is not enabled in the embodiment.

[0047] Fig.12 This is a comparison chart of the model accuracy when the CIFAR10 poisoning frequency is 1 / 5 and the prototype identification defense is turned on in the embodiment.

[0048] Fig.13 This is a comparison chart of the model accuracy when the CIFAR10 poisoning frequency is 1 / 3 and the prototype identification defense is enabled in the embodiment.

[0049] Fig.14 This is a comparison chart of the model accuracy when the CIFAR10 poisoning frequency is 1 and the prototype identification defense is enabled in the embodiment. DETAILED DESCRIPTION

[0050] The present invention is further described below in conjunction with the accompanying drawings and examples. The embodiments of the present invention include but are not limited to the following examples.

[0051] Federated learning is a distributed machine learning method that allows users in different geographical locations (called federated learning terminal nodes) to jointly train neural network models based on their own private data, while ensuring that each terminal node does not need to share its private data with other nodes (and edge servers). Specifically, FL is jointly participated by an edge server and several terminal nodes, and includes multiple rounds of iterative training. The detailed calculation process of any round of iteration is as follows:

[0052] 1. Participating node selection: The edge server selects some terminal nodes from all terminal nodes to participate in the iterative calculation of the current round;

[0053] 2. Node local training: Each selected terminal node trains the local model based on local private data (based on the global model parameters of the previous round) and uploads the local model parameters to the edge server;

[0054] 3. Model parameter aggregation: The edge server weightedly aggregates the model parameters from all terminal nodes participating in the current round of calculation to form the global model of the current round;

[0055] 4. Local model update: The edge server sends the global model parameters aggregated in the current round to each terminal node.

[0056] Federated learning solves the problems of high communication overhead and poor privacy data protection in traditional centralized machine learning, but it faces two major shortcomings: model heterogeneity and data heterogeneity across terminal nodes. Model heterogeneity means that due to the different computing and storage resources between different terminal nodes participating in federated learning, the depth of the neural network model they maintain and the dimension of each layer of the network are different. The edge server cannot aggregate the local models of terminal nodes with different structures, and it is often difficult to obtain an accurate global model. Data heterogeneity means that the local data of different terminal nodes are non-independent and identically distributed (non-IID), which means that these terminal nodes hold different data labels, resulting in different scopes of application of the trained local models. If the edge server directly aggregates these local models, the resulting global model will have extremely low prediction accuracy for data on different terminal nodes.

[0057] In order to solve the above shortcomings, Tan proposed a new feature-based heterogeneous federated learning framework FedProto, which uses prototypes to replace the model gradient-based aggregation method of traditional federated learning. Prototype refers to the intermediate result generated by the original data passing through the middle layer of a multi-layer neural network (for example, the neural network layer between the second layer and the penultimate layer). Therefore, even if the number of model layers of each terminal node and the dimension of each layer of the neural network are different, as long as the dimensions of the prototypes generated based on the middle layer are the same, the edge server can still successfully aggregate the prototypes, thereby solving the model heterogeneity problem. At the same time, FedProto only uses the global prototypes that are consistent with the labels of the local prototypes of the terminal nodes to calculate the regularization term of the loss function (objective function) of the model optimization, and discards the global prototypes that are inconsistent with the local prototype labels of the terminal nodes (these global prototypes with inconsistent labels are heterogeneous data) to avoid the convergence of heterogeneous data for the local model of the terminal node, so the data heterogeneity in FedProto does not affect the normal convergence of the local model. Due to the advantage of FedProto's tolerance of data and model heterogeneity, it has received widespread attention and welcome from scholars and industry. The above-mentioned loss function for model optimization refers to the use of the model to calculate the distance between the predicted label of the feature data and its true label (common loss functions include cross entropy function, KL divergence, and mean square error). The larger the distance, the less similar the predicted label and the true label. We need to reduce the value of the loss function in each round of training to optimize the model parameters.

[0058] Specifically, assume that the complete neural network model is F(θ), the middle layer is f(θ), where θ is the model parameter. At the same time, the training data is recorded as (x, y), where x is the original data and y is the corresponding label. The final training output of the model is F(θ, x), and the prototype of the original data x is f(θ, x). The calculation steps of the rth round of FedProto iteration are as follows:

[0059] 1. The terminal node receives the global prototype sent by the edge server in the r-1th round.

[0060] 2. The terminal node generates a prototype for each label based on the local training data. Specifically, the local training data of the terminal node includes feature data x and label y. Each label contains a large amount of different feature data. The feature data will generate a corresponding prototype f(θ, x) after passing through the intermediate layer f(θ) of the local model F(θ). The terminal node calculates the average value of the prototypes corresponding to all feature data of each label to obtain the local prototype of the terminal node in round r. For example, the local prototype of terminal node i with label j in round r is

[0061] 3. The terminal node calculates the mean square error between the global prototype and the local prototype in the r-1th round, and uses this as a regular term σ in the loss function of the model optimization i For example, the loss function of the model optimization of terminal node i is CE(F(θ,x),y)+λσ i , CE is the cross entropy function, λ is the weight (the larger the λ value, the larger the regular term σ i The greater the impact on the loss function, it is set to 1 in the experiment).

[0062] 4. The edge server receives the local prototype uploaded by the terminal node, and performs weighted aggregation for each label to generate the global prototype of round r. For example, the edge server calculates the global prototype of label j The weight is calculated as Where |D i,j | is the amount of data with label j at terminal node i, N j is the sum of the data volume of all terminal nodes with label j.

[0063] FedProto repeatedly iterates the above steps until the local model of the terminal node reaches a convergence state, at which point the global model prediction accuracy reaches the optimal level. In addition to solving the problems of model heterogeneity and data heterogeneity, FedProto also has the following advantages: 1) It is difficult for edge servers to restore the original training data through prototypes, thereby protecting user data privacy; 2) Compared with local model parameters, the data scale of prototypes is often smaller, which helps to reduce the communication overhead of federated learning.

[0064] However, FedProto does not consider the correctness of the prototype, which plays a key role. That is, the terminal node may be controlled by the attacker and transmit the wrong prototype; the effect of poisoning attacks using wrong prototypes will become more significant as the number of poisoned prototypes increases. Specifically, the malicious terminal node uploads the wrong prototype in each round of communication. When the cumulative number of poisoned prototypes gradually increases, the model accuracy of other terminal nodes will gradually decrease. In addition, when the number of malicious terminal nodes increases, the accuracy of the local model will also decrease faster.

[0065] In order to deal with the harm caused by prototype poisoning attacks to FedProto, the present invention proposes a dual defense method against prototype poisoning attacks (Defend Feature Map Poisoning Attack, De-FMPA).

[0066] For ease of understanding, Table 1 shows the relevant symbol marks of the present invention, and the characteristic diagram is the prototype.

[0067]

[0068] Table 1 Symbol table

[0069] like Figures 1 to 5 As shown in FIG. 1 , the dual defense method for the federated learning poisoning attack is based on the federated learning framework and includes the following steps:

[0070] S100, full knowledge distillation defense stage: Use the full knowledge distillation method to transfer the knowledge of the local model of the terminal node to the student model on the edge server, then test the learning rate of the student model, and determine whether the terminal node is a malicious node based on the set threshold. If so, prevent the terminal node from participating in the federated learning framework;

[0071] S200, prototype identification defense stage: A prototype identification model is maintained on the edge server to identify whether the prototype uploaded by the terminal node is correct. If it is incorrect, the prototype is rejected from participating in the aggregation. If it is correct, the prototype is allowed to participate in the aggregation and the prototype identification model is continuously trained using the prototype.

[0072] Most of the data poisoning attacks in federated learning use chaotic feature data and labels to train local models, and then use incorrect local model parameters to affect the model accuracy of other terminal nodes. In order to prevent these malicious terminal nodes from joining the federated prototype learning FedProto, while protecting user privacy, we consider using knowledge distillation to determine whether the local training data of the terminal node is scrambled. The full knowledge distillation method used in the present invention aims to transfer the knowledge of the local model to the student model to the maximum extent while avoiding the knowledge of the public data set from being learned by the student model. The accuracy of the local training model of the terminal node is indirectly detected by using full knowledge distillation technology, rather than uploading the model of the terminal node directly to the edge server. This method well ensures the confidentiality of the local model of the terminal node and the privacy of user data.

[0073] The following uses a formula to show the calculation of full knowledge distillation. Using a public dataset, each feature data will be inferred by the teacher model (the local model of the terminal node) and the student model (maintained by the edge server) and output logits. Full knowledge distillation uses the teacher model to output soft labels of probability distribution to help the student model learn. The soft labels are generated by the softmax function, which is

[0074]

[0075] In the formula, q i represents the soft label generated by terminal node i, z i represents the original output value of the teacher model, and j is the label category. In order to prevent the soft labels generated by softmax from contributing too little to the loss function of training the student model, the concept of temperature T is introduced here to improve the knowledge transfer effect. The softmax function after introducing the temperature parameter T is expressed as:

[0076]

[0077] The above formula is used to generate the soft label soft_label output by the teacher model, which is used to supervise the training of the student model. The loss function F of the student model is full_kd It is expressed as:

[0078]

[0079] Where KL represents the KL divergence function, soft_label represents the soft label generated by the teacher model, and softmax(logits / T) represents the predicted label of the student model.

[0080] In order to fully learn the knowledge of the local model of the terminal node, full knowledge distillation is performed by setting different temperatures T. After comparison, it is found that when T is 0.1, the learning rate of the student model for the local model of the terminal node reaches a maximum of 83.50%.

[0081] Specifically, the edge server in step S100 verifies the accuracy of the student model using the test data set, and calculates the learning rate of the student model = student model accuracy / teacher model accuracy;

[0082] Determine whether the learning rate of the student model is lower than the pre-configured learning rate threshold. If so, the terminal node is considered to be a malicious node. Otherwise, the terminal node is considered to be an honest terminal node, and the honest terminal node is allowed to upload the trained prototype to the edge server.

[0083] The full knowledge distillation defense algorithm includes two functions: Local_Pre-train() and Full_KD_Defense(). Before the terminal node joins FedProto, it is necessary to execute the Local_Pre-train() function to pre-train the local model using local data. After pre-training, the local model of the terminal node is used as a teacher model to guide the student model of the edge server to learn. The edge server executes Full_KD_Defense() to transfer the knowledge of the terminal node model to the student model, and then test the accuracy of the student model to determine whether the training data of the terminal node is available based on its accuracy.

[0084] Defense Program 1 Pseudocode of full knowledge distillation defense:

[0085] Input: Dataset D of malicious terminal node k k , a public training dataset D used by edge servers for knowledge extraction train , the edge server is used to test the public test dataset D of the student model test , the temperature T in KD, the accuracy standard target_acc of the student model taught by the honest terminal node.

[0086] Output: Whether the local model training process of the terminal node has been maliciously poisoned.

[0087]

[0088]

[0089] Specifically, the prototype identification model in step S200 includes three convolutional layers and two fully connected layers arranged in sequence, the input is the prototype data, and the output is the label corresponding to the prototype, which is used to identify whether the prototype uploaded by the terminal node is correct;

[0090] First, the prototype identification model is pre-trained using the public data set on the edge server. During the federated learning process, the prototype identification model is continuously trained using the prototypes uploaded by the correct terminal nodes to improve the accuracy of the prototype identification model.

[0091] Specifically, the process of the prototype identification model in step S200 identifying whether the prototype uploaded by the terminal node is correct is as follows:

[0092] Input the prototype uploaded by the terminal node into the trained prototype identification model to obtain the predicted label of the prototype of the terminal node;

[0093] Compare whether the predicted label of the prototype of the terminal node is consistent with the true label. If not, reject the prototype from participating in the subsequent global prototype aggregation. If consistent, accept the prototype from participating in the subsequent global prototype aggregation. At the same time, use the prototype to continuously train the prototype identification model to improve the prediction accuracy.

[0094] Pseudo code of defense program 2 prototype identification defense:

[0095] Input: the prototype identification model feature_map_model maintained locally by the edge server, the prototype feature_map and label target_label delivered by the terminal node.

[0096] Output: Whether the local prototype uploaded by the terminal node is correct.

[0097]

[0098] Based on the above dual defense method for federated learning poisoning attacks, the present invention also provides a dual defense system for federated learning prototype poisoning attacks, including:

[0099] Full knowledge distillation defense module: Use the full knowledge distillation method to transfer the knowledge of the local model of the terminal node to the student model on the edge server, then test the learning rate of the student model, determine whether the terminal node is a malicious node based on the set threshold, and prevent malicious terminal nodes from participating in the federated learning framework;

[0100] Prototype identification defense module: A prototype identification model is maintained on the edge server to identify whether the prototype uploaded by the terminal node is correct, and to reject incorrect prototypes from participating in aggregation, while allowing correct prototypes to participate in aggregation and using the correct prototypes to continuously train the prototype identification model.

[0101] In order to better reflect the effect of the present invention, the federated learning poisoning attack is first explained and tested below.

[0102] Firstly, the threat model of the prototype poisoning attack is determined based on the behaviors of edge servers and terminal nodes in FedProto. Secondly, the attack process is introduced and the effectiveness of the attack is tested.

[0103] This example mainly focuses on the problem that the security vulnerabilities of the prototype may be exploited by malicious attackers and cause the accuracy of the model to decrease. The following two assumptions are made here:

[0104] Assumption 1: All terminal nodes participating in federated learning agree that the edge server will allocate scheduling tasks, and the terminal nodes agree that the edge server will aggregate the global prototypes based on the local prototypes and publish them to each terminal node;

[0105] Assumption 2: The edge server is honest, that is, it will not maliciously generate wrong global prototypes, and its behavior is only to aggregate local prototypes from terminal nodes and send them down.

[0106] Like other studies on poisoning attacks, the goal of the attacker in this embodiment is to maliciously manipulate the local models of other terminal nodes to make them have a high error rate on test data. This attack is also called an untargeted attack, which makes the learned local model unavailable and ultimately prevents users from enjoying the service. The attacker knows the local training data set, local model, and training code of the malicious terminal node, but it cannot know the local training data and models of other terminal nodes. At the same time, the attacker knows the edge server's aggregation rules for the prototype. For example, the edge server will disclose the aggregation rules to improve the transparency and trust of the FedProto system. The attacker can use the known aggregation rules and the local prototypes that can be arbitrarily manipulated to attack, breaking the correctness of the global prototype. Assume that the attacker controls k terminal nodes, that is, the attacker can re-inject k malicious terminal nodes into the FedProto system, or invade and control k honest terminal nodes. Although the malicious terminal node will perform specific steps according to the protocol, it cannot guarantee the correctness of the local training data and the correctness of the local prototype it uploads. For example, the malicious terminal node may randomly scramble the labels of the training data before training the local model, causing errors in the correspondence between feature data and labels. Moreover, malicious terminal nodes may also generate poisoned prototypes locally and upload them to the edge servers, and these erroneous local poisoned prototypes will seriously affect the correctness of the global prototypes aggregated by the edge servers.

[0107] In FedProto, the model optimization loss function of terminal node i is L i =CE(F(θ,x),y)+λσ i, it can be observed that the loss function is affected by two parts: one part is the cross entropy loss CE(F(θ,x),y) generated by the local training data, and the other part is the mean square error regularization term σ calculated by the global prototype and the local prototype i ,λ is used to adjust the regularization term for L i to expand or reduce its effect. If the regularization term σ i If L is larger than CE(F(θ,x),y), then i The optimization of the model will be mainly affected by the regularization term. On the contrary, if the regularization term σ i Compared with CE(F(θ,x),y), which is smaller, the optimization of the model will also be mainly affected by the cross entropy loss.

[0108] To this end, this embodiment proposes a poisoning attack FMPA (Feature Map Poisoning Attack, FMPA) using wrong prototypes. The malicious terminal node arbitrarily manipulates the local prototype to make it unreliable, and uploads these tampered prototypes to the edge server. If there are enough poisoned prototypes, errors will occur when the edge server aggregates these prototypes, which will cause the honest terminal node to calculate the wrong regularization term σ locally, thereby interfering with the optimization convergence process of the model. In order to more clearly demonstrate the FMPA attack, here is a description of the calculation method of local poisoned prototypes, global prototypes, regularization terms, and loss functions. Assume that there is a set of malicious terminal nodes k∈U among the terminal nodes participating in FedProto, and take the feature data labeled j as an example. The local poisoned prototype with label j calculated by the malicious terminal node k in the rth round is:

[0109]

[0110] Where l is any label except label j. Since the labels are scrambled, the label calculated at this time is the prototype of j is the poisoned prototype, where j' represents the local poisoned prototype with label j. In order to enhance the effect of FMPA attack, the malicious terminal node multiplies the value of the poisoned prototype before uploading it to the edge server. So that it can more effectively affect the honest terminal nodes. By numerically amplifying the poisoned prototypes, malicious terminal nodes can change the distribution of global prototypes, thereby affecting the results of honest terminal nodes when calculating regularization terms locally. This enhanced FMPA attack can cause honest terminal nodes to be more disturbed during model optimization, thereby affecting the convergence and performance of the model. Where η is a real number greater than 1. In the test experiment, the corresponding η values ​​for different data sets may also be different, for example, η = 3 in MNIST and η = 5 in CIFAR10.

[0111] The participation of the local poisoned prototype in the aggregation of the global prototype will cause the calculation deviation of the global prototype. The following takes the global prototype with label j aggregated by the edge server in round r as an example:

[0112]

[0113] Finally, for the honest terminal node i, the regularization term is calculated using the global poisoned prototype sent by the edge server in the rth round and the local prototype in the r+1th round This regularization term will be an important item in the model optimization loss function. Due to the manipulation of poisoned prototype amplification by malicious terminal nodes, the calculation result of the regularization term will be enlarged and occupy the main body of the local model optimization target of the honest terminal node, which will lead to deviation in the model optimization direction and eventually cause the model to fail to converge or even become unusable. In the r+1th round, the local objective function of the honest terminal node i is:

[0114]

[0115] The following attack program shows the basic process of FMPA attack. The program includes two functions. The Local_Update() function is executed at the terminal node, and its input is the terminal node number k and the data set D. k , which is used by the terminal node to update the model and calculate the local prototype at the same time; the Global_Proto_Agg() function is executed on the edge server, and its input is the local prototype calculated by all terminal nodes This function is used to aggregate all local prototypes by weighted average to calculate the global prototype. The execution order should be that the terminal node first executes Local_Update() (the global prototype is initialized to 0 in the initial stage), and then the edge server executes Global_Proto_Agg(); then the above execution order is alternated in each round of training.

[0116] Attack program pseudo code:

[0117] Input: Local dataset D of malicious terminal node k k , the set of all terminal nodes V, the local prototype of each terminal node with label j

[0118] Output: Malicious terminal node k calculates the local poisoning prototype The edge server aggregates local prototypes to form a global prototype.

[0119]

[0120]

[0121] Through a large number of experiments on the MNIST dataset, we can draw the following conclusions: when there are no malicious terminal nodes, the average accuracy of 20 terminal nodes after 100 rounds of training is 98.36%; when there are 2 malicious terminal nodes and poison is administered once every 5 rounds of training, the average accuracy drops to 98.06%; when there are 4 malicious terminal nodes and poison is administered once every 3 rounds, the accuracy drops to 88.63%; when there are 10 malicious terminal nodes and poison is administered once every 1 round, the accuracy drops to 16.64%, which is 81.72% lower than the accuracy when not attacked.

[0122] This proves that FMPA attacks can effectively affect the normal operation of FedProto, resulting in low model accuracy of the terminal nodes involved, and the greater the number of poisoned prototypes, the lower the model accuracy.

[0123] Based on the above prototype poisoning attack experiment, the effect of the dual defense method De-FMPA of the present invention is verified by experiments as follows.

[0124] The De-FMPA defense experiment uses the MNIST dataset as the main dataset and the CIFAR10 dataset as the auxiliary dataset. Other related parameters are consistent with the configuration of the attack experiment. The MNIST dataset contains 10 label categories, with a total of 60,000 training samples and 10,000 test samples. Each category has 6,000 training samples and 1,000 test samples. Each sample is a 28*28 handwritten digital grayscale image. The CIFAR10 dataset contains 10 label categories, with a total of 50,000 training samples and 10,000 test samples. Each category has 5,000 training samples and 1,000 test samples. Each sample is a 32*32 three-channel color image.

[0125] The basic configuration of FedProto uses a virtual machine to simulate 30 terminal nodes as participants in federated learning, among which the terminal nodes numbered 21-30 are malicious poisoning terminal nodes. Non-independent and identically distributed data are assigned to each terminal node, and different label data in MNIST and CIFAR10 are deployed on different terminal nodes. CNNMNIST and ResNet18 models are used to learn the above two datasets, and the learning rate is set to an empirical value of 0.001. In order to realize the heterogeneous scenario of the neural network model, some hyperparameters of the neural network model deployed on the terminal node are modified as in previous work. In MNIST, the number of output channels of the convolutional layer is set to 18, 20, or 22. In CIFAR10, different convolutional layer strides are set for different terminal nodes. A high-performance server with a 3090Ti graphics card and 24GB of memory is selected for the experiment.

[0126] 1. Results of the full knowledge distillation defense experiment

[0127] Optimal distillation temperature: In the pre-screening stage, the full knowledge distillation method proposed in this invention is used to quickly screen out poisoned terminal nodes that use toxic data to train local models, reducing the computational overhead of the subsequent prototype identification defense stage. In order to explore the optimal temperature T for full knowledge distillation, we set T to different values ​​and found that when T = 0.1, the knowledge of the teacher model can be extracted to the maximum extent; at different temperatures, we randomly select 2 terminal nodes as teacher models to help train the student model of the server. Table 2 shows the learning effect of the student model at different temperatures, where the teacher model is the local model of the terminal node and the student model is deployed on the edge server.

[0128]

[0129] Table 2 Learning effect of student model at different temperatures

[0130] It can be seen from Table 2 that when T is set to 0.1, the knowledge extraction effect of full knowledge distillation is the best, and the learning rate can reach up to 83.88% (student model accuracy divided by teacher model accuracy) without poisoning; the accuracy of the student model is used to determine whether the teacher model is poisoned. When the accuracy of the student model is lower than a certain threshold, the terminal node is considered to be a malicious terminal node. In this experiment, the learning rate threshold for the MNIST data set is set to 60%, and the thresholds for different data sets may also be different.

[0131] Full knowledge distillation at the optimal temperature: 100 virtual terminal nodes are simulated, of which 60 are malicious terminal nodes, which will scramble the training data locally, and the remaining 40 are honest terminal nodes. Table 3 records the identification accuracy of the full knowledge distillation defense phase when malicious or honest terminal nodes are added.

[0132] Poisoning Number of terminal nodes Identify the correct amount Identification success rate yes 60 60 100％ no 40 37 92.5％

[0133] Table 3. Full knowledge distillation identification success rate

[0134] The identification of malicious terminal nodes is the main task of the pre-screening stage. As can be seen from Table 3, when 60 malicious terminal nodes are simulated, the pre-screening stage identifies all of them with an accuracy of 100%; since the local model accuracy of malicious terminal nodes is mostly between 10% and 50%, they contain less knowledge, so the learning rate of the student model is low (when T = 0.1, the learning rate is only about 54%), and the final accuracy of the student model will be lower (usually lower than 60%), so the full knowledge distillation has a high accuracy rate for identifying malicious terminal nodes. When 40 honest terminal nodes are simulated, the pre-screening stage can identify 37 of them with an accuracy rate of 92.5%; for a few terminal nodes that fail to be identified, the reason may be that the amount of data allocated to the terminal node is small, and the data distribution is quite different from the public data set used for full knowledge distillation, resulting in a low learning rate of the student model and a low accuracy rate of the student model.

[0135] Knowledge distillation and full knowledge distillation: As mentioned above, the loss function of the student model in knowledge distillation contains two items, namely soft target and knowledge distillation loss. The specific loss function is as follows:

[0136]

[0137] In the formula, y represents the true label of the data. The difference between the above formula and the loss function of full knowledge distillation is that the student model not only learns the knowledge of the teacher model (soft target) but also learns the knowledge of the data set (knowledge distillation loss) through the above formula, while full knowledge distillation only learns the knowledge of the teacher model. When the weight α of the knowledge distillation loss term is 0, knowledge distillation is transformed into full knowledge distillation. In order to enable the student model to better reflect the teacher model, that is, to judge whether the teacher model is poisoned by the accuracy of the student model, the following experiment is conducted: the temperature is set to 0.1, 60 malicious terminal nodes are successively invested in FedProto, and the knowledge of the local model of the terminal node is transferred to the student model of the edge server by distributing the knowledge distillation and full knowledge distillation. Table 4 shows the comparison of the identification success rate of knowledge distillation and full knowledge distillation in the above experiment.

[0138]

[0139] Table 4 Knowledge distillation and full knowledge distillation

[0140] It can be seen from Table 4 that the lower the value of the hyperparameter α in the student model loss function of knowledge distillation, the higher the identification success rate of malicious terminal nodes. When α is 0, the loss function becomes full knowledge distillation, and the identification success rate rises to 100%. It can be seen that full knowledge distillation has a huge advantage in identifying malicious terminal nodes in the pre-screening stage.

[0141] 2. Results of prototype identification and defense experiments

[0142] In order to demonstrate the effectiveness of prototype identification defense, we conducted an ablation experiment. In the experiment, while enabling full knowledge distillation defense, the prototype identification defense was distributed and enabled or not, and a malicious terminal node was put into communication every 10 rounds, with poisoning frequencies of 1 / 5, 1 / 3, and 1, that is, poisoning once every 5 rounds, poisoning once every 3 rounds, and poisoning every round. In order to fully reflect the necessity and effectiveness of prototype identification defense, we need to set up some malicious terminal nodes that can partially participate in FedProto. The malicious terminal nodes put in here use clean data sets for local training, but upload poisoned prototype data. Because it uses clean data sets to train the model, the local model of the malicious terminal node can achieve a higher accuracy, and can bypass the full knowledge distillation defense with a greater probability and participate in the subsequent FedProto.

[0143]

[0144] Table 5 Average accuracy when prototype identification defense is enabled or not

[0145] Table 5 shows the average accuracy of all terminal nodes in the final communication round after 100 global training rounds and a malicious terminal node is added every 10 rounds of communication. It can be seen that under different poisoning frequencies, due to the high local model accuracy of some malicious terminal nodes, the full knowledge distillation defense is bypassed and added to FedProto for poisoning. The average accuracy of the model after enabling prototype identification defense is always higher than that without enabling it; especially when the poisoning frequency is 1, the prototype identification defense mechanism can improve the average accuracy of the model by 39.84% and 80.84% ​​on the CIFAR10 and MNIST datasets, respectively, which fully demonstrates the effectiveness and necessity of prototype identification defense.

[0146] In order to clearly show the impact of prototype identification defense on the model accuracy of terminal nodes, the following comparison charts of accuracy changes with and without defense are drawn for different data sets with poisoning frequencies of 1 / 5, 1 / 3, and 1. In the figure with the defense mechanism turned on, only the model accuracy change chart of the honest terminal node is drawn.

[0147] MNIST Dataset: Figures 3 to 5 The following table shows the accuracy changes of the local models of all terminal nodes participating in FedProto when the poisoning frequency is 1 / 5, 1 / 3, and 1, respectively, without enabling prototype identification defense. When the poisoning frequency is 1 / 5, the vertical line represents the injection of a malicious terminal node in the current round. Since the poisoning frequency is low once every 5 rounds, the poisoned prototype does not affect the accuracy of the honest terminal node model. Figure 6When the poisoning frequency is 1 / 3, not turning on the prototype identification defense mechanism will significantly reduce the model accuracy. When the seventh malicious terminal node is added in 70 rounds of communication, the poisoning frequency rises to once every 3 rounds. At this time, the large number of poisoned prototypes contained in FedProto seriously affects the calculation of the global prototype. The average model accuracy of the honest terminal nodes also begins to decline and eventually drops to 54.93%. Figure 7 When the prototype identification defense mechanism is enabled, the average model accuracy of the terminal nodes is reduced by 42.71%. When the poisoning frequency is 1, FedProto contains a large number of poisoned prototypes in just a few rounds. Figure 5 It shows that after the third malicious terminal node was added in the 30th round of communication, the model experienced gradient explosion due to the influence of a large number of poisoned prototypes, and the accuracy began to drop sharply and finally remained at 15%, which is far lower than Figure 8 The average model accuracy of terminal nodes after the prototype identification defense mechanism is enabled.

[0148] CIFAR10 dataset: We also conducted related experiments on the CIFAR10 dataset. Figures 9 to 11 The graph shows the change in the accuracy of the local model of the terminal node when the poisoning frequencies are 1 / 5, 1 / 3, and 1 respectively and the prototype identification defense is not enabled. Figure 12 to Figure 14 The figure shows the change in the accuracy of the local model of the terminal node when the prototype identification defense is turned on with the poisoning frequencies of 1 / 5, 1 / 3, and 1 respectively. It can be seen that although the poisoned prototypes generated on the malicious terminal node did not cause the model gradient explosion on the CIFAR10 dataset, it also greatly affected the convergence of the local models of other honest terminal nodes. Since the CIFAR10 dataset is more complex and the loss of local classification task calculation is large, in order to make the prototype poisoning attack effective, the malicious terminal node is enlarged 10 times before uploading the poisoned prototype, which makes the poisoning efficiency higher and the experimental results more significant. Figures 9 to 14 From the comparison, it can be seen that when prototype identification defense is applied in FedProto, the local model accuracy of honest terminal nodes can be significantly improved.

[0149] 3. The necessity of a dual defense mechanism

[0150] Necessity of full knowledge distillation defense: In order to reduce the computing resource consumption in the prototype identification defense stage, we choose to use full knowledge distillation defense to perform a round of pre-screening before the terminal node joins FedProto to prevent malicious terminal nodes that use poisoned data to train the model from joining, reducing the additional running time of the prototype identification model when reasoning about the prototype data. At the same time, it can also directly prevent the above-mentioned malicious terminal nodes from uploading poisoned prototypes to improve the model accuracy.

[0151] Assuming that a total of 30 terminal nodes participate in FedProto, 20 of which are honest terminal nodes and 10 are malicious terminal nodes, Table 6 shows the average model accuracy of the terminal nodes when using full knowledge distillation defense or not.

[0152]

[0153] Table 6 Average accuracy of the model with or without full knowledge distillation defense enabled

[0154] It can be seen that after turning on the full knowledge distillation defense, the average model accuracy of the terminal node is higher than that without turning it on, especially when the poisoning frequency is 1, turning on the full knowledge distillation defense increases the model accuracy by 5.19%; the reason why the model accuracy is low without turning on the full knowledge distillation defense is that all malicious terminal nodes will enter FedProto and continue to upload poisoned prototypes to the edge server. Although the edge server maintains a prototype identification model, the accuracy of the model cannot reach 100%, and a few poisoned prototypes will be misjudged and allowed to join the calculation of the global prototype. Without the pre-screening of full knowledge distillation, all malicious terminal nodes will join FedProto, and the number of poisoned prototypes will increase. Due to the misjudgment of the prototype identification model, the number of poisoned prototypes participating in the global prototype aggregation will also increase, which will have a greater impact on the model accuracy.

[0155] On the other hand, if all malicious terminal nodes are added to FedProto without screening, when they upload a large number of poisoned prototypes to the edge server at a high poisoning frequency, they need to go through the reasoning of the prototype identification model to identify their correctness. However, the reasoning of a large number of prototypes will seriously slow down the running time of the system and slow down the convergence speed of the model. From the average running time of 100 rounds of communication recorded in Table 7, the average communication time of the system is reduced by 4.07s when the full knowledge distillation defense is turned on than when it is not turned on.

[0156] Whether to enable full knowledge distillation defense Average round of communication time yes 38.49s no 42.56s

[0157] Table 7 Average system running time whether full knowledge distillation defense is enabled

[0158] Necessity of prototype identification defense: The full knowledge distillation defense stage is to quickly screen out malicious terminal nodes that use poisoned data to train local models to prevent these malicious terminal nodes from joining FedProto. However, this stage does not guarantee the correctness of the prototypes uploaded to the edge server by the terminal nodes participating in FedProto. Therefore, it is necessary to use the prototype identification defense mechanism to continuously identify local prototypes to prevent the poisoned prototype from affecting the calculation of the global prototype, thereby affecting the accuracy of the local model of the honest terminal node.

[0159] The above embodiments are only preferred embodiments of the present invention and are not intended to limit the protection scope of the present invention. Any changes made by adopting the design principles of the present invention and performing non-creative work on this basis should fall within the protection scope of the present invention.

Claims

1. A dual defense method for federated learning poisoning attacks, characterized in that: Based on the federated learning framework, the following steps are included: S100, full knowledge distillation defense stage: Use the full knowledge distillation method to transfer the knowledge of the local model of the terminal node to the student model on the edge server, then test the learning rate of the student model, and determine whether the terminal node is a malicious node based on the set threshold. If so, prevent the terminal node from participating in the federated learning framework; S200, prototype identification defense stage: A prototype identification model is maintained on the edge server to identify whether the prototype uploaded by the terminal node is correct. If it is incorrect, the prototype is rejected from participating in the aggregation. If it is correct, the prototype is allowed to participate in the aggregation and the prototype identification model is continuously trained using the prototype.

2. The dual defense method for federated learning poisoning attacks according to claim 1 is characterized in that: The method for full knowledge distillation in step S100 includes: The local model of the terminal node is used as a soft label of the probability distribution of the teacher model output to help the student model learn; Introducing a temperature parameter in the generation of soft labels to improve knowledge extraction from the teacher model; The soft labels output by the teacher model are used to supervise the training of the student model so that the student model can learn the knowledge of the teacher model.

3. The dual defense method for federated learning poisoning attacks according to claim 2 is characterized in that: In step S100, the soft label is generated by a softmax function, which is expressed as: In the formula, q i represents the soft label generated by terminal node i, z i Represents the original output value of the teacher model, j is the label category; The softmax function after introducing the temperature parameter T is expressed as: The loss function F of the student model fullkd It is expressed as: Where KL represents the KL divergence function, soft_label represents the soft label generated by the teacher model, and softmax(logits / T) represents the predicted label of the student model.

4. The dual defense method for federated learning poisoning attacks according to claim 3 is characterized in that: The edge server in step S100 verifies the accuracy of the student model using the test data set, and calculates the learning rate of the student model = student model accuracy / teacher model accuracy; Determine whether the learning rate of the student model is lower than the pre-configured learning rate threshold. If so, the terminal node is considered to be a malicious node. Otherwise, the terminal node is considered to be an honest terminal node, and the honest terminal node is allowed to upload the trained prototype to the edge server.

5. The dual defense method for federated learning poisoning attack according to any one of claims 1 to 4, characterized in that: The prototype identification model in step S200 includes three convolutional layers and two fully connected layers arranged in sequence, the input is the prototype data, and the output is the label corresponding to the prototype; First, the prototype identification model is pre-trained using the public data set on the edge server. During the federated learning process, the prototype identification model is continuously trained using the prototypes uploaded by the correct terminal nodes.

6. The dual defense method for federated learning poisoning attacks according to claim 5 is characterized in that: The process of the prototype identification model in step S200 identifying whether the prototype uploaded by the terminal node is correct is as follows: Input the prototype uploaded by the terminal node into the trained prototype identification model to obtain the predicted label of the prototype of the terminal node; Compare whether the predicted label of the prototype of the terminal node is consistent with the true label. If not, reject the prototype from participating in the subsequent global prototype aggregation. If consistent, accept the prototype from participating in the subsequent global prototype aggregation.

7. A dual defense system for federated learning poisoning attacks, characterized in that: include Full knowledge distillation defense module: Use the full knowledge distillation method to transfer the knowledge of the local model of the terminal node to the student model on the edge server, then test the learning rate of the student model, determine whether the terminal node is a malicious node based on the set threshold, and prevent malicious terminal nodes from participating in the federated learning framework; Prototype identification defense module: A prototype identification model is maintained on the edge server to identify whether the prototype uploaded by the terminal node is correct, and to reject incorrect prototypes from participating in aggregation, while allowing correct prototypes to participate in aggregation and using the correct prototypes to continuously train the prototype identification model.

Images