Dual defense method and system for federal learning poisoning attacks

Abstract

The application discloses a double defense method and system for a federal learning poisoning attack, based on a federal learning framework, comprising S100, a full knowledge distillation defense stage: using a full knowledge distillation method to transfer the knowledge of a terminal node local model to a student model on an edge server, then testing the learning rate of the student model, judging whether the terminal node is a malicious node according to a set threshold, if yes, preventing the terminal node from participating in the federal learning framework; S200, a prototype discrimination defense stage: maintaining a prototype discrimination model on the edge server, used to identify whether the prototype uploaded by the terminal node is correct, if not, refusing the prototype to participate in aggregation, if correct, allowing the prototype to participate in aggregation and continuously training the prototype discrimination model using the prototype. The application can effectively prevent the prototype poisoning attack of federal prototype learning by excluding the wrong prototype through full knowledge distillation and prototype discrimination.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of federated learning, in particular to a double defense method and system for poisoning attacks in federated learning. BACKGROUND

[0002] Federated learning is widely welcomed and applied due to its advantages in data privacy protection. Unlike traditional centralized machine learning, federated learning does not need to transfer data from terminal nodes to other untrusted third-party processing centers, but transfers machine learning models to third-party processing centers for analysis and processing, and the processing center cannot obtain the original data through the model.

[0003] Although federated learning has natural advantages in learning efficiency and privacy protection, it still faces two major challenges of data heterogeneity and model heterogeneity in the model optimization process. There are many studies trying to alleviate the threat brought by these two challenges. For the data heterogeneity challenge, adding an additional regularization term to the loss function during local model optimization is a common solution, which effectively reduces the parameter "drift" by constraining the parameter update of the local model, so that the optimal points of the local model and the global model are approximated, and finally the model accuracy is improved. Another strategy is to generate a personalized model for each terminal by using the global model and the local personalized data. For model heterogeneity, the usual solution is to use knowledge distillation (KD). Federated learning based on knowledge distillation uses public data sets and local private data sets of terminal nodes to transfer the knowledge of teacher models (pre-trained models that produce soft labels for training data for student models to learn) with different model structures to a student model (which uses training data and soft labels and real labels provided by the teacher model for training), so that the student model has different teacher knowledge, achieving the purpose of knowledge aggregation in federated learning, which is an effective method commonly used to solve model heterogeneity.

[0004] The above studies improve data heterogeneity and model heterogeneity in federated learning to varying degrees, but these improvements are usually only for one of them. In response, Tan et al. proposed the federated prototype learning method (FedProto) in 2022, which converts the aggregation of gradients in traditional federated learning to the aggregation of prototypes, eliminating the gradient misalignment caused by differences in data distribution and model structure between end nodes, making model aggregation independent of end node local model structure and data distribution, optimizing model convergence speed and generalization ability, and improving the tolerance of federated learning to data heterogeneity and model heterogeneity. Essentially, FedProto converts model gradient aggregation to prototype aggregation, supporting local model optimization of end nodes while being influenced by both local data classification tasks and prototype fitting. Since the local data classification task of the end node occupies the main part of the optimization target, the end node can learn and generate a more efficient model through prototype fitting while ensuring the normal convergence of the local model. Due to the tolerance of FedProto to data and model heterogeneity and the efficiency of learning, it has been applied to fields such as clinical medicine, metaverse, and autonomous driving.

[0005] The inventors found that FedProto does not consider the correctness of the prototype, which plays a key role, i.e., end nodes may be controlled by attackers to deliver incorrect prototypes, and it faces the threat of data poisoning attacks. A data poisoning attack refers to an attacker intentionally injecting incorrect or biased data during the data preparation phase to reduce the usability of the trained model. Assuming that some end nodes participating in federated learning are malicious, these end nodes first construct poisoned data locally, and then train local models based on the poisoned data. The edge server aggregates the local model parameters of all end nodes to obtain a global model. Since the local model and parameters of the malicious end nodes are flawed, the global model is also flawed. After several rounds of iteration, a flawed inference model is finally trained. As can be seen, when the existing federated learning model is subjected to a prototype poisoning attack, the incorrect prototype uploaded to the edge server will destroy the correctness of the global prototype, causing the local model to update in the wrong direction, thereby causing the prediction accuracy of all local models on the entire data set to decrease; and experiments have also shown that the effect of a poisoning attack using an incorrect prototype will become significant as the number of poisoned prototypes increases. How to effectively defend against such prototype poisoning attacks is an important problem faced by current federated learning methods. SUMMARY

[0006] In view of the problem of prototype poisoning attacks reducing the accuracy of federated learning methods in the prior art, the present application provides a dual defense method for federated learning poisoning attacks.

[0007] To achieve the above purpose, the technical scheme adopted by the present application is as follows:

[0008] A double defense method of a federal learning poisoning attack, based on a federal learning framework, comprising the following steps:

[0009] S100, full knowledge distillation defense stage: using the method of full knowledge distillation to transfer the knowledge of the local model of the terminal node to the student model on the edge server, and then testing the learning rate of the student model, judging whether the terminal node is a malicious node according to the set threshold, if so, preventing the terminal node from participating in the federal learning framework;

[0010] S200, prototype discrimination defense stage: maintaining a prototype discrimination model on the edge server to identify whether the prototype uploaded by the terminal node is correct, if not, rejecting the prototype from participating in aggregation, if correct, allowing the prototype to participate in aggregation and continuously training the prototype discrimination model using the prototype.

[0011] Specifically, the method of full knowledge distillation in step S100 comprises:

[0012] The local model of the terminal node is output as a soft label of the probability distribution of the teacher model to help the student model learn;

[0013] A temperature parameter is introduced in the generation of the soft label to improve the extraction of the knowledge of the teacher model;

[0014] The soft label output by the teacher model is used to supervise the training of the student model, so that the student model learns the knowledge of the teacher model.

[0015] Specifically, the soft label in step S100 is generated by a softmax function, which is expressed as:

[0016]

[0017] In the formula, q i represents the soft label generated by the terminal node i, z i represents the original output value of the teacher model, and j is the label category;

[0018] The softmax function after introducing the temperature parameter T is expressed as:

[0019]

[0020] The loss function F full_kd of the student model is expressed as:

[0021]

[0022] In the formula, KL represents the KL divergence function, soft_label represents the soft label generated by the teacher model, and softmax(logits / T) represents the predicted label of the student model.

[0023] Specifically, the edge server in the step S100 verifies the accuracy of the student model by using the test data set, and calculates the learning rate of the student model = student model accuracy / teacher model accuracy.

[0024] It is judged whether the learning rate of the student model is lower than the pre-configured learning rate threshold, if yes, the terminal node is considered as a malicious node, otherwise, the terminal node is considered as an honest terminal node, and the honest terminal node is allowed to upload the trained prototype to the edge server.

[0025] Specifically, the prototype discrimination model in the step S200 includes three convolution layers and two fully connected layers arranged in sequence, and the input is the prototype data, and the output is the label corresponding to the prototype.

[0026] Firstly, the prototype discrimination model is pre-trained by using the public data set on the edge server, and in the federated prototype learning process, the prototype discrimination model is continuously trained by using the prototype uploaded by the correct terminal node.

[0027] Specifically, the process that the prototype discrimination model in the step S200 discriminates whether the prototype uploaded by the terminal node is correct is as follows:

[0028] The prototype uploaded by the terminal node is input into the trained prototype discrimination model to obtain the predicted label of the prototype of the terminal node.

[0029] It is compared whether the predicted label of the prototype of the terminal node is consistent with the real label, if not, the prototype is rejected to participate in the subsequent aggregation of the global prototype, if yes, the prototype is accepted to participate in the subsequent aggregation of the global prototype.

[0030] Based on the double defense method against the poisoning attack in the federated learning, the application further provides a double defense system against the prototype poisoning attack in the federated learning, which comprises

[0031] The full knowledge distillation defense module uses the full knowledge distillation method to transfer the knowledge of the local model of the terminal node to the student model on the edge server, then tests the learning rate of the student model, judges whether the terminal node is a malicious node according to the set threshold, and prevents the malicious terminal node from participating in the federated learning framework.

[0032] The prototype discrimination defense module maintains a prototype discrimination model on the edge server, which is used to discriminate whether the prototype uploaded by the terminal node is correct, rejects the incorrect prototype to participate in the aggregation, and allows the correct prototype to participate in the aggregation and continuously trains the prototype discrimination model by using the correct prototype.

[0033] Compared with the prior art, the application has the following beneficial effects:

[0034] (1) The application designs a double defense mechanism against prototype poisoning attacks through full knowledge distillation technology and prototype discrimination technology. First, the full knowledge distillation technology is used to quickly screen out malicious nodes to prevent them from uploading prototypes. Then, the prototype discrimination technology is used to judge whether the prototype uploaded by the terminal node to the edge server is correct. The incorrect prototype is discarded, thereby effectively avoiding the influence of the poisoned prototype on the global prototype calculation, and further affecting the local model accuracy of the correct terminal node. Experiments on real data sets show that the application can improve the prediction accuracy of the attacked model by 1-5 times, and only increases the system running time by 2%.

[0035] (2) The application first uses the full knowledge distillation defense stage to pre-screen before the terminal node joins the federated learning, to prevent malicious terminal nodes that use poisoned data to train models from joining, reduce the additional running time of the prototype discrimination model when reasoning prototypes, and also directly avoid uploading poisoned prototypes by malicious terminal nodes to improve model accuracy. BRIEF DESCRIPTION OF DRAWINGS

[0036] Figure 1 It is the overall flow framework schematic diagram of the embodiment of the application.

[0037] Figure 2 It is the flow framework schematic diagram of the prototype discrimination defense of the embodiment of the application.

[0038] Figure 3 It is the model accuracy comparison chart of the embodiment in which the MNIST poisoning frequency is 1 / 5 and the prototype discrimination defense is not started.

[0039] Figure 4 It is the model accuracy comparison chart of the embodiment in which the MNIST poisoning frequency is 1 / 3 and the prototype discrimination defense is not started.

[0040] Figure 5 It is the model accuracy comparison chart of the embodiment in which the MNIST poisoning frequency is 1 and the prototype discrimination defense is not started.

[0041] Figure 6 It is the model accuracy comparison chart of the embodiment in which the MNIST poisoning frequency is 1 / 5 and the prototype discrimination defense is started.

[0042] Figure 7 It is the model accuracy comparison chart of the embodiment in which the MNIST poisoning frequency is 1 / 3 and the prototype discrimination defense is started.

[0043] Figure 8 It is the model accuracy comparison chart of the embodiment in which the MNIST poisoning frequency is 1 and the prototype discrimination defense is started.

[0044] Figure 9Figure 1 shows the accuracy comparison of the model in the embodiment with CIFAR10 poisoning frequency of 1 / 5 and without turning on the prototype discrimination defense.

[0045] Figure 10 Figure 2 shows the accuracy comparison of the model in the embodiment with CIFAR10 poisoning frequency of 1 / 3 and without turning on the prototype discrimination defense.

[0046] Figure 11 Figure 3 shows the accuracy comparison of the model in the embodiment with CIFAR10 poisoning frequency of 1 and without turning on the prototype discrimination defense.

[0047] Figure 12 Figure 4 shows the accuracy comparison of the model in the embodiment with CIFAR10 poisoning frequency of 1 / 5 and turning on the prototype discrimination defense.

[0048] Figure 13 Figure 5 shows the accuracy comparison of the model in the embodiment with CIFAR10 poisoning frequency of 1 / 3 and turning on the prototype discrimination defense.

[0049] Figure 14 Figure 6 shows the accuracy comparison of the model in the embodiment with CIFAR10 poisoning frequency of 1 and turning on the prototype discrimination defense. DETAILED DESCRIPTION

[0050] The present application will be further described below in conjunction with the accompanying drawings and embodiments, which include but are not limited to the following embodiments.

[0051] Federated learning is a distributed machine learning method that allows users distributed in different geographical locations (referred to as terminal nodes of federated learning) to jointly train a neural network model based on their respective private data, while ensuring that each terminal node does not need to share its private data with other nodes (and edge servers). Specifically, FL is jointly participated by an edge server and a plurality of terminal nodes, and includes a plurality of rounds of iterative training processes. The calculation process of any round of iteration is described in detail as follows:

[0052] 1. Selection of participating nodes: the edge server selects part of the terminal nodes from all the terminal nodes to participate in the iterative calculation of the current round;

[0053] 2. Local training of nodes: each selected terminal node trains a local model based on the local private data (based on the global model parameters of the last round) and uploads the local model parameters to the edge server;

[0054] 3. Aggregation of model parameters: the edge server aggregates the model parameters from all the terminal nodes participating in the calculation of the current round to form a global model of the current round;

[0055] 4. Update of local model: the edge server issues the global model parameters aggregated in the current round to each terminal node.

[0056] Federated learning solves the problem of large communication overhead and poor privacy data protection of traditional centralized machine learning, but faces two major shortcomings of model heterogeneity and data heterogeneity across terminal nodes. Model heterogeneity refers to the fact that different terminal nodes participating in federated learning have different computing and storage resources, resulting in different depths and dimensions of the neural network models they maintain. Edge servers cannot aggregate different local models of terminal nodes, often making it difficult to obtain an accurate global model. Data heterogeneity refers to the fact that local data of different terminal nodes is non-IID, meaning that these terminal nodes hold different data labels, resulting in different application ranges of the local models trained. If the edge server directly aggregates these local models, the prediction accuracy of the global model for data on different terminal nodes will be very low.

[0057] To solve the above shortcomings, Tan proposed a new heterogeneous federated learning framework FedProto based on features, which uses prototypes to replace the traditional federated learning model gradient-based aggregation method. A prototype is an intermediate result produced by the original data after passing through multiple layers of a neural network (e.g., the neural network layer between the second layer and the second-to-last layer). Therefore, even if the number of layers and the dimension of each layer of the neural network of each terminal node differ, as long as the dimensions of the prototypes produced by the intermediate layers are the same, the edge server can still successfully aggregate the prototypes, thereby solving the model heterogeneity problem. At the same time, FedProto only uses global prototypes with the same label as the local prototypes of the terminal nodes to calculate the regularization term of the loss function (objective function) of model optimization, and discards global prototypes with inconsistent labels (these inconsistent global prototypes are heterogeneous data), avoiding the convergence of heterogeneous data on the local model of the terminal node. Therefore, data heterogeneity in FedProto does not affect the normal convergence of the local model. Due to the advantages of FedProto in tolerating data and model heterogeneity, it has received widespread attention and welcome from academics and industry. The above-mentioned model optimization loss function refers to calculating the distance (common loss functions include cross-entropy function, KL divergence, mean square error) between the predicted label and the true label of the feature data using the model, and the greater the distance, the less similar the predicted label and the true label. We need to reduce the value of the loss function in each training round to optimize the model parameters.

[0058] Specifically, assume a complete neural network model F(θ), with an intermediate layer f(θ). Here, θ is the model parameter. At the same time, the training data is denoted as (x, y), where x is the original data and y is the corresponding label. The final training output of the model is F(θ, x), and the prototype of the original data x is f(θ, x). The detailed FedProto iteration calculation steps in the rth round are as follows:

[0059] 1. The terminal node receives the global prototype of the r-1th round issued by the edge server.

[0060] 2. The terminal node generates a prototype for each label based on the local training data. Specifically, the local training data of the terminal node includes feature data x and labels y, each label contains a large number of different feature data, and the feature data will generate a corresponding prototype f(θ, x) after passing through the intermediate layer f(θ) of the local model F(θ). The terminal node calculates the average value of the prototype corresponding to all feature data of each label to obtain the local prototype of the terminal node in the rth round. For example, the local prototype of the terminal node i in the rth round for the label j is

[0061] 3. The terminal node calculates the mean square error of the global prototype of the r-1th round and the local prototype as a regularization term σ of the loss function of model optimization. i For example, the loss function of model optimization of the terminal node i is CE(F(θ, x), y) + λσ i , CE is the cross-entropy function, λ is the weight (the greater the value of λ, the greater the influence of the regularization term σ i on the loss function, which is set to 1 in the experiment.

[0062] 4. The edge server receives the local prototype uploaded from the terminal node, and generates the global prototype of the rth round by weighted aggregation for each label. For example, the edge server calculates the global prototype of the label j The calculation method of the weight is where |D i,j | is the data amount of the label j of the terminal node i, N j is the sum of the data amount of the label j of all terminal nodes.

[0063] FedProto repeatedly iterates the above steps until the local model of the terminal node reaches a convergent state, at which time the prediction accuracy of the global model reaches the optimum. In addition to solving the problems of model heterogeneity and data heterogeneity, FedProto also has the following advantages: 1) it is difficult for the edge server to restore the original training data through the prototype, thereby protecting the privacy of user data; 2) compared with the local model parameters, the data size of the prototype is usually smaller, which helps to reduce the communication overhead of federated learning.

[0064] However, FedProto does not consider the correctness of the key role prototype, that is, the terminal node may be controlled by the attacker to transfer the wrong prototype; the effect of the poisoning attack using the wrong prototype will become significant as the number of poisoned prototypes increases. Specifically, malicious terminal nodes upload wrong prototypes in each round of communication, and when the number of cumulative invested poisoned prototypes gradually increases, the model accuracy of other terminal nodes will gradually decrease. In addition, when the number of malicious terminal nodes increases, the local model accuracy will also decrease faster.

[0065] To deal with the harm of the prototype poisoning attack to FedProto, the present application proposes a double defense method (Defend Feature Map Poisoning Attack, De-FMPA) against the prototype poisoning attack.

[0066] For the convenience of understanding, Table 1 shows the relevant symbol marks of the present application, and the feature map is the prototype.

[0067]

[0068] Table 1 Symbol Table

[0069] As shown in Figures 1 to 5 The double defense method of the federated learning poisoning attack based on the federated learning framework includes the following steps:

[0070] S100, full knowledge distillation defense stage: using the method of full knowledge distillation to transfer the knowledge of the local model of the terminal node to the student model on the edge server, and then testing the learning rate of the student model, according to the set threshold value to determine whether the terminal node is a malicious node, if so, prevent the terminal node from participating in the federated learning framework;

[0071] S200, prototype discrimination defense stage: maintaining a prototype discrimination model on the edge server, which is used to identify whether the prototype uploaded by the terminal node is correct, if not, the prototype is refused to participate in aggregation, if correct, the prototype is allowed to participate in aggregation and the prototype discrimination model is continuously trained using the prototype.

[0072] Data poisoning attacks in federated learning mostly use chaotic feature data and labels to train local models, and then use incorrect local model parameters to affect the model accuracy of other terminal nodes. In order to avoid these malicious terminal nodes from joining the federated prototype learning FedProto, while protecting user privacy, we consider using knowledge distillation to determine whether the local training data of the terminal node is disordered. The full knowledge distillation method used by the invention aims to maximize the transfer of knowledge from the local model to the student model while avoiding the knowledge of the public dataset being learned by the student model. Using full knowledge distillation technology indirectly detects the accuracy of the local training model of the terminal node, rather than directly uploading the model of the terminal node to the edge server, which effectively guarantees the confidentiality of the local model of the terminal node and the privacy of the user data.

[0073] The following shows the calculation of full knowledge distillation using a formalized way, using a public dataset, each feature data in which will be inferred by the teacher model (local model of the terminal node) and the student model (maintained by the edge server) and output logits. Full knowledge distillation uses the soft label of the teacher model output probability distribution to help the student model learn, where the soft label is generated by the softmax function, and the softmax function is

[0074]

[0075] In the formula, q i represents the soft label generated by the terminal node i, z i represents the original output value of the teacher model, and j is the label category. In order to prevent the contribution of the soft label generated by softmax to the loss function of the training student model from being too small, the concept of temperature T is introduced here to improve the knowledge transfer effect. The softmax function after introducing the temperature parameter T is represented as:

[0076]

[0077] The above formula is used to generate the soft label soft_label output by the teacher model, which is used to supervise the training of the student model, and the loss function F full_kd of the student model is represented as:

[0078]

[0079] In the formula, KL represents the KL divergence function, soft_label represents the soft label generated by the teacher model, and softmax(logits / T) represents the predicted label of the student model.

[0080] In order to fully learn the knowledge of the local model of the terminal node, full knowledge distillation is performed by setting different temperatures T. It is found through comparison that when T is 0.1, the learning rate of the student model to the local model of the terminal node reaches the highest 83.50%.

[0081] Specifically, the edge server in the step S100 verifies the accuracy of the student model by using the test data set, and calculates the learning rate of the student model = student model accuracy / teacher model accuracy.

[0082] It is judged whether the learning rate of the student model is lower than the pre-configured learning rate threshold. If yes, the terminal node is considered as a malicious node, otherwise, the terminal node is considered as an honest terminal node, and the honest terminal node is allowed to upload the trained prototype to the edge server.

[0083] The algorithm of full knowledge distillation defense includes two functions of Local_Pre-train() and Full_KD_Defense(). Before the terminal node joins FedProto, the Local_Pre-train() function needs to be executed first to pre-train the local model by using the local data. After pre-training, the local model of the terminal node is used as a teacher model to guide the learning of the student model of the edge server. The edge server executes Full_KD_Defense(), which aims to transfer the knowledge of the model of the terminal node to the student model, and then tests the accuracy of the student model to determine whether the training data of the terminal node is available.

[0084] Pseudocode of defense program 1 full knowledge distillation defense:

[0085] Input: data set D of malicious terminal node k k , public training data set D used by the edge server for knowledge extraction train , public test data set D used by the edge server for testing the student model test , temperature T in KD, accuracy standard target_acc of the student model taught by the honest terminal node.

[0086] Output: whether the local model training process of the terminal node is maliciously poisoned.

[0087]

[0088]

[0089] Specifically, the prototype discrimination model in the step S200 includes three convolution layers and two fully connected layers arranged in sequence, the input is the prototype data, and the output is the label corresponding to the prototype, which is used to discriminate whether the prototype uploaded by the terminal node is correct.

[0090] First, the prototype discrimination model is pre-trained using the public dataset on the edge server. During the federated learning process, the prototype discrimination model is continuously trained using the correct prototype uploaded by the terminal node to improve the accuracy of the prototype discrimination model.

[0091] Specifically, the process of identifying whether the prototype uploaded by the terminal node is correct by the prototype discrimination model in step S200 is as follows:

[0092] The prototype uploaded by the terminal node is input into the trained prototype discrimination model to obtain the predicted label of the prototype of the terminal node.

[0093] The predicted label of the prototype of the terminal node is compared with the true label. If they are inconsistent, the prototype is rejected from participating in the subsequent aggregation of global prototypes. If they are consistent, the prototype is accepted to participate in the subsequent aggregation of global prototypes, and the prototype is used to continuously train the prototype discrimination model to improve the prediction accuracy.

[0094] Pseudocode for defense program 2 prototype discrimination defense:

[0095] Input: The prototype discrimination model feature_map_model maintained locally by the edge server, and the prototype feature_map and label target_label transmitted by the terminal node.

[0096] Output: Whether the local prototype uploaded by the terminal node is correct.

[0097]

[0098] Based on the double defense method against the poisoning attack in federated learning described above, the present application also provides a double defense system against the poisoning attack of prototypes in federated learning, comprising:

[0099] Full knowledge distillation defense module: using the method of full knowledge distillation to transfer the knowledge of the local model of the terminal node to the student model on the edge server, and then testing the learning rate of the student model, and determining whether the terminal node is a malicious node according to the set threshold, and preventing the malicious terminal node from participating in the federated learning framework;

[0100] Prototype discrimination defense module: maintaining a prototype discrimination model on the edge server to identify whether the prototype uploaded by the terminal node is correct, and rejecting incorrect prototypes from participating in aggregation, and allowing correct prototypes to participate in aggregation and continuously training the prototype discrimination model using correct prototypes.

[0101] In order to better reflect the effect of the present application, the poisoning attack in federated learning is described and tested as follows.

[0102] First, the threat model of prototype poisoning attack is determined according to the behavior of edge server and terminal node in FedProto; second, the attack process is introduced, and the effectiveness of the attack is tested.

[0103] In this embodiment, the security vulnerabilities of the prototype that may be exploited by malicious attackers and cause the model accuracy to decrease are mainly concerned, and the following two assumptions are made:

[0104] Assumption 1: All terminal nodes participating in federated learning agree to be scheduled by the edge server, and the terminal nodes agree that the global prototype aggregated by the edge server according to the local prototype is not released to each terminal node.

[0105] Assumption 2: The edge server is honest, that is, it will not maliciously produce an incorrect global prototype, and its behavior is only to aggregate the local prototype from the terminal node and issue it.

[0106] Like other studies on poisoning attacks, the goal of the attacker in this embodiment is to maliciously manipulate the local model of other terminal nodes to have a high error rate on the test data. This attack also becomes a non-targeted attack, which makes the learned local model unusable and ultimately causes users to be unable to enjoy the service. The attacker knows the local training data set, the local model and the training code of the malicious terminal node, but it cannot know the local training data and the model of other terminal nodes. At the same time, the attacker knows the aggregation rule of the edge server for the prototype. For example, the edge server will disclose the aggregation rule to improve the transparency and trustworthiness of the FedProto system. The attacker can use the known aggregation rule and can arbitrarily manipulate the local prototype to attack and break the correctness of the global prototype. It is assumed that the attacker controls k terminal nodes, that is, the attacker can re-inject k malicious terminal nodes into the FedProto system, or hack and control k honest terminal nodes. Although the malicious terminal node will perform specific steps according to the protocol, it cannot guarantee the correctness of the local training data and the correctness of the local prototype uploaded by it. For example, the malicious terminal node may randomly shuffle the labels of the training data before performing local model training, so that the correspondence between the feature data and the label is wrong. And the malicious terminal node may also generate a poisoned prototype locally and upload it to the edge server, and these incorrect local poisoned prototypes will seriously affect the correctness of the global prototype aggregated by the edge server.

[0107] In FedProto, the model optimization loss function of terminal node i is L i = CE(F(θ, x), y) + λσ iIt can be observed that the loss function is affected by two parts: one part is the cross-entropy loss CE(F(θ, x), y) generated by the local training data, and the other part is the mean square error regular term σ i , λ is to adjust the influence of the regular term on L i to expand or shrink its role. If the regular term σ i is larger than CE(F(θ, x), y), then L i will be mainly affected by the regular term, and the optimization of the model will also be mainly affected by the regular term; on the contrary, if the regular term σ i is smaller than CE(F(θ, x), y), the optimization of the model will also be mainly affected by the cross-entropy loss.

[0108] To this end, the embodiment proposes a poisoning attack FMPA (Feature Map Poisoning Attack, FMPA) using error prototypes. Malicious terminal nodes arbitrarily manipulate local prototypes to make them unreliable and upload these tampered prototypes to the edge server. If there are enough poisoned prototypes, the edge server will make mistakes when aggregating these prototypes, which will cause honest terminal nodes to calculate the wrong regular term σ locally, thereby interfering with the optimization convergence process of the model. In order to more clearly show the FMPA attack, the calculation method of the local poisoned prototype, the global prototype, the regular term, and the loss function is described here. Assuming that there is a set of malicious terminal nodes k ∈ U among the terminal nodes participating in FedProto, and taking feature data with label j as an example, the local poisoned prototype of label j calculated by the malicious terminal node k in the rth round is:

[0109]

[0110] Where l is any label other than label j. Since the label is shuffled, the prototype calculated at this time is the poisoned prototype , where j' represents the local poisoned prototype of label j. In order to enhance the effect of the FMPA attack, the malicious terminal node performs multiple expansion manipulation on the value of the poisoned prototype before uploading it to the edge server, so that it can more effectively affect the honest terminal nodes. By expanding the value of the poisoned prototype, the malicious terminal node can change the distribution of the global prototype, thereby affecting the result of the honest terminal node when calculating the regular term locally. This enhanced FMPA attack can cause the honest terminal node to be more disturbed during the model optimization process, thereby affecting the convergence and performance of the model. Where η is a real number greater than 1. In the test experiment, the η value corresponding to different data sets can also be different, for example, η = 3 in MNIST and η = 5 in CIFAR10.

[0111] The local poisoning prototype participates in the aggregation of the global prototype, which will cause the calculation deviation of the global prototype. The following is an example of the edge server in the rth round of aggregating the global prototype of label j:

[0112]

[0113] Finally, for the honest terminal node i, the regularization term is calculated using the global poisoning prototype issued by the edge server in the rth round and the local prototype in the r+1th round The regularization term will be an important part of the model optimization loss function. Due to the manipulation of malicious terminal nodes to amplify poisoning prototypes, the calculation result of the regularization term will be expanded and will occupy the main body of the local model optimization goal of honest terminal nodes, thereby causing the deviation of the model optimization direction, and finally leading to the model unable to converge or even unusable. In the r+1th round, the local objective function of the honest terminal node i is:

[0114]

[0115] The following attack program shows the basic process of FMPA attack. The program includes two functions, the Local_Update() function is executed at the terminal node, and the input is the terminal node number k, the data set D k , which is used for terminal node to update the model while calculating the local prototype; the Global_Proto_Agg() function is executed at the edge server, and the input is the local prototype calculated by all terminal nodes , which is used to aggregate all local prototypes by weighted average to calculate the global prototype. The execution sequence should be that the terminal node executes Local_Update() first (the global prototype is initialized to 0 in the initial stage), and the edge server executes Global_Proto_Agg() later; then the above execution sequence is alternated every round of training.

[0116] Attack program pseudo code:

[0117] Input: Local data set D of malicious terminal node k k , the set of all terminal nodes V, the local prototype of each terminal node with label j

[0118] Output: The malicious terminal node k calculates the local poisoning prototype The edge server aggregates the local prototype to form the global prototype.

[0119]

[0120]

[0121] Through a large number of experiments on the MNIST dataset, the following conclusions can be drawn: when there is no malicious terminal node, the average accuracy of 20 terminal nodes after 100 rounds of training is 98.36%; when there are 2 malicious terminal nodes and each poisons every 5 rounds of training, the average accuracy decreases to 98.06%; when there are 4 malicious terminal nodes and each poisons every 3 rounds, the accuracy decreases to 88.63%; when there are 10 malicious terminal nodes and each poisons every 1 round, the accuracy decreases to 16.64%, which is 81.72% lower than the accuracy without attack.

[0122] Therefore, it can be proved that the FMPA attack can effectively affect the normal operation of FedProto, resulting in low model accuracy of terminal nodes participating in it, and the larger the number of poisoned prototypes, the lower the model accuracy.

[0123] Based on the above prototype poisoning attack experiment, the following experiment verifies the effect of the double defense method De-FMPA of the application.

[0124] The De-FMPA defense experiment mainly uses the MNIST dataset and the CIFAR10 dataset as auxiliary, and other related parameters and attack experiment configurations remain the same. The MNIST dataset contains 10 label classifications, a total of 60,000 training samples and 10,000 test samples, 6,000 training samples and 1,000 test samples for each class, and each sample is a 28*28 grayscale handwritten number picture. The CIFAR10 dataset contains 10 label classifications, a total of 50,000 training samples and 10,000 test samples, 5,000 training samples and 1,000 test samples for each class, and each sample is a 32*32 three-channel color picture.

[0125] The basic configuration of FedProto uses a virtual machine to simulate 30 terminal nodes as participants in federated learning, of which terminal nodes numbered 21-30 are malicious poisoning terminal nodes. Each terminal node is assigned non-identically distributed data, and different label data in MNIST and CIFAR10 are deployed on different terminal nodes. The CNNMNIST and ResNet18 models are used to learn the above two datasets, and the learning rate is set to the experience value 0.001. In order to realize the heterogeneous scene of neural network model, the part of the hyperparameters of the neural network model deployed on the terminal node is modified as in the previous work. In MNIST, the output channel number of the convolutional layer is set to 18, 20 or 22. In CIFAR10, different convolutional layer strides are set for different terminal nodes. A high-performance server with a 3090Ti graphics card and 24GB of memory is selected for the experiment.

[0126] I. Full knowledge distillation defense experiment effect

[0127] Optimal distillation temperature: in the pre-screening stage, the full knowledge distillation method proposed in the application is used to quickly screen out the poisoning terminal node for training the local model with toxic data, and to reduce the calculation overhead in the subsequent prototype identification defense stage. In order to explore the best temperature T for full knowledge distillation, we set T to different values, and found that when T = 0.1, the knowledge of the teacher model can be extracted to the maximum extent; under different temperatures, we randomly select 2 terminal nodes as the teacher model to help train the student model of the edge server. Table 2 shows the learning effect of the student model under different temperatures, where the teacher model is the local model of the terminal node, and the student model is deployed on the edge server.

[0128]

[0129] Table 2 Learning effect of student model under different temperatures

[0130] From table 2, it can be seen that when T is set to 0.1, the knowledge extraction effect of full knowledge distillation is the best, and the highest learning rate (student model accuracy divided by teacher model accuracy) can reach 83.88% without poisoning; according to the accuracy of the student model to determine whether the teacher model is poisoned, when the accuracy of the student model is lower than a certain threshold, it is considered that the terminal node is a malicious terminal node, and in this experiment, the learning rate threshold is set to 60% for MNIST dataset, and the threshold for different datasets can also be different.

[0131] Full knowledge distillation under optimal temperature: simulate 100 virtual terminal nodes, of which 60 are malicious terminal nodes that will perform local training data scrambling, and the remaining 40 are honest terminal nodes. Table 3 records the identification accuracy of the full knowledge distillation defense stage when adding malicious or honest terminal nodes.

[0132] Poisoning Number of end nodes Number of correct identifications Identification success rate Yes 60 60 100％ No 40 37 92.5％

[0133] Table 3 Full knowledge distillation identification success rate

[0134] The identification of malicious terminal nodes is the main task of the pre-screening stage. As can be seen from Table 3, when 60 malicious terminal nodes are simulated, the pre-screening stage identifies all of them, with an accuracy of 100%; since the local model accuracy of malicious terminal nodes is mostly between 10% and 50%, they contain less knowledge, so the learning rate of the student model is lower (when T = 0.1, the learning rate is only about 54%), and the final accuracy of the student model will be lower (usually less than 60%), so full knowledge distillation has a high accuracy in identifying malicious terminal nodes. When 40 honest terminal nodes are simulated, the pre-screening stage can identify 37 of them, with an accuracy of 92.5%; for a small number of terminal nodes that failed to be identified, the reason may be that the amount of data allocated to the terminal node is small, and the data distribution is quite different from the public dataset used for full knowledge distillation, resulting in a lower learning rate of the student model and a lower final accuracy of the student model.

[0135] Knowledge distillation and full knowledge distillation: As previously known, the loss function of the student model in knowledge distillation contains two items, namely the soft target and the knowledge distillation loss, and the specific loss function is as follows:

[0136]

[0137] In the formula, y represents the true label of the data. The difference between the above formula and the loss function of full knowledge distillation is that the student model learns not only the knowledge of the teacher model (soft target) but also the knowledge of the dataset (knowledge distillation loss) through the above formula, while full knowledge distillation only learns the knowledge of the teacher model. When the weight α of the knowledge distillation loss term is 0, knowledge distillation becomes full knowledge distillation. In order to enable the student model to better reflect the teacher model, i.e. to judge whether the teacher model is poisoned by the accuracy of the student model, the following experiment is done: set the temperature to 0.1, and sequentially input 60 malicious terminal nodes into FedProto, and transfer the local model knowledge of the terminal nodes to the student model of the edge server using knowledge distillation and full knowledge distillation. Table 4 shows the comparison of the identification success rate of knowledge distillation and full knowledge distillation in the above experiment.

[0138]

[0139] Table 4 Knowledge distillation and full knowledge distillation

[0140] As can be seen from Table 4, the lower the value of the hyperparameter α in the loss function of the student model of knowledge distillation, the higher the identification success rate of malicious terminal nodes, and when α is 0, the loss function becomes full knowledge distillation, and the identification success rate rises to 100%, so full knowledge distillation has a great advantage in identifying malicious terminal nodes in the pre-screening stage.

[0141] II. Effect of prototype identification defense experiment

[0142] To demonstrate the effectiveness of the prototype discrimination defense, we make ablation experiments. The experiment opens the full knowledge distillation defense while the distribution setting opens or does not open the prototype discrimination defense, and every 10 rounds of communication invests a malicious terminal node, and the poisoning frequency distribution is 1 / 5, 1 / 3 and 1, that is, every 5 rounds of poisoning, every 3 rounds of poisoning and every round of poisoning. In order to fully reflect the necessity and effectiveness of the prototype discrimination defense, we need to set some malicious terminal nodes to participate in FedProto. Here, the malicious terminal nodes use clean data sets for local training, but upload poisoned prototype data. Because they use clean data sets to train the model, the local model of the malicious terminal node can achieve high accuracy and can bypass the full knowledge distillation defense with a higher probability and participate in subsequent FedProto.

[0143]

[0144] Table 5 Average accuracy under the condition of whether the prototype discrimination defense is opened

[0145] Table 5 shows the average accuracy of all terminal nodes at the final communication round after 100 global training rounds and every 10 rounds of communication invests a malicious terminal node. It can be seen that under different poisoning frequencies, because part of the malicious terminal node local model has high accuracy and bypasses the full knowledge distillation defense and joins FedProto to poison, the average accuracy of the model after opening the prototype discrimination defense is always higher than that without opening. Especially when the poisoning frequency is 1, the prototype discrimination defense mechanism can increase the average accuracy of the model by 39.84% and 80.84% on the CIFAR10 and MNIST data sets, respectively, fully reflecting the effectiveness and necessity of the prototype discrimination defense.

[0146] In order to clearly show the influence of the prototype discrimination defense on the model accuracy of the terminal node, the following compares the accuracy change of opening and not opening the defense under different data sets and poisoning frequencies of 1 / 5, 1 / 3 and 1. In the graph of opening the defense mechanism, only the model accuracy change graph of the honest terminal node is drawn.

[0147] MNIST data set: Figures 3 to 5 Respectively show the accuracy change of the local model of all terminal nodes participating in FedProto without opening the prototype discrimination defense when the poisoning frequency is 1 / 5, 1 / 3 and 1. When the poisoning frequency is 1 / 5, the vertical line represents the investment of a malicious terminal node at the current round. Because the poisoning frequency is low, poisoning every 5 rounds, the poisoned prototype does not affect the accuracy of the model of the honest terminal node, and the accuracy of the model of the honest terminal node is basically the same as that of the clean data set. Figure 6The average model accuracy of the terminal node is slightly different when the prototype identification defense mechanism is turned on. When the poisoning frequency is 1 / 3, the model accuracy will be significantly reduced without turning on the prototype identification defense mechanism. When the 7th malicious terminal node is added in the 70th communication, the poisoning frequency increases to once every 3 rounds, and at this time, the large number of poisoned prototypes contained in FedProto seriously affects the calculation of the global prototype, and the average model accuracy of the honest terminal node also begins to decrease and eventually decreases to 54.93%, compared with Figure 7 When the prototype identification defense mechanism is turned on, the average model accuracy of the terminal node decreases by 42.71%. When the poisoning frequency is 1, a large number of poisoned prototypes are contained in FedProto in a few rounds, Figure 5 It is shown that after the third malicious terminal node is added in the 30th communication, the model gradient explosion occurs due to the influence of a large number of poisoned prototypes, and the accuracy begins to drop sharply and finally maintains at 15%, which is much lower than Figure 8 The average model accuracy of the terminal node after turning on the prototype identification defense mechanism.

[0148] CIFAR10 dataset: We also do related experiments on the CIFAR10 dataset, Figures 9 to 11 The local model accuracy of the terminal node without turning on the prototype identification defense when the poisoning frequencies are 1 / 5, 1 / 3 and 1, respectively, Figures 12 to 14 The local model accuracy of the terminal node when turning on the prototype identification defense when the poisoning frequencies are 1 / 5, 1 / 3 and 1, respectively. It can be seen that although the poisoned prototypes produced by the malicious terminal node do not cause model gradient explosion on the CIFAR10 dataset, they also greatly affect the convergence of the local model of other honest terminal nodes. Since the CIFAR10 dataset is more complex, the loss of local classification task calculation is larger. In order to make the prototype poisoning attack effective, the malicious terminal node will enlarge 10 times before uploading the poisoned prototype, so that the poisoning efficiency is higher and the experimental results are more significant. Through Figures 9 to 14 It can be known from the comparison that when the prototype identification defense is applied in FedProto, the local model accuracy of the honest terminal node can be significantly improved.

[0149] III. The necessity of double defense mechanism

[0150] The necessity of full knowledge distillation defense: In order to reduce the consumption of computing resources in the prototype identification defense stage, we choose to use the full knowledge distillation defense to perform a round of pre-screening before the terminal node joins FedProto, so as to prevent the malicious terminal node trained with poisoned data from joining, reduce the additional running time of the prototype identification model when reasoning the prototype data, and also directly avoid the above-mentioned malicious terminal node uploading the poisoned prototype to improve the model accuracy.

[0151] Assuming that there are 30 terminal nodes participating in FedProto, 20 of which are honest terminal nodes and 10 of which are malicious terminal nodes, Table 6 shows the average model accuracy of the terminal nodes when using full-knowledge distillation defense or not.

[0152]

[0153] Table 6 Model average accuracy with or without full-knowledge distillation defense

[0154] It can be seen that after turning on the full-knowledge distillation defense, the average model accuracy of the terminal nodes is higher than that without turning on the full-knowledge distillation defense. Especially when the poisoning frequency is 1, turning on the full-knowledge distillation defense makes the model accuracy increase by 5.19%. The reason why the model accuracy is low without turning on the full-knowledge distillation defense is that all malicious terminal nodes will enter FedProto and continuously upload poisoned prototypes to the edge server. Although the edge server maintains a prototype identification model, the accuracy of the model cannot reach 100%, and a small number of poisoned prototypes will be misjudged and allowed to join the global prototype calculation. If there is no pre-screening through full-knowledge distillation, all malicious terminal nodes will join FedProto, and the number of poisoned prototypes will also increase. Due to the misjudgment of the prototype identification model, more poisoned prototypes will participate in the global prototype aggregation, and the impact on the model accuracy will also be greater.

[0155] On the other hand, if all malicious terminal nodes join FedProto without screening, when they upload a large number of poisoned prototypes to the edge server at a high poisoning frequency, they all need to go through the reasoning of the prototype identification model to identify their correctness. However, the reasoning of a large number of prototypes will seriously slow down the running time of the system and slow down the model convergence speed. From the average running time of the system recorded in Table 7 for 100 rounds of communication, the average communication time of the system when turning on the full-knowledge distillation defense is 4.07s lower than when not turning on the full-knowledge distillation defense.

[0156] Whether to open the full knowledge distillation defense Average one-round communication time Yes 38.49s No 42.56s

[0157] Table 7 System average running time with or without full-knowledge distillation defense

[0158] Necessity of prototype identification defense: The full-knowledge distillation defense stage is to quickly screen out malicious terminal nodes that use poisoned data to train local models, so as to avoid these malicious terminal nodes from joining FedProto. However, this stage does not guarantee the correctness of the prototypes uploaded by the terminal nodes participating in FedProto to the edge server. Therefore, it is necessary to use the prototype identification defense mechanism to continuously identify local prototypes to avoid the impact of poisoned prototypes on the calculation of global prototypes, and thus affect the accuracy of the local models of honest terminal nodes.

[0159] The above embodiments are only the preferred embodiments of the present application, and are not intended to limit the protection scope of the present application, and any change made on the basis of the design principles of the present application and non-creative labor shall belong to the protection scope of the present application.

Claims

1. A double defense method of a federated learning poisoning attack, characterized in that, Based on the federated learning framework, the following steps are included: S100, full knowledge distillation defense stage: using the method of full knowledge distillation to transfer the knowledge of the local model of the terminal node to the student model on the edge server, and then testing the learning rate of the student model, judging whether the terminal node is a malicious node according to the set threshold, if so, preventing the terminal node from participating in the federated learning framework; The method of full knowledge distillation includes: The local model of the terminal node is used as a teacher model to output a soft label of a probability distribution, to help the student model learn; A temperature parameter is introduced in the generation of the soft label to improve the knowledge extraction of the teacher model; The soft label output by the teacher model is used to supervise the training of the student model, so that the student model learns the knowledge of the teacher model; In the above process, the soft label is generated by a softmax function, which is expressed as: In the formula, q i The soft label generated by the terminal node i is represented by z i The original output value of the teacher model is represented by j, and the label category is represented by j. The softmax function after introducing the temperature parameter T is expressed as: Loss function F of student model full_kd is represented as: In the formula, KL represents the KL divergence function, soft_label represents the soft label generated by the teacher model, and softmax(logits / T) represents the predicted label of the student model; The edge server verifies the accuracy of the student model using a test data set, and calculates the learning rate of the student model = student model accuracy / teacher model accuracy; Determine whether the learning rate of the student model is lower than the pre-configured learning rate threshold, if so, consider that the terminal node is a malicious node, otherwise, consider that the terminal node is an honest terminal node, and allow the honest terminal node to upload the trained prototype to the edge server; S200, prototype discrimination defense stage: maintaining a prototype discrimination model on the edge server to identify whether the prototype uploaded by the terminal node is correct, if not, rejecting the prototype from participating in aggregation, if correct, allowing the prototype to participate in aggregation and using the prototype to continuously train the prototype discrimination model.

2. The double defense method of the federal learning poisoning attack according to claim 1, characterized in that, The prototype discrimination model in step S200 includes three convolutional layers and two fully connected layers arranged in sequence, and the input is prototype data and the output is the label corresponding to the prototype; First, pre-train the prototype discrimination model using a public data set on the edge server, and continuously train the prototype discrimination model using the prototype uploaded by the correct terminal node during the federated learning process.

3. The double defense method of the federal learning poisoning attack according to claim 2, characterized in that, The process of identifying whether the prototype uploaded by the terminal node is correct in step S200 is: Input the prototype uploaded by the terminal node into the trained prototype discrimination model to obtain the predicted label of the prototype of the terminal node; Compare the predicted label of the prototype of the terminal node with the true label, if not consistent, reject the prototype from participating in subsequent global prototype aggregation, if consistent, accept the prototype to participate in subsequent global prototype aggregation.

4. A double defense system against a federated learning poisoning attack, characterized in that, It includes Full knowledge distillation defense module: using the method of full knowledge distillation to transfer the knowledge of the local model of the terminal node to the student model on the edge server, and then testing the learning rate of the student model, judging whether the terminal node is a malicious node, and preventing the malicious terminal node from participating in the federated learning framework; The prototype discrimination defense module: maintaining a prototype discrimination model on the edge server, which is used to identify whether the prototype uploaded by the terminal node is correct, and to reject the incorrect prototype from participating in aggregation, and to allow the correct prototype to participate in aggregation and continuously train the prototype discrimination model using the correct prototype; The full-knowledge distillation method includes: The local model of the terminal node is used as a soft label of the probability distribution output by the teacher model to help the student model learn; A temperature parameter is introduced in the generation of the soft label to improve the knowledge extraction of the teacher model; The soft label output by the teacher model is used to supervise the training of the student model, so that the student model learns the knowledge of the teacher model; In the above process, the soft label is generated by a softmax function, which is expressed as: In the formula, q i The soft label generated by the terminal node i is represented by z i The original output value of the teacher model is represented by j, and j is the label category. The softmax function after introducing the temperature parameter T is expressed as: Loss function F of student model full_kd is represented as: In the formula, KL represents the KL divergence function, soft_label represents the soft label generated by the teacher model, and softmax(logits / T) represents the predicted label of the student model; The edge server verifies the accuracy of the student model using a test data set and calculates the learning rate of the student model = student model accuracy / teacher model accuracy; Determine whether the learning rate of the student model is lower than the pre-configured learning rate threshold. If yes, the terminal node is considered to be a malicious node, otherwise, the terminal node is considered to be an honest terminal node, and the honest terminal node is allowed to upload the trained prototype to the edge server.

Images