Federal learning defense method, electronic equipment and storage medium

Abstract

The invention belongs to the technical field of information security, and particularly relates to a federal learning defense method, electronic equipment and a storage medium. The defense method is suitable for defending poisoning attacks in a non-independent identically distributed data environment, a teacher model is constructed by fusing a historical global model, knowledge distillation and personalized training are carried out at a client, and a K-means aggregation method is used in combination with a server to jointly defend model poisoning attacks. By simulating an attack and defense mechanism in a training process, the method and system can identify and reject malicious clients, so that the defense capability of the federated learning framework on model poisoning attacks is improved, the performance of a final global model is remarkably improved, and the overall robustness and security of the federated learning system are ensured.

Description

Technical Field

[0001] The present invention belongs to the technical field of information security, and in particular relates to a federated learning defense method, electronic equipment and storage medium. Background Art

[0002] Federated learning is a distributed machine learning framework that allows multiple clients to train models locally. After local training is completed, the client sends the model update to the server for aggregation to obtain a global model. The biggest advantage of federated learning is that the client does not need to share local data, thus protecting privacy. In the federated learning scenario, the local data of the client is often non-independent and identically distributed. This imbalance of data poses a huge challenge to federated learning, because the traditional aggregation method FedAvg works well when processing independent and identically distributed data, but under the condition of non-independent and identically distributed data, the convergence and accuracy of the global model will be affected, resulting in model performance degradation or deviation. At the same time, federated learning also faces many security threats, especially attacks from malicious clients. One of the most common types of attacks is poisoning attacks. Poisoning attacks are divided into data poisoning attacks that manipulate training data and model poisoning attacks that tamper with local model updates. Both of these attack methods will interfere with the training of the global model and even cause the model to output specific biased results. In order to defend against these attacks, additional mechanisms need to be introduced in the federated learning framework to detect, isolate and filter malicious updates. Traditional defense methods include redundancy-based model validation, anomaly detection, or aggregation-based robust algorithms, but these methods may not always be effective when faced with complex Non-IID data.

[0003] Chinese patent document CN118761455A discloses a personalized federated learning method based on knowledge distillation, including the following steps: initializing the global model and the number of local devices of the central server; broadcasting the initialized global model to local devices; all local devices use local private data for model training; and uploading local model parameters to the central server to aggregate and generate a new global model; pruning the aggregated new global model through unstructured pruning technology to turn the global model into a student-friendly teacher model; passing the final prediction and attention map knowledge in the teacher model obtained after pruning to the local model, which then acts as a student model and is updated according to the transferred knowledge. However, this method does not explicitly mention the defense mechanism against malicious clients or model poisoning attacks, especially when facing data poisoning or model poisoning attacks, there is a lack of additional detection and elimination mechanisms; at the same time, this method does not propose a special solution to the special challenges of non-independent and identically distributed data, which may cause the performance of the global model to deteriorate when the data distribution is inconsistent.

[0004] Chinese patent document CN118246009A discloses a method for defending against poisoning attacks in federated learning. The method first initializes parameters. The server initializes the global model and learning rate and broadcasts them to each client. Secondly, the client receives the global model of the server, uses local data to calculate local gradients and loss values, and then uploads the local gradients and loss values ​​to the server. Then, based on the gradient, the server uses the local outlier factor algorithm to remove malicious parameter clusters and retain normal parameter clusters. Finally, the server performs score calculations based on the gradients and loss values ​​uploaded by each client in the normal parameter cluster, calculates the global model of the t+1th round, and continuously iterates and updates. However, this method only relies on local outlier factors to judge malicious clients. This defense strategy may not cover all attack modes when facing diversified and complex poisoning attacks, especially for data poisoning attacks or misleading attacks. The effect is limited; and this method only relies on gradients and loss values ​​for defense, lacking effective knowledge transfer between the client and the global model.

[0005] In order to solve the challenges of federated learning when facing non-independent and identically distributed data and poisoning attacks, the present invention proposes a federated learning defense method, electronic device and storage medium. Summary of the invention

[0006] The present invention aims to overcome at least one defect of the above-mentioned prior art and provides a federated learning defense method, which fuses global models of different rounds by constructing a historical global model storage pool, uses these historical models as teacher models, and performs local model training on the client through knowledge distillation technology; in addition, the server side uses the K-means clustering algorithm to cluster the model updates uploaded by the client to detect and eliminate potential malicious clients; by combining these technologies, the system can not only enhance the resistance to poisoning attacks, but also improve the performance of the global model in a non-independent and identically distributed environment, thereby ensuring the stability and accuracy of the model.

[0007] The invention also discloses an electronic device for implementing the method.

[0008] The invention also discloses a machine-readable storage medium for implementing the method.

[0009] In order to achieve the above object, the technical solution of the present invention is as follows: A federated learning defense method, the method comprising: S1. Initial work on the server: S11, constructing a historical global model storage pool on the server side, and randomly initializing a global model as an initialization global model and putting it into the historical global model storage pool; S12, sending the initialized global model as the teacher model and the global model to the clients randomly selected to participate in the training in the first round; S2. Client training: S21. After the client receives the global model and the teacher model sent by the server, the client performs local training on the global model based on the local data set to generate a client model and generate a local loss. At the same time, the client performs knowledge distillation training on the teacher model based on the local data set to generate a student model and generate a knowledge distillation loss. S22, jointly training the client model and the student model generated in step S21 to generate an updated local model, and sending the updated local model back to the server to participate in the next round of global model aggregation; S3, Server-side Aggregation: S31, after receiving the updated local model in step S22, the server generates a new round of global model by aggregating the local model and stores it in the historical global model storage pool; S32, obtaining a new round of teacher models by aggregating the global models in the historical global model storage pool; S4. Simultaneously send the new round of global model generated in step S31 and the new round of teacher model generated in step S32 to the client selected for training in the next round, and repeat steps S2 and S3 until the specified round is reached or the global model converges and the training ends.

[0010] Preferably, the data set in step S21 is an image data set.

[0011] Preferably, in step S21, the client performs local training on the global model based on the local data set to generate the client model and generate the local loss. Specifically, the client performs training based on its local data set and fine-tunes the global model. The local loss calculation formula (1) generated by the difference between the client model and the global model is as follows: (1) in Refers to the Round Client local losses, is the local dataset of client k, Refers to the The global model sent by the server. is the loss function, , denote the input and output vectors respectively, Indicates the use of the global model The calculated model output, Indicates that from the distribution The random variable pairs sampled from expected value.

[0012] Preferably, in step S21, the client performs knowledge distillation training on the teacher model based on the local data set to generate a student model and generates knowledge distillation loss specifically as follows: The client transfers the knowledge in the teacher model to the student model by means of knowledge distillation, which is achieved by minimizing the difference between the output of the student model and the teacher model. This process will generate knowledge distillation loss, and the student model is optimized by minimizing this loss: (2) in Refers to the Round Client The knowledge distillation loss is Refers to the The teacher model sent by the server. yes The distillation loss based on Kullback-Leibler divergence measures the difference between the probability distributions output by the teacher model and the student model. 、 denote the input and output vectors respectively; Indicates the use of the teacher model The calculated model output, Indicates that from the distribution The random variable pairs sampled from expected value.

[0013] Preferably, in step S22, the client model and the student model generated in step S21 are jointly trained to first generate a local model loss, and then an updated local model is obtained by calculating the local model loss.

[0014] Further preferably, the local model loss calculation formula is as follows: (3) in Refers to the Round Client Calculate the local model loss, is a weighted coefficient that controls the balance between local loss and distillation loss. It is Round Client local losses, It is Round Client Knowledge distillation loss of The generating of the updated local model is specifically as follows: (4) in represents the learning rate, It is Round Client The local model of , that is, the updated local model, It is Round Client The local model of Represents the local model loss About the local model gradient.

[0015] Preferably, the step S31 of generating a new round of global models by aggregating the local models is specifically as follows: After receiving the updated local model uploaded by the client in step S22, the server generates a global model by minimizing the clustering loss through the K-means aggregation method, and identifies and removes abnormal models: (5) in Refers to the total loss function of K-means, Represents the number of clusters for clustering. The number of attackers is smaller than the number of benign clients, so the local model in the largest cluster is selected for global model aggregation; Representation Cluster Local model in , Indicates t Round Client The local model of Representation Cluster The center of mass, Represents the L2 norm, which is used to measure the local model With cluster centroid The distance between them is 1, which makes similar models be classified into the same cluster, reducing the probability of malicious clients participating in training. The new global model generated : （ 6) in Refers to the The global model of the wheel, Indicates the number of clients in the largest cluster participating in this round of training, Refers to the Round Client local model.

[0016] Preferably, the step S32 of obtaining a new round of teacher models by aggregating the global models in the historical global model storage pool is specifically as follows: Use the FedAvg aggregation method to get the The teacher model sent to the client in turn : (7) in, Refers to the The teacher model sent by the server. Refers to the first A global model.

[0017] In another aspect of the present invention, there is also provided an electronic device, comprising: at least one processor; and A memory storing instructions, which, when executed by the at least one processor, causes the at least one processor to perform the federated learning defense method as described above.

[0018] In another aspect of the present invention, a machine-readable storage medium is provided, which stores executable instructions, and when the instructions are executed, the machine executes the federated learning defense method as described above.

[0019] Compared with the prior art, the present invention has the following beneficial effects: (1) The present invention proposes a defense method for resisting poisoning attacks suitable for non-independent and identically distributed data environments. The method constructs a teacher model by integrating historical global models, performs knowledge distillation and personalized training on the client, and uses the K-means aggregation method on the server to jointly defend against model poisoning attacks, which effectively resists model poisoning attacks in federated learning. It is mainly used to delete malicious clients in federated learning and improve the performance of the final global model. By simulating the attack and defense mechanisms during the training process, this method system can identify and eliminate malicious clients, which not only improves the defense capability of the federated learning framework against model poisoning attacks, but also significantly improves the performance of the final global model, ensuring the overall robustness and security of the federated learning system.

[0020] (2) The establishment of the "historical global model storage pool" of the present invention ensures that the system can call historical models for fusion at any time, thereby building a more stable and robust teacher model to provide support for the subsequent knowledge distillation process. At the same time, it avoids the requirement that the server side needs to rely on small data sets in traditional knowledge distillation, and further protects the privacy of the client.

[0021] (3) In each round of training, the server not only sends the global model screened and generated by the K-means clustering aggregation algorithm (in the first round of training, it is a randomly initialized global model), but also sends the teacher model fused by the FedAvg aggregation algorithm. These two models play different roles: the global model is used to guide the local training of the client, while the teacher model transfers its own knowledge to the client's student model through knowledge distillation. After the client receives these two models, it will apply knowledge distillation technology to extract the knowledge in the teacher model and transfer it to the client's student model. This process effectively reduces the model deviation caused by inconsistent data distribution. At the same time, the client also integrates the knowledge of the global model into its own model to further optimize the global model.

[0022] (4) After completing local training, the client uploads the updated local model to the server to participate in a new round of model aggregation. The server uses the K-means clustering aggregation algorithm to aggregate the models uploaded by the client, and generates a new round of global models after screening and eliminating potential malicious models. Combined with the use of the K-means clustering algorithm, the server can effectively identify abnormal client models, which are often uploaded by malicious clients with the intention of poisoning. Through clustering analysis, the similarities and differences of each client model can be effectively evaluated, thereby deleting those potential attack models and further ensuring the robustness of the global model. Through such continuous iteration and optimization, the entire federated learning system gradually improves the overall performance of the global model while resisting poisoning attacks, ensuring the security and reliability of the model.

[0023] (5) The present invention aims to resist model poisoning attacks in the federated learning process and improve the performance of the global model by combining the collaboration of the client and the server. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] Figure 1 It is a schematic diagram of the framework of the defense method of the present invention; Figure 2 It is a schematic diagram of the local model training process of the client of the present invention; Figure 3 This is a comparison chart of the number of training rounds and the test accuracy of the global model in the MNIST data set in Example 1 of the present invention, comparing the three defense methods and the case of no defense method under the gradient ascent attack mode; Figure 4 This is a comparison chart of the number of training rounds and the test accuracy of the global model in the MNIST data set in Example 1 of the present invention, comparing the three defense methods and the case of no defense method under the gradient misleading attack mode; Figure 5This is a comparison chart of the number of training rounds and test accuracy of the global model in the CIFAR-10 dataset in Example 1 of the present invention, comparing three defense methods and no defense method in a gradient ascent attack mode; Figure 6 This is a comparison chart of the number of training rounds and test accuracy of the global model in Example 1 of the present invention in the CIFAR-10 dataset, comparing three defense methods and no defense method under the gradient misleading attack mode. DETAILED DESCRIPTION

[0025] Example 1 The present invention provides a federated learning defense method, which realizes effective defense against federated learning attacks. Figure 1 , Figure 2 As shown, the method includes: S1. Initial work on the server: S11, constructing a historical global model storage pool on the server side, and randomly initializing a global model as an initialization global model and putting it into the historical global model storage pool for subsequent use; Specifically, the historical global model storage pool is used to store the global model generated after each round of federated learning aggregation.

[0026] S12, sending the initialized global model as the teacher model and the global model to the clients randomly selected to participate in the training in the first round; S2. Client training: S21, after the client receives the global model and the teacher model sent by the server, the client performs local training on the global model based on the local data set to generate a client model and generate a local loss. At the same time, the client performs knowledge distillation training on the teacher model based on the local data set to generate a student model and generate a knowledge distillation loss, and the data set is an image data set; Specifically, in step S21, the client performs local training on the global model based on the local data set to generate a client model and generate a local loss as follows: First, the client trains based on its local dataset to fine-tune the global model, and the difference between the client model and the global model generates a local loss: (1) in Refers to the Round Client local losses, Is the client k The local dataset, Refers to the The global model sent by the round server, is the loss function, ， denote the input and output vectors respectively, Indicates the use of global model weights The calculated model output, Indicates that from the distribution The random variable pairs sampled from Here, we calculate the difference loss between the global model and the client model output.

[0027] Specifically, in step S21, the client performs knowledge distillation training on the teacher model based on the local data set to generate a student model and generates knowledge distillation loss as follows: The client transfers the knowledge in the teacher model to the student model by means of knowledge distillation, which is achieved by minimizing the difference between the output of the student model and the teacher model. This process will generate knowledge distillation loss, and the student model is optimized by minimizing this loss: (2) in Refers to the Round Client The knowledge distillation loss is Refers to the The teacher model sent by the server. It is a distillation loss based on the Kullback-Leibler divergence, which measures the difference between the probability distributions output by the teacher model and the student model. 、 denote the input and output vectors respectively; Indicates the use of the teacher model The calculated model output, Indicates that from the distribution The random variable pairs sampled from The expected value of . This formula calculates the knowledge distillation loss between the output of the student model and the teacher model.

[0028] Knowledge distillation can not only improve the performance of local models, but also effectively reduce learning bias caused by non-IID environments. In addition, by learning from the teacher model, the client can better resist attacks from malicious participants or contaminated local models. Even if some clients are attacked or upload unreliable models, the stability of the teacher model and the knowledge integration based on multi-round global models can effectively filter out the influence of malicious or abnormal models, thereby ensuring the robustness and security of the model.

[0029] S22. The client model and the student model generated in step S21 are jointly trained to generate an updated local model, and the updated local model is sent back to the server to participate in a new round of global model aggregation.

[0030] Specifically, in step S22, the client model and the student model generated in step S21 are jointly trained to first generate a local model loss, and then an updated local model is calculated through the local model loss.

[0031] Specifically, the generated local model loss calculation formula is as follows: (3) in Refers to the Round Client Calculate the local model loss, is a weighted coefficient that controls the balance between local loss and distillation loss. It is Round Client local losses, It is Round Client Knowledge distillation loss of Finally got the client No. Round training of the local model uploaded to the server That is, the updated local model: (4) in represents the learning rate, It is Round Client The local model of , that is, the updated local model, It is Round Client The local model of Represents the local model loss About the local model This local model not only combines the characteristics of local data, but also learns the shared knowledge in the global and teacher models through distillation.

[0032] S3, Server-side Aggregation: S31, after receiving the updated local model in step S22, the server generates a new round of global model by aggregating the local model and stores it in the historical global model storage pool; Specifically, the step S31 of generating a new round of global models by aggregating the local models is as follows: After receiving the updated local model uploaded by the client in step S22, the server generates a global model by minimizing the clustering loss through the K-means aggregation method, and identifies and removes abnormal models. The specific process is as follows: (5) in Refers to the total loss function of K-means, Represents the number of clusters for clustering. The number of attackers is smaller than the number of benign clients, so we select the local model in the largest cluster for global model aggregation. Representation Cluster Local model in , Represents the client The local model of Representation Cluster The center of mass, Represents the L2 norm, which is used to measure the local model With cluster centroid This formula allows similar models to be grouped into the same cluster, reducing the probability of malicious clients participating in training.

[0033] The next step is to aggregate the local models in all the largest clusters to obtain a new round of global models. : (6) in Refers to the The global model of the wheel, is the number of clients in the largest cluster, Refers to the Round Client local model.

[0034] S32, obtaining a new round of teacher models by aggregating the global models in the historical global model storage pool; Specifically, the FedAvg aggregation method is used to obtain The teacher model sent to the client in turn : (7) in, Refers to the The teacher model sent by the server. Refers to the first A global model is created. Using this method, the server does not need to have additional data sets, which can further protect the privacy of the data.

[0035] S4. Simultaneously send the new round of global model generated in step S31 and the new round of teacher model generated in step S32 to the client selected for training in the next round, and repeat steps S2 and S3 until the specified round is reached or the global model converges and the training ends.

[0036] The present invention proposes an effective federated learning defense method by combining the knowledge distillation of the historical global model and the K-means clustering algorithm. By constructing a historical global model storage pool, the effective fusion of the historical global model is realized on the server side to form a stable teacher model, which overcomes the reliance of the traditional knowledge distillation method on the server side to have an additional data set with the same distribution as the client, and protects the privacy of the client. On the client side, the global model guidance and the teacher model knowledge distillation are combined to make the local model not only integrate the local data, but also absorb the shared knowledge of the global model, effectively reduce the deviation caused by inconsistent data distribution, and enhance the ability to resist malicious attacks. At the same time, the server side uses the K-means clustering aggregation algorithm to accurately identify and eliminate potential models to ensure the robustness of the global model.

[0037] Specifically, experiments were conducted on two public datasets, including MNIST and CIFAR-10. MNIST is a grayscale image dataset of handwritten digits. The size of each image is 28×28 pixels. The training set contains 60,000 images and the test set contains 10,000 images. They are divided into 10 categories, corresponding to the numbers 0-9, and the image content is relatively simple. CIFAR-10 is a color natural image dataset with 10 categories. The size of each image is 32×32 pixels and contains 3 color channels (RGB). The training set contains 50,000 images and the test set contains 10,000 images. It is also divided into 10 categories, but it is more difficult than MNIST.

[0038] In each round of experiments, 50 clients were set up and a fully connected neural network was used, including one hidden layer. The hidden layer size of the neural network model of each client was 1024. A total of 200 rounds of training were conducted. Each client was trained locally for two rounds, and the learning rate was is 0.001. Initially, the initial parameters of each client model are consistent with the global model. The attacker ratio is 0.4, and the attacker starts attacking from the first round. For the defense mechanism of the present invention, the weight of local loss is set. is 0.3, the corresponding distillation loss weight is 0.7, and the number of clusters It is 2, which means that the clusters are divided into two clusters. The number of attackers is smaller than the number of benign clients, so we choose the larger category of the two categories for global model aggregation.

[0039] The experiment of this invention compares two attack methods: Gradient Ascent Attack and Mislead Attack, and three defense methods: Median Aggregation Method (Median), Trimmed Mean Aggregation Method (Trimmed Mean), Our Defense Method (Our Defense) and No Defense Method (Baseline).

[0040] In a gradient ascent attack, a malicious client intentionally increases the loss of the global model by updating the gradient in reverse, making the model inaccurate.

[0041] Gradient misleading attack is a combined attack strategy that combines label flipping and gradient ascent. Its purpose is to disrupt the normal training process of the model by combining normal training gradients with the local model accumulated gradients.

[0042] The median aggregation method is a method that selects the middle value in the training data to offset the impact of anomalies or attack points and enhance the robustness of the model.

[0043] The modified mean aggregation method maintains the consistency of model weights by using the trained average weights when updating model parameters to reduce the risk of being disturbed by extreme data points.

[0044] Figure 3 and Figure 4 The test accuracy of the global model of the defense method of the present invention, the median aggregation method, the modified mean aggregation method and the no defense method is compared in the MNIST data set under the gradient rise attack and the gradient misleading attack. In the gradient rise attack, the test accuracy of the global model without the defense method fluctuates with the rounds, indicating that the model is susceptible to gradient perturbations; in the gradient misleading attack, the test accuracy decreases slightly, and stabilizes at about 96% after 200 rounds, because the misleading attack directly interferes with the model decision boundary. The median aggregation method has a relatively good defense effect against the gradient rise attack, and the test accuracy is maintained at 92.5% after 200 rounds, but the effect on the gradient misleading attack is limited because the median aggregation cannot filter malicious updates with the same direction. The modified mean aggregation method is relatively stable under the gradient misleading attack, and the test accuracy is not much different from that of the defense method of the present invention, but in the gradient rise attack, the effect does not meet expectations, which may be due to the excessive loss of effective updates due to extreme value pruning. The defense method of the present invention performs best in gradient ascent attacks, with the test accuracy remaining above 97.5% after 200 rounds. It is also significantly robust to gradient misleading attacks, and can effectively distinguish malicious updates through historical model fusion, bringing the test accuracy to over 98%.

[0045] Figure 5 and Figure 6The test accuracy of the global model with three defense methods and no defense method is shown in the CIFAR-10 dataset under gradient ascent attack and gradient misleading attack. Due to the high dimensionality of CIFAR-10 data, the attack impact is more significant and the defense is more difficult. The test accuracy of the defenseless method under misleading attack fluctuates around 60%, and further drops from 65% to 22% under gradient ascent attack, indicating that complex data is more susceptible to attacks. The median aggregation method has limited effect on gradient ascent attack and gradient misleading attack, because the gradient direction of high-dimensional data is dispersed and the median cannot effectively denoise. Figure 6 The medium corrected mean aggregation method has a high initial test accuracy under gradient misleading attacks, but it continues to fluctuate with the increase in rounds and still cannot converge after 200 rounds. It may be due to excessive pruning that causes the model to converge slowly. The defense method of the present invention combines knowledge distillation and dynamic aggregation to alleviate the spread of attacks under high-dimensional data. The test accuracy is stable at more than 73% in gradient rising attacks and remains at 75% under gradient misleading attacks, which is significantly better than other methods.

[0046] In summary, the present invention proposes an effective federated learning defense method by combining the knowledge distillation of the historical global model and the K-means clustering algorithm. It improves the performance of the model in a non-independent and identically distributed data environment while enhancing the defense capability against model poisoning attacks, thereby ensuring the security and robustness of the federated learning system. At the same time, it avoids the requirement in traditional knowledge distillation that the server needs to rely on a small data set, further protecting the privacy of the client.

[0047] Example 2 This embodiment also provides an electronic device, including: at least one processor; and A memory storing instructions, which, when executed by the at least one processor, causes the at least one processor to perform the federated learning defense method as described above.

[0048] In this embodiment, the electronic device may include, but is not limited to: personal computers, server computers, workstations, desktop computers, laptop computers, notebook computers, mobile computing devices, smart phones, tablet computers, cellular phones, personal digital assistants (PDAs), handheld devices, messaging devices, wearable computing devices, consumer electronic devices, and the like.

[0049] Example 3 This embodiment also provides a machine-readable storage medium storing executable instructions, which, when executed, cause the machine to perform the above-described federated learning defense method.

[0050] Specifically, a system or device equipped with a readable storage medium can be provided, on which software program codes that implement the functions of any of the above-mentioned embodiments are stored, and a computer or processor of the system or device can read and execute instructions stored in the readable storage medium.

[0051] In this case, the program code itself read from the machine-readable medium can realize the function of any one of the above embodiments, and thus the machine-readable code and the machine-readable storage medium storing the machine-readable code constitute part of this specification.

[0052] Examples of readable storage media include floppy disks, hard disks, magneto-optical disks, optical disks (such as CD-ROM, CD-R, CD-RW, DVD-ROM, DVD-RAM, DVD-RW, DVD-RW), magnetic tapes, non-volatile memory cards, and ROMs. Optionally, the program code may be downloaded from a server computer or a cloud via a communication network.

[0053] Obviously, the above embodiments of the present invention are merely examples for clearly illustrating the technical solution of the present invention, and are not intended to limit the specific implementation methods of the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the claims of the present invention shall be included in the protection scope of the claims of the present invention.

Claims

1. A federated learning defense method, characterized in that: The method comprises: S1. Initial work on the server: S11, constructing a historical global model storage pool on the server side, and randomly initializing a global model as an initialization global model and putting it into the historical global model storage pool; S12, sending the initialized global model as the teacher model and the global model to the clients randomly selected to participate in the training in the first round; S2. Client training: S21, after the client receives the global model and the teacher model sent by the server, the client performs local training on the global model based on the local data set to generate a client model and generate a local loss. At the same time, the client performs knowledge distillation training on the teacher model based on the local data set to generate a student model and generate a knowledge distillation loss; S22, jointly training the client model and the student model generated in step S21 to generate an updated local model, and sending the updated local model back to the server to participate in the next round of global model aggregation; S3, Server-side Aggregation: S31, after receiving the updated local model in step S22, the server generates a new round of global model by aggregating the local model and stores it in the historical global model storage pool; S32, obtaining a new round of teacher models by aggregating the global models in the historical global model storage pool; S4. Simultaneously send the new round of global model generated in step S31 and the new round of teacher model generated in step S32 to the client selected for training in the next round, and repeat steps S2 and S3 until the specified round is reached or the global model converges and the training ends.

2. The federated learning defense method according to claim 1, characterized in that: In step S21, the client performs local training on the global model based on the local data set to generate a client model and generate a local loss. Specifically, the client performs training based on its local data set and fine-tunes the global model. The local loss calculation formula (1) generated by the difference between the client model and the global model is as follows: （1） in Refers to the Round Client k local losses, Is the client k The local dataset, Refers to the The global model sent by the server. is the loss function, , denote the input and output vectors respectively, Indicates the use of the global model The calculated model output, Indicates that from the distribution The random variable pairs sampled from expected value.

3. The federated learning defense method according to claim 1, characterized in that: In step S21, the client performs knowledge distillation training on the teacher model based on the local data set to generate a student model and generate knowledge distillation loss, specifically: The client transfers the knowledge in the teacher model to the student model by means of knowledge distillation, which is achieved by minimizing the difference between the output of the student model and the teacher model. This process will generate knowledge distillation loss, and the student model is optimized by minimizing this loss: （2） in Refers to the Round Client The knowledge distillation loss is Refers to the The teacher model sent by the server. It is a distillation loss based on the Kullback-Leibler divergence, which measures the difference between the probability distributions output by the teacher model and the student model. 、 denote the input and output vectors respectively; Indicates the use of the teacher model The calculated model output, Indicates that from the distribution The random variable pairs sampled from expected value.

4. The federated learning defense method according to claim 1, characterized in that: In step S22, the client model and the student model generated in step S21 are jointly trained to first generate a local model loss, and then an updated local model is obtained through the local model loss calculation.

5. The federated learning defense method according to claim 4, characterized in that: The local model loss calculation formula is as follows: （3） in Refers to the Round Client Calculate the local model loss, is a weighted coefficient that controls the balance between local loss and distillation loss. It is Round Client local losses, It is Round Client Knowledge distillation loss of The generating of the updated local model is specifically as follows: （4） in represents the learning rate, It is Round Client The local model of , that is, the updated local model, It is Round Client The local model of Represents the local model loss About the local model gradient.

6. The federated learning defense method according to claim 1, characterized in that: The step S31 of generating a new round of global models by aggregating the local models is specifically as follows: After receiving the updated local model uploaded by the client in step S22, the server generates a global model by minimizing the clustering loss through the K-means aggregation method, and identifies and removes abnormal models: （5） in Refers to the total loss function of K-means, Represents the number of clusters for clustering. The number of attackers is smaller than the number of benign clients, so the local model in the largest cluster is selected for global model aggregation; Representation Cluster Local model in , Indicates t Round Client The local model of Representation Cluster The center of mass, Represents the L2 norm, which is used to measure the local model With cluster centroid The distance between them is 1, which makes similar models be classified into the same cluster, reducing the probability of malicious clients participating in training. The new global model generated : （6） in Refers to the The global model of the wheel, Indicates the number of clients in the largest cluster participating in this round of training, Refers to the Round Client local model.

7. The federated learning defense method according to claim 1, characterized in that: The step S32 of obtaining a new round of teacher models by aggregating the global models in the historical global model storage pool is specifically as follows: Use the FedAvg aggregation method to get The teacher model sent to the client in turn : （7） in, Refers to the The teacher model sent by the server. Refers to the first A global model.

8. An electronic device, characterized in that: The electronic device comprises: at least one processor; and A memory storing instructions, which, when executed by the at least one processor, causes the at least one processor to execute the federated learning defense method according to any one of claims 1 to 7.

9. A machine-readable storage medium storing executable instructions, characterized in that: When the instructions are executed, the machine performs the federated learning defense method as described in any one of claims 1-7.

Images