A federated learning backdoor erasing method based on adversarial knowledge distillation
By employing adversarial knowledge distillation in a federated learning environment, and utilizing adversarial training loss function and backdoor regularization techniques, the backdoor defense problem on the server side is solved. This enables the effective erasure of backdoor attacks without relying on high-quality datasets, protecting data privacy while maintaining model performance.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- SHANGHAI JIAOTONG UNIV
- Filing Date
- 2023-09-12
- Publication Date
- 2026-07-31
AI Technical Summary
Existing backdoor defenses for federated learning are ineffective at erasing backdoor attacks in privacy-preserving environments, especially when servers cannot access training data and rely on high-quality datasets, leading to degraded model performance or privacy breaches.
An adversarial knowledge distillation-based approach is adopted. On the server side, adversarial data-free knowledge distillation and backdoor regularization techniques are used to suppress the output and attention mutations of the backdoor model by utilizing the adversarial training loss function, backdoor response suppression, and attention graph alignment, thereby achieving backdoor erasure.
It effectively eliminates backdoor attacks, reduces model accuracy loss, and maintains model performance without relying on high-quality datasets. It is applicable to any communication round of federated learning and protects data privacy.
Smart Images
Figure CN117195208B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of network security, and in particular to a federated learning backdoor erasure method based on adversarial knowledge distillation. Background Technology
[0002] Federated Learning (FL) is a distributed machine learning method that allows multiple clients to collaboratively train neural network models while ensuring local data privacy. However, data poisoning attacks in FL scenarios have become a widely concerned security threat. Backdoor attacks, as a form of targeted data poisoning, induce the global model to generate correct outputs on benign samples while simultaneously outputting incorrect predictions specified by the attacker on specific samples.
[0003] Existing backdoor defense measures can be divided into four categories: poisoned data detection, robust learning, backdoor detection, and backdoor erasure. In the FL scenario, due to the limited computing resources of terminal devices and the complexity of trust relationships, backdoor defense measures need to be deployed on the server. Under the monolithic model, the above backdoor defense measures have considerable effects, but when they are deployed to the FL server, they still face the following challenges: (1) The server only participates in model aggregation and distribution, and does not participate in local training, which makes poisoned data detection and robust learning measures unavailable; (2) Privacy protection requirements make it impossible for the server to access training data, while trigger reverse engineering and backdoor erasure measures often rely on high-quality clean data; (3) Existing FL backdoor defense methods that detect and remove malicious local updates need to check model weights, which may also lead to privacy violations. In addition, most of them are only effective for specific attack models and may fail if the assumptions about the data distribution or the attacker’s strategy are not met (rejecting benign weights or accepting malicious weights); (4) Other backdoor defense efforts in FL dilute the backdoor effect through pruning, noise or smoothing. Although these strategies do not require specific attack model assumptions, they may significantly reduce the benign performance of the model.
[0004] Knowledge distillation (KD) promises to overcome the aforementioned challenges and provide new insights into backdoor erasure in fine-tuned models (FL). Existing backdoor erasure methods based on knowledge distillation include: (1) knowledge distillation based on model response; (2) attention-based knowledge distillation, which erases backdoors by distilling attention-based knowledge from a teacher model finely tuned to clean data into the backdoor model; and (3) self-attention distillation, which uses observations of backdoors being hidden in deeper layers rather than shallower layers to allow poorly performing deep layers to learn goodly performing shallow layers. These methods largely rely on high-quality datasets and are therefore unavailable in privacy-preserving FL environments. Summary of the Invention
[0005] The purpose of this invention is to provide a backdoor erasure method for federated learning based on adversarial knowledge distillation. This method overcomes the limitations of existing technologies that rely on high-quality datasets by balancing knowledge transfer from the backdoor teacher to the student model and backdoor regularization used to eliminate potential backdoor responses in the student model. It is suitable for privacy-preserving FL environments.
[0006] The objective of this invention can be achieved through the following technical solutions:
[0007] A backdoor erasure method based on adversarial knowledge distillation in federated learning is used for server-side erasure of implicit backdoors in the global model in a federated learning environment. The method includes the following steps:
[0008] Step 1) The server uses the client's local model to update and initialize the teacher and student models;
[0009] Step 2) Construct an adversarial training loss function and perform knowledge transfer from the teacher model to the student model through adversarial knowledge distillation;
[0010] Step 3) The server uses the samples generated by knowledge transfer in Step 2) to perform backdoor regularization using two methods: backdoor response suppression and attention map alignment, and determines the backdoor regularization term.
[0011] Step 4) The server alternately repeats steps 2) and 3) to train the model based on the adversarial training loss function and the backdoor regularization term until the student model accuracy converges, thus obtaining the clean student model after erasure. The clean student model is then distributed to the client as a global model.
[0012] The specific round in which backdoor erasure is performed on the federated server is determined by the defender. If the server performs backdoor erasure during a certain communication round of federated learning, then step 1 is executed.
[0013] In step 1), the model initialization method is as follows: the server collects local updates from the client and aggregates them to obtain a global model containing potential backdoors, and copies its model parameters to the teacher model and student model that have the same model structure.
[0014] Step 2) includes the following steps:
[0015] Step 2-1) Use data-free adversarial distillation to impart the teacher's knowledge to the student, and use a sample generator to dynamically reconstruct samples that cause huge differences between the teacher and student models;
[0016] Step 2-2) Reconstruct samples using a sample generator. The student model minimizes the difference between itself and the teacher model based on the adversarial training loss function.
[0017] The adversarial training loss function is:
[0018]
[0019] Where z is a vector randomly sampled from a normal distribution. Indicates a sample generator. Representing the teacher model, Representing the student model, z ~ p z (z) indicates that z follows a normal distribution p. z A vector randomly sampled from (z), It represents the expected value of a mathematical calculation.
[0020] Step 3) includes the following steps:
[0021] Step 3-1) Use the trigger generator to generate triggers that are most sensitive to the student model;
[0022] Step 3-2) Backdoor response suppression:
[0023] To suppress backdoor responses in the student model output, the backdoor features at the model output level are described as follows: on the original image, the model outputs the correct label; on the image containing the trigger, the model outputs a backdoor target label that is different from the correct label; using the trigger generated in step 3-1), the student model eliminates backdoor responses at the model output level by minimizing the backdoor response suppression regularization term.
[0024] Step 3-3) Attention map alignment:
[0025] To suppress backdoor responses in the intermediate layers of the student model, the backdoor features at the intermediate attention level of the model are described as follows: on the original image, the intermediate layer attention is distributed across all image features that contribute to image classification; on the image containing triggers, the intermediate layer attention is focused on the trigger regions in the image; using the triggers generated in step 3-1), the student model eliminates backdoor responses at the intermediate attention level by minimizing the attention map alignment regularization term.
[0026] Steps 3-4) Combine the backdoor response suppression regularization term and the attention map alignment regularization term to determine the total backdoor regularization term.
[0027] The backdoor response suppression regularization term is expressed as follows:
[0028]
[0029] in, This indicates a trigger generator, whose generated triggers are limited to the size of hw; Indicates the sample generator; Representing the teacher model, Representing the student model, This indicates that x is generated by the sample generator. Randomly generated samples, z ~ p z (z) indicates that z follows a normal distribution p. z A vector randomly sampled from (z), It represents the expected value of a mathematical calculation.
[0030]
[0031] in, This represents the measurement of two different input images x and x′ in the middle layer of the model. The function that differentiates the two attention maps is defined as follows:
[0032]
[0033] in, This represents a function that maps a feature map to an attention map.
[0034] The total backdoor regularization term is:
[0035]
[0036] Where μ represents the weight of the attention graph alignment regularization term.
[0037] In step 4), the overall adversarial process of model training based on the adversarial training loss function and the backdoor regularization term is described as follows:
[0038]
[0039] in, Represents the adversarial training loss function. Let λ represent the backdoor regularization term, and λ represent the weight of the backdoor regularization term. Indicates a trigger generator. Indicates a sample generator. Representing the teacher model, This represents the student model.
[0040] In the technical solution of this invention, the characteristics of backdoor responses are first analyzed, and a backdoor regularization loss term combining backdoor response suppression and attention map alignment is proposed. The key idea is that backdoor models exhibit significantly different outputs and layer-by-layer attention on benign samples and samples with triggers. By utilizing adversarial optimization to minimize these two differences, abrupt changes in model output and attention caused by any potential backdoor activation can be suppressed, ultimately resulting in a clean student model without backdoor responses.
[0041] Compared with the prior art, the present invention has the following beneficial effects:
[0042] (1) Compared with backdoor erasure methods that require clean data, this invention supports backdoor erasure in a federated environment without the server having any training data by deploying adversarial data-free knowledge distillation and backdoor regularization techniques on the server side.
[0043] (2) This invention minimizes the damage to model accuracy caused by backdoor erasure by fully transferring the positive knowledge of the teacher model to the student model in adversarial knowledge distillation.
[0044] (3) Since the backdoor defense measures proposed in this invention only involve the server and global model, and are independent of the training, distribution and aggregation process of the local model in federated learning, this invention is compatible with the original federated learning framework and can be used as a plug-and-play defense measure in any communication round of federated learning. Attached Figure Description
[0045] Figure 1 This is a flowchart of the method of the present invention;
[0046] Figure 2 This is a comparison diagram of generated samples and real training samples during the adversarial knowledge distillation process in one embodiment.
[0047] Figure 3 This is a comparison of the intermediate layer attention map results after backdoor erasure is performed in one embodiment;
[0048] Figure 4 This is a graph showing the impact of different erasure cycles on model performance metrics in one embodiment of federated learning. Detailed Implementation
[0049] The present invention will now be described in detail with reference to the accompanying drawings and specific embodiments. These embodiments are based on the technical solution of the present invention and provide detailed implementation methods and specific operating procedures. However, the scope of protection of the present invention is not limited to the following embodiments.
[0050] This embodiment provides a backdoor erasure method based on adversarial knowledge distillation in federated learning, used by the server to erase hidden backdoors in the global model within a federated learning environment. Figure 1 As shown, it includes the following steps:
[0051] Step 1) The server uses the client's local model to update and initialize the teacher and student models.
[0052] The specific round in which backdoor erasure is performed on the federated server is determined by the defender. If the server performs backdoor erasure during a certain communication round of federated learning, then step 1 is executed.
[0053] Specifically, the model initialization method is as follows: the server collects local updates from the client and aggregates them to obtain a global model containing potential backdoors, and copies its model parameters to the teacher model and student model with the same model structure.
[0054] Step 2) Construct an adversarial training loss function and perform knowledge transfer from the teacher model to the student model through adversarial knowledge distillation.
[0055] Step 2) includes the following steps:
[0056] Step 2-1) To avoid infringing on data privacy, a non-adversarial distillation method is used to transfer the teacher's knowledge to the students, utilizing a sample generator. Dynamic reconstruction resulted in samples with significant differences between teacher and student models;
[0057] Step 2-2) Reconstruct samples using a sample generator. The student model minimizes the difference between itself and the teacher model based on the adversarial training loss function.
[0058] In this embodiment, the adversarial training loss function is:
[0059]
[0060] Where z is a vector randomly sampled from a normal distribution. Indicates a sample generator. Representing the teacher model, Representing the student model, z ~ p z (z) indicates that z follows a normal distribution p. z A vector randomly sampled from (z), It represents the expected value of a mathematical calculation.
[0061] like Figure 2 As shown, to demonstrate that this invention does not infringe on data privacy, this embodiment compares the model's real training samples (first row) and the generated samples (second row) during the adversarial knowledge distillation process on two datasets: CIFAR10 (first column) and CIFAR100 (second column). The figure shows that the generated samples used in this invention have extremely low similarity to the real samples, thus ensuring the client's local data privacy to a certain extent.
[0062] Step 3) The server uses the samples generated by knowledge transfer in Step 2) to perform backdoor regularization using two methods: backdoor response suppression and attention map alignment. The backdoor regularization term is determined to suppress the transfer of backdoor knowledge to the student model in Step 2).
[0063] Step 3) includes the following steps:
[0064] Step 3-1) Using a trigger generator Generate triggers that are most sensitive to the student model, with the size of the generated triggers limited to hw.
[0065] Step 3-2) Backdoor response suppression
[0066] To suppress backdoor responses in the student model output, the backdoor feature at the model output level is described as follows: on the original image, the model outputs the correct label; on the image containing the trigger, the model outputs a backdoor target label that differs from the correct label. Using the trigger generated in step 3-1), the student model eliminates backdoor responses at the model output level by minimizing the Backdoor Response Suppression (BRS) regularization term.
[0067] The backdoor response suppression regularization term is expressed as:
[0068]
[0069] in, Indicates a trigger generator; Indicates the sample generator; This indicates that x is generated by the sample generator. Randomly generated samples, z ~ p z (z) indicates that z follows a normal distribution p. z A vector randomly sampled from (z), It represents the expected value of a mathematical calculation.
[0070] Step 3-3) Attention Map Alignment
[0071] The neural network model with hidden backdoors not only exhibits backdoor features at the model output level but also at the intermediate attention level. To suppress backdoor responses in the intermediate layers of the student model, the backdoor features at the intermediate attention level are described as follows: on the original image, the intermediate layer attention is distributed across all image features contributing to image classification; on images containing triggers, the intermediate layer attention is focused on the trigger regions within the image. Using the triggers generated in step 3-1), the student model eliminates backdoor responses at the intermediate attention level by minimizing the Attention Map Alignment (AMA) regularization term.
[0072]
[0073] in, This represents the measurement of two different input images x and x′ in the middle layer of the model. The function that differentiates the two attention maps is defined as follows:
[0074]
[0075] in, This represents a function that maps a feature map to an attention map. In this embodiment, the mapping form is chosen as follows:
[0076]
[0077] Where C l express The number of channels in the layer.
[0078] Steps 3-4) Combine the backdoor response suppression regularization term and the attention map alignment regularization term to determine the total backdoor regularization term.
[0079] The total backdoor regularization terms are:
[0080]
[0081] Where μ represents the weight of the attention graph alignment regularization term.
[0082] like Figure 3 As shown, after implementing this embodiment on a backdoor model, the backdoor response at the attention level of the model is basically eliminated. That is, on poisoned images containing real triggers, the intermediate layer attention of the student model is highly similar to the attention of the clean model without backdoors on clean data.
[0083] Step 4) The server alternately repeats steps 2) and 3) to train the model based on the adversarial training loss function and the backdoor regularization term until the student model accuracy converges, thus obtaining the clean student model after erasure. The clean student model is then distributed to the client as a global model.
[0084] In step 4), the overall adversarial process of model training based on the adversarial training loss function and the backdoor regularization term is described as follows:
[0085]
[0086] in, Represents the adversarial training loss function. Let λ represent the backdoor regularization term, and let λ represent the weight of the backdoor regularization term.
[0087] During the adversarial distillation process, the model accuracy is continuously observed to see if it reaches a convergence state. If it converges, the clean model obtained by performing backdoor erasure is distributed to the client as the global model.
[0088] like Figure 4As shown in the figure, this embodiment assumes that the attacker begins continuous poisoning in the 250th round of federated learning. The defender plots the model accuracy and backdoor attack success rate curves for four defense deployment methods: no defense measures, erasure period T=10, erasure period T=20, and erasure period T=30. The figure shows that, while ensuring minimal damage to model accuracy, this invention can reduce the backdoor attack success rate to below 1%, and the shorter the erasure period, the lower the maximum achievable backdoor attack success rate.
[0089] The preferred embodiments of the present invention have been described in detail above. It should be understood that those skilled in the art can make numerous modifications and variations based on the concept of the present invention without creative effort. Therefore, all technical solutions that can be obtained by those skilled in the art based on the concept of the present invention through logical analysis, reasoning, or limited experimentation on the basis of existing technology should be within the scope of protection defined by the claims.
Claims
1. A backdoor erasure method for federated learning based on adversarial knowledge distillation, characterized in that, The method for removing hidden backdoors in the global model on the server side in a federated learning environment includes the following steps: Step 1) The server uses the client's local model to update and initialize the teacher and student models; Step 2) Construct an adversarial training loss function and perform knowledge transfer from the teacher model to the student model through adversarial knowledge distillation; Step 3) The server uses the samples generated by knowledge transfer in Step 2) to perform backdoor regularization using two methods: backdoor response suppression and attention map alignment, and determines the backdoor regularization term. Step 4) The server alternately repeats steps 2) and 3) to train the model based on the adversarial training loss function and the backdoor regularization term until the student model accuracy converges, and obtains the clean student model after erasure. The clean student model is then distributed to the client as a global model. Step 3) includes the following steps: Step 3-1) Use the trigger generator to generate triggers that are most sensitive to the student model; Step 3-2) Backdoor response suppression: To suppress backdoor responses in the student model output, the backdoor features at the model output level are described as follows: on the original image, the model outputs the correct label; on the image containing the trigger, the model outputs a backdoor target label that is different from the correct label; using the trigger generated in step 3-1), the student model eliminates backdoor responses at the model output level by minimizing the backdoor response suppression regularization term. Step 3-3) Attention Map Alignment: To suppress backdoor responses in the intermediate layers of the student model, the backdoor features at the intermediate attention level of the model are described as follows: on the original image, the intermediate layer attention is spread across all image features that contribute to image classification; on the image containing triggers, the intermediate layer attention is focused on the trigger regions in the image; using the triggers generated in step 3-1), the student model eliminates backdoor responses at the intermediate attention level by minimizing the attention map alignment regularization term. Steps 3-4) Combine the backdoor response suppression regularization term and the attention map alignment regularization term to determine the total backdoor regularization term.
2. The backdoor erasure method based on adversarial knowledge distillation according to claim 1, characterized in that, The specific round in which backdoor erasure is performed in the federated server is determined by the defender. If the server performs backdoor erasure in a certain communication round of federated learning, then step 1 is executed.
3. The backdoor erasure method based on adversarial knowledge distillation according to claim 1, characterized in that, In step 1), the model initialization method is as follows: the server collects local updates from the client and aggregates them to obtain a global model containing potential backdoors, and copies its model parameters to the teacher model and student model that have the same model structure.
4. The backdoor erasure method based on adversarial knowledge distillation according to claim 1, characterized in that, Step 2) includes the following steps: Step 2-1) Use data-free adversarial distillation to impart the teacher's knowledge to the students, and use a sample generator to dynamically reconstruct samples that cause huge differences between the teacher and student models; Step 2-2) Reconstruct samples using a sample generator. The student model minimizes the difference between itself and the teacher model based on the adversarial training loss function.
5. The backdoor erasure method based on adversarial knowledge distillation according to claim 4, characterized in that, The adversarial training loss function is: in, Representing the teacher model, Representing the student model, This indicates that z follows a normal distribution. A vector randomly sampled from the middle. This indicates the calculation of mathematical expectation. This indicates a sample generator.
6. The backdoor erasure method based on adversarial knowledge distillation according to claim 1, characterized in that, The backdoor response suppression regularization term is expressed as follows: in, This represents a trigger generator, whose generated trigger size is limited to... ; Indicates the sample generator; Representing the student model; This indicates that x is generated by the sample generator. Randomly generated samples This indicates that z follows a normal distribution. A vector randomly sampled from the middle. It represents the expected value of a mathematical calculation.
7. The backdoor erasure method based on adversarial knowledge distillation according to claim 1, characterized in that, The attention map alignment regularization term is represented as follows: in, Indicates a trigger generator. Indicates a sample generator. Representing the student model, This indicates that x is generated by the sample generator. Randomly generated samples This indicates that z follows a normal distribution. A vector randomly sampled from the middle. This indicates the calculation of mathematical expectation. This represents measuring two different input images. In the middle layer of the model The function that differentiates the two attention maps is defined as follows: in, This represents a function that maps a feature map to an attention map.
8. The backdoor erasure method based on adversarial knowledge distillation according to claim 1, characterized in that, The total backdoor regularization term is: in, This represents the weight of the attention map alignment regularization term. This indicates a backdoor response suppression regularization term. This indicates the attention map alignment regularization term. Indicates a trigger generator. Indicates a sample generator. This represents the student model.
9. The backdoor erasure method based on adversarial knowledge distillation according to claim 1, characterized in that, In step 4), the overall adversarial process of model training based on the adversarial training loss function and the backdoor regularization term is described as follows: in, Represents the adversarial training loss function. This represents the backdoor regularization term. This indicates the weight of the backdoor regularization term. Indicates a trigger generator. Indicates a sample generator. Representing the teacher model, This represents the student model.