Convolutional neural network federated learning backdoor defense method based on model bias
By using the out-of-distributed sample set to calculate model bias and perform reverse learning in federated learning, the hidden and destructive problems of backdoor attacks in federated learning are solved, and the effectiveness and efficiency of defense are improved.
Patent Information
- Application Number
- CN202510295767.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-03-13
- Publication Date
- 2025-06-27
AI Technical Summary
The concealment and destructiveness of backdoor attacks in federated learning puts its security at severe challenges, and the existing technology has weak defense capabilities and low efficiency.
A convolutional neural network federated learning backdoor defense method is adopted to calculate the bias of each local model using a distributed sample set through a central server, detect the backdoor model, and remove the backdoor features through the inverse learning mechanism.
It improves the detection effectiveness and defense efficiency of the backdoor model, and can identify benign models and backdoor models in complex or non-independent and homogeneous environments, reducing computing resource consumption and improving detection efficiency.
Smart Images

Figure CN120217383A_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the technical field of neural network security, and relates to a backdoor defense method for federated learning of convolutional neural networks, specifically a backdoor defense method for federated learning of convolutional neural networks based on model bias, which can be applied to fields such as autonomous driving, face recognition, finance, chemistry, and biology. Background Art
[0002] In the field of machine learning, backdoor attacks are a highly concealed and destructive form of attack. Attackers implant specific "trigger" samples in the training data and tamper with the labels of these samples, causing the model to perform well under normal circumstances but outputting the wrong results preset by the attacker when encountering the trigger condition. The forms of triggers are diverse and may be tiny perturbations in images, specific color markings, or special symbols in text, etc. Since triggers do not appear in most input samples, backdoor attacks are extremely concealed and difficult to detect through conventional security detection methods. The threat of this attack lies in that attackers can manipulate the decisions of the model under specific conditions, leading to serious security risks.
[0003] With the increasing demand for data privacy protection, federated learning, as a distributed machine learning method, has gradually become an important technology for solving the data silo problem. In federated learning, multiple devices (or participants) train models based on local data without uploading the data to a central server. After each device completes local training, it uploads the model updates (such as weights or gradients) to the central server, and the server generates a global model by aggregating these weights. The global model is then distributed back to each device for the next round of local training. The advantage of this method is that the data always remains on the local device, avoiding the risk of data privacy leakage and making full use of the computing power of distributed devices. Federated learning is widely used in fields such as healthcare and finance and has become an important tool for promoting data collaboration.
[0004] However, the distributed nature of federated learning also makes it a new target for backdoor attacks. In a federated learning environment, attackers can control certain local devices, implant malicious samples with triggers into their training data, and tamper with the labels of these samples. Due to the privacy protection mechanism of federated learning, the central server cannot directly access the local data, so it is difficult to detect these malicious samples. Local devices will learn the association between the trigger and the wrong label during the training process and upload these abnormal model updates to the central server. When the central server aggregates these updates, it cannot effectively distinguish between normal updates and malicious updates, resulting in the global model outputting wrong results as expected by the attacker when encountering inputs with triggers. The concealment and destructiveness of this attack pose a severe challenge to the security of federated learning.
[0005] Backdoor defense in federated learning faces additional difficulties compared to backdoor defense in a centralized learning environment. In a centralized scenario, the server can directly access the training data, enabling the use of methods such as data cleaning and anomaly detection to identify and filter malicious samples. In federated learning, due to data privacy restrictions, the server cannot directly review the local data or the model training process, making it difficult to directly apply traditional data-level defense measures. Additionally, attackers can inject backdoors on some clients while others remain normal, making the attack more concealed and difficult to detect through abnormal performance of a single global model. To address these challenges, backdoor defense in federated learning relies on statistical analysis of model updates, robust aggregation methods (such as pruning, weighted aggregation), or anomaly detection strategies based on historical update behavior.
[0006] Existing backdoor defense methods for federated learning of convolutional neural networks generally include: the central server detects the backdoor model based on the model parameters uploaded by the clients, and then removes the backdoor in the model through pruning or anti-learning. For example, Xi'an Jiaotong University discloses a defense method for backdoor attacks in federated learning in its patent document "A Defense Method for Backdoor Attacks in Federated Learning" (Patent Application No.: CN202410242568.8, Publication No.: CN118036770A). The main steps of this method are: (1) statistically analyze the parameter distribution of the uploaded model; (2) preliminarily filter potential backdoor models through an adaptive filtering method; (3) perform trigger reverse engineering on the potential backdoor models and conduct outlier detection to detect the backdoor model and the class labels implanted with backdoors; (4) remove the backdoor in the backdoor model through the technology of backdoor forgetting. This method detects the backdoor model and removes the backdoor in the model through statistical analysis of model parameters and trigger reverse engineering technology. However, since the data samples of different participants in federated learning usually have non-independent and identically distributed situations, backdoor detection methods based on model parameter differences are often difficult to effectively distinguish between backdoor models and benign models in such an environment, and cannot guarantee the effectiveness of the defense method; at the same time, performing trigger reverse engineering on backdoor models has a large computational cost and high consumption of computing resources, and the efficiency of applying this defense method on large-scale datasets is low. Summary of the Invention
[0007] The object of the present invention is to overcome the defects existing in the above-mentioned prior art, and propose a backdoor defense method for federated learning of convolutional neural networks based on model bias, which is used to solve the technical problems of weak defense ability and low defense efficiency existing in the prior art.
[0008] To achieve the above object, the technical solution adopted by the present invention includes the following steps:
[0009] (1) Construct a federated learning system:
[0010] Construct a federated learning system including a central server and a total of B clients consisting of N benign clients and K malicious clients; the central server obtains an out-of-distribution sample set D including U RGB images OOD , and sends a convolutional neural network as the global model to each client c b , where K≥1, N≥10, U≥200;
[0011] (2) Each client obtains a training sample set:
[0012] Each client c b labels the targets in the obtained M RGB images including P target categories; then each benign client c n forms a first training sample set D with the M RGB images and their labels n train , and each malicious client c k implants the trigger T into S RGB images and modifies the target labels, and then forms a second training sample set D with the S RGB images containing the trigger and their modified labels and the J = M - S RGB images without the trigger and their labels k train , where P≥2, M≥200, and the s-th RGB image implanted with the trigger T is
[0013] (3) Each client iteratively trains the local model:
[0014] Each client c b uses the global model sent by the central server as the local model and iteratively trains it through its own training sample set, and uploads the model weights ω b of the trained local model to the central server;
[0015] (4) The central server calculates the bias of the local model and detects the backdoor model:
[0016] The central server calculates the bias Γ OOD of each local model through the out-of-distribution sample set D b , and takes the O local models when the largest element l in Γ b satisfies l≥ξ as the backdoor models, where 1≥ξ≥0, O≥0;
[0017] (5) The central server obtains the backdoor defense result:
[0018] The central server passes through the out-of-distribution sample set D OODAnti-learn the weights of O backdoor models to obtain, and aggregate the weights of the O backdoor-removed model weights and the weights of H = B - O non-backdoor models after anti-learning to obtain the aggregated weight ω global The global model without backdoors.
[0019] Compared with the prior art, the present invention has the following advantages:
[0020] 1. The central server of the present invention detects the backdoor model through the bias of each local model calculated by the out-of-distribution sample set. The bias characteristics of the out-of-distribution data are not restricted by the data distribution, and can effectively identify benign models and backdoor models in complex or non-independent and identically distributed environments. Compared with the prior art, the effectiveness of defense is improved, and it can adapt to different data distributions and attack methods.
[0021] 2. The central server of the present invention removes the backdoor by anti-learning the weights of the backdoor model. Since the out-of-distribution data can effectively approximate the characteristics of the backdoor samples, it can simulate the potential impact of the backdoor samples, and the model gradually forgets the backdoor characteristics through the anti-learning mechanism, more efficiently weakening the backdoor effect, avoiding the defect of large computational overhead of reverse engineering in the prior art, and improving the defense efficiency. BRIEF DESCRIPTION OF THE DRAWINGS
[0022] Figure 1 is the implementation flowchart of the present invention. DETAILED DESCRIPTION OF THE INVENTION
[0023] The present invention will be further described in detail below with reference to the drawings and specific embodiments.
[0024] Refer to Figure 1 The present invention includes the following steps:
[0025] Step 1) Construct a federated learning system:
[0026] Construct a federated learning system including a central server and a total of B clients composed of N benign clients and K malicious clients; the central server obtains an out-of-distribution sample set D including U RGB images OOD and sends the convolutional neural network as the global model to each client c b where K ≥ 1, N ≥ 10, U ≥ 200;
[0027] In this embodiment, N = 99, K = 1, U = 200.
[0028] The global model in this step includes a stacked input layer, multiple convolutional-pooling layers, and a fully connected layer.
[0029] In this embodiment, the convolutional neural network takes RGB images as input. The parameters of this convolutional neural network instance include the weights of the convolutional layer and the fully connected layer, as well as the bias terms of each layer. The first convolutional layer consists of 16 3×3 convolutional kernels. Each convolutional kernel receives 3 input channels (RGB) and has a bias. The total number of parameters is 448. After the first convolutional layer, there is a 2×2 max-pooling layer, which halves the size of the feature map. The second convolutional layer contains 32 3×3 convolutional kernels. Each kernel corresponds to 16 input channels and has an additional bias, with a total of 4,640 parameters. After the second convolutional layer, there is also a 2×2 max-pooling layer to further reduce the size of the feature map. After the flattening operation, the feature dimension becomes 8×8×32 = 2,048, which is connected to a fully connected layer with 128 hidden units. Adding the bias, the total number of parameters is 262,272. Finally, the output layer connects 128 hidden units to 10 classes and includes the corresponding bias, with a total of 1,290 parameters.
[0030] In this embodiment, the out-of-distribution dataset contains 200 RGB images, which are from the network and cover multiple categories such as plants, animals, vehicles, and landscapes.
[0031] Step 2) Each client obtains the training sample set:
[0032] Each client c b Labels the targets in the obtained M RGB images including P target categories; then each benign client c n Forms the first training sample set D from the M RGB images and their labels n train Each malicious client c k Implants the trigger T into S RGB images and modifies the target labels, and then forms the second training sample set D from the S RGB images containing the trigger and their modified labels and the J = M - S RGB images without the trigger and their labels, where P≥2, M≥200, and the s-th RGB image implanted with the trigger T is k train In this step, each malicious client c
[0033] Implants the trigger T into S RGB images and modifies the annotation of the images. The image obtained by implanting the trigger T into the s-th RGB image is x k s The implantation method is that each malicious client c k Implants the trigger T into S RGB images, where the s-th RGB image x s The image after implanting the trigger T
[0034]
[0035] Among them, ⊙ represents the Hadamard product, x represents the image without the implanted trigger T, and δ is the position mask of the trigger.
[0036] The local training sample set adopted in this embodiment is sourced from the CIFAR-10 dataset. This dataset contains a color image training set for general object recognition, covering 10 categories, namely: airplane, car, bird, cat, deer, dog, frog, horse, ship, and truck. The dataset contains a total of 50,000 images. The size of each image is fixed, and each category contains 5,000 images. In this embodiment, each client is assigned 500 images of 10 categories, and it is ensured that the data between different clients does not repeat.
[0037] In this embodiment, S = 100. The selection of S has an important impact on the effect of the backdoor attack. If the value of S is too large, although it can enhance the effect of the backdoor attack, it may reduce the accuracy of the model on the main task, and even damage the performance of the overall model when exceeding a certain threshold. On the contrary, if the value of S is too small, although it can improve the concealment of the backdoor attack, it may weaken the attack intensity. In this embodiment, the position matrix used corresponds to the 3×3 area in the upper right corner of the image, and the selected trigger is a red block. After the trigger is implanted, a 3×3 red square will appear in the upper right corner of the sample image.
[0038] Step 3) Each client iteratively trains the local model:
[0039] Each client c b uses the global model sent by the central server as the local model and iteratively trains it through its own training sample set, and uploads the model weights ω b of the trained local model to the central server;
[0040] In this step, each client iteratively trains the local model, and the implementation steps are as follows:
[0041] (3a) Initialize the iteration number as v, the maximum iteration number as V, V ≥ 2, and the weight of the local model at the v-th iteration is ω v , and let v = 1;
[0042] (3b) Each benign client c n and each malicious client c k respectively use D n train and D k trainAs the input of its own local model, it successively performs feature extraction in the convolutional layer, dimensionality reduction in the pooling layer, multiple convolutional-pooling operations to extract high-level features, outputs the probability distribution in the fully connected layer, and selects the category with the highest probability as the prediction label to obtain the prediction label of each first training sample x m and the prediction label of each second training sample implanted with the trigger and the prediction label of each second training sample as well as the prediction label of each second training sample x that is not implanted j
[0043] (3c) Adopt the cross-entropy loss function. For each benign client c n calculate the loss value L1 of the local model through the prediction label and its corresponding true class label y m For each malicious client c k calculate the backdoor loss value L2 of the local model through the prediction label of the non-injected trigger samples and its corresponding true class label y j and the prediction label of the samples injected with the trigger and its modified label Then, adopt the stochastic gradient descent method to update ω through L1 and L2 v
[0044] ;
[0045] (3d) Judge whether v = V holds. If so, obtain the trained local model with weights ω b Otherwise, set v = v + 1 and execute step (3b).
[0045] In this step, the loss value L1 of the local model and the backdoor loss value L2 of the local model are calculated by the following formulas respectively:
[0046]
[0047] where, ∑ represents the summation operation, and log represents the logarithmic operation.
[0048] Update ω through L1 and L2 v The update formulas are as follows:
[0049]
[0050] where, ω v+1 represents the update result of ω v represents the partial derivative operation, η represents the learning rate, and in this embodiment, η = 0.05.
[0051] An attacker embeds a specific trigger in the training data, causing the model to make incorrect predictions when it sees inputs with the trigger. Since normal samples in the training data account for the vast majority, the model mainly learns the features of these normal samples during training, and the impact of the trigger is almost invisible under normal circumstances. This method exploits the high accuracy of the model for normal samples and its vulnerability to abnormal inputs, making the attack concealed and difficult to detect through conventional tests.
[0052] Step 4) The central server calculates the bias of the local model and detects the backdoor model:
[0053] The central server uses the out-of-distribution sample set D OOD to calculate the bias Γ of each local model b , and regards the O local models where the largest element l in Γ b satisfies l≥ξ as the backdoor models, where 1≥ξ≥0 and O≥0;
[0054] In this step, the central server uses the out-of-distribution sample set D OOD to calculate the bias Γ of each local model b , and the calculation method is as follows:
[0055] The central server takes D OOD as the input of each local model in turn to extract features through the convolutional layer, reduce the dimension through the pooling layer, extract high-level features through multiple convolutional-pooling operations, and output the probability distribution Q through the fully connected layer u b , and calculates the bias of the local model from the U output probability distributions Q u b , and the calculation formula is:
[0056]
[0057] In this embodiment, ξ = 0.8.
[0058] Backdoor samples are usually regarded as out-of-distribution samples. When these samples with backdoors are input into the model for training, they will construct a mapping in the feature space of the model that is significantly different from the distribution of normal training data. Due to this significant difference in mapping from in-distribution data, when the model encounters inputs with backdoor triggers, it often has an extremely high confidence in the backdoor class, while for normal inputs, a benign model will show a more balanced class distribution. Therefore, when out-of-distribution data enters the backdoor model, it can generate an abnormal probability distribution at the output layer, showing a strong bias, while a benign model behaves more naturally and reasonably.
[0059] By analyzing the bias of the model on out-of-distribution data, we can avoid the steps of clustering, filtering, etc. that rely on statistical values in traditional methods, and instead directly utilize the characteristics of the model itself to identify backdoors. In this way, it no longer depends on external training data distribution information, enhancing the flexibility of the detection method and enabling it to better adapt to different application scenarios. This detection method based on the inherent properties of the model can not only be applicable to various data environments, but also reduce the need for manual intervention and improve the automation level of backdoor identification.
[0060] Especially in the non-independent and identically distributed (Non-IID) federated learning environment, due to the large differences in the data distributions of each client, this method can effectively adapt to the data diversity, enhancing the stability and robustness of the detection. Moreover, when calculating the model bias, only the distribution of the model output needs to be analyzed, avoiding the complex calculations in traditional methods, greatly reducing the consumption of computing resources, and at the same time improving the detection efficiency. The high efficiency and adaptability of this method make backdoor detection more operable and universal in practical applications.
[0061] The backdoor model has formed a bias towards the trigger during the training process. This bias not only exists in the training data but may also affect the model's prediction of out-of-distribution data. When the characteristics of the out-of-distribution data are similar to the backdoor characteristics in the training data, the model may misclassify it as the backdoor class, thus showing a relatively high prediction probability. Therefore, the backdoor model will make abnormal predictions for the backdoor class on out-of-distribution data without the trigger.
[0062] Step 5) The central server obtains the backdoor defense result:
[0063] The central server performs anti-learning on the weights of the O backdoor models through the out-of-distribution sample set D OOD to obtain, and aggregates the weights of the O backdoor-removed models after anti-learning and the weights of the H = B - O non-backdoor models to obtain the aggregated weight ω global of the global model without backdoors.
[0064] In this step, the way of anti-learning is:
[0065] The central server uses D OOD as the input of each backdoor model to predict each out-of-distribution sample, and adopts the cross-entropy loss function. Through the predicted out-of-distribution sample label and the index y of the maximum value of the backdoor model bias max calculate the loss value L of the backdoor model backdoor , and then adopt the stochastic gradient ascent method. Through L backdoor update ω z , and the update formula is:
[0066]
[0067] Among them, ω z+1 represents the updated result of ω z , and α represents the learning rate of anti-learning.
[0068] In this embodiment, α = 0.0001.
[0069] In this step, for the O model weights ω o after removing the backdoor and the model weights ω h of H = B - O which are not backdoor models, aggregation is performed, and the aggregation formula is:
[0070]
[0071] Out-of-distribution data and backdoor samples may have similar characteristics. Since backdoor attacks usually change the behavior of the model by introducing specific triggers into the training data, and the characteristics of these triggers may be similar to some patterns in natural data. Although out-of-distribution data does not contain triggers, its characteristics may have a certain overlap with the characteristics of backdoor samples. This enables out-of-distribution data to effectively simulate the characteristics of backdoor samples, thus providing a "reverse" learning signal for the model. By using out-of-distribution data to perform anti-learning on the model, the model can be adjusted on these data, thereby reducing the impact of trigger characteristics on the model. This anti-learning mechanism helps the model gradually forget the wrong patterns related to backdoor samples, and thus effectively removes or alleviates the backdoor effect.
Claims
1. A convolutional neural network federated learning backdoor defense method based on model bias, characterized in that: The steps include: (1) Building a federated learning system: Construct a federated learning system including a central server and B clients consisting of N benign clients and K malicious clients; the central server obtains an out-of-distribution sample set D including U RGB images OOD , and sends the convolutional neural network as a global model to each client c b , where K ≥ 1, N ≥ 10, U ≥ 200; (2) Each client obtains a training sample set: Each client c b The targets in the acquired M RGB images including P target categories are labeled; then each benign client c n M RGB images and their labels form the first training sample set D n train , each malicious client c k The trigger T is implanted into S RGB images and the target label is modified. Then, the S RGB images containing the trigger and their modified labels and the J=MS RGB images without the trigger and their labels form the second training sample set D. k train , where P ≥ 2, M ≥ 200, and the RGB image of the sth implanted trigger T is (3) Each client iteratively trains the local model: Each client c b The global model sent by the central server is used as the local model and iteratively trained through its own training sample set, and the model weights ω of the trained local model are b Upload to the central server; (4) The central server calculates the bias of the local model and detects the backdoor model: The central server distributes the external sample set D OOD Calculate the bias Γ for each local model b , and Γ b The O local models whose largest element l and the preset threshold ξ satisfy l≥ξ are used as backdoor models, where 1≥ξ≥0, O≥0; (5) The central server obtains the backdoor defense results: The central server distributes the external sample set D OOD The weights of the O backdoor models are reversed and learned to obtain, and the weights of the O models after reverse learning without backdoors and the weights of the H = BO non-backdoor model are aggregated to obtain the aggregated weight ω global A global model without backdoors.
2. The method according to claim 1, characterized in that The global model described in step (1) includes a stacked input layer, multiple convolution-pooling layers, and a fully connected layer.
3. The method according to claim 1, characterized in that The sth RGB image of the implanted trigger T in step (2) is Its expression is: Where ⊙ represents the Hadamard product, x s represents the sth RGB image without the trigger T implanted, and δ is the position mask of the trigger.
4. The method according to claim 1, characterized in that: Each client in step (3) iteratively trains the local model, and the implementation steps are as follows: (3a) The number of initial iterations is v, the maximum number of iterations is V, V ≥ 2, and the weight of the local model in the vth iteration is ω v , let v = 1; (3b) Each benign client c n Each malicious client c k D n train , D k train As the input of its own local model, the convolution layer extracts features, the pooling layer reduces dimensions, and multiple convolution-pooling extracts high-level features. The fully connected layer outputs the probability distribution and selects the category with the highest probability as the predicted label to obtain each first training sample x m The predicted label and every second training sample with a trigger implanted Predicted Label And each second training sample x that is not implanted j The predicted label (3c) Using the cross entropy loss function, each benign client c n By predicting the label and its corresponding true class label y m Calculate the loss value L1 of the local model, each malicious client c k Predicted labels for samples without trigger injection and its corresponding true class label y j and the predicted labels of the samples injected with triggers and its modified label Calculate the backdoor loss value L2 of the local model, and then use the stochastic gradient descent method to calculate the loss value L2 of the local model through L1 and L2. v Make updates; (3d) Determine whether v = V. If so, the trained weight is ω. b Otherwise, let v=v+1 and execute step (3b).
5. The method according to claim 4, characterized in that The loss value L1 of the local model described in step (3c) and the backdoor loss value L2 of the local model are calculated by the following formulas: Among them, ∑ represents a summation operation, and log represents a logarithmic operation.
6. The method according to claim 4, characterized in that The step (3c) described in step (3c) is to use L1 and L2 to v To update, the update formulas are: Among them, ω v+1 Represents ω v The updated results, represents the partial derivative operation, and η represents the learning rate.
7. The method according to claim 1, characterized in that The out-of-distribution sample set D described in step (4) OOD Calculate the bias Γ for each local model b , calculated as: The central server will OOD As the input of each local model, the convolution layer extracts features, the pooling layer reduces dimensions, multiple convolution-pooling extracts high-level features, and the fully connected layer outputs the probability distribution Q u b , and the U output probability distributions Q u b Calculate the bias of the local model using the formula:
8. The method according to claim 1, characterized in that: The reverse learning of the weights of the O backdoor models described in step (5) is implemented as follows: The central server will OOD As the input of each backdoor model, each out-of-distribution sample is predicted, and the cross entropy loss function is used to predict the out-of-distribution sample label and the index y of the maximum bias of the backdoor model max Calculate the loss value L of the backdoor model backdoor , and then use the stochastic gradient ascent method to pass L backdoor Yes z Update, the update formula is: Among them, ω z+1 Represents ω z The update result of , α represents the learning rate of anti-learning.
9. The method according to claim 1, characterized in that: The weights ω of the model after removing the backdoor described in step (5) are o And H = BO is not the model weight ω of the backdoor model h Aggregation is performed, and the aggregation formula is:
Citation Information
Patent Citations
Federal learning backdoor attack-oriented defense method
CN118036770A
