Double-stage data-independent backdoor defense method in federal learning

Abstract

The invention discloses a dual-stage data-independent backdoor defense method and system in federal learning and a server, and belongs to the technical field of artificial intelligence security. In order to solve the problems that an existing defense method depends on auxiliary data, is poor in performance in a non-IID environment and cannot effectively defend early attacks, the method comprises the steps that in the training stage, a server can recognize and filter malicious models without accessing client data by analyzing statistical data of a batch normalization layer in a model, and the malicious models can be effectively defended; only aggregating a feature extractor of the benign model, and storing a classifier of the feature extractor; in the reasoning stage, features of input samples are sent into all stored benign classifiers in parallel, multiple outputs are aggregated to suppress potential backdoors, and final prediction is generated. According to the invention, backdoor attacks can be effectively resisted, and the security of the federated learning model is improved.

Description

Technical Field

[0001] This invention relates to the field of computer technology, and more specifically, to a model security defense method in distributed machine learning, particularly federated learning (FL) environments, and especially to a backdoor attack defense method and system that does not require access to local client data. Background Technology

[0002] Federated learning, as an emerging distributed privacy-preserving machine learning framework, allows multiple clients (such as mobile phones and hospitals) to collaboratively train a powerful global model without exposing their local raw data. However, because the server cannot directly access and verify the client's local data and training process, federated learning systems are inherently vulnerable to malicious attacks, with backdoor attacks being a serious threat.

[0003] In backdoor attacks, malicious clients inject poisoned samples with specific triggers into their local data and relabel them with the target label, thereby training a local model with a backdoor. When these malicious models are aggregated into a global model by the server, the global model is then implanted with the backdoor. During inference, once a sample with the same trigger is input, the model will misclassify it with extremely high confidence as the attacker's preset target label, while the classification of normal samples remains unaffected. This makes backdoor attacks highly stealthy and extremely harmful.

[0004] Existing backdoor defense methods can be mainly divided into several categories:

[0005] 1. Robust aggregation methods: These methods filter or reduce the weight of malicious models by detecting anomalies in model updates. They typically rely on distance or similarity metrics between model parameters. However, in scenarios where client data is not independently and identically distributed (non-IID), the parameter differences between benign and malicious models may be insignificant, leading to high detection failure rates.

[0006] 2. Model Refinement Methods: These methods purify the aggregated global model on the server side through fine-tuning or knowledge distillation. The main limitation of these methods is that they usually require an additional, clean auxiliary dataset, which is difficult to obtain in many real-world scenarios, thus violating the data-insensitive principle of federated learning.

[0007] 3. Robustness of authentication methods: These methods provide theoretical security guarantees, but usually require strong assumptions, such as the support of a Trusted Execution Environment (TEE), or require a large amount of additional verification overhead, thus limiting their practicality.

[0008] Furthermore, a critical issue that is often overlooked is that backdoor tasks can be learned very rapidly in the initial stages of model training. Most existing defense mechanisms focus on anomaly detection in the later stages of training, and are insufficient to defend against backdoor attacks launched early in the training process.

[0009] In summary, existing technologies face a dilemma: they either violate the original intent of federated learning by relying on auxiliary data, perform poorly in real-world non-IID data environments, or are difficult to deploy in practice due to overly strong assumptions, and generally lack effective defense against early-stage attacks. Therefore, there is an urgent need for a new defense technology that requires no auxiliary data, remains robust in non-IID environments, and can effectively resist backdoor attacks from the initial training phase. Summary of the Invention

[0010] The present invention aims to address the shortcomings of the aforementioned background technology and provide a backdoor defense method, system, and server in federated learning. Its purpose is to achieve a highly efficient defense mechanism that is data-independent, robust to non-IID data distributions, and effectively defends against backdoor attacks in the early stages of training.

[0011] This invention provides a method for backdoor defense in a federated learning system consisting of a central server and multiple clients, the method comprising the following steps:

[0012] During the training phase:

[0013] 1. The central server receives local models from multiple clients and divides each local model into a feature extractor and a classifier.

[0014] 2. The central server performs a model checking operation: it analyzes all received feature extractors to identify a set of benign clients. This analysis is based on the neuron activation statistics within each feature extractor, without accessing any client's local training data.

[0015] 3. The central server aggregates the corresponding feature extractors based on the set of benign clients to generate a global feature extractor and stores all classifiers corresponding to the set of benign clients.

[0016] During the reasoning stage:

[0017] 4. The central server receives an input sample to be predicted and processes the input sample using the global feature extractor to generate a feature vector.

[0018] 5. The central server inputs the feature vector into all the stored classifiers to obtain multiple parallel classification results.

[0019] 6. The central server performs an inference suppression operation: aggregating the multiple classification results to generate a final, robust prediction result, thereby neutralizing the potential backdoor influence.

[0020] In a preferred embodiment, the model checking operation utilizes statistical parameters of the batch normalization (BN) layers in the feature extractors. Specifically, the central server extracts the mean and variance statistics of all BN layers in each feature extractor and uses these statistics as proxies for the neuron activation distribution.

[0021] In a preferred embodiment, the model checking operation identifies anomalies by calculating the differences in the statistical distributions of the BN layers among different feature extractors. Specifically, the central server uses Kullback-Leibler divergence (KLD) to measure the differences in the statistical distributions of the corresponding BN layers between any two feature extractors. The total divergence between two feature extractors is the sum of the divergences of all their corresponding BN layers.

[0022] In a further preferred embodiment, the step of identifying benign clients employs a voting-based mechanism. Specifically, for each client, the server identifies the k other clients with the smallest KLD value and votes for these k clients. Finally, the server counts the votes for all clients and identifies the client with the highest number of votes as a benign client.

[0023] In a preferred embodiment, the inference suppression operation includes: first, applying a Softmax function to the output of each classifier to convert it into a probability distribution; then, for each category, selecting the maximum probability value of that category from the probability distributions of all classifiers to form a final probability distribution; and finally, selecting the category with the highest probability value in the final probability distribution as the final prediction result.

[0024] The present invention also provides a server for defending against backdoors in federated learning, the server comprising: a processor, a memory, and a network interface. The memory stores a computer program, which, when executed by the processor, implements any of the methods described above.

[0025] Specifically, the server may include:

[0026] A model receiving and partitioning module is used to receive client models and partition them into feature extractors and classifiers.

[0027] A model checking module is used to identify benign clients by analyzing neuron activation statistics from a feature extractor.

[0028] An extractor aggregation module is used to aggregate benign feature extractors to generate a global feature extractor.

[0029] A classifier storage module for storing classifiers for benign clients.

[0030] An inference suppression module is used to aggregate the outputs of multiple classifiers during the inference phase to generate the final prediction result.

[0031] The present invention also provides a federated learning system, including the aforementioned server and multiple client devices. Attached Figure Description

[0032] Figure 1 This is a schematic diagram of a federated learning system architecture according to an embodiment of the present invention.

[0033] Figure 2 This is a diagram of a two-stage backdoor defense system architecture deployed on the server side according to an embodiment of the present invention.

[0034] Figure 3 This is a schematic diagram of the working mechanism of an inference suppressor according to an embodiment of the present invention. Detailed Implementation Plan

[0035] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0036] Example 1: System and Architecture

[0037] Reference Figure 1 The federated learning system used in this invention comprises a central aggregation server 101 and multiple client devices 102 (e.g., Client 1, Client 2, ..., Client n). Server 101 can be a cloud server or server cluster with powerful computing and storage capabilities. Clients 102 can be any device with local datasets and computing capabilities, such as smartphones, personal computers, IoT devices, or servers in medical institutions. Among these clients, there may be benign clients 102a and malicious clients 102b.

[0038] Server 101 is responsible for coordinating the entire federated learning training process, including distributing the global model, collecting local models uploaded by clients, performing model aggregation, and implementing the backdoor defense mechanism proposed in this invention. Client 102 trains the model using its local data and only uploads the model parameters to server 101.

[0039] like Figure 2 As shown, the core defense mechanism of this invention is deployed on server 101. When a client uploads a local model, the server first divides each model (e.g., a convolutional neural network CNN) into a feature extractor and a classifier. The feature extractor typically contains the first few layers of the model (such as convolutional and pooling layers) and is responsible for extracting high-level features from the input data; the classifier typically contains the last few layers of the model (such as fully connected layers) and is responsible for classifying based on the extracted features.

[0040] The defense architecture mainly consists of three core components: Model Inspector 201, Extractor Aggregator 202, and Inference Inhibitor 203. These components work together to form the defense for both the training and inference phases.

[0041] Example 2: Detailed Method Implementation

[0042] Reference Figure 3 The detailed steps of the two-stage backdoor defense method of the present invention are as follows:

[0043] Training Stage

[0044] Step S301: Model reception and partitioning

[0045] At the start of each communication round, server 101 from Each client receives its locally trained model. The server then processed each model. The structure is divided into feature extractors. Classifier .

[0046] Step S302: Model Check

[0047] This is the core of the first phase of defense. Model inspector 201 checks all received feature extractors. A review process was conducted to identify and filter out backdoor models uploaded by the malicious client 102b. This process was completely data-agnostic; its key lay in leveraging the statistical properties within the model.

[0048] The basic principle is that the activation distribution of neurons in a model trained on a mixture of benign data and a small amount of poisoned data will exhibit a subtle but measurable deviation from the activation distribution of a model trained only on benign data. This invention cleverly utilizes batch normalization (BN) layers, common in deep learning models, to capture this deviation. During training, the BN layer calculates and updates the mean of each feature channel. and variance The running average is the average value. These statistics can be viewed as a compact, privacy-preserving "fingerprint" of the data distribution that the model "sees" during training. When a malicious client trains with poisoned data, trigger patterns can cause aberrant activation of specific neurons, which will be reflected in the mean and variance statistics of the final BN layer.

[0049] The specific implementation of model checking is as follows:

[0050] 1. BN layer statistics extraction: For each feature extractor The server extracts the running average of all its BN layers. and operating variance ,in Indicates the first One BN layer.

[0051] 2. Divergence Calculation: The server calculates the divergence between any two feature extractors. and The difference between them. This embodiment uses Kullback-Leibler divergence (KLD) to measure the difference in their BN layer statistical distributions. Assuming that neuron activation values ​​follow a Gaussian distribution, with their mean and variance given by the BN layer parameters, then the two distributions... and The KLD between them has a closed-form solution:

[0052]

[0053] Two feature extractors and The total divergence between them is the sum of the divergences of all L corresponding BN layers:

[0054]

[0055] in Representative model The Statistical distribution of each BN layer.

[0056] 3. Benign Client Identification: Based on the calculated pairwise KLD values, the server employs a two-layer voting mechanism to identify the set G of benign clients. According to common threat models of federated learning backdoor attacks, it is assumed that the number of malicious clients is less than half of the total. Based on this assumption, for each client The server selects the smallest KLD value among its own and all other clients' KLD values. A value, and consider this A corresponding client and client "Similar". The server provides this. Each client casts one vote. After iterating through all clients, the server counts the total number of votes for each client. Clients that receive more votes are considered benign clients and are added to the benign set G.

[0057] Step S303: Global Model Aggregation and Distribution

[0058] After model checker 201 outputs a set of benign clients G, extractor aggregator 202 discards all models from non-benign clients. Then, it extracts features only from the benign set G. Perform aggregation to generate a new global feature extractor. Meanwhile, all benign classifiers The updated global feature extractor is stored by the server for use during the inference phase. Finally, the server updates the global feature extractor. The broadcast is sent back to all clients to begin the next round of local training.

[0059] Inference Stage

[0060] When the system needs to make predictions on new input samples, a second-stage defense mechanism is triggered. This stage aims to neutralize any potential backdoors that might have escaped model checks during the training phase.

[0061] Step S304: Feature Extraction

[0062] The server receives an input sample to be predicted. First, the sample It is fed into a global feature extractor obtained through multiple rounds of robust aggregation. The high-dimensional feature representation vector is calculated. .

[0063] Step S305: Parallel Classification

[0064] The server will use the feature vector The data is fed in parallel into all the benign classifiers stored during the training phase. In the middle. Because these classifiers come from different benign clients, their decision boundaries differ, especially in non-IID data environments. Therefore, this step will yield... There are 10 different classification results, each of which is a probability distribution vector. .

[0065] Step S306: Inference Suppression and Aggregation

[0066] Reference Figure 3 Inference suppressor 203 for this The parallel classification results are aggregated to generate a final, more reliable prediction. This process effectively leverages the idea of ​​ensemble learning to enhance robustness. If a classifier contains an undetected backdoor, it might output an abnormally high probability for the target class when the input includes a trigger. However, the vast majority of other benign classifiers are unaffected and will predict the true class with a higher probability. The aggregation mechanism of the inference suppressor takes advantage of this.

[0067] The specific aggregation process is as follows:

[0068] 1. The output of each classifier Apply the Softmax function to transform it into a standardized probability distribution. .

[0069] 2. For each of the K categories From all From the probability distributions, select the category The maximum probability value.

[0070]

[0071] 3. This results in a final aggregate probability distribution. .

[0072] 4. Final prediction results yes The category corresponding to the category with the highest probability value.

[0073]

[0074] In this way, the weight of high-confidence erroneous predictions generated by a single backdoor classifier is effectively suppressed, while the consistent high-confidence predictions of multiple benign classifiers for the correct category are highlighted, thus neutralizing the influence of the backdoor and achieving defense during the inference stage.

[0075] The above description is merely a preferred embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A two-stage data-agnostic backdoor defense method in federated learning, characterized in that, The method comprises the following steps of: In the training phase: a) the central server receives local models trained by the plurality of clients respectively, and divides each local model into a feature extractor and a classifier; b) the central server performs a model checking operation based on neuron activation statistics inside each feature extractor to identify a set of benign clients, wherein the model checking operation does not need to access the local training data of any client; c) the central server aggregates the feature extractors corresponding to the set of benign clients to generate a global feature extractor, and stores all the classifiers corresponding to the set of benign clients; in the inference phase: d) the central server receives an input sample, and processes the input sample using the global feature extractor to generate a feature vector; e) the central server inputs the feature vector into all the stored classifiers to obtain a plurality of classification results; f) the central server performs an inference suppression operation to aggregate the plurality of classification results to generate a final prediction result. The neuron activation statistics are represented by statistical parameters of one or more Batch Normalization (BN) layers in the feature extractor.

2. The method of claim 1, wherein, The statistical parameters of the BN layer include running mean and running variance recorded by each BN layer.

3. The method of claim 2, wherein, The model checking operation comprises calculating the difference between the distributions represented by the statistical parameters of the corresponding BN layers between any two feature extractors.

4. The method of claim 3, wherein, The difference measure is Kullback-Leibler divergence (KLD).

5. The method of claim 4, wherein, The step of identifying a set of benign clients comprises, for each client, determining whether it is a benign client through a voting mechanism based on the difference measure between its feature extractor and the feature extractors of all other clients.

6. The method according to claim 4 or 5, characterized in that, The voting mechanism comprises, for each client, identifying a preset number k of other clients with the smallest difference measure, and adding one vote for each of the k clients, counting the final vote number of all clients, and identifying the clients with a vote number exceeding a preset threshold as benign clients.

7. The method of claim 6, wherein, The inference suppression operation comprises converting the output of each classifier into a probability distribution, and for each class, selecting the maximum probability value of the class from the probability distributions of all classifiers to combine into a final aggregated probability distribution, and selecting the class with the maximum probability value in the final aggregated probability distribution as the final prediction result.

8. The method of claim 1, wherein, The method comprises the following steps of:

9. A server for federated learning backdoor defense, comprising: a model receiving and dividing module for receiving local models from a plurality of clients, and dividing each local model into a feature extractor and a classifier; a model checking module for identifying a set of benign clients based on neuron activation statistics inside each feature extractor, and the process does not need to access the local training data of the clients; ​ an extractor aggregation module configured to aggregate feature extractors corresponding to the set of benign clients to generate a global feature extractor; a classifier storage module configured to store all classifiers corresponding to the set of benign clients; and an inference suppression module configured to, when inferring, process an input sample using the global feature extractor to obtain a feature vector, input the feature vector into all stored classifiers to obtain a plurality of classification results, and aggregate the plurality of classification results to generate a final prediction result.

10. The server of claim 9, wherein, The model checking module identifies the benign clients by analyzing statistical parameters of one or more batch normalization (BN) layers in the feature extractors, the statistical parameters including running mean and running variance, and uses Kullback-Leibler divergence (KLD) as a measure of distribution difference.

11. A federated learning system, comprising: The server of claim 9 or 10 and a plurality of client devices.

12. A computer readable storage medium having stored thereon a computer program, which, when executed by a processor, implements the method of any one of claims 1 to 8.

Images