Backdoor attack defense method based on interpretable feature map and symbol fine-grained information

Abstract

The invention discloses a backdoor attack defense method based on an interpretable feature map and symbol fine-grained information, and belongs to the field of artificial intelligence and machine learning. Two detection modules are included for interpretable feature detection (IFD) and dynamic main parameter difference detection (DMPD). The IFD captures the global difference of gradient information by means of an interpretable technology, the IFD does not lose the defense effect along with model iteration, and a subset can be obtained after IFD screening. Because the IFD only captures the global difference of gradient information and ignores the importance of local gradient information, the IFD may be crossed. Gradient symbol information is used in the DMPD, the symbol information can conduct more detailed evaluation on a model submitted by a client and is not affected by iteration, and a subset is obtained after DMPD screening. And finally, taking an intersection of two subsets obtained by the two modules to obtain a final set, and carrying out norm clipping on gradient updating on the final set to aggregate a new global model. The method has excellent defense capability for backdoor attacks under federated learning.

Description

Technical Field

[0001] This invention belongs to the field of artificial intelligence and machine learning, and in particular relates to defending against backdoor attacks launched by attackers under a federated learning framework. Background Technology

[0002] In today's era, technologies such as big data and cloud computing are booming, and artificial intelligence technologies such as machine learning and deep learning have been widely adopted across various industries. Currently, Large Language Modeling (LLM) is experiencing rapid development. Since OpenAI launched ChatGPT in 2022, LLM has seen a surge in popularity. LLM possesses broad application capabilities, such as knowledge-based question answering and text generation. From pre-training to fine-tuning, LLM relies heavily on massive amounts of data, demonstrating the rich value contained within this data, which is crucial for driving the development of artificial intelligence. However, the surge in data volume has also brought some problems. First, the massive amount of data increases communication costs when users upload data, and it's difficult to find a central server capable of storing such a large amount of data. Second, third-party platforms cannot guarantee data privacy, increasing the risk of data leakage and discouraging data holders from sharing their data. Finally, the data held by data holders may contain sensitive information that cannot be disclosed. Model training is inseparable from data, but data holders are unwilling or unable to disclose this data, resulting in increasingly confidential data and creating the problem of data silos. Data silos and security issues have become major bottlenecks affecting the development of artificial intelligence.

[0003] To address the issues of data silos and security, McMahan et al. first proposed federated learning, a novel distributed machine learning paradigm that allows multiple clients to collaboratively train a model through multiple iterations while protecting the data privacy of each client. In each iteration, each client trains a local model using local data and uploads its gradient updates to the server. The server receives all gradient updates, aggregates them into a new global model, and then distributes the new global model to the clients for a new round of iterations. Due to its ability to protect data privacy, federated learning has become a fundamental framework for building machine learning models in fields such as transportation, healthcare, the Internet of Things, and word prediction.

[0004] However, while federated learning can protect client data privacy, it is highly vulnerable to attacks, especially backdoor attacks, due to the server's unknowability of the gradient updates submitted by the client. Attackers manipulate malicious clients to embed backdoor triggers in their local data, thereby injecting malicious updates into the global model. The corrupted global model will identify samples with embedded backdoor triggers as backdoor labels set by the attacker, while behaving normally on other samples. Compared to non-targeted attacks that only aim to reduce the performance of the global model, backdoor attacks are much more difficult to defend against because the difference between the malicious model and the benign model is very small. Attackers can even manipulate the parameters of the local model, further increasing the difficulty of defense. Therefore, identifying and defending against backdoor attacks is particularly important.

[0005] Currently, academia and industry have proposed various defense methods against backdoor attacks on federated learning. These methods utilize different information from gradient updates to detect and defend against backdoor attacks, and can be broadly categorized as follows:

[0006] Scoring-based backdoor attack defense methods utilize specific metrics (such as cosine similarity, gradient norm, variance, and sign information) to assign different weights to each client in order to reduce the impact of malicious gradients on the model. These methods can achieve good results on simple backdoor attacks because malicious gradients and benign gradients can be easily distinguished. However, these defense methods are not effective against attacks that are carefully designed by attackers, who can generate malicious gradients that are very similar to benign gradients.

[0007] Differential privacy-based backdoor attack defense methods weaken the connection between model parameters and backdoor triggers by adding Gaussian noise during the training or aggregation phase. Such methods can be effective even against more covert backdoor attacks. However, adding noise to the model will reduce its performance, and the magnitude of the noise is difficult to control, which can easily cause the model training to crash.

[0008] Backdoor attack defense methods based on elimination further mine gradient information, such as Euclidean distance, Manhattan distance, cosine similarity, gradient sign, subspace, etc., and use this information to filter malicious clients or malicious parameters. These methods can defend against some complex attacks, but they can still be bypassed by well-designed attacks.

[0009] Current defense methods still have limitations; therefore, exploring and developing new federated learning methods for backdoor attack defense is of great significance. To explore more robust backdoor attack defense methods while utilizing multiple pieces of information from gradient updates, this invention designs a novel and effective backdoor attack defense method, FLID, based on interpretable feature maps and fine-grained symbolic information. Summary of the Invention

[0010] This invention proposes a backdoor attack defense method, FLID, based on interpretable feature maps and fine-grained symbolic information. FLID is a defense method based on a dual detection mechanism, comprising two detection modules: Interpretable Feature Detection (IFD) and Dynamic Master Parameter Difference Detection (DMPD). IFD leverages interpretability to capture the global variability of gradient information. Since interpretability reflects the importance of each pixel in the input image, and backdoor attacks are strongly correlated with backdoor features, this results in low similarity between the interpretable saliency maps generated by the malicious model and those generated by the global model. Furthermore, IFD does not rely on defense methods based on traditional metrics and does not lose its defensive effectiveness with model iteration. A subset can be obtained after IFD filtering. In strong backdoor attacks, as model parameters increase, the proportion of malicious parameters from the attacker becomes extremely small. Since IFD only captures the global differences in gradient information and ignores the importance of local gradient information, IFD may be bypassed. To capture the local differences in gradient information, this invention introduces the DMPD module as a second layer of detection for FLID. Unlike IFD, DMPD uses the sign information of the gradient. Sign information allows for a more detailed evaluation of the model submitted by the client and is not affected by iteration compared to traditional metrics. Because backdoor tasks cause differences in the update direction between malicious and benign models, a subset can be obtained after DMPD filtering. Finally, the intersection of the two subsets obtained from the two modules yields the final set. ,right The gradient updates are pruned according to norm and then aggregated to form a new global model. To verify the effectiveness of the proposed method, FLID's defense capabilities against five attacks—Badnet with Constraint-and-Scale, DBA Attack, PGDAttack, Neurotoxin Attack, and BCL (Backdoor critical layer)—were tested. FLID can reduce the attack success rate to below 4% in all cases. Experimental results demonstrate that FLID possesses excellent defense capabilities. Attached Figure Description

[0011] Figure 1 Backdoor attack defense methods based on interpretable feature maps and fine-grained symbol information

[0012] Figure 2 Interpretable saliency maps of malicious and benign models on backdoor and clean labels, respectively, for clean and noisy samples.

[0013] Figure 3Differences in the interpretable saliency map distributions of malicious and benign models on normal and backdoor labels, respectively. Detailed Implementation

[0014] This invention proposes a backdoor attack defense method FLID based on interpretable feature maps and symbolic fine-grained information. This method utilizes multiple information from gradient updates to defend against more threatening backdoor attacks without infringing on data privacy, while ensuring that attackers find it difficult to carry out elaborate attacks.

[0015] like Figure 1 As shown, the FLID framework mainly consists of three parts: Interpretable Feature Detection (IFD), Dynamic Master Parameter Difference Detection (DMPD), and Norm Clipping.

[0016] Step 1: Interpretable Feature Detection (IFD)

[0017] Interpretable techniques can help understand how neural networks make decisions and how input data affects the output. In computer vision, backpropagation of an input image yields a saliency map, which reflects how each pixel in the image influences the output. Therefore, the saliency map can be used to assess the importance of image features. For a given image... and tags The output of the model can be represented as In convolutional neural networks It is about It is a nonlinear function, and therefore can be approximated by a linear function using a first-order Taylor expansion:

[0018]

[0019] Therefore, the gradient with respect to the input image on the output label can be calculated:

[0020]

[0021] Formula (2) describes the function Regarding input gradient Therefore, we can obtain and Having the same size is key to obtaining a saliency map. Representation function In the label The output, Indicates information about the original image The partial derivatives of . The above formula reflects the partial derivatives of the output category. Original image How each pixel affects the output.

[0022] In federated learning scenarios, the existence of non-independent and identically distributed data leads to differences in optimization objectives among different clients. This results in traditional metrics (such as cosine similarity and Euclidean distance) calculated based on gradient updates showing very little difference among different clients. Methods using these metrics will fail to properly filter out malicious clients and are easily bypassed by targeted attacks, such as PGD Attack, Neurotoxin Attack, and BCL Attack.

[0023] Explainable techniques reflect the importance of each pixel in the input image. For a backdoor attack to succeed, the model must learn backdoor features; otherwise, the model will not be able to identify triggers as backdoor labels. Therefore, explainable techniques can be used to determine whether a backdoor has been implanted in the current model. In a normal model's decision-making process, the output label is strongly correlated with the features of the input image. Therefore, if the model classifies based on irrelevant features, it indicates that the current model may be submitted by a malicious client. To achieve this goal, without infringing on client data privacy, an interpretable saliency map is generated for each category using uniformly noisy data. This map is then compared with the saliency map generated by the global model to obtain a filtered subset.

[0024] To improve the accuracy of the generated interpretable saliency maps, this invention improves the generation method by using Guided Backpropagation (GBP) instead of traditional Backpropagation (BP). Guided Backpropagation uses the ReLU function to propagate only positive gradient values. This approach eliminates noise and avoids displaying features that negatively impact the output class. Furthermore, a linear layer is used as the output instead of a softmax layer, as softmax layers reduce the influence of non-output labels, which is detrimental to generating interpretable saliency maps. Backdoor features are only revealed in the saliency maps generated by the malicious model on the backdoor labels; GBP can highlight these regions.

[0025] To highlight the role of interpretable saliency maps, saliency maps calculated by the malicious model (benign model) on uniformly noisy samples (clean samples) based on normal labels (backdoor labels) were obtained, as follows: Figure 2As shown, the backdoor model, based on the backdoor label, calculates a saliency map that shows significant focus on the top left corner of the input image on both uniformly noisy and clean samples, as this is the location of the backdoor trigger. The backdoor features are even more prominent on uniformly noisy samples, where the malicious model ignores the normal features of the original image. On samples with normal labels, the saliency map generated by the backdoor model focuses on the main feature regions of the original image, similar to the focus areas of the normal model. Leveraging these characteristics of interpretable saliency maps and protecting client data privacy, interpretable saliency maps for different models are calculated using uniformly noisy data.

[0026] For a containing The task of classifying tags, server It will generate A uniform noise sample, where each label contains One sample. Because the server cannot see the backdoor label for the malicious client, but the interpretable saliency map of the malicious model on normal labels differs little from that of the benign model (e.g., ...). Figure 3 As shown, the left image represents the difference on the normal label, and the right image represents the difference on the backdoor label (where the vertical axis represents the density of pixel values). Therefore, the impact of uniformly distributing the same number of samples to each label on the screening results can be ignored. It is assumed that the server randomly selects [the appropriate data] in each round of training. Several clients participated in the training, including A malicious client, One benign client, among which The saliency graph representation for each client. ,in Indicates the first One client, Indicates the first Each sample. In each round of training, the server will provide the global model. The data is distributed to each client. This invention assumes that the global model is initially uncontaminated. Therefore, the cosine similarity between the saliency map generated by each client and the saliency map generated by the global model can be calculated to determine whether the current model is contaminated. The formula is defined as follows:

[0027]

[0028]

[0029]

[0030] in Indicates the first The saliency map calculated by each client on all noise samples. Indicates server The saliency map calculated on all noise samples. Indicates the first A client's salience graph and the server Cosine similarity of saliency maps, followed by thresholding. Filter out subsets ,in .

[0031]

[0032] By comparing the similarity between the interpretable saliency maps of each client and the interpretable saliency map of the global model, malicious clients with low relevance can be filtered out. The filtering criterion is that backdoor attacks launched by malicious clients will reduce the relevance of the saliency maps. In this way, the global differences in gradient information can be captured because the corresponding local model is aggregated by directly using the gradient updates submitted by the clients. Moreover, unlike the magnitude-based defense method IFD, it does not become ineffective with training iterations because attackers cannot abandon the learned backdoor features.

[0033] Step 2: Dynamic Master Parameter Difference Detection (DMPD)

[0034] IFD, which relies solely on global gradient information to screen for malicious clients, is easily bypassed by sophisticated attack strategies. For example, in strong backdoor attacks, as model parameters increase, the attacker's malicious parameters account for a very small portion. Capturing only global gradient information often overlooks the important role of local gradient information in detecting malicious clients. However, the sign direction can provide a more detailed evaluation of the model submitted by the client. Especially when the model is under attack, the sign direction of malicious updates will differ from that of benign updates, even if their traditional metrics are similar. Therefore, this invention considers using the sign direction of gradient updates as the second stage for screening malicious models.

[0035] Therefore, this invention proposes Dynamic Master Parameter Difference Detection (DMPD) to examine the differences in gradient update signs among various clients when the core parameters of the global model are uniformly positioned. In the federated learning training process, the global model changes dynamically in each round; therefore, in the t-th round of training, a temporary global model is first aggregated using FedAvg. Since FedAvg is a defenseless method, if a malicious client exists... It will be contaminated. In order to obtain... The core parameter location is obtained by backpropagation of noise data generated by the IFD module. Backpropagation is not for training, but only to find the gradient with the largest change. One position, due to Contaminated This location contains both parameters that function in the backdoor task and normal parameters without infringing on data privacy; at this point, it has been obtained. Main parameter location Subsequent calculations of differences between clients will revolve around Expand.

[0036] Definition 1 ( For a vector , Represents the real number field, indicating It is by A vector consisting of n real numbers, where It is gradient update The flattened vector is defined as follows: Represents a vector Each in the set Take the union of the positions in the set, where , Yes Sort and take the first The set after each position, where Representing this Each position corresponds to a vector The subscript in, where express Cannot exceed the vector length Not less than 1 express It is in ascending order. .

[0037] This represents the current global model. The parameter region of interest, in each round It is dynamic, which aligns with the non-independent and identically distributed nature of federated learning, and at the same time... The standardized parameter locations across clients make it easier to filter out malicious clients. To quantify the differences between different clients... To address the differences in signatures, this invention introduces the Master Parameter Sign Difference Value (MPSV). Prior to this, to better extract the signatures of each client... The symbols on, with the help of Get the mask .

[0038]

[0039]

[0040] in Is The mask generated based on this, if the m-th position is in If it is true, it is 1; otherwise, it is 0. Depend on 1 and It consists of 0s; This is the gradient update for the i-th client in the t-th round after straightening; The function that retrieves the sign of its parameters takes values ​​of -1, 0, and 1. This is an XOR function; it returns 1 if the two parameters have different signs, and 0 otherwise. It is a Hadama pile; Used to calculate the distance between two clients The number of parameters with different signs; This represents the sign difference of the primary parameter between the i-th client and the j-th client.

[0041] To obtain the total difference between each client and other clients, the following definition is used:

[0042]

[0043] in This represents the total sign difference between client i and other clients. The final result is... ,express Depend on It consists of P real numbers, because each round of training involves P clients. It contains P real numbers. DMPD can effectively distinguish between malicious and benign clients by calculating MPSV through the master parameter symbol.

[0044] After obtaining the MPSV, anomaly filtering is required. This invention uses a median-based Z-score (MZ-score) for filtering. Compared to the traditional Z-score, the median-based MZ-score is better at handling non-normally distributed data and accurately removing outliers, reducing the impact of extreme values.

[0045] Definition 2 ( For a set Assuming that For set the median of For set The standard deviation, for any Its MZ_score can be obtained as follows: .

[0046] DMPD with Process MPSV and set a threshold. Delete clients exceeding the threshold to obtain the set. ,in The malicious client's calculated MPSV is smaller due to the consistency of the backdoor task. Anomalies will occur. In this way, the present invention can capture local differences in gradient information.

[0047] Step 3: Norm Clipping

[0048] Two sets can be obtained through IFD and DMPD. and Subsequently, the present invention obtains the intersection of the two sets. This set represents the selected benign clients. To prevent the selected clients from implementing ScaleAttack, since ScaleAttack does not affect the detection of IFD and DMPD, we introduce norm pruning. Specifically, we calculate the median of the L2-norm of the selected gradient updates, use this median as a threshold to rescale these gradient updates, and finally aggregate the pruned gradient updates into the global model.

Claims

1. A backdoor attack defense method FLID based on interpretable feature maps and symbolic fine-grained information, characterized in that: Step 1: Interpretable Feature Detection (IFD) For a containing The task of classifying tags, server It will generate A uniform noise sample, where each label contains Noisy samples; since the server cannot see the backdoor label of the malicious client, but the interpretable saliency map of the malicious model on the normal label is not much different from that of the benign model, the impact of uniformly distributing the same number of samples to each label on the screening results can be ignored; it is assumed that the server randomly selects samples in each round of training. Several clients participated in the training, including A malicious client, One benign client, among which ;Saliency graph representation for each client ,in Indicates the first One client, Indicates the first Each sample; in each round of training, the server will generate the global model. The data is distributed to each client. Assuming the global model is initially uncontaminated, the cosine similarity between the saliency map generated by each client and the saliency map generated by the global model can be calculated to determine whether the current model is contaminated. The formula is defined as follows: ; ; in Indicates the first The saliency map calculated by each client on all noise samples contains One data point, Indicates server The saliency map calculated on all noise samples contains One data point, for, Indicates the first A client's salience graph and the server Cosine similarity of saliency maps, followed by thresholding. Filter out subsets ,in ; Step 2: Dynamic Master Parameter Difference Detection (DMPD) IFD, which only uses global gradient information to screen for malicious clients, is easily bypassed by well-designed attack strategies. For example, in strong backdoor attacks, as the model parameters continue to increase, the malicious parameters of the attacker account for a very small part. Only capturing global gradient information often ignores the important role of local gradient information in detecting malicious clients. The sign direction can provide a more detailed evaluation of the model submitted by the client. In particular, when the model is attacked, the sign direction of malicious updates will be different from that of benign updates, even if their traditional indicators are small. Therefore, we consider using the sign direction of gradient updates as the second stage for screening malicious models. Therefore, a Dynamic Master Parameter Difference Detection (DMPD) is proposed to examine the differences in gradient update signs among various clients when the core parameters of the global model are uniformly positioned. Since the global model is dynamically changing in each round of federated learning training, a temporary global model is first aggregated using FedAvg in the t-th round of training. Since FedAvg is a defenseless method, if a malicious client exists... It will be contaminated; in order to obtain The core parameter locations are determined by using noise data generated by the IFD module to temporarily modify the global model. Gradient updates are obtained by backpropagation. Backpropagation is not for training, but only to find the gradient with the largest change. One position, due to Contaminated This location contains both parameters that function in the backdoor task and normal parameters without infringing on data privacy; at this point, it has been obtained. Main parameter location Subsequent calculations of differences between clients will revolve around Expand; Definition 1 ( For a vector , Represents the real number field, indicating It is by A vector consisting of n real numbers, where It is gradient update The flattened vector is defined as follows: Represents a vector Each in the set Take the union of the positions in the set, where , Yes Sort and take the first The set after each position, where Representing this Each position corresponds to a vector The subscript in, where express Cannot exceed the vector length Not less than 1 express It is in ascending order. ; This represents the current global model. The parameter region of interest, in each round It is dynamic, which aligns with the non-independent and identically distributed nature of federated learning, and at the same time... The standardized parameter locations across clients make it easier to filter out malicious clients; to quantify the differences between different clients... To address the differences in the parameters, the Master Parameter Signed Difference Value (MPSV) was introduced; prior to this, in order to better extract the differences for each client... The symbols on, with the help of Get the mask ; ; ;in Is The mask generated based on this, if the m-th position is in If it is true, it is 1; otherwise, it is 0. Depend on 1 and It consists of 0s; This is the gradient update for the i-th client in the t-th round after straightening; The function that retrieves the sign of its parameters takes values ​​of -1, 0, and 1. This is an XOR function; it returns 1 if the two parameters have different signs, and 0 otherwise. It is a Hadama pile; Used to calculate the distance between two clients The number of parameters with different signs; This represents the sign difference of the primary parameter between the i-th client and the j-th client; To obtain the total difference between each client and other clients, the following definition is used: ;in This represents the total sign difference between client i and other clients; ultimately, we obtain... ,express Depend on It consists of P real numbers, because each round of training involves P clients. It contains P real numbers; DMPD can effectively distinguish between malicious and benign clients by calculating MPSV through the master parameter symbol. After obtaining the MPSV, it is necessary to perform anomaly filtering using median-based Z_score (MZ_score). MZ_score leverages the median, which is better at handling non-normally distributed data and accurately removing outliers compared to the traditional Z_score, thus reducing the impact of extreme values. Definition 2 ( For a set Assuming that For set the median of For set The standard deviation, for any Its MZ_score is obtained as ; DMPD with Process MPSV and set a threshold. Delete clients exceeding the threshold to obtain the set. ,in The malicious client's calculated MPSV is smaller due to the consistency of the backdoor task. An anomaly will occur.

Images