Backdoor attack defense methods, programs, devices, and storage media based on model parameter similarity detection

Abstract

This invention belongs to the field of federated learning security defense technology, specifically involving a backdoor attack defense method, program, device, and storage medium based on model parameter similarity detection. This invention analyzes the update magnitude and output changes of each model to reflect whether the distribution of client data samples has changed, thereby detecting abnormal model updates. This invention maintains records of historical model update magnitudes and historical output classification results for each model, and calculates the average similarity with the current update magnitude and output results. Based on these average similarities, it comprehensively evaluates whether the distribution of client data samples has changed. Combined with model parameter normalization technology, it can filter out backdoor poisoned updates while retaining benign model updates.

Description

Technical Field

[0001] This invention belongs to the field of federated learning security defense technology, specifically involving a backdoor attack defense method, program, device, and storage medium based on model parameter similarity detection. Background Technology

[0002] Federated learning, based on multi-party collaborative model training, aims to protect data privacy. However, it faces the security threat of backdoor attacks. These attacks have the potential to implant malicious functionality. For example, a backdoor attack might manipulate a local model, affecting the output of the global model, causing it to behave absurdly in the presence of specific inputs (such as road signs with specific trigger labels). Due to the imbalanced distribution of client training data and the attacker's sophisticated design, such as using loss function constraints to hide malicious updates, these attacks are difficult to defend against through anomaly detection. Summary of the Invention

[0003] The purpose of this invention is to provide a backdoor attack defense method, program, device and storage medium based on model parameter similarity detection. By analyzing the update magnitude and output changes of each model, it reflects whether the distribution of client data samples has changed, thereby detecting abnormal model updates.

[0004] The backdoor attack defense method based on model parameter similarity detection involves each user participating in this round of federated learning uploading their updated local model to the central server after receiving the global model parameters and completing local training. The central server then performs the following steps for each user's updated local model:

[0005] (1) Determine whether the updated local model parameters meet the normalization conditions; if they do, the first evaluation score is 1 point; if they do not, the first evaluation score is 0 points.

[0006] (2) Calculate the update magnitude of the output layer neurons of the updated local model in this round and perform normalization; according to the user's historical update list, calculate the similarity between the update magnitude of this round and the update magnitude of each benign model in the historical update list, and take the average; if the average similarity is lower than the predetermined threshold, the second evaluation score is 0; otherwise, the second evaluation score is 1.

[0007] (3) Use random input vectors to construct a random dataset and input it into the updated local model to obtain the classification results of each random sample and obtain the probability distribution of each classification result; according to the user's historical model probability distribution list, calculate the similarity between the probability distribution of the classification result in this round and the probability distribution of the classification result of each benign model in the historical model probability distribution list, and take the average value; if the average similarity value is lower than the predetermined threshold, the third evaluation score is 0; otherwise, the third evaluation score is 1.

[0008] (4) If the total evaluation score is 2 or 3, the user is determined to be a benign client and the updated local model uploaded by the user is determined to be a benign model.

[0009] In this round of federated learning, the central server only aggregates benign models uploaded by benign clients.

[0010] Furthermore, the determination of whether the updated local model parameters meet the normalization condition specifically involves:

[0011] After a healthy client participating in federated learning completes local training, it will normalize the model parameters to update the model.

[0012]

[0013] in, w represents the original model parameters for the j-th user in the t-th round of federated learning. ' j This represents the normalized model parameters.

[0014] Furthermore, the specific update magnitude of the output layer neurons of the local model in this round after computation is as follows:

[0015]

[0016] Among them, E t,k,i This represents the update magnitude of the output layer neuron i of the local model uploaded by user k in the t-th round of federated learning; H represents the number of connections from the output layer neuron to the previous layer neuron; b t,k,i This represents the bias of user k on neuron i of the output layer of the updated local model uploaded in round t of federated learning; and This indicates that it comes from the global model G. t The bias and weight of the neurons;

[0017] The normalized update magnitude ε of the output layer neuron i of the local model uploaded by user k in round t of federated learning. t,k,i for:

[0018]

[0019] Where P is the number of output layer neurons in user k's local model;

[0020] Based on user k's historical update list Hlist k,i The update magnitude {ε} of each neuron i in this round is calculated based on cosine similarity. t,k,i} and the historical update list Hlist k,iThe benign model update magnitude {ε} j,k,i The similarity between} is calculated and the average value is taken.

[0021]

[0022] Furthermore, the step of using random input vectors to construct a random dataset and inputting it into the updated local model to obtain the classification results of each random sample and the probability distribution of each classification result is as follows:

[0023]

[0024] Where n represents the sum of the number of times the classification result appears. k represents the total number of classification results; x i p represents the number of times the i-th classification result appears; i This represents the probability of the i-th classification result occurring.

[0025] A computer device / apparatus / system includes a memory, a processor, and a computer program stored in the memory, wherein the processor executes the computer program to implement the steps of the backdoor attack defense method based on model parameter similarity detection described above.

[0026] A computer-readable storage medium having a computer program / instructions stored thereon, which, when executed by a processor, implements the steps of the aforementioned backdoor attack defense method based on model parameter similarity detection.

[0027] A computer program product includes a computer program / instructions that, when executed by a processor, implement the steps of the aforementioned backdoor attack defense method based on model parameter similarity detection.

[0028] The beneficial effects of this invention are as follows:

[0029] This invention analyzes the update magnitude and output changes of each model to reflect whether the distribution of client-side data samples has changed, thereby detecting abnormal model updates. This invention maintains records of historical model update magnitudes and historical output classification results for each model, and calculates the average similarity with the current update magnitude and output results. Based on these average similarities, it comprehensively evaluates whether the distribution of client-side data samples has changed. Combined with model parameter normalization techniques, it can filter out backdoor poisoned updates while retaining benign model updates. Attached Figure Description

[0030] Figure 1 This is a schematic diagram illustrating the defense against covert backdoor attacks provided by the present invention.

[0031] Figure 2A diagram illustrating the backdoor attack defense scheme provided by this invention.

[0032] Figure 3 This is a diagram illustrating the defensive effect of the present invention against covert backdoor attacks.

[0033] Figure 4 This is a comparison diagram of the defensive effects of the method in this invention with other existing methods. Detailed Implementation

[0034] The present invention will now be further described with reference to the accompanying drawings.

[0035] This invention provides a backdoor attack defense method based on model parameter similarity detection, which effectively achieves the defense against covert backdoor attacks. This invention analyzes the update magnitude and output changes of each model to reflect whether the distribution of client data samples has changed, thereby detecting abnormal model updates.

[0036] (1) Parameter Normalization Processing. After participants receive the global model parameters and complete local training, this invention processes model updates by normalizing the model parameters. Participants who are normally involved in system training will follow this process, adjusting the model parameters to the same scale range. When the central server receives the normalized model update, it can judge whether the model update is normal by evaluating this update.

[0037] in, These are the original model parameters for the j-th participant in round t, w ' j These are the normalized model parameters. This normalization operation helps ensure that the model parameters are scaled consistently, thus making model updates more stable.

[0038]

[0039] When the central server receives a model update from a participant, it performs a normalization check on the received model update. If the model update does not meet the normalization condition, i.e., the norm of the parameters is not 1, the aggregator will give an evaluation score of 0; otherwise, it will give 1 point.

[0040] (2) Model Parameter Update Similarity Evaluation. The second metric used to identify malicious model update clients is the consistency of model update distribution. It analyzes the output layer parameter updates and extracts information about the label distribution in the underlying training data of the model. During training, the parameters of the output layer neurons corresponding to each class of samples are adjusted. Since this is repeated for each sample, neurons corresponding to the class of a sample are often updated multiple times, resulting in the accumulation of individual changes in these neurons into updates with high amplitude. In order to successfully inject a backdoor into the global model, a large number of toxic samples containing triggers are injected into the malicious client, so the data distribution of the client will change. In order to measure the magnitude of the update and deduce this distribution change in reverse, this invention calculates the update amplitude of the output layer neurons. Let H represent the number of connections from the output layer neurons to the previous layer neurons, b t,k,i w represents the deviation on neuron i in the output layer of model k after round t. t,k,i,h This represents the weights of the connections to the previous layer's neurons, h. and Similarly, this represents the data from the global model G. t The bias and weights of the neurons. Then, the magnitude E of the update of the output layer neuron i of the model submitted by client k in round t. t,k,i The definition is as follows:

[0041]

[0042] This invention normalizes the update magnitudes of all output layer neurons in the model to highlight the changing trends of the update magnitudes. Therefore, the normalized update magnitude ε of neuron i at round t, in client k, is... t,k,i The definition is as follows:

[0043]

[0044] To assess whether the update magnitude has changed drastically, this invention provides a per-model G... i A historical update list Hlist was created i Hlist i This includes all historically considered benign update magnitudes of client i prior to the current training cycle, thus providing a benchmark for new updates. This historical update magnitude list, Hlist, is used to define this benchmark. i The model update magnitude {ε} of model k in this round is calculated based on cosine similarity. t,k,i ,...ε t,k,n} and history list Hlist k The benign model update magnitude {ε} j,k,i ,...ε j,k,nThe similarity between} is calculated and the average value is taken. If the average similarity is lower than a predetermined threshold, it usually reflects that the data sample distribution of client i may have changed significantly, and there is a risk of backdoor attack being injected. In this case, the second evaluation score is 0. Conversely, if the update shows good consistency, the second evaluation score is 1.

[0045]

[0046] (3) Model Parameter Update Similarity Evaluation. When training a deep learning model, each specific sample includes an input x and an output class y. During training, the parameters of the deep learning neural network are iteratively adjusted to maximize the probability of the current model predicting class y for input x. When the model is close to convergence, the changes in input x and output class y become very small. However, changes in the distribution of client samples may affect the classification result output by input x. For example, in a computer vision scenario, x can be an image, and y can be the label corresponding to the image. However, for another input x... * Its tag y * The probability of y predicted by the model will also change slightly, even though y ≠ y * Because malicious attackers need to inject backdoor samples for backdoor training, the number of target category samples increases, which may cause changes in the probability distribution of the model's output. Therefore, this invention leverages this by using the same sample inputs to the model to obtain a classification result vector of these samples, thus predicting whether the sample distribution during model training has changed drastically. One problem is that the server lacks input data to evaluate the model, as this invention assumes the server has neither training nor test data. This invention constructs a random dataset D using random input vectors. random Instead of using actual data to solve this problem, the goal of this invention is to find the local model for the current round. The method focuses on the difference between the distribution of the classification results for the samples and the distribution of the corresponding historical model output classification results, rather than finding the category with the highest prediction probability. Therefore, it is not necessary to obtain meaningful predictions.

[0047] This invention will use all random samples x random The classification vectors are summed to obtain an overall classification vector. Since the classification vectors output by the model are discrete, this invention uses the overall distribution of the classification vectors to construct a multinomial probability distribution. This distribution is then used to measure the distribution of participant samples.

[0048]

[0049] Where n represents the sum of the number of times the classification result appears (e.g., the number of times the dice are rolled). k represents the total number of classification results (e.g., the number of faces of a die or the number of categories); x i p represents the number of times the i-th classification result appears; i This represents the probability of the i-th classification result occurring.

[0050] This invention constructs a historical model probability distribution list HPlist for each participant i. i This list records the probability distributions of classification results from local models that have been accepted and considered safe and reliable over various historical periods. By comparing the probability distributions of historical local model classification results with the current local model... The similarity of the probability distribution of the classification results is used to assess whether the distribution of the classification results of the uploaded model has changed drastically.

[0051] As a standard for measuring the consistency of model output distribution, this invention calculates the distribution of the current model i classification result vector and the historical list Hlist. i The similarity of the distribution of classification results from the normal model is evaluated using the mean of the cosine similarity as an indicator of model update consistency. If the calculated mean similarity is below a predetermined threshold, it usually reflects a significant change in the distribution of the client's data samples, indicating a risk of backdoor attacks. In this case, the third evaluation score is 0; conversely, if the update shows good consistency, the third evaluation score is 1.

[0052] (4) Deep Filtering. A voting filter is performed based on all obtained evaluation scores. If the overall score falls below 2, it may indicate a change in the client's sample distribution, suggesting the participant might be attempting malicious interference or a backdoor attack. If the model passes the filtering mechanism (i.e., the total evaluation score is 2 or 3), the model update and output distribution for this round are recorded. This new information will serve as an important reference for future training and evaluation. Finally, the central server aggregates only benign models uploaded by benign clients in this round of federated learning to obtain the global model.

[0053] Because attackers meticulously design malicious model updates and the uneven distribution of client data samples leads to more dispersed model updates, existing backdoor attack defense methods rely on the difference between benign and anomalous model updates to filter out anomalous updates, making it difficult for existing methods to identify backdoor attacks.

[0054] This invention analyzes the update magnitude and output changes of each model to reflect whether the distribution of client-side data samples has changed, thereby detecting abnormal model updates. This invention maintains records of historical model update magnitudes and historical output classification results for each model, and calculates the average similarity with the current update magnitude and output results. Based on these average similarities, it comprehensively evaluates whether the distribution of client-side data samples has changed. Combined with model parameter normalization techniques, it can filter out backdoor poisoned updates while retaining benign model updates.

[0055] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A backdoor attack defense method based on model parameter similarity detection, characterized in that: For each user participating in this round of federated learning, after receiving the global model parameters and completing local training, the updated local model is uploaded to the central server. The central server then performs the following steps for each user's uploaded updated local model: (1) Determine whether the updated local model parameters meet the normalization conditions; if they do, the first evaluation score is 1 point; if they do not, the first evaluation score is 0 points. (2) Calculate the update magnitude of the output layer neurons of the updated local model in this round and perform normalization; according to the user's historical update list, calculate the similarity between the update magnitude of this round and the update magnitude of each benign model in the historical update list, and take the average value. If the mean similarity is lower than a predetermined threshold, the second evaluation score is 0; otherwise, the second evaluation score is 1. (3) Use random input vectors to construct a random dataset and input it into the updated local model to obtain the classification results of each random sample and obtain the probability distribution of each classification result; according to the user's historical model probability distribution list, calculate the similarity between the probability distribution of the classification results in this round and the probability distribution of the classification results of each benign model in the historical model probability distribution list, and take the average value. If the mean similarity is lower than a predetermined threshold, the third evaluation score is 0; otherwise, the third evaluation score is 1. (4) If the total evaluation score is 2 or 3, the user is determined to be a benign client and the updated local model uploaded by the user is determined to be a benign model. In this round of federated learning, the central server only aggregates benign models uploaded by benign clients.

2. The backdoor attack defense method based on model parameter similarity detection according to claim 1, characterized in that: The determination of whether the updated local model parameters meet the normalization condition is as follows: After a healthy client participating in federated learning completes local training, it will normalize the model parameters to update the model. in, w' represents the original model parameters of the j-th user in the t-th round of federated learning. j This represents the normalized model parameters.

3. The backdoor attack defense method based on model parameter similarity detection according to claim 1, characterized in that: The specific update magnitude of the output layer neurons of the local model in this round after calculation and updating is as follows: Among them, E t,k,i This represents the update magnitude of the output layer neuron i of the local model uploaded by user k in the t-th round of federated learning; H represents the number of connections from the output layer neuron to the previous layer neuron; b t,k,i This represents the bias of user k on neuron i of the output layer of the updated local model uploaded in round t of federated learning; and This indicates that it comes from the global model G. t The bias and weight of the neurons; The normalized update magnitude ε of the output layer neuron i of the local model uploaded by user k in round t of federated learning. t,k,i for: Where P is the number of output layer neurons in user k's local model; Based on user k's historical update list Hlist k,i The update magnitude {ε} of each neuron i in this round is calculated based on cosine similarity. t,k,i } and the historical update list Hlist k,i The benign model update magnitude {ε} j,k,i The similarity between} is calculated and the average value is taken.

4. The backdoor attack defense method based on model parameter similarity detection according to claim 1, characterized in that: The process involves using random input vectors to construct a random dataset, which is then input into the updated local model to obtain the classification results for each random sample. The probability distribution of each classification result is then obtained, specifically as follows: Where n represents the sum of the number of times the classification result appears. k represents the total number of classification results; x i p represents the number of times the i-th classification result appears; i This represents the probability of the i-th classification result occurring.

5. A computer device, comprising a memory, a processor, and a computer program stored in the memory, characterized in that: The processor executes the computer program to implement the steps of the method according to any one of claims 1 to 4.

6. A computer-readable storage medium having a computer program stored thereon, characterized in that: When executed by a processor, the computer program implements the steps of the method according to any one of claims 1 to 4.

7. A computer program product comprising computer instructions, characterized in that: When executed by a processor, the computer instructions implement the steps of the method according to any one of claims 1 to 4.

Images