Backdoor attack defense method based on model parameter similarity detection, program, equipment and storage medium

Abstract

The invention belongs to the technical field of federated learning security defense, and particularly relates to a backdoor attack defense method based on model parameter similarity detection, a program, equipment and a storage medium. According to the method, the change of the updating amplitude and the output of each model is analyzed to reflect whether client data sample distribution changes or not, so that abnormal model updating is found. According to the method, records of historical model updating amplitude and historical output classification results are maintained for each model, the average similarity between the historical model updating amplitude and the historical output classification results is calculated, whether data sample distribution of a client changes or not is comprehensively evaluated based on the average similarity, and a model parameter normalization technology is combined to determine whether the data sample distribution of the client changes or not. Back door poisoning updates may be filtered while preserving benign model updates.

Description

Technical Field

[0001] The present invention belongs to the field of federated learning security defense technology, and specifically relates to a backdoor attack defense method, program, device and storage medium based on model parameter similarity detection. Background Art

[0002] Federated learning is based on multi-party collaborative training models and aims to protect data privacy. However, it faces the security threat of backdoor attacks. These attacks have the potential to implant potentially malicious functions. For example, a backdoor attack may manipulate the local model to affect the output of the global model, causing it to behave abnormally when a specific input (such as a street sign with a specific trigger label) exists. These attacks are difficult to defend against through anomaly detection due to the unbalanced distribution of client training data and the attacker's careful design, such as using loss function constraints to hide malicious updates. Summary of the invention

[0003] The purpose of the present invention is to provide a backdoor attack defense method, program, device and storage medium based on model parameter similarity detection, which reflects whether the distribution of client data samples has changed by analyzing the update amplitude and output changes of each model, so as to discover abnormal model updates.

[0004] The backdoor attack defense method based on model parameter similarity detection is as follows: for each user participating in this round of federated learning, after receiving the global model parameters and completing local training, the updated local model is uploaded to the central server. The central server performs the following steps for each updated local model uploaded by the user:

[0005] (1) Determine whether the updated local model parameters meet the normalization condition; if so, the first evaluation score is 1; if not, the first evaluation score is 0;

[0006] (2) Calculate the update amplitude of the output layer neurons of the updated local model in this round and perform normalization; calculate the similarity between the update amplitude of this round and the update amplitude of each benign model in the historical update list according to the user's historical update list, and take the average; if the similarity average is lower than a predetermined threshold, the second evaluation score is 0; otherwise, the second evaluation score is 1;

[0007] (3) Use the random input vector to construct a random data set and input it into the updated local model to obtain the classification results of each random sample and the probability distribution of each classification result; according to the user's historical model probability distribution list, calculate the similarity between the probability distribution of the classification results of this round and the probability distribution of the classification results of each benign model in the historical model probability distribution list, and take the average; if the similarity average is lower than the predetermined threshold, the third evaluation score is 0; otherwise, the third evaluation score is 1;

[0008] (4) All evaluation scores are combined. If the total evaluation score is 2 or 3, the user is judged to be a benign client, and the updated local model uploaded by the user is judged to be a benign model;

[0009] The central server only aggregates the benign models uploaded by benign clients in this round of federated learning.

[0010] Furthermore, the determination of whether the updated local model parameters meet the normalization condition is specifically as follows:

[0011] After completing local training, benign clients that normally participate in federated learning will normalize the model parameters to update the model;

[0012]

[0013] in, represents the original model parameters of the jth user in the tth round of federated learning, w' j represents the normalized model parameters.

[0014] Furthermore, the update amplitude of the output layer neurons of the local model after calculation and update in this round is specifically:

[0015]

[0016] Among them, E t,k,i represents the update amplitude of the output layer neuron i of the updated local model uploaded by user k in the tth round of federated learning; H represents the number of connections from the output layer neuron to the previous layer neuron; b t,k,i represents the deviation of the output layer neuron i of the updated local model uploaded by user k in the tth round of federated learning; w t,k,i,h express and Represents the global model G t The biases and weights of the neurons in (this explanation has no t, i, h);

[0017] The normalized update amplitude ε of the output layer neuron i of the updated local model uploaded by user k in the tth round of federated learning t,k,i for:

[0018]

[0019] Where P is the number of output layer neurons in the local model of user k;

[0020] Update list Hlist based on user k's history k,i , based on cosine similarity, the update amplitude {ε t,k,i} and the historical update list Hlist k,i The benign model update amplitude {ε j,k,i} and take the mean

[0021]

[0022] Furthermore, the random data set is constructed by using the random input vector and inputted into the updated local model to obtain the classification result of each random sample and obtain the probability distribution of each classification result, specifically:

[0023]

[0024] Among them, n represents the sum of the number of occurrences of the classification results. k represents the total number of classification results; x i Indicates the number of occurrences of the i-th classification result; p i Represents the probability of the i-th classification result occurring.

[0025] A computer device / equipment / system includes a memory, a processor, and a computer program stored in the memory, wherein the processor executes the computer program to implement the steps of the backdoor attack defense method based on model parameter similarity detection.

[0026] A computer-readable storage medium stores a computer program / instruction, which, when executed by a processor, implements the steps of the above-mentioned backdoor attack defense method based on model parameter similarity detection.

[0027] A computer program product includes a computer program / instruction, which, when executed by a processor, implements the steps of the above-mentioned backdoor attack defense method based on model parameter similarity detection.

[0028] The beneficial effects of the present invention are:

[0029] The present invention analyzes the update amplitude and output changes of each model to reflect whether the distribution of client data samples has changed, so as to discover abnormal model updates. The present invention maintains records of historical model update amplitudes and historical output classification results for each model, and calculates the average similarity with the current update amplitude and output results. Based on these average similarities, it comprehensively evaluates whether the distribution of client data samples has changed. Combined with model parameter normalization technology, backdoor poisoning updates can be filtered while retaining benign model updates. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] Figure 1 A schematic diagram of the covert backdoor attack defense provided by the present invention.

[0031] Figure 2 A diagram of the backdoor attack defense solution provided by the present invention.

[0032] Figure 3 This is a diagram showing the defensive effect of the present invention against hidden backdoor attacks.

[0033] Figure 4 The figure is a comparison chart of the defense effect of the method in the present invention and other existing methods. DETAILED DESCRIPTION

[0034] The present invention is further described below in conjunction with the accompanying drawings.

[0035] The present invention provides a backdoor attack defense method based on model parameter similarity detection, which actually and effectively realizes hidden backdoor attack defense. The present invention reflects whether the distribution of client data samples has changed by analyzing the update amplitude and output changes of each model, so as to discover abnormal model updates.

[0036] (1) Parameter normalization. After the participants receive the global model parameters and complete local training, the present invention processes the model update by normalizing the model parameters. Participants who normally participate in system training will follow this process and adjust the model parameters to the same scale range. When the central server receives the normalized model update, it can judge whether the model update is normal by judging the model update.

[0037] in, is the original model parameter of the jth participant in round t, w' j are the normalized model parameters. This normalization operation helps ensure that the scale of the model parameters is consistent, making the model update more stable.

[0038]

[0039] When the central server receives the model update from the participant, it will normalize the received model update. If the model update does not meet the normalization conditions, that is, the norm of the parameter is not 1, the aggregator will give an evaluation score of 0, otherwise it will give 1.

[0040] (2) Model parameter update similarity assessment. The second metric used to identify malicious model update clients is model update distribution consistency. It analyzes the output layer parameter updates and extracts information about the label distribution in the underlying training data of the model. During the training process, the parameters of the output layer neurons corresponding to each type of sample will be adjusted. Since this is repeated for each sample, neurons corresponding to the category of samples that often appear will be updated multiple times, causing the individual changes of these neurons to accumulate into updates with high amplitudes. In order to successfully inject a backdoor into the global model, a large number of poisonous samples containing triggers will be injected into the malicious client, so the data distribution of the client will change. In order to measure the amplitude of the update and reversely deduce this distribution change, the present invention calculates the update amplitude of the output layer neurons. Let H represent the number of connections from the output layer neurons to the neurons in the previous layer, b t,k,i represents the bias of neuron i in the output layer of model k after round t, w t,k,i,h represents the weight of the connection with the previous layer of neurons, h, and Similarly, we express the global model G t Then, the update amplitude E of the output layer neuron i of the model submitted by client k in round t is t,k,i The definition is as follows:

[0041]

[0042] The present invention normalizes the update amplitudes of all output layer neurons of the model to highlight the changing trend of the update amplitudes. Therefore, the normalized update amplitude ε of the update of neuron i at client k in round t is t,k,i The definition is as follows:

[0043]

[0044] In order to evaluate whether the update amplitude has changed dramatically, the present invention provides each model G i Create a historical update list Hlist i , Hlist i This includes all the historical update amplitudes that were identified as benign by client i before the current training cycle, thus providing a reference standard for new updates. i , based on cosine similarity, calculate the model update amplitude {ε t,k,i ,...ε t,k,n} and the history list Hlist k The benign model update amplitude {ε j,k,i ,...ε j,k,n}, and take the average; if the similarity average is lower than the predetermined threshold, this usually reflects that the data sample distribution of client i may have changed significantly, and there is a risk of implanting a backdoor attack. In this case, the second evaluation score is 0; on the contrary, if the update shows good consistency, the second evaluation score is 1;

[0045]

[0046] (3) Model parameter update similarity evaluation. When training a deep learning model, each specific sample includes an input x and an output category y. During the training process, the parameters of the deep learning neural network are iteratively adjusted to maximize the probability of the current model predicting category y for input x. When the model is close to convergence, the change in input x and an output category y becomes very small. However, when the client sample distribution changes, it may affect the classification result output by input x. For example, in a computer vision scenario, x can be an image and y is the label corresponding to the image. However, for another input x * , whose label y * The probability of y predicted by the model will also change slightly, even though y ≠ y * . Since malicious attackers need to inject backdoor samples for backdoor training, the number of target category samples of the backdoor increases, which may cause changes in the probability distribution of the model output. Next, the present invention takes advantage of this and uses the same samples to input into the model to obtain a model classification result vector for these samples to predict whether the sample distribution of training the model has changed dramatically. One problem is that the server has no input data to evaluate the model because the present invention assumes that the server has neither training data nor test data. The present invention constructs a random data set D by using random input vectors random Rather than actual data to solve this problem. Because the goal of the present invention is to find the current round of local model The difference between the distribution of the classification results of the sample and the distribution of the corresponding historical model output classification results, rather than finding the category with the highest prediction probability, is not required to obtain meaningful predictions.

[0047] The present invention converts all random samples x random The classification vectors of the model are added to obtain an overall classification vector. Since the classification vector output by the model is discrete, the present invention uses the classification vector of the overall distribution to construct a multinomial probability distribution. This distribution is used to measure the distribution of the participant sample.

[0048]

[0049] Where n is the sum of the number of times the classification results appear (for example, the number of times a dice is rolled), k represents the total number of classification results (for example, the number of faces of a dice or the number of categories); x i Indicates the number of occurrences of the i-th classification result; p i Represents the probability of the i-th classification result occurring.

[0050] The present invention constructs a historical model probability distribution list HPlist for each participant i i This list records the probability distribution of classification results of local models that were accepted in various periods in the past and were considered safe and reliable. By comparing the probability distribution of classification results of historical local models with the current local model The similarity of the probability distribution of the classification results of the uploaded model is used to evaluate whether the distribution of the classification results of the uploaded model has changed drastically.

[0051] As a criterion for measuring the consistency of model output distribution, the present invention calculates the distribution of the classification result vector of the current model i and the historical list Hlist i The similarity of the distribution of the normal model classification results in the , the mean of the cosine similarity is used as an indicator to evaluate the consistency of the model update. If the obtained similarity mean is lower than the predetermined threshold, this usually reflects that the data sample distribution of the client may have changed significantly, and there is a risk of implanting a backdoor attack. In this case, the third evaluation score is 0; conversely, if the update shows good consistency, the third evaluation score is 1;

[0052] (4) Deep filtering. Voting filtering is performed based on all the evaluation scores obtained. Once the comprehensive score is lower than 2 points, it may indicate that the sample distribution of the client is changing, and the participant may be trying to conduct malicious interference and launch a backdoor attack. If the model passes the filtering mechanism, that is, if the total evaluation score is 2 or 3 points, then the model update and output distribution of this round will be recorded. This new information will serve as an important reference for future training and evaluation. Finally, the central server only aggregates the benign models uploaded by benign clients in this round of federated learning to obtain the global model.

[0053] Since attackers carefully design malicious model updates and the uneven distribution of client data samples makes model updates more dispersed, existing backdoor attack defense methods screen abnormal updates based on the difference between benign model updates and abnormal model updates. Existing methods find it difficult to identify hidden backdoor attacks.

[0054] The present invention analyzes the update amplitude and output changes of each model to reflect whether the distribution of client data samples has changed, so as to discover abnormal model updates. The present invention maintains records of historical model update amplitudes and historical output classification results for each model, and calculates the average similarity with the current update amplitude and output results. Based on these average similarities, it comprehensively evaluates whether the distribution of client data samples has changed. Combined with model parameter normalization technology, backdoor poisoning updates can be filtered while retaining benign model updates.

[0055] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A backdoor attack defense method based on model parameter similarity detection, characterized in that: For each user participating in this round of federated learning, after receiving the global model parameters and completing local training, the updated local model is uploaded to the central server. The central server performs the following steps for each updated local model uploaded by the user: (1) Determine whether the updated local model parameters meet the normalization condition; if so, the first evaluation score is 1; if not, the first evaluation score is 0; (2) Calculate the update amplitude of the output layer neurons of the updated local model in this round and perform normalization; calculate the similarity between the update amplitude of this round and the update amplitude of each benign model in the historical update list according to the user's historical update list, and take the average; If the similarity mean is lower than a predetermined threshold, the second evaluation score is 0; otherwise, the second evaluation score is 1; (3) Use random input vectors to construct random data sets and input them into the updated local model to obtain the classification results of each random sample and the probability distribution of each classification result; according to the user's historical model probability distribution list, calculate the similarity between the probability distribution of the classification results of this round and the probability distribution of the classification results of each benign model in the historical model probability distribution list, and take the average; If the similarity mean is lower than a predetermined threshold, the third evaluation score is 0; otherwise, the third evaluation score is 1; (4) All evaluation scores are combined. If the total evaluation score is 2 or 3, the user is judged to be a benign client, and the updated local model uploaded by the user is judged to be a benign model; The central server only aggregates the benign models uploaded by benign clients in this round of federated learning.

2. The backdoor attack defense method based on model parameter similarity detection according to claim 1 is characterized in that: The determination of whether the updated local model parameters meet the normalization condition is specifically as follows: After completing local training, benign clients that normally participate in federated learning will normalize the model parameters to update the model; in, represents the original model parameters of the jth user in the tth round of federated learning, w' j Represents the normalized model parameters.

3. The backdoor attack defense method based on model parameter similarity detection according to claim 1 is characterized in that: The update amplitude of the output layer neurons of the local model after calculation and update in this round is specifically: Among them, E t,k,i represents the update amplitude of the output layer neuron i of the updated local model uploaded by user k in the tth round of federated learning; H represents the number of connections from the output layer neuron to the previous layer neuron; b t,k,i represents the deviation of the output layer neuron i of the updated local model uploaded by user k in the tth round of federated learning; w t,k,i,h express and Represents the global model G t The biases and weights of the neurons in (this explanation has no t, i, h); The normalized update amplitude ε of the output layer neuron i of the updated local model uploaded by user k in the tth round of federated learning t,k,i for: Where P is the number of output layer neurons in the local model of user k; Update list Hlist based on user k's history k,i , based on cosine similarity, the update amplitude {ε t,k,i } and the historical update list Hlist k,i The benign model update amplitude {ε j,k,i } and take the mean 4. The backdoor attack defense method based on model parameter similarity detection according to claim 1, characterized in that: The random input vector is used to construct a random data set and input it into the updated local model to obtain the classification results of each random sample and the probability distribution of each classification result, specifically: Among them, n represents the sum of the number of occurrences of the classification results. k represents the total number of classification results; x i Indicates the number of occurrences of the i-th classification result; p i Represents the probability of the i-th classification result occurring.

5. A computer device / equipment / system comprising a memory, a processor and a computer program stored in the memory, characterized in that: The processor executes the computer program to implement the steps of the method according to any one of claims 1 to 4.

6. A computer-readable storage medium having a computer program / instruction stored thereon, characterized in that: When the computer program / instructions are executed by a processor, the steps of the method according to any one of claims 1 to 4 are implemented.

7. A computer program product comprising a computer program / instructions, characterized in that: When the computer program / instructions are executed by a processor, the steps of the method according to any one of claims 1 to 4 are implemented.

Images