Federal learning persistent backdoor attack method based on dynamic optimization trigger

Abstract

The invention discloses a federal learning persistent backdoor attack method based on a dynamic optimization trigger, and the method comprises the steps: firstly reading an attack feedback index from the second attack round, and adjusting the attack parameters of the round; secondly, dynamically optimizing the backdoor trigger, simulating a benign global model after the attack is stopped by using local data while global dynamic updating is carried out by an attacker, minimizing the potential difference between the trigger characteristics during the attack and after the attack, and training the memory ability of the global model to the trigger characteristics; then, adaptively adding noise to a model full-connection layer to enhance the attack effectiveness; and finally, updating an attack feedback index, adopting a redundant neuron as a feedback index for judging whether the attack is successful or not, initializing in a first attack round, and updating in other attack rounds. According to the method, the persistent backdoor attack aiming at the federated learning framework is realized by adopting a dynamic trigger optimization strategy and an attack feedback index based on redundant neurons.

Description

Technical Field

[0001] The present invention belongs to the field of artificial intelligence security, and specifically relates to a federated learning persistent backdoor attack method based on dynamic optimization triggers. Background Art

[0002] Federated learning can jointly train models without each client sharing data, which protects the privacy and security of each client's private data to a certain extent. However, although the data involved in the training and the local model training are privately controlled by each client, the server's invisibility to the local training process makes it impossible to verify the authenticity of the uploaded model parameters. Malicious users may exploit this defect to attack the server, especially by uploading malicious models during the communication process to destroy or affect the accuracy of the global model. Due to the privacy of data in federated learning, attackers usually attack the global model by poisoning the model. In particular, the backdoor attack trains a poisoned model with carefully designed triggers, so that the model only outputs the target label in the scenario containing the trigger, without destroying the model's normal classification task facing other benign data. At present, common triggers include image pixels, natural attributes of data, and self-added noise.

[0003] A large number of studies have demonstrated the feasibility of implanting backdoors in federated learning. However, these methods often use a single fixed trigger throughout the attack, ignoring the dynamic update characteristics of federated learning. A large number of benign models are aggregated with a few malicious models, which may cause the features of the fixed trigger to be covered by the features of the benign model, and eventually the influence is eliminated and the attack capability is lost. At the same time, when the trigger is stopped, the effect of the backdoor attack will quickly weaken and lose its effect due to the "catastrophic forgetting" characteristic of the neural network. In order to maintain the high accuracy of the attack, the attack has to be launched continuously, which greatly increases the cost of the attack and increases the risk of attack exposure. The existing dynamic adjustment strategy only focuses on the attack effect of the local model and the global model at the attacker's current time, and does not really discuss the scenario where the trigger does not participate in the model training. Therefore, how to dynamically adjust the trigger is still a technical problem worth exploring and needs to be solved. Summary of the invention

[0004] Purpose of the invention: The present invention proposes a method for persistent backdoor attack on federated learning based on dynamic optimization triggers, which can maintain the accuracy of the main task of federated learning.

[0005] Technical solution: The method for persistent backdoor attack based on federated learning of dynamic optimization triggers described in the present invention specifically includes the following steps:

[0006] (1) Starting from the second attack round, the attacker first reads the global model received in this round to obtain the feedback indicator set, and then adjusts the attack parameters of this round according to the feedback indicator set;

[0007] (2) The attacker simulates a benign global model using a local benign data set to predict the model after the attack stops; optimizes the trigger based on the idea of ​​minimizing the potential difference between the poisoned data in the current global model and the simulated benign global model; and uses the optimized trigger to train the poisoned model;

[0008] (3) Noise is considered as an optimizable target, and noise is added to the fully connected layer of the model to avoid over-concentrated neuron updates in the fully connected layer, thus obtaining a poisoned model after adding noise;

[0009] (4) Redundant neurons are used as feedback indicators of whether the attack is successful. The indicator set is initialized in the first attack round and the value of the indicator set is updated in the remaining attack rounds. After the update, the poisoned model is uploaded to the server.

[0010] Furthermore, the step (1) comprises the following steps:

[0011] (11) Obtaining the feedback indicator set: The attacker first receives the global model G of this round t , for the attacker L c , using the indicator set represents the position of the index neuron, model L c Index I c The parameter value at is expressed as Representation model L c Compared to Model G t At index I c The model update value, if the received global model G t Existence If the poisoning fails, it is recorded as r, otherwise it is considered that the poisoning is effective and recorded as a; according to the above judgment criteria, the feedback indicator set array is obtained;

[0012] (12) Adjusting attack parameters: The attack parameters that need to be adjusted include α and β. α is used to balance the ratio between trigger effectiveness and persistence tasks. If the feedback indicator set includes a, it indicates that the direction of attack effectiveness is satisfied. The optimization of the trigger task should focus on enhancing the persistence of the attack. Then the value of α is increased by a certain amount. β is used to balance the two goals of the noise task, reducing the variance of the update energy of the fully connected layer and constraining all added noise and satisfying The norm is 0; if the number of r contained in the feedback indication set is greater than or equal to half of the number of indication sets, it indicates that the attacker has failed to attack successfully, and there may still be significant differences in the output layer. Therefore, the noise task should focus on reducing the differences, and the β value must increase; the increase is 0.01.

[0013] Furthermore, the resources used by the attacker in step (2) only include the global model sent by the server and the local data set of the poisoned client, that is, the attack occurs in a black box environment;

[0014] Furthermore, the step (2) comprises the following steps:

[0015] (21) Training a benign global model: In each round, the attacker accepts the global model sent down and trains a benign global model locally using the local dataset controlled by the attacker to simulate the global model after the attack stops;

[0016] (22) Trigger optimization: The attacker uses the trained benign global model and the global model received and sent in this round to minimize the difference in the potential representation of the data implanted with the trigger in the two models. The loss function used is:

[0017]

[0018] in, Indicates that the trigger is in the current global model G t The attack task loss, α is used to balance the ratio of trigger effectiveness and persistence tasks, x * =x⊕ξ represents the poisoned data implanted into the trigger, ξ represents the backdoor trigger, y * Indicates the target poisoning label; Calculated by the following formula:

[0019]

[0020] Among them, σ(·) is the activation function of the first activation layer of the model, which is used to represent the potential representation of the trigger-containing data; represents the cross entropy loss; is the simulated attack stop global model;

[0021] (24) Poisoning training: Use optimized triggers to poison data, and use the poisoned data for local training to obtain the poisoned model.

[0022] Furthermore, the step (3) is implemented by the following formula:

[0023]

[0024] in, is the loss function of the noise optimization method, represents the variance of the output layer neurons, which is used to reduce the discreteness of the output neuron update energy; the second term of the loss is used to constrain the l2 norm of all added noise and to be 0, m represents the number of output layer neurons; β is used to balance the two objectives of the noise task.

[0025] Furthermore, the step (4) comprises the following steps:

[0026] (41) Find the neuron with the smallest absolute gradient and curvature: Use local data D train =

[0027] {D benign ,D backdoor}Get approximate gradient and approximate curvature;

[0028] (42) Construct an index set: Select neurons with the smallest absolute gradient and curvature as indicator neurons and record their index sets As the indicator set, change the parameter value by k times, After modifying the model parameters, upload it to the server as the final poisoned model.

[0029] Furthermore, the backdoor trigger ξ described in step (22) is initialized to a pixel block of size 5×5 and value 0.5; the optimization process does not change the size of the trigger, but only changes the value of the trigger.

[0030] Furthermore, the set size of the indicator set described in step (42) is equal to the number of attackers, and each attacker corresponds to an index.

[0031] Beneficial effects: Compared with the prior art, the present invention has the following beneficial effects: the present invention proposes a new strategy for dynamically optimizing triggers, discusses the scenario where triggers do not participate in model training, optimizes triggers before each round of aggregation in federated learning, trains the global model's ability to remember trigger features, and extends the persistence of triggers; the present invention combines the indication mechanism to adaptively add noise to the fully connected layer, and by treating the noise as an optimizable variable, balances the significant differences in the output layer of the poisoned model, thereby improving the effectiveness of backdoor attacks; the present invention does not undermine the accuracy of the main task of federated learning. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] Figure 1 is a flow chart of the present invention;

[0033] Figure 2 The following are the results of the attack effectiveness experiment, where (a) is the experimental result of the MNIST dataset; (b) is the experimental result of the CIFAR-100 dataset;

[0034] Figure 3Figure 2 is a diagram of the attack persistence effect; (a) is the experimental result of the CIFAR-10 dataset, and (b) is the experimental result of the CIFAR-100 dataset. DETAILED DESCRIPTION

[0035] The present invention is further described in detail below in conjunction with the accompanying drawings.

[0036] The present invention proposes a method for persistent backdoor attack in federated learning based on dynamically optimized triggers, which focuses on solving the problem that triggers are prone to failure in backdoor attacks. Specifically, in the scenario where the prediction triggers in each round of dynamic updates of federated learning do not participate in model training, the backdoor triggers are synchronously and dynamically optimized so that the triggers can adapt to the global model after the attack stops in advance. At the same time, a reasonable noise addition method is designed, redundant neurons are integrated as an attack indication mechanism, the poisoning effect is monitored, and finally poisoning is initiated based on the optimized triggers, and noise is used to alleviate the prominent features to achieve a concealed and persistent federated learning backdoor attack. Figure 1 As shown, the specific steps include:

[0037] Step 1: Attack strategy adjustment. Except for the first attack round, the attacker first reads the global model received in this round to obtain the feedback indicator set, and then adjusts the attack parameters of this round according to the feedback indicator set.

[0038] Get the feedback indicator set. The attacker first receives the global model of this round. For the attacker, the indicator set is used to represent the position of the indicator neuron. The parameter value at the index of the model is represented as , which represents the update value of the model compared to the model at the index. If it exists in the received global model, it indicates that the poisoning failed and is recorded as r. Otherwise, it is considered that the poisoning is valid and is recorded as a. According to the above judgment criteria, the feedback indicator set array is obtained.

[0039] Adjust the attack parameters. The attack parameters that need to be adjusted include and . It is used to balance the ratio of trigger effectiveness and persistence tasks. If the feedback indicator set contains a, it indicates that the direction of attack effectiveness is satisfied. The optimization of the trigger task should focus on enhancing the persistence of the attack, and the value is increased by 0.01. It is used to balance the two goals of the noise task, reduce the variance of the fully connected layer update energy and constrain all added noise and meet the norm of 0. If the number of r contained in the feedback indicator set is greater than or equal to half of the number of indicator sets, it indicates that there is an attacker who failed to attack successfully, and there may still be significant differences in the output layer. Therefore, the noise task should focus on reducing the difference, and the value is increased by 0.01.

[0040] Step 2: Trigger optimization and training of poisoned models. The attacker simulates a benign global model using a local benign data set to predict the model after the attack stops. The trigger is optimized based on the idea of ​​minimizing the potential difference between the poisoned data in the current global model and the simulated benign global model. The poisoned model is trained using the optimized trigger.

[0041] Train a benign global model. In each round, the attacker accepts the global model sent down and trains a benign global model locally using the local dataset controlled by the attacker to simulate the global model after the attack stops.

[0042] Optimize triggers. The attacker uses the trained benign global model and the global model received and sent in this round to minimize the difference in the potential representation of the data implanted with the trigger in the two models. The loss function used is:

[0043]

[0044] in, Indicates that the trigger is in the current global model G t The attack task loss, α is used to balance the ratio of trigger effectiveness and persistence tasks, x * =x⊕ξ represents the poisoned data implanted into the trigger, ξ represents the trigger, y * Indicates the target poisoning label. Calculated by the following formula:

[0045]

[0046] where σ(·) is the activation function of the first activation layer of the model, which is used here to represent the latent representation of the trigger-containing data; represents the cross entropy loss; is the simulated attack stopping global model.

[0047] Poisoning training. Use optimized triggers to poison data, and use poisoned data for local training to obtain a poisoned model.

[0048] Step 3: Consider the noise as an optimizable target, add noise to the fully connected layer of the model to avoid over-concentrated neuron updates in the fully connected layer, and use the following loss function for the model after adding noise:

[0049]

[0050] in, represents the variance of the output layer neurons, which is used to reduce the discreteness of the output neuron update energy; the second term of the loss is used to constrain the l2 norm of all added noise and 0, m represents the number of output layer neurons; β is the attack parameter that balances the influence of the two.

[0051] Step 4: Update the indicator set. Redundant neurons are used as feedback indicators of whether the attack is successful. The indicator set is initialized in the first attack round and the value of the indicator set is updated in the remaining attack rounds. After the update, the poisoning model is uploaded to the server.

[0052] Find redundant neurons. Redundant neurons need to satisfy the requirement that the gradient update is significantly small; small changes in parameters do not affect the main task loss, that is, find neurons with the smallest absolute gradient and curvature. Because it is in a black box environment, it is impossible to directly obtain the current global model G t The specific gradient update uses the local data D train =

[0053] {D benign ,D backdoor}Get the approximate gradient and approximate curvature.

[0054] Construct an index set; select neurons with the smallest absolute gradient and curvature as indicator neurons, and record their index sets As an indicator set, change the parameter value here to k times, After modifying the model parameters, upload it to the server as the final poisoned model.

[0055] Figure 2 The attack success effect of the present invention is shown. The attack success rate of the present invention and the three comparative methods is the average of the attack success rates of the last 10 attack rounds. The attack was successful under 2 types of data and 5 types of defense. Figure 2 (a) shows the experimental results of the MNIST dataset. Figure 2 (b) shows the experimental results of the CIFAR-100 dataset, which can achieve a nearly 100% attack success rate.

[0056] Figure 3 Demonstrates the attack persistence effect of the present invention, Figure 3 (a) shows the experimental results of the CIFAR-10 dataset. Figure 3 (b) is the experimental result of CIFAR-100 dataset. The two types of icons of the present invention represent respectively: the symbol "*" indicates that the present invention adds a scaling factor, and the one without "*" indicates that the scaling factor is not added. The scaling factor refers to the attack strategy adopted by DBA to amplify the model update parameters, which is set to 100 here. DBA points out that only one complete attack is performed, and the attacker needs to scale his malicious update to overwhelm other benign updates and ensure that the backdoor survives the aggregation step. The results show that although the present invention without scaling cannot gain an advantage, after adding the scaling parameter, our method is significantly better than the advanced distributed trigger in a single attack.

[0057] By comparison, the method provided by the present invention can launch effective and persistent attacks in the common image recognition field, and has excellent robustness against several popular backdoor defense methods. In particular, in the multi-classification scenarios of images for the three data sets of MNIST, CIFAR-10, and CIFAR-100, the present invention dynamically optimizes triggers to ensure the effectiveness of the attack while extending the persistence of the attack, that is, the method proposed by the present invention strengthens the attacker's ability and has better attack performance.

[0058] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principle of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A method for persistent backdoor attack based on federated learning with dynamic optimization triggers, characterized in that: The following steps are involved: (1) Starting from the second attack round, the attacker first reads the global model received in this round to obtain the feedback indicator set, and then adjusts the attack parameters of this round according to the feedback indicator set; (2) The attacker simulates a benign global model using a local benign dataset to predict the model after the attack stops; the trigger is optimized based on the idea of ​​minimizing the potential difference between the poisoned data in the current global model and the simulated benign global model; Use the optimized trigger training to obtain the poisoning model; (3) Noise is considered as an optimizable target, and noise is added to the fully connected layer of the model to avoid over-concentrated neuron updates in the fully connected layer, thus obtaining a poisoned model after adding noise; (4) Redundant neurons are used as feedback indicators of whether the attack is successful. The indicator set is initialized in the first attack round and the value of the indicator set is updated in the remaining attack rounds. After the update, the poisoned model is uploaded to the server.

2. According to claim 1, a method for persistent backdoor attack based on federated learning with dynamic optimization triggers is characterized in that: The step (1) comprises the following steps: (11) Obtaining the feedback indicator set: The attacker first receives the global model G of this round t , for the attacker L c , using the indicator set represents the position of the index neuron, model L c Index I c The parameter value at is expressed as Representation model L c Compared to Model G t At index I c The model update value, if the received global model G t Existence If the poisoning fails, it is recorded as r, otherwise it is considered that the poisoning is effective and recorded as a; according to the above judgment criteria, the feedback indicator set array is obtained; (12) Adjusting attack parameters: The attack parameters that need to be adjusted include α and β. α is used to balance the ratio between trigger effectiveness and persistence tasks. If the feedback indicator set includes a, it indicates that the direction of attack effectiveness is satisfied. The optimization of the trigger task should focus on enhancing the persistence of the attack. Then the value of α is increased by a certain amount. β is used to balance the two goals of the noise task, reducing the variance of the update energy of the fully connected layer and constraining all added noise and satisfying The norm is 0; if the number of r contained in the feedback indication set is greater than or equal to half of the number of indication sets, it indicates that the attacker has failed to attack successfully, and there may still be significant differences in the output layer. Therefore, the noise task should focus on reducing the differences, and the β value must increase.

3. According to claim 1, a method for persistent backdoor attack based on federated learning with dynamic optimization triggers is characterized in that: The resources used by the attacker in step (2) only include the global model sent by the server and the local data set of the poisoned client, that is, the attack occurs in a black box environment.

4. According to claim 1, a method for persistent backdoor attack based on federated learning with dynamic optimization triggers is characterized in that: The step (2) comprises the following steps: (21) Training a benign global model: In each round, the attacker accepts the global model sent down and trains a benign global model locally using the local dataset controlled by the attacker to simulate the global model after the attack stops; (22) Trigger optimization: The attacker uses the trained benign global model and the global model received and sent in this round to minimize the difference in the potential representation of the data implanted with the trigger in the two models. The loss function used is: in, Indicates that the trigger is in the current global model G t The attack task loss is used to balance the ratio of trigger effectiveness and persistence tasks. represents the poisoned data implanted with the trigger, ξ represents the backdoor trigger, and y * Indicates the target poisoning label; Calculated by the following formula: Among them, σ(·) is the activation function of the first activation layer of the model, which is used to represent the potential representation of the trigger-containing data; represents the cross entropy loss; is the simulated attack stop global model; (23) Poisoning training: Use optimized triggers to poison data, and use the poisoned data for local training to obtain the poisoned model.

5. According to claim 1, a method for persistent backdoor attack based on federated learning with dynamic optimization triggers is characterized in that: The step (3) is implemented by the following formula: in, is the loss function of the noise optimization method, represents the variance of the output layer neurons, which is used to reduce the discreteness of the output neuron update energy; the second term of the loss is used to constrain the l2 norm of all added noise and to be 0, m represents the number of output layer neurons; β is used to balance the two objectives of the noise task.

6. According to claim 1, a method for persistent backdoor attack based on federated learning with dynamic optimization triggers is characterized in that: The step (4) comprises the following steps: (41) Find the neuron with the smallest absolute gradient and curvature: Use local data D train = {D benign ,D backdoor }Get approximate gradient and approximate curvature; (42) Construct an index set: Select neurons with the smallest absolute gradient and curvature as indicator neurons and record their index sets As the indicator set, change the parameter value by k times, After modifying the model parameters, upload it to the server as the final poisoned model.

7. According to claim 4, a method for persistent backdoor attack based on federated learning with dynamic optimization triggers is characterized in that: The backdoor trigger ξ described in step (22) is initialized to a pixel block of size 5×5 and a value of 0.5; the optimization process does not change the size of the trigger, but only changes the value of the trigger.

8. According to claim 6, a method for persistent backdoor attack based on federated learning with dynamic optimization triggers is characterized in that: The set size of the index set described in step (42) is equal to the number of attackers, and each attacker corresponds to an index.

9. According to claim 2, a method for persistent backdoor attack based on federated learning with dynamic optimization triggers is characterized in that: The increase is 0.01.

Images