Backdoor robustness evaluation method for non-iid federated learning model based on generative adversarial network

Abstract

The application relates to a non-IID federated learning model backdoor robustness evaluation method based on a generative adversarial network, which comprises the following steps: selecting a client in federated learning as a test node, taking a downloaded server-side global model as a discriminator, and designing a generator locally to form a generative adversarial network model; a backdoor attack target category is specified, a class representative sample is reconstructed by using the generator in each round of global training, and the target category is marked as pre-poisoning data participating in training; a supplementary data set is generated locally; a source category of the backdoor attack is specified, a backdoor trigger is optimized, the supplementary data set is used for class-specific backdoor training, the discriminator is updated and uploaded to the server, and the global model is updated; the non-IID degree of data and the number of specified source categories are adjusted, and the robustness of the federated learning global model to the backdoor attack is observed. Compared with the prior art, the application can verify the effect of the backdoor attack on the federated learning model under different degrees of data heterogeneity.

Description

Technical Field

[0001] This invention relates to the field of network security, and in particular to a method for evaluating the robustness of backdoors in a non-IID federated learning model based on generative adversarial networks. Background Technology

[0002] In recent years, machine learning has achieved remarkable success in many fields, which is inseparable from the support of massive amounts of high-quality training data. However, with increasing attention to privacy protection and the enactment of privacy laws in various countries, much training data is becoming increasingly difficult to circulate, forming "data silos" of distributed datasets. Federated learning is a distributed machine learning framework designed for privacy protection. A federated learning system consists of a central server and multiple clients, enabling clients to coordinate and collaborate to train machine learning models through the central server without sharing local data. Therefore, federated learning can effectively alleviate the conflict between data privacy and availability.

[0003] While federated learning offers better privacy protection compared to traditional centralized learning frameworks, it also exposes new attack surfaces. The most common of these is poisoning attacks launched by malicious clients. Poisoning attacks against federated learning can be categorized into two types based on their objective: non-targeted poisoning and targeted poisoning. The former aims to reduce the overall performance of the model, while the latter aims to manipulate the model to misclassify samples from a specific distribution, while maintaining performance on the original task unaffected. Therefore, targeted poisoning attacks are also known as backdoor attacks. Backdoor attacks can be implemented through two poisoning methods: data poisoning and model poisoning. Data-based backdoor attacks indirectly implant backdoors into the model by manipulating training samples, while model-based backdoor attacks directly mislead the model's learning process to achieve the effect of backdoor implantation.

[0004] Another key and common challenge in federated learning lies in the non-independent, identically distributed (non-IID) nature of data across different clients. Due to the differences in data distribution between local and global datasets, local training can significantly deviate from the global objective. The greater the heterogeneity, the more difficult it is to obtain a global model with the expected performance through model aggregation. Furthermore, data heterogeneity also presents new security challenges for federated learning. Research has found that in non-IID data federated learning scenarios, the feasibility and effectiveness of backdoor attacks are both affected. Summary of the Invention

[0005] The purpose of this invention is to provide a backdoor robustness evaluation method for non-IID federated learning models based on generative adversarial networks, which can verify the effect of backdoor attacks on federated learning models under different levels of data heterogeneity.

[0006] The objective of this invention can be achieved through the following technical solutions:

[0007] A backdoor robustness evaluation method for non-IID federated learning models based on generative adversarial networks includes the following steps:

[0008] Step 1) Select a client in federated learning as a test node, back up the global model downloaded from the server as a discriminator, and design a generator locally to form a generative adversarial network model.

[0009] Step 2) The tester specifies the backdoor attack target category. In each round of global training, the generator is used to reconstruct the class representative samples of all category data participating in the training and mislabel them as the target category. These samples are then used as pre-poisoned data for training. This step is repeated until the generator achieves the expected effect.

[0010] Step 3) Use the trained generator to generate a supplementary dataset offline locally to improve the imbalanced data distribution of a single client.

[0011] Step 4) The tester specifies the source category of the backdoor attack, optimizes the backdoor trigger to obtain the enhanced trigger, and uses the supplementary dataset to train the category-specific backdoor based on the enhanced trigger or random trigger. The discriminator after the malicious attack is updated and uploaded to the server, and the global model is updated on the server side.

[0012] Step 5) Adjust the non-IID level of the federated learning data and the number of source categories specified by the backdoor attack, and observe the robustness to backdoor attacks on the global federated learning model.

[0013] In step 1), the generator G corresponding to the discriminator D is designed locally using an interpolation-based generation structure. The optimization process follows the following objective function:

[0014]

[0015] Where x represents the distribution of the true sample p real The sample obtained from (x) is z, which represents the sample obtained from the Gaussian distribution p. z Random noise sampled from (z).

[0016] The condition for determining whether the generator has achieved the expected results in step 2) is: if the confidence level of the downloaded global model in classifying the generated samples reaches the expected threshold, then the generator is considered to have completed training and the iteration stops.

[0017] The samples generated by the generator after training in step 3) simultaneously meet the requirements of realism and diversity. Diversity includes category diversity and sample diversity. Category diversity means that the generated samples are evenly distributed in the global data categories, while sample diversity means that the samples in the same category show obvious differences from each other.

[0018] The reinforcement trigger in step 4) is a backdoor trigger that is generated using the projected gradient descent algorithm or the fast gradient descent algorithm and only applies to the specified category.

[0019] The objectives of the category-specific backdoor training include main task accuracy, attack success rate, and unspecified class accuracy, wherein the unspecified class accuracy is the accuracy at which a category not specified by the attacker can still be correctly classified after a backdoor trigger is added.

[0020] The category-specific backdoor training adopts a phased multi-task training approach. First, the attack success rate of the specified source category is used as the training objective. Then, the accuracy rate of the original main task is used as the training objective. Finally, robust training is performed with the immunity of non-specified source categories to backdoors as the training objective.

[0021] The method for controlling the non-IID level of federated learning data in step 5) is to control the number of data categories obtained by each client, while keeping the total amount of data consistent with the amount of data in each category.

[0022] The minimum number of data categories obtained by each client is 1, and the maximum is the total number of global categories.

[0023] The workflow of federated learning in each global training round is divided into the following three stages: 1) Local training: Each participating node receives the global model from the server, performs iterative training using several data batches, and obtains updated model parameters; 2) Parameter sharing: Nodes calculate the difference between the new model parameters and the old model parameters as model updates and upload them to the server; 3) Parameter merging: The server collects the model parameters of all participants and uses the following federated aggregation algorithm to obtain global model parameters for the next round of model distribution.

[0024]

[0025] in Let $\mathbf{T}$ represent the global model parameters in the $t$ round of federated learning and the model update from the $i$-th client, respectively, where $K$ is the total number of clients. This represents the ratio of the dataset size of the i-th client to the global dataset size.

[0026] Compared with the prior art, the present invention has the following beneficial effects:

[0027] (1) This invention verifies the security threats of federated learning in non-IID data scenarios, making up for the lack of research on the security of federated learning. Unlike traditional backdoor poisoning, this invention takes a different approach by using generative adversarial networks to reconstruct locally missing data in order to control the scope of poisoning, which will provide a new idea for building a covert backdoor attack and defense mechanism in highly heterogeneous federated learning scenarios that are more in line with reality.

[0028] (2) This invention proposes a specific robustness evaluation method for backdoor threats in federated learning models. It can use generative adversarial networks to reconstruct high-quality alternative data on data-constrained client nodes, thereby testing the robustness of model backdoors under different data heterogeneity and different source category ranges.

[0029] (3) This invention not only verifies the feasibility of backdoor attacks in non-IID federated learning scenarios, but also evaluates their robustness under different degrees of heterogeneity and attack range, providing support for improving the awareness of the threat of backdoors in federated learning models and enhancing artificial intelligence security awareness. Attached Figure Description

[0030] Figure 1 This is a flowchart of the method of the present invention;

[0031] Figure 2 This is a schematic diagram of the specific backdoor implantation process in non-IID federated learning.

[0032] Figure 3 A diagram illustrating the training process for federated learning and a backdoor attack model;

[0033] Figure 4 This diagram illustrates the backdoor effect of the model under different levels of data heterogeneity in federated learning.

[0034] Figure 5 This is a schematic diagram illustrating the backdoor effect of the model under different source categories and their scope of influence. Detailed Implementation

[0035] The present invention will now be described in detail with reference to the accompanying drawings and specific embodiments. These embodiments are based on the technical solution of the present invention and provide detailed implementation methods and specific operating procedures. However, the scope of protection of the present invention is not limited to the following embodiments.

[0036] This embodiment provides a method for evaluating the backdoor robustness of a non-IID federated learning model based on generative adversarial networks, such as... Figure 1 As shown, it includes the following steps:

[0037] Step 1) Select a client in federated learning as a test node, back up the downloaded global model from the server as a discriminator, and design a generator locally to form a generative adversarial network model.

[0038] The discriminator in this embodiment differs from traditional generative adversarial networks. It is a federated learning global model, which is updated with global training and is not only affected by the generator. This allows the generator to learn the distribution of global training data.

[0039] This embodiment uses an interpolation-based generation structure to design a generator G corresponding to the discriminator D. The optimization process follows the following objective function:

[0040]

[0041] Where x represents the distribution of the true sample p real The sample obtained from (x) is z, which represents the sample obtained from the Gaussian distribution p. z Random noise sampled from (z).

[0042] Step 2) The tester specifies the backdoor attack target category. In each round of global training, the generator is used to reconstruct class representative samples of all category data participating in the training and mislabel them as the target category. These samples are then used as pre-poisoned data for training. This step is repeated until the generator achieves the expected effect.

[0043] The pre-poisoning strategy used in this step provides negative samples for the global model's discrimination task, prompting it to learn the ability to distinguish between real and fake samples, thereby better guiding the generator's generation task. The method of action is to feed the incorrectly labeled samples generated by the generator into the local training to generate toxic updates and upload them to the server.

[0044] During step 2), the tester iterates through the pre-poisoning and sample generation operations in each round of federated learning and checks the quality of the generated samples in each round. If the classification confidence of the downloaded global model for the generated samples reaches the expected threshold, the generator is considered to have been trained and the iteration stops; otherwise, step 2 will be repeated until the standard is met.

[0045] The generator training process in this step differs from that of traditional generative adversarial networks. It is not only influenced by feedback from the discriminator (global model), but also by statistical information of the generated samples and the activation values ​​of the generator model's hidden layers. The aim is to guide the generator to produce samples that are both realistic and diverse in order to address the data missing problem in non-IID federated learning.

[0046] Step 3) Use the trained generator to generate a supplementary dataset offline locally to improve the imbalanced data distribution of a single client.

[0047] The generated samples from the trained generator simultaneously meet the requirements of realism and diversity, forming a supplementary dataset to improve the local data distribution and provide data for subsequent backdoor testing. The diversity includes class diversity and sample diversity. Class diversity refers to the uniform distribution of generated samples among global data classes, while sample diversity refers to the significant differences between samples within the same class.

[0048] Step 4) The tester specifies the source category of the backdoor attack, optimizes the backdoor trigger to obtain an enhanced trigger, and uses a supplementary dataset to train a category-specific backdoor based on the enhanced trigger or random trigger. The discriminator update after the malicious attack is uploaded to the server, and the global model is updated on the server side.

[0049] The reinforcement trigger is a backdoor trigger generated using the projective gradient descent algorithm that only applies to a specified class and is used in subsequent source class-specific backdoor training. In another embodiment, other algorithms for generating adversarial examples, such as the fast gradient descent algorithm, can also be used.

[0050] Traditional backdoor attacks only specify the target category, while source category specificity refers to further specifying the source category for the attacker. Only the specified category will be affected by the backdoor attack, while the unspecified category will be immune to the backdoor.

[0051] The objectives of the category-specific backdoor training include main task accuracy, attack success rate, and unspecified class accuracy, wherein the unspecified class accuracy is the accuracy at which a category not specified by the attacker can still be correctly classified after a backdoor trigger is added.

[0052] The category-specific backdoor training adopts a phased multi-task training approach. First, the attack success rate of the specified source category is used as the training objective. Then, the accuracy rate of the original main task is used as the training objective. Finally, robust training is performed with the immunity of non-specified source categories to backdoors as the training objective.

[0053] Step 5) Adjust the non-IID level of the federated learning data and the number of source categories specified by the backdoor attack, and observe the robustness to backdoor attacks on the global federated learning model.

[0054] The method to control the non-IID degree of federated learning data is to control the number of data categories obtained by each client, while keeping the total amount of data and the amount of data in each category consistent (that is, to divide the evenly distributed global dataset equally among each client). The minimum number of data categories obtained by each client is 1, and the maximum is the total number of global categories.

[0055] The architecture and vulnerabilities of federated learning, such as Figure 2As shown, the workflow of federated learning in each global training round consists of the following three stages: 1) Local training: Each participating node receives the global model from the server, iterates through several data batches to obtain updated model parameters; 2) Parameter sharing: Nodes calculate the difference between the new and old model parameters and upload it to the server as model updates; 3) Parameter merging: The server collects the model parameters from all participants and uses the following federated aggregation algorithm to obtain global model parameters for the next round of model distribution:

[0056]

[0057] in Let $\mathbf{T}$ represent the global model parameters in the $t$ round of federated learning and the model update from the $i$-th client, respectively, where $K$ is the total number of clients. This represents the ratio of the dataset size of the i-th client to the global dataset size.

[0058] However, due to the inherent vulnerabilities of federated learning, each stage faces corresponding attack threats: during the local training stage, there is a data poisoning threat caused by untrusted nodes or external attackers; during parameter sharing, there is a model poisoning threat caused by untrusted nodes or external attackers; and during parameter merging, there is a threat of global model tampering and privacy theft executed by malicious servers.

[0059] Specifically, such as Figure 3 As shown, based on a survey and analysis of security threats in federated learning, this invention designs a backdoor robustness evaluation method for non-IID federated learning models based on generative adversarial networks. By changing the degree of data heterogeneity and the range of backdoor attack categories in federated learning, the feasibility and robustness of launching backdoor attacks with higher concealment in real-world scenarios are evaluated. The scheme consists of two main stages: data reconstruction and backdoor implantation. First, the missing data categories are reconstructed locally. Then, backdoors with source category specificity are implanted based on the reconstructed data and the original data. The necessary steps included in this process are:

[0060] Generative Adversarial Network Construction: The client backs up the global model received in each round as a discriminator D; then, a corresponding generator G is designed using an interpolation-based generative structure.

[0061] Pre-poisoning process: The fake samples generated by the current generator in each round are incorrectly labeled and used as poisoned data for training, providing negative samples for the global model's discrimination task.

[0062] Supplementary dataset construction: Reconstruct a uniformly distributed global dataset using the trained generator to provide a data source for subsequent backdoor attack tests.

[0063] Trigger optimization: Based on the supplementary dataset, the random triggers are optimized using the projective gradient descent algorithm so that they only act on specific source classes.

[0064] Source category-specific backdoor training: A phased multi-task training approach is adopted. First, the attack success rate of a specified source category is used as the training target. Then, the accuracy rate of the original main task is used as the training target. Finally, robust training is performed with the immunity of non-specified source categories to backdoors as the training target.

[0065] Based on the above description, this invention, for the first time, implements a backdoor robustness evaluation method for non-IID federated learning models based on generative adversarial networks. It verifies the feasibility of backdoor attacks in non-IID federated learning scenarios and evaluates their robustness under different degrees of heterogeneity and attack scope, thus filling the gaps in research on the security of federated learning. Figure 4 Experimental results show that the degree of data heterogeneity in non-IID federated learning has a significant impact on the effectiveness of backdoor attacks. After data reconstruction, the experiment on controlling the source category scope of the backdoor is as follows: Figure 5 As shown, in three public datasets, setting the scope of the source category to 5, 3, and 1 respectively reveals that the backdoor attacks exhibit stronger targeting and stealth. Combined with non-IID federated learning scenarios, this type of backdoor attack is more feasible in its attack assumptions and more covert in its attack effects, providing a new approach to improving awareness of the threat posed by backdoors in federated learning models and enhancing AI security awareness.

[0066] The preferred embodiments of the present invention have been described in detail above. It should be understood that those skilled in the art can make numerous modifications and variations based on the concept of the present invention without creative effort. Therefore, all technical solutions that can be obtained by those skilled in the art based on the concept of the present invention through logical analysis, reasoning, or limited experimentation on the basis of existing technology should be within the scope of protection defined by the claims.

Claims

1. A method for evaluating the backdoor robustness of a non-IID federated learning model based on a generative adversarial network, characterized in that, The method comprises the following steps: Step 1) selecting a client in federated learning as a test node, backing up the downloaded global model of the server as a discriminator, and designing a generator locally to form a generative adversarial network model; Step 2) the tester specifies a backdoor attack target class, and in each round of global training, the generator is used to reconstruct the class representative samples of all class data participating in training, and mark the errors as the target class as pre-poisoning data participating in training, and the step is repeated until the generator reaches the expected effect; Step 3) using the trained generator to generate a supplementary data set locally offline to improve the unbalanced data distribution of a single client; Step 4) the tester specifies the source class of the backdoor attack, optimizes the backdoor trigger to obtain an enhanced trigger, and based on the enhanced trigger or a random trigger, uses the supplementary data set to perform class-specific backdoor training, updates the discriminator after the malicious attack to the server, and updates the global model on the server; Step 5) adjusting the non-IID degree of federated learning data and the number of source classes specified by the backdoor attack, and observing the robustness of the global model of federated learning to the backdoor attack.

2. The non-IID federated learning model backdoor robustness evaluation method based on a generative adversarial network according to claim 1, wherein In step 1), the generator is designed locally based on an interpolation-based generation structure, and a generator G corresponding to the discriminator D is designed, and the optimization process follows the following objective function: where x represents a sample drawn from the real sample distribution p real (x) and z represents random noise drawn from the Gaussian distribution p z (z).

3. The non-IID federated learning model backdoor robustness evaluation method based on a generative adversarial network according to claim 1, wherein In step 2), the judgment condition for the generator to reach the expected effect is that if the classification confidence of the downloaded global model for the generated samples reaches the expected threshold, the generator is considered to be trained, and the iteration is stopped.

4. The non-IID federated learning model backdoor robustness evaluation method based on a generative adversarial network according to claim 1, wherein In step 3), the samples generated by the trained generator meet the requirements of authenticity and diversity, wherein the diversity includes class diversity and sample diversity, the class diversity means that the generated samples are uniformly distributed in the global data classes, and the sample diversity means that the samples in the same class are significantly different from each other.

5. The non-IID federated learning model backdoor robustness evaluation method based on a generative adversarial network according to claim 1, wherein In step 4), the enhanced trigger is a backdoor trigger generated by using a projection gradient descent algorithm or a fast gradient descent algorithm, and only the specified class is effective.

6. The non-IID federated learning model backdoor robustness evaluation method based on a generative adversarial network according to claim 1, wherein The target of the class-specific backdoor training includes the main task accuracy, the attack success rate and the non-specified class accuracy, wherein the non-specified class accuracy is the accuracy of the classes not specified by the attacker that can still be correctly classified after the backdoor trigger is added.

7. The non-IID federated learning model backdoor robustness evaluation method based on a generative adversarial network according to claim 6, wherein The class-specific backdoor training adopts a multi-task training method in stages, first taking the attack success rate of the specified source class as the training target, then taking the original main task accuracy as the training target, and finally taking the immunity of the non-specified source class to the backdoor as the training target for robust training.

8. The non-IID federated learning model backdoor robustness evaluation method based on a generative adversarial network according to claim 1, wherein In step 5), the method for controlling the non-IID degree of federated learning data is to control the number of data classes obtained by each client, while keeping the total data amount and the data amount of each class consistent.

9. The method of claim 8, wherein the method is based on a generative adversarial network (GAN) for non-IID federated learning model backdoor robustness evaluation. The number of data classes obtained by each client is at least 1 and at most the total number of global classes.

10. The non-IID federated learning model backdoor robustness evaluation method based on a generative adversarial network according to claim 1, wherein The workflow of the federated learning in each global training round is divided into the following three stages: 1) local training: each learning participating node receives the global model issued by the server, iteratively trains using several data batches, and obtains updated model parameters; 2) parameter sharing: the node calculates the difference between the new model parameters and the old model parameters as the model update and uploads to the server; 3) parameter merging: the server collects all the model parameters of the participants, and uses the following federated aggregation algorithm to obtain the global model parameters for the next round of model distribution: wherein represent the global model parameters of the t-th round of federated learning and the model update from the i-th client, respectively, and K is the total number of clients, represents the ratio of the data set size of the i-th client to the global data set size.

Images