Optimizing the performance evaluation method of federated learning model defense mechanism under backdoor attacks

Abstract

The present invention discloses a method for evaluating the performance of a federated learning model defense mechanism under optimized backdoor attacks, and belongs to the field of network security. The method comprises: in the early stage of federated learning training, the attacker participates in the training normally; in the middle stage of training, the attacker obtains the global model; while participating in the training normally, the attacker uses the obtained global model in advance to simulate normal training and backdoor model training, and optimizes the trigger in advance by using the update difference; in the late stage of training, the optimized trigger is used to carry out a backdoor attack, and the global model obtained again using the same method is used to continuously optimize the trigger, thereby improving the attack effect and efficiency; a backdoor attack test is carried out based on the final optimized trigger to evaluate the performance of the federated learning model defense mechanism. The method for evaluating the vulnerability of a federated learning model under optimized backdoor attacks proposed by the present invention proposes a new set of attack ideas and effective evaluation methods for existing defenses, revealing the loopholes in existing defenses.

Description

Technical Field

[0001] The present invention relates to the field of network security, and in particular to a method for evaluating the performance of a federated learning model defense mechanism under optimized backdoor attacks. Background Art

[0002] In today's digital age, federated learning, as an emerging distributed machine learning technology, is gradually showing great potential in various fields. It allows multiple participants to jointly train models without sharing the original data, thereby protecting data privacy.

[0003] However, this technology also faces security threats, with federated learning backdoor attacks becoming a significant concern. A federated learning backdoor attack involves an attacker implanting a specific "backdoor" in a federated learning system by tampering with data, optimizing triggers, or disrupting the model update process. This allows attackers to manipulate model outputs under specific conditions to achieve malicious objectives. While some adaptive attack methods have been developed, backdoor attack methods are constantly evolving and highly stealthy in federated learning environments. Attackers can cleverly inject malicious samples with specific trigger patterns into local training data. To combat backdoor attacks, researchers have proposed numerous defense mechanisms. These include robust aggregation algorithms designed to identify and filter out anomalous model updates; improved solutions based on differential privacy techniques that enhance resistance to malicious data injection while maintaining model training effectiveness; and solutions based on trusted execution environments (TEEs) that secure the model training process at the hardware level. However, a systematic, comprehensive, and effective method for evaluating the performance of these defense mechanisms is currently lacking. Different evaluation metrics have their advantages and disadvantages in different scenarios, with some focusing on detection accuracy and others on the impact on normal model training. Without a unified evaluation method, it's difficult to accurately compare the pros and cons of different defense mechanisms, nor can it determine the applicability of a particular defense mechanism in complex real-world environments. This hinders the further optimization and widespread application of defense technologies. In summary, addressing the current lack of unified evaluation methods and metrics, we propose a method for optimizing the performance of federated learning models' defenses against backdoor attacks, addressing this gap in the field. Summary of the Invention

[0004] In order to overcome the problems in the prior art, the purpose of the present invention is to provide a method for evaluating the performance of a federated learning model defense mechanism under optimized backdoor attacks to evaluate the performance of a defense mechanism based on a detection algorithm.

[0005] This paper presents a method for evaluating the performance of a federated learning model defense mechanism under optimized backdoor attacks. First, after understanding the detection mechanism, the method optimizes triggers to reduce the distribution difference between the poisoned and original datasets, thereby reducing the difference between normal and malicious updates and circumventing the defense. The attack also employs a pre-optimization strategy, leveraging the global model mid-training to pre-optimize triggers. The attacker then conducts a backdoor attack in the later stages of training, alternating between trigger optimizations to adapt to the dynamic changes in the global model. Repeatable experiments are then used to more accurately evaluate the performance of the defense method.

[0006] In order to achieve the above object, the specific technical solutions adopted by the present invention are as follows:

[0007] On the one hand, the present invention proposes a method for evaluating the performance of a federated learning model defense mechanism under optimized backdoor attacks, comprising:

[0008] S1: Establish a federated learning evaluation system consisting of an initiator and multiple participants. The initiator owns the global model to be evaluated, and the participants each own a local training dataset for training the global model. One of the participants is defined as the attacker, and the total number of training rounds, initial training rounds, and mid-term training rounds are preset. During the federated learning training process between the initiator and the participants, a defense mechanism for resisting backdoor attacks to be evaluated is provided.

[0009] S2: Each participant performs initial federated learning training on the initiator's global model based on the local training dataset until the initial training round is reached to obtain the initial global model;

[0010] S3: Each participant conducts mid-term federated learning training on the initial global model based on the local training dataset until the mid-term training round is reached, resulting in a mid-term global model. During the mid-term federated learning training, the attacker initializes the trigger and adds the trigger to the local training dataset. The attacker uses the training dataset without the trigger to participate in the federated learning training. At the same time, the global model of each round of federated learning training is backed up and the backup model is trained in parallel using the training dataset with the trigger. Based on the difference in model parameter updates between the two trainings, the trigger parameters are optimized and updated using an adaptive optimization method.

[0011] S4: Each participant conducts subsequent federated learning training on the mid-term global model based on the local training dataset until the total number of training rounds is reached. During the subsequent federated learning training, the attacker adds the current trigger to the local training dataset and uses the training dataset with the added trigger to participate in the federated learning training. At the same time, the global model of each round of federated learning training is backed up and the training dataset without the added trigger is used for parallel training on the backup model. Based on the difference in model parameter updates between the two trainings, the trigger parameters are optimized and updated using an adaptive optimization method.

[0012] S5: Build a test dataset based on the final trigger, test the trained global model, calculate the accuracy of the model output, and evaluate the performance of the defense mechanism based on the accuracy.

[0013] Furthermore, the global model to be trained is an image recognition model, and the training data set includes image samples and their labels; adding a trigger to the local training data set refers to modifying the image samples in the training data set according to the trigger parameters.

[0014] Furthermore, each round of federated learning training includes: distributing the global model to each participant, each participant training the global model locally on a local training dataset, uploading the updated model parameters after local training to the initiator, and the initiator updating the global model based on all uploaded model parameters;

[0015] Furthermore, the defense mechanism includes: during each round of federated learning training, the initiator detects and filters the model parameters uploaded by each participant before comprehensively updating the global model with all the uploaded model parameters, and updates the global model with the comprehensively filtered model parameters. The updated model is the trained global model.

[0016] Furthermore, the federated learning training specifically includes: the attacker uses the training data set participating in the federated learning training to perform local training on the currently received global model, and uploads the updated model parameters to the initiator after each round of local training; the parallel training specifically includes: in each round of local training, the attacker uses the training data set participating in the federated learning training to perform local training on the backup global model, and the updated model parameters are not uploaded to the initiator; the rounds of parallel training are synchronized with the rounds of federated learning training.

[0017] Furthermore, in step S6, the backdoor attack test is performed on the federated learning model based on the final trigger, specifically by obtaining a test data set, adding the final trigger to the test data set, and using the test data set with the added trigger to test the federated learning model.

[0018] On the other hand, the present invention also proposes a performance evaluation system for optimizing the federated learning model defense mechanism under backdoor attacks to implement the above method.

[0019] Compared with the prior art, the beneficial effects of the present invention include at least:

[0020] (1) By optimizing the triggers and understanding the mechanism of the detection method, the distribution difference between the poisoned data set and the original data set is reduced, thereby reducing the difference between normal updates and malicious updates, thereby successfully evading the defense. This creates conditions for more accurate analysis of the defense mechanism's response to attacks.

[0021] (2) The attack adopts an early optimization strategy, using the global model in the middle of training to optimize the trigger in advance. At the same time, the attacker chooses to conduct a backdoor attack in the late stage of training. In order to adapt to the dynamic changes of the global model, the triggers are optimized alternately. This innovative attack strategy makes the evaluation process more suitable for actual complex attack scenarios. In addition, through repeated experiments, the performance of the defense method can be more accurately evaluated, thereby providing more reliable data support for improving and optimizing the defense mechanism. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0023] Figure 1 This is a flowchart of the performance evaluation method of the federated learning model defense mechanism provided in this embodiment;

[0024] Figure 2 This is a schematic diagram of the trigger optimization process based on model update differences provided in this embodiment. DETAILED DESCRIPTION

[0025] In order to make the purpose, technical solutions and advantages of the present invention more clearly understood, the present invention and the reverse operation are further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and do not limit the scope of protection of the present invention.

[0026] In view of the current situation in which academia and industry are deficient in evaluating the performance of federated learning backdoor attack defense, this embodiment provides an evaluation method based on advance optimization of triggers. This method is committed to accurately evaluating the performance of defense mechanisms in responding to backdoor attacks, thereby providing a reliable basis for measuring the effectiveness of existing defense mechanisms. Specifically, by carefully optimizing the triggers and fully considering the model structure and data characteristics of federated learning, more complex triggers that are difficult to be identified by existing defense mechanisms are designed. During the federated learning process, it is observed whether the defense mechanism can effectively detect when an attack is launched using this optimized trigger. At the same time, combined with adaptive attack strategies, dynamic changes in real scenarios are simulated, and the attack method is adjusted according to the changes to test the performance of the defense mechanism in complex attack situations. In addition, by deeply analyzing the backdoor attack defense mechanism, it is evaluated whether the defense mechanism can accurately identify and defend when facing attack methods that specifically reduce the risk of detection, so as to determine whether it can effectively respond to backdoor attacks during training, thereby completing a comprehensive evaluation of the defense performance. As Figure 1 As shown in FIG, the flowchart of the method for optimizing the performance evaluation of the federated learning model defense mechanism under backdoor attacks provided in this embodiment includes the following steps:

[0027] Step 1: Establish a performance evaluation system for a federated learning model defense mechanism, including an initiator and multiple participants, one of whom is the attacker. The initiator possesses a global model to be trained, and the participants possess their own training datasets. In this embodiment, the global model to be trained is an image recognition model, and the training dataset includes image samples and their labels. A total number of training rounds, initial training rounds, and mid-term training rounds are predefined, dividing the federated learning training process into initial, mid-term, and final stages. A round specifically refers to the number of times a participant exchanges model parameters with the initiator.

[0028] In this embodiment, the central server is the initiator, and the training data set of the participants is obtained by splitting and distributing a large data set. The splitting is performed according to Dirichlet sampling Dir(α) to simulate the situation in which the distribution of the data sets of each participant is not independent and different in actual situations. Suppose N:{P1,P2,…P N} participants, the server initializes the global model parameters θ 0 , and sent to the participants.

[0029] Step 2: The attacker participates in the initial federated learning training as normal, allowing the model to access data.

[0030] In the initial stages of federated learning, the attacker joins the training process as a normal participant. This step is crucial because it lays the foundation for subsequent attack operations. Through training, the model is fully exposed to the data and begins to understand its characteristics and distribution. In this embodiment, the federated learning process includes: the server distributes the current global model, the participants perform local training and upload model updates, the server updates the model according to the aggregation rules, and distributes the updated model to each participant. This constitutes a normal round of federated learning training. In this embodiment, each participant performs two rounds of local training during each round of federated learning training.

[0031] Participants in the local dataset D i Training is performed by minimizing the loss function L i (θ) Update local model parameters:

[0032]

[0033] Where η is the learning rate, and The local model parameters before and after the update of the t+1th round of training for the i-th participant, Represents the loss function of the model. After completing the initial federated learning training, each participant uploads the locally trained model parameters (or model updates) to the central server, which updates the new global model parameters based on the aggregation rule, such as FedAvg:

[0034]

[0035] where θ t+1 The server uses the parameters of the global model updated after t+1 rounds of training by each participant.

[0036] Step 3: In the middle of training, the attacker obtains the global model and optimizes the triggers while continuing to participate in training normally, thus preparing for subsequent attacks.

[0037] Step 3-1: After the server updates the global model parameters, the attacker obtains the global model for parallel training. This global model is the result of the initial training of all participants. It contains important information about the data and the structure of the model. For example, when the total number of global training rounds is set to 220, the initial federated learning training ends at round 100. The server updates the global model parameters, and the attacker obtains the global model parameters θ of the 100th round. 100 .

[0038] Step 3-2: The attacker continues to participate in the training as normal. The attacker continues to participate in the federated learning training as a normal participant. This can reduce the impact on the normal federated learning training.

[0039] Step 3-3: While the attacker continues training, they begin optimizing the trigger using the acquired global model. Adding triggers to the local training dataset involves modifying image samples in the training dataset based on the trigger parameters. Triggers are a core component of backdoor attacks, determining the conditions for triggering the backdoor and the stealthiness of the attack. Through parallel training, the attacker can assign optimal trigger parameters, making them more stealthy and effective in subsequent simulated attacks.

[0040] Specifically, the attacker simulates normal training and parallel training, fine-tuning the trigger parameters based on the differences in model parameters obtained from the two trainings. The parallel training in step 3 is backdoor training, which requires data contamination before local training using the contaminated dataset.

[0041] Data contamination means that the attacker will modify some data in the local dataset to inject a backdoor. For example, the normal local dataset is The attacker will select some data samples, such as k samples {x j1 ,y j1}, {x j2 ,y j2},…,{x jk ,y jk}, and modify its label to the attack target label. The modified local dataset is the contaminated dataset. For example, in an image classification task, if the normal label is "cat", the attacker may modify some selected labels to "dog". At the same time, some subtle and imperceptible disturbances are added to these images through triggers. In this embodiment, the trigger parameter is defined as trigger, and the original image pixel value is image. Then the image pixel value after adding the disturbance is 0.9*image+0.1*trigger. The specific way to add a trigger can also be to inject pixel blocks into the image sample or add image filters. In this case, the trigger parameters include injection position, pixel block size, etc. The attacker then backs up the global model obtained during the current round of federated learning training, uses the dataset for local training, and updates the model parameters of the backup model:

[0042]

[0043] in, is the backup model parameter updated during the t+1th round of training, θ t is the global model parameter obtained by the parameter in the t+1th round of training, is the gradient, L poison (θ t ) is the loss function.

[0044] The optimization process for triggers is as follows Figure 2As shown in the figure, the received global model is used to perform normal federated local training and backdoor training on the clean dataset and the contaminated dataset respectively. After each round of federated local training and backdoor training updates the model parameters, the trigger is optimized based on the difference between the two updates:

[0045]

[0046] in is the gradient of the optimization method, The difference in updating parameters for normal training and backdoor training models, and is the trigger value, α is the learning rate, and cosine similarity is used here to measure the update difference

[0047]

[0048] Where cosine_similarity(·) represents cosine similarity.

[0049] Step 4: The attacker alternates between backdoor attacks and trigger optimization in the later stages of training. When training reaches a late stage, such as epoch 200 (out of a total of 220 epochs), the attacker begins the backdoor attack. This phase is crucial for the attack, requiring the attacker to fully leverage previous preparations to execute an effective backdoor attack. During the backdoor attack, the attacker continuously optimizes the trigger based on the progress of federated learning and changes in the model, ensuring that it remains highly effective.

[0050] Specifically, the backdoor attack involves the attacker adding triggers to their own training dataset during federated local training, rather than using a clean dataset (one without triggers). This creates a contaminated dataset, which they then train on. While conducting the backdoor attack, the attacker continues parallel training on the clean dataset, synchronizing this training with the federated local training to optimize trigger updates.

[0051] The model parameters are updated in each round when performing normal training and backdoor attack on the clean dataset and the contaminated dataset respectively as follows:

[0052]

[0053]

[0054] in are the model parameters after the tth round of training on the poisoned dataset, is the gradient, L poison (.) is the loss function.

[0055]

[0056]

[0057] where y i is the true label, is the model prediction value, n is the number of training samples in this round of training; L poison (θ t ) has the same form, where y i ′ is the target label. The method for optimizing trigger parameters in this step is the same as that described in step 3. Both methods optimize trigger parameters based on the difference in model parameter updates between the two training steps.

[0058] Step 5: During the final round of federated learning training, the initiator uses an anomaly detection algorithm to detect and filter the model parameters uploaded by each participant. The filtered model parameters are then used to update the parameters of the global model. The updated model is the trained global model.

[0059] The anomaly detection algorithm includes clustering methods such as HDBACN and PCA.

[0060] Step 6: Perform backdoor attack tests on the trained global model based on the final trigger, and evaluate the performance of the defense mechanism based on the test results.

[0061] In this example, a test dataset is first obtained. The final optimized trigger is added to the test dataset. The federated learning model is then tested using the test dataset with the added trigger. The number of times the model outputs normal and abnormal is recorded. The percentage of normal outputs is calculated as the attack success rate, with a preset threshold of ε = 50%. If the attack success rate is greater than 50%, the model's defense mechanism is considered to be performing well and not susceptible to backdoor attacks. Otherwise, the model's defense mechanism is considered to be performing poorly and susceptible to backdoor attacks.

[0062] This approach, designed based on an in-depth analysis of existing defense methods, detects anomalies in uploaded model updates. The server detects anomalies in these updates. The differences between updates are essentially due to differences in the distribution of the datasets. For example, when the poisoning rate increases, malicious updates are more easily detected. Due to the high-dimensional continuous nature of the data and the low-dimensional discrete nature of the data labels, the dimensionality difference is significant, making it difficult to directly and accurately establish their joint distribution.

[0063] However, during the training process, the model itself is also learning the joint distribution of data. Even when it is in a convergent state, its model update can capture the differences between data distributions. In particular, when the data distribution tends to be consistent, the model update difference also tends to 0. Based on this, the attacker can directly use the received global model to simulate normal training and backdoor training, thereby calculating their differences to optimize the trigger. And use the gaps in training to optimize the trigger in advance, giving the trigger a good initial value, thereby improving the efficiency of the subsequent backdoor attack. Taking into account this backdoor attack method, the present invention proposes the above-mentioned evaluation method to achieve a more targeted vulnerability assessment, which will help relevant researchers take effective defense measures to ensure the safe and reliable operation of the federated learning system, and provide a solid guarantee for the safe use and innovative development of data.

[0064] In summary, this method provides new ideas and methods for detecting and defending backdoor attacks in federated learning, and is expected to play an important role in practical applications.

[0065] This embodiment also provides a system for optimizing the performance evaluation of a federated learning model defense mechanism under backdoor attacks, which is used to implement the above-mentioned embodiment. The terms "module", "unit", etc. used below can be a combination of software and / or hardware that implements a predetermined function. Although the system described in the following embodiments is preferably implemented in software, it is also possible to implement hardware, or a combination of software and hardware. The system includes:

[0066] The federated learning training module includes an initiator and multiple participants for conducting federated learning training, where one participant uses a trigger to implement a backdoor attack in the later stage of the federated learning training;

[0067] The trigger optimization module is used to initialize and optimize triggers during federated learning training.

[0068] The defense module is used to detect and filter the model parameters obtained after local training of each participant;

[0069] The test and evaluation module builds a test dataset based on the final trigger, and uses the test dataset to test the trained global model and evaluate the performance of the defense mechanism.

[0070] As for the system embodiment, since it basically corresponds to the method embodiment, the relevant parts can be referred to the partial description of the method embodiment, and the implementation methods of the remaining modules will not be repeated here. The system embodiment described above is only illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the solution of the present invention. Ordinary technicians in this field can understand and implement it without paying any creative work.

[0071] Embodiments of the system of the present invention can be applied to any device with data processing capabilities, such as a computer or other device. System embodiments can be implemented through software, hardware, or a combination of software and hardware. For example, a software implementation, as a logical device, is implemented by a processor of any device with data processing capabilities, reading corresponding computer program instructions from non-volatile memory into internal memory and executing them.

[0072] The above-described embodiments merely illustrate several implementations of the present invention, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the present invention. Persons skilled in the art will readily appreciate that variations and modifications may be made without departing from the scope of the present invention, all of which fall within the scope of protection of the present invention.

Claims

1. A method for evaluating the performance of a federated learning model defense mechanism under optimized backdoor attacks, characterized in that: include: S1: Establish a federated learning evaluation system consisting of an initiator and multiple participants. The initiator owns the global model to be evaluated, and the participants each own a local training dataset for training the global model. One of the participants is defined as the attacker, and the total number of training rounds, initial training rounds, and mid-term training rounds are preset. During the federated learning training process between the initiator and the participants, a defense mechanism for resisting backdoor attacks to be evaluated is provided. S2: Each participant performs initial federated learning training on the initiator's global model based on the local training dataset until the initial training round is reached to obtain the initial global model; S3: Each participant conducts mid-term federated learning training on the initial global model based on the local training dataset until the mid-term training round is reached, resulting in a mid-term global model. During the mid-term federated learning training, the attacker initializes the trigger and adds the trigger to the local training dataset. The attacker uses the training dataset without the trigger to participate in the federated learning training. At the same time, the global model of each round of federated learning training is backed up and the backup model is trained in parallel using the training dataset with the trigger. Based on the difference in model parameter updates between the two trainings, the trigger parameters are optimized and updated using an adaptive optimization method. S4: Each participant conducts subsequent federated learning training on the mid-term global model based on the local training dataset until the total number of training rounds is reached. During the subsequent federated learning training, the attacker adds the current trigger to the local training dataset and uses the training dataset with the added trigger to participate in the federated learning training. At the same time, the global model of each round of federated learning training is backed up and the training dataset without the added trigger is used for parallel training on the backup model. Based on the difference in model parameter updates between the two trainings, the trigger parameters are optimized and updated using an adaptive optimization method. S5: Build a test dataset based on the final trigger, test the trained global model, calculate the accuracy of the model output, and evaluate the performance of the defense mechanism based on the accuracy.

2. The method for evaluating the performance of a federated learning model defense mechanism under optimized backdoor attacks according to claim 1, characterized in that: The global model is an image recognition model, and the training data set includes image samples and their labels; adding a trigger to the local training data set refers to modifying the image samples in the training data set according to the trigger parameters.

3. The method for evaluating the performance of a federated learning model defense mechanism under optimized backdoor attacks according to claim 1, wherein: Each round of federated learning training includes: sending the global model to each participant, each participant performing local training on the global model on the local training dataset, uploading the updated model parameters after local training to the initiator, and the initiator updating the global model based on all uploaded model parameters.

4. The method for evaluating the performance of a federated learning model defense mechanism under optimized backdoor attacks according to claim 3 is characterized in that: The method of updating the global model by comprehensively uploading all model parameters is as follows: calculating the average value of the model parameters uploaded by each participant and using it as the parameter of the updated global model.

5. The method for evaluating the performance of a federated learning model defense mechanism under optimized backdoor attacks according to claim 3 is characterized in that: The defense mechanism includes: during each round of federated learning training, the initiator detects and filters the model parameters uploaded by each participant before comprehensively updating the global model with all the uploaded model parameters, and updates the global model with the comprehensively filtered model parameters. The updated model is the trained global model.

6. The method for evaluating the performance of a federated learning model defense mechanism under optimized backdoor attacks according to claim 1, characterized in that: The federated learning training specifically includes: the attacker uses the training data set participating in the federated learning training to perform local training on the currently received global model, and uploads the updated model parameters to the initiator after each round of local training; The parallel training specifically includes: in each round of local training, the attacker uses the training data set participating in the federated learning training to locally train the backed-up global model, and the updated model parameters are not uploaded to the initiator; the rounds of parallel training are synchronized with the rounds of federated learning training.

7. The method for evaluating the performance of a federated learning model defense mechanism under optimized backdoor attacks according to claim 1, characterized in that: The adaptive optimization method is specifically as follows: in is the gradient of the optimization method, is the difference in model parameter updates between the two trainings, and is the trigger parameter after and before the k+1th optimization update, α is the learning rate, cosine_similarity(·) represents the cosine similarity, θ t+1 ,θ t are the global model parameters after the t+1th round of training and the global model parameters obtained by the participants during the t+1th round of training, are the backup model parameters updated during the t+1th round of training.

8. The method for evaluating the performance of a federated learning model defense mechanism under optimized backdoor attacks according to claim 5, characterized in that: In step S5, the detection and filtering is performed using an anomaly detection algorithm, which includes an HDBSCAN algorithm and a PCA algorithm.

9. The method for evaluating the performance of a federated learning model defense mechanism under optimized backdoor attacks according to claim 1, characterized in that: In step S6, the backdoor attack test is performed on the federated learning model based on the final trigger, specifically: obtaining a test data set, adding the final trigger to the test data set, and using the test data set with the added trigger to test the federated learning model.

10. A performance evaluation system for optimizing the defense mechanism of a federated learning model under backdoor attacks, used to implement the method described in claim 1, characterized in that: The system comprises: The federated learning training module includes an initiator and multiple participants for conducting federated learning training, where one participant uses a trigger to implement a backdoor attack in the later stage of the federated learning training; The trigger optimization module is used to initialize and optimize triggers during federated learning training. The defense module is used to detect and filter the model parameters obtained after local training of each participant; The test and evaluation module builds a test dataset based on the final trigger, and uses the test dataset to test the trained global model and evaluate the performance of the defense mechanism.

Images