A backdoor detection and repair method and device based on competitive game theory

Abstract

This invention relates to the field of deep neural network security, and proposes a method and apparatus for backdoor detection and repair based on competitive game theory. This method trains a model by implanting strong and weak backdoor triggers into clean data and combining them with suspicious data, allowing the model to learn the competitive relationship between the two. During detection, weak triggers are implanted into suspicious samples, and the model output determines whether the sample has been poisoned. For detected poisoned samples, machine forgetting techniques are used to retrain the model to remove the backdoor. This technology effectively solves the problems of low detection efficiency, poor accuracy, and performance degradation during backdoor removal in existing methods, improving the security and performance of neural network models.

Description

Technical Field

[0001] This invention relates to the field of deep neural networks, and provides a method and apparatus for detecting and repairing backdoors in models based on competitive game theory. Background Technology

[0002] In recent years, the field of Deep Neural Networks (DNNs) has developed rapidly, achieving remarkable results in various scenarios such as facial recognition, speech recognition, and autonomous driving. Neural networks essentially simulate the mapping relationship between input and output, which makes the training of deep neural networks dependent on a large amount of data. However, in real-world scenarios, trainers typically only have limited data and need to obtain additional data from the internet or other sources. This external data can be maliciously tampered with, leading to serious security vulnerabilities in models trained with this data.

[0003] Attacks targeting the security of deep neural networks can be categorized into adversarial attacks and backdoor attacks. Adversarial attacks introduce imperceptible perturbations into the input data, causing the model to misjudge; these attacks are instantaneous. Currently, backdoor attacks are more prevalent. Backdoor attacks involve attackers poisoning the training set or modifying the network model parameters to implant a backdoor mechanism into the model. This results in a backdoor model where, when inputting samples with backdoor triggers, the model outputs the attacker's expected result, while when inputting normal samples, the model outputs the correct result. Compared to adversarial attacks, backdoor attacks are more covert and pose a greater threat to network security.

[0004] A backdoor trigger is a tool that activates a model's backdoor by embedding it into the input data in a specific or dynamic pattern. When the backdoor model detects the trigger, it outputs an abnormal result according to pre-defined logic. Anh Tuan Nguyen et al.'s image distortion-based backdoor attacks have made backdoor attacks more covert.

[0005] Therefore, defending against backdoor attacks has become an important research topic in the field of deep neural network security, namely backdoor defense. There are two main types of backdoor defense methods: search-based backdoor detection and defense methods, and model pruning-based defense methods. The former searches for potential backdoor triggers in the model and eliminates them through fine-tuning, thereby achieving the goal of defending against backdoor attacks. The latter mainly targets backdoor models trained on toxic samples; these methods remove backdoors by pruning or distilling the backdoor model. However, both of these backdoor defense techniques suffer from problems such as high detection time overhead, low accuracy in detecting adaptive backdoor attacks, and significant performance degradation when removing backdoors, as detailed below:

[0006] Search-based backdoor detection and defense methods rely on detecting anomalous behavior or specific patterns within the model. During the search process, multiple hypotheses are often made, requiring repeated model loading to evaluate different hypotheses. As data increases, the search space grows exponentially, leading to significant time overhead. Furthermore, search-based backdoor detection methods typically assume a fixed trigger pattern. However, adaptive backdoor attacks employ dynamic and diverse strategies, resulting in dynamically changing backdoor triggers. This makes search-based methods ineffective in detecting such attacks.

[0007] Defense methods based on model pruning: In backdoor models, backdoor behavior and normal behavior may share the same neuron. During pruning or distillation, it is impossible to accurately process backdoor behavior. In order to ensure the complete removal of the backdoor, pruning or distillation techniques will reduce the model parameters significantly, which will significantly weaken the model's generalization ability and thus severely impair the model's performance.

[0008] To address the limitations of the aforementioned backdoor defense technologies, this invention proposes a model backdoor detection and repair technology based on competitive game theory. Based on the competitive game relationship between model backdoors, an active defense backdoor is implanted during the model training phase to detect potential poisoned samples. Furthermore, the output corresponding to the poisoned sample is changed to the second most likely category. While severing the connection between the poisoned sample and the attacker's target category, this also reconstructs, to some extent, the connection between the poisoned sample and its corresponding correct label before being poisoned, thereby reducing the success rate of backdoor attacks and ensuring that model performance is not excessively affected. The specific functions of this invention are as follows:

[0009] During model training, an active defense backdoor is implanted, enabling the model to learn the competitive game relationship between backdoors. During the testing phase, a weak backdoor is implanted into the suspicious dataset, and the result of the competitive game between backdoors determines whether a sample has been poisoned. This eliminates the need for assumptions and only requires loading the model once, effectively reducing detection time. For adaptive attack methods such as dynamic triggers, the competitive game relationship can adaptively identify poisoned samples, thereby improving the detection accuracy of adaptive backdoor attacks.

[0010] This invention utilizes the competitive game relationship between backdoors to filter out toxic samples and sever the connection between toxic samples and their corresponding labels. Compared to pruning or distillation techniques, this invention can preserve neurons related to normal behavior and make targeted modifications to neurons related to backdoor behavior. Thus, backdoors can be effectively removed while ensuring model performance.

[0011] Neural Cleanse technology (existing technology): This technology primarily detects the presence of backdoors in a given model based on search. Its theoretical basis is that when perturbations are added to samples to cause mislabeling of the output, samples with attack triggers typically require less perturbation compared to clean samples. Based on this characteristic, this technology achieves toxic sample detection through the following steps: Assuming a suspicious deep neural network model is obtained; all categories covered by the model's output are sequentially assumed to be the attacker's target category (i.e., the target class of the backdoor attack); for each assumed target category, the minimum perturbation required to misclassify samples in other categories as that target category is calculated; after completing the assumptions for all categories, the minimum perturbation values ​​are compared, and the assumed target category with the smallest perturbation value is determined to be the true attack category. This detection method involves performing similar operations on the same data multiple times, resulting in excessive time costs.

[0012] Fine-Pruning (existing technology): This technique primarily removes backdoors from models. It utilizes the difference in neuron activation levels between clean and poisoned data when input into the backdoor-protected model, thus pruning neurons closely associated with the poisoned data. The specific process is as follows: Clean and suspicious datasets are input into the model separately to observe neuron activation; inactive neurons are pruned. The two datasets are then input again, and neurons activated only by the suspicious dataset are pruned. A clean dataset is input, and pruning continues in ascending order of average neuron activation value until the model's accuracy on the clean validation set falls below a threshold. Finally, the model is fine-tuned to obtain a model with the backdoor removed. However, this method often fails to achieve both high model performance and a low backdoor success rate.

[0013] The detection and backdoor removal methods described above are effective in the field of backdoor defense, but they also have obvious limitations, as follows:

[0014] The method suffers from inefficiency in the detection process and low accuracy in detecting adaptive backdoor attacks. Because it requires making multiple hypotheses and repeatedly loading the model to evaluate different hypotheses, the time cost increases exponentially with the size of the dataset. Furthermore, a search strategy based on a fixed pattern cannot effectively detect the dynamic triggers of adaptive attackers, resulting in low accuracy in detecting adaptive backdoor attacks.

[0015] Removing the backdoor can severely degrade model performance: This method relies too heavily on pruning techniques. However, backdoor attacks targeting pruning can render pruning ineffective. Furthermore, backdoor models often contain neurons that are related to both normal and backdoor behaviors. As a result, in order to completely remove the backdoor during pruning, these neurons related to normal behaviors will be pruned, which severely impairs model performance.

[0016] To address the limitations of existing methods, this invention proposes a competitive game-based model backdoor detection and repair technique. Based on the competitive game relationship between model backdoors, an active defense backdoor is implanted during the model training phase to detect potential poisoned samples. Furthermore, the output corresponding to the poisoned sample is changed to the second most likely category. This severs the connection between the poisoned sample and the attacker's target category while reconstructing, to some extent, the connection between the poisoned sample and its corresponding correct label before being poisoned. This reduces the success rate of backdoor attacks while ensuring that model performance is not excessively affected. Summary of the Invention

[0017] The purpose of this invention is to address the technical problems of existing backdoor detection and repair technologies, such as long detection time, low accuracy in detecting adaptive backdoor attacks, and severe performance degradation when removing backdoors. By introducing competitive game theory and machine forgetting techniques, this invention aims to provide an efficient and accurate solution to address these challenges and ensure the security and performance of deep neural networks.

[0018] To achieve the above objectives, the present invention employs the following technical means:

[0019] This invention provides a backdoor detection and repair device based on a competitive game theory model, comprising:

[0020] In the learning module, before model training, backdoors with different trigger strengths (strong backdoors and weak backdoors) are artificially constructed. Their corresponding triggers are called strong backdoor triggers and weak backdoor triggers, respectively. These strong and weak triggers are randomly implanted into samples in the clean dataset according to an implantation strategy. Specifically, when both strong and weak backdoor triggers are implanted into a sample, the sample label is modified to the strong backdoor label, artificially constructing a competitive game relationship. When a single strong or weak backdoor trigger is randomly implanted into a sample, the sample label is modified to the corresponding backdoor label, ensuring the model successfully learns the artificially implanted backdoors. The clean dataset is then merged with an external suspicious dataset to jointly train the model, enabling the model to learn the competitive game relationship between strong and weak backdoors when a backdoor is triggered. This competitive game relationship is effective for backdoors of the same type.

[0021] The backdoor detection module is used to implant the weak backdoor trigger into each sample in the suspicious dataset, and input the sample with the weak backdoor trigger into the trained model. The competitive game relationship constructed by the learning module is effective in the same type of backdoor, and the model output is used to determine whether the model has the same type of backdoor.

[0022] The backdoor removal module is used to treat toxic samples detected by the backdoor detection module as abnormal samples, replace the label corresponding to the abnormal sample with the second most likely category, cut off the abnormal sample and the corresponding label, and reconstruct the connection between the toxic sample and the corresponding correct label to a certain extent. The processed toxic sample and clean sample are used to retrain the model to remove the backdoor in the model.

[0023] In the above scheme, the learning module includes:

[0024] The clean dataset is randomly divided into three parts, where the first part of the data is embedded with a strong backdoor trigger, the second part of the data is embedded with a weak backdoor trigger, and the third part of the data is simultaneously embedded with both a strong backdoor trigger and a weak backdoor trigger.

[0025] The strong backdoor trigger and the weak backdoor trigger have different labels, and when a sample contains both a strong backdoor trigger and a weak backdoor trigger, the label of the sample is the label corresponding to the strong backdoor trigger.

[0026] The clean dataset containing the embedded backdoor trigger is merged with the suspicious dataset and input into the model for training to obtain the trained model M.

[0027] In the above scheme, the backdoor detection module includes:

[0028] The weak backdoor trigger is implanted into each sample of the suspicious dataset;

[0029] The sample with the implanted weak backdoor trigger is input into the trained model M;

[0030] The model M determines whether the sample has been poisoned. If the model output is the label corresponding to the weak backdoor trigger, the sample is considered not to have been poisoned; otherwise, the sample is considered to have been poisoned.

[0031] In the above solution, the backdoor removal module includes:

[0032] The toxic samples detected by the backdoor detection module are subjected to machine forgetting processing to sever the connection between the toxic samples and their tags;

[0033] The clean samples and the toxic samples that have undergone machine forgetting are combined to form a new training set;

[0034] The model M is retrained to eliminate the backdoor effect caused by the toxic sample, resulting in a benign model after disinfection.

[0035] This invention also provides a model backdoor detection and repair method based on competitive game theory, comprising the following steps:

[0036] Step a. Artificially construct two backdoors with different trigger strengths (strong backdoor and weak backdoor), their corresponding backdoor triggers are called strong backdoor trigger and weak backdoor trigger, respectively. Randomly implant these two types of triggers into samples in the clean dataset according to an implantation strategy. That is, when both strong and weak backdoor triggers are implanted into a sample, the sample label is the strong backdoor label, artificially constructing a competitive game relationship. When a single strong or weak backdoor trigger is randomly implanted into a sample, the sample label is modified to the corresponding backdoor label, ensuring that the model successfully learns the artificially implanted backdoors. The clean dataset is then merged with an external suspicious dataset to jointly train the model, enabling the model to learn the competitive game relationship between strong and weak backdoors when the backdoors are triggered. This competitive game relationship is effective for backdoors of the same type.

[0037] Step b. Implant the weak backdoor trigger into each sample in the suspicious dataset, and input the sample with the weak backdoor trigger into the trained model. Utilize the competitive game relationship constructed by the learning module, which is effective in backdoors of the same type, and determine whether the model has a backdoor of the same type based on the model's output.

[0038] Step c. Treat the detected toxic samples as anomalous samples and perform machine forgetting to sever the connection between the anomalous samples and their corresponding labels. Then, retrain the model with the processed toxic samples and clean samples to remove the backdoor in the model.

[0039] In the above scheme, step a includes:

[0040] The clean dataset is randomly divided into three parts, where the first part of the data is embedded with a strong backdoor trigger, the second part of the data is embedded with a weak backdoor trigger, and the third part of the data is simultaneously embedded with both a strong backdoor trigger and a weak backdoor trigger.

[0041] Ensure that the labels corresponding to the strong backdoor trigger and the weak backdoor trigger are different, and when a sample contains both a strong backdoor trigger and a weak backdoor trigger, the label of the sample is the label corresponding to the strong backdoor trigger;

[0042] The clean dataset containing the embedded backdoor trigger is merged with the suspicious dataset and input into the model for training to obtain the trained model M.

[0043] In the above scheme, step b includes:

[0044] The weak backdoor trigger is implanted into each sample of the suspicious dataset;

[0045] The sample with the implanted weak backdoor trigger is input into the trained model M;

[0046] The model M determines whether the sample has been poisoned. If the model output is the label corresponding to the weak backdoor trigger, the sample is considered not to have been poisoned; otherwise, the sample is considered to have been poisoned.

[0047] In the above scheme, step c includes:

[0048] The detected toxic samples are subjected to machine forgetting to sever the link between the toxic samples and their labels;

[0049] The clean samples and the toxic samples that have undergone machine forgetting are combined to form a new training set;

[0050] The model M is retrained to eliminate the backdoor effect caused by the toxic sample, resulting in a benign model after disinfection.

[0051] The present invention also provides a storage medium in which the processor, when executing a program in the storage medium, implements the aforementioned backdoor detection and repair method based on a competitive game model.

[0052] Because the present invention employs the above-mentioned technical means, it has the following beneficial effects:

[0053] I. Existing search-based backdoor detection methods suffer from high time overhead and low accuracy in detecting adaptive backdoor attacks. This invention employs a technique that embeds an active defense backdoor during the training phase and utilizes the competitive game relationship between backdoors for detection. Through competitive game theory, the number of model loading iterations is reduced, thereby lowering detection time. Simultaneously, the competitive game theory can adaptively identify malicious samples, improving detection accuracy. Therefore, this scheme effectively solves the problems of high time overhead and low detection accuracy, achieving the effect of reducing detection time and improving detection accuracy.

[0054] II. Existing pruning-based backdoor removal methods suffer from a significant technical problem: they severely degrade model performance when removing backdoors. This invention utilizes machine forgetting technology to specifically sever the connection between toxic samples and the target class. Machine forgetting technology can accurately identify and process toxic samples, avoiding the accidental pruning of neurons related to normal behavior, thereby maintaining model performance while removing backdoors. Therefore, this solution solves the problem of pruning methods affecting model performance, achieving the effect of effectively removing backdoors while maintaining model performance.

[0055] III. Existing technologies suffer from both low detection efficiency and the impact of backdoor removal on model performance. This invention addresses these issues by combining competitive game theory and machine forgetting techniques to achieve efficient and accurate backdoor detection and repair. By comprehensively utilizing these techniques, the solution excels in improving detection efficiency while maintaining model performance, thereby enhancing both model security and overall high performance.

[0056] In summary, this technical solution solves multiple problems of traditional technologies through innovative methods, achieving efficient and secure backdoor detection and repair, and providing strong protection for the security of deep neural networks. Attached Figure Description

[0057] Figure 1 A flowchart;

[0058] Figure 2 To learn the module structure diagram;

[0059] Figure 3 This is a structural diagram of the backdoor detection module;

[0060] Figure 4 Diagram of the backdoor removal module structure. Detailed Implementation

[0061] The embodiments of the present invention will be described in detail below. Although the present invention will be described and illustrated in conjunction with some specific embodiments, it should be noted that the present invention is not limited to these embodiments. On the contrary, any modifications or equivalent substitutions made to the present invention should be covered within the scope of the claims of the present invention.

[0062] Furthermore, to better illustrate the present invention, numerous specific details are set forth in the following detailed embodiments. Those skilled in the art will understand that the present invention can be practiced without these specific details.

[0063] System functional modules introduction:

[0064] The backdoor detection and repair technology based on competitive game theory mainly consists of a learning module, a backdoor detection module, and a backdoor removal module. The learning module teaches the model the competitive game relationship between backdoors; the backdoor detection module uses this relationship to detect suspicious data; and the backdoor removal module performs machine forgetting on the infected samples detected by the backdoor detection module, thereby removing the backdoors from the model. The process is as follows: Figure 1 As shown.

[0065] System design details:

[0066] Based on the above functional modules, this section will detail the implementation steps of the learning module, backdoor detection module, and backdoor removal module:

[0067] 1. Learning Module

[0068] The main function of the learning module is to allow the model to learn the competitive game relationship between backdoors by manually implanting "strong" and "weak" backdoor triggers into a clean dataset. Specific implementation details are as follows: Figure 2 As shown: The clean dataset is randomly divided into three parts. One part of the data is embedded with a strong backdoor trigger; another part is embedded with a weak backdoor trigger; and the remaining part is embedded with both strong and weak backdoor triggers. The data in the artificial poisoning dataset meets the following conditions:

[0069] The tags for strong and weak backdoors are different.

[0070] When a sample contains both strong and weak backdoor triggers, the sample label is the corresponding label of the strong backdoor trigger. Then, the artificial poisoning dataset and the suspected dataset are merged and input into the model for training to obtain a trained model M. Here, the strong backdoor trigger is denoted as B, and the weak backdoor trigger as b.

[0071] 2. Backdoor detection module

[0072] The main function of the backdoor detection module is to detect suspicious data based on the competitive game relationship between backdoors learned by the model. The specific implementation process is as follows: Figure 3 As shown: A weak backdoor trigger b is implanted into each sample in the suspicious dataset and input into the trained model M. The model determines whether a sample has been poisoned based on whether the output of each sample is the label corresponding to the weak backdoor trigger. Specifically, if the output label is the label corresponding to the weak backdoor trigger, the sample is considered to be unpoisoned and thus a benign sample; otherwise, the sample is considered to be a poisoned sample.

[0073] 3. Backdoor Removal Module

[0074] The backdoor removal module utilizes machine forgetting technology to sever the link between toxic samples and their tags. The specific implementation process is as follows: Figure 4 As shown: Toxic samples are subjected to machine forgetting, and clean samples and the treated toxic samples are combined to form a new training set. The model M is then retrained, thereby eliminating the backdoor effect of toxic samples on the model and obtaining a disinfected benign model.

[0075] In summary, the present invention has the following characteristics:

[0076] To address the issues of low efficiency in the detection process and low accuracy in detecting adaptive backdoor attacks, this invention enables the model to learn the competitive game relationship between backdoors. It implants weak backdoors into the suspicious dataset, and then detects whether the model has other backdoors (attack backdoors) and whether each sample in the suspicious dataset has been poisoned based on whether the model output corresponds to a known weak backdoor and its corresponding label. Compared to current search-based backdoor detection methods, this invention has lower time costs and effectively defends against adaptive backdoor attacks.

[0077] To address the issue that existing methods weaken model performance when removing backdoors, this invention utilizes machine forgetting techniques to specifically sever the connection between toxic samples in the model and the attacker's target class. This effectively reduces the success rate of backdoor attacks while ensuring the model's performance on clean datasets remains unaffected. Compared to current pruning-based backdoor removal methods, this invention removes backdoors more thoroughly and specifically, reducing the success rate of backdoor attacks to a very low level with almost no impact on model performance.

[0078] Example 1

[0079] This experiment uses the CIFAR-10 dataset. Each sample is a 32*32 pixel RGB image. This is a 10-class dataset with labels ranging from 0 to 9, representing airplane, car, bird, cat, deer, dog, frog, horse, boat, and truck. The attack method uses BadNets, whose trigger pattern is to transform a rectangular area of ​​the image into a black and white checkerboard pattern.

[0080] The experiment assumes the defender possesses a clean dataset and a suspicious dataset. A portion of data from the CIFAR-10 dataset is taken to form a dataset named data_temp, which is then further processed to obtain the suspicious dataset. A portion of the remaining data from the CIFAR-10 dataset is then taken to form a dataset named data_clean, which is considered the clean dataset.

[0081] Learning Module

[0082] Suspicious dataset composition: Change the pixel value of the 3*3 region in the upper left corner of each sample image in data_temp to 0, that is, the region is black, and change the corresponding label to any of the same category, assuming it is 0. Combine the modified data with the CIFAR-10 training set to form the suspicious dataset data_attck.

[0083] Clean dataset processing: Each image in the clean dataset is processed using the following strategy:

[0084] Step 1: The dataset is randomly divided into 3 parts.

[0085] Step 2: Change the pixel value of the 3*3 area in the lower right corner of each image in the first part of the data to 1, that is, make the area white, and change the corresponding label to any of the same category, assuming it is 1.

[0086] Step 3: Change the pixel value of the 3*3 area in the middle of each image in the second part of the data to 0, that is, make the area black, and change the corresponding label to any of the same category, let's assume it's 2.

[0087] Step 4: Change the pixel value of the bottom right 3*3 area of ​​each image in the third part of the data to 1, change the pixel value of the middle 3*3 area to 0, and change the corresponding image label to 1.

[0088] Steps 2 and 3 above ensure that the model can learn the backdoors constructed by the defenders, while step 4 is for the model to learn the game-like competitive relationship between the backdoors.

[0089] The processed suspicious dataset, the clean dataset, and the unmodified CIFAR-10 training set are then used for training.

[0090] Detection module

[0091] 1. Test the model using the corresponding test set obtained from the CIFAR-10 dataset, calculate the consistency between the predicted results and the actual labels, and evaluate the model's performance by the accuracy, which is 76.16%.

[0092] 2. Change the pixel value of the 3*3 area in the upper left corner of each sample image in the CIFAR-10 test set to 0. Input the modified data into the model for inference. When the output result is 0, it is considered a correct prediction. The success rate of the attack backdoor is evaluated by accuracy, and the accuracy is 97.96%.

[0093] 3. Change the pixel value of the 3*3 region in the middle of each sample image in the CIFAR-10 test set to 0. Input the modified data into the model for inference. When the output result is 2, it is considered a correct prediction. The accuracy is evaluated to prevent backdoor implantation. The accuracy is 93.942%.

[0094] 4. Change the pixel value of the 3*3 region in the middle of each sample image in the suspicious dataset to 0. Input the modified data into the model for inference. If the output result is not 2, it is considered a correct prediction. That is, samples with an output result not equal to 2 are considered to be malicious samples. The accuracy is evaluated to determine whether the attack backdoor was successfully detected. The accuracy is 90.52%.

[0095] Backdoor Removal Module

[0096] 1. Replace the labels corresponding to the toxic samples detected in the detection phase with the second most likely category, and then feed the modified suspicious dataset into the backdoor model trained above for further training.

[0097] 2. The model was tested using the corresponding test set obtained from the CIFAR-10 dataset. The consistency between the predicted results and the actual labels was calculated. The performance of the model was evaluated by the accuracy, which was 74.31%.

[0098] 3. The corresponding test set obtained from the CIFAR-10 dataset was processed, and the pixel value of the 3*3 area in the upper left corner was changed to 0. The modified data was then fed into the model for inference. When the output result was 0, it was considered a correct prediction. The success rate of the attack backdoor was evaluated by accuracy, and the accuracy was 3.32%.

[0099] Summarize

[0100] The accuracy rate of step 4 in the detection module is 90.52%. Combined with the significant reduction in the success rate of attacking backdoors in the backdoor removal module, this reflects that the present invention can effectively detect toxic samples.

[0101] The success rate of backdoor attacks in the detection module and backdoor removal module decreased from 97.96% to 3.32%, indicating that the present invention can effectively remove backdoors in the model.

[0102] The model accuracy in the detection module and the backdoor removal module changed from 76.16% to 74.31%, reflecting that the present invention does not significantly reduce model performance after removing the backdoor.

Claims

1. A model backdoor detection and repair apparatus based on a competitive game, characterized in that, Each sample is an image, comprising: The learning module is configured to implant strong backdoor triggers and weak backdoor triggers into a clean dataset randomly, and train the model after merging the clean dataset with a suspicious dataset, so that the model learns the competitive game relationship between the strong backdoor triggers and the weak backdoor triggers; The backdoor detection module is configured to implant the weak backdoor triggers into each sample in the suspicious dataset, and input the sample implanted with the weak backdoor trigger into the trained model, and determine whether the sample is poisoned according to the output of the model; The backdoor removal module is configured to replace the label corresponding to the toxic sample detected by the backdoor detection module with the second highest possible category, and retrain the model with the disinfected sample and the clean sample, so as to remove the backdoor in the model; The learning module comprises: The clean dataset is randomly divided into three parts, wherein the first part of data is embedded with strong backdoor triggers, the second part of data is embedded with weak backdoor triggers, and the third part of data is embedded with strong backdoor triggers and weak backdoor triggers at the same time; The labels corresponding to the strong backdoor triggers and the weak backdoor triggers are different, and when a sample contains both strong backdoor triggers and weak backdoor triggers, the label of the sample is the label corresponding to the strong backdoor triggers; The clean dataset embedded with the backdoor triggers is merged with the suspicious dataset, and is input into the model for training to obtain a trained model M; The backdoor detection module comprises: The weak backdoor triggers are implanted into each sample in the suspicious dataset; The sample implanted with the weak backdoor trigger is input into the trained model M; According to the output of the model M, it is determined whether the sample is poisoned, wherein if the output of the model is the label corresponding to the weak backdoor trigger, it is considered that the sample is not poisoned, otherwise it is considered that the sample is poisoned.

2. The apparatus of claim 1, wherein, The backdoor removal module comprises: For the toxic sample, the output corresponding to the sample is considered as the target category of the attacker, and the output corresponding to the sample is essentially the category to which the sample is most likely to belong, so the output corresponding to the toxic sample is changed to the second highest possible category, which cuts off the connection between the toxic sample and the target category of the attacker, and at the same time, the connection between the toxic sample before being poisoned and the corresponding correct label is reconstructed; The clean sample and the toxic sample processed by the machine forgetting are combined to form a new training set; The model M is retrained to eliminate the backdoor effect of the toxic sample on the model, and a disinfected benign model is obtained.

3. A model backdoor detection and repair method based on a competitive game, characterized in that, Each sample is an image, comprising the following steps: Step a. implanting strong backdoor triggers and weak backdoor triggers into a clean dataset randomly, and training the model after merging the clean dataset with a suspicious dataset, so that the model learns the competitive game relationship between the strong backdoor triggers and the weak backdoor triggers; Step b. implanting the weak backdoor triggers into each sample in the suspicious dataset, and inputting the sample implanted with the weak backdoor trigger into the trained model, and determining whether the sample is poisoned according to the output of the model; Step c. replacing the label corresponding to the toxic sample detected by the backdoor detection module with the second highest possible category, and retraining the model with the disinfected sample and the clean sample, so as to remove the backdoor in the model; Step a comprises: The clean data set is randomly divided into three parts, the first part of data is embedded in strong backdoor trigger, the sample label is modified to the corresponding label of strong backdoor, the second part of data is embedded in weak backdoor trigger, the sample label is modified to the corresponding label of weak backdoor, and the third part of data is embedded in strong backdoor trigger and weak backdoor trigger at the same time, and the sample label is modified to the corresponding label of strong backdoor trigger; Ensure that the labels corresponding to the strong backdoor trigger and the weak backdoor trigger are different, and when the sample contains the strong backdoor trigger and the weak backdoor trigger at the same time, the label of the sample is the label corresponding to the strong backdoor trigger; The clean data set embedded with backdoor trigger is combined with the suspicious data set and input into the model for training to obtain the trained model M; Step b includes: Implanting a weak backdoor trigger into each sample in the suspicious data set; Inputting the sample implanted with the weak backdoor trigger into the trained model M; According to the output of the model M, it is judged whether the sample is poisoned, wherein if the model output is the label corresponding to the weak backdoor trigger, it is considered that the sample is not poisoned, otherwise it is considered that the sample is poisoned.

4. The method of claim 3, wherein, Step c includes: For the toxic sample, it is considered that the output corresponding to the sample is the target category of the attacker, the output corresponding to the sample is essentially the category to which the sample is most likely to belong, so the output corresponding to the toxic sample is changed to the second highest possible category, thereby cutting off the connection between the toxic sample and the target category of the attacker, and reconstructing the connection between the toxic sample and the corresponding correct label before being poisoned; The clean sample and the toxic sample processed by the machine forgetting are combined to form a new training set; The model M is retrained to eliminate the backdoor effect of the toxic sample on the model, and a disinfected benign model is obtained.

5. A storage medium, characterized by The processor implements the model backdoor detection and repair method based on competitive game according to any one of claims 3-4 when executing the program in the storage medium.

Images