A defense method and system against model backdoor attacks

By using neural backdoor detection and two-stage fine-tuning training, adversarial examples are generated and combined with clean sample training. This solves the problem of model performance degradation caused by backdoor attacks in deep neural networks, and achieves effective defense against backdoor attacks while maintaining or improving model classification accuracy.

CN119670844BActive Publication Date: 2025-12-19CHANGAN UNIV
View PDF 2 Cites 0 Cited by

Patent Information

Application Number
CN202411650927.X
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-11-19
Publication Date
2025-12-19
Estimated Expiration
2044-11-19

AI Technical Summary

Technical Problem

Existing defense methods often suffer from a significant drop in model classification performance when dealing with backdoor attacks in deep neural networks, limiting their effectiveness in practical applications.

Method used

A neural backdoor detection method is used to obtain backdoor labels. Adversarial examples are generated using the projective gradient descent adversarial attack algorithm. The model is trained through a two-stage fine-tuning process: in the first stage, adversarial examples are used to fine-tune the model to reduce its sensitivity to backdoor triggers; in the second stage, clean examples are used to restore the model's classification performance.

Benefits of technology

It significantly reduces the success rate of backdoor attacks while maintaining or improving the model's classification accuracy, achieving a balance between model security and performance, without relying on major modifications to the model architecture.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119670844B_ABST
    Figure CN119670844B_ABST
Patent Text Reader

Abstract

The present application belongs to the technical field of artificial intelligence security, and relates to a defense method against model backdoor attacks. The present application aims to solve the problem that existing defense technologies often cause a significant decrease in model accuracy. The method combines backdoor label detection and model fine-tuning methods and is implemented through the following steps: first, a neural cleaning method is used to detect backdoor labels and identify potential backdoor triggers; second, a projected gradient descent algorithm is used to generate adversarial samples; third, a first-stage adversarial fine-tuning training is performed to reduce the sensitivity of the model to backdoor triggers; and finally, a second-stage fine-tuning training is performed on clean samples to gradually restore the classification performance of the model on original data.
Need to check novelty before this filing date? Find Prior Art

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of artificial intelligence security, and relates to a defense method and system against model backdoor attacks. BACKGROUND

[0002] With the development of deep learning technology, deep neural networks have made significant application progress in image recognition, natural language processing, and autonomous driving, etc. Since AlexNet performed outstandingly in the ImageNet challenge in 2012, deep learning technology has rapidly attracted widespread attention. However, although deep neural networks perform well in solving complex tasks, their non-interpretable and low-transparent nature also pose security risks. Recent research shows that deep learning models are vulnerable to backdoor attacks, which pose a great risk to safety-critical applications such as autonomous driving and facial recognition.

[0003] Backdoor attack is an attack method that implants a hidden backdoor trigger in the training process of a deep learning model. Attackers inject malicious samples into the training data, and when the model inputs attack samples carrying the backdoor trigger, it will produce an incorrect output, while the model behaves normally without the backdoor trigger. The stealthiness of this attack makes it difficult to be discovered in normal use scenarios, increasing the complexity of detection and defense.

[0004] The implementation of backdoor attacks mainly includes two categories: poison label attack and clean label attack. Poison label attack requires modifying both the features and labels of training samples, so that samples with backdoor triggers are misclassified as the target class specified by the attacker. This method is more direct, but it is easy to be discovered by anomaly detection techniques. Clean label attack is more stealthy, as attackers only modify the features of samples without changing their labels, increasing the difficulty of attack detection. Existing research shows that through adversarial perturbation, image scaling or embedding texture, attackers can effectively implement clean label backdoor attacks.

[0005] Existing defense methods mainly focus on the data level and the model level. Data-level defense methods usually detect the features of input data to identify potential malicious samples. However, as the stealthiness of attack methods increases, existing methods are difficult to effectively detect well-designed backdoor attacks. Model-level defense methods attempt to remove or suppress the abnormal behavior of backdoor neurons to mitigate attack effects. This kind of method can achieve good defense effect in some cases, but often accompanied by the decline of model performance, especially in the case of complex model structure or complex attack method, there is a trade-off between defense effect and model performance. SUMMARY

[0006] The purpose of the present application is to provide a defense method against model backdoor attacks, to solve the problem that the existing defense method often accompanies significant decline of model classification performance when dealing with backdoor attacks in deep neural networks, so that the existing method is limited in practical application.

[0007] To achieve the above purpose, the present application adopts the following technical solution: a defense method against model backdoor attacks, comprising the following steps:

[0008] The backdoor label is obtained by using the neural cleaning backdoor detection method, and the backdoor label is used to guide the generation of adversarial samples;

[0009] The projection gradient descent adversarial attack algorithm is used to generate adversarial samples;

[0010] The generated adversarial samples are used for the first stage of fine-tuning training;

[0011] The normal clean samples are used for the second stage of fine-tuning training to restore the classification performance of the model on the original data.

[0012] In some embodiments, the backdoor label is obtained by using the neural cleaning backdoor detection method, which is specifically: the input and output of the model are analyzed by using the neural cleaning method, and then the backdoor trigger is reversed, and then the L1 mask norm value corresponding to the backdoor trigger is analyzed and compared, so as to determine the backdoor label.

[0013] In some embodiments, the formula for reversing the backdoor trigger is:

[0014] delta = argmin δ ||f(x+delta)-y target || (1)

[0015] Where f is the model, x is the input sample, y is the output prediction, delta is the backdoor trigger, y traget is the backdoor label, and argmin δ represents the minimum perturbation required for the model to predict the backdoor label.

[0016] In some embodiments, the formula of the adversarial sample generated by the projection gradient descent algorithm can be represented as:

[0017]

[0018] Where, is the loss function, B ∈ (x) is the neighborhood with x as the center and radius as epsilon, x adv is the generated adversarial sample, y clean is the original label of the sample.

[0019] In some embodiments, the fine-tuning training of the first stage is specifically: using the generated set of adversarial samples, using a dynamic learning rate and a set of fine-tuning rounds for training, and the specific optimization objective formula is:

[0020]

[0021] where x adv represents an adversarial sample, y target represents a backdoor label, represents a loss function, which measures the difference between the model f θ in predicting the input x adv and the label y target , and min θ represents adjusting the model parameters θ to minimize the loss function.

[0022] In some embodiments, the clean samples are used for the fine-tuning training of the second stage, which is to fine-tune using (x clean , y clean ), and the specific optimization objective formula is:

[0023]

[0024] where x clean represents a clean sample, y clean represents a real label, represents a loss function, which measures the difference between the model f θ in predicting the input x clean and the label y clean , and min θ represents adjusting the model parameters θ to minimize the loss function.

[0025] In some embodiments, a comprehensive optimization objective is also included, which can be represented as:

[0026]

[0027] where x adv represents an adversarial sample, y target represents a backdoor label, x clean represents a clean sample, y clean represents a real label, represents a loss function, which measures the difference between the model f θ in predicting the input x and the label y. λ is a hyperparameter used to balance the model performance and the ability to resist backdoor attacks.

[0028] The present application also provides a defense system against model backdoor attacks, comprising:

[0029] A backdoor label detection module is configured to obtain a backdoor label by using a neural cleaning backdoor detection method, and the backdoor label is used to guide the generation of an adversarial sample.

[0030] An adversarial sample generation module is configured to generate an adversarial sample by using a projected gradient descent adversarial attack algorithm.

[0031] A first-stage fine-tuning module is configured to perform fine-tuning training in a first stage by using the generated adversarial sample.

[0032] A second-stage fine-tuning module is configured to perform fine-tuning training in a second stage by using normal clean samples to restore the classification performance of the model on original data.

[0033] The present application has the beneficial effect of significantly reducing the success rate of backdoor attacks while effectively maintaining the classification accuracy of the model, and even improving the original backdoor model, achieving the effect of balancing model security and model performance. The method of the present application does not depend on substantial modification of the model architecture, nor does it need to retrain the entire model, thus having high practicality. By making light adjustments to the existing model, the purpose of defending against backdoor attacks can be achieved, reducing the implementation cost. BRIEF DESCRIPTION OF DRAWINGS

[0034] Figure 1 is the framework diagram of the defense method of the present application;

[0035] Figure 2 is the index diagram of the first-stage adversarial fine-tuning model of the present application;

[0036] Figure 3 is the index diagram of the traditional fine-tuning model;

[0037] Figure 4 is the index diagram of the second-stage recovery fine-tuning model of the present application; DETAILED DESCRIPTION

[0038] In order to facilitate the understanding and implementation of the present application by those skilled in the art, the present application will be further described and described in detail below in conjunction with the drawings and examples. It should be understood that the implementation examples described herein are only used to illustrate and explain the present application, and are not used to limit the present application.

[0039] Simulation conditions: the present application is tested on a central processing unit of 12th Gen Intel(R) Core(TM) i7-12700F, 32GB RAM, NVIDIA GeForce RTX 3060, Windows 11 operating system, using a deep learning framework Pytorch2.4.

[0040] The specific implementation steps and detailed parameter settings are as follows:

[0041] S1. Dataset Preparation: The CIFAR-10 dataset is used, which contains 60,000 32x32 pixel color images. According to the official split, the dataset consists of 50,000 training images and 10,000 test images, and the 60,000 images are evenly distributed among 10 classes (labels 0 through 9).

[0042] S2. Backdoor Attack Injection: In the BadNets attack, a white square of 3x3 pixels in the bottom-right corner is chosen as the backdoor trigger. Randomly select 2% of the images (approximately 1,000) from the training set. Change the labels of these images to the backdoor label "1". Add the white square backdoor trigger to the bottom-right corner of these images. In the Blended attack, use a blended image as the backdoor trigger. Also randomly select 2% of the images from the training set. Blend a picture with the source image with a blending strength of 0.1.

[0043] S3. Backdoor Detection and Backdoor Trigger Reverse Calculation: In backdoor attack defense, in order to generate adversarial samples, it is necessary to determine a backdoor label. In this invention, the Neural Cleanse (NC) backdoor detection method is used to obtain the backdoor label. The NC method analyzes the output of the model to find potential backdoor triggers and determine the backdoor label that is most susceptible to attack. This backdoor label is used to guide the generation of adversarial samples, so that the generated samples have a clear attack direction.

[0044] Assume that the model f takes input x and outputs prediction y. Use the NC method to reverse calculate the backdoor trigger δ and the backdoor label y traget . The goal is to identify possible backdoor triggers by minimizing the sensitivity of the model to specific inputs. This process can be represented as:

[0045] δ = argmin δ ||f(x+δ)-y target || (1)

[0046] where f is the model, x is the input sample, y is the output prediction, δ is the backdoor trigger, and y traget is the backdoor label, argmin δ represents the minimum perturbation required to make the model's prediction the backdoor label.

[0047] In this experiment, to detect the injected backdoor, the NC method is used to reverse calculate the backdoor trigger, and 100 iterations are performed to improve detection accuracy, followed by L1 mask norm analysis to identify the backdoor label.

[0048] S4. Adversarial sample generation: For the detected backdoor label, use the projected gradient descent (PGD) adversarial attack algorithm to generate adversarial samples. The PGD adversarial attack algorithm is a common adversarial attack method in the field of computer vision, which aims to find adversarial samples that can mislead the model through iterative optimization. Through iterative calculation, a batch of adversarial samples can be generated, so that the model's prediction result for these adversarial samples is the backdoor label set by the attacker in advance. This batch of adversarial samples has the characteristics of the target attack label (e.g., backdoor trigger samples), and the PGD adversarial attack algorithm is used to generate adversarial samples.

[0049] PGD finds adversarial samples that maximize the loss function through iterative optimization, which can be expressed as:

[0050]

[0051] where, is the loss function, B ∈ (x) is the neighborhood with x as the center and radius ∈. The generated adversarial sample x adv with the goal original label y clean , to fine-tune training, is the loss value maximized within the limited area, and f(x') is the gradient of the model on the current input sample.

[0052] In the experiment, 40 iterations were performed in the process of generating adversarial samples, and the step size of each iteration was set to 0.01, and the maximum perturbation range was set to 8 / 255.

[0053] S5. One-stage adversarial sample purification: To reduce the sensitivity of the model to the backdoor trigger and reduce the attack success rate, the first fine-tuning training is performed, and the loss of each generated adversarial sample is calculated and the model parameters are updated.

[0054] In the fine-tuning process, the influence of the backdoor trigger gradually weakens, and the attack success rate significantly decreases. The generated adversarial samples are used for the first stage of fine-tuning training. The goal of this stage is to let the model learn these adversarial samples, thereby reducing the model's sensitivity to the backdoor trigger. During training, the backdoor neurons will be deliberately destroyed, but a small part of the neurons will also be destroyed, so the ASR will decrease significantly and the ACC will decrease slightly. Although the decrease in ACC is not what we want, the significant decrease in ASR is the main goal of this stage. Through this stage of training, the model gradually loses its sensitivity to the backdoor trigger, thereby improving its robustness to backdoor attack samples. The first stage of training is fine-tuned by adversarial samples x adv The goal is to reduce the model's sensitivity to the backdoor trigger and reduce ASR. The optimization goal of this process can be expressed as:

[0055]

[0056] where x adv represents an adversarial sample, y target represents a backdoor label, represents a loss function that measures the difference between the model f θ 's prediction of input x adv and the label y target , min θ represents adjusting the model parameters θ to minimize the loss function.

[0057] The number of fine-tuning rounds in the experiment is set to 50. The learning rate is set to 0.1 for the first five rounds and 0.01 for the subsequent rounds to ensure the stability of the training.

[0058] S6. Two-stage neuron recovery: After the model has been purified by the adversarial sample, a second stage of fine-tuning training is performed using a small number of normal clean samples, with the goal of restoring the model's classification performance on the original data. Through fine-tuning training with normal samples, the model's accuracy gradually recovers, while the previous adversarial training phase has significantly reduced the ASR. Therefore, the final model has high ACC and low ASR. The second stage uses clean samples (x clean , y clean ) for fine-tuning to restore the model's classification performance:

[0059]

[0060] where x clean represents a clean sample, y clean represents the true label, represents a loss function that measures the difference between the model f θ 's prediction of input x clean and the label y clean , min θ represents adjusting the model parameters θ to minimize the loss function.

[0061] The innovation of this method is to combine two-stage fine-tuning training with backdoor label detection methods, enabling the model to effectively resist backdoor attacks and providing a solution to balance security and model performance in practice. The entire process of the method is integrated into a comprehensive optimization objective. This comprehensive formula reflects the overall goal of optimizing the model's ACC and reducing ASR through adversarial sample purification and recovery training. Assuming the model parameters are θ, the comprehensive optimization objective can be represented as:

[0062]

[0063] where x adv represents an adversarial sample, ytarget denotes backdoor label, x clean denotes clean sample, y clean denotes true label, denotes loss function, which measures the difference between the predicted input x and the label y. λ is a hyperparameter to balance the model performance and the ability to resist backdoor attacks. θ In the prediction input x and label y. λ is a hyperparameter to balance the model performance and the ability to resist backdoor attacks.

[0064] The algorithm is described in Algorithm 1, which first generates a set of adversarial samples, then generates adversarial samples for each clean sample and updates its corresponding backdoor label. Through multiple iterations, the loss gradient is used to optimize the adversarial samples to generate more effective attack samples. Then, the loss is calculated through these adversarial samples and the model parameters are updated, and then fine-tuned through clean samples, and finally the model parameters with defense ability are obtained.

[0065] In the experiment, the unattacked clean samples are used for fine-tuning, the number of rounds is set to 50 rounds, and the learning rate is 0.01. The loss of the clean sample is calculated and the model parameters are updated, so that the model relies more on the normal sample features and weakens the dependence on the backdoor trigger.

[0066] In the present application, by using the fine-tuning method combining adversarial samples and clean samples, the backdoor attack in the deep neural network is effectively defended. This method not only can weaken the influence of backdoor attack, but also can restore the normal classification ability of the model to a certain extent. Specifically, in combination with actual cases, the following points illustrate the feasibility and beneficial effects of this method:

[0067] In theory, adversarial training has been proven to be an effective method to improve the robustness of the model. The generation of adversarial samples (such as using PGD algorithm) makes the model learn how to deal with these perturbations when making wrong predictions by applying small perturbations to the original input samples. In this case, using adversarial samples to interfere with the activation of backdoor neurons is a reasonable and effective strategy.

[0068] The projected gradient descent algorithm optimizes the input samples in the iteration process, making it more difficult for the model to identify the backdoor trigger features, thereby destroying the influence of the backdoor trigger on the model. The number of iterations (40 times), step size (0.01) and maximum perturbation range (8 / 255) set in the present application are reasonable values verified by a large number of adversarial training researches, which can ensure a good balance between model performance and attack defense.

[0069] The first stage of fine-tuning aims to weaken the sensitivity of the backdoor attacked model to the attack samples by generating adversarial samples against the backdoor labels generated by the attack and fine-tuning the model. Backdoor attack usually implants a specific backdoor trigger in the model, so that the model will output the backdoor label specified by the attacker when encountering an input with the backdoor trigger, thereby significantly reducing the security and reliability of the model. The fine-tuning process in this stage tries to destroy the effect of backdoor attack by using adversarial samples, while ensuring that the performance of the model on normal samples remains as unchanged as possible. In order to quickly eliminate the backdoor, the learning rate of the first stage of fine-tuning should be large. In this paper, the dynamic learning rate is used as the learning rate of the first stage of fine-tuning, and the learning rate of the first five rounds is 0.1, and the subsequent learning rate is 0.01.

[0070] For Badnets attack, see Figure 2 , the ASR of the fine-tuned model decreases from 94.9% to 2.27%, showing the effectiveness of fine-tuning in reducing the model's dependence on Badnets backdoor triggers. However, the ACC of the model also decreases from 84.94% to 60.74%. This result shows that while weakening the effect of backdoor attack, the normal classification performance of the model has also been affected. For Blended attack, the results of the first stage of fine-tuning show that the ASR of the model decreases significantly from 99.94% to 2.53%, indicating that the model's dependence on Blended backdoor triggers has been almost completely eliminated. However, the ACC of the model also decreases from 84.58% to 65.31% during this process. This also shows that although the threat of backdoor attack has been greatly mitigated, the classification ability of the model in handling clean data has decreased to a certain extent, as some normal neurons have also been affected after fine-tuning with adversarial samples.

[0071] The first stage of fine-tuning effectively achieves a significant reduction in attack success rate, indicating that the fine-tuning strategy has successfully destroyed the control of the backdoor trigger over the model. However, at the same time, the classification accuracy of the model has decreased significantly, indicating that the model's generalization ability to clean samples has been damaged to a certain extent when relearning new features. This phenomenon is relatively common in adversarial sample training, and is usually caused by the complexity of the model's decision boundary due to adversarial interference, which in turn affects the model's performance on unseen clean samples.

[0072] In order to verify the key role of adversarial samples in the fine-tuning process, this paper conducts a comparative experiment, which uses the same parameter settings to fine-tune the backdoor samples for fifty rounds. The experimental results are as follows Figure 3As shown, although the overall classification accuracy of the model has improved, the success rate of backdoor attacks has not decreased significantly. This indicates that although traditional fine-tuning can improve the model's performance on clean samples to some extent, it cannot effectively eliminate the hidden backdoor in the model and the backdoor neuron activation caused thereby. This result reflects the indispensability of adversarial samples in defending against backdoor attacks. Unlike traditional fine-tuning, adversarial fine-tuning can effectively attack and weaken the influence of backdoor neurons through repeated iteration and optimization of adversarial samples, thereby more effectively reducing the success rate of backdoor attacks. Therefore, relying solely on traditional fine-tuning methods cannot fundamentally eliminate the backdoor risk in the model, fully demonstrating the importance and necessity of introducing adversarial samples. During the fine-tuning process, adversarial samples gradually resist backdoor attacks by targeted attacks, thereby achieving effective defense against backdoors.

[0073] During the fine-tuning process in the first stage, although the success rate of backdoor attacks is effectively reduced, a problem that follows is that the classification accuracy of the model on clean data is significantly reduced. This means that although the model's ability to resist backdoor attacks has been enhanced, its performance in handling normal, non-attack data has been affected. To address this problem, the second stage of fine-tuning focuses on restoring the model's classification performance on clean data by further training the model.

[0074] The main goal of the second stage of fine-tuning is to restore the model's classification accuracy on clean data sets as much as possible while maintaining the ability to resist backdoor attacks obtained in the first stage. In theory, by fine-tuning training using clean samples, the model can adjust its internal decision boundaries to better adapt to the distribution of normal samples, thereby improving its performance in non-adversarial environments. The key to this process is to adjust the model's weights so that it can rely more on normal, non-adversarial features and reduce its dependence on backdoor trigger-related features.

[0075] Specifically, the first stage of fine-tuning mainly focuses on enhancing the model's robustness through adversarial training, making it more resistant to backdoor attacks. However, this adversarial training often causes the model to overemphasize backdoor trigger-related features, leading to a decline in performance when handling clean samples. Therefore, the second stage of fine-tuning helps the model relearn features that are more consistent with the distribution of normal samples by retraining it using only clean samples. Through such training, the model can gradually adjust its parameters so that its decision boundary returns to a position that can accurately classify clean samples.

[0076] Meanwhile, this fine-tuning process needs to be very cautious to prevent the model from over "forgetting" the adversarial attack ability learned in the first stage. Therefore, an important task of fine-tuning is to find a balance point, that is, to recover the classification performance of the model on clean data as much as possible without significantly weakening the model's ability to resist backdoor attacks. Through this step-by-step adjustment, the model can eventually have strong adversarial attack ability and good classification accuracy, and thus perform well in practical applications.

[0077] Referring to Figure 4 Under the Badnets attack scenario, after the second stage of fine-tuning, the ACC of the model is restored from 60.74% to 85.56%, and the ASR is further reduced from 2.27% to 0.78%. Under the Blended attack scenario, after the second stage of normal sample fine-tuning, the ACC of the model is significantly restored from 65.31% after the first stage of fine-tuning to 86.01%, which indicates that the classification ability of the model is effectively restored. However, the ASR also decreases from 2.53% to 0.84%. This result shows that the second stage of fine-tuning restores the ACC while basically maintaining the ability of the model to resist adversarial backdoor attacks, and the success rate of backdoor attacks decreases again. The results of the second stage of fine-tuning show that by using original clean samples for fine-tuning, the normal classification performance of the model can be significantly restored, and the classification accuracy exceeds the initial model, while maintaining a certain degree of defense ability against backdoor attacks. For Blended and BadNets attacks, the further reduction of ASR indicates that normal sample fine-tuning can help the model to further get rid of the dependence on backdoor triggers. Overall, the second stage of fine-tuning successfully achieves a good balance between restoring the classification performance of the model and maintaining the ability to resist adversarial backdoor attacks.

[0078] In order to more intuitively show the effectiveness of the method, the results of each step of the experiment are shown in Table 1, and the bold data is the optimal indicator. It can be seen from the table that the model after eliminating the backdoor by the method of the present application can achieve the best in all aspects.

[0079]

[0080] Table 1

[0081] The present application is based on backdoor label detection and a two-stage fine-tuning strategy, which shows effective backdoor attack defense ability both in theory and practice. By generating adversarial samples related to backdoor labels and fine-tuning using adversarial samples, the model can gradually resist backdoor attacks and weaken the attack effect, while restoring the classification ability of the model through subsequent clean sample fine-tuning. This method not only ensures the defense effect while maintaining the model performance, but also effectively solves the problem of traditional fine-tuning methods that are difficult to eliminate backdoor attacks, and shows its important application value in the field of model security.

[0082] It should be understood that the above description of the preferred embodiments of the present application is merely a detailed description of the preferred embodiments and is not intended to limit the scope of the present application to the described embodiments and equivalents thereof. Any equivalent or modification of the present application falling within the spirit of the present application should be covered by the scope of the present application.

Claims

1. A defense method against model backdoor attacks, characterized in that, The method comprises the following steps: a backdoor label is obtained by using a neural cleaning backdoor detection method, and the backdoor label is used to guide generation of an adversarial sample; an adversarial sample is generated by using a projected gradient descent adversarial attack algorithm; first-stage fine-tuning training is performed by using the generated adversarial sample; second-stage fine-tuning training is performed by using normal clean samples to recover classification performance of the model on original data; the trained model is used for face recognition, and data used for model training is an image; a formula of the adversarial sample generated by the projected gradient descent algorithm is: ; wherein, is a loss function, is a neighborhood centered at with a radius of is a generated adversarial sample, is a sample original label;​ the first-stage fine-tuning training is specifically: the generated adversarial sample set is used for training by using a dynamic learning rate and a set number of fine-tuning rounds, and a specific optimization target formula is: ; wherein, represents an adversarial sample, represents a backdoor label, represents a loss function that measures how well the model predicts the input versus the label , the difference, represents adjusting the model parameters to minimize the loss function; The clean samples are subjected to a second phase of fine-tuning training for use for fine-tuning, the specific optimization objective formula is: ; wherein represents a clean sample, represents a true label, represents a loss function that measures how well the model predicts the input versus the label ; and represents adjusting the model parameters to minimize the loss function; a comprehensive optimization target after the first-stage fine-tuning training and the second-stage fine-tuning training is also included, and the comprehensive optimization target is represented as: ; wherein denotes an adversarial sample, denotes a backdoor label, denotes a clean sample, denotes a true label, denotes a loss function that measures how well the model predicts the input versus the label , the difference, is a hyperparameter that balances the model's performance and its ability to resist backdoor attacks.

2. The defense method against model backdoor attacks according to claim 1, characterized in that, the backdoor label is specifically obtained by using a neural cleaning method to analyze input and output of the model, and then a backdoor trigger is reversely obtained, and then L1 mask norm values corresponding to the backdoor trigger are analyzed and compared, so that the backdoor label is determined.

3. The defense method against model backdoor attacks according to claim 2, characterized in that, a formula for reversely obtaining the backdoor trigger is: ; wherein is the model, is the input sample, and δ is the backdoor trigger, is the backdoor label, denotes the minimum perturbation required to make the model predict the backdoor label.

4. A defense system against model backdoor attacks, employing the defense method of any one of claims 1-3, characterized in that, The method comprises the following steps: a backdoor label detection module is configured to obtain a backdoor label by using a neural cleaning backdoor detection method, and the backdoor label is used to guide generation of an adversarial sample; an adversarial sample generation module is configured to generate an adversarial sample by using a projected gradient descent adversarial attack algorithm; a first-stage fine-tuning module is configured to perform first-stage fine-tuning training by using the generated adversarial sample; a second-stage fine-tuning module is configured to perform second-stage fine-tuning training by using normal clean samples to recover classification performance of the model on original data; the trained model is used for face recognition, and data used for model training is an image.

Citation Information

Patent Citations

  • Method, device and system for improving backdoor defense capability of model

    CN114266041A

  • Classification neural network backdoor clearing method and system based on adversarial fine tuning

    CN116432177A