Backdoor defense method based on forgetting learning and implicit hypergradient

Abstract

The application discloses a backdoor defense method based on forgetting learning and implicit super gradient, and the method comprises the following steps: assuming a backdoor label, generating a label corresponding possible backdoor trigger and performing parameter search, obtaining a static backdoor trigger, adding disturbance, making an optimizer optimize the backdoor label in the direction deviating from the correct label, and adapting a dynamic backdoor trigger; using balanced samples to perform probability superposition; according to the parameters of the static trigger and the dynamic trigger and the predicted probability cumulative distribution, the comprehensive score of the backdoor label is calculated, and the target label is calculated; extracting model features and other information as a training set, using the trained backdoor detection model for model backdoor detection; constructing a mixed data set, performing data enhancement on the mixed data set; using the enhanced data set to perform forgetting training and multi-stage implicit super gradient training on the classification model, and deleting the backdoor of the classification model. The application effectively deletes the backdoor from the neural network model to ensure the safety and reliability of the model.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of information security, and particularly relates to a backdoor defense method based on forgetting learning and implicit super-gradient. BACKGROUND

[0002] With the rapid development of artificial intelligence technology, neural network models have been widely applied in the fields of image recognition, natural language processing, medical diagnosis, etc. However, the security problems of these models have gradually been exposed. AI backdoor problem is one of them, which poses a serious threat to the security and reliability of the model. AI backdoor, or neural network backdoor, is a security vulnerability. Attackers implant specific triggers (backdoors) in the training process, so that the model produces a preset false output when encountering certain specific input patterns. The existence of AI backdoor not only threatens the security of a single model, but also may affect the reliability of the entire system. For example, if the neural network model in the automatic driving system is implanted with a backdoor, the vehicle may misidentify traffic signals, pedestrians or obstacles, leading to traffic accidents. Therefore, how to detect and defend against AI backdoor attacks has become an important research direction in the field of artificial intelligence security.

[0003] The backdoor identification in the model can be divided into two categories. The first category is based on the reverse reconstruction of the trigger. The possible triggers are generated by reverse engineering and input into the detected model. If the trigger can be successfully triggered and meet certain conditions (such as trigger success rate, trigger area size, etc.), it is considered that the model has a backdoor model. The second category is based on the internal feature representation of the model. The features are extracted from the output of the internal specific layer of the model, and statistical, machine learning, etc. are used to identify whether there is an abnormality in the output of the model, so as to achieve the purpose of model detection. There are three main methods to delete the backdoor in the model. The first is backdoor forgetting training. The backdoor sample is introduced based on the normal sample to form the backdoor adversarial sample, so that the model forgets / anti-learns the backdoor feature, and achieves the purpose of ignoring the influence of the backdoor trigger. The second is to identify and delete the backdoor neurons through the backdoor sample. This method usually needs to reverse engineer the backdoor trigger and restore the backdoor sample by generating possible backdoor triggers. Through the backdoor sample, the neurons activated by the trigger are pruned to eliminate the influence of the backdoor in the model. The third is data set purification. The backdoor sample is identified and filtered to avoid the model from contacting the backdoor sample as much as possible during the training process, so as to prevent the model from learning the backdoor. For example: the neural network backdoor detection method and device patent (patent application number: CN202211202494.2) proposes a neural network backdoor detection method. The method first generates detection samples for each label in the model, updates the initial samples by iteration to minimize the classification loss, and then inputs these samples into the neural network to obtain the activation vector of the feature layer, which is spliced to form a feature matrix. Then, statistical analysis is performed on the feature matrix, such as principal component analysis, to determine the detection index of each label. Then, the abnormal value detection algorithm is used to calculate the abnormal index and compare it with the threshold value to determine whether the model has a backdoor and identify the target label of the attack. The neural network backdoor deletion method, device, equipment and medium patent (patent application number: CN202410224335.5) proposes a neural network backdoor deletion method. The method receives an initial model and a clean sample set, and then updates the model weight through multiple rounds of adversarial learning. In each round of adversarial learning, the model to be updated is first obtained, and then critical disturbance patterns are generated. These patterns are superimposed with the clean sample set to form a contaminated sample set, so that the prediction error rate of the model on the contaminated sample set reaches or exceeds the preset threshold. Then, the model is trained using these contaminated samples to reduce its sensitivity to disturbance patterns, and finally the model after weight update is output, effectively deleting the backdoor in the neural network and improving the security of the model.

[0004] However, the existing backdoor defense methods still have some deficiencies, such as the above: the neural network backdoor detection method and device (patent application number: CN202211202494.2) inputs the detection sample into the model when identifying the model, and the features of the model activation layer are more likely to capture the feature distribution or traces of the backdoor. The feature information and feature distribution are highly related to the task, structure and data set of the model, so it is not universal. In addition, this patent only uses the features of the sample for calculation, the judgment angle is single, and it cannot guarantee that the features are highly related to the backdoor; the neural network backdoor deletion method, device, equipment and medium (application number: CN202410224335.5) describe a critical disturbance pattern in each round of adversarial learning, which is generated for the current round of to-be-updated model. Because the disturbance pattern does not change with the change of individual clean samples, the trigger in each round is static, and after being superimposed with the clean sample set, a contaminated sample set is formed. These contaminated samples are used to train the model, the purpose is to make the model reduce the sensitivity to these fixed disturbance patterns, so as to weaken or delete the backdoor. Therefore, this method can only guarantee its effectiveness for static triggers, but cannot delete dynamic triggers.

[0005] Therefore, how to improve the universality of backdoor feature recognition and delete dynamic triggers is a problem that needs to be solved urgently at present. SUMMARY

[0006] The main purpose of the present application is to overcome the shortcomings and deficiencies of the prior art, and to provide a backdoor defense method based on forgetting learning and implicit super gradient. Through implicit super gradient training and forgetting training in stages, the backdoor can be effectively deleted from the neural network model to ensure the safety and reliability of the model.

[0007] In order to achieve the above purpose, the present application adopts the following technical scheme:

[0008] In a first aspect, the present application provides a backdoor defense method based on forgetting learning and implicit super gradient, comprising the following steps:

[0009] S1, input the classification model into the backdoor detection model to detect whether there is a backdoor, if the classification model has a backdoor, execute step S2, otherwise quit;

[0010] S2, assume all labels as backdoor labels, generate label corresponding possible backdoor triggers, adjust trigger success rate and search through joint optimization of trigger cost, obtain static backdoor triggers, add disturbance, make the optimizer optimize the backdoor label in the direction away from the correct label, obtain the optimized backdoor label, and adapt the dynamic backdoor trigger; use balanced samples to accumulate and normalize the prediction probability of each label to obtain the prediction probability accumulation distribution;

[0011] S3, calculating a comprehensive score of the backdoor label by using the backdoor label prediction model according to the parameters of the static trigger and the dynamic trigger and the predicted probability cumulative distribution, and taking the backdoor label with the highest score as the target label;

[0012] S4, generating a backdoor picture sample by using the backdoor prediction label of the multiple categories with higher prediction, adding a real label to construct a mixed data set, performing data enhancement on the mixed data set to obtain an enhanced data set, and performing multi-stage forgetting training and implicit hyper-gradient training on the classification model by using the enhanced data set to delete the backdoor of the classification model.

[0013] As a preferred technical solution, the step of assuming all labels as backdoor labels and generating labels corresponding possible backdoor triggers comprises:

[0014] Assuming all labels of the clean sample as backdoor labels;

[0015] Generating a mask and a pattern with the same dimension as the input sample, and finding a trigger (mask, pattern);

[0016] The trigger (mask, pattern) classifies the original picture as a backdoor label, and obtains a predicted backdoor label; a cross-entropy loss of the predicted backdoor label and the backdoor label is calculated, an L1 norm regularization term of the mask is added, and the mask and the pattern are optimized by using gradient descent until the cross-entropy loss and the L1 norm of the mask reach a balance, and the optimized mask and the pattern are obtained.

[0017] The optimized mask and the pattern are applied to the clean sample to generate possible backdoor triggers.

[0018] The minimum value of the L1 norm is deleted, and the L1 norm is averaged, and the L1 norm of the possible trigger corresponding to each label is normalized.

[0019] As a preferred technical solution, the step of optimizing the mask and the pattern by using gradient descent is as follows:

[0020]

[0021] wherein, indicates a target label, f(·) indicates a prediction function of the DNN, indicates a loss function for measuring classification error, λ indicates a weight of a second target, X indicates a group of clean images for solving an optimization task, and m and P respectively indicate the mask and the pattern.

[0022] As a preferred technical solution, the adjustment of the trigger success rate and the joint optimization search of the trigger cost include:

[0023] Initialize the learning rate and regularization term weights, and set the success rate range for triggering the target attack;

[0024] Calculate the loss of possible backdoor triggers corresponding to the backdoor label, use stochastic gradient descent to optimize the parameters of possible backdoor triggers, and perform forward propagation on all training samples to calculate the predicted label of the classification model and obtain the static trigger.

[0025] Check the current attack success rate and the size of the static trigger. If the current attack success rate is lower than the set lower limit, increase the learning rate and decrease the weight of the regularization term. If the current attack success rate is higher than the set upper limit, decrease the learning rate and increase the weight of the regularization term.

[0026] Save the static trigger and its parameters, including the L2 norm;

[0027] Remove the minimum value of the L2 norm, then take the mean of the L2 norm, and normalize the norm of the static trigger corresponding to each label.

[0028] As a preferred technical solution, the addition of perturbation, causing the optimizer to optimize the backdoor label in a direction deviating from the correct label direction, includes:

[0029] Initialize the learning rate and regularization term weights, generate a mask and pattern with the same dimensions as the input samples, and find the corresponding triggers (mask, pattern); optimize the triggers using gradient descent, and check the attack trigger success rate and trigger size;

[0030] The loss for the backdoor label is calculated by performing n rounds of search based on the learning rate and the weight of the regularization term; the backdoor label loss is as follows:

[0031] ;

[0032] ;

[0033] in, This represents the L2 norm of the trigger corresponding to label i in the classification model m. For real labels With predictive labels cross-entropy, This represents the loss value calculated by model m on the input sample x;

[0034] Set the attack success rate in each round to 85%-100%, select the label with the maximum probability distribution in n rounds of search as the backdoor label, obtain the optimal mask and pattern, and save the one-hot encoding of the label and the size of the trigger.

[0035] As a preferred technical solution, the prediction probability of each label is accumulated and normalized using balanced samples, including:

[0036] Extract samples of each category from the original data set to obtain a balanced sample set;

[0037] Input the clean samples into the trained classification model to obtain the prediction probability of each sample corresponding to each label ;

[0038] The prediction probability of each label is accumulated to obtain the accumulated value of each label ;

[0039] The accumulated value of each label is normalized, as follows:

[0040] ;

[0041] wherein, represents the L1 norm of the trigger corresponding to all labels of the same model, represents the L1 norm of the trigger corresponding to label i in the classification model m.

[0042] As a preferred technical solution, the comprehensive score of the backdoor label is calculated, and the backdoor label with the highest score is taken as the target label, as follows:

[0043]

[0044] wherein, , , , represents a hyperparameter, represents the optimized backdoor label, represents the result of the prediction probability accumulation of each label;

[0045] The label with the highest comprehensive score is the predicted backdoor label, and the backdoor label is taken as the predicted label of the backdoor label prediction model.

[0046] As a preferred technical solution, the target label is used to generate a backdoor picture sample, a real label is added to construct a mixed data set, the mixed data set is subjected to data enhancement to obtain an enhanced data set, and specifically:

[0047] According to the probability prediction of the plurality of categories of backdoor prediction labels, the minimum trigger corresponding to the backdoor label in the preceding step is matched and superimposed on the clean sample picture to generate a backdoor picture;

[0048] The generated backdoor sample is mixed with the real label to obtain a backdoor forgetting training data set for training the model to delete the backdoor, the backdoor sample picture, the backdoor label and the real label are mixed to obtain a backdoor test set for evaluating the attack success rate and the label accuracy, and the clean sample picture and the real label are mixed to obtain an original data set; the original data set includes an original training set and an original test set;

[0049] The backdoor forgetting training data set, the backdoor test set and the original data set are subjected to enhancement processing, including random distortion disturbance, random horizontal flip, rotation and pixel normalization;

[0050] The backdoor forgetting training data set and the original training set after data enhancement are mixed to obtain a first training set, and the original training set and the original test set after data enhancement are mixed to obtain a second training set.

[0051] As a preferred technical solution, the backdoor deletion of the classification model includes: multi-stage mixed training of the classification model, n rounds of implicit super-gradient training and 1 round of forgetting training are used in each round of mixed training; the backdoor deletion effect of the classification model is evaluated through the comprehensive score of the model; the best model result in the backdoor deletion effect evaluation is selected; n rounds of implicit super-gradient training are added with forgetting training, and the classification model is subjected to multi-stage mixed training again;

[0052] The forgetting training includes:

[0053] The first training set is input into the classification model, the problematic weight is determined and updated through training, the classification model is caused to forget the original label corresponding to the backdoor trigger through forgetting training, and the normal label of the backdoor sample is relearned;

[0054] The n rounds of implicit super-gradient training include:

[0055] The second training set is input into the classification model, the internal loss of the classification model is calculated, the backdoor trigger with the highest trigger factor is found out, the external loss of the classification model is calculated to optimize the model parameters, and the effect of the trigger on the classification model is reduced;

[0056] The weight of the model is updated through the fixed point iteration method to obtain a trained classification model, as follows:

[0057]

[0058] wherein, represents a real label, a predicted label representing a model, a sample, a perturbation, a model parameter, and L represents a loss function;

[0059] The trained classification model is evaluated by using the backdoor test set and the original data test set, and the comprehensive score score of the current classification model is calculated as follows:

[0060] score = test_acc – test_asr + test_ra

[0061] Wherein, test_acc represents the test accuracy, test_asr represents the backdoor attack success rate, and test_ra represents the recovery attack success rate; if the comprehensive score score of the current classification model is higher than the maximum score, the maximum score and the best model state are updated, and the best model state is saved.

[0062] As a preferred technical solution, the step S1 comprises:

[0063] The training set is input into the backdoor detection model, and the backdoor detection model weight parameters are updated by supervised learning, and the loss between the model output and the true label is calculated using the binary classification cross-entropy loss function, so as to judge whether there is a backdoor in the model; the training set includes normal model, classification model and backdoor state, and the backdoor state includes whether there is a backdoor model and backdoor label category.

[0064] Compared with the prior art, the present application has the following advantages and beneficial effects:

[0065] (1) The present application can use the restored trigger feature as a classification feature by optimizing the size of the prediction label and the backdoor label cross-entropy trigger, and by optimizing the two parameters of the balanced success rate and the trigger size relationship, and by ASR and trigger cost joint optimization search, accurate static trigger restoration and trigger cost evaluation are realized.

[0066] (2) The present application uses an optimizer to autonomously select a backdoor label method, which requires input of a normal data set and a classification model, adds perturbation on the normal data set, calculates the loss between the predicted label and the true label, optimizes the predicted label as much as possible to deviate from the original correct label direction, so as to find the optimal label, and select different original samples for multiple times, adapt dynamic trigger, so that according to the input of different samples, the direction of easy deviation from the original label is optimized autonomously, and the problem of not being able to restore static trigger is solved.

[0067] (3) The present application accumulates the prediction probability of each label, uses the prediction probability distribution as a feature, inputs the clean sample, and averages the probability of predicting the label after superposition, without restoring the trigger, thereby avoiding the problem of inaccuracy or high concealment of the restoring trigger.

[0068] (4) The method for constructing a backdoor detection model and a backdoor label prediction model after training can be evaluated for different data sets and different models, solving the problem of insufficient adaptability of traditional backdoor detection methods. The enhancement of the data set can enhance the generalization ability of the model while ensuring the diversity of the sample. The use of multi-stage implicit hyper-gradient training and forgetting training solves the problem of model capability degradation when deleting the model backdoor, which can prevent the catastrophic forgetting of the model while ensuring the accuracy of the model. BRIEF DESCRIPTION OF DRAWINGS

[0069] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed in the embodiment description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0070] Figure 1 The flowchart of the backdoor defense method of the present application embodiment based on forgetting learning and implicit hyper-gradient;

[0071] Figure 2 The flowchart of the backdoor detection model of the present application;

[0072] Figure 3 The flowchart of the backdoor label method of the present application embodiment;

[0073] Figure 4 The flowchart of the parameter search method of the present application;

[0074] Figure 5 The flowchart of the optimizer autonomous selection backdoor label method of the present application;

[0075] Figure 6 The flowchart of the probability superposition method of the present application;

[0076] Figure 7 The flowchart of the backdoor label calculation of the present application;

[0077] Figure 8 The flowchart of the mixed data set construction of the present application;

[0078] Figure 9 The flowchart of the n-round implicit hyper-gradient training and 1-round direct forgetting training and evaluation of the present application embodiment;

[0079] Figure 10 For multi-round mixed training flowchart. DETAILED DESCRIPTION

[0080] In order to enable personnel in the technical field to better understand the scheme of the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below in combination with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the present application.

[0081] In the present application, the phrase "embodiments" means that the specific features, structures or characteristics described in combination with the embodiments can be included in at least one embodiment of the present application. The appearance of this phrase at various places in the specification does not necessarily mean the same embodiment, nor is it an independent or alternative embodiment to other embodiments. Those skilled in the art explicitly and implicitly understand that the embodiments described in the present application can be combined with other embodiments.

[0082] Implicit super-gradient: here refers to a gradient that estimates the gradient of the calculation model without direct calculation but through iteration and approximation techniques.

[0083] Please refer to Figure 1 The present embodiment provides a backdoor defense method based on forgetting learning and implicit super-gradient, comprising the following steps:

[0084] S1, before deleting the backdoor of the classification model, pre-training a backdoor detection model through supervised learning, as shown in Figure 2 The pre-trained backdoor detection model is used to determine whether the model has a backdoor, including the following steps: assuming that there is a neural network model as a classification model, inputting the classification model into the backdoor detection model to determine whether the model has a backdoor, if it does, calculating the backdoor label and generating a backdoor picture according to the assumed backdoor label method, parameter search method, optimizer autonomous selection backdoor label method and probability superposition method; using the generated backdoor picture and clean samples to construct an enhanced data set; then using forgetting training and implicit super-gradient training to delete the backdoor in the classification model; if not, exit. The above "assumed backdoor label method", "parameter search method", "optimizer autonomous selection" and "probability superposition method" will be described one by one later.

[0085] Next, the backdoor label prediction and backdoor deletion part based on forgetting learning and implicit super-gradient is executed, as steps S2-S3.

[0086] S2, all label hypotheses are assumed to be backdoor labels, generate label corresponding possible backdoor triggers, adjust trigger success rate and search through joint optimization of trigger cost to obtain static backdoor triggers, add perturbations, make the optimizer optimize the backdoor label in the direction away from the correct label, obtain the optimized backdoor label, and adapt the dynamic backdoor trigger; the prediction probability of each label is accumulated and normalized using balanced samples to obtain the prediction probability cumulative distribution; the balanced samples include normal samples and error samples in the same proportion.

[0087] First, please refer to Figure 3 , assume the backdoor label, that is, assume that all labels may be backdoors, optimize the size of the trigger using the cross-entropy between the predicted label and the backdoor label, obtain the possible trigger and its L1 norm, and normalize the L1 norm.

[0088] Traverse all labels, set as target labels, and perform the following operations on clean samples:

[0089] S201, initialize a mask and a pattern with the same dimension as the input sample. The pattern is a three-dimensional matrix with the same dimension as the input image (including height, width and color channel). Mask is a two-dimensional matrix that determines how much of the original image the trigger can cover.

[0090] S202, find a trigger (mask, pattern) that will incorrectly classify the original image as the target label; define the objective function as the cross-entropy loss between the predicted label and the target label, plus the L1 norm regularization term of mask. The L1 norm is used to control the sparsity of mask, so that the trigger is as small as possible.

[0091] Use gradient descent or other optimization algorithms to optimize mask and pattern until the cross-entropy loss and the L1 norm of mask reach equilibrium, i.e. the loss function converges.

[0092] The optimization formula (1) is as follows:

[0093]

[0094] where, is the target label, f(·) is the prediction function of DNN, is the loss function that measures the classification error, and λ is the weight of the second objective; a smaller λ has a lower weight on the control of trigger size, but a higher success rate of producing misclassification, and the optimization process will dynamically adjust λ. X is a set of clean images that solve the optimization task, and m and P represent mask and pattern, respectively.

[0095] S203. After optimization, the mask and pattern for the current target label are obtained. The mask and pattern are applied to clean samples to generate possible triggers. The formula (2) for generating the triggers is as follows:

[0096]

[0097] Where A represents the function that superimposes the trigger onto the original sample x, and p represents the pattern of the trigger. These represent the height, width, and color channels, respectively. m is a 2D matrix called mask, with values ​​ranging from 0 to 1, used for specific pixels (...). )of When =1, the trigger completely rewrites the original color.

[0098] S204. Delete the minimum value of the L1 norm, then take the average of the L1 norms, and normalize the L1 norm of the possible triggers corresponding to each label.

[0099] To enable comparison of different models on the same dataset, the minimum value of the L1 norm is removed, and then the mean of the L1 norm is taken. The L1 norm of each label's corresponding trigger is divided by the mean norm, and then multiplied by 100 to obtain the normalized L1 norm value that can be compared across models.

[0100]

[0101] in, This represents the L1 norm of the triggers corresponding to all labels in the same model. This represents the L1 norm of the trigger labeled i in model m.

[0102] Secondly, testing revealed a positive correlation between the size of the backdoor trigger and the attack success rate. Starting from a 97% attack success rate, the increase in backdoor trigger size varied significantly for every 0.1% increase in success rate. For ease of comparison, the trigger success rate was set to 95%. Furthermore, by optimizing two parameters (lr parameter and cost parameter) to balance the relationship between success rate and trigger size, a single-epoch backdoor trigger capable of meeting the 95%-96% attack success rate constraint was obtained, improving search speed.

[0103] Please see Figure 4The embodiment adopts a parameter search method, which obtains a backdoor trigger capable of meeting the restriction condition of 95%-96% attack success rate by optimizing two parameters of the balance between success rate and trigger size, and specifically, the static backdoor trigger is obtained by adjusting trigger success rate and searching through joint optimization of trigger cost, and the static backdoor trigger comprises the following steps:

[0104] S205, initialize learning rate lr and regularization term cost weight, and set the trigger success rate ASR range of the target attack.

[0105] S206, calculate the loss of the possible backdoor trigger corresponding to the backdoor label, use stochastic gradient descent to optimize the parameters of the possible backdoor trigger, and perform forward propagation on all training samples to calculate the predicted label of the classification model, and obtain the static trigger. The loss of the possible backdoor trigger is as follows:

[0106]

[0107] Wherein, represents the L2 norm of the trigger corresponding to the label i in the classification model m. is the target label and the predicted label cross entropy, represents the loss value calculated by the model m on the input sample x. mean() represents the average loss of the entire batch.

[0108] S207, check the current trigger success rate and static trigger size, if the current attack success rate is lower than the set lower limit, increase the learning rate and reduce the weight of the regularization term, if the current attack success rate is higher than the set upper limit, reduce the learning rate and increase the weight of the regularization term.

[0109] S208, save the static trigger and its parameters, including the L2 norm.

[0110] S209, delete the minimum value of the L2 norm, and then take the mean value of the L2 norm, and normalize the norm of the static trigger corresponding to each label. The normalization process is as formula (3).

[0111] The above "hypothetical backdoor label" and "parameter search" both use the restored trigger features as classification features, and through the joint optimization search of ASR and trigger cost, accurate static trigger restoration and trigger cost evaluation are realized.

[0112] Next, please refer to Figure 5This embodiment employs an optimizer that autonomously selects backdoor labels. It requires inputting a normal dataset and a classification model, adding perturbations to the normal dataset, calculating the loss between the predicted label and the true label, and optimizing the predicted label as far as possible away from the original correct label, thereby finding the optimal label. This process involves multiple selections from different original samples to adapt to dynamic triggers.

[0113] It's worth explaining that "adding perturbation" means adding triggers to the original image, adding different triggers for different images to generate backdoor images with dynamic triggers; also, "optimizer" here refers to the algorithm in the PyTorch library used to update the parameters of a neural network model to minimize the loss function. The optimizer helps the model learn by updating the network weights during training, optimizing in a direction that deviates from the original correct label, thereby finding the optimal label.

[0114] Specifically, adding perturbation to cause the optimizer to optimize the backdoor label in a direction deviating from the correct label, and obtaining the optimized backdoor label, includes the following steps:

[0115] S210, Initialize parameters and generate initial triggers.

[0116] S211. Calculate the loss of the trigger, as follows:

[0117]

[0118] in, This represents the L2 norm of the trigger with label i in model m. For real labels With predictive labels cross-entropy, This represents the loss value calculated by model m on the input sample x. To optimize as far as possible away from the original correct label, this is where... Invert the value. mean() represents the average loss for the entire batch.

[0119] S212. Optimize the trigger and check the attack success rate and trigger size.

[0120] After step S212, the learning rate lr and the regularization term cost under the constraints are obtained.

[0121] S213. According to formulas (6) and (7), n rounds of search are performed under the constraints of learning rate lr and regularization term cost. Under the condition that the attack success rate in each round is above 90%, the label with the largest probability distribution in the n rounds of search is selected as the backdoor label, and the optimal mask and pattern are obtained.

[0122] S214, save the one-hot encoding of the label and the size of the trigger.

[0123] Finally, please refer to Figure 6 , the embodiment also adopts the method of probability superposition, that is, the balanced sample is input into the classification model, the prediction probability of each label is accumulated and normalized, and the prediction probability accumulation distribution is obtained. This method uses the prediction probability distribution as a feature, inputs the clean sample and averages the probability superposition of the predicted label, without restoring the trigger.

[0124] The method assumes that the probability corresponding to the normal label is relatively low when predicting other categories, that is, after inputting the normal sample into the normal model, the probability corresponding to the correct prediction label is larger, the probability of being divided into other error labels is smaller, and even negative, but the overall distribution probability is relatively balanced. In order to trigger the label by modifying a small amount of triggers, the backdoor label needs to keep a high probability when predicting normal samples, so that a small amount of value change can make the prediction probability of the backdoor label the largest, thereby triggering the backdoor.

[0125] Specifically, the following steps are included:

[0126] S215, extract samples of each category from the original data set to obtain a balanced sample set.

[0127] In order to ensure that the number of samples of each class is the same, first, samples of each category need to be extracted from the original data set. Let the original data set contain categories, and the sample set of each category is denoted as where . Randomly extract s samples from each to form a balanced data set D′. Merge the balanced samples of all categories to form a batch data set:

[0128] D′ = , where represents the i-th sample extracted from the k-th category.

[0129] S216, input the clean sample into the trained classification model to obtain the prediction probability of each sample corresponding to each label , as follows:

[0130]

[0131] S217, accumulate the prediction probability of each label to obtain the accumulation value of each label , as follows:

[0132]

[0133] S218, normalize the accumulated value of each label according to formula (3)

[0134] S219, calculate the loss between the model output and the true label using the cross-entropy loss function of binary classification, and obtain the trained backdoor detection model.

[0135] Regarding the training of the backdoor detection model, the following steps are included:

[0136] (1) Define the network structure of the backdoor detection model: the feature vector of the input layer is x e , where d is the dimension of the input feature. The hidden layer is a number of neurons, using ReLU activation function. The output layer has two neurons, outputting the probability of the model having a backdoor or not. Let the parameters of the model be , the input feature vector be x, and the output of the model y be y = softmax( x). Where softmax is an activation function for binary classification.

[0137] (2) Construction of the training set: extract the features of the normal model and the model with a backdoor, and use the model features and the backdoor status (whether the model has a backdoor, the backdoor label) as the training set.

[0138] (3) Model training: training through supervised learning, using the cross-entropy loss function of binary classification to calculate the loss between the model output and the true label.

[0139] S3, according to the parameters of the static trigger and the dynamic trigger, and the accumulated distribution of the prediction probability, calculate the comprehensive score of the backdoor label, and take the multiple backdoor labels with the highest score as the target label.

[0140] In this embodiment, please refer to Figure 7 , assuming that the task of the classification model has n classification labels. Assume that the trigger L1 norm normalization result L1 obtained by the backdoor label method, the trigger L2 norm normalization result L2 obtained by the parameter search method, the one-hot encoding of the most probable label PL of the model obtained by the optimizer autonomous selection backdoor label method, and the normalized prediction probability NP of the signature obtained by the probability stacking method. The values of all data are scaled to the value range of 0-1.

[0141] The comprehensive score for label i is:

[0142]

[0143] ​​Finally, the label with the highest comprehensive score is taken as the predicted backdoor label, and the backdoor label is taken as the predicted label of the backdoor label prediction model. In order to be able to contain more backdoor features in the process of generating backdoor samples, the embodiment selects the images corresponding to the top three class target labels with higher prediction as the training set.

[0144] S4, generating backdoor picture samples by using the backdoor prediction labels of multiple classes with higher prediction, adding real labels to construct a mixed data set, performing data enhancement on the mixed data set to obtain an enhanced data set, and using the enhanced data set to perform multi-stage forgetting training and implicit super-gradient training on the classification model to delete the backdoor of the classification model.

[0145] Please refer to Figure 8 The so-called mixed data set includes four data sets, namely, a first training set, a second training set, a backdoor test set, and an original data test set.

[0146] In order to obtain the four data sets, the embodiment adopts the following steps:

[0147] S401, generating backdoor picture samples: according to the top three target labels with the highest probability generated by the label prediction model, matching the minimum trigger corresponding to the target label in the foregoing step, and superimposing the clean sample picture to generate a backdoor picture.

[0148] S402, constructing a data set: the generated backdoor samples and real labels are combined to form a backdoor forgetting training data set for forgetting training, the backdoor samples, backdoor labels, and real labels are combined to form a backdoor test set for evaluating attack success rate (ASR) and label accuracy (RA), and clean samples and real labels are combined to form original training and test data sets.

[0149] S403, data enhancement: the data enhancement of the training set includes random distortion disturbance, random horizontal flip, and rotation. The test set only performs basic size conversion and pixel value normalization.

[0150] The enhancement of the data set can enhance the generalization ability of the model while ensuring the diversity of the samples.

[0151] S404, constructing a mixed data set, namely:

[0152] The backdoor forgetting training data set after data enhancement and the original training set are mixed to obtain a first training set, and the original training set after data enhancement and the original test set are mixed to obtain a second training set.

[0153] Next, please refer to Figure 9 and Figure 10 The mixed data set is used to retrain the classification model, thereby solving the problem of degradation of the classification model when deleting the model backdoor.

[0154] To solve this problem, the embodiment attempts to use a method combining multi-stage forgetting learning and n rounds of implicit super-gradient training to delete the backdoor of the classification model, including: performing multi-stage mixed training on the classification model, using n rounds of implicit super-gradient training and 1 forgetting training in each round of mixed training; evaluating the backdoor deletion effect of the classification model through the comprehensive score of the model; selecting the best model result in the backdoor deletion effect evaluation; adding one forgetting training in n rounds of implicit super-gradient training, and performing multi-stage mixed training on the classification model again. Implicit super-gradient dynamically clears the backdoor sensitivity of the classification model with great effort, and forgetting training mainly suppresses the backdoor while restoring the normal classification ability of the model through the restored trigger features.

[0155] Further, the specific steps are as follows:

[0156] The forgetting learning includes step S405.

[0157] S405, input the first training set into the classification model, and the model determines which weights have problems and updates through training. Let the model forget the original label corresponding to the backdoor trigger training, and relearn the normal label of these backdoor samples, so as to forget the backdoor, thereby achieving the effect of deleting the model backdoor.

[0158] The implicit super-gradient training includes steps S406-S407.

[0159] S406, input the second training set into the classification model, calculate the inner loss of the classification model, find out the backdoor trigger with the highest trigger factor, calculate the outer loss of the classification model to optimize the model parameters, and reduce the effect of the trigger on the classification model.

[0160] In this case, the inner loss of the classification model is calculated by formula (11) to obtain the optimal perturbation The weight of the classification model is updated by the fixed point iteration method. It should be explained that in the embodiment, the maximization of the inner loss is that the difference between the label of the backdoor picture generated by superimposing the generated trigger on the original picture and input into the detection model and the true label is larger, and the minimization of the outer loss is to find a suitable model parameter θ to minimize the loss value of the function H(·). The purpose is to optimize the model parameters to minimize the influence of the trigger on the model, and realize the forgetting of the backdoor trigger features.

[0161] S407, update the weight of the classification model by the fixed point iteration method, and obtain the trained classification model, as follows:

[0162]

[0163] wherein, is the true label, a predicted label of the model, is a model parameter.

[0164] The backdoor removal effect of the classification model is evaluated, including step S408.

[0165] S408, using the backdoor test set and the third test set to evaluate the trained classification model, and calculating the comprehensive score score of the current classification model as follows:

[0166]

[0167] Wherein, test_acc represents the test accuracy, test_asr represents the backdoor attack success rate, and test_ra represents the recovery attack success rate; if the comprehensive score score of the current classification model is higher than the maximum score, update the maximum score and the best model state, and save the best model state.

[0168] Since direct forgetting training reduces the success rate of backdoor attacks while also reducing the task prediction accuracy, implicit hyper-gradient training has less impact on task prediction accuracy, and also has the effect of improving the original task prediction accuracy to some extent, so one forgetting training is added in n rounds of implicit hyper-gradient training. In order to effectively find and optimize the best model performance in limited resources and time, the performance of the model is gradually improved by gradually increasing the depth and breadth of training. The specific steps are as follows:

[0169] (1) Preliminary training: each model is trained for k rounds to obtain the comprehensive score score_i of the model, i=1,2,...,n.

[0170] (2) Selection: select the best m models.

[0171] (3) Increase training rounds: additional l rounds of training are performed on the m models.

[0172] (4) Final evaluation: evaluate the performance of the m models again, and select the best model.

[0173] (5) Long round training: the best model selected finally is trained for t rounds.

[0174] (6) Model saving: save the best model selected according to the backdoor removal effect evaluation result, not the model trained finally.

[0175] After the above 6 steps, the final model is obtained.

[0176] It should be noted that, for the foregoing method embodiments, for the sake of brief description, they are all expressed as a series of action combinations, but those skilled in the art should know that the present application is not limited to the action sequence described, because according to the present application, certain steps can be performed in other sequences or simultaneously.

[0177] The technical features of the above embodiments can be combined in any manner. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described, but as long as the combination of the technical features does not exist contradictory, it should be considered as the scope of the present disclosure.

[0178] The above embodiments are the preferred embodiments of the present application, but the embodiments of the present application are not limited to the above embodiments, and any changes, modifications, substitutions, combinations, simplifications made without departing from the spirit and principles of the present application shall be equivalent replacement methods and shall be within the scope of the present application.

Claims

1. A backdoor defense method based on forgetting learning and implicit supergradients, characterized in that, Includes the following steps: S1. Input the classification model into the backdoor detection model to check if a backdoor exists. If the classification model has a backdoor, proceed to step S2; otherwise, exit. S2. Assume all labels are backdoor labels, generate possible backdoor triggers for each label, adjust the trigger success rate and search through joint optimization of trigger cost to obtain static backdoor triggers, add perturbations to make the optimizer optimize backdoor labels in a direction that deviates from the correct labels, obtain optimized backdoor labels, and adapt them to dynamic backdoor triggers. The predicted probabilities of each label are accumulated and normalized using balanced samples to obtain the accumulated distribution of predicted probabilities. S3. Based on the parameters of the static backdoor trigger and the dynamic backdoor trigger, as well as the cumulative distribution of the predicted probability, calculate the comprehensive score of the backdoor label using the backdoor label prediction model, and take the multiple backdoor labels with the highest scores as the target labels. S4. Generate backdoor image samples using target labels, add real labels to construct a hybrid dataset, perform data augmentation on the hybrid dataset, and obtain the augmented dataset. By using augmented datasets to perform multi-stage forgetting training and implicit supergradient training on the classification model, the backdoor of the classification model is removed.

2. The backdoor defense method based on forgetting learning and implicit supergradients according to claim 1, characterized in that, The step of assuming all tags are backdoor tags and generating possible backdoor triggers for each tag includes: Iterate through all labels of the clean samples and assume that each label is a backdoor label; Generate a mask and pattern with the same dimensions as the input sample, and find the backdoor trigger (mask, pattern); The backdoor trigger (mask, pattern) misclassifies the original image as a backdoor label and obtains the predicted backdoor label; calculates the cross-entropy loss between the predicted backdoor label and the backdoor label, adds the L1 norm regularization term of the mask, and uses gradient descent to optimize the mask and pattern until the cross-entropy loss and the L1 norm of the mask reach a balance, and obtains the optimized mask and pattern. The optimized mask and pattern are applied to clean samples to generate potential backdoor triggers; Remove the minimum value of the L1 norm, then take the mean of the L1 norm, and normalize the L1 norm of the possible backdoor triggers for each label.

3. The backdoor defense method based on forgetting learning and implicit supergradients according to claim 1, characterized in that, The process of adjusting the trigger success rate and obtaining a static backdoor trigger through joint optimization search of trigger costs includes: Initialize the learning rate and regularization term weights, and set the success rate range for triggering the target attack; Calculate the loss of possible backdoor triggers corresponding to the backdoor label, use stochastic gradient descent to optimize the parameters of possible backdoor triggers, and perform forward propagation on all training samples to calculate the predicted label of the classification model and obtain the static backdoor trigger. Check the current attack success rate and the size of the static backdoor trigger. If the current attack success rate is lower than the set lower limit, increase the learning rate and decrease the weight of the regularization term. If the current attack success rate is higher than the set upper limit, decrease the learning rate and increase the weight of the regularization term. Save the static backdoor trigger and its parameters, including the L2 norm; Remove the minimum value of the L2 norm, then take the mean of the L2 norm, and normalize the norm of the static backdoor trigger corresponding to each label.

4. The backdoor defense method based on forgetting learning and implicit supergradients according to claim 1, characterized in that, The addition of perturbation, causing the optimizer to optimize the backdoor label in a direction deviating from the correct label, includes: Initialize the learning rate and regularization term weights, generate a mask and pattern with the same dimensions as the input samples, and find the corresponding triggers (mask, pattern); optimize the triggers using gradient descent, and check the attack trigger success rate and trigger size; Calculate the loss for the backdoor label and perform n rounds of search based on the learning rate and the weights of the regularization term; Set the attack success rate in each round to 85%~100%, select the label with the highest probability distribution in n rounds of search as the backdoor label, obtain the optimal mask and pattern; save the one-hot encoding of the label and the trigger size.

5. The backdoor defense method based on forgetting learning and implicit supergradients according to claim 1, characterized in that, The step of accumulating and normalizing the predicted probabilities of each label using balanced samples includes: Extract samples from each category from the original dataset to obtain a balanced sample set; Input clean samples into the trained classification model to obtain each sample. Corresponding tags The predicted probability; The predicted probabilities of each label are summed to obtain the probability of each label. The cumulative value ; For each tag The cumulative value Normalize.

6. The backdoor defense method based on forgetting learning and implicit supergradients according to claim 1, characterized in that, The process involves generating backdoor image samples using target labels, adding real labels to construct a hybrid dataset, and then performing data augmentation on the hybrid dataset to obtain the augmented dataset. Specifically: Based on the backdoor prediction labels of multiple categories with high probability prediction, match the smallest trigger corresponding to the backdoor label, and overlay it onto the clean sample image to generate the backdoor image. The generated backdoor samples are mixed with the real labels to obtain the backdoor forgotten training dataset, which is used to train the classification model to remove the backdoor. The backdoor sample images, backdoor labels, and real labels are mixed to obtain the backdoor test set, which is used to evaluate the attack success rate and label accuracy. The clean sample images are mixed with the real labels to obtain the original dataset. The original dataset includes the original training set and the original test set. The backdoor forgetting training dataset, backdoor test dataset, and original dataset are augmented with random twisting perturbations, random horizontal flips, rotations, and pixel normalization. The augmented backdoor forgetting training dataset is mixed with the original training set to obtain the first training set. The augmented original training set is mixed with the original test set to obtain the second training set.

7. The backdoor defense method based on forgetting learning and implicit supergradients according to claim 6, characterized in that, The backdoor removal of the classification model includes: performing multi-stage hybrid training on the classification model, using n rounds of implicit supergradient training and forgetting training in each round of hybrid training; evaluating the backdoor removal effect of the classification model through the comprehensive score of the classification model; selecting the classification model result with the best backdoor removal effect in the evaluation; adding forgetting training in the n rounds of implicit supergradient training, and performing multi-stage hybrid training on the classification model again. The forgetting training includes: The first training set is input into the classification model. The weights of the problematic samples are determined and updated through training. The classification model is trained by forgetting the original backdoor trigger and then trained to relearn the normal labels of the backdoor samples. The n rounds of implicit supergradient training include: Input the second training set into the classification model, calculate the internal loss of the classification model, find the backdoor trigger with the highest triggering factor, calculate the external loss of the classification model, optimize the model parameters, and reduce the effect of the trigger on the classification model. The weights of the model are updated using a fixed-point iteration method to obtain a trained classification model. The evaluation of the backdoor removal effect of the classification model through a comprehensive score includes: The trained classification model is evaluated using both the backdoor test set and the original data test set. The overall score of the current classification model is calculated as follows: score = test_acc – test_asr + test_ra Where test_acc represents the test accuracy, test_asr represents the backdoor attack success rate, and test_ra represents the recovery attack success rate; if the current classification model's overall score is higher than the maximum score, then the maximum score and the best model state are updated, and the best model state is saved.

8. The backdoor defense method based on forgetting learning and implicit supergradients according to claim 1, characterized in that, Step S1 includes: The training set is input into the backdoor detection model, and the model is trained through supervised learning. The weight parameters of the backdoor detection model are updated, and the cross-entropy loss function of binary classification is used to calculate the loss between the output of the backdoor detection model and the true label, thereby determining whether there is a backdoor in the classification model. The training set includes the normal model, the classification model, and the backdoor state. The backdoor state includes whether there is a backdoor and the backdoor label category.

Images