Back door defense method based on forgetting learning and implicit hypergradient

Abstract

The invention discloses a back door defense method based on forgetting learning and implicit hypergradient. The method comprises the steps of assuming a back door label, generating a possible back door trigger corresponding to the label and performing parameter search, obtaining a static back door trigger, adding disturbance, enabling an optimizer to optimize the back door label in a direction deviating from a correct label, and adapting to a dynamic back door trigger. Carrying out probability superposition by using the balanced sample; according to the parameters of the static trigger and the dynamic trigger and the prediction probability accumulative distribution, calculating a comprehensive score of the backdoor tag, and calculating a target tag; information such as model features is extracted as a training set, and the trained backdoor detection model is used for model backdoor detection; constructing a mixed data set, and performing data enhancement on the mixed data set; and performing forgetting training and multi-stage implicit super-gradient training on the classification model by using the enhanced data set, and deleting the back door of the classification model. According to the method, the backdoor is effectively deleted from the neural network model, so that the safety and reliability of the model are ensured.

Description

Technical Field

[0001] The present invention belongs to the technical field of information security, and in particular relates to a backdoor defense method based on forgetting learning and implicit super gradient. Background Art

[0002] With the rapid development of artificial intelligence technology, neural network models have been widely used in image recognition, natural language processing, medical diagnosis and other fields. However, the security issues of these models have gradually been exposed. The AI ​​backdoor problem is one of them, which poses a serious threat to the security and reliability of the model. AI backdoor, or neural network backdoor, is a security vulnerability. The attacker implants a specific trigger (backdoor) during the training process, causing the model to produce a preset error output when encountering certain specific input patterns. The existence of AI backdoors not only threatens the security of a single model, but is also likely to affect the reliability of the entire system. For example, if a neural network model in an autonomous driving system is implanted with a backdoor, the vehicle may misidentify traffic signals, pedestrians or obstacles, leading to traffic accidents. Therefore, studying how to detect and defend against AI backdoor attacks has become an important research direction in the field of artificial intelligence security.

[0003] The identification of backdoors in models can be divided into two categories. The first category is based on the reverse reconstruction of triggers. Possible triggers are generated through reverse engineering and input into the detected model. If the trigger can be successfully triggered and meets specific conditions (such as the trigger success rate, the size of the trigger area, etc.), the model is considered to have a backdoor model. The second category is based on the internal feature representation of the model. Features are extracted from the output of a specific internal layer of the model. Statistics, machine learning and other methods are used to identify whether there are anomalies in the output of the model, thereby achieving the purpose of model detection. There are three mainstream methods for deleting backdoors in models. The first is backdoor forgetting training. Backdoor samples are introduced on the basis of normal samples to form backdoor adversarial samples, so that the model forgets / anti-learns the backdoor features, so that the model ignores the influence of the backdoor trigger. The second is to identify and delete the backdoor neurons through the backdoor samples. This method usually requires reverse engineering of the backdoor trigger and generating possible backdoor triggers to restore the backdoor samples. The neurons activated by the trigger are further determined through the backdoor samples, and they are pruned to eliminate the backdoor influence in the model. The third is data set purification, which identifies and filters backdoor samples to avoid contact with backdoor samples as much as possible during model training, thereby preventing the model from learning backdoors. For example: The patent for neural network backdoor detection method and device (patent application number: CN202211202494.2) proposes a neural network backdoor detection method. This method first generates detection samples for each label in the model, iteratively updates the initial samples to minimize the classification loss, and then inputs these samples into the neural network to obtain the activation vector of the feature layer, splicing to form a feature matrix; then the feature matrix is ​​statistically analyzed, such as principal component analysis, to determine the detection index of each label; then the outlier detection algorithm is used to calculate the anomaly index and compare it with the threshold to determine whether the model has a backdoor and identify the target label of the attack. The patent for neural network backdoor removal method, device, equipment and medium (patent application number: CN202410224335.5) proposes a neural network backdoor removal method, which receives an initial model and a set of clean samples, and then updates the model weights through multiple rounds of adversarial learning. In each round of adversarial learning, the model to be updated is first obtained, and then critical perturbation patterns are generated. These patterns are superimposed on the clean sample set to form a contaminated sample set, so that the prediction error rate of the model on the contaminated samples reaches or exceeds the preset threshold. Subsequently, these contaminated samples are used to train the model to reduce its sensitivity to perturbation patterns, and finally the model with updated weights is output, which effectively removes the backdoor in the neural network and improves the security of the model.

[0004] However, the existing backdoor defense methods still have some shortcomings, such as the above: the neural network backdoor detection method and device (patent application number: CN202211202494.2) inputs the detection sample into the model when identifying the model, and the features of the model activation layer are more likely to capture the feature distribution or traces of the backdoor. The feature information and feature distribution are highly related to the task, structure, and data set of the model, so it is not universal. In addition, this patent only uses the features of the sample for calculation, the judgment angle is single, and it cannot guarantee that its features are highly related to the backdoor; the neural network backdoor deletion method, device, equipment and medium (application number: CN202410224335.5) The critical perturbation pattern described in each round of adversarial learning is generated for the model to be updated in the current round. Because the perturbation pattern does not change with the change of individual clean samples, the trigger in each round is static, and after superimposing with the clean sample set, a contaminated sample set is formed. These contaminated samples are used to train the model, the purpose is to reduce the sensitivity of the model to these fixed perturbation patterns, thereby weakening or deleting the backdoor. Therefore, this method can only guarantee its effectiveness for static triggers, but cannot delete dynamic triggers.

[0005] Therefore, how to improve the universality of backdoor feature recognition and delete dynamic triggers is an urgent problem that needs to be solved. Summary of the invention

[0006] The main purpose of the present invention is to overcome the shortcomings and deficiencies of the prior art and to provide a backdoor defense method based on forgetting learning and implicit super gradient. Through staged implicit super gradient training and forgetting training, the backdoor can be effectively deleted from the neural network model to ensure the security and reliability of the model.

[0007] In order to achieve the above object, the present invention adopts the following technical solutions: In a first aspect, the present invention provides a backdoor defense method based on forgetting learning and implicit super gradient, comprising the following steps: S1. Input the classification model into the backdoor detection model to detect whether there is a backdoor. If the classification model has a backdoor, execute step S2, otherwise exit; S2. Assume all labels as backdoor labels, generate possible backdoor triggers corresponding to the labels, adjust the trigger success rate, and obtain static backdoor triggers through joint optimization search of trigger costs. Add disturbances to make the optimizer optimize the backdoor labels in the direction of deviating from the correct labels, obtain optimized backdoor labels, and adapt dynamic backdoor triggers. Use balanced samples to accumulate and normalize the predicted probabilities of each label to obtain the cumulative distribution of predicted probabilities. S3, according to the parameters of the static trigger and the dynamic trigger, and the cumulative distribution of the predicted probability, the comprehensive score of the backdoor label is calculated using the backdoor label prediction model, and the backdoor label with the highest score is used as the target label; S4. Generate backdoor image samples using backdoor prediction labels of multiple categories with higher predictions, add real labels to construct a mixed data set, perform data enhancement on the mixed data set to obtain an enhanced data set; use the enhanced data set to perform multi-stage forgetting training and implicit super gradient training on the classification model to delete the backdoor of the classification model.

[0008] As a preferred technical solution, assuming all tags as backdoor tags and generating possible backdoor triggers corresponding to the tags includes: Traverse all labels of clean samples and assume the labels to be backdoor labels; Generate a mask and pattern with the same dimensions as the input sample, and find the trigger (mask, pattern); The trigger (mask, pattern) misclassifies the original image as a backdoor label and obtains the predicted backdoor label; calculates the cross entropy loss between the predicted backdoor label and the backdoor label, adds the L1 norm regularization term of the mask, and uses gradient descent to optimize the mask and pattern until the cross entropy loss and the L1 norm of the mask reach a balance, and obtains the optimized mask and pattern; Apply the optimized mask and pattern to the clean sample to generate possible backdoor triggers; Delete the minimum value of the L1 norm, take the average of the L1 norm, and normalize the L1 norm of the possible triggers corresponding to each label.

[0009] As a preferred technical solution, the mask and pattern are optimized using gradient descent, as shown in the following formula: in, represents the target label, f(·) represents the prediction function of DNN, represents the loss function that measures the classification error, λ represents the weight of the second objective, X represents a set of clean images that solve the optimization task, and m and P represent mask and pattern, respectively.

[0010] As a preferred technical solution, the adjustment of the trigger success rate and the joint optimization search through the trigger cost include: Initialize the learning rate and regularization term weights, and set the trigger success rate range of the target attack; Calculate the loss of possible backdoor triggers corresponding to the backdoor labels, use stochastic gradient descent to optimize the parameters of possible backdoor triggers, perform forward propagation on all training samples, calculate the predicted labels of the classification model, and obtain static triggers; Check the current trigger success rate and static trigger size. If the current attack success rate is lower than the set lower limit, increase the learning rate and reduce the weight of the regularization term. If the current attack success rate is higher than the set upper limit, reduce the learning rate and increase the weight of the regularization term. Save static triggers and their parameters, including L2 norm; Delete the minimum value of the L2 norm, take the average of the L2 norm, and normalize the norm of the static trigger corresponding to each label.

[0011] As a preferred technical solution, the adding of disturbance to make the optimizer optimize the backdoor label in a direction deviating from the correct label includes: Initialize the learning rate and regularization weights, generate masks and patterns with the same dimensions as the input samples, and find the corresponding triggers (mask, pattern); use gradient descent to optimize the triggers, and check the attack trigger success rate and trigger size of the triggers; Calculate the loss of the backdoor label and perform n rounds of search based on the learning rate and the regularization term weight; the backdoor label loss is as follows: ; ; in, It represents the L2 norm of the trigger corresponding to label i in the classification model m. The true label With predicted labels The cross entropy of Represents the loss value calculated by model m on input sample x; Set the attack success rate in each round to 85%~100%, select the label with the largest probability distribution in n rounds of search as the backdoor label, obtain the optimal mask and pattern; save the one-hot encoding and trigger size of the label.

[0012] As a preferred technical solution, the method of accumulating and normalizing the predicted probability of each label using balanced samples includes: Extract samples of each category from the original data set to obtain a balanced sample set; Input clean samples into the trained classification model and obtain each sample The corresponding tags The predicted probability of Accumulate the predicted probability of each label to get The cumulative value of ; For each tag The cumulative value of Normalize it as follows: ; in, It represents the L1 norm of all labels corresponding to triggers of the same model. It represents the L1 norm of the trigger corresponding to label i in the classification model m.

[0013] As a preferred technical solution, the comprehensive score of the backdoor tag is calculated, and the backdoor tag with the highest score is used as the target tag, as shown in the following formula: in, , , , represents the hyperparameter, Indicates the optimized backdoor tag, Represents the result of the accumulation of predicted probabilities for each label; The label with the highest comprehensive score is the predicted backdoor label, and the backdoor label is used as the predicted label of the backdoor label prediction model.

[0014] As a preferred technical solution, the target label is used to generate backdoor image samples, real labels are added to construct a mixed data set, and the mixed data set is enhanced to obtain an enhanced data set, specifically: According to the backdoor prediction labels of multiple categories with higher probability prediction, the minimum trigger corresponding to the backdoor label in the previous step is matched and superimposed on the clean sample image to generate the backdoor image; The generated backdoor samples are mixed with the real labels to obtain a backdoor forgetting training data set for training the model to delete the backdoor; the backdoor sample images, backdoor labels and real labels are mixed to obtain a backdoor test set for evaluating the attack success rate and label accuracy; the clean sample images and real labels are mixed to obtain the original data set; the original data set includes the original training set and the original test set; The backdoor forgotten training dataset, backdoor test dataset and original dataset are enhanced, including random distortion perturbation, random horizontal flipping, rotation and pixel normalization; The data-enhanced backdoor forgotten training data set is mixed with the original training set to obtain a first training set, and the data-enhanced original training set is mixed with the original test set to obtain a second training set.

[0015] As a preferred technical solution, the backdoor of the classification model is deleted, including: performing multi-stage hybrid training on the classification model, using n rounds of implicit super gradient training and 1 forgetting training in each round of hybrid training; evaluating the backdoor deletion effect of the classification model through the comprehensive score of the model; selecting the best model result in the backdoor deletion effect evaluation; adding forgetting training to the n rounds of implicit super gradient training, and performing multi-stage hybrid training on the classification model again; The forgetting training includes: The first training set is input into the classification model. The problematic weights are determined and updated through training. The classification model is made to forget the labels corresponding to the original backdoor trigger training through forgetting training, and the normal labels of the backdoor samples are re-learned. The n rounds of implicit super gradient training include: Input the second training set into the classification model, calculate the internal loss of the classification model, find the backdoor trigger with the highest trigger factor, calculate the external loss of the classification model to optimize the model parameters, and reduce the effect of the trigger on the classification model; Update the model weights through the fixed point iteration method to obtain the trained classification model, as shown below: in, represents the true label, represents the predicted label of the model, Represents a sample, represents disturbance, represents the model parameters, L represents the loss function; The trained classification model is evaluated using the backdoor test set and the original data test set, and the comprehensive score of the current classification model is calculated as follows: score = test_acc – test_asr + test_ra Among them, test_acc represents the test accuracy, test_asr represents the backdoor attack success rate, and test_ra represents the recovery attack success rate; if the comprehensive score of the current classification model is higher than the maximum score, the maximum score and the best model state are updated, and the best model state is saved.

[0016] As a preferred technical solution, step S1 includes: The training set is input into the backdoor detection model, and the training is performed through supervised learning. The weight parameters of the backdoor detection model are updated, and the loss between the model output and the true label is calculated using the cross entropy loss function of the two classifications to determine whether there is a backdoor in the model; the training set includes a normal model, a classification model and a backdoor status, and the backdoor status includes whether there is a backdoor model and a backdoor label category.

[0017] Compared with the prior art, the present invention has the following advantages and beneficial effects: (1) The present invention optimizes the size of the trigger by utilizing the cross entropy of the predicted label and the backdoor label, and optimizes the two parameters of the relationship between the balance success rate and the trigger size. It can use the restored trigger features as classification features, and realize accurate static trigger restoration and trigger cost evaluation through the joint tuning search of ASR and trigger cost.

[0018] (2) The present invention uses a method for autonomously selecting backdoor labels through an optimizer. It needs to input a normal data set and a classification model, add disturbances to the normal data set, calculate the loss between the predicted label and the true label, and optimize the predicted label as far as possible away from the original correct label, thereby finding the optimal label. Multiple selections are made for different original samples, and dynamic triggers are adapted. In this way, the optimization is easily deviated from the original label according to the different autonomous selections of the input samples, thereby solving the problem of being unable to restore static triggers.

[0019] (3) The present invention accumulates the predicted probability of each label, uses the predicted probability distribution as a feature, inputs a clean sample and superimposes and averages the probabilities of its predicted labels. It does not need to restore the trigger, thus avoiding the problem of inaccurate restored triggers or high concealment of the triggers.

[0020] (4) The method of constructing a training set to train a backdoor detection model and a backdoor label prediction model can be evaluated for different data sets and different models, solving the problem of insufficient adaptability of traditional backdoor detection methods. The enhancement of the data set can enhance the generalization ability of the model while ensuring sample diversity. The use of multi-stage implicit super gradient training and forgetting training solves the problem of model capability degradation when deleting the model backdoor, and can prevent catastrophic forgetting of the model while ensuring the accuracy of the model while ensuring the backdoor removal effect. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0022] Figure 1 A flowchart of a backdoor defense method based on forgetting learning and implicit super gradient according to an embodiment of the present invention; Figure 2 A flow chart of the backdoor detection model implemented in the present invention; Figure 3 This is a flow chart of a hypothetical backdoor tag method according to an embodiment of the present invention; Figure 4 A flow chart of the parameter search method for implementing the present invention; Figure 5 A flow chart of the method for implementing the optimizer to autonomously select backdoor labels in the present invention; Figure 6 A flow chart of the probability superposition method for implementing the present invention; Figure 7 A flow chart for calculating backdoor tags for the present invention; Figure 8 A flow chart for constructing a hybrid data set for the implementation of the present invention; Fig. 9 This is a flow chart of n rounds of implicit super gradient training and 1 round of direct forgetting training and evaluating their effects according to an embodiment of the present invention; Fig.10 This is a flowchart of multiple rounds of mixed training. DETAILED DESCRIPTION

[0023] In order to enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without making creative work are within the scope of protection of the present application.

[0024] Reference to "embodiments" in this application means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present application. The appearance of the phrase in various locations in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described in this application may be combined with other embodiments.

[0025] Implicit super gradient: Here it refers to a gradient that is not directly calculated but estimated through iteration and approximation techniques to calculate the model gradient.

[0026] See also Figure 1 , this embodiment provides a backdoor defense method based on forgetting learning and implicit super gradient, comprising the following steps: S1. Before deleting the backdoor of the classification model, a backdoor detection model is pre-trained through supervised learning, such as Figure 2As shown. Using the pre-trained backdoor detection model to determine whether the model has a backdoor, the following steps are included: assuming that there is a neural network model as a classification model, input the classification model into the backdoor detection model to determine whether the model has a backdoor. If so, the backdoor label is calculated and the backdoor image is generated based on the data obtained by the assumed backdoor label method, the parameter search method, the optimizer autonomously selects the backdoor label method, and the probability superposition method; the generated backdoor image and the clean sample are mixed to construct an enhanced data set; then forgetting training and implicit super gradient training are used to delete the backdoor in the classification model; if it does not exist, exit. The above-mentioned "assumed backdoor label method", "parameter search method", "optimizer autonomous selection" and "probability superposition method" will be explained one by one later.

[0027] Next, the backdoor label prediction and backdoor deletion parts based on forgetting learning and implicit super gradient are performed, such as steps S2-S3.

[0028] S2. Assume all labels as backdoor labels, generate possible backdoor triggers corresponding to the labels, adjust the trigger success rate and obtain static backdoor triggers through joint tuning search of trigger costs, add disturbances, and let the optimizer optimize the backdoor labels in the direction of deviating from the correct labels, obtain the optimized backdoor labels, and adapt the dynamic backdoor triggers; use balanced samples to accumulate and normalize the predicted probability of each label to obtain the cumulative distribution of predicted probabilities; the balanced samples include normal samples and error samples in the same proportion.

[0029] First, see Figure 3 , assuming a backdoor label, that is, assuming that all labels may be backdoors, the size of the trigger is optimized using the cross entropy of the predicted label and the backdoor label, the possible trigger of the label and its L1 norm are obtained, and the L1 norm is normalized.

[0030] Traverse all labels, set them as target labels, and perform the following operations on clean samples: S201, initialize a mask and pattern with the same dimensions as the input sample. Pattern is a three-dimensional matrix with the same dimensions as the input image (including height, width and color channels). Mask is a two-dimensional matrix that determines how much ratio of the original image the trigger can cover.

[0031] S202. Find a trigger (mask, pattern) that will misclassify the original image as the target label; define the objective function as the cross entropy loss between the predicted label and the target label, and add the L1 norm regularization term of the mask. The L1 norm is used to control the sparsity of the mask to make the trigger as small as possible.

[0032] Use gradient descent or other optimization algorithms to optimize the mask and pattern until the cross entropy loss and the L1 norm of the mask reach a balance, that is, the loss function converges.

[0033] The optimization formula (1) is as follows: in, is the target label, f(·) is the prediction function of DNN, is the loss function that measures the classification error, and λ is the weight of the second objective; a smaller λ has a lower weight on the control of the trigger size, but will have a higher success rate of misclassification, and the optimization process will dynamically adjust λ. X is a set of clean images to solve the optimization task, and m and P represent mask and pattern respectively.

[0034] S203, after optimization, obtain the mask and pattern for the current target label. Apply the mask and pattern to the clean sample to generate possible triggers. The formula (2) for generating triggers is as follows: Where A represents the function of superimposing the trigger on the original sample x, p represents the pattern of the trigger, Represents the height, width and color channel respectively, and m is a 2D matrix called mask, whose values ​​range from 0 to 1. When used for a specific pixel ( )of = 1, the trigger completely overwrites the original color.

[0035] S204, delete the minimum value of the L1 norm, then take the average of the L1 norm, and normalize the L1 norm of the possible trigger corresponding to each label.

[0036] In order to compare different models under the same data set, the minimum value of the L1 norm is deleted, and then the L1 norm is averaged. The L1 norm of each label corresponding to the trigger is divided by the mean norm and then multiplied by 100 to obtain the normalized L1 norm value that can be compared across models.

[0037] in, It represents the L1 norm of all labels corresponding to triggers of the same model. It represents the L1 norm of the trigger corresponding to label i in model m.

[0038] Secondly, through testing, we found that the size of the backdoor trigger is positively correlated with the attack success rate, and starting from 97% attack success rate, the size of the corresponding backdoor trigger increases greatly with each 0.1% increase in attack success rate. In order to facilitate horizontal comparison, the trigger success rate of the backdoor trigger is set to 95%. At the same time, by optimizing the two parameters (lr parameter and cost parameter) that balance the relationship between the success rate and the trigger size, a single epoch backdoor trigger that can meet the restriction of 95%-96% attack success rate is obtained, thereby improving the search speed.

[0039] See also Figure 4 This embodiment adopts a parameter search method, which optimizes two parameters of the relationship between the balance success rate and the trigger size to obtain a backdoor trigger that can meet the restriction condition of 95%-96% attack success rate. Specifically, the adjustment of the trigger success rate and the joint optimization search of the trigger cost to obtain a static backdoor trigger include the following steps: S205, initializing the learning rate lr and the regularization term cost weight, and setting the trigger success rate ASR range of the target attack.

[0040] S206, calculate the loss of the possible backdoor trigger corresponding to the backdoor label, use stochastic gradient descent to optimize the parameters of the possible backdoor trigger, and perform forward propagation on all training samples to calculate the predicted label of the classification model and obtain the static trigger. The loss of the possible backdoor trigger is as follows: in, It represents the L2 norm of the trigger corresponding to label i in the classification model m. Target label With predicted labels The cross entropy of Represents the loss value calculated by model m on input sample x. mean() represents the average loss of the entire batch.

[0041] S207, check the current trigger success rate and static trigger size. If the current attack success rate is lower than the set lower limit, increase the learning rate and reduce the weight of the regularization term. If the current attack success rate is higher than the set upper limit, reduce the learning rate and increase the weight of the regularization term.

[0042] S208. Save the static trigger and its parameters, where the parameters include the L2 norm.

[0043] S209, delete the minimum value of the L2 norm, then take the average of the L2 norm, and normalize the norm of the static trigger corresponding to each label. The normalization process is as shown in formula (3).

[0044] The above "hypothetical backdoor label" and "parameter search" both use the restored trigger features as classification features, and achieve accurate static trigger restoration and trigger cost evaluation through the joint tuning search of ASR and trigger cost.

[0045] Next, see Figure 5 ,This embodiment adopts a method in which the optimizer autonomously selects the backdoor label, and ,it is necessary to input a normal data set and a classification model, ,add perturbations to the normal data set, and calculate the loss between the ,predicted label and the true label so as to optimize the ,predicted label as far as possible from the original correct label to find the ,optimal label, and make multiple selections for different original samples to ,adapt the dynamic trigger.

[0046] It is worth explaining that "adding disturbance" means adding triggers to the original image, adding different triggers according to different images, and generating backdoor images with dynamic triggers; and the "optimizer" here refers to the algorithm in the pytorch library used to update the parameters of the neural network model to minimize the loss function. The optimizer helps the model learn by updating the network weights during the training process, optimizing in the direction of deviating from the original correct label, thereby finding the optimal label.

[0047] Specifically, the adding of disturbances to make the optimizer optimize the backdoor label in a direction deviating from the correct label to obtain the optimized backdoor label includes the following steps: S210, initializing parameters and generating initial triggers.

[0048] S211. Calculate the loss of the trigger as follows: in, It represents the L2 norm of the trigger corresponding to label i in model m. The true label With predicted labels The cross entropy of Represents the loss value calculated by the model m on the input sample x. In order to optimize as much as possible to deviate from the original correct label direction, Negate. mean() represents the average loss of the entire batch.

[0049] S212. Optimize the trigger and check the attack success rate and trigger size.

[0050] After step S212, the learning rate lr and the regularization term cost under the constraint condition are obtained.

[0051] S213. According to formulas (6) and (7), perform n rounds of search under the constraints of the learning rate lr and the regularization term cost. Under the condition that the attack success rate in each round is above 90%, select the label with the largest probability distribution in the n rounds of search, and consider it to be the backdoor label, and obtain the optimal mask and pattern.

[0052] S214. Save the one-hot encoding of the label and the trigger size.

[0053] Finally, see Figure 6 This embodiment also adopts a probability superposition method, that is, inputting balanced samples into the classification model, accumulating and normalizing the predicted probability of each label, and obtaining the predicted probability cumulative distribution. This method uses the predicted probability distribution as a feature, inputs clean samples and superimposes and averages the probabilities of their predicted labels, without the need to restore the trigger.

[0054] This method assumes that the probability corresponding to the normal label is relatively low when predicting other categories. That is, after a normal sample is input into a normal model, the probability of being assigned to the correct predicted label will be greater, and the probability of being assigned to other wrong labels will be smaller, or even negative, but the overall distribution probability is relatively balanced. In order to trigger the backdoor label by modifying a small number of triggers, it is necessary to keep a relatively high probability when predicting normal samples, so that a small change in value can maximize the predicted probability of the backdoor label, thereby triggering the backdoor.

[0055] Specifically, the following steps are included: S215. Extract samples of each category from the original data set to obtain a balanced sample set.

[0056] In order to ensure that the number of samples in each class is the same, we first need to extract samples of each category from the original dataset. Assume that the original dataset contains categories, and the sample set of each category is recorded as in, From each Randomly extract s samples from to form a balanced data set D′. Combine balanced samples of all categories to form a batch data set: D′ = ,in, It means extracting the i-th sample from the k-th class.

[0057] S216. Input clean samples into the trained classification model to obtain each sample The corresponding tags The predicted probability is as follows: S217, accumulate the predicted probability of each label to obtain the predicted probability of each label The cumulative value of , as follows: S218, according to formula (3), for each label The cumulative value of Normalize.

[0058] S219. Use the binary cross entropy loss function to calculate the loss between the model output and the true label to obtain the trained backdoor detection model.

[0059] The training of the backdoor detection model includes the following steps: (1) Define the network structure of the backdoor detection model: the feature vector of the input layer is x∈ , where d is the dimension of the input feature. The hidden layer is a number of layers of neurons, using the ReLU activation function. The output layer has two neurons, which output the probability of the model having a backdoor or not having a backdoor. Let the model parameters be , the input feature vector is x, then the output y of the model is y=softmax( x). Among them, softmax is the activation function for binary classification.

[0060] (2) Construction of training set: Extract the features of the normal model and the model with backdoor, and use the model features and backdoor status (whether the model has a backdoor, backdoor label) as the training set.

[0061] (3) Model training: Training is performed through supervised learning, using the binary cross entropy loss function to calculate the loss between the model output and the true label.

[0062] S3. Calculate the comprehensive score of the backdoor label according to the parameters of the static trigger and the dynamic trigger, and the cumulative distribution of the predicted probability, and use the multiple backdoor labels with the highest scores as the target labels.

[0063] In this example, see Figure 7 , assuming that the classification model has n classification labels. Assume that the trigger L1 norm normalization result L1 obtained by the backdoor label method, the trigger L2 norm normalization result L2 obtained by the parameter search method, the optimizer autonomously selects the backdoor label method to obtain the most likely probability label PL of the model, and the predicted probability accumulation NP of the normalized signature obtained by the probability superposition method. All data values ​​are uniformly scaled to the range of 0-1.

[0064] For the comprehensive score of label i: Finally, the label with the highest comprehensive score is used as the predicted backdoor label, and the backdoor label is used as the predicted label of the backdoor label prediction model. In order to include more backdoor features in the process of generating backdoor samples, this embodiment selects images corresponding to the first three classes of target labels with higher predictions as training sets.

[0065] S4. Generate backdoor image samples using backdoor prediction labels of multiple categories with higher predictions, add real labels to construct a mixed data set, perform data enhancement on the mixed data set to obtain an enhanced data set; use the enhanced data set to perform multi-stage forgetting training and implicit super gradient training on the classification model to delete the backdoor of the classification model.

[0066] See also Figure 8 ,The so-called mixed data set consists of 4 data sets, namely the first training set, the second training set, the backdoor test set, and the original data test set.

[0067] In order to obtain the four data sets, this embodiment adopts the following steps: S401, generating backdoor image samples: according to the top three target labels with the highest probability generated by the label prediction model, matching the minimum trigger corresponding to the target label in the previous step, superimposing them on the clean sample image, and generating a backdoor image.

[0068] S402, constructing a data set: The generated backdoor samples and the real labels form a backdoor forgetting training data set for forgetting training; the backdoor samples, backdoor labels, and real labels form a backdoor testing set for evaluating the attack success rate (ASR) and label accuracy (RA); the clean samples and real labels form the original training and testing data sets.

[0069] S403, data enhancement: The data enhancement of the training set includes random distortion perturbation, random horizontal flipping, and rotation. The test set only undergoes basic size conversion and pixel value normalization.

[0070] The enhancement of the dataset can enhance the generalization ability of the model while ensuring the diversity of samples.

[0071] S404, constructing a mixed data set, namely: The data-enhanced backdoor forgotten training data set is mixed with the original training set to obtain a first training set, and the data-enhanced original training set is mixed with the original test set to obtain a second training set.

[0072] Next, see Fig. 9 and Fig.10 ,The classification model is retrained using a mixed dataset, thereby solving the problem of classification model capability degradation when the model backdoor is deleted.

[0073] In order to solve this problem, this embodiment attempts to delete the backdoor of the classification model by combining multi-stage forgetting learning and n-round implicit super gradient training, including: multi-stage hybrid training of the classification model, using n-round implicit super gradient training and 1 forgetting training in each round of hybrid training; evaluating the backdoor deletion effect of the classification model through the comprehensive score of the model; selecting the best model result in the backdoor deletion effect evaluation; adding a forgetting training in n-round implicit super gradient training, and performing multi-stage hybrid training on the classification model again. Implicit super gradient uses a large force to dynamically clear the backdoor sensitivity of the classification model, and forgetting training mainly suppresses the backdoor by restoring the trigger features, while restoring the normal classification ability of the model.

[0074] Furthermore, the specific steps are as follows: The forgetting learning includes step S405.

[0075] S405, input the first training set into the classification model, and the model determines which weights have problems through training and updates them. Let the model forget the labels corresponding to the original backdoor trigger training, re-learn the normal labels of these backdoor samples, and forget the backdoor, so as to achieve the effect of model backdoor deletion.

[0076] The implicit super gradient training includes steps S406-S407.

[0077] S406, input the second training set into the classification model, calculate the internal loss of the classification model, find the backdoor trigger with the highest trigger factor, calculate the external loss of the classification model to optimize the model parameters, and reduce the effect of the trigger on the classification model.

[0078] In this process, formula (11) is used to calculate the internal loss of the classification model and obtain the optimal perturbation , and the weights of the classification model are updated through the fixed point iteration method. It should also be explained that in this embodiment, the internal loss maximization is to generate a trigger superimposed on the original image, and the difference between the label of the backdoor image generated and input into the detection model and the real label is as large as possible, and the external loss minimization is to find a suitable model parameter θ so that the loss value of the function H(·) is minimized. The purpose is to optimize the model parameters so that the impact of the trigger on the model is minimized, and to achieve the forgetting of the backdoor trigger feature.

[0079] S407, updating the weight of the classification model through the fixed point iteration method to obtain the trained classification model, as shown in the following formula: in, is the true label, is the predicted label of the model, are model parameters.

[0080] The step of evaluating the backdoor removal effect of the classification model includes step S408.

[0081] S408: Use the backdoor test set and the third test set to evaluate the trained classification model and calculate the comprehensive score of the current classification model, as shown in the following formula: Among them, test_acc represents the test accuracy, test_asr represents the backdoor attack success rate, and test_ra represents the recovery attack success rate; if the comprehensive score of the current classification model is higher than the maximum score, the maximum score and the best model state are updated, and the best model state is saved.

[0082] Since direct forgetting training will reduce the task prediction accuracy while reducing the success rate of backdoor attacks, and implicit super gradient training has little effect on task prediction accuracy, and can also improve the original task prediction accuracy to a certain extent, a forgetting training is added to n rounds of implicit super gradient training. In order to effectively find and optimize the best performance model within limited resources and time. By gradually increasing the depth and breadth of training, the performance of the model is gradually improved. The specific steps are as follows: (1) Initial training: Each model undergoes 𝑘 rounds of training to obtain the model’s comprehensive score score_i, 𝑖=1,2,…,𝑛.

[0083] (2) Selection: Select the 𝑚 models with the best performance.

[0084] (3) Increase the number of training rounds: Perform an additional l rounds of training on this 𝑚 model.

[0085] (4) Final evaluation: Evaluate the performance of these 𝑚 models again and select the model with the best performance.

[0086] (5) Long-round training: The best model finally selected is trained for t rounds.

[0087] (6) Model preservation: Select the best model result based on the backdoor removal effect evaluation and save it instead of the last trained model.

[0088] After the above 6 steps, the final model is obtained.

[0089] It should be noted that, for the sake of convenience, the aforementioned method embodiments are all expressed as a series of action combinations, but those skilled in the art should know that the present invention is not limited to the described order of actions, because according to the present invention, certain steps can be performed in other orders or simultaneously.

[0090] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0091] The above embodiments are preferred implementation modes of the present invention, but the implementation modes of the present invention are not limited to the above embodiments. Any other changes, modifications, substitutions, combinations, and simplifications that do not deviate from the spirit and principles of the present invention should be equivalent replacement methods and are included in the protection scope of the present invention.

Claims

1. A backdoor defense method based on forgetting learning and implicit super gradient, characterized in that: The steps include: S1. Input the classification model into the backdoor detection model to detect whether there is a backdoor. If the classification model has a backdoor, execute step S2, otherwise exit; S2. Assume all labels as backdoor labels, generate possible backdoor triggers corresponding to the labels, adjust the trigger success rate, and obtain static backdoor triggers through joint optimization search of trigger costs. Add disturbances to make the optimizer optimize the backdoor labels in the direction of deviating from the correct labels, obtain the optimized backdoor labels, and adapt to dynamic backdoor triggers. The predicted probability of each label is accumulated and normalized using balanced samples to obtain the cumulative distribution of the predicted probability; S3, according to the parameters of the static trigger and the dynamic trigger, and the cumulative distribution of the predicted probability, the comprehensive score of the backdoor label is calculated using the backdoor label prediction model, and the backdoor label with the highest score is used as the target label; S4. Generate backdoor image samples using backdoor prediction labels of multiple categories with higher predictions, add real labels to construct a mixed data set, perform data enhancement on the mixed data set, and obtain an enhanced data set; The classification model is trained with enhanced data sets through multi-stage forgetting and implicit super gradient training to remove the backdoor of the classification model.

2. The backdoor defense method based on forgetting learning and implicit super gradient according to claim 1 is characterized in that: The method assumes all tags to be backdoor tags and generates possible backdoor triggers corresponding to the tags, including: Traverse all labels of clean samples and assume the labels to be backdoor labels; Generate a mask and pattern with the same dimensions as the input sample, and find the trigger (mask, pattern); The trigger (mask, pattern) misclassifies the original image as a backdoor label and obtains the predicted backdoor label; calculates the cross entropy loss between the predicted backdoor label and the backdoor label, adds the L1 norm regularization term of the mask, and uses gradient descent to optimize the mask and pattern until the cross entropy loss and the L1 norm of the mask reach a balance, and obtains the optimized mask and pattern; Apply the optimized mask and pattern to the clean sample to generate possible backdoor triggers; Delete the minimum value of the L1 norm, take the average of the L1 norm, and normalize the L1 norm of the possible triggers corresponding to each label.

3. The backdoor defense method based on forgetting learning and implicit super gradient according to claim 2 is characterized in that: The gradient descent is used to optimize the mask and pattern as follows: ； in, represents the target label, f(·) represents the prediction function of DNN, represents the loss function that measures the classification error, λ represents the weight of the second objective, X represents a set of clean images that solve the optimization task, and m and P represent mask and pattern, respectively.

4. The backdoor defense method based on forgetting learning and implicit super gradient according to claim 1 is characterized in that: The adjusting of the trigger success rate and the joint optimization search through the trigger cost include: Initialize the learning rate and regularization term weights, and set the trigger success rate range of the target attack; Calculate the loss of possible backdoor triggers corresponding to the backdoor labels, use stochastic gradient descent to optimize the parameters of possible backdoor triggers, perform forward propagation on all training samples, calculate the predicted labels of the classification model, and obtain static triggers; Check the current trigger success rate and static trigger size. If the current attack success rate is lower than the set lower limit, increase the learning rate and reduce the weight of the regularization term. If the current attack success rate is higher than the set upper limit, reduce the learning rate and increase the weight of the regularization term. Save static triggers and their parameters, including L2 norm; Delete the minimum value of the L2 norm, take the average of the L2 norm, and normalize the norm of the static trigger corresponding to each label.

5. The backdoor defense method based on forgetting learning and implicit super gradient according to claim 1 is characterized in that: The adding of disturbances to make the optimizer optimize the backdoor label in a direction deviating from the correct label includes: Initialize the learning rate and regularization weights, generate masks and patterns with the same dimensions as the input samples, and find the corresponding triggers (mask, pattern); use gradient descent to optimize the triggers, and check the attack trigger success rate and trigger size of the triggers; Calculate the loss of the backdoor label and perform n rounds of search based on the learning rate and the regularization term weight; the backdoor label loss is as follows: ； ； in, It represents the L2 norm of the trigger corresponding to label i in the classification model m. The true label With predicted labels The cross entropy of Represents the loss value calculated by model m on input sample x; Set the attack success rate in each round to 85%~100%, select the label with the largest probability distribution in n rounds of search as the backdoor label, obtain the optimal mask and pattern; save the one-hot encoding and trigger size of the label.

6. The backdoor defense method based on forgetting learning and implicit super gradient according to claim 1, characterized in that: The method of accumulating and normalizing the predicted probability of each label by using balanced samples includes: Extract samples of each category from the original data set to obtain a balanced sample set; Input clean samples into the trained classification model and obtain each sample The corresponding tags The predicted probability of Accumulate the predicted probability of each label to get The cumulative value of ; For each tag The cumulative value of Normalize it as follows: ； in, It represents the L1 norm of all labels corresponding to triggers of the same model. It represents the L1 norm of the trigger corresponding to label i in the classification model m.

7. The backdoor defense method based on forgetting learning and implicit super gradient according to claim 1, characterized in that: The comprehensive score of the backdoor tag is calculated, and the backdoor tag with the highest score is used as the target tag, as shown in the following formula: ； in, , , , represents the hyperparameter, Indicates the optimized backdoor tag, Represents the result of the accumulation of predicted probabilities for each label; The label with the highest comprehensive score is the predicted backdoor label, and the backdoor label is used as the predicted label of the backdoor label prediction model.

8. The backdoor defense method based on forgetting learning and implicit super gradient according to claim 1, characterized in that: The method uses the target label to generate the backdoor image sample, adds the real label to construct the mixed data set, and performs data enhancement on the mixed data set to obtain the enhanced data set, specifically: According to the backdoor prediction labels of multiple categories with higher probability prediction, the minimum trigger corresponding to the backdoor label in the previous step is matched and superimposed on the clean sample image to generate the backdoor image; The generated backdoor samples are mixed with the real labels to obtain a backdoor forgetting training data set for training the model to delete the backdoor; the backdoor sample images, backdoor labels and real labels are mixed to obtain a backdoor test set for evaluating the attack success rate and label accuracy; the clean sample images and real labels are mixed to obtain the original data set; the original data set includes the original training set and the original test set; The backdoor forgotten training dataset, backdoor test dataset and original dataset are enhanced, including random distortion perturbation, random horizontal flipping, rotation and pixel normalization; The data-enhanced backdoor forgotten training data set is mixed with the original training set to obtain a first training set, and the data-enhanced original training set is mixed with the original test set to obtain a second training set.

9. The backdoor defense method based on forgetting learning and implicit super gradient according to claim 8, characterized in that: The backdoor of the classification model is deleted, including: performing multi-stage hybrid training on the classification model, using n rounds of implicit super gradient training and forgetting training in each round of hybrid training; evaluating the backdoor deletion effect of the classification model through a comprehensive score of the model; selecting the best model result in the backdoor deletion effect evaluation; adding forgetting training to the n rounds of implicit super gradient training, and performing multi-stage hybrid training on the classification model again; The forgetting training includes: The first training set is input into the classification model. The problematic weights are determined and updated through training. The classification model is made to forget the labels corresponding to the original backdoor trigger training through forgetting training, and the normal labels of the backdoor samples are re-learned. The n rounds of implicit super gradient training include: Input the second training set into the classification model, calculate the internal loss of the classification model, find the backdoor trigger with the highest trigger factor, calculate the external loss of the classification model to optimize the model parameters, and reduce the effect of the trigger on the classification model; Update the model weights through the fixed point iteration method to obtain the trained classification model, as shown below: ； in, represents the true label, represents the predicted label of the model, Represents a sample, represents disturbance, represents the model parameters, L represents the loss function; The backdoor removal effect of the classification model is evaluated by the comprehensive score of the model, including: The trained classification model is evaluated using the backdoor test set and the original data test set, and the comprehensive score of the current classification model is calculated as follows: score = test_acc – test_asr + test_ra Among them, test_acc represents the test accuracy, test_asr represents the backdoor attack success rate, and test_ra represents the recovery attack success rate; if the comprehensive score of the current classification model is higher than the maximum score, the maximum score and the best model state are updated, and the best model state is saved.

10. The backdoor defense method based on forgetting learning and implicit super gradient according to claim 1, characterized in that: The step S1 comprises: The training set is input into the backdoor detection model, and the training is performed through supervised learning. The weight parameters of the backdoor detection model are updated, and the loss between the model output and the true label is calculated using the cross entropy loss function of the two classifications to determine whether there is a backdoor in the model; the training set includes a normal model, a classification model and a backdoor status, and the backdoor status includes whether there is a backdoor model and a backdoor label category.

Images