A neural network capacity-based deep learning backdoor defense method

Abstract

The present application relates to a kind of neural network capacity-based deep learning backdoor defense method, belong to artificial intelligence security field.The method includes: according to reverse generation trigger, poison data set is generated, according to poison data set, train and select suitable low capacity neural network, utilize selected low capacity neural network, data labeling is carried out to training set, and suspicious sample in data set is put into collection summary;Again, backdoor screening is carried out to labeled sample, and the sample with classification accuracy greater than set threshold can be labeled as backdoor sample, and put into backdoor sample set, then retraining is carried out to data set on high capacity neural network by importance weight, and high capacity network is selectively extracted by using the way of reweighting, so that model can also be protected from backdoor attack on toxic data set.The present application can greatly reduce the threat of backdoor attack to image recognition, and prevent the intrusion of backdoor when using the data set provided by others.

Description

TECHNICAL FIELD

[0001] The application belongs to the field of artificial intelligence security, relates to the field of deep learning security, and particularly relates to a method for training a clean model on a backdoor dataset based on high and low capacity neural networks and deep learning. BACKGROUND

[0002] In recent years, due to the development of the field of deep neural networks, deep neural network models are used in more and more industries, including some fields with high security requirements (for example: autonomous driving, medical treatment, object detection, etc.). For a neural network, if the model is attacked, it may bring unimaginable dangerous consequences. At present, deep neural networks have been proved to be vulnerable to attacks, among which backdoor attack is a new type of attack on deep neural networks. Backdoor attack mainly includes two types. One is through the method of poisoning the dataset, that is, injecting a trigger into some samples in the dataset, and then training the model, the model will misclassify the samples with the trigger. Its essence is to learn the backdoor information in the samples through the strong learning ability of the deep neural network. The other type of backdoor attack mainly modifies the model parameters directly to make the model learn the backdoor samples directly. After embedding the backdoor, when the backdoor sample with the trigger appears in the inference stage after the model is deployed, the sample will be misclassified. The characteristic of backdoor attack lies in its concealment. First of all, the model itself may misclassify, and it is difficult to determine whether there is a backdoor sample in the dataset. Secondly, on the basis of misclassifying the specified target, the backdoor attack can keep the accuracy at about the same level as the original model, that is, it is difficult to determine whether the model has been attacked by the backdoor.

[0003] Current backdoor defense mainly includes two types. The first type is to detect whether there is a backdoor sample in the dataset before the model is trained. However, the defect of this method lies in that it only focuses on detecting whether there is a backdoor, but does not give a good method for removing the backdoor. The second type focuses on how to remove the backdoor information in the model. The common method is to perform pruning operation on the model, but this type of method generally has a great impact on the performance of the model itself. Therefore, there is still a great research space in the field of backdoor at present, and how to safely remove the backdoor after detecting the backdoor is a key challenge in current backdoor defense. SUMMARY

[0004] Therefore, the purpose of the present application is to provide a deep learning backdoor defense method based on neural network capacity, which greatly reduces the threat of backdoor attack in image recognition, and prevents the invasion of backdoors when using data sets provided by others.

[0005] To achieve the above purpose, the present application provides the following technical scheme:

[0006] A deep learning backdoor defense method based on neural network capacity, specifically comprising the following steps:

[0007] S1: reverse generation trigger Δ for the high-capacity neural network to be protected, and embed the trigger Δ into a clean picture to generate a backdoor sample, after generating a plurality of backdoor samples, put them into the originally clean data set to obtain a poisoned data set D={x i ,y i}, wherein x i is the input of the poisoned data set, y i is the corresponding label;

[0008] S2: train the initial low-capacity neural network M with the generated poisoned data set D={x i ,y i}, adjust the low neural network capacity according to the classification accuracy of the trained backdoor data, and obtain the required low-capacity neural network M l :

[0009] S3: train the obtained low-capacity network M l on the training set D={x i ,y i}, label the samples x i with classification success rate higher than the threshold , and put them into the set D i ;

[0010] S4: train the labeled set D i with other low-capacity neural networks, and put the samples in D i with classification accuracy higher than the threshold in the backdoor set D b ;

[0011] S5: importance weight labeling is performed on the entire data set D={x i ,y i}, that is, different importance weights are assigned to the clean samples and the backdoor sample set D b , and the high-capacity neural network is trained according to the importance weight information provided by the low-capacity neural network;

[0012] S6: use common backdoor attacks to attack the high-capacity neural network M h trained, and verify the defense effect of the network model.

[0013] Further, in step S1, the backdoor sample is generated, specifically including: using the method in Neural Cleanse to do reverse engineering on different labels to obtain the backdoor trigger Δ of the high-capacity neural network model that needs to be protected, and then selecting a set of clean data D' = {x, y} to embed the generated trigger Δ into the clean picture x in the data D' to generate a backdoor sample That is, the backdoor data is generated by using the following formula:

[0014]

[0015] Wherein, represents the backdoor sample, x represents the clean sample, and y is the label corresponding to x.

[0016] Further, in step S2, the low-capacity neural network M needed is obtained l , specifically including: first selecting a conventional low-capacity neural network such as Lenet, denoted as M, and training the current neural network M on the poisoned data set D = {x i ,y i}; then through the backdoor verification set , the classification accuracy of the backdoor sample is counted, and if all the backdoor samples can be accurately classified into the backdoor label , the model capacity is reduced, that is, one convolutional layer is reduced, and if there is a phenomenon that the backdoor sample classification is wrong, the neural network capacity is increased, that is, one convolutional layer is added, and the above operation is repeated to find the lowest capacity neural network M that can guarantee all the backdoor features l .

[0017] Further, in step S3, the suspicious sample is labeled by using the low-capacity network M l The method for labeling suspicious samples is specifically: using the low-capacity network to label the backdoor samples in the data set. The essence of the backdoor belongs to a shortcut, and the low-capacity network can easily learn the shortcut in the data set. Therefore, the low-capacity network, which should have poor learning ability, can learn the features of the backdoor sample, and the classification accuracy for clean samples will be low. Therefore, the success rate of the backdoor sample in the low-capacity network task will be maintained at a high level. For a given poisoned data set D = {x i ,y i}, first train it in the low-capacity network, and label the samples with a classification success rate higher than a threshold in the training process to set D i .

[0018] Further, in step S4, the possible backdoor samples are determined, including: reclassifying the D i samples. The samples with high classification accuracy are not completely backdoor samples (that is, they may also contain normal samples), and need to be further classified D iThe medium samples are rescreened, and other low-capacity network M l , D i is trained, and D i is obtained The input of the medium samples is put into the set D b .

[0019] Further, in step S5, the process of removing the backdoor information by the high-capacity model specifically comprises: performing importance weight distribution on the samples in D b by using an importance weight method, and given a data set D b ={x i ,y i}, the importance weight of the training image x i is IW(w i ), wherein (w i ) is the error classification probability given by the low-capacity neural network, and the calculation formula is w i =1-p(y i |x i ); then, when the high-capacity neural network is trained, the loss function of each sample in the data set needs to be multiplied by the importance weight, and here, normalization is performed, and the entire loss in the mini-batch is calculated as follows:

[0020]

[0021] wherein, B represents the entire mini-batch set, k represents the kth sample in the mini-batch, and L k represents the loss of the kth sample in the mini-batch; through the transmission of the importance weight, the high-capacity neural network M h can intentionally filter the backdoor samples.

[0022] Further, step S6 specifically comprises: using common backdoor attack construction methods such as badNets or waNet to construct a poisoned data set and a verification set of backdoor samples, attacking the trained high-capacity neural network M h , if the model obtains a normal label for the input of the backdoor sample, it represents that the defense is successful, and the success rate of the defense is counted for each classification of the data set; then, the classification accuracy of the model is tested on normal samples, and the classification precision of the model on normal samples is observed. Compared with the current mainstream backdoor defense method, the performance of the backdoor defense method of the present application is evaluated.

[0023] The beneficial effects of the present application are that the present application aims at the poisoning problem of backdoor attacks, that is, the existing backdoor defense strategy has the problems of being difficult to determine the backdoor and the original model precision after eliminating the backdoor, and the present application proposes a method for removing the backdoor by using the capacity of the network, and the network model in the method can effectively use various data sets and effectively defend against possible backdoor attacks in the process of defending the backdoor.

[0024] Other advantages, objects, and features of the present application will be understood in view of the following specification, and to some extent will be apparent to those skilled in the art from the examination of the following specification, or can be learned from the practice of the present application. The objects and other advantages of the present application can be realized and obtained by the following specification. BRIEF DESCRIPTION OF DRAWINGS

[0025] In order to make the objects, technical solutions and advantages of the present application clearer, the preferred detailed description of the present application will be combined with the drawings, in which:

[0026] Figure 1 The flowchart of the method of the present application is shown in the figure;

[0027] Figure 2 The graph of the poisoned data set is shown in the figure;

[0028] Figure 3 The schematic diagram of selecting a suitable low-capacity network is shown in the figure;

[0029] Figure 4 The schematic diagram of filtering backdoor information by using a low-capacity network is shown in the figure;

[0030] Figure 5 The schematic diagram of verifying the defense effect of a high-capacity network is shown in the figure. DETAILED DESCRIPTION

[0031] The embodiments of the present application will be described below by specific examples, and those skilled in the art can easily understand other advantages and effects of the present application from the disclosure of the present specification. The present application can also be implemented or applied by different specific embodiments, and the details in the specification can be modified or changed in different ways without departing from the spirit of the present application. It should be noted that the diagrams provided in the following examples only illustrate the basic concept of the present application in a schematic manner, and the following examples and features in the examples can be combined with each other without conflict.

[0032] Please refer to Figures 1-5 The present application provides a deep learning backdoor defense method based on neural network capacity, as shown in Figure 1 The specific steps include the following steps:

[0033] S1: generate backdoor trigger Δ and embed trigger Δ into pictures to generate backdoor samples, mix backdoor samples with clean data to obtain poisoned data set.

[0034] Specifically, using the backdoor sample characteristics mentioned in Neural Cleanse, the trigger Δ is obtained by reverse engineering the labels of the data set in the model, and the backdoor sample is constructed as shown in Figure 2 The clean picture is selected, the obtained trigger is embedded into the picture to construct the backdoor sample, and then the backdoor sample is put into the data set to obtain the poisoned data set.

[0035] S2: use the generated poisoned data set to train the low-capacity network, and find the lowest neural network model that can learn all backdoors.

[0036] The specific steps are shown in Figure 3 , including:

[0037] S21: use the common low-capacity network Lenet as the initial model M, and use the poisoned data set D={x i ,y i} to train the model M, and see if all backdoor samples can be learned, that is, after training, use the backdoor verification set to test whether all backdoor samples can be classified into backdoor labels, that is, Where f(·) represents the inference of the low-capacity network M.

[0038] S22: if the above is true, reduce the capacity of the neural network M, and the method of reducing the capacity of the network is to reduce a convolutional layer, if there are backdoor samples that are not correctly classified into backdoor labels, that is, , increase the network capacity of M, that is, add a convolutional layer, repeat the above operation until the network that cannot classify all backdoor samples after the last training is found, and increasing the capacity can successfully classify all backdoor samples, then the network is the required M l , or M can successfully classify all backdoor samples, and after reducing the capacity of M, it cannot successfully classify all backdoor samples, then M is the required low-capacity neural network M l .

[0039] S3: label suspicious samples during the training of the low-capacity model and put them into set M l .

[0040] Specifically, during the training of the low-capacity network, the data in the data set is labeled, and the samples with classification accuracy higher than 0.99 in multiple epochs are put into set D i .

[0041] S4: Replace the D marked in step S3. i The samples from the set are processed through other low-capacity networks M. b Retraining, in M b During training, label D i Samples with a classification accuracy still higher than 0.99 are placed into the backdoor sample set D. b ={x i ,y i}middle.

[0042] S5: Label the importance weights of the samples in the backdoor sample set and train the high-capacity network using the weight redistribution method;

[0043] Specific steps are as follows: Figure 5 As shown, this includes: using an importance weighting method to evaluate D. b Assign importance weights to samples in a dataset D. b ={x i ,y i Training image x i The corresponding importance weight is IW(w) i ), of which (w i This is the misclassification probability given for a low-capacity neural network, calculated using the formula w. i =1-p(y i |x i Then, when training a high-capacity network, the loss function for each sample in the dataset needs to be multiplied by its importance weight. Here, normalization has been performed, and the total loss in the mini-batch is calculated as follows: in, B represents the entire mini-batch set, k represents the k-th sample in this mini-batch, and L... k This represents the loss of the k-th sample in this mini-batch; through the propagation of importance weights, the high-capacity neural network M... h It can intentionally filter backdoor samples.

[0044] S6: The defensive effect of generating a poisoned dataset through common backdoor attacks on a trained high-capacity network attack detection model.

[0045] Specifically, first, the poisoned data set and the validation set of the backdoor sample are constructed by using the common backdoor attack of badNets, waNet, etc., the high-capacity network after training is attacked, if the normal label is obtained for the backdoor sample input model, it represents that the defense is successful, the success rate of the defense is counted for each classification of the data set, then the classification accuracy of the model is tested for the normal sample, and the classification accuracy of the model for the normal sample is observed. Compared with the current mainstream backdoor defense method, the performance of the backdoor defense method of the present application is evaluated.

[0046] Finally, it should be pointed out that the above embodiments are only used to illustrate the technical solutions of the present application and are not limiting. Although the present application has been described in detail with reference to the preferred embodiments, it should be understood by those skilled in the art that the technical solutions of the present application can be modified or replaced by equivalents without departing from the purpose and scope of the technical solutions, which should be covered in the scope of the claims of the present application.

Claims

1. A deep learning-based backdoor defense method based on neural network capacity, characterized in that, The method specifically includes the following steps: S1: Reverse-generate triggers for the high-capacity neural network that needs protection. and trigger Backdoor samples are generated by embedding them into clean images. After generating multiple backdoor samples, they are mixed with the original clean dataset to obtain the poisoned dataset. ,in As input to the poisoning dataset, Its corresponding tag; S2: Using the generated poisoning dataset For the initial low-capacity neural network M The low-capacity neural network is trained, and its capacity is adjusted based on the classification accuracy of the backdoor data after training to obtain the desired low-capacity neural network. : S3: Low-capacity network available For the training set Training, labeling those with a classification success rate higher than a threshold samples Add to collection ; S4: Set of annotations Train with other low-capacity neural networks, The accuracy of classification by other networks is higher than the threshold. The samples were put into the backdoor set. middle; S5: For the entire dataset Perform importance weight labeling, i.e., assign importance weights to clean samples and backdoor sample sets. Different importance weights are assigned to medium-sized samples, and the high-capacity neural network is trained by assigning weights based on the importance weight information provided by the low-capacity neural network. Using importance weighting method Assign importance weights to samples in a given dataset. Training images The corresponding importance weight is IW ( ),in The formula for calculating the misclassification probability of a low-capacity neural network is as follows: Then, when training a high-capacity neural network, the loss function for each sample in the dataset needs to be multiplied by its importance weight. The total loss in the mini-batch is calculated as follows: in, , B This represents the entire mini-batch collection. k Represented as the first in the mini-batch k One sample, This indicates the first in the mini-batch k The loss per sample; through the transfer of importance weights, high-capacity neural networks It can intentionally filter backdoor samples; S6: Exploiting a high-capacity neural network trained using a backdoor attack. To verify the defensive effectiveness of the network model.

2. The deep learning backdoor defense method according to claim 1, characterized in that, In step S1, generating backdoor samples specifically includes: using methods in Neural Cleanse to reverse engineer different labels to obtain the backdoor trigger of the high-capacity neural network model that needs to be protected. Then select a clean set of data. The generated trigger Embedded into data Clean images x Generate backdoor samples That is, backdoor data is generated using the following formula: in, Representative backdoor sample, x This represents a clean sample. y for x The corresponding tags.

3. The deep learning backdoor defense method according to claim 1 or 2, characterized in that, In step S2, the required low-capacity neural network is obtained. Specifically, this includes: first, selecting the conventional low-capacity neural network Lenet, and setting it as... M In the poisoning dataset Train the current neural network M Then verify the set via the backdoor. The classification accuracy of backdoor samples is calculated. If all backdoor samples can be accurately classified into the backdoor label, then... If the model capacity is reduced, i.e., one less convolutional layer is removed. If backdoor samples are misclassified, the neural network capacity is increased, i.e., one more convolutional layer is added. This process is repeated to find the neural network with the lowest capacity that guarantees all backdoor features. .

4. The deep learning backdoor defense method according to claim 1, characterized in that, Step S6 specifically includes: constructing a poisoned dataset and a validation set of backdoor samples using backdoor attacks on badNets or waNet, and then testing the trained high-capacity neural network. The attack is carried out. If the model receives a normal label when inputting a backdoor sample, the defense is considered successful. The success rate of the defense is statistically analyzed for each category in the dataset. Then, the model's classification accuracy is tested on normal samples to observe the model's classification precision for normal samples.

Images