A robust training method against poisoning attacks on neural network algorithms

Abstract

The present invention relates to a robust training method for poisoning attacks on neural network algorithms, and belongs to the field of artificial intelligence security technology. The method includes the steps of: constructing a neural network; constructing a robust training loss function; using a training set containing poisoned samples to pre-train the neural network to obtain a neural network pre-training model; according to the neural network pre-training model, obtaining the probability that the training set containing poisoned samples is predicted to be a true label, filtering samples with a prediction probability lower than a filtering threshold, thereby eliminating poisoned samples and obtaining a clean training set; using the clean training set to retrain the neural network pre-training model to obtain a neural network model; and performing a performance test on the neural network model. The present invention utilizes the model prediction results and a monotonically increasing function to realize the adaptive update of sample weights, increases the weight difference between poisoned samples and clean samples, reduces the risk of the neural network model fitting the poisoned samples, and improves the accuracy of the neural network model classification.

Description

Technical Field

[0001] The present invention belongs to the field of artificial intelligence security technology, and in particular relates to a robust training method for neural network algorithm poisoning attacks. Background Art

[0002] In recent years, with the rapid development of artificial intelligence (AI), AI has become increasingly integrated into our daily lives. Humanity is increasingly reliant on the efficiency and convenience it brings, particularly in technologies like facial recognition, voice recognition, and autonomous driving. Simultaneously, the security of deep neural networks themselves has attracted increasing attention. A key attack type is "poisoning attacks," in which attackers inject samples with carefully crafted triggers and labels into the training set. These attacks can manipulate the neural network into misclassifying samples containing triggers while simultaneously behaving normally with normal samples. The high concealment and feasibility of poisoning attacks pose significant security risks to numerous AI applications. Therefore, developing robust training methods to protect against poisoning attacks on neural network algorithms is of great practical significance.

[0003] Huazhong University of Science and Technology proposed a robust training method for a neural network in its patent document "Image classification model training method, classification method and system for suppressing label noise" (patent application number: 202010567241.X, publication number: CN111832627A). The method first divides the data set into a training set and a validation set, and uses the training set to perform supervised training on the image classification model. The training set is traversed to obtain the gradient generated by each sample in the model output layer, and the gradient modulus distribution is estimated to calculate the probability that the sample in the training set is miscalibrated. The incorrectly calibrated sample subset and the correctly calibrated sample subset are thereby divided. If the number of incorrectly calibrated samples is too small, the model is supervised training using the correctly calibrated sample subset. Otherwise, after discarding the category labels of the incorrectly calibrated samples, the two sample subsets are combined with a semi-supervised learning algorithm to update the model parameters, and then the steps of traversing the training set to update the model parameters are repeated until the model converges. This invention can reduce the risk of the neural network model fitting miscalibrated samples from the sample level and improve the accuracy of image classification. However, this method still has the disadvantage that it is only effective for incorrectly labeled samples without triggers and cannot perform robust training on incorrectly labeled samples with triggers.

[0004] Wuhan University proposed a robust neural network training method in its patent application, "Method and System for Resisting Neural Network Backdoor Attacks Based on Image Feature Analysis" (Patent Application No.: 202110398727.X, Publication No.: CN113205115A). This method first processes data and initializes the model to obtain a clean dataset. Based on the initial deep neural network model, it performs a commonality analysis of benign data features, including feature selection and extraction. Feature difference analysis is then performed, with a centroid defense strategy used to initially screen for malicious data. Finally, a deep KNN defense strategy is used to secondary screen for suspicious data. This invention addresses the problem that traditional manual methods for screening poisoned samples are unsuitable for backdoor attacks based on covert triggers. By constructing commonality in the features of benign training data, this method uses a centroid and deep KNN defense strategy to dually screen the test data for significant differences from the commonalities of the features of its corresponding labels, effectively reducing the success rate of backdoor attacks and addressing the shortcomings of traditional defense strategies. However, this method still has some drawbacks: First, clean sample data is difficult to extract for datasets such as speech and images, and feature difference analysis based on Euclidean distance is not applicable to all data types. Summary of the Invention

[0005] (1) Technical issues to be resolved

[0006] The technical problem to be solved by the present invention is: how to design a robust training method for poisoning attacks on neural network algorithms to reduce the success rate of poisoning attacks, prevent the model from being injected with backdoors, and not affect the recognition of normal samples.

[0007] (2) Technical solution

[0008] In order to solve the above technical problems, the present invention provides a robust training method for neural network algorithm poisoning attacks, comprising the following steps:

[0009] (1) Constructing a neural network:

[0010] (1a) Design the neural network structure and parameters according to the actual application scenario;

[0011] (1b) Constructing a neural network according to the designed neural network structure and parameters;

[0012] (2) Constructing a robust training loss function:

[0013] The robust training loss function is Where k is the number of sample categories, p i Refers to the probability of the sample being input into the neural network model and predicted to be the i-th category, Represents the One-Hot encoding of the sample label, and the probability threshold θ is a small value. It is a probability mask that filters out values ​​with probability values ​​lower than the probability threshold θ to prevent the loss value from being too large when the probability is too small. g(*) is a monotonically increasing function. The input value is the model prediction probability, and the output value is the weight of the current sample, which makes the weight of the sample with small probability smaller and the weight of the sample with large probability larger.

[0014] (3) Based on steps 1 and 2, the neural network is pre-trained to obtain a neural network pre-training model:

[0015] (3a) Randomly shuffle the N clean sample data sets and then divide them into N train =N*p pieces of data training set, and N test =N*(1-p) test set N test , randomly select N from the original sample categories of the training set poision =N train *r pieces of data are added with poisoning triggers and put back into the target category of the training set as the neural network training set N train , 0<p<1, represents the proportion of the neural network training set, 0<r<1, represents the proportion of poisoned samples in the training set; (3b) Select the robust training loss function as the objective function of the neural network;

[0016] (3c) Divide the neural network training set into batches, b is the number of data input each time, and the robust training loss function is used to calculate the loss for back propagation, and according to Update the network weights, where W k Represents the current weight, W k+1 represents the updated weight, Represents the loss function when W = W k The gradient at time t, lr represents the learning rate, and the neural network pre-training model is obtained after iterating T1 rounds, where Represents the rounding operation;

[0017] (4) Based on step 3, the poisoned samples are removed to obtain a clean sample dataset:

[0018] (4a) The neural network training set N train Input into the neural network pre-training model to obtain the probability of the sample prediction being the true label. The sample filtering threshold ρ is selected according to the histogram method. The histogram method selects the sample filtering threshold ρ, which means that the probability distribution of the neural network training set samples is displayed in the form of a histogram. The probability characteristics show a bimodal characteristic. The trough between the two peaks is selected as the filtering threshold. This point represents the intersection of normal samples and poisoned samples.

[0019] (4b) The neural network training set N trainInput into the neural network pre-training model to obtain the probability of the sample prediction being the true label, and delete the samples whose probability is lower than the sample filtering threshold ρ to obtain a clean neural network training set N clear_train ;

[0020] (5) Based on step 4, the neural network pre-trained model is retrained to obtain a neural network model:

[0021] (5a) Load the pre-trained model, set the model training parameters to be consistent with the pre-training parameters, and change the iteration round to T2;

[0022] (5b) Using the clean neural network training set N clear_train Train the pre-trained model for a total of After that, the training is stopped and a neural network model is generated, where Represents the rounding operation;

[0023] (6) Perform neural network model performance test based on step 5:

[0024] (6a) The neural network test set N test Input into the neural network model, test the recognition accuracy ACC of the neural network model for clean samples, and then select the neural network test set N test Randomly select clean samples from the original sample category, add poisoning triggers and put them back into the target category to obtain the poisoned sample test set N poision_test , test the attack success rate ACC_Attack of the poisoned sample on the neural network model and the recognition accuracy ACC_Poision of the neural network model on the poisoned sample;

[0025] (6b) Replace the robust training loss function with the ordinary cross entropy loss function and compare the changes in test indicators of different loss functions.

[0026] (3) Beneficial effects

[0027] First, compared with other traditional robust training algorithms, the present invention improves the loss function and does not require additional processing of the original data set. The robust training process is simpler and is effective for commonly used data types, with a wider range of adaptability.

[0028] Second, the present invention uses the predicted probability of the neural network model to design the loss function, without the need for redundant calculations, and realizes the adaptive update of sample weights based on probabilistic positive feedback, thereby increasing the weight difference between poisoned samples and normal samples, greatly reducing the risk of the neural network model fitting poisoned samples without affecting the recognition of normal samples. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] Figure 1 is a flow chart of the method of the present invention;

[0030] Figure 2 It is the probability distribution histogram of poisoned samples and clean samples;

[0031] Figure 3 Graph of application scenarios for robust training. DETAILED DESCRIPTION

[0032] In order to make the purpose, content and advantages of the present invention more clear, the specific implementation methods of the present invention are further described in detail below with reference to the accompanying drawings and examples.

[0033] The present invention addresses the security protection issues of neural network poisoning attacks, studies poisoning attacks based on neural network algorithms, and provides a robust training method for neural network algorithm poisoning attacks. This method uses the low output probability of poisoned samples to make the output probability positively correlated with the sample weight. Robust training for poisoning attacks is achieved by reducing the weight of the poisoned samples. This method improves the loss function and does not require additional processing of the original samples. The robust training process is simpler and is effective for commonly used data types, with a wider range of adaptability. The loss function is designed using the predicted probability of the neural network model, without the need for redundant calculations. The sample weights are adaptively updated based on probabilistic positive feedback, increasing the weight difference between the poisoned samples and the normal samples, greatly reducing the risk of the neural network model fitting the poisoned samples, and does not affect the recognition of normal samples.

[0034] refer to Figure 1 、 Figure 2 、 Figure 3 The robust training method of the present invention specifically includes the following steps:

[0035] Step 1: Build a neural network.

[0036] A BiLSTM neural network was constructed, including an input layer, a hidden layer, a fully connected layer, and an output layer. The input layer size was set to 64×16000; the number of neurons in the hidden layer was 64; the fully connected layer included a fully connected and ReLU activation function, and the number of neurons in the fully connected layer was 1024; and the number of neurons in the output layer was 8.

[0037] The neural network is constructed according to the designed neural network structure and parameters.

[0038] Step 2: Construct a robust training loss function.

[0039] The robust training loss function is Where k = 8 is the number of sample categories, p i Refers to the probability of the sample being input into the neural network model and predicted to be the i-th category, Indicates the One-Hot encoding of the sample label, with a probability threshold of θ = 0.01. The larger the probability threshold θ (used to prevent attacks during training), the greater the suppression of poisoned samples, but the slower the training speed. The smaller the probability threshold θ, the smaller the suppression of poisoned samples and the faster the training speed. It is a probability mask. When constructing the robust training loss function, it filters out the values ​​whose probability values ​​are lower than the probability threshold θ to prevent the loss value from being too large when the probability is too small. It is a monotonically increasing function, which makes the weight of samples with small probability smaller and the weight of samples with large probability larger.

[0040] Step 3: Use the training set containing the poisoned samples to pre-train the BiLSTM neural network to obtain a neural network pre-training model.

[0041] The 10,000 clean voice command data sets are randomly shuffled and then divided into a training set with 8,000 data and a test set with 2,000 data. 12% of the samples in the training set's original sample category (left) are randomly selected to add poison triggers to generate poisoned samples, and then put back into the training set's category (right) as the neural network training set N. train (Since data poisoning, subsequent training sets have included poisoned samples). The voice command dataset includes voice commands such as down, up, go, stop, left, right, off, and on.

[0042] The robust training loss function is selected as the objective function of the neural network.

[0043] The neural network training set is divided into 125 batches, 64 data are input each time, the loss is calculated using the robust training loss function, and back propagation is performed, and according to Update the network weights, where W k Represents the current weight, W k+1 represents the updated weight, Represents the loss function when W = W k The gradient at time , lr = 1e-3 represents the learning rate, and the above operations (calculation of loss, back propagation, network weight update) are iterated for 20 rounds to obtain the neural network pre-training model.

[0044] In step 4, based on the neural network pre-training model, the probability of the training set containing the poisoned samples being predicted as the true label is obtained, and samples with a predicted probability lower than the filtering threshold are filtered out, thereby eliminating the poisoned samples and obtaining a clean training set.

[0045] The neural network training set N trainInput into the BiLSTM neural network pre-training model to obtain the probability of the sample prediction being the true label. The sample filtering threshold ρ is selected according to the histogram method. The histogram method selects the sample filtering threshold ρ, which means that the probability distribution of the neural network training set samples is displayed in the form of a histogram. The probability characteristics show a bimodal feature. The trough between the two peaks is selected as the filtering threshold. This point represents the intersection of normal samples and poisoned samples, such as Figure 2 As shown in FIG, the dividing point between normal samples and poisoned samples is 0.2, so the sample filtering threshold ρ is set to 0.2. The sample filtering threshold is used to filter out poisoned samples after pre-training is completed.

[0046] The neural network training set N train Input into the BiLSTM neural network pre-training model to obtain the probability of the sample prediction being the true label, and delete the samples with probability lower than the sample filtering threshold ρ = 0.2 to obtain a clean neural network training set N clear_train .

[0047] Step 5: Use the clean training set to retrain the neural network pre-trained model to obtain a neural network model.

[0048] Load the BiLSTM pre-trained model, set the model training parameters to be consistent with the pre-trained parameters, and change the iteration round to T2 = 5.

[0049] Using a clean neural network training set N clear_train The BiLSTM pre-trained model is retrained and training is stopped after a total of 600 iterations to generate a BiLSTM neural network model.

[0050] Step 6: Perform performance test on the neural network model.

[0051] The neural network test set N test Input into the neural network model, test the recognition accuracy ACC of the neural network model for clean samples, and then select the neural network test set N test Randomly select clean samples from the original sample category left, add poisoning triggers and put them back into the target category right to obtain the poisoned sample test set N poision_test , test the attack success rate ACC_Attack of the poisoned sample on the neural network model and the recognition accuracy ACC_Poision of the neural network model on the poisoned sample.

[0052] Replace the robust training loss function with the ordinary cross entropy loss function and compare the changes in test indicators of different models.

[0053] The effects of the present invention are further described below in conjunction with simulation experiments.

[0054] Simulation experiment conditions:

[0055] The hardware platform of the simulation experiment of the present invention is: Intel Core (TM) i7-6700K@4.0GHZ×8, GPU NVIDIA GeForce GTX 1080Ti, 11GB video memory.

[0056] The software platform for the simulation experiment of the present invention is: Windows 10 operating system and Spyder integrated development environment.

[0057] The simulation experiment of the present invention is as follows: first, a portion of samples are selected from the clean sample training set and triggers are added to generate poisoned samples with incorrect labels. These samples are then mixed into the original clean sample training set to obtain a training set containing poisoned samples. The training set containing poisoned samples is then used to obtain a neural network model through two-stage training. The clean sample test set is then used to test the neural network model's recognition accuracy for the clean sample test set, i.e., the proportion of correct labels identified in the clean sample test set. Finally, a trigger is added to the clean sample test set to generate a poisoned sample test set. The success rate of the poisoning attack on the neural network model and the recognition accuracy of the poisoned samples are tested, i.e., the proportion of incorrect labels and the proportion of poisoned samples identified as correct labels in the poisoned sample test set. The results are shown in Table 1.

[0058] Table 1 Robust training test results of neural network algorithm poisoning attack

[0059]

[0060] Comparing the results in the third column of Table 1, under the same poisoning ratio, compared with the traditional cross-entropy loss function, the recognition accuracy of the present invention for clean samples increased from 93.34% to 94.32%. Comparing the results in the fourth column of Table 1, under the same poisoning ratio, compared with the traditional cross-entropy loss function, the success rate of poisoning attacks by the present invention decreased from 83.84% to 0.80%. Comparing the results in the fifth column of Table 1, under the same poisoning ratio, compared with the traditional cross-entropy loss function, the recognition accuracy of poisoned samples by the present invention increased from 15.76% to 94.39%. Therefore, the method of the present invention is suitable for the robust training task of poisoning attacks.

[0061] It can be seen that the present invention uses the model prediction results and the monotonically increasing function to realize the adaptive update of the sample weights, increase the weight difference between the poisoned samples and the clean samples, reduce the risk of the neural network model fitting the poisoned samples, and improve the accuracy of the neural network model classification.

[0062] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the technical principles of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A robust training method for neural network algorithm poisoning attacks, characterized in that: The following steps are involved: Step 1: Build a neural network; Step 2: Construct a robust training loss function; Step 3: Based on steps 1 and 2, pre-train the neural network using a training set containing poisoned samples to obtain a neural network pre-training model; Step 4: Based on step 3, remove the poisoned samples to obtain a clean neural network training set; Step 5: Based on step 4, retrain the neural network pre-training model to obtain a neural network model; Step 6: Perform neural network model performance test based on step 5; The robust training loss function described in step 2 is specifically: Where k is the number of sample categories, p i Refers to the probability of the sample being input into the neural network model and predicted to be the i-th category, Represents the One-Hot encoding of the sample label, It is a probability mask. When constructing the robust training loss function, it filters out values ​​with probability values ​​lower than the probability threshold θ. g(*) is a monotonically increasing function. The input value is the model prediction probability, and the output value is the weight of the current sample, which makes the weight of the sample with small probability smaller and the weight of the sample with large probability larger. Step 3 is as follows: Step 3a: Randomly shuffle the N clean sample data sets and then divide them into N train =N*p pieces of data training set, and N test =N*(1-p) test set N test , randomly select N from the original sample categories of the training set poision =N train *r pieces of data are added with poisoning triggers and then put back into the target category of the training set as the neural network training set N train , 0<p<1, represents the proportion of neural network training set, 0<r<1, represents the proportion of poisoned samples in the training set; Step 3b: Select the robust training loss function as the objective function of the neural network; Step 3c: Divide the neural network training set into batches, b is the number of data input each time, the robust training loss function is used to calculate the loss, and back propagation is performed, and then according to Update the network weights, where W k Represents the current weight, W k+1 represents the updated weight, Represents the robust training loss function when W = W k The gradient at the time, lr represents the learning rate, and the neural network pre-training model is obtained after T1 rounds of iterative operation, where Represents the round-up operation.

2. The method according to claim 1, wherein Step 1 is as follows: Step 1a: Design the structure and parameters of the neural network according to the actual application scenario; Step 1b: Construct a neural network according to the structure and parameters of the designed neural network.

3. The method according to claim 1, wherein Step 4 is as follows: Step 4a: The neural network training set N train Input into the neural network pre-training model to obtain the probability that the sample is predicted to be the true label, and select the sample filtering threshold ρ according to the histogram method; Step 4b: The neural network training set N train Input into the neural network pre-training model to obtain the probability of the sample being predicted as the true label, and delete the samples whose predicted probability is lower than the sample filtering threshold ρ to obtain a clean neural network training set N clear_train .

4. The method according to claim 3, wherein Step 5 is as follows: Step 5a: Load the neural network pre-trained model, set the model training parameters to be consistent with the pre-trained parameters, and change the iteration round to T2; Step 5b: Use the clean neural network training set N clear_train Retrain the neural network pre-trained model for a total of After that, the training is stopped and a neural network model is generated, where Represents the round-up operation.

5. The method according to claim 4, wherein Step 6 is as follows: Step 6a: Test the neural network set N test Input into the neural network model, test the recognition accuracy ACC of the neural network model for clean samples, and then select the neural network test set N test Randomly select clean samples from the original sample category, add poisoning triggers, and put them back into the target category to obtain the poisoned sample test set N poision_test , test the attack success rate ACC_Attack of the poisoned sample on the neural network model and the recognition accuracy ACC_Poision of the neural network model on the poisoned sample; Step 6b: Replace the robust training loss function with the cross entropy loss function, execute steps 3 to 6a again, and compare the changes in test indicators of different loss functions.

6. The method according to claim 1, wherein In step 2, the value range of the probability threshold θ is [1e-2, 1e-3]. The larger the probability threshold θ, the greater the ability to suppress poisoned samples, but the slower the training speed. The smaller the probability threshold θ, the smaller the ability to suppress poisoned samples and the faster the training speed.

7. The method according to claim 3, wherein In step 4a, the histogram method for selecting the sample filtering threshold ρ refers to presenting the probability distribution of the neural network training set samples in the form of a histogram, where the probability characteristics show a bimodal feature, and selecting the trough between the two peaks as the filtering threshold. The point where the trough is located represents the intersection of the normal sample and the poisoned sample.

8. The method according to claim 5, wherein In step 6a, the recognition accuracy ACC of the neural network model for clean samples represents the clean sample test set N test In the test set, the ratio of samples that can be correctly identified by the neural network model is ACC_Attack, which represents the ratio of samples that can be identified as attack categories by the neural network model in the poisoned sample test set. The recognition accuracy of poisoned samples is ACC_Poision, which represents the ratio of samples that can be correctly identified by the neural network model in the poisoned sample test set N. poision_test The proportion of samples that can be correctly identified by the neural network model.

Images