A backdoor attack defense method and defense system based on security training
By adopting a backdoor attack defense method based on security training in the deep neural network model, using filtering indicators and cross-entropy loss function to train the model, the problem that deep neural network models are susceptible to backdoor attacks during training is solved, and the effect of reducing attack success rate and maintaining model accuracy is achieved.
Patent Information
- Application Number
- CN202111571909.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2021-12-21
- Publication Date
- 2025-05-02
- Estimated Expiration
- 2041-12-21
AI Technical Summary
Deep neural network models are susceptible to backdoor attacks during training, resulting in the model performing well on normal test samples but classification of test samples with backdoor triggers is incorrect, affecting the security and reliability of the model.
The backdoor attack defense method based on security training is adopted, and the backdoor poisoning data set is obtained by obtaining the backdoor poisoning data set, and the poisoning samples are filtered out using the two filtering indicators of "average in-class distance" and "subject neighbor tag" to obtain a clean training sample set, and the neural network model is trained using a cross-entropy loss function containing two regularization terms, and the sample filtering and model training steps are performed alternately to optimize the results.
It greatly reduces the success rate of backdoor attacks, maintains the accuracy of the model on normal samples and the robust accuracy of the poisoned samples, and ensures the safety and reliability of the model.
Smart Images
Figure CN114238975B_ABST
Abstract
Description
Technical Field
[0001] The invention relates to a backdoor attack defense method and a defense system based on security training, and belongs to the field of neural network security technology. Background Art
[0002] Deep neural network models are currently widely used in many security-related tasks, such as face recognition, autonomous driving, and malware detection. However, training a deep neural network model usually requires a large amount of data. To achieve state-of-the-art performance, users often purchase data sets from third-party data providers or collect some open source data sets. However, these data collected without reliable human supervision are extremely vulnerable to backdoor attacks. In a backdoor attack, the attacker adds a backdoor trigger to a clean sample to obtain a poisoned sample, and the model is injected with a backdoor during training. The model with the backdoor performs well on normal test samples, but will classify the test samples with the backdoor trigger added as the target category specified by the attacker. Therefore, backdoor attacks can pose a hidden and serious security threat.
[0003] Current backdoor attack methods can be divided into poisoned label attacks and clean label attacks according to the nature of the target label. According to the nature of the trigger, poisoned label attacks can be further divided into sample-related attacks and sample-independent attacks. At the same time, there are also some defense methods to eliminate the threat of backdoor attacks. The mainstream defenses can be divided into two categories, including: (1) Detection-based defense attempts to detect whether the model contains a backdoor, and if so, refuse to use it; (2) Elimination-based defense attempts to eliminate the backdoor in the attacked model and convert it into a clean model. Among them, for the second type of defense, either use additional clean samples to eliminate the backdoor, or distinguish between clean samples and poisoned samples based on their different performance in the model containing the backdoor or in the process of injecting the backdoor into the model, and then eliminate the backdoor. However, since the backdoor has been injected into the model, even if it is eliminated, it will still leave traces in the model, thereby affecting the model's detection accuracy for normal samples. Summary of the invention
[0004] The present invention provides a backdoor attack defense method and defense system based on security training, which can significantly reduce the success rate of backdoor attacks and maintain the accuracy of the model on normal samples and the robust accuracy on poisoned samples.
[0005] In one aspect, the present invention provides a backdoor attack defense method based on security training, the method comprising:
[0006] Step 1: Obtain the backdoor poisoned dataset;
[0007] Step 2: Filter out the poisoned samples in the backdoor poisoned data set using a filtering index to obtain a training sample set;
[0008] Step 3: training a neural network model using the training sample set;
[0009] Step 4: Update the filtering index according to the trained neural network model;
[0010] Step 5: Repeat steps 2 to 4 for a preset number of times.
[0011] Optionally, the filtering index includes a first index and a second index;
[0012] The step 2 specifically includes:
[0013] Using the first indicator, the backdoor poisoned data set is divided into samples labeled as target categories and samples labeled as non-target categories;
[0014] Using the second indicator, the samples labeled as the target category are divided into poisoned samples and clean samples, and after filtering out the poisoned samples, clean samples labeled as the target category are obtained;
[0015] The samples labeled as non-target categories and the clean samples labeled as target categories form a training sample set.
[0016] Optionally, the first indicator is the average intra-class distance The calculation formula is:
[0017]
[0018] Where D i is a data set consisting of samples with label i; h(x) is the feature of sample x; D i The center point in feature space; The set of all categories;
[0019] The second indicator is the subject neighbor label The calculation formula is:
[0020]
[0021] In the formula, For sample x i k nearest neighbors in feature space; is an indicative function.
[0022] Optionally, the step 3 specifically includes:
[0023] The training sample set is input into a neural network model, and the neural network model is trained using a cross entropy loss function including two regularization terms.
[0024] Optionally, the cross entropy loss function for:
[0025]
[0026] In the formula, is a sample whose label is a non-target category; [f(x)] y is the yth element of f(x); e is the number of current training rounds; is a clean sample whose label is the target category.
[0027] Optionally, the two regularization terms are respectively the inter-class similarity and intra-class distance in,
[0028]
[0029] In the formula, is the total number of categories; is the center point of category i in the feature space;
[0030]
[0031] In the formula, is the target category.
[0032] Optionally, the total loss function of the neural network model is for:
[0033]
[0034] In the formula, λ 1 and λ 2 Represent the weight coefficients of the two regularization terms respectively.
[0035] On the other hand, an embodiment of the present invention provides a backdoor attack defense system based on security training, the system comprising:
[0036] The acquisition module is used to obtain the backdoor poisoned data set;
[0037] A sample filtering module, used to filter out poisoned samples in the backdoor poisoned data set using filtering indicators to obtain a training sample set;
[0038] A model training module, used for training a neural network model using the training sample set;
[0039] An index updating module, used for updating the filtering index according to the trained neural network model;
[0040] The control module is used to control the sample filtering module, the model training module and the indicator updating module to repeat the execution of a preset number of times in sequence.
[0041] Optionally, the filtering index includes a first index and a second index;
[0042] The sample filtering module is specifically used for:
[0043] Using the first indicator, the backdoor poisoned data set is divided into samples labeled as target categories and samples labeled as non-target categories;
[0044] Using the second indicator, the samples labeled as the target category are divided into poisoned samples and clean samples, and after filtering out the poisoned samples, clean samples labeled as the target category are obtained;
[0045] The samples labeled as non-target categories and the clean samples labeled as target categories form a training sample set.
[0046] Optionally, the model training module is specifically used for:
[0047] The training sample set is input into a neural network model, and the neural network model is trained using a cross entropy loss function including two regularization terms.
[0048] The beneficial effects that the present invention can produce include:
[0049] The backdoor attack defense method and defense system based on security training provided by the present invention are designed based on the observation of the feature space of the backdoor poisoned data set under the model that has not been attacked, and the method of security training including the sample filtering module and the model training module is designed. In the sample filtering module, the target category and the poisoned sample are respectively distinguished by the two filtering indicators of "average intra-class distance" and "subject neighbor label", so that the backdoor poisoned data set can be filtered into a clean training sample set. In the model training module, the model is trained by using a cross entropy loss function containing two regularization terms so that the backdoor is not injected. The sample filtering module and the model training module are alternately performed and mutually optimized. Therefore, even if the model is directly trained on the backdoor poisoned data set, a clean model without backdoor injection can be obtained. The method proposed by the present invention has strong practical operability, can greatly reduce the attack success rate, and maintain the accuracy of the model on normal samples and the robust accuracy on poisoned samples. BRIEF DESCRIPTION OF THE DRAWINGS
[0050] Figure 1 A flow chart of a backdoor attack defense method based on security training provided by an embodiment of the present invention;
[0051] Figure 2A schematic diagram of a backdoor attack defense system based on security training provided in an embodiment of the present invention. DETAILED DESCRIPTION
[0052] The present invention is described in detail below in conjunction with embodiments, but the present invention is not limited to these embodiments.
[0053] The backdoor defense considered in this invention can be classified as defense based on poisoning suppression, which mainly solves the problem of how to train a clean model on a backdoor poisoned data set. In this scenario, the attacker sets the target category and trigger, and then publishes the poisoned data on a third-party data platform; the victim downloads the poisoned data to train a local model; the purpose of this invention is to provide a safe and reliable training method to protect the victim's trained model from predicting the test sample containing the trigger as the target category, or even as the real category, and maintain its high accuracy on normal test samples.
[0054] Specifically, the embodiment of the present invention provides a backdoor attack defense method based on security training, such as Figure 1 and Figure 2 As shown, the method includes:
[0055] Step 1: Get the backdoor poisoned dataset.
[0056] Step 2: Use filtering indicators to filter out poisoned samples in the backdoor poisoned data set to obtain a training sample set.
[0057] Specifically, the filtering index includes a first index and a second index;
[0058] The step 2 specifically includes:
[0059] The first indicator is used to identify the target category, thereby dividing the backdoor poisoned dataset into samples labeled as the target category and samples labeled as the non-target category;
[0060] The second indicator is used to identify poisoned data, so that samples labeled as the target category are divided into poisoned samples and clean samples. After filtering out the poisoned samples, clean samples labeled as the target category are obtained.
[0061] The samples labeled as non-target categories and the clean samples labeled as target categories form a training sample set.
[0062] Among them, the first indicator is the average intra-class distance The calculation formula is:
[0063]
[0064] Where D i is a data set consisting of samples with label i; h(x) is the feature of sample x; D i The center point in feature space; is a set of all categories; in this embodiment, it can be considered The largest category is the target category
[0065] The second indicator is the subject neighbor label The calculation formula is:
[0066]
[0067] In the formula, For sample x i k nearest neighbors in feature space; is an indicative function; in this embodiment, it is considered The sample is a clean sample labeled as the target category, The sample is a poisoned sample.
[0068] According to the feature space of the current neural network model, two indicators are used to filter the poisoned samples in the dataset, thereby providing a cleaner dataset for training the model.
[0069] Step 3: Use the training sample set to train the neural network model.
[0070] The step 3 specifically includes:
[0071] The training sample set is input into the neural network model, and the neural network model is trained using a cross entropy loss function containing two regularization terms.
[0072] Among them, the cross entropy loss function for:
[0073]
[0074] In the formula, is a sample whose label is a non-target category; [f(x)] y is the yth element of f(x); e is the number of current training rounds; is a clean sample whose label is the target category. The cross entropy loss function It is possible to separate non-target categories from each other in the feature space. In this embodiment, e 0 It can be taken as 90, which means that the consideration of The cross entropy loss function.
[0075] The two regularization terms are the inter-class similarity and intra-class distance
[0076]
[0077] In the formula, is the total number of categories; is the center point of category i in the feature space; the similarity between categories It enables all categories, including the target category, to be separated from each other in the feature space.
[0078]
[0079] In the formula, is the target category; intra-class distance It can make non-target categories more compact in the feature space.
[0080] The neural network model is trained by using the training sample set after filtering out the poisoned samples, and the cross entropy loss function containing two regularization terms is used to train the neural network model, thereby providing a cleaner model to filter the samples.
[0081] In this embodiment of the present invention, the total loss function of the neural network model is for:
[0082]
[0083] In the formula, λ 1 and λ 2 Respectively represent the weight coefficients of the two regularization terms. In this embodiment, λ 1 can be taken as 1.0, λ 2 It can be taken as 0.1.
[0084] Step 4: Update the filtering index according to the trained neural network model.
[0085] Step 5: Repeat steps 2 to 4 for a preset number of times.
[0086] By alternating the steps of sample filtering and model training, they continuously optimize each other and eventually a clean model can be obtained.
[0087] refer to Figure 2As shown, the sample filtering module can filter the poisoned samples in the data set according to the neural network model obtained by the model training module. The model training module can train a clean model according to the clean data set obtained by the sample filtering module. Under the action of the sample filtering module, the model obtained by the model training module is cleaner. Under the action of the model training module, the data set obtained by the sample filtering module is cleaner. As the training process progresses, the two modules gradually optimize each other, and finally a clean model can be obtained. Among them, the preset number of times is a pre-set number of executions, which can be set by those skilled in the art according to actual conditions. The embodiment of the present invention does not limit this. For example, the model can be trained for 100 rounds, that is, the two modules are called 100 times respectively.
[0088] Another embodiment of the present invention provides a backdoor attack defense system based on security training, the system comprising:
[0089] The acquisition module is used to obtain the backdoor poisoned data set;
[0090] A sample filtering module is used to filter out poisoned samples in the backdoor poisoned data set using filtering indicators to obtain a training sample set;
[0091] A model training module is used to train a neural network model using a training sample set;
[0092] An indicator updating module, used to update the filtering indicators according to the trained neural network model;
[0093] The control module is used to control the sample filtering module, the model training module and the indicator updating module to repeat the preset number of times in sequence.
[0094] Further, the filtering index includes a first index and a second index;
[0095] The sample filtering module is specifically used for:
[0096] The first indicator is used to divide the backdoor poisoned dataset into samples labeled as target categories and samples labeled as non-target categories;
[0097] The second indicator is used to divide the samples labeled as the target category into poisoned samples and clean samples. After filtering out the poisoned samples, clean samples labeled as the target category are obtained.
[0098] The samples labeled as non-target categories and the clean samples labeled as target categories form a training sample set.
[0099] Furthermore, the model training module is specifically used for:
[0100] The training sample set is input into the neural network model, and the neural network model is trained using a cross entropy loss function containing two regularization terms.
[0101] The specific description of each module in the above-mentioned defense system can refer to the description of each step in the defense method, which will not be repeated here. The above-mentioned defense system can achieve the same function as the defense method side.
[0102] Aiming at the scenario where an attacker uses a backdoor to poison a data set, the present invention designs a secure training method including a sample filtering module and a model training module, so that a clean model can be directly trained from a data set poisoned by a backdoor. Specifically, first in the sample filtering module, the poisoned samples in the data set are filtered according to two filtering indicators, thereby providing a cleaner data set for another module to train the model; then in the model training module, the model is trained according to a cross entropy loss function including two regularization terms, thereby providing a cleaner model for another module to filter samples; finally, the sample filtering module and the model training module are executed alternately so that they optimize each other, and finally a clean model can be obtained. The present invention can effectively reduce the success rate of an attacker in implementing a backdoor attack, while maintaining a high accuracy rate of the prediction results of normal test samples and a high robust accuracy rate of the prediction results of poisoned test samples.
[0103] The above are only a few embodiments of the present application and do not constitute any form of limitation to the present application. Although the present application is disclosed as above with preferred embodiments, it is not intended to limit the present application. Any technician familiar with the profession, without departing from the scope of the technical solution of the present application, using the technical content disclosed above to make slight changes or modifications are equivalent to equivalent implementation cases and fall within the scope of the technical solution.
Claims
1. A backdoor attack defense method based on security training, characterized in that: The method comprises: Step 1: Obtain the backdoor poisoned dataset; Step 2: Filter out the poisoned samples in the backdoor poisoned data set using a filtering index to obtain a training sample set; Step 3: training a neural network model using the training sample set; Step 4: Update the filtering index according to the trained neural network model; Step 5, repeating steps 2 to 4 for a preset number of times; The filtering index includes a first index and a second index; the step 2 specifically includes: Using the first indicator, the backdoor poisoned data set is divided into samples labeled as target categories and samples labeled as non-target categories; Using the second indicator, the samples labeled as the target category are divided into poisoned samples and clean samples, and after filtering out the poisoned samples, clean samples labeled as the target category are obtained; The samples labeled as non-target categories and the clean samples labeled as target categories form a training sample set; Among them, the first indicator is the average intra-class distance The calculation formula is: Where D i is a data set consisting of samples with label i; h(x) is the feature of sample x; D i The center point in feature space; The set of all categories; The largest category is the target category The second indicator is the subject neighbor label The calculation formula is: In the formula, For sample x i k nearest neighbors in feature space; is the indicative function; The sample is a clean sample labeled as the target category, The sample is a poisoned sample.
2. The backdoor attack defense method according to claim 1, characterized in that: The step 3 specifically includes: The training sample set is input into a neural network model, and the neural network model is trained using a cross entropy loss function including two regularization terms.
3. The backdoor attack defense method according to claim 2, characterized in that: The cross entropy loss function for: In the formula, is a sample whose label is a non-target category; [f(x)] y is the yth element of f(x); e is the number of current training rounds; is a clean sample whose label is the target category.
4. The backdoor attack defense method according to claim 3, characterized in that: The two regularization terms are respectively the inter-class similarity and intra-class distance in, In the formula, is the total number of categories; is the center point of category i in the feature space; In the formula, is the target category.
5. The backdoor attack defense method according to claim 4, characterized in that: The total loss function of the neural network model for: Where λ1 and λ2 represent the weight coefficients of the two regularization terms respectively.
6. A backdoor attack defense system based on security training, characterized in that: The system comprises: The acquisition module is used to obtain the backdoor poisoned data set; A sample filtering module, used to filter out poisoned samples in the backdoor poisoned data set using filtering indicators to obtain a training sample set; A model training module, used for training a neural network model using the training sample set; An index updating module, used for updating the filtering index according to the trained neural network model; A control module, used to control the sample filtering module, the model training module and the indicator updating module to repeat the execution of a preset number of times in sequence; The filtering index includes a first index and a second index; the sample filtering module is specifically used for: Using the first indicator, the backdoor poisoned data set is divided into samples labeled as target categories and samples labeled as non-target categories; Using the second indicator, the samples labeled as the target category are divided into poisoned samples and clean samples, and after filtering out the poisoned samples, clean samples labeled as the target category are obtained; The samples labeled as non-target categories and the clean samples labeled as target categories form a training sample set; Among them, the first indicator is the average intra-class distance The calculation formula is: Where D i is a data set consisting of samples with label i; h(x) is the feature of sample x; D i The center point in feature space; The set of all categories; The largest category is the target category The second indicator is the subject neighbor label The calculation formula is: In the formula, For sample x i k nearest neighbors in feature space; is the indicative function; The sample is a clean sample labeled as the target category, The sample is a poisoned sample.
7. The backdoor attack defense system according to claim 6, characterized in that: The model training module is specifically used for: The training sample set is input into a neural network model, and the neural network model is trained using a cross entropy loss function including two regularization terms.
Citation Information
Patent Citations
Accurate retrieval method for target on the basis of deep metric learning
CN106897390A
Learning device, learning method and learning program
JP2020119091A
Cited By
AI model backdoor defense system and detection method thereof
CN122093173A