Image confrontation sample detection method based on hidden layer feature difference
By building an adversarial robust model and adversarial sensitive model, extracting the feature differences of hidden layers and designing adversarial detectors, the problem of insufficient differences between clean samples and adversarial samples and difficulty in detection of adversarial samples is solved, and higher detection is achieved accuracy.
Patent Information
- Application Number
- CN202411292771.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-09-14
- Publication Date
- 2025-01-10
AI Technical Summary
The prior art is difficult to effectively distinguish clean samples from adversarial samples, especially the difficulty in detecting adversarial samples, which leads to the threat of the reliability and security of deep neural networks.
By constructing adversarial robust models and adversarial sensitive models, extracting hidden layer feature differences, designing adversarial detectors, using adversarial sensitivity models and adversarial robust models for adversarial sample detection, amplifying the differences between clean samples and adversarial samples. Improve detection accuracy.
Effectively separating clean samples from adversarial samples improves the accuracy of adversarial sample detection, especially in the detection of minimal perturbation adversarial sample better than traditional methods.
Smart Images
Figure CN119295801A_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the field of artificial intelligence security technology, and in particular to adversarial sample detection in deep neural networks. Background Art
[0002] Adversarial samples refer to malicious samples formed by adding some perturbations that are difficult for the human eye to detect to the original samples. Such malicious samples can cause deep neural networks to make incorrect predictions. The existence of adversarial samples seriously threatens the reliability and security of deep neural networks. Adversarial sample detection, as an important way to defend against adversarial samples, has the advantages of low time and resource costs and little impact on the classification accuracy of clean samples. However, the field of adversarial sample detection still faces some urgent problems, mainly including the lack of obvious differences between clean samples and adversarial samples and the difficulty in detecting adversarial samples with extremely small perturbations.
[0003] (1) There is no obvious difference between clean samples and adversarial samples.
[0004] Clean samples refer to original input data that has not been subjected to any malicious perturbations and can correctly reflect its true category. Adversarial samples, on the other hand, are malicious samples generated by adding carefully designed tiny perturbations. These perturbations are almost invisible changes at the image pixel level. Although they are imperceptible to the human eye, they are enough to cause significant impacts on deep neural networks. Since the perturbations are very subtle, intuitively, adversarial samples are extremely similar to the corresponding clean samples. Existing distance measurement methods often cannot effectively distinguish the difference between the two, which poses a challenge to adversarial sample detection.
[0005] (2) It is difficult to detect adversarial samples with extremely small perturbations.
[0006] When designing adversarial samples, the size of the adversarial perturbation is usually based on the standard that it is imperceptible to the human eye. In the adversarial sample generation method, the adversarial perturbation size is usually limited to less than 8 / 255. This perturbation size is sufficient to ensure that the human eye cannot distinguish the difference between clean samples and adversarial samples, and it can mislead the deep neural network to produce incorrect outputs with a high probability. Existing adversarial sample detection methods usually use this perturbation size setting and design algorithms for adversarial sample detection. However, extremely small perturbation adversarial samples (such as perturbation sizes of 1 / 255 and 2 / 255) still have a considerable probability of misleading deep neural networks to produce incorrect outputs, and this type of adversarial sample is even more difficult to detect because the perturbation amplitude is extremely small. Summary of the invention
[0007] The purpose of the present invention is to provide an image adversarial sample detection method based on hidden layer feature differences, so as to solve the technical problems that the difference between clean samples and adversarial samples is not obvious and the detection of adversarial samples with extremely small perturbations is difficult.
[0008] To achieve the above purpose, the specific technical solutions of the present invention are as follows:
[0009] A method for detecting adversarial samples of an image based on hidden layer feature differences comprises the following steps:
[0010] Step 1: Based on the original classification model, train the adversarial sensitive model and adversarial robust model for extracting hidden layer features, build an adversarial sample dataset, divide the dataset into training set, validation set and test set, and use the dataset to train the adversarial robust model and adversarial sensitive model in adversarial training and adversarial sensitive training ways;
[0011] Step 2: Design an adversarial detector structure based on the adversarial sensitive model, adversarial robust model, and hidden layer feature difference extraction method;
[0012] Step 3: Construct an adversarial sample detection dataset based on the clean sample dataset and the adversarial sample dataset;
[0013] Step 4: Build an adversarial detector based on hidden layer feature differences, use the adversarial sensitive model and adversarial robust model to extract the hidden layer features of the adversarial sample detection dataset, calculate the feature differences, and train the adversarial detector;
[0014] Step 5: Implement an adversarial sample detection framework based on hidden layer feature differences. First, input the sample to be tested into the adversarial sensitive model and the adversarial robust model to extract the hidden layer features, extract the differences between the hidden layer features in the adversarial sensitive model and the adversarial robust model, and then input the extracted feature differences into the adversarial detector based on hidden layer feature differences to realize the detection of adversarial samples.
[0015] Furthermore, the optimization goal of the adversarial training is:
[0016]
[0017] Where x is the training sample, y is the label of the training sample, D is the distribution of the training sample, L is the loss function, r is the adversarial perturbation, ε is the maximum perturbation size, and θ is the model parameter;
[0018] The objective function of the adversarial sensitivity training is:
[0019]
[0020] Among them, x is the training sample, y is the label of the training sample, D is the distribution of the training sample, L is the loss function, r is the adversarial perturbation, ε is the maximum perturbation size, and θ is the model parameter. In adversarial sensitive training, the training data set contains clean samples with correct labels and adversarial samples with incorrect labels.
[0021] Furthermore, step three specifically includes:
[0022] The clean sample dataset and the adversarial sample dataset are merged into an adversarial sample detection dataset, where the label of the clean sample is set to 0 and the label of the adversarial sample is set to 1. When the clean sample is correctly classified and the adversarial sample formed by adding adversarial perturbations to the clean sample is misclassified, the clean sample and the corresponding adversarial sample are retained.
[0023] Furthermore, step four specifically includes:
[0024] The training samples are input into the adversarial robust model and the adversarial sensitive model, and the features of the specified hidden layers of the two models are obtained. The hidden layer features of the two models are extracted and their differences are calculated. A machine learning model is used to detect adversarial samples. The input of the machine learning model is the difference in the hidden layer features of the two models, and the output is whether it is an adversarial sample. The machine learning model is an adversarial detector, and the optimization goal of the adversarial detector is:
[0025]
[0026] x′=D(Hrob(x),Hsen(x))
[0027] Among them, x is the training sample, x' is the difference in hidden layer features between the two models, y is the label of the training sample, D is the distribution of the training sample, L is the loss function, θ is the model parameter, Hsen and Hrob are the hidden layer feature functions of the adversarial sensitive model and the adversarial robust model respectively, and D is the calculation function of the hidden layer feature difference between the two models.
[0028] Furthermore, step five specifically includes: the components of the adversarial sample detection framework based on hidden layer feature differences include an adversarial robust model, an adversarial sensitive model and an adversarial detector. When performing adversarial sample detection on the sample to be tested, the sample to be tested is input into the adversarial robust model and the adversarial sensitive model respectively, and the hidden layer features of specific hidden layers of the two models are extracted. After the difference calculation, the hidden layer features are directly input into the adversarial detector to output whether the sample is an adversarial sample.
[0029] The present invention has the following technical effects:
[0030] The present invention trains adversarial sensitive models and adversarial robust models based on the original model, utilizes the characteristics of adversarial samples themselves, and amplifies the difference between clean samples and adversarial samples by extracting the hidden layer features of the model, so as to better separate clean samples from adversarial samples and facilitate the detection of adversarial samples. At the same time, minimal perturbation adversarial samples are more difficult to detect than adversarial samples with traditional parameter settings. For minimal perturbation adversarial samples, the present invention has a better effect in detecting minimal perturbation adversarial samples. BRIEF DESCRIPTION OF THE DRAWINGS
[0031] Figure 1 This is a flow chart of the image adversarial sample detection method of the present invention;
[0032] Figure 2 A diagram of a method for generating an adversarial robust model and an adversarial sensitive model according to the present invention;
[0033] Figure 3 A diagram of a method for constructing an adversarial sample detection dataset of the present invention;
[0034] Figure 4 A diagram of a method for constructing an adversarial detector of the present invention;
[0035] Figure 5 Schematic diagram of the adversarial sample detection framework structure of the present invention. DETAILED DESCRIPTION
[0036] In order to better understand the purpose, technical solution and function of the present invention, the present invention is further described in detail below in conjunction with the accompanying drawings, but the present invention can be implemented in a variety of different ways as defined and covered by the claims. The accompanying drawings constituting a part of the present invention are used to provide a further understanding of the present invention, and the exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention.
[0037] like Figure 1 As shown, the present invention is an adversarial sample detection method based on hidden layer feature differences. The core idea of the method is to construct an adversarial robust model and an adversarial sensitive model by adversarial training and adversarial sensitive training, extract and amplify the feature differences of the hidden layer after the sample to be tested is input into the two models, and construct an adversarial detector based on the hidden layer feature differences to detect adversarial samples. Adversarial robust models refer to deep learning models that are specially designed or trained to resist adversarial sample attacks. In contrast, adversarial sensitive models are models that have strong adversarial sensitivity to adversarial samples, and such models are more susceptible to adversarial sample perturbations. Since the adversarial robust model and the adversarial sensitive model have similar output results for clean samples, the adversarial robust model is not easily affected by adversarial perturbations, and the adversarial sensitive model is easily affected by adversarial perturbations. Therefore, when clean samples are input into the two models, the hidden layer features of the two models are similar, and when adversarial samples are input into the two models, the hidden layer features of the two models will have a large difference. The present invention realizes the detection of adversarial samples by extracting the hidden layer output differences of the two models for clean samples and adversarial samples, and training a small adversarial detector to learn the feature differences between clean samples and adversarial samples.
[0038] Specifically, the present invention first generates an adversarial sample data set for the original classification model and uses the data set to train an adversarial robust model and an adversarial sensitive model. Then, the adversarial detector structure is determined according to the hidden layer feature extraction position and the feature difference extraction method. Then, the clean sample data set and the adversarial sample data set are used to generate an adversarial sample detection data set, and the data set is used to train the adversarial detector. Finally, the adversarial sensitive model, adversarial robust model and adversarial detector are used to detect adversarial samples.
[0039] The specific steps of the present invention are as follows:
[0040] Step 1: Train the adversarial sensitive model and adversarial robust model for extracting hidden layer features based on the original classification model. First, construct a series of adversarial samples for the original classification model to form an adversarial sample dataset, and divide the dataset into a training set, a validation set, and a test set. Use this dataset to train the adversarial robust model and adversarial sensitive model in adversarial training and adversarial sensitive training.
[0041] Step 2: Design the adversarial detector structure based on the adversarial sensitive model and adversarial robust model and the hidden layer feature difference extraction method. According to the structure of the adversarial sensitive model and adversarial robust model and the hidden layer position of the extracted features, perform hidden layer feature extraction and feature difference calculation. Design the size of the adversarial detector input and the model structure according to the size of the feature difference.
[0042] Step 3: Construct an adversarial sample detection dataset based on the clean sample dataset and the adversarial sample dataset. The clean sample dataset and the adversarial sample dataset are merged into an adversarial sample detection dataset. The clean sample label is 0, and the adversarial sample label is 1.
[0043] Step 4: Construct an adversarial detector based on hidden layer feature differences. Use the adversarial sensitive model and adversarial robust model to extract the hidden layer features of the adversarial sample detection dataset, calculate the feature differences, and use them to train the adversarial detector.
[0044] Step 5: Implement an adversarial sample detection framework based on hidden layer feature differences. First, input the sample to be tested into the adversarial sensitive model and the adversarial robust model to extract the hidden layer features, extract the difference between the hidden layer features of the two models, and then input the extracted feature differences into the adversarial detector based on hidden layer feature differences to detect adversarial samples.
[0045] like Figure 2As shown, the present invention extracts hidden layer features of the adversarial robust model and the adversarial sensitive model and calculates feature differences for adversarial sample detection, requiring that the adversarial robust model and the adversarial sensitive model should be as similar as possible to the original classification model. Therefore, the training of the adversarial robust model and the adversarial sensitive model both uses the original classification model as a pre-training model, and fine-tuning is performed on the basis of the model. Using the original classification model as a pre-training model can also reduce training consumption.
[0046] The training method of the adversarial robust model is conventional adversarial training, that is, adding adversarial samples to the training set, and the training model correctly classifies the adversarial samples. The core idea of this method is to solve a saddle point problem, find the local maximum point of the loss function through the gradient, that is, the adversarial sample, and then add the adversarial sample to the training set to minimize the loss there. The model trained by adversarial training will have strong adversarial robustness to adversarial samples and will not be easily affected by adversarial perturbations. The optimization goal of adversarial training is:
[0047]
[0048] Among them, x is the training sample, y is the label of the training sample, D is the distribution of the training sample, L is the loss function, r is the adversarial perturbation, ε is the maximum perturbation size, θ is the model parameter, and E represents the mathematical expectation.
[0049] The present invention proposes a method for training an adversarial sensitive model, namely adversarial sensitive training. Adversarial sensitive training refers to a method in which adversarial samples are added to a training set, but the labels of the samples are modified to incorrect labels to increase the sensitivity of the model to adversarial samples. The method obtains adversarial samples and their attack targets through targeted adversarial attacks, modifies the labels of the samples to attack targets, and then adds them to the training data set to train the model. The principle of this method is to optimize the parameters of the model so that the model maximizes the local maximum of the loss function, thereby making the model susceptible to adversarial perturbations. The objective function of adversarial sensitive training is:
[0050]
[0051] Where x is the training sample, y is the label of the training sample, D is the distribution of the training sample, L is the loss function, r is the adversarial perturbation, ε is the maximum perturbation size, and θ is the model parameter. In adversarial sensitive training, the training dataset should contain clean samples with correct labels and adversarial samples with incorrect labels, so as to improve the sensitivity of the model to adversarial samples while ensuring the classification accuracy of the model for clean samples.
[0052] like Figure 3As shown in the figure, to build an adversarial detector based on hidden layer features, it is necessary to first build an adversarial sample detection dataset for training. The clean sample dataset and the adversarial sample dataset are merged into an adversarial sample detection dataset, where the label of the clean sample is set to 0 and the label of the adversarial sample is set to 1. To ensure data quality, the data needs to be cleaned. When the clean sample is correctly classified and its corresponding adversarial sample is misclassified, the sample is retained. The purpose of data cleaning is to ensure that the adversarial sample is misclassified due to adversarial perturbations. A high-quality training dataset can enable the trained adversarial detector to learn more accurate features.
[0053] like Figure 4 As shown, after obtaining the adversarial sample detection data set, the training samples are first input into the adversarial robust model and the adversarial sensitive model, and the features of the specified hidden layers of the two models are obtained. Since the adversarial robust model is more robust to adversarial perturbations and the adversarial sensitive model is more sensitive to adversarial perturbations, the hidden layer features of the two models when receiving adversarial sample inputs are quite different. The two models have similar classification performance for clean samples, so the hidden layer features of the two models when receiving clean sample inputs are relatively similar. This method uses this feature to extract the hidden layer features of the two models and calculate their differences. Since the difference is still a high-dimensional abstract data and is difficult to be used directly to detect adversarial samples, this method uses a small machine learning model to detect adversarial samples. The input of the model is the difference in the hidden layer features of the two models, and the output is whether it is an adversarial sample. The machine learning model is an adversarial detector, and its optimization goal is:
[0054]
[0055] x′=D(Hrob(x),Hsen(x))
[0056] Among them, x is the training sample, x' is the difference in hidden layer features between the two models, y is the label of the training sample, D is the distribution of the training sample, L is the loss function, θ is the model parameter, Hsen and Hrob are the hidden layer feature functions of the adversarial sensitive model and the adversarial robust model respectively, and D is the calculation function of the hidden layer feature difference between the two models.
[0057] like Figure 5 As shown, the main components of the adversarial sample detection framework based on hidden layer feature differences implemented by the present invention include an adversarial robust model, an adversarial sensitive model, and an adversarial detector. When performing adversarial sample detection on a sample to be tested, the sample to be tested first needs to be input into the adversarial robust model and the adversarial sensitive model respectively, and the output of the specific hidden layer of the two models, that is, the hidden layer features, is extracted. The hidden layer features extracted by the two models are directly input into the adversarial detector after the difference calculation is performed, and it can output whether the sample is an adversarial sample.
[0058] The present invention trains adversarial sensitive models and adversarial robust models based on the original model, utilizes the characteristics of adversarial samples, and extracts the hidden layer features of the model to amplify the difference between clean samples and adversarial samples. Therefore, clean samples can be better separated from adversarial samples, making it easier to detect adversarial samples.
[0059] Compared with adversarial samples with traditional parameter settings, minimal perturbation adversarial samples are more difficult to detect. The present invention designs an adversarial sample detection method based on model hidden layer differences for minimal perturbation adversarial samples. Therefore, the effect of detecting minimal perturbation adversarial samples is better.
[0060] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the technical principles of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.
Claims
1. A method for detecting adversarial samples of images based on hidden layer feature differences, characterized in that: The following steps are involved: Step 1: Based on the original classification model, train the adversarial sensitive model and adversarial robust model for extracting hidden layer features, build an adversarial sample dataset, divide the dataset into training set, validation set and test set, and use the dataset to train the adversarial robust model and adversarial sensitive model in adversarial training and adversarial sensitive training ways; Step 2: Design an adversarial detector structure based on the adversarial sensitive model, adversarial robust model, and hidden layer feature difference extraction method; Step 3: Construct an adversarial sample detection dataset based on the clean sample dataset and the adversarial sample dataset; Step 4: Build an adversarial detector based on hidden layer feature differences, use the adversarial sensitive model and adversarial robust model to extract the hidden layer features of the adversarial sample detection dataset, calculate the feature differences, and train the adversarial detector; Step 5: Implement an adversarial sample detection framework based on hidden layer feature differences. First, input the sample to be tested into the adversarial sensitive model and the adversarial robust model to extract the hidden layer features, extract the differences between the hidden layer features in the adversarial sensitive model and the adversarial robust model, and then input the extracted feature differences into the adversarial detector based on hidden layer feature differences to realize the detection of adversarial samples.
2. The method for detecting adversarial samples based on hidden layer feature differences according to claim 1, characterized in that: The optimization goal of the adversarial training is: Where x is the training sample, y is the label of the training sample, D is the distribution of the training sample, L is the loss function, r is the adversarial perturbation, ε is the maximum perturbation size, and θ is the model parameter; The objective function of the adversarial sensitivity training is: Among them, x is the training sample, y is the label of the training sample, D is the distribution of the training sample, L is the loss function, r is the adversarial perturbation, ε is the maximum perturbation size, and θ is the model parameter. In adversarial sensitive training, the training data set contains clean samples with correct labels and adversarial samples with incorrect labels.
3. The method for detecting adversarial samples based on hidden layer feature differences according to claim 2, characterized in that: Step three specifically includes: The clean sample dataset and the adversarial sample dataset are merged into an adversarial sample detection dataset, where the label of the clean sample is set to 0 and the label of the adversarial sample is set to 1. When the clean sample is correctly classified and the adversarial sample formed by adding adversarial perturbations to the clean sample is misclassified, the clean sample and the corresponding adversarial sample are retained.
4. The method for detecting adversarial samples based on hidden layer feature differences according to claim 3, characterized in that: Step 4 specifically includes: The training samples are input into the adversarial robust model and the adversarial sensitive model, and the features of the specified hidden layers of the two models are obtained. The hidden layer features of the two models are extracted and their differences are calculated. A machine learning model is used to detect adversarial samples. The input of the machine learning model is the difference in the hidden layer features of the two models, and the output is whether it is an adversarial sample. The machine learning model is an adversarial detector, and the optimization goal of the adversarial detector is: x′=D(Hrob(x),Hsen(x)) Among them, x is the training sample, x' is the difference in hidden layer features between the two models, y is the label of the training sample, D is the distribution of the training sample, L is the loss function, θ is the model parameter, Hsen and Hrob are the hidden layer feature functions of the adversarial sensitive model and the adversarial robust model respectively, and D is the calculation function of the hidden layer feature difference between the two models.
5. The method for detecting adversarial samples of an image based on hidden layer feature differences according to claim 4, characterized in that: Step five specifically includes: the components of the adversarial sample detection framework based on hidden layer feature differences include an adversarial robust model, an adversarial sensitive model, and an adversarial detector. When performing adversarial sample detection on the sample to be tested, the sample to be tested is input into the adversarial robust model and the adversarial sensitive model respectively, and the hidden layer features of the specific hidden layers of the two models are extracted. After the difference calculation, the hidden layer features are directly input into the adversarial detector to output whether the sample is an adversarial sample.