Integrated defense method based on adversarial sample detection and matrix decomposition

Abstract

The present invention discloses an integrated defense method based on adversarial sample detection and matrix decomposition. For the input sample of the target model, a threshold-based adversarial sample detection method is first used to perform adversarial sample detection. If the input sample is detected to be an adversarial sample, data is reconstructed for the adversarial sample based on matrix decomposition to obtain a reconstructed sample. When the input sample is a normal sample, the input sample is directly input into the target model for prediction to obtain the prediction result of the input sample; when the input sample is an adversarial sample, the reconstructed sample is input into the target model for prediction as the prediction result of the input sample. The present invention effectively defends against adversarial samples and improves the robustness of the target model by combining the threshold-based adversarial sample detection method and the matrix decomposition-based input sample reconstruction method.

Description

Technical Field

[0001] The present invention belongs to the field of artificial intelligence security technology, and more specifically, relates to an integrated defense method based on adversarial sample detection and matrix decomposition. Background Art

[0002] With the rapid development of artificial intelligence, deep learning has achieved significant applications in many fields, especially in image classification. However, recent studies have shown that deep learning has significant security issues. The emergence of adversarial examples can cause deep learning models to output incorrect classification results with high confidence, posing a significant challenge to AI security. Currently, methods for combating adversarial examples fall into three main categories:

[0003] 1) Adversarial sample defense methods based on enhancing model robustness: Adversarial training is commonly used in this type of method. This method adds adversarial samples to the training dataset to train the deep learning model together, giving the model the ability to defend against adversarial samples. The advantage of this method is that it has good defense effects, but the disadvantage is that the defense cost is relatively high. The model often only has good defense effects on adversarial samples generated by the adversarial sample generation algorithm added to the training set.

[0004] 2) Adversarial sample defense methods based on image preprocessing: Commonly used methods in this category include feature compression and randomization. In feature compression, the input sample is first compressed, and then the compressed image is reconstructed. The compressed and reconstructed image can effectively defend against adversarial samples. Randomization and other methods randomly scale the input sample to a certain size, and then randomly pad around the scaled image to achieve a fixed input size. Finally, the processed image is input into the model for prediction. The advantages of this type of method are simplicity, effectiveness, and low defense cost. The disadvantage is that the introduction of a series of operations such as image preprocessing will reduce the recognition accuracy of normal samples.

[0005] 3) Defense method based on adversarial sample detection: This method is used to separate adversarial samples from input samples so that different samples can be processed separately. The current mainstream detection schemes include feature compression and other methods. The advantage of existing methods is that they can separate adversarial samples before the samples are input into the model. The disadvantage is that the current detection methods have a high false positive rate and also have certain limitations. Summary of the Invention

[0006] The purpose of the present invention is to overcome the shortcomings of the existing technology and provide an integrated defense method based on adversarial sample detection and matrix decomposition. By combining the two technologies of threshold-based adversarial sample detection method and matrix decomposition-based input sample reconstruction method, it can effectively defend against adversarial samples and improve the robustness of the target model.

[0007] To achieve the above-mentioned object, the integrated defense method based on adversarial sample detection and matrix decomposition of the present invention includes the following steps:

[0008] S1: For the input sample of the target model, we first use the threshold-based adversarial sample detection method to detect adversarial samples. The specific method is as follows:

[0009] S1.1: Copy the input sample X M times and then generate M Gaussian noise signals σ that obey the preset distribution m , m=1,2,…,M, and M Gaussian noise signals σ m Add them to the copied input samples respectively to get M interference samples X_adv m ;

[0010] S1.2: Input the original input sample X into the target model for prediction to obtain the prediction result P0, and input the M interference samples X_adv m Input the target model separately for prediction to obtain M prediction results P m ;

[0011] S1.3: Calculate the information entropy H0 for the prediction result of the original input sample X, and calculate the information entropy H0 for the M interference samples X_adv m The prediction result P m Calculate the information entropy H separately m , and then use the following formula to calculate the information entropy score S of each interference sample m :

[0012] S m =|H m -H0|

[0013] S1.4: From M information entropy scores S m Filter out the maximum value S max , if the maximum value S max Greater than the preset threshold S th , then the input sample X is an adversarial sample, otherwise it is a normal sample;

[0014] S2: Determine whether the input sample is an adversarial sample based on the detection result in step S1. If not, proceed to step S4; otherwise, proceed to step S3.

[0015] S3: Data reconstruction based on matrix decomposition for adversarial samples is performed as follows:

[0016] S3.1: Split the input sample data X into N data blocks x n , n=1,2,…,N;

[0017] S3.2: For each data block x n Using different ranks Rn Perform tensor decomposition to obtain three factor matrices A n 、B n 、C n ;

[0018] S3.3: For each data block x n The three factor matrices A n 、B n 、C n Perform rank truncation to obtain the processed factor matrix A′ n , B′ n , C′ n ;

[0019] S3.4: According to the factor matrix A′ n , B′ n , C′ n Reconstruct the data block x′ n Then, according to the data block division method in step S3.1, N data blocks x′ are divided into n Reconstruct the reconstructed sample X′;

[0020] S4: When the input sample X is a normal sample, directly input the input sample X into the target model for prediction to obtain the prediction result of the input sample X; when the input sample X is an adversarial sample, input the reconstructed sample X′ into the target model for prediction as the prediction result of the input sample X.

[0021] The present invention is based on an integrated defense method of adversarial sample detection and matrix decomposition. For the input sample of the target model, a threshold-based adversarial sample detection method is first used to perform adversarial sample detection. If the input sample is detected to be an adversarial sample, data is reconstructed for the adversarial sample based on matrix decomposition to obtain a reconstructed sample. When the input sample is a normal sample, the input sample is directly input into the target model for prediction to obtain the prediction result of the input sample; when the input sample is an adversarial sample, the reconstructed sample is input into the target model for prediction as the prediction result of the input sample.

[0022] The present invention has the following beneficial effects:

[0023] 1) This paper uses a threshold-based adversarial sample detection method to detect input samples. Based on the different output feedback of clean samples and adversarial samples under subtle noise interference, information entropy is used as an indicator to measure adversarial samples, thereby separating adversarial samples.

[0024] 2) The present invention adopts an input sample reconstruction method based on matrix decomposition, which utilizes the feature extraction capability of matrix decomposition to decompose and reconstruct the input sample, thereby demonstrating that the reconstructed sample can destroy the special structure of the adversarial perturbation, effectively defending against adversarial samples and mitigating the harm caused by adversarial samples;

[0025] 3) This invention achieves effective adversarial defense and improves the robustness of the target model by combining two technologies: a threshold-based adversarial sample detection method and a matrix decomposition-based input sample reconstruction method.

[0026] 4) The present invention does not require retraining or adjustment of the target model and can be directly used as a pre-module of the existing target model to improve the adversarial defense effect of the target model with low defense cost. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] Figure 1 This is a flowchart of a specific implementation of the integrated defense method based on adversarial sample detection and matrix decomposition of the present invention;

[0028] Figure 2 This is a comparison chart of the confidence of clean samples and adversarial samples after adding random noise;

[0029] Figure 3 is a flowchart of the threshold-based adversarial sample detection method of the present invention;

[0030] Figure 4 It is a flow chart of data reconstruction of input samples based on matrix decomposition in the present invention;

[0031] Figure 5 is a sample noise comparison diagram before and after data reconstruction in this embodiment;

[0032] Figure 6 3 is a sample comparison diagram before and after data reconstruction in this embodiment. DETAILED DESCRIPTION

[0033] The following describes the specific embodiments of the present invention in conjunction with the accompanying drawings so that those skilled in the art can better understand the present invention. It should be noted that in the following description, when detailed descriptions of known functions and designs may dilute the main content of the present invention, such descriptions will be omitted here.

[0034] Example

[0035] Figure 1 This is a flowchart of a specific implementation of the integrated defense method based on adversarial sample detection and matrix decomposition of the present invention. Figure 1 As shown, the specific steps of the integrated defense method based on adversarial sample detection and matrix decomposition of the present invention include:

[0036] S101: Threshold-based adversarial sample detection:

[0037] For the input samples of the target model, a threshold-based adversarial sample detection method is first used to detect adversarial samples.

[0038] Adversarial examples are created by adding carefully designed perturbations to input samples. Their purpose is to cause deep learning models to output incorrect predictions with high confidence, such as misidentifying a cat as a dog. This obviously poses significant security risks in the real world. Typically, trained models are robust to common noise. This means that even when subjected to common noise, the model can still correctly identify an object. Adversarial examples are created by adding carefully designed perturbations to clean samples. These perturbations typically have a unique structure that causes the model to misidentify the object. Adding noise to adversarial examples often alters this unique structure, significantly changing the model's recognition results. Figure 2 This is a comparison chart of the confidence of clean samples and adversarial samples after adding random noise. Figure 2 It can be seen that when random noise is added to the clean sample, its confidence is not greatly affected, but when random noise is added to the adversarial sample, the confidence changes significantly.

[0039] Based on the above principles, the present invention proposes a threshold-based adversarial sample detection method. Figure 3 This is a flowchart of the threshold-based adversarial sample detection method in the present invention. Figure 3 As shown, the specific steps of the threshold-based adversarial sample detection method in the present invention include:

[0040] S301: Generate interference samples:

[0041] Copy the input sample X M times, and then generate M Gaussian noise signals σ that obey the preset distribution m , m=1,2,…,M, and M Gaussian noise signals σ m Add them to the copied input samples respectively to get M interference samples X_adv m .

[0042] S302: Sample prediction:

[0043] The original input sample X is input into the target model for prediction to obtain the prediction result P0, and the M interference samples X_adv m Input the target model separately for prediction to obtain M prediction results P m The prediction result is a vector of probability values ​​that the sample belongs to each category to be detected by the target model.

[0044] S303: Calculate information entropy score:

[0045] The information entropy H0 is calculated based on the prediction result of the original input sample X, and the information entropy H0 is calculated based on the M interference samples X_adv m The prediction result P m Calculate the information entropy H separately m, and then use the following formula to calculate the information entropy score S of each interference sample m :

[0046] S m =|H m -H0|

[0047] S304: Adversarial Sample Determination:

[0048] From M information entropy scores S m Filter out the maximum value S max , if the maximum value S max Greater than the preset threshold S th , then the input sample X is an adversarial sample, otherwise it is a normal sample. Threshold S th Can be obtained through experiments.

[0049] S102: Determine whether the input sample is an adversarial sample based on the detection result in step S101. If not, proceed to step S104; otherwise, proceed to step S103.

[0050] S103: Reconstruct data of adversarial samples based on matrix decomposition:

[0051] For adversarial samples, data reconstruction is performed based on matrix decomposition. Figure 4 This is a flow chart of data reconstruction of input samples based on matrix decomposition in the present invention. Figure 4 As shown, the specific steps of reconstructing the input sample data based on matrix decomposition in the present invention include:

[0052] S401: Data segmentation:

[0053] Split the input sample data X into N data blocks x n , n=1,2,…,N.

[0054] S402: Tensor decomposition:

[0055] Pre-set N ranks R n , for the nth data block x n Using rank R n Perform tensor decomposition. Since the input sample in the present invention is a third-order tensor, three factor matrices A can be obtained through tensor decomposition: n 、B n 、C n .

[0056] For tensor decomposition, the current tensor decomposition method is to initialize the factor matrix first, and then iteratively update to obtain the final factor matrix. The calculation process is relatively complex and time-consuming. In order to improve the efficiency of the algorithm, this embodiment proposes a tensor decomposition method based on a neural network. This method can utilize existing GPU resources when performing tensor decomposition, greatly improving the efficiency of the algorithm. The specific method is: construct N neural networks, each neural network corresponds to a rank R n , whose input is a data block and output is a rank R n The factor matrix obtained by decomposition. For each neural network, several tensor decomposition samples are obtained in advance. Each tensor decomposition sample includes a tensor with the same size as the data block, and the tensor adopts rank R n Decompose the factor matrix obtained, use the tensor as the input of the neural network, and the corresponding factor matrix as the expected output to train the neural network. Then use the data block x n Input the nth trained neural network to get the data block x n Using rank R n Factor matrix A for tensor decomposition n 、B n 、C n .

[0057] The setting of the loss function is very important for the training effect of the neural network. In order to improve the training effect of the tensor decomposition neural network, the loss function in this embodiment adopts the sum of squares of the residuals, that is, the calculation formula of the loss function L is as follows:

[0058]

[0059] in, Represents the input training sample tensor The value of the element with coordinates (i, j, k) in the . Represents the training sample tensor Output factor matrix The element at position (i, r) in Represents the training sample tensor Output factor matrix The element at position (j,r) in Represents the training sample tensor Output factor matrix The element at position (k,r) in .

[0060] Then, the gradient descent method is used to update the parameters of the neural network based on the calculated loss function to complete the training of the neural network.

[0061] S403: Rank Truncation:

[0062] For each data block x n The three factor matrices A n 、B n 、C n Perform rank truncation to obtain the processed factor matrix A′ n , B′ n , C′ n Since the data in the front columns of the factor matrix are more important, the core features of the factor matrix can be extracted and redundant data can be eliminated through rank truncation.

[0063] S404: Data reconstruction:

[0064] According to the factor matrix A′ n , B′ n , C′ n Reconstruct the data block x′ n Then, according to the data block division method in step S401, N data blocks x′ are divided into n Reconstruct the reconstructed sample X′.

[0065] S104: Target model prediction:

[0066] When the input sample X is a normal sample, the input sample X is directly input into the target model for prediction, which is used as the prediction result of the input sample X. When the input sample X is an adversarial sample, the reconstructed sample X′ is input into the target model for prediction, which is used as the prediction result of the input sample X.

[0067] In order to better illustrate the technical effects of the present invention, specific examples are used to experimentally verify the present invention.

[0068] First, the effectiveness of the threshold-based adversarial sample detection method used in the present invention is experimentally verified. In this experimental verification, the Feature Squeezing method (see the document Xu, Weilin & Evans, David & Qi, Yanjun. (2018). Feature Squeezing: Detecting Adversarial Examples in Deep NeuralNetworks.10.14722 / ndss.2018.23210.0) and the MagNet method (see the document Meng D, Chen H. MagNet: a Two-Pronged Defense against Adversarial Examples [J]) were selected as comparison methods. The samples generated by the PGD attack and the DeepFool attack were detected by the present invention and the comparison method respectively, and the accuracy and false positive rate were compared. In order to better compare the false positive rate, the accuracy of the present invention and the comparison method was adjusted to be basically the same.

[0069] The datasets used are the MNIST and CIFAR10 datasets. The MNIST dataset comes from the National Institute of Standards and Technology. The training set consists of handwritten digits from 250 different individuals, 50% of whom are high school students and 50% are Census Bureau staff. The test set also contains handwritten digits from the same proportions, but the author sets of the test and training sets are kept disjoint. The MNIST dataset contains 70,000 images, 60,000 of which are training and 10,000 are testing. Each image is a 28×28 image of handwritten digits from 0 to 9. Each image is a black background with white text, where the black background is represented by 0 and the white text is represented by a floating-point number between 0 and 1, with the closer to 1 the whiter the text. CIFAR-10 is a computer vision dataset for pervasive object recognition, collected by Hinton's students Alex Krizhevsky and Ilya Sutskever. It contains 60,000 32×32 RGB color images with 10 categories. Among them, 50,000 images are used for training set and 10,000 images are used for testing set.

[0070] Table 1 is a comparison table of the detection results of the threshold-based adversarial sample detection method and the comparison method on the MNIST dataset in this embodiment.

[0071]

[0072] Table 1

[0073] Table 2 is a comparison table of the detection results of the threshold-based adversarial sample detection method of the present invention and the comparison method on the CIFAR10 dataset in this embodiment.

[0074]

[0075] Table 2

[0076] As can be seen from Tables 1 and 2, on the MNIST dataset, the accuracy of our threshold-based adversarial example detection method is roughly comparable to that of the two comparison methods, with a false positive rate between that of the Feature Squeezing method and the MagNet method. On the CIFAR10 dataset, the Feature Squeezing method achieved higher accuracy, but also a higher false positive rate. Our threshold-based adversarial example detection method achieved similar accuracy to the Feature Squeezing method, but with a lower false positive rate.

[0077] Next, we experimentally validate the effectiveness of the matrix decomposition-based data reconstruction technique in this invention. In this experiment, we added perturbation noise generated by the FGSM algorithm to clean samples and compared the added noise with Gaussian noise and the noise of the samples reconstructed from the data in this invention. Figure 5 This is a sample noise comparison diagram before and after data reconstruction in this embodiment. Figure 5 As shown in the figure, the disturbance noise generated by the FGSM algorithm is quite different from Gaussian noise. This adversarial disturbance noise has a special structure that causes the model to recognize errors. Through the input sample reconstruction method based on matrix decomposition of the present invention, this adversarial disturbance is converted into ordinary noise similar to Gaussian noise, thereby effectively resisting the harm caused by adversarial samples.

[0078] Figure 6 This is a sample comparison diagram before and after data reconstruction in this embodiment. Figure 6 As shown, the first row is the adversarial sample of the FGSM algorithm, and the disturbance noise gradually increases from left to right. The second row is the reconstructed sample obtained by the input sample reconstruction method based on matrix decomposition of the present invention. It can be seen that the disturbance information in the adversarial sample is invisible to the human eye, and the reconstructed sample obtained by the present invention based on matrix decomposition can effectively remove the disturbance noise and destroy the special structure of the adversarial disturbance, which helps to improve the prediction accuracy of the target model and achieve adversarial defense.

[0079] Finally, the anti-defense effect of the present invention was experimentally verified. The comparison methods used in this experimental verification include:

[0080] Random is a method proposed by Cihang et al. that mitigates the effects of adversarial examples through randomization. Specifically, the authors used two random operations: random resizing (resizing the input image to a random size) and random padding (filling the input image with zeros in a random manner).

[0081] Label smoothing is a regularization strategy that mainly adds noise through soft one-hot, reducing the weight of the category of the real sample label when calculating the loss function, and ultimately suppressing overfitting.

[0082] Feature Squeezing is a method proposed by Weilin et al. in 2018. This method studies two feature compression methods: reducing the color depth of each pixel and spatial smoothing, in order to improve the model's ability to deal with adversarial samples.

[0083] AT (Adversarial Training) is one of the classic methods for dealing with adversarial samples. This method trains the network model by adding adversarial samples to the training set to improve the robustness of the model, making the model have a certain degree of defense against adversarial samples, but slightly reducing the recognition rate of clean samples.

[0084] Table 3 is a comparison table of the PGD attack defense experimental results of the present invention and four defense methods on the MNIST dataset in this embodiment.

[0085]

[0086] Table 3

[0087] Table 4 is a comparison table of the experimental results of the PGD attack defense of the present invention and four defense methods on the CIFAR10 dataset in this embodiment.

[0088]

[0089] Table 4

[0090] The numbers in the PGD subscripts in Tables 3 and 4 represent the number of iterations, and the attack perturbation thresholds ζ are selected as 0.02 and 0.04. As can be seen from Tables 3 and 4, the present invention can significantly improve the recognition rate of adversarial samples while maintaining the recognition rate of clean samples, thereby enhancing the robustness of the target model.

[0091] Although the above describes the illustrative specific embodiments of the present invention to facilitate understanding of the present invention by those skilled in the art, it should be clear that the present invention is not limited to the scope of the specific embodiments. For those skilled in the art, as long as various changes are within the spirit and scope of the present invention as defined and determined by the appended claims, these changes are obvious, and all inventions and creations using the concepts of the present invention are protected.

Claims

1. An integrated defense method based on adversarial sample detection and matrix decomposition, characterized in that: The following steps are involved: S1: For the input image sample of the target model, we first use the threshold-based adversarial sample detection method to detect adversarial samples. The specific method is as follows: S1.1: Copy the input image sample X M times, and then generate M Gaussian noise signals σ that obey the preset distribution m , m=1,2,…,M, and M Gaussian noise signals σ m Add them to the copied input image samples respectively to obtain M interference image samples X_adv m ; S1.2: Input the original input image sample X into the target model for prediction to obtain the prediction result P0, and input the M interference image samples X_adv m Input the target model separately for prediction to obtain M prediction results P m ; S1.3: Calculate the information entropy H0 based on the prediction result of the original input image sample X, and calculate the information entropy H0 for the M interference image samples X_adv m The prediction result P m Calculate the information entropy H separately m , and then use the following formula to calculate the information entropy score S of each interference image sample m : S m =|H m -H0| S1.4: From M information entropy scores S m Filter out the maximum value S max , if the maximum value S max Greater than the preset threshold S th , then the input image sample X is an adversarial sample, otherwise it is a normal sample; S2: Determine whether the input image sample is an adversarial sample based on the detection result in step S1. If not, proceed to step S4; otherwise, proceed to step S3. S3: Data reconstruction based on matrix decomposition for adversarial samples is performed as follows: S3.1: Split the input image sample data X into N data blocks x n , n=1,2,…,N; S3.2: For each data block x n Using different ranks R n Perform tensor decomposition to obtain three factor matrices A n 、B n 、C n ; The specific method of tensor decomposition is: Construct N neural networks, each corresponding to a rank R n , whose input is a data block and output is a rank R n The factor matrix obtained by decomposition; for each neural network, several tensor decomposition samples are obtained in advance, each tensor decomposition sample includes a tensor with the same size as the data block, and the tensor adopts rank R n The factor matrix obtained by decomposition is used as the input of the neural network, and the corresponding factor matrix is ​​used as the expected output to train the neural network; then the data block x n Input the nth trained neural network to get the data block x n Using rank R n Factor matrix A for tensor decomposition n 、B n 、C n ; S3.3: For each data block x n The three factor matrices A n 、B n 、C n Perform rank truncation to obtain the processed factor matrix A n ′、B n ′、C n '; S3.4: According to the factor matrix A n ′、B n ′、C n 'Reconstruct the data block x' n Then, according to the data block division method in step S3.1, N data blocks x′ are divided into n Reconstruct the reconstructed image sample X′; S4: When the input image sample X is a normal sample, the input image sample X is directly input into the target model for prediction to obtain the prediction result of the input image sample X; when the input image sample X is an adversarial sample, the reconstructed image sample X′ is input into the target model for prediction as the prediction result of the input image sample X.

2. The integrated defense method according to claim 1, characterized in that: The calculation formula of the loss function L during the neural network training process is as follows: in, Represents the input training sample tensor The value of the element with coordinates (i, j, k) in the . Represents the training sample tensor Output factor matrix The element at position (i, r) in Represents the training sample tensor Output factor matrix The element at position (j,r) in Represents the training sample tensor Output factor matrix The element at position (k,r) in .

Images