A Neural Network-Based Adversarial Sample Detection Method

Abstract

The present invention relates to the field of artificial intelligence confrontation, and specifically to an adversarial sample detection method based on a neural network, comprising: using an adversarial attack algorithm to attack two sample sets to obtain adversarial sample sets, which become new training sets and test sets; inputting cyclic samples corresponding to input samples into a neural network model to obtain feature matrices of feature maps of each layer generated by each of them after an average pooling layer; calculating the attention weights of each channel of the feature matrix after the average pooling layer and performing weighted processing to obtain a feature matrix; horizontally splicing the feature matrices obtained by the weighted processing to obtain feature vectors; using the maximum mean difference and a kernel function to measure the consistency of the feature vectors; obtaining a feature difference vector obtained by the consistency measurement, inputting the feature difference vector into a detector model, and outputting labels 0 and 1; and achieving more efficient adversarial sample detection by optimizing the feature extraction process and the feature consistency measurement.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence confrontation, and specifically to a method for detecting adversarial samples based on a neural network. Background Art

[0002] In today's digital age, deep learning has achieved outstanding results in many fields with the help of neural network models. For example, it has demonstrated extremely high accuracy and powerful performance in image recognition, speech recognition, and natural language processing, and is widely used in key scenarios such as medical diagnosis, autonomous driving, and financial risk control. However, the emergence of adversarial samples has brought severe challenges to the security and reliability of deep learning.

[0003] Adversarial samples are special samples that are generated by adding subtle perturbations that are difficult for humans to detect to normal samples, but these samples can mislead neural networks to make incorrect classification decisions. For example, in an autonomous driving scenario, adversarial samples formed by adding tiny perturbations to traffic sign images may cause the vehicle's recognition system to misjudge them, causing serious traffic accidents; in the field of medical diagnosis, adversarial samples may lead to misdiagnosis of diseases and endanger the life and health of patients. This not only highlights the seriousness of the adversarial sample problem, but also shows that it is crucial to ensure the robustness of neural networks when facing adversarial samples.

[0004] At present, although a variety of adversarial sample detection methods have been proposed, most of them have limitations.

[0005] First, the design space of adversarial samples is very large. Facing all possible variants, it is difficult to fully capture the characteristics of adversarial samples. Second, some detection methods may be limited to specific machine learning models or fields, have poor adaptability to new attacks, and have high requirements for data labeling, so their robustness and security need to be further enhanced.

[0006] Therefore, people need a neural network-based adversarial sample detection method to solve the above problems. Summary of the invention

[0007] The purpose of the present invention is to provide an adversarial sample detection method based on a neural network to solve the problems raised in the above background technology.

[0008] In order to solve the above technical problems, the present invention provides the following technical solutions:

[0009] A method for detecting adversarial samples based on a neural network, the method comprising:

[0010] S100, obtaining RGB color image data to form a data set; dividing the data set into a training sample set and a test sample set, and using the training sample set as input to train the neural network model ; Use the adversarial attack algorithm to attack the above two types of sample sets to obtain adversarial training sample sets and adversarial test sample sets respectively, and divide the test sample sets and adversarial test sample sets into new training sets and test sets again;

[0011] The dataset contains RGB color images of 10 categories: airplane, automobile, bird, cat, deer, dog, frog, horse, ship, and truck. The image size is 32×32, and there are 50,000 training images and 10,000 test images in the dataset.

[0012] S200, using the new training set as input sample , Generate adversarial network CycleGAN through cycle consistency, and find input samples The corresponding cycle sample ;

[0013] S300: input the loop sample corresponding to the sample Input to the neural network model , get the feature matrix of each layer of feature map generated by the average pooling layer and ;

[0014] S400, calculate the attention weight of each channel for the feature matrix after the average pooling layer and perform weighted processing to obtain the feature matrix and ;

[0015] S500, horizontally concatenate the feature matrices obtained by weighted processing to obtain feature vectors and ;

[0016] S600, for feature vector and , the consistency measure of feature vectors is performed using the maximum mean difference and kernel function;

[0017] S700, obtain the feature difference vector obtained by the consistency measurement, then input the feature difference vector into the detector model, and output labels 0 and 1 for the feature difference vectors corresponding to the normal sample and the adversarial sample respectively. Label 1 represents that the input sample is an adversarial sample, otherwise it is a normal sample.

[0018] Preferably, S200 includes:

[0019] S201, obtain new training set , , then the input sample Through the generator G, according to the formula: ; Get the corresponding cycle sample ;

[0020] S202: Input sample Through the generator F, according to the formula: ; Get the cycle sample .

[0021] Preferably, S300 includes:

[0022] S301, divide the two types of cyclic samples into multiple non-overlapping patches, perform linear projection and position encoding on each patch, then according to the formula: , get the projected and encoded patch representation ;

[0023] in, represents a learnable linear projection matrix, Indicates input patch, represents the position encoding matrix;

[0024] S302, feature extraction through Transformer encoder:

[0025] Based on multiple layers of Transformer encoder, The calculation formula of the layer is:

[0026] ;

[0027] in, , , ; , and Represents a learnable weight matrix, represents the dimension of the key vector;

[0028] The calculation formula of the feedforward neural network is: ;

[0029] in, , , , represents learnable parameters;

[0030] No. The output of the layer is , the final feature map is obtained by stacking multiple layers and ;

[0031] S303: feature map and Perform average pooling to obtain the feature matrix and .

[0032] Preferably, S400 includes:

[0033] S401, feature matrix and Input into a weight generation network composed of a multi-layer perceptron, first through the first fully connected layer and ReLU activation function: ;

[0034] Then pass through the second fully connected layer and the Sigmoid activation function to get the attention weight vector :

[0035] ;

[0036] S402, performing feature weighting:

[0037] ;

[0038] .

[0039] Preferably, S600 includes:

[0040] Get feature vector and , use the maximum mean difference and kernel function to measure the consistency of the feature vector and get the feature difference value , the calculation formula is: ;

[0041] in, Represents the eigenmapping function mapped to the reproducing kernel Hilbert space.

[0042] A computer-readable storage medium stores a computer program, which, when executed by a processor, implements the steps in the above-mentioned neural network-based adversarial sample detection method.

[0043] A computer device includes a memory, a processor, and a computer program stored in the memory and running on the processor. When the processor executes the program, the steps in the above-mentioned adversarial sample detection method based on a neural network are implemented.

[0044] Compared with the prior art, the beneficial effects achieved by the present invention are:

[0045] The present invention achieves more efficient and accurate adversarial sample detection by optimizing the feature extraction process and feature consistency measurement. It uses the Transformer neural network to fully extract sample feature information, and uses a cycle consistency generative adversarial network to perform multi-layer feature map difference detection between normal samples and adversarial samples. It can automatically learn the complex features of the data, capture deep and hidden feature differences, and integrate multi-dimensional feature information to comprehensively judge feature map differences. In addition, the attention mechanism is introduced to adjust the weight of feature information, highlight important information, and improve the accuracy of adversarial sample detection and the stability of the neural network model.

[0046] Adversarial samples are identified by analyzing the consistency difference between the feature representation of normal samples and adversarial samples in the neural network; in the context of adversarial sample detection, normal samples and adversarial samples can be regarded as coming from different "domains"; the feature maps generated by normal samples in the neural network have a certain internal consistency, while adversarial samples may destroy this consistency due to the added disturbance; the cycle-consistent generative adversarial network can realize the conversion between image domains under unsupervised conditions, and its cycle consistency ensures that the data before and after the conversion are similar; if normal samples and adversarial samples are regarded as different domains, the cycle-consistent generative adversarial network is used to analyze the features Figure 1 Consistency may effectively detect adversarial samples, which can not only mine the deep feature relationships of the data, but also have stronger robustness against various types of adversarial attacks, opening up a new and effective way for adversarial sample detection. In addition, in the feature extraction stage, the Transformer-based network architecture is adopted, which can better capture the long-distance dependencies and complex feature information in the samples, making the extracted features more comprehensive and rich. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] The accompanying drawings are used to provide a further understanding of the present invention and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the present invention and do not constitute a limitation of the present invention. In the accompanying drawings:

[0048] Figure 1 It is a flow chart of a neural network-based adversarial sample detection method of the present invention;

[0049] Figure 2 is a sample image used in the simulation in the embodiment of the present invention;

[0050] Figure 3 It is the performance comparison result of simulation in the embodiment of the present invention. DETAILED DESCRIPTION

[0051] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0052] See also Figure 1-Figure 3 , the present invention provides a technical solution:

[0053] Example 1

[0054] A method for detecting adversarial samples based on a neural network, the method comprising:

[0055] S100, obtaining RGB color image data to form a data set; dividing the data set into a training sample set and a test sample set, and using the training sample set as input to train the neural network model ; Use the adversarial attack algorithm to attack the above two types of sample sets to obtain adversarial training sample sets and adversarial test sample sets respectively, and divide the test sample sets and adversarial test sample sets into new training sets and test sets again;

[0056] The dataset contains RGB color images of 10 categories: airplane, automobile, bird, cat, deer, dog, frog, horse, ship, and truck. The image size is 32×32, and there are 50,000 training images and 10,000 test images in the dataset.

[0057] Preferably, S100 includes:

[0058] S101. Divide the data set X into a training set and test set , then the training set As input, train the neural network model ;

[0059] S102: training sample set Conduct adversarial attacks to obtain adversarial sample sets , for the test sample set Attack to obtain adversarial sample set ;

[0060] Among them, the above-mentioned adversarial attack methods respectively use three methods: Generative Adversarial Perturbation GAP, Jacobian Matrix-based Saliency Map Attack Method JSMA and Fast Gradient Sign Method FGSM. GAP consists of a generator and a discriminator. The goal of the generator is to generate adversarial perturbations that can interfere with the target model, while the goal of the discriminator is to distinguish between the original samples and the samples with added adversarial perturbations.

[0061] JSMA generates the Jacobian matrix of the input sample image pixels based on the input sample, calculates the saliency score of each pixel based on the Jacobian matrix element value, forms a saliency map, selects the pixels to be modified on the input image according to a certain strategy, makes slight modifications to them, and recalculates the Jacobian matrix and saliency map. When the perturbation successfully causes the sample to be misclassified, the attack is exited. On the test set, the hyperparameters The values ​​are 0.1, 0.2 and 0.3 respectively;

[0062] In the process of generating adversarial perturbations, FGSM only calculates the gradient once, and then multiplies the sign of the obtained gradient by the hyperparameter ε to obtain the final adversarial perturbation. As it increases, the generated adversarial perturbation noise will also increase. The values ​​are 4 / 255, 6 / 255 and 8 / 255 respectively;

[0063] S103, respectively and Divide the data set in a certain ratio to obtain a new training sample set , , and the new test set , .

[0064] S200, using the new training set as input sample , Generate adversarial network CycleGAN through cycle consistency, and find input samples The corresponding cycle sample ;

[0065] Preferably, S200 includes:

[0066] S201, obtain new training set , , then the input sample Through the generator G, according to the formula: ; Get the corresponding cycle sample ;

[0067] Generator G adds a constraint layer based on the encoder and decoder structure to improve the feature extraction capability of the generator. The structure consists of a convolution layer, an instance normalization layer, a ReLU activation function, a deconvolution layer, a 9-layer residual block, and a Tanh activation function.

[0068] S202: Input sample Through the generator F, according to the formula: ; Get the cycle sample .

[0069] S300: input the loop sample corresponding to the sample Input to the neural network model , get the feature matrix of each layer of feature map generated by the average pooling layer and ;

[0070] Preferably, S300 includes:

[0071] S301, divide the two types of cyclic samples into multiple non-overlapping patches, perform linear projection and position encoding on each patch, then according to the formula: , get the projected and encoded patch representation ;

[0072] in, represents a learnable linear projection matrix, Indicates input patch, represents the position encoding matrix;

[0073] S302, feature extraction through Transformer encoder:

[0074] Based on multiple layers of Transformer encoder, The calculation formula of the layer is:

[0075] ;

[0076] in, , , ; , and Represents a learnable weight matrix, represents the dimension of the key vector;

[0077] The calculation formula of the feedforward neural network is: ;

[0078] in, , , , represents learnable parameters;

[0079] No. The output of the layer is , the final feature map is obtained by stacking multiple layers and ;

[0080] S303: feature map and Perform average pooling to obtain the feature matrix and .

[0081] S400, calculate the attention weight of each channel for the feature matrix after the average pooling layer and perform weighted processing to obtain the feature matrix and ;

[0082] Preferably, S400 includes:

[0083] S401, feature matrix and Input into a weight generation network composed of a multi-layer perceptron, first through the first fully connected layer and ReLU activation function: ;

[0084] Then pass through the second fully connected layer and the Sigmoid activation function to get the attention weight vector :

[0085] ;

[0086] S402, performing feature weighting:

[0087] ;

[0088] .

[0089] S500, horizontally concatenate the feature matrices obtained by weighted processing to obtain feature vectors and ;

[0090] ;

[0091] .

[0092] S600, for feature vector and , the consistency measure of feature vectors is performed using the maximum mean difference and kernel function;

[0093] Preferably, S600 includes:

[0094] Get feature vector and , use the maximum mean difference and kernel function to measure the consistency of the feature vector and get the feature difference value , the calculation formula is: ;

[0095] in, Represents the eigenmapping function mapped to the reproducing kernel Hilbert space.

[0096] S700, obtaining a feature difference vector obtained by the consistency measurement, inputting the feature difference vector into the detector model, and outputting labels 0 and 1 for the feature difference vectors corresponding to the normal sample and the adversarial sample, respectively, where label 1 represents that the input sample is an adversarial sample, and vice versa, it is a normal sample;

[0097] Characteristic difference vector As a detector model The detector model is trained with input from , where the detector model consists of two fully connected layers. The weight matrix size of fully connected layer 1 is 1008×128, using ReLU activation function, and the weight matrix size of fully connected layer 2 is 128×2.

[0098] Normal sample and adversarial examples The characteristic difference vector They correspond to labels 0 and 1 respectively, that is, label 0 represents the input sample is a normal sample, and vice versa is an adversarial sample. and adversarial sample test set Measuring adversarial sample detection models performance of indicators.

[0099] In the model training process of the above steps, the least squares loss is used instead of the traditional cross entropy loss to make the training more stable. For the discriminator, the least squares loss is , for the generator , where x is the input original image sample, z is random noise, G is the generator, and D is the discriminator.

[0100] Adjustment for cycle consistency loss. In CycleGAN, the total loss function is , adjusted through experiments The value of , balances the adversarial loss and cycle consistency loss, and finds the weight that makes the model perform best.

[0101] The adversarial sample detection effect of the present invention can be further illustrated by the following simulation:

[0102] 1. Simulation content

[0103] The present invention takes the CIFAR-10 dataset as an example. The dataset contains images of 10 categories, including airplanes, automobiles, birds, cats, dogs, horses, and trucks. Each category has 6,000 images, and each image size is 32×32. 3,000 images are selected from each category as the training sample set, and the remaining images are used as the test set. Some image samples are shown in Figure 2 CycleGAN is used to generate cycle samples for the data set. The original samples and cycle samples are extracted through the transformer neural network and the attention mechanism is introduced to adjust the weights to increase the attention of important features, thereby obtaining the feature difference vector and finally obtaining the detection and classification results.

[0104] This simulation selected two existing adversarial sample detection models as comparison models. Detection model 1 directly uses a binary classification network to distinguish between normal samples and adversarial samples. Detection model 2 uses a binary classification network model that uses the convolutional feature map output by the middle layer of the neural network as input to distinguish between normal samples and adversarial samples.

[0105] This simulation uses accuracy and average accuracy as model performance evaluation indicators.

[0106] 2. Simulation results

[0107] The simulation results are as follows Figure 3 The adversarial sample detection algorithm proposed in this invention achieved the highest accuracy in the three adversarial attack scenarios, verifying the effectiveness of the algorithm model.

[0108] The present invention adopts the feature based on CycleGAN Figure 1 The consistency detection method is used to detect adversarial samples in the field of image recognition. By finding cycle samples related to the input samples, the weight of important information is increased by introducing the attention mechanism, and the difference between the feature maps of the input samples and the cycle samples is used as the input detection model for adversarial sample discrimination. CycleGAN can be used as a black box model with unknown parameters, which enhances the difficulty of attack. In addition, the detection model has an ideal model complexity and has a high detection accuracy for a variety of adversarial attack samples.

[0109] Example 2

[0110] The computer-readable storage medium of this embodiment stores a computer program, which, when executed by a processor, implements the steps of a neural network-based adversarial sample detection method in Example 1.

[0111] The computer-readable storage medium of this embodiment may be an internal storage unit of the terminal, such as a hard disk or memory of the terminal; the computer-readable storage medium of this embodiment may also be an external storage device of the terminal, such as a plug-in hard disk, a smart memory card, a secure digital card, a flash memory card, etc. equipped on the terminal; further, the computer-readable storage medium may also include both an internal storage unit of the terminal and an external storage device.

[0112] The computer-readable storage medium of this embodiment is used to store computer programs and other programs and data required by the terminal. The computer-readable storage medium can also be used to temporarily store data that has been output or is to be output.

[0113] Example 3

[0114] The computer device of this embodiment includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, the steps of a neural network-based adversarial sample detection method of Embodiment 1 are implemented.

[0115] In this embodiment, the processor may be a central processing unit, or other general-purpose processors, digital signal processors, application-specific integrated circuits, readily available programmable gate arrays or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc. The memory may include a read-only memory and a random access memory, and provide instructions and data to the processor. A portion of the memory may also include a non-volatile random access memory. For example, the memory may also store information about the device type.

[0116] Those skilled in the art will appreciate that the contents disclosed in the embodiments may be provided as methods, systems, or computer program products. Therefore, the present solution may take the form of a hardware embodiment, a software embodiment, or an embodiment combining software and hardware. Moreover, the present solution may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage and optical storage, etc.) containing computer-usable program codes.

[0117] The present solution is described with reference to the method according to the embodiment of the present solution and the flowchart and / or block diagram of the computer program product. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of the processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions; these computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 one or more processes and / or methods Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0118] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 one or more processes and / or methods Figure 1 A function specified in one or more boxes.

[0119] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process in the computer or other programmable device. Figure 1 one or more processes and / or methods Figure 1 The steps for the functions specified in one or more boxes.

[0120] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be implemented by instructing related hardware through a computer program, and the program can be stored in a computer-readable storage medium, and when the program is executed, it can include the processes of the embodiments of the above-mentioned methods. The storage medium can be a disk, an optical disk, a read-only memory (ROM) or a random access memory (RAM), etc.

[0121] Finally, it should be noted that the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art can still modify the technical solutions described in the aforementioned embodiments or replace some of the technical features therein by equivalents. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A method for detecting adversarial samples based on a neural network, characterized in that: The method comprises: S100, obtaining RGB color image data to form a data set; dividing the data set into a training sample set and a test sample set, and using the training sample set as input to train the neural network model f1; using an adversarial attack algorithm to attack the above two types of sample sets to obtain an adversarial training sample set and an adversarial test sample set, respectively, and dividing the test sample set and the adversarial test sample set into a new training set and a test set again; S200, using the new training set as input sample x, generate adversarial network CycleGAN through cycle consistency, and find the cycle sample corresponding to the input sample x S300: input the loop sample corresponding to the sample Input into the neural network model f1, and obtain the feature matrix [V1, V2, …, V n ]and S400, calculate the attention weight of each channel for the feature matrix after the average pooling layer and perform weighted processing to obtain the feature matrix [V'1, V'2, ..., V' n ]and S500, horizontally concatenate the feature matrices obtained by weighted processing to obtain feature vectors V and S600, for the feature vector V and The consistency measure of feature vectors is performed using the maximum mean difference and kernel function; S700, obtaining a feature difference vector obtained by the consistency measurement, inputting the feature difference vector into the detector model, and outputting labels 0 and 1 for the feature difference vectors corresponding to the normal sample and the adversarial sample, respectively, where label 1 represents that the input sample is an adversarial sample, and vice versa, it is a normal sample; The S300 includes: S301, divide the two types of cyclic samples into multiple non-overlapping patches, perform linear projection and position encoding on each patch, then according to the formula: Get the projected and encoded patch representation Where E represents the learnable linear projection matrix, x p represents the input patch, E pos represents the position encoding matrix; S302, feature extraction through Transformer encoder: Based on the multiple layers of the Transformer encoder, the calculation formula for the lth layer is: Where Q = W q z l-1 , K=W k z l-1 , V=W v z l-1 ; W q , W k and W v Represents a learnable weight matrix, d k represents the dimension of the key vector; The calculation formula of the feedforward neural network is: FFN(z)=max(0,zW1+b1)W2+b2; Among them, W1, b1, W2, and b2 represent learnable parameters; The output of layer 1 is z l =FFN(Attention(z l-1 ))+z l-1 , the final feature map H is obtained by stacking multiple layers l and S303, the feature map H l and Perform average pooling to obtain the feature matrix [V1, V2, …, V n ]and 2. The adversarial sample detection method based on a neural network as claimed in claim 1, characterized in that: The S200 includes: S201, get new training set X TRAIN , Then input sample X TRAIN Through the generator G, according to the formula: Get the corresponding cycle sample S202: Input sample Through the generator F, according to the formula: Get loop sample 3. The adversarial sample detection method based on a neural network as claimed in claim 1, characterized in that: The S400 includes: S401, the feature matrix [V1, V2, ..., V n ]and Input into a weight generation network composed of a multi-layer perceptron, first through the first fully connected layer and the ReLU activation function: x = ReLU (W1V n +b1); Then, after the second fully connected layer and the Sigmoid activation function, the attention weight vector a is obtained. c : a c =Sigmoid(W2x+b2); S402, performing feature weighting: [V′1,V′2,…,V′ n ]=a c [V1,V2,…,V n ]; 4. The neural network-based adversarial sample detection method according to claim 1, characterized in that: The S600 includes: Get the eigenvector V and Use the maximum mean difference and kernel function to measure the consistency of the feature vector and get the feature difference value The calculation formula is: Where φ represents the feature mapping function mapped to the reproducing kernel Hilbert space.

5. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the steps of a neural network-based adversarial sample detection method as described in any one of claims 1 to 4 are implemented.

6. A computer device comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that: When the processor executes the program, the steps of the adversarial sample detection method based on a neural network as described in any one of claims 1-4 are implemented.

Images