Adversarial image detection method and device based on convolutional layer activation difference

Abstract

The application discloses a kind of based on the activation difference of convolution layer and the detection method and device of image confrontation, comprising: obtaining image to be processed, and image to be processed is input to image classification model;Obtain the feature map of the output of preset convolution layer in image classification model, and determine the activation proportion of feature map;After the activation proportion is input detection model, whether the classification result of image classification model to image to be processed exists abnormality is judged according to the output of detection model;Wherein, detection model is the fully connected neural network model obtained after training with the activation proportion of normal image and confrontation sample in image classification model as training sample in advance.This application calculates the activation proportion of feature map in the normal forward propagation process of image classification model, while almost without causing the time delay of online application, so that detection model judges whether the classification result exists abnormality according to the activation proportion, to carry out security reinforcement to the image classification model that has been online.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of artificial intelligence, and particularly relates to a convolutional layer activation difference-based adversarial image detection method and device. BACKGROUND

[0002] In recent years, a series of industries with image classification as the technical core, such as face recognition, image content review, intelligent security and automatic driving, have developed rapidly and covered important scenes such as authentication, payment, medical treatment and travel in daily life. However, researchers have found that the security of the image classification model is insufficient and is vulnerable to attacks by attackers.

[0003] The image adversarial sample is an attack launched by an attacker against an image classification model that is providing services, also known as an "escape attack". The goal of the attacker is to add perturbations to an existing picture so that the modified picture does not visually change the category, but the image classification model will misclassify it. In this process, the attacker does not need to tamper with the parameters of the image classification model, but greatly reduces the classification accuracy. For example, in the network image content supervision scene, the attacker may bypass the detection system through the adversarial sample and then publish pictures containing illegal content; in the intelligent security monitoring, the attacker may wear clothes with a specific pattern to avoid detection; in the automatic driving scene, the attacker may add specific content to the traffic sign to make the car's sign recognition system make mistakes, causing traffic safety hazards.

[0004] In related technologies, the image adversarial sample detection method such as NIC needs to train multiple sub-models, or the CDRP needs to obtain relevant information by backstepping after the image classification model makes a prediction result, and then detect it, which causes the image adversarial sample detection method to have a high time delay when the image classification model makes a final judgment, and makes it impossible for the online application to handle user requests in high concurrency. SUMMARY

[0005] In order to solve the above problems in the prior art, the application provides a convolutional layer activation difference-based adversarial image detection method and device. The technical problems to be solved by the application are solved by the following technical solutions:

[0006] In a first aspect, the application provides a convolutional layer activation difference-based adversarial image detection method, comprising:

[0007] obtaining a to-be-processed image and inputting the to-be-processed image into an image classification model; the image classification model is a pre-trained convolutional neural network model, comprising a convolutional layer, a pooling layer and a fully connected layer;

[0008] obtaining a feature map output by a preset convolutional layer in the image classification model, and determining an activation ratio of the feature map;

[0009] After inputting the activation ratio into the detection model, it is determined whether the classification result of the image classification model for the image to be processed is abnormal according to the output of the detection model; wherein the detection model is a full connection neural network model representing the abnormal probability of the input image, which is obtained after pre-training with the activation ratio of the normal image and the adversarial sample in the image classification model as training samples.

[0010] In an embodiment of the present application, the step of obtaining the feature map output by the preset convolution layer in the image classification model and determining the activation ratio of the feature map comprises:

[0011] After selecting two convolution layers in the image classification model, a first feature map and a second feature map output by the two convolution layers are obtained respectively;

[0012] The activation ratios of the first feature map and the second feature map are calculated respectively.

[0013] In an embodiment of the present application, the step of calculating the activation ratios of the first feature map and the second feature map respectively comprises:

[0014] The activation ratios of each channel in the first feature map are calculated to obtain a first activation ratio;

[0015] The activation ratios of each channel in the second feature map are calculated to obtain a second activation ratio.

[0016] In an embodiment of the present application, the activation ratio of the feature map is determined according to the following formula:

[0017]

[0018] wherein M represents the feature map output by the convolution layer, h and w represent the height and width of the feature map respectively, φ represents a preset threshold value, e represents the matrix element in the feature map, the matrix element greater than the preset threshold value in the feature map is regarded as activated, and R represents the calculated activation ratio of the feature map.

[0019] In an embodiment of the present application, the detection model is trained and obtained by the following steps:

[0020] A first type of training sample is obtained, which includes a plurality of normal images, normal activation ratios corresponding to the normal images, a plurality of adversarial samples corresponding to each normal image, and abnormal activation ratios corresponding to the adversarial samples;

[0021] inputting the preset number of first type training samples into the full connection neural network model to be trained, and determining a loss value according to an output result of the full connection neural network model to be trained and a preset loss function;

[0022] determining whether the full connection neural network model to be trained converges according to the loss value; if yes, the full connection neural network model to be trained is a detection model trained completely;

[0023] if no, adjusting network parameters of the full connection neural network model to be trained, and returning to the step of inputting the preset number of first type training samples into the full connection neural network model to be trained.

[0024] In an embodiment of the present application, the first type training samples are obtained by the following steps:

[0025] obtaining first training samples; the first training samples include a plurality of normal images;

[0026] after inputting the first training samples into the image classification model, obtaining feature maps output by all convolution layers in the image classification model, and determining a normal activation proportion of the feature maps;

[0027] generating a plurality of adversarial samples corresponding to each normal image to obtain second training samples;

[0028] after inputting the second training samples into the image classification model, obtaining feature maps output by all convolution layers in the image classification model, and determining an abnormal activation proportion of the feature maps;

[0029] generating first type training samples, the first type training samples including the normal activation proportion of each normal image and the abnormal activation proportion of the plurality of adversarial samples corresponding to each normal image.

[0030] In a second aspect, the present application provides an adversarial image detection device based on convolution layer activation difference, comprising:

[0031] a first obtaining module for obtaining an image to be processed and inputting the image to be processed into an image classification model; the image classification model is a pre-trained convolution neural network model, including a convolution layer, a pooling layer and a full connection layer;

[0032] a second obtaining module for obtaining a feature map output by a preset convolution layer in the image classification model, and determining an activation proportion of the feature map;

[0033] The input module is configured to input the activation ratio into the detection model, and determine whether the classification result of the image classification model for the image to be processed is abnormal according to an output of the detection model.

[0034] Compared with the prior art, the present application has the following beneficial effects:

[0035] In view of the threat of the adversarial image to the image classification model, the present application provides an adversarial image detection method and device based on the activation difference of the convolution layer, which calculates the activation ratio of the feature map in the normal forward propagation process of the image classification model, so that the detection model determines whether the classification result of the image classification model for the image to be processed is abnormal according to the activation ratio, thereby performing security reinforcement on the online image classification model. In addition, since the calculation of the activation ratio of the feature map ends in the normal forward propagation process, it almost does not cause time delay to the online application, thereby solving the problem of high time delay of the existing detection method.

[0036] The present application will be further described in detail below with reference to the accompanying drawings and embodiments. BRIEF DESCRIPTION OF DRAWINGS

[0037] Figure 1 is a flowchart of the adversarial image detection method based on the activation difference of the convolution layer provided by the embodiment of the present application;

[0038] Figure 2 is a schematic diagram of the image classification model provided by the embodiment of the present application;

[0039] Figure 3 is a schematic diagram of the pooling operation provided by the embodiment of the present application;

[0040] Figure 4 is a visual result schematic diagram of the feature map provided by the embodiment of the present application;

[0041] Figure 5 is a schematic diagram of the adversarial image detection method based on the activation difference of the convolution layer provided by the embodiment of the present application;

[0042] Figure 6 is a structural schematic diagram of the adversarial image detection device based on the activation difference of the convolution layer provided by the embodiment of the present application. DETAILED DESCRIPTION

[0043] The present application will be further described in detail below with reference to the accompanying drawings and embodiments.

[0044] Figure 1 is a flow chart of the adversarial image detection method based on the activation difference of the convolutional layer provided by the embodiment of the present application. Please refer to Figure 1 , the embodiment of the present application provides an adversarial image detection method based on the activation difference of the convolutional layer, comprising:

[0045] S1, obtaining a to-be-processed image, and inputting the to-be-processed image into an image classification model; the image classification model is a pre-trained convolutional neural network model, comprising: a convolutional layer, a pooling layer and a fully connected layer;

[0046] S2, obtaining a feature map output by a preset convolutional layer in the image classification model, and determining an activation ratio of the feature map;

[0047] S3, inputting the activation ratio into a detection model, and determining whether the classification result of the to-be-processed image by the image classification model is abnormal according to the output of the detection model; wherein the detection model is a fully connected neural network model representing the abnormal probability of the input image, which is obtained by pre-training with the activation ratio of the normal image and the adversarial sample in the image classification model as training samples.

[0048] Figure 2 is a schematic diagram of the image classification model provided by the embodiment of the present application. Specifically, as shown in Figure 2 , the image classification model used for classifying the to-be-processed image in the embodiment is a pre-trained convolutional neural network model, comprising: a convolutional layer, a pooling layer and a fully connected layer, wherein the convolutional layer and the pooling layer can be stacked in multiple layers. It should be noted that the activation function brings non-linear expression ability to the convolutional neural network model, but since it is an element-level operation rather than a network structure, it is not regarded as a separate layer structure in Figure 2 , but is merged into the convolutional layer and the like.

[0049] It should be understood that the above image classification model processes the image by extracting features layer by layer and calculating the probability of the to-be-processed image belonging to each category. Specifically, the convolutional layer extracts features of the to-be-processed image through the convolution kernel, a plurality of convolution kernels in the same layer extract different features and generate a corresponding number of feature maps after being mapped by the activation function, and these feature maps are then subjected to higher-level feature extraction by the subsequent convolutional layer. Figure 3 is a schematic diagram of the pooling operation provided by the embodiment of the present application. Further, as shown in Figure 3As shown, the pooling layer down-samples the feature map, and common pooling operations include max pooling and average pooling; for example, assuming that the size of the pooling window is 2x2 and the step is 2, then the size of the feature map after pooling is reduced to one quarter of the original size; the pooling operation can effectively reduce the calculation amount of the image classification model, and also makes the feature map in the middle of the image classification model have certain invariance to deformation operations such as slight translation of the input picture. The fully connected layer is at the end of the image classification model, can flatten the feature map into a one-dimensional vector, and then establish a connection with each neuron of the fully connected layer; the last of the fully connected layer is a number of neurons equal to the number of categories, and the values output by these neurons are converted into probabilities of each category for the processed image after a softmax operation.

[0050] After the attacker adds perturbations to the processed image, the modified processed image does not visually change the category, but the image classification model incorrectly classifies it. Figure 4 is a schematic diagram of a visualization result of a feature map provided by an embodiment of the present application. Referring to Figure 4 , the inventors found in the research process that the attacker adds noise to the normal processed image according to a certain strategy, and this noise affects the forward propagation calculation of the image classification model and induces the model to make an incorrect category prediction; and this effect specifically acts on the feature map obtained by the convolution operation in the image classification model; compared with the state of the normal processed image in the image classification model, the adversarial perturbation first changes the input processed image, and then the perturbation causes the feature map calculated by the convolution kernel to change in the first convolution layer, including the number and position of activated elements in the feature map, and these changes gradually cause the subsequent extracted feature maps to change.

[0051] Based on the above analysis of the influence of the adversarial perturbation on the feature map, the inventors believe that the vulnerability of the convolutional neural network model is related to the inability of the model to actively verify the rationality of the activation state of the feature map during the forward propagation process, that is, since the training of the image classification model is completed on a data set composed of normal images, the model only learns the activation mode of the normal image and cannot effectively suppress abnormal activation modes.

[0052] Therefore, in this embodiment, the activation ratio of the normal image and the adversarial sample in the image classification model is used as a training sample to train a fully connected neural network model representing the abnormal probability of the input image, i.e., a detection model. After the feature map output by the preset convolution layer in the image classification model is obtained in step S2, the activation ratio of the feature map is determined, and then the activation ratio is input into the detection model; the detection model determines whether the classification result of the image classification model for the processed image is abnormal according to the output of the detection model.

[0053] Optionally, in the step S2, the activation ratio refers to a proportion of elements greater than the preset threshold in the feature map M, and the calculation formula is as follows:

[0054]

[0055] wherein M represents a feature map output by a convolution layer, the feature map can be a matrix, h and w respectively represent height and width of the feature map, φ represents a preset threshold, e represents a matrix element in the feature map, a matrix element greater than the preset threshold in the feature map is regarded as activation, and R represents the calculated activation ratio of the feature map.

[0056] In the embodiment, the value of the activation ratio R reflects the significance of the corresponding feature, although the number of activated elements can also reflect the feature extraction state of a certain layer, but the size of the output feature map in the convolutional neural network model decreases from front to back, and the use of the activation ratio compared with the activation number can better meet the needs of combining the feature extraction states of two convolutional layers to extract the activation mode. In addition, even if the target feature appears in any area of the input feature map, the activation ratio can still effectively depict the feature extraction state.

[0057] Further, in the step S2, the step of obtaining the feature map output by the preset convolution layer in the image classification model and determining the activation ratio of the feature map comprises:

[0058] S201, after selecting two convolutional layers in the image classification model, obtaining a first feature map and a second feature map output by the two convolutional layers, respectively;

[0059] S202, calculating the activation ratio of the first feature map and the second feature map, respectively.

[0060] Specifically, in the step S201, the two convolutional layers selected from the image classification model can be adjacent or spaced apart by a certain distance. It should be noted that different convolutional layer selection methods will result in different activation ratios, although the overall strategy is to select two adjacent convolutional layers, but in actual application, it can be flexibly determined according to experience or trial, and the present application does not limit this.

[0061] Further, after obtaining the first feature map and the second feature map output by the two preset convolutional layers in the image classification model, the activation ratio of each channel in the first feature map is calculated to obtain a first activation ratio, and the activation ratio of each channel in the second feature map is calculated to obtain a second activation ratio. For example, if the first feature map and the second feature map are both 64 channels, 64 first activation ratios and 64 second activation ratios will be calculated, and the input of the detection model has a total of 128 activation ratios.

[0062] Figure 5is a schematic diagram of the method for detecting adversarial images based on activation difference of convolutional layers provided by an embodiment of the present application. Optionally, as shown in Figure 5 In the method for detecting adversarial images based on activation difference of convolutional layers, the detection model is trained by the following steps:

[0063] S301, obtaining first type training samples, the first type training samples comprising: a plurality of normal images, normal activation ratios corresponding to the normal images, a plurality of adversarial samples corresponding to each normal image, and abnormal activation ratios corresponding to the adversarial samples;

[0064] S302, inputting a preset number of first type training samples to a to-be-trained fully connected neural network model, and determining a loss value according to an output result of the to-be-trained fully connected neural network model and a preset loss function;

[0065] S303, determining whether the to-be-trained fully connected neural network model converges according to the loss value; if the to-be-trained fully connected neural network model converges, the to-be-trained fully connected neural network model is a trained detection model;

[0066] S304, if the to-be-trained fully connected neural network model does not converge, adjusting network parameters of the to-be-trained fully connected neural network model, and returning to the step of inputting the preset number of first type training samples to the to-be-trained fully connected neural network model.

[0067] Optionally, please continue to refer to Figure 5 The first type training samples are obtained by the following steps:

[0068] S401, obtaining first training samples; the first training samples comprising a plurality of normal images;

[0069] S402, after inputting the first training samples to an image classification model, obtaining feature maps output by all convolutional layers in the image classification model, and determining normal activation ratios of the feature maps;

[0070] S403, generating a plurality of adversarial samples corresponding to each normal image, to obtain second training samples;

[0071] S404, after inputting the second training samples to the image classification model, obtaining feature maps output by all convolutional layers in the image classification model, and determining abnormal activation ratios of the feature maps;

[0072] S405, generating first type training samples, the first type training samples comprising: normal activation ratios of the normal images and abnormal activation ratios of the plurality of adversarial samples corresponding to each normal image.

[0073] In this embodiment, firstly, for the image classification model to be protected, a certain number of images that can be correctly classified are collected, referred to as normal images, and these normal images are used as first training samples. Since the adversarial samples generated by different attack methods may have different activation characteristics, in order to detect multiple attacks, adversarial samples generated under multiple attacks need to be prepared as training samples, so that the detection model learns a more accurate decision boundary. In this embodiment, existing adversarial attack algorithms can be used to generate multiple adversarial samples corresponding to each normal image as second training samples.

[0074] Next, the first training samples are input into the image classification model, and the feature maps output after all convolution layer activations in the image classification model are copied during the forward propagation process. The normal activation proportion of the feature maps is calculated and saved to a file. Similarly, the second training samples are input into the image classification model again, and the feature maps output after all convolution layer activations in the image classification model are copied. The abnormal activation proportion of the feature maps is calculated and saved to a file. Obviously, the normal activation proportion of each normal image and the abnormal activation proportion of the multiple adversarial samples corresponding to each normal image can constitute the first type of training samples for training the detection model.

[0075] Optionally, the detection model includes an input layer, three hidden layers and an output layer, and the specific parameters are shown in the following table:

[0076] Table 1

[0077]

[0078] It should be understood that the present application combines the detection model with high recognition accuracy and the image classification model together to provide online services. For the image to be processed uploaded by the user, on the one hand, the normal forward propagation is performed to obtain the classification result, and on the other hand, the monitoring points are set in the preset convolution layer of the image classification model to obtain the feature maps and the activation proportion, so as to evaluate by the detection model whether the classification result of the image classification model this time has an abnormal risk. If there is an abnormal risk, the classification result this time is discarded, otherwise the classification result is trusted and the subsequent task processing is normally performed.

[0079] Next, the above-mentioned adversarial image detection method based on convolution layer activation difference is further illustrated through a simulation experiment.

[0080] Specifically, the second training sample used in the simulation experiment, i.e., the adversarial sample, is generated by the torchattacks library according to certain parameters. The hardware environment of the experiment is 1 RTX 2080Ti graphics card and 32GB memory. The images used to test the attack effect are selected from the classic ILSVRC competition image classification test set, which contains 1000 classes. After removing single-channel grayscale images and misclassified images, 5000 images covering all classes are randomly selected as the data set for this simulation. Optionally, the image preprocessing first normalizes the image size to 224 pixels square, then normalizes the pixels to [0, 1], and finally normalizes the RGB three channels according to the mean and variance of (0.485, 0.229), (0.456, 0.224) and (0.406, 0.225), respectively. In addition, the versions of the image classification models VGG-19 and ResNet-50 used as protection objects are torchvision 0.9.

[0081] FGSM attack, PGD-l2 and PGD-linf attack with Euler distance and infinite norm limit perturbation amplitude, and C&W attack with Euler distance limit are selected for the two image classification models VGG-19 and ResNet-50 to generate non-directional adversarial samples. This simulation sets a more difficult detection scenario, for example, the hyperparameter eps that controls the maximum modification amplitude is set to be small, which means that the adversarial sample has less perturbation than the original image. In the iterative attack projection descent method PGD, the option random_start allows adding initial random noise to the original image before the attack starts. Although the attack strength will be improved, the noise addition will be excessive, which is not conducive to the detection model to learn more complex activation patterns, so in this experiment it is set to False.

[0082] Randomly select some adversarial images generated according to Table 2, and it can be seen that there is no particularly prominent perturbation in the adversarial images. Compared with the parameter setting with larger perturbation amplitude, it is reasonable to believe that such experimental setting makes the difference in the activation of the generated adversarial sample on the feature map smaller, increasing the detection difficulty.

[0083] Table 2

[0084]

[0085] This embodiment will compare the effect of adversarial image detection with existing SafetyNet and CDRP. SafetyNet detects adversarial images at the last fully connected layer of the model, encodes the output of each neuron in the fully connected layer as a feature after a certain threshold, and trains an SVM detector. Selecting n-1 thresholds means encoding as an n-ary code. When selecting thresholds, first collect all output values of the last fully connected layer of the image classification model in the training set, sort them and divide them into 4 groups using 3 quartile points as thresholds. Finally, encode each output value as the median of the group it belongs to, and train a Gaussian kernel SVM detector. When detecting, use the thresholds and corresponding code words determined by the training set.

[0086] CDRP detects adversarial samples after the model completes the calculation of the picture category. According to the probability distribution of each category output by the model, it tries to keep the judgment category unchanged after suppressing as many convolutional layer outputs as possible, thereby obtaining the key path of the image to be processed in the identification process of the image classification model. Optionally, a random forest is used as a detector.

[0087] The hyperparameters of the detection model, the division of the training set and the test set, and the training time are shown in Table 3. The corresponding detection model hyperparameters of VGG and ResNet models are different. In the experiment, it was found that it was more difficult for the detection model of ResNet50 to achieve high precision on the training set, so 16 image feature activation states were used for each update, while VGG used 32 images. Since the detection model is a fully connected neural network containing three hidden layers, the training speed is very fast. It only takes about 72.71 seconds to train the detection model of ResNet for 100 rounds.

[0088] Table 3

[0089]

[0090] Table 4

[0091]

[0092] The accuracy of the above-mentioned adversarial image detection method based on convolution layer activation difference is shown in Table 4. For FGSM and PGD-linf attacks using infinite norm as the perturbation limit, the adversarial image detection method provided by the present application has a detection accuracy of more than 98%, which shows that the detection model has well extracted the difference between the activation mode of the adversarial sample and the normal image. In the horizontal comparison in Table 4, the detection effect of the adversarial image detection method provided by the present application on the VGG model is generally better than that of the ResNet, which may be related to the fact that the ResNet model has more network layers and the two selected convolution layers are far apart. From the vertical comparison, it can be seen that the detection success rate of the adversarial attack using infinite norm as the limit is higher than that of the Euler norm, which shows that the infinite norm is easy to bring more obvious activation difference.

[0093] For the problem that the above-mentioned adversarial image detection method based on convolution layer activation difference is difficult to detect PGD-l2 and C&W adversarial samples, the inventors found through attempts that combining JPEG compression with the present method can greatly improve the detection effect, i.e., the method marked as “JPEG+ACTPD” in Table 4. Specifically, during training, the Pillow image processing library is used to call JPEG with a quality parameter of 75 to compress the normal image and the adversarial sample, and then the feature map activation ratio is extracted to train the detection model. During detection, the image to be processed is first compressed by JPEG in the same way as during training, and then the adversarial image detection method provided by the present application is used for detection. It can be seen that for the VGG-19 model, the detection accuracy of PGD-l2 and C&W adversarial images is improved by about 19 and 11 percentage points, respectively; for the ResNet-50 model, the detection accuracy of PGD-l2 and C&W adversarial images is improved by about 43 and 26 percentage points, respectively.

[0094] Table 5 and Table 6 are the detection accuracy and time delay comparison results of the adversarial image detection method based on convolution layer activation difference provided by the present application and SafetyNet and CDRP. Obviously, the detection accuracy of the above-mentioned adversarial image detection method based on convolution layer activation difference is better than that of the comparative scheme for FGSM on VGG and ResNet models; the detection success rate for PGD-linf is basically the same as that of the comparative scheme on VGG-19, and is better than that of the comparative scheme on ResNet-50; the detection for PGD-l2 is weaker than that of the comparative scheme, but the scheme using “JPEG+ACTPD” can greatly narrow the gap; the detection for C&W is better than that of SafetyNet, and is weaker than that of CDRP.

[0095] Table 5

[0096]

[0097] Table 6

[0098]

[0099] In the calculation of the delay of the comparative scheme, since SafetyNet is detected by SVM, and the SVM trained from different attack data sets contains different numbers of support vectors, the detection delay of different attacks is not the same. The present application takes the average of the delays under different attacks as the final delay. CDRP also finds the critical path according to each picture individually, but experiments show that the processing time of CDRP for different attack images under the same model is basically the same. The present application calculates the feature map activation ratio under different attacks in the same step, so the delay is basically the same. As can be seen from Table 6, although CDRP has a relatively balanced detection accuracy for different attacks, its processing time for each image is close to 2 seconds, which cannot meet the high concurrency requirements in actual applications.

[0100] Figure 6 is a structural schematic diagram of an adversarial image detection device based on convolution layer activation difference provided by an embodiment of the present application. As shown in Figure 6 , the present application provides an adversarial image detection device based on convolution layer activation difference, comprising:

[0101] A first acquisition module 610 is configured to acquire a to-be-processed image and input the to-be-processed image into an image classification model; the image classification model is a pre-trained convolutional neural network model, comprising a convolutional layer, a pooling layer and a fully connected layer;

[0102] A second acquisition module 620 is configured to acquire a feature map output by a preset convolutional layer in the image classification model and determine an activation ratio of the feature map;

[0103] An input module 630 is configured to input the activation ratio into a detection model, and determine whether the classification result of the to-be-processed image by the image classification model is abnormal according to the output of the detection model; wherein the detection model is a fully connected neural network model representing the abnormal probability of an input image, which is obtained by training the activation ratio of normal images and adversarial samples in the image classification model as training samples in advance.

[0104] From the above embodiments, the present application has the following beneficial effects:

[0105] In view of the threat of the adversarial image to the image classification model, the application provides a kind of adversarial image detection method and device based on convolution layer activation difference, by calculating the feature map activation proportion in the normal forward propagation process of image classification model, so that the detection model judges whether the classification result of image classification model to the image to be processed is abnormal according to the activation proportion, to carry out security reinforcement to the image classification model that has been online;In addition, since the calculation of feature map activation proportion ends in the normal forward propagation process, it will hardly cause time delay to online application, solve the problem of high time delay of existing detection method.

[0106] In the description of the application, the terms "first", "second" are only for the purpose of description, and cannot be understood as indicating or implying relative importance or implicitly indicating the number of indicated technical features. Therefore, the features limited by "first", "second" can be explicitly or implicitly included one or more of the features. In the description of the application, the meaning of "a plurality of" is two or more, unless otherwise specifically limited.

[0107] In the description of the present application, the description of the terms "one embodiment", "some embodiments", "example", "specific example" or "some examples" means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present application. In the present application, the illustrative description of the above terms is not necessarily for the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art can combine and combine different embodiments or examples described in the present application.

[0108] Although the present application is described herein in conjunction with various embodiments, those skilled in the art can understand and implement other variations of the disclosed embodiments by viewing the drawings, disclosure and attached claims during implementation of the claimed application. In the claims, the word "comprising" does not exclude other components or steps, and "one" or "an" does not exclude multiple cases. A single processor or other unit can implement several functions listed in the claims. Some measures are recorded in mutually different dependent claims, but this does not mean that these measures cannot be combined to produce good results.

[0109] The above is further detailed description of the present application in combination with specific preferred embodiments, and cannot be deemed as limitation of the specific implementation of the present application to these descriptions. For ordinary skilled in the art to which the present application belongs, several simple deductions or substitutions can be made without departing from the concept of the present application, and all of them shall be deemed as falling within the protection scope of the present application.

Claims

1. An adversarial image detection method based on convolutional layer activation difference, characterized in that, include: The image to be processed is acquired and input into the image classification model; The image classification model is a pre-trained convolutional neural network model, including: convolutional layers, pooling layers, and fully connected layers; Obtain the feature map output by the preset convolutional layer in the image classification model, and determine the activation ratio of the feature map; After inputting the activation ratio into the detection model, the image classification model is used to determine whether there is an anomaly in the classification result of the image to be processed based on the output of the detection model; wherein, the detection model is: a fully connected neural network model that represents the anomaly probability of the input image after being trained in advance with the activation ratio of normal images and adversarial samples in the image classification model as training samples; The steps of obtaining the feature map output by the preset convolutional layer in the image classification model and determining the activation ratio of the feature map include: selecting two convolutional layers in the image classification model and obtaining the first feature map and the second feature map output by the two convolutional layers respectively; calculating the activation ratio of the first feature map and the second feature map respectively; the activation ratio is: the proportion of elements in the feature map that are greater than a preset threshold; The step of calculating the activation ratios of the first feature map and the second feature map respectively includes: calculating the activation ratio of each channel in the first feature map to obtain a first activation ratio; and calculating the activation ratio of each channel in the second feature map to obtain a second activation ratio.

2. The adversarial image detection method based on convolutional layer activation difference according to claim 1, characterized in that, The activation ratio of the feature map is determined according to the following formula: in, This represents the feature map output by the convolutional layer. , These represent the height and width of the feature map, respectively. Indicates the preset threshold. This represents a matrix element in the feature map, where matrix elements in the feature map that are greater than the preset threshold are considered active. This represents the activation ratio of the calculated feature map.

3. The adversarial image detection method based on convolutional layer activation difference according to claim 1, characterized in that, The detection model was trained using the following steps: Obtain a first type of training samples, which includes: multiple normal images, normal activation ratios corresponding to the normal images, multiple adversarial samples corresponding to each normal image, and abnormal activation ratios corresponding to the adversarial samples. A predetermined number of first-class training samples are input into the fully connected neural network model to be trained, and the loss value is determined based on the output of the fully connected neural network model to be trained and the predetermined loss function. The loss value is used to determine whether the fully connected neural network model to be trained has converged; if it has converged, the fully connected neural network model to be trained is a detection model that has been trained. If convergence is not achieved, adjust the network parameters of the fully connected neural network model to be trained, and return to the step of inputting a preset number of first-class training samples into the fully connected neural network model to be trained.

4. The adversarial image detection method based on convolutional layer activation difference according to claim 3, characterized in that, The first type of training samples were obtained using the following steps: Obtain a first training sample; the first training sample includes multiple normal images; After inputting the first training sample into the image classification model, the feature maps output by all convolutional layers in the image classification model are obtained, and the normal activation ratio of the feature map is determined. Multiple adversarial samples corresponding to each of the normal images are generated to obtain the second training samples; After inputting the second training sample into the image classification model, the feature map output by all convolutional layers in the image classification model is obtained, and the abnormal activation ratio of the feature map is determined. Generate a first type of training samples, which includes: the normal activation ratio of each normal image and the abnormal activation ratio of multiple adversarial samples corresponding to each normal image.

5. An adversarial image detection device based on convolutional layer activation difference, characterized in that, include: The first acquisition module is used to acquire the image to be processed and input the image to be processed into the image classification model; The image classification model is a pre-trained convolutional neural network model, including: convolutional layers, pooling layers, and fully connected layers; The second acquisition module is used to acquire the feature map output by the preset convolutional layer in the image classification model and determine the activation ratio of the feature map. The input module is used to input the activation ratio into the detection model and then determine whether the classification result of the image classification model for the image to be processed is abnormal based on the output of the detection model; wherein, the detection model is a fully connected neural network model that represents the probability of anomalies in the input image after being trained in advance with the activation ratios of normal images and adversarial samples in the image classification model as training samples. The second acquisition module is specifically used to select two convolutional layers in the image classification model, and then obtain the first feature map and the second feature map output by the two convolutional layers respectively; calculate the activation ratio of the first feature map and the second feature map respectively; the activation ratio is: the proportion of elements in the feature map that are greater than a preset threshold; The activation ratios of the first feature map and the second feature map include: a first activation ratio obtained based on the activation ratio of each channel in the first feature map, and a second activation ratio obtained based on the activation ratio of each channel in the second feature map.

Images