Adversarial sample detection method and device based on divide-and-conquer strategy

Abstract

The application provides a kind of based on the detection method and equipment of adversarial sample of divide-and-conquer strategy.The method comprises the following steps: step 1: test sample input target model, obtain feature map;Step 2: the feature map is input into feature map detector FMD, and feature map detector FMD detects whether the sample is an adversarial sample;Step 3: the feature map is input into adversarial noise eraser ANE for detection;Step 4: the detection results of feature map detector FMD and adversarial noise eraser ANE are integrated, if one detector detects that the test sample is an adversarial sample, it is determined that the test sample is an adversarial sample.The application can accurately detect all unknown attack method generated adversarial samples by training on the adversarial samples generated by fast gradient sign method, achieve an average AUC value of 97%, and the proposed detection algorithm can be applied to the real world.

Description

TECHNICAL FIELD

[0001] Embodiments of the present application relate to the technical field of information security, and particularly relate to a method and device for detecting adversarial samples based on a divide-and-conquer strategy. BACKGROUND

[0002] With some breakthroughs in deep learning, deep neural networks (DNNs) have been applied in various fields such as object recognition, autonomous driving, machine translation, and speech recognition. However, in recent years, researchers have found that DNNs are also subject to various forms of security threats, especially adversarial sample attacks. Adversarial attacks add some noise that does not affect human observation to normal samples, causing the model to make classification errors. These samples that fool DNNs are called adversarial samples. The existence of adversarial samples poses a serious threat to the application of neural networks, especially in some security-sensitive applications.

[0003] There are many methods for detecting adversarial samples at present, but these methods cannot detect a wide variety of adversarial samples and cannot effectively detect unknown attacks. Therefore, developing a method and device for detecting adversarial samples based on a divide-and-conquer strategy can effectively overcome the defects in the related art, and has become a technical problem to be solved in the industry. SUMMARY

[0004] To solve the above problems in the prior art, embodiments of the present application provide a method and device for detecting adversarial samples based on a divide-and-conquer strategy.

[0005] In a first aspect, embodiments of the present application provide a method for detecting adversarial samples based on a divide-and-conquer strategy, comprising: step 1, inputting a test sample into a target model to obtain a feature map; step 2, inputting the feature map into a feature map detector FMD to detect whether the sample is an adversarial sample; step 3, inputting the feature map into an adversarial noise eraser ANE for detection; and step 4, integrating the detection results of the feature map detector FMD and the adversarial noise eraser ANE, and if one of the detectors detects that the test sample is an adversarial sample, determining that the test sample is an adversarial sample.

[0006] On the basis of the method embodiment content, the method for detecting adversarial samples based on the divide-and-conquer strategy provided in the embodiment of the application specifically comprises the following steps: step 2.1: feature selection, for each layer, the feature map detector FMD randomly selects half of the feature maps for monitoring; step 2.2: feature extraction, one feature map contains a plurality of statistical features, and the feature map detector FMD selects the mean value of the plurality of statistical features as the statistical feature; step 2.3: training of a random forest, for each layer, the feature map detector FMD trains a random forest classifier using the extracted statistical features, and the random forest classifier gives a prediction value for input; and step 2.4: training of an integrated model, the feature map detector FMD trains a multi-layer perception model to take the output of the random forest as input and give a final decision result.

[0007] On the basis of the method embodiment content, the method for detecting adversarial samples based on the divide-and-conquer strategy provided in the embodiment of the application specifically comprises the following steps: step 3.1: the adversarial sample is destroyed by using image transformation; step 3.2: the destroyed adversarial sample is input into the target model to obtain a label thereof; and step 3.3: the label change rate LCR is calculated, and if the label change rate LCR is greater than a preset threshold, the test sample is considered to be an adversarial sample.

[0008] On the basis of the method embodiment content, the method for detecting adversarial samples based on the divide-and-conquer strategy provided in the embodiment of the application specifically comprises the following steps: step 3.1: the adversarial sample is destroyed by using image transformation; step 3.2: the destroyed adversarial sample is input into the target model to obtain a label thereof; and step 3.3: the label change rate LCR is calculated, and if the label change rate LCR is greater than a preset threshold, the test sample is considered to be an adversarial sample.

[0009] On the basis of the method embodiment content, the method for detecting adversarial samples based on the divide-and-conquer strategy provided in the embodiment of the application specifically comprises the following steps: step 3.1: the adversarial sample is destroyed by using image transformation; step 3.2: the destroyed adversarial sample is input into the target model to obtain a label thereof; and step 3.3: the label change rate LCR is calculated, and if the label change rate LCR is greater than a preset threshold, the test sample is considered to be an adversarial sample. i |i=1,...,n} represents image transformation, wherein t i is the ith image transformation, and the LCR is represented as:

[0010]

[0011] wherein |T| represents the number of elements of T, and is used to measure the sensitivity of the test sample x to image transformation, and the greater the LCR value, the higher the possibility that the test sample is an adversarial sample.

[0012] In a second aspect, embodiments of the present application provide an adversarial sample detection device based on a divide-and-conquer strategy, comprising: a first main module for implementing step 1: inputting a test sample into a target model to obtain a feature map; a second main module for implementing step 2: inputting the feature map into a feature map detector FMD, and the feature map detector FMD detects whether the sample is an adversarial sample; a third main module for implementing step 3: inputting the feature map into an adversarial noise eraser ANE for detection; and a fourth main module for implementing step 4: integrating the detection results of the feature map detector FMD and the adversarial noise eraser ANE, and if one of the detectors detects that the test sample is an adversarial sample, it is determined that the test sample is an adversarial sample.

[0013] In a third aspect, embodiments of the present application provide an electronic device, comprising:

[0014] at least one processor; and

[0015] at least one memory in communication with the processor, wherein:

[0016] The memory stores program instructions executable by the processor, and the processor invoking the program instructions can execute the adversarial sample detection method based on the divide-and-conquer strategy provided in any one of the various implementation manners of the first aspect.

[0017] In a fourth aspect, embodiments of the present application provide a non-transitory computer readable storage medium, which stores computer instructions, and the computer instructions cause a computer to execute the adversarial sample detection method based on the divide-and-conquer strategy provided in any one of the various implementation manners of the first aspect.

[0018] The adversarial sample detection method and device based on the divide-and-conquer strategy provided by the embodiments of the present application can accurately detect all unknown attack method generated adversarial samples by training on the adversarial samples generated by the fast gradient sign method, and achieve an average AUC value of 97%. The detection algorithm proposed can be applied to the real world. BRIEF DESCRIPTION OF DRAWINGS

[0019] In order to more clearly illustrate the technical solutions of the embodiments of the present application or the prior art, the following will briefly introduce the drawings needed to be used in the embodiments or prior art description. Obviously, the drawings described below are some embodiments of the present application, and those skilled in the art can obtain other drawings according to these drawings without creative labor.

[0020] Figure 1 The flowchart of the adversarial sample detection method based on the divide-and-conquer strategy provided by the embodiments of the present application;

[0021] Figure 2A structure schematic diagram of the adversarial sample detection device based on the divide-and-conquer strategy provided by the embodiment of the present application is provided.

[0022] Figure 3 A structure schematic diagram of the electronic device provided by the embodiment of the present application is provided.

[0023] Figure 4 A framework principle schematic diagram of the adversarial sample detection method based on the divide-and-conquer strategy provided by the embodiment of the present application is provided. DETAILED DESCRIPTION

[0024] In order to make the purpose, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are some of the embodiments of the present application, but not all the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the present application. In addition, the technical features in each embodiment or single embodiment provided by the present application can be combined with each other arbitrarily to form a feasible technical solution, and this combination is not restricted by the order of steps and / or structure mode, but it must be based on the implementation by those skilled in the art. When the combination of technical solutions appears contradictory or unfeasible, it should be considered that the combination of technical solutions does not exist, and is not within the scope of protection required by the present application.

[0025] In order to solve the problem of detecting adversarial samples, it is assumed that different types of adversarial samples have different characteristics, and the adversarial samples are divided into fragile adversarial samples and robust adversarial samples. Then, the divide-and-conquer strategy is adopted, and two different methods are proposed to detect the two types of adversarial samples, and finally the best effect is achieved by integrating the two methods. Based on this idea, the embodiment of the present application provides an adversarial sample detection method based on the divide-and-conquer strategy, which is described with reference to Figure 1 The method comprises the following steps: step 1: inputting a test sample into a target model (in another embodiment, the target model can be a deep neural network model DNN) to obtain a feature map; step 2: inputting the feature map into a feature map detector FMD, and the feature map detector FMD detects whether the sample is an adversarial sample; step 3: inputting the feature map into an adversarial noise eraser ANE for detection; step 4: integrating the detection results of the feature map detector FMD and the adversarial noise eraser ANE, and if one of the detectors detects that the test sample is an adversarial sample, it is determined that the test sample is an adversarial sample.

[0026] Specifically, the fragile adversarial sample only makes slight modifications to the normal sample, and thus is difficult to detect from the image perturbation level and the feature map perturbation level. However, due to the less modification and insufficient robustness, the fragile adversarial sample is easily destroyed by some image transformations. Inspired by this, the adversarial noise eraser ANE is proposed to detect the fragile adversarial sample.

[0027] The robust adversarial sample usually introduces adversarial noise to the normal sample to make it robust, thereby resisting image transformations. These perturbations change the values of the activated neurons of the next layer, and the perturbations are amplified with the increase of the layer number, and finally change the prediction value. The feature map is composed of neurons, and the change of the neuron activation value naturally causes the change of the statistical characteristics of the feature map. Therefore, the feature map detector FMD is proposed to detect the robust adversarial sample.

[0028] The two detectors are integrated, and finally all types of adversarial samples are detected. The flowchart of the whole framework is shown in Figure 4 The general idea is that a test sample is input into the model, the feature map detector FMD detects whether the sample is an adversarial sample according to the feature map, and the ANE determines whether the sample is an adversarial sample by comparing the prediction difference of the model before and after the image is transformed. As long as one detector detects that the sample is an adversarial sample, the integrated method considers that the sample is an adversarial sample.

[0029] Based on the content of the above method embodiment, as an optional embodiment, the adversarial sample detection method based on the divide-and-conquer strategy provided in the embodiment of the application, step 2 specifically comprises: step 2.1: feature selection, for each layer, the feature map detector FMD randomly selects half of the feature maps for monitoring; step 2.2: feature extraction, a feature map contains multiple statistical characteristics, and the feature map detector FMD selects the mean value of the multiple statistical characteristics as the statistical characteristic; step 2.3: training a random forest, for each layer, the feature map detector FMD trains a random forest classifier with the extracted statistical characteristics, and the random forest classifier gives a prediction value for the input; step 2.4: training an integrated model, the feature map detector FMD trains a multilayer perception model to take the output of the random forest as the input and gives the final decision result.

[0030] Based on the content of the above method embodiment, as an optional embodiment, the adversarial sample detection method based on the divide-and-conquer strategy provided in the embodiment of the application, step 3 specifically comprises: step 3.1: destroying the adversarial sample by using image transformation; step 3.2: inputting the destroyed adversarial sample into the target model to obtain its label; and step 3.3: calculating the label change rate LCR, and if the label change rate LCR is greater than a preset threshold, the test sample is considered to be an adversarial sample.

[0031] Based on the content of the above method embodiment, as an optional embodiment, the adversarial sample detection method based on the divide-and-conquer strategy provided in the embodiment of the application adopts image transformation to destroy the adversarial sample, and the image transformation includes: adopting JPEG compression, rotation, bit depth reduction and / or mean blur to destroy the adversarial sample.

[0032] Based on the content of the above method embodiment, as an optional embodiment, the adversarial sample detection method based on the divide-and-conquer strategy provided in the embodiment of the application adopts image transformation to destroy the adversarial sample, and the image transformation includes: adopting JPEG compression, rotation, bit depth reduction and / or mean blur to destroy the adversarial sample. i |i=1,...,n} represents image transformation, wherein t i is the ith image transformation, and the LCR is represented as:

[0033]

[0034] wherein |T| represents the number of elements of T, and is used to measure the sensitivity of the test sample x to the image transformation, and the greater the LCR value, the higher the possibility that the test sample is an adversarial sample.

[0035] The AUC (Area under Curve) value is used as an evaluation index, which represents that a pair of samples (one positive sample and one negative sample) are randomly extracted, and the classifier trained is used to predict the two samples, and the probability of the positive sample is greater than that of the negative sample. The AUC value directly evaluates the good and bad of the classifier, and the greater the value, the better the effect of the classifier. The integrated method can obtain AUC values of 99.30% and 99.62% on the CIFAR-10 and ImageNet data sets respectively when detecting adversarial samples of 7 different attack intensities.

[0036] The adversarial sample detection method based on the divide-and-conquer strategy provided in the embodiment of the application can accurately detect all unknown adversarial samples generated by other attack methods by training on the adversarial samples generated by the fast gradient sign method, and can achieve an average AUC value of 97%. The detection algorithm can be applied to the real world.

[0037] The implementation basis of each embodiment of the application is that the processing is realized by the device with the processor function. Therefore, in engineering practice, the technical solutions and functions of each embodiment of the application can be packaged into various modules. Based on this actual situation, on the basis of the above embodiments, the embodiment of the application provides an adversarial sample detection device based on the divide-and-conquer strategy, which is used to execute the adversarial sample detection method based on the divide-and-conquer strategy in the above method embodiment. Referring to Figure 2The device comprises: a first main module for implementing step 1: inputting a test sample into a target model to obtain a feature map; a second main module for implementing step 2: inputting the feature map into a feature map detector FMD, and the feature map detector FMD detects whether the sample is an adversarial sample; a third main module for implementing step 3: inputting the feature map into an adversarial noise eraser ANE for detection; and a fourth main module for implementing step 4: integrating the detection results of the feature map detector FMD and the adversarial noise eraser ANE, and if one of the detectors detects that the test sample is an adversarial sample, it is determined that the test sample is an adversarial sample.

[0038] The device for detecting adversarial samples based on the divide-and-conquer strategy provided in the embodiments of the present application adopts Figure 2 several modules in the device, and can accurately detect adversarial samples generated by all unknown attack methods by training on the adversarial samples generated by the fast gradient sign method, so that an average AUC value of 97% is achieved, and the detection algorithm can be applied to the real world.

[0039] It should be noted that the device in the device embodiment provided in the present application can be used to implement the methods in the method embodiments provided in the present application, and can also be used to implement the methods in other method embodiments provided in the present application, the difference is only that corresponding function modules are set, and the principle is basically the same as that of the above-mentioned device embodiments provided in the present application, as long as the person skilled in the art can obtain the corresponding technical means by combining technical features on the basis of the above-mentioned device embodiments, and the technical solutions formed by these technical means, on the premise of ensuring the practicability of the technical solutions, the device in the above-mentioned device embodiment can be improved, so as to obtain the corresponding device class embodiment, and the method in the other method class embodiment is implemented. For example:

[0040] Based on the content of the above-mentioned device embodiment, as an optional embodiment, the device for detecting adversarial samples based on the divide-and-conquer strategy provided in the embodiments of the present application further comprises: a first sub-module for implementing step 2, which specifically comprises: step 2.1: feature selection, for each layer, the feature map detector FMD randomly selects half of the feature maps for monitoring; step 2.2: feature extraction, a feature map contains a plurality of statistical features, and the feature map detector FMD selects the mean value of the plurality of statistical features as the statistical feature; step 2.3: training a random forest, for each layer, the feature map detector FMD trains a random forest classifier with the extracted statistical features, and the random forest classifier gives a prediction value for the input; and step 2.4: training an integrated model, the feature map detector FMD trains a multi-layer perception model to take the output of the random forest as input and gives a final decision result.

[0041] As an optional embodiment based on the above-mentioned device embodiment, the device for detecting adversarial samples based on the divide-and-conquer strategy provided in the embodiment of the present application further comprises a second sub-module for implementing step 3, which specifically comprises: step 3.1: destroying the adversarial sample by image transformation; step 3.2: inputting the destroyed adversarial sample into the target model to obtain the label thereof; and step 3.3: calculating the label change rate LCR, and if the label change rate LCR is greater than a preset threshold, the test sample is considered to be an adversarial sample.

[0042] As an optional embodiment based on the above-mentioned device embodiment, the device for detecting adversarial samples based on the divide-and-conquer strategy provided in the embodiment of the present application further comprises a third sub-module for implementing the image transformation to destroy the adversarial sample, which comprises: destroying the adversarial sample by using JPEG compression, rotation, bit depth reduction and / or mean blur.

[0043] As an optional embodiment based on the above-mentioned device embodiment, the device for detecting adversarial samples based on the divide-and-conquer strategy provided in the embodiment of the present application further comprises a fourth sub-module for implementing the label change rate LCR, which comprises: given a test sample x and a DNN deep neural network model f, defining f(x) as the label output by the model, and using T={t i |i=1,...,n} to represent image transformation, wherein t i is the i-th image transformation, and the LCR is represented as:

[0044]

[0045] wherein |T| represents the number of elements of T, and is used to measure the sensitivity of the test sample x to image transformation, and the greater the LCR value, the higher the possibility that the test sample is an adversarial sample.

[0046] The method of the embodiment of the present application is implemented by relying on an electronic device, and therefore it is necessary to introduce the related electronic device. For this purpose, the embodiment of the present application provides an electronic device, as shown in Figure 3 The electronic device comprises at least one processor, a communications interface, at least one memory and a communications bus, wherein the at least one processor, the communications interface and the at least one memory complete mutual communication through the communications bus. The at least one processor can invoke the logical instructions in the at least one memory to execute all or part of the steps of the method provided by the above-mentioned various method embodiments.

[0047] In addition, the logic instructions in the at least one memory described above can be implemented in the form of a software function unit and sold or used as an independent product, and can be stored in a computer readable storage medium. Based on such understanding, the technical solutions of the present application essentially or the part that contributes to the prior art or part of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium, and includes several instructions to make a computer device (which can be a personal computer, a server, or a network device, etc.) execute all or part of the steps of the methods described in various method embodiments of the present application. The aforementioned storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), a magnetic disk or an optical disk, and various program code storage media.

[0048] The device embodiments described above are only schematic, wherein the units described as separate components can or can not be physically separated, and the components displayed as units can or can not be physical units, i.e., can be located in one place, or can be distributed on multiple network units. Part or all of the modules can be selected to achieve the purpose of the present embodiment according to actual needs. Those skilled in the art can understand and implement without creative labor.

[0049] Through the description of the above embodiments, those skilled in the art can clearly understand that each embodiment can be realized by means of software and the necessary general hardware platform, and of course can also be realized by hardware. Based on such understanding, the above technical solutions essentially or the part that contributes to the prior art can be embodied in the form of a software product. The computer software product can be stored in a computer readable storage medium, such as a ROM / RAM, a magnetic disk, an optical disk, etc., and includes several instructions to make a computer device (which can be a personal computer, a server, or a network device, etc.) execute the methods described in various embodiments or some parts of the embodiments.

[0050] The computer program product of the present application can be a computer program implemented on one or more various apparatuses, e.g., general purpose computers or dedicated computers. When implemented on a general purpose computer, the computer program can be stored in a tangible computer readable storage medium, e.g., ROM, floppy disk, etc., or transmitted from a website, server or other remote source using a transmission medium, e.g., a modem, or the Internet. Within the scope of the present application, programs can be implemented using any programming language, e.g., C, C++, Java, Pascal, etc. It will also be appreciated that future programming languages can be used.

[0051] It should be noted that the terms "comprising", "including", or any other similar term as used herein are intended to be taken in their broadest sense as meaning that they include the stated elements, but not excluding additional elements. In some embodiments, the terms "comprising" and "including" are used interchangeably. In some embodiments, the term "comprising" is used to mean one or more (i.e., at least one) of the listed elements can be present or can be used.

[0052] Finally, it should be noted that the above-mentioned embodiments are merely used to illustrate the technical solutions of the present application, rather than limiting the same. Although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that the technical solutions recorded in the foregoing embodiments can be modified or equivalent replacements can be made to some of the technical features, and these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A method for detecting adversarial samples based on a divide-and-conquer strategy, characterized in that, The method comprises the following steps: Step 1: inputting a test sample into a target model to obtain a feature map; Step 2: inputting the feature map into a feature map detector FMD, and the feature map detector FMD detecting whether the sample is an adversarial sample; Step 3: inputting the feature map into an adversarial noise eraser ANE for detection; Step 4: integrating the detection results of the feature map detector FMD and the adversarial noise eraser ANE, and determining that the test sample is an adversarial sample if one of the detectors detects that the test sample is an adversarial sample. Step 2 specifically comprises the following steps: Step 2.1: feature selection, for each layer, the feature map detector FMD randomly selects half of the feature maps for monitoring; Step 2.2: feature extraction, one feature map contains multiple statistical features, and the feature map detector FMD selects the mean value of the multiple statistical features as a statistical feature; Step 2.3: training a random forest, for each layer, the feature map detector FMD trains a random forest classifier using the extracted statistical features, and the random forest classifier gives a prediction value for the input; Step 2.4: training an integrated model, the feature map detector FMD trains a multi-layer perception model to take the output of the random forest as input and give a final decision result; Step 3 specifically comprises the following steps: Step 3.1: destroying the adversarial sample by image transformation; Step 3.2: inputting the destroyed adversarial sample into the target model to obtain a label; Step 3.3: calculating a label change rate LCR, and regarding the test sample as an adversarial sample if the label change rate LCR is greater than a preset threshold; The label change rate LCR comprises: Given a test sample x and a DNN deep neural network model f, define f(x) as the label output by the model, using represents an image transformation, where t i is the ith image transformation, and LCR is defined as: wherein, represents the number of T elements, which is used to measure the sensitivity of the test sample x to the image transformation, the larger the LCR value, the higher the possibility that the test sample is an adversarial sample.

2. The divide-and-conquer strategy based adversarial sample detection method of claim 1, wherein, The method for destroying the adversarial sample by image transformation comprises: destroying the adversarial sample by JPEG compression, rotation, bit depth reduction and / or mean blur. 3.A divide-and-conquer strategy based adversarial sample detection device using the divide-and-conquer strategy based adversarial sample detection method according to any one of claims 1 to 2, characterized in that, The method comprises the following steps: A first main module is configured to implement step 1: inputting a test sample into a target model to obtain a feature map; A second main module is configured to implement step 2: inputting the feature map into a feature map detector FMD, and the feature map detector FMD detecting whether the sample is an adversarial sample; A third main module is configured to implement step 3: inputting the feature map into an adversarial noise eraser ANE for detection; A fourth main module is configured to implement step 4: integrating the detection results of the feature map detector FMD and the adversarial noise eraser ANE, and determining that the test sample is an adversarial sample if one of the detectors detects that the test sample is an adversarial sample.

4. An electronic device, comprising: The device comprises at least one processor, at least one memory and a communication interface; wherein The processor, the memory and the communication interface communicate with each other; The memory stores program instructions executable by the processor, and the processor invokes the program instructions to execute the method of any one of claims 1 to 2. The non-transitory computer readable storage medium stores computer instructions, and the computer instructions cause the computer to execute the method of any one of claims 1 to 2.

5. A non-transitory computer-readable storage medium, comprising: ​

Images