Efficient black-box adversarial attacks exploiting input data structure

Abstract

Efficient black-box adversarial attack exploiting input data structure. Markov random field parameters are identified for covariance modeling of correlations between gradient terms of a classifier's loss function. A subset of images are sampled from an image dataset according to a normal distribution to estimate gradient terms. Black-box gradient estimates are used to infer values ​​of the Markov random field parameters from the sampling. Fourier basis vectors are generated from the inferred values. The original image is perturbed using the Fourier basis vectors to obtain loss function values. An estimate of the gradient is obtained from the loss function values. Image perturbations are created using the estimated gradients. The image perturbations are added to the original input to generate a candidate adversarial input that maximizes the loss in the classifier-identified image. A neural network classifier is queried to determine the classifier predictions for the candidate adversarial input.

Description

Technical Field

[0001] The present disclosure relates to attacking classifiers based on deep neural networks in an adversarial manner, and more particularly to efficient black-box adversarial attacks that exploit the structure of input data to the classifier. Background Art

[0002] Black-box adversarial attacks are an active area of ​​research. The following three references each describe a method for crafting black-box adversarial examples. One method using a natural evolution strategy is described in International Conference on Machine Learning ( arXiv:1804.08598 ) is found in Ilyas, A., Engstrom, L., Athalye,A., & Lin, J. (July 2018). The reference defines three realistic threat models that more accurately characterize many real-world classifiers: the query-restricted setting, the partial information setting, and the label-only setting. The reference develops new attacks that fool classifiers under these stricter threat models, where previous approaches would be impractical or ineffective. The reference shows that our approach is effective against the ImageNet classifier under our proposed threat model. The reference also demonstrates targeted black-box attacks against commercial classifiers, overcoming the challenges of limited query access, partial information, and other practical issues to break the Google Cloud Vision API.

[0003] A method for estimating the gradient using a gradient prior and then performing gradient descent is in Ilyas, A., Engstrom, L., & Madry, A. (2018). Prior convictions: Black-box adversarialattacks with bandits and priors ( arXiv preprint arXiv:1807.07978). This reference studies the problem of generating adversarial examples in a black-box setting, where only loss-oracle access to the model is available. This reference introduces a framework that conceptually unifies much of the existing work on black-box attacks, and shows that current state-of-the-art methods are optimal in a natural sense. Despite this optimality, this reference shows how black-box attacks can be improved by bringing a new element to the problem, namely the gradient prior. This reference gives a bandit optimization-based algorithm that allows seamless integration of any such prior, and explicitly identifies and combines two examples.

[0004] A method using submodule optimization can be found in Moon, S., An, G., & Song, HO (2019) Parsimonious Black-Box Adversarial Attacks via Efficient Combinatorial Optimization ( arXiv preprint arXiv:1905.06635 ). This reference proposes an efficient discrete alternative for the optimization problem that does not require estimating gradients and thus becomes unnecessary to tune first-order update hyperparameters. Experiments on Cifar-10 and ImageNet show black-box attack performance at significantly reduced query requirements compared to several recently proposed methods. Summary of the invention

[0005] According to one or more illustrative examples, a method for performing a single-step adversarial attack on a neural network classifier includes: identifying Markov random field parameters for modeling the covariance of correlations between gradient terms of a loss function of the classifier; sampling a subset of images from an image dataset according to a normal distribution to estimate the gradient terms; inferring values ​​of the Markov random field parameters using black-box gradient estimation based on the sampling; generating Fourier basis vectors from the inferred values; perturbing the original image using the Fourier basis vectors to obtain loss function values; obtaining an estimate of the gradient from the loss function values; creating an image perturbation using the estimated gradient; adding the image perturbation to the original input to generate a candidate adversarial input that maximizes the loss in the classifier-identified image; querying the neural network classifier to determine the classifier prediction for the candidate adversarial input; calculating a score for the classifier prediction; and accepting the candidate adversarial input as a successful adversarial attack in response to the classifier prediction being incorrect.

[0006] The method may further include adding the image perturbation to the original input using a fast gradient sign method. The method may further include, wherein the fast gradient sign method utilizes a pre-specified Perturbation bounds to add image perturbations to the original input.

[0007] The method could further include, wherein the Markov random field parameters include first parameters governing diagonal terms and second parameters governing neighboring pixels corresponding to adjacent indices in the image.

[0008] The method may further include, wherein the parameter values ​​are inferred using Newton's method. The method may further include, wherein the gradient is estimated using a predefined maximum number of queries for the classifier. The method may further include, wherein the image comprises video image data. The method may further include, wherein the image comprises still image data.

[0009] According to one or more illustrative examples, a computing system for performing a single-step adversarial attack on a neural network classifier includes a memory storing instructions of a black-box adversarial attack algorithm of a software program; and a processor programmed to execute the instructions to perform operations, the operations including: identifying Markov random field parameters for covariance modeling of correlations between gradient terms of a loss function of the classifier; sampling a subset of images from an image dataset according to a normal distribution to estimate the gradient terms; inferring values ​​of the Markov random field parameters using the black-box gradient estimates based on the sampling; generating Fourier basis vectors from the inferred values; perturbing the original image using the Fourier basis vectors to obtain loss function values; obtaining estimates of the gradients from the loss function values; creating image perturbations using the estimated gradients; adding the image perturbations to the original input to generate a candidate adversarial input that maximizes the loss in the image identified by the classifier; querying the neural network classifier to determine the classifier prediction for the candidate adversarial input; calculating a score for the classifier prediction; in response to the classifier prediction being incorrect, accepting the candidate adversarial input as a successful adversarial attack; and in response to the classifier prediction being correct, rejecting the candidate adversarial input.

[0010] The computing system can further be programmed to add the image perturbation to the original input using a fast gradient sign method. The computing system can further include, wherein the fast gradient sign method utilizes a pre-specified Perturbation bounds to add image perturbations to the original input.

[0011] The computing system may further include, wherein the Markov random field parameters include a first parameter governing diagonal terms and a second parameter governing neighboring pixels corresponding to neighboring indices in the image. The computing system may further include, wherein the parameter values ​​are inferred using Newton's method. The computing system may further include, wherein the gradient is estimated using a predefined maximum number of queries for the classifier.

[0012] According to one or more illustrative examples, a non-transitory computer-readable medium includes instructions for performing a single-step adversarial attack on a neural network classifier, which, when executed by a processor, causes the processor to: identify Markov random field parameters for covariance modeling of correlations between gradient terms of a loss function of the classifier; sample a subset of images from an image dataset according to a normal distribution to estimate the gradient terms; infer values ​​of the Markov random field parameters using black-box gradient estimation based on the sampling; generate Fourier basis vectors from the inferred values; perturb the original image using the Fourier basis vectors to obtain loss function values; obtain estimates of gradients from the loss function values; create image perturbations using the estimated gradients; add the image perturbations to the original input to generate a candidate adversarial input that maximizes the loss in the classifier-identified image; query the neural network classifier to determine the classifier's prediction for the candidate adversarial input; calculate a score for the classifier's prediction; in response to the classifier's prediction being incorrect, accept the candidate adversarial input as a successful adversarial attack; and in response to the classifier's prediction being correct, reject the candidate adversarial input.

[0013] The medium may further include instructions that, when executed by the processor, cause the processor to use a fast gradient sign method to add an image perturbation to the original input. The medium may further include, wherein the fast gradient sign method utilizes a pre-specified Perturbation bounds to add image perturbations to the original input.

[0014] The medium may further include, wherein the Markov random field parameters include first parameters governing diagonal terms and second parameters governing neighboring pixels corresponding to neighboring indices in the image. The medium may further include, wherein parameter values ​​are inferred using Newton's method. The medium may further include, wherein the gradient is estimated using a predefined maximum number of queries for the classifier. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Figure 1 is an example data flow graph for performing a single-step black-box adversarial attack;

[0016] Figure 2 is an example of a parameter of a two-dimensional image; and

[0017] Figure 3is a schematic diagram of a computing platform that can be exploited to implement a single-step black-box adversarial attack. DETAILED DESCRIPTION

[0018] Embodiments of the present disclosure are described herein. However, it should be understood that the disclosed embodiments are merely examples, and other embodiments may take various forms as well as alternative forms. The figures are not necessarily to scale; some features may be enlarged or minimized to show the details of a particular component. Therefore, the specific structural and functional details disclosed herein should not be interpreted as restrictive, but only as a representative basis for teaching those skilled in the art to adopt the embodiments in different ways. As will be understood by those of ordinary skill in the art, the various features illustrated and described with reference to any one of the figures may be combined with the features illustrated in one or more other figures to produce embodiments that are not explicitly illustrated or described. The combination of illustrated features provides representative embodiments for typical applications. However, various combinations and modifications of features consistent with the teachings of the present disclosure may be desirable for specific applications or implementations.

[0019] The present disclosure relates to attacking a classifier based on a deep neural network in an adversarial manner. The method takes an input image and adds perturbations to the image so that the perturbed image has minimal visual distortion to the human eye relative to the original image. However, the perturbed image causes the deep neural network-based classifier to misclassify the image. The concept of minimal distortion is achieved by requiring the perturbation to be within The pre-specified maximum value on the norm Limits have been formalized.

[0020] Adversarial attacks can be divided into two categories, namely white-box attacks, in which one assumes complete knowledge of the neural network and associated parameters, and black-box attacks, in which no such information is available. In addition, the classification of black-box attacks can be further subdivided into multi-step and single-step attacks. In multi-step attacks, perturbations are generated over multiple iterations, whereas in single-step attacks, perturbations are generated in only one iteration. The present disclosure is particularly directed to single-step black-box attacks, which differ significantly in implementation compared to multi-step attacks.

[0021] In a black-box attack setting, obtaining information about a model involves querying the model. Right now , provide input to the model and obtain predictions for the classes or probability distributions over the classes. In real-world attack scenarios, it is unrealistic to assume that the model can be queried as much as one wants. Therefore, in a real setting, the number of queries available is limited. Therefore, a fixed query budget for evaluating black-box adversarial attack scenarios can be ascertained. As described herein, the present disclosure is directed to maximizing the percentage of images that are misclassified for a given query budget using a single iteration of queries.

[0022] Techniques used in the present disclosure include parameter inverse covariance estimation using Gaussian Markov random fields and fast gradient signed methods. Relative to existing methods, the methods in the present disclosure are able to achieve higher attack accuracy while making perturbations in only one step as opposed to multiple steps in other solutions. The disclosed methods can thus be used to check for vulnerabilities in deployable deep learning models. As another application, the disclosed methods can be used to generate data for adversarial training of deep neural networks to improve the robustness of the models. Therefore, the computer system, computer-readable medium, and method aspects disclosed herein provide non-abstract technical improvements to known methods for identifying model weaknesses and addressing those weaknesses.

[0023] Figure 1 Figure 1 shows an example data flow graph for performing a single-step black-box adversarial attack. Figure 1 In many black-box attacks, the gradient terms (of the loss with respect to the input image) are implicitly assumed to be independent of each other. However, when visually inspecting adversarial examples, it becomes clear that this is not the case. The gradient terms can be largely correlated. The disclosed method exploits this correlation to construct query-efficient black-box adversarial attacks.

[0024] In form, let is the input to the deep classifier, and let is the gradient of the associated gradient with respect to the estimated loss function. Then, as shown in Equation (1), Put a prior distribution on top:

[0025]

[0026] in is some (non-identity) covariance that models the correlation between terms. The inverse covariance matrix can be modeled as follows:

[0027]

[0028] Also known as the Gaussian Markov random field setting, where The non-zero entries of correspond exactly to the edges in the graphical model that describes the distribution. In general, the vertices in a Gaussian Markov random field represent random variables, and the edges impose statistical constraints on those random variables.

[0029] Parameterized Gaussian Markov random fields can be used with relatively few free parameters. Figure 2 As an example of a parameter, if is a 2D image ( For example , 2 channels: black and white), then there is a parameter For all i that dominate the diagonal terms , and another parameter governs the neighboring pixels corresponding to the neighboring indices in the original image for i, j All parameters of this model can be referred to as , so in this case The result is Can be called For 3D images ( For example , with 3 channels: red, blue, and green), additional parameters can be used to capture the relationship of pixels in other channels. To infer the parameters, Newton's method can be used as shown in equation (2) to solve the following optimization problem:

[0030]

[0031] in is a gradient sample To sample the gradient, we use , where u is drawn from a normal distribution, and is the input image. In a non-limiting example, 50 images from a dataset and 50 draws from a normal distribution can be used to estimate the gradient to be used in (2). In Newton's method, at each step, the iteration can be updated as follows:

[0032]

[0033] in Iteration t of iterations, and It's Hessian.

[0034] To efficiently compute the objective in equation (2), techniques from circular convolution and fast Fourier transform are used. Under this framework, black-box gradient estimation is interpreted as a Gaussian inference problem with the prior imposed in equation (1), and then, using the fact that the value of the loss function at some point x' is a noisy observation of the gradient:

[0035] .

[0036] Therefore, given that basically there are m queries and their corresponding function values The sample point set ,distributed is instructed, among others:

[0037]

[0038] And the expected value of the posterior distribution is given by equation (3):

[0039] .

[0040] In particular, using the inverse covariance matrix The eigenvector of the perturbed image is The feature vectors of in this case are exactly the Fourier basis vectors. Note that the m queries extracted fit within the query budget. The query budget indicates the maximum permissible number of queries that can be used to attack the image.

[0041] To avoid the potentially tedious inversion of equation (3), the Woodbury matrix identity can be used to yield:

[0042] .

[0043] Using the expected value of the gradient in (3) as the gradient estimate, the fast gradient sign method can be used to generate adversarial images as follows:

[0044]

[0045] in is pre-specified perturbation bounds. The fast gradient sign method is implemented by using The estimated gradient of From the input image Creating adversarial examples (The sign function gives a value of 1 for positive numbers, -1 for negative numbers, and zero for zero). Using the input image, the method uses the estimated gradient of the loss with respect to the input image to create a new image that maximizes the loss. This new image can be called an adversarial image. As noted in the paper, the goal is to find an image As The disturbance of will be Error classification.

[0046] Assumptions is used for K Classifier for class classification problems. For soft label output, neural network The output is K dimensional vector, and the predicted class is The index of the maximum value of Given. quilt When correctly classified, let the predicted class In another example, the network outputs only the predicted class ( Right now , only the index of the largest class is output with soft labels). If Predicted class , then this method can be achieved by using the objective function to adapt to the hard label case and is 0 otherwise.

[0047] If the image Leading to deep network based classifiers Generate a target Misclassified Score ( For example , ), indicating a successful attack. Otherwise, if the image If it is not misclassified, the attack is unsuccessful.

[0048] Variations on the method are possible. In one example variation, a Laplace prior may be used in place of the Gaussian prior in equation (1). Furthermore, the above description is particularly applicable to the image domain, since x is specifically assumed to be an image. It should be noted that the disclosed method can be extended to video, where the data exhibits a structure similar to that of images.

[0049] like Figure 3 As shown in , a computing platform is used to implement the algorithms and / or methods of one or more embodiments. The computing platform 300 may include a memory 302, a processor 304, and a non-volatile storage device 306. The processor 304 may include one or more devices selected from a high-performance computing (HPC) system, including a high-performance core, a microprocessor, a microcontroller, a digital signal processor, a microcomputer, a central processing unit, a field programmable gate array, a programmable logic device, a state machine, a logic circuit, an analog circuit, a digital circuit, or any other device that manipulates a signal (analog or digital) based on computer executable instructions residing in the memory 302. The memory 302 may include a single memory device or multiple memory devices, including but not limited to random access memory (RAM), volatile memory, non-volatile memory, static random access memory (SRAM), dynamic random access memory (DRAM), flash memory, cache memory, or any other device capable of storing information. The non-volatile storage device 306 may include one or more persistent data storage devices, such as a hard drive, an optical drive, a tape drive, a non-volatile solid-state device, a cloud storage, or any other device capable of persistently storing information.

[0050] The processor 304 may be configured to read into the memory 302 and execute computer executable instructions that reside in the software module 308 of the non-volatile storage device 306 and embody the algorithms and / or methods of one or more embodiments. The software module 308 may include an operating system and an application program. The software module 308 may be compiled or interpreted from a computer program created using a variety of programming languages ​​and / or technologies, including, without limitation and alone or in combination, the following: Java, C, C++, C#, Objective C, Fortran, Pascal, JavaScript, Python, Perl, and PL / SQL.

[0051] When executed by processor 304, the computer executable instructions of software module 308 may cause computing platform 300 to implement one or more algorithms and / or methods disclosed herein. Non-volatile storage 306 may also include data 310 that supports the functions, features, and processes of one or more embodiments described herein.

[0052] The program code embodying the algorithms and / or methods described herein can be distributed individually or collectively as a program product in a variety of different forms. The program code can be distributed using a computer-readable storage medium having computer-readable program instructions thereon, which is used to cause the processor to implement aspects of one or more embodiments. Intrinsically non-transitory computer-readable storage media may include volatile and non-volatile, and removable and non-removable tangible media implemented in any method or technology for information storage, such as computer-readable instructions, data structures, program modules, or other data. The computer-readable storage medium may further include RAM, ROM, erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other solid-state memory technology, portable compact disk read-only memory (CD-ROM) or other optical storage device, cassette, tape, disk storage device or other magnetic storage device, or any other medium that can be used to store the desired information and can be read by a computer. The computer-readable program instructions can be downloaded from the computer-readable storage medium to a computer, another type of programmable data processing device, or another device, or downloaded to an external computer or external storage device via a network.

[0053] Computer-readable program instructions stored in a computer-readable medium can be used to instruct a computer, other types of programmable data processing devices, or other devices to operate in a particular manner so that the instructions stored in the computer-readable medium produce an article of manufacture including instructions for implementing the functions, actions, and / or operations specified in the flowchart or diagram. In certain alternative embodiments, the functions, actions, and / or operations specified in the flowchart and diagram can be reordered, processed serially, and / or processed concurrently in accordance with one or more embodiments. In addition, any flowchart and / or diagram may include more or fewer nodes or boxes than those illustrated in accordance with one or more embodiments.

[0054] Although exemplary embodiments are described above, it is not intended that these embodiments describe all possible forms included in the claims. The words used in the specification are descriptive words, not restrictive words, and it should be understood that various changes can be made without departing from the spirit and scope of the present disclosure. As previously described, the features of various embodiments can be combined to form other embodiments of the present invention that may not be explicitly described or illustrated. Although various embodiments may have been described as providing advantages over other embodiments or prior art implementations in terms of one or more desired characteristics or being preferred over other embodiments or prior art implementations, it is recognized by those of ordinary skill in the art that one or more features or characteristics may be compromised to achieve the desired overall system properties depending on the specific application and implementation. These properties may include, but are not limited to, cost, strength, durability, life cycle cost, marketability, appearance, packaging, size, applicability, weight, manufacturability, ease of assembly, etc. As such, to the extent that any embodiment is described as less desirable than other embodiments or prior art implementations in terms of one or more characteristics, these embodiments are not outside the scope of the present disclosure and may be desirable for specific applications.

Claims

1. A method for performing a single-step adversarial attack on a neural network classifier, comprising: Identifying Markov random field parameters to model the covariance of correlations between gradient terms of a loss function for a classifier; Sampling a subset of images from an image dataset according to a normal distribution to estimate a gradient term; Infer the values ​​of Markov random field parameters using black-box gradient estimation based on sampling; Generate Fourier basis vectors from the inferred values; Perturb the original image using Fourier basis vectors to obtain the loss function value; Get an estimate of the gradient from the loss function value; Create image perturbations using the estimated gradients; adding the image perturbations to the original input to generate candidate adversarial inputs that maximize the loss in the classifier identification image; querying the neural network classifier to determine the classifier prediction for the candidate adversarial input; Calculate the scores of the classifier's predictions; as well as In response to the classifier predicting an incorrect result, the candidate adversarial input is accepted as a successful adversarial attack.

2. The method of claim 1 , further comprising adding image perturbations to the original input using a fast gradient sign method.

3. The method according to claim 2, wherein the fast gradient sign method utilizes a pre-specified Perturbation bounds to add image perturbations to the original input.

4. The method of claim 1, wherein the Markov random field parameters include first parameters governing diagonal terms and second parameters governing neighboring pixels corresponding to adjacent indices in the image. The method of claim 1 , wherein the value of the parameter is inferred using Newton's method. The method of claim 1 , wherein the gradient is estimated using a predefined maximum number of queries for the classifier. The method of claim 1 , wherein the image comprises video image data. The method of claim 1 , wherein the image comprises still image data.

9. A computing system for performing a single-step adversarial attack on a neural network classifier, the system comprising: a memory storing instructions for a black-box adversarial attack algorithm of a software program; as well as A processor programmed to execute instructions to perform operations including Identifying Markov random field parameters to model the covariance of correlations between gradient terms of a loss function for a classifier; Sampling a subset of images from an image dataset according to a normal distribution to estimate a gradient term; Infer the values ​​of Markov random field parameters using black-box gradient estimation based on sampling; Generate Fourier basis vectors from the inferred values; Perturb the original image using Fourier basis vectors to obtain the loss function value; Get an estimate of the gradient from the loss function value; Create image perturbations using the estimated gradients; adding the image perturbations to the original input to generate candidate adversarial inputs that maximize the loss in the classifier identification image; querying the neural network classifier to determine the classifier prediction for the candidate adversarial input; Calculate the scores of the classifier's predictions; In response to the classifier predicting incorrectly, accepting the candidate adversarial input as a successful adversarial attack; as well as In response to the classifier predicting that it is correct, the candidate adversarial input is rejected.

10. The computing system of claim 9, wherein the processor is further programmed to add image perturbations to the original input using a fast gradient sign method.

11. The computing system of claim 10, wherein the fast gradient symbol method utilizes a pre-specified Perturbation bounds to add image perturbations to the original input.

12. The computing system of claim 9, wherein the Markov random field parameters include first parameters governing diagonal terms and second parameters governing neighboring pixels corresponding to adjacent indices in the image.

13. The computing system of claim 9, wherein the value of the parameter is inferred using Newton's method.

14. The computing system of claim 9, wherein the gradient is estimated using a predefined maximum number of queries for the classifier.

15. A non-transitory computer-readable medium comprising instructions for performing a single-step adversarial attack on a neural network classifier, the instructions, when executed by a processor, causing the processor to: Identifying Markov random field parameters to model the covariance of correlations between gradient terms of a loss function for a classifier; Sampling a subset of images from an image dataset according to a normal distribution to estimate a gradient term; Infer the values ​​of Markov random field parameters using black-box gradient estimation based on sampling; Generate Fourier basis vectors from the inferred values; Perturb the original image using Fourier basis vectors to obtain the loss function value; Get an estimate of the gradient from the loss function value; Create image perturbations using the estimated gradients; adding the image perturbations to the original input to generate candidate adversarial inputs that maximize the loss in the classifier identification image; querying the neural network classifier to determine the classifier prediction for the candidate adversarial input; Calculate the scores of the classifier's predictions; In response to the classifier predicting incorrectly, accepting the candidate adversarial input as a successful adversarial attack; as well as In response to the classifier predicting that it is correct, the candidate adversarial input is rejected.

16. The medium of claim 15, further comprising instructions that, when executed by the processor, cause the processor to add image perturbations to the original input using a fast gradient sign method.

17. The medium of claim 16, wherein the fast gradient sign method utilizes a pre-specified Perturbation bounds to add image perturbations to the original input.

18. The medium of claim 15, wherein the Markov random field parameters include first parameters governing diagonal terms and second parameters governing neighboring pixels corresponding to adjacent indices in the image.

19. The medium of claim 15, wherein the value of the parameter is inferred using Newton's method.

20. The medium of claim 15, wherein the gradient is estimated using a predefined maximum number of queries for the classifier.

Images