An adversarial sample generation method based on image spectrum enhancement

Abstract

The present invention discloses an adversarial sample generation method based on image spectrum enhancement, comprising the following steps: Step 1, obtain the original image x from the target dataset clean as the adversarial sample x0; Step 2, input the adversarial sample x during the iterative process t , and obtain the compensatory image sample x t‑enhance ; Step 3, use x t‑enhance and x t to obtain the high-frequency compensatory image spectrum F t‑high ; Step 4, randomly discard the low-frequency information of F t‑high to obtain the spectrum-enhanced adversarial sample set; Step 5, obtain the average aggregated gradient; Step 6, update the adversarial perturbation; Step 7, repeat Steps 2 to 6 until the maximum number of iterations T is reached, and output x T as the adversarial sample x adv . The present invention proposes an adversarial sample generation method that compensates for the high-frequency information and randomly discards the low-frequency components of the adversarial samples during the gradient iteration process, alleviates the overfitting phenomenon of the adversarial samples to the white-box model, and improves the transferability of the adversarial samples.

Description

Technical Field

[0001] The present invention belongs to the field of deep learning image adversarial attacks, and particularly relates to a method for generating adversarial samples based on image spectrum enhancement. Background Art

[0002] Neural networks have shown excellent performance in image classification tasks, achieving classification efficiency and accuracy that are difficult for humans to match. However, deliberately adding carefully crafted perturbations to clean images will induce neural networks to make incorrect classification decisions. Such artificially created perturbations that are difficult for the human visual system to detect are called adversarial noises, and the contaminated images are called adversarial samples. According to the directivity of the induced image labels, adversarial attacks can be divided into targeted attacks and non-targeted attacks. A targeted attack means that the label to which the adversarial sample is misclassified by the neural network is artificially determined, while for a non-targeted attack, it only needs the misclassified label to be different from the true label of the clean sample, and there is no restriction on which specific category it is classified into.

[0003] To improve the generation efficiency of adversarial samples, white-box attacks are currently the most common. In this scenario, the attacker can obtain the complete information of the original model, including the network structure and parameters. Gradient-based attack methods obtain the gradient of the white-box model with respect to the input image and use it as the direction guidance for applying adversarial perturbations, maximizing the model loss to generate adversarial samples, which is a research hotspot in white-box attacks. To generate adversarial samples using existing models and then attack unknown black-box models, improving the transferability of adversarial samples is one of the desirable approaches. The method based on input transformation is a direction to solve this problem. By performing operations such as translation, shearing, and scale transformation on the input image, the overfitting phenomenon of adversarial noises to the original model is reduced, and thus model enhancement can be achieved, which can effectively improve the attack effect of adversarial samples on black-box models. Since black-box attacks can only obtain the decision results of the model with respect to the input image and the available information is very limited, they are more challenging and have practical research value. Query-based attack methods are an important branch of black-box attacks. By inputting a large number of images into the model to obtain the corresponding labels and gradually changing the direction and magnitude of the adversarial perturbations, the decision boundary of the model is simulated and crossed. Such methods can effectively utilize label information to generate adversarial samples.

[0004] Current adversarial attack methods generate adversarial samples through different information acquisition methods. Generally speaking, there are still the following defects: (1) The overfitting phenomenon of adversarial samples to the original model is obvious, with low success rate and poor transferability when attacking black-box models; (2) Most of the input transformations on images stay in the spatial domain, without deliberately changing the image spectrum, making it difficult for the direction of adversarial perturbations to be detected by black-box models; (3) Against adversarial training models and defense models with measures such as adding denoising modules, the attack effect of adversarial samples is significantly reduced. Summary of the Invention

[0005] To solve the above technical problems, the present invention provides an adversarial sample generation method based on image spectrum enhancement. Based on the differences in the ability of different models to extract image spectrum components, this method compensates for the high-frequency components and randomly discards the low-frequency components of the image spectrum during the iterative process, and applies adversarial noise using the average aggregated gradient, enabling the white-box model to extract richer high-frequency information for backpropagation gradient information and simulating the low-frequency information extracted by the black-box model, alleviating the overfitting phenomenon of adversarial samples to the white-box model, thereby achieving image spectrum enhancement, making the adversarial samples have both a high white-box attack success rate and high transferability, and improving the attack effect of the adversarial samples on the defense model.

[0006] The technical method adopted by the present invention is: an adversarial sample generation method based on image spectrum enhancement, characterized by including the following steps:

[0007] Step 1: Obtain the original image x with the class label y from the target dataset clean , and use it as the adversarial sample x at the iteration number t = 0 t , that is, x0;

[0008] Step 2: Input the adversarial sample x during the iterative process t , and use low-pass filtering and high-frequency enhancement filtering to obtain the compensatory image sample x t-enhance :

[0009] Step 201: Select a low-pass convolution kernel W to perform a convolution operation on the adversarial sample, that is, low-pass filtering, to obtain the low-pass image sample x t-low :

[0010] x t-low = x t * W

[0011] Step 202: Subtract the low-frequency image sample x t from the adversarial sample x t-low to obtain the unsharp mask x t-mask :

[0012] x t-mask = x t - x t-low

[0013] Step 203: Set the weight factor k, and use the unsharp mask x t-mask to implement high-frequency enhancement filtering to obtain the high-frequency enhanced image sample x t-high :

[0014] x t-high = x t + k × x t-mask

[0015] Step 204: Subtract the high-frequency enhanced image sample x t-high from the low-frequency image sample x t-low to obtain the compensatory image sample x t-enhance :

[0016] x t-enhance = x t-high - x t-low

[0017] Step 3: Use the compensatory image sample x t-enhance to compensate for the high-frequency components of the adversarial sample x t to obtain the high-frequency compensated image spectrum F t-high :

[0018] Step 301: Set a high-frequency mask M H with the same size (n, n) as the image to extract the high-frequency part of the image spectrum:

[0019]

[0020] where r is the high-low frequency segmentation parameter, satisfying 0 < r < n;

[0021] Step 302: Use the discrete cosine transform to transform the compensatory image sample x t-enhance to the frequency domain and perform a Hadamard product with the high-frequency mask M H to obtain the high-frequency spectrum F t-enhance of the compensatory image:

[0022] F t-enhance = DCT(x t-enhance ) ⊙ M H

[0023] Step 303: Set a high-frequency compensation factor β > 0 and use the high-frequency spectrum F t-enhance of the compensatory image to enhance the spectrum of the adversarial sample x t to obtain the high-frequency compensated adversarial spectrum F t-high :

[0024] F t-high = DCT(x t ) + β × F t-high

[0025] Step 4: Randomly discard the low-frequency information of the high-frequency compensated adversarial spectrum F t-high multiple times to obtain the complete spectrum-enhanced adversarial sample set:

[0026] Step 401: Set a low-frequency mask M L with the same size (n, n) as the image to randomly discard the low-frequency components of the image spectrum:

[0027]

[0028] Where R(0, 1, p) represents 0 with probability p and 1 with probability 1 - p, and 0 ≤ p ≤ 1;

[0029] Step 402: Perform Hadamard product on the high-frequency compensation adversarial spectrum F t-high and the low-frequency mask M L and use the inverse discrete cosine transform to convert the spectrum with low-frequency information discarded into the spatial domain to obtain the complete spectrum enhanced adversarial sample x′ t :

[0030] x′ t = IDCT(F t-high ⊙ M L )

[0031] Step 403: Repeat steps 401 - 402 until the number of spectrum enhancement transformations N is reached to obtain the complete spectrum enhanced adversarial sample set x′ t-1 , x′ t-2 , …, x′ t-N ;

[0032] Step Five: Input the images in the complete spectrum enhanced adversarial sample set into the neural network respectively to obtain the average aggregated gradient:

[0033] Step 501: Input the complete spectrum enhanced adversarial samples x′ t-1 , x′ t-2 , …, x′ t-N into the neural network respectively, and use the network loss to take the derivative of the gradient of the image sample to obtain the gradient G i :

[0034] G i = ▽ x J(x′ t-i , y)

[0035] Step 502: Obtain the average aggregated gradient based on the obtained G i where λ

[0036]

[0037] is the weight assigned by the neural network to the complete spectrum enhanced adversarial sample, satisfying: i

[0038]

[0039] Step Six: Update the momentum using the average aggregated gradient to guide the update direction of the adversarial sample:

[0040] Step 601: Set the momentum delay factor μ ∈ [0, 1], and use the average aggregated gradient to update the momentum g t+1 :

[0041]

[0042] Step 602: Set the perturbation noise threshold ξ and the maximum number of iterations T to obtain the magnitude of a single adversarial perturbation

[0043] Step 603: Using g t+1 as the application direction of the perturbation, update the pixel values of the adversarial sample:

[0044] x t+1 = x t + α·sign(g t+1 )

[0045] Step 604: Centering on the pixel values of the clean image sample, use the perturbation noise threshold ξ to clip the pixel values of x t+1 , and limit the pixel values of the clipped image to the normal display range [0, 255] to obtain the updated adversarial sample:

[0046]

[0047] Step Seven: Repeat Steps Two to Six until the maximum number of iterations T is reached, and output the updated image sample x T as the adversarial sample x adv .

[0048] Compared with the prior art, the present invention mainly has the following advantages:

[0049] First, the present invention proposes an adversarial sample generation method based on image spectrum enhancement. By respectively making up for information and randomly losing information in the high-frequency and low-frequency components of the image spectrum, the white-box model can extract richer spectrum information during the iterative process, alleviating the overfitting phenomenon of the adversarial noise to the white-box model while enhancing the simulation effect of the information extracted by the black-box model, and significantly enhancing the transferability of the adversarial sample;

[0050] Second, the present invention takes into account the differences in the ability of different models to extract image spectrum information, performs relevant transformations on the image in the frequency domain, realizes spectrum enhancement, makes the adversarial perturbation no longer concentrated in a fixed frequency range, and enhances the attack effect of the adversarial sample on the defense model.

[0051] The technical solution of the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. Description of the Drawings

[0052] Figure 1 This is the flowchart of the method of the present invention. Detailed implementation manners

[0053] The method of the present invention will be further described in detail below in conjunction with the accompanying drawings and embodiments of the present invention.

[0054] It should be noted that, without conflict, the embodiments in the present application and the features in the embodiments may be combined with each other. The present invention will be described in detail below with reference to the accompanying drawings and embodiments.

[0055] It should be noted that the terms used herein are only for describing specific implementation manners and are not intended to limit the exemplary embodiments according to the present application. As used herein, unless the context clearly indicates otherwise, the singular forms are also intended to include the plural forms. In addition, it should be understood that when the terms "comprise" and / or "include" are used in this specification, they specify the presence of features, steps, operations, devices, components, and / or combinations thereof.

[0056] It should be noted that the terms "first", "second", etc. in the description and claims of the present application and the above-mentioned drawings are used to distinguish similar objects and do not necessarily have to be used to describe a specific order or sequence. It should be understood that such data can be interchanged under appropriate circumstances so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "comprise" and "have" and any variations thereof are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or device that comprises a series of steps or units does not necessarily have to be limited to those steps or units clearly listed, but may include other steps or units not clearly listed or inherent to these processes, methods, products, or devices.

[0057] For ease of description, spatial relative terms such as "above...", "over...", "on the upper surface of...", "above" etc. may be used herein to describe the spatial positional relationship of a device or feature shown in the drawings with other devices or features. It should be understood that the spatial relative terms are intended to include different orientations in use or operation in addition to the orientation of the device shown in the drawings. For example, if the device in the drawing is inverted, the device described as "above other devices or structures" or "over other devices or structures" will then be positioned as "below other devices or structures" or "under other devices or structures". Thus, the exemplary term "above..." may include both the orientations of "above..." and "below...". The device may also be positioned in other different ways (rotated 90 degrees or in other orientations), and corresponding interpretations will be made for the spatial relative descriptions used herein.

[0058] AsFigure 1 As shown, taking the publicly available ILSVRC2017 image dataset as an example, the rationality and effectiveness of the present invention are illustrated, and the specific steps are as follows:

[0059] Step 1: Obtain the original image x with the class label y from the target dataset clean , and use it as the adversarial sample x at the iteration number t = 0 t , that is, x0;

[0060] Step 2: Input the adversarial sample x during the iteration process t , and use low-pass filtering and high-frequency enhancement filtering to obtain the compensatory image sample x t-enhance :

[0061] Step 201: Select a Gaussian convolution kernel W with a size of 3×3 to perform a convolution operation on the adversarial sample, that is, low-pass filtering, to obtain the low-pass image sample x t-low :

[0062] x t-low =x t *W

[0063] Step 202: Subtract the low-frequency image sample x t from the adversarial sample x t-low to obtain the unsharp mask x t-mask :

[0064] x t-mask =x t -x t-low

[0065] Step 203: Set the weight factor k = 2, and use the unsharp mask x t-mask to implement high-frequency enhancement filtering to obtain the high-frequency enhanced image sample x t-high :

[0066] x t-high =x t +k×x t-mask

[0067] Step 204: Subtract the low-frequency image sample x t-high from the high-frequency enhanced image sample x t-low to obtain the compensatory image sample x t-enhance :

[0068] x t-enhance =x t-high -x t-low

[0069] Step 3: Use the compensatory image sample x t-enhance to compensate for the high-frequency components of the adversarial sample x t to obtain the high-frequency compensated image spectrum Ft-high :

[0070] Step 301: Set a high-frequency mask M with the same size as the image (299, 299) H for extracting the high-frequency part of the image spectrum:

[0071]

[0072] where r = 100 is the high-low frequency segmentation parameter;

[0073] Step 302: Use the discrete cosine transform to transform the compensatory image sample x t-enhance to the frequency domain and perform a Hadamard product with the high-frequency mask M H to obtain the high-frequency spectrum F of the compensatory image t-enhance :

[0074] F t-enhance = DCT(x t-enhance ) ⊙ M H

[0075] Step 303: Set the high-frequency compensation factor β = 0.025 and use the high-frequency spectrum F of the compensatory image t-enhance to enhance the spectrum of the adversarial sample x t to obtain the high-frequency compensated adversarial spectrum F t-high :

[0076] F t-high = DCT(x t ) + β × F t-high

[0077] Step 4: Randomly discard the low-frequency information of the high-frequency compensated adversarial spectrum F t-high multiple times to obtain a complete spectrum-enhanced adversarial sample set:

[0078] Step 401: Set a low-frequency mask M with the same size as the image (299, 299) L for randomly discarding the low-frequency components of the image spectrum:

[0079]

[0080] where R(0, 1, 0.1) represents a probability of 0 with a probability of 0.1 and a probability of 1 with a probability of 0.9;

[0081] Step 402: Perform a Hadamard product on the high-frequency enhanced adversarial spectrum F t-high and the low-frequency mask M L and use the inverse discrete cosine transform to transform the spectrum with the low-frequency information discarded to the spatial domain to obtain a complete spectrum-enhanced adversarial sample x' t :

[0082] x′ t = IDCT(F t-high ⊙M L )

[0083] Step 403: Repeat steps 301 - 302 until the number of spectral enhancement transformations N = 20 is reached to obtain the complete set of spectral enhancement adversarial samples x′ t-1 , x′ t-2 , …, x′ t-20 ;

[0084] Step Five: Input the images in the complete set of spectral enhancement adversarial samples into the neural network respectively to obtain the average aggregated gradient:

[0085] Step 501: Input the complete spectral enhancement adversarial samples x′ t-1 , x′ t-2 , …, x′ t-20 into the neural network respectively, and obtain the gradient G i by taking the derivative of the gradient of the image sample with respect to the network loss:

[0086] G i =▽ x J(x′ t-i , y)

[0087] Step 502: Obtain the average aggregated gradient i based on the obtained G

[0088]

[0089] where λ i is the weight assigned by the neural network to the complete spectral enhancement adversarial sample, satisfying:

[0090]

[0091] Step Six: Update the momentum using the average aggregated gradient to guide the update direction of the adversarial sample:

[0092] Step 601: Set the momentum delay factor μ = 1 and update the momentum g using the average aggregated gradient t+1 :

[0093]

[0094] Step 602: Set the perturbation noise threshold ξ = 16 and the maximum number of iterations T = 10 to obtain the magnitude of the single - time adversarial perturbation application

[0095] Step 603: Update the pixel values of the adversarial sample with the momentum g t+1 as the application direction of the perturbation:

[0096] x t+1 = x t + α·sign(g t+1 )

[0097] Step 604: Centering on the pixel values of the clean image sample, shear the pixel values of x t+1 using the perturbation noise threshold ξ, and limit the pixel values of the sheared image to the normal display range [0, 255] to obtain the updated adversarial sample:

[0098]

[0099] Step Seven: Repeat Step Two to Step Six until the maximum number of iterations T is reached, and output the updated image sample x T as the adversarial sample x adv .

[0100] The above are only embodiments of the present invention and do not impose any limitations on the present invention. Any simple modifications, changes, and equivalent structural changes made to the above embodiments based on the technical essence of the present invention still fall within the protection scope of the technical solution of the present invention.

Claims

1. An adversarial sample generation method based on image spectrum enhancement, characterized in that: Including the following steps: Step 1: Obtain the original image x with the class label y from the target dataset clean , and use it as the adversarial sample x at the iteration number t = 0 t , that is, x0; Step 2: Input the adversarial sample x during the iterative process t , and obtain the compensatory image sample x by using low-pass filtering and high-frequency enhancement filtering t-enhance : Step 201: Select a low-pass convolution kernel W to perform a convolution operation on the adversarial sample, that is, low-pass filtering, to obtain a low-pass image sample x t-low : x t-low = x t * W Step 202: Subtract the adversarial sample x t from the low-frequency image sample x t-low to obtain the unsharp mask x t-mask : x t-mask = x t -x t-low Step 203, set the weight factor k, and use unsharp masking x t-mask to implement high-frequency enhancement filtering and obtain the high-frequency enhanced image sample x t-high : x t-high = x t + k × x t-mask Step 204: Subtract the high-frequency enhanced image sample x t-high from the low-frequency image sample x t-low to obtain a compensatory image sample x t-enhance : x t-enhance = x t-high -x t-low Step 3: Use the compensatory image sample x t-enhance to compensate for the high-frequency components of the adversarial sample x t and obtain the high-frequency compensated image spectrum F t-high : Step 301: Set a high-frequency mask M with the same size as the image (n, n) H , which is used to extract the high-frequency part of the image spectrum: where r is the high-low frequency segmentation parameter, satisfying 0 < r < n; Step 302: Use the discrete cosine transform to transform the compensatory image sample x t-enhance to the frequency domain, and perform a Hadamard product with the high-frequency mask M H to obtain the high-frequency spectrum F of the compensatory image t-enhance : F t-enhance = DCT(x t-enhance )M H Step 303: Set the high-frequency compensation factor β > 0, and use the compensated image high-frequency spectrum F t-enhance to perform spectrum enhancement on the adversarial sample x t to obtain the high-frequency compensated adversarial spectrum F t-high : F t-high = DCT(x t ) + β × F t-high Step 4. Randomly discard the low-frequency information of the high-frequency compensated adversarial spectrum F multiple times to obtain a complete spectrum enhanced adversarial sample set: t-high ​ Step 401: Set a low-frequency mask M with the same size as the image (n, n) L , which is used to randomly discard the low-frequency components of the image spectrum: where R(0, 1, p) represents 0 with probability p and 1 with probability 1 - p, and 0 ≤ p ≤ 1; Step 402: Perform Hadamard product on the high-frequency compensation adversarial spectrum F t-high and the low-frequency mask M L to convert the spectrum with low-frequency information discarded to the spatial domain by using inverse discrete cosine transform, and obtain the complete spectrum enhanced adversarial sample x′ t : x′ t = IDCT(F t-high ⊙M L ) Step 403: Repeat steps 401 - 402 until the number of spectral enhancement transformation times N is reached, and obtain the complete set of spectral enhancement adversarial samples \(x'\) t-1 , \(x'\) t-2 , …, \(x'\) t-N ; Step 5: Input the images in the complete spectrum enhanced adversarial sample set into the neural network respectively to obtain the average aggregated gradient: Step 501: Respectively input the complete spectrum enhanced adversarial samples \(x'\) t-1 , \(x'\) t-2 , …, \(x'\) t-N into the neural network, and obtain the gradient \(G\) i : Step 502. According to the obtained G i Obtain the average aggregation gradient where λ i is the weight assigned by the neural network to the complete spectrum enhanced adversarial sample, and satisfies: Step 6: Use the average aggregated gradient to update the momentum and guide the update direction of the adversarial samples: Step 601: Set the momentum delay factor μ ∈ [0, 1], and use the average aggregated gradient to update the momentum g t+1 : Step 602: Set the perturbation noise threshold ξ and the maximum number of iterations T to obtain the magnitude of the single adversarial perturbation applied Step 603: Update the pixel values of the adversarial sample with the momentum g t+1 as the application direction of the perturbation: x t+1 = x t + α·sign(g t+1 ) Step 604: Centering on the pixel values of the clean image sample, shear the pixel values of x t+1 using the perturbation noise threshold ξ, and restrict the pixel values of the sheared image to the normal display range [0, 255] to obtain the updated adversarial sample: Step 7: Repeat Steps 2 to 6 until the maximum number of iterations T is reached, and output the updated image sample x T as the adversarial sample x adv .

Images