A zero-shot self-supervised purification method for adversarial samples

Abstract

The present application relates to the technical field of adversarial defense, and particularly relates to a zero-shot self-supervised purification method for an adversarial sample, comprising the following steps: obtaining an adversarial sample and a blurred image thereof, and extracting feature embeddings of the two; performing rough alignment processing according to the feature embeddings to obtain a rough movement result; performing fine alignment processing according to the rough movement result to obtain a purified sample; the method is flexible and lightweight, and greatly saves time and computing resources.

Description

Technical Field

[0001] This invention relates to adversarial defense technology, specifically to a zero-sample self-supervised purification method for adversarial samples. Background Technology

[0002] Adversarial attacks are one of the main risks faced by deep neural networks. These attacks create adversarial examples by adding imperceptible perturbations to normal input using specific algorithms, misleading the deep neural network into making incorrect decisions. To eliminate the impact of adversarial attacks, adversarial defense techniques have emerged. Mainstream adversarial defense techniques fall into two main categories: one is adversarial training, which primarily uses adversarial examples to train the neural network, aiming to adapt the model to the specific properties of the adversarial examples and empirically defend against adversarial attacks; however, this method requires extremely high computational resources and cannot defend against unknown attack methods. The other category is adversarial purification, which uses specific purification modules to purify adversarial examples; because this method is completely independent of the neural network to be protected, it can defend against unknown attacks in most cases.

[0003] However, existing adversarial purification techniques mainly rely on generative models and auxiliary loss functions. First, existing generative model-dependent adversarial purification techniques include: Samangouei et al. used GAN networks to simulate the probability distribution of clean samples; Song et al. assumed adversarial samples were distributed in low-probability-density regions of the training data distribution, involving PixelDefend to approximate this distribution; more recently, Nie and Wang et al. used diffusion models to purify adversarial samples, which mainly adds Gaussian noise to the adversarial samples to cover the adversarial perturbations, and uses the denoising capability of the diffusion model to recover clean samples. This type of adversarial purification technique requires training a dedicated generative model for purifying adversarial samples using a large amount of data, turning adversarial samples into normal samples. Second, existing adversarial purification techniques relying on auxiliary loss functions include: Shi et al. added a new unsupervised loss function during neural network training, and then purified adversarial samples by minimizing this unsupervised loss function during the execution phase; Mao et al. used contrastive learning to minimize the distance between feature maps in the intermediate layers of the neural network to purify adversarial samples. This type of adversarial purification technique requires retraining the neural network model to be protected using a standardized loss function.

[0004] In summary, existing adversarial purification methods are not flexible, requiring both training of the generative model and retraining of the neural network to be protected. However, training a generative model with excellent generative capabilities requires a large amount of data, and retraining a neural network trained on a large dataset also consumes significant computational resources and time. Summary of the Invention

[0005] To address the above problems, this invention provides a zero-sample self-supervised purification method for adversarial examples, characterized by comprising the following steps:

[0006] S1. Obtain the adversarial sample and its blurred image, and extract the feature embeddings of both. Perform coarse alignment processing based on the feature embeddings to obtain a coarse movement result.

[0007] S2. Perform fine alignment processing based on the coarse movement results to obtain the purified sample.

[0008] Further, step S1 acquires the adversarial sample and its blurred image, extracts the feature embeddings of both, and performs coarse alignment processing based on the feature embeddings to obtain a coarse movement result, including:

[0009] S11. Random noise is added to the adversarial sample, and then a blurring operator is applied to it to obtain a blurred image;

[0010] S12. Input the adversarial sample and the blurred image into the feature extraction network respectively to obtain the adversarial sample feature embedding and the blurred image feature embedding;

[0011] S13. Calculate the cosine similarity between the adversarial example feature embedding and the blurred image feature embedding, and calculate the movement distance based on the cosine similarity;

[0012] S14. Move the adversarial sample in a specified direction according to the moving distance, and then project the moving result into a limited pixel space to obtain a new adversarial sample;

[0013] S15. Determine if the maximum number of iterations K has been reached. c If yes, the current adversarial sample is output as a coarse movement result; otherwise, the iteration count is incremented by 1, the current adversarial sample is processed by the fuzzing operator to obtain a new fuzzy image, and then the process returns to step S12.

[0014] Further, step S13 calculates the cosine similarity between the adversarial example feature embedding and the blurred image feature embedding, expressed as:

[0015]

[0016] Wherein d(z) adv ,z′ adv ) represents the adversarial example feature embedding z adv and blurred image feature embedding z` adv Cosine similarity between them.

[0017] Furthermore, the distance traveled, calculated based on cosine similarity, is expressed as:

[0018]

[0019] Where D represents the distance traveled, α c Indicates the update step size; sgn() is a symbolic operator. This represents the gradient.

[0020] Furthermore, step S2 performs fine alignment processing based on the coarse movement results, including:

[0021] S21. Initialize the adversarial examples to obtain the initial purification results;

[0022] S22. Perform multi-layer intermediate difference processing on the purification results and the roughing results to obtain the difference value;

[0023] S23. Calculate the fine-shift distance based on the difference value, and subtract the fine-shift distance from the purification result to obtain the fine-shift result;

[0024] S24. Project the fine-movement result into a defined pixel space to obtain a new purification result;

[0025] S25. Determine if the maximum number of iterations K has been reached. f If yes, the current purification result is output as the purified sample; otherwise, the iteration count is incremented by 1 and the process returns to step S22.

[0026] Furthermore, step S22 performs multi-layer intermediate difference processing on the purification result and the roughing result to obtain difference values, including:

[0027] S221. Input the current purification result, coarse moving result, and adversarial example into the classifier; as these three flow from the input layer of the classifier network to the output layer, they will pass through all intermediate layers, and these intermediate layers will calculate the feature maps of the three.

[0028] S222. Extract the feature map of the purified result of the l-th layer, the feature map of the coarse shift result of the l-th layer, and the feature map of the adversarial example of the l-th layer from the output of the l-th layer of the classifier;

[0029] S223. Calculate the first projection difference of the l-th layer based on the feature map of the l-th layer purification result and the feature map of the l-th layer adversarial example; calculate the second projection difference of the l-th layer based on the feature map of the l-th layer coarse movement result and the feature map of the l-th layer adversarial example;

[0030] S224. Calculate the difference of the l-th layer based on the first projection difference and the second projection difference of the l-th layer; add the difference of the l-th layer to the sum of the differences of the previous l-1 layers to obtain a new sum of differences;

[0031] S225. Determine if the maximum level has been reached. If yes, output the current sum of differences as the difference value. If not, let l = l + 1 and return to step S222.

[0032] Furthermore, the difference value The calculation formula is:

[0033]

[0034] L l =-Δu″ l ·Δu′ l

[0035] Among them, L l Let Δu″ represent the difference at level l, S represent the maximum number of levels, and Δu″ represent the difference at level l. l Δu′ represents the first projection difference of the l-th layer. l This represents the second projection difference of the l-th layer.

[0036] The beneficial effects of this invention are:

[0037] This invention proposes a zero-shot unsupervised adversarial purification method based on the natural image manifold hypothesis. This method employs a strategy of first coarse movement, then fine alignment. In short, given an adversarial example, a blurring operator is used to obtain its corresponding blurred image, which is then used to guide the original adversarial example to move coarsely within the network's feature embedding space. However, the blurred image itself limits the network's feature extraction capabilities, thus limiting the distance of the coarse movement. Therefore, the adversarial example needs to be adaptively adjusted based on the coarse movement to achieve fine alignment. The entire process does not rely on any generative model, nor does it require retraining the network to be protected based on a specified loss function. Therefore, there is no additional training time, and it achieves the same purification effect as existing methods. This method is flexible and lightweight, significantly saving time and computational resources. Attached Figure Description

[0038] Figure 1 This is a flowchart of a zero-sample self-supervised purification method for adversarial examples according to the present invention;

[0039] Figure 2 This is a schematic diagram of the method framework of the present invention. Detailed Implementation

[0040] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0041] This invention provides a zero-sample self-supervised purification method for adversarial examples, which adopts a strategy of first coarsely moving and then finely aligning.

[0042] In the coarse movement section, given an adversarial example, a blurring operator is first used to obtain its corresponding blurred image. This invention first experiments with directly using the blurred image to guide the adversarial example. The experiments demonstrate that after blurring the adversarial example using the blurring operator, the ability of the adversarial example to mislead the neural network into making incorrect decisions sometimes fails, but this phenomenon is not significant. To enhance this phenomenon, this invention further obtains the feature embeddings of the adversarial example and the blurred image after blurring the adversarial example in the neural network during the coarse movement. Then, by minimizing the cosine similarity between the feature embeddings, the adversarial example is guided to move towards the blurred image. The entire process is iterative, thereby amplifying the effect of the blurred adversarial example.

[0043] However, blurring operators can also disrupt the integrity of image embedding. Therefore, coarse movement cannot completely purify adversarial examples into clean ones, and the adversarial examples tend to become blurred. Therefore, we further propose a fine alignment part. In this part, we utilize the differences of multiple intermediate layers to provide a reference direction for the adversarial examples, but no longer use the blurred samples for guidance, allowing the adversarial examples to adaptively move towards the clean samples.

[0044] In one embodiment, a zero-sample self-supervised purification method for adversarial examples, such as Figure 1 , Figure 2 As shown, it includes the following steps:

[0045] S1. Obtain the adversarial sample and its blurred image, and extract the feature embeddings of both. Perform coarse alignment processing based on the feature embeddings to obtain the coarse movement result.

[0046] Specifically, in coarse movement (i.e., coarse alignment), this invention attempts to move the adversarial sample closer to its blurred image in the feature embedding space, thereby moving the adversarial sample to the vicinity of the clean image. Furthermore, to prevent individual adversarial samples from becoming so blurred that they cannot be recognized by the classifier, we propose performing a single blurring process on the adversarial sample, followed by iteratively reducing the distance; the pseudocode for coarse movement is shown in Table 1.

[0047] Table 1. Coarse Moving Algorithm

[0048]

[0049]

[0050] Specifically, step S1 acquires the adversarial sample and its blurred image, extracts the feature embeddings of both, and performs coarse alignment based on the feature embeddings to obtain a coarse movement result, including:

[0051] S11. Random noise is added to the adversarial sample, and then a blurring operator is applied to it to obtain a blurred image;

[0052] S12. Input the adversarial sample and the blurred image into the feature extraction network respectively to obtain the adversarial sample feature embedding and the blurred image feature embedding;

[0053] Specifically, the feature extraction network is ResNet18 or WideResNet-28-10.

[0054] S13. Calculate the cosine similarity between the adversarial example feature embedding and the blurred image feature embedding, and calculate the movement distance based on the cosine similarity;

[0055] Specifically, step S13 calculates the cosine similarity between the adversarial example feature embedding and the blurred image feature embedding, expressed as:

[0056]

[0057] Wherein d(z) adv ,z′ adv ) represents the adversarial example feature embedding z adv and blurred image feature embedding z` adv Cosine similarity between them.

[0058] The distance traveled, calculated based on cosine similarity, is expressed as follows:

[0059]

[0060] Where D represents the distance traveled, α c This indicates that the step size is updated. `sgn()` is the signed operator, which takes the sign of the internal elements to obtain the direction of the gradient. This represents the gradient.

[0061] S14. Move the adversarial sample in the specified direction according to the moving distance (that is, add the adversarial sample to the moving distance, corresponding to step 6 in Table 1), and then project the moving result into the limited pixel space (corresponding to step 7 in Table 1) to ensure the integrity of the image semantic information and obtain a new adversarial sample.

[0062] S15. Determine if the maximum number of iterations K has been reached. c If yes, the current adversarial sample is output as a coarse movement result; otherwise, the iteration count is incremented by 1, the current adversarial sample is processed by the fuzzing operator to obtain a new fuzzy image, and then the process returns to step S12.

[0063] However, without data augmentation to support the classifier's adaptation to blurred images, coarse movement will be limited by the low-quality embedding of the blurred image, failing to move to the exact location of its corresponding natural image. But at least the direction is reasonable. We can demonstrate this through Table 2:

[0064] Table 2 Responses of different data augmentation strategies to coarse movement

[0065]

[0066] The main objective of this invention is to overcome the limitations of low-quality embedding of blurred images without the support of strong data augmentation; therefore, we have included a fine alignment module.

[0067] S2. Perform fine alignment processing based on the coarse movement results to obtain the purified sample.

[0068] Specifically, given the feature map f of the l-th layer of the classifier l (·), we define the following objective function:

[0069]

[0070] Where x″ is the final purification result to be obtained, Δu″ l and Δu′ l The tiled feature map is defined as follows:

[0071] Δu″ l =f l (x″)-f l (x adv )

[0072] Δu′ l =f l (x′)-f l (x adv )

[0073] Because ||u′ l Since || is a constant, maximizing the above objective function is equivalent to maximizing u″. l in u′ l The projection in the direction. An increase in the magnitude of this projection means that the adversarial sample is no longer affected by the low-quality embedding of the blurred sample and spontaneously moves towards the reference direction.

[0074] We empirically found that using multi-layer intermediate difference (MILD) during fine alignment can more effectively move adversarial examples. Define L = {l1, l2, ..., l...} m Let} be the set of m layers of classifier f, which we carefully select To calculate the above objective function, MILD can be defined as:

[0075]

[0076] st||x″-x adv ||≤δ pfy

[0077] Where ||S|| is the number of elements in set S, δ pfy This is the maximum pixel distance that can be purified and altered in the image. The pseudocode for the entire fine alignment process is shown in Table 3:

[0078] Table 3 Fine Alignment Algorithm

[0079]

[0080] Step S2 performs fine alignment processing based on the coarse movement results, including:

[0081] S21. Initialize the adversarial examples to obtain the initial purification results;

[0082] S22. Perform multi-layer intermediate difference processing on the purification results and the roughing results to obtain the difference value;

[0083] Step S22 performs multi-level intermediate difference processing on the purification results and the roughing results to obtain difference values, including:

[0084] S421. Input the purification result feature map of layer l-1, the coarse shift result feature map of layer l-1, and the adversarial example feature map of layer l-1 into the l-th layer of the classifier, and output the purification result feature map of layer l, the coarse shift result feature map of layer l, and the adversarial example feature map of layer l; wherein the purification result feature map of layer 0, the coarse shift result feature map of layer 0, and the adversarial example feature map of layer 0 corresponding to the l-th layer of the classifier are the purification result, the coarse shift result, and the adversarial example, respectively;

[0085] This step involves inputting the current purification result and the coarse shift result into the classifier, and then sequentially extracting the feature maps corresponding to each layer of the classifier to calculate the intermediate differences. The classifier includes an input layer, multiple intermediate layers, and an output layer.

[0086] S222. Calculate the first projection difference of the l-th layer based on the feature map of the l-th layer purification result and the feature map of the l-th layer adversarial example; calculate the second projection difference of the l-th layer based on the feature map of the l-th layer coarse movement result and the feature map of the l-th layer adversarial example;

[0087] S223. Calculate the difference of the l-th layer based on the first projection difference and the second projection difference of the l-th layer; add the difference of the l-th layer to the sum of the differences of the previous l-1 layers to obtain a new sum of differences;

[0088] S224. Determine if the maximum level has been reached. If yes, output the current sum of differences as the difference value. If not, let l = l + 1 and return to step S221.

[0089] S23. Calculate the fine-shift distance based on the difference value, and subtract the fine-shift distance from the purification result to obtain the fine-shift result;

[0090] S24. Project the fine-movement result adversarial sample into a limited pixel space to ensure the integrity of the image semantic information and obtain a new purification result;

[0091] S25. Determine if the maximum number of iterations K has been reached. f If yes, the current purification result is output as the purified sample; otherwise, the iteration count is incremented by 1 and the process returns to step S22.

[0092] In this invention, unless otherwise explicitly specified and limited, the terms "installation," "setting," "connection," "fixing," "rotation," etc., should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral part; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; they can refer to the internal connection of two components or the interaction between two components. Unless otherwise explicitly limited, those skilled in the art can understand the specific meaning of the above terms in this invention according to the specific circumstances.

[0093] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A zero-sample self-supervised purification method for adversarial examples, characterized in that, Includes the following steps: S1. Obtain the adversarial sample and its blurred image, and extract the feature embeddings of both. Perform coarse alignment based on the feature embeddings to obtain a coarse movement result, including: S11. Random noise is added to the adversarial sample, and then a blurring operator is applied to it to obtain a blurred image; S12. Input the adversarial sample and the blurred image into the feature extraction network respectively to obtain the adversarial sample feature embedding and the blurred image feature embedding; S13. Calculate the cosine similarity between the adversarial example feature embedding and the blurred image feature embedding, and calculate the movement distance based on the cosine similarity; S14. Move the adversarial sample in a specified direction according to the moving distance, and then project the moving result into a limited pixel space to obtain a new adversarial sample; S15. Determine if the maximum number of iterations K has been reached. c If yes, the current adversarial sample is output as a coarse movement result; otherwise, the iteration count is incremented by 1, the current adversarial sample is processed by the fuzzing operator to obtain a new fuzzy image, and then the process returns to step S12. S2. Based on the coarse movement results, perform fine alignment processing to obtain the purified sample; Step S2 performs fine alignment processing based on the coarse movement results, including: S21. Initialize the adversarial examples to obtain the initial purification results; S22. Perform multi-layer intermediate difference processing on the purification results and the roughing results to obtain the difference value; S23. Calculate the fine-shift distance based on the difference value, and subtract the fine-shift distance from the purification result to obtain the fine-shift result; S24. Project the fine-movement result into a defined pixel space to obtain a new purification result; S25. Determine if the maximum number of iterations K has been reached. f If yes, output the current purification result as the purified sample; otherwise, increment the iteration count by 1 and return to step S22. Step S22 performs multi-level intermediate difference processing on the purification results and the roughing results to obtain difference values, including: S221. Input the current purification results, coarse shift results, and adversarial examples into the classifier; S222. Extract the feature map of the purified result of the l-th layer, the feature map of the coarse shift result of the l-th layer, and the feature map of the adversarial example of the l-th layer from the output of the l-th layer of the classifier; S223. Calculate the first projection difference of the l-th layer based on the feature map of the l-th layer purification result and the feature map of the l-th layer adversarial example; calculate the second projection difference of the l-th layer based on the feature map of the l-th layer coarse movement result and the feature map of the l-th layer adversarial example; S224. Calculate the difference of the l-th layer based on the first projection difference and the second projection difference of the l-th layer; add the difference of the l-th layer to the sum of the differences of the previous l-1 layers to obtain a new sum of differences; S225. Determine if the maximum level has been reached. If yes, output the current sum of differences as the difference value. If not, let l = l + 1 and return to step S222. Difference value The calculation formula is: Among them, L l Let S represent the difference at level l, and let S represent the maximum number of levels. This represents the difference in the first projection of the l-th layer. This represents the second projection difference of the l-th layer.

2. The zero-sample self-supervised purification method for adversarial examples according to claim 1, characterized in that, The feature extraction network is either ResNet18 or WideResNet-28-10.

3. The zero-sample self-supervised purification method for adversarial examples according to claim 1, characterized in that, Step S13 calculates the cosine similarity between the adversarial example feature embedding and the blurred image feature embedding, expressed as: in, The adversarial example feature embedding z adv and blurred image feature embedding z` adv Cosine similarity between them.

4. A zero-sample self-supervised purification method for adversarial examples according to claim 3, characterized in that, The distance traveled, calculated based on cosine similarity, is expressed as follows: Where D represents the distance traveled, α c represents the update step size, sgn() is the symbolic operator, and ▽ represents the gradient.

Images