An improved method of backdoor attack against adversarial perturbation

Abstract

The application discloses a Gaussian distribution-based adversarial perturbation backdoor attack method capable of improving interference disturbance stability and concealment and effectively evading detection of a defense model. The Gaussian distribution-based adversarial perturbation backdoor attack method first superimposes noise satisfying Gaussian distribution in adversarial attack, which makes the adversarial perturbation generated by the adversarial attack in clean samples and clean models smaller, and then superimposes the perturbation into general adversarial perturbation. Then, the perturbation is added to part of the clean samples, and the labels of the clean samples are changed to construct toxic samples. The toxic samples are used for model training, thereby generating a backdoor attack. The Gaussian distribution-based adversarial perturbation backdoor attack method can not only reduce data distribution anomaly and improve the stability of perturbation generation, but also greatly improve the detection resistance.

Description

Technical Field

[0001] This invention relates to the field of backdoor attacks, and in particular to an improved method for countering perturbation backdoor attacks. Background Technology

[0002] In recent years, with the rise of deep learning, deep neural networks (DNNs) have achieved remarkable results in various fields, including autonomous driving, intrusion detection, and facial recognition. However, these applications still face the threat of backdoor attacks. Backdoor attacks involve embedding backdoor triggers into deep neural network models. When these models encounter backdoor triggers, they will misclassify the data into a predetermined target class, but they can still maintain high accuracy on benign data.

[0003] Typically, backdoor attacks use a patch-based backdoor pattern as a trigger, and the selection of these triggers does not take into account the data distribution of the dataset or the applicable scenario. Therefore, samples with these triggers lead to anomalous data distribution, and DNNs trained on these triggers will exhibit anomalous activations during inference. These anomalous activations can be detected using current advanced backdoor detection techniques, which can also filter out toxic data that causes anomalous activations.

[0004] Therefore, these patch-based backdoors are ineffective because they cannot bypass the detection of the defense model. Although recent studies by Nguyen TA, Tran A. Input-aware dynamic backdoor attack[J]. Advances in Neural Information Processing Systems, 2020, 33:3454-3464. and Shokri R. Bypassing backdoor detection algorithms in deep learning[C] / / 2020IEEE European Symposium on Security and Privacy (EuroS&P).IEEE, 2020:175-183. have attempted to bypass verification in other ways, they are based on the assumption that the attacker can completely manipulate the model's training process. Around the same time, Zhong H, Liao C, Squicciarini AC, et al. Backdoor embedding in convolutional neural network models via invisible perturbation[C] / / Proceedings of the TenthACM Conference on Data and Application Security and Privacy.2020:97-108. proposed using adversarial perturbations to design triggers. They employ a Universal Adversarial Perturbation (UAP) approach to generate an imperceptible perturbation. This method involves generating multiple tiny perturbations from a training dataset using an iterative adversarial attack algorithm, and then stacking these tiny adversarial perturbations together to generate a universal trigger that achieves universality on the training dataset.Zhang Q, Ding Y, Tian Y, et al. AdvDoor: adversarial backdoor attack of deep learning system[C] / / Proceedings of the 30th ACM SIGSOFT International Symposium on Software Testing and Analysis.2021:127-138. Similar to the UAP work, a Targeted Universal Adversarial Perturbation (TUAP) is proposed to achieve targeted backdoor attacks, and the authors prove that the triggering mode based on TUAP is more difficult to be attacked by Tran B, Li J, Mądry A. Spectralsignatures in backdoor attacks[C] / / Proceedings of the 32nd International Conference on Neural Information Processing Systems.2018:8011-8021. and Chen B, Carvalho W, Baracaldo N, et al. Detecting backdoor attacks on deep neuralnetworks by activation clustering[J].arXiv preprint The defense model proposed in arXiv:1811.03728,2018 was detected. Although the triggers generated by TUAP can bypass the detection of the defense model, the perturbation amplitude is quite obvious, and the general perturbation triggers generated for different sample categories are very unstable. Therefore, we studied the individual perturbations generated by TUAP in each iteration of adversarial attacks and found that they did not consider the directionality of the perturbation in the perturbation generation method. Therefore, the perturbations generated by this method have adversarial instability, which leads to the randomness of the generated triggers. Most of the triggers can be easily identified by the human eye. Summary of the Invention

[0005] The technical problem to be solved by the present invention is to provide an improved method for countering backdoor attacks that can improve the stability and concealment of interference and disturbance and more effectively evade the detection of defense models.

[0006] The technical solution adopted by this invention to solve its technical problem is: an improved method for resisting backdoor attacks, comprising the following steps:

[0007] S1. Establish a planar coordinate system. Draw a circle tangent to H(x) with (x0, y0) as the center. Then draw a triangle with H(x) as the hypotenuse that intersects the circle. At this time, any point on the circle is the minimum perturbation from (x0, y0) to the hyperplane H(x).

[0008] The direction from (x0, y0) to the hyperplane H(x) can be arbitrary, and different directions have different perturbation quantities. The orthogonal projection direction from (x0, y0) to the hyperplane H(x) has the smallest perturbation quantity.

[0009] Where H(x)=ω T x+b; where ω T It is a row vector representing the model weights, x is a column vector of input features, and b is the bias.

[0010] Noise with a Gaussian distribution is added to stimulate the perturbation direction; the point (x0, y0) is used as the mean to make the perturbation direction centered on the perturbation, and the variance is controlled to make the perturbation direction as much as possible towards the orthogonal projection direction, so that the generated perturbation reaches the optimal value with the highest probability, and each time a different perturbation is generated, each generated perturbation can reach the optimal value with the highest probability.

[0011] The following formula is obtained:

[0012] ;

[0013] In the formula: It uses the 𝐹𝐺 adversarial attack algorithm to generate adversarial perturbations for the i-th sample; It is calculated at the i-th sample. Gaussian noise; The perturbation for the i-th sample;

[0014] S2, Through Algorithm generates universal perturbation Select a clean dataset and an attack model, iterate through the i-th sample, and then select an attack target. The perturbation method is iteratively applied using adversarial attacks, superimposing the perturbation of the i-th sample and the perturbation of the (i+1)-th sample to obtain the general perturbation V; specifically, the following steps are included:

[0015] S21, if ;

[0016] The sample perturbation is iterated using the following formula;

[0017] ;

[0018] in, This refers to adversarial attacks; the input to an adversarial attack includes clean samples, an adversarial attack algorithm, and an attack target. It is Gaussian noise; Represent the target model, Let i represent the i-th sample. Indicates the category of the attack target;

[0019] The following formula is used to superimpose the results of each iteration. , This refers to the perturbation of the i-th sample. The perturbations of each sample are then superimposed together by an algorithm to finally generate a V.

[0020] ;

[0021] The following formula is used to limit the overall perturbation and optimize its magnitude;

[0022] ;

[0023] ;

[0024] ;

[0025] In the formula, Represents the radius of the projected sphere; This indicates an adversarial attack algorithm; Indicates the direction of the disturbance; Indicates the minimum disturbance amount; This means that the final generated V is composed of both the direction and amount of the disturbance;

[0026] S22, Loop step S21; The maximum number of loop iterations is n;

[0027] S23, loop steps S21 and S22; and satisfy the loop count requirement. , And satisfy ;in The specified maximum number of loops; and Define a disturbance quantity , It is composed of the sum of the values ​​of the additional perturbations generated after the attack, where, This represents the value of a disturbance. X represents the sum of all perturbations; Xs represents the dataset of the target.

[0028] S24, obtain a list containing all ; and the disturbance V has been concealed;

[0029] S3, Data poisoning; randomly select clean samples and use... The algorithm generates a toxic dataset, then puts the toxic dataset into a clean dataset, and uses it to train the model to produce a toxic model.

[0030] Furthermore, in step S21, control The magnitude of the mean and variance of Gaussian noise, where the mean is 0 and the variance is 0.01; a function that generates a small perturbation to interfere with the perturbation direction, with the input being the sample and the adversarial perturbation.

[0031] Furthermore, in step S2, P refers to the adversarial attack algorithm, which employs the DeepFool attack.

[0032] Furthermore, in step S2, the clean dataset is selected as the dataset that needs to be contaminated in the backdoor attack; the clean model is selected as the target model of the attack.

[0033] The beneficial effects of the present invention are: the improved anti-disturbance backdoor attack method described in the present invention can improve the stability and concealment of the disturbance in terms of trigger generation, and can easily evade the human visual recognition system;

[0034] Secondly, regarding resistance to detection, all patch-based methods were marked as poisoned models, and 30% of TUAP models were also marked as poisoned models, while our GN-TUAP model evaded detection completely, achieving a 100% pass rate. The improved anti-perturbation backdoor attack method described in this invention not only reduces data distribution anomalies and improves the stability of perturbation generation, but also significantly enhances resistance to detection. Furthermore, the improved anti-perturbation backdoor attack method described in this invention must be used within legal limits. Attached Figure Description

[0035] Figure 1 This is a schematic diagram of the decision hyperplane for a two-dimensional binary classification task in an embodiment of the present invention;

[0036] Figure 2 This is a data distribution diagram based on patch triggers in an embodiment of the present invention;

[0037] Figure 3 This is a schematic diagram illustrating the instability of disturbances in an embodiment of the present invention;

[0038] Figure 4 This is a graph showing different variances in the probability density function in an embodiment of the present invention;

[0039] Figure 5 This is a schematic diagram of the data poisoning process in an embodiment of the present invention;

[0040] Figure 6 This is a schematic diagram of the backdoor detection process in an embodiment of the present invention;

[0041] Figure 7 The diagram shows triggers (a), (b), and (c) generated using GN-TUAP and triggers (d), (e), and (f) generated using TUAP, as well as toxic samples (A), (B), and (C) generated using GN-TUAP and toxic samples (D), (E), and (F) generated using TUAP in embodiments of the present invention.

[0042] Figure 8 This is a schematic diagram of toxic samples (a), (b), (c), (d), (e), (f) generated using GN-TUAP on Fashion MNIST and toxic samples (A), (B), (C), (D), (E), (F) generated using TUAP on MNIST in an embodiment of the present invention;

[0043] Figure 9 This is a schematic diagram comparing accuracy evaluation indicators in embodiments of the present invention;

[0044] Figure 10 This is a comparative diagram of recall rate evaluation indicators in the embodiments of the present invention;

[0045] Figure 11 This is a schematic diagram comparing the F1 value evaluation indicators in the embodiments of the present invention;

[0046] Figure 12 This is a schematic diagram comparing the average values ​​of three indicators in an embodiment of the present invention. Detailed Implementation

[0047] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0048] The theoretical basis of the improved method for resisting perturbation backdoor attacks described in this invention is as follows:

[0049] 1. Counterattack

[0050] In recent years, numerous studies have shown that DNNs are highly vulnerable to adversarial attacks in their application scenarios. Suppose we have a prediction function f and a clean sample x, and the goal of an adversarial attack is to find a perturbation v such that f(x + v) ≠ f(X). x + v is called an adversarial sample. Typically, v is a very small perturbation, so small that it is imperceptible to the human eye.

[0051] None of the existing adversarial attack algorithms can satisfy all attack scenarios; different adversarial attack algorithms have their own advantages and disadvantages in different scenarios. Based on our objectives, we selected the classic DeepFool algorithm, such as Moosavi-Dezfooli SM, Fawzi A, Frossard P. Deepfool: a simple and accurate method to fool deep neural networks[C] / / Proceedings of the IEEE conference on computer vision and pattern recognition.2016:2574-2582. It is based on gradient iteration methods that generate the smallest perturbation and has a high attack success rate. Since DeepFool can be used for binary and multi-class classification tasks, and our goal is to mispredict class A as class B, which is a binary classification task, such as... Figure 1 As shown.

[0052] Suppose there is a decision hyperplane Ax + By + C = 0. Let y = Ax + By + C. When y > 0, the classification result is A, and when y < 0, the classification result is B. The purpose of the adversarial attack is to make the point (x0, y0) move towards the decision hyperplane. When the point (x0, y0) crosses the decision hyperplane, it will cause the model to misclassify. r is the distance that 0 (x0, y0) moves from the line y, which is also called the perturbation. We can calculate the magnitude of r from the formula for the distance from a point to a line, as shown in formula (1).

[0053]

[0054] If the decision hyperplane is fixed, moving the point (x0, y0) in the y direction will yield formula (2).

[0055]

[0056] In the formula, "-" represents the direction of movement, and e represents a unit vector.

[0057] When the two-dimensional binary classification task is extended to the high-dimensional binary classification task, formula (3) can be obtained from formula (2).

[0058]

[0059] Note that r(x) here is linearly differentiable. We can then use an iterative method to find the optimal value of r, and the optimization function is shown in equation (4).

[0060]

[0061] In the formula f (x i ) T This is the gradient we need to iterate over. Adversarial attacks continuously search for the optimal gradient to generate perturbations, thereby generating triggers with minimal perturbation.

[0062] 2. Backdoor attack

[0063] Backdoor attacks are a classic type of attack in traditional security. They primarily exploit system vulnerabilities to gain system access through covert channels. In 2017, Gu et al. proposed Badnets, introducing backdoor attacks into deep learning models for the first time. This involved replacing clean data with malicious data to poison a portion of the training data, thus altering the training process. The model would then memorize triggers from the malicious data during learning, allowing backdoors to be injected. The specific definition of a backdoor attack is as follows:

[0064] D={(x1,y1) ,(x2,y2) , ……(x n y n Let} represent a clean training dataset, (x1, x2, ..., xn). n Let ∈ X represent a clean sample, (y1, y2, ..., y...). n )∈Y represents the source label. To carry out a backdoor attack, during the generation of the poisoned dataset, the attacker first selects a small number of clean samples D. C ={(x i y i ) │(x1)∈X, (y1)∈Y}where Dc∈D, then the trigger is added to the clean sample x and the corresponding value y is modified, and finally the toxic samples (x%, y%) are obtained. These toxic samples are placed in the clean dataset to become the toxic dataset Dp. The generation of toxic samples is shown in formula (5).

[0065] Dp={(x1%,y1%) ,(x2%,y2%) ,……(x n %, y n %)} (5)

[0066] Generally speaking, the percentage of toxic samples x% and clean samples x% after adding triggers i The models will be very similar, making them difficult for the human visual recognition system to detect, thus achieving a stealthy effect. During the training phase, attackers use the generated malicious dataset to manipulate the clean model f. c After training, a poisoning model f will be generated. pThe stealth of backdoor attacks lies in the fact that even if the model is poisoned, it will not affect the normal clean model prediction, as shown in formula (6).

[0067] f p (x)=y,f p (x%) = y% (6)

[0068] When a clean sample x is input, the model will classify it as y correctly. However, when a sample x% carrying a trigger is input into the model, the poisoned model will misclassify it as y.

[0069] 3. Gaussian noise

[0070] In image classification, the accuracy is often affected by random errors, which we consider noise. Noise can be generated during image capture, transmission, and processing. Noise is generally characterized by its frequency characteristics. Ideal noise is called white noise. If the noise amplitude distribution follows a Gaussian distribution and the power spectral density follows a uniform distribution, it is called Gaussian white noise. The Gaussian aspect of Gaussian white noise refers to its probability distribution being Gaussian, and white noise is noise that satisfies the condition of uncorrelated second moments and constant first moments. From a temporal perspective, uncorrelated second-order matrices mean that at a certain point on the time axis, the noise is independent and unrelated to other noise; its amplitude is unaffected by preceding or following noise. From a frequency perspective, this means that the noise amplitude appears randomly at any given time, i.e., the noise satisfies a uniform distribution.

[0071] 4. Backdoor detection

[0072] To prevent backdoor poisoning attacks, Chen B, Carvalho W, Baracaldo N, et al. Detecting backdoor attacks on deep neural networks by activation clustering[J]. arXiv preprint arXiv:1811.03728,2018 proposed a classic detection method, which mainly detects backdoors based on anomalous activations. First, it provides all inputs to the backdoor model and collects their activation values. Then, after dimensionality reduction, it uses the K-means algorithm to cluster the activation values ​​into two clusters. If the number of activations in one cluster is lower than a certain value, the cluster is identified as poisoned. Once a cluster is identified as poisoned, the current model is marked as poisoned, and the corresponding activation data in the poisoned cluster is deleted. Activation clustering detection methods are suitable for most backdoor attacks because they construct triggers based on patch patterns, which leads to anomalous data distribution, thus detecting the anomalies. Figure 2 As shown.

[0073] This invention provides an improved method for resisting perturbation backdoor attacks, comprising the following steps:

[0074] S1. Establish a planar coordinate system. Draw a circle tangent to H(x) with (x0, y0) as the center. Then draw a triangle with H(x) as the hypotenuse that intersects the circle. At this time, any point on the circle is the minimum perturbation from (x0, y0) to the hyperplane H(x).

[0075] The direction from (x0, y0) to the hyperplane H(x) can be arbitrary, and different directions have different perturbation quantities. The orthogonal projection direction from (x0, y0) to the hyperplane H(x) has the smallest perturbation quantity.

[0076] Where H(x)=ω T x+b; where ω T It is a row vector representing the model weights, x is a column vector of input features, and b is the bias.

[0077] Noise with a Gaussian distribution is added to stimulate the perturbation direction; the point (x0, y0) is used as the mean to make the perturbation direction centered on the perturbation, and the variance is controlled to make the perturbation direction as much as possible towards the orthogonal projection direction, so that the generated perturbation reaches the optimal value with the highest probability, and each time a different perturbation is generated, each generated perturbation can reach the optimal value with the highest probability.

[0078] The following formula is obtained:

[0079] ;

[0080] In the formula: It is to utilize Adversarial attack algorithms are used to generate adversarial perturbations for the i-th sample; It is calculated at the i-th sample. Gaussian noise; The perturbation for the i-th sample;

[0081] S2. Generate a general perturbation V using the GN-TUAP algorithm; select a clean dataset and a clean model, iterate through the i-th sample, and then select an attack target. The perturbation method is iteratively applied using adversarial attacks, superimposing the perturbation of the i-th sample and the perturbation of the (i+1)-th sample to finally obtain the universal perturbation V; specifically, the following steps are included:

[0082] S21, if ;

[0083] The sample perturbation is iterated using the following formula;

[0084] ;

[0085] in, This refers to adversarial attacks; the input to an adversarial attack includes clean samples, an adversarial attack algorithm, and an attack target. It is Gaussian noise; Let x represent the target model. i Let i represent the i-th sample. Indicates the category of the attack target;

[0086] The following formula is used to superimpose the results of each iteration. , This refers to the perturbation of the i-th sample. The perturbations of each sample are then superimposed together by an algorithm to finally generate a V.

[0087] ;

[0088] The following formula is used to limit the overall perturbation and optimize its magnitude;

[0089] ; refers to the sign function of V, which in this algorithm means the direction of the perturbation offset.

[0090] ; refers to taking the minimum disturbance amount of V.

[0091] The final generated V is composed of both the direction and amount of the disturbance.

[0092] In the formula, Represents the radius of the projected sphere. This indicates an adversarial attack algorithm;

[0093] S22, Loop step S21; The maximum number of loop iterations is n;

[0094] S23, repeat steps S21 and S22; number of iterations is... ;in, as well as ;in The specified maximum number of loops; and Define a disturbance quantity , It is composed of the sum of the values ​​of the additional perturbations generated after the attack, where, This represents the value of a disturbance. X represents the sum of all perturbations; Xs represents the dataset of the target.

[0095] S24, obtain a list containing all ; and the disturbance V has been concealed;

[0096] S3. Data poisoning: Randomly select clean samples, use the GN-TUAP algorithm to generate a poisoned dataset, and then put the poisoned dataset into the dataset of clean samples, and use it to train the model to generate a poisoned model.

[0097] Specifically, in step S21, the mean and variance of the GN Gaussian noise are controlled, with a mean of 0 and a variance of 0.01; a function with a very small perturbation to interfere with the perturbation direction is generated, with the sample and the adversarial perturbation as inputs. In step S2, P refers to the adversarial attack algorithm, using the DeepFool attack. The clean dataset mentioned in step S2 is selected as the dataset that needs to be contaminated in the backdoor attack; the clean model is selected as the target model of the attack.

[0098] In specific applications:

[0099] 1. Generation of disturbances

[0100] 1.1 Counter-perturbation based on Gaussian distribution;

[0101] Disturbances can occur in any direction, and if left uncontrolled, they can lead to instability. For example... Figure 3 As shown, draw a circle with (x0, y0) as the center and tangent to H(x), and then draw a triangle with H(x) as the hypotenuse that intersects the circle. At this time, any point on the circle is the minimum perturbation from (x0, y0) to the hyperplane H(x).

[0102] The direction from (x0, y0) to the hyperplane H(x) can be arbitrary, and different directions have different perturbation quantities. The orthogonal projection direction from (x0, y0) to the hyperplane H(x) has the smallest perturbation quantity.

[0103] The shaded area S represents the additional perturbation from (x0, y0) to the decision hyperplane. From a probabilistic perspective, the phenomenon that every perturbation is a minimum perturbation is rare in nature, leading to instability. As shown in the diagram, the larger the area of ​​S, the more directions of perturbation occur. Since (x0, y0) and H(x) are fixed, only by reducing the area of ​​S can the instability of the perturbation be controlled.

[0104] To stabilize the direction of the disturbance, noise following a Gaussian distribution is added to stimulate the direction of the disturbance; for example... Figure 4 As shown, μ represents the mean, which controls the axis of symmetry in the image, and σ 2 It represents variance, which controls the amplitude of the image.

[0105] Using the point (x0, y0) as the mean, the perturbation direction is centered on the perturbation. The variance is then controlled to make the perturbation direction as much as possible toward the orthogonal projection direction, so that the generated perturbation reaches the optimal value with the highest probability. Furthermore, each time a different perturbation is generated, each generated perturbation can reach the optimal value with the highest probability.

[0106] Gaussian noise not only follows a Gaussian distribution in amplitude but also a uniform power spectrum. This means the noise is more random at sample distribution points. The advantage of this is that it reduces the occurrence of anomalous sample distributions, making the generated triggers more covert. The following formula is derived:

[0107] ; (7)

[0108] In the formula: It is to utilize Adversarial attack algorithms are used to generate adversarial perturbations for the i-th sample; It is calculated at the i-th sample. Gaussian noise; The perturbation for the i-th sample;

[0109] 1.2 An adversarial perturbation generation algorithm based on Gaussian distribution;

[0110] This invention employs the GN-TUAP algorithm. The goal of the GN-TUAP algorithm is to find a universal perturbation V. Specifically, it selects a clean dataset and a clean model. The clean dataset is the dataset that needs to be contaminated in the backdoor attack; the clean model is the target model for the attack. The i-th sample is iterated over, and then an attack target is selected. The perturbation method is iteratively applied using adversarial attacks, superimposing the perturbation of the i-th sample and the perturbation of the (i+1)-th sample to finally obtain the universal perturbation V; specifically, the following steps are included:

[0111] 1.21, if ;

[0112] The sample perturbation is iterated using the following formula;

[0113] ;

[0114] in, This refers to an adversarial attack algorithm; the input to the adversarial attack includes a clean sample, the adversarial attack algorithm, and the attack target. The DeepFool attack is employed. GN is Gaussian noise; the mean and variance of the GN Gaussian noise are controlled, with a mean of 0 and a variance of 0.01; a function is generated to generate a small perturbation to interfere with the perturbation direction, with the input being the sample and the adversarial perturbation.

[0115] The following formula is used to superimpose the results of each iteration. , This refers to the perturbation of the i-th sample. The perturbations of each sample are then superimposed together by an algorithm to finally generate a V.

[0116] ;

[0117] The following formula is used to limit the overall perturbation and optimize its magnitude;

[0118] ;

[0119] ;

[0120] ;

[0121] In the formula, Represents the radius of the projected sphere; This indicates an adversarial attack algorithm; Indicates the direction of the disturbance; Indicates the minimum disturbance amount; This means that the final generated V is composed of both the direction and amount of the disturbance;

[0122] 1.22, Loop step S21; the maximum number of loop iterations is n;

[0123] 1.23. Loop steps S21 and S22; and satisfy the loop count. , ;in The specified maximum number of loops; and Define a perturbation quantity p, which is the sum of the values ​​of the additional perturbations generated after the attack, where, This represents the value of a disturbance. X represents the sum of all perturbations; Xs represents the dataset of the target.

[0124] 1.24. Obtain a result containing all ; and the disturbance V has been concealed;

[0125] 2. Data poisoning

[0126] The GN-TUAP generated using the above method can be used to demonstrate a simple data poisoning scenario, such as... Figure 5 As shown, first, a portion of clean samples are randomly selected, and GN-TUAP is added to the data samples to generate a portion of the toxic dataset. Then, the toxic dataset is put into the dataset of clean samples and used for model training, which will generate a toxic model.

[0127] For users, such as Figure 6 As shown. When using third-party datasets, users cannot guarantee that the datasets have not been compromised by attackers. Therefore, detection models are generally used to detect the datasets, and only data that passes the security test can be used for model training.

[0128] The above methods were experimentally verified;

[0129] 1. Dataset selection and experimental environment;

[0130] Experiments were conducted using the publicly available CIFAR10, MNIST, and Fashion MNIST datasets. CIFAR10 comprises 50,000 training samples and 10,000 test samples, a 32×32 pixel dataset with 10 categories. MNIST and Fashion MNIST are similar, both consisting of 28×28 pixel grayscale images and 60,000 training samples and 10,000 test samples. MNIST is a collection of handwritten digits from 0 to 9, while Fashion MNIST is a classification dataset for clothing categories. Both datasets contain 10 categories.

[0131] The development environment primarily uses Python 3.6 in Anaconda3, and a framework based on the Tensorflow-GPU version and the Keras library are created for experiments. The operating system environment is the default Ubuntu 20.04.3 LTS; the hardware configuration is an Intel Core i5-6300H and an NVIDIA GeForce GTX 950 graphics card; the experimental model uses a custom network model with a structure similar to VGG16.

[0132] 2. Evaluation Indicators

[0133] To evaluate backdoor attacks on deep neural networks, the following three aspects can typically be considered.

[0134] (1) Effectiveness: The effectiveness of deep neural network backdoor attacks can be measured by the attack success rate (ASR). The model defines ASR by the proportion of any sample x% with a backdoor trigger that is classified as the target class y%, as shown in formula (8). Π means that the value is 1 when both sides of the equal sign are equal and 0 when they are not equal.

[0135] ASR= (f(x%)=y%) (8)

[0136] (2) Stealth: The stealth of backdoor attacks on deep neural networks is a measure of the difference in performance between the deep neural network with the backdoor implanted and the original network when facing benign samples. It can characterize the impact of implanting a backdoor on the performance of deep neural networks.

[0137] (3) Resistance to detection: The resistance to detection of backdoor attacks generally refers to whether the triggers embedded in the sample are easily identified by humans or detected by defense algorithms. Activation clustering is used to detect whether a backdoor is contained. In backdoor detection, three metrics are used to evaluate the backdoor situation: precision, recall, and F1 score.

[0138]

[0139] In the formula: TP is determined as positive, FP as negative, TN as positive, and FN as negative. In the anti-detection experiment, all input samples are toxic samples. Generally speaking, the lower the value of these three indicators, the stronger the anti-detection ability of the toxic sample, and the less likely it is to be detected. The determination of activation clustering as a backdoor detection is also based on these indicators, using the same indicators as in the literature Zhang Q, Ding Y, Tian Y, et al. AdvDoor: adversarial backdoor attack of deep learning system[C] / / Proceedings of the 30th ACM SIGSOFT International Symposium on Software Testing and Analysis.2021:127-138. as the threshold (35%). When the detection indicators are less than 35%, the detection model will not mark it as an abnormal type.

[0140] 3. Experimental Procedure and Results

[0141] GN-TUAP is implemented on the existing TUAP published by Zhang Q, Ding Y, Tian Y, et al. AdvDoor: adversarial backdoor attack of deep learning system [C] / / Proceedings of the 30th ACM SIGSOFT International Symposium on Software Testing and Analysis. 2021:127-138. Both are perturbations generated by adversarial attacks based on DeepFool, such as... Figure 7 As shown. In Figure 7(a) to Figure 7 (c) demonstrates that the triggers generated using the GN-TUAP method exhibit a highly uniform noise distribution, better reflecting natural noise, and are also relatively stable across different sample categories. Figure 7 (A) to Figure 7 (C) shows a toxic sample with added GN-TUAP, where the abnormally distributed shape is difficult to discern with the human eye. Figure 7 (d) to Figure 7 (f) shows the triggers generated using TUAP. It's clear that the noise points in the triggers are uneven, the noise looks very unnatural, and there are significant differences in the triggers across different sample categories. Figure 7 (D) to Figure 7 As can be seen in (F), the toxic samples after adding TUAP also have obvious abnormal data distribution, which is very easy to identify even by the human eye.

[0142] Furthermore, we validated our GN-TUAP-enhanced images on the Fashion MNIST and MNIST datasets, such as... Figure 8 As shown. Figure (a) to Figure 8 (f) demonstrates the use of the generated toxic sample. Figure 8 (A) to Figure 8 (F) shows a toxic sample generated using the TUAP method. Overall, the method described in this invention has the advantages of being very stable and stealthy in trigger generation.

[0143] At the same time, we also conducted a large number of experiments on the resistance to detection, and used the defense detection method based on activation clustering to detect backdoors in the toxic samples generated by the method described in this invention.

[0144] To facilitate comparison with similar research, the same 10 pairs of samples were used in CIFAR10 as in the literature Zhang Q, Ding Y, Tian Y, et al. AdvDoor: adversarial backdoor attack of deep learning system[C] / / Proceedings of the 30th ACM SIGSOFT International Symposium on Software Testing and Analysis.2021:127-138. The specific data are shown in Table 1.

[0145]

[0146] We compared the resistance to detection in the defense model using TUAP-based and patch-based triggers. In the defense model detection, the highest detection rate for toxic samples generated using the GN-TUAP method was for identifying the Deer category as the Bird category, with detection metrics of Precision, Recall, and F1 at 31.53%, 22.93%, and 26.55%, respectively. However, using the TUAP method, the detection rates for toxic samples in three categories all exceeded our highest metric: Airplane identified as Deer, Bird identified as Dog, and Deer identified as Bird. Almost all toxic samples generated using the patch method were detected by the defense model.

[0147] exist Figure 9 , Figure 10 , Figure 11 It is clearly visible that the GN-TUAP method detected six outliers, but all were within the normal range (outlier rate less than 35%), resulting in an average detection rate of only 7.4% for the three detection metrics. In contrast, the TUAP method detected three outliers, all of which were labeled as backdoor models, achieving an average detection rate of 20.53%. Furthermore, the patch method detected all outliers as backdoor models, achieving an average detection rate as high as 91.01%. Figure 12 As can be seen, the method described in this invention improves the detection resistance by approximately 2.77 times compared to the TUAP trigger and by approximately 12.29 times compared to the patch trigger.

[0148] Experimental conclusions

[0149] Since different datasets produce different perturbations, this invention aims to find a stable and covert generation method that can produce good stability even when generating triggers on different datasets. Figure 7 and Figure 8 It can be seen that the perturbation distribution of toxic samples using GN-TUAP is significantly better than that of TUAP, and the stability is also greatly improved. Multiple experiments have shown that the perturbation distribution generated by the method described in this invention is relatively stable, while the perturbation generated by TUAP varies greatly. In terms of resistance to detection, GN-TUAP and TUAP are significantly higher than patch. The patch-based mode does not take into account the data distribution and applicable scenarios of the dataset when selecting triggers, so it cannot escape detection models based on abnormal data activation, such as activation clustering. Therefore, the TUAP-based method is significantly better than the patch-based method in terms of resistance to detection. GN-TUAP has the advantage of more stable and uniform data distribution, so it has better resistance to detection than TUAP.

[0150] In summary, the improved anti-disturbance backdoor attack method described in this invention can improve the stability and concealment of the disturbance in terms of trigger generation, and can easily evade human visual recognition systems.

[0151] Secondly, regarding resistance to detection, all patch-based methods were marked as poisoned models, and 30% of TUAP models were also marked as poisoned models, while our GN-TUAP model evaded detection completely, achieving a 100% pass rate. The improved anti-perturbation backdoor attack method described in this invention not only reduces data distribution anomalies and improves the stability of perturbation generation, but also significantly enhances resistance to detection.

Claims

1. An improved method for countering perturbation backdoor attacks, characterized in that, Includes the following steps: S1. Establish a planar coordinate system. Draw a circle tangent to H(x) with (x0, y0) as the center. Then draw a triangle with H(x) as the hypotenuse that intersects the circle. At this time, any point on the circle is the minimum perturbation from (x0, y0) to the hyperplane H(x). The direction from (x0, y0) to the hyperplane H(x) can be arbitrary, and different directions have different magnitudes of perturbation. The orthogonal projection direction from (x0, y0) to the hyperplane H(x) is the smallest perturbation. Where H(x)=ω T x+b; where ω T It is a row vector representing the model weights, x is a column vector of input features, and b is the bias. Noise with a Gaussian distribution is added to stimulate the perturbation direction; the point (x0, y0) is used as the mean to make the perturbation direction centered on the perturbation, and the variance is controlled to make the perturbation direction as much as possible towards the orthogonal projection direction, so that the generated perturbation reaches the optimal value with the highest probability, and each time a different perturbation is generated, each generated perturbation can reach the optimal value with the highest probability. The following formula is obtained: ； In the formula: It is to utilize Adversarial attack algorithms are used to generate adversarial perturbations for the i-th sample; It is calculated at the i-th sample. Gaussian noise; The perturbation is for the i-th sample; the sample is an image; S2. Generate a universal perturbation V using the GN-TUAP algorithm; select a clean dataset and a clean model, iterate through the i-th sample, and then select an attack target. The perturbation method is iteratively applied using adversarial attacks, superimposing the perturbation of the i-th sample and the perturbation of the (i+1)-th sample to finally obtain the universal perturbation V; specifically, the following steps are included: S21, if ; The sample perturbation is iterated using the following formula; ； Where P refers to the adversarial attack algorithm; the input to the adversarial attack includes a clean sample, the adversarial attack algorithm, and the attack target. It is Gaussian noise; Let x represent the target model. i Let i represent the i-th sample. Indicates the category of the attack target; The following formula is used to superimpose the results of each iteration. , This refers to the perturbation of the i-th sample. Then, the perturbations of each sample are superimposed together by an algorithm to finally generate a perturbation V. ； The following formula is used to limit the overall perturbation and optimize its magnitude; ； ； ； In the formula, Represents the radius of the projected sphere; This indicates an adversarial attack algorithm; Indicates the direction of the disturbance; Indicates the minimum disturbance amount; This means that the final generated V is composed of both the direction and amount of the disturbance; S22, Loop step S21; The maximum number of loop iterations is n; S23, loop steps S21 and S22; and satisfy the loop count requirement. , And satisfy ;in The specified maximum number of loops; and Define a disturbance quantity , It is composed of the sum of the values ​​of the additional perturbations generated after the attack, where, This represents the value of a disturbance. Xs represents the sum of all perturbations; Xs represents the dataset of the target. S24, obtain a list containing all ; and the disturbance V has been concealed; S3. Data poisoning: Randomly select clean samples, use the GN-TUAP algorithm to generate a poisoned dataset, and then put the poisoned dataset into the dataset of clean samples, and use it to train the model to generate a poisoned model.

2. An improved method for countering disturbance backdoor attacks as described in claim 1, characterized in that: In step S21, the mean and variance of the GN Gaussian noise are controlled, with a mean of 0 and a variance of 0.01; a function with a very small perturbation amount to interfere with the perturbation direction is generated, with the input being the sample and the adversarial perturbation.

3. An improved method for countering disturbance backdoor attacks according to claim 2, characterized in that: In step S2, P refers to the adversarial attack algorithm, which employs the DeepFool attack.

4. An improved method for countering disturbance backdoor attacks according to claim 3, characterized in that: In step S2, the clean dataset is selected as the dataset that needs to be contaminated in the backdoor attack; the clean model is selected as the target model of the attack.

Images