Deep neural network model back door defense method based on frequency domain disturbance

Abstract

The invention discloses a deep neural network model back door defense method based on frequency domain disturbance. According to the method, frequency domain disturbance and lightweight retraining are combined, a retraining sample is constructed through frequency domain disturbance, and a back door in a suspicious model is removed through lightweight retraining. Specifically, a frequency domain variant dataset is generated by applying perturbations to the amplitude spectrum of the image and keeping the phase spectrum unchanged. In combination with a lightweight retraining process, the model is guided to ignore high-frequency-band information possibly containing a backdoor attack trigger, and finally a healing model is obtained. The repaired model can effectively resist backdoor attacks initiated by opponents, and pollution samples containing triggers can be identified as correct categories with high probability. Compared with most related works, the scheme provided by the invention has obvious advantages.

Description

Technical Field

[0001] The present invention relates to the field of computer security, and in particular to the security of deep neural network models, and in particular to a method for defending against backdoor attacks on deep neural network pre-training models. Background Art

[0002] In recent years, deep neural networks (DNNs) have developed rapidly and are widely used in numerous fields, including facial recognition and self-driving cars. However, the resulting security issues have also attracted widespread attention, with backdoor attacks being particularly prominent. A backdoor attack occurs when an attacker intentionally inserts a backdoor during the training of a deep learning model, causing the model to output incorrect outputs under specific conditions. Specifically, the attacker first slightly modifies some samples to embed specific triggers. These samples with embedded triggers are then mixed with normal samples for model training, thereby implanting the backdoor. Furthermore, the attacker can also insert the backdoor by modifying model parameters.

[0003] Backdoor attacks are extremely subtle and dangerous. A model embedded with a backdoor produces normal output when receiving normal input; however, if the input contains a specific trigger pattern, the model will incorrectly output a specific label. For example, in a facial recognition system containing a backdoor, an attacker can simply activate the attack by wearing or holding a physical trigger object (such as sunglasses, a tie, a white belt, a headscarf, earrings, etc.). This attack method allows the attacker to manipulate the model to produce specific erroneous outputs, posing a serious threat to system security.

[0004] In reality, adversaries have numerous opportunities to launch backdoor attacks, and defending against them is not easy. Deep learning model training requires vast amounts of computing resources and training data, which is beyond the reach of most ordinary users. Consequently, many users choose to rely on third-party platforms and / or use third-party datasets of unknown origin to train their models. Or they even directly deploy pre-trained models provided by third parties. This presents ample opportunities for malicious third parties to insert backdoors into users' models. They can do this by contaminating datasets, manipulating the model training process, or even directly modifying model parameters. Because the adversary has already implanted the backdoor before the user or defender has access to the model, they possess a significant advantage over the defender. Furthermore, the inherently poor interpretability of deep learning models makes the backdoors they implant highly concealed. Consequently, defending against backdoor attacks is extremely difficult.

[0005] Academics have proposed a variety of strategies for defending against backdoor attacks, including those based on sample preprocessing, model diagnosis, model reconstruction, and trigger synthesis. However, these strategies have their own limitations and face numerous challenges in practical applications. For example, while reconstruction strategies such as FP (Fine-pruning) can theoretically effectively combat backdoors, in practice they often inadvertently weaken the model's performance on the original task due to excessive weight reduction during model pruning. Trigger synthesis strategies, such as DeepInspect, attempt to reconstruct triggers through reverse engineering, but their execution requires enormous computational resources. Therefore, it is particularly important to develop a universally applicable and efficient method for defending deep neural network models from backdoors. Summary of the Invention

[0006] The technical problem to be solved by the present invention is to propose a new backdoor defense method for deep neural network models in response to the shortcomings of the existing technology.

[0007] The technical solution of the present invention is:

[0008] A backdoor defense method for deep neural network models based on frequency domain perturbation, comprising the following steps:

[0009] (1) Retraining dataset construction: The retraining dataset consists of two parts, namely the original dataset or a subset X of a similar dataset P , and the subset X P The frequency domain variant dataset X constructed based on V ; Frequency domain variant dataset X V The samples in the dataset X P Based on the sample in X P The amplitude spectrum of the sample in is generated after the perturbation is applied;

[0010] (2) Suspect model repair: A lightweight retraining process is used to repair the suspected model M that may contain a backdoor. s Repair; during the repair process, the original dataset or a subset of a similar dataset X P And the frequency domain variant dataset X constructed based on this subset V will be used as training data; in this retraining process, the model is guided to ignore the high-frequency information related to noise and details and that may contain backdoor attack triggers, and finally the healing model M is obtained. H .

[0011] Furthermore, the specific steps of step (1) are as follows:

[0012] (1.1) Data subset X P : Data subset X P From training suspect model Ms The original dataset used when generating the dataset, or another dataset similar to the original dataset;

[0013] (1.2) Frequency domain variant X V : Frequency domain variant X V The samples in the subset X P The specific generation process is as follows:

[0014] (1.2.1) From subset X P Extract amplitude spectrum and phase spectrum information from samples

[0015] Using discrete Fourier transform (DFT), from the subset X P The amplitude spectrum and phase spectrum information are extracted from the sample; in specific implementation, discrete Fourier transform (DFT) can be realized by fast Fourier transform FFT; let subset X P The sample in is denoted as f(x, y), where (x, y) represents the spatial domain coordinates, which are used to describe the spatial position of the image; the size of the sample is W×H, and its image F(u, v) in the two-dimensional frequency domain can be obtained by discrete two-dimensional Fourier transform, as shown below:

[0016]

[0017] Where (u, v) represents the coordinates in the frequency domain, which is used to describe the frequency components of the image; e iθ represents the complex exponential term in Euler's formula, θ is the angle; the amplitude spectrum A(u,v) and phase spectrum φ(u,v) can be obtained from the Fourier transform result F(u,v); the amplitude spectrum can be obtained by solving the modulus of the complex frequency domain value, that is, A(u,v) = |F(u,v)|; the phase spectrum can be obtained by solving the argument of the complex frequency domain value, that is, φ(u,v) = arg(F(u,v));

[0018] (1.2.2) Applying disturbance

[0019] By applying perturbation, a frequency domain variant sample is constructed; the perturbation is applied to the high frequency part of the sample amplitude spectrum A(u, v); by applying perturbation to the sample amplitude spectrum, an adjusted amplitude spectrum A'(u, v) is obtained; the manner of applying perturbation includes but is not limited to adding noise; as an example, the implementation process of applying perturbation by adding noise is as follows:

[0020] First, set the frequency range of the high frequency area [F min , F max ];

[0021] As an optimization, in order to ensure that the noise perturbation mainly affects the high-frequency area, it is recommended to set the high-frequency range to one-quarter to one-half of the frequency domain image size;

[0022] Then, a frequency mask M is constructed. The frequency mask M is used to ensure that only the amplitude values ​​within the specified frequency range are affected by random perturbations. The size of the frequency mask is the same as the size of the frequency domain image obtained by discrete Fourier transform. The frequency mask takes the value of 1 in the aforementioned high-frequency region and takes the value of 0 in other regions. It is defined as follows:

[0023]

[0024] Next, a random perturbation matrix R is constructed; the size of the perturbation matrix is ​​the same as the size of the frequency domain image obtained by discrete Fourier transform; the values ​​of the perturbation matrix elements obey the uniform distribution U(-1,1);

[0025] Finally, construct the adjusted amplitude spectrum A'(u, v); the calculation formula is as follows:

[0026] A'(u,v)=A(u,v)(1+βMR)

[0027] Where β is the disturbance intensity parameter, which is used to control the mixing degree of noise and adjust the influence of noise in the amplitude spectrum;

[0028] (1.2.3) Synthetic frequency domain variant samples

[0029] Given the phase spectrum φ(u, v) and the adjusted amplitude spectrum A'(u, v), the noise perturbation image f'(x, y) can be reconstructed using the inverse discrete Fourier transform (IDFT);

[0030] First, construct the complex spectrum using the amplitude spectrum A'(u, v) and phase spectrum φ(u, v):

[0031] F'(u, v) = A'(u, v)·e iφ(u,v)

[0032] Next, the complex spectrum F'(u, v) is inversely discretely transformed into a two-dimensional Fourier transform to obtain the spatial domain representation of the image:

[0033]

[0034] in Represents the inverse discrete two-dimensional Fourier transform; given a frequency domain image F(u, v) of size W×H, its inverse discrete two-dimensional Fourier transform f'(x, y) is defined as:

[0035]

[0036] Among them, the coefficient Used for normalization to ensure the correct restoration of signal amplitude during the transformation process. It is the conversion factor from frequency domain to time domain in the inverse discrete Fourier transform.

[0037] Furthermore, the specific steps of step (2) are as follows:

[0038] (2.1) Training data: During the restoration process, the original dataset or a subset X of a similar dataset P And the frequency domain variant dataset X constructed based on this subset V will be used as training data;

[0039] (2.2) Retraining process: A lightweight retraining process is used to implement the suspect model M s The repair process for the model repair will be a lightweight retraining process with fewer rounds; the suspect model M as the pre-trained model s It has already achieved relatively ideal convergence in the original dataset; the samples in the dataset used for the retraining process are highly similar to some samples in the original dataset; these similar samples only have a small difference in the perturbation level in the high-frequency part of the amplitude spectrum of their images, and their labels are also consistent; in the retraining process, the input of these samples will only lead to a small fine-tuning of the model parameters, so the retraining process for model repair will converge very quickly; in this retraining process, the model is guided to ignore those high-frequency band information related to noise and details, and which may contain backdoor attack triggers, and finally obtain the healing model M H ;

[0040] (2.3) Transfer learning: As a further optimization, in order to accelerate the model retraining process, we can refer to the transfer learning strategy and fix the suspect model M before the retraining process begins. s The parameters of the nodes in certain layers will reduce the total number of model parameters to be trained; specifically, the initial layer weight parameters of the network are selectively fixed; according to deep learning theory, these shallow layers are mainly responsible for capturing the basic components of the image and some simple low-level features, which are usually less relevant to specific backdoor triggers; by freezing them, it can not only promote the model to quickly adapt to the new training set, but also focus on deepening the model's recognition of high-level semantic features.

[0041] As an example, the specific implementation process of the present invention is demonstrated below in combination with specific scenarios.

[0042] (1) The adversary creates and shares a model; the model created by the adversary may contain a preset backdoor, which is called the suspected model M s ; Use a specific trigger to input the suspected model M containing the backdoor s, the preset backdoor will be activated;

[0043] (2) The user obtains the suspicion model M shared by the adversary s ;

[0044] (3) The user obtains some training samples, perturbs the amplitude spectrum of the training samples, constructs frequency domain perturbation variant samples, adds them to the training data set, and then removes the suspected model M through lightweight retraining. s The possible backdoor in the system is repaired to obtain a healthy model, referred to as the healing model M. h ;

[0045] (3.1) The dataset used for the retraining process consists of two parts, one of which is a random subset X of the original dataset X P , and secondly, with X P Based on the frequency domain variant, module I is constructed t , the frequency domain variant subset X obtained after processing V ;

[0046] (3.2) For each input sample, I t Perturbations are added to the high-frequency part of the sample's amplitude spectrum to generate multiple different frequency domain variant samples and added to the frequency domain variant dataset X. V ;

[0047] (3.2.1) Use discrete Fourier transform to extract the amplitude spectrum A(u,v) and phase spectrum φ(u,v) from the training samples;

[0048] (3.2.2) Apply perturbation to the sample amplitude spectrum A(u, v) to obtain the adjusted amplitude spectrum A'(u, v);

[0049] (3.2.3) Based on the phase spectrum φ(u, v) and the adjusted amplitude spectrum A'(u, v), the image is reconstructed using the inverse Fourier transform (IDFT) to obtain the frequency domain variant sample;

[0050] (3.2.4) Repeat steps (3.2.1) to (3.2.3), and adjust the perturbation amplitude or method in (3.2.2) appropriately to obtain multiple different frequency domain variant samples; add these frequency domain variant samples to the frequency domain variant dataset X V ;

[0051] (3.3) During the restoration process, the subset X of the original dataset P And the frequency domain variant dataset X constructed based on this subset V will be used as training data;

[0052] (3.4) In order to speed up the model retraining process, the transfer learning strategy is introduced, and the suspect model M is fixed before the retraining process begins. s Parameters of nodes in certain layers to reduce the total number of model parameters to be trained.

[0053] (3.5) Remove the suspected model M through lightweight retraining s During the retraining process, the model is guided to ignore the high-frequency information related to noise and details that may contain backdoor attack triggers, and a repaired healthy model is obtained, referred to as the healing model M. h ;

[0054] (4) The healing model M deployed by the user in a specific application scenario h ; The healing model M deployed in the application scenario used by the adversary and ordinary users h .

[0055] Beneficial effects:

[0056] The present invention proposes a backdoor defense method for deep neural network models based on frequency domain perturbation, which combines frequency domain perturbation with lightweight retraining. Retraining samples are constructed through frequency domain perturbation, and backdoors in suspicious models are removed through lightweight retraining. Specifically, a frequency domain variant dataset is generated by perturbing the amplitude spectrum of the image and keeping the phase spectrum unchanged. Combined with the lightweight retraining process, the model is guided to ignore high-frequency band information that may contain backdoor attack triggers, and finally a healing model is obtained. The repaired model can effectively resist backdoor attacks launched by the adversary, and contaminated samples containing triggers can be identified as the correct category with a high probability. Compared with most related works, the proposed scheme has obvious advantages. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] Figure 1 The basic process of deep neural network backdoor attack and defense

[0058] Figure 2 Comparison of the behavior of the suspect model and the healing model

[0059] Figure 3 The overall process of the deep neural network model backdoor attack defense method

[0060] Figure 4 Generation process of frequency domain variant images

[0061] Figure 5 Comparison results of defense performance on the first attack model

[0062] Figure 6 Comparison results of defense performance on the second attack model

[0063] Figure 7 Noise ratio β adjustment and its impact on performance (CIFAR-10 dataset)

[0064] Figure 8 Impact of noise ratio on defense performance (CIFAR-100 and Tiny-ImageNet datasets)

[0065] Figure 9 The impact of sample ratio on defense performance

[0066] Figure 10 The impact of retraining rounds on defense performance

[0067] Figure 11 Comparison of computational overhead of different solutions DETAILED DESCRIPTION

[0068] The specific implementation process of the present invention is as follows:

[0069] 1. Attack Model and Basic Hypothesis

[0070] We define the attack model and the attacker's capabilities through two basic assumptions.

[0071] Hypothesis 1: Assume that an attacker is able to create a model with a backdoor and deliver it to the end user through some means.

[0072] For example, an attacker could use a contaminated dataset to train a model containing a backdoor and publish it online for users to download and use. Alternatively, an attacker could intercept the original, backdoor-free model and, before it reaches the end user, somehow embed a backdoor. Training deep neural networks typically requires a large number of training samples and computing power, both of which are unaffordable for ordinary users. It's common for ordinary users to upload data to third-party platforms for model training, even use questionable datasets provided by third parties, or download pre-trained models that haven't been reviewed or verified. Therefore, Hypothesis 1 is relatively easy to satisfy in reality.

[0073] According to Hypothesis 1, pre-trained models downloaded by users, or models trained on third-party platforms, may be suspected of containing backdoors. Our goal is to provide an effective defense against backdoor attacks by repairing these suspected backdoor models and rendering them ineffective.

[0074] Hypothesis 2: This paper assumes that attackers cannot interfere when users perform backdoor repairs on suspicious models and when the repaired models are deployed and used.

[0075] Hypothesis 2 is also easily met in reality. Once a user downloads a suspicious model locally, they gain control of the local model. As long as the user handles it properly, an attacker cannot modify the user's local model again, whether during the repair phase, deployment phase, or use phase.

[0076] Hypothesis 3: After the repaired model is deployed, an attacker can use samples containing backdoor triggers as input to attempt to launch a backdoor attack on the deployed model.

[0077] II. Overview of Backdoor Attack Defense Methods

[0078] Figure 1 Combined with typical application scenarios, the basic process of backdoor attack and defense of deep neural networks is briefly summarized, which is divided into 5 steps in total. Step 1, the adversary creates and shares a backdoor attack model. In reality, the creation of a backdoor attack model can also be indirect. For example, the adversary contaminates the model training data set, or the adversary invades the model training platform, or the adversary attacks the model hosting platform. Step 2, the user deploys the suspected model to a specific application scenario, such as a face recognition system, a license plate recognition system, and so on. The model obtained by the user may have a backdoor, which is referred to as a suspected model in this invention and is marked as M in the figure. s In step 3, we assume that both the adversary and ordinary users have access to the suspect model M deployed in the application scenario. s Ordinary users input normal samples into the model, while the adversary inputs malicious samples containing triggers into the model. In step 3, the adversary attempts to activate the backdoor in the suspicious model by using the malicious samples containing triggers as input, thereby completing the backdoor attack. In step 4, the backdoor defense solution proposed in this invention is used to implement backdoor defense. Suspect model M deployed in the application scenario s Will be replaced by Healing Model M h The difference between step 5 and step 3 is that the adversary and normal users access the healing model M h , rather than the suspect model M s .

[0079] Figure 2 Right Figure 1 The behavior of the suspect model in step 3 and Figure 1 The behaviors of the healing models in step 5 are demonstrated separately. s , the trigger in the malicious sample provided by the adversary will activate the backdoor embedded in the model, and the attack will be successful. Figure 2 As shown in the first part (left sub-figure), for the sample containing the trigger (frog), the suspect model M s The output is the wrong result (bird) preset by the adversary. The normal samples provided by normal users will not activate the backdoor of the model. The suspected model Ms Still following the normal processing logic to output the correct result (frog), this is also the hiddenness of the backdoor attack. h , the backdoor contained in it has been removed in step 4. Although the sample input by the attacker still embeds the backdoor trigger, for the healing model M with the backdoor removed h , the trigger does not work and the backdoor attack fails. Figure 2 As shown in the second part (right subfigure), the adversary input sample still contains the backdoor trigger, but obtains the correct result (frog).

[0080] Figure 3 This paper describes the overall process of the deep neural network model backdoor attack defense method proposed in this paper. s The possible backdoor in the final healing model M h Specifically, we will analyze the suspect model M s Implement a specific lightweight retraining process to remove M s Possible backdoors in the system, and a repaired healing model M is obtained. h The dataset used for the retraining process consists of two parts, one of which is a random subset X of the original dataset X. P , and secondly, with X P Based on the frequency domain variant, module I is constructed t , the frequency domain variant subset X obtained after processing V For each input sample, I t Different disturbances are added to the low-level information of the sample (random noise is added to the high-frequency part of the amplitude spectrum) to generate N different frequency domain variant samples (for example, N=4).

[0081] 3. Construction of retraining dataset

[0082] The retraining dataset consists of two parts, namely the original dataset or a subset X of a similar dataset P , and the frequency domain variant X constructed based on this subset V .

[0083] 1. Subset data X P

[0084] The present invention assumes that a portion of the original training data has been obtained before the user repairs the suspect model. This assumption can be met in reality. If the user's own dataset is hosted on a third-party platform for training, the user himself owns the original dataset. If the user downloads a third-party pre-trained model. Pre-trained models trained with public datasets generally indicate the source of the dataset and the download link. Pre-trained models trained with non-public datasets usually provide a sample dataset when shared with the outside world, or provide descriptive information on the composition rules of the training set, so users have the opportunity to obtain a certain amount of original training sets.

[0085] The goal of retraining isn't to restart the entire training process from scratch. Instead, it's to fine-tune model parameters, guiding the model to ignore noise and trigger information hidden in high-frequency bands, thereby improving model robustness. Retraining doesn't require the entire original dataset, but rather a smaller subset. Compared to the original training set, a smaller dataset makes retraining more efficient and reduces computational resources.

[0086] The present invention does not require that the subset X P The samples are all clean samples from the original dataset. The present invention does not use this subset of samples for retraining alone, but performs a series of processing on the samples to generate an additional frequency domain variant dataset X V In fact, even if the data subset X P A certain number of backdoor trigger samples are included. Due to the existence of the frequency domain variant dataset, the retraining process can still guide the healing model to ignore the backdoor trigger information, and the present invention can still achieve the purpose of model repair.

[0087] 2. Frequency Domain Variant X V

[0088] Frequency Domain Variant X V Take subset X P Generate as the basis. Subset X P Each sample of can be decomposed into two parts: the amplitude spectrum and the phase spectrum. The amplitude spectrum represents the amplitude of each frequency component in the frequency domain and is usually used to describe the texture and edge information of the image. The phase spectrum describes the phase information of each frequency component in the frequency domain and is usually used for image restoration and synthesis. When synthesizing frequency domain variant samples, we will perform a multiplication of the subset X. P Apply different degrees of disturbance to the high-frequency part of the amplitude spectrum of the sample.

[0089] Figure 4The generation process of frequency domain variant data is shown, which mainly includes three core steps: (1) extracting amplitude spectrum and phase spectrum information through discrete Fourier transform (DFT); (2) applying perturbations to the high-frequency part of the amplitude spectrum, such as adding noise; (3) synthesizing a new variant image through inverse discrete Fourier transform (IDFT). The synthesized image will be added to the frequency domain variant dataset X V The following are detailed descriptions of each.

[0090] (1) Extracting amplitude spectrum and phase spectrum information

[0091] The amplitude spectrum and phase spectrum information of the image sample can be obtained from the discrete Fourier transform (DFT) result. For a given image f(x,y) of size W×H, its image F(u,v) in the two-dimensional frequency domain can be obtained by discrete Fourier transform, which is defined as follows:

[0092]

[0093] Where (x, y) represents the coordinates in the spatial domain, which is used to describe the spatial position of the image. (u, v) represents the coordinates in the frequency domain, which is used to describe the frequency components of the image. iθ represents the complex exponential term in Euler's formula, θ is an angle. In a specific implementation, the discrete Fourier transform can be implemented by fast Fourier transform FFT.

[0094] The amplitude spectrum A(u,v) and phase spectrum φ(u,v) can be obtained from the Fourier transform result F(u,v). The amplitude spectrum can be obtained by solving the modulus of the complex frequency domain value, that is,

[0095] A(u, v)=|F(u, v)|

[0096] The phase spectrum can be obtained by solving the argument value of the complex frequency domain value, that is,

[0097] φ(u, v)=arg(F(u, v))

[0098] (2) Apply disturbance

[0099] The perturbation used to construct the frequency-domain variant sample is added to the high-frequency portion of the amplitude spectrum. By applying the perturbation to the original amplitude spectrum A(u, v), the adjusted amplitude spectrum A'(u, v) is obtained. Adding noise is one of the simplest ways to implement perturbation. As an example, the implementation process is as follows.

[0100] First, set the frequency range of the high frequency area [F min , F max ]. To ensure that the noise perturbation mainly affects the high-frequency area, it is recommended to set the high-frequency range to one-quarter to one-half of the frequency domain image size.

[0101] Then, a frequency mask M is constructed. The frequency mask M is used to ensure that only the amplitude values ​​within the specified frequency range are affected by the random perturbation. The size of the mask is the same as the size of the frequency domain image obtained by the discrete Fourier transform. The mask takes the value of 1 in the specified high-frequency region and the value of 0 in other regions. It is defined as follows:

[0102]

[0103] Next, construct a random perturbation matrix R. The size of the perturbation matrix is ​​the same as the size of the frequency domain image obtained by discrete Fourier transform. The values ​​of the perturbation matrix elements follow the uniform distribution U(-1, 1).

[0104] Finally, construct the adjusted amplitude spectrum A'(u, v). The calculation formula is as follows:

[0105] A'(u, v) = A(u, v)(1+βMR)

[0106] Where β is the disturbance intensity parameter, which is used to control the mixing degree of noise and adjust the influence of noise in the amplitude spectrum.

[0107] (3) Synthesize frequency domain variant samples.

[0108] Given the phase spectrum φ(u, v) and the adjusted amplitude spectrum A'(u, v), the noise-perturbed image can be reconstructed using the Inverse Discrete Fourier Transform (IDFT).

[0109] First, construct the complex spectrum using the amplitude spectrum A'(u, v) and phase spectrum φ(u, v):

[0110] F'(u, v) = A'(u, v)·e iφ(u,v)

[0111] Next, the complex spectrum F'(u, v) is inversely discretely transformed into a two-dimensional Fourier transform to obtain the spatial domain representation of the image:

[0112]

[0113] in Represents the inverse discrete two-dimensional Fourier transform. Given a frequency domain image F(u, v) of size W×H, its inverse discrete two-dimensional Fourier transform f(x, y) is defined as:

[0114]

[0115] Among them, the coefficient Used for normalization to ensure the correct restoration of signal amplitude during the transformation process. It is the conversion factor from frequency domain to time domain in the inverse discrete Fourier transform.

[0116] 4. Repair of Suspect Model

[0117] Suspect Model M s The restoration process is achieved through a lightweight retraining process. During the restoration process, the original dataset or a subset of the similar dataset X P And the frequency domain variant dataset X constructed based on this subset V will be used as training data. In this retraining process, the model is guided to ignore the high-frequency information related to noise and details, which may contain backdoor attack triggers, and finally obtain the healing model M H .

[0118] The retraining process for model repair will be a lightweight retraining process with fewer rounds. The suspect model M is the pre-trained model. s The original dataset itself has already achieved relatively ideal convergence. The samples in the dataset used for retraining are highly similar to some samples in the original dataset. These similar samples differ only in the high-frequency portion of their image amplitude spectra, with minor perturbation levels, and their labels are consistent. During retraining, the input of these samples results in only minor adjustments to the model parameters. Therefore, the retraining process used for model repair converges very quickly.

[0119] At the same time, we optimized the training process, reducing the consumption of computing resources and time cost. In order to speed up the model retraining process, we borrowed the strategy of transfer learning and fixed the suspect model M before the retraining process began. s We will freeze the parameters of nodes in certain layers, reducing the total number of model parameters required for training. Specifically, we will selectively freeze the weight parameters of the network's initial layers. According to deep learning theory, these shallow layers are primarily responsible for capturing the essential components of the image and simple, low-level features, which are typically less relevant to specific backdoor triggers. By freezing these layers, we not only facilitate rapid model adaptation to new training sets, but also focus on deepening the model's recognition of high-level semantic features.

[0120] 5. Analysis of Defense Principles

[0121] This section will analyze the principles of the defense solution proposed in this invention from three perspectives: reducing the attack success rate, ensuring classification accuracy, and achieving model repair.

[0122] 1. The principle of reducing the success rate of attacks.

[0123] To evade detection, triggers are often extremely small and subtle. With the advancement of backdoor attack techniques, triggers have become even more invisible, becoming virtually undetectable in the spatial domain. Despite this, in the frequency domain, the embedding of triggers still causes significant changes, particularly in the high-frequency portion of the amplitude spectrum. For example, many existing methods for generating trigger samples result in high-frequency artifacts.

[0124] The present invention guides the model retraining process through frequency domain variant samples. Compared with the original image, the high-frequency part of the amplitude spectrum of the frequency domain variant sample image has been disturbed. The retraining process will perceive the change in the high-frequency characteristics of the amplitude spectrum, and then update the parameter information related to the high-frequency characteristics of the amplitude spectrum in the model. The features related to the original suspect model and the trigger usually act on the high-frequency part of the amplitude spectrum, and therefore will be destroyed with a high probability during the update process. The retrained model will focus on the new perturbed pattern features rather than the original pattern features related to the trigger. Therefore, the poisoned samples containing the pattern features of the backdoor trigger will not be able to affect the repaired model, and the success rate of the attack will be significantly reduced.

[0125] 2. Principles for ensuring classification accuracy.

[0126] The classification accuracy of the repaired model is another important metric for measuring the performance of a defense solution. This can be further divided into two aspects: the classification accuracy (ACC) of normal samples and the recovery and recognition ability (RA) of poisoned samples. The ACC evaluates the repaired model's ability to correctly classify normal samples, while the RA evaluates its ability to correctly identify the true class of samples containing triggers.

[0127] Next, we analyze the principles by which the present invention's solution ensures the classification accuracy of normal and poisoned samples. According to image processing theory, the amplitude spectrum of an image describes the intensity of each frequency component in the image, corresponding to the basic outline of the image and detailed texture from low frequency to high frequency. The phase spectrum records the relative position and arrangement of these frequency components. The phase spectrum is crucial to the image and is the key to reconstructing the original spatial structure of the image. Even when the amplitude information is partially lost, as long as the high-level semantic information of the phase spectrum is preserved, the human eye can still recognize the main content and objects of the image. In contrast, although the high-frequency information in the amplitude spectrum contains detailed information about the image, it is also more susceptible to noise and subtle changes. The semantic information contained in the phase information provides new perspectives and possibilities for improving the robustness and security of neural networks. During the generation of frequency domain variant samples in the defense solution of the present invention, we keep the phase spectrum of the image unchanged, which is conducive to maintaining the correct classification ability (ACC) of the repaired model for normal samples. At the same time, by perturbing the high-frequency part of the amplitude spectrum in the frequency domain variant sample, the repaired model can reduce its attention to the high-frequency part of the amplitude spectrum and reduce the influence of trigger-related patterns, which will improve the recovery and recognition ability (RA) of the repaired model for poisoned samples.

[0128] 3. The principle of implementing model repair.

[0129] In the present invention, the repair of the backdoor model is achieved through a retraining process. Next, we analyze from the principle level why the repair goal can be achieved through the retraining method. Traditional neural networks are highly dependent on the amplitude spectrum of the image during training, especially its high-frequency components. This feature facilitates the implementation of backdoor attacks and also provides the possibility of implementing backdoor attack defense through frequency domain processing. As mentioned above, the embedding of the backdoor trigger will cause changes in the high-frequency part of the amplitude spectrum, and this change will be captured by the backdoor model during the training process of the backdoor model. In the backdoor defense scheme of the present invention, by applying perturbations to the high-frequency part of the image amplitude spectrum, the model can be guided to pay attention to the changes in the characteristics of this high-frequency part during retraining and learn from it. The influence of the trigger-related pattern characteristics associated with the high-frequency part will be alleviated or even eliminated during the retraining process.

[0130] 6. Analysis of the Principles of Lightweight Heavy Training

[0131] The retraining process in this invention is remarkably lightweight, which not only reduces computational overhead but also improves defense efficiency. We will analyze the theoretical basis for this lightweight feature from three perspectives: model, sample, and optimization principle.

[0132] 1. From the model perspective

[0133] The original backdoor model itself is already in a converged state. The retraining process is essentially a fine-tuning of the converged model rather than starting from scratch. Therefore, the retraining process converges faster.

[0134] 2. From the perspective of samples

[0135] Because the perturbations applied when constructing the retraining samples are small, the samples used in the retraining process are highly similar to the original model training samples. Furthermore, the number of retraining samples is small, resulting in lower training overhead.

[0136] 3. From the perspective of optimization principles

[0137] According to deep learning theory, nodes near the input layer are primarily responsible for capturing basic image components and simple, low-level features, which are typically less relevant to specific backdoor triggers. Our goal is to deepen the model's ability to discern high-level semantic features during retraining. Freezing neurons associated with these shallow features facilitates rapid retraining adaptation to new training sets, allowing it to better focus on our objectives and reducing computational resource consumption.

[0138] VII. Experimental Evaluation

[0139] This section presents a comprehensive experimental evaluation of our backdoor defense scheme.

[0140] 1. Experimental Setup

[0141] (1) Dataset and main evaluation indicators

[0142] We will use ResNet-18 as the base model to train the backdoor attack model on three benchmark datasets (i.e., CIFAR-10, CIFAR-100, and Tiny ImageNet) for subsequent backdoor defense scheme performance evaluation.

[0143] We use three metrics to evaluate the performance of defense schemes: normal sample classification accuracy (ACC) (i.e., the prediction accuracy of normal / clean samples), attack success rate (ASR) (i.e., the prediction accuracy of poisoned samples for the target class), and robust accuracy (RA) (i.e., the prediction accuracy of poisoned samples for the original class). Generally speaking, the principle is that ASR + RA <= 1. For an ideal defense scheme, the ASR value should be as small as possible, while the ACC and RA values ​​should be as large as possible. The smaller the ASR and the larger the RA, the better the defense performance; the larger the ACC, the less impact the defense scheme has on the prediction of clean samples.

[0144] (2) Backdoor attack methods and settings

[0145] This section will select six representative backdoor attack scenarios and construct backdoor attack models for each to evaluate the performance of the backdoor defense solutions. These six backdoor attack scenarios are BadNets (2017), TrojanNN (2018), Blended (2017), Sinusoidal Signal Backdoor Attack (SIG) (2019), Specific Sample Backdoor Attack (ISSBA) (2020), and Image Distortion-Based Backdoor Attack (WaNet) (2021). Based on the level of defense difficulty, the six attack models are divided into two groups: BadNets, TrojanNN, and WaNet in one group, and ISSBA, SIG, and Blended in the other.

[0146] During the training of all backdoor attack models, we set the labels of backdoor trigger samples to 0, the sample backdoor injection rate to 0.1, the learning rate to 0.01, and the optimizer to stochastic descent gradient SGD. On the CIFAR-10 dataset containing backdoor triggers, the number of training epochs was set to 100. On the CIFAR-100 and ImageNet subsets containing backdoor triggers, the number of training epochs was set to 200.

[0147] (3) Backdoor defense methods and settings

[0148] This section compares the performance of four representative backdoor defense schemes: standard fine-tuning (FT), fine pruning (FP), pattern chaining repair (MCR), and neural attention distillation (NAD). We follow the original configurations and recommendations for these schemes. For FP, we pruned the last convolutional layer of the model. For MCR, we trained the backdoor model for 100 epochs using the loss curve as the endpoint and evaluated the model's defense performance on the loss curve. For NAD, the student model was trained for 10 epochs under the guidance of the teacher model.

[0149] To ensure fair evaluation and comparison, each solution was tested under the same setup. We used the same retraining rounds for all defenses. When clean samples were required for training or fine-tuning, we used 2% of the training dataset. Furthermore, data augmentation (including random cropping and random flipping) was applied to all defenses.

[0150] 2. Defense Performance Comparison

[0151] This section compares the performance of our proposed defense scheme against four other defense schemes based on six different backdoor attack models. In this experiment, we set the noise ratio to 0.1. In subsequent experiments, we will analyze the impact of varying the noise ratio on performance.

[0152] Figure 5 and Figure 6 They are the comparison results of the defense performance on these two groups of attack models. Tables 1 and 2 involve three data sets: CIFAR-10, CIFAR-100 and Tiny ImageNet (column 1). Six different types of backdoor attack models are constructed on each data set (column 2). Column 3 (No Defense) is the performance result of each backdoor attack model when no defense scheme is applied, which only includes two indicators, ACC and ASR, and does not involve RA. Columns 4 to 8 contain three indicators, ACC, ASR and RA, which are the performance results of the schemes after applying different types of defense schemes, respectively. Column 8 (ours) is the performance result of the scheme proposed in this invention. AVG1 and AVG2 are the average performance results of different defense schemes on the two groups of backdoor attack models.

[0153] according to Figure 5 , different defense schemes achieved good defense effects on the three attack models of Group 1 (i.e., BadNets, TrojanNN, and WaNet), and the scheme of the present invention achieved better performance than other schemes. Taking the CIFAR-10 dataset as an example, according to the avg1 line (row 6), the performance of the scheme of the present invention is the best, where ASR drops from 0.91 (column 4) to 0.03 (column 18), and RA reaches 0.87 (column 19), which shows that the defense effect of the scheme of the present invention is significant. At the same time, ACC is 0.92 (column 17), which is slightly lower than 0.97 (column 3) when No Defense is used. This shows that the implementation of the defense scheme usually has a small negative impact on the prediction accuracy of clean samples, but the accuracy is still very high.

[0154] according to Figure 6 , under the default parameters, the solution of the present invention did not achieve better performance than other defense solutions on the three attack models of Group 2 (i.e., SIG, ISSBA, and Blended). At the same time, it is not difficult to find that the defense effects of all defense methods on Group 2 are not satisfactory. Taking ASR as an example, on the CIFAR-10 and CIFAR-100 datasets, FP achieved the best ASR values ​​(9th column), which are 0.20 and 0.36 respectively. On the TinyImageNet dataset, NAD achieved the best ASR value (0.69, 15th column). However, these three ASR values ​​are very large, which means that the probability of the adversary's attack success is still very high, so the defense effect is not good.

[0155] 3. Impact of noise ratio on defense performance

[0156] This section examines the impact of the noise ratio parameter on the defense performance of our proposed solution. Previous experimental results show that our proposed solution, when using the default noise ratio parameter, performs poorly against three backdoor attack schemes in group 2 (ISSBA, SIG, and Blended), so we will focus on group 2. Furthermore, our proposed solution's performance on the CIFAR-10 dataset differs significantly from that on the CIFAR-100 and Tiny-ImageNet datasets, so we will examine these separately.

[0157] (1) CIFAR-10 dataset

[0158] According to the previous experimental results, the solution of the present invention has a good defense effect on ISSBA and SIG on the CIFAR-10 dataset. Therefore, we only conduct further analysis on Blended to explore the impact of the noise ratio parameter in the solution of the present invention on the defense effect. We still follow the other experimental settings in the previous article. Figure 7 From the results, it is not difficult to find that as the noise ratio increases, ACC decreases very slowly, while ASR decreases and RA increases significantly faster. For example, when the noise ratio increases to 0.8, ASR decreases to 0.2, while RA increases to 0.63, and the ACC value changes very little.

[0159] Experimental results show that properly adjusting the noise ratio parameter can significantly improve the defense performance of the proposed solution. By setting an appropriate noise ratio parameter for the proposed defense solution, it is possible to effectively repair the Blended backdoor attack model on the CIFAR-10 dataset.

[0160] (2) CIFAR-100 and Tiny-ImageNet datasets

[0161] According to the previous experimental results, the proposed solution has poor defense performance against the three attack models of ISSBA, SIG, and Blended on the CIFAR-100 and Tiny-ImageNet datasets. In this section, we will re-evaluate the impact of the noise ratio parameter on the defense performance of the proposed solution in the three attack models. The evaluation results are as follows: Figure 8 shown.

[0162] according to Figure 8 It is not difficult to find that from the perspective of average value, when the noise ratio parameter increases to 1, the ASR value on the CIFAR-100 dataset drops from the original 0.74 to 0.14, which is a very large drop. This means that the probability of the adversary's attack success is greatly reduced and the defense effect is significantly improved. At the same time, the RA value also increases from the original 0.11 to 0.34. Figure 6 The RA values ​​of other defense solutions in these three backdoor attack models are not difficult to find, which is a very significant improvement. Of course, this performance improvement also has a certain price, and the ACC value has also decreased to a certain extent. Figure 8 The performance results for different attack models show that increasing the noise ratio brings different improvements to different models, with the most significant improvement for Blended. The performance improvement on the Tiny-ImageNet dataset is similar to that on the CIFAR-100 dataset.

[0163] Experimental results show that by setting a suitable value for the noise ratio parameter, the defense scheme of the present invention can, to a certain extent, effectively repair the three backdoor attack models of ISSBA, SIG, and Blended on the CIFAR-100 and Tiny-ImageNet datasets.

[0164] 4. Impact of Sample Ratio on Defense Performance

[0165] This section uses the BadNets backdoor attack model on the CIFAR-10 dataset as an example to evaluate the impact of sample ratio changes on the defense performance of the proposed solution. We evaluate the impact of sample ratio on defense performance by increasing the proportion of original samples used in the retraining process (2%, 6%, 8%, 10%). The experimental results are shown in Figure 2. Figure 9 shown.

[0166] Experimental results show that as the proportion of the original dataset used increases, the adversary attack success rate (ASR) decreases, while the model accuracy (ACC) and recovery accuracy (RA) increase modestly. If sample acquisition capabilities and computing resources permit, the defensive performance of the proposed defense scheme can be further enhanced by increasing the number of samples used in the retraining process, but the increase is not particularly significant.

[0167] 5. Impact of Retraining Rounds on Defense Performance

[0168] The number of retraining rounds directly determines the computational overhead of the defense scheme of the present invention. To this end, we analyze how the defense performance of the present invention changes with the increase in the number of retraining rounds. The experiments in this section are based on CIFAR10. The model was retrained for 100 rounds using the present invention, and the accuracy (ACC), attack success rate (ASR), and RA data after each round of training were recorded. The results are shown in Figure 2. Figure 10As shown in the figure, the performance curves for all models show little change after 30 retraining rounds, so the horizontal axis in the figure only includes the first 30 rounds. Experimental results show that for all backdoor attack models, in most cases, our defense solution only requires two to three retraining rounds before the performance curve converges to a relatively stable value, achieving good performance.

[0169] Experimental results show that a small amount of retraining can significantly enhance the model's resistance to backdoor attacks, which means that the solution of the present invention is significantly lightweight in terms of computational overhead.

[0170] 6. Computational Cost Comparison

[0171] In order to more intuitively experience the lightweight computing features of the solution of the present invention, this section uses the BadNet backdoor attack model constructed in the CIFAR-10 dataset as a basis to record the implementation time required for different defense methods. Figure 11 The time unit is minutes. The basic information of the experimental platform is as follows: Intel Xeon(R) Gold 6226R @ 2.9GHz*32 processor, 64GB of memory, GeForce GTX 3090 graphics card, Python 3.7, and PyTorch 1.11.0. The results show that compared with several other defense solutions, the proposed solution has a significant advantage in computational overhead.

Claims

1. A backdoor defense method for deep neural network models based on frequency domain perturbation, characterized in that: The following steps are involved: (1) Retraining dataset construction: The retraining dataset consists of two parts, namely the original dataset or a subset X of a similar dataset P , and the subset X P The frequency domain variant dataset X constructed based on V ; Frequency domain variant dataset X V The samples in the dataset X P Based on the sample in X P The amplitude spectrum of the sample in is generated after the perturbation is applied; (2) Suspect model repair: A lightweight retraining process is used to repair the suspected model M that may contain a backdoor. s Repair; during the repair process, the original dataset or a subset of a similar dataset X P And the frequency domain variant dataset X constructed based on this subset V will be used as training data; in this retraining process, the model is guided to ignore the high-frequency information related to noise and details and that may contain backdoor attack triggers, and finally the healing model M is obtained. H .

2. The backdoor defense method for deep neural network models based on frequency domain perturbation according to claim 1 is characterized in that: The specific steps of step (1) are as follows: (1.1) Data subset X P : Data subset X P From training suspect model M s The original dataset used when generating the dataset, or another dataset similar to the original dataset; (1.2) Frequency domain variant X V : Frequency domain variant X V The samples in the subset X P The specific generation process is as follows: (1.2.1) From subset X P Extract amplitude spectrum and phase spectrum information from samples Using discrete Fourier transform (DFT), from the subset X P Extract amplitude spectrum and phase spectrum information from the sample; In specific implementation, discrete Fourier transform (DFT) can be realized by fast Fourier transform FFT; let subset X P The sample in is denoted as f(x, y), where (x, y) represents the spatial domain coordinates, which are used to describe the spatial position of the image; the size of the sample is W×H, and its image F(u, v) in the two-dimensional frequency domain can be obtained by discrete two-dimensional Fourier transform, as shown below: Where (u, v) represents the coordinates in the frequency domain, which is used to describe the frequency components of the image; e iθ represents the complex exponential term in Euler's formula, θ is the angle; the amplitude spectrum A(u, v) and phase spectrum φ(u, v) can be obtained from the Fourier transform result F(u, v); the amplitude spectrum can be obtained by solving the modulus of the complex frequency domain value, that is, A(u, v) = |F(u, v)|; the phase spectrum can be obtained by solving the argument of the complex frequency domain value, that is, φ(u, v) = arg(F(u, v)); (1.2.2) Applying disturbance By applying perturbation, a frequency domain variant sample is constructed; the perturbation is applied to the high frequency part of the sample amplitude spectrum A(u, v); by applying perturbation to the sample amplitude spectrum, an adjusted amplitude spectrum A'(u, v) is obtained; the manner of applying perturbation includes but is not limited to adding noise; as an example, the implementation process of applying perturbation by adding noise is as follows: First, set the frequency range of the high frequency area [F min , F max ]; Then, a frequency mask M is constructed; the frequency mask M is used to ensure that only the amplitude values ​​within the specified frequency range are affected by the random perturbation; The size of the frequency mask is the same as the size of the frequency domain image obtained by discrete Fourier transform. The frequency mask takes the value of 1 in the high-frequency region specified above and takes the value of 0 in other regions. It is defined as follows: Next, a random perturbation matrix R is constructed: the size of the perturbation matrix is ​​the same as the size of the frequency domain image obtained by discrete Fourier transform; the values ​​of the perturbation matrix elements obey the uniform distribution U(-1,1); Finally, construct the adjusted amplitude spectrum A'(u, v); the calculation formula is as follows: A'(u, v) = A(u, v)(1+βMR) Where β is the disturbance intensity parameter, which is used to control the mixing degree of noise and adjust the influence of noise in the amplitude spectrum; (1.2.3) Synthetic frequency domain variant samples Given the phase spectrum φ(u, v) and the adjusted amplitude spectrum A'(u, v), the noise perturbation image f'(x, y) can be reconstructed using the inverse discrete Fourier transform (IDFT); First, construct the complex spectrum using the amplitude spectrum A'(u, v) and phase spectrum φ(u, v): F’(u,v)=A'(u,v)·e iφ(u,v) Next, the complex spectrum F'(u, v) is inversely discretely transformed into a two-dimensional Fourier transform to obtain the spatial domain representation of the image: in Represents the inverse discrete two-dimensional Fourier transform; given a frequency domain image F(u, v) of size W×H, its inverse discrete two-dimensional Fourier transform f'(x, y) is defined as: Among them, the coefficient Used for normalization to ensure the correct restoration of signal amplitude during the transformation process. It is the conversion factor from frequency domain to time domain in the inverse discrete Fourier transform.

3. The backdoor defense method for deep neural network models based on frequency domain perturbation according to claim 1 is characterized in that: The specific steps of step (2) are as follows: (2.1) Training data: During the restoration process, the original dataset or a subset X of a similar dataset P And the frequency domain variant dataset X constructed based on this subset V will be used as training data; (2.2) Retraining process: A lightweight retraining process is used to implement the suspect model M s The repair process for the model repair will be a lightweight retraining process with fewer rounds; the suspect model M as the pre-trained model s It has already achieved relatively ideal convergence in the original dataset; and the samples in the dataset used for the retraining process are highly similar to some samples in the original dataset; These similar samples only have a small difference in the perturbation level in the high-frequency part of their image amplitude spectrum, and their labels are also consistent; during the retraining process, the input of these samples will only lead to a small fine-tuning of the model parameters. Therefore, the retraining process for model repair will converge very quickly; during this retraining process, the model is guided to ignore the high-frequency band information related to noise and details and that may contain backdoor attack triggers, and finally the healing model M is obtained. H ; (2.3) Transfer learning: As a further optimization, in order to accelerate the model retraining process, we can refer to the transfer learning strategy and fix the suspect model M before the retraining process begins. s The parameters of the nodes in certain layers will reduce the total number of model parameters to be trained; specifically, the initial layer weight parameters of the network are selectively fixed; according to deep learning theory, these shallow layers are mainly responsible for capturing the basic components of the image and some simple low-level features, which are usually less relevant to specific backdoor triggers; by freezing them, it can not only promote the model to quickly adapt to the new training set, but also focus on deepening the model's recognition of high-level semantic features.

Images