A face recognition diffusion denoising defense method based on attention mechanism

Abstract

The present invention discloses a face recognition diffusion denoising defense method based on an attention mechanism, which relates to the field of artificial intelligence security technology and includes the following steps: obtaining a data set; constructing a face recognition diffusion denoising network, including a face recognition model, a conditional denoising diffusion model, and an attention model; constructing a total loss based on attention constraints and image quality constraints, and training the face recognition diffusion denoising network to obtain a face recognition diffusion denoising model for denoising adversarial samples. During the training process, the present invention combines image quality constraints and attention constraints, and utilizes an attention mechanism to capture the difference in focus points between clean samples and denoised samples of the face recognition model. Guided by this, the conditional denoising diffusion model learns to restore key semantic information in clean images, ensuring that denoised samples have high visual quality and recognition accuracy. In the denoising stage, the optimal diffusion step size is adaptively generated, avoiding the need for repeated adjustments and greatly reducing defense time.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence security technology, and in particular to a face recognition diffusion denoising defense method based on an attention mechanism. Background Art

[0002] Existing GAN-based adversarial defense methods commonly suffer from unstable training processes and are prone to mode collapse, failing to meet the performance and reliability requirements for practical applications. Training a GAN is a complex process, involving a dynamic game between two neural networks: the generator and the discriminator. This game requires a proper balance between the two during training. However, maintaining this balance in practice is challenging, leading to unstable training. Unbalanced training can prevent the generator from learning the true distribution of the data, resulting in low-quality or repetitive samples.

[0003] Existing adversarial sample defense methods based on diffusion models require multiple adjustments to the noise intensity for different datasets to achieve the optimal defense denoising effect. This process is not only complex but also extremely time-consuming because the ideal noise intensity is not static, but is highly dependent on the specific dataset characteristics, the form of the adversarial attack, and the desired defense effect.

[0004] Existing adversarial example denoising methods generally adopt a blanket strategy, applying uniform denoising to the entire image. However, this approach overlooks a key characteristic of adversarial attacks: attacks are often concentrated in specific regions of the image rather than uniformly distributed across the entire image. These key attack areas are often carefully selected by the attacker because they have a decisive influence on the model's final classification decision. Therefore, when the denoising process fails to specifically identify and address these key areas, the overall defense performance is limited.

[0005] LPIPS loss (Learned Perceptual Image Patch Similarity) is a method for measuring image similarity. LPIPS uses pre-trained deep networks (such as VGG and AlexNet) to extract multi-layer features of two images, and then calculates the distance between the corresponding layer features of the two images to evaluate the perceptual similarity between the images.

[0006] The Frechet Inception Distance score (FID) is a metric used to evaluate the performance of generative models, particularly image generation models. It measures the difference between the distribution of generated images and the distribution of real images, typically by comparing features extracted using a pre-trained neural network (typically Inception v3). Generally speaking, a lower FID score indicates a closer distribution of generated images to real images, indicating better generative performance. Summary of the Invention

[0007] The purpose of the present invention is to provide a face recognition diffusion denoising defense method based on the attention mechanism to solve the problems of unstable model training, long defense time and decreased recognition rate of clean samples after defense.

[0008] In order to achieve the above-mentioned object, the technical solution adopted by the present invention is as follows: a face recognition diffusion denoising defense method based on attention mechanism, comprising the following steps;

[0009] S1, obtain the data set X, including the clean sample set X clean and adversarial sample set X adv , X clean Clean sample x clean and X adv The adversarial samples x0 correspond one to one;

[0010] S2, constructing a face recognition diffusion denoising network, including a pre-trained face recognition model, a conditional denoising diffusion model, and an attention model;

[0011] The face recognition model includes a feature extraction network for extracting features from the input image to obtain deep features;

[0012] The conditional denoising diffusion model includes a forward denoising process and a reverse generation process;

[0013] The forward denoising process is used to add noise to the adversarial sample x0 time step by time step to generate a noisy image corresponding to each time step, where the noisy image at time step t is x t , 1≤t≤T, T is the optimal diffusion step length;

[0014] The reverse generation process is based on conditional guidance, starting from x T Remove noise step by step, and generate denoised image at time step t , the final denoised image As the denoised sample, the According to the following formula:

[0015] ,

[0016] Where, 、 、 are the mean function, variance function, and standard deviation function of time step t respectively; is the deep feature extracted by the feature extraction network, λ is a hyperparameter, for norm, ϵ is random noise, is a standard normal distribution;

[0017] The attention model is used to extract denoised samples and clean sample x clean Extract the corresponding attention map 、 ;

[0018] S3, construct the total loss L of the face recognition diffusion denoising network total ;

[0019] ,

[0020] ,

[0021] Where, L attention is the attention loss; L1 is the L1 loss between the adversarial sample and the denoised sample, L lpips is the LPIPS loss between adversarial samples and denoised samples, To calculate the mean square error, λ1 and λ2 are the first weight hyperparameter and the second weight hyperparameter respectively;

[0022] S4, based on the optimal diffusion step size T, uses the data set X to minimize L total Train the face recognition diffusion denoising network until convergence to obtain the face recognition diffusion denoising model;

[0023] S5: Obtain the adversarial sample to be purified and generate the corresponding denoised sample based on the face recognition diffusion denoising model.

[0024] Preferably, the attention model uses a gradient integration method to extract an attention map, specifically including steps Sa1 to Sa3;

[0025] Sa1, select a deep network F and a completely black image as the reference input x baseline , take the denoised sample or clean sample as the actual input x;

[0026] Sa2, the attribution value IG of the i-th pixel in x i (x) is obtained according to the following formula:

[0027] ,

[0028] is the interpolation parameter, is x baseline The RGB value of the i-th pixel, x i is the RGB value of the i-th pixel in x, For deep networks;

[0029] Sa3, replace the RGB value of the i-th pixel in the actual input x with the attribution value of the i-th pixel to obtain the attention map of x.

[0030] As a preference, the L1, L lpips Calculate according to the following formula respectively;

[0031] ,

[0032] Where N is the total number of pixels in the adversarial sample x0, For adversarial examples The nth pixel in For denoising samples The nth pixel in ;

[0033] ,

[0034] Where, L lpips The multi-layer feature extraction network extracts x0 and After calculating the feature map, H l 、W l are the total height and total width of the feature map extracted by the feature extraction network at layer l, w l is the weight of the l-th layer feature extraction network, is the eigenvalue of the feature map of x0 at height h and width w after the feature extraction network of the first layer, for The feature value of the feature map of the l-th layer feature extraction network at height h and width w is 1≤h≤H, 1≤w≤W, is an element-wise multiplication operation, is the L2 norm.

[0035] Preferably, in S4, training the face recognition diffusion denoising network is specifically as follows;

[0036] Preset the iteration round and batch size, input the dataset X into the face recognition diffusion denoising network according to the batch size, and calculate a total loss L for each batch total , and back-propagate to adjust the network parameters of the conditional denoising diffusion model until the iteration round is obtained.

[0037] Preferably, the optimal diffusion step length T is adaptively generated through steps Sb1 to Sb4;

[0038] Sb1, from the clean sample set X clean and adversarial sample set X adv , respectively sample subsets of the same size and ;

[0039] Sb2, set D different step sizes T1~T D , where the dth step length is T d , 1≤d≤D;

[0040] Sb3, let the diffusion step length be T d ,right The adversarial samples are generated by the conditional denoising diffusion model to form a denoising dataset. , and calculate and FID score;

[0041] Sb4, T1~T D Corresponding to D FID scores, the step size with the smallest FID score is taken as the optimal diffusion step size T.

[0042] Compared with the prior art, the advantages of the present invention are:

[0043] (1) The conditional denoising diffusion model of the present invention performs denoising on adversarial samples. In the training process of the conditional denoising diffusion model, the image distance constraint is introduced. The image distance constraint is based on the conditional denoising diffusion model. and This ensures that the distance between the two in feature space is small during the denoising process, thereby preserving the semantic information in the image. By introducing image distance constraints, the conditional denoising diffusion model not only learns how to restore the image during training but also controls the magnitude of the reconstruction error, helping to improve the quality and stability of the denoising results.

[0044] (2) This paper combines the attention mechanism with the conditional denoising diffusion model. By using the attention model's denoised samples and the attention areas of the adversarial samples (i.e., attention maps), the conditional denoising diffusion model is guided to generate denoising results that are closer to clean samples. This method uses the difference between attention maps as a supervisory signal, prompting the conditional denoising diffusion model to learn how to effectively remove noise from the image while retaining key identity feature information and reducing damage to clean areas. This can effectively improve the accuracy of defense and reduce the impact on clean samples.

[0045] (3) Image quality constraints are also introduced, through L1, L lpipsThe L1 loss and LPIPS loss are calculated for the adversarial samples and denoised samples, and the denoised samples are constrained to ensure the visual quality of the denoised image. The L1 loss is used to measure the similarity between the adversarial samples and the denoised samples in the pixel dimension, ensuring that the pixel-level difference between the two is not too large, thereby maintaining the visual consistency of the denoised image. The LPIPS loss is used to measure the similarity between the adversarial samples and the denoised samples at the perceptual level, and further preserves the key structure and detail information of the original image by capturing the differences in high-level semantic features.

[0046] (4) An adaptive diffusion step size selection mechanism is used to shorten the time to obtain the best denoising results. This mechanism can calculate the optimal diffusion step size based on the characteristics of the dataset and the form of the adversarial attack, thereby significantly reducing the time overhead while ensuring the denoising effect.

[0047] (5) The present invention demonstrates significant defensive effects against a variety of adversarial attacks and exhibits excellent generalization capabilities, such as those against FGSM, PGD, and ATA. Its defensive effectiveness can be transferred to other face recognition models, including FaceNet, IR152, and IRSE50, in a plug-and-play manner. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] Figure 1 It is the overall framework diagram of the present invention;

[0049] Figure 2 A comparison chart of defense denoising results of different defense methods;

[0050] Figure 3 A comparison of attention maps with and without the use of the attention mechanism;

[0051] Figure 4 Comparison diagram of attention maps for another group with and without the use of the attention mechanism. DETAILED DESCRIPTION

[0052] The present invention will be further described below with reference to the embodiments and accompanying drawings.

[0053] Example 1: See Figure 1 ,A face recognition diffusion denoising defense method based on attention ,mechanism, includes the following steps;

[0054] S1, obtain the data set X, including the clean sample set X clean and adversarial sample set X adv , X clean Clean sample x clean and X adv The adversarial samples x0 correspond one to one;

[0055] S2, constructing a face recognition diffusion denoising network, including a pre-trained face recognition model, a conditional denoising diffusion model, and an attention model;

[0056] The face recognition model includes a feature extraction network for extracting features from the input image to obtain deep features;

[0057] The conditional denoising diffusion model includes a forward denoising process and a reverse generation process;

[0058] The forward denoising process is used to add noise to the adversarial sample x0 time step by time step to generate a noisy image corresponding to each time step, where the noisy image at time step t is x t , 1≤t≤T, T is the optimal diffusion step length;

[0059] The reverse generation process is based on conditional guidance, starting from x T Remove noise step by step, and generate denoised image at time step t , the final denoised image As the denoised sample, the According to the following formula:

[0060] ,

[0061] Where, 、 、 are the mean function, variance function, and standard deviation function of time step t respectively; is the deep feature extracted by the feature extraction network, λ is a hyperparameter, for norm, ϵ is random noise, is a standard normal distribution;

[0062] The attention model is used to extract denoised samples and clean sample x clean Extract the corresponding attention map 、 ;

[0063] S3, construct the total loss L of the face recognition diffusion denoising network total ;

[0064] ,

[0065] ,

[0066] Where, L attention is the attention loss; L1 is the L1 loss between the adversarial sample and the denoised sample, L lpips is the LPIPS loss between adversarial samples and denoised samples, To calculate the mean square error, λ1 and λ2 are the first weight hyperparameter and the second weight hyperparameter respectively;

[0067] S4, based on the optimal diffusion step size T, uses the data set X to minimize L total Train the face recognition diffusion denoising network until convergence to obtain the face recognition diffusion denoising model;

[0068] S5: Obtain the adversarial sample to be purified and generate the corresponding denoised sample based on the face recognition diffusion denoising model.

[0069] In this embodiment, the attention model uses the gradient integration method to extract the attention map, which specifically includes steps Sa1 to Sa3;

[0070] Sa1, select a deep network F and a completely black image as the reference input x baseline , take the denoised sample or clean sample as the actual input x;

[0071] Sa2, the attribution value IG of the i-th pixel in x i (x) is obtained according to the following formula:

[0072] ,

[0073] is the interpolation parameter, is x baseline The RGB value of the i-th pixel, x i is the RGB value of the i-th pixel in x, For deep networks;

[0074] Sa3, replace the RGB value of the i-th pixel in the actual input x with the attribution value of the i-th pixel to obtain the attention map of x.

[0075] The L1, L lpips Calculate according to the following formula respectively;

[0076] ,

[0077] Where N is the total number of pixels in the adversarial sample x0, For adversarial examples The nth pixel in For denoising samples The nth pixel in ;

[0078] ,

[0079] Where, L lpips The multi-layer feature extraction network extracts x0 and After calculating the feature map, H l 、Wl are the total height and total width of the feature map extracted by the feature extraction network at layer l, w l is the weight of the l-th layer feature extraction network, is the eigenvalue of the feature map of x0 at height h and width w after the feature extraction network of the first layer, for The feature value of the feature map of the l-th layer feature extraction network at height h and width w is 1≤h≤H, 1≤w≤W, is an element-wise multiplication operation, is the L2 norm.

[0080] In this embodiment S4, the face recognition diffusion denoising network is trained as follows:

[0081] Preset the iteration round and batch size, input the dataset X into the face recognition diffusion denoising network according to the batch size, and calculate a total loss L for each batch total , and back-propagate to adjust the network parameters of the conditional denoising diffusion model until the iteration round is obtained.

[0082] The optimal diffusion step length T is adaptively generated through steps Sb1 to Sb4;

[0083] Sb1, from the clean sample set X clean and adversarial sample set X adv , respectively sample subsets of the same size and ;

[0084] Sb2, set D different step sizes T1~T D , where the dth step length is T d , 1≤d≤D;

[0085] Sb3, let the diffusion step length be T d ,right The adversarial samples are generated by the conditional denoising diffusion model to form a denoising dataset. , and calculate and FID score;

[0086] Sb4, T1~T D Corresponding to D FID scores, the step size with the smallest FID score is taken as the optimal diffusion step size T.

[0087] In this embodiment, S5 is specifically as follows: the adversarial sample to be purified is input as a condition into the conditional denoising diffusion model in the face recognition diffusion denoising model, Gaussian noise is gradually added in the forward denoising process, and a noisy image is generated at each time step. Then, in the reverse generation process, the noise is gradually removed and the identity information of the image is restored, and finally the corresponding denoised sample is obtained.

[0088] Example 2: See Figure 2 , Figure 2 This figure shows the denoising effects of different defense methods, using samples from the CelebA-HQ dataset. Columns 1 through 7 in the figure represent clean samples, adversarial samples, the Defense-GAN method, the TVM method, the MagNet method, the DiffPure method, and the proposed method. Comparison shows that the denoised samples generated by the proposed method effectively restore the identity information in the clean image, maintaining high visual clarity. This demonstrates that the proposed method can effectively eliminate adversarial perturbations while better preserving key image information, improving defense accuracy.

[0089] Example 3: See Figure 3 、 Figure 4 In order to study the generalization and defense effectiveness of the present invention against different adversarial attacks, the present invention is used to defend against a variety of adversarial attacks. At the same time, the present invention is compared with some mainstream adversarial defense methods. The performance of different defense methods is evaluated by defense accuracy, and Table 1 is obtained.

[0090] Table 1. Defense accuracy of different defense methods on the CelebA-HQ dataset

[0091] method No attack FGSM PGD ATA C&W DeepFool No defense 97.8 2.1 5.9 18.4 0.3 1.8 Defense-GAN 45.2 22.6 18.3 25.7 11.5 15.9 TVM 68.7 38.9 29.5 42.6 21.8 34.1 MagNet 76.6 54.5 53.1 44.9 26.4 44.1 DifPure 93.7 89.5 65.3 63.9 60.2 77.4 The present invention 95.4 91.8 58.7 71.3 58.3 82.7

[0092] Table 1 lists five typical adversarial example generation methods: FGSM, PGD, ATA, C&W, and DeepFool. The results in Table 1 demonstrate that the proposed method demonstrates significant defense against a variety of adversarial attacks and excellent generalization, effectively improving the model's robustness against a variety of adversarial attacks. Furthermore, the proposed method not only maintains high recognition accuracy close to that of the original model in the absence of an attack, but also outperforms other defense methods in the face of FGSM, ATA, and DeepFool.

[0093] To verify the impact of the model’s attention mechanism on defense accuracy, we trained the conditional denoising diffusion model with and without the attention mechanism, and then performed denoising defense on the adversarial samples generated by the FGSM adversarial attack on the CelebA-HQ dataset, resulting in Table 2.

[0094] Table 2. Defense accuracy with and without attention mechanism

[0095] method No attack FGSM PGD ATA C&W DeepFool No defense 97.8 2.1 5.9 18.4 0.3 1.8 No attention mechanism 91.5 86.5 50.2 65.2 43.5 74.5 Using the Attention Mechanism 95.4 91.8 58.7 71.3 58.3 82.7

[0096] From the data in Table 2, we can see that after using the model attention mechanism, the overall defense accuracy is higher than the result without using the model attention mechanism.

[0097] Also, see Figure 3 、 Figure 4 , showing the comparison of attention maps between denoised samples generated using the attention mechanism and not using the attention mechanism. It can be seen from the figure that although some adversarial noise can be removed without using the attention mechanism, the face recognition model's attention area on the denoised image is still somewhat scattered, and there is still a certain difference compared with the attention map of the clean image; after using the attention mechanism, the attention maps of the denoised image and the clean image show a higher similarity, so that the face recognition model can extract features more effectively, thereby improving the recognition accuracy.

[0098] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A face recognition diffusion denoising defense method based on attention mechanism, characterized by: The following steps are included: S1, obtain the data set X, including the clean sample set X clean and adversarial sample set X adv , X clean Clean sample x clean and X adv The adversarial samples x0 correspond one to one; S2, constructing a face recognition diffusion denoising network, including a pre-trained face recognition model, a conditional denoising diffusion model, and an attention model; The face recognition model includes a feature extraction network for extracting features from the input image to obtain deep features; The conditional denoising diffusion model includes a forward denoising process and a reverse generation process; The forward denoising process is used to add noise to the adversarial sample x0 time step by time step to generate a noisy image corresponding to each time step, where the noisy image at time step t is x t , 1≤t≤T, T is the optimal diffusion step length; The reverse generation process is based on conditional guidance, starting from x T Remove noise step by step, and generate denoised image at time step t , the final denoised image As the denoised sample, the According to the following formula: ， Where, 、 、 are the mean function, variance function, and standard deviation function of time step t respectively; is the deep feature extracted by the feature extraction network, λ is a hyperparameter, for norm, ϵ is random noise, is a standard normal distribution; The attention model is used to extract denoised samples and clean sample x clean Extract the corresponding attention map 、 ; S3, construct the total loss L of the face recognition diffusion denoising network total ; ， ， Where, L attention is the attention loss; L1 is the L1 loss between the adversarial sample and the denoised sample, L lpips is the LPIPS loss between adversarial samples and denoised samples, To calculate the mean square error, λ1 and λ2 are the first weight hyperparameter and the second weight hyperparameter respectively; S4, based on the optimal diffusion step size T, uses the data set X to minimize L total Train the face recognition diffusion denoising network until convergence to obtain the face recognition diffusion denoising model; S5: Obtain the adversarial sample to be purified and generate the corresponding denoised sample based on the face recognition diffusion denoising model.

2. The face recognition diffusion denoising defense method based on the attention mechanism according to claim 1 is characterized in that: The attention model uses a gradient integration method to extract an attention map, specifically including steps Sa1 to Sa3; Sa1, select a deep network F and a completely black image as the reference input x baseline , take the denoised sample or clean sample as the actual input x; Sa2, the attribution value IG of the i-th pixel in x i (x) is obtained according to the following formula: ， is the interpolation parameter, is x baseline The RGB value of the i-th pixel, x i is the RGB value of the i-th pixel in x, For deep networks; Sa3, replace the RGB value of the i-th pixel in the actual input x with the attribution value of the i-th pixel to obtain the attention map of x.

3. The face recognition diffusion denoising defense method based on attention mechanism according to claim 1 is characterized in that: The L1, L lpips Calculate according to the following formula respectively; ， Where N is the total number of pixels in the adversarial sample x0, For adversarial examples The nth pixel in For denoising samples The nth pixel in ; ， Where, L lpips The multi-layer feature extraction network extracts x0 and After calculating the feature map, H l 、W l are the total height and total width of the feature map extracted by the feature extraction network at layer l, w l is the weight of the l-th layer feature extraction network, is the eigenvalue of the feature map of x0 at height h and width w after the feature extraction network of the first layer, for The feature value of the feature map of the l-th layer feature extraction network at height h and width w is 1≤h≤H, 1≤w≤W, is an element-wise multiplication operation, is the L2 norm.

4. The face recognition diffusion denoising defense method based on attention mechanism according to claim 1 is characterized in that: In S4, the face recognition diffusion denoising network is trained as follows; Preset the iteration round and batch size, input the dataset X into the face recognition diffusion denoising network according to the batch size, and calculate a total loss L for each batch total , and back-propagate to adjust the network parameters of the conditional denoising diffusion model until the iteration round is obtained.

5. The face recognition diffusion denoising defense method based on attention mechanism according to claim 1 is characterized in that: The optimal diffusion step length T is adaptively generated through steps Sb1 to Sb4; Sb1, from the clean sample set X clean and adversarial sample set X adv , respectively sample subsets of the same size and ; Sb2, set D different step sizes T1~T D , where the dth step length is T d , 1≤d≤D; Sb3, let the diffusion step length be T d ,right The adversarial samples are generated by the conditional denoising diffusion model to form a denoising dataset. , and calculate and FID score; Sb4, T1~T D Corresponding to D FID scores, the step size with the smallest FID score is taken as the optimal diffusion step size T.

Images