Face adversarial sample defense method based on diffusion model hybrid sampling

Abstract

The present application relates to the technical field of computer, and particularly relates to a face adversarial sample defense method based on diffusion model hybrid sampling. The method improves the reverse denoising process of the traditional diffusion model and adopts a phased sampling strategy. In the first phase, DDIM sampling is introduced to realize fast and slight denoising, effectively weaken the adversarial perturbation and maximize the retention of face structure features, so as to improve the accuracy of identity recognition. In the second phase, DPM++2M Karras sampling is introduced for high-quality fine reconstruction, eliminating residual perturbation and enhancing image clarity, thereby significantly improving the defense success rate. A dynamic purification ratio regulation mechanism is adopted to adaptively adjust the denoising and reconstruction strength, achieving a balance between perturbation suppression and feature retention. The present application improves the face adversarial sample defense effect while reducing the damage of the purification process to the face identity features, improving the security and stability of the face recognition system in the adversarial attack environment.

Description

Technical Field

[0001] This invention relates to the field of computer technology, and more specifically to a method for defending against adversarial examples of faces based on diffusion model hybrid sampling. Background Technology

[0002] With the development of artificial intelligence and deep learning technologies, facial recognition technology, as an important biometric identification technology, has been widely used in important scenarios such as security monitoring, financial payments, and social entertainment. Facial recognition technology has advantages such as being contactless, having high recognition efficiency, and being convenient to use, and can improve the automation level and security of identity verification to a certain extent. However, existing deep learning-based facial recognition models are vulnerable to adversarial example attacks. Attackers can add minute perturbations that are imperceptible to the human eye to the original facial image, causing the model to produce incorrect recognition results, leading to misidentification or recognition failure, thereby affecting the reliability and security of the facial recognition system.

[0003] Existing methods for defending against adversarial examples in face recognition mainly include adversarial training, gradient masking, input preprocessing, and image sanitization based on generative models. Among these, adversarial training typically has high training costs and limited generalization ability against unknown attacks; gradient masking methods are unstable and prone to failure against adaptive attacks; while input preprocessing methods are simple to implement, they struggle to simultaneously remove perturbations and preserve identity features. In recent years, image sanitization based on generative adversarial networks has been increasingly used for adversarial example defense, such as reconstruction methods based on generative adversarial networks and noise-adding / denoising methods based on diffusion models. However, the former is prone to generation instability and image structure distortion, while the latter often employs a single sampling strategy or fixed denoising intensity, lacking fine-grained control mechanisms specific to face recognition tasks.

[0004] In facial recognition scenarios, model recognition results depend not only on the overall facial structure but also on identity-related details such as eye shape, nose contour, lip curvature, facial boundaries, and facial texture. If the denoising intensity is too low, residual adversarial perturbations in the image are difficult to remove sufficiently; if the denoising intensity is too high, it may damage facial identity features, resulting in the purified image being unable to be correctly recognized. Therefore, existing technologies struggle to achieve a good balance between adversarial perturbation suppression and identity feature preservation. It is necessary to provide an adversarial example defense method suitable for facial recognition scenarios to effectively weaken adversarial perturbations while preserving facial geometry and identity-related local features as much as possible. Summary of the Invention

[0005] To address the problems of insufficient removal of adversarial perturbations, easy distortion of image structure after purification, and easy loss of key identity features in existing generative model-based face adversarial example defense methods, this invention provides a face adversarial example defense method based on diffusion model hybrid sampling. This method designs the inverse denoising sampling process of the diffusion model in stages and introduces a dynamic purification ratio adjustment mechanism, enabling the purification process to achieve a balance between the strength of adversarial perturbation suppression and the degree of preservation of facial identity features, thereby improving the security and stability of the face recognition system under adversarial attack scenarios.

[0006] The technical solution adopted by the present invention to achieve the above objectives is as follows:

[0007] A face adversarial example defense method based on diffusion model hybrid sampling includes the following steps:

[0008] S1: Obtain the face image to be defended and input the face image to be defended into the diffusion model purification module.

[0009] The face image to be defended includes the original clean face image or the face adversarial sample image generated after processing by the adversarial attack method. The diffusion model purification module includes a forward diffusion unit, a first-stage DDIM sampling purification unit, a second-stage DPM++2M Karras sampling reconstruction unit, and a dynamic purification ratio control unit.

[0010] S2, the purification control parameters corresponding to the current input image are determined using the dynamic purification ratio adjustment unit.

[0011] The dynamic descaling ratio control unit determines the descaling risk level of the current input image based on the attack detection results, image quality evaluation results, and recognition confidence level output by the target face recognition model. It then determines the effective descaling intensity, DDIM sampling ratio, and minimum noise scaling ratio based on the descaling risk level.

[0012] The effective purification intensity is used to control the forward diffusion noise intensity; the DDIM sampling ratio is used to determine the proportion of the first-stage DDIM sampling purification process in the total number of sampling steps; the minimum noise scaling ratio is used to control the noise scheduling range in the second-stage DPM++2M Karras sampling reconstruction process.

[0013] Furthermore, the dynamic purification ratio control unit selects the corresponding detail preservation mode based on the degree of purification risk. The detail preservation modes include ultra-high preservation mode, high preservation mode, balanced mode, and medium purification mode. When the purification risk is low, the DDIM sampling ratio is increased and the purification intensity is reduced to prioritize preserving facial identity features; when the purification risk is high, the DPM++2M Karras sampling ratio is increased and the purification intensity is enhanced to prioritize removing residual adversarial perturbations.

[0014] S3, using a forward diffusion unit to add controlled Gaussian noise to the face image to be defended, so that the face image to be defended diffuses from the initial time to the intermediate diffusion time.

[0015] The intermediate diffusion time is determined based on the effective descaling intensity. A higher effective descaling intensity results in a later intermediate diffusion time and a higher forward diffusion noise intensity; conversely, a lower effective descaling intensity results in an earlier intermediate diffusion time and a lower forward diffusion noise intensity. This controlled forward diffusion process weakens or masks the original adversarial perturbations in the input face image and provides an initial state for subsequent reverse denoising sampling.

[0016] S4. Using the first-stage DDIM sampling purification unit, deterministic reverse denoising sampling is performed on the face image after forward diffusion and noise addition to obtain the first purified face image.

[0017] The first-stage DDIM sampling purification unit is used to initially weaken adversarial perturbations while preserving the geometric structure, facial contours, face shape boundaries, and identity-related local features of the face image. The deterministic reverse denoising sampling can reduce the impact of random sampling on facial identity details, allowing the first-stage purification process to focus on structure preservation and mild perturbation reduction.

[0018] S5. Using the second-stage DPM++2M Karras sampling and reconstruction unit, the first cleaned face image is used as input, and the reverse denoising and reconstruction is performed according to the Karras noise scheduling strategy to obtain the second cleaned face image.

[0019] The second-stage DPM++2M Karras sampling and reconstruction unit is used to further remove residual adversarial perturbations based on the first-stage DDIM sampling and cleansing results, and improve the clarity, naturalness, and local detail quality of the cleaned face image. The second-stage DPM++2M Karras sampling and reconstruction process does not directly perform strong reconstruction on the original input image, but rather performs further reconstruction based on the first-stage output image, thereby reducing the risk of excessive alteration of identity features.

[0020] S6, the second purified face image is used as the output image after defense processing.

[0021] The second purified face image is used as input to the target face recognition model for identity verification. When the target face recognition model's recognition result of the second purified face image matches the original identity category, the adversarial defense is deemed successful. The second purified face image can also be output as an independent face purification result to other face recognition systems or identity authentication systems.

[0022] Furthermore, the dynamic purification ratio control unit determines the number of DDIM sampling steps in the first stage based on the total number of sampling steps and the DDIM sampling ratio, and determines the number of DPM++2M Karras sampling steps in the second stage based on the difference between the total number of sampling steps and the number of DDIM sampling steps in the first stage, thereby enabling a continuous connection between the first stage structure-preserving purification process and the second stage fine reconstruction process.

[0023] Furthermore, the attack detection result includes at least one of the following: attack detection score, perturbation anomaly score, frequency domain anomaly score, or stability analysis result output by the target face recognition model; the image quality evaluation result includes at least one of the following: image sharpness, face region integrity, image noise level, face alignment quality, or compression distortion degree; the recognition confidence includes the highest identity category confidence output by the target face recognition model or the confidence corresponding to the original identity category.

[0024] Furthermore, the target face recognition model includes a convolutional neural network face recognition model, a deep metric learning face recognition model, a Transformer-based face recognition model, or other face recognition models capable of outputting identity recognition results. The diffusion model includes a pixel space diffusion model, a latent space diffusion model, or a diffusion model trained or fine-tuned using face image data.

[0025] Compared with the prior art, the present invention has at least the following beneficial effects:

[0026] 1. The method provided by this invention divides the reverse denoising process of the diffusion model into the DDIM structure preservation stage and the DPM++2M Karras fine reconstruction stage, so that the adversarial examples on the face can effectively weaken the adversarial perturbation during the purification process, while maintaining the geometric structure and key identity features of the face. This alleviates the problems of excessive denoising leading to loss of identity information and insufficient denoising leading to attack residue in traditional generative defense methods.

[0027] 2. The method provided by this invention introduces a dynamic purification ratio control mechanism, which adaptively adjusts the effective purification intensity, noise scaling ratio, and two-stage sampling step ratio according to different defense requirements, so that the purification process can achieve a better balance between defense success rate and identity recognition rate, thereby improving the robustness of the face recognition system in adversarial attack scenarios.

[0028] 3. The method described in this invention can be used as a defense preprocessing module at the front end of a face recognition system. It does not require changing the main structure of the target face recognition model, has good applicability and deployment convenience, and can be used to improve the robustness of face recognition systems in adversarial attack scenarios such as FGSM, PGD, and BIM.

[0029] In summary, the face adversarial example defense method based on diffusion model hybrid sampling provided by this invention can achieve the hierarchical optimization goal of "preserving structure first and then refining quality" in the process of face image adversarial perturbation purification. While improving the success rate of adversarial example defense, it maintains a high accuracy rate of identity recognition, thereby improving the security, stability and practicality of the face recognition system in complex attack environments. Attached Figure Description

[0030] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0031] Figure 1 A schematic diagram of the face adversarial sample defense method based on diffusion model hybrid sampling provided in this embodiment of the invention;

[0032] Figure 2 This is a flowchart of the dynamic purification mechanism of the present invention. Detailed Implementation

[0033] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the following will be described in conjunction with the appendix of the present invention. Figure 1-2 The present invention will be clearly and completely described in conjunction with its embodiments. These embodiments are only a part of, not all, of the embodiments of the present invention. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort are within the scope of protection of the present invention.

[0034] This invention provides a method for defending against adversarial examples in faces based on diffusion model hybrid sampling. This method can be used as a preprocessing method for defense at the front end of a face recognition system. That is, before the face image is input into the target face recognition model, the input image is first subjected to diffusion model purification processing to weaken or remove possible adversarial perturbations and preserve the geometric structure and identity-related local features in the face image as much as possible.

[0035] In this embodiment, the face image to be defended can be a raw, clean face image or a face adversarial sample image generated after processing by an adversarial attack method. The adversarial attack method includes, but is not limited to, FGSM, PGD, BIM, or other gradient-based or non-gradient-based face adversarial attack methods. The target face recognition model can be a convolutional neural network model, a deep metric learning model, a Transformer-based face recognition model, or other face recognition models capable of outputting identity recognition results and recognition confidence.

[0036] like Figure 1 As shown in the figure, this embodiment provides a face adversarial sample defense method based on diffusion model hybrid sampling, which includes the following steps.

[0037] S101, acquire the face image to be defended, and input the face image to be defended into the diffusion model purification module; specifically, acquire the input face image. Before inputting the diffusion model purification module, the input image can be processed by face detection, face alignment, face region cropping, and size normalization to ensure that the input image meets the input requirements of the diffusion model purification module and the target face recognition model.

[0038] Specifically, the diffusion model purification module includes a forward diffusion unit, a first-stage DDIM sampling purification unit, a second-stage DPM++2M Karras sampling reconstruction unit, and a dynamic purification ratio control unit.

[0039] The forward diffusion unit adds controlled Gaussian noise to the input face image; the first-stage DDIM sampling purification unit performs structure-preserving inverse denoising on the noisy face image; the second-stage DPM++2M Karras sampling reconstruction unit performs fine reconstruction on the first-stage output image; and the dynamic purification ratio control unit determines the forward diffusion noise intensity, the ratio of the two-stage sampling steps, and the noise scaling ratio based on the input image state.

[0040] S102, the dynamic purification ratio control unit determines the purification control parameters.

[0041] The dynamic descaling ratio control unit determines the descaling risk level of the current input image based on the attack detection results, image quality evaluation results, and the recognition confidence level output by the target face recognition model, and accordingly determines the effective descaling intensity. DDIM sampling ratio and minimum noise scaling .

[0042] S103, the forward diffusion unit performs controlled noise addition on the input face image.

[0043] The forward diffusion unit is based on the effective purification intensity. The intermediate diffusion time is determined, and the input face image is diffused from the initial time to the intermediate diffusion time, thereby weakening or masking the original adversarial perturbation in the input image.

[0044] S104, the first stage DDIM sampling and purification unit performs deterministic reverse denoising sampling.

[0045] The first-stage DDIM sampling and denoising unit takes the forward-diffused noisy face image as input and performs deterministic reverse denoising sampling to obtain the first denoising face image. This stage is primarily used to preserve the overall structure of the facial image, the layout of facial features, facial boundaries, and identity-related local features.

[0046] S105, the second-stage DPM++2M Karras sampling reconstruction unit performs fine reconstruction.

[0047] The second-stage DPM++2M Karras sampling and reconstruction unit uses the first purified face image. As input, the reverse denoising and reconstruction is performed according to the Karras noise scheduling strategy to obtain the second cleaned face image. This stage is mainly used to further remove residual adversarial perturbations in the output image of the first stage, and to improve the clarity, naturalness, and local detail quality of the cleaned face image.

[0048] S106, output the second purified face image.

[0049] The second purified facial image The output image after defensive processing. The second purified face image. It can be used to input a target face recognition model for identity verification, or it can be output to an independent face recognition system, identity authentication system, or image quality verification system.

[0050] like Figure 2 As shown, in this embodiment, the dynamic purification ratio control unit is used to determine the control parameters in the diffusion model purification process based on the attack risk and identity preservation requirements of the input image.

[0051] An attack detection module is used to obtain an attack detection score for the input face image. The attack detection score It can be obtained by disturbance detector, abnormal feature detector, frequency domain anomaly detection method, input stability analysis method or target face recognition model output fluctuation analysis method. The larger the value, the more likely the input image is to contain strong adversarial perturbations.

[0052] The image quality score of the input face image is obtained using the image quality evaluation module. The image quality score It can be determined based on image sharpness, face region integrity, image noise level, face alignment quality, compression distortion, or other image quality evaluation indicators. The larger the value, the higher the quality of the input image.

[0053] The recognition confidence level of the target face recognition model is obtained using the recognition confidence level acquisition module. The identification confidence level It can be the highest confidence score for the identity category output by the target face recognition model, or it can be the confidence score corresponding to the original identity category. The larger the value, the more stable the target face recognition model is for the current recognition result.

[0054] The mode selection module is used based on the attack detection score. Image quality score and identification confidence Calculate the purification risk score :

[0055]

[0056] in, , , Let represent the weight coefficients corresponding to the attack detection score, recognition confidence score, and image quality score, respectively, and satisfy the following:

[0057]

[0058] In this embodiment, , , All are normalized to the range of 0 to 1. When A smaller value indicates a lower risk of attack on the input image or a higher need for identity preservation; when... A larger value indicates that the input image may contain strong adversarial perturbations, requiring enhanced cleansing.

[0059] The mode selection module selects the detail preservation mode according to the following rules:

[0060] when When needed, select the ultra-high hold mode;

[0061] when When needed, select the high hold mode;

[0062] when When needed, select Balanced mode;

[0063] when When needed, select the medium purification mode.

[0064] Furthermore, the parameters corresponding to different detail preservation modes can be set as shown in Table 1:

[0065] Table 1 Parameter Adjustment of Dynamic Purification Mechanism

[0066]

[0067] Among them, effective purification intensity Used to control the intensity of forward diffusion noise; DDIM sampling ratio Used to control the proportion of the first-stage structure-preserving sampling in the total number of sampling steps; minimum noise scaling ratio. Used to control the minimum noise level during the second phase of the Karras noise scheduling process.

[0068] When the attack risk is low or the need to maintain identity features is high, the dynamic cleanup ratio control unit increases the DDIM sampling ratio to reduce the change of key identity features during the cleanup process; when the attack risk is high or the identification confidence is low, the dynamic cleanup ratio control unit increases the DPM++2M Karras sampling ratio to enhance the ability to remove residual adversarial disturbances.

[0069] Understandably, a higher DDIM sampling ratio indicates a greater tendency for the cleansing process to preserve facial structure and identity details; similarly, a higher DPM++2M Karras sampling ratio indicates a greater tendency for the cleansing process to improve perturbation removal capabilities and image reconstruction quality. Therefore, this invention dynamically adjusts the two-stage sampling ratio to achieve a balance between fidelity and robustness in the cleansing process, adapting to different attack intensities and application requirements.

[0070] Furthermore, the dynamic purification ratio control unit adjusts the ratio based on the total number of sampling steps. and DDIM sampling ratio Calculate the number of two-stage sampling steps:

[0071]

[0072]

[0073] in, This indicates the number of sampling steps in the first-stage DDIM sampling purification unit. This indicates the number of sampling steps in the second-stage DPM++2M Karras sampling and reconstruction unit. This method ensures a continuous connection between the first and second-stage sampling processes.

[0074] In a further implementation, when the face recognition system has high requirements for identity preservation, or when the attack detection score of the input image is low, the dynamic sanitization ratio control unit selects an ultra-high preservation mode or a high preservation mode, so that the DDIM sampling sanitization unit occupies a high sampling ratio, thereby preserving facial identity features to the greatest extent. When the attack detection score of the input image is detected to be high, or when the target face recognition model has low confidence in recognizing the input image, the dynamic sanitization ratio control unit selects a balanced mode or a medium sanitization mode, so that the DPM++2M Karras sampling reconstruction unit occupies a high sampling ratio, thereby enhancing the ability to remove residual adversarial perturbations. The dynamic sanitization ratio control unit can also select a detail preservation mode based on the input image quality evaluation results. For example, when the input image has high clarity and the requirement for identity detail preservation is strong, the DDIM sampling ratio is increased; when the input image is suspected of containing strong adversarial perturbations and the sanitization priority is high, the DPM++2MKarras sampling ratio is increased.

[0075] In this embodiment, the forward diffusion unit determines the effective purification intensity based on the dynamic purification ratio control unit. Determine the intermediate diffusion time :

[0076]

[0077] in, This represents the maximum diffusion time step in the diffusion model. Indicates the effective purification intensity. This indicates rounding down to the nearest integer.

[0078] Specifically, the forward diffusion noise addition stage includes: processing the input face image... Add an appropriate amount of Gaussian noise to make it equal to the initial time. Diffusion to the intermediate moment This reduces the impact of the original adversarial perturbation on the image feature space. The noise addition process can be represented as:

[0079]

[0080] in, , It is the first in the diffusion model noise time table One noise figure, It is a standard Gaussian distribution The noise vector sampled in the middle.

[0081] It should be understood that the forward diffusion noise-adding stage is not intended to completely destroy the input image, but rather to mask adversarial perturbations in the face image through controlled noise addition, enabling the subsequent reverse denoising sampling process to gradually restore the image to a state close to the clean face data distribution. If the noise intensity is too low, it will be difficult to sufficiently destroy the original adversarial perturbations; if the noise intensity is too high, it may destroy the key identity features of the face image. Therefore, this invention controls the noise intensity through a dynamic purification ratio adjustment unit.

[0082] The first stage, the DDIM sampling and purification stage, includes: [removing the noise from the forward diffusion stage]. Input is sent to the DDIM sampling and purification unit, and deterministic sampling is used from... Sampled to The first purified face image was obtained. .

[0083] Furthermore, the randomness parameters of the DDIM sampling and purification unit Set to 0.

[0084] Specifically, the DDIM sampling and purification unit predicts the current noise using a denoising model during the reverse denoising process, and then gradually restores the image based on the predicted noise. The DDIM sampling steps can be represented as follows:

[0085]

[0086] in, , express Diffusion image at time, express Image restoration at any given moment express Noise figure at time step This represents the noise predicted by the denoising model.

[0087] The first-stage DDIM sampling and cleansing unit is primarily used for mild cleansing of the input face image. Because DDIM sampling has good structure preservation capabilities, it can initially smooth out abrupt disturbances while retaining the facial geometry and identity-related details in the input image, such as eye shape, nose contour, lip curvature, facial boundaries, and local texture information. This avoids the facial structure distortion and identity information loss problems that are easily caused by traditional strong cleansing methods.

[0088] The second stage, the DPM++2M Karras sampling and reconstruction stage, includes: converting the first cleaned face image output from the first stage... As input, a Karras noise schedule is generated again, which will... Starting with the DPM++2M Karras sampler, from Sampling until The second purified face image was obtained. .

[0089] further, Set to 0.

[0090] Specifically, in the second stage, the DPM++2M Karras sampling and reconstruction unit generates a noise sequence according to the Karras noise scheduling strategy. The noise sequence can be generated in the following manner:

[0091]

[0092]

[0093] in, This indicates the number of Karras sampling steps. Indicates the minimum noise level. Indicates the maximum noise level. This represents the noise scheduling hyperparameter. Indicates the first The linear interpolation coefficients corresponding to each sampling position.

[0094] further, Set to 7.0.

[0095] Furthermore, the minimum noise level is adjusted according to the minimum noise scaling factor:

[0096]

[0097] in, This indicates the adjusted minimum noise level. This indicates the minimum noise scaling ratio.

[0098] It should be understood that the DPM++2M Karras sampling and reconstruction unit utilizes a high-order solver and an adaptive noise scheduling strategy to further remove residual perturbations in the first cleaned face image and perform fine reconstruction of the image. Compared with the first-stage DDIM sampling and cleansing unit, the second-stage DPM++2M Karras sampling and reconstruction unit focuses more on improving the clarity, naturalness, and local detail quality of the cleaned image, thereby enhancing the usability of the cleaned image in face recognition tasks.

[0099] After purification is complete, the second purified face image output from the second stage will be... The input is fed into the target face recognition model to obtain the corresponding face identity recognition result.

[0100] It should be understood that if the target face recognition model is effective in cleaning up face images... If the output identity category matches the original face identity category, the adversarial example defense is considered successful; if the target face recognition model successfully defends against the purified face image... If the output identity category is inconsistent with the original face identity category, the adversarial sample defense for that face is deemed to have failed.

[0101] It should be understood that the facial recognition stage can be a subsequent application step of the method of the present invention. In actual deployment, the method of the present invention may also output only the purified facial image. Then, an independent facial recognition system performs identity authentication.

[0102] In a specific experimental embodiment, the LFW and CelebA-HQ datasets were selected as the face image datasets, and ResNet50 was chosen as the target face recognition model. The Protogenx 3.4 model was used as the diffusion model, and FGSM, PGD, and BIM were selected as adversarial attack methods. The attack perturbation was set to 4 / 255. High-preservation mode was selected for detail preservation. The experimental environment used the PyTorch 1.11.0 deep learning framework, with an RTX 3090 GPU.

[0103] To evaluate the defensive effectiveness of the method provided in this embodiment, the Defense Success Rate (DSR) is used as the evaluation metric. The Defense Success Rate (DSR) measures the ability of a defensive method to suppress adversarial disturbances and can be expressed as:

[0104]

[0105] In this experimental embodiment, the face images in the dataset are first input into the target face recognition model to obtain the original identity recognition result; then, corresponding adversarial face sample images are generated through FGSM, PGD, or BIM; subsequently, the adversarial face sample images are input into the diffusion model purification module of this invention, and forward diffusion noise addition, DDIM sampling purification, DPM++2M Karras sampling reconstruction, and dynamic scaling are performed in sequence; finally, the purified face images are input into the target face recognition model to obtain the defended face identity recognition result.

[0106] To verify the effectiveness of the two-stage hybrid sampling structure, ablation tests were conducted on different sampling strategies. Specifically, three methods were employed for defense: DDIM-only sampling, DPM++2M Karras-only sampling, and a hybrid sampling of DDIM and DPM++2M Karras combined with a dynamic purification ratio control mechanism. The specific defense success rates are shown in the table below:

[0107] Table 2 Defense success rate under different sampling strategies

[0108]

[0109] The results show that while DDIM sampling alone can preserve facial structure well, it has limited ability to eliminate residual perturbations; while DPM++2M Karras sampling alone can improve reconstruction quality, it is prone to loss of identity details; and the present invention can fully combine the advantages of both through two-stage hybrid sampling.

[0110] Because face recognition tasks differ from ordinary image classification tasks, their recognition results highly depend on fine-grained identity features such as eye shape, lip curvature, facial contours, and local textures. If the cleansing intensity is too high, key facial features may be smoothed or altered, causing the cleaned image to escape the attack target but not return to the original identity category. If the cleansing intensity is too low, it is difficult to fully eliminate adversarial perturbations, and attacks may still succeed. This invention adopts a hybrid sampling approach of "preserving structure first, then refining quality." The DDIM sampling stage is responsible for maintaining the overall facial structure and identity skeleton, while the DPM++2MKarras sampling stage is responsible for eliminating residual perturbations and improving image quality. A dynamic cleansing ratio adjustment mechanism is used to adjust the sampling ratio according to different application requirements, thereby achieving a balance between adversarial perturbation suppression and identity feature preservation.

[0111] Those skilled in the art will readily understand that the above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A face adversarial example defense method based on diffusion model hybrid sampling, characterized in that, The method includes the following steps: S1. Obtain the face image to be defended and input the face image to be defended into the diffusion model purification module; S2. Use the dynamic purification ratio control unit to determine the purification control parameters corresponding to the current input image; S3. Controlled Gaussian noise is added to the face image to be defended using a forward diffusion unit, so that the face image to be defended diffuses from the initial time to the intermediate diffusion time. S4. Using the first-stage DDIM sampling purification unit, perform deterministic reverse denoising sampling on the face image after forward diffusion and noise addition to obtain the first purified face image. S5. Using the second-stage DPM++2M Karras sampling and reconstruction unit, the first cleaned face image is used as input, and the reverse denoising and reconstruction is performed according to the Karras noise scheduling strategy to obtain the second cleaned face image. S6. Use the second purified face image as the output image after defense processing.

2. The face adversarial example defense method based on diffusion model hybrid sampling according to claim 1, characterized in that, The face image to be defended in step S1 includes the original clean face image or the face adversarial sample image generated after processing by the adversarial attack method. The diffusion model purification module includes a forward diffusion unit, a first-stage DDIM sampling purification unit, a second-stage DPM++2M Karras sampling reconstruction unit, and a dynamic purification ratio control unit.

3. The face adversarial example defense method based on diffusion model hybrid sampling according to claim 1, characterized in that, In step S2, the dynamic purification ratio control unit determines the purification risk level of the current input image based on the attack detection result of the input face image, the image quality evaluation result, and the recognition confidence level output by the target face recognition model, and determines the effective purification intensity, DDIM sampling ratio, and minimum noise scaling ratio based on the purification risk level. The effective purification intensity is used to control the forward diffusion noise intensity; the DDIM sampling ratio is used to determine the proportion of the first-stage DDIM sampling purification process in the total number of sampling steps; the minimum noise scaling ratio is used to control the noise scheduling range in the second-stage DPM++2M Karras sampling reconstruction process.

4. The face adversarial example defense method based on diffusion model hybrid sampling according to claim 3, characterized in that, The dynamic purification ratio control unit determines the number of DDIM sampling steps in the first stage based on the total number of sampling steps and the DDIM sampling ratio, and determines the number of DPM++2M Karras sampling steps in the second stage based on the difference between the total number of sampling steps and the first stage DDIM sampling steps, so that the first stage structure-preserving purification process and the second stage fine reconstruction process are continuously connected. The dynamic purification ratio control unit selects the corresponding detail preservation mode according to the degree of purification risk; the detail preservation mode includes ultra-high preservation mode, high preservation mode, balanced mode and medium purification mode; when the degree of purification risk is low, the DDIM sampling ratio is increased and the purification intensity is reduced to prioritize the preservation of facial identity features; when the degree of purification risk is high, the DPM++2M Karras sampling ratio is increased and the purification intensity is enhanced to prioritize the removal of residual anti-perturbations.

5. A face adversarial example defense method based on diffusion model hybrid sampling according to claim 1, characterized in that, The intermediate diffusion time in step S3 is determined based on the effective purification intensity; the higher the effective purification intensity, the later the intermediate diffusion time, and the higher the forward diffusion noise intensity; the lower the effective purification intensity, the earlier the intermediate diffusion time, and the lower the forward diffusion noise intensity; through controlled forward diffusion processing, the original adversarial perturbations in the input face image can be weakened or masked, and an initial state can be provided for subsequent reverse denoising sampling.

6. The face adversarial example defense method based on diffusion model hybrid sampling according to claim 1, characterized in that, In step S4, the first-stage DDIM sampling purification unit is used to maintain the geometric structure, facial contours, face shape boundaries, and identity-related local features of the face image while initially weakening adversarial perturbations. The deterministic reverse denoising sampling can reduce the impact of random sampling on facial identity details, making the first-stage purification process focus on structure preservation and mild perturbation reduction.

7. The face adversarial example defense method based on diffusion model hybrid sampling according to claim 1, characterized in that, In step S5, the second-stage DPM++2M Karras sampling and reconstruction unit is used to remove residual adversarial perturbations based on the first-stage DDIM sampling and purification results, and to improve the clarity, naturalness, and local detail quality of the purified face image. The second-stage DPM++2M Karras sampling and reconstruction process does not directly perform strong reconstruction on the original input image, but rather performs further reconstruction based on the first-stage output image, thereby reducing the risk of excessive alteration of identity features.

8. The face adversarial example defense method based on diffusion model hybrid sampling according to claim 1, characterized in that, In step S6, the second purified face image is used as input to the target face recognition model for identity recognition; when the recognition result of the target face recognition model on the second purified face image is consistent with the original identity category, the adversarial defense is determined to be successful; the second purified face image is output as an independent face purification result to other face recognition systems or identity authentication systems.

9. A face adversarial example defense method based on diffusion model hybrid sampling according to claim 3, characterized in that, The attack detection results include at least one of the following: attack detection score, perturbation anomaly score, frequency domain anomaly score, or stability analysis results output by the target face recognition model; the image quality evaluation results include at least one of the following: image sharpness, face region integrity, image noise level, face alignment quality, or compression distortion; the recognition confidence includes the highest identity category confidence output by the target face recognition model or the confidence corresponding to the original identity category.

10. A face adversarial example defense method based on diffusion model hybrid sampling according to claim 3, characterized in that, The target face recognition model includes a convolutional neural network face recognition model, a deep metric learning face recognition model, a Transformer-based face recognition model, or other face recognition models capable of outputting identity recognition results; the diffusion model includes a pixel space diffusion model, a latent space diffusion model, or a diffusion model trained or fine-tuned by face image data.

Images