A face recognition system security test method based on diffusion model

Abstract

The present application relates to a kind of face recognition system security test method based on diffusion model, comprising: using preset encoder to encode source image, obtain source image latent code;According to the corresponding identity sensitive region image of source image, using preset encoder to encode identity sensitive region image, obtain identity sensitive region image latent code;Using conditional latent diffusion model, source image latent code is gradually forward diffusion processing and step-by-step reverse denoising processing, generates adversarial sample;Adversarial sample is input into the face recognition system to be tested to carry out identification, and the security of face recognition system is determined according to the identification result.The present application method uses conditional latent diffusion model, can reach the best attack performance while generating high-quality imperceptible adversarial sample, and the security of face recognition system is tested using the adversarial sample, improve the credibility of the identification result of face recognition system.

Description

Technical Field

[0001] This invention belongs to the field of face recognition technology, specifically relating to a security testing method for face recognition systems based on a diffusion model. Background Technology

[0002] With the development of deep learning technology, facial recognition systems, as the most mature application of artificial intelligence, have been widely implemented and are used in people's daily lives, such as facial payment, check-in registration, identity verification, and identity verification. However, along with large-scale commercialization, facial recognition technology also faces more threats.

[0003] Research has found that these deep neural network-based models are susceptible to adversarial perturbations added to the original clean images. These carefully crafted perturbation images, also known as adversarial examples, can cause specific deep learning models to make incorrect classifications. Therefore, adversarial examples can be used to detect security issues in deep learning models.

[0004] Existing gradient-based methods cannot generate imperceptible adversarial samples, and their performance depends on the attack threshold. Patch-based methods, on the other hand, synthesize adversarial patches that typically possess specific color and texture patterns, making them easier to distinguish but also difficult to achieve stealth. Stealth-based methods generally fail to achieve good attack performance. None of the above methods achieve a satisfactory balance between attack performance and imperceptibility. Therefore, generating adversarial samples that simultaneously possess high attack performance and imperceptibility to test the security of face recognition systems and thus verify the reliability of their recognition results is of great significance. Summary of the Invention

[0005] To address the aforementioned problems in existing technologies, this invention provides a security testing method for face recognition systems based on a diffusion model. The technical problem to be solved by this invention is achieved through the following technical solution:

[0006] This invention provides a security testing method for a face recognition system based on a diffusion model, comprising:

[0007] The source image is encoded using a preset encoder to obtain the latent code of the source image;

[0008] The corresponding identity-sensitive region image is obtained from the source image, and the identity-sensitive region image is encoded using the preset encoder to obtain the latent encoding of the identity-sensitive region image.

[0009] The latent coding of the source image is subjected to progressive forward diffusion and progressive reverse denoising using a conditional latent diffusion model to generate adversarial samples. In each step of the reverse denoising process, the latent coding of the identity-sensitive region image is used as a constraint to obtain denoised data. Semantic adversarial perturbation is added to the denoised data, and the denoised data with semantic adversarial perturbation is used as the input data for the next step of the reverse denoising process.

[0010] The adversarial sample is input into the face recognition system under test for recognition, and the security of the face recognition system is determined based on the recognition result.

[0011] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0012] The security testing method for face recognition systems based on diffusion models of the present invention utilizes a conditional latent diffusion model to generate semantic adversarial perturbations in the latent space of the diffusion model rather than the original pixel space. Moreover, with identity-sensitive regions as constraints, it can achieve the best attack performance while generating high-quality, imperceptible adversarial samples. The security of the face recognition system is tested using these adversarial samples, thereby improving the reliability of the recognition results of the face recognition system.

[0013] The above description is merely an overview of the technical solution of the present invention. In order to better understand the technical means of the present invention and to implement it in accordance with the contents of the specification, and to make the above and other objects, features and advantages of the present invention more apparent and understandable, preferred embodiments are described in detail below with reference to the accompanying drawings. Attached Figure Description

[0014] Figure 1 This is a flowchart of a security testing method for a face recognition system based on a diffusion model, provided by an embodiment of the present invention.

[0015] Figure 2 This is a flowchart of a process for generating adversarial samples provided in an embodiment of the present invention. Detailed Implementation

[0016] To further illustrate the technical means and effects adopted by the present invention to achieve the intended purpose, the following describes in detail, with reference to the accompanying drawings and specific embodiments, a security testing method for a face recognition system based on a diffusion model proposed in accordance with the present invention.

[0017] The foregoing and other technical contents, features, and effects of the present invention will be clearly presented in the following detailed description of specific embodiments in conjunction with the accompanying drawings. Through the description of the specific embodiments, a more in-depth and concrete understanding can be gained of the technical means and effects adopted by the present invention to achieve its intended purpose. However, the accompanying drawings are for reference and illustration only and are not intended to limit the technical solutions of the present invention.

[0018] First, we introduce the latent diffusion model, which learns a lower-dimensional compressed latent space to reduce computational cost. We first define an autoencoder that encodes the input image x into a latent code z = ε(x), and then decodes z into... In the forward phase, the T-step forward diffusion process aims to gradually transform the latent code z to match a Gaussian distribution: The forward process can be defined as:

[0019]

[0020] Where, β t ∈(0,1] is a hyperparameter, and I is the identity matrix. The subsequent inverse denoising process can be defined as:

[0021]

[0022] Where, μ θ and Σ θ Let represent the mean and standard deviation, respectively. α t For setting hyperparameters, ∈ θ This is a parameterized deep learning model. During the inference phase, the latent encoding is randomly initialized. Then it is gradually updated until z0, using the reverse process described above. Finally, the synthesized high-quality image x0 can be generated by the latent decoder.

[0023] This invention provides a security testing method for a face recognition system based on a diffusion model. It mainly utilizes a conditional latent diffusion model to generate high-quality adversarial samples, and uses these adversarial samples to perform security testing on the face recognition system.

[0024] Please refer to the above. Figure 1 , Figure 1 This is a flowchart of a security testing method for a face recognition system based on a diffusion model, provided by an embodiment of the present invention. Figure 1 As shown in this embodiment, the security testing method for a face recognition system based on a diffusion model includes:

[0025] Step 1: Encode the source image using a preset encoder to obtain the latent encoding of the source image;

[0026] In an optional embodiment, the source image is a face image that the face recognition system under test can recognize.

[0027] Step 2: Obtain the corresponding identity-sensitive region image based on the source image, and encode the identity-sensitive region image using a preset encoder to obtain the latent encoding of the identity-sensitive region image;

[0028] In an optional embodiment, an identity-sensitive region image corresponding to the source image is obtained by performing face analysis on the source image.

[0029] Inspired by cognitive psychology, most identity-sensitive information is concentrated in identity-sensitive areas, such as the eyes, nose, cheeks, and lips. For example, an image of an identity-sensitive area is an image of the face. Identity-insensitive areas, such as hairstyle, accessories, and background, contain less discriminative information for human recognition, but can be captured by facial recognition models.

[0030] Step 3: Use the conditional latent diffusion model to perform progressive forward diffusion and progressive backward denoising on the latent coding of the source image to generate adversarial samples;

[0031] In each step of the reverse denoising process, the latent encoding of the identity-sensitive region image is used as a constraint to obtain denoised data. Semantic adversarial perturbation is added to the denoised data, and the denoised data with semantic adversarial perturbation is used as the input data for the next step of reverse denoising.

[0032] In an optional embodiment, the adversarial sample is obtained by decoding the denoised data with semantic adversarial perturbations obtained by progressive reverse denoising until the final denoised data is obtained.

[0033] Please refer to the above. Figure 2 , Figure 2 This is a flowchart illustrating the generation of adversarial samples according to an embodiment of the present invention, such as... Figure 2 As shown, in the process of generating adversarial examples, for the source image x s First, the latent encoding of the source image is subjected to noise addition through a T-step forward diffusion process to obtain z. T The initial value is the noisy image. Then, the source image x is calculated using a face parsing model. s The binary mask M of the mid-face region is used to calculate the identity-sensitive region image x. m =x s ⊙(1-M), and then use the preset encoder to map the identity-sensitive area image to the latent space c=ε(x mThis yields the latent encoding of the identity-sensitive region image. Next, the robust patching capabilities of the Conditional Latent Diffusion Model (LDM) are utilized to ensure that most adversarial semantic appearance perturbations around the identity-sensitive region image are avoided.

[0034] The process of performing a one-step reverse denoising on a noisy image is illustrated using a noisy image as an example. Let z... T The input data is connected to c and used as the input data for the UNet model, i.e., the inverse denoising process. Then, the input data is subjected to inverse denoising using the patching inference process to obtain denoised data under the constraint of the latent encoding of identity-sensitive regions. After that, semantic adversarial perturbation is added to the denoised data to complete one step of inverse denoising.

[0035] In each step of the reverse denoising process, the process of obtaining denoised data by using the latent encoding of the identity-sensitive region image as a constraint can be represented as follows:

[0036]

[0037] Where, p θ (·) represents the parameterized network model, z t-1 For the latent variables of the image at step t-1, Let c be the latent variable of the image at step t, and c be the constraint condition. Let μ represent a normal distribution, t be the number of steps in the stepwise reverse denoising process, and μ be the number of steps in the reverse denoising process. θ For the mean, Σ θ The standard deviation is denoted as .

[0038] After adding semantic adversarial perturbations, the denoised data with semantic adversarial perturbations is represented as:

[0039]

[0040] in, For denoised data with semantically adversarial perturbations, To counteract disturbances.

[0041] It is worth noting that the Conditional Latent Diffusion (LDD) model primarily focuses on generating visual information suitable for the central region and attempting to preserve identity-sensitive areas, which is determined by the patching capabilities of constraint c and LDM. In this embodiment of the invention, learned semantic adversarial perturbations are added to the latent space during the anti-cartoon process, thereby generating less conspicuous perturbations in identity-sensitive areas. Since the latent space is perceptually equivalent to the original pixel space, the perturbations added to the latent space introduce different semantic appearances.

[0042] Please continue reading Figure 2 In an optional embodiment, the process of generating semantic adversarial perturbations includes:

[0043] Step ①: For the input data of the current step's reverse denoising process, reverse denoising is performed using the latent encoding of the identity-sensitive region image as a constraint to obtain denoised data;

[0044] Step 2: Decode the denoised data to obtain the corresponding decoded image;

[0045] Step 3: Based on the decoded image and the target image, generate the semantic adversarial perturbation for the current step using a preset face recognition model.

[0046] In an optional embodiment, the target image is a face image that the face recognition system under test is expected to misidentify.

[0047] In an optional embodiment, the preset face recognition model is a face recognition model with known network parameters.

[0048] Optionally, a gradient-based adversarial example algorithm is used to generate semantic adversarial perturbations. Specifically, the semantic adversarial perturbation can be represented as:

[0049]

[0050] in, For semantic adversarial perturbations, w t Attack strength, expressed as s is a hyperparameter used to control attack strength, x r For the target image, For the decoded image, F(·) is a preset face recognition model. The preset face recognition model is used to learn the identity similarity between the faces in the decoded image and the target image, and the similarity between the two is output. This is a gradient algorithm. From z t-1 The predicted approximate result is expressed as

[0051] It should be noted that, due to As the reverse denoising process increases, the attack strength w t By adaptively reducing the attack strength, an adaptive attack strength strategy can help improve image quality.

[0052] Step 4: Input the adversarial sample into the face recognition system to be tested for recognition, and determine the security of the face recognition system based on the recognition results.

[0053] Optionally, adversarial sample sets can be formed by acquiring adversarial samples from source images of different faces. These adversarial sample sets and the corresponding source image sets are then jointly input into the face recognition system under test as a test set, ultimately obtaining the classification probability of each adversarial sample. The security performance of the face recognition system under test can be evaluated using metrics such as recognition accuracy; a lower recognition accuracy indicates poorer security performance.

[0054] The security testing method for face recognition systems based on diffusion models in this invention utilizes a conditional latent diffusion model to generate semantic adversarial perturbations with high attack capability and low perceptibility in the latent space of the diffusion model rather than the original pixel space. Moreover, by using identity-sensitive regions as constraints, it ensures the most adversarial appearance around the identity-sensitive regions, ensuring the transmissibility and concealment of the attack. It can achieve the best attack performance while generating high-quality, imperceptible adversarial samples. Using these adversarial samples to test the security of the face recognition system improves the credibility of the recognition results of the face recognition system.

[0055] Furthermore, in order to verify the effectiveness of the face recognition system security testing method based on the diffusion model in this embodiment of the invention, an adversarial sample generated by the method of this invention was used to attack the black box model through simulation experiments.

[0056] Simulation experiment:

[0057] The method of the present invention was evaluated on two publicly available high-quality face datasets, CelebA-HQ and FFHQ, denoted as Adv-Diffusion.

[0058] The FFHQ dataset is a widely used high-quality face dataset (Karras, Laine, and Aila2019) containing nearly 70,000 high-quality face images at a resolution of 1024×1024.

[0059] The CelebA-HQ dataset is also a high-quality face dataset (Karras et al. 2018), built on the CelebA dataset, and contains nearly 30,000 face images with a resolution of 512×512.

[0060] To evaluate the attack performance of the adversarial samples generated by the method of this invention against black-box models, four commonly used face recognition models were selected: IR152 (He et al. 2016), IRSE50 (Hu, Shen, and Sun 2018), FaceNet (Schroff, Kalenichenko, and Philbin 2015), and MobileFace (Deng et al. 2019) as target attack models. It is important to note that these face recognition models were pre-trained on large-scale face datasets and all achieved satisfactory recognition performance. When one model was selected as the black-box target model, the other three models were used as white-box target models to generate the adversarial samples. Simultaneously, three types of attack methods were selected as comparison methods: gradient-based methods, patch-based methods, and stealth-based methods. See Table 1, which shows the comparison results of the attack performance of different methods on the CelebA-HQ and FFHQ datasets. Experimental results demonstrate that the adversarial samples generated by the method of this invention achieve the highest attack performance.

[0061] Table 1: Experimental results using Attack Success Rate (ASR) as the metric for comparison when parameter T is set to 50.

[0062]

[0063] It should be noted that, in this document, the terms "comprising," "including," or any other variations are intended to cover non-exclusive inclusion, such that an article or device that comprises a list of elements includes not only those elements but also other elements not expressly listed. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the article or device that includes said element.

[0064] The above description, in conjunction with specific preferred embodiments, provides a further detailed explanation of the present invention. It should not be construed that the specific implementation of the present invention is limited to these descriptions. For those skilled in the art, various simple deductions or substitutions can be made without departing from the concept of the present invention, and all such modifications and substitutions should be considered within the scope of protection of the present invention.

Claims

1. A security testing method for a face recognition system based on a diffusion model, characterized in that, include: The source image is encoded using a preset encoder to obtain the latent code of the source image; The corresponding identity-sensitive region image is obtained from the source image, and the identity-sensitive region image is encoded using the preset encoder to obtain the latent encoding of the identity-sensitive region image. The latent coding of the source image is subjected to progressive forward diffusion and progressive reverse denoising using a conditional latent diffusion model to generate adversarial samples. In each step of the reverse denoising process, the latent coding of the identity-sensitive region image is used as a constraint to obtain denoised data. Semantic adversarial perturbation is added to the denoised data, and the denoised data with semantic adversarial perturbation is used as the input data for the next step of the reverse denoising process. The adversarial sample is input into the face recognition system under test for recognition, and the security of the face recognition system is determined based on the recognition result.

2. The security testing method for a face recognition system based on a diffusion model according to claim 1, characterized in that, The source image is a face image that the face recognition system under test can recognize.

3. The security testing method for a face recognition system based on a diffusion model according to claim 1, characterized in that, Based on the source image, the corresponding identity-sensitive region image is obtained, including: The source image is subjected to face analysis to obtain the identity-sensitive region image corresponding to the source image.

4. The security testing method for a face recognition system based on a diffusion model according to claim 3, characterized in that, The identity-sensitive area image is a face region image.

5. The security testing method for a face recognition system based on a diffusion model according to claim 1, characterized in that, In each step of the reverse denoising process, the process of obtaining denoised data by using the latent encoding of the identity-sensitive region image as a constraint condition is represented as follows: ； in, For parameterized network models, For the first Latent variables in the image of the step, For the first Latent variables in the image of the step, As constraints, Indicates a normal distribution. The first step of the progressive reverse denoising process step, The mean, The standard deviation is denoted as .

6. The security testing method for a face recognition system based on a diffusion model according to claim 5, characterized in that, The denoised data with semantically adversarial perturbations is represented as follows: ； in, For denoised data with semantically adversarial perturbations, This is a semantic adversarial perturbation.

7. The security testing method for a face recognition system based on a diffusion model according to claim 1, characterized in that, The generation process of the semantic adversarial perturbation includes: Step ①: For the input data of the reverse denoising process in the current step, reverse denoising is performed with the latent encoding of the identity-sensitive region image as a constraint condition to obtain denoised data; Step 2: Decode the denoised data to obtain the corresponding decoded image; Step 3: Based on the decoded image and the target image, generate the semantic adversarial perturbation for the current step using a preset face recognition model.

8. The security testing method for a face recognition system based on a diffusion model according to claim 7, characterized in that, The target image is the face image that the face recognition system is expected to misidentify.

9. The security testing method for a face recognition system based on a diffusion model according to claim 7, characterized in that, The semantic adversarial perturbation is represented as: ； in, For semantic adversarial perturbations, For attack strength, For the target image, To decode the image, For the preset face recognition model, This is a gradient algorithm. The first step of the progressive reverse denoising process step, For the first Latent variables in the image of the step, From Predicted latent variables in the image.

Images