A face anti-spoofing method based on diffusion model

Abstract

The present invention discloses a face anti-spoofing method based on a diffusion model, and relates to the technical field of face anti-spoofing. The present invention learns to reconstruct fake textures through the training of a diffusion model, and then introduces various real face identity information in the sampling process, so as to generate diversified and effective fake faces; specifically, the present invention uses a fake sample as a condition to guide the generation of new fake faces through a conditional diffusion model; to ensure the learning of fake textures, only fake data is used to train the diffusion model, and by designing a fake style transfer module, the interference of the face information carried by the fake data on the generation process is reduced; in the sampling stage, a noisy version of the real face is used as initialization, and denoising sampling is performed to obtain a generated sample with the same fake texture as the guided fake sample and the same identity as the real face. The final generated data can be trained together with the original data through a classification detection network to obtain a classification model.

Description

Technical Field

[0001] The present invention relates to the technical field of face anti-spoofing, and in particular to a face anti-spoofing method based on a diffusion model. Background Art

[0002] In the field of emotional micro-gesture recognition, it is crucial to accurately identify subtle changes in facial expressions. However, the data collection process is inevitably mixed with static, non-real face data, such as masks or photos, which is extremely unfavorable for model training because it cannot accurately reflect the dynamics and complexity of human emotions. To meet this challenge, Face Anti-Spoofing Technology (FAS) has become a viable solution, which excludes non-real data through deep learning and biometric recognition technology, thereby ensuring the quality and authenticity of the training set.

[0003] With the emergence of academic datasets in the past decade, deep learning-based FAS techniques have achieved significant performance improvements; however, these methods often overfit to data with limited scenes or lighting conditions, resulting in a sharp drop in performance in unseen scenes. In order to improve generalization capabilities, a series of cross-domain training methods based on multi-source domain data have emerged. However, the application of multi-source domain data has brought about the problem of domain shift, which is mainly reflected in the differences in image style and image quality between datasets. Image quality can be quantified by factors such as image degradation, blur, and low resolution, while image style is described by the statistical characteristics of local images, reflecting the color and texture differences between different attack types. The difference in quality and style introduces incorrect prior information to the network, posing a huge challenge to learning accurate deception cues.

[0004] Existing domain generalization methods attempt to solve the domain shift problem by introducing human prior knowledge, such as domain-specific features and identity information should be regarded as interference, and then design a dedicated loss function to make the network ignore it. Such human prior knowledge cannot be quantified to make the network accurately ignore irrelevant information, resulting in difficulty in learning some key forgery clues. Image quality can be used as a quantifiable metric feature to directly combat domain shift, because high-quality forged images present richer forgery details for network learning; for image style factors that are difficult to quantify in domain shift, explicit generation methods can be used as a solution; and existing FAS systems are easily affected by new attack methods, but academic datasets are difficult to quickly collect and update new attack methods, and explicit generation methods can effectively improve this problem, using a small number of new attack samples to generate a large number of new samples of the same type. Existing explicit generation methods are mainly based on generative adversarial networks (GANs) and variational autoencoders (VAEs). GAN-based methods perform well in transferring delicate attack textures, but cannot solve the domain shift problem between print and replay attacks, making them unsuitable for domain generalization tasks of these attack types. On the other hand, VAE-based methods often cause local texture distortion due to their limited generation capabilities, which limits their application in RGB-based FAS methods. Existing explicit methods are prone to overfitting to limited data; for example, when a face image is input, the spoofed face generated by the current model is highly similar to the original spoofed mask in the dataset. This generation method leads to the loss of identity features from the face and poor generation diversity, so it is of limited help to FAS tasks.

[0005] Diffusion models provide a feasible solution, showing stronger generation quality and diversity compared to GAN and VAE. However, the denoising process of the diffusion model is reflected in the weakening process of high-frequency information, which also includes texture information as a key discriminant clue in the FAS task. The data generated by directly using the fine-tuned stable diffusion model does not have the correct attack texture and cannot bring performance improvement to the cross-domain FAS task; therefore, designing a diffusion model that focuses on texture generation is a research direction worth exploring. Summary of the invention

[0006] The purpose of the present invention is to propose a face anti-spoofing method based on a diffusion model to solve the following problems existing in the existing cross-domain FAS data generation method:

[0007] (1) Since academic datasets are usually small in size, existing generation methods suffer from severe overfitting. The generated data are highly similar to the forged masks in the original dataset and lose the identity information of the original face, thus limiting the diversity of the generated data and failing to improve the performance of cross-domain FAS tasks.

[0008] (2) Due to the inherent limitations of the denoising process, the existing generation method based on the diffusion model cannot generate fake faces with fake textures in a targeted manner, resulting in the generated data not having practical application value. In addition, fine-tuning all parameters on academic datasets also faces the problem of overfitting.

[0009] In order to achieve the above object, the present invention adopts the following technical solutions:

[0010] A face anti-spoofing method based on a diffusion model comprises the following steps:

[0011] S1. Crop the original face images in the data set using a face detection algorithm, remove the background information of the image, and adjust the processed face images to the same size;

[0012] S2. According to the forged labels provided by the dataset, a residual neural network encoder is trained to classify the forged samples and obtain the features of the middle layer as the potential expression of the forged texture information;

[0013] S3, performing data augmentation on the processed forged face images by randomly horizontally flipping them to enrich data diversity;

[0014] S4, adopting the improved diffusion model (IDDPM) as the diffusion model backbone, by introducing the latent expression of forged information in the cross attention of each block as a conditional control, and training with the reconstruction of various forged samples as the learning goal;

[0015] S5, extracting the predicted noise of the backbone output of the diffusion model, and calculating the mean square error loss function with the actual noise;

[0016] S6, optimizing all parameters of the diffusion model by gradient backpropagation, so that the diffusion model fits the forged texture information;

[0017] S7, after the diffusion model training converges, the real face image is denoised according to the forward process formula of the diffusion model as the initialization state of the sampling;

[0018] S8, input the real image after adding noise into the diffusion model backbone, and input a forged sample into the ResNet encoder to obtain the forged texture encoding of the forged sample, and inject it into the network backbone through cross attention;

[0019] S9, iterating the denoising process of the diffusion model to obtain the final sampling sample;

[0020] S10. After sampling to obtain a large number of generated samples, the generated samples are mixed with the original data set samples to train the classification model. Quality priors are added to the network during the classification process to combat the domain shift problem from the perspective of quality and style.

[0021] Preferably, the training of a residual neural network encoder to classify forged samples in S2 specifically refers to: supervising the network through a cross entropy loss function, mapping the forgery clues provided by a single image into a set of multi-level features of different sizes, and the specific formula is expressed as:

[0022] Cond1,Cond2,…,Cond n =Resnet(X)

[0023] Among them, Resnet represents the residual neural network encoder used; X represents the forged sample RGB image to be encoded; Cond 1～n Represents the multi-level feature maps of different sizes extracted.

[0024] Preferably, the S4 further includes the following contents:

[0025] S4.1. Use the diffusion model to add noise to the sample. The prior noise probability q(y t / y (t-1) )The specific formula is as follows:

[0026]

[0027] Among them, t~[1,T]; β1, β2,…,β T Represents a fixed variance sequence, iterating until the original image y0 is degraded to Gaussian noise y t ; posterior p(y (t-1) / y t ) by training a U-shaped neural network model ε θ (y t ,t) is used to fit and predict the initial noise ε, and the mean square error loss is used as the loss function. The specific formula is as follows:

[0028]

[0029] S4.2. During the denoising process, Gaussian noise is sampled. The specific formula is as follows:

[0030] P θ (y (t-1) / y t )=N(y (t-1) ; ε θ (yt ,t))

[0031] S4.3. The generation goal is summarized as follows: using the attack method from one domain as a guiding sample, a live sample from another domain is used to generate a fake sample with similar fake texture, while maintaining the facial features of the live sample; only the fake samples are used for training during the diffusion process, so that the model fits the texture information of various attacks;

[0032] S4.4. Change the self-attention layer of U-Net to the cross-attention layer, and introduce the obtained multi-level features; use the style transfer method to expand the feature map into blocks, and then calculate the mean and variance of each block respectively, and use the obtained mean-variance sequence as the representation of the forged texture style, and calculate the cross-attention with the mean-variance sequence obtained in the same way as the backbone feature map of the diffusion model. The formula of the above process is expressed as follows:

[0033] μ f =Attn(Q(μ x ),KV(μ cond ))σ f =Attn(Q(σ x ),KV(σ cond ))

[0034] Among them, Attn represents the attention operation; Q, K, V represent the queue, key and value in the attention calculation respectively; μ and σ represent the mean and variance respectively; x, cond, f represent the backbone feature, conditional feature and fusion feature respectively;

[0035] S4.5. After obtaining the combined features, the original feature map is normalized, and the normalized feature map is normalized using the mean variance sequence obtained in S4.4; then, the convolution layer is passed and the residual connection is adopted. The specific formula is as follows:

[0036] X f =X+conv(norm(X)*σ f +μ f )

[0037] S4.5. Introduce a classifier-free guidance method to achieve controllability of forgery strength. Specifically, calculate the influence of the conditional branch at each time step at the same time during sampling, and adjust the forgery strength by increasing the multiplier. The specific formula is as follows:

[0038] p θ (X (t-1) X t )=N(X (t-1) ; ε θ (X t ,t))+γ*(N(X (t-1) ; εθ (X t ,t,X s ))-N(X (t-1) ; ε θ (X t ,t)))

[0039] Among them, X s represents the fake face as a condition; N(X (t-1) ; ε θ (X t ,t)) represents the single-step denoising result under unconditional guidance, γ*(N(X (t-1) ; ε θ (X t ,t,X s ))-N(X (t-1) ; ε θ (X t ,t))) represents the difference between the single-step denoising result under conditional guidance and the single-step denoising result under unconditional guidance, that is, the influence of the conditional branch on the sampling process; by adjusting the influence with the γ scaling factor, the control strength is adjusted.

[0040] Preferably, the S5 specifically includes the following contents:

[0041] The error between the predicted noise and the actual noise is reduced by the mean square error loss function, thereby improving the performance of the denoising process. The specific formula is as follows:

[0042]

[0043] In the formula, ε represents the actual noise; ε θ (y t ,t) represents the given noise image y t and the predicted noise of the model under the condition of time step t; the formula as a whole represents the expectation of the L2 loss for the predicted noise and the actual noise.

[0044] Preferably, the noise-added forward q(y t / y (t-1) ) The probability formula specifically refers to:

[0045]

[0046] Among them, t~[1,T]; β1, β2,…,β T Represents a fixed variance sequence, iterating until the original image y0 is degraded to a blurred face image y with random Gaussian noise T .

[0047] Preferably, the S9 specifically includes the following contents:

[0048] S8 is executed repeatedly at t~[T,1], where T is the same as the noise addition number in S7.

[0049] Preferably, the S10 specifically includes the following contents:

[0050] S10.1. Calculate the quality score b of each image in the training batch using the BRISQUE method aq , the mass fraction b aq Proportional to image quality;

[0051] S10.2. Calculate the standardized quality score b of the images in each batch norm , the standardized quality score b norm It is proportional to the image quality. The specific formula is as follows:

[0052] b norm =(b mean -b aq ) / (b std +eps)

[0053] Where b mean 、b std They represent the mean and variance of the batch scores respectively; eps represents the stability coefficient;

[0054] S10.3. According to the 3σ principle, the standardized mass fraction is compressed into the range of [-n1 / N, n2 / N], denoted as b rq , the specific formula is as follows:

[0055] b rq = torch.clip(b norm / 3,-n1 / N,n2 / N)

[0056] Where N is the total number of samples; n1 is the number of samples with scores lower than the average score; n2 is the number of samples with scores higher than the average score; torch.clip means to normalize and compress the sequence values ​​into the range of [-n1 / N,n2 / N];

[0057] S10.4. Calculate the loss and apply a larger classification threshold for high-quality images. The specific formula is as follows:

[0058] ψ=-s×b rq ,ω=s×(1+b rq ),

[0059] g(θ)=cos(θ-s×b rq )-s×(1+b rq ),

[0060]

[0061] Among them, s represents the threshold coefficient; m represents the loss scaling coefficient; compared with the cross entropy loss function, L RQ Adjust cos(θ) to g(θ) and embed quality-related information into the function, so that for samples with higher relative quality scores, the equivalent classification threshold cos(θ)-g(θ) is larger, thereby achieving the goal of applying a larger classification threshold to samples with higher quality, and mining forgery discrimination clues from high-quality samples;

[0062] S10.5. By backpropagating and optimizing the neural network, the trained neural network is used to monitor new counterfeit scenarios or unknown attack methods.

[0063] Compared with the prior art, the present invention provides a face anti-spoofing method based on a diffusion model, which has the following beneficial effects:

[0064] (1) The present invention is different from the previous forged face generation method by learning forgery traces from real-forged face pairs. It is not limited by the face alignment requirement and can effectively reduce the limitation of overfitting on the diversity of generated forged faces, thereby generating new forged samples with different forgery strengths and diverse identity information of the original face.

[0065] (2) Different from the previous FAS method that improves the generalization performance by injecting human priors into the classification neural network (such as hindering the learning of domain-specific information and face information), the present invention measures the network's information mining intensity for the image by image quality, and then reduces the domain style deviation between data sets by style fusion shuffling between multiple data sets, thereby improving the domain generalization ability of the trained classification model.

[0066] (3) Different from the previous methods of improving classification performance by introducing quality priors, the present invention takes into account the uneven quality distribution of the FAS dataset, improves the robustness to uneven sampling during training, and enables the network to adaptively focus on the samples with the highest quality in the batch, achieving more effective results.

[0067] (4) Considering that in actual deployment scenarios, forged samples require a lot of manpower and material resources to collect, while real faces can be found everywhere and are easy to collect on a large scale, the present invention can work with only 10-20 samples of new attack methods, thereby greatly reducing the difficulty for the industry to collect this type of data set when a new attack method appears, and improving the detection performance of new attack methods in actual deployment scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0068] Figure 1 This is a flow chart of the design framework of a diffusion model-based face anti-spoofing method proposed in Example 1 of the present invention;

[0069] Figure 2This is a diagram showing the generation effect of the cross-domain and cross-attack proposed in Example 2 of the present invention. DETAILED DESCRIPTION

[0070] The technical solutions in the embodiments of the present invention will be described clearly and completely below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments.

[0071] The present invention designs an adaptation module dedicated to the generation of fake faces that takes into account the characteristics of FAS fake textures. The diffusion model is efficiently trained only through a small-scale academic data set to overcome the overfitting problem and learn accurate FAS fake textures. The present invention designs a method for generating face anti-spoofing data based on a few-sample training of a diffusion model. Different from the previous method based on GAN and VAE that strips and learns fake clues by aligning real faces and fake faces, the present invention breaks this inherent thinking and explores for the first time the training of a diffusion model to learn fake texture reconstruction, and then introduces a variety of real face information in the sampling process to generate diversified and effective fake faces; specifically, the present invention uses a fake sample as a condition to guide the generation of new fake faces through a conditional diffusion model; to ensure the learning of fake textures, only fake data is used to train the diffusion model, and a fake style transfer module is designed to reduce the interference of the face information carried by the fake data on the generation process; the sampling stage uses a noisy version of the real face as initialization, performs denoising sampling, and obtains a generated sample with the same fake texture as the guided fake sample. The final generated data can be trained together with the original data through a classification detection network to obtain a classification model.

[0072] The patent of this invention is funded by the National Natural Science Foundation of China - Human Micro-gesture Recognition and Emotion Analysis Based on Self-supervised Learning 62171309. The following is a description of a human face anti-spoofing method based on a diffusion model protected by the present invention in conjunction with relevant drawings and specific examples. The specific contents are as follows.

[0073] Embodiment 1:

[0074] See also Figure 1 ,This invention proposes a face anti-spoofing method based on diffusion model, and the overall process is divided into three parts:

[0075] 1) Conditional encoder module: The input fake face as a guide is mapped into a set of multi-level texture features of size 32×32×512, 16×16×512, and 8×8×512 for subsequent guidance diffusion generation.

[0076] 2) Diffusion model module. A cross-attention layer is embedded in each sub-block of the diffusion model to obtain the forged texture information from the conditional encoder. The local mean and variance of the forged texture feature map are learned separately through the forged style fusion module, and the learned local mean variance parameters are used to replace the local mean variance of the backbone feature map, so that the diffusion model backbone can better learn the forged texture features and adapt to the forged texture generation task.

[0077] 3) Sampling module: The real face is denoised, and a specified fake guidance image is encoded. The denoised real face and the encoding result are input into the model for iterative sampling to obtain the generated face.

[0078] For the details in each of the above modules, the specific implementation steps are as follows:

[0079] Step 1: Use the Retinaface face detector commonly used in the field of face recognition, and use the same cropping method for live and fake images to crop out the face information and remove the interference of background information. Then resize the cropped face image to 256×256.

[0080] Step 2: According to the forged labels provided by the dataset, the conditional encoder (ResNet) is trained to classify the forged samples and obtain a set of multi-level texture features of sizes 32×32×512, 16×16×512, and 8×8×512 in its middle layer as the potential expression of the forged texture information.

[0081] Step 3: In order to enrich the data diversity, the transformed face images are augmented. The data augmentation method includes randomly flipping the horizontal image.

[0082] Step 4: Using IDDPM as the backbone of the diffusion model, we introduce the potential expression of forged information in the cross-attention of each block as a conditional control and train it with the reconstruction of various forged samples as the learning goal. Specifically, the diffusion model first adds noise to the sample according to the following formula:

[0083]

[0084] Among them, t~[1,T]; β1, β2,…,β T Represents a fixed variance sequence, iterating until the original image y0 is degraded to Gaussian noise y t ; posterior p(y (t-1) / y t ) by training a U-shaped neural network model ε θ To fit, it is used to predict the initial noise, and the mean square error (MSE) loss is used as the loss function. The specific formula is as follows:

[0085]

[0086] Later in the denoising process, the Gaussian noise is sampled using the following formula:

[0087] P θ (y (t-1) / y t )=N(y (t-1) ; ε θ (y t ,t))

[0088] The generation goal can be summarized as: using the attack method from one domain as a guide sample, the live sample from another domain generates a fake sample with similar fake texture, while maintaining the facial features of the live sample. Directly using the diffusion model for this task will lead to local texture degradation and edge blurring problems. Therefore, only fake samples are used for training in the diffusion process to make the model fit the texture information of various attacks.

[0089] In order to enable the model to generate samples based on conditional images, the self-attention layer of U-Net is changed to a cross-attention layer, and the multi-level feature map extracted in step 2 is introduced; at the same time, in order to reduce the impact of forged face information on the diffusion model learning texture information, the style transfer method is used to expand the feature map into blocks, and then the mean and variance of each block are calculated respectively to obtain the mean variance sequence as the representation of the forged texture style, and the cross-attention is calculated by the mean variance sequence obtained in the same way as the backbone feature map of the diffusion model. This process can be expressed by the following formula:

[0090] μ f =Attn(Q(μ x ),KV(μ cond ))σ f =Attn(Q(σ x ),KV(σ cond ))

[0091] Among them, Attn represents the attention operation; since the n-neighborhood variance reflects the image texture characteristics, and the mean, similar to pooling, tends to retain the image color style, the variance contains richer texture characteristics than the mean, so we use a larger block size for the mean. This operation can minimize the dependence of the diffusion model on forged face information. After obtaining the combined features, the original feature map is normalized, and the normalized feature map is normalized using the obtained mean variance sequence. Then it passes through the convolution layer and uses a residual connection. It can be expressed as follows:

[0092] X f =X+conv(norm(X)*σ f +μ f )

[0093] The above operations constitute the cross attention unit of the diffusion model, which is implemented in a network layer with a size of less than 32×32. Since too high forgery strength will cause texture distortion, a classifier-free guidance method is introduced to achieve controllability of forgery strength. Specifically, the control condition is set to 0 with probability p, so that the network can learn to generate with and without conditions at the same time. In this way, the influence of the conditional branch at each time step can be calculated simultaneously during sampling, and the forgery strength can be adjusted by increasing the multiplier. The formula for this process is expressed as follows:

[0094] p θ (X (t-1) |X t )=N(X (t-1) ; ε θ (X t ,t))+γ*(N(X (t-1) ; ε θ (X t ,t,X s ))-N(X (t-1) ; ε θ (X t ,t)))

[0095] Among them, X s represents the fake face as a condition; N(X (t-1) ; ε θ (X t ,t)) represents the single-step denoising result under unconditional guidance, γ*(N(X (t-1) ; ε θ (X t ,t,X s ))-N(X (t-1) ; ε θ (X t ,t))) represents the difference between the single-step denoising result under conditional guidance and the single-step denoising result under unconditional guidance, that is, the influence of the conditional branch on the sampling process; by adjusting the influence with the γ scaling factor, the control strength is adjusted.

[0096] Step 5: Extract the noise predicted by the diffusion model backbone and calculate the MSE loss with the actual noise. Specifically, the MSE loss is used to reduce the error between the predicted noise and the actual noise, thereby improving the performance of the denoising process. The formula is as follows:

[0097]

[0098] In the formula, ε represents the actual noise; ε θ (y t ,t) represents the given noise image y tand the predicted noise of the model under the condition of time step t; the formula as a whole represents the expectation of the L2 loss for the predicted noise and the actual noise.

[0099] Step 6: Optimize all parameters of the diffusion model through gradient backpropagation so that the diffusion model fits the forged texture information. Specifically, since we only want the model to fit the forged attack texture, this method starts full parameter training from random initialization.

[0100] Step 7: After the diffusion model training converges, the real face image is denoised according to the forward process formula of the diffusion model as the initial state of sampling. Specifically, the real face image is denoised according to the following formula as the initialization of the denoising process to obtain identity information from the real face:

[0101]

[0102] Step 8: Input the real image after adding noise into the diffusion model backbone, and input a fake sample into the ResNet encoder to obtain the fake texture encoding of the fake sample, and inject it into the network backbone through cross attention. Specifically, as described in step 4, the conditional and unconditional noise prediction values ​​are calculated at each time step, and then the specific impact of the conditional branch is calculated by difference.

[0103] Step 9: Iterate the denoising process of the diffusion model to obtain the final sample. Specifically, step 8 is executed repeatedly at t~[T,1], where T is the number of noise addition steps described in step 7.

[0104] Step 10: After sampling to obtain a large number of generated samples, the generated samples and the original data set are compared using the BRISQUE method to calculate the quality score b of the images in each training batch. aq , the mass fraction b aq Proportional to image quality; sample mixing training classification model, the classification model uses residual neural network Resnet18. In the classification process, image quality prior information is added to the network, so that the network pays more attention to high-quality samples. In this way, the domain shift problem is combated from the perspective of quality and style. Specifically, the quality score b of each image in each training batch is first calculated by the BRISQUE method. aq , the mass fraction b aq Proportional to the image quality; the standardized quality score b of the images in each batch is then calculated using the following algorithm norm , standardized quality score b norm Proportional to image quality:

[0105] b norm =(b meam -b aq ) / (b std +eps)

[0106] Where b meam 、b std They represent the mean and variance of the batch score respectively; eps represents the stability coefficient; then the standardized quality score is compressed into the range of [-n1 / N, n2 / N] according to the 3σ principle. The specific formula is as follows:

[0107] B rq = torch.clip(b norm / 3,-n1 / N,n2 / N)

[0108] Among them, N is the total number of samples; n1 is the number of samples with scores lower than the average score; n2 is the number of samples with scores higher than the average score; torch.clip means to standardize and compress the sequence values ​​into the range of [-n1 / N,n2 / N].

[0109] The loss is then calculated according to the following formula, applying a larger classification threshold for high-quality images, where s is the threshold coefficient:

[0110] ψ=-s×b rq ,ω=s×(1+b rq ),

[0111] g(θ)=cos(θ-s×b rq )-s×(1+b rq ),

[0112]

[0113] Finally, the neural network is optimized through backpropagation. The trained neural network can be used to detect new counterfeit scenarios or unknown attack methods.

[0114] Embodiment 2:

[0115] Based on Example 1, but with the following differences:

[0116] This paper selects OULU-NPU, CASIA-MFSD, MSU-MFSD, Idiap-Replayattack (OCIM) datasets for cross-domain generation experiments, and PADISI-USC for cross-attack generation experiments. The cross-domain generation experiment refers to the transfer of the attack style of one domain to the live face of another domain. The experiment uses four datasets, including video attacks and print attacks of different styles. The cross-domain generation results are listed in Figure 2 First three rows: Cross-attack generation experiment targets a large number of new attack methods. More than ten attack methods provided in PADISI-USC are transferred to living faces to obtain a large number of forged augmented samples. The cross-attack generation results are listed in Figure 2 The last three lines.

[0117] For various types of mask attacks on PADISI-USC, our method is able to learn specific texture features on masks with high homogeneity in the training set and replace skin texture with attack texture while maintaining identity consistency, which can effectively enrich the mask diversity in the dataset. For face editing attacks on PADISI-USC, our method can partially edit the facial attack part without modifying the sampling algorithm. In addition, in PADISI-USC, despite being constrained by limited image pairs (20-30) during training, our model demonstrates strong texture learning capabilities. In the OCIM cross-domain generation experiment, we achieve cross-domain style transfer by leveraging universal texture features, thus emphasizing the universality of our texture-based method.

[0118] The above description is only a preferred specific implementation manner of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can make equivalent replacements or changes according to the technical scheme and inventive concept of the present invention within the technical scope disclosed by the present invention, which should be covered by the protection scope of the present invention.

Claims

1. A face anti-spoofing method based on a diffusion model, characterized in that: The following steps are involved: S1. Crop the original face images in the data set using a face detection algorithm, remove the background information of the image, and adjust the processed face images to the same size; S2. According to the forged labels provided by the data set, a residual neural network encoder is trained to classify the forged samples, and the features of the middle layer are obtained as the potential expression of the forged texture information; wherein, the training of a residual neural network encoder to classify the forged samples specifically refers to: supervising the network through a cross entropy loss function, and mapping the forged clues provided by a single image into a set of multi-level features of different sizes; S3, performing data augmentation on the processed forged face images by randomly horizontally flipping them to enrich data diversity; S4. An improved diffusion model (IDDPM) is used as the diffusion model backbone. By introducing the potential expression of forged information in the cross attention of each block as a conditional control, the reconstruction of various forged samples is used as the learning goal for training; including the following: S4.1, add noise to the sample using the diffusion model; S4.2, during the denoising process, the Gaussian noise is sampled; S4.

3. The generation goal is summarized as follows: using the attack method from one domain as a guiding sample, a live sample from another domain is used to generate a fake sample with similar fake texture, while maintaining the facial features of the live sample; only the fake samples are used for training during the diffusion process, so that the model fits the texture information of various attacks; S4.

4. Change the self-attention layer of U-Net to the cross-attention layer, and introduce the obtained multi-level features; use the style transfer method to expand the feature map into blocks, and then calculate the mean and variance of each block respectively, and use the obtained mean-variance sequence as the representation of the forged texture style, and calculate the cross-attention with the mean-variance sequence obtained in the same way as the backbone feature map of the diffusion model; S4.5, after obtaining the combined features, the original feature map is normalized, and the normalized feature map is normalized using the mean variance sequence obtained in S4.4; then, the convolution layer is passed and residual connection is adopted; S4.6, introduce a classifier-free guidance method to achieve controllability of forgery strength; S5, extracting the predicted noise of the backbone output of the diffusion model, and calculating the mean square error loss function with the actual noise; S6, optimizing all parameters of the diffusion model by gradient backpropagation, so that the diffusion model fits the forged texture information; S7, after the diffusion model training converges, the real face image is denoised according to the forward process formula of the diffusion model as the initialization state of the sampling; S8, input the real image after adding noise into the diffusion model backbone, and input a forged sample into the ResNet encoder to obtain the forged texture encoding of the forged sample, and inject it into the network backbone through cross attention; S9, iterating the denoising process of the diffusion model to obtain the final sampling sample; S10. After sampling to obtain a large number of generated samples, the generated samples are mixed with the original data set samples to train the classification model. Quality priors are added to the network during the classification process to combat the domain shift problem from the perspective of quality and style.

2. According to the diffusion model-based face anti-spoofing method of claim 1, it is characterized in that: The training of a residual neural network encoder to classify forged samples described in S2 specifically refers to: supervising the network through the cross entropy loss function, mapping the forgery clues provided by a single image into a set of multi-level features of different sizes, and the specific formula is expressed as: Cond1,Cond2,…,Cond n =Resnet(X) Among them, Resnet represents the residual neural network encoder used; X represents the forged sample RGB image to be encoded; Cond 1～n Represents the multi-level feature maps of different sizes extracted.

3. According to the diffusion model-based face anti-spoofing method of claim 2, it is characterized in that: The S4 further includes the following contents: S4.

1. Use the diffusion model to add noise to the sample. The prior noise probability q(y t / y (t-1) )The specific formula is as follows: Among them, t~[1,T]; β1, β2,…,β T Represents a fixed variance sequence, iterating until the original image y0 is degraded to Gaussian noise y t ; posterior p(y (t-1) / y t ) by training a U-shaped neural network model ε θ (y t ,t) is used to fit and predict the initial noise ε, and the mean square error loss is used as the loss function. The specific formula is as follows: S4.

2. During the denoising process, Gaussian noise is sampled. The specific formula is as follows: P θ (and (t-1) / and t )=N(and (t-1) ;e θ (and t ,t)) S4.

3. The generation goal is summarized as follows: using the attack method from one domain as a guiding sample, a live sample from another domain is used to generate a fake sample with similar fake texture, while maintaining the facial features of the live sample; only the fake samples are used for training during the diffusion process, so that the model fits the texture information of various attacks; S4.

4. Change the self-attention layer of U-Net to the cross-attention layer, and introduce the obtained multi-level features; use the style transfer method to expand the feature map into blocks, and then calculate the mean and variance of each block respectively, and use the obtained mean-variance sequence as the representation of the forged texture style, and calculate the cross-attention with the mean-variance sequence obtained in the same way as the backbone feature map of the diffusion model. The formula of the above process is expressed as follows: μ f =Attn(Q(μ x ),KV(μ cond ))σ f =Attn(Q(σ x ),KV(σ cond )) Among them, Attn represents the attention operation; Q, K, V represent the queue, key and value in the attention calculation respectively; μ and σ represent the mean and variance respectively; x, cond, f represent the backbone feature, conditional feature and fusion feature respectively; S4.

5. After obtaining the combined features, the original feature map is normalized, and the normalized feature map is normalized using the mean variance sequence obtained in S4.4; then, the convolution layer is passed and the residual connection is adopted. The specific formula is as follows: X f =X+conv(norm(X)*σ f +m f ) S4.

6. Introduce a method without classifier guidance to achieve controllability of forgery strength. Specifically, calculate the influence of conditional branches at each time step at the same time during sampling, and adjust the forgery strength by increasing the multiplier. The specific formula is as follows: p θ (X (t-1) X t )=N(X (t-1) ;ε θ (X t ,t))+γ*(N(X (t-1) ;ε θ (X t ,t,X s ))-N(X (t-1) ;ε θ (X t ,t))) Among them, X s represents the fake face as a condition; N(X (t-1) ; ε θ (X t ,t)) represents the single-step denoising result under unconditional guidance, γ*(N(X (t-1) ; ε θ (X t ,t,X s ))-N(X (t-1) ; ε θ (X t ,t))) represents the difference between the single-step denoising result under conditional guidance and the single-step denoising result under unconditional guidance, that is, the influence of the conditional branch on the sampling process; by adjusting the influence with the γ scaling factor, the control strength is adjusted.

4. The face anti-spoofing method based on diffusion model according to claim 1, characterized in that: The S5 specifically includes the following contents: The error between the predicted noise and the actual noise is reduced by the mean square error loss function, thereby improving the performance of the denoising process. The specific formula is as follows: Where, ε represents the actual noise; ε θ (y t ,t) represents the given noise image y t and the predicted noise of the model under the condition of time step t; the formula as a whole represents the expectation of the L2 loss for the predicted noise and the actual noise.

5. The face anti-spoofing method based on diffusion model according to claim 1, characterized in that: The noise-added forward q(y t / y (t-1) ) The probability formula specifically refers to: Among them, t~[1,T]; β1, β2,…,β T Represents a fixed variance sequence, iterating until the original image y0 is degraded to a blurred face image y with random Gaussian noise T .

6. The face anti-spoofing method based on diffusion model according to claim 1, characterized in that: The S9 specifically includes the following contents: S8 is executed repeatedly at t~[T,1], where T is the same as the noise addition number in S7.

7. The face anti-spoofing method based on diffusion model according to claim 1, characterized in that: The S10 specifically includes the following contents: S10.

1. Calculate the quality score b of each image in the training batch using the BRISQUE method aq , the mass fraction b aq Proportional to image quality; S10.

2. Calculate the standardized quality score b of the images in each batch norm , the standardized quality score b norm It is proportional to the image quality. The specific formula is as follows: b norm =(b mean -b aq ) / (b std +eps) Where b mean 、b std They represent the mean and variance of the batch scores respectively; eps represents the stability coefficient; S10.

3. According to the 3σ principle, the standardized mass fraction is compressed into the range of [-n1 / N, n2 / N], denoted as b rq , the specific formula is as follows: b rq =torch.clip(b norm / 3,-n1 / N,n2 / N) Where N is the total number of samples; n1 is the number of samples with scores lower than the average score; n2 is the number of samples with scores higher than the average score; torch.clip means to normalize and compress the sequence values ​​into the range of [-n1 / N,n2 / N]; S10.

4. Calculate the loss and apply a larger classification threshold for high-quality images. The specific formula is as follows: ψ=-s×b rq ,ω=s×(1+b rq ), g(θ)=cos(θ-s×b rq )-s×(1+b rq ), Among them, s represents the threshold coefficient; m represents the loss scaling coefficient; compared with the cross entropy loss function, L RQ Adjust cos(θ) to g(θ) and embed quality-related information into the function, so that for samples with higher relative quality scores, the equivalent classification threshold cos(θ)-g(θ) is larger, thereby achieving the goal of applying a larger classification threshold to samples with higher quality, and mining forgery discrimination clues from high-quality samples; S10.

5. By backpropagating and optimizing the neural network, the trained neural network is used to monitor new counterfeit scenarios or unknown attack methods.

Images