Face forgery detection method based on abnormal clue guidance

Abstract

The invention discloses a face counterfeiting detection method based on abnormal clue guidance, and relates to the technical field of image processing, and the method is characterized in that the method comprises the following steps: S1, obtaining a data set; according to the method, a FaceForensics + + face data set is used, the data set comprises two subsets, namely, an origin sequence and a manidia sequence, the origin sequence is a real face data set, and the manidia sequence is a forged face data set; s2, model training based on the de-noising diffusion probability; s21, data preprocessing; s22, performing a forward diffusion process; s23, performing a reverse denoising process; and S3, face forgery detection. According to the method, the prior distribution information of a real face can be utilized, so that the method has a good detection effect on different types of forged faces, and has relatively high robustness on image post-processing operations such as compression and filtering.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and more specifically, to a method for detecting face forgery based on abnormal clues. Background Technology

[0002] With the rapid development of deep learning technology, generative models such as generative adversarial networks, variational autoencoders, and autoregressive models have been widely applied to image generation tasks, achieving breakthroughs, especially in the fields of face generation and editing. However, the misuse of these technologies has led to serious security risks, such as fake face videos, fabricated fake news, and forged identity authentication, posing a significant threat to social trust systems and information security.

[0003] Deepfakes originated in 2017 with a video posted by Reddit user "deepfakes". In recent years, with the development of generative models such as generative adversarial networks and diffusion models, deepfake technology has made significant progress, capable of generating highly realistic fake face images and videos. These technologies have wide applications in entertainment, film and television production, and other fields, but have also brought serious problems such as the spread of misinformation, invasion of personal privacy, and social security. The rapid development of deepfake technology has made the forged content difficult to identify with the naked eye, and traditional detection methods (such as identifying visual artifacts or motion inconsistencies) are no longer sufficient. Therefore, developing efficient and reliable deepfake detection technology has become an urgent task to address security threats such as the spread of misinformation and identity fraud. However, current deepfake detection methods often rely on specific data distributions and have poor generalization ability to unseen forgery types. Therefore, in this new method, we consider how to better utilize existing real face datasets to learn the distribution of real faces, thereby improving the generalization ability and robustness of the face forgery detection model.

[0004] Existing methods for detecting face spoofing can be mainly divided into several categories: 1. Classification methods based on convolutional neural networks: These methods utilize convolutional neural networks to learn the differences between real and fake faces, achieving a binary classification task. However, these methods often rely on specific data distributions and have poor generalization ability for unseen fake types.

[0005] 2. Biometric analysis-based methods: These methods detect forged images or videos by analyzing facial physiological features (such as blinking frequency, facial muscle movements, and lighting consistency). However, these methods are mainly applicable to dynamic videos and have limited ability to detect static forged images.

[0006] 3. Frequency domain analysis-based methods: These methods detect forgery traces in fake images by analyzing high-frequency information using Fourier transform and discrete cosine transform. These methods perform well on specific datasets, but are less robust to forged images that have undergone post-processing (such as compression and filtering).

[0007] Given the limitations of the above methods, there is an urgent need for a face forgery detection method with strong generalization ability, high robustness, and strong interpretability. Summary of the Invention

[0008] The technical problem this invention aims to solve is to provide a face forgery detection method based on anomaly clues. This method utilizes a denoising diffusion probability model to recover a "real" version of the forged face, and achieves high-precision forgery detection by calculating multi-scale, multi-frequency domain, and multi-feature spatial differences between the input forged image and the recovered "real" version. This method can leverage prior distribution information of real faces, thus exhibiting good detection performance against different types of forged faces, and demonstrates strong robustness to image post-processing operations such as compression and filtering.

[0009] The present invention achieves its objective by employing the following technical solution: A face forgery detection method based on abnormal clues, characterized by the following steps: S1: Obtaining the dataset; The FaceForensics++ face dataset is used, which contains two subsets: original_sequence and manipulated_sequence. The former is a real face dataset, and the latter is a fake face dataset. S2: Model training based on denoising diffusion probability; S21: Data preprocessing; S22: Forward diffusion process; S23: Reverse noise reduction process; S3: Face spoofing detection.

[0010] As a further limitation of this technical solution, the specific steps of S22 are as follows: Gradually add noise to real face images This forms a Markov chain: (1); in: Indicates the previous time step status Under the condition that the current time step status The probability distribution; This indicates a normal distribution; It is a predefined hyperparameter that represents the time step. The variance of the added noise; It represents the identity matrix, which serves as the covariance matrix of noise in a normal distribution; After multiple time steps, the real face image gradually evolves into pure Gaussian noise.

[0011] As a further limitation of this technical solution, the specific steps of S23 are as follows: Train the U-Net model to learn the denoising process and recover realistic face images from noise: (2); in: Indicates at the given current time step Data Under the condition of predicting the previous time step Data The probability distribution; Indicates at time step and given Under the condition of prediction The mean; Indicates at time step and given Under the condition of prediction The covariance matrix; After training, given a pure noise input, the real human face can be gradually restored by removing noise.

[0012] As a further limitation of this technical solution, the specific steps of S3 are as follows: S31: Input face image ; S32: Generating a realistic version of the input face image based on a model with denoising diffusion probability. ; S33: Calculate the input face image Real version of the face image The multidimensional differences between them; S331: Multi-scale structural similarity: (3); in: It is a multi-scale structural similarity measure; S332: Frequency Domain Analysis: (4); in: It is the discrete cosine transform; The Euclidean norm is used to calculate the length of the frequency domain difference vector, and the formula is: ,in, It is a frequency domain difference vector; S333: Deep feature differences; (5); in: This represents a function used for feature extraction using a VGG network; S334: Multi-feature difference; (6); in: It is an adjustable hyperparameter; S34: Classification decision based on converter model; S341: Feature Encoding: [This will be used in the following context] As input features, they are mapped to a feature space of the same dimension through a linear transformation; S342: Self-attention calculation: Utilizes multi-head attention to capture the interaction relationships between features and calculates feature weights. ; S343: Feature fusion: Use a feedforward neural network to further integrate information and employ residual connections to improve training stability; S344: Final classification: Use the softmax function to output the classification result and determine whether the input image is fake; (7); in: It is a trainable weight matrix with dimensions of . ,in It is the hidden layer dimension of the Transformer. It is the number of categories; Indicates that the given input image is At that time, the model predicts the output category probability; This represents the category label predicted by the model.

[0013] Compared with the prior art, the advantages and positive effects of the present invention are: 1. This invention utilizes a diffusion model to recover the "real" version of a forged face, achieving high-precision face forgery detection by analyzing the differences between the input image and the recovered image. This method leverages prior distribution information of real faces, thus demonstrating good detection performance against different types of forged faces, and exhibits strong robustness to image post-processing operations such as compression and filtering.

[0014] 2. Compared with traditional deepfake detection methods, this invention does not rely on specific data distributions and proposes an anomaly clue-guided face forgery detection framework. It uses a denoising probability diffusion model to recover the "real" version of the forged face image and achieves high-precision face forgery detection by analyzing the differences between the input image and the recovered image, thus having better generalization ability. Attached Figure Description

[0015] Figure 1 This describes the training process of the diffusion model in this invention.

[0016] Figure 2 This is a flowchart of the forgery detection process based on abnormal clues of the present invention. Detailed Implementation

[0017] A specific embodiment of the present invention is described in detail below with reference to the accompanying drawings, but it should be understood that the protection scope of the present invention is not limited by the specific embodiment.

[0018] This invention includes the following steps: S1: Obtaining the dataset.

[0019] The FaceForensics++ face dataset is used, which contains two subsets: original_sequence and manipulated_sequence. The former is a real face dataset, used in this invention for training the diffusion model, while the latter is a fake face dataset, including four fake methods: Deepfakes, Face2Face, FaceSwap, and NeuralTextures, used as input to generate "real" versions of fake images in the diffusion model.

[0020] S2: Model training based on denoising diffusion probability.

[0021] A denoising probabilistic diffusion model is trained on a real face dataset, enabling it to learn the data distribution of real faces. This model is then used to generate a "real" version of the input fake face image.

[0022] S21: Data preprocessing.

[0023] A large number of high-quality face images are sampled from FaceForensics++'s original_sequence and then normalized. S22: Forward diffusion process.

[0024] The specific steps of S22 are as follows: Gradually add noise to real face images This forms a Markov chain: (1); in: Indicates the previous time step status Under the condition that the current time step status The probability distribution; This indicates a normal distribution; It is a predefined hyperparameter that represents the time step. The variance of the added noise; It represents the identity matrix, which serves as the covariance matrix of noise in a normal distribution; After multiple time steps, the real face image gradually evolves into pure Gaussian noise.

[0025] S23: Reverse denoising process.

[0026] The specific steps of S23 are as follows: The U-Net model is trained to learn the denoising process and recover the real face image from the noise (the recovered face image is the "real" version of the input face image; assuming the input face image is a fake image, the face image recovered from the noise is the "real" version of the input face image, that is, closer to the real face image): (2); in: Indicates at the given current time step Data Under the condition of predicting the previous time step Data The probability distribution; Indicates at time step and given Under the condition of prediction The mean; Indicates at time step and given Under the condition of prediction The covariance matrix; After training, given a pure noise input, the real human face can be gradually restored by removing noise.

[0027] S3: Face spoofing detection.

[0028] The specific steps of S3 are as follows: S31: Input face image ; S32: Generating a realistic version of the input face image based on a model with denoising diffusion probability. ; S33: Calculate the input face image Real version of the face image The multidimensional differences between them; By calculating the multi-scale structural similarity, frequency domain difference, and depth feature difference between the input image and the generated "real" image (the smaller the difference, the greater the probability that the input image is judged as real; conversely, the greater the difference, the greater the probability that the input image is judged as fake).

[0029] S331: Multiscale Structural Similarity (MS-SSIM): (3); in: It is a multi-scale structural similarity metric, and its value ranges from... Between these values, the closer the value is to 1, the higher the structural similarity between the two images across multiple scales, meaning the images are more similar. S332: Frequency Domain Analysis (DCT High-Frequency Residual Signal): (4); in: It is the Discrete Cosine Transform, applied to the input image. and Perform discrete cosine transforms on them respectively to convert them from the spatial domain to the frequency domain; The Euclidean norm is used to calculate the length of the frequency domain difference vector, and the formula is: ,in, It is a frequency domain difference vector. By calculating the Euclidean norm, the frequency domain difference is converted into a scalar value. ; S333: Deep feature difference (feature distance extracted by VGG); (5); in: This represents a function that uses the VGG network for feature extraction. It takes the input image and extracts the corresponding depth feature vectors by passing it through multiple layers of convolution and pooling operations of the VGG network. S334: Multi-feature difference; (6); in: It is an adjustable hyperparameter; S34: Classification decision based on transformer model; Multi-feature fusion is performed based on transformer, and finally the softmax function is used to output the classification result to determine whether the input image is fake.

[0030] S341: Feature Encoding: [This will be used in the following context] As input features, they are mapped to a feature space of the same dimension through a linear transformation; S342: Self-attention calculation: Utilizes multi-head attention to capture the interaction relationships between features and calculates feature weights. ; S343: Feature fusion: Use a feedforward neural network to further integrate information and employ residual connections to improve training stability; S344: Final classification: Use the softmax function to output the classification result and determine whether the input image is fake; (7); in: It is a trainable weight matrix with dimensions of . ,in It is the hidden layer dimension of the Transformer. It is the number of categories (in the face forgery detection task, ); Indicates that the given input image is At that time, the model predicts the output category probability; The class label represented by the model prediction, when When, it indicates that the input face image is real; when When the input face image is fake, it indicates that the input face image is fake.

[0031] The above-disclosed embodiments are merely specific examples of the present invention. However, the present invention is not limited thereto, and any variations that can be conceived by those skilled in the art should fall within the protection scope of the present invention.

Claims

1. A face forgery detection method based on abnormal clues, characterized in that, Includes the following steps: S1: Obtaining the dataset; The FaceForensics++ face dataset is used, which contains two subsets: original_sequence and manipulated_sequence. The former is a real face dataset, and the latter is a fake face dataset. S2: Model training based on denoising diffusion probability; S21: Data preprocessing; S22: Forward diffusion process; S23: Reverse noise reduction process; S3: Face forgery detection.

2. The face forgery detection method based on abnormal clues according to claim 1, characterized in that: The specific steps of S22 are as follows: Gradually add noise to real face images This forms a Markov chain: （1）； in: Indicates the previous time step status Under the condition that the current time step status The probability distribution; This indicates a normal distribution; It is a predefined hyperparameter that represents the time step. The variance of the added noise; It represents the identity matrix, which serves as the covariance matrix of noise in a normal distribution; After multiple time steps, the real face image gradually evolves into pure Gaussian noise.

3. The face forgery detection method based on abnormal clues according to claim 2, characterized in that: The specific steps of S23 are as follows: Train the U-Net model to learn the denoising process and recover realistic face images from noise: （2）； in: Indicates at the given current time step Data Under the condition of predicting the previous time step Data The probability distribution; Indicates that at time step and given Under the condition of prediction The mean; Indicates that at time step and given Under the condition of prediction The covariance matrix; After training, given a pure noise input, the real human face can be gradually restored by removing noise.

4. The face forgery detection method based on abnormal clues according to claim 2, characterized in that: The specific steps of S3 are as follows: S31: Input face image ; S32: Generating a realistic version of the input face image based on a model with denoising diffusion probability. ; S33: Calculate the input face image The real version of the face image The multidimensional differences between them; S331: Multi-scale structural similarity: （3）； in: It is a multi-scale structural similarity measure; S332: Frequency Domain Analysis: （4）； in: It is the discrete cosine transform; The Euclidean norm is used to calculate the length of the frequency domain difference vector, and the formula is: ,in, It is a frequency domain difference vector; S333: Deep feature differences; （5）； in: This represents a function used for feature extraction using a VGG network; S334: Multi-feature difference; （6）； in: It is an adjustable hyperparameter; S34: Classification decision based on converter model; S341: Feature Encoding: [The following text appears to be incomplete and requires further context:] As input features, they are mapped to a feature space of the same dimension through a linear transformation; S342: Self-attention calculation: Utilizes multi-head attention to capture the interaction relationships between features and calculates feature weights. ; S343: Feature fusion: Use a feedforward neural network to further integrate information and employ residual connections to improve training stability; S344: Final classification: Use the softmax function to output the classification result and determine whether the input image is fake; （7）； in: It is a trainable weight matrix with dimensions of . ,in It is the hidden layer dimension of the Transformer. It is the number of categories; Indicates that the given input image is At that time, the model predicts the output category The probability of; This represents the category label predicted by the model.

Images