Training-free deep counterfeit face detection method based on disturbance

Abstract

The invention discloses a disturbance-based training-free deep-counterfeit face detection method, and the method comprises the steps: carrying out the up-sampling processing of a detected face image, and adding noise after the up-sampling processing; respectively performing down-sampling processing on the image after up-sampling processing and the image after noise addition, and extracting features of the two images; detecting the detected face image according to the calculated similarity of the two image features; the problem that a traditional disturbance-based training-free method cannot detect images with different resolutions in a robust mode is solved, the sensitivity of low-resolution samples to disturbance is reduced, and therefore the detection performance is improved. Compared with a traditional method for extracting features based on the original input size, the authenticity detection efficiency is improved through down-sampling processing, and computing resources are saved. According to the method, the DINOv2 model is adopted as an image feature extractor to perform feature extraction on the detected image, and training-free robust and efficient deep fake face detection is realized.

Description

Technical Field

[0001] The present invention belongs to the technical field of image processing, and in particular relates to a perturbation-based training-free deep fake face detection method. Background Art

[0002] In recent years, deep generation methods, such as generative adversarial networks (GANs) and diffusion-based models (DMs), have attracted increasing attention in the field of computer vision and created a large number of realistic synthetic images. However, the use of these generated contents, especially fine-grained face images, raises serious risks of abuse and poses a major challenge to security in many fields. To address this issue, researchers have proposed a variety of detection methods to distinguish whether face images are real or forged.

[0003] In terms of deep fake face detection, traditional deep fake detection methods usually train deep neural networks, such as convolutional neural networks (CNNs) and visual transformers (ViTs), to extract features to solve the task of binary classification (i.e., real or fake). These methods can be roughly divided into two categories according to the final features used for classification, namely, space-based methods and frequency-based methods. The former only mines local or global traces for detection. Specifically, Chai et al. used patch-based classifiers to mine areas that are easier to detect and have good generalization ability. Similarly, researchers extracted fake clues from facial parts for detection. Multi-attention network uses spatial attention mechanism to focus on different local parts. Patch-DFD extracts five key patches around the face based on prior knowledge. In order to improve detection robustness, Mandelli et al. used multiple orthogonal networks to calculate patch-level scores, which were then aggregated and finally classified. At the same time, global spatial features are also beneficial for detection. For example, CNN-Aug achieves certain generalization ability through data augmentation. Recently, researchers applied visual-language models to deep fake detection and demonstrated its advantages in extracting global spatial features for detection. UnivFD uses the fixed visual feature space of CLIP to achieve universal forged image detection. Complementary to spatial features, frequency features describe forgery traces from another perspective. Researchers proposed a frequency-based masking strategy and achieved better results on universal forged image detection. In addition, local-global spatial feature fusion or spatial-frequency feature fusion significantly improved the detection performance. However, all these methods need to be trained or fine-tuned on training samples, consuming a lot of computing resources to obtain satisfactory detection performance. This is extremely unfavorable for large-scale training datasets. In order to save computing resources, training-free deep forgery detection methods have emerged in recent years. UnivFD uses the nearest neighbor algorithm to determine whether the test sample is real or synthetic. However, the nearest neighbor search requires labeled training samples. AEROBLADE detects authenticity by calculating the similarity between the input reconstructed by the autoencoder of pre-trained DMs and the original input. Obviously, AEROBLADE only shows satisfactory detection performance on images synthesized by DMs. Therefore, RIGID proposes a perturbation-based method to measure the similarity between the original input and the perturbed image with the help of a model. This method has achieved good detection performance on natural photos, but when it is applied to deep fake face detection, especially for face images of different resolutions, the detection effectiveness is significantly reduced, and there are problems such as poor robustness (i.e., unstable detection performance for images of different resolutions) and low efficiency (i.e., the detection speed drops sharply as the image size increases). In addition, due to the high computational complexity of the basic model (such as DINOv2), this method still consumes a lot of computing resources. Summary of the invention

[0004] The purpose of the present invention is to provide a perturbation-based, training-free deep fake face detection method.

[0005] In a first aspect, the present invention provides a perturbation-based training-free deep fake face detection method, which comprises the following steps: Step 1: Get the face image to be tested, perform upsampling on the face image to obtain an interpolated image. ; Step 2: Add noise to the interpolated image to obtain the perturbed sample ; Step 3: Interpolate the images and the perturbation sample Perform downsampling to obtain the processed image and ; Step 4: Extract images separately through image feature extractor and features, and obtain the similarity R between the two features; if the similarity R is less than the identification threshold , then the tested face image is a forged image; otherwise, the tested face image is a real image.

[0006] Preferably, in the step 1, the interpolation image It is expressed as: in, is the face image of the person being tested; is the resolution of the tested face image; is the threshold for triggering the upsampling operation; is an upsampling operation.

[0007] Preferably, in step 1, the threshold The value is 1024×1024.

[0008] Preferably, in step 2, the sample is disturbed It is expressed as: in, is additive noise; To control the intensity.

[0009] Preferably, the control intensity is 0.08; the identification threshold is 0.95.

[0010] Preferably, in the step 4, the similarity R between the two features is obtained by a cosine similarity function.

[0011] Preferably, in step 4, the image feature extractor adopts the DINOv2 model.

[0012] Preferably, in step 2, the noise is randomly sampled from a standard normal Gaussian distribution.

[0013] In a second aspect, the present invention provides a computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the memory stores the computer program; and the processor executes the above-mentioned perturbation-based training-free deep fake face detection method.

[0014] In a third aspect, the present invention provides a readable storage medium storing a computer program; when the computer program is executed by a processor, it is used to implement the above-mentioned perturbation-based training-free deep fake face detection method.

[0015] The present invention has the following beneficial effects: 1. The present invention processes the face image to be tested by upsampling-perturbation downsampling in sequence, thereby solving the problem that the traditional perturbation-based training-free method cannot robustly detect images of different resolutions, reducing the sensitivity of low-resolution samples to perturbations, and thus improving the detection performance.

[0016] 2. Compared with the traditional method of extracting features based on the original input size, the present invention improves the authenticity detection efficiency through downsampling processing, saves computing resources, and realizes efficient and robust deep fake face detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 It is the overall flow chart of the present invention.

[0018] Figure 2 It is a structural schematic diagram of the DINOv2 model in the present invention.

[0019] Figure 3 This is a comparison chart of the accuracy of the present invention and the RIGID model.

[0020] Figure 4 The figures are comparison diagrams of detection results for different hyperparameter values; (a) is a comparison diagram of detection results corresponding to different control strengths; (b) is a comparison diagram of detection results corresponding to different identification thresholds; (c) is a comparison diagram of detection results corresponding to different thresholds for triggering upsampling operations. DETAILED DESCRIPTION

[0021] The present invention will be further described below in conjunction with the accompanying drawings.

[0022] like Figure 1As shown, a perturbation-based training-free deep fake face detection method includes the following steps: Step 1: Get the face image to be tested, and obtain the interpolated image by upsampling the face image to be tested. , avoiding low smoothness of the input content. Interpolated image It is expressed as:

[0023] (1) in, is the face image of the person being tested; is the face image under test Resolution; is the threshold for triggering the upsampling operation; is an upsampling operation.

[0024] Step 2: From the standard normal distribution In the sampled noise, in the interpolated image Add noise to obtain perturbation samples for: (2) in, is additive noise; is the control strength; I is the unit matrix.

[0025] Step 3: Use downsampling to process the interpolated image and the perturbation sample respectively to obtain the processed image and It is expressed as: (3) (4) in, is a downsampling operation.

[0026] Step 4: Figure 2 In order to measure the similarity between the original input and the perturbed input, the DINOv2 model is used to extract the images and features; the DINOv2 model includes a patch embedding module (PatchEmbedding), a position embedding module (Position Embedding) and a Transformer encoder connected in sequence; the block embedding module processes the image input to the DINOv2 model through convolution and flattening in sequence, and splices the output result with the class embedding (Class token) set by the model, and then performs position encoding through the position embedding module to obtain a spliced ​​feature map; the spliced ​​feature map is discarded and input into the Transformer encoder.

[0027] The Transformer encoder includes twenty-four layers of encoder modules connected in sequence; the output of the Transformer encoder is processed by layer normalization to obtain the output feature map; the class embedding in the output feature map is extracted as the output of the DINOv2 model.

[0028] Step 5: Use the cosine similarity function to obtain the image and The similarity R between the extracted features, the expression of similarity R is: (5) in, Represents the DINOv2 model; Represents the cosine similarity function.

[0029] If the similarity R is less than the identification threshold , the tested face image is a forged image; otherwise, the tested face image is a real image.

[0030] Step 6: Experimental results and analysis 6-1. Obtaining a dataset of fake face images, DeepFakeFaceForensics is a challenging dataset for deep fake face detection in real-world scenarios, which takes into account common post-processing (e.g., compression, blurring, adversarial samples, manipulation, etc.) and advanced generative models (e.g., GAN-based, DM-based, ViT-based, etc.). The dataset includes 6 subsets, namely original samples, common post-processing samples, face mixing, anti-forensics, and multi-image compression, with a total of 46,400 synthetic images. For evaluation, 1,000 real images were randomly selected from the FFHQ real face image dataset.

[0031] Table 1 Comparison between subsets and enhanced subsets

[0032] As shown in Table 1, the original subset only contains 1024×1024 high-resolution real images. The enhanced DeepFakeFace Forensics is a dataset that is reconstructed by downsampling and enhancing the original real images (i.e., 1024×1024 resolution) to generate samples with lower resolutions of 512×512 and 256×256. In addition, more real images are selected from the FFHQ real face image dataset to make the number of real and fake images equal, forming a mixed face subset. Since the resolution of the samples usually exceeds 2560×2560, the computational burden of RIGID (i.e., minutes per image) is large, so this subset is not enhanced. This enhanced and balanced dataset can fairly evaluate the robustness of the detection method.

[0033] The present invention adopts the DINOv2 model as the image feature extractor and sets the identification thresholds respectively. is 0.95, the noise intensity λ is 0.08, and the threshold R thr The present invention and RIGID (based on simple perturbation) are used to identify the forged face image and the real image respectively. The identification results are as follows: Figure 3 As shown in Table 2;

[0034] Table 2 Identification results of the present invention and RIGID

[0035] As can be seen from the figure, although the perturbation-based training-free method shows good results in detecting natural images, its detection effectiveness is significantly reduced when applied to deep fake face detection, especially for face images of different resolutions. Specifically, high-resolution face images are more easily recognized as real. In addition, due to the high computational complexity of the basic model, it usually takes a lot of computing resources. For example, RIGID takes an average of 268 milliseconds to process an image in the DeepFakeFaceForensics dataset, which seriously hinders the application of this method in real scenarios. When simply using downsampling operations for enhancement, the existing methods perform poorly on the enhanced dataset. RIGID has an accuracy (Acc) of only 0.144 on real samples with a resolution of 256×256. The results show that the existing perturbation-based training-free methods perform poorly in deep fake face detection.

[0036] The overall accuracy (OA) and area under the ROC curve (AUC) are used to evaluate the identification results. Since the accuracy and AUC are fully measured in each subset, it is different from GLFF (a method based on global and local fusion) in which the average of all generated models in each subset is calculated. Different evaluation indicators of different models are shown in Tables 3 and 4.

[0037] Table 3 AUC results on the enhanced dataset

[0038] Table 4 OA results on enhanced datasets

[0039] As can be seen from Tables 3 and 4, the proposed method achieves the best detection performance in both OA (overall accuracy) and AUC, even surpassing the training-based method. Secondly, the proposed method achieves the top 2 scores and 1 second-place score in 5 subsets. For the anti-forensics and hybrid subsets, GLFF has the highest AUC and OA scores, while RIGID performs consistently with the proposed method, indicating that the training-free method has limitations in detecting these challenging deep fake face images.

[0040] 3-2. Ablation experiment Detailed experiments were performed with different values ​​of each hyperparameter and Figure 4 The detection results (i.e., AUC and OA) are shown in Figure 2. Experiments show that the method is robust to changes in the noise intensity λ. However, the performance will decrease with the threshold S. thr and R thr In this embodiment, the noise intensity λ, the threshold S thr and R thr The values ​​of are set to 0.08, 0.95, and 1024×1024 respectively.

Claims

1. A perturbation-based, training-free deep fake face detection method, characterized in that: The following steps are involved: Step 1: Get the face image to be tested, perform upsampling on the face image to obtain an interpolated image. ; Step 2: Add noise to the interpolated image to obtain the perturbed sample ; Step 3: Interpolate the images and the perturbation sample Perform downsampling to obtain the processed image and ; Step 4: Extract images separately through image feature extractor and features, and obtain the similarity R between the two features; if the similarity R is less than the identification threshold , then the tested face image is a forged image; otherwise, the tested face image is a real image.

2. The perturbation-based training-free deep fake face detection method according to claim 1, characterized in that: In the step 1, the interpolation image It is expressed as: ； in, is the face image of the person being tested; is the resolution of the tested face image; is the threshold for triggering the upsampling operation; is an upsampling operation.

3. The perturbation-based training-free deep fake face detection method according to claim 2, characterized in that: In step 1, the threshold The value is 1024×1024.

4. The perturbation-based training-free deep fake face detection method according to claim 1, characterized in that: In the step 2, the perturbation sample It is expressed as: ； in, is additive noise; To control the intensity.

5. The perturbation-based training-free deep fake face detection method according to claim 4, characterized in that: The control intensity is 0.08; the identification threshold is 0.

95.

6. The perturbation-based training-free deep fake face detection method according to claim 1, characterized in that: In the step 4, the similarity R between the two features is obtained by using a cosine similarity function.

7. The perturbation-based training-free deep fake face detection method according to claim 1, characterized in that: In the step 4, the image feature extractor adopts the DINOv2 model.

8. The perturbation-based training-free deep fake face detection method according to claim 1, characterized in that: In the step 2, the noise is randomly sampled from a standard normal distribution Gaussian distribution.

9. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: The memory stores computer programs; the processor executes a perturbation-based, training-free deep fake face detection method as described in any one of claims 1-8.

10. A readable storage medium storing a computer program; characterized in that: When the computer program is executed by a processor, it is used to implement a perturbation-based, training-free deep fake face detection method as described in any one of claims 1 to 8.

Images