Face depth forgery detection method based on middle-level forgery feature guidance

Abstract

The invention provides a face depth forgery detection method based on middle-layer forgery feature guidance. The method comprises the following steps: firstly, extracting a middle-layer counterfeit feature FM and a middle-layer and high-layer counterfeit feature FMH of an input face image through a four-stage architecture; then, a context-guided dynamic convolution block is constructed, channel compression and spatial upsampling are performed on the FMH, and channel cascade is performed after the FMH is aligned with the FM resolution; cross-layer context information is extracted through depth separable convolution, and under semantic guidance of FM, dynamic convolution weight is utilized to adaptively focus on a key counterfeit area; and finally, inputting the fused features into a classifier, and outputting an authenticity judgment result. According to the method, irrelevant semantic interference is effectively inhibited through middle-layer feature guidance, and effective expression of forged features is enhanced; meanwhile, the guide mechanism has irrelevance of counterfeit types, can adapt to various counterfeit technologies and cross-dataset scenes, shows excellent generalization ability, and provides an effective and universal technical scheme for deep counterfeit detection in practical application.

Description

Technical Field

[0001] This invention relates to image processing technology, specifically to a method for detecting deepfake faces based on mid-layer forgery features. Background Technology

[0002] With the rapid development of artificial intelligence technologies such as Generative Adversarial Networks (GANs) and diffusion models, deepfake technology can generate highly realistic facial images and video content. The improper use of this technology poses a serious threat to individual rights and social stability. Therefore, developing efficient and accurate deepfake detection methods is of significant practical importance.

[0003] Deepfake detection aims to determine whether facial content in video images has been altered using deepfake techniques. Existing deepfake detection methods are mainly based on spatial, frequency, and temporal domain analysis, identifying anomalies in color distribution, illumination consistency, texture patterns, and biometric features to distinguish between genuine and fake images. While these methods have achieved good results on specific datasets, the task of deepfake detection still faces serious challenges due to the diversification of generative models and the increasing complexity of post-processing techniques. Firstly, there is the problem of semantic interference. In deep neural networks, high-level features often contain rich semantic information, but this information may include a large amount of content irrelevant to forgery detection, such as facial identity and facial expressions. This irrelevant semantics can interfere with the model's recognition of forgery traces. Secondly, there is the issue of feature alignment bias. During multi-scale feature fusion, features at different levels are spatially and semantically misaligned, leading to low feature fusion efficiency and the potential loss of important forgery information.

[0004] To address the aforementioned issues, existing research has proposed introducing attention mechanisms to enhance the model's focus on key regions. However, most existing attention mechanisms calculate attention weights based on features of the current layer, lacking cross-layer semantic guidance and failing to effectively utilize the rich structural information contained in mid-layer features. Therefore, this invention aims to design a deep face forgery detection method guided by mid-layer features. By designing a context-guided dynamic convolution mechanism, it utilizes the semantic context information provided by mid-layer features to adaptively guide the learning process of high-level features, enabling the model to focus on key regions related to forgery detection, effectively improving detection accuracy and generalization ability. Summary of the Invention

[0005] The purpose of this invention is to solve the problem of severe high-level semantic interference and feature alignment deviation in deepfake detection, and to provide a face deepfake detection method guided by mid-level features. This method can effectively utilize the semantic information of mid-level features to guide the learning of high-level features, suppress irrelevant semantic interference, and improve the model's sensitivity to forgery traces and detection accuracy.

[0006] To achieve the above objectives, this invention provides a deepfake face detection method guided by mid-level forgery features, mainly comprising the following six parts: the first part is preprocessing the video dataset; the second part is extracting shallow, mid-level, and mid-to-high-level forgery features from face images using a four-stage convolutional neural network; the third part is context-guided feature fusion; the fourth part is extracting high-level forgery features; the fifth part is inputting the high-level forgery features into a classifier for binary classification of true and false images; the sixth part is end-to-end training and testing of the constructed deepfake detection model, ultimately achieving image authenticity identification, specifically: The first part consists of two steps: Step 1: Obtain the public benchmark dataset FaceForensics++ for deepfake detection. Extract 30 image samples evenly from each video. Use MTCNN for face detection, face alignment and cropping. Crop the face region to 224×224 pixels. Step 2: Divide the processed images into training, validation and test sets in a ratio of 7:2:1, and perform further data augmentation on the training image set; Part Two consists of six steps: Step 3: The face image training set samples obtained in Step 2 are processed through a 4×4 convolutional layer with a stride of 4 to reduce the image resolution to 1 / 4 of the original size, thereby compressing the feature dimension. Step 4: The feature map from Step 3 is processed by the feature extraction module to obtain the shallow forgery feature FL; Step 5: The shallow forgery features obtained in Step 4 are processed by a downsampling module to reduce the image resolution by half. Step 6: The feature map from Step 5 is processed by the feature extraction module to generate the mid-layer forgery feature FM; Step 7: The mid-layer forgery feature FM from Step 6 is passed through a downsampling module to reduce the image resolution by half; Step 8: The feature map obtained in step 7 is processed by the feature extraction module to generate the mid-to-high-level forgery feature FMH; Part Three includes a step: Step 9: The mid-level forged feature FM from Step 6 and the mid-to-high-level forged feature FMH from Step 8 are taken as input, passed through a context-guided dynamic convolution block, and finally output the enhanced fusion feature. Part Four consists of two steps: Step 10: The fusion features from step 9 are processed by a downsampling module to reduce the image resolution by half; Step 11: The feature map from step 10 is processed by the feature extraction module to obtain the high-level forgery feature FH; Part Five includes one step: Step 12: Input the high-level forgery feature FH from Step 11 into the classifier to obtain the authenticity probability of the face image. For video input, obtain the video-level forgery probability by weighted summation of the probability values ​​of the frame sequence; Part VI consists of two steps: Step 13: Train the network end-to-end using the training and validation sets from Step 2, with SGD as the optimizer and a learning rate of 1e. -3 The loss function chosen is BCEWithLogitsLoss with class weights. The network is then trained to obtain the final deepfake detection model. Step 14: Evaluate the model performance on the test set to achieve deepfake detection at both the image and video levels.

[0007] The four-stage network described in this invention can adopt commonly used convolutional neural network structures, including but not limited to backbone networks such as ResNet and EfficientNet.

[0008] This invention presents a deep face forgery detection method guided by mid-level forgery features. The core innovation lies in the context-guided dynamic convolutional block, which guides high-level learning from mid-level features through semantic alignment, contextual information extraction, dynamic weight adjustment, and adaptive feature focusing. First, spatial upsampling and channel adjustment align features at different levels in both spatial and channel dimensions, creating conditions for cross-layer information interaction. Second, depthwise separable convolutions extract cross-layer contextual information from cascaded features, encoding semantic relationships between features at different scales. Third, dynamic convolutions are introduced into the residual units, with weights adaptively adjusted based on the semantic content of the mid-level features, enabling the network to focus on regions relevant to forgery detection. Finally, a combination of regularization and dynamic convolutions enhances the response to key forgery regions while preserving original feature information. This invention, through mid-level feature guidance, makes the model focus more on regions relevant to forgery detection, reducing irrelevant semantic interference and thus improving detection accuracy. The feature fusion mechanism based on mid-level feature guidance is independent of specific forgery types and can adapt to various forgery techniques and cross-dataset scenarios. Attached Figure Description

[0009] Figure 1 This is a diagram of the overall network framework of the present invention; Figure 2 This is a context-guided dynamic convolutional block for the present invention; Figure 3 Examples of four types of face spoofing techniques derived from the FaceForensics++ dataset; Figure 4 To utilize the present invention for processing Figure 3Grad-CAM heatmap for deepfake detection. Detailed Implementation

[0010] To better understand this invention, the face deepfake detection method based on mid-layer forgery features is described in more detail below with reference to specific embodiments. In the following description, detailed descriptions of existing technologies may obscure the subject matter of this invention, and such descriptions will be omitted here.

[0011] Step 1: Download the deepfake detection benchmark dataset FaceForensics++, uniformly extract 30 image samples from each video, and use MTCNN for face detection, face alignment and cropping, uniformly cropping the face region to 224×224 pixels; Step 2: The cropped images are divided into training, validation, and test sets in a 7:2:1 ratio to ensure that each set contains a considerable number of real and fake samples. Further data augmentation is performed on the training image set, including transformations such as random flipping, rotation, and compression, to generate the final training samples. Figure 1 This is a network framework diagram of the deepfake face detection method based on mid-layer forgery features of the present invention. In this embodiment, it is carried out according to the following steps: Step 3: The face image training set samples obtained in Step 2 are processed through a 4×4 convolutional layer with a stride of 4, so that the image is mapped from the low-dimensional space of RGB channels to the high-dimensional space of 32 channels, and its resolution is reduced to 1 / 4 of the original size, thus achieving feature dimension compression. Step 4: The feature map from Step 3 is processed by the feature extraction module to generate information-rich shallow forgery features FL; Step 5: The shallow forgery feature FL obtained in Step 4 is processed by a downsampling module to reduce the image resolution by half. Step 6: The feature map from Step 5 is processed by the feature extraction module to generate the mid-layer forgery feature FM; Step 7: The mid-layer forged feature FM from Step 6 is processed by a downsampling module, which doubles the number of channels in the generated feature map and halves the resolution. Step 8: Use the feature map feature extraction module obtained in Step 7 to generate the mid-to-high-level forgery feature FMH; Step 9: Taking the mid-level fake feature FM from Step 6 and the mid-to-high-level fake feature FMH from Step 8 as input, and passing them through a context-guided dynamic convolutional block, the final output is an enhanced fusion feature. The specific implementation is as follows: Step 9.1, context-guided dynamic convolutional blocks, such as Figure 2As shown in (a), FMH first compresses the number of channels using a 1×1 convolutional layer to reduce computational complexity. Then, it performs a 2x spatial upsampling through bilinear interpolation to make its spatial size consistent with that of FM. Finally, it is concatenated with FM along the channel dimension to generate the feature map FC; Step 9.2: After the FC undergoes a 3×3 depthwise separable convolution, it is added to the FC through a residual connection to obtain an enhanced feature map containing contextual information; Step 9.3, dynamic convolutional perceptron block as follows Figure 2 As shown in (b), the enhanced feature map generated in step 9.2 is passed sequentially through a residual unit that combines regularization and dynamic convolution, and a residual unit that combines regularization and convolutional feedforward network. Under the semantic guidance of FM, the enhanced feature map adaptively adjusts the receptive field and feature response mode through dynamic convolution, so as to better capture forgery traces related to mid-level features, thereby effectively perceiving and focusing on key forgery regions in the image.

[0012] Step 9.4: Divide the output features from Step 9.3 into two parts along the channel dimension, and match the number of channels of FMH and FM respectively to obtain the fused features FMH' and FM'. Step 10: The fusion feature FMH' from step 9 is passed through a downsampling module to reduce the image resolution by half; Step 11: The feature map from step 10 is processed by the feature extraction module to obtain the deep forgery feature FH; Step 12: Input the deep forgery feature FH from step 11 into the classifier to obtain the probability of the face image being real or fake. Step 13: Input the training and validation set samples from Step 2 into the network from Step 3 to Step 12, and set the network hyperparameters: learning rate 1e-3, training epochs 60, batch size 8, and SGD optimizer with weight decay of 1e-7 and gradient clipping with a threshold of 5. The learning rate is scheduled using the StepLR strategy, decaying to 0.1 times the original rate every 15 epochs, including a 5-epoch warm-up phase. The loss function is BCEWithLogitsLoss with class weights, and the weights are automatically calculated based on the distribution of the training set samples. By training the network, the final deepfake detection training model is obtained. Step 14: Input the frame images from the test set into the training model obtained in Step 13 to obtain the forgery probability value of each frame image. By weighted summing of the forgery probability values ​​of the frame image sequence, the overall forgery probability at the video level is generated.

[0013] This invention proposes a deepfake face detection method guided by mid-level forgery features. By designing a context-guided dynamic convolution mechanism, it uses mid-level forgery features rich in detail and semantics as contextual information to adaptively guide and enhance the representation learning of high-level forgery features. Through semantic guidance of mid-level forgery features, this invention effectively suppresses interference from irrelevant semantic features such as background and identity, enabling the model to adaptively focus on key forgery-related regions such as facial alteration and fusion boundaries. This significantly enhances the effective expression of forgery features and improves the accuracy of the model's authenticity judgment. Furthermore, the mid-level feature guidance mechanism relied upon by this invention is forgery type-independent, allowing the model to better adapt to various forgery techniques and cross-dataset scenarios, exhibiting excellent generalization ability. Moreover, this invention does not rely on a specific basic network structure and is easily integrated into existing detection processes. It provides a general technical solution with significant theoretical value and practical application prospects for addressing the increasingly complex and diverse deepfake threats.

[0014] Although specific embodiments of the present invention have been described above, it should be understood that the present invention is not limited to the scope of the specific embodiments. For those skilled in the art, various changes will be obvious as long as they are within the spirit and scope of the present invention as defined and determined by the appended claims. All inventions utilizing the concept of the present invention are protected.

Claims

1. A deep face forgery detection method based on mid-level forgery features is proposed. By designing a context-guided dynamic convolution mechanism, the method effectively utilizes the semantic information of mid-level features to guide high-level feature learning, thereby suppressing interference from irrelevant semantic features. This allows the model to adaptively focus on key regions related to forgery detection, thus improving the accuracy and generalization ability of deep face forgery detection. The method comprises six parts: dataset preprocessing, extraction of mid-level and mid-to-high-level forgery features from face images, context-guided feature fusion, high-level feature extraction, real / fake classification, and network model training and testing. The first part consists of two steps: Step 1: Download the FaceForensics++ public benchmark dataset for deepfake detection, uniformly extract 30 image samples from each video, and use MTCNN for face detection, face alignment and cropping, uniformly cropping the face region to 224×224 pixels; Step 2: The cropped images are divided into training, validation, and test sets in a 7:2:1 ratio to ensure that each set contains a considerable number of real and fake samples. Further data augmentation is performed on the training image set, including transformations such as random flipping, rotation, and compression, to generate the final training samples. Part Two consists of six steps: Step 3: The face image training set samples obtained in Step 2 are processed through a 4×4 convolutional layer with a stride of 4 to reduce the image resolution to 1 / 4 of the original size, thereby compressing the feature dimension. Step 4: The feature map from Step 3 is processed by the feature extraction module to generate information-rich shallow forgery features FL; Step 5: The shallow forgery features obtained in Step 4 are processed by a downsampling module to reduce the image resolution by half. Step 6: The feature map from Step 5 is processed by the feature extraction module to generate the mid-layer forgery feature FM; Step 7: The mid-layer forgery feature FM from Step 6 is passed through a downsampling module to reduce the image resolution by half; Step 8: The feature map obtained in step 7 is processed by the feature extraction module to generate the mid-to-high-level forgery feature FMH; Part Three includes a step: Step 9: Taking the mid-level fake feature FM from Step 6 and the mid-to-high-level fake feature FMH from Step 8 as input, and passing them through a context-guided dynamic convolutional block, the final output is an enhanced fusion feature. The specific implementation is as follows: (1) The FMH is compressed by a 1×1 convolutional layer to reduce computational complexity. Then, it is spatially upsampled to make it a feature map with the same resolution as the FM. After that, it is concatenated with the FM along the channel dimension to generate the feature map FC. (2) After the FC undergoes a 3×3 depthwise separable convolution, it is added to the FC to obtain an enhanced feature map containing contextual information; (3) The enhanced feature map obtained in (2) is sequentially passed through a residual unit that combines regularization and dynamic convolution, and a residual unit that combines regularization and convolutional feedforward network, so that under the semantic guidance of FM, the enhanced feature map can adaptively perceive and focus on the key forgery region in the image through dynamic convolution weights. (4) Subsequently, the output features are segmented along the number of channels to match the number of channels of FMH and FM, resulting in fused features FMH' and FM'; Part Four consists of two steps: Step 10: The fusion feature FMH' from step 9 is passed through a downsampling module to reduce the image resolution by half; Step 11: The feature map from step 10 is processed by the feature extraction module to obtain the deep forgery feature FH; Part Five includes one step: Step 12: Take the deep forgery feature FH from step 11 as input, pass it through a classifier, and obtain the probability of the face image being real or fake. Part VI consists of two steps: Step 13: Input the training and validation set samples from Step 2 into the network from Step 3 to Step 12, and set the network hyperparameters: learning rate 1e-3, training epochs 60, batch size 8, and SGD optimizer with weight decay of 1e-7 and gradient clipping with a threshold of 5. The learning rate is scheduled using the StepLR strategy, decaying to 0.1 times the original rate every 15 epochs, including a 5-epoch warm-up phase. The loss function is BCEWithLogitsLoss with class weights, and the weights are automatically calculated based on the distribution of the training set samples. By training the network, the final deepfake detection training model is obtained. Step 14: Input the frame images from the test set into the training model obtained in Step 13 to obtain the forgery probability value of each frame image. By weighted summing of the forgery probability values ​​of the frame image sequence, the overall forgery probability at the video level is generated.

2. The face deepfake detection method based on mid-layer forgery features as described in claim 1, characterized in that, In step 9, the mid-level forgery features serve as contextual information to guide the enhancement process of mid-to-high-level forgery features, focusing on key forgery regions in the image.

3. The face deepfake detection method based on mid-layer forgery features as described in claim 1, characterized in that, In step 9 (1), the mid-to-high-level forged features are subjected to channel compression and spatial upsampling so that their spatial dimensions are consistent with those of the mid-level forged features. Then, the two are concatenated along the channel dimension to generate a fused feature map.

4. The face deepfake detection method based on mid-layer forgery features as described in claim 1, characterized in that, In step 9(2), the fused feature map is subjected to depthwise separable convolution, and the processing result is added to the fused feature map to obtain an enhanced feature map containing contextual information.

5. The face deepfake detection method based on mid-layer forgery features as described in claim 1, characterized in that, In step 9 (3), the enhanced feature map is processed sequentially through the first residual unit and the second residual unit; wherein, the first residual unit combines regularization and dynamic convolution, and the second residual unit combines regularization and convolutional feedforward network, so that the enhanced feature map adaptively generates dynamic convolution weights under the semantic guidance of the mid-layer forged features, so as to better highlight the key forged regions in the image.

6. The face deepfake detection method based on mid-layer forgery features as described in claim 1, characterized in that, In step 9 (4), the segmentation operation along the channel dimension specifically involves dividing the output feature into two parts, the number of which is the same as the original number of channels of the mid-level fake features and the mid-to-high-level fake features of the context-guided dynamic convolution block input in step 9 (1).

Images