Video depth forgery detection method based on space-time inconsistency and frequency domain analysis

Abstract

The invention provides a video depth forgery detection method based on space-time inconsistency and frequency domain analysis, the forgery detection method constructs a video depth forgery detection network of an unknown environment image, and the video depth forgery detection network comprises a space-time inconsistency module, a frequency domain analysis module and a video level classifier. The space-time inconsistency module captures space-time features in video frames, through a double-flow cooperation mechanism integrating space-time inconsistency modeling and frequency domain artifact analysis, the space-time features in videos are efficiently captured, dynamic information interaction between key frames is achieved, and therefore an enough feature basis is provided for deep counterfeiting detection. And the frequency domain analysis module combines a deep neural network with time-frequency analysis and maps video frames to a frequency domain so as to realize end-to-end identification of an unknown tampering mode and provide support for understanding of fine-grained content and improvement of classification precision.

Description

Technical Field

[0001] This invention relates to the field of video deepfake detection, and more particularly to a video deepfake detection method based on spatiotemporal inconsistency and frequency domain analysis. Background Technology

[0002] The rapid development of Generative Adversarial Networks (GANs), diffusion models, and large-scale visual-language models has propelled facial generation and manipulation techniques beyond the limits of realism. Deepfake videos exhibit near-perfect visual realism in identity replacement, facial expression manipulation, and dynamic behavior synthesis, even capable of misleading professional forensic analysis. Against this backdrop, research into deepfake detection technology has become an urgent priority for protecting the credibility of digital content.

[0003] Early detection methods primarily focused on single-frame spatial artifacts, but their performance in dynamic forgery scenarios was significantly limited due to neglecting temporal consistency. In recent years, research has gradually shifted towards joint spatiotemporal modeling. However, dual-branch networks face insufficient coordination, LSTM / 3D-CNN is limited by computational complexity, and the Transformer framework is unsuitable for edge computing due to its high computational cost. While methods like Face X-ray and LipForensics alleviate dataset-specific overfitting by analyzing local features (such as blended boundary artifacts and lip movement anomalies), their generalization ability for unknown attack patterns remains insufficient. The core challenges of current deepfake video detection lie in the deep fusion of spatiotemporal features and the effective extraction of cross-domain forgery trace representations.

[0004] Currently, numerous deepfake detection methods have emerged. TALL-Swin reconstructs video frames using thumbnail layout and combines it with the Swin Transformer, achieving efficient spatiotemporal feature modeling and demonstrating high performance in cross-dataset detection tasks. The AltFreezing method improves detection generalization by alternately freezing spatiotemporal weights to enable the model to simultaneously capture spatial and temporal artifacts. The X-CLIP method extends pre-trained image models to video recognition tasks by introducing a cross-frame attention mechanism to capture long-term dependencies between video frames. The STIL framework mines video forgery traces through collaborative spatial-temporal modules (SIM / TIM) and feature compensation mechanisms (ISM). The TimeSformer method significantly improves video classification performance by applying the Transformer model to video understanding tasks through the introduction of a spatiotemporal attention mechanism. However, the main problem with this method is its high computational complexity, especially when dealing with large-scale video datasets. I3D, a two-stream dilated 3D convolutional network, significantly improves action recognition performance by expanding 2D convolutional kernels to 3D and utilizing ImageNet pre-trained parameters. FTCN, combined with the Temporal Transformer framework, detects video face forgery by fully leveraging the temporal consistency of the video.

[0005] The existing technology suffers from the following three shortcomings. First, while existing deep learning-based forgery detection techniques have shown considerable performance improvements in practical applications, the high computational overhead during model training is becoming a key bottleneck restricting technological development. Second, some mainstream methods focus on single-frame spatial artifacts, but their performance in dynamic forgery scenarios is significantly limited due to neglecting temporal consistency. Third, existing methods exhibit dual limitations in generalization performance and detection accuracy when dealing with complex or unknown tampering methods. This is mainly due to two key factors: on the one hand, it is difficult to effectively extract subtle feature representations of local regions in the spatial domain; on the other hand, the mining of frequency domain features is not deep enough, failing to fully integrate tampering trace information in multi-scale frequency bands. Summary of the Invention

[0006] To address the shortcomings of existing technologies, this invention proposes a video depth forgery detection method based on spatiotemporal inconsistency and frequency domain analysis. The method constructs a video depth forgery detection network for unknown environment images. This network comprises three parts: a spatiotemporal inconsistency module, a frequency domain analysis module, and a video-level classifier. The spatiotemporal inconsistency module captures spatiotemporal features in video frames. The frequency domain analysis module uses a two-dimensional fast Fourier transform module to map video frames to the frequency domain and enhance the dynamic range. Then, a ResNet50 network with fixed parameters is used to extract frequency domain features. Time dependencies are modeled through gated recurrent units. Finally, the frequency domain analysis features are integrated, and the video-level classifier performs a comprehensive analysis of the input probability values, including: Step 1: Prepare the dataset required for video deepfake detection, including the training set and the test set; Step 2: Preprocess the acquired dataset. The training and test sets are saved as videos. First, the dataset is processed by frame extraction. Then, the Dlib face detection algorithm is used to detect faces in each video frame. The Dlib shape predictor model is used to align and crop faces based on the detected facial landmarks. The processed face images are saved in a separate folder. Step 3: Perform data augmentation on the preprocessed training and test sets. Simultaneously, construct and initialize a deep forgery detection network based on spatiotemporal inconsistency and frequency domain analysis. The detection network consists of a spatiotemporal inconsistency module, a frequency domain analysis module, and a video-level classifier. The spatiotemporal inconsistency module uses a pre-trained X-CLIP video encoder as the backbone network and consists of two cascaded cross-frame communication transformers and a multi-frame integrated transformer. The frequency domain analysis module combines deep neural networks with time-frequency analysis to map video frames to the frequency domain, enabling end-to-end identification of unknown tampering patterns. The video-level classifier extracts features from each frame, providing support for fine-grained content understanding and improved classification accuracy, and finally outputs a probability value to predict the authenticity of the input video. Step 4: Input the video frame training set processed in Step 3 into the constructed detection network to train the network. The training process specifically includes: Step 421: The frequency domain analysis module takes the input video data, which has a batch size of B and contains T video frames, and uses a two-dimensional fast Fourier transform module to convert each RGB channel to the frequency domain, obtaining the spectrum of the t-th video frame. .

[0007] Step 422: Graph the spectrum The DC component is centered by FFT shifting, and the dynamic range is adjusted and normalized using logarithmic compression to obtain the spectral tensor of the t-th video frame. .

[0008] Step 423: Convert the spectral tensor Feature maps of the t-th video frame are obtained by extracting features using a ResNet50 backbone network pre-trained on ImageNet. ; Step 43: Analyze the spatiotemporal inconsistency features output from Steps 41 and 42. S and frequency domain analysis features F The fusion features are obtained by integrating the learned weighted fusion. Step 44: Input the fused features into the video-level classifier for processing. Through the calculation of the video-level classifier, the encoded features are transformed using a fully connected layer. The forgery detection network performs a comprehensive analysis of the input fused features to generate a probability value for predicting whether the feature is real or fake. Step 5: Using the probability values ​​output by the deepfake detection network, calculate the loss between the network output value and the real label using the cross-entropy loss function, and optimize the network parameters through backpropagation; Step 6: Steps 4 and 5 are completed after a preset number of training rounds. After each training round, the test set is sent to the trained network for testing. The objective evaluation index of the deep fake detection network at the current test is calculated, and the network parameters when the objective evaluation index is best are saved. Step 7: Select the network that performs best during the test, then feed the test set into the network, calculate the objective evaluation indicators such as accuracy, area under the curve, mean precision and F1 score, and save the detection results of the final test samples.

[0009] According to a preferred embodiment, the processing flow of the frequency domain analysis module in step 42 includes: Step 421: The frequency domain analysis module takes the input video data, which has a batch size of B and contains T video frames, and uses a two-dimensional fast Fourier transform module to convert each RGB channel to the frequency domain, obtaining the spectrum of the t-th video frame. ; Step 422: Graph the spectrum The DC component is centered by FFT shifting, and the dynamic range is adjusted and normalized using logarithmic compression to obtain the spectral tensor of the t-th video frame. ; Step 423: Convert the spectral tensor Feature maps of the t-th video frame are obtained by extracting features using a ResNet50 backbone network pre-trained on ImageNet. ; Step 424: Extract the feature maps from the T video frames. Constructing a time series { The model is constructed using a 3-layer gated recurrent unit, and then global average pooling is used to generate the final frequency domain analysis feature F.

[0010] Compared with the prior art, the beneficial effects of the present invention are as follows: 1. This invention designs a deep forgery detection method based on spatiotemporal inconsistency and frequency domain analysis. By integrating the dual-stream collaborative mechanism of spatiotemporal inconsistency modeling and frequency domain artifact analysis, it significantly outperforms existing technologies in terms of cross-domain generalization ability.

[0011] 2. The method of this invention introduces a pre-trained X-CLIP visual model as a spatiotemporal inconsistency module to efficiently capture spatiotemporal features in the video and realize dynamic information interaction between key frames, thereby providing a sufficient feature basis for deepfake detection.

[0012] 3. The frequency domain analysis module of the present invention combines deep neural networks with time-frequency analysis to map video frames to the frequency domain, so as to achieve end-to-end identification of unknown tampering patterns. Attached Figure Description

[0013] Figure 1 This is a schematic diagram of the counterfeit detection network structure of the present invention; Figure 2 This is a schematic diagram of the Dlib face feature extraction process of the present invention; Figure 3 This is a visual representation of the Dlib face feature extraction process of this invention. Detailed Implementation

[0014] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to specific embodiments and the accompanying drawings. It should be understood that these descriptions are merely exemplary and not intended to limit the scope of the invention. Furthermore, descriptions of well-known structures and techniques are omitted in the following description to avoid unnecessarily obscuring the concept of the invention.

[0015] The following is a detailed explanation with reference to the accompanying drawings.

[0016] The 2D FFT of this invention refers to a two-dimensional fast Fourier transform module.

[0017] This invention relates to the field of deepfake detection, primarily addressing the cross-domain generalization ability of deepfake video detection and the effective detection of complex forgery patterns. Deep learning, as one of the mainstream methods in deepfake video detection, has demonstrated superior performance compared to traditional methods. However, most existing neural network architectures in deep learning have limited generalization capabilities, are insufficient in capturing local details of forged videos, and are inadequate in extracting and utilizing spectral information. These limitations result in models failing to meet practical application requirements in terms of generalization ability and detection accuracy when dealing with complex or unknown forgery patterns.

[0018] Dlib is a modern C++ toolkit that contains machine learning algorithms and tools.

[0019] To address the limitations of existing deepfake detection methods in generalization and effective detection of complex forgery patterns, this invention proposes a deepfake detection network based on spatial frequency feature integration and dynamic edge optimization. Figure 1 This is a schematic diagram of the forgery detection network structure proposed in this invention. Its input data is the original video frames, and its output is the predicted real / fake result of the video frames. The resolution of the original video frames is T×3×224×224, where T represents the time dimension, 3 represents the number of channels, and 224 represents the height and width of the image, respectively.

[0020] The core and main innovation of the counterfeit detection method proposed in this invention lies in the frequency domain analysis module.

[0021] This invention proposes a forgery detection method that constructs a video deep forgery detection network for images in unknown environments. The network comprises three parts: a spatiotemporal inconsistency module, a frequency domain analysis module, and a video-level classifier. The spatiotemporal inconsistency module uses a pre-trained X-CLIP visual model as its backbone network, efficiently capturing spatiotemporal features in video frames through a cross-frame communication transformer (CCT) and a multi-frame ensemble transformer (MIT). The frequency domain analysis module first maps video frames to the frequency domain and enhances dynamic range using a 2D Fast Fourier Transform (2D FFT) module. Then, it extracts frequency domain features using a fixed-parameter ResNet50 network, models temporal dependencies through gated recurrent units (GRUs), and finally generates frequency domain analysis features that integrate spatiotemporal information and forgery traces through global average pooling. The video-level classifier extracts features from each frame, supporting fine-grained content understanding and improved classification accuracy.

[0022] Step 1: Prepare the datasets required for video deepfake detection, including training and testing sets, namely FaceForensics++, CelebDF-v1, and DFDCP datasets. FaceForensics++ is used for the training set, and CelebDF-v1 and DFDCP datasets are used for the testing set.

[0023] Download the FaceForensics++ dataset, the Celeb-DF face forgery dataset, and the DFDCP Preview dataset from the internet. All datasets contain videos ranging from 10 to 30 seconds in length. The FaceForensics++ dataset is a face forgery dataset that allows researchers to train deep learning-based methods in a supervised manner. The dataset contains operations created using four state-of-the-art methods: Face2Face, FaceSwap, DeepFakes, and NeuralTextures. Celeb-DF is a large-scale, challenging dataset for Deepfake Forensics, comprising 590 real videos (from subjects of different ages, ethnicities, and genders) collected from YouTube, along with 5639 corresponding DeepFake videos. The DFDC Preview dataset is a preview dataset for deepfake detection, containing approximately 5000 videos, some of which are real videos processed with deepfake techniques. This dataset aims to help researchers develop and evaluate deepfake detection algorithms.

[0024] Step 2: Preprocess the acquired dataset; both the training and test sets are saved as videos, so first, frame extraction is performed, and the Dlib face detection algorithm is used to detect faces in each video frame. The Dlib shape predictor model is used to align and crop faces based on the detected facial landmarks, and the processed face images are saved in a separate folder.

[0025] Figure 2 This is a schematic diagram of the Dlib face feature extraction process of the present invention; Figure 3 This is a visual representation of the Dlib face feature extraction process of this invention. Figure 3The face images used in this dataset are from a publicly available dataset. Specifically, the input video is first read frame by frame to obtain the current frame image, and then a pre-loaded Dlib face detector is used to detect faces. This detector can quickly return the position information of the face, typically defining the approximate area of ​​the face in the image as a rectangular bounding box. Subsequently, within the detected face area, Dlib's 68-point facial landmark detector is used to detect feature points, accurately locating 68 key points of the face, including feature points of eyebrows, eyes, nose, mouth, etc., providing crucial reference information for face alignment. Based on this key point information, a specific alignment algorithm (such as affine transformation or similarity transformation) is used to adjust the face to a standard pose, making the poses of different faces more consistent and effectively eliminating the impact of pose variations on subsequent processing. Based on the aligned face, reasonable cropping boundaries are set, and the size and range of the cropping area are determined according to actual needs, ensuring that the cropped face image contains complete facial features and has no redundant background interference. Then, according to the set cropping boundaries, the face is extracted separately from the original image to form an independent face image. Finally, the processed face images are saved in a separate folder according to certain naming rules and file formats for easy management and use later.

[0026] Step 3: Perform data augmentation on the training set and the test set, and simultaneously construct and initialize a deep forgery detection network based on spatiotemporal inconsistency and frequency domain analysis.

[0027] For the dataset, data augmentation preprocessing is performed using methods such as flip_prob, rotate_prob, rotate_limit, blur_prob, blur_limit, brightness_prob, brightness_limit, contrast_limit, quality_lower, and quality_upper.

[0028] Data augmentation is performed on the training and test sets. Simultaneously, a deepfake detection network based on spatiotemporal inconsistency and frequency domain analysis is constructed and initialized. The detection network consists of a spatiotemporal inconsistency module, a frequency domain analysis module, and a video-level classifier. The spatiotemporal inconsistency module uses a pre-trained X-CLIP video encoder as its backbone network and comprises two cascaded visual transformer modules. Through a cross-frame communication transformer (CCT) and a multi-frame ensemble transformer (MIT), it efficiently captures spatiotemporal features in the video, providing a sufficient feature base for deepfake detection. Meanwhile, the frequency domain analysis module combines deep neural networks with time-frequency analysis, mapping video frames to the frequency domain to achieve end-to-end identification of unknown tampering patterns. The video-level classifier extracts features from each frame, supporting fine-grained content understanding and improved classification accuracy. Through a series of complex calculations, it ultimately outputs a probability value for authenticity to predict the authenticity of the input video.

[0029] Step 4: Input the video frame training set processed in Step 3 into the constructed detection network to train the network. The specific training process is as follows.

[0030] Step 41: Input the data-augmented training set into the spatiotemporal inconsistency module. This module is equipped with a pre-trained Xclip video encoder. After processing by the position encoding, cross-frame communication transformer (CCT), and multi-frame integrated transformer (MIT) modules in the video encoder, the spatiotemporal inconsistency features of the input video data are obtained. S .

[0031] Step 411: Input the video data containing T video frames in the training set into the spatiotemporal inconsistency module, perform batch embedding operation on each video frame, divide each video frame into multiple non-overlapping blocks, and use the block embedding matrix to map the non-overlapping blocks to a higher-dimensional feature space. Then, add a learnable embedding vector before the sequence of embedded blocks to obtain the frame representation of the t-th frame.

[0032] Step 412: Integrate the frame representation extracted in step 411 into the cross-frame communication transformer (CCT) to capture the time dependency between frames and output the frame-level representation of the cross-frame communication transformer.

[0033] Step 414: Combine the T frame-level representations to form a total frame-level representation, and input it into the multi-frame integrated transformer (MIT) for processing to aggregate inter-frame information and generate comprehensive spatiotemporal inconsistency features. S。

[0034] Step 42: Input the augmented training set into the frequency domain analysis module to obtain the frequency domain analysis features of the input data. F The frequency domain analysis module includes: 2D FFT, a pre-trained ResNet50 network, and a gated recurrent unit (GRU).

[0035] Step 421: The frequency domain analysis module will input data in batch B, containing T video frames. Where H and W represent spatial resolution, each RGB channel is transformed to the frequency domain using a two-dimensional Fast Fourier Transform (2D FFT) module, resulting in the spectrogram of the t-th video frame, represented as follows: .

[0036] Step 422: Graph the spectrum The DC component is centered by FFT shifting, and the dynamic range is adjusted and normalized using logarithmic compression to obtain the spectral tensor of the t-th video frame. .

[0037] Step 423: Convert the spectral tensor Feature maps of the t-th video frame are obtained by extracting features using a ResNet50 backbone network pre-trained on ImageNet. ; Step 424: Finally, construct a time series from the feature maps of the T video frames. The model is constructed using a 3-layer gated recurrent unit (GRU), and then global average pooling is used to generate the final frequency domain analysis features. F .

[0038] Step 43: Analyze the spatiotemporal inconsistency features output from Steps 41 and 42. S and frequency domain analysis features F The fusion feature L is obtained by integrating the features through learnable weighted fusion.

[0039] The mathematical expression is:

[0040] Step 44: The fused feature L is input into the video-level classifier for processing. Through the calculations of the video-level classifier, the encoded features are transformed using a fully connected layer. The forgery detection network then performs a comprehensive analysis of the input fused feature L, generating a probability value predicting whether the video is real or fake. This probability value represents the likelihood that the input video belongs to the real or fake category. Ultimately, this probability value is used as the basis for judging the authenticity of the video, achieving efficient identification of video authenticity.

[0041] Step 5: Using the output probability of the deepfake detection network, calculate the loss between the network output value and the real label using the cross-entropy loss function, and optimize the network parameters through backpropagation.

[0042] Step 6: Steps 4 and 5 are completed after 10 rounds of training. After each round of training, the test set is sent to the trained network for testing. The objective evaluation index of the deep fake detection network at the current test is calculated, and the network parameters when the objective evaluation index is the best are saved.

[0043] Step 7: Select the network that performs best during the test, then feed the test set into the network, calculate the objective evaluation indicators such as accuracy, area under the curve, mean precision and F1 score, and save the detection results of the final test samples.

[0044] To further illustrate the effectiveness of the method of this invention, it is compared with other existing methods. For a fair comparison, the officially released code of other methods is used, and their experimental settings are followed, with all methods implemented in the same computational environment and subjected to both quantitative and qualitative analysis. The seven methods specifically include: Method 1 is the FTCN method, which utilizes the temporal consistency of video to detect face forgery by leveraging the Temporal Transformer framework. Method 2 is I3D, a dual-stream dilated 3D convolutional network that significantly improves action recognition performance by expanding 2D convolutional kernels to 3D and utilizing ImageNet pre-trained parameters. Method 3 is STIL, which proposes a Spatiotemporal Inconsistency Learning Block (STIL) that jointly captures forgery traces in video through a Spatial Inconsistency Module (SIM), a Temporal Inconsistency Module (TIM), and an Information Supplement Module (ISM). Method 4 is the Transformer method, which significantly improves video classification performance by introducing a spatiotemporal attention mechanism and applying the Transformer model to video understanding tasks. Method 5, X-CLIP, introduces a cross-frame attention mechanism to capture long-term dependencies between video frames, thereby extending the pre-trained image model to video recognition tasks. Method 6, AltFreezing, alternately freezes spatiotemporal weights to encourage the model to simultaneously capture spatial and temporal artifacts, thus improving detection generalization ability. Method 7, TALL-Swin, reconstructs video frames using thumbnail layout and combines it with the Swing Transformer, achieving efficient spatiotemporal feature modeling and demonstrating high performance in cross-dataset detection tasks.

[0045] Since the method of this invention mainly targets deepfake detection of videos in unknown environments, the forgery techniques used in the training and test sets are from different sources. Tables 1, 2, 3, and 4 present quantitative comparison results of accuracy (ACC), area under the curve (AUC), mean precision (AP), and F1 score for eight different network structures on different video frame lengths in the two datasets.

[0046] Table 1. Comparison results of different methods on CelebDF-v1 (input video length of 8 frames).

[0047] Table 2 Comparison results of different methods in DFDCP (input video length of 8 frames).

[0048] Table 3 Comparison results of different methods on CelebDF-v1 (input video length of 16 frames).

[0049] Table 4 Comparison results of different methods in DFDCP (input video length of 16 frames).

[0050] In Tables 1, 2, 3, and 4, the percentage counting method was used, expanding the measurement indicators by 100 times. A comparison of the results in Tables 1, 2, 3, and 4 shows that the method of this invention has higher detection accuracy compared to all other methods.

[0051] Furthermore, the effectiveness of each proposed module was verified through ablation experiments. The training set for the ablation experiments was FaceForensics++, and the test sets were the CelebDF-v1 and DFDCP datasets.

[0052] Table 5 Comparison results of ablation experiments on CelebDF-v1 (8 frames)

[0053] Table 6 Comparison results of ablation experiments in DFDCP (8 frames)

[0054] Table 7 Comparison results of ablation experiments on CelebDF-v1 (16 frames)

[0055] Table 8 Comparison results of ablation experiments in DFDCP (16 frames)

[0056] Quantitative results are shown in Tables 5, 6, 7, and 8. Experimental results demonstrate that removing any module degrades detection performance when applying the method of this invention to remove modules one by one. This proves the effectiveness of the proposed modules. Extensive experimental evidence shows that the deepfake detection method based on spatiotemporal inconsistency and frequency domain analysis proposed in this invention achieves the best results among many mainstream detection algorithms in terms of cross-domain generalization ability and its ability to handle complex forgery patterns.

[0057] It should be noted that the specific embodiments described above are exemplary, and those skilled in the art can devise various solutions inspired by the disclosure of this invention. These solutions all fall within the scope of this invention and its protection. Those skilled in the art should understand that this specification and its accompanying drawings are illustrative and not intended to limit the scope of the claims. The scope of protection of this invention is defined by the claims and their equivalents.

Claims

1. A video deepfake detection method based on spatiotemporal inconsistency and frequency domain analysis, characterized in that, The forgery detection method constructs a video deep forgery detection network for images in unknown environments, comprising three parts: a spatiotemporal inconsistency module, a frequency domain analysis module, and a video-level classifier. The spatiotemporal inconsistency module captures spatiotemporal features in video frames. The frequency domain analysis module uses a two-dimensional fast Fourier transform module to map video frames to the frequency domain and enhance dynamic range. Then, a ResNet50 network with fixed parameters is used to extract frequency domain features. Time dependencies are modeled through gated recurrent units. Finally, the frequency domain analysis features are integrated, and the video-level classifier performs a comprehensive analysis of the input probability values, including: Step 1: Prepare the dataset required for video deepfake detection, including the training set and the test set; Step 2: Preprocess the acquired dataset. The training and test sets are saved as videos. First, the dataset is processed by frame extraction. Then, the Dlib face detection algorithm is used to detect faces in each video frame. The Dlib shape predictor model is used to align and crop faces based on the detected facial landmarks. The processed face images are saved in a separate folder. Step 3: Perform data augmentation on the preprocessed training and test sets. Simultaneously, construct and initialize a deep forgery detection network based on spatiotemporal inconsistency and frequency domain analysis. The detection network consists of a spatiotemporal inconsistency module, a frequency domain analysis module, and a video-level classifier. The spatiotemporal inconsistency module uses a pre-trained X-CLIP video encoder as the backbone network and consists of two cascaded cross-frame communication transformers and a multi-frame integrated transformer. The frequency domain analysis module combines deep neural networks with time-frequency analysis to map video frames to the frequency domain, enabling end-to-end identification of unknown tampering patterns. The video-level classifier extracts features from each frame, providing support for fine-grained content understanding and improved classification accuracy, and finally outputs a probability value to predict the authenticity of the input video. Step 4: Input the video frame training set processed in Step 3 into the constructed detection network to train the network. The training process specifically includes: Step 41: Input the augmented video training set into the spatiotemporal inconsistency module. This module is equipped with a pre-trained Xclip video encoder. After processing by the position encoding, cross-frame communication transformer module, and multi-frame integrated transformer module in the video encoder, the spatiotemporal inconsistency features of the input video data are obtained. S ; Step 42: Input the augmented video training set into the frequency domain analysis module to obtain the frequency domain analysis features of the input data. F The frequency domain analysis module mainly includes: a two-dimensional fast Fourier transform module, a pre-trained ResNet50 network, and a gated recurrent unit (GRU). Step 43: Analyze the spatiotemporal inconsistency features output from Steps 41 and 42. S and frequency domain analysis features F The fusion features are obtained by integrating the learned weighted fusion. Step 44: Input the fused features into the video-level classifier for processing. Through the calculation of the video-level classifier, the encoded features are transformed using a fully connected layer. The forgery detection network performs a comprehensive analysis of the input fused features to generate a probability value for predicting whether the feature is real or fake. Step 5: Using the probability values ​​output by the deepfake detection network, calculate the loss between the network output value and the real label using the cross-entropy loss function, and optimize the network parameters through backpropagation; Step 6: Steps 4 and 5 are completed after a preset number of training rounds. After each training round, the test set is sent to the trained network for testing. The objective evaluation index of the deep fake detection network at the current test is calculated, and the network parameters when the objective evaluation index is best are saved. Step 7: Select the network that performs best during the test, then feed the test set into the network, calculate the objective evaluation indicators such as accuracy, area under the curve, mean precision and F1 score, and save the detection results of the final test samples.

2. The video deepfake detection method as described in claim 1, characterized in that, Step 42, the frequency domain analysis module's processing flow includes: Step 421: The frequency domain analysis module takes the input video data, which has a batch size of B and contains T video frames, and uses a two-dimensional fast Fourier transform module to convert each RGB channel to the frequency domain, obtaining the spectrum of the t-th video frame. ; Step 422: Graph the spectrum The DC component is centered by FFT shifting, and the dynamic range is adjusted and normalized using logarithmic compression to obtain the spectral tensor of the t-th video frame. ; Step 423: Convert the spectral tensor Feature maps of the t-th video frame are obtained by extracting features using a ResNet50 backbone network pre-trained on ImageNet. ; Step 424: Extract the feature maps from the T video frames. Constructing a time series { The model is constructed using a 3-layer gated recurrent unit, and then global average pooling is used to generate the final frequency domain analysis feature F.

Images