A method for detecting a fake face based on double-flow spatial domain and frequency domain feature fusion

Abstract

The present application relates to the field of computer vision, in particular to a kind of based on double-flow spatial domain and frequency domain feature fusion fake face detection method.The purpose is to improve model detection precision and generalization ability.The method comprises: video data is preprocessed, original image is obtained, the original image is converted into frequency domain;Low-rank parameter adaptation mechanism is introduced in spatial domain branch to adjust the training parameters of pre-training CLIP visual encoder, and middle-high layer features are extracted and fused, and then spatial domain features are obtained;Frequency domain branch includes direction related extraction module, and the key direction channel of frequency domain is adaptively weighted, and the frequency domain features for representing image high-frequency abnormal information are extracted by ResNet;Design cross-domain bidirectional cross attention module, the feature alignment and fusion of the spatial domain features and the frequency domain features are carried out, and the joint feature representation is obtained, the true and false prediction of the fused features is carried out by multilayer perceptron classification head, and the detection result is output.

Description

Technical Field

[0001] This invention relates to the fields of computer vision and artificial intelligence, specifically to a method for detecting fake faces based on the fusion of dual-stream spatial domain and frequency domain features. Background Technology

[0002] With the rapid development of deep learning technology, generative model-based face synthesis and editing techniques have been widely applied in image and video generation. These technologies, through learning facial structure, texture, and semantic features, have enabled various applications such as identity replacement, facial expression replay, attribute editing, and full-face generation. The generated results are continuously improving in terms of visual realism and semantic consistency, making them difficult to detect manually. The widespread dissemination of deepfake faces poses potential risks to privacy protection, identity authentication, and social security. Therefore, developing high-precision and robust face detection technologies is of significant practical importance.

[0003] Existing methods for detecting fake faces primarily rely on deep learning models to distinguish between image or video content. Detection typically involves learning the differences between real and fake samples in terms of texture, color distribution, edge structure, and local artifacts. Some methods extract local texture features based on convolutional neural networks, which can identify artifacts introduced during the generation process to some extent, but their ability to model long-range dependencies and global semantic consistency is limited. Other methods introduce feature extraction structures based on self-attention mechanisms to enhance the ability to model global structural relationships, but they still fall short in capturing subtle local forgery traces.

[0004] To overcome the limitations of a single feature domain, some studies have attempted to combine spatial and frequency domain information for forgery detection. Frequency domain analysis is used to extract high-frequency anomaly features or generate fingerprint information to assist spatial domain discrimination. However, existing frequency domain methods often rely on fixed frequency transforms or predefined filters, limiting their ability to express frequency domain features and making it difficult to adapt to variations in forgery traces under different generation models and complex scenarios. Furthermore, some dual-stream detection methods simply concatenate spatial and frequency domain features at the high-level feature stage, lacking effective feature alignment and interaction mechanisms, thus failing to fully leverage the complementary advantages between the two types of features.

[0005] Meanwhile, with the continuous emergence of new generative models, fake faces are becoming more diverse in terms of texture details, noise distribution, and global consistency. Existing detection methods are still insufficient in terms of generalization ability across datasets and generation methods.

[0006] Therefore, there is an urgent need for a fake face detection method that can simultaneously take into account the global semantic structure of the face and fine-grained frequency domain anomaly features, and achieve effective fusion of spatial and frequency domain features, so as to improve the robustness and generalization ability of the detection system in complex scenarios and unknown forgery methods. Summary of the Invention

[0007] The purpose of this invention is to provide a deep fake face detection method based on the fusion of dual-stream spatial domain and frequency domain features, aiming to solve the problems existing in the prior art and improve detection accuracy and generalization ability.

[0008] The present invention provides a deep fake face detection method based on the fusion of dual-stream spatial domain and frequency domain features, which specifically includes the following steps:

[0009] Step 1: Preprocess the video data to obtain the original image, and transform the original image from the spatial domain to the frequency domain;

[0010] Step 2: The spatial domain branch introduces a low-rank parameter adaptation mechanism to adjust the training parameters of the pre-trained CLIP visual encoder, extracts and fuses mid-to-high-level spatial features, and then extracts spatial domain features to represent the overall structure and semantic information of the face.

[0011] Step 3: The frequency domain branch includes a direction correlation extraction module, which adaptively weights the key direction channels in the frequency domain and extracts frequency domain features to characterize high-frequency anomalies in the image using ResNet;

[0012] Step 4: Through the cross-domain bidirectional cross-attention module, the spatial domain features and frequency domain features are aligned and fused to obtain a joint feature representation. Finally, the fused features are used to predict whether they are true or false by a multilayer perceptron classification head, and the detection result is output.

[0013] Furthermore, step one includes the following steps:

[0014] The preprocessing includes extracting video frames from the original real and fake face videos, dividing them into training and test sets, locating face regions, standardizing image sizes, and normalizing the data.

[0015] The process involves transforming the original image from the spatial domain to the frequency domain. Through dual-tree complex wavelet transform, the input face image is decomposed into high-frequency sub-band components at multiple scales. Each high-frequency sub-band corresponds to texture variations and edge information in different directions (±15°, ±45°, ±65°), thereby obtaining a frequency domain feature representation with direction selectivity while maintaining good spatial localization capabilities.

[0016] ;

[0017] ;

[0018] In the formula, This indicates that the input is a face image. This represents the dual-tree complex wavelet transform operator. This represents the low-frequency subband component of the J-th layer. Indicates the j-th layer and direction. High-frequency subband components, and Let represent the real and imaginary parts of the complex wavelet coefficients, respectively. This represents the set of six directional sub-bands.

[0019] Furthermore, step two includes the following steps:

[0020] The face image obtained in step one is input into the pre-trained CLIP visual coding network. By introducing a low-rank parameter adaptation mechanism, the parameters of the feedforward neural network layer of the visual coding network are adjusted. While keeping the original backbone network parameters unchanged, efficient modeling of fake face features is achieved, thereby extracting spatial domain features.

[0021] Specifically, the visual encoding network includes a multi-layer attention module and a feedforward network, and its original weight matrix is ​​denoted as follows. During parameter tuning, instead of directly updating the weight matrix W, a low-rank parameter increment term ΔW is introduced.

[0022] ;

[0023] ;

[0024] In the formula, matrix A is the dimensionality reduction mapping matrix, used to map the input features from the original high-dimensional space to a low-dimensional latent subspace. Its function is to compress the original features to reduce the parameter size and computational complexity. Matrix B is the dimensionality increase mapping matrix, used to remap the low-dimensional features mapped by matrix A back to the original feature space. Its function is to reconstruct the low-dimensional features so that they are consistent with the dimension of the original network weights. The parameter r represents the rank used in low-rank parameter adaptation, used to control the dimension of the low-rank subspace.

[0025] The weights after low-rank parameter adaptation are expressed as follows:

[0026] ;

[0027] In the formula, This is a scaling factor used to adjust the degree of influence of the low-rank increment on the original weights.

[0028] facial images Inputting the visual encoding network after low-rank parameter adaptation yields the feature representations of the intermediate and high-level layers.

[0029] ;

[0030] In the formula, This represents different middle and high-level levels in the network.

[0031] Furthermore, multiple mid-to-high-level features are fused to obtain the final spatial domain feature representation.

[0032] ;

[0033] In the formula, This indicates a feature fusion operation.

[0034] By adopting the above approach, while maintaining the general representation capability of the pre-trained model, a small number of trainable parameters are introduced, enabling spatial domain features to more sensitively characterize the overall structure of the face, semantic consistency, and potential forgery traces, thus providing a reliable spatial semantic representation for subsequent fusion with frequency domain features.

[0035] Furthermore, step three includes the following steps:

[0036] The direction correlation extraction module first performs... The spectrum is decomposed into amplitude and phase spectra, and anomalies in the frequency domain energy distribution and spatial structure consistency of fake faces are further extracted.

[0037] ;

[0038] ;

[0039] In the formula, Indicates the first Each scale and direction is Phase spectrum characteristics, To prevent the introduction of a minimal constant when the denominator is zero, Indicates the first Each scale and direction is The amplitude spectrum characteristics.

[0040] Then, local convolution processing is performed on subbands in different directions to enhance the ability to detect fake features such as local texture discontinuities and edge breaks while maintaining directional selectivity.

[0041] ;

[0042] ;

[0043] ;

[0044] In the formula, This represents the convolution operation. This indicates normalization processing. This represents the ReLU activation function.

[0045] Furthermore, considering both local features within each orientation pattern and the correlation between features across different orientation patterns, an improvement is made to the convolutional block attention module by designing a multi-scale channel attention module. First, 3x3, 5x5, and 7x7 convolutional kernels are used to perform multi-scale convolutional mapping on the input features, respectively.

[0046] ;

[0047] In the formula, Represents the features of the input. This represents the features obtained after convolution at different receptive field scales.

[0048] Then to Simultaneously perform max pooling and average pooling operations, and input the max-pooled features and average-pooled features into a shared MLP layer.

[0049] ;

[0050] ;

[0051] ;

[0052] ;

[0053] In the formula, Indicates global average pooling. This indicates global max pooling.

[0054] The multi-scale weights are then fused to obtain the final channel weights, which are multiplied element-wise and channel-wise with the input image to obtain the output features.

[0055] ;

[0056] ;

[0057] ;

[0058] In the formula, This represents channel-wise element-wise multiplication. Finally, the extracted features are input into the ResNet network for aggregation processing to obtain the frequency domain feature representation, which is then used as the output feature of the frequency domain branch for subsequent alignment and fusion with the spatial domain features.

[0059] Furthermore, step four includes the following steps:

[0060] The spatial domain features obtained in step two are represented as a set of spatial feature sequences. The frequency domain features obtained in step three are represented as a set of frequency domain feature sequences. To integrate the correlation between the two feature domains, the cross-attention mechanism is improved by designing a cross-domain bidirectional cross-attention mechanism to achieve mutual complementarity between spatial and frequency domain features and enhance the generalization ability of the model detection.

[0061] Linear mappings are performed on spatial domain features and frequency domain features respectively to obtain their respective query, key, and value representations, which are used for subsequent cross-domain attention calculations.

[0062] ;

[0063] ;

[0064] In the formula, The query matrix is ​​used to guide the selection and aggregation of feature information in another feature domain, while the value matrix... and The key matrix represents the feature information extracted in the spatial domain and frequency domain, respectively. and It is used to measure the correlation between two different feature domains.

[0065] Then, the matching relationship between the two feature domains is measured by the relevance between the query and the key.

[0066] ;

[0067] ;

[0068] Taking the spatial domain as an example, let's first look at the query representation of spatial domain features. Key representation of frequency domain features Correlation calculations are performed, and the attention weights of the spatial domain to the frequency domain features are obtained through normalization operations. The frequency domain value features are then processed. Weighted aggregation is performed to obtain a supplementary feature representation of the spatial domain from the frequency domain, and the spatial domain features are updated through residual connections. Similarly, frequency domain attention is obtained. And through spatial value features The frequency domain features are supplemented, and then updated through residual connections to obtain the desired result. .

[0069] Finally, the updated frequency domain features are added to the spatial domain features to obtain the output result.

[0070] ;

[0071] The correlation between the two feature domains is used to integrate the information of the two-stream network and serve as the input to the classification module.

[0072] Based on the multilayer perceptron, the fused features in step four are analyzed. To make a true / false prediction, firstly... The dimensionality is reduced by using a global average pooling layer and a normalization layer, and then the features are predicted by a multilayer perceptron to output the deepfake detection result.

[0073] ;

[0074] In the formula, and These represent the connection weight matrices between neurons in the input layer and the hidden layer, and between neurons in the hidden layer and the output layer, respectively. and Indicates the offset of each layer. This represents the GELU activation function.

[0075] The beneficial effects of this invention are:

[0076] This invention constructs a dual-stream feature extraction framework that combines spatial and frequency domains to achieve joint modeling of multi-level features of forged faces. At the same time, it utilizes the overall structure and semantic consistency information of the face in the spatial domain, as well as the high-frequency anomalies and directional artifacts introduced by the generation process in the frequency domain. This effectively makes up for the problem of insufficient information expression in the single feature domain in the forgery detection task, thereby improving the ability of the detection model to represent complex forgery traces.

[0077] This invention utilizes a more directional dual-tree complex wavelet transform to perform frequency domain decomposition on face images at multiple scales and directions, and jointly models amplitude spectrum and phase spectrum information. At the same time, it improves the block attention module and designs a multi-scale channel attention module, which can more effectively capture subtle features such as texture discontinuities, edge anomalies and directional distribution disorder generated during the forgery process, thereby improving the ability to identify forged samples.

[0078] This invention improves the cross-attention mechanism by designing a cross-domain bidirectional cross-attention module, which realizes the mutual complementarity and alignment of spatial and frequency domain features. This effectively avoids the information redundancy and feature mismatch problems caused by simple feature splicing, making the fused feature representation more complete and stable. Attached Figure Description

[0079] Fig. 1 This is a schematic diagram of the method flow of the present invention.

[0080] Fig. 2 Schematic diagram of multi-scale channel attention

[0081] Fig. 3 Schematic diagram of cross-domain bidirectional cross-attention module Detailed Implementation Plan

[0082] Reference Figs. 1-3 As shown, this embodiment provides a deep fake face detection method based on the fusion of dual-stream spatial domain and frequency domain features, including the following steps:

[0083] Step 1: Preprocess the video data to obtain the original image, and transform the original image from the spatial domain to the frequency domain;

[0084] Step 2: The spatial domain branch introduces a low-rank parameter adaptation mechanism to adjust the training parameters of the pre-trained CLIP visual encoder, extracts and fuses mid-to-high-level spatial features, and then extracts spatial domain features to represent the overall structure and semantic information of the face.

[0085] Step 3: The frequency domain branch includes a direction correlation extraction module, which adaptively weights the key direction channels in the frequency domain and extracts frequency domain features to characterize high-frequency anomalies in the image using ResNet;

[0086] Step 4: Through the cross-domain bidirectional cross-attention module, the spatial domain features and frequency domain features are aligned and fused to obtain a joint feature representation. Finally, the fused features are used to predict whether they are true or false by a multilayer perceptron classification head, and the detection result is output.

[0087] Furthermore, step one includes the following steps:

[0088] The preprocessing includes extracting video frames from the original real and fake face videos, dividing them into training and test sets, and standardizing the image size.

[0089] Based on the selected dataset FaceForensics++, which contains 1000 original video sequences and fake video sequences obtained through four facial manipulation algorithms: Deepfakes, Face2Face, FaceSwap, and NeuralTextures, 32 video frames were extracted from the video sequences. The extracted video frames were then divided according to the dataset partitioning file provided by the dataset publisher. The images were standardized to a uniform size of 224x224 pixels and converted into tensor format.

[0090] The input face image is transformed using the dual-tree complex wavelet transform. It is decomposed into high-frequency sub-band components at multiple scales, with each high-frequency sub-band corresponding to texture variations and edge information in different directions (±15°, ±45°, ±65°), thereby obtaining a frequency domain feature representation with direction selectivity while maintaining good spatial localization capabilities.

[0091] ;

[0092] ;

[0093] In the formula, This indicates that the input is a face image. This represents the dual-tree complex wavelet transform operator. This represents the low-frequency subband component of the J-th layer. Indicates the j-th layer and direction. High-frequency subband components, and Let represent the real and imaginary parts of the complex wavelet coefficients, respectively. This represents the set of six directional sub-bands.

[0094] Furthermore, step two includes the following steps:

[0095] The face image obtained in step one is input into a pre-trained visual coding network. By introducing a low-rank parameter adaptation mechanism, the parameters of the visual coding network are fine-tuned. While keeping the original backbone network parameters unchanged, efficient modeling of fake face features is achieved, thereby extracting spatial domain features.

[0096] Specifically, the visual encoding network includes a multi-layer attention module and a feedforward network, and its original weight matrix is ​​denoted as follows:

[0097] ;

[0098] During fine-tuning, instead of directly updating the weight matrix W, a low-rank parameter increment term ΔW is introduced.

[0099] ;

[0100] ;

[0101] In the formula, matrix A is the dimensionality reduction mapping matrix, used to map the input features from the original high-dimensional space to a low-dimensional latent subspace. Its function is to compress the original features to reduce the parameter size and computational complexity. Matrix B is the dimensionality increase mapping matrix, used to remap the low-dimensional features mapped by matrix A back to the original feature space. Its function is to reconstruct the low-dimensional features so that they are consistent with the dimension of the original network weights. The parameter r represents the rank used in low-rank parameter adaptation, used to control the dimension of the low-rank subspace.

[0102] The weights after low-rank parameter adaptation are expressed as follows:

[0103] ;

[0104] In the formula, This is a scaling factor used to adjust the degree of influence of the low-rank increment on the original weights.

[0105] facial images Inputting the visual encoding network after low-rank parameter adaptation yields the feature representations of the intermediate and high-level layers.

[0106] ;

[0107] In the formula, This represents different middle and high-level levels in the network.

[0108] Furthermore, multiple mid-to-high-level features are fused to obtain the final spatial domain feature representation.

[0109] ;

[0110] In the formula, This indicates a feature fusion operation.

[0111] By adopting the above approach, while maintaining the general representation capability of the pre-trained model, a small number of trainable parameters are introduced, enabling spatial domain features to more sensitively characterize the overall structure of the face, semantic consistency, and potential forgery traces, thus providing a reliable spatial semantic representation for subsequent fusion with frequency domain features.

[0112] Furthermore, step three includes the following steps:

[0113] The direction correlation extraction module first performs... The spectrum is decomposed into amplitude and phase spectra, and anomalies in the frequency domain energy distribution and spatial structure consistency of fake faces are further extracted.

[0114] ;

[0115] ;

[0116] In the formula, Indicates the first Each scale and direction is Phase spectrum characteristics, To prevent the introduction of a minimal constant when the denominator is zero, Indicates the first Each scale and direction is The amplitude spectrum characteristics.

[0117] Then, local convolution processing is performed on subbands in different directions to enhance the ability to detect fake features such as local texture discontinuities and edge breaks while maintaining directional selectivity.

[0118] ;

[0119] ;

[0120] ;

[0121] In the formula, This represents the convolution operation. This indicates normalization processing. This represents the ReLU activation function, concatenating the amplitude and phase spectrum features, and compressing the number of channels. .

[0122] Furthermore, considering both the local features within each orientation pattern and the correlation between features across different orientation patterns, an improvement is made to the convolutional block attention module by designing a multi-scale channel attention module. First, features from different directions at the same scale are concatenated to obtain... Multi-scale convolution mapping was performed on the input features using 3x3, 5x5, and 7x7 convolution kernels, respectively.

[0123] ;

[0124] In the formula, This represents the features obtained after convolution at different receptive field scales.

[0125] Then to Simultaneously perform max pooling and average pooling operations, and input the max-pooled features and average-pooled features into a shared MLP layer.

[0126] ;

[0127] ;

[0128] ;

[0129] ;

[0130] In the formula, Indicates global average pooling. This indicates global max pooling.

[0131] The multi-scale weights are then fused to obtain the final channel weights, which are multiplied element-wise and channel-wise with the input image to obtain the output features.

[0132] ;

[0133] ;

[0134] ;

[0135] In the formula, This represents channel-wise element-wise multiplication. Finally, the extracted features are input into the ResNet network for aggregation processing to obtain the frequency domain feature representation, which is then used as the output feature of the frequency domain branch for subsequent alignment and fusion with the spatial domain features.

[0136] Furthermore, step four includes the following steps:

[0137] The spatial domain features obtained in step two are represented as a set of spatial feature sequences. The frequency domain features obtained in step S3 are represented as a set of frequency domain feature sequences. To integrate the correlation between the two feature domains, the cross-attention mechanism is improved by designing a cross-domain bidirectional cross-attention mechanism to achieve mutual complementarity between spatial and frequency domain features and enhance the generalization ability of the model detection.

[0138] Linear mappings are performed on spatial domain features and frequency domain features respectively to obtain their respective query, key, and value representations, which are used for subsequent cross-domain attention calculations.

[0139] ;

[0140] ;

[0141] In the formula, The query matrix is ​​used to guide the selection and aggregation of feature information in another feature domain, while the value matrix... and The key matrix represents the feature information extracted in the spatial domain and frequency domain, respectively. and It is used to measure the correlation between two different feature domains.

[0142] Then, the matching relationship between the two feature domains is measured by the relevance between the query and the key.

[0143] ;

[0144] ;

[0145] Taking the spatial domain as an example, let's first look at the query representation of spatial domain features. Key representation of frequency domain features Correlation calculations are performed, and the attention weights of the spatial domain to the frequency domain features are obtained through normalization operations. The frequency domain value features are then processed. Weighted aggregation is performed to obtain a supplementary feature representation of the spatial domain from the frequency domain, and the spatial domain features are updated through residual connections. Similarly, frequency domain attention is obtained. And through spatial value features The frequency domain features are supplemented, and then updated through residual connections to obtain the desired result. .

[0146] Finally, the updated frequency domain features are added to the spatial domain features to obtain the output result.

[0147] ;

[0148] The correlation between the two feature domains is used to integrate the information of the two-stream network and serve as the input to the classification module.

[0149] Based on the multilayer perceptron, the fused features in step four are analyzed. To make a true / false prediction, firstly... The dimensionality is reduced by using a global average pooling layer and a normalization layer, and then the features are predicted by a multilayer perceptron to output the deepfake detection result.

[0150]

[0151] In the formula, and These represent the connection weight matrices between neurons in the input layer and the hidden layer, and between neurons in the hidden layer and the output layer, respectively. and Indicates the offset of each layer. This represents the GELU activation function.

Claims

1. A deepfake face detection method based on the fusion of dual-stream spatial domain and frequency domain features, characterized in that, Includes the following steps: S1: Preprocess the video data to obtain the original image, and convert the original image into the frequency domain; S2: The spatial domain branch introduces a low-rank parameter adaptation mechanism to adjust the training parameters of the pre-trained CLIP visual encoder, extract and fuse mid-to-high-level spatial features, and then extract spatial domain features to represent the overall structure and semantic information of the face. S3: The frequency domain branch includes a direction correlation extraction module, which adaptively weights the key direction channels in the frequency domain and extracts frequency domain features to characterize high-frequency anomalies in the image through ResNet. S4: Through the cross-domain bidirectional cross-attention module, the spatial domain features and frequency domain features are aligned and fused to obtain a joint feature representation. Finally, the fused features are used to predict whether they are true or false by a multilayer perceptron classification head, and the detection result is output.

2. The deepfake face detection method based on the fusion of dual-stream spatial domain and frequency domain features according to claim 1, characterized in that, The preprocessing includes extracting video frames from the original video, dividing it into training and testing sets, locating face regions, and standardizing image size to obtain the original image. The process involves transforming the original image from the spatial domain to the frequency domain. Through dual-tree complex wavelet transform and multi-scale, multi-directional frequency decomposition, the input face image is decomposed into high-frequency sub-band components at several scales to obtain a frequency domain feature representation with direction selectivity. ; In the formula, This represents the low-frequency subband component of the j-th layer. Indicates the j-th layer and direction. The high-frequency subband components are used to characterize the frequency domain information of the image in different directions.

3. The deepfake face detection method based on the fusion of dual-stream spatial domain and frequency domain features according to claim 1, characterized in that, The original image obtained in step S1 is input into the pre-trained CLIP visual coding network. The feedforward network training parameters of the visual coding network are adjusted by introducing a low-rank parameter adaptation mechanism to enhance the modeling ability of fake face features without significantly increasing the model parameter scale. Specifically, the original weight matrix of the feedforward network layer of the CLIP visual coding network is denoted as... Introduce a low-rank parameter increment term for it. This leads to the adjusted weight matrix. , ; In the formula, This is a scaling factor used to adjust the degree of influence of the low-rank increment on the original weights; facial images Inputting the visual encoding network after low-rank parameter adaptation yields the feature representations of the intermediate and high-level layers. ; In the formula, This represents different middle and high-level layers in the network; Furthermore, multiple mid-to-high-level features are fused to obtain the final spatial domain feature representation. ; In the formula, This indicates a feature fusion operation.

4. The deepfake face detection method based on the fusion of dual-stream spatial domain and frequency domain features according to claim 1, characterized in that, The direction correlation extraction module first performs... The spectrum is decomposed into amplitude and phase spectra, and anomalies in the frequency domain energy distribution and spatial structure consistency of fake faces are further extracted. ; In the formula, Indicates the first Scale and direction are Phase spectrum characteristics, Indicates the first Each scale and direction is Amplitude spectrum characteristics; Then, local convolution processing is performed on sub-bands in different directions to enhance the ability to perceive fake features such as local texture discontinuities and edge breaks. ; ; ; In the formula, This represents the convolution operation. This indicates normalization processing. Represents the ReLU activation function; Furthermore, considering both the local features within each orientation pattern and the correlation between features across different orientation patterns, an improvement is made to the convolutional block attention module by designing a multi-scale channel attention module. First, convolutional operations with different receptive fields are used to perform multi-scale mapping on the input features. ; In the formula, Represents the features of the input. This represents the features obtained after convolution at different receptive field scales; Then to Simultaneously, max pooling and average pooling operations are performed, and the features obtained from max pooling and average pooling are input into a shared MLP to obtain the correlation weights between radiation patterns at different scales. Then, the weights at different scales are fused to obtain the final channel weights, which are then multiplied with the input image channel by channel and element by element to obtain the output features. ; ; In the formula, This represents channel-wise and element-wise multiplication. Finally, the extracted features are input into the residual network for aggregation processing to obtain the frequency domain feature representation, which is then used as the output feature of the frequency domain branch for subsequent alignment and fusion with the spatial domain features.

5. The deepfake face detection method based on the fusion of dual-stream spatial domain and frequency domain features according to claim 1, characterized in that, The spatial domain features obtained in step S2 are represented as a set of spatial feature sequences. The frequency domain features obtained in step S3 are represented as a set of frequency domain feature sequences. To integrate the correlation between the two feature domains, the cross-attention mechanism is improved by designing a cross-domain bidirectional cross-attention mechanism to achieve mutual complementarity between spatial and frequency domain features and enhance the generalization ability of the model detection. Linear mappings are performed on spatial domain features and frequency domain features respectively to obtain their respective query, key, and value representations, which are used for subsequent cross-domain attention calculations. ; ; In the formula, The query matrix is ​​used to guide the selection and aggregation of feature information in another feature domain, while the value matrix... and The key matrix represents the feature information extracted in the spatial domain and frequency domain, respectively. and Used to measure the correlation between two different feature domains; Then, the matching relationship between the two feature domains is measured by the relevance between the query and the key; ; ; Taking the spatial domain as an example, let's first look at the query representation of spatial domain features. Key representation of frequency domain features Correlation calculations are performed, and the attention weights of the spatial domain to the frequency domain features are obtained through normalization operations. The frequency domain value features are then processed. Weighted aggregation is performed to obtain a supplementary feature representation of the spatial domain from the frequency domain, and the spatial domain features are updated through residual connections. Similarly, frequency domain attention is obtained. And through spatial value features The frequency domain features are supplemented, and then updated through residual connections to obtain the desired result. ; Finally, the updated frequency domain features are added to the spatial domain features to obtain the output result. ; The correlation between the two feature domains is used to integrate the information of the two-stream network and serve as the input to the classification module.

6. The method for predicting the true or false nature of fused features based on a multilayer perceptron classification head according to claim 5, characterized in that, Based on the multilayer perceptron, the fused features in step S4 are analyzed. To make a true / false prediction, firstly... The dimensionality is reduced by using a global average pooling layer and a normalization layer, and then the features are predicted by a multilayer perceptron to output the deepfake detection result. ; In the formula, and These represent the connection weight matrices between neurons in the input layer and the hidden layer, and between neurons in the hidden layer and the output layer, respectively. and Indicates the offset of each layer. This represents the GELU activation function.

Images