A method for identity data analysis based on space-frequency perception fusion network

Abstract

The present invention relates to the field of data analysis and processing technology, and specifically to an identity data analysis method based on a spatial-frequency perception fusion network. The method comprises the following steps: collecting identity data to be detected and labeling the real identity information; preprocessing the collected identity data and dividing it into a training set and a test set in proportion; constructing a spatial-frequency perception multi-scale network for real-time, in-depth identity analysis and training the network; inputting the identity data in the test set into the trained network for forgery detection prediction, outputting a probability score for each identity being "true" or "false," and thereby obtaining an identity analysis result. By constructing a lightweight deep neural network structure that combines spatial and frequency feature perception and possesses multi-scale fusion capabilities, the present invention can be used to efficiently detect identity information in practical scenarios such as video conferencing and social media.

Description

Technical Field

[0001] The present invention relates to the technical field of data analysis and processing, and in particular to an identity data analysis method based on a space-frequency perception fusion network. Background Art

[0002] Identity information, as the core link connecting individuals with social activities and information exchange, has a profound impact on security, social governance, daily life convenience, economic activities, and other dimensions. It is the fundamental support for the efficient operation and risk prevention of modern society. Researchers have proposed a variety of models for identity data analysis, primarily relying on structures such as convolutional neural networks (CNNs) and visual transformers to classify and identify identity data.

[0003] While existing detection methods demonstrate high accuracy on public benchmark datasets, they often rely on complex model structures and extensive computing resources, making them difficult to meet real-time processing requirements. This is particularly challenging to implement on mobile devices or in edge computing environments. Furthermore, identity information can exhibit different classification patterns at different levels, such as high-frequency artifacts in the frequency domain or edge distortion in the spatial domain. Traditional methods often neglect the joint modeling of multi-scale and multi-modal forgery features, resulting in poor robustness against high-quality forgeries and limited generalization capabilities.

[0004] Therefore, the present invention proposes an identity data analysis method based on a space-frequency perception fusion network to solve the above problems. Summary of the Invention

[0005] In response to the shortcomings of the existing technology, the present invention develops an identity data analysis method based on a spatial-frequency perception fusion network. By constructing a multi-scale fusion network structure with spatial-frequency perception, the present invention can solve the problem of balancing detection accuracy and computing resource overhead in current identity data analysis.

[0006] The technical solution to the technical problem of the present invention is an identity data analysis method based on a spatial frequency perception fusion network, which is specifically as follows:

[0007] S1. Collect identity data from an existing database. The collected identity data is marked with real identity information, specifically a facial image of a person;

[0008] S2. Preprocess the collected identity data to obtain preprocessed identity data, adjust the identity data to a fixed size and save it in an efficient reading format, and then divide the preprocessed identity data into a training set and a test set in proportion;

[0009] S3. Build a spatial frequency-aware multi-scale network for real-time, in-depth identity data analysis. Train the network, input the identity data from the training set into the network, output binary classification probabilities, use the true labels of the identity data as supervision signals, use a binary cross-entropy loss function to calculate the error between the predicted results and the true labels, calculate the gradients through the back-propagation algorithm, and use the Adam optimizer to update the network parameters to obtain a trained network.

[0010] The network includes a spatial-frequency hybrid perception module, a fuzzy pooling-based downsampling module, a token selective cross-attention module, and a classification detection module;

[0011] S4. Input the identity data in the test set into the trained network for identity analysis, output the probability score of the identity being "true" or "false", and then obtain the identity analysis result.

[0012] The operations in the spatial-frequency hybrid perception module are as follows:

[0013] The spatial-frequency hybrid perception module includes a convolutional neural network, a parallel spatial perception branch and a frequency domain perception branch, and a spatial-frequency hybrid perception layer.

[0014] Use the identity data in the training set as the input identity data of the spatial frequency-aware multi-scale network First, it is input into the spatial frequency domain hybrid perception module, and then the convolutional neural network is used to extract features layer by layer and perform downsampling operations to obtain three identity features with different semantic levels, namely low-level identity features , middle-level identity characteristics and high-level identity characteristics ;

[0015] Then, the low-level identity features Input into the spatial perception branch and frequency domain perception branch respectively to obtain the spatial perception attention features and frequency attention features ;

[0016] Finally, the spatial perception attention features and frequency attention features Input to the spatial frequency domain mixed perception layer, and convert and Perform weighted splicing to generate unified mixed perception features ;

[0017] Mixed perception features Perform a two-dimensional convolution operation, and then generate fused gated features through the Sigmoid function ;

[0018] Fusing gated features and low-class identity characteristics Multiply element by element and add the result to the low-level identity feature Output low-level output features through residual connections ;

[0019] Similarly, the middle-level identity characteristics and high-level identity characteristics Input into the spatial frequency domain hybrid perception module respectively, and output the middle layer output features and high-level output features .

[0020] The spatial perception branch and the frequency domain perception branch are as follows:

[0021] (1) Spatial perception branch:

[0022] First, the low-level identity features Compress in the channel dimension and then calculate the average pooling features separately and max pooling features ;

[0023] Then, the coordinate enhancement module is introduced to generate a normalized coordinate channel. and Perform stitching to generate location-aware enhanced features ;

[0024] Finally, features that enhance location awareness Apply a two-dimensional convolution operation to obtain spatial perception attention features ;

[0025] (2) Frequency domain perception branch:

[0026] Low-class identity characteristics After applying the convolution operation, the Haar wavelet transform is performed to Each channel is calculated independently to generate frequency sub-band features , which contains four sub-bands, namely double low sub-band LL, low high sub-band LH, high low sub-band HL and double high sub-band HH;

[0027] Then, the contextual dependency between frequency sub-bands is mined in the channel-space joint dimension through three-dimensional convolution operation, the discriminative information across frequency bands is integrated, and the compressed single-channel attention weight features are extracted through two-dimensional convolution to generate frequency attention features. , the calculation formula is as follows:

[0028] ,

[0029] in, represents the upsampling operation, represents a three-dimensional convolution operation, The number of channels is 1, represents a non-linear activation function.

[0030] The operation of the downsampling module based on fuzzy pooling is as follows:

[0031] Introducing blur operation and residual information compensation mechanism in the downsampling process;

[0032] (1) For low-level output features :

[0033] First, the low-level output features of the input are processed by fuzzy pooling operation Smoothing, then Standard convolution operation and ReLU activation function are used to obtain the semantic expression features under the backbone path. ;

[0034] Design an information-compensated residual path that shares the fuzzy pooling operation with the backbone path and The convolution operation realizes linear mapping and finally generates the residual signal ;

[0035] Finally, the semantic expression features output by the main path are The residual signal output by the residual path Add element by element and pass through the ReLU activation function again to generate the final downsampled features ;

[0036] (2) For high-level output features :

[0037] Introducing feature projection module on high-level features to output high-level features Perform dimension mapping and structural adjustment to generate high-level projection features ,The feature projection module consists of a convolution layer, a normalization layer and an activation function;

[0038] (3) For the middle-level output features :

[0039] Downsample the features and the middle-level output features Perform fusion and output fusion features .

[0040] The operation of the token selective crisscross attention module is as follows:

[0041] Through three independent convolution operators 、 、 Generate query feature Query, key feature Key and value feature Value;

[0042] in, Extract convolutional layer for query features, which is used to extract convolutional layer from fusion features. Extract query features ; Extract convolutional layers for key features, used to project features from higher layers Extract key features ; Convolution is used to extract features from high layers. Extracting value features ;

[0043] A token selection mechanism based on spatial average pooling is used to select features Channel averaging is performed separately and reconstructed into a token sequence. The token sequence is represented as follows:

[0044] ,

[0045] in, , , Indicates the number of tokens selected, is the size of spatial pooling, Indicates the total number of channels, Indicates the total number of channels The index of Represents a reconstruction operation, denote the query, key, and value token sequences obtained after spatial pooling and reconstruction operations, respectively;

[0046] Use the standard scaled dot product attention mechanism to model the interaction between tokens and calculate the semantically enhanced attention output , the calculation formula is as follows:

[0047] ,

[0048] in, Represents the token after cross-scale fusion, with dimension , represents transpose, represents the normalized exponential function;

[0049] Finally, the cross-scale fused tokens Reshape into The spatial structure of the fusion feature is then restored by upsampling. And projection features Same size, then pass Convolution adjusts the number of channels and finally performs residual fusion with the original fusion features to generate the final fusion features , the calculation formula is as follows:

[0050] ,

[0051] in, Learnable residual weight coefficients, represents the convolution mapping operation, represents the spatial upsampling operation, Represents a refactoring operation.

[0052] The classification detection operations are as follows:

[0053] The final fusion feature The data is fed into a multi-layer perceptron (MLP), which consists of one or more fully connected layers, and inserts batch normalization and ReLU activation functions. The final output layer will output the probability score of each identity to be detected as "true" or "false", thereby obtaining the identity data analysis result.

[0054] The effects provided in the summary of the invention are only the effects of the embodiments, rather than all the effects of the invention. The above technical solution has the following advantages or beneficial effects:

[0055] (1) Enhanced perception of forged identity information: This paper designs a space-frequency hybrid perception module that combines the frequency features extracted by wavelet transform with the spatial information generated by the coordinate attention mechanism. It enhances the perception of forged identity information through dynamic gating fusion and can effectively detect tiny artifacts in high-quality forged videos.

[0056] (2) Efficient cross-scale feature interaction mechanism: A selective cross-attention module is introduced to achieve efficient and low-redundancy information interaction between different scales through dynamic sampling and fusion of feature tokens, thereby improving the alignment and discrimination capabilities of features for forged areas while controlling computational complexity, making it suitable for real-time scenarios.

[0057] (3) Downsampling module with strong ability to preserve structure and edge details: The present invention adopts a residual-enhanced fuzzy pooling structure to effectively alleviate the aliasing problem caused by traditional downsampling during the multi-scale feature extraction process, retain key edges and semantic clues, and help improve the robustness of the detection model.

[0058] (4) Balancing accuracy and efficiency, with strong adaptability: Compared with existing mainstream deep fake detection models, the present invention has achieved excellent performance on multiple public benchmark datasets. In addition, the model structure is lightweight and has low computational overhead, making it easy to deploy on edge devices or mobile terminals, meeting the needs of application scenarios such as video conferencing and live broadcast monitoring that have high requirements for real-time performance and resource constraints.

[0059] (5) Wide applicability and strong generalization ability: The method of the present invention has demonstrated good generalization ability on multiple data sets and is suitable for a variety of complex forgery detection tasks, providing reliable technical support for the security protection of deep forgery content.

[0060] Therefore, the space-frequency-aware multi-scale fusion network structure proposed in the present invention can significantly reduce the computational complexity while maintaining detection accuracy, and is suitable for fast and accurate detection of identity information in actual real-time application scenarios. It uses a frequency-aware gating module to capture fine-grained forgery traces, introduces a selective cross-attention mechanism to improve the efficiency of cross-layer feature interaction, and adopts a residual-enhanced fuzzy pooling structure in the downsampling process to avoid semantic information loss, thereby achieving efficient and robust recognition of identity information while maintaining a lightweight structure. BRIEF DESCRIPTION OF THE DRAWINGS

[0061] The accompanying drawings are used to provide further understanding of the present invention and constitute a part of the specification. They are used to explain the present invention together with the embodiments of the present invention and do not constitute a limitation of the present invention.

[0062] Figure 1 Schematic diagram of the method of the present invention.

[0063] Figure 2 t-SNE visualization diagram of the present invention and different models.

[0064] Figure 3 The figure shows the relationship between the average area under the curve (Avg AUC) of the present invention and different models and the number of floating-point operations (FLOPs) on multiple benchmark data sets. DETAILED DESCRIPTION

[0065] In order to clearly illustrate the technical features of this solution, the present invention is described in detail below through specific implementation methods and in conjunction with the accompanying drawings.

[0066] Example 1

[0067] A method for analyzing identity data based on a spatial frequency perception fusion network comprises the following steps:

[0068] S1. Collect identity data from an existing database. The collected identity data is marked with real identity information, specifically a facial image of a person;

[0069] Existing databases include the FaceForensics++ (FF++) dataset, the DeepFake Detection (DFD) dataset, the Celeb-DF v2 dataset, the DFDCP (DeepFake Detection Challenge Preview) dataset, the DFDC (DeepFake Detection Challenge) dataset, and the UADFV dataset. All of these datasets provide clear true and false labels.

[0070] S2. Preprocess the collected identity data to obtain preprocessed identity data, adjust the identity data to a fixed size and save it in an efficient reading format, and then divide the preprocessed identity data into a training set and a test set in proportion;

[0071] S3. Build a spatial frequency-aware multi-scale network for real-time, in-depth identity data analysis. Train the network, input the identity data from the training set into the network, output binary classification probabilities, use the true labels of the identity data as supervision signals, use a binary cross-entropy loss function to calculate the error between the predicted results and the true labels, calculate the gradients through the back-propagation algorithm, and use the Adam optimizer to update the network parameters to obtain a trained network.

[0072] The network includes a spatial-frequency hybrid perception module, a fuzzy pooling-based downsampling module, a token selective cross-attention module, and a classification detection module;

[0073] S4. Input the identity data in the test set into the trained network for identity analysis, output the probability score of the identity being "true" or "false", and then obtain the identity analysis result.

[0074] In a specific implementation, the operations in the spatial-frequency hybrid perception module are as follows:

[0075] The spatial-frequency hybrid perception module includes a convolutional neural network, a parallel spatial perception branch and a frequency domain perception branch, and a spatial-frequency hybrid perception layer.

[0076] Use the identity data in the training set as the input identity data of the spatial frequency-aware multi-scale network First, it is input into the spatial frequency domain hybrid perception module, and then the convolutional neural network is used to extract features layer by layer and perform downsampling operations to obtain three identity features with different semantic levels, namely low-level identity features , middle-level identity characteristics and high-level identity characteristics ,in Indicates input identity data The number of channels is 3, Indicates input identity data height, Indicates input identity data The width, 、 、 Representing low-level identity characteristics Number of channels, height and width, 、 、 Representing the middle-level identity characteristics The number of channels, height and width, 、 、 Represent high-level identity features Number of channels, height, and width;

[0077] The spatial resolution of identity features decreases layer by layer, while the level of semantic abstraction increases. Low-level identity features primarily retain fine-grained information such as texture and edges, which helps detect subtle forgery traces. Mid-level identity features combine spatial structure and semantic expression capabilities, making them suitable for forgery identification in mesoscale areas. High-level identity features contain rich contextual semantic information, making them suitable for perceiving global forgery behavior.

[0078] Then, the low-level identity features Input into the spatial perception branch and frequency domain perception branch respectively to obtain the spatial perception attention features and frequency attention features ;

[0079] Finally, the spatial perception attention features and frequency attention features Input to the spatial frequency domain mixed perception layer, and convert and Perform weighted splicing to generate unified mixed perception features , the calculation formula is as follows:

[0080] ,

[0081] in, and Represents a learnable scaling parameter that adjusts the ratio of frequency domain and spatial domain attention fusion;

[0082] Mixed perception features Perform a two-dimensional convolution operation, and then generate fused gated features through the Sigmoid function , the calculation formula is as follows:

[0083] ,

[0084] in, represents the Sigmoid function, Represents a two-dimensional convolution operation;

[0085] Fusing gated features and low-class identity characteristics Multiply element by element and add the result to the low-level identity feature Output low-level output features through residual connections , the calculation formula is as follows:

[0086] ,

[0087] in, Represents element-wise multiplication;

[0088] Similarly, the middle-level identity characteristics and high-level identity characteristics Input into the spatial frequency domain hybrid perception module respectively, and output the middle layer output features and high-level output features .

[0089] The above-mentioned space-frequency fusion gating mechanism can effectively realize the joint modeling and dynamic perception of spatial structure and frequency domain artifact information, and improve the model's response ability to forged identity information in different levels of features. Compared with the single-domain modeling method, the hybrid perception design can enhance feature recognition while ensuring computational efficiency, thereby achieving consistent modeling and extraction of multi-scale forged identity information, providing a solid foundation for subsequent fusion and discrimination.

[0090] In a specific implementation, the spatial sensing branch and the frequency domain sensing branch are specifically as follows:

[0091] (1) Spatial perception branch:

[0092] First, the low-level identity features Compress in the channel dimension, and then calculate the average pooling feature and the maximum pooling feature respectively. The calculation formula is as follows:

[0093] ,

[0094] ,

[0095] in, Indicates the Two-dimensional low-level identity features of channels; represents the average pooling feature, represents the maximum pooling feature, and The dimensions are ;

[0096] Then, the coordinate enhancement module is introduced to generate a normalized coordinate channel. and Splicing is performed to generate location-aware enhanced features. The calculation formula is as follows:

[0097] ,

[0098] in, Indicates the characteristics of enhanced location awareness, , The number of channels is 4; Represents the operation of the coordinate enhancement module; Represents the splicing operation in the channel dimension;

[0099] Finally, features that enhance location awareness Apply a two-dimensional convolution operation to obtain spatial perception attention features , the calculation formula is as follows:

[0100] ,

[0101] in, Represents a two-dimensional convolution operator, which can be set to different sizes according to different scales to obtain multi-scale perception capabilities; , The number of channels is 1;

[0102] (2) Frequency domain perception branch:

[0103] Low-class identity characteristics After applying the convolution operation, Haar wavelet transform is performed on Each channel is calculated independently, and the calculation formula is as follows:

[0104] ,

[0105] in, represents a two-dimensional convolution operation, represents the wavelet transform operation; Represents the frequency subband features, which includes four subbands: double low subband LL, low high subband LH, high low subband HL and double high subband HH, which can be used to characterize the structural information and detail changes in different directions and frequency bands;

[0106] Then, the contextual dependency between frequency sub-bands is mined in the channel-space joint dimension through three-dimensional convolution operation, the discriminative information across frequency bands is integrated, and the compressed single-channel attention weight features are extracted through two-dimensional convolution, which can be used to characterize the importance of different spatial positions in the frequency domain and generate frequency attention features. , the calculation formula is as follows:

[0107] ,

[0108] in, represents a two-dimensional convolution operation, represents the upsampling operation, represents a three-dimensional convolution operation, , The number of channels is 1, It can adaptively perceive abnormal areas in the frequency domain of features and emphasize the interference caused by forged identity information on the frequency structure.

[0109] In a specific implementation, the operation of the downsampling module based on fuzzy pooling is as follows:

[0110] Introducing blurring and residual information compensation mechanisms during the downsampling process can effectively alleviate aliasing during feature downsampling, effectively improving the model's feature integrity and stability while maintaining low computational complexity, and enhancing the ability to retain detailed information such as feature structure edges.

[0111] (1) For low-level output features :

[0112] First, the low-level output features of the input are processed by fuzzy pooling operation Smoothing can reduce high-frequency artifacts and aliasing risks, and then Standard convolution operation and ReLU activation function are used to obtain the semantic expression features under the backbone path. , the calculation formula is as follows:

[0113] ,

[0114] in, express Convolution operation, represents the fuzzy pooling operation, represents the batch normalization operation, represents a nonlinear activation function;

[0115] Design an information-compensated residual path that shares the fuzzy pooling operation with the backbone path and The convolution operation implements linear mapping, thereby capturing local texture information, which can avoid serious loss of feature information during the downsampling process and ultimately generate a residual signal. The calculation formula is as follows:

[0116] ,

[0117] in, represents the residual signal, Represents mapping compression and detail supplementation for low-level feature channels Convolution operation;

[0118] Finally, the semantic expression features output by the main path are The residual signal output by the residual path Add element by element and pass through the ReLU activation function again to generate the final downsampled features , the calculation formula is as follows:

[0119] ;

[0120] While maintaining feature compactness, this module effectively enhances the perception of key information such as structure and edges during the downsampling process. Compared with traditional maximum pooling or strided convolution methods, the fuzzy pooling-based structure has obvious advantages in maintaining stable training and anti-aliasing capabilities. It is particularly suitable for task scenarios with high requirements for detail preservation, such as deep fake identity information detection.

[0121] (2) For high-level output features :

[0122] Introducing feature projection module on high-level features to output high-level features Perform dimension mapping and structural adjustment. The feature projection module consists of a convolution layer, a normalization layer, and an activation function. It can map high-level features to the same scale and channel dimensions as the middle and low layers. The calculation formula is as follows:

[0123] ,

[0124] in, Represents a projection operation, Represents high-level projection features;

[0125] (3) For the middle-level output features :

[0126] Downsample the features and the middle-level output features Perform fusion and output fusion features , the calculation formula is as follows:

[0127] ,

[0128] in, Represents a fusion operation.

[0129] In a specific implementation, the token selective crisscross attention module operates as follows:

[0130] To balance high-level semantic perception with low-level detail preservation during multi-scale feature fusion while reducing computational complexity, a token-selective cross-attention module is used. This module compresses spatial information and selects key tokens to perform cross-scale attention modeling, improving the semantic discrimination of low-level forged regions.

[0131] Through three independent convolution operators 、 、 Generate query feature Query, key feature Key and value feature Value;

[0132] in, Extract convolutional layer for query features, which is used to extract convolutional layer from fusion features. Extract query features ; Extract convolutional layers for key features, used to project features from higher layers Extract key features ; Convolution is used to extract features from high layers. Extracting value features , the calculation formula is as follows:

[0133] ,

[0134] in, , Represents the query features, represents the key characteristics, Represents the value characteristics, and the number of channels is ;

[0135] The token selection mechanism based on spatial average pooling can reduce the overhead of attention calculation and improve the feature Channel averaging is performed separately and reconstructed into a token sequence. The token sequence is represented as follows:

[0136] ,

[0137] in, , , Indicates the number of tokens selected, is the size of spatial pooling, Indicates the total number of channels, Indicates the total number of channels The index of Represents a reconstruction operation, denote the query, key, and value token sequences obtained after spatial pooling and reconstruction operations, respectively;

[0138] Use the standard scaled dot product attention mechanism to model the interaction between tokens and calculate the semantically enhanced attention output , the calculation formula is as follows:

[0139] ,

[0140] in, Represents the token after cross-scale fusion, with dimension , represents transpose, represents the normalized exponential function;

[0141] Finally, the cross-scale fused tokens Reshape into The spatial structure of the fusion feature is then restored by upsampling. And projection features Same size, then pass Convolution adjusts the number of channels and finally performs residual fusion with the original fusion features to generate the final fusion features , the calculation formula is as follows:

[0142] ,

[0143] in, Learnable residual weight coefficients, represents the convolution mapping operation, represents the spatial upsampling operation, Represents a refactoring operation.

[0144] The TSCA module described in the present invention reduces computational costs through a token selection mechanism, and uses cross-layer attention modeling to achieve an effective fusion of low-level details and high-level semantics, thereby enhancing the model's ability to identify complex forged identity information. It is suitable for deep fake detection scenarios that require both real-time performance and accuracy.

[0145] In a specific implementation, the classification detection operation is as follows:

[0146] The final fusion feature The data is fed into a multi-layer perceptron (MLP), which consists of one or more fully connected layers, and inserts batch normalization and ReLU activation functions. The final output layer will output the probability score of each identity to be detected as "true" or "false", thereby obtaining the identity data analysis result.

[0147] Example 2

[0148] The effectiveness and generalization ability of the identity data analysis method described in this paper are verified based on public datasets. Identity analysis is performed based on forged facial images. The experiment uses the FaceForensics++ (FF++, facial forensics enhanced dataset) dataset as the main training and test dataset, covering a variety of common forgery techniques. The FF-DF dataset, FF-F2F dataset, FF-FS dataset, and FF-NT dataset are also used.

[0149] FF-DF is a forged subset of the FF++ dataset generated using DeepFakes technology, used to test the model's ability to detect DeepFakes forgeries;

[0150] FF-F2F is a forged subset of the FF++ dataset generated using Face2Face technology. It mainly evaluates the model's detection effect on expression redirection forgeries.

[0151] FF-FS is a forged subset of the FF++ dataset generated using FaceSwap technology, used to test the performance of the model in face-swapping forgery scenarios;

[0152] FF-NT is a forged subset of the FF++ dataset generated using NeuralTextures technology, focusing on the model's ability to identify forged texture reconstruction.

[0153] To further evaluate the generalization performance of the proposed method in cross-domain scenarios, the experiment also introduced multiple datasets from different sources for testing, including:

[0154] The DeepFake Detection Dataset (DFD) released by the Google team contains a large number of real and synthetic face videos;

[0155] Celeb-DF v2 (CDF2, Celebrity Fake Face Dataset Version 2), features more natural and realistic fake videos;

[0156] DeeperForensics-1.0 Compression Pack (DFDCP), which simulates the effects of different compression levels on forgery detection;

[0157] DeepFake Detection Challenge Dataset (DFDC), released by Facebook, is one of the largest and most comprehensive deepfake datasets available.

[0158] and UADFV (University at Albany Deep Fake Video Dataset), which contains fake video samples generated by Deepfakes tools.

[0159] The comparative experiments selected a variety of typical deepfake detection methods and basic image classification models, covering both general feature extraction and specialized detection techniques. Common models include RegNet, GoogLeNet, AlexNet, and ResNet18, representing scalable networks, multi-scale convolutional networks, classic convolutional networks, and residual networks, respectively, and are widely used in image classification tasks.

[0160] CNN-Aug is a forgery detection method based on convolutional neural networks and data augmentation strategies, which aims to improve the adaptability of the detection model to different forgery samples.

[0161] Xception is a network model that uses depthwise separable convolutions. It can improve feature extraction while reducing the number of parameters and is widely used in forgery detection tasks.

[0162] Capsule is a method for image and video forgery detection that leverages the feature modeling capabilities of capsule networks.

[0163] FWA is a method that effectively identifies abnormal features of DeepFake videos by detecting deformation artifacts during the face forgery process;

[0164] X-ray is a method for universal detection of facial forgeries based on feature visualization analysis. FFD is a method that focuses on feature extraction and forgery detection of digital face manipulation.

[0165] F3Net is a method that combines spatial information with frequency features to mine the frequency artifact characteristics of forged images;

[0166] UCF is a method that uses universal feature extraction as its core to improve the performance of deepfake detection across datasets;

[0167] SPSL is a method that enhances the model's ability to detect frequency domain artifacts through shallow feature learning of spatial and phase information.

[0168] The image preprocessing process follows the DeepfakeBench standard, including face cropping and normalization operations. The experiment is based on the PyTorch Lightning framework and runs on an NVIDIA V100 GPU. The input image size is 256×256, the batch size is 64, the optimizer uses the Adam optimizer, and the initial learning rate is set to 0.0002.

[0169] During training, performance is evaluated using the area under the curve (AUC) metric on the validation set. The number of floating-point operations (FLOPs) (measured in gigabytes, meaning billions of floating-point operations per second) and the number of parameters (measured in millions, meaning millions) required for each forward inference pass of the model are recorded. The final test uses the model weights that achieve the best AUC on the test set to evaluate its generalization performance and practical applicability.

[0170] Considering that the present invention is suitable for real-time detection scenarios, this embodiment focuses on comparing deep fake detection models with lower computational complexity. Although some detection methods based on visual language or diffusion models (such as the ASAP model) have high accuracy, they have high inference costs, with a computational load exceeding 100 GFLOPs and billions of parameters, making them unsuitable for delay-sensitive application scenarios. In contrast, detection methods based on the spatial frequency domain, such as the UCF model and the X-ray model, have a computational load of less than 10 GFLOPs and a parameter load within 100 million, achieving a good balance between efficiency and accuracy.

[0171] Therefore, for a fair comparison, this example selected several mainstream methods with similar computational requirements as control models. Under a unified training setup, detection performance was evaluated on the FaceForensics++ dataset. The results, as shown in Table 1, demonstrate that the proposed method achieves superior detection results for most forgery types, with an average AUC of 0.9867. Even on the more complex FF-NT subset, the AUC reached 0.9748, demonstrating good robustness and adaptability.

[0172] Table 1 Comparison of the present invention and other models in the FF++ dataset domain

[0173]

[0174] In addition, if Figure 2 As shown in the figure, t-SNE (t-Distributed Stochastic Neighbor Embedding, a nonlinear algorithm for dimensionality reduction and visualization of high-dimensional data) is used to visualize the features learned by each model. As shown in Figure 2, the blue dots represent "real", the orange dots represent deepfakes, which is the FF-DF subset of the FF++ dataset; the green dots represent face transfer forgeries (Face2Face), which is the FF-F2F subset of the FF++ dataset; the red dots represent face swaps (FaceSwap), which is the FF-FS subset of the FF++ dataset; the purple dots represent neural texture forgeries (NeuralTextures), which is the FF-NT subset of the FF++ dataset; Figure 2It can be seen that the features of the AlexNet model and the Xception model have a large overlap between real and forged samples, and their discriminability is insufficient; F3Net has improved, but there is still confusion about forged types; the method of the present invention can effectively distinguish samples of different categories and has stronger feature expression capabilities.

[0175] The method of the present invention was compared with existing mainstream models on six public datasets, as shown in Table 2. The average AUC reached 0.8682, which is a 4.99% improvement over the UCF model, a 6.35% improvement over the FFD model, and a 7.79% improvement over the lightweight AlexNet model, demonstrating stronger detection accuracy. It outperformed the UCF model on the FF++, DFD, CDF2, DFDCP, and DFDC datasets, only slightly lower on UADFV. In terms of computational efficiency, this method uses only 1.27 G FLOPs and 6.64 million parameters, reducing computational overhead by 87.9% and 78.9% compared to the UCF model and FFD model, respectively. This significantly reduces resource consumption while maintaining high performance, making it suitable for real-time or resource-constrained applications.

[0176] Table 2 Comparison of the generalization ability of the inspection model across different datasets

[0177]

[0178] To further demonstrate the advantages of the present invention between performance and efficiency, the relationship between the average AUC and FLOPs on multiple benchmark datasets is constructed. The results are as follows: Figure 3 It shows that the method of the present invention still achieves an average AUC of 0.8682 with only 1.27 G FLOPs and 6.64 million parameters, which is far better than most models with similar computational costs, demonstrating its superiority in resource-constrained scenarios.

[0179] Although the above describes the specific implementation methods of the invention in conjunction with the accompanying drawings, it does not limit the scope of protection of the invention. Based on the technical solution of the present invention, various modifications or variations that can be made by those skilled in the art without creative work are still within the scope of protection of the present invention.

Claims

1. A method for analyzing identity data based on a spatial frequency perception fusion network, characterized in that: The following steps are involved: S1. Collect identity data from an existing database. The collected identity data is marked with real identity information, specifically a facial image of a person; S2. Preprocess the collected identity data to obtain preprocessed identity data, adjust the identity data to a fixed size and save it in an efficient reading format, and then divide the preprocessed identity data into a training set and a test set in proportion; S3. Build a spatial frequency-aware multi-scale network for real-time, in-depth identity data analysis. Train the network, input the identity data from the training set into the network, output binary classification probabilities, use the true labels of the identity data as supervision signals, use a binary cross-entropy loss function to calculate the error between the predicted results and the true labels, calculate the gradients through the backpropagation algorithm, and use the Adam optimizer to update the network parameters to obtain a trained network. The network includes a spatial-frequency hybrid perception module, a fuzzy pooling-based downsampling module, a token selective cross-attention module, and a classification detection module; The operations in the spatial-frequency hybrid perception module are as follows: The spatial-frequency hybrid perception module includes a convolutional neural network, a parallel spatial perception branch and a frequency domain perception branch, and a spatial-frequency hybrid perception layer. Use the identity data in the training set as the input identity data of the spatial frequency-aware multi-scale network First, it is input into the spatial frequency domain hybrid perception module, and then the convolutional neural network is used to extract features layer by layer and perform downsampling operations to obtain three identity features with different semantic levels, namely low-level identity features , middle-level identity characteristics and high-level identity characteristics ; Then, the low-level identity features Input into the spatial perception branch and frequency domain perception branch respectively to obtain the spatial perception attention features and frequency attention features ; Finally, the spatial perception attention features and frequency attention features Input to the spatial frequency domain mixed perception layer, and convert and Perform weighted splicing to generate unified mixed perception features ; Mixed perception features Perform a two-dimensional convolution operation, and then generate fused gated features through the Sigmoid function ; Fusing gated features and low-class identity characteristics Multiply element by element and add the result to the low-level identity feature Output low-level output features through residual connections ; Similarly, the middle-level identity characteristics and high-level identity characteristics Input into the spatial frequency domain hybrid perception module respectively, and output the middle layer output features and high-level output features ; The operation of the downsampling module based on fuzzy pooling is as follows: Introducing blur operation and residual information compensation mechanism in the downsampling process; (1) For low-level output features : First, the low-level output features of the input are processed by fuzzy pooling operation Smoothing, then Standard convolution operation and ReLU activation function are used to obtain the semantic expression features under the backbone path. ; Design an information-compensated residual path that shares the fuzzy pooling operation with the backbone path and The convolution operation realizes linear mapping and finally generates the residual signal ; Finally, the semantic expression features output by the main path are The residual signal output by the residual path Add element by element and pass through the ReLU activation function again to generate the final downsampled features ; (2) For high-level output features : Introducing feature projection module on high-level features to output high-level features Perform dimension mapping and structural adjustment to generate high-level projection features ,The feature projection module consists of a convolution layer, a normalization layer and an activation function; (3) For the middle-level output features : Downsample the features and the middle-level output features Perform fusion and output fusion features ; The operation of the token selective crisscross attention module is as follows: Through three independent convolution operators 、 、 Generate query feature Query, key feature Key and value feature Value; in, Extract convolutional layer for query features, which is used to extract convolutional layer from fusion features. Extract query features ; Extract convolutional layers for key features, used to project features from higher layers Extract key features ; Convolution is used to extract features from high layers. Extracting value features ; A token selection mechanism based on spatial average pooling is used to select features Channel averaging is performed separately and reconstructed into a token sequence. The token sequence is represented as follows: ， in, , , Indicates the number of tokens selected, is the size of spatial pooling, Indicates the total number of channels, Indicates the total number of channels The index of Represents a reconstruction operation, denote the query, key, and value token sequences obtained after spatial pooling and reconstruction operations, respectively; Use the standard scaled dot product attention mechanism to model the interaction between tokens and calculate the semantically enhanced attention output , the calculation formula is as follows: ， in, Represents the token after cross-scale fusion, with dimension , represents transpose, represents the normalized exponential function; Finally, the cross-scale fused tokens Reshape into The spatial structure of the fusion feature is then restored by upsampling. And projection features Same size, then pass Convolution adjusts the number of channels and finally performs residual fusion with the original fusion features to generate the final fusion features , the calculation formula is as follows: ， in, Learnable residual weight coefficients, represents the convolution mapping operation, represents the spatial upsampling operation, Represents a reconstruction operation; The operations of the classification detection module are as follows: The final fusion feature The data is fed into a multi-layer perceptron (MLP), which consists of one or more fully connected layers, with batch normalization and ReLU activation functions inserted. The final output layer will output the probability score of each identity to be detected as "true" or "false", thereby obtaining the identity data analysis result. S4. Input the identity data in the test set into the trained network for identity analysis, output the probability score of the identity being "true" or "false", and then obtain the identity analysis result.

2. The identity data analysis method based on a spatial frequency perception fusion network according to claim 1 is characterized in that: The spatial perception branch and the frequency domain perception branch are as follows: (1) Spatial perception branch: First, the low-level identity features Compress in the channel dimension and then calculate the average pooling features separately and max pooling features ; Then, the coordinate enhancement module is introduced to generate a normalized coordinate channel. and Perform stitching to generate location-aware enhanced features ; Finally, features that enhance location awareness Apply a two-dimensional convolution operation to obtain spatial perception attention features ; (2) Frequency domain perception branch: Low-class identity characteristics After applying the convolution operation, the Haar wavelet transform is performed to Each channel is calculated independently to generate frequency sub-band features , which contains four sub-bands, namely double low sub-band LL, low high sub-band LH, high low sub-band HL and double high sub-band HH; Then, the contextual dependency between frequency sub-bands is mined in the channel-space joint dimension through three-dimensional convolution operation, the discriminative information across frequency bands is integrated, and the compressed single-channel attention weight features are extracted through two-dimensional convolution to generate frequency attention features. , the calculation formula is as follows: ， in, represents a two-dimensional convolution operation, represents the upsampling operation, represents a three-dimensional convolution operation, The number of channels is 1, represents a non-linear activation function.

Images