An adaptive deep fake face detection method and system based on spatial-frequency domain graph learning

Abstract

The present invention discloses an adaptive deep fake face detection method and system based on spatial-frequency domain graph learning. The method randomly extracts image frames from a video and intercepts a facial image. After adjusting the feature dimensions of the facial image, the facial image is fed into a deep adaptive wavelet module and a normalized residual isomorphism graph neural network module, respectively. The deep adaptive wavelet module extracts the frequency features of the facial image; the normalized residual isomorphism graph neural network module extracts the spatial domain features of the facial image; an adaptive feature fusion module based on gated convolution is used to perform weighted fusion of the frequency domain features and spatial domain features, using gated convolution to achieve attention-like guidance, dynamically adjusting the weights of the feature map's channels and spatial dimensions, and ultimately obtaining fused features; and a classifier is used to perform classification based on the fused features. The present invention enhances the ability to extract forgery detail clues and significantly improves the detection accuracy and stability of the model.

Description

Technical Field

[0001] The present invention relates to the field of face fraud detection technology, and in particular to an adaptive deep fake face detection method and system based on space-frequency domain graph learning. Background Art

[0002] Deepfake technology uses artificial intelligence to generate or replace real images and videos. While it opens up new avenues for artistic creation, it can also be used to spread false media, disrupt social order, and even be used in warfare. Its potential harm should not be underestimated. Deepfake detection technology helps people verify the authenticity of digital content and maintain the credibility of digital information by identifying and analyzing subtle features and patterns in tampered or forged digital content.

[0003] Existing deepfake detection technologies primarily rely on deep learning models, such as convolutional neural networks (CNNs) and graph neural networks (GNNs). These models extract and identify subtle features such as texture, color, and lighting variations in forged content, as well as information such as character movements and facial expressions in videos, by learning from large amounts of real and forged image, video, or audio data. Existing face forgery detection models typically employ a combination of spatial and frequency domain approaches to detect forged faces, using spatial domain features to identify distortion or mixing inconsistencies, and frequency analysis to identify GAN fingerprints. However, existing face forgery detection models suffer from the following problems: First, they use discrete cosine transforms (DCTs) or fixed filter banks to extract frequency features, which cannot provide signals in both the spatial and frequency domains and cannot adapt to data changes; second, the computational complexity of constructing the graph structure is high; and third, they lack discriminability and have poor cross-domain generalization. Summary of the Invention

[0004] The present invention aims to provide an adaptive deep fake face detection method and system based on space-frequency domain graph learning. Through a dynamic adaptive wavelet module, a normalized residual isomorphic graph neural network module, an adaptive feature fusion module and a comprehensive loss optimization function, the problems existing in the prior art are solved and the detection accuracy and generalization ability are improved.

[0005] The present invention adopts the following technology to provide an adaptive deep fake face detection method based on spatial frequency domain graph learning, which includes the following steps:

[0006] Step 1: Randomly extract image frames from the video and capture the face image. After adjusting the feature dimension of the face image, it is sent to the deep adaptive wavelet module and the normalized residual isomorphic graph neural network module respectively;

[0007] Step 2: The deep adaptive wavelet module extracts frequency features of facial images at different scales; the normalized residual isomorphic graph neural network module uses an approximate nearest neighbor algorithm to construct the graph topology and uses multi-layer normalized residual isomorphic graph convolution to update node features and extract spatial features of facial images;

[0008] Step 3: Use the adaptive feature fusion module based on gated convolution to perform weighted fusion of frequency domain features and spatial domain features, use gated convolution to implement attention-like guidance, dynamically adjust the weights of the channel and spatial dimensions of the feature map, and finally obtain the fused features. Use the classifier to perform classification based on the fused features.

[0009] Further preferably, the collected facial images are enhanced to expand the facial image samples, construct a facial image dataset, and divide it into training set and test set, and train the adaptive deep fake face detection model composed of a deep adaptive wavelet module, a normalized residual isomorphic graph neural network module, an adaptive feature fusion module and a classifier.

[0010] Further preferably, the deep adaptive wavelet module includes several layers of 2D adaptive wavelet transform and channel attention modules, the 2D adaptive wavelet transform includes a horizontal lifting step and two independent vertical lifting steps, generating four wavelet transform subband signals: the wavelet transform subband signal LL of horizontal low frequency and vertical low frequency decomposition, the wavelet transform subband signal LH of horizontal low frequency and vertical high frequency decomposition, the wavelet transform subband signal HL of horizontal high frequency and vertical low frequency decomposition, and the wavelet transform subband signal HH of horizontal high frequency and vertical high frequency decomposition; the input of each layer of 2D adaptive wavelet transform is the wavelet transform subband signal LL of horizontal low frequency and vertical low frequency decomposition of the previous layer of 2D adaptive wavelet transform; all the final wavelet transform subband signals obtained by the decomposition of each layer of 2D adaptive wavelet transform are spliced ​​and sent to the channel attention module for weighted fusion.

[0011] Further preferably, the channel attention module is composed of two multi-layer perceptrons and two activation functions, and its formula is expressed as follows:

[0012] ;

[0013] Where, Represent the learnable weights of two multilayer perceptrons, is the RELU activation function, is the Sigmoid activation function, is the attention weight, is the feature sequence output by 2D adaptive wavelet transform, It is the fused frequency domain feature sequence output by the channel attention module.

[0014] Further preferably, the objective loss function of the deep adaptive wavelet module training is:

[0015] ;

[0016] Where M is the number of wavelet decomposition layers, It is The concatenation of the wavelet transform subband signals (HH, HL, LH) of the layer, is the Huber norm, It is The mean of the layer input signal, It is The mean of the wavelet transform subband signal LL of the horizontal low-frequency and vertical low-frequency decomposition output of the layer.

[0017] Further preferably, the approximate nearest neighbor algorithm is improved as follows: when searching for neighbors of a node, the average pooling method is used to downsample the face image samples, and the number of nodes in the topological graph is calculated by Decreased to ,in is the downsampling ratio; after downsampling The nodes are representative nodes of each region. When querying, you only need to The representative point closest to the query node is found among the nodes; the distance measurement method used is cosine similarity instead of Euclidean distance.

[0018] Further preferably, the normalized residual isomorphic graph convolution is expressed as:

[0019] ;

[0020] Where, is a node In the The feature representation of the layer, is a node In the The feature representation of the layer, For the Layers of multi-layer perceptrons, are learnable parameters, is a node The neighbor set of Is a neighbor node In the Feature representation of the layer; is the learnable weight of the neighbor node; ,in, is a learnable weight matrix, is a learnable weight vector, yes The transpose of Is a neighbor node In the The feature representation of the layer, is a leaky linear rectifier function.

[0021] Further preferably, , where BN stands for batch normalization, represents the weight matrix of the first fully connected layer of the kth layer, Represents the weight matrix of the second fully connected layer of the kth layer.

[0022] Further preferably, the adaptive feature fusion module based on gated convolution has two inputs: spatial features and frequency domain characteristics ; First, the features of both inputs are convolved by 3×3 to convert the feature dimensions to , spatial characteristics One of the branches undergoes a 3×3 convolution and is combined with the frequency domain features Add pixel by pixel to get the first fusion feature map , After gated convolution and activation function, adaptive features are obtained ; Airspace characteristics The other branch is transformed by gated convolution and then combined with the adaptive feature Perform pixel-by-pixel multiplication to obtain a mixed feature map with an attention-like guidance mechanism Finally, the mixed feature map Add pixel by pixel with spatial features to obtain fusion features .

[0023] Furthermore, a dynamic asymmetric triplet loss function is introduced to optimize the adaptive deep fake face detection model. The mathematical expression of the dynamic asymmetric triplet loss function is:

[0024] ;

[0025] Where, is the triplet loss, It is Gaussian weights of triplets, and They are The distance between the negative sample pairs and the positive sample pairs in the triples, is the boundary value;

[0026] ;

[0027] Where, is a negative sample pair Compared with the positive sample The relative distance difference, and are the mean and standard deviation of the Gaussian function;

[0028] ;

[0029] Where, is the initial boundary value, is the rate of change is the number of iterations, is the switching threshold, is the linear decay rate.

[0030] Further preferably, the total loss function is expressed as follows:

[0031] ;

[0032] Where, is the binary cross entropy loss, is the dynamic asymmetric triplet loss, is the wavelet optimization objective loss, are the weights of the dynamic asymmetric triplet loss function, is the weight of the adaptive wavelet constraint function.

[0033] The present invention also provides an adaptive deep fake face detection system based on space-frequency domain graph learning, comprising a memory and a processor, wherein computer instructions are stored on the processor, and the steps of the adaptive deep fake face detection method described in the computer instructions are performed.

[0034] The present invention has the following advantages:

[0035] 1. This invention uses a dynamic adaptive wavelet transform to extract frequency features. It possesses multi-scale analysis capabilities and can adaptively adjust the scale and position parameters of the wavelet basis functions to adapt to changes in the input signal, thereby more effectively detecting forgery details. The extracted multi-scale subband signals are weighted and fused using a channel attention module, further enhancing the ability to extract forgery detail clues and significantly improving the model's detection accuracy.

[0036] 2. This invention leverages the powerful complex relationship representation capabilities of graph neural networks (GNNs) to capture subtle clues in forged faces. By using an improved approximate nearest neighbor algorithm (ANN), this invention reduces the computational complexity of graph structure construction and improves image composition efficiency. Furthermore, the introduction of normalized residual isomorphic graph convolutions avoids the vanishing gradient problem, improves model stability, and accelerates training convergence.

[0037] 3. This invention introduces a dynamic asymmetric triplet loss function, which weights samples using a Gaussian function, assigning higher weights to pairs of difficult-to-learn samples, thereby improving the detection performance of difficult-to-classify samples. Furthermore, the method of dynamically adjusting boundary values ​​further enhances the model's data adaptability, enabling rapid model training and stable convergence. Unilateral domain generalization technology facilitates the aggregation of real samples from different domains, effectively eliminating the distribution gap between domains and significantly improving the model's domain generalization capability and detection accuracy.

[0038] 4. The present invention uses an adaptive fusion module based on gated convolution to deeply fuse the frequency domain features extracted by dynamic adaptive wavelet with the spatial domain features extracted by graph neural network. Gated convolution learns a dynamic feature selection mechanism for each channel and spatial position, achieving attention-like guidance and adaptively selecting task-related features. This fusion method not only enhances the data adaptability of the model, but also improves the generalization and robustness of deep fake face detection. Through the complementarity of frequency domain and spatial domain features, the present invention further enhances the data representation capability of the model. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] Figure 1 This is a framework diagram of the adaptive deep fake face detection model of the present invention.

[0040] Figure 2 This is the structural diagram of the adaptive feature fusion module based on gated convolution.

[0041] Figure 3 Schematic diagram for dynamic asymmetric triple optimization. DETAILED DESCRIPTION

[0042] The present invention will be further described in detail below with reference to the accompanying drawings.

[0043] Reference Figure 1 , an adaptive deep fake face detection method based on spatial-frequency domain graph learning, the steps are as follows:

[0044] Step 1: Randomly extract image frames from the video and capture the face image. After adjusting the feature dimension of the face image, it is sent to the deep adaptive wavelet module and the normalized residual isomorphic graph neural network module respectively;

[0045] Step 2: The deep adaptive wavelet module extracts frequency features of facial images at different scales; the normalized residual isomorphic graph neural network module uses an approximate nearest neighbor algorithm to construct the graph topology and uses multi-layer normalized residual isomorphic graph convolution to update node features and extract spatial features of facial images;

[0046] Step 3: Use the adaptive feature fusion module based on gated convolution to perform weighted fusion of frequency domain features and spatial domain features, use gated convolution to implement attention-like guidance, dynamically adjust the weights of the channel and spatial dimensions of the feature map, and finally obtain the fused features. Use the classifier to perform classification based on the fused features.

[0047] The computational cost of a single-frame image is relatively low, and the detection is fast, so the present invention selects a single-frame image to carry out forged face detection. Figure 1 As shown in the figure, the sample video includes data from different domains (data domain 1 and data domain 2). The present invention first randomly extracts image frames from the video and uses the MTCNN algorithm to capture facial images. Then, each facial image is independently enhanced by random rotation, brightness adjustment, Gaussian noise and other methods. The enhanced facial image samples are passed through two sets of convolutional networks (Conv-BN-ReLU) to transform the feature dimension from Promote to ,H is the height, W is the width, and C is the number of channels.

[0048] like Figure 1 As shown, the present invention uses a deep adaptive wavelet module to extract frequency-domain features of fraudulent faces. Existing face forgery detection models typically use discrete cosine transforms (DCTs) to extract frequency-domain features of forged faces. However, the wavelet transform (WT) can provide local signal information in both the spatial and frequency domains, while the DCT only provides frequency-domain information and cannot provide local spatial features. Furthermore, the wavelet transform can provide frequency information at different scales, better capturing local signal details, while the DCT lacks this multi-scale analysis capability. Therefore, the present invention uses a second-generation wavelet transform to extract frequency-domain features of faces and replaces the updater and predictor with a nonlinear function represented by a neural network. This adaptively adjusts the scale and position parameters of the wavelet basis function to better adapt to signal characteristics at different frequencies and spatial locations.

[0049] The second generation wavelet transform transforms the signal As input, its output is an approximation of the wavelet transform and detail subband signals . The second-generation wavelet transform includes three stages: signal separation, updater, and predictor. Drawing on the deep adaptive wavelet module (DAWN), the present invention uses a nonlinear function represented by a neural network to replace the updater and predictor of the second-generation wavelet transform to adapt to changes in the input signal, thereby obtaining a 2D adaptive wavelet transform. The 2D adaptive wavelet transform includes a horizontal lifting step and two independent vertical lifting steps. These lifting steps are applied in sequence to decompose the signal to generate four wavelet transform subband signals: the wavelet transform subband signal LL of the horizontal low-frequency and vertical low-frequency decomposition, the wavelet transform subband signal LH of the horizontal low-frequency and vertical high-frequency decomposition, the wavelet transform subband signal HL of the horizontal high-frequency and vertical low-frequency decomposition, and the wavelet transform subband signal HH of the horizontal high-frequency and vertical high-frequency decomposition. Each lifting step has its own predictor and updater. The internal structure of the updater and predictor is the same in the vertical and horizontal directions, and both include convolutional layers and nonlinear activations. The updater and predictor workflow begins with a padding module that replicates and expands the signal. This is followed by a 1x3 convolution and RELU activation, followed by a 1x1 convolution and Tanh activation. After the update and prediction, a 2D adaptive wavelet transform performs spatial pooling, reducing the spatial size of the output by half relative to the input.

[0050] Such a 2D adaptive wavelet decomposition layer can have at most Layers, where W is the width of the input image. The input of each layer corresponds to the wavelet transform subband signal LL of the horizontal low-frequency and vertical low-frequency decomposition of the previous level. Each layer will output four wavelet transform subband signals (LL, LH, HL and HH), so M decomposition layers produce a total of wavelet transformed subband signals.

[0051] Finally, these sub-wavelet transform sub-bands are spliced ​​and sent to the channel attention module for weighted fusion. The splicing formula is as follows:

[0052] ;

[0053] Where, Indicates the The wavelet transform subband signal is decomposed into low-frequency in the horizontal direction and high-frequency in the vertical direction. Indicates the The wavelet transform subband signal is decomposed into high frequency in horizontal direction and low frequency in vertical direction. Indicates the The wavelet transform subband signal of the horizontal high frequency and vertical high frequency decomposition, i=[1,2,…,M], Indicates the The wavelet transform subband signal of the horizontal low-frequency and vertical low-frequency decomposition is obtained, X represents the characteristic sequence of the 2D adaptive wavelet transform output, and M is the number of wavelet decomposition layers.

[0054] In order to extract more useful frequency features, the present invention feeds each wavelet transform subband signal into a channel attention module for weighted fusion. The channel attention module of the present invention is composed of two multilayer perceptrons and two activation functions, and its formula is expressed as follows:

[0055] ;

[0056] Where, Represent the learnable weights of two multilayer perceptrons, is the RELU activation function, Is the Sigmoid activation function. Through the training of two layers of multilayer perceptron, the importance of each wavelet transform subband signal can be learned, and then the normalized weight between 0 and 1 can be obtained through the Sigmoid function. , and finally multiply the weighted feature sequence output by the 2D adaptive wavelet transform by the channel width On the other hand, the weighted fusion of features in the channel dimension is completed to obtain the fused frequency domain feature sequence .

[0057] When training the deep adaptive wavelet module, on the one hand, it is required that the input signal of each 2D adaptive wavelet transform and its output low-frequency approximation (horizontal low-frequency and vertical low-frequency) have the same average value in order to preserve the mean value of the input signal; on the other hand, it is required that the detail coefficient output by each wavelet module is as small as possible. Therefore, the wavelet optimization objective loss function of the deep adaptive wavelet module is:

[0058] ;

[0059] Where, is the target loss for wavelet optimization, M is the number of wavelet decomposition layers, It is The concatenation of the wavelet transform subband signals HH, HL and LH of the layer (which can be called the detail subband signal), is the Huber norm, and the first term in the objective loss function is to minimize the sum of the detail coefficients on all decomposition layers. It is The mean of the layer input signal, It is The second term in the objective loss function is to minimize the sum of the L2 norms of the differences between the input signal and the output wavelet transform subband signal LL.

[0060] like Figure 1 As shown, another branch of the present invention is to use the normalized residual isomorphic graph neural network module to extract the spatial domain features of the fraudulent face. This module first passes the face image through the convolutional network (3 groups of Conv-BN-ReLU) to obtain the feature vector , the feature dimension is given by becomes ,in is the number of nodes, is the number of channels, the eigenvector Adding learnable positional encoding , , and then fed into the Normalized Residual Homomorphic Graph Neural Module (NRGIN) to extract spatial features. The NRGIN consists of a multi-layer network. Each layer dynamically constructs a topological graph using an improved approximate nearest neighbor algorithm, updates node features using the Normalized Residual Homomorphic Graph convolution, and further enriches node features using a feedforward neural network (FFN).

[0061] Graph neural networks only accept topological graph inputs, so images need to be converted into topological graphs. Topological graphs are defined as ,in is the set of nodes in the topology graph, The topology graph is a collection of edges. The topology graph can be viewed as a collection of nodes and edges. In the process of constructing a topology graph, the K-nearest neighbor algorithm (KNN) is usually used to select the K nearest neighbor nodes. , thereby constructing edges connecting nodes.

[0062] When using the KNN algorithm to construct a topological graph, the computational complexity is high due to the need to calculate the distances between all nodes. Therefore, this paper improves the Approximate Nearest Neighbor (ANN) algorithm to quickly find neighboring nodes. ANN sacrifices a certain degree of accuracy in exchange for faster search speed. There are various ANN implementations, including the inverted file index (IVF), an indexing technique for efficiently retrieving similar vectors. It is a spatial partitioning index that divides the data into multiple regions, each represented by a representative point. When performing a query, the representative point closest to the query point is first found, and then these regions are scanned.

[0063] Inverted file indexing requires clustering to divide the index area, but the computational complexity of clustering algorithms is already relatively high and is not suitable for the needs of this invention. Therefore, this invention does not use clustering algorithms to divide the index area. Instead, it uses the principle of high correlation between adjacent pixels in the image to simply divide the image into blocks, each corresponding to an index area. Specifically, when searching for node neighbors, the average pooling method is used to downsample the face image samples. The number of nodes in the topological graph is determined by Decreased to ,in is the downsampling ratio. The nodes are representative nodes of each region. When querying, you only need to Find the representative point closest to the query node among the nodes, so the computational cost is much lower than that of KNN. , the computational complexity of the improved approximate nearest neighbor (ANN) algorithm will be reduced to In addition, the present invention uses cosine similarity to replace Euclidean distance, further reducing the amount of calculation for finding the nearest neighbor.

[0064] Graph neural networks use graph convolution operations to update the features of the current node using the features of neighboring nodes. Among various graph convolution operations, the original isomorphic graph convolution network can effectively capture the topological structure of the graph and achieve a discrimination capability comparable to the Weisfeiler-Lehman (WL) graph isomorphism test. The original isomorphic graph convolution network updates node features through multi-layer perceptrons and summation aggregation mechanisms. The node feature update formula of the layer is:

[0065] ;

[0066] Where, is a node In the The feature representation of the layer, is a node In the The feature representation of the layer, For the Layers of multi-layer perceptrons, It is a learnable parameter used to adjust the weight of its own features and neighbor aggregation features. is a node The neighbor set of Is a neighbor node In the Layer feature representation. Traditional graph neural network (GCN) aggregation methods primarily focus on information about local neighborhood nodes, but fail to capture the global structure of the graph. When two graphs have similar local structures but different global structures, GCNs may struggle to distinguish between these structures. GCN's discriminative / representational capabilities have been shown to be equivalent to those of the WL test, enabling it to distinguish between graphs with a wider range of different structures.

[0067] However, the original isomorphic graph convolutional network has a limited number of stacked layers. As the network deepens, the gradient gradually disappears or explodes during the backpropagation process, making it difficult to effectively train deep networks. Therefore, the present invention introduces residual connections and normalization into the isomorphic graph convolutional network, and automatically focuses on important neighbor nodes through adaptive attention weights, making it a normalized residual isomorphic graph convolution. Its formula is expressed as:

[0068] ;

[0069] In the formula, add The transformed features are added to the input features. This residual connection method can directly retain the input features and avoid smoothing of key information. It is the learnable weight of the neighbor node, which uses the correlation between the neighbor node and the central node to give the neighbors different weights, so that the model pays attention to the more important neighbor nodes. The weights are normalized using the Softmax function. ,in, is a learnable weight matrix, is a learnable weight vector, yes The transpose of Is a neighbor node In the The feature representation of the layer, is a leaky linear rectifier function.

[0070] , where BN stands for batch normalization, represents the weight matrix of the first fully connected layer of the kth layer, Represents the weight matrix of the second fully connected layer at the kth level. Its main function is to activate and normalize the transformed features, further improving model stability. The original GIN does not normalize node features. The input feature distribution changes during training, resulting in internal covariate shift. Since the input distribution of each layer constantly changes, network parameters must be frequently adjusted to adapt to the new distribution, which complicates the training process and increases the difficulty. Normalized Residual Isomorphic Graph Convolution uses neighbor weight normalization and batch normalization to stabilize the input distribution of each layer, accelerate training convergence, and reduce the training process's sensitivity to hyperparameters.

[0071] In order to improve the feature selection ability of the model, the present invention introduces an adaptive feature fusion module based on gated convolution to perform weighted fusion of frequency features and spatial features. The module structure is as follows: Figure 2 As shown in Figure 2. Gated convolution enhances the model's expressiveness and flexibility for input data by introducing a gating mechanism. Its core idea is to learn a dynamic feature selection mechanism for each channel and each spatial location, thereby better capturing important features in the input data and suppressing unimportant features. Gated convolution consists of two convolutional layers: one for generating feature maps and the other for generating gating signals. Its core formula is:

[0072] ;

[0073] in, is the input of the gated convolution, Y is the output of the gated convolution, and They are convolution kernel and gated convolution kernel, is the Sigmoid activation function, which is used to ensure that the gate value is between 0 and 1.

[0074] The present invention uses gated convolution to achieve attention-like guidance, that is, dynamically adjust the channel and spatial dimensions of the feature map through gated convolution to enhance the model's attention to important features. The adaptive feature fusion module based on gated convolution has two inputs: spatial features and frequency domain characteristics ; First, the features of both inputs are convolved by 3×3 to convert the feature dimensions to , spatial characteristics One of the branches undergoes a 3×3 convolution and is combined with the frequency domain features Add pixel by pixel to get the first fusion feature map , After gated convolution and activation function, adaptive features are obtained ; Airspace characteristics The other branch is transformed by gated convolution and then combined with the adaptive feature Perform pixel-by-pixel multiplication to obtain a mixed feature map with an attention-like guidance mechanism Finally, the mixed feature map Add pixel by pixel with spatial features to obtain fusion features , the fused features will be fed into the classifier for end-to-end training.

[0075] To improve the generalization and class discrimination of the adaptive deepfake face detection model, this paper introduces a dynamic asymmetric triplet loss function to optimize the adaptive deepfake face detection model. The following describes the method from two perspectives: the sample mining strategy and the target loss function.

[0076] Reference Figure 3, the training samples of the present invention come from different data domains (data domain 1, data domain 2 and data domain 3), and have two categories (real samples and forged samples). In order to improve the data domain generalization ability of the model, it is necessary to eliminate the gap between data domains as much as possible. However, different data domains use different forgery methods, and the distribution of their forged features is quite different. The difference in forged features between different data domains is difficult and unnecessary to eliminate; while the feature distributions of real samples in different data domains are similar, and the difference in real features of different data domains is easier and necessary to eliminate. Therefore, the present invention uses unilateral real samples to carry out data domain generalization optimization, and forged samples are only used to promote inter-class discrimination. Since different optimization strategies are used for positive samples and negative samples, this method is called asymmetric triplet optimization. The mining strategy of dynamic asymmetric triples is as follows Figure 3 As shown in , the specific mining strategy is: (1) When the anchor sample is a real sample, the real samples of other data domains are selected as positive samples, and the forged samples of the same data domain are selected as negative samples, so as to eliminate the gap between the real samples of different data domains as much as possible and promote the separation between classes; (2) When the anchor sample is a forged sample ( Figure 3 The bold middle border is the anchor sample), then the forged samples in the same data domain are selected as positive samples, and the real samples in the same data domain are selected as negative samples to promote compactness within the class and separation between classes.

[0077] The goal of the triplet loss function is to maximize the relative difference between the distance between the anchor sample and the negative sample and the distance between the anchor sample and the positive sample. The dynamic asymmetric triplet loss function weights the samples based on the triplet loss function. By assigning higher weights to sample pairs that are difficult to learn, the model pays more attention to these samples during training. The mathematical expression of the dynamic asymmetric triplet loss function is:

[0078] ;

[0079] Where, is the dynamic asymmetric triplet loss, It is Gaussian weights of triplets, and They are The distance between the negative sample pairs and the positive sample pairs in the triples, is the boundary value.

[0080] The present invention uses the Gaussian function method to weight the samples, and the Gaussian weight can be calculated by the following formula:

[0081] ;

[0082] Where, is a negative sample pair Compared with the positive sample The relative distance difference, , and is the mean and standard deviation of the Gaussian function, which can be adjusted dynamically according to the data. When the distance difference is small, it means that the current triple is a difficult-to-classify sample and will be given a larger weight.

[0083] In addition, the objective function The boundary value can control the relative distance interval between the positive and negative sample pairs. The setting of the boundary value is crucial to the performance and training process of the model, and different data sets and tasks may require different boundary values. If the boundary value is too small, it cannot guarantee that the model will learn feature representations with sufficient discriminative power; if the boundary value is too large, it means that the model needs to pull the distance between the positive sample and the anchor point closer, and push the distance between the negative sample and the anchor point farther, which will increase the difficulty of model training and make it difficult for the network to converge. To this end, the present invention introduces a dynamic boundary value. In the early stage of training, a larger boundary value is used to prompt the model to learn quickly in the early stage of training, and in the later stage, the boundary value is reduced to make more fine adjustments to the model, thereby improving the performance and generalization ability of the model. The calculation formula of the boundary value is:

[0084] ;

[0085] Where, is the initial boundary value, is the rate of change, which can be set to 0.05 in the experiment. is the number of iterations. As the iteration progresses, the boundary value decays exponentially with the number of iterations, It is the lower limit to prevent the boundary value from being too small. It can be set to 1e-5 in the experiment. In addition, when the number of training steps reaches a certain requirement, the boundary value switches to linear decay to prevent the feature distance from converging too much. The calculation formula is:

[0086] ;

[0087] Where, For the switching threshold, the experiment can be set to 300. The linear decay rate is set to 0.05 in the experiment. This setting can force the model to learn robust feature representations in the early training stage, while allowing the model to be fine-tuned in a reasonable feature space in the later stage, so that the model pays more attention to subtle differences in features and improves detection accuracy.

[0088] To collaboratively optimize the deep adaptive wavelet module, the normalized residual isomorphic graph neural network module, and the gated convolution-based adaptive feature fusion module, this paper integrates multi-task loss functions and jointly supervises model training. The total loss function consists of three parts: the wavelet optimization target loss function, the dynamic asymmetric triple loss function, and the binary cross entropy loss function, which is expressed as follows:

[0089] ;

[0090] Where, is the binary cross entropy loss, It is a dynamic asymmetric triplet loss, whose optimization goal is to eliminate the gap between domains as much as possible while promoting compactness within the class and separation between classes; It is the wavelet optimization target loss, and its optimization goal is to force the input signal and the output signal to have the same mean, while the detail coefficient is as small as possible. and It is a hyperparameter used to balance the impact of each objective loss function on the model. are the weights of the dynamic asymmetric triplet loss function, is the weight of the adaptive wavelet constraint function.

[0091] The present invention also provides an adaptive deep fake face detection system based on space-frequency domain graph learning, comprising a memory and a processor, wherein computer instructions are stored on the processor, and the steps of the adaptive deep fake face detection method described in the computer instructions are performed.

[0092] The above description merely represents preferred embodiments of the present invention and is not intended to limit the present invention in any other manner. Any person skilled in the art may utilize the above disclosure to modify or modify the present invention into equivalent embodiments. However, any simple modifications, equivalent variations, and modifications to the above embodiments that do not depart from the technical content of the present invention and are based on the technical essence of the present invention remain within the scope of protection of the present invention.

Claims

1. An adaptive deep fake face detection method based on spatial frequency domain graph learning, characterized by the following steps: as follows: Step 1: Randomly extract image frames from the video and capture the face image. After adjusting the feature dimension of the face image, it is sent to the deep adaptive wavelet module and the normalized residual isomorphic graph neural network module respectively; Step 2: The deep adaptive wavelet module extracts the frequency features of the face image at different scales; the normalized residual isomorphic graph neural network module adopts the approximate nearest neighbor algorithm to construct the graph topology structure, and uses multi-layer normalized residual isomorphic graph convolution to update the node features and extract the spatial domain features of the face image; the deep adaptive wavelet module includes several layers of 2D adaptive wavelet transform and channel attention module, the 2D adaptive wavelet transform includes a horizontal lifting step and two independent vertical lifting steps to generate four wavelet transform subband signals: horizontal low frequency and vertical low frequency decomposition subband signals. The 2D adaptive wavelet transform subband signal LL, the wavelet transform subband signal LH of horizontal low frequency and vertical high frequency decomposition, the wavelet transform subband signal HL of horizontal high frequency and vertical low frequency decomposition, and the wavelet transform subband signal HH of horizontal high frequency and vertical high frequency decomposition; the input of each layer of 2D adaptive wavelet transform is the wavelet transform subband signal LL of horizontal low frequency and vertical low frequency decomposition of the previous layer of 2D adaptive wavelet transform; all the final wavelet transform subband signals obtained by the 2D adaptive wavelet transform decomposition of each layer are spliced ​​and sent to the channel attention module for weighted fusion; Step 3: Use the adaptive feature fusion module based on gated convolution to perform weighted fusion of frequency domain features and spatial domain features, use gated convolution to implement attention-like guidance, dynamically adjust the weights of the channel and spatial dimensions of the feature map, and finally obtain the fused features. Use the classifier to perform classification based on the fused features.

2. The adaptive deep fake face detection method according to claim 1, characterized in that: The collected facial images are enhanced to expand the facial image samples, a facial image dataset is constructed, and the training set and test set are divided. The adaptive deep fake face detection model composed of a deep adaptive wavelet module, a normalized residual isomorphic graph neural network module, an adaptive feature fusion module and a classifier is trained.

3. The adaptive deep fake face detection method according to claim 1, characterized in that: The channel attention module consists of two multi-layer perceptrons and two activation functions, and its formula is as follows: ； Where, Represent the learnable weights of two multilayer perceptrons, is the RELU activation function, is the Sigmoid activation function, is the attention weight, is the feature sequence output by 2D adaptive wavelet transform, It is the fused frequency domain feature sequence output by the channel attention module.

4. The adaptive deep fake face detection method according to claim 1, characterized in that: The wavelet optimization objective loss function for deep adaptive wavelet module training is: ； Where, is the target loss of wavelet optimization, M is the number of wavelet decomposition layers, It is The concatenation of the wavelet transform subband signals HH, HL and LH of the layer, is the Huber norm, It is The mean of the layer input signal, It is The mean of the wavelet transform subband signal LL of the horizontal low-frequency and vertical low-frequency decomposition output of the layer.

5. The adaptive deep fake face detection method according to claim 1, characterized in that: The approximate nearest neighbor algorithm is improved as follows: when searching for the nearest neighbor of a node, the average pooling method is used to downsample the face image samples, and the number of nodes in the topological graph is Decreased to ,in is the downsampling ratio; after downsampling The nodes are representative nodes of each region. When querying, you only need to The representative point closest to the query node is found among the nodes; the distance measurement method used is cosine similarity instead of Euclidean distance.

6. The adaptive deep fake face detection method according to claim 1, characterized in that: The normalized residual isomorphic graph convolution is expressed as: ； Where, is a node In the The feature representation of the layer, is a node In the The feature representation of the layer, For the Layers of multilayer perceptrons, are learnable parameters, is a node The neighbor set of Is a neighbor node In the Feature representation of the layer; is the learnable weight of the neighbor node; ,in, is a learnable weight matrix, is a learnable weight vector, yes The transpose of Is a neighbor node In the The feature representation of the layer, is a linear rectification function with leakage; , where BN stands for batch normalization, represents the weight matrix of the first fully connected layer of the kth layer, Represents the weight matrix of the second fully connected layer of the kth layer.

7. The adaptive deep fake face detection method according to claim 1, characterized in that: The adaptive feature fusion module based on gated convolution has two inputs: spatial features and frequency domain characteristics ; First, the features of both inputs are convolved by 3×3 to convert the feature dimensions to , spatial characteristics One of the branches undergoes a 3×3 convolution and is combined with the frequency domain features Add pixel by pixel to get the first fusion feature map , After gated convolution and activation function, adaptive features are obtained ; Airspace characteristics The other branch is transformed by gated convolution and then combined with the adaptive feature Perform pixel-by-pixel multiplication to obtain a mixed feature map with an attention-like guidance mechanism ,Finally, the mixed feature map Add pixel by pixel with spatial features to obtain fusion features .

8. The adaptive deep fake face detection method according to claim 1 is characterized by introducing A dynamic asymmetric triplet loss function is used to optimize the adaptive deep fake face detection model. The mathematical expression of the dynamic asymmetric triplet loss function is: ； Where, is the dynamic asymmetric triplet loss, It is Gaussian weights of triplets, and They are The distance between the negative sample pairs and the positive sample pairs in the triples, is the boundary value; ； Where, is a negative sample pair Compared with the positive sample The relative distance difference, and are the mean and standard deviation of the Gaussian function; ； Where, is the initial boundary value, is the rate of change, is the number of iterations, is the switching threshold, is the linear decay rate; The total loss function is expressed as follows: ； Where, is the binary cross entropy loss, is the dynamic asymmetric triplet loss, is the adaptive wavelet constraint function, is the weight of the dynamic asymmetric triplet loss, is the weight of the adaptive wavelet constraint function.

9. An adaptive deep fake face detection system based on spatial-frequency domain graph learning, characterized by: It includes a memory and a processor, and the processor stores computer instructions, and the computer instructions are used to execute the steps of the adaptive deep fake face detection method according to any one of claims 1 to 8.

Images