Deepfake detection method and system based on priori knowledge guidance and dual-domain representation optimization

Abstract

The present application relates to the technical field of computer vision, in particular to a priori knowledge guided and dual-domain representation optimized deep fake detection method and system, specifically as follows: obtaining a video containing a face identity to be detected; preprocessing the video data to be detected to obtain a face image; inputting the face image into a pre-trained deep fake detection model to extract deep fake evidence features of the image; the deep fake detection model is guided by prior knowledge, guiding the feature extractor to learn the potential knowledge of the real image in the pre-trained model, and establishing the perception ability of the real world; then through the adaptive data augmentation mechanism of the feature hidden space and the image space, the robustness and generalization of the detection model are improved; the features are used as fake evidence to determine whether the face image has been deep faked. The present application has lower test cost, higher efficiency and better generalization, and can widely cope with deep fake detection scenes.

Description

Technical Field

[0001] This invention relates to the field of computer vision technology, and in particular to a deep forgery detection method and system guided by prior knowledge and optimized by dual-domain representation. Background Technology

[0002] In generative artificial intelligence, deepfake technology has become one of the most influential applications, proposed for creating incredibly realistic fake images such as face reconstruction, face swapping, and face editing. In recent years, with the advancement of Generative Adversarial Networks (GANs), high-quality facial image processing no longer requires manual operation (e.g., Adobe Photoshop). With minimal training, models can easily generate highly realistic images, which are then blended with the original images using various mixing strategies to achieve deepfakes. The realistic generated images and sophisticated blending strategies allow forgery clues to be hidden. Furthermore, the emergence of diffusion models (DM) makes generated images even more realistic with fewer artificial artifacts, posing a challenge to the task of identifying deepfake images. The malicious misuse and spread of deepfake technology pose a significant threat to national and societal security.

[0003] To address this issue, numerous deepfake detection methods have been proposed in academia, most of which are based on ordinary binary classifiers. The general process involves: first, using a backbone network to extract features from suspicious images; then, inputting these features into a binary classifier to determine authenticity. However, limited by monotonous and undiversified training sets, these naive binary classifiers often overfit, exhibiting poor generalization and insufficient robustness when faced with unknown forgery methods or external perturbations. To address this, in recent years, a large number of deepfake detection methods have been proposed, detecting forgeries from multiple perspectives such as spatial artifacts, frequency domain, temporal inconsistencies, or multimodal conflicts, achieving some success on specific datasets. These features are then input into a binary classification network to identify real or fake features. While these detection methods perform well when the forgery method is known (e.g., evaluation within a dataset), their generalization is limited when encountering unseen forgeries (e.g., evaluation across datasets). Therefore, improving the generalization performance of detection algorithms is a key focus and challenge in current research.

[0004] Therefore, this invention proposes a deepfake detection method and system guided by prior knowledge and optimized by dual-domain representation to solve the above problems. Summary of the Invention

[0005] To address the shortcomings of existing technologies, this invention develops a deep forgery detection method and system guided by prior knowledge and optimized by dual-domain representation. This invention extracts features from forged images through a prior knowledge guidance module, a biomimetic feature enhancement module, and an adversarial-driven image mixing module, which can improve the accuracy of forged image detection and the generalization performance of the detection algorithm.

[0006] On the one hand, the technical solution of this invention to solve the technical problem is a deepfake detection method guided by prior knowledge and optimized by dual-domain representation, comprising the following steps:

[0007] S1. Collect real human face images, perform deepfake processing on them, and obtain face fake images and self-face swap images;

[0008] S2. Construct a deep fake image detection model. Input real human face images and images obtained through deepfake processing into the deep fake image detection model, extract fake image features, detect the authenticity of the images, calculate the loss function, and optimize the model through training.

[0009] The deep image spoofing detection model includes a prior knowledge-guided module, a genetically inspired bio-inspired feature enhancement module, an adversarial-driven image fusion module, and a classifier.

[0010] S3. Input the image to be detected into the trained deep fake image detection model, input the extracted fake image features into the classifier, and output the final fake image detection result.

[0011] The specific structural components of the deep image fabrication detection model are as follows:

[0012] (1) Prior knowledge guidance module:

[0013] It includes two encoders, each of which includes a teacher network and a student network. The teacher network and the student network have the same structure. The two encoders have the same initial weights and share weights.

[0014] During the training of the deep fake image detection model, the parameters in the teacher network remain fixed, while the parameters in the student network are fine-tuned during the training process.

[0015] (2) Feature enhancement module for biomimetic relic:

[0016] This includes crossover and mutation operations;

[0017] (3) Adversarial-driven image blending module:

[0018] Includes generators and discriminators similar to UNet;

[0019] A UNet-like generator consists of an input layer, a feature extraction layer, a noise mixing layer, a mask generation layer, and an output layer.

[0020] The discriminator consists of a prior knowledge-guided module, a biomimetic feature enhancement module, and a classifier. The classifier uses a normalized index... The function calculates whether an image is real or fake.

[0021] The specific operations in the depth-based image fabrication detection model are as follows:

[0022] (1) Using fake facial images and real human face images The images are input into two encoders, and the teacher network and student network of the encoders output the corresponding image features respectively. The knowledge distillation loss guided by prior knowledge is calculated based on the image loss output by the teacher network and student network, and the knowledge in the teacher network is transferred to the student network.

[0023] (2) Input the image features output by the two encoder student networks into the genetic bionic feature enhancement module, perform crossover and mutation operations, and obtain the enhanced features;

[0024] (3) Input the image features output by the student network and the enhanced features output by the genetic bionic feature enhancement module into the classifier and output the image forgery detection results;

[0025] (4) Use real human face images The input is fed into the adversarial-driven mixing module, where a generator produces a mixing mask. Real human face images Hybrid Mask and self-face swapped images Perform fusion to generate a mixed image The mixed image is processed by a discriminator, which outputs the final forgery detection result.

[0026] The specific operations in the prior knowledge guidance module are as follows:

[0027] Real facial images and fake face images Input into the prior knowledge guidance module. and Through two encoders respectively, Teacher features obtained from real images through an encoder and student characteristics , Teacher characteristics obtained from forged images using an encoder. and student characteristics ;

[0028] Calculate teacher features from real images and student characteristics Knowledge distillation loss between them, calculating teacher features of forged images and student characteristics The knowledge distillation loss between them transfers knowledge from the teacher network to the student network, and the prior knowledge-guided module ultimately outputs the student features of the forged image. Student characteristics and real images ,use Indicates student characteristics, Student characteristics express and .

[0029] The specific operations in the genetically-inspired biomimetic feature enhancement module are as follows:

[0030] Student characteristics The input is fed into the genetically-inspired biomimetic feature enhancement module for enhancement operations, student features Batch size is Specifically includes eigenvectors, denoted as , Indicate student characteristics The Middle 1 eigenvector Indicates transpose;

[0031] Enhanced operations include crossover Operations and mutations operate;

[0032] (1) Cross operate:

[0033] Using roulette wheel Randomly select two different feature vectors and , , , , Indicate student characteristics The Middle 1 eigenvector Indicate student characteristics The Middle 1 eigenvector , , Representing dimension, , Representing the eigenvector Mid-dimensional The element at that location, Representing the eigenvector Mid-dimensional Elements at the location;

[0034] Set hyperparameters to control the length of the intersection segment. , Indicates cross strength. Calculate the length of the exchange segment based on the cross strength. , , Indicates rounding down;

[0035] Randomly generate the starting position index of the crossover. ,satisfy Termination position index After crossing The feature vector of the operation is represented as and , and They represent the number of times after the crossover. The eigenvector and the eigenvector The elements in the cross-formed feature vectors are defined as follows:

[0036] ,

[0037] ,

[0038] in, and These represent the feature vectors after the crossover. and Mid-dimensional The element at that location, Indicates other situations;

[0039] (2) Variation operate:

[0040] from Randomly select a different feature vector , , Indicate student characteristics The Middle 1 eigenvector Representing the eigenvector Mid-dimensional Elements at the location;

[0041] Set hyperparameters to control the length of the variant segment. , Indicates the intensity of variation. Calculate the length of the variant segment based on the variation intensity. , ;

[0042] Randomly generate the starting position index of the mutation. ,satisfy Termination position index Let the random normal distribution tensor be... ,in , , Represents the random normal distribution tensor No. A tensor, after mutation The feature vector of the operation is represented as , Indicates the mutated th The elements of the mutated eigenvectors are defined as follows:

[0043] ,

[0044] in, Represents the feature vector after crossover Mid-dimensional Elements at the location;

[0045] Student characteristics After crossover and mutation operations, the enhanced features are obtained. , , Indicates the enhanced first 1 eigenvector.

[0046] The specific operations in the adversarial-driven hybrid data module are as follows:

[0047] 1) Use real human facial images The input to the adversarial-driven hybrid data module first passes through a UNet-like generator, where the specific operations are as follows:

[0048] (1) Real human face images First, it passes through the input layer;

[0049] (2) Real human face images The input layer feeds into the feature extraction layer, which includes a double convolution (DoubleConv) operation and a downsampling stage.

[0050] The channels are expanded to 64 through the DoubleConv operation, which consists of two ReLU convolutional blocks.

[0051] During the downsampling stage, a pooling layer of size 2 is used to compress the spatial dimension to half of the original size, and a double convolution (DoubleConv) is used to double the number of feature channels. The image undergoes four downsampling operations to obtain the high-level feature information of the image. ;

[0052] (3) The input is fed into a noise mixing layer, which includes noise injection and feature mixing operations.

[0053] The noise injection operation involves sampling a noise tensor of the same size as the feature from a standard normal distribution. ;

[0054] Feature blending operation is to use a The convolutional layers mix high-level feature information with noise tensors to obtain conditional tensors. ;

[0055] (4) The mask generation layer includes an upsampling stage and residual connections;

[0056] During the upsampling stage, a deconvolutional structure is used to restore the spatial dimension to twice the current scale, while double convolution is used to compress the feature channels to twice the original size.

[0057] Each stage of upsampling performs a residual connection between the current feature map and the feature map of the corresponding stage in downsampling, and uses a convolutional layer to fuse the features.

[0058] random tensor After five upsampling passes and a final double convolution to integrate channel information, a hybrid mask is finally generated. ;

[0059] (5) Output layer output mixing mask ;

[0060] 2) Use real human facial images Hybrid Mask and self-face swapped images Blend to obtain a blended image. ;

[0061] Blended Images The calculation formula is as follows:

[0062] ;

[0063] Self-face swapping image The face in the image is still the face of the target identity; a self-face-swapped image is obtained using the Simswap face-swapping model. The purpose of this is to both use forgery methods to construct fake images, containing faint forgery traces of deepfake models for model training, and ensure the consistency of identity between real and fake images, thereby avoiding identity leakage issues.

[0064] 3) Mix the images Input to the discriminator, blended image The two encoders in the prior knowledge-guided module output the features of the mixed image. The mixed image features then pass through the genetically biomimetic feature enhancement module, which outputs the enhanced features. The mixed image features and the enhanced features are then input into the classifier to output the detection results.

[0065] The model is optimized and trained using a loss function, which is calculated as follows:

[0066] 1) Calculate the total loss function of the classifier for the prior knowledge-guided module and the adversarial data hybrid module. The calculation formula is as follows:

[0067] ,

[0068] in, This represents the distillation loss guided by prior knowledge. Represents class centrality loss. This represents the classification loss guided by prior knowledge. This represents the discriminator classification loss of the adversarial-driven data mixing module. , , , The trade-off parameter represents the balance between the contributions of each loss function;

[0069] (1) Distillation loss guided by prior knowledge :

[0070] Student characteristics guided by calculating distillation loss in real-world images With teacher characteristics Student characteristics of close-up, fake images With teacher characteristics The centers are far apart, and the calculation formula is as follows:

[0071]

[0072] ,

[0073] in, This represents the total number of input image samples. Represents cosine similarity. express The center Represents the vector dot product. Representing vectors Norm;

[0074] (2) Class center loss :

[0075] The class center loss method brings features of samples belonging to the same class closer to their corresponding class centers, while maintaining distance between feature centers of samples from different classes. The calculation formula is as follows:

[0076] ,

[0077] in, Indicates the first Student characteristics of a sample express The corresponding counterfeit category label, express Category center, Represents the square of the Euclidean distance;

[0078] (3) Prior knowledge-guided module classification loss :

[0079] ;

[0080] in, The prior knowledge guidance module indicates the first... The predicted probability of the true label of a sample;

[0081] (4) Discriminator classification loss of adversarial-driven data mixing module :

[0082] ,

[0083] in, The first in the adversarial-driven image blending module The mixed image features corresponding to each sample The adversarial-driven image blending module represents the first... The predicted probability of the true value of a sample;

[0084] 2) Calculate the total loss function of the adversarial-driven data mixing module generator. The calculation formula is as follows:

[0085] ,

[0086] in, This represents the image distortion loss of the adversarial-driven data mixing module generator. This indicates the loss of the adversarial discriminator. and A trade-off parameter representing the balance between the contributions of each entity;

[0087] (1) Image distortion loss of adversarial-driven data mixing module generator :

[0088] Structural similarity loss is calculated using real human facial images and composite images. This ensures that the mixed mask generated by the generator differs structurally from real human face images; through various loss mechanisms. and This generates diverse blending masks and reduces image distortion loss. as follows:

[0089] ,

[0090] Structural similarity loss :

[0091]

[0092] ,

[0093] in, Indicates a channel. Indicates three channels; Represents structural similarity operations; Indicates that the channel is The A real human face image, Indicates that the channel is The Zhang composite image, express In the The average value of each channel pixel. express In the The average value of each channel pixel. express and In the Covariance of pixel values ​​in each channel express In the The variance of pixel values ​​in each channel express In the The variance of pixel values ​​in each channel and Represents a constant term;

[0094] By comprehensively considering the structural similarity of each channel, the structural differences between two three-channel images can be measured more comprehensively, guiding the model to generate images that are structurally more different from the original images.

[0095] Diversity loss:

[0096] The first diverse loss function :

[0097] ,

[0098] in, Represents the information entropy function. Indicates the first A mixed mask vector;

[0099] The second diverse loss function :

[0100] ,

[0101] in, Indicates the first A mixed mask vector;

[0102] in, This represents the diversity loss function based on information entropy. This represents the diversity loss based on inter-graph differences. This represents the generated mask;

[0103] (2) Discriminator loss :

[0104]

[0105] in, Represents a random tensor. Represents the random tensor Expectations Represents a generator. This represents the discriminator.

[0106] On the other hand, the present invention also provides a deepfake detection system guided by prior knowledge and optimized by dual-domain representation, for executing a deepfake detection method guided by prior knowledge and optimized by dual-domain representation, comprising the following units:

[0107] Data acquisition unit: acquires the human face video image data to be detected. The data to be detected is obtained from different data sources through the detection interface provided by the system.

[0108] Feature extraction unit: Inputs the facial image to be detected into the pre-trained deep forgery detection model and extracts deep forgery evidence features of the image;

[0109] The feature extraction unit includes a prior knowledge-guided module, a genetically inspired feature enhancement module, and an adversarial-driven image blending module;

[0110] Result recognition unit: Input the deep forgery evidence features of the image into the classifier to obtain the detection result.

[0111] The effects described in the invention are merely those of the embodiments, and not all the effects of the invention. The above technical solutions have the following advantages or beneficial effects:

[0112] The prior knowledge-guided and dual-domain representation optimization deepfake detection method of the present invention uses a prior knowledge-guided module to perform knowledge distillation on the feature extractor using a teacher network pre-trained with massive, real and diverse data, guiding the model to perceive the data distribution of the real world, thereby reducing the generalization defects caused by data traps and improving the robustness and accuracy of feature extraction.

[0113] By using a biomimetic feature enhancement module, which draws on the population inheritance and gene crossover mutation mechanism in genetic algorithms, a rich variety of feature individuals are generated at the latent space feature level. By enhancing the diversity and robustness of features, the true and false decision boundaries of the classifier are broadened, thereby significantly improving the generalization performance of the model.

[0114] By using an adversarial-driven image blending module, the image blending mask is adaptively constructed based on the adversarial game mechanism between the generator and the discriminator in the generative adversarial network, which effectively alleviates the feature bias problem caused by the single training data or the limitation of the blending strategy.

[0115] In summary, this invention improves the accuracy of detecting forged images and the generalization performance of the detection algorithm. Attached Figure Description

[0116] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used together with the embodiments of the invention to explain the invention and do not constitute a limitation thereof.

[0117] Figure 1 This is a schematic diagram of the method flow of the present invention. Detailed Implementation

[0118] To clearly illustrate the technical features of this solution, the invention will be described in detail below through specific implementation methods and in conjunction with the accompanying drawings.

[0119] Example 1

[0120] A deepfake detection method guided by prior knowledge and optimized by dual-domain representation includes the following steps:

[0121] S1. Collect real human face images, perform deepfake processing on them, and obtain face fake images and self-face swap images;

[0122] S2. Construct a deep fake image detection model. Input real human face images and images obtained through deepfake processing into the deep fake image detection model, extract fake image features, detect the authenticity of the images, calculate the loss function, and optimize the model through training.

[0123] The deep image spoofing detection model includes a prior knowledge-guided module, a genetically inspired bio-inspired feature enhancement module, an adversarial-driven image fusion module, and a classifier.

[0124] S3. Input the image to be detected into the trained deep fake image detection model, input the extracted fake image features into the classifier, and output the final fake image detection result.

[0125] In a specific implementation, the structural composition of the depth pseudo-image detection model is as follows:

[0126] (1) Prior knowledge guidance module:

[0127] It includes two encoders, each of which includes a teacher network and a student network. The teacher network and the student network have the same structure. The two encoders have the same initial weights and share weights.

[0128] During the training of the deep fake image detection model, the parameters in the teacher network remain fixed, while the parameters in the student network are fine-tuned during the training process.

[0129] The input to this module is a face image to be identified, which contains real and fake images. The feature extractor is subjected to knowledge distillation by a pre-trained model with frozen parameters, so that the student features of the real image are close to the teacher features extracted by the pre-trained model, while the student features of the fake image are far from the teacher features. The student features are finally obtained and used in subsequent calculations.

[0130] (2) Feature enhancement module for biomimetic relic:

[0131] This includes crossover and mutation operations;

[0132] This module takes the student features output by the prior knowledge guidance module as input and defines them as the initial population. Based on the crossover and mutation operations between individuals (single sample features) in the initial population, more complex and diverse approximate features are constructed. Finally, the classifier is guided to construct a more generalized decision boundary, breaking out of the limitations of the training data. The output of this module is the enhanced student features.

[0133] (3) Adversarial-driven image blending module:

[0134] Includes generators and discriminators similar to UNet;

[0135] A UNet-like generator consists of an input layer, a feature extraction layer, a noise mixing layer, a mask generation layer, and an output layer.

[0136] The discriminator consists of a prior knowledge-guided module, a biomimetic feature enhancement module, and a classifier. The classifier uses a normalized index... The function calculates whether an image is true or false.

[0137] This module consists of a generator and a discriminator. Leveraging the powerful generative capabilities of generative adversarial networks (GANs), it overcomes the limitations of traditional manual masking and constructs richer and more diverse hybrid strategies. The input to this module is the original image and a self-swapped image. Its core objective is to guide the model to accurately focus on key information during the adversarial learning process between the generator and the discriminator, avoiding excessive focus on information with weak relevance to distinguishing between real and fake images. This allows the model to learn more robust, discriminative, and generalizable visual features.

[0138] In a specific implementation, the operations in the depth-based fake image detection model are as follows:

[0139] (1) Using fake facial images and real human face images The images are input into two encoders, and the teacher network and student network of the encoders output the corresponding image features respectively. The knowledge distillation loss guided by prior knowledge is calculated based on the image loss output by the teacher network and student network, and the knowledge in the teacher network is transferred to the student network.

[0140] (2) Input the image features output by the two encoder student networks into the genetic bionic feature enhancement module, perform crossover and mutation operations, and obtain the enhanced features;

[0141] (3) Input the image features output by the student network and the enhanced features output by the genetic bionic feature enhancement module into the classifier and output the image forgery detection results;

[0142] (4) Use real human face images The input is fed into the adversarial-driven mixing module, where a generator produces a mixing mask. Real human face images Hybrid Mask and self-face swapped images Perform fusion to generate a mixed image The mixed image is processed by a discriminator, which outputs the final forgery detection result.

[0143] In a specific implementation, the operations in the prior knowledge guidance module are as follows:

[0144] Real facial images and fake face images Input into the prior knowledge guidance module. and Through two encoders respectively, Teacher features obtained from real images through an encoder and student characteristics , Teacher characteristics obtained from forged images using an encoder. and student characteristics ;

[0145] Calculate teacher features from real images and student characteristics Knowledge distillation loss between them, calculating teacher features of forged images and student characteristics The knowledge distillation loss between them transfers knowledge from the teacher network to the student network, and the prior knowledge-guided module ultimately outputs the student features of the forged image. Student characteristics and real images ,use Indicates student characteristics, Student characteristics express and ;

[0146] in, , , , , , , 3 indicates that the number of image channels is 3. Indicates altitude, Indicates width, This indicates the batch size, 1 indicates that the number of feature channels is 1, and 1792 indicates that the dimension is 1792.

[0147] In a specific implementation, the operations in the genetically-inspired biomimetic feature enhancement module are as follows:

[0148] Student characteristics The input is fed into the genetically-inspired biomimetic feature enhancement module for enhancement operations, student features Batch size is Specifically includes eigenvectors, denoted as , Indicate student characteristics The Middle 1 eigenvector Indicates transpose;

[0149] Enhanced operations include crossover Operations and mutations operate;

[0150] (1) Cross operate:

[0151] Using roulette wheel Randomly select two different feature vectors and , , , , Indicate student characteristics The Middle 1 eigenvector Indicate student characteristics The Middle 1 eigenvector , , Representing dimension, , , Representing the eigenvector Mid-dimensional The element at that location, Representing the eigenvector Mid-dimensional Elements at the location;

[0152] Set hyperparameters to control the length of the intersection segment. , Indicates cross strength. Calculate the length of the exchange segment based on the cross strength. , , Indicates rounding down;

[0153] Randomly generate the starting position index of the crossover. ,satisfy Termination position index After crossing The feature vector of the operation is represented as and , and They represent the number of times after the crossover. The eigenvector and the eigenvector The elements in the cross-formed feature vectors are defined as follows:

[0154] ,

[0155] ,

[0156] in, and These represent the feature vectors after the crossover. and Mid-dimensional The element at that location, Indicates other situations;

[0157] (2) Variation operate:

[0158] from Randomly select a different feature vector , , Indicate student characteristics The Middle 1 eigenvector Representing the eigenvector Mid-dimensional Elements at the location;

[0159] Set hyperparameters to control the length of the variant segment. , Indicates the intensity of variation. Calculate the length of the variant segment based on the variation intensity. , ;

[0160] Randomly generate the starting position index of the mutation. ,satisfy Termination position index Let the random normal distribution tensor be... ,in , , Represents the random normal distribution tensor No. A tensor, after mutation The feature vector of the operation is represented as , Indicates the mutated th The elements of the mutated eigenvectors are defined as follows:

[0161] ,

[0162] in, Represents the feature vector after crossover Mid-dimensional Elements at the location;

[0163] Student characteristics After crossover and mutation operations, the enhanced features are obtained. , , Indicates the enhanced first 1 eigenvector.

[0164] In a specific implementation, the operations in the adversarial-driven hybrid data module are as follows:

[0165] 1) Use real human facial images The input to the adversarial-driven hybrid data module first passes through a UNet-like generator, where the specific operations are as follows:

[0166] (1) Real human face images First, it passes through the input layer;

[0167] (2) Real human face images The input layer feeds into the feature extraction layer, which includes a double convolution (DoubleConv) operation and a downsampling stage.

[0168] The channels are expanded to 64 through the DoubleConv operation, which consists of two ReLU convolutional blocks.

[0169] During the downsampling stage, a pooling layer of size 2 is used to compress the spatial dimension to half of the original size, and a double convolution (DoubleConv) is used to double the number of feature channels. The image undergoes four downsampling operations to obtain the high-level feature information of the image. , ;

[0170] (3) The input is fed into a noise mixing layer, which includes noise injection and feature mixing operations.

[0171] The noise injection operation involves sampling a noise tensor of the same size as the feature from a standard normal distribution. ;

[0172] Feature blending operation is to use a The convolutional layers mix high-level feature information with noise tensors to obtain conditional tensors. , ;

[0173] (4) The mask generation layer includes an upsampling stage and residual connections;

[0174] During the upsampling stage, a deconvolutional structure is used to restore the spatial dimension to twice the current scale, while double convolution is used to compress the feature channels to twice the original size.

[0175] Each stage of upsampling performs a residual connection between the current feature map and the feature map of the corresponding stage in downsampling, and uses a convolutional layer to fuse the features.

[0176] random tensor After five upsampling passes and a final double convolution to integrate channel information, a hybrid mask is finally generated. , ;

[0177] (5) Output layer output mixing mask ;

[0178] 2) Use real human facial images Hybrid Mask and self-face swapped images Blend to obtain a blended image. , ;

[0179] Blended Images The calculation formula is as follows:

[0180] ;

[0181] Self-face swapping image The face in the image is still the face of the target identity; a self-face-swapped image is obtained using the Simswap face-swapping model. The purpose of this is to both use forgery methods to construct fake images, containing faint forgery traces of deepfake models for model training, and ensure the consistency of identity between real and fake images, thereby avoiding identity leakage issues.

[0182] 3) Mix the images Input to the discriminator, blended image The two encoders in the prior knowledge-guided module output the features of the mixed image. The mixed image features then pass through the genetically biomimetic feature enhancement module, which outputs the enhanced features. The mixed image features and the enhanced features are then input into the classifier to output the detection results.

[0183] In a specific implementation, the model is optimized and trained using a loss function, which is calculated as follows:

[0184] 1) Calculate the total loss function of the classifier for the prior knowledge-guided module and the adversarial data hybrid module. The calculation formula is as follows:

[0185] ,

[0186] in, This represents the distillation loss guided by prior knowledge. Represents class centrality loss. This represents the classification loss guided by prior knowledge. This represents the discriminator classification loss of the adversarial-driven data mixing module. , , , The trade-off parameter represents the balance between the contributions of each loss function;

[0187] (1) Distillation loss guided by prior knowledge :

[0188] Student characteristics guided by calculating distillation loss in real-world images With teacher characteristics Student characteristics of close-up, fake images With teacher characteristics The centers are far apart, and the calculation formula is as follows:

[0189]

[0190] ,

[0191] in, This represents the total number of input image samples. Represents cosine similarity. express The center Represents the vector dot product. Representing vectors Norm;

[0192] (2) Class center loss :

[0193] The class center loss method brings features of samples belonging to the same class closer to their corresponding class centers, while maintaining distance between feature centers of samples from different classes. The calculation formula is as follows:

[0194] ,

[0195] in, Indicates the first Student characteristics of a sample express The corresponding counterfeit category label, express Category center, Represents the square of the Euclidean distance;

[0196] (3) Prior knowledge-guided module classification loss :

[0197] ;

[0198] in, The prior knowledge guidance module indicates the first... The predicted probability of the true label of a sample;

[0199] (4) Discriminator classification loss of adversarial-driven data mixing module :

[0200] ,

[0201] in, The first in the adversarial-driven image blending module The mixed image features corresponding to each sample The adversarial-driven image blending module represents the first... The predicted probability of the true value of a sample;

[0202] 2) Calculate the total loss function of the adversarial-driven data mixing module generator. The calculation formula is as follows:

[0203] ,

[0204] in, This represents the image distortion loss of the adversarial-driven data mixing module generator. This indicates the loss of the adversarial discriminator. and A trade-off parameter representing the balance between the contributions of each entity;

[0205] (1) Image distortion loss of adversarial-driven data mixing module generator :

[0206] Structural similarity loss is calculated using real human facial images and composite images. This ensures that the mixed mask generated by the generator differs structurally from real human face images; through various loss mechanisms. and This generates diverse blending masks and reduces image distortion loss. as follows:

[0207] ,

[0208] Structural similarity loss :

[0209]

[0210] ,

[0211] in, Indicates a channel. Indicates three channels; Represents structural similarity operations; Indicates that the channel is The A real human face image, Indicates that the channel is The Zhang composite image, express In the The average value of each channel pixel. express In the The average value of each channel pixel. express and In the Covariance of pixel values ​​in each channel express In the The variance of pixel values ​​in each channel express In the The variance of pixel values ​​in each channel and This represents a constant term introduced to avoid a zero denominator, used to stabilize numerical calculations;

[0212] By comprehensively considering the structural similarity of each channel, the structural differences between two three-channel images can be measured more comprehensively, guiding the model to generate images that are structurally more different from the original images.

[0213] Diversity loss:

[0214] The first diverse loss function :

[0215] ,

[0216] in, The information entropy function is used to measure the uncertainty or randomness of a mixed mask vector. Indicates the first A mixed mask vector;

[0217] The second diverse loss function :

[0218] ,

[0219] in, Indicates the first A mixed mask vector;

[0220] in, This represents the diversity loss function based on information entropy. This represents the diversity loss based on inter-graph differences. This represents the generated mask;

[0221] (2) Discriminator loss :

[0222]

[0223] in, Represents a random tensor. Represents the random tensor Expectations Represents a generator. This represents the discriminator.

[0224] Example 2

[0225] A deepfake detection system guided by prior knowledge and optimized by dual-domain representation is provided for executing a deepfake detection method guided by prior knowledge and optimized by dual-domain representation, comprising the following units:

[0226] Data acquisition unit: acquires the human face video image data to be detected. The data to be detected is obtained from different data sources through the detection interface provided by the system.

[0227] Feature extraction unit: Inputs the facial image to be detected into the pre-trained deep forgery detection model and extracts deep forgery evidence features of the image;

[0228] The feature extraction unit includes a prior knowledge-guided module, a genetically inspired feature enhancement module, and an adversarial-driven image blending module;

[0229] Result recognition unit: Input the deep forgery evidence features of the image into the classifier to obtain the detection result.

[0230] Example 3

[0231] To demonstrate that the method in this invention outperforms existing methods, this invention compares its performance with mainstream models (Meso4, MesoInncep, CNN-Aug, Xception, EfficientB4, Capsule, FWA, X-ray, FFD, CORE, RECCE, UCF, LSDA, F3Net, SPSL, and SRM, all of which are mainstream feature extraction networks) on multiple public datasets (CDFv1, CDFv2, DFD, DFDC, Fsh, and UADFV). The comparison results are shown in Table 1.

[0232] Table 1. Performance comparison of the method of this invention with existing methods on multiple datasets.

[0233]

[0234] As shown in Table 1, the average AUC of the present invention reaches 0.8258, which is 2.63% higher than the LSDA model and 5.69% higher than the baseline method EfficientB4. The method of the present invention exhibits stronger detection accuracy and model generalization, indicating that the method in the present invention is superior to other advanced existing deepfake detection algorithms.

[0235] The specific public datasets are as follows:

[0236] CDFv1 (Celeb-DF v1): is a celebrity deepfake dataset that contains real celebrity videos and fake videos generated using DeepFaceLab technology;

[0237] CDFv2 (Celeb-DF v2): Contains 590 original real videos and 5,639 fake videos, covering a wider range of celebrity samples and fake scenarios;

[0238] DFD (Deepfake Detection Dataset): Contains 363 original video clips (real-world scenes) and over 3,000 fake videos generated by DeepFakes, with significant differences in video resolution and duration;

[0239] DFDC (Deepfake Detection Challenge): A public challenge dataset initiated by Facebook, containing 119,197 videos (the ratio of real to fake videos is approximately 1:5).

[0240] Fsh (FaceShifter Dataset): The Fsh dataset is generated based on FaceShifter technology and contains approximately 1,000 face-swapped videos;

[0241] UADFV (University of Adelaide Deepfakes Video Dataset): Contains 49 real videos and 49 fake videos, covering samples from ordinary users and celebrities.

[0242] Although the specific embodiments of the invention have been described above in conjunction with the accompanying drawings, this is not intended to limit the scope of protection of the invention. Based on the technical solutions of the invention, various modifications or variations that can be made by those skilled in the art without creative effort are still within the scope of protection of the invention.

Claims

1. A deepfake detection method guided by prior knowledge and optimized by dual-domain representation, characterized in that, Includes the following steps: S1. Collect real human face images, perform deepfake processing on them, and obtain face fake images and self-face swap images; S2. Construct a deep fake image detection model. Input real human face images and images obtained through deepfake processing into the deep fake image detection model, extract fake image features, detect the authenticity of the images, calculate the loss function, and optimize the model through training. The deep image fabrication detection model includes a prior knowledge-guided module, a genetically-inspired biomimetic feature enhancement module, an adversarial-driven image fusion module, and a classifier. The specific operations in the prior knowledge guidance module are as follows: Real facial images and fake face images Input into the prior knowledge guidance module. and Through two encoders respectively, Teacher features obtained from real images via encoder and student characteristics , Teacher characteristics obtained from forged images using an encoder. and student characteristics ; Calculate teacher features from real images and student characteristics Knowledge distillation loss between them, calculating teacher features of forged images and student characteristics The knowledge distillation loss between them transfers knowledge from the teacher network to the student network, and the prior knowledge-guided module ultimately outputs the student features of the forged image. Student characteristics and real images ,use Indicates student characteristics, Student characteristics express and ; The specific operations in the genetically-inspired biomimetic feature enhancement module are as follows: Student characteristics The input is fed into the genetically-inspired biomimetic feature enhancement module for enhancement operations, student features Batch size is Specifically includes eigenvectors, denoted as , Indicate student characteristics The Middle 1 eigenvector Indicates transpose; Enhanced operations include crossover Operations and mutations operate; (1) Cross operate: Using roulette wheel Randomly select two different feature vectors and , , , , Indicate student characteristics The Middle 1 eigenvector Indicate student characteristics The Middle 1 eigenvector , , Representing dimension, , Representing the eigenvector Mid-dimensional The element at that location, Representing the eigenvector Mid-dimensional Elements at the location; Set hyperparameters to control the length of the intersection segment. , Indicates cross strength. Calculate the length of the exchange segment based on the cross strength. , , Indicates rounding down; Randomly generate the starting position index of the crossover. ,satisfy Termination position index After crossing The feature vector of the operation is represented as and , and They represent the number of times after the crossover. The eigenvector and the eigenvector The elements in the cross-formed feature vectors are defined as follows: ， ， in, and These represent the feature vectors after the crossover. and Mid-dimensional The element at that location, Indicates other situations; (2) Variation operate: from Randomly select a different feature vector , , Indicate student characteristics The Middle 1 eigenvector Representing the eigenvector Mid-dimensional Elements at the location; Set hyperparameters to control the length of the variant segment. , Indicates the intensity of variation. Calculate the length of the variant segment based on the variant intensity. , ; Randomly generate the starting position index of the mutation. ,satisfy Termination position index Let the random normal distribution tensor be... ,in , , Represents the random normal distribution tensor No. A tensor, after mutation The feature vector of the operation is represented as , Indicates the mutated th The elements of the mutated eigenvectors are defined as follows: ， in, Represents the feature vector after crossover Mid-dimensional Elements at the location; Student characteristics After crossover and mutation operations, the enhanced features are obtained. , , Indicates the enhanced first 1 eigenvector; The specific operations in the adversarial-driven hybrid data module are as follows: 1) Use real human facial images The input to the adversarial-driven hybrid data module first passes through a UNet-like generator, where the specific operations are as follows: (1) Real human face images First, it passes through the input layer; (2) Real human face images The input layer feeds into the feature extraction layer, which includes a double convolution (DoubleConv) operation and a downsampling stage. The channels are expanded to 64 through the DoubleConv operation, which consists of two ReLU convolutional blocks. During the downsampling stage, a pooling layer of size 2 is used to compress the spatial dimension to half of the original size, and a double convolution (DoubleConv) is used to double the number of feature channels. The image undergoes four downsampling operations to obtain the high-level feature information of the image. ; (3) The input is fed into a noise mixing layer, which includes noise injection and feature mixing operations. The noise injection operation involves sampling a noise tensor of the same size as the feature from a standard normal distribution. ; Feature blending operation is to use a The convolutional layers mix high-level feature information with noise tensors to obtain conditional tensors. ; (4) The mask generation layer includes an upsampling stage and residual connections; During the upsampling stage, a deconvolutional structure is used to restore the spatial dimension to twice the current scale, while double convolution is used to compress the feature channels to twice the original size. Each stage of upsampling performs a residual connection between the current feature map and the feature map of the corresponding stage in downsampling, and uses a convolutional layer to fuse the features. random tensor After five upsampling passes and a final double convolution to integrate channel information, a hybrid mask is finally generated. ; (5) Output layer output mixing mask ; 2) Use real human facial images Hybrid Mask and self-face swapped images Blend to obtain a blended image. ; Blended Images The calculation formula is as follows: ； Self-face swapping image The face in the image is still the face of the target identity; a self-face-swapped image is obtained using the Simswap face-swapping model. ; 3) Mix the images Input to the discriminator, blended image The two encoders in the prior knowledge-guided module output the features of the mixed image. The mixed image features then pass through the genetic biomimetic feature enhancement module, which outputs the enhanced features. The mixed image features and the enhanced features are then input into the classifier to output the detection results. S3. Input the image to be detected into the trained deep fake image detection model, input the extracted fake image features into the classifier, and output the final fake image detection result.

2. The deepfake detection method guided by prior knowledge and optimized by dual-domain representation as described in claim 1, characterized in that, The specific structural components of the deep image fabrication detection model are as follows: (1) Prior knowledge guidance module: It includes two encoders, each of which includes a teacher network and a student network. The teacher network and the student network have the same structure. The two encoders have the same initial weights and share weights. During the training of the deep fake image detection model, the parameters in the teacher network remain fixed, while the parameters in the student network are fine-tuned during the training process. (2) Genetic biomimetic feature enhancement module: This includes crossover and mutation operations; (3) Adversarial-driven image blending module: Includes generators and discriminators similar to UNet; A UNet-like generator consists of an input layer, a feature extraction layer, a noise mixing layer, a mask generation layer, and an output layer. The discriminator consists of a prior knowledge-guided module, a genetically-inspired feature enhancement module, and a classifier. The classifier utilizes a normalized index... The function calculates whether an image is real or fake.

3. The deepfake detection method guided by prior knowledge and optimized by dual-domain representation according to claim 2, characterized in that, The specific operations in the depth-based image fabrication detection model are as follows: (1) Using fake facial images and real human face images The images are input into two encoders, and the teacher network and student network of the encoders output the corresponding image features respectively. The knowledge distillation loss guided by prior knowledge is calculated based on the image loss output by the teacher network and student network, and the knowledge in the teacher network is transferred to the student network. (2) Input the image features output by the two encoder student networks into the genetic bionic feature enhancement module, perform crossover and mutation operations, and obtain the enhanced features; (3) Input the image features output by the student network and the enhanced features output by the genetic bionic feature enhancement module into the classifier and output the image forgery detection results; (4) Use real human face images The input is fed into the adversarial-driven mixing module, where a generator produces a mixing mask. Real human face images Hybrid Mask and self-face swapped images Perform fusion to generate a mixed image The mixed image is processed by a discriminator, which outputs the final forgery detection result.

4. The deepfake detection method guided by prior knowledge and optimized by dual-domain representation according to claim 3, characterized in that, Calculate the loss function, and then use the loss function to optimize and train the model. The specific calculation of the loss function is as follows: 1) Calculate the total loss function of the classifier for the prior knowledge-guided module and the adversarial data hybrid module. The calculation formula is as follows: ， in, This represents the distillation loss guided by prior knowledge. Represents class centrality loss. This represents the classification loss guided by prior knowledge. This represents the discriminator classification loss of the adversarial-driven data mixing module. , , , The trade-off parameter represents the balance between the contributions of each loss function; (1) Distillation loss guided by prior knowledge : Student characteristics guided by calculating distillation loss in real-world images With teacher characteristics Student characteristics of close-up, fake images With teacher characteristics The centers are far apart, and the calculation formula is as follows: ， in, This represents the total number of input image samples. Represents cosine similarity. express The center Represents the vector dot product. Representing vectors Norm; (2) Class center loss : The class center loss method brings features of samples belonging to the same class closer to their corresponding class centers, while maintaining distance between feature centers of samples from different classes. The calculation formula is as follows: ， in, Indicates the first Student characteristics of a sample express The corresponding counterfeit category label, express Category center, Represents the square of the Euclidean distance; (3) Prior knowledge-guided module classification loss : ； in, The prior knowledge guidance module indicates the first... The predicted probability of the true label of a sample; (4) Discriminator classification loss of adversarial-driven data mixing module : ， in, The first in the adversarial-driven image blending module The mixed image features corresponding to each sample The adversarial-driven image blending module represents the first... The predicted probability of the true value of a sample; 2) Calculate the total loss function of the adversarial-driven data mixing module generator. The calculation formula is as follows: ， in, This represents the image distortion loss of the adversarial-driven data mixing module generator. This indicates the loss of the adversarial discriminator. and A trade-off parameter representing the balance between the contributions of each entity; (1) Image distortion loss of adversarial-driven data mixing module generator : Structural similarity loss is calculated using real human facial images and composite images. This ensures that the mixed mask generated by the generator differs structurally from real human face images; through various loss mechanisms. and This generates diverse blending masks and reduces image distortion loss. as follows: ， Structural similarity loss : ， in, Indicates a channel. Indicates three channels; Represents structural similarity operations; Indicates that the channel is The A real human face image, Indicates that the channel is The Zhang composite image, express In the The average value of each channel pixel. express In the The average value of each channel pixel. express and In the Covariance of pixel values ​​in each channel express In the The variance of pixel values ​​in each channel express In the The variance of pixel values ​​in each channel and Represents a constant term; By comprehensively considering the structural similarity of each channel, the structural differences between two three-channel images can be measured more comprehensively, guiding the model to generate images that are structurally more different from the original images. Diversity loss: The first diverse loss function : ， in, The information entropy function is represented by... Indicates the first A mixed mask vector; The second diverse loss function : ， in, Indicates the first A mixed mask vector; in, This represents the diversity loss function based on information entropy. This represents the diversity loss based on inter-graph differences. This represents the generated mask; (2) Discriminator loss : in, Represents a random tensor. Represents the random tensor Expectations Represents a generator. This represents the discriminator.

5. A deepfake detection system guided by prior knowledge and optimized by dual-domain representation, used to execute the deepfake detection method guided by prior knowledge and optimized by dual-domain representation as described in any one of claims 1-4, characterized in that, Includes the following units: Data acquisition unit: acquires the human face video image data to be detected. The data to be detected is obtained from different data sources through the detection interface provided by the system. Feature extraction unit: Inputs the facial image to be detected into the pre-trained deep forgery detection model and extracts deep forgery evidence features of the image; The feature extraction unit includes a prior knowledge-guided module, a genetically-inspired feature enhancement module, and an adversarial-driven image blending module; Result recognition unit: Input the deep forgery evidence features of the image into the classifier to obtain the detection result.

Images