Deep forgery detection method and system, storage medium and computer equipment

Abstract

The invention relates to the technical field of deep counterfeit image detection, and discloses a deep counterfeit detection method and system, a storage medium and computer equipment. The method comprises the following steps: firstly, constructing a reference data set containing a forged image and an original real image; secondly, through an integrated model, generating antagonistic samples for the reference data set, and integrating the successfully attacked antagonistic samples into an antagonistic sample set; and finally, merging the reference data set and the adversarial sample set, and constructing a robustness enhanced data set containing four types of samples. In the model training stage, multi-classification cross entropy loss and comparative learning loss are combined, and expression of the model in a feature space is optimized through comparative learning constraint, so that the model learns discriminative features with more compact intra-class features and more dispersed inter-class features. The model trained by the method not only can effectively defend against attack and improve robustness, but also surpasses original detection performance on clean samples, and has remarkable technical advantages and application value.

Description

Technical Field

[0001] This invention relates to the field of deepfake image detection technology, and in particular to a method for improving the robustness of deepfake detection models. Background Technology

[0002] Currently, with breakthroughs in deep learning technology, AI-generated content (AIGC) has experienced rapid development. Among them, deepfake technology, an emerging technology aimed at manipulating facial media through deep neural networks, has attracted much attention due to its ability to generate highly realistic images and videos. These technologies, such as Generative Adversarial Networks (GANs) and diffusion models, enable seamless face swapping, facial expression manipulation, and voice replication, demonstrating enormous application potential in film and television entertainment, virtual social networking, and digital marketing. However, the widespread availability and ease of use of deepfake tools have also created the risk of malicious abuse, including the creation of fake news, identity fraud, and violations of personal privacy. This seriously erodes public trust in digital media and poses a severe challenge to social security. Therefore, developing technologies capable of automatically, accurately, and reliably identifying deepfake content has become a critical issue urgently needing to be addressed in the fields of information security and computer vision.

[0003] To address this challenge, researchers have proposed various deepfake detection methods. Early detectors primarily relied on convolutional neural networks (CNNs) to identify local artifacts in synthetic content, such as unnatural textures, compression marks, or splicing boundaries between different source images. These methods performed well when the training and test data distributions were consistent. However, in real-world open-world scenarios, forgery techniques are constantly evolving, and detectors often encounter novel, untrained forgery samples, leading to insufficient generalization capabilities. Therefore, subsequent research has focused on improving the generalization ability of models, such as using visual transformers (ViTs) to capture global inconsistencies in images, or introducing multimodal models (such as CLIP) to utilize richer semantic information for judgment. These efforts have significantly improved the accuracy of detectors in cross-dataset evaluations and begun to achieve a balance between accuracy, generalization, and inference speed, paving the way for deployment in practical downstream applications.

[0004] Existing detection models reveal a fatal weakness in complex real-world environments: vulnerability to adversarial attacks. Adversarial attacks, by injecting minute perturbations imperceptible to the human eye into the input image, can easily deceive even state-of-the-art deep learning models, causing them to make incorrect judgments. Numerous studies have shown that even highly accurate deepfake detectors can experience a sharp decline in performance, or even complete failure, when faced with these adversarial deepfake samples. Attackers can even create effective attack samples under a "black box" setting where only the detection results can be queried, exposing deployed detection systems to significant and unpredictable security risks.

[0005] To address the threat of adversarial attacks, existing technologies have explored various defense strategies, but these strategies themselves have significant limitations. A common defense approach is input preprocessing, such as image denoising, compression, or feature transformation, to attempt to eliminate adversarial perturbations. However, deepfake detection tasks rely on capturing subtle differences between genuine and fake images. While such preprocessing operations suppress perturbations, they are also highly likely to destroy or distort these crucial discriminative artifacts, often severely impairing the model's detection accuracy on normal, clean samples. Another approach is adversarial training, which introduces adversarial samples during the model's training phase to enhance its robustness. While direct, this method significantly increases the learning difficulty of the model and requires modifications to model parameters or architecture. For already deployed detection models, retraining at a high cost is impractical. Furthermore, there are iterative defense strategies that cleanse the input through multiple iterations, but this typically leads to significant inference latency, failing to meet the real-time requirements of many real-world applications.

[0006] The current technological situation is as follows: on the one hand, deepfake detectors are generally not robust enough in the face of adversarial attacks; on the other hand, existing defense methods either sacrifice the original detection performance or lack feasibility and efficiency in actual deployment. Summary of the Invention

[0007] To address the issues of insufficient robustness and compromised original detection performance in the face of adversarial attacks in existing detection models, this invention provides a deepfake detection method, system, storage medium, and computer device.

[0008] In a first aspect, the present invention provides a deepfake detection method, comprising the following steps: S1. Construct a benchmark dataset A, which includes "real" samples and "fake" samples. The "real" samples are original real images, and the "fake" samples are fake images generated based on the original real images using various forgery methods. S2. Construct an ensemble model composed of multiple heterogeneous benchmark detection models, train the ensemble model based on the benchmark dataset A to obtain a trained ensemble model; use adversarial attack methods to attack the samples in the benchmark dataset A, and filter out the successfully attacked adversarial samples; construct an adversarial sample set B, which includes "adversarial real" samples and "adversarial fake" samples, where the "adversarial real" samples are the successfully attacked adversarial samples corresponding to the "real" samples, and the "adversarial fake" samples are the successfully attacked adversarial samples corresponding to the "fake" samples; S3. Integrate the baseline dataset A and the adversarial sample set B into a robust enhancement dataset C, wherein the robust enhancement dataset C includes the "real" samples, the "fake" samples, the "adversarial real" samples, and the "adversarial fake" samples; S4. Based on the robustness enhancement dataset C, a joint training strategy combining multi-class cross-entropy loss and contrastive learning loss is used to train the deep forgery detection model to be trained, thereby obtaining a trained deep forgery detection model.

[0009] Secondly, the present invention provides a deepfake detection system, comprising the following modules: A data construction module is used to construct a benchmark dataset A, which includes "real" samples and "fake" samples. The "real" samples are original real images, and the "fake" samples are fake images generated based on the original real images through various forgery methods. An adversarial sample generation module is used to construct an ensemble model composed of multiple heterogeneous benchmark detection models. The ensemble model is trained based on the benchmark dataset A to obtain a trained ensemble model. The trained ensemble model uses adversarial attack methods to attack samples in the benchmark dataset A, and successfully attacks adversarial samples are selected. An adversarial sample set B is constructed, which includes "adversarial real" samples and "adversarial fake" samples. The "adversarial real" samples are the successfully attacked adversarial samples corresponding to the "real" samples, and the "adversarial fake" samples are the successfully attacked adversarial samples corresponding to the "fake" samples. The data integration module is used to integrate the baseline dataset A and the adversarial sample set B into a robust enhancement dataset C, wherein the robust enhancement dataset C includes the "real" samples, the "fake" samples, the "adversarial real" samples, and the "adversarial fake" samples; The joint training module is used to train the deep forgery detection model to be trained based on the robustness enhancement dataset C, using a joint training strategy that combines multi-class cross-entropy loss and contrastive learning loss, to obtain a trained deep forgery detection model.

[0010] Thirdly, the present invention provides a storage medium storing a computer program, which, when executed by a processor, enables the implementation of the deepfake detection method.

[0011] Fourthly, the present invention provides a computer device including a memory and a processor, wherein the memory stores computer-readable instructions, and when the computer-readable instructions are executed by the processor, the processor performs the deepfake detection method described above.

[0012] Compared with existing technologies, its advantages are as follows: This invention generates diverse forged images using various forgery methods, constructing a benchmark dataset that is then combined with original images. Multiple attack methods are then applied to the model ensemble to obtain a highly transferable adversarial sample set. This results in an augmented dataset containing four categories of samples: "real," "forged," "adversarial real," and "adversarial forged." An innovative multi-classification and contrastive learning joint training strategy is employed, enabling the model not only to learn to distinguish between real and fake content but also to explicitly perceive and identify adversarial perturbations. This method significantly improves the model's robustness to unknown adversarial attacks while maintaining its original detection accuracy on clean samples to the maximum extent, without requiring lossy input preprocessing or additional computation during inference. Attached Figure Description

[0013] Figure 1 This is a flowchart of a deepfake detection method proposed in this invention; Figure 2 This is a schematic diagram illustrating the process of generating adversarial sample set B in an embodiment of the present invention; Figure 3 This is a schematic diagram of integrating the benchmark dataset A and the adversarial sample set B into a four-class robustness enhancement dataset C in an embodiment of the present invention; Figure 4 This is a framework diagram of a joint training strategy that combines multi-class cross-entropy loss and contrastive learning loss in an embodiment of the present invention. Detailed Implementation

[0014] The specific embodiments of the present invention will be described in further detail below with reference to the accompanying drawings and examples. In the description of the present invention, unless otherwise stated, "a plurality of" means two or more. The following examples are used to illustrate the present invention, but are not intended to limit the scope of the invention.

[0015] like Figure 1 As shown, this embodiment provides a deepfake detection method, the steps of which are as follows: S1. Construct a benchmark dataset A, which includes "real" samples and "fake" samples. The "real" samples are original real images, and the "fake" samples are fake images generated based on the original real images using various forgery methods. S2. Construct an ensemble model composed of multiple heterogeneous benchmark detection models, train the ensemble model based on the benchmark dataset A to obtain a trained ensemble model; use adversarial attack methods to attack the samples in the benchmark dataset A, and filter out the successfully attacked adversarial samples; construct an adversarial sample set B, which includes "adversarial real" samples and "adversarial fake" samples, where the "adversarial real" samples are the successfully attacked adversarial samples corresponding to the "real" samples, and the "adversarial fake" samples are the successfully attacked adversarial samples corresponding to the "fake" samples; S3. Integrate the baseline dataset A and the adversarial sample set B into a robust enhancement dataset C, wherein the robust enhancement dataset C includes the "real" samples, the "fake" samples, the "adversarial real" samples, and the "adversarial fake" samples; S4. Based on the robustness enhancement dataset C, a joint training strategy combining multi-class cross-entropy loss and contrastive learning loss is used to train the deep forgery detection model to be trained, thereby obtaining a trained deep forgery detection model.

[0016] This embodiment constructs a four-class classification dataset containing real, fake, and their corresponding highly transferable adversarial samples, and employs an innovative joint training strategy. This enables the model to explicitly perceive and learn adversarial perturbation patterns while learning to distinguish between real and fake features. This method aims to address the problem of drastic performance degradation of existing detection models when facing adversarial attacks, significantly improving the model's adversarial robustness without sacrificing original detection performance.

[0017] Specifically, this embodiment innovatively extends the traditional "true / false" binary classification problem into a four-class classification problem: "true / fake / adversarial true / adversarial fake". In this way, the model is explicitly guided to identify the presence of adversarial perturbations. Simultaneously, the introduced supervised contrastive learning loss further forces the model to learn more compact intra-class and more dispersed inter-class discriminative representations in the feature space, thereby constructing a robust feature space that is insensitive to minor perturbations, fundamentally enhancing the model's defensive capabilities.

[0018] Step S1 constructs a benchmark dataset A, which includes "real" samples and "fake" samples. The "real" samples are original real images, and the "fake" samples are fake images generated based on the original real images using various forgery methods.

[0019] The original real images are sourced from several industry-recognized large-scale public datasets, mainly from high-quality face or scene datasets such as CelebA-HQ, FF++, and FFHQ.

[0020] The forged images are generated by applying various advanced forgery methods to the original real images. These methods cover the current mainstream forgery paradigms, including face swapping, face reenactment, full-face synthesis, or face editing. Specifically: Face swapping: The core objective of this technology is to seamlessly transplant a person's face (source identity) onto a person in another video or image (target identity) while preserving the target person's original facial expressions, head posture, and lighting conditions.

[0021] Classic methods like DeepFake and FaceSwap are typically based on an autoencoder architecture. A typical implementation involves training two independent autoencoders: one for the source identity A and the other for the target identity B. Crucially, these two autoencoders share the same encoder but have their own independent decoders. The shared encoder learns to extract core facial features independent of identity, such as pose, expression, and lighting, and compresses them into a latent feature vector. During the swap, an image frame of the target person B is input into the shared encoder to obtain the feature vector, which is then fed into the decoder of the source person A. Since decoder A has only learned how to reconstruct A's face from the feature vector, it will derive A's face using B's pose and expression, thus achieving identity swapping. Artifacts left by this method typically appear at the boundaries between the swapped face and the original hair and neck, and may include color, lighting, or resolution mismatches.

[0022] Face reenactment: This technique aims to manipulate a static target image of a person by using the facial expressions and head postures of a source image of a person, making it mimic the movements of the source image of a person.

[0023] Face2Face is a 3D model-based method. It first reconstructs 3D face models from both the source and target images. Then, it extracts expression, pose, and lighting parameters from the source image and transfers these parameters to the 3D model of the target person, generating a new 3D rendering result. Finally, a rendering network blends this new 3D rendering result with the background of the original target image to make it look more realistic.

[0024] NeuralTextures is a method based on neural network rendering. It also estimates parameters such as the pose of the source and target images, but instead of directly rendering the entire 3D model, it maps the texture of the target image onto a neural texture, samples and warps this texture using parameters from the source image, and then renders it back as a 2D image. This method is more flexible than traditional rendering when handling extreme expressions and poses, but may produce unnatural distortions or blurring in texture details.

[0025] Entire Face Synthesis: This technology does not rely on any existing target image, but creates a completely non-existent yet highly realistic human face image from scratch.

[0026] Methods based on generative adversarial networks, such as StyleGAN and ProGAN, are representative of this field. They are trained through an adversarial game between a generator and a discriminator. The generator attempts to generate realistic faces from a random noise vector, while the discriminator tries to distinguish real faces from those created by the generator. After sufficient training, the generator becomes capable of creating high-quality faces. StyleGAN, in particular, achieves unprecedented precision and generation quality by decoupling different levels of facial features (such as pose, identity, hairstyle, freckles, etc.) in the latent space. The artifacts produced by these methods are often global, such as illogical backgrounds, symmetrical details (like earrings), or unique frequency characteristics observed in spectral analysis.

[0027] Methods based on diffusion models, such as Stable Diffusion and Midjourney, represent a more recent technological approach. They generate images through a "denoising" process: first, noise is progressively added to a real image until it becomes completely random noise; then, a neural network is trained to learn the inverse process—that is, to progressively and iteratively remove noise from the random noise, ultimately recovering a clear image. By guiding the process, the denoising process can be directed towards the desired outcome, resulting in highly controllable and extremely high-quality images.

[0028] Face Editing: This technology focuses on modifying specific attributes of an existing facial image, such as changing age, gender, hair color, adding glasses, or changing facial expressions.

[0029] StarGAN and StyleCLIP are typical methods in this field. They usually employ GANs with encoder-decoder structures. The core idea is that during training, the input includes not only an image but also a vector or label representing the target attribute. The model learns to encode both the image and attribute information into a unified latent space. During decoding, the attributes of the output image can be controlled by changing the input attribute vector. For example, given a photo of a person not smiling and an attribute vector for "smiling," the model can output a photo of that person smiling. These methods may produce unnatural traces at the transition between modified and unmodified regions, or cause slight, unintended changes to identity features when modifying attributes.

[0030] The forged images are generated by sampling samples in equal proportions from multiple datasets generated by the various forgery methods, to ensure the diversity and balance of forgery types in the benchmark dataset A.

[0031] Step S2: Construct an ensemble model composed of multiple heterogeneous benchmark detection models. Train the ensemble model on the benchmark dataset A to obtain a trained ensemble model. Use an adversarial attack method to attack the samples in the benchmark dataset A and filter out the successfully attacked adversarial samples. Construct an adversarial sample set B, which includes "adversarial real" samples and "adversarial fake" samples. The "adversarial real" samples are the successfully attacked adversarial samples corresponding to the "real" samples, and the "adversarial fake" samples are the successfully attacked adversarial samples corresponding to the "fake" samples. In this embodiment, the ensemble model consists of nine heterogeneous benchmark detection models. To generate adversarial samples with high transferability (i.e., the ability to simultaneously deceive multiple different models), this embodiment selects nine heterogeneous models and constructs an ensemble system containing these nine models. The heterogeneity of the models is crucial, ensuring that attacks do not target the weaknesses of a single model; the heterogeneity of the ensemble model is reflected in the diversity of the backbone network, the diversity of the analysis domain, the diversity of core principles, and the diversity of training paradigms.

[0032] Specifically, the nine models were selected as follows: Model 1: Models based on frequency domain analysis search for forgery traces in the frequency space. The input image is transformed to the frequency domain through Discrete Cosine Transform (DCT) or Fourier Transform (FFT), and a specially designed network (such as ResNet) is used to learn and identify anomalous frequency artifacts introduced by GANs or other generative models, such as unnatural periodic patterns. It focuses on global inconsistencies in the frequency of the image.

[0033] Model 2: Models based on CLIP pre-training and local features leverage CLIP's powerful pre-trained visual-language representation capabilities. They employ a Vision Transformer or ResNet initialized by CLIP as the backbone network to capture high-level semantic features. Combined with strategies for weighting or masking local image regions, they enhance sensitivity to local tampering.

[0034] Model 3: The model based on synthetic frequency pattern injection employs a unique training paradigm. Instead of directly using a large number of fake samples, it injects artificially synthesized frequency artifacts that simulate the fake process into real images. This allows the model to learn a more generalized ability to detect frequency anomalies, rather than overfitting to traces of a specific generative model.

[0035] Model 4: The frequency and spatial domain co-model extracts features from the spatial and frequency domains in parallel. It consists of two independent branches, one processing RGB images and the other processing their spectrum. The features from these two branches are fused through a co-model module, thereby making judgments by utilizing both local texture artifacts and global frequency anomalies.

[0036] Model 5: The model based on image gradient analysis first converts the input image into its gradient map, because forgery processes often destroy the natural gradient distribution of the original image, leaving discontinuous or abnormally smooth gradient patterns. The model learns to identify anomalies in these gradient maps by training on a ResNet-based detector.

[0037] Model Six: The residual network model based on noisy printing uses a deep residual network to extract and learn the noisy fingerprint of the input image, and classifies it by comparing the fingerprint with the fingerprint of a real image or a known forgery model.

[0038] Model 7: The model based on diffusion model reconstruction error utilizes a pre-trained diffusion model. A diffusion model trained on real images will have a small reconstruction error when reconstructing a real image using a "noise-addition-denoising" process; however, when reconstructing a forged image, the reconstruction error will significantly increase because the distribution of the forged image does not match the real distribution learned by the model. This model detects such errors by calculating and analyzing their magnitude and patterns.

[0039] Model 8: The standard ResNet model is a deep forgery detector using the ResNet-50 architecture. It represents a benchmark component in ensemble models and primarily relies on the powerful feature extraction capabilities of deep convolutional networks to learn discriminative features directly from pixels.

[0040] Model Nine: Model 9 adds knowledge distillation optimization to Model 7. It adopts the knowledge distillation technique, which trains a lighter student model to directly predict or imitate the reconstruction error feature map calculated by the teacher model. This makes the model greatly improve the inference speed while maintaining the high accuracy of the reconstruction error method.

[0041] Specifically, the trained ensemble model uses adversarial attack methods to attack samples in the benchmark dataset A, and filters out successfully attacked adversarial samples, including: S2.1. Using a sample from the benchmark dataset A as an input sample, a benchmark detection model is randomly selected from the trained ensemble model as an attack model. The attack model uses an adversarial attack method to attack the input sample and generate an adversarial sample. S2.2. Input the adversarial sample into the other benchmark detection models in the trained ensemble model (excluding the attack model) for inference, and obtain the inference result; S2.3. Count the number of models whose inference results are opposite to the original labels of the input sample. When the number of models is greater than or equal to a preset threshold, the attack is considered successful. If the input sample is the "real" sample, the successfully attacked adversarial sample is an "adversarial real" sample. If the input sample is the "fake" sample, the successfully attacked adversarial sample is an "adversarial fake" sample.

[0042] The process of selecting successful adversarial samples is an ingenious "attack-verify-vote" loop designed to generate highly transferable adversarial samples that are effective against most members of the ensemble model. This process relies not only on powerful attack algorithms but, more importantly, on leveraging the heterogeneity of the ensemble model to filter out truly universally deceptive samples.

[0043] The adversarial attack methods can employ a variety of advanced approaches based on different principles, including Projected Gradient Descent (PGD), Fast Signed Gradient (FGSM), C&W attack, and Automatic Attack (AA). In practice, these methods can be selected randomly or alternately to maximize the diversity of generated samples.

[0044] In this embodiment, as Figure 2 As shown, for any sample image in the benchmark dataset A (Its real label is) ) and attack models randomly selected from the ensemble model Different adversarial attack methods generate adversarial samples The specific methods are as follows: Projected gradient descent (PGD) is a powerful, multi-step iterative white-box attack method that finds the optimal attack direction through multiple iterations within a perturbation budget. Its main process is as follows: Initialization: In Centered on Norm radius is (For example Randomly select a point inside the hypercube. As the starting point of the attack.

[0045]

[0046] Iterative attack: Perform For example Iterative updates. In the... In the next iteration:

[0047] in, Let cross-entropy be the loss function. It is the gradient of the loss with respect to the current input. It is the step size of a single-step attack (e.g.) ), It's a projection operation that ensures the updated sample is always in... of Within the neighborhood.

[0048] Final sample:

[0049] Fast Gradient Sign Method (FGSM), as an efficient single-step attack method, adds the maximum perturbation all at once along the direction of gradient ascent of the loss function. This method generates adversarial examples extremely quickly and is suitable for large-scale generation of adversarial examples. However, the perturbations it generates are relatively easy to detect and defend against. The attack method is as follows:

[0050] C&W attacks, as a powerful optimization-based attack method, aim to find ways to perturb the object itself while satisfying the condition of "misclassification". norm (e.g.) The smallest adversarial sample (norm). C&W attacks are usually slower than PGD, but they can generate more subtle and harder-to-detect perturbations. They focus on the stealth of the perturbation rather than its magnitude, making them an excellent choice for generating high-quality hard samples.

[0051] Automated Attack (AA) is not a single algorithm, but a standardized, parameter-tuning-free combination of attacks. It automatically runs a series of complementary attack methods and reports the result of the most effective one. AA typically includes two PGD-based variants (APGD-CE and APGD-DLR), a boundary-based attack (FAB), and a black-box attack (SquareAttack). In this invention, using AA ensures that the generated adversarial sample set B includes samples generated by the most state-of-the-art and comprehensive attack methods currently available.

[0052] Screening process: For each sample in dataset A One of the four attack methods (PGD, FGSM, C&W, AA) is randomly selected. Simultaneously, one of the nine heterogeneous models is randomly selected. Generate adversarial samples as targets for attack. .

[0053] The generated Input into the ensemble model except Inference verification was performed on the remaining eight models, and statistics were collected on them. The prediction result is judged as The number of models in the opposite category, denoted as Additionally, let the total number of models in the ensemble model be . ,but:

[0054] like:

[0055] Then it is believed This is a high-quality, highly transferable adversarial sample, considered a successful attack. and its corresponding original tags These samples are saved and together form the adversarial sample set B.

[0056] This approach, based on heterogeneous integrated model voting and diverse attack strategies, ensures that the final selected adversarial sample set B possesses unprecedented diversity and challenge. It does not merely exploit vulnerabilities in a single model or the characteristics of a single attack algorithm, but rather obtains highly transferable samples by discovering more fundamental blind spots shared by the model group, thus guaranteeing the universal adversarial nature of the adversarial sample set.

[0057] Step S4 integrates the baseline dataset A and the adversarial sample set B into a robust enhancement dataset C. The robust enhancement dataset C includes the "real" samples, the "fake" samples, the "adversarial real" samples, and the "adversarial fake" samples, such as... Figure 3 As shown.

[0058] Step S5 uses the robustness enhancement dataset C as a basis to train the deep forgery detection model to be trained by a joint training strategy that combines multi-class cross-entropy loss and contrastive learning loss, so as to obtain a trained deep forgery detection model.

[0059] One of the core technical contributions of this invention is the design and implementation of a joint training framework based on dual supervision signals. This framework combines traditional classification tasks with contrastive learning, forcing the model to construct a geometrically robust and structurally sound feature space that is highly discriminative for all four categories (real, fake, adversarial real, and adversarial fake).

[0060] The deepfake detection model to be trained includes a feature extractor. and a classification head The feature extractor The classification head is used to obtain the feature vector of the sample. The predicted probabilities are used to obtain the sample classification, which includes "real", "fake", "adversarial real", and "adversarial fake".

[0061] In this embodiment, the deepfake detection model to be trained It can be built based on mainstream deep learning network architectures in the industry, such as convolutional neural networks (CNN) or vision transformers (ViT). To demonstrate the universality and efficiency of the framework of this invention, the following lists three specific and selectable implementation methods that represent the current cutting-edge technical routes in the field of forgery detection: Architecture Option 1: The advantage of the detector based on BNext and fusing multimodal features lies in its multimodal input fusion mechanism. This model uses a high-performance BNext network as the feature extraction backbone. However, the input undergoes careful preprocessing and channel expansion before being fed into the backbone network: Calculate the magnitude of the Sobel gradient in the image; this channel is designed to capture unnatural edges and texture transitions in the image. Perform a Fourier transform on the image and use the logarithmic magnitude of the spectrum as one channel, because many generative models leave unique, identifiable periodic artifacts in the frequency domain when generating images through upsampling operations. Local Binary Patterns (LBPs) are powerful texture descriptors that are extremely sensitive to microscopic texture changes in images. Due to pixel-level generation or tampering, the microscopic texture statistical characteristics of forged images often differ significantly from those of real photos.

[0062] This architecture fuses the original RGB three channels with one or more auxiliary channels through an adapter convolutional layer, enabling the model to simultaneously obtain information from the pixel domain, gradient domain, frequency domain, and texture domain, constructing an extremely rich and three-dimensional input representation, which greatly enhances its ability to capture diverse forgery traces.

[0063] Architecture Option Two: Dedicated networks based on bi-branch frequency domain analysis completely abandon spatial domain analysis and focus on finding evidence of forgery in the frequency space, because the traces left by the forgery process in the frequency domain are more universal and stable than those in the spatial domain.

[0064] A dual-branch structure was designed: The low-frequency branch processes the central region of the frequency spectrum, focusing on analyzing the overall structure and content-related low-frequency information of the image. The high-frequency branch processes the outer region of the frequency spectrum, focusing on analyzing high-frequency noise and artifacts introduced by upsampling, compression, or generation models.

[0065] The model concatenates the features extracted from the two branches and then feeds them into the classification head, enabling the model to decouple and learn fake features in different frequency bands separately, thus achieving in-depth mining of frequency domain information.

[0066] Architecture Option 3: The core of the detector based on semantic and underlying bias inconsistency lies in detecting the inconsistency between high-level semantics and underlying physical traces in a fake image. This is because the semantic content of a real image is harmoniously consistent with its underlying frequency or noise bias, while a fake image will disrupt this harmony.

[0067] First, high-level semantic features of the image are extracted using the powerful EfficientNet backbone network; Then, the low-level frequency bias or noise fingerprint of the image is extracted through specific network layers or preprocessing steps.

[0068] The model learns to judge whether there is inconsistency between semantic features and bias features through attention mechanisms or other fusion methods. For example, a region may be a clear face in semantics, but it may present a noise pattern similar to the background in the underlying bias, which is a strong fake signal.

[0069] In joint training, regardless of which architecture is chosen, the final classification layer will be temporarily set aside, allowing it to function as a pure feature extractor. Output a high-dimensional feature vector for subsequent loss calculation. .

[0070] like Figure 4 As shown, when a batch of data is sampled from the four-class robustness enhancement dataset C and input into any of the above model architectures, its forward propagation and loss calculation process is unified as follows: An image Feature extractor of the selected model A batch of corresponding, processed Normalized eigenvectors:

[0071] Feature vector They are simultaneously fed into two parallel computational paths with complementary objectives: The goal in the classification path is to improve classification accuracy, and the feature vector... It is fed into a fully connected classification head Output the probability distribution predicted by the model:

[0072] Then calculate the probability distribution predicted by the model. With real labels Standard cross-entropy loss between:

[0073] in, The true label is denoted by ; C represents the number of sample categories, specifically 4 in this case. The predicted probability for classifying a sample.

[0074] It is the main supervisory signal driving the model to complete the classification task. It ensures the correctness of the classification boundaries. Then, The limitation of this approach is that it does not directly concern itself with the geometric structure of the feature space. As long as the samples can be correctly classified, it does not require that samples of the same class be closely clustered in the feature space, or that samples of different classes be far apart. This may result in a loose distribution of feature clusters and fragile inter-class boundaries, making them vulnerable to adversarial attacks.

[0075] Therefore, a contrastive learning path was introduced in order to structure the feature space.

[0076] Contrastive learning groups all relevant positive samples together and separates irrelevant negative samples. For each feature vector within a batch... Its corresponding positive sample set For all items within the batch Other sample index sets with the same labels, negative sample set For all items within the batch A set of sample indexes with different labels. For Any positive sample in The number of positive samples in a batch is denoted as The comparative loss is calculated as follows:

[0077] in, The set of positive samples; For negative sample sets; positive sample set The feature vector of any positive sample in the set; zi is the feature vector of the current sample, and za is the set of negative samples. The feature vector of any negative sample in the dataset; This is the temperature coefficient.

[0078] The final The loss is calculated for all anchor points and all positive sample pairs within the batch. The average value.

[0079] Ultimately, the model's total loss The weighted sum of these two losses:

[0080] In this embodiment, The value is set to 0.4.

[0081] The calculated total loss is used for backpropagation, and the parameters of the entire model are updated simultaneously.

[0082] Through this innovative joint training method, a feature space with excellent geometric properties is actively constructed during the training process, in which high cohesion within classes and high separability between classes are achieved.

[0083] After obtaining the trained deepfake detection model, the method further includes an evaluation step: loading the checkpoints of the trained deepfake detection model and testing it on the test sets of the benchmark dataset A, the adversarial sample set B, and the robustness enhancement dataset C (which is a combination of the two) generated by the method of the present invention, respectively, to obtain indicators such as accuracy, robustness, and practicality, thereby verifying the effectiveness of the deepfake detection method.

[0084] All model training and inference in this embodiment were performed on an NVIDIA GeForce RTX 3090. All networks were implemented in PyTorch and optimized using the default Adam settings. The training epochs were uniformly set to 50. The batch size during training was adjusted based on the model size and memory usage, typically set to 32 or 64. The initial learning rate was set to 0.0001, and a cosine annealing learning rate scheduler with 5 warm-up epochs was used.

[0085] In summary, this invention provides a deepfake detection method, system, storage medium, and computer device. It constructs a four-class classification dataset containing real, fake, and their corresponding highly transferable adversarial samples, and employs an innovative joint training strategy. This enables the model to learn to distinguish between real and fake features while explicitly perceiving and learning adversarial perturbation patterns, thereby obtaining a robust detection model with strong generalization defense capabilities against unknown attacks. This invention innovatively extends the traditional "real / fake" binary classification problem into a four-class classification problem and introduces supervised contrastive learning loss, forcing the model to learn more compact intra-class and more dispersed inter-class discriminative representations in the feature space. This constructs a structurally stable feature space that is insensitive to minor perturbations, fundamentally enhancing the model's defense capabilities. This method is universally applicable to various mainstream detector architectures and can systematically improve their stability and reliability in complex adversarial environments. The model trained by this method is more robust than many models that rely solely on data augmentation or a single defense strategy, and it also demonstrates that jointly optimizing data distribution and feature space geometry is an effective way to improve adversarial robustness.

[0086] The above description is only a preferred embodiment of the present invention. It should be noted that for those skilled in the art, several improvements and substitutions can be made without departing from the technical principles of the present invention, and these improvements and substitutions should also be considered within the scope of protection of the present invention.

Claims

1. A deepfake detection method, characterized in that, The steps are as follows: S1, a reference data set A is constructed, the reference data set A includes "real" samples and "fake" samples, the "real" samples are original real images, and the "fake" samples are fake images generated based on the original real images through multiple fake methods; S2, an integrated model composed of multiple heterogeneous reference detection models is constructed, the integrated model is trained based on the reference data set A to obtain a trained integrated model; the trained integrated model uses an adversarial attack method to attack the samples of the reference data set A, and screens out adversarial samples that are successfully attacked; an adversarial sample set B is constructed, the adversarial sample set B includes "adversarial real" samples and "adversarial fake" samples, the "adversarial real" samples are adversarial samples that are successfully attacked and correspond to the "real" samples, and the "adversarial fake" samples are adversarial samples that are successfully attacked and correspond to the "fake" samples; S3, the reference data set A and the adversarial sample set B are integrated into a robustness-enhanced data set C, the robustness-enhanced data set C includes the "real" samples, the "fake" samples, the "adversarial real" samples and the "adversarial fake" samples; S4, based on the robustness-enhanced data set C, a joint training strategy combining multi-classification cross-entropy loss and contrast learning loss is used to train a to-be-trained deep fake detection model to obtain a trained deep fake detection model.

2. The method of claim 1, wherein, In step S1, the multiple fake methods include face exchange, face replay, full-face synthesis or face editing.

3. The method of claim 1, wherein, In step S2, the trained integrated model uses an adversarial attack method to attack the samples of the reference data set A, and screens out adversarial samples that are successfully attacked, including: S2.1, taking one sample of the reference data set A as an input sample, randomly selecting one reference detection model from the trained integrated model as an attack model, the attack model using an adversarial attack method to attack the input sample to generate an adversarial sample; S2.2, inputting the adversarial sample into other reference detection models in the trained integrated model except the attack model to perform reasoning to obtain a reasoning result; S2.3, counting the number of models whose reasoning results are opposite to the original labels of the input samples, when the number of models is greater than or equal to a preset threshold, it is determined that the attack is successful; if the input sample is the "real" sample, the adversarial sample that is successfully attacked is the "adversarial real" sample; if the input sample is the "fake" sample, the adversarial sample that is successfully attacked is the "adversarial fake" sample.

4. The method of claim 3, wherein, In step S2.1, the adversarial attack method includes a projected gradient descent method, a fast gradient sign method, a C&W attack or an automatic attack.

5. The method of claim 1, wherein, In step S4, the deep fake detection model to be trained comprises a feature extractor and a classification head ; the feature extractor is configured to obtain a feature vector of a sample, and the classification head is configured to obtain a predicted probability of a sample classification, the sample classification comprising "real", "fake", "adversarial real", and "adversarial fake".

6. The method of claim 1, wherein, In step S4, the total loss function of the joint training strategy is L total a weighted sum of one multi-class cross-entropy loss L CE and one contrastive learning loss L con The expression of the total loss L total is: wherein, is a hyperparameter for balancing the two losses.

7. The method of claim 6, wherein, In step S4, it is characterized in that the multi-class cross-entropy loss L CE The expression is: wherein, is the true label; C is the number of sample classification categories, which is specifically 4 here; is the predicted probability of sample classification. In the robustness-enhanced dataset C, a sample is randomly selected as an anchor sample, and a sample is randomly selected from other samples belonging to the same sample category as the anchor sample as a positive sample; all samples different from the sample category of the anchor are negative samples; the contrastive learning loss is obtained by maximizing the similarity of the anchor sample and the positive sample in the feature space and minimizing the similarity of the anchor sample and the negative sample in the feature space L con , the expression of the contrastive learning loss L con is: wherein, is a positive sample set; is a negative sample set; is a positive sample set is a feature vector of any one positive sample in the positive sample set; zi is a feature vector of the current sample, za is a feature vector of any one negative sample in the negative sample set is a feature vector of any one negative sample in the negative sample set; is a temperature coefficient.

8. A deepfake detection system, comprising: The following modules are included: A data construction module is configured to construct a benchmark dataset A, which includes "real" samples and "fake" samples, the "real" samples being original real images, and the "fake" samples being fake images generated based on the original real images through multiple fake methods; An adversarial sample generation module is configured to construct an ensemble model composed of multiple heterogeneous benchmark detection models, train the ensemble model based on the benchmark dataset A to obtain a trained ensemble model, use an adversarial attack method to attack samples of the benchmark dataset A by using the trained ensemble model, and screen out adversarial samples with successful attacks; an adversarial sample set B is constructed, which includes "adversarial real" samples and "adversarial fake" samples, the "adversarial real" samples being adversarial samples with successful attacks corresponding to the "real" samples, and the "adversarial fake" samples being adversarial samples with successful attacks corresponding to the "fake" samples; A data integration module is configured to integrate the benchmark dataset A and the adversarial sample set B into a robustness-enhanced dataset C, which includes the "real" samples, the "fake" samples, the "adversarial real" samples, and the "adversarial fake" samples; A joint training module is configured to train a to-be-trained deep fake detection model based on the robustness-enhanced dataset C by using a joint training strategy combining a multi-classification cross-entropy loss and a contrastive learning loss, to obtain a trained deep fake detection model.

9. A storage medium, characterized by A storage medium stores a computer program, which, when executed by a processor, can implement the deep fake detection method of any one of claims 1-7.

10. A computer device, comprising: A device includes a memory and a processor, and the memory stores computer readable instructions, wherein the computer readable instructions, when executed by the processor, cause the processor to perform the deep fake detection method of any one of claims 1-7.

Images