Face image forgery detection method based on diffusion inversion trajectory geometry supervision
Patent Information
- Application Number
- CN202611524162.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-09-29
- Publication Date
- 2026-10-30
AI Technical Summary
[0009]本发明提供了一种基于扩散反演轨迹几何监督的人脸图像伪造检测方法，解决的技术问题包括：第一，现有伪造检测方法将扩散反演结果压缩为单一终点残差，丢失了反演过程中按步骤变化的轨迹信息，导致跨域泛化能力受限；第二，扩散反演需要多步迭代，推理延迟和算力消耗较高，难以在在线场景中部署；第三，已有方法在训练与测试伪造类型不一致时容易把数据集特定纹理、压缩和采样偏差误判为伪造证据，造成明显的泛化退化
[0064]（1）本发明将扩散反演过程作为训练期的离线轨迹特征监督源，通过对训练图像执行多步骤扩散反演获得按步骤排列的潜空间状态张量序列，并将各步骤的潜空间状态张量之间的变化关系向量化后计算五类过程性轨迹几何特征，组成轨迹特征监督序列，弥补了终点残差方法对中间过程信息丢失的不足。
Smart Images
Figure CN123016533A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to a face image forgery detection method based on diffusion inversion trajectory geometric supervision, belonging to the fields of image processing, information security and computer vision. Background Technology
[0002] With the rapid development of deepfake technologies such as face generation, face swapping, and expression transfer, the security risks posed by highly realistic forged content are becoming increasingly prominent. Deepfake detection has become a key task in trusted vision systems. The core objective of deepfake detection is to automatically distinguish between real face images and face images synthesized by algorithms such as face generation, face swapping, and expression-driven techniques.
[0003] Current methods for detecting face forgery can be broadly categorized into four types. The first type is based on visual classification backbones, such as using deep networks like Xception, ResNet, or MesoNet to treat forgery detection as a standard binary classification problem. The second type focuses on methods that fuse boundaries or fine-grained artifacts, such as Face X-Ray and FakeDiffer, which address edge inconsistencies caused by face swapping or splicing. The third type is based on frequency domain features, such as F3-Net, SPSL, and methods based on SRM filters or high-pass filters, which use upsampling, compression, and residual fusion to detect high-frequency anomalies. The fourth type is based on reconstruction learning or diffusion priors, which achieves detection by comparing the differences in reconstruction distribution between real and forged samples.
[0004] However, the aforementioned methods generally suffer from significant generalization degradation in cross-domain scenarios. Publicly available experiments have shown that many mainstream detection methods can achieve an AUC of over 90% within the FaceForensics++ (FF++) source domain, but when the trained detectors are transferred to external datasets such as Celeb-DF and WildDeepfake, the AUC typically drops by 20 to 35 percentage points. This phenomenon indicates that the detector does not always learn the essential differences in the forged content, but may include texture, compression, or sampling biases specific to the training set.
[0005] To incorporate diffusion priors into forgery detection, researchers proposed the Diffusion Reconstruction Error (DIRE) method. This method performs a deterministic diffusion implicit model (DDIM) inversion on the input image to generate a reconstructed image, and then uses the residual between the input and reconstructed images as the criterion. The DIRE method shows that genuine samples conforming to the natural image prior yield smaller residuals near the inversion endpoint, while forged samples deviating from the natural image manifold have larger residuals.
[0006] However, diffusion reconstruction error methods only use the endpoint residual after inversion, making it difficult to characterize the local shifts, directional changes, and stability of the latent space trajectory in intermediate steps. The endpoint residual essentially describes the difference in the inversion results, not the difference in the inversion process. For cross-domain forgery detection, trajectory variables during the inversion process may be closer to the generation mechanism itself than a single endpoint residual, thus possessing stronger transfer potential.
[0007] Furthermore, diffusion inversion typically requires dozens of iterations, placing high demands on computational resources and inference latency. Performing a complete diffusion inversion for every image to be detected during the testing phase would significantly increase the burden on the detection system, making it difficult to meet the requirements for online deployment.
[0008] While existing technologies involve diffusion model reconstruction, spatial and frequency feature fusion, and face forgery detection, they still lack a complete technical solution that constructs the latent space state changes corresponding to multiple diffusion inversion steps as process supervision and directly obtains the process features through the prediction module during the inference stage, thereby avoiding online diffusion inversion. Summary of the Invention
[0009] This invention provides a face image forgery detection method based on diffusion inversion trajectory geometric supervision, addressing the following technical problems: First, existing forgery detection methods compress the diffusion inversion result into a single endpoint residual, losing the trajectory information that changes step-by-step during the inversion process, resulting in limited cross-domain generalization ability; Second, diffusion inversion requires multiple iterations, leading to high inference latency and computational consumption, making it difficult to deploy in online scenarios; Third, existing methods are prone to misjudging specific textures, compression, and sampling biases in the dataset as forgery evidence when the forgery types in training and testing are inconsistent, causing significant generalization degradation. This invention solves these three problems simultaneously by explicitly modeling the diffusion inversion process as a step-by-step trajectory feature supervision sequence and embedding a trajectory feature prediction module to transform offline process supervision into online forward prediction.
[0010] To solve the above-mentioned technical problems, the technical solution adopted by the present invention is as follows:
[0011] A face image forgery detection method based on diffusion inversion trajectory geometric supervision:
[0012] Acquire the image of the face to be detected;
[0013] The face image to be detected is input into a pre-trained forgery detection model. The trajectory feature prediction module in the forgery detection model obtains the predicted trajectory feature sequence and extracts the image features of the face image to be detected.
[0014] The predicted trajectory feature sequence is interactively fused with the image features of the face image to be detected to obtain fused features;
[0015] The forgery detection model outputs the authenticity detection result of the face image to be detected based on the fusion features;
[0016] The training process of the forgery detection model includes:
[0017] Obtain training images with real-world category labels;
[0018] The training images are latent space encoded, and a multi-step diffusion inversion process is performed based on a pre-trained diffusion generation model to obtain multiple latent space state tensors arranged according to the inversion steps.
[0019] The state changes between adjacent latent space state tensors are vectorized into state increment vectors. Based on the numerical relationship between the state increment vectors, trajectory features corresponding to each inversion step are generated, and a trajectory feature supervision sequence is formed according to the order of the inversion steps.
[0020] The training image is input into the forgery detection model, and the trajectory feature prediction module in the forgery detection model obtains the predicted trajectory feature sequence and extracts the image features of the training image.
[0021] By fusing the predicted trajectory feature sequence and the image features of the training images, a realistic prediction result is output;
[0022] The forgery detection model is trained based on the classification loss between the authenticity prediction result and the authenticity category label, and the trajectory alignment loss between the predicted trajectory feature sequence and the trajectory feature supervision sequence.
[0023] Further, the training images are latent space encoded, and a multi-step diffusion inversion process is performed based on the pre-trained diffusion generation model, including:
[0024] Preprocessed training images Variational autoencoder encoding is performed to obtain the initial latent space state tensor. ,in A variational autoencoder is built into the pre-trained diffusion generative model. Based on the pre-trained diffusion generative model, a deterministic diffusion implicit model inversion method is adopted, and T inversion steps are executed sequentially with an inversion step count of T=20 to obtain a sequence of latent space state tensors arranged according to the inversion steps: ;
[0025] The update formula for the deterministic diffusion implicit model inversion method is:
[0026]
[0027] in, This represents the cumulative product of the noise scheduling parameters over t inversion steps. t represents the noise prediction network in the pre-trained diffusion generation model, and t is the inversion step number.
[0028] Furthermore, the state increment vector is ;
[0029] in, This represents the operation of flattening a tensor into a vector along all dimensions. , , These represent the height, width, and number of channels of the latent space state tensor, respectively. .
[0030] Furthermore, the trajectory features corresponding to each inversion step include: first-order state difference magnitude, second-order state difference magnitude, increment direction consistency, increment direction deviation, and latent space state magnitude.
[0031] First-order state difference module ;
[0032] Second-order state difference module ;
[0033] Incremental directional consistency ;
[0034] Where t≥2, Representing vectors transpose, To avoid pre-defined positive numbers with a denominator of zero;
[0035] Incremental directional deviation ;
[0036] in, Represents the unit vector of the reference direction. , Indicates the direction of the historical average increment. k is the index variable for the historical inversion step, with values k=1,2,…,t-1;
[0037] Latent space state modulus .
[0038] Furthermore, the trajectory feature supervision sequence ,
[0039] in, This represents the trajectory feature vector at the position corresponding to the t-th inversion step;
[0040] , .
[0041] Furthermore, the predicted trajectory feature sequence is represented as:
[0042] ,in, This represents the trajectory feature vector predicted at the position corresponding to the t-th inversion step, and... One-to-one correspondence.
[0043] Furthermore, the image features fused with the predicted trajectory feature sequence and the training image include:
[0044] The training images are encoded using an image feature extraction module to obtain a sequence of visual feature tokens. ,in, Indicates the number of visual feature tokens. Indicates the feature dimension;
[0045] The high-frequency noise residual R is extracted by applying a residual filter bank to the training image. A two-dimensional discrete Fourier transform is performed on R to obtain the frequency domain response tensor F=|FFT2(R)|, which is then encoded into a frequency domain feature token sequence by a frequency encoder network. ,in, Indicates the number of frequency domain feature tokens. Indicates the feature dimension;
[0046] Predicted trajectory feature sequence Mapped to trajectory feature token sequence via geometric encoder ;
[0047] Using visual feature tokens as queries and trajectory feature tokens and frequency domain feature tokens as keys and values, cross-branch and cross-attention computation is performed to obtain fused feature tokens. ;
[0048] The fused feature tokens are then input into the classifier after global average pooling, and the output is the authenticity prediction result, which is the forgery probability. .
[0049] Furthermore, the classification loss is defined as the binary cross-entropy:
[0050] ;
[0051] in, For classifying losses, Let be the real class label for the i-th training sample, where the label for fake samples is 1 and the label for real samples is 0. Let be the forgery probability output by the classifier for the i-th training sample.
[0052] Furthermore, the trajectory alignment loss is defined as the mean squared error between the predicted trajectory feature sequence and the trajectory feature supervision sequence:
[0053] ;
[0054] in, Let T be the trajectory alignment loss, and T be the number of inversion steps.
[0055] Furthermore, the overall loss function is defined as:
[0056] ;
[0057] When extracting frequency domain features, the overall loss function also includes frequency information loss. and its weighting coefficients The overall loss function becomes:
[0058] ;
[0059] in, These are the weighting coefficients for the trajectory alignment loss;
[0060] Defined as frequency branch auxiliary output Binary cross-entropy with the real label:
[0061] ;
[0062] in, This is the spoofing probability output by the frequency branch auxiliary classification head for the i-th training sample.
[0063] Compared with the prior art, the present invention has the following beneficial effects:
[0064] (1) This invention uses the diffusion inversion process as the offline trajectory feature supervision source during the training period. By performing multi-step diffusion inversion on the training image, a sequence of latent space state tensors arranged in steps is obtained. The change relationship between the latent space state tensors of each step is vectorized and then five types of process trajectory geometric features are calculated to form a trajectory feature supervision sequence, which makes up for the lack of intermediate process information loss in the endpoint residual method.
[0065] (2) The present invention sets up a trajectory feature prediction module, which directly predicts the predicted trajectory feature sequence corresponding to the trajectory feature supervision sequence based on the image features of the image to be detected, and generates a realism detection result based on the predicted trajectory feature sequence and the image features. In the inference stage, there is no need to call the diffusion generation model or perform multi-step diffusion inversion processing, thereby reducing the computational overhead caused by diffusion inversion iteration and intermediate latent space state storage. Attached Figure Description
[0066] Figure 1 This is a schematic diagram of the overall process of the method described in this invention;
[0067] Figure 2 This is a schematic diagram comparing the geometric features of the diffusion inversion trajectories of real samples and fake samples in this invention; Figure 2 (a) is a schematic diagram of the geometric features of the actual sample trajectory; Figure 2 (b) is a schematic diagram of the geometric features of the fake sample trajectory; Figure 2 (c) is a schematic diagram showing the differences in geometric features between the fake sample and the real sample;
[0068] Figure 3 This is a schematic diagram of the internal structure of the geometric encoder in this invention;
[0069] Figure 4 This is a schematic diagram of the internal structure of the frequency encoder in this invention;
[0070] Figure 5 A schematic diagram comparing the frequency domain anomaly feature processing examples of real samples and fake samples in this invention. Detailed Implementation
[0071] The present invention will be further described below with reference to the accompanying drawings. The following embodiments are only used to more clearly illustrate the technical solution of the present invention, and should not be used to limit the scope of protection of the present invention.
[0072] like Figure 1 As shown, this invention provides a face image forgery detection method based on diffusion inversion trajectory geometric supervision, comprising the following steps:
[0073] Step 1, Training Image Acquisition and Preprocessing: Obtain images with real-world class labels. Training images Where H and W represent the height and width of the image, respectively. Face detection, alignment, and cropping are performed on the training images, the resolution is unified to 256×256 pixels, and channel-normalized to the [-1,1] interval to obtain the preprocessed training images. .
[0074] Step 2, diffusion inversion to obtain the latent space state tensor sequence: This involves processing the preprocessed training images... Variational autoencoder encoding is performed to obtain the initial latent space state tensor. ,in This is a variational autoencoder built into the pre-trained diffusion generation model. Based on the pre-trained diffusion generation model (e.g., Stable Diffusion V1.5 or a similarly sized text-based graph diffusion model), a deterministic diffusion implicit model inversion method is used. T inversion steps are executed sequentially with a total inversion step count of T=20, resulting in a sequence of latent space state tensors arranged according to the inversion steps. .
[0075] Specifically, the update formula for the deterministic diffusion implicit model inversion is:
[0076] ;
[0077] in, This represents the cumulative product of the noise scheduling parameters over each inversion step. The noise prediction network in the pre-trained diffusion generation model is denoted by t, which represents the inversion step number. In this embodiment, the inversion process uses empty text prompts, the number of inversion steps is T=20, and the parameters of the diffusion generation model are kept frozen.
[0078] Step 3, Trajectory Feature Supervision Sequence Construction: Vectorize the changes between adjacent latent space state tensors and define the state increment vector:
[0079] ;
[0080] in, This represents the operation of flattening a tensor into a vector along all dimensions. , , These represent the height, width, and number of channels of the latent space state tensor, respectively. Based on state increment vector The five types of trajectory features corresponding to each inversion step are calculated sequentially, including: first-order state difference magnitude, second-order state difference magnitude, increment direction consistency, increment direction deviation, and latent space state magnitude.
[0081] First-order state difference module The first-order state difference modulus is used to characterize the overall drift intensity between adjacent latent space states.
[0082] Second-order state difference module The second-order state difference modulus is used to reflect the acceleration or deceleration changes of the trajectory.
[0083] Incremental directional consistency Incremental direction consistency is used to measure the stability of the inversion direction between adjacent steps.
[0084] Where t≥2, Representing vectors transpose, To avoid a pre-defined positive number with a denominator of zero, (in this embodiment, we take...) When the image maintains a stable directional motion during the inversion process, Approaching 1; when the direction changes significantly, Significant decrease.
[0085] Incremental directional deviation ;
[0086] in, Represents the unit vector of the reference direction. , Indicates the direction of the historical average increment. k is the index variable for the historical inversion step, with values k=1,2,…,t-1.
[0087] Reference direction unit vector It is calculated from the historical average state increment direction prior to the current step. In other implementations, Alternatively, the state increment direction of the previous inversion step or the average state increment direction of all inversion steps can be used.
[0088] Latent space state modulus ;
[0089] Where t=2,3,…,T; the L2 norm is calculated along all dimensions after vectorization.
[0090] Arrange the above five types of procedural geometric quantities in the order of inversion steps to obtain a (T-1)×5 trajectory feature supervision sequence: ;
[0091] in, This represents the trajectory feature vector at the position corresponding to the t-th inversion step;
[0092] , .
[0093] When the number of inversion steps T changes, the above calculation process is re-executed on the new step sequence; if the number of inversion steps is too large, intra-step aggregation can be performed on adjacent steps before calculating various geometric quantities.
[0094] Trajectory feature supervision sequence During the training phase, three methods can be used: pre-computed and stored in local cache, real-time computation during training, or batch online generation during training. This invention adopts the pre-computed and stored method to reduce data loading overhead during training.
[0095] like Figure 2As shown in the figure, "latent variable," "velocity," "acceleration," "velocity cosine," and "projection error" correspond to the latent space state magnitude, first-order state difference magnitude, second-order state difference magnitude, increment direction consistency, and increment direction deviation, respectively. Each feature is visualized after Z-score standardization. Figure 2 (a) and Figure 2 As shown in (b), the trajectory characteristics of both real and fake samples change in stages with the inversion steps of the deterministic diffusion implicit model (DDIM), but the two types of samples differ in the magnitude and directional stability of changes in some steps; for example Figure 2 As shown in (c), the differences are mainly concentrated in the velocity cosine in the early stage of inversion, and the latent variables, velocity, and acceleration characteristics in the later stage of inversion. Therefore, the diffusion inversion trajectory can reflect the process differences between real and fake samples in terms of the intensity of state changes and the direction of motion. Using the geometric features of the trajectory as supervisory information can compensate for the insufficient characterization of intermediate inversion process information when only the endpoint residual is used.
[0096] Step 4, Forward pass of trajectory feature prediction module: feed the training images... Input trajectory feature prediction module to obtain predicted trajectory feature sequence ,in, This represents the trajectory feature vector predicted at the position corresponding to the t-th inversion step, and... One-to-one correspondence.
[0097] The trajectory feature prediction module consists of three parts: an image backbone network, a feature aggregation layer, and a geometric regression head. The image backbone network uses EfficientNet-B4 (in some implementations, ResNet-50, ConvNeXt-T, or a backbone network of similar size can also be used) to encode the input image into a two-dimensional feature map. Here, h' and w' represent the height and width of the two-dimensional feature map, respectively, and d' represents the number of channels in the two-dimensional feature map, i.e., the feature dimension. The feature aggregation layer uses global average pooling to transform the two-dimensional feature map into a global feature vector. The geometric regression head consists of several fully connected layers, used to map global feature vectors to... Dimensional predicted trajectory feature sequence.
[0098] The geometric regression head consists of the following layers connected sequentially: LayerNorm(d'), Linear(d',32), ReLU, Dropout(0.1), Linear(32,128), ReLU, Linear(128,(T-1)·5), and Reshape(T-1,5). The final output... Dimensional predicted trajectory feature sequence .
[0099] Input preprocessed training images Before entering the trajectory feature prediction module, the image is further scaled to 224×224 pixels (matching the EfficientNet-B4 pre-trained weights), and channel normalization uses the mean and variance corresponding to the ImageNet pre-trained weights. The visual feature extraction module of the main detection network uses a 256×256 resolution input, and the two resolutions are processed independently by their respective modules.
[0100] Predicted trajectory feature sequence With trajectory feature supervision sequence Regression supervision is performed using mean squared error loss, and parameter updates are completed through backpropagation of the overall loss function.
[0101] The trajectory feature prediction module also has a regularization effect. The predicted trajectory feature sequence may contain detailed noise caused by the number of inversion steps, the version of the diffusion model, or the preprocessing process; the trajectory feature prediction module recovers the main trajectory trend from the global features of the image, prompting the network to learn more stable geometric patterns, rather than completely memorizing local perturbations in the supervised target.
[0102] The inference phase does not involve any inversion iterations. The trajectory feature prediction of a single image takes about 8ms on the EfficientNet-B4 backbone (including TensorRT acceleration), which is more than an order of magnitude lower than the overhead of the 20-step deterministic diffusion implicit model inversion.
[0103] Step 5, Multi-branch Feature Interaction and Fusion: The detection network adopts a three-branch fusion architecture of visual features, frequency domain features, and predicted trajectory features. It should be noted that the frequency domain feature branch is an optional enhancement module. In other implementations, only visual features and predicted trajectory features can be used for two-branch fusion without affecting the integrity of the basic technical solution.
[0104] (5.1) Visual feature extraction: The training images are encoded using an image feature extraction module (e.g., a convolutional neural network based on EfficientNet-B4 or of equivalent size) to obtain a sequence of visual feature tokens. ,in, Indicates the number of visual feature tokens. Indicates the feature dimension;
[0105] (5.2) Frequency domain feature extraction (optional enhancement module): Apply a residual filter bank to the training image to extract the high-frequency noise residual R, perform a two-dimensional discrete Fourier transform on R to obtain the frequency domain response tensor F=|FFT2(R)|, and encode it into a frequency domain feature token sequence through a frequency encoder network. ,in, Indicates the number of frequency domain feature tokens. Indicates the feature dimension;
[0106] The frequency domain feature extraction module is an optional enhancement module. Its goal is to capture local high-frequency anomalies from the input image to compensate for the shortcomings of trajectory features in the local response of the spatial domain.
[0107] A residual filter bank containing multiple preset high-pass convolution kernels is used to extract high-frequency residuals from the input image in different directions and at different orders. The residual filter bank contains 30 3×3 high-pass convolution kernels.
[0108] For input grayscale image By applying a residual filter bank, multi-channel filtering results are obtained. The absolute values are then concatenated along the channel dimension to form a multi-channel noise feature map. ,in This represents the number of filter channels.
[0109] Perform a two-dimensional discrete Fourier transform on the noise feature map R along the spatial dimension to obtain the frequency domain response tensor. .
[0110] like Figure 4 As shown, the frequency encoder consists of four concatenated convolutional blocks, each containing a 3×3 convolution, batch normalization, and ReLU activation. The four convolutional blocks are progressively scaled up to d (default d=128) in the channel dimension. Adaptive pooling aligns the spatial dimensions to a uniform size, and after Flattening, a linear layer maps the result to a frequency domain feature token sequence. .
[0111] To capture anomalous responses of different frequency components, F can be divided into blocks and pooled according to frequency ranges after FFT2 to obtain three sets of sub-band frequency tokens for low frequency, mid frequency, and high frequency, which are then input into the subsequent fusion module.
[0112] Besides the main branch used for fusion, the intermediate layer features of the frequency encoder are fed into a binary classification head after global average pooling, outputting a frequency auxiliary probability q. This auxiliary head is used for frequency information loss during the training phase. Direct supervision; can be removed during the inference phase to reduce forward overhead.
[0113] like Figure 5 As shown, after the input sample is processed by the residual filter bank, the corresponding SRM residual map and high-frequency filtered residual map can be obtained, and the FFT logarithmic amplitude spectrum can be obtained through two-dimensional discrete Fourier transform. Real samples and fake samples exhibit distinguishable differences in frequency domain response and high-frequency residual information.
[0114] It should be noted that, in order to avoid the example images in the attached figures involving personal identification information, the input examples shown in the attached figures have only been anonymized during the display stage. This anonymization process does not participate in model training, frequency domain anomaly feature extraction, trajectory geometric feature calculation, or authenticity determination.
[0115] (5.3) Trajectory feature encoding: The predicted trajectory feature sequence Mapped to trajectory feature token sequence via geometric encoder .like Figure 3 As shown, the geometric encoder is composed of a layer normalization layer, a first linear layer, a ReLU activation layer and a second linear layer cascaded in sequence. The input feature dimension of the layer normalization layer is 5, the first linear layer maps the feature dimension from 5 to 32, and the second linear layer maps the feature dimension from 32 to 128.
[0116] (5.4) Cross-branch and cross-attention fusion: Using visual feature tokens as queries and trajectory feature tokens and frequency domain feature tokens as keys and values, cross-branch and cross-attention calculations are performed to obtain fused feature tokens. .
[0117] Specifically: visual feature tokens Trajectory feature token Frequency domain feature token (When frequency branching is enabled) the tokens are concatenated into a unified set in a unified d-dimensional space; a multi-head cross-attention mechanism is used for computation: the visual feature tokens are used as query Q, the concatenated unified tokens are used as key K and value V, and the fused visual feature tokens are output after attention computation. Number of attention heads Single-head dimension Scaling factor is The multi-head outputs are spliced together and then mapped back to d-dimensionality via a Linear layer.
[0118] (5.5) Reliability-weighted pooling: Learn the reliability weights for each inversion step. The trajectory feature tokens of each step are weighted and pooled using a lightweight scoring function to obtain a weighted summary vector of trajectory features.
[0119] Not all inversion steps are equally important: early steps may be significantly affected by noise initialization, while later steps may be too close to the inversion endpoint to show process differences. Therefore, this invention learns a reliability weight for each inversion step t. .
[0120] Specifically, the trajectory feature token Input lightweight scoring function (Linear(d,1)) yields the raw scores for each step. , This represents a lightweight scoring function, implemented in this embodiment using Linear(d,1), which is then normalized using Softmax to obtain the reliability weights.
[0121] ;
[0122] Based on reliability weight The trajectory feature tokens of each step are weighted and pooled:
[0123] ;
[0124] Fusion Feature Token After global average pooling, the result is The trajectory features are summarized after reliability pooling. By concatenating the layers along the channel dimension and inputting them into a two-layer fully connected network (Linear(2d,d), ReLU, Linear(d,1), and Sigmoid), the final forgery probability is obtained. .
[0125] For genuine samples, visual texture, frequency response, and trajectory geometry are generally consistent with each other; for fake samples, local textures may appear visually plausible, but the frequency response or diffusion trajectory may reveal anomalies. The fusion module transforms this cross-branch inconsistency into the final classification criterion, which is the key difference between this invention and single-branch or simple splicing methods.
[0126] (5.6) Authenticity determination: The fused feature token is input into the classifier after global average pooling, and the output is the forgery probability. .
[0127] Step 6, Multi-task Joint Training:
[0128] The overall loss function is defined as: ;
[0129] in, The classification loss is defined as the binary cross-entropy:
[0130] ;
[0131] The trajectory alignment loss is defined as the mean squared error between the predicted trajectory features and the supervised trajectory features:
[0132] ;
[0133] These are the weighting coefficients for the trajectory alignment loss. In this embodiment, .
[0134] When further extracting frequency domain features, the overall loss function also includes frequency information loss. and its weighting coefficients The overall loss function becomes . Defined as frequency branch auxiliary output Binary cross-entropy with the real label:
[0135] ;
[0136] The optimizer uses AdamW with an initial learning rate of Weight decay The batch size is N=16, and the training lasts for 30 epochs. The learning rate uses a cosine annealing strategy.
[0137] The following details the training process:
[0138] The training set consists of four classes of fake samples (DeepFakes, Face2Face, FaceSwap, and NeuralTextures) and their corresponding real samples, all compressed at FF++ c40 level. The training subset is selected according to the official classification. Inference is performed on FF++, Celeb-DF, and WildDeepfake respectively.
[0139] The pre-trained diffusion generation model uses Stable Diffusion V1.5 (other diffusion generation models of the same scale can also be used), and the parameters are kept frozen during training.
[0140] The deterministic diffusion implicit model inversion uses empty text prompts, with an inversion step count of T=20, and the diffusion scheduler is DDIMScheduler, η=0 (deterministic inversion).
[0141] The optimizer uses AdamW with an initial learning rate of Weight decay The batch size is N=16, and the training lasts for 30 epochs. The learning rate uses a cosine annealing strategy.
[0142] Classification loss Its weighting coefficient is 1.0; trajectory alignment loss Its weighting coefficient When frequency branching is enabled, frequency information is lost. Its weighting coefficient .
[0143] During training, data augmentation is applied to the training images, including random horizontal flipping, random ±10° rotation, random brightness jitter (±0.1), and random contrast jitter (0.9-1.1); random cropping is not performed to avoid damaging the face region.
[0144] Using FP16 mixed precision training acceleration, the training time for a single A100 GPU is approximately 18 hours.
[0145] The first 5 epochs train only the classification loss and frequency information loss (if enabled), allowing the backbone network to converge to a reasonable true / false discrimination representation; from epochs 6 to 30, trajectory alignment loss is gradually introduced and trained according to a cosine curve. The speed was increased from 0 to 1.0 to avoid introducing geometric supervision too early, which could cause training instability.
[0146] In each epoch, the AUC is calculated on the FF++ validation subset, and the model with the highest AUC on the validation set is retained as the final model.
[0147] Step 7, Inference Stage Processing: In the inference stage, the method described in this invention does not require offline computation of diffusion inversion and trajectory feature supervision sequences. Specifically, the detection device only loads the image feature extraction module, frequency feature extraction module (optional), trajectory feature prediction module, feature fusion module, and authenticity discrimination module. Given a face image to be detected... The inverted trajectory feature sequence is directly predicted by the trajectory feature prediction module. The frequency feature extraction module (optional) directly extracts frequency domain anomaly features. Both and visual features The input feature fusion module processes the data through cross-attention interaction, reliability-weighted pooling, and classifier forward propagation, ultimately outputting the forgery probability. .when If the image is fake, it is considered a fake image; otherwise, it is considered a real image.
[0148] The inference process involves only one forward propagation of the detection network, without diffusion inversion iteration, saving intermediate inversion states, or loading trajectory supervision data. Compared with diffusion reconstruction error-type methods that require 20 steps of deterministic diffusion implicit model inversion, the inference latency of a single image is significantly reduced.
[0149] The method described in this invention can be widely applied to the following scenarios:
[0150] (1) Social media platform facial image content review, automatically identify whether uploaded facial images have been AI face swapped or generated;
[0151] (2) Facial identity verification in financial and government scenarios, detecting facial forgery during remote account opening or facial login;
[0152] (3) Image authenticity verification in judicial evidence collection provides technical basis for the review of electronic evidence;
[0153] (4) Content anti-counterfeiting in the film and advertising industry to protect the portraits of public figures from being misused;
[0154] (5) Face verification for video conferencing and online education platforms to detect face forgery in real-time video streams.
[0155] This invention utilizes diffusion inversion process supervision during the training phase and skips inversion iterations through a prediction module during the inference phase. This allows the method to possess both the procedural discrimination capability for generating priors and the real-time capability for engineering deployment. The online detection system can provide services externally via API, taking a single-frame face image as input and outputting the forgery probability and the authenticity judgment result at the corresponding threshold.
[0156] The above description is only a preferred embodiment of the present invention. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the technical principles of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.
[0157] The above are merely embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention are included within the scope of the claims of the present invention pending approval.
Claims
1. A face image forgery detection method based on diffusion inversion trajectory geometric supervision, characterized in that: Acquire the image of the face to be detected; The face image to be detected is input into a pre-trained forgery detection model. The trajectory feature prediction module in the forgery detection model obtains the predicted trajectory feature sequence and extracts the image features of the face image to be detected. The predicted trajectory feature sequence is interactively fused with the image features of the face image to be detected to obtain fused features; The forgery detection model outputs the authenticity detection result of the face image to be detected based on the fusion features; The training process of the forgery detection model includes: Obtain training images with real-world category labels; The training images are latent space encoded, and a multi-step diffusion inversion process is performed based on a pre-trained diffusion generation model to obtain multiple latent space state tensors arranged according to the inversion steps. The state changes between adjacent latent space state tensors are vectorized into state increment vectors. Based on the numerical relationship between the state increment vectors, trajectory features corresponding to each inversion step are generated, and a trajectory feature supervision sequence is formed according to the order of the inversion steps. The training image is input into the forgery detection model, and the trajectory feature prediction module in the forgery detection model obtains the predicted trajectory feature sequence and extracts the image features of the training image. By fusing the predicted trajectory feature sequence and the image features of the training images, a realistic prediction result is output; The forgery detection model is trained based on the classification loss between the authenticity prediction result and the authenticity category label, and the trajectory alignment loss between the predicted trajectory feature sequence and the trajectory feature supervision sequence.
2. The face image forgery detection method based on diffusion inversion trajectory geometric supervision according to claim 1, characterized in that, The latent space encoding of the training images and the multi-step diffusion inversion process based on the pre-trained diffusion generation model include: Preprocessed training images Variational autoencoder encoding is performed to obtain the initial latent space state tensor. ,in A variational autoencoder is built into the pre-trained diffusion generative model. Based on the pre-trained diffusion generative model, a deterministic diffusion implicit model inversion method is adopted, and T inversion steps are executed sequentially with an inversion step count of T=20 to obtain a sequence of latent space state tensors arranged according to the inversion steps: ; The update formula for the deterministic diffusion implicit model inversion method is: ； in, This represents the cumulative product of the noise scheduling parameters over t inversion steps. t represents the noise prediction network in the pre-trained diffusion generation model, and t is the inversion step number.
3. The face image forgery detection method based on diffusion inversion trajectory geometric supervision according to claim 2, characterized in that, The state increment vector is ; in, This represents the operation of flattening a tensor into a vector along all dimensions. , , These represent the height, width, and number of channels of the latent space state tensor, respectively. .
4. The face image forgery detection method based on diffusion inversion trajectory geometric supervision according to claim 3, characterized in that, The trajectory features corresponding to each inversion step include: first-order state difference modulus, second-order state difference modulus, increment direction consistency, increment direction deviation, and latent space state modulus; First-order state difference module ; Second-order state difference module ; Incremental directional consistency ; Where t≥2, Representing vectors transpose, To avoid pre-defined positive numbers with a denominator of zero; Incremental directional deviation ; in, Represents the unit vector of the reference direction. , Indicates the direction of the historical average increment. k is the index variable for the historical inversion step, with values k=1,2,…,t-1; Latent space state modulus .
5. The face image forgery detection method based on diffusion inversion trajectory geometric supervision according to claim 4, characterized in that, The trajectory feature supervision sequence , in, This represents the trajectory feature vector at the position corresponding to the t-th inversion step; ， 。 6. The face image forgery detection method based on diffusion inversion trajectory geometric supervision according to claim 5, characterized in that, The predicted trajectory feature sequence is represented as follows: ,in, This represents the trajectory feature vector predicted at the position corresponding to the t-th inversion step, and... One-to-one correspondence.
7. The face image forgery detection method based on diffusion inversion trajectory geometric supervision according to claim 5, characterized in that, Image features that fuse the predicted trajectory feature sequence and the training image include: The training images are encoded using an image feature extraction module to obtain a sequence of visual feature tokens. ,in, Indicates the number of visual feature tokens. Indicates the feature dimension; The high-frequency noise residual R is extracted by applying a residual filter bank to the training image. A two-dimensional discrete Fourier transform is performed on R to obtain the frequency domain response tensor F=|FFT2(R)|, which is then encoded into a frequency domain feature token sequence by a frequency encoder network. ,in, Indicates the number of frequency domain feature tokens. Indicates the feature dimension; Predicted trajectory feature sequence Mapped to trajectory feature token sequence via geometric encoder ; Using visual feature tokens as queries and trajectory feature tokens and frequency domain feature tokens as keys and values, cross-branch and cross-attention computation is performed to obtain fused feature tokens. ; The fused feature tokens are then input into the classifier after global average pooling, and the output is the authenticity prediction result, which is the forgery probability. .
8. The face image forgery detection method based on diffusion inversion trajectory geometric supervision according to claim 7, characterized in that, The classification loss is defined as binary cross-entropy: ； in, For classifying losses, Let be the real class label for the i-th training sample, where the label for fake samples is 1 and the label for real samples is 0. This represents the forgery probability output by the forgery detection model for the i-th training sample.
9. The face image forgery detection method based on diffusion inversion trajectory geometric supervision according to claim 8, characterized in that, The trajectory alignment loss is defined as the mean square error between the predicted trajectory feature sequence and the trajectory feature supervision sequence: ； in, Let T be the trajectory alignment loss, and T be the number of inversion steps.
10. The face image forgery detection method based on diffusion inversion trajectory geometric supervision according to claim 9, characterized in that, The overall loss function is defined as: ； in, These are the weighting coefficients for the trajectory alignment loss; When extracting frequency domain features, the overall loss function also includes frequency information loss. and its weighting coefficients The overall loss function becomes: ； Defined as frequency branch auxiliary output Binary cross-entropy with the real label: ； in, This is the spoofing probability output by the frequency branch auxiliary classification head for the i-th training sample.