A gif face image active forensics method based on space-time watermarking

Abstract

This invention discloses a proactive forensic method for GIF face images based on spatiotemporal watermarking, belonging to the fields of image forensics and deep learning technology. This invention constructs a spatiotemporal adaptive residual encoder (STARE) and a deep integrity recovery decoder (DIRD), utilizing 3D convolution and adaptive attention mechanisms to capture the global spatiotemporal dependencies of the GIF, achieving coherent embedding of the watermark signal in the temporal dimension, eliminating visual flicker and enhancing resistance to temporal attacks. Simultaneously, this invention introduces an adversarial training mechanism, forcing the watermark information to be embedded in a deep spatiotemporal region robust to semantic reconstruction. Therefore, while ensuring high-fidelity visual quality, this significantly improves the watermark survival rate and forensic accuracy of GIFs when facing deepfake tampering.

Description

Technical Field

[0001] This invention belongs to the field of image forensics and deep learning technology, and particularly relates to an active forensics method for GIF face images based on spatiotemporal watermarking. Background Technology

[0002] With the widespread adoption of social networks, the Graphical Interchange Format (GIF) has become a fundamental medium for visual communication. GIFs fall between still images and videos, conveying richer dynamic content than still images while maintaining a smaller file size and higher transmission efficiency than videos. In particular, GIFs centered on faces (such as social media emojis and reaction clips) are widely used because they can capture subtle facial expressions and emotional cues.

[0003] However, the widespread use and ease of editing of GIFs, coupled with the lack of effective source tracing and content authentication mechanisms, make facial content within them highly susceptible to malicious tampering. In particular, with the rapid development of deepfake technology, generative attacks targeting GIF facial images pose an unprecedented threat to the authenticity of digital media. Therefore, establishing a proactive evidence-gathering mechanism (such as digital watermarking) capable of verifying the authenticity and source of GIFs has become a critical technical problem that urgently needs to be solved.

[0004] To combat deepfakes and protect copyright, existing technologies are mainly divided into two categories: proactive forensics for static images and robust watermarking for traditional videos / GIFs.

[0005] (1) Spatial domain-based active forensic technology for static images

[0006] Most existing active forensics methods are designed for static images, using deep learning frameworks to embed invisible and robust signals in the spatial domain of the image.

[0007] SepMark (Wu X, Liao X, Ou B. Sepmark: Deep separable watermarking for unified source tracing and deepfake detection[C] / / Proceedings of the 31st ACM International Conference on Multimedia. 2023: 1190-1201.) proposed a deep separable watermarking framework to address the contradiction between "source tracing robustness" and "detection vulnerability" that a single watermark cannot simultaneously achieve. Its implementation involves embedding a single watermark signal in the encoder, but employing a dual-branch structure at the decoder: one branch uses robust features to extract the source ID, while the other branch uses vulnerable features to respond to and locate deepfake regions, thus achieving functional decoupling and unification without increasing the embedding burden.

[0008] The core of FakeTagger (Wang R, Juefei-Xu F, Luo M, et al. Faketagger: Robustsafeguards against deepfake dissemination via provenance tracking[C] / / Proceedings of the 29th ACM international conference on multimedia. 2021:3546-3555.) lies in using Redundant Encoding to combat generative attacks. It employs a GAN-based encoder-decoder architecture, combining channel coding techniques to repeatedly map identity information (ID) to image features. During training, a Deepfake generation simulation layer is introduced as a noise channel, forcing the model to learn a feature representation that remains even after the face is reconstructed, thus achieving passive recovery of the victim's identity.

[0009] WaveGuard (He, Y. et al. "WaveGuard: Wavelet-based DeepfakeDetection," in CVPR, 2025.) abandons traditional pixel-domain embedding and proposes a frequency-domain defense mechanism based on complex wavelet transform. It decomposes the image into multi-scale, multi-directional frequency sub-bands and leverages the phase stability and direction selectivity of complex wavelets to embed watermark information into specific frequency bands that are sensitive to generative artifacts but visually insignificant, thereby enhancing the robustness of the watermark structure against deepfake texture reconstruction.

[0010] (2) Time-series watermarking technology for videos or GIFs To address the issue of temporal consistency, traditional video watermarking techniques have introduced temporal modeling in an attempt to maintain the stability of the watermark across consecutive frames.

[0011] GIFMarking (Liao X, Peng J, Cao Y. GIFMarking: The robust watermarking for animated GIF based deep learning[J]. Journal of Visual Communication and Image Representation, 2021, 79: 103244.) is a pioneering work in the field of GIF. This scheme constructs an end-to-end watermarking framework based on a three-dimensional convolutional neural network (3D-CNN). It treats GIF as a spatiotemporal three-dimensional tensor for overall feature extraction and specifically designs a color quantization layer and a frame rate adjustment simulation layer in the training loop, forcing the model to adapt to the 256-color palette limitation unique to GIF and common frame deletion attacks when embedding watermarks.

[0012] DVMark (Luo X, Li Y, Chang H, et al. Dvmark: a deep multiscale framework for video watermarking[J]. IEEE Transactions on Image Processing, 2023.) proposes a video watermarking strategy based on multiscale feature fusion. It constructs a multi-level 3D convolutional pyramid structure to extract features at different spatiotemporal resolution scales of the video and embed the watermark signal. By leveraging the complementarity of multi-scale features, it resists the block artifacts and lossy compression distortion introduced by video coding standards such as H.264 / HEVC, thereby improving the survivability of the watermark in complex video channels.

[0013] Analysis of the above reveals that existing technologies, when processing animated GIF images, suffer from a lack of consistency in the watermark signal over time, leading to decreased visual quality and vulnerability to temporal attacks. When existing proactive forensics solutions for static images are applied to GIFs, the resulting watermarked GIFs often exhibit flickering or jittering (visual instability). Furthermore, because each frame's watermark is independent, attackers can easily compromise the integrity of the forensics system by simply scrambling, deleting, or replacing frames. The root cause of this problem lies in the fact that existing technologies often employ a "frame-independent" processing paradigm, primarily designed for single static images. They treat the GIF sequence merely as a discrete set of images, ignoring the inherent spatiotemporal correlation between GIF frames. This embedding strategy, lacking global temporal modeling, cannot guarantee the continuity of the watermark signal across consecutive frames.

[0014] Current technologies suffer from low watermark survival rates when facing semantic-level tampering attacks such as deepfakes, making effective tracing and evidence collection impossible. When a face in a GIF is reconstructed by a deepfake algorithm, the originally embedded watermark signal is easily "washed away," resulting in the extraction end being unable to detect the watermark or experiencing a very high error rate, thus rendering the evidence collection mechanism ineffective. The root cause of this problem lies in the fact that existing watermark embedding strategies rely on fragile shallow features or high-frequency components. Traditional robust watermarking techniques typically hide information in the high-frequency texture or pixel-domain redundancy of the image, while deepfake technology involves generative semantic reconstruction of the face. During the reconstruction process, non-semantic high-frequency details (i.e., where the watermark is hidden) are automatically filtered or rewritten by the generator because existing embedding methods fail to couple the watermark to semantically robust deep spatiotemporal features.

[0015] To address the aforementioned issues, this invention proposes an active evidence collection method for GIF face images based on spatiotemporal watermarking. Summary of the Invention

[0016] The purpose of this invention is to propose an active forensic method for GIF face images based on spatiotemporal watermarking to solve the technical problems of existing technologies lacking spatiotemporal consistency and being unable to effectively resist deepfake attacks when processing dynamic GIFs.

[0017] To achieve the above objectives, the technical solution adopted by the present invention is as follows: A method for proactive evidence collection from GIF face images based on spatiotemporal watermarking includes the following steps: S1. Data preprocessing: Standardize the GIF format preprocessing of the source video data containing faces, including color quantization and jitter processing, face detection and sequence extraction, and identity labeling, to obtain the original watermark message to be embedded. S2, Watermark Embedding: The obtained original watermark message along with the original GIF face image sequence is input into the spatiotemporal adaptive residual encoder. The encoder performs spatiotemporal information diffusion, 3D convolutional backbone network and hierarchical spatiotemporal residual generation, and outputs a residual map superimposed on the original GIF to generate a GIF sequence with watermark. S3. Watermark Extraction and Authentication: The obtained watermarked GIF sequence is input into the deep integrity restoration decoder. The decoder performs attention feature denoising, spatiotemporal integrity reconstruction, and adaptive global projection on the input sequence, and finally outputs the restored watermark bitstream. The authenticity of the content and the source are authenticated by comparing the watermark bitstream with the original watermark message.

[0018] Preferably, S1 specifically includes the following: S1.1 Color Quantization and Dithering: The Floyd-Steinberg dithering algorithm is used to compress continuous tone video frames to a 256-color palette, and LZW lossless compression encoding is used to preserve the inherent discrete index and temporal dithering characteristics of GIF. S1.2 Face Detection and Sequence Extraction: Perform frame-by-frame face detection on the original video data to remove invalid data with poor quality or face occlusion; extract several frames of images at equal intervals from each video sequence and uniformly crop them to H×W resolution to construct a GIF face image sequence in tensor form; S1.3 Identification Labeling: Assign a unique identifier to each GIF face image sequence, and convert the identifier into a binary bit stream as a watermark message.

[0019] Preferably, S2 specifically includes the following: S2.1 Spatiotemporal Information Diffusion: The binary watermark vector is mapped to a high-dimensional feature space through a fully connected layer. After dimensional reshaping and normalization, it is broadcast in the three dimensions of time, height and width using trilinear interpolation technology to generate a watermark feature tensor with the same resolution as the carrier GIF. S2.2, 3D attention-guided feature fusion: The feature map of the carrier GIF is spliced ​​with the watermark feature tensor obtained in S2.1 in channel dimension 1 to form a joint feature; the joint feature is then subjected to channel attention weight calculation and feature recalibration through the 3D extrusion module to obtain the enhanced feature; S2.3, Layered Spatiotemporal Residual Generation: The enhanced features obtained in S2.2 are input into a layered spatiotemporal refinement network based on a four-level depth 3D U-Net architecture. Downsampling and feature extraction are performed through a spatiotemporal contraction path, and upsampling and feature recovery are performed through a spatiotemporal expansion path combined with long-range skip connections. At the end of the layered spatiotemporal refinement network, a residual map constrained in the range of (-1,1) is generated through convolutional layers and Tanh activation functions. The residual map is multiplied by a scaling factor and then superimposed onto the original carrier GIF to obtain a watermarked sequence.

[0020] Preferably, S3 specifically includes the following: S3.1 Attention Feature Denoising: The spatiotemporal resolution of the input sequence is gradually reduced by a shrinking path containing 3D convolution, and a 3D squeezing excitation module is integrated at each level to perform channel-level feature denoising and enhancement. S3.2 Spatiotemporal integrity reconstruction: The spatiotemporal resolution of features is gradually restored through an expansion path that includes 3D transposed convolution, and deep semantic information is used to guide and repair feature breaks caused by temporal or spatial distortion. S3.3 Adaptive Global Projection: The reconstructed high-dimensional features are flattened and projected into Hamming space through a fully connected projection head that dynamically generates a mapping matrix based on the input feature dimensions. The predicted watermark bitstream is then output via the Sigmoid function.

[0021] Preferably, the spatiotemporal adaptive residual encoder and the depth integrity recovery decoder are jointly optimized through an adversarial training framework that includes a realistic distortion simulator; The realistic distortion simulator is connected during the training phase to apply differentiable distortion simulation to the watermarked sequence. The distortion simulation includes: deepfake generation simulation, geometric distortion simulation, and signal-level distortion simulation. The total loss function for the adversarial training is:

[0022] in, L imp To constrain the loss of visual imperceptibility; L msg To limit the loss of accuracy in watermark message recovery; L adv_g To combat the losses.

[0023] Preferably, the adversarial training adopts an adaptive course learning strategy based on a realistic distortion simulator, dynamically adjusting the trigger probability of each distortion operator through a probability ramping mechanism; slight distortion is applied in the early stage of training, and the sampling probability of random pruning and deepfake simulation is gradually increased as the training rounds increase.

[0024] The present invention further protects a computer device, the computer device including a processor and a memory, wherein the memory stores at least one instruction, at least one program, code set or instruction set, the instruction, program, code set or instruction set being loaded and executed by the processor to implement the above-mentioned method for proactive evidence collection of GIF face images based on spatiotemporal watermarking.

[0025] The present invention further protects a computer-readable storage medium storing at least one instruction, at least one program, code set, or instruction set, wherein the instruction, program, code set, or instruction set is loaded and executed by a processor to implement the above-mentioned method for proactive evidence collection of GIF face images based on spatiotemporal watermarking.

[0026] Compared with the prior art, the beneficial effects of the present invention are as follows: (1) Significantly improved defense capabilities and robustness against deepfake semantic reconstruction attacks: This invention employs an adversarial training mechanism integrating a Realistic Distortion Simulator (RDS). Unlike existing techniques that simulate attacks simply by adding Gaussian noise or compression, this invention directly integrates a real depth forgery generation model (such as SimSwap) with frozen parameters into the end-to-end training loop. This technique constructs a highly challenging adversarial environment, forcing the encoder to abandon hiding information in easily reconstructable shallow pixels or high-frequency textures, and instead automatically learn to embed the watermark into deep spatiotemporal features that are invariant to semantic reconstruction. This ensures that the watermark remains intact even if the face is "face-swapped" or reconstructed.

[0027] (2) It achieves spatiotemporal consistency of watermark signals in GIF dynamic image sequences, effectively eliminating visual flicker and improving concealment: This invention employs a global spatiotemporal coding strategy (STARE) based on 3D convolution and attention mechanisms. Unlike existing technologies that treat GIFs as discrete frames and embed them frame by frame (leading to independent and abrupt watermarks between frames), this invention utilizes trilinear interpolation technology to holographically broadcast the discrete watermark bitstream in both temporal and spatial dimensions, and combines this with a 3D excitation squeeze module (3DSE-Block) to capture the dynamic correlation between frames. This approach ensures that the watermark energy transitions smoothly and continuously between consecutive frames, fundamentally avoiding visual jitter caused by differences in watermarks between frames.

[0028] (3) Possesses the ability to self-repair and extract watermarks under severe temporal distortions (such as dropped frames and out-of-order frames) and geometric cropping: This invention employs a Deep Integrity Recovery Decoder (DIRD) with "reverse repair" functionality. Unlike existing decoders that directly classify features from incomplete images, this invention designs a symmetrical "spatiotemporal hourglass" architecture and introduces a spatiotemporal feature reconstruction module at the decoding front end. This technique utilizes the contextual information of undamaged frames as "anchor points" to complete and repair the spatiotemporal structures lost due to frame drops or cropping in the feature space through 3D transposed convolution, thereby ensuring accurate extraction of watermark information even when the input data is incomplete. Attached Figure Description

[0029] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the accompanying drawings involved in the embodiments are now briefly described. Obviously, the drawings in the following description are merely illustrative of some embodiments of the present invention. For those skilled in the art, other forms of drawings can be constructed based on these drawings without creative effort.

[0030] Figure 1 This is the overall network structure diagram of the GIF face image active evidence collection method based on spatiotemporal watermarking proposed in this invention. Detailed Implementation

[0031] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0032] This invention proposes a proactive forensic method for GIF face images based on spatiotemporal watermarking, primarily addressing the technical problem that existing proactive forensic techniques struggle to effectively protect GIF animated images from deepfake tampering. Specifically, this manifests in two aspects: firstly, existing technologies lack adaptability to the temporal characteristics of GIFs (spatiotemporal inconsistency); secondly, traditional watermarks are ill-suited to resist semantic-level generation corruption (poor resistance to deepfakes). Based on these problems, this invention proposes: 1. A Global Spatiotemporal Watermarking Embedding Strategy Based on 3D Convolution and Attention Mechanisms Most existing GIF or video watermarking schemes follow the processing logic of static images, adopting a "frame-independent embedding" approach. The technical means usually involves adding the watermark signal independently in the two-dimensional space (2D Spatial Domain) or transform domain (such as DCT / DWT) of each frame, without the frames being aware of each other.

[0033] This invention employs a spatiotemporal adaptive residual encoder (STARE), treating the GIF as a unified five-dimensional tensor (Batch, Time, Channel, Height, Width). Technically, through 3D convolution and a 3D excitation squeeze module (3D SE-Block), the discrete watermark bitstream is transformed into a feature volume of the same dimension as the carrier through "trilinear broadcasting" and "fully connected mapping," and global pooling is used during embedding to perceive the dynamic changes across the entire timeline.

[0034] In contrast, existing technologies sever the connection of the time dimension, resulting in uneven distribution of watermarks on the timeline; while this invention realizes the joint diffusion of watermark energy in the spatiotemporal dimension, ensuring the temporal consistency of the watermark from the algorithmic level, fundamentally eliminating visual flicker and enhancing the ability to resist temporal editing.

[0035] 2. Adversarial semantic defense mechanism based on differentiable deepfake simulator Existing robust watermarking techniques primarily defend against signal-level distortion. Their training methods typically involve adding Gaussian noise, JPEG compression, or random cropping layers to the network. This approach can only protect high-frequency textures or pixel redundancy; once a face undergoes semantic-level reconstruction via deepfake (i.e., the pixels are completely rewritten by the generation model), the watermark information is erased.

[0036] This invention constructs a Realistic Distortion Simulator (RDS) and embeds it into an end-to-end training loop. Technically, it utilizes a "fixed proxy strategy" to directly connect a real face-swapping model with frozen parameters (such as SimSwap) as a differentiable distortion operator between the encoder and decoder. This forces the encoder, guided by backpropagation gradients, to automatically find deep semantic feature regions that remain unchanged during deepfake reconstruction and embed watermarks therein.

[0037] In comparison, existing technologies passively resist noise interference, while this invention actively simulates the attacker's generation logic; existing technologies protect shallow pixel textures, while this invention protects deep semantic topology structures, thus achieving specific defense against AI generative attacks.

[0038] 3. Integrity recovery decoding technology based on spatiotemporal hourglass architecture Existing decoders typically employ simple downsampling classifiers, which directly perform convolutional dimensionality reduction on the received image and output the prediction result. This approach assumes that the input signal is structurally complete. Once the GIF suffers from dropped frames, out-of-order sequences, or partial tampering, the network fails to extract features because it cannot align them.

[0039] This invention designs a Deep Integrity Recovery Decoder (DIRD) that employs a symmetrical "Spatiotemporal Hourglass" architecture. Technically, before extracting the watermark bits, inverse feature repair is performed using 3D Transpose Convolution and long-range skip connections, utilizing the contextual information of undamaged frames to fill the feature gaps in damaged frames.

[0040] In contrast, existing technologies "read directly" and are powerless against incomplete input; this invention "repairs first and then reads," which has the ability to reconstruct spatiotemporal integrity in the feature space and significantly improves the watermark detection rate under severe temporal attacks.

[0041] 4. Adaptive global projection mechanism based on dynamic graph construction Existing decoding networks typically use fixed-dimensional fully connected layers at the end, which limits the network to processing input images of a specific resolution and lacks flexibility.

[0042] This invention designs an adaptive global projection head. Technically, it employs a dynamic graph construction strategy, deriving the topology of the feature manifold through a virtual forward propagation (Dummy Pass) before inference, and dynamically generating the mapping matrix.

[0043] In contrast, existing technologies require retraining or forced scaling for inputs of different sizes; this invention achieves resolution-agnostic decoding, which can flexibly handle GIF inputs of any size and duration without retraining.

[0044] In summary, this invention constructs a spatiotemporally adaptive residual encoder (STARE) and a depth integrity recovery decoder (DIRD), utilizing 3D convolution and adaptive attention mechanisms to capture the global spatiotemporal dependencies of GIFs, achieving coherent embedding of watermark signals in the temporal dimension, eliminating visual flicker, and enhancing resistance to temporal attacks. Simultaneously, this invention introduces an adversarial training mechanism, forcing watermark information to be embedded in a deep spatiotemporal region robust to semantic reconstruction. This significantly improves the watermark survival rate and forensic accuracy of GIFs when facing deepfake tampering, while maintaining high-fidelity visual quality.

[0045] The following description, in conjunction with the accompanying drawings and relevant examples, further illustrates the proactive evidence collection method for GIF face images based on spatiotemporal watermarking proposed in this invention.

[0046] Example 1: Please see Figure 1 This invention proposes an active evidence collection method for GIF face images based on spatiotemporal watermarking, which mainly includes three core subsystems: a watermark embedding method (encoding end), a realistic distortion simulator (training end), and a watermark extraction and authentication method (decoding end). Watermarking embedding method: A spatiotemporal adaptive residual encoder (STAR) is used to covertly embed the identity tracing message (Message, M) into the original GIF image sequence (Cover GIF, G) to generate a watermarked GIF sequence (Watermarked GIF, Gw).

[0047] Realistic Distortion Simulator: Also known as the Realistic Distortion Simulator (RDS), it is only accessed during the model training phase. As a differentiable proxy module, it simulates the deepfake manipulation, temporal distortion, and lossy signal compression that GIFs may suffer during their spread on social networks, and is used for adversarial training of the codec.

[0048] Watermark extraction and authentication method: The Deep Integrity Restoration Decoder (DIRD) is used to extract the hidden watermark signal from the GIF sequence to be detected and compare it with the original message to verify the authenticity and source of the content.

[0049] The active evidence collection method for GIF face images based on spatiotemporal watermarking proposed in this invention specifically includes the following: 1. Data Preprocessing and Benchmark Building (GIF-Face Data Processing Workflow) Before training the model, the source video data needs to be preprocessed into a standardized GIF format. The specific steps are as follows: 1.1 Color Quantization and Dithering: To simulate the color characteristics of a real GIF, continuous-tone video frames are compressed to a 256-tone color palette. During this process, the Floyd-Steinberg dithering algorithm is introduced to eliminate color banding, and LZW lossless compression encoding is used to preserve the inherent discrete indexing and temporal dithering characteristics of the GIF.

[0050] 1.2 Face Detection and Sequence Extraction: Perform frame-by-frame face detection on the original video, removing invalid data that is of poor quality or contains occluded faces. Extract faces at equal intervals from each video sequence. T Frame (e.g., 10 frames) of image, uniformly cropped to H× W (e.g. 256) 256) resolution, constructed as a tensor ( T , H , W ,3).

[0051] 1.3 Identity Labeling: Assign a unique identity ID to each GIF sample, and convert the ID into a binary bit stream as the watermark message M.

[0052] 2. Detailed Implementation of Watermark Embedding Process (STARE Encoder): The specific execution flow of the spatiotemporal adaptive residual encoder is as follows... Figure 1 As shown, it includes the following three key technical steps: 2.1 Spatiotemporal Message Diffusion Due to the original watermark message... It is a one-dimensional discrete bit stream (e.g.) L (Note: The last part about "2-bit" and "4D" seems unrelated and likely refers to a different implementation. It has been omitted from the translation.) Fully Connected Mapping: The binary watermark vector M is input into the fully connected layer, which maps it to a high-dimensional latent feature space, and non-linear features are introduced using the LeakyReLU activation function.

[0053] Reshaping and Normalization: The mapped feature vectors are reshaped and 3D batch normalization is performed to normalize the feature distribution and accelerate model convergence.

[0054] Trilinear Broadcasting: Utilizing trilinear upsampling technology, features are broadcast along three dimensions: time, height, and width, generating a watermark feature tensor M with the same resolution as the carrier GIF. feat This step ensures that the watermark energy is redundantly distributed across every frame and every local area of ​​the GIF, thus providing redundancy to resist subsequent frame deletion and random cropping attacks.

[0055] 2.2 3D Attention-Guided Feature Fusion To accurately inject watermark features into imperceptible and robust regions of the carrier, this embodiment introduces a 3D extrusion excitation module (3D SE-Block). The specific steps are as follows: Feature stitching: Combining the feature map of the carrier GIF with the watermark feature tensor M feat Concatenation is performed along the channel dimension to form a joint feature X.

[0056] Global compression (Squeeze): Performs 3D global average pooling on the joint feature X to compress the dynamic context information across the entire GIF time period, calculating the first... c Global statistics for each channel z c .

[0057] Adaptive excitation: This involves applying statistics... z c The input contains a gating mechanism with two fully connected layers, and channel attention weights s are generated by the Sigmoid function.

[0058] Feature recalibration: Multiply the weights s back into the original feature X to achieve... This mechanism can automatically suppress visually sensitive smooth areas (such as facial skin) and significantly enhance the response of channels containing rich texture or violent motion information, thereby improving robustness against deepfake tampering.

[0059] 2.3 After feature fusion in Hierarchical Spatiotemporal Residual Learning, the feature is fed into a hierarchical spatiotemporal refinement backbone network, which adopts a four-level deep 3D U-Net architecture: Spatiotemporal contraction path (downsampling): Contains four consecutive encoding modules. A double convolutional structure is used in the Conv2 to Conv4 stages: the first convolutional layer uses a stride of 2 to gradually reduce the spatiotemporal resolution and build a feature pyramid; the second convolutional layer further extracts deep semantics.

[0060] Spatiotemporal dilation path (upsampling): The spatiotemporal dimension of the feature map is gradually restored through trilinear interpolation. Long-range skip connections are introduced in this process to establish an "information highway" from the shallow layers of the encoder to the deep layers of the decoder. This not only solves the gradient vanishing problem, but more importantly, it directly transmits the original high-frequency micro-texture and edge information, undamaged by depth convolution, to the restoration layer, providing an original structural reference for adversarial generative artifacts.

[0061] Residual Prediction and Stacking: A refined projection layer is placed at the output. First, a 3×3×3 convolutional layer is used for smoothing, followed by a 1×1×1 convolutional layer to map the high-dimensional features back to the 3-channel residual space. The Tanh activation function is used to strictly constrain the residual values ​​to the range (-1, 1), and then multiply by a scaling factor. (e.g., 0.2).

[0062] Finally, the generated residual image is overlaid back onto the original carrier GIF to obtain the watermarked GIF sequence. .

[0063] 3. Realistic Distortion Simulation Process (RDS Simulator) The RDS simulator constructed in this invention integrates multiple differentiable operators and applies an attack on \mathbf{G}_{w} during training, forcing the encoder to learn robust features. For example... Figure 1 As shown, it contains the following three branches: Deepfake Simulation branch: Directly integrates face-swapping models such as SimSwap, MobileFaceSwap, and Ghost as distortion operators into the training loop. A "fixed surrogate strategy" is adopted, which freezes the parameters of these pre-trained models and uses only the texture reconstruction distortion generated by their forward propagation to calculate the gradient, thereby prompting the watermark to be embedded in the reconstructive feature space.

[0064] Geometric Simulation Branch: Timing attack: Introducing random frame drop and frame shuffle operators to simulate the timeline manipulation commonly seen in GIF propagation.

[0065] Spatial attack: Random cropping is introduced to simulate boundary loss caused by secondary composition. This forces the model to holographically distribute the watermark energy in a local spatiotemporal region.

[0066] Signal Simulation: Addressing the unique issue of non-differentiable color quantization in GIFs, a composite approximation strategy is employed. Specifically, a differentiable JPEG layer is constructed using block-wise Discrete Cosine Transform (DCT) combined with soft-rounding approximations. This layer simulates high-frequency information loss and block artifacts while maintaining the integrity of the gradient chain, enabling end-to-end adversarial training.

[0067] 4. Detailed Implementation of Watermark Extraction and Recovery Process (DIRD Decoder) The Deep Integrity Recovery Decoder (DIRD / DARE) adopts a symmetrical "spatiotemporal hourglass" architecture: 4.1 Attentive Feature Denoising: The decoder's input is the attacked GIF video V. atk It is often filled with Deepfake artifacts and noise.

[0068] In the decoder's contraction path, 3D convolution is used to progressively reduce the spatiotemporal resolution. A 3D excitation module (3D SE-Block) is integrated at each level. Here, the SE-Block acts as a "soft threshold denoiser," automatically reducing the weights of high-frequency noise channels by sensing the global noise distribution, while enhancing the weights of feature channels containing the watermark signal.

[0069] 4.2 Spatiotemporal Integrity Reconstruction addresses common temporal misalignment issues in GIFs: The spatiotemporal resolution of features is gradually restored using 3D transpose convolution. Guided by deep semantic information, discrete and incomplete local features are reconnected to repair feature breaks caused by frame drops or out-of-order sequences, thus restoring the complete spatiotemporal structure of the watermark signal in the feature space.

[0070] 4.3 Adaptive Global Projection Head: To enable the decoder to handle GIF inputs of arbitrary resolution and duration: Dynamic graph construction: During the network initialization phase or before inference, the topology of the current input feature manifold is automatically derived through a virtual forward propagation (Dummy Pass).

[0071] Dynamic matrix generation: Dynamically constructing a mapping matrix based on feature dimensions Instead of using a fixed-dimensional fully connected layer.

[0072] Bit prediction: The reconstructed five-dimensional feature volume is flattened into feature vectors, projected onto Hamming space through a mapping matrix, and finally output as the predicted watermark bitstream through a Sigmoid activation function. .

[0073] 5. Model Training and Loss Function Design This invention employs a multi-task joint optimization strategy, and the training process is modeled as an encoder. E decoder D With discriminator A The game between them.

[0074] Total loss function The definition is as follows:

[0075] The specific definitions of each loss are as follows: Unperceptible loss ( It consists of two parts: pixel consistency loss and perceptual similarity loss.

[0076]

[0077] Pixel consistency loss ( L mse ): Calculate G and G w The mean square error between them constrains the restoration of low-frequency content.

[0078] Perceptual similarity loss ( L lpips ): Utilize a pre-trained AlexNet to extract deep features and calculate the Euclidean distance in the feature space to preserve texture details.

[0079] Robust information recovery loss ( L msg ): Minimize the decoder's predicted bits using binary cross-entropy loss (BCE) The difference between this loss and the real watermark M. The gradient flow of this loss is backpropagated through the Deepfake model in RDS, directly optimizing the encoder's anti-forgery capabilities.

[0080]

[0081] Combating loss ( L adv ): Introducing a 3D discriminator based on Spectral Normalization. The generator attempts to generate samples that can fool the discriminator, while the discriminator attempts to distinguish between the original GIF and the watermarked GIF, thereby improving the visual realism of the watermarked GIF.

[0082]

[0083] Dynamic Weight Strategy: For training, an RDS-based adaptive curriculum learning approach is employed. Specifically, a probability climbing mechanism dynamically adjusts the trigger probability of distortion operators. In the early stages of training, RDS applies distortion with low probability or only slight distortion to establish a baseline for the model's restoration accuracy. As training epochs increase, the sampling probability of random cropping and deepfake simulators is gradually increased. This adversarial path, from easy to difficult, forces the model to progressively transform watermark energy into a topologically invariant representation that is insensitive to semantic changes.

[0084] Through the above embodiments, the present invention can effectively embed a spatiotemporally consistent watermark into a dynamic GIF image sequence and maintain an extremely low watermark error rate when facing deepfake reconstruction attacks, thus achieving proactive evidence collection and copyright protection for GIFs.

[0085] Example 2: Based on Embodiment 1, but with a difference: to verify the effectiveness of the proposed spatiotemporal watermarking-based active forensics method for GIF face image sequences, this invention underwent extensive testing on the constructed GIF-Face benchmark dataset and was compared with representative solutions in the prior art. Prior art solutions include: GIFMarking, a steganography scheme designed for GIF format (as comparison file 1), and REVMark, a robust watermarking scheme designed for video format (as comparison file 2). Specifically, they include the following: 7.1 Verification of Visual Imperceptibility (High Fidelity) First, the visual quality of the watermarked GIFs was verified. Peak signal-to-noise ratio (PSNR), structural similarity (SSIM), and perceptual patch similarity (LPIPS) were used as evaluation metrics. The test results are shown in Table 1.

[0086] Table 1 Visual Quality Comparison Data Table

[0087] Results Analysis: As shown in Table 1, the watermarked GIF generated by this invention achieves a PSNR of 49.54 dB, significantly outperforming existing technologies. The extremely low LPIPS (0.0035) indicates that the watermark embedding process of this invention has almost no impact on human vision, achieving extremely high visual concealment.

[0088] 7.2 Robustness Verification Against Deepfake Attacks The bit error rate (BER) of this invention is compared with that of comparison file 2 (REVMark) under attacks from three mainstream face-swapping models: Ghost, MobileFaceSwap, and SimSwap. Simultaneously, to verify the vulnerability of traditional image-based watermarking to semantic reconstruction, comparison file 1 (GIFMarking) was retrained and its structural similarity (SSIM) was tested. RivaGAN Table 2 Robustness Comparison under Deepfake Attacks (Bit Error Rate BER)

[0089] Table 3. Failure Verification of Existing Image-Level Watermarks under Deepfakes

[0090] Results Analysis: Table 2 shows that existing video watermarking technologies (comparison file 2) suffer from a bit error rate exceeding 12% when facing deepfake attacks, essentially becoming ineffective. In contrast, this invention, through adversarial training and deep semantic embedding strategies, reduces the average bit error rate to 0.0106%, achieving near-complete immunity to face-swapping tampering. Table 3 shows that after deepfake reconstruction, the traditional image-domain-based watermark (comparison file 1) exhibits a SSIM of 0.52 for the extracted watermark image, with a visual information fidelity (VIF) approaching zero, demonstrating that traditional methods cannot resist semantic-level reconstruction attacks.

[0091] 7.3 Verification against timing attacks and conventional signal distortion The error rate performance of this invention was tested against common GIF attacks such as frame deletion, random cropping, and compression. The results are shown in Table 4.

[0092] Table 4 Comparison of Bit Error Rates under Complex Spatiotemporal Attacks

[0093] Results Analysis: Existing technologies (Comparative Document 2) perform poorly against attacks that disrupt temporal structure (such as frame deletion). In contrast, this invention, benefiting from 3D convolution and a spatiotemporal attention mechanism (SE-Block), can utilize inter-frame redundancy to recover information, achieving a bit error rate of only 0.0167% under frame deletion attacks, demonstrating the superiority of this invention in protecting the temporal sequence of dynamic GIFs.

[0094] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.

Claims

1. A method for proactive evidence collection from GIF face images based on spatiotemporal watermarking, characterized in that, Includes the following steps: S1. Data preprocessing: Perform standardized GIF format preprocessing on the source video data containing faces to obtain the original watermark message to be embedded and the source GIF face image sequence. S2, Watermark Embedding: The obtained original watermark message, together with the original GIF face image sequence, is input into the spatiotemporal adaptive residual encoder. The encoder adopts a three-dimensional convolutional backbone network and captures the global spatiotemporal dependency of the GIF through an adaptive channel recalibration mechanism to generate a GIF sequence containing watermark. S3. Watermark Extraction and Authentication: The obtained watermarked GIF sequence is input into the deep integrity restoration decoder. The decoder adopts a spatiotemporal hourglass architecture that combines a three-dimensional attention mechanism to perform latent feature recovery and spatiotemporal integrity reconstruction on the input sequence, and finally outputs the restored watermark bitstream. The authenticity of the content and the source are authenticated by comparing the watermark bitstream with the original watermark message. 2.S4 Model Training Steps: The STARE encoder and DIRD decoder are trained end-to-end adversarially using a joint loss function, which includes visual quality loss, watermark extraction loss, and adversarial robustness loss against deepfake operations.

3. The method for proactive evidence collection from GIF face images based on spatiotemporal watermarking according to claim 1, characterized in that, S1 specifically includes the following: S1.1 Color Quantization and Dithering: The Floyd-Steinberg dithering algorithm is used to compress continuous tone video frames to a 256-color palette, and LZW lossless compression encoding is used to preserve the inherent discrete index and temporal dithering characteristics of GIF. S1.2 Face Detection and Sequence Extraction: Perform frame-by-frame face detection on the original video data to remove invalid data with poor quality or face occlusion; extract several frames of images at equal intervals from each video sequence and uniformly crop them to H×W resolution to construct a GIF face image sequence in tensor form; S1.3 Identification Labeling: Assign a unique identifier to each GIF face image sequence, and convert the identifier into a binary bit stream as a watermark message.

4. The method for proactive evidence collection from GIF face images based on spatiotemporal watermarking according to claim 1, characterized in that, S2 specifically includes the following: S2.1 Spatiotemporal Information Diffusion: The binary watermark vector is mapped to a high-dimensional feature space through a fully connected layer. After dimensional reshaping and normalization, it is broadcast in the three dimensions of time, height and width using trilinear interpolation technology to generate a watermark feature tensor with the same resolution as the carrier GIF. S2.2, 3D Convolutional Backbone Network: The feature map of the carrier GIF is concatenated with the watermark feature tensor obtained in S2.1 in the channel dimension 1 to form a joint feature; The enhanced features are obtained by calculating channel attention weights and recalibrating features of the joint features through a three-dimensional extrusion excitation module. S2.3, Layered Spatiotemporal Residual Generation: The enhanced features obtained in S2.2 are input into a layered spatiotemporal refinement network based on a four-level depth 3D U-Net architecture. Downsampling and feature extraction are performed through a spatiotemporal contraction path, and upsampling and feature recovery are performed through a spatiotemporal expansion path combined with long-range skip connections. At the end of the layered spatiotemporal refinement network, a residual map constrained in the range of (-1,1) is generated through convolutional layers and Tanh activation functions. The residual map is multiplied by a scaling factor and then superimposed onto the original carrier GIF to obtain a watermarked sequence.

5. The method for proactive evidence collection from GIF face images based on spatiotemporal watermarking according to claim 1, characterized in that, S3 specifically includes the following: S3.1, Attention Feature Denoising: The spatiotemporal resolution of the input sequence is gradually reduced by a shrinking path that includes 3D convolution, and a 3D squeezing excitation module is integrated at each level to perform channel-level feature denoising and enhancement. S3.2 Spatiotemporal integrity reconstruction: The spatiotemporal resolution of features is gradually restored through an expansion path that includes 3D transposed convolution, and deep semantic information is used to guide and repair feature breaks caused by temporal or spatial distortion. S3.3 Adaptive Global Projection: The reconstructed high-dimensional features are flattened and projected into Hamming space through a fully connected projection head that dynamically generates a mapping matrix based on the input feature dimensions. The predicted watermark bitstream is then output via the Sigmoid function.

6. The method for proactive evidence collection from GIF face images based on spatiotemporal watermarking according to claim 1, characterized in that, The spatiotemporal adaptive residual encoder and the deep integrity recovery decoder are jointly optimized through an adversarial training framework that includes a realistic distortion simulator; The realistic distortion simulator is connected during the training phase to apply differentiable distortion simulation to the watermarked sequence. The distortion simulation includes: deepfake generation simulation, geometric distortion simulation, and signal-level distortion simulation. The total loss function for the adversarial training is: in, L imp To constrain the loss of visual imperceptibility; L msg To limit the loss of accuracy in watermark message recovery; L adv_g To combat the losses.

7. The method for proactive evidence collection from GIF face images based on spatiotemporal watermarking according to claim 5, characterized in that, The adversarial training adopts an adaptive course learning strategy based on a realistic distortion simulator, which dynamically adjusts the trigger probability of each distortion operator through a probability ramping mechanism; slight distortion is applied in the early stage of training, and the sampling probability of random pruning and deepfake simulation is gradually increased as the training rounds increase.

8. A computer device, characterized in that, The computer device includes a processor and a memory, wherein the memory stores at least one instruction, at least one program, code set, or instruction set, and the instruction, program, code set, or instruction set is loaded and executed by the processor to implement the active evidence collection method for GIF face images based on spatiotemporal watermarking as described in any one of claims 1-6.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores at least one instruction, at least one program, code set, or instruction set, which is loaded and executed by a processor to implement the active evidence collection method for GIF face images based on spatiotemporal watermarking as described in any one of claims 1-6.

Images