Depth attributable watermarking method oriented to multi-person scene

Abstract

The invention discloses a depth attributable watermarking method oriented to a multi-person scene, and belongs to the technical field of image evidence obtaining and deep learning. According to the method, a multi-face encoder-decoder architecture is constructed, an end-to-end full convolutional network is adopted, an original image is used as a whole to be input, parallel recognition is performed, watermarks are embedded into all target face regions, one-time efficient parallel processing and cross-region feature extraction of multiple face targets are realized, and preprocessing steps are simplified fundamentally; in addition, a selective region is constructed to supervise loss, a forged face positioning mask is introduced as a loss weight, the watermark extraction loss of a compulsory decoder is only calculated in a tampered region, the watermark extraction capability is coupled with the positioning of a forged region, and pixel-level interpretable positioning judgment is realized. The invention also provides a dual attribution mechanism, and provides accurate positioning and attributable source fingerprints provided by watermarks. Through successful recovery of attributable watermarks, a complete evidence closed loop of extracting traceability watermarks from positioning watermarks is realized.

Description

Technical Field

[0001] This invention belongs to the field of image forensics and deep learning technology, and in particular relates to a deep attributable watermarking method for multi-person scenarios. Background Technology

[0002] Deepfake countermeasures are a key technology in digital image forensics, designed to ensure the authenticity of multimedia content by detecting the genuineness and provenance of images. With the increasing prevalence of deepfakes and image editing tools, deepfake countermeasures are becoming increasingly important in areas such as news verification, forensic investigation, and social media content moderation.

[0003] Traditional deepfakes generally focus on single-face scenes, ignoring the dual problems that exist in multi-face scenes: the number of real and fake faces is unknown, and compared with an ideal single face image, each face generally has lower resolution, greater pose diversity, and more severe occlusion or deformation, resulting in a significant decrease in face quality, making it difficult to identify which one has been tampered with.

[0004] Currently, existing technical solutions mainly fall into two categories: (1) Passive evidence collection method Representative approaches include SBIs (Shiohara et al., CVPR 2022), S-MIL (Li et al., In Proceedings of the 28th ACM international conference on multimedia 2020), and MRL+GFA (Lin et al., IEEE Transactions on Information Forensics and Security 2024). These approaches guide classifiers to learn general forgery features by generating "self-mixing images," employ spatiotemporal coding instance methods to capture intra- and inter-frame inconsistencies, and explore correlations between faces by combining Multi-Face Relationship Learning (MRL) and Global Feature Aggregation (GFA). However, they rely on specific data distributions and compression standards, which often leads to false positives or false negatives when dealing with high-quality deepfake content. Especially when the forgery content is very sophisticated, or when images / videos have undergone compression or noise interference, existing detection methods still exhibit weak robustness.

[0005] (2) Active defense methods Representative solutions include CMUA (Huang et al., In proceedings of the AAAI conference on artificial intelligence 2022), KAD-Net (He et al., Knowledge-Based Systems 2025), and MEA+AIS (Jia et al., AAAI 2025). These solutions defend against deepfakes through a cross-model general adversarial watermarking framework. They further improve proactive forensics by separating source tracing and forgery detection into two dedicated branches. They also define and verify the key vulnerability of multiple embedding attacks (MEA) in the proactive forensic watermarking field and propose an adversarial interference simulation (AIS) model-independent training paradigm. However, as deepfakes achieve breakthroughs in image detail fidelity, existing watermarking technologies often cannot effectively combat forgeries caused by minute changes, leading to a significant reduction in watermark stability and extractability, especially in multi-face scenarios where deepfake detection and source tracing are difficult.

[0006] To address the aforementioned issues, this invention proposes a deep attributable watermarking method for multi-user scenarios. Summary of the Invention

[0007] The purpose of this invention is to propose a deep attributable watermarking method for multi-person scenarios. This addresses the problem that existing methods, when faced with multi-person scenarios closer to the real world, heavily rely on a single-face setting, making it difficult to effectively solve copyright protection issues. This invention successfully achieves deepfake localization and identity source tracking by constructing a multi-face codec and selective region supervised loss. To achieve the above objectives, the technical solution adopted by the present invention is as follows: A deep attributable watermarking method for multi-user scenarios includes the following steps: A deep attributable watermarking framework is constructed, which includes a multi-face encoder, a random perturbation module, a multi-face decoder, and a discriminator. The multi-face encoder is used to process multiple face regions detected in the input image in parallel. The corresponding message to be embedded is simultaneously embedded into each face region in the form of residual through a steganalysis kernel with shared weights, generating a watermarked encoded image. Using the random perturbation module, various random perturbations, including arbitrary distortion, normal distortion, and malicious distortion, are applied to the generated watermarked encoded image to generate a corresponding perturbation image; wherein the malicious distortion process records the corresponding tampering operation and generates a mask image; The generated perturbation image is processed using the multi-face decoder, and the message vectors of all face regions are recovered in one go through a decoding network with shared weights. The discriminator is used during training to distinguish between the original face image patch and the face image patch after watermarking by the multi-face encoder. Adversarial training drives the multi-face encoder to minimize the distribution difference between the two, thereby precisely limiting the imperceptible constraint of the watermark to the face region.

[0008] Preferably, the processing procedure of the multi-face encoder specifically includes: Given an input image and normalized face bounding box coordinates, compute pixel-level coordinates at the whole-image scale; Each face window is resampled to a fixed size using differentiable bilinear interpolation to obtain a batch tensor. The message to be embedded and the resampled face image tensor are input together into a U-shaped steganography kernel with shared weights; the steganography kernel injects information layer by layer on multi-scale features and outputs a residual map. The resulting residual map is inversely interpolated back to the original coordinates, and the image is updated in the form of residuals; The formula for the above operation is expressed as follows:

[0009] in, I en Represents an encoded image. I co Represents the original image. This represents the residual image.

[0010] Preferably, the multi-face decoder includes a source tracer and a locator; The source tracer recovers messages from perturbed images subjected to arbitrary distortion, enabling robust watermark extraction and identity attribution; The locator achieves accurate localization of forged regions in face images by comparing the differences between messages recovered from images subjected to ordinary distortion and messages recovered from images subjected to malicious distortion, combined with selective region supervision loss.

[0011] Preferably, the selective region supervision loss is used to train the localizer, and it is constructed as follows: The face regions in the image that have been tampered with by deepfakes are defined as fake ground truth boxes, and the regions processed by the multi-face encoder are defined as face embedding boxes. Calculate the IoU (Intersection over Union) between the fake ground truth bounding boxes and the face embedding boxes, and determine a localization mask to mark the set of all face embedding boxes affected by malicious tampering; Watermark recovery loss is applied only to the face regions marked as true by the positioning mask, guiding the watermark in those regions to recover to a preset failure state; The logical conditions for the above operation are:

[0012] in, B mask Represents a mask box; E i Represents the embedding box of the i-th face; F j This represents the j-th fake truth box; Used to measure the degree of overlap between the i-th face embedding and the j-th fake ground truth box; Indicates the preset threshold for the degree of overlap; When the overlap exceeds the threshold When the time is right, mark the corresponding face embedding box as True and proceed to loss calculation:

[0013] in, Indicates selective area monitoring loss; A collection representing fake faces; This represents the watermark information of the i-th fake face.

[0014] Preferably, both the steganography kernel and the decoding network in the multi-face decoder adopt a U-Net structure.

[0015] The present invention further protects a computer device, the computer device including a processor and a memory, the memory storing at least one instruction, at least one program, code set or instruction set, the instruction, program, code set or instruction set being loaded and executed by the processor to implement the above-mentioned deep attributable watermarking method for multi-user scenarios.

[0016] The present invention further protects a computer-readable storage medium storing at least one instruction, at least one program, code set, or instruction set, wherein the instruction, program, code set, or instruction set is loaded and executed by a processor to implement the aforementioned deep attributable watermarking method for multi-user scenarios.

[0017] Compared with the prior art, the beneficial effects of the present invention are as follows: (1) This invention achieves highly efficient and robust multi-user scene processing: Unlike existing technologies, current active defense methods focus on single faces and require inefficient pipeline operations of face cropping, alignment, individual processing, and stitching, which are difficult to cope with the complexity and efficiency requirements of multi-person scenarios.

[0018] Implementation method: A multi-face encoder-decoder architecture was constructed, which adopts an end-to-end fully convolutional network. The original image is used as the whole input, and watermarks are identified and embedded into all target face regions in parallel. This achieves efficient parallel processing and cross-region feature extraction for multiple face targets in one go, which fundamentally simplifies the preprocessing steps.

[0019] (2) This invention can provide an interpretable and reliable chain of evidence: Unlike existing technologies, current black-box detection methods output the overall "true / false" probability, but cannot provide the precise area where forgery occurred (pixel-level or region-level), and lack a legally interpretable chain of evidence.

[0020] Implementation method: A selective region-supervised loss is constructed, and a fake face localization mask is introduced as a loss weight, forcing the decoder's watermark extraction loss to be calculated only in the tampered region. This mechanism couples the watermark extraction capability with the localization of the fake region, achieving pixel-level, interpretable localization determination.

[0021] (3) Possesses the dual ability of locating and attributing the root cause: Unlike existing technologies, which are limited in function and, even if they can locate the source, cannot link the location results to the specific forgery generation model, thus failing to provide a complete chain of evidence for tracing the source.

[0022] Implementation method: A dual attribution mechanism was proposed, providing precise location and source fingerprints from attributable watermarks. Through the successful recovery of the attributable watermark, a complete evidence loop for extracting and tracing the source watermark from the location watermark was achieved. Attached Figure Description

[0023] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the accompanying drawings involved in the embodiments are now briefly described. Obviously, the drawings in the following description are merely illustrative of some embodiments of the present invention. For those skilled in the art, other forms of drawings can be constructed based on these drawings without creative effort.

[0024] Figure 1 This is the overall flowchart of the deep attributable watermarking method for multi-user scenarios proposed in Embodiment 1 of the present invention. Figure 2 It is the multi-scene encoded image proposed in Embodiment 2 of the present invention. Detailed Implementation

[0025] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0026] This invention proposes a deep attributable watermarking method for multi-person scenarios. It addresses the shortcomings of existing passive forensics methods, which are mostly based on black-box detection and cannot provide a complete and interpretable chain of evidence. Therefore, they lack credibility in both legal terms and practical application in combating deepfakes. Furthermore, existing active defense methods focus on single-face scenarios, neglecting multi-person interaction scenarios that are closer to the real world, making it difficult to locate and trace deepfakes in multi-person scenarios. The reasons for this are as follows: 1. Existing detection models (such as deep learning-based classifiers) typically focus on classification accuracy rather than the basis for decision-making. They treat images or videos as a whole, outputting a "true / false" probability, but cannot precisely indicate whether the model's judgment is based on a specific pixel on a face, artifacts, or temporal inconsistencies in the video. In legal evidence collection, it is necessary to clearly demonstrate "where the forgery occurred" and "why it was determined to be forgery." Because black-box models only provide results and not processes, their conclusions are difficult to accept as a "complete and interpretable chain of evidence" in legal and judicial practice. Most passive detection relies on tiny, fragile traces or artifacts left by generative models during the synthesis process. These traces are easily destroyed or obscured after post-processing (such as video compression, format conversion, network transmission, etc.). The features on which the detection model relies disappear after adversarial processing, causing the model to fail and unable to continuously lock in evidence of forgery, naturally making it impossible to establish a complete chain from "forgery traces" to "forgery source."

[0027] In multi-person scenarios, the occlusion of faces, significant scale differences (distant and close-up faces), and complex temporal relationships among multiple targets exponentially increase the difficulty of extracting stable facial features and tracking forged parts. Current localization and source tracing algorithms, whether active or passive, are primarily designed and validated to overcome the challenges of single-face forgery (such as resolution and universality). When these algorithms are directly applied to multi-person scenarios, they cannot effectively perform multi-target detection and tracking, let alone handle different forgery types on multiple targets simultaneously. Even if a face is successfully located, it is difficult to establish forgery associations or source tracing relationships between multiple faces or between a face and the background.

[0028] To address the aforementioned issues, this invention proposes a deep attributable watermarking method for multi-person scenarios. This method overcomes the efficiency and accuracy problems of traditional methods when handling multi-person scenes. By employing a novel multi-face encoder-decoder architecture, it fundamentally simplifies the preprocessing steps required for traditional forensic investigation. This architecture aims to achieve parallel watermark embedding for all multi-face targets in an image and, through complex cross-region processing, ensures efficient and stable watermark embedding and extraction even under complex conditions such as multi-person interaction, occlusion, and scale variations.

[0029] This invention achieves precise and fine-grained localization of forged regions by proposing an innovative mechanism of selective region supervision loss. This mechanism forces the decoder to selectively decode only the facial regions corrupted by deepfakes. In this way, the invention aims to provide "interpretable" evidence, that is, to clearly answer which facial region has been tampered with, thereby solving the problem that existing black-box detection cannot provide a complete chain of evidence.

[0030] Based on the aforementioned precise positioning mechanism, this invention achieves efficient identity attribution. Through the deep attributable watermarking framework (DAWF), it is possible to clearly distinguish and identify who has been forged, precisely pinpointing the forgery to a specific facial target. This not only enhances the granularity of evidence collection but also provides direct and clear identity evidence for legal accountability and platform governance.

[0031] In summary, this invention constructs a highly efficient and interpretable method for proactive evidence collection, location, and tracing of deepfakes by utilizing the DAWF framework and its core mechanisms. This method is designed for complex multi-user scenarios and simultaneously achieves accurate localization of forged areas and effective attribution of forged identities.

[0032] The following description, in conjunction with the accompanying drawings and relevant examples, further illustrates the deep attributable watermarking method for multi-user scenarios proposed in this invention.

[0033] Example 1: Please see Figure 1 This invention proposes a method for deepfake localization and attribution in multi-person scenarios, the core of which is the Deep Attributable Watermarking Framework (DAWF). This framework, through an innovative multi-face encoder-decoder architecture and selective region supervision loss, achieves accurate localization of forged regions and effective attribution of forged identities in complex multi-person scenarios.

[0034] The core improvements of this invention include: By employing an end-to-end fully convolutional network, the face detection, preprocessing, and watermark embedding / extraction processes are parallelized, eliminating the complex and inefficient single-target cropping and alignment steps in traditional forensics, and achieving efficient parallel embedding and cross-region processing.

[0035] During decoder training, a fake face localization mask is introduced to provide weighted guidance for the watermark reconstruction loss. This forces the decoder to learn watermark extraction only in the faked regions, achieving a strong coupling between watermark extraction capability and fake face localization.

[0036] Combining a multi-image encryption mechanism and attributable watermarking, it accurately answers "which" facial region was forged; successful recovery of the attributable watermark proves "who" was forged, providing a reliable and complete chain of evidence.

[0037] This invention innovatively achieves parallel embedding of multiple independent messages across regions within a single image. The original face image is processed by a face detector to obtain a coordinate matrix and bounding box image. After layered encoding by a steganalysis kernel, an encoded image with multiple face steganalysis patches is output. This image is then fed into a noise pool to generate different types of noisy images. Subsequently, a source tracing unit recovers robust messages from the noisy images, while a locator extracts messages from ordinary distortion and deepfake distortion, determining whether a region has been maliciously forged based on message differences.

[0038] The specific design of this invention includes: 1. Multifaceted encoder Given an image and a normalized face bounding box, this invention first calculates pixel-level coordinates at the whole-image scale, then uses differentiable bilinear interpolation to resample each window to a fixed 64×64, obtaining a batch tensor. The message to be embedded is fed into a weighted steginary kernel along with the face image. The U-shaped structure of the steginary kernel injects the message layer by layer across multi-scale features, outputting a residual map. The residual map is then inversely interpolated back to the original coordinates, and the image is updated in residual form.

[0039] 2. Random perturbation In each forward propagation, this invention independently selects distortion from three pools—arbitrary distortion, normal distortion, and malicious distortion—to generate three different randomly perturbed images for the encoded image. During the generation of forged images, the module captures and records malicious tampering operations and converts them into mask images. This captured tampering information serves as the ground truth for subsequent selective region supervision loss, guiding the training of the localizer.

[0040] 3. Multi-faceted decoder The multifaceted decoder uses the same "differentiable window sampling" strategy as the multifaceted encoder, reconstructing the messages of all faces at once using a U-Net with shared weights, resulting in a message vector for each face. The multifaceted source and multifaceted localizer share the same U-Net network. The multifaceted source accepts perturbed images with arbitrary distortion, and its training objective is... The multi-faceted positioner accepts common disturbances. and malicious disturbance The images are compared to achieve tamper detection. Furthermore, we introduce a selective region supervision loss to optimize the deep forgery localization objective.

[0041] 4. Discriminator During training, the discriminator distinguishes facial patches rather than the entire image. This drives the multifaceted encoder to minimize the distributional differences between the encoded facial patches and the original facial patches, thus precisely limiting the imperceptible constraints to the facial regions.

[0042] 5. Selective area monitoring loss Selective region supervision loss guides the watermark recovery target under malicious attacks to a pre-defined "failure state" to achieve sensitivity to deepfakes. Supervision is applied only to face regions in the image that have been tampered with by deepfakes. We need to determine a localization mask to mark all face embeddings affected by the tampering. We construct this mask by calculating the IoU (Intersection over Union) between the tampered ground truth bounding boxes and the face embeddings (encoder processing areas). The logical conditions are as follows:

[0043] Here, IoU(Ei, Fj) measures the degree of overlap between the i-th embedding box and the j-th fake ground truth box. Only when the overlap exceeds a threshold will the corresponding embedding box Ei be marked as True and included in the set for loss calculation.

[0044]

[0045] It is calculated only on the set of regions marked above.

[0046] Based on the above, it can be seen that the present invention proposes: (1) High-efficiency parallel processing architecture for multi-face scenes: This refers to the DAWF multi-face encoder-decoder architecture. It employs an end-to-end fully convolutional network to achieve parallel watermark embedding and extraction of all facial targets in an image, avoiding the inefficient steps of cropping, aligning, and processing each face individually in traditional methods.

[0047] Existing proactive defense methods typically employ a pipeline operation of single-face cropping, processing, and stitching. The model focuses only on a single cropped, high-resolution face region, offering poor support for parallel processing of multiple faces.

[0048] This invention employs an architecture of overall image input and parallel feature processing, fundamentally simplifying the preprocessing complexity of multi-person scenes and improving processing efficiency. Existing technologies require complex loops and post-processing logic to handle multiple targets, resulting in poor efficiency and robustness.

[0049] (2) Coupling mechanism between forged location and watermark extraction: A selective region-supervised loss is proposed, which introduces a fake face localization mask as a weighting factor during decoder training. This mechanism forces the loss calculation for watermark extraction to focus only on the faked region, thus forcibly coupling the watermark extraction capability with the accurate localization of the fake region.

[0050] Existing passive detection methods typically use classification loss (such as cross-entropy) to output the overall true / false probability; or use unweighted reconstruction loss to extract watermarks, which cannot distinguish which region in the image contributes to the loss.

[0051] The selective region supervision loss proposed in this invention achieves a unified loss level for extracting forensic features (watermarks) and locating forged regions. This enables the invention to output interpretable localization maps, while existing black-box methods or non-selective watermarking methods cannot provide region-level forensic evidence.

[0052] (3) Dual attribution ability: The combination of watermarking and selective region supervision loss determines "which" face region was forged through selective region supervision loss, and achieves a complete chain of evidence for location and identity attribution through the recovered attributable watermark.

[0053] Existing methods focus on single-dimensional functionality. Passive detection can only answer "whether it is forged"; traditional watermarking can only answer "whether the watermark has been destroyed"; even if it can locate the source, it is difficult to correlate the location result with the specific source that was forged.

[0054] The invention integrates precise location and source tracing into a single framework through technological means. This design, which couples location and source tracing features, provides a more complete and legally significant dual attribution capability than existing technologies.

[0055] Example 2: Based on Example 1, but with a difference: given the scarcity of dedicated datasets for multi-face manipulation detection, this invention systematically investigated existing resources and selected two of the most relevant and authoritative benchmarks in computer vision (i.e., Wideface and COCO2017) for rigorous validation. Their size and diversity are widely considered sufficient to rigorously validate the effectiveness of the proposed method. The Wideface dataset size was adjusted to 1024. The resolution is 700. Specifically, 15406 / 1083 / 3000 face images were used for training, evaluation, and testing, respectively. To comprehensively evaluate the model's generalizability, this invention extracted 3,000 multi-face images from the COCO2017 dataset and conducted source tracking and depth pseudo-localization tests under various distortions, including common and malicious distortions. To detect faces in multi-face images for further processing and feature extraction, this invention used the MobileFaceswap face detector. Furthermore, the detected individual faces were resampled to a size of 64×64 as input to the steganalysis kernel.

[0056] The DAWF proposed in this invention is implemented in PyTorch and executed on an NVIDIA A40. The network optimizer uses the ADAM optimizer with a learning rate of 0.0008. The model is trained for 50 epochs, with 64 training images per batch. The discriminator weights, encoded image weights, source information weights, common distortion information weights, and malicious distortion weights are 15, 10, 10, 10, and 0.01, respectively.

[0057] This invention employs a comprehensive set of metrics to evaluate the performance of the proposed framework, including imperceptibility, message robustness, and unique deep pseudo-localization capabilities. PSNR measures image reconstruction accuracy, SSIM evaluates image structural similarity, and bit error rate quantifies the accuracy of watermark extraction. Precision, recall, and F1 score are used to evaluate deep pseudo-localization accuracy by calculating the IOU values ​​between predicted pseudo-face bounding boxes and ground truth pseudo-face bounding boxes.

[0058] To ensure objective and comprehensive comparisons in complex multi-face environments, four representative active forensic watermarking methods were selected as baselines: CIN, SepMark, FaceSigns, and WaveGuard. This invention uniformly scales images from the WideRFace dataset to 256. The image resolution is 256, with a message capacity of 128 bits. All images were retrained and evaluated on all distortion levels. For FaceSigns, this invention uses its publicly disclosed pre-trained model, cropping and resizing faces extracted by the face detection model to 256×256 pixels according to the method described in the paper, and then embedding the watermark into each face. This invention directly trains the model on 1024×700 resolution images to compare with a pre-trained baseline model, demonstrating the effectiveness of the proposed model in tamper location and source tracing.

[0059] Table 1. Quantitative visual quality assessment of coded images

[0060] As shown in Table 1 above, this invention uses PSNR and SSIM as the main objective indicators to evaluate the visual quality of coded images. DAWF achieves a high PSNR value, second only to FaceSigns.

[0061] In addition, such as Figure 2 As shown, although the proposed method achieves high peak signal-to-noise ratio and SSIM, subtle embedding residuals exist in the encoded image under high-intensity embedding settings. However, these residuals represent an acceptable trade-off aimed at achieving higher depth pseudo-localization accuracy and message robustness in complex multi-person scenes.

[0062] This invention compares the performance of each model through two tasks: source tracing and detection, targeting 11 common types of distortion and 3 types of malicious distortion. This invention uses the bit error rate (BER) as a metric for watermark robustness, where a lower average BER indicates better robustness of the watermark under various distortions.

[0063] Table 2. Quantitative comparison of Bit Error Rate (BER) of watermarks on the Wideface dataset under common distortion conditions.

[0064] Table 3. Quantitative comparison of Bit Error Rate (BER) of watermarks on the Wideface dataset under malicious distortion.

[0065] Please refer to Table 2-3, where DAWF achieves robustness second only to WaveGuard in both the source tracer and locator. This invention evaluated the bit error rate under typical deepfake operations: for most deepfake operations, the proposed source tracer achieved a low bit error rate, while the locator exhibited strong deepfake sensitivity (bit error rate close to 50%). This strongly confirms the special effectiveness of the selective region supervision loss of this invention. In contrast, SepMark exhibits a high bit error rate under various common distortions, indicating that its detector sacrifices resistance to common distortions in pursuit of high sensitivity to deepfake operations. FaceSigns exhibits a high bit error rate under common distortions due to its sensitivity to severe scaling. WaveGuard's detection bit error rate is close to 0% under malicious distortion conditions, indicating that its watermarking mechanism is insensitive to locally damaged regions and cannot trigger detection.

[0066] The core advantage of DAWF lies in its deepfake localization accuracy. This advantage is not accidental, but rather an organic combination of three systematic innovations built upon this invention: attributable watermarking, multi-faceted encoder-decoder, and selective region supervision loss—each indispensable. It is worth noting that the localization accuracy proposed in this invention does not simply refer to the localization of geometric regions in an image; more importantly, for tracing the individual identity of the tampered party (source tracing), this invention calculates the IoU value between the predicted tamper bounding box and the ground-based real tamper bounding box, introducing three parameters—precision, recall, and F1 score—to evaluate the accuracy of deepfake localization. Since all baseline models lack an IoU matching mechanism required for individual-level tamper localization, these three metrics are unique performance evaluation dimensions of the DAWF framework.

[0067] Table 4. Quantitative results of deepfake localization technology on the WIDER-Face dataset.

[0068] As shown in Table 4, the DAWF framework proposed in this invention achieves an average F1 score of 84.95% under malicious Deepfake attacks. The framework performs particularly well in dealing with Ghost attacks, achieving an F1 score of 92.37%.

[0069] Table 5. Quantitative comparison of bit error rate (BER) of watermarks under different distortion conditions in the COCO2017 dataset. (PSNR=51.3179, SSIM=0.9957)

[0070] Table 6. Quantitative results of deepfake localization on the COCO2017 dataset.

[0071] In Tables 5-6, the performance on the COCO2017 dataset, compared to the WideRFace dataset, showed only slight fluctuations in the PSNR of the encoded image, the bit error rate per distortion, and the F1 score for deepfake localization. This high consistency in performance strongly validates the framework's good cross-dataset generalization ability.

[0072] This invention conducts an ablation study on the IOU threshold. This threshold defines a criterion for evaluating whether a pre-determined forged face bounding box constitutes a valid match.

[0073] Table 7 Ablation experiments; IOU value range is Gain: "↑" indicates better

[0074] In Table 7, smaller thresholds show a slight increase in accuracy but lead to a decrease in the F1 score. Conversely, increasing the threshold by a higher threshold results in a decrease in all three metrics. Therefore, to ensure that the DAWF can fully capture deep pseudo-regions, 0.1 was ultimately chosen as the final threshold.

[0075] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.

Claims

1. A deep attributable watermarking method for multi-person scenarios, characterized in that, The method comprises the following steps: A deep attributable watermark framework is constructed, which comprises a multi-face encoder, a random perturbation module, a multi-face decoder and a discriminator; The multi-face encoder is used to perform parallel processing on multiple face regions detected in an input image, and a corresponding message to be embedded is embedded in each face region in the form of a residual through a steganographic kernel with shared weights to generate a watermark-containing encoded image; The random perturbation module is used to apply multiple random perturbations including arbitrary distortion, common distortion and malicious distortion to the generated watermark-containing encoded image to generate a corresponding perturbed image; wherein the malicious distortion process records the corresponding tampering operation and generates a mask image; The multi-face decoder is used to process the generated perturbed image, and a message vector of all face regions is recovered through a decoding network with shared weights at one time; The discriminator is used to distinguish between an original face image block and a face image block embedded with a watermark by the multi-face encoder during the training process, and the multi-face encoder is driven to minimize the distribution difference between the two through adversarial training, so as to accurately limit the imperceptibility of the watermark to the face region.

2. The method of claim 1, wherein, The processing process of the multi-face encoder specifically comprises: Given an input image and normalized face frame coordinates, pixel-level coordinates are calculated at the whole image scale; Each face window is resampled to a fixed size using a differentiable bilinear interpolation to obtain a batch tensor; The message to be embedded is input into a U-shaped structure steganographic kernel with shared weights together with the resampled face image tensor; the steganographic kernel injects information layer by layer on multi-scale features and outputs a residual map; The obtained residual map is inversely interpolated back to the original coordinates, and the image is updated in the form of a residual; The formula of the above operation is: wherein, I en denotes the encoded image, I co denotes the original image, denotes the residual image.

3. The method of claim 1, wherein, The multi-face decoder comprises a tracer and a locator; The tracer recovers the message from the perturbed image subjected to arbitrary distortion, and realizes robust extraction and identity attribution of the watermark; The locator realizes accurate positioning of the forged region in the face image by comparing the difference between the message recovered from the image subjected to common distortion and the message recovered from the image subjected to malicious distortion, and combining a selective region supervision loss.

4. The method of claim 3, wherein, The selective region supervision loss is used to train the locator, and is constructed in the following manner: The face region in the image that is deeply forged and tampered is defined as a forged ground truth box, and the region processed by the multi-face encoder is defined as a face embedding box; The IoU between the forged ground truth box and the face embedding box is calculated to determine a positioning mask to mark all face embedding box sets affected by malicious tampering; Only in the face region marked as true by the positioning mask, a watermark recovery loss is applied to guide the watermark recovery of the region to a preset failure state; The logic condition of the above operation is: wherein, B mask denotes a mask box; E i denotes an i-th face embedding box; F j denotes a j-th fake ground truth box; for measuring an overlap degree between the i-th face embedding box and the j-th fake ground truth box; denotes a preset overlap degree threshold; When the degree of overlap exceeds the threshold The corresponding face embedding box is marked as True, and the loss calculation is entered: wherein, denotes a selective region supervision loss; denotes a set of fake human faces; denotes an information watermark of the i-th fake human face.

5. The method of claim 1, wherein, The steganographic kernel and the decoding network in the multi-face decoder both adopt a U-Net structure.

6. A computer device, comprising: The computer device comprises a processor and a memory, and the memory stores at least one instruction, at least one program, a code set or an instruction set, which are loaded and executed by the processor to implement the deep attributable watermark method in the multi-person scene as claimed in any one of claims 1-5.

7. A computer-readable storage medium, characterized in that, The computer readable storage medium stores at least one instruction, at least one program, a code set or an instruction set, which are loaded and executed by the processor to implement the deep attributable watermark method in the multi-person scene as claimed in any one of claims 1-5.

Images