Image editing traceable spatial adaptive plug-and-play watermarking method and device

Abstract

The application discloses a space adaptive plug-and-play watermarking method and device for image editing traceability, and the method comprises the following steps: obtaining the latent space feature representation of the edited image through an image editing model according to the input original image and editing conditions; constructing a structured watermark grayscale image for carrying the core information of image editing traceability; encoding the structured watermark grayscale image into watermark latent features through a watermark encoder; realizing watermark embedding with content and structure perception in the latent space based on the watermark latent features, and obtaining the edited image latent space feature representation with watermark; inputting the edited image latent space feature representation into an image decoder to decode it into an edited image with watermark; inputting the edited image with watermark into a watermark decoder to extract the embedded watermark grayscale image, and recovering the complete traceability metadata information based on the watermark grayscale image. The device comprises a processor and a memory. The application realizes reliable traceability while maintaining high visual quality of the image.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of artificial intelligence generated content (AIGC) security, in particular to a space adaptive plug-and-play watermarking method and device for image editing traceability. BACKGROUND

[0002] The rapid development of AIGC, especially diffusion models represented by Stable Diffusion, DALL-E and Imagen, is reshaping the landscape of digital content creation. These technologies not only significantly improve the quality and efficiency of image generation, but also become a new engine to promote the development of creative economy. However, this technological revolution has brought unprecedented challenges. With the improvement of the quality of generated content, copyright disputes and other issues have become increasingly prominent, making content authenticity verification, copyright ownership and privacy protection urgent problems to be solved.

[0003] Under this background, watermarking technology as a key solution to this problem provides reliable identity authentication and traceability for AI generated content. Existing watermarking research mainly focuses on text-to-image models, and according to different training strategies, it can be mainly divided into the following three categories: joint training, fine-tuning and plug-and-play methods. Joint training watermarking method realizes high consistency embedding by optimizing the model and watermark module together, but the training cost is high and it is difficult to generalize. Fine-tuning watermarking method embeds watermark by adjusting diffusion model parameters during generation, which has strong robustness but high deployment cost. Plug-and-play watermarking method, as the current research hotspot, does not need to modify any model parameters and can directly embed watermark in input, latent space or output layer, which has excellent flexibility.

[0004] However, the existing image watermarking scheme has fundamental limitations when applied to editing scenarios, mainly facing three key challenges: (1) Local sensitivity. Unlike the generation task of creating new content, editing operations usually modify only specific image regions while maintaining the integrity of unmodified regions. This local modification mode requires the watermarking scheme to adapt flexibly to the spatial changes of the edited image. (2) Complex traceability chain. Unlike the generation task that only needs to identify information, the editing task must have a complete chain of responsibility traceability, which requires more comprehensive information coding, greatly exceeding the capacity limit of traditional binary watermarking methods. (3) Trade-off between capacity and invisibility. In the editing scenario, users have higher requirements for the perceived quality of images, and increasing watermark capacity and robustness usually means stronger watermark embedding strength, but too strong embedding will damage the editing quality and produce image artifacts. Therefore, the purpose of the present application is to overcome the limitations of existing technologies and provide an image watermarking scheme that can achieve the best balance among high watermark capacity, high robustness and high visual quality. SUMMARY

[0005] The application provides a spatial adaptive plug-and-play watermarking method and device for image editing traceability, aiming to solve three core problems in existing image watermarking technology under editing scenarios: first, existing methods are difficult to adapt to the local characteristics of editing operations, and there is a contradiction between the inherent global processing mode and the watermark spatial distribution requirements required by local editing; second, the capacity of traditional binary watermarking is limited, and it is difficult to carry multi-dimensional metadata required by image editing traceability, and it is difficult to establish a complete responsibility traceability chain; third, under the premise of ensuring visual quality, existing technology is difficult to simultaneously realize high-capacity watermark embedding and strong robustness, and cannot meet the dual demands of image quality and traceability in actual applications; by systematically solving the above three core problems, the application ensures reliable traceability in complex editing environment while maintaining high visual quality of images, providing a feasible technical solution for the safe governance and copyright protection of AIGC content, as described in detail below:

[0006] In a first aspect, a spatial adaptive plug-and-play watermarking method for image editing traceability is provided, the method comprising:

[0007] For the input original image and editing conditions, obtain the latent space feature representation of the edited image through an image editing model; construct a structured watermark grayscale image for carrying core information of image editing traceability, and encode the structured watermark grayscale image into watermark latent features through a watermark encoder;

[0008] Input the latent space feature representation of the edited image into an attention-guided progressive network, and based on the watermark latent features, realize content and structure-aware watermark embedding in the latent space to obtain the watermark-edited image latent space feature representation;

[0009] Input the watermark-edited image latent space feature representation into an image decoder to decode it into a watermark-edited image, input the watermark-edited image into a watermark decoder to extract the embedded watermark grayscale image, and based on the watermark grayscale image, recover the complete traceability metadata information.

[0010] The method comprises constructing a joint loss function to train the watermark encoder, watermark decoder and attention-guided progressive network end-to-end; the joint loss function includes: visual quality loss for ensuring imperceptibility of watermark embedding; watermark recoverability loss for ensuring reliable extraction of watermark information; and smoothing regularization loss for preventing visual artifacts caused by sudden changes in embedding strength.

[0011] The structured watermark grayscale image is composed of source image identification, editing instructions, user identity and time information, adopts block encoding design, and ensures the security and uniqueness of watermark information through a hash encryption mechanism.

[0012] The structured watermark grayscale image for carrying image editing traceability core information is specifically:

[0013] The watermark image is designed as The structured watermark grayscale image is constructed by dividing the watermark grayscale image into non-overlapping blocks with a size of N is the resolution of the watermark grayscale image, i.e., the width and height of the watermark grayscale image, and the watermark grayscale image is used to carry multi-dimensional metadata information. The structured watermark grayscale image realizes comprehensive traceability of the editing process through the synergistic effect of four types of core metadata.

[0014] The method divides the watermark grayscale image into non-overlapping blocks with a size of Each non-overlapping block encodes one character, and multiple metadata fields are stored in a single structured watermark grayscale image. The watermark grayscale image is represented as:

[0015]

[0016] Each corresponds to a character in the b b image block. The effective watermark capacity is represented as:

[0017] .

[0018] The watermark embedding in the latent space for content and structure perception is:

[0019] The latent space feature representation of the edited image is input into the attention-guided progressive network to generate a spatially adaptive watermark embedding strength matrix. The watermark latent feature is intensity-modulated in the latent space and additively fused with the latent space feature of the edited image to obtain the watermark-edited image latent space feature representation.

[0020] The watermark embedding strategy in the latent space for content and structure perception is executed to realize watermark embedding by maintaining semantic content consistency and adaptive fusion of spatial structure.

[0021] The attention-guided progressive network is used to realize adaptive watermark embedding with content and structure perception capability in the latent space.

[0022] By learning the feature distribution in the latent space, the optimal watermark embedding mode is obtained.

[0023] Given the edited image latent space feature representation and the watermark latent feature , the embedding process is represented as:

[0024]

[0025] wherein the symbol " denotes an element-wise modulation operation, denotes an attention-guided progressive network with parameters , whose output is normalized by a Sigmoid activation function to obtain an adaptive intensity matrix for regulating the embedding strength of the watermark feature, is a latent space encoder, is a watermark encoder, and W is a structured watermark grayscale image, is a watermarked edited image latent space feature representation, is an image editing model, and c is an editing condition, and x is an original image.

[0026] In a second aspect, a spatially adaptive plug-and-play watermarking device for image editing provenance is provided, the device comprising: a processor and a memory, the memory storing program instructions, the processor invoking the program instructions stored in the memory to cause the device to perform the method of any one of the first aspect.

[0027] In a third aspect, a computer-readable storage medium is provided, the computer-readable storage medium storing a computer program, the computer program comprising program instructions, the program instructions being executed by a processor to cause the processor to perform the method of any one of the first aspect.

[0028] The technical scheme provided by the present application has the following beneficial effects:

[0029] 1. The present application uses a structured watermark grayscale image as an information carrier, which expands the traditional low-capacity bit string into a two-dimensional grayscale image encoded by blocks. Under the settings of size N = 256 and block size b = 4, the single embedding capacity can reach 32,768 bits, which is significantly higher than the common generated model watermark (usually 32-1024 bits). The block-based regional storage makes local editing or destruction not cause the overall decoding failure, and can simultaneously carry multiple field metadata such as source image identification, editing instructions, user identity and time information, meeting the requirements of image editing provenance for information completeness and robustness.

[0030] 2. The present application proposes a content and structure-aware embedding strategy, which introduces an attention-guided progressive network APNet in the latent space of the diffusion editing model, so that it can autonomously learn and predict a spatially adaptive intensity matrix, thereby realizing dynamic regulation of the watermark embedding strength. This method presents the spatial optimal regulation characteristics of "enhancing embedding in high-frequency regions and suppressing embedding in smooth regions", which can effectively suppress the image artifacts introduced by watermark embedding while ensuring the high decoding stability of the watermark, significantly improving the visual quality and editing fidelity of the edited image. ​

[0031] 3、The application adopts modular design, and the watermark framework can be flexibly integrated into various image editing models based on the Stable Diffusion architecture as an independent plug-in, without modifying the backbone network parameters to realize plug-and-play, has cross-model compatibility and engineering feasibility, and is convenient for quick landing application in the existing image editing system.

[0032] Therefore, the application can realize high-capacity embedding, high-precision extraction and multi-dimensional tracing of watermarks in the image editing scene, balance the requirements of visual quality and robustness, realize the verifiability and traceability of the editing process, and provide reliable technical support for copyright protection, safety supervision and responsibility attribution of artificial intelligence generated content (AIGC). BRIEF DESCRIPTION OF DRAWINGS

[0033] Figure 1 Fig. 1 is a flowchart of a spatial adaptive plug-and-play watermark method for image editing traceability;

[0034] Figure 2 Fig. 2 is a schematic diagram of the framework of the spatial adaptive plug-and-play watermark method for image editing traceability;

[0035] Figure 3 Fig. 3 is a robustness test result diagram of the spatial adaptive plug-and-play watermark method for image editing traceability. DETAILED DESCRIPTION

[0036] In order to make the purpose, technical scheme and advantages of the application more clear, the embodiments of the application are further described in detail below.

[0037] Embodiment 1

[0038] The embodiment of the application discloses a spatial adaptive plug-and-play watermark method for image editing traceability, referring to Figure 1 The method comprises the following steps:

[0039] Step 101: For the input original image and editing conditions (for example: text instructions, masks or sketches), the latent space feature representation of the edited image is obtained through the image editing model based on the Stable Diffusion architecture (for example: InstructPix2Pix, MagicBrush, PFB-diff and other image editing models); at the same time, a structured watermark grayscale image for carrying the core information of image editing traceability is constructed, and the structured watermark grayscale image is encoded into a watermark latent feature through a watermark encoder;

[0040] The structured watermark grayscale image is composed of multi-dimensional metadata fields such as source image identification, editing instruction, user identity and time information, adopts block coding design to improve watermark capacity and robustness, and ensures the security and uniqueness of watermark information through a hash encryption mechanism.

[0041] Step 102: based on the watermark latent feature, realizing content and structure-aware watermark embedding in the latent space to obtain a watermark-edited image latent feature representation;

[0042] Specifically, the latent feature representation of the edited image is input into the attention-guided progressive network, which adopts a U-Net backbone architecture and contains an encoder-decoder structure and integrates a multi-head self-attention mechanism to finally generate a spatially adaptive watermark embedding strength matrix. The strength matrix is then used to spatially modulate the watermark latent feature, specifically by element-wise multiplication to achieve adaptive control of embedding strength in different regions. The modulated watermark latent feature and the latent feature representation of the edited image are additively fused to obtain a watermark-edited image latent feature representation.

[0043] Step 103: input the watermark-edited image latent feature representation generated in step 102 into the image decoder to decode it into a watermark-edited image, and then input the watermark-edited image into the watermark decoder to extract the embedded watermark grayscale image and recover the complete provenance metadata information based on the watermark grayscale image;

[0044] The watermark-edited image maintains high visual consistency with the original edited image. The provenance metadata information includes source image identification, editing instruction, user identity and time information, thereby realizing the traceability and identity authentication of the edited content.

[0045] Step 104: construct a joint loss function to train the watermark encoder, watermark decoder and attention-guided progressive network end-to-end.

[0046] The joint loss function contains three parts: visual quality loss to ensure imperceptibility of watermark embedding; watermark recoverability loss to ensure reliable extraction of watermark information; and smooth regularization loss to prevent visual artifacts caused by sudden changes in embedding strength. By minimizing the weighted joint loss function, the parameters of the three trainable modules are optimized while keeping the parameters of the image editing model fixed, realizing the plug-and-play feature of the watermark framework.

[0047] In summary, the embodiment of the present application completes the training of the watermark framework through the above steps 101-104, realizes the spatial adaptive plug-and-play watermark method for image editing traceability, completes the reliable embedding and extraction of high-capacity watermark under the premise of maintaining the quality of the edited image, provides complete traceability for the image editing scene, and meets the needs of copyright protection, safety supervision and responsibility attribution in actual application.

[0048] Embodiment 2

[0049] The scheme in embodiment 1 will be further introduced in combination with specific calculation formulas, and details are described below:

[0050] 201: For the input original image and editing condition, obtain the latent space feature representation of the edited image through the image editing model based on the Stable Diffusion architecture; at the same time, construct a structured watermark grayscale image for carrying the core information of image editing traceability, and encode it into a watermark latent feature through a watermark encoder;

[0051] The watermark grayscale image is composed of multi-dimensional metadata fields such as source image identifier, editing instruction, user identity and time information, adopts block encoding design to improve the watermark capacity and robustness, and ensures the security and uniqueness of the watermark information through a hash encryption mechanism.

[0052] Given the input original image x , editing condition c, the latent space encoder of the image editing model based on the Stable Diffusion architecture obtains the latent space feature representation of the edited image:

[0053]

[0054] Where H is the height of the input image, and W is the width of the input image.

[0055] At the same time of obtaining the latent space feature representation of the edited image, a structured watermark grayscale image is constructed and encoded, where N is the resolution of the watermark grayscale image, i.e. the width and height of the watermark grayscale image. For the traceability requirement of the image editing task, the watermark grayscale image is used to carry multi-dimensional metadata information to support a complete responsibility traceability chain. Unlike the binary watermark of the generation model [2] , the structured watermark grayscale image of the embodiment of the present application realizes comprehensive traceability of the editing process through the synergistic effect of the following four types of core metadata:

[0056] (1) Source image identifier: find the original image edited and modified, which can be used to clearly define the responsibility attribution and accurately bind the edited source image;

[0057] (2) Edit instruction: record modification intention, distinguish legal creative editing and illegal operation;

[0058] (3) User identity: realize accountability and attribution;

[0059] (4) Time information: provide time evidence for copyright tracking and dispute resolution. These fields together form a complete traceability chain, which can accurately locate which image is edited, who edits the image, how to modify and when to modify.

[0060] Specifically, the embodiment of the present application designs a watermark image as a watermark grayscale image with a size of , which is divided into non-overlapping blocks with a size of . Each block encodes a character, so that multiple metadata fields such as source image identification, edit instruction, user identity and time information can be stored in a single structured watermark grayscale image. Formally, the watermark grayscale image can be represented as:

[0061]

[0062] where each corresponds to a character in the b b image block. Since each character carries 8 bits of information, the effective watermark capacity can be represented as:

[0063]

[0064] In the embodiment of the present application, , the watermark capacity reaches:

[0065]

[0066] Compared with the capacity of traditional generative model watermarking methods (usually 32–1024 bits), the capacity of the embodiment of the present application is increased by 32 - 1024 times. This design enables a single watermark image to store complete traceability metadata without worrying about capacity limitations. In addition, the block design of the watermark grayscale image can realize regional storage of information. Even if some local areas are damaged during editing or attack, other areas can still retain information, unlike binary strings that fail to decode due to local errors, which significantly improves the robustness of watermark decoding.

[0067] Finally, the structured watermark grayscale image is encoded into watermark latent features by a watermark encoder :

[0068]

[0069] 202: Achieving content- and structure-aware watermark embedding in latent space;

[0070] Specifically, the latent space feature representation of the edited image is input into an attention-guided progressive network to generate a spatially adaptive watermark embedding intensity matrix. The latent features of the watermark are intensity modulated in the latent space and additively fused with the latent space feature representation of the edited image to obtain the latent space feature representation of the watermarked edited image.

[0071] In obtaining the latent space feature representation of the edited image With watermark potential features Subsequently, embodiments of the present invention execute a content- and structure-aware watermark embedding strategy in the latent space, achieving robust and imperceptible watermark embedding by maintaining semantic content consistency and adaptive fusion of spatial structure.

[0072] While existing methods have demonstrated the feasibility of watermark fusion in latent space, these methods typically rely on fixed watermark perturbation allocation, making it difficult to balance image quality and watermark robustness under complex editing conditions. They are also prone to producing visible artifacts or degrading watermark recovery performance. To address this issue, this invention proposes an attention-guided progressive network (APNet) for implementing content- and structure-aware adaptive watermark embedding in latent space. Its core innovation lies in its ability to autonomously discover the optimal watermark embedding pattern by learning the feature distribution in latent space, without explicit supervision of texture richness or edited regions.

[0073] APNet employs a U-Net backbone architecture combined with a multi-head attention mechanism. The encoder progressively moves from latent space feature representations... Semantic and geometric information is extracted from the data, while the multi-head attention layer facilitates information interaction between global semantics and local details, thereby helping the model establish an effective connection between "what to embed" and "where to embed". The decoder reconstructs these fused features into an adaptive intensity matrix. It is used to control the watermark embedding intensity at various locations in the latent space.

[0074] Given the latent space feature representation of the edited image With watermark potential features The embedding process can be represented as:

[0075]

[0076] Among them, the symbol " "Indicates element-wise modulation operation, The parameter is The attention-guided progressive network (APNet) whose output is activated by the Sigmoid function. After normalization, it is the adaptive intensity matrix: for regulating the embedding intensity of the watermark latent feature.

[0077] Through this embedding strategy, APNet can adaptively allocate watermark embedding energy in the latent space, achieving a spatial optimal distribution of "high-frequency enhancement and low-frequency suppression". In summary, this strategy realizes dual perception of content and structure in the embedding process. Specifically, content perception focuses on maintaining the semantic consistency and perceptual quality of the image. By limiting the disturbance amplitude, the watermark signal is constrained within the perceptually acceptable range, thus completing watermark embedding without compromising semantic authenticity and visual naturalness. Relatively, structure perception emphasizes the spatial organization and geometric stability in the latent space. By analyzing the distribution structure of the latent space feature, APNet can adaptively adjust the embedding intensity between different regions, so that the texture edges or high-frequency regions generated during editing carry stronger watermark signals, while maintaining lower embedding intensity in smooth regions.

[0078] 203: input the watermark-ed edited image latent space feature representation generated in step 202 into the image decoder to decode it into a watermark-ed edited image; input the watermark-ed edited image into the watermark decoder to extract the embedded watermark grayscale image, recover the complete traceability metadata information, including: source image identifier, editing instruction, user identity and time information, thereby realizing the traceability and identity authentication of the edited content.

[0079] After watermark embedding is completed, to realize the recovery and verification of traceability information, the embodiment of the present application inputs the watermark-ed edited image latent space feature representation to the image decoder of the editing model to decode it from the compressed latent space back to the high-resolution image space:

[0080]

[0081]

[0082] wherein, represents the final output watermark-ed edited image. The image decoder reconstructs the latent representation of dimension step by step into ​​RGB image. Since the watermark is embedded in the latent space and modulated by an adaptive intensity matrix, the decoded watermarked edited image is visually consistent with the original edited image without watermark. Experimental results show that the PSNR of the decoded edited image is more than 35 dB, the SSIM is more than 98%, and the LPIPS is less than , ensuring the imperceptibility of the watermark and the preservation of the editing quality.

[0083] Subsequently, the watermarked edited image is input into a watermark decoder , which is used to extract the embedded structured watermark grayscale image from the image and recover the provenance metadata information. The watermark decoder amplifies and inverts the watermark signal in the image by combining the convolution feature extraction and the deconvolution reconstruction module, so as to reconstruct the watermark grayscale image at the output end:

[0084]

[0085] wherein, is the watermark grayscale image recovered from the watermarked edited image. By performing hash check and field analysis on the reconstructed watermark grayscale image , the complete provenance metadata information can be recovered, including: source image identifier, editing instruction, user identity and time information. The metadata can be compared with the information recorded by the generation platform to confirm the image source, verify the legality of the editing operation, and trace the responsible subject, realizing the full-link traceability from the generated content to the editing behavior.

[0086] 204: An end-to-end training is performed on the watermark encoder, the watermark decoder and the attention-guided progressive network by constructing a joint loss function, the joint loss function including three parts: a visual quality loss ensuring the imperceptibility of the watermark embedding; a watermark recoverability loss ensuring the reliable extraction of the watermark information; and a smooth regularization loss preventing visual artifacts caused by the mutation of the embedding strength; by minimizing the weighted joint loss function, the parameters of the three trainable modules are optimized while the parameters of the image editing model are kept fixed, realizing the plug-and-play feature of the watermark framework.

[0087] To balance the visual quality, watermark recoverability and watermark robustness, the watermark embedding process is modeled as a multi-objective optimization problem in the embodiments of the present application, and the watermark encoder, the decoder and the attention-guided progressive network (APNet) are jointly trained. Under the fixed embedding strength, the model first establishes a stable watermark encoder-decoder cycle to ensure the image quality and the watermark decodability; subsequently, the APNet is introduced to predict an adaptive intensity matrix, which dynamically adjusts the embedding strength according to the latent features, so as to achieve a better balance between the visual quality and the robustness.

[0088] The embodiments of the present application introduce three loss components to jointly guide the optimization. First, the visual quality loss limits the perceptual difference between the watermarked edited image and the original edited image , ensuring that watermark embedding does not introduce visible artifacts. This loss function combines pixel-level reconstruction and perceptual consistency, and is specifically formulated as:

[0089]

[0090] where is the perceptual loss computed by a pre-trained feature extractor, which is used to capture high-level semantic similarity.

[0091] Second, the watermark recoverability loss is used to ensure that the watermark can be stably recovered in the decoder. Specifically, the recovered watermark grayscale image is consistent with the original watermark grayscale image, and is specifically formulated as:

[0092]

[0093] The smoothness regularization loss is imposed on the adaptive intensity matrix . Since abrupt changes in embedding intensity can create perceptible edges, the local discontinuities in the adaptive intensity matrix are penalized, and are specifically formulated as:

[0094]

[0095] Finally, the overall training objective integrates the three components into a unified optimization problem:

[0096]

[0097] where , controls the trade-off between watermark recoverability and visual imperceptibility. Through this joint optimization, the embodiments of the present application not only learn to implicitly embed provenance information, but also adaptively adjust the embedding intensity of spatial consistency, ensuring that even under perturbations, the watermark signal can still be recovered.

[0098] In the watermark embedding strategy, a progressive network (APNet) based on a U-Net backbone architecture and integrating multi-head self-attention mechanism is used to realize watermark embedding in the latent space. The core of the strategy is that the APNet takes the latent feature representation of the edited image as input, and through its forward propagation, it learns and predicts a spatially adaptive intensity matrix. The matrix is normalized by the Sigmoid function and used to modulate the watermark latent features element by element. Finally, the embedding is completed through additive fusion. In this process, content-aware is embodied by using the Sigmoid function to globally constrain the embedding strength in the range of [0, 1], and combining with the visual quality loss to jointly limit the amplitude of watermark disturbance, thereby ensuring semantic consistency and visual naturalness; structure-aware is embodied by APNet analyzing the latent feature distribution autonomously, so that the trained network can spontaneously allocate higher intensity in the high-frequency editing region with complex texture, and suppress the intensity in the smooth low-frequency background region, realizing the spatial optimal allocation of embedding energy.

[0099] In summary, the embodiment of the present application completes the training of the watermark framework through the above steps 201-204, realizes the spatially adaptive plug-and-play watermarking method for image editing traceability, and realizes reliable embedding and extraction of high-capacity watermarks under the premise of maintaining the quality of edited images, providing complete traceability for image editing scenarios, and meeting the needs of copyright protection, safety supervision and responsibility attribution in actual applications.

[0100] Embodiment 3

[0101] The feasibility of embodiments 1 and 2 is verified by specific experiments as follows:

[0102] The embodiment of the present application is verified by experiments on two public data sets: InstructPix2Pix data set and MagicBrush data set. 39,553 triplets (original image, editing instruction, edited image) are selected from the InstructPix2Pix data set as the training set, and 7,650 triplets are selected as the test set; 6,000 triplets are selected from the MagicBrush data set as an additional test set for evaluating the generalization ability across data sets. To verify the universality of the method, it is integrated into five representative image editing models based on the Stable Diffusion architecture: InstructPix2Pix, HIVE, InstructCV, MagicBrush and PFB-diff, although these models differ in editing mechanism and input modality, the embodiment of the present application embeds and decodes the watermark directly in the latent space of the editing output without any modification to the original editing process, ensuring the compatibility of plug-and-play.

[0103] The embodiments of the present application evaluate performance from three dimensions, including: image quality, watermark recoverability, distribution consistency. For image quality, peak signal-to-noise ratio (PSNR), structural similarity (SSIM), perceptual image block similarity (LPIPS) and CLIPScore are used to evaluate the pixel-level fidelity, structural consistency, perceptual similarity and semantic alignment of the watermark image. For the consistency of the distribution of the watermark image, the FID, which measures the offset effect of watermark embedding on the distribution of the edited image, and further reports p FID, which measures the offset effect of watermark embedding on the distribution of the edited image, and further reports p FID, which measures the offset effect of watermark embedding on the distribution of the edited image, and further reports p. For watermark recoverability, normalized correlation (NC), character account (CA) and character edit ratio (CER) are used to jointly measure the robustness of the watermark at the pixel level, character level and information recovery rate.

[0104] Embodiment 4

[0105] An image editing traceability-oriented spatial adaptive plug-and-play watermarking device, the device comprising: a memory storing program instructions, a processor calling the program instructions stored in the memory to make the device execute the following method steps in embodiment 1:

[0106] For the input original image and editing conditions, the latent space feature representation of the edited image is obtained through the image editing model; a structured watermark grayscale image for carrying the core information of image editing traceability is constructed, and the structured watermark grayscale image is encoded into a watermark latent feature through a watermark encoder;

[0107] The latent space feature representation of the edited image is input into the attention-guided progressive network, and based on the watermark latent feature, the watermark embedding is realized in the latent space with content and structure perception, and the watermark-edited image latent space feature representation is obtained;

[0108] The watermark-edited image latent space feature representation is input into the image decoder to be decoded into the watermark-edited image, and the watermark-edited image is input into the watermark decoder to extract the embedded watermark grayscale image, and the complete traceability metadata information is recovered based on the watermark grayscale image.

[0109] Wherein, the device comprises: constructing a joint loss function to train the watermark encoder, the watermark decoder and the attention-guided progressive network end-to-end; the joint loss function includes: a visual quality loss for ensuring the imperceptibility of watermark embedding; a watermark recoverability loss for ensuring the reliable extraction of watermark information; a smooth regularization loss for preventing visual artifacts caused by sudden changes in embedding strength.

[0110] The structured watermark grayscale image is composed of a source image identifier, an editing instruction, a user identity, and time information, is designed by block coding, and security and uniqueness of the watermark information are ensured by a hash encryption mechanism.

[0111] The structured watermark grayscale image for carrying image editing traceability core information is specifically:

[0112] The watermark image is designed as a watermark grayscale image with a size of The structured watermark grayscale image is constructed and coded, N is the resolution of the watermark grayscale image, that is, the width and height of the watermark grayscale image, the watermark grayscale image is used to carry multi-dimensional metadata information, and the structured watermark grayscale image realizes comprehensive traceability of the editing process through the synergistic effect of four types of core metadata.

[0113] The watermark grayscale image is divided into non-overlapping blocks with a size of Each non-overlapping block encodes a character, a plurality of metadata fields are stored in a single structured watermark image, and the watermark grayscale image is represented as:

[0114]

[0115] Each corresponds to a character in the b b image block, and the effective watermark capacity is represented as:

[0116] .

[0117] The watermark embedding realizing content and structure perception in the latent space is:

[0118] The latent space feature representation of the edited image is input into the attention-guided progressive network to generate a spatially adaptive watermark embedding strength matrix, the watermark latent feature is intensity-modulated in the latent space, and is additively fused with the latent space feature representation of the edited image to obtain a watermark-edited image latent space feature representation;

[0119] The watermark embedding strategy realizing content and structure perception in the latent space is executed, semantic content consistency and spatial structure adaptive fusion are maintained, and watermark embedding is realized.

[0120] The attention-guided progressive network is used to realize adaptive watermark embedding with content and structure perception capability in the latent space;

[0121] By learning the feature distribution in the latent space, an optimal watermark embedding mode is obtained;

[0122] Given the edited image latent space feature representation and the watermark latent feature The embedding process is represented as:

[0123]

[0124] where the symbol '' denotes an element-wise modulation operation, denotes an attention guided progressive network with parameters whose output is normalized by a sigmoid activation function to obtain an adaptive intensity matrix that is used to regulate the embedding strength of the watermark feature, is a latent space encoder, is a watermark encoder, and W is a structured watermark grayscale image, is the watermarked edited image latent space feature representation, is an image editing model, and c is an editing condition and x is an original image.

[0125] It should be noted that the apparatus described in the above embodiments corresponds to the method described in the embodiments, and the embodiments of the present application will not be described here.

[0126] The execution subject of the processor and the memory described above can be a computer, a single-chip microcomputer, a microcontroller, or other devices with computing functions. In actual implementation, the embodiments of the present application do not limit the execution subject, and the execution subject is selected according to actual application needs.

[0127] The memory and the processor transmit data signals through a bus, and the embodiments of the present application will not be described here.

[0128] Based on the same inventive concept, the embodiments of the present application also provide a computer readable storage medium. The storage medium includes a stored program. When the program runs, the device where the storage medium is located executes the method steps in the above embodiments.

[0129] The computer readable storage medium includes, but is not limited to, a flash memory, a hard disk, a solid state disk, and the like.

[0130] It should be noted that the readable storage medium described in the above embodiments corresponds to the method described in the embodiments, and the embodiments of the present application will not be described here.

[0131] In the above embodiments, all or part of the embodiments can be realized by software, hardware, firmware, or any combination thereof. When realized by software, all or part of the embodiments can be realized in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions according to the embodiments of the present application are generated.

[0132] ​The computer can be a general purpose computer, a special purpose computer, a computer network, or other programmable apparatus. Computer instructions can be stored in or transmitted by a computer-readable storage medium, such as a magnetic or semiconductor memory, etc.

[0133] Embodiment 5

[0134] A spatial adaptive plug-and-play watermarking method for image editing provenance, a robustness test result graph, see Figure 3 ,

[0135] In order to systematically evaluate the robustness of the watermark, the embodiments of the present application tested the performance under six common perturbations on the InstructPix2Pix dataset, including: Gaussian blur, random cropping, brightness adjustment, Gaussian noise, salt and pepper noise and rotation transformation. The result graph shows that the embodiments of the present application have very strong robustness to Gaussian blur, brightness adjustment and salt and pepper noise, and the NC value remains above 0.9; in contrast, Gaussian noise and random cropping have more obvious impact on watermark recovery, as the intensity or cropping size increases, the NC slowly decreases and the CER slightly rises, but the overall is still in an acceptable range. Rotation has the most significant impact on the watermark, with NC dropping to about 0.82 and CER rising to 0.35 at larger angles, indicating that spatial geometric perturbation has the greatest impact on watermark recoverability. In summary, the embodiments of the present application show good robustness under most common attacks, and only have certain performance decline under extreme geometric perturbation conditions.

[0136] The model of each device in the embodiments of the present application is not limited unless otherwise specified, and any device that can complete the above functions can be used.

[0137] Those skilled in the art can understand that the accompanying drawings are only schematic diagrams of a preferred embodiment, and the above-mentioned serial numbers of the embodiments of the present application are only for description, not representing the advantages and disadvantages of the embodiments.

[0138] The above is only a preferred embodiment of the present application, and does not limit the present application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principles of the present application shall be included in the protection scope of the present application.

Claims

1. A spatially adaptive plug-and-play watermarking method for image editing provenance, characterized in that, The method comprises: For the input original image and the editing condition, obtain the latent space feature representation of the edited image through the image editing model; construct a structured watermark grayscale image for carrying the image editing traceability core information, and encode the structured watermark grayscale image into a watermark latent feature through a watermark encoder; The structured watermark grayscale image for carrying the image editing traceability core information is specifically constructed as follows: The watermark image is designed to be a structured watermark gray image of a size of The structured watermark gray image is constructed by using a watermark gray image of a size of and is encoded, N is a resolution of the watermark gray image, that is, a width and a height of the watermark gray image, and the watermark gray image is used to carry multi-dimensional metadata information. The method divides the watermark grayscale image into non-overlapping blocks of size Each non-overlapping block encodes a character, and multiple metadata fields are stored in a single structured watermark grayscale image, which is represented as: ； wherein each corresponding to b The effective watermark capacity for one character in the b image block is represented as: ; input the latent space feature representation of the edited image into the attention-guided progressive network, realize watermark embedding in the latent space based on the watermark latent feature, and obtain the latent space feature representation of the image with watermark; The watermark embedding with content and structure perception in the latent space is as follows: Input the latent space feature representation of the edited image into an attention-guided progressive network to generate a spatially adaptive watermark embedding strength matrix, modulate the strength of the watermark latent feature in the latent space, and additively fuse the latent space feature of the edited image to obtain a watermark-edited image latent space feature representation; Perform a content and structure perception-aware watermark embedding strategy in the latent space; The attention-guided progressive network is used for adaptive watermark embedding with content and structure perception capability in the latent space; By learning the feature distribution in the latent space, the optimal watermark embedding mode is obtained; Given edited image latent space representation With watermark latent feature The embedding process is represented as: ； where the symbol represents an element-wise modulation operation, represents an attention-guided progressive network with parameters whose output is normalized by a Sigmoid activation function to obtain an adaptive intensity matrix that is used to regulate the embedding strength of the watermark feature, is a latent space encoder, is a watermark encoder, and W is a structured watermark grayscale image, is a watermarked edited image latent space feature representation, is an image editing model, c is an editing condition, and x is an original image. Input the watermark-edited image latent space feature representation into an image decoder to decode it into a watermark-edited image, input the watermark-edited image into a watermark decoder to extract the embedded watermark grayscale image, and recover the complete traceability metadata information based on the watermark grayscale image; The method comprises: constructing a joint loss function to perform end-to-end training on the watermark encoder, the watermark decoder, and the attention-guided progressive network; the joint loss function includes: a visual quality loss for ensuring imperceptibility of watermark embedding; a watermark recoverability loss for ensuring reliable extraction of watermark information; and a smooth regularization loss for preventing visual artifacts caused by sudden changes in embedding strength.

2. The spatial adaptive plug-and-play watermarking method for image editing provenance according to claim 1, wherein, The structured watermark grayscale image is composed of a source icon, editing instructions, user identity, and time information, adopts block encoding design, and ensures the security and uniqueness of the watermark information through a hash encryption mechanism.

3. A spatially adaptive plug-and-play watermarking apparatus oriented towards image editing provenance, characterized in that, The device comprises a processor and a memory, the memory stores program instructions, and the processor invokes the program instructions stored in the memory to enable the device to perform the method of any one of claims 1-2.

4. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program, the computer program comprises program instructions, and the program instructions are executed by the processor to enable the processor to perform the method of any one of claims 1-2.

Images