Diffusion model oriented generative image watermarking processing method

Abstract

The application provides a diffusion model-oriented generative image watermark processing method. It relates to the fields of artificial intelligence and information security technology, and in particular to the fields of artificial intelligence generated content and digital watermark technology. The method comprises the following steps: encoding watermark information to obtain a bit sequence of the watermark information; determining a dual-purpose key based on the bit sequence mapping and a preset shared redundant bit string; converting the shared redundant bit string into initial latent noise by pseudo-random scrambling and distribution-preserving sampling based on the dual-purpose key; and inputting the initial latent noise into a diffusion model to generate a target watermark image embedded with the watermark information in combination with a generation condition.

Description

Technical Field

[0001] This invention relates to the fields of artificial intelligence and information security, and more particularly to the fields of Artificial Intelligence Generated Content (AIGC) and digital watermarking. More specifically, it relates to a generative image watermarking processing method oriented towards a diffusion model. Background Technology

[0002] Generative techniques based on diffusion models, such as Stable Diffusion and Imagen, have made groundbreaking progress, enabling the large-scale generation of photorealistic synthetic images and demonstrating significant application value in creative design and digital content production. However, the widespread application of these technologies has also raised a series of security issues, including content authenticity verification, copyright ownership confirmation, and content traceability. Reliable watermarking technology has become a key solution to ensure the traceability of content generated by diffusion models.

[0003] In the process of realizing the concept of this invention, the inventors discovered that there are at least the following problems in the related technologies: traditional watermarking schemes are difficult to balance between robustness and capacity. Summary of the Invention

[0004] In view of this, the present invention provides a generative image watermarking processing method oriented towards a diffusion model.

[0005] One aspect of the present invention provides a generative image watermarking method for a diffusion model, comprising: encoding watermark information to obtain a bit sequence of watermark information; determining a dual-purpose key based on the bit sequence and a preset shared redundant bit string; converting the shared redundant bit string into initial latent noise based on the dual-purpose key through pseudo-random shuffling and distribution-maintaining sampling; inputting the initial latent noise into a diffusion model and, in combination with generation conditions, generating a target watermark image with embedded watermark information.

[0006] Another aspect of the present invention provides a generative image watermarking processing apparatus for a diffusion model, comprising: a watermark information encoding module for encoding watermark information to obtain a bit sequence of watermark information; a dual-purpose key determination module for determining a dual-purpose key based on the bit sequence and a preset shared redundant bit string; a shared redundant bit string conversion module for converting the shared redundant bit string into initial latent noise based on the dual-purpose key through pseudo-random shuffling and distribution-maintaining sampling; and a target watermark image generation module for inputting the initial latent noise into the diffusion model and, in combination with generation conditions, generating a target watermark image with embedded watermark information.

[0007] Another aspect of the present invention provides an electronic device comprising: one or more processors; and a memory for storing one or more programs, wherein, when the one or more programs are executed by the one or more processors, the one or more processors implement the diffusion-oriented generative image watermarking method of the present invention.

[0008] Another aspect of the present invention provides a computer-readable storage medium storing computer-executable instructions, which, when executed, are used to implement the generative image watermarking method of the present invention based on a diffusion-oriented model.

[0009] Another aspect of the present invention provides a computer program product comprising computer-executable instructions, which, when executed, implement the diffusion-oriented generative image watermarking processing method of the present invention.

[0010] According to the technical solution of the present invention, a key-guided high-capacity embedding mechanism is adopted. Redundant patterns are encoded into the bit space itself through a dual-purpose key, so that latent noise variables simultaneously carry the effective payload and redundant information, avoiding explicit bit repetition. Without reducing robustness, the watermark capacity is increased from 256 bits in the prior art to more than 10,000 bits, which enhances the robustness to common distortions such as compression, cropping, and rotation, without affecting the image generation quality. It can carry multi-dimensional fine-grained traceability metadata such as model identifier, user identity, generation instructions, and timestamps, and is widely used in the traceability and copyright protection of artificial intelligence generated images. Attached Figure Description

[0011] The above and other objects, features and advantages of the present invention will become more apparent from the following description of embodiments of the invention with reference to the accompanying drawings, in which:

[0012] Figure 1 An exemplary system architecture for applying a diffusion-oriented generative image watermarking method according to an embodiment of the present invention is shown;

[0013] Figure 2 A flowchart of a generative image watermarking embedding method based on a diffusion-oriented model according to an embodiment of the present invention is shown.

[0014] Figure 3 A flowchart of a generative image watermark extraction method based on a diffusion-oriented model according to an embodiment of the present invention is shown.

[0015] Figure 4A A schematic diagram illustrating the overall process of generative image watermark embedding and extraction based on a diffusion-oriented model according to an embodiment of the present invention is shown.

[0016] Figure 4BA schematic diagram illustrating correction based on DIRN according to an embodiment of the present invention is shown;

[0017] Figure 5 The robustness test results of the generative image watermarking method based on the diffusion model according to an embodiment of the present invention are shown in the figure.

[0018] Figure 6 A block diagram of a generative image watermarking processing apparatus based on a diffusion-oriented model according to an embodiment of the present invention is shown.

[0019] Figure 7 A block diagram of an electronic device suitable for implementing the above-described generative image watermarking method based on a diffusion-oriented model, according to an embodiment of the present invention, is shown. Detailed Implementation

[0020] Hereinafter, embodiments of the present invention will be described with reference to the accompanying drawings. However, it should be understood that these descriptions are exemplary only and are not intended to limit the scope of the invention. In the following detailed description, numerous specific details are set forth to provide a thorough understanding of the embodiments of the invention for ease of explanation. However, it will be apparent that one or more embodiments may be practiced without these specific details. Furthermore, descriptions of well-known structures and techniques are omitted in the following description to avoid unnecessarily obscuring the concept of the invention.

[0021] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the invention. The terms “comprising,” “including,” etc., as used herein indicate the presence of the stated features, steps, operations, and / or components, but do not exclude the presence or addition of one or more other features, steps, operations, or components.

[0022] All terms used herein (including technical and scientific terms) have the meanings commonly understood by those skilled in the art, unless otherwise defined. It should be noted that the terms used herein are to be interpreted in a manner consistent with the context of this specification, and not in an idealized or overly rigid way.

[0023] When using expressions such as "at least one of A, B and C", they should generally be interpreted in accordance with the meaning that is commonly understood by those skilled in the art (e.g., "a system having at least one of A, B and C" should include, but is not limited to, a system having A alone, a system having B alone, a system having C alone, a system having A and B, a system having A and C, a system having B and C, and / or a system having A, B and C, etc.).

[0024] In the embodiments of this invention, the collection, updating, analysis, processing, use, transmission, provision, disclosure, and storage of data (e.g., including but not limited to user personal information) comply with relevant laws and regulations, are used for legitimate purposes, and do not violate public order and good morals. In particular, necessary measures have been taken to prevent unauthorized access to user personal information data and maintain information security.

[0025] In the embodiments of the present invention, the user's authorization or consent is obtained before acquiring or collecting the user's personal information.

[0026] In related technologies, diffusion model watermarking methods are mainly divided into three categories: post-generation watermarking methods, in-process watermarking methods, and inversion-based watermarking methods. Post-generation watermarking methods embed the watermark signal after image generation, which is simple to deploy but can easily lead to a decrease in visual quality. In-process watermarking methods integrate the watermark target into the diffusion model training process, offering strong robustness but requiring access to model weights, resulting in high computational overhead and poor generalization. Inversion-based watermarking methods embed watermark information into the initial latent noise and subsequently recover the watermark through diffusion inversion, combining high fidelity and invisibility, making it a highly promising technical approach.

[0027] However, inversion-based watermarking methods face core technical bottlenecks in generative image scenarios, primarily due to two main issues: First, the inherent contradiction between capacity and robustness. Related methods typically introduce redundancy by repeating watermark bits to ensure recovery stability, severely limiting the watermark capacity and making it difficult to accommodate the multi-dimensional metadata required for fine-grained traceability. Second, insufficient error compensation during inversion. The inverse transform of the diffusion model is susceptible to external image distortion (such as compression and cropping) and internal iterative errors. The superposition of errors from external image distortion and internal iteration leads to a deviation between the recovered latent representation and the original embedded latent representation. Related methods lack effective error correction mechanisms, resulting in a significant decrease in watermark recovery accuracy under complex distortion scenarios.

[0028] Therefore, the present invention aims to overcome the limitations of related technologies and provide a diffusion model generative image watermarking scheme that takes into account high capacity, strong robustness and plug-and-play characteristics.

[0029] This invention provides a generative image watermarking method for a diffusion model. Specifically, the method may include at least one of a watermark embedding process and a watermark extraction process. For the watermark embedding process, the watermark information can be encoded to obtain a bit sequence of the watermark information; a dual-purpose key is determined based on the bit sequence and a preset shared redundant bit string; based on the dual-purpose key, the shared redundant bit string is converted into initial latent noise through pseudo-random shuffling and distribution-maintaining sampling; the initial latent noise is input into a diffusion model and combined with the text description of the image to be generated to generate a target watermark image with embedded watermark information, wherein the image content of the target watermark image is adapted to the content of the text description. For the watermark extraction process, a distorted watermark image can be inversely transformed to obtain distorted latent noise, wherein the distorted watermark image is obtained by distorting the target watermark image; the distorted latent noise is input into a distortion-aware inverse correction network to obtain corrected latent noise; based on the dual-purpose key, the corrected latent noise is decoded to extract the watermark information. Based on these two processes, the system addresses the two core issues mentioned above, ensuring reliable embedding and extraction of high-capacity watermarks in complex distortion environments while maintaining high visual quality of images. This provides a practical and feasible technical solution for copyright protection and secure traceability of AIGC content.

[0030] Figure 1 An exemplary system architecture for applying a diffusion-oriented generative image watermarking method according to an embodiment of the present invention is shown.

[0031] It is important to note that Figure 1 The examples shown are merely examples of system architectures that can be applied to embodiments of the present invention, in order to help those skilled in the art understand the technical content of the present invention, but do not mean that embodiments of the present invention cannot be used in other devices, systems, environments or scenarios.

[0032] like Figure 1 As shown, the system architecture 100 according to this embodiment may include a first terminal device 101, a second terminal device 102, a third terminal device 103, a network 104, and a server 105. The network 104 serves as a medium for providing communication links between the first terminal device 101, the second terminal device 102, the third terminal device 103, and the server 105. The network 104 may include various connection types, such as wired and / or wireless communication links, etc.

[0033] Users can use the first terminal device 101, the second terminal device 102, and the third terminal device 103 to interact with the server 105 via the network 104 to receive or send messages, etc. Various communication client applications can be installed on the first terminal device 101, the second terminal device 102, and the third terminal device 103, such as shopping applications, web browser applications, search applications, instant messaging tools, email clients, and / or social media platform software, etc. (for example only).

[0034] The first terminal device 101, the second terminal device 102, and the third terminal device 103 can be various electronic devices with displays and support web browsing, including but not limited to smartphones, tablets, laptops, and desktop computers.

[0035] Server 105 can be a server that provides various services, such as a backend management server that supports websites browsed by users using the first terminal device 101, the second terminal device 102, and the third terminal device 103 (this is just an example). The backend management server can analyze and process data such as received user requests, and feed back the processing results (such as web pages, information, or data obtained or generated according to user requests) to the terminal devices.

[0036] It should be noted that the generative image watermarking method for diffusion-oriented models provided in this embodiment of the invention can generally be executed by server 105. Correspondingly, the generative image watermarking device for diffusion-oriented models provided in this embodiment of the invention can generally be located in server 105. The generative image watermarking method for diffusion-oriented models provided in this embodiment of the invention can also be executed by a server or server cluster that is different from server 105 and capable of communicating with the first terminal device 101, the second terminal device 102, the third terminal device 103, and / or server 105. Correspondingly, the generative image watermarking device for diffusion-oriented models provided in this embodiment of the invention can also be located in a server or server cluster that is different from server 105 and capable of communicating with the first terminal device 101, the second terminal device 102, the third terminal device 103, and / or server 105. Alternatively, the generative image watermarking method for diffusion models provided in this embodiment of the invention can also be executed by the first terminal device 101, the second terminal device 102, or the third terminal device 103, or by other terminal devices different from the first terminal device 101, the second terminal device 102, or the third terminal device 103. Correspondingly, the generative image watermarking apparatus for diffusion models provided in this embodiment of the invention can also be disposed in the first terminal device 101, the second terminal device 102, or the third terminal device 103, or in other terminal devices different from the first terminal device 101, the second terminal device 102, or the third terminal device 103.

[0037] For example, the watermark information and text description, and / or the distorted watermark image may originally be stored in any one of the first terminal device 101, the second terminal device 102, or the third terminal device 103 (e.g., the first terminal device 101, but not limited thereto), or stored on an external storage device and can be imported into the first terminal device 101. Then, the first terminal device 101 may locally execute the generative image watermarking method for the diffusion model provided in this embodiment of the invention, or send the watermark information and text description, and / or the distorted watermark image to other terminal devices, servers, or server clusters, and have the other terminal devices, servers, or server clusters that receive the watermark information and text description, and / or the distorted watermark image execute the generative image watermarking method for the diffusion model provided in this embodiment of the invention.

[0038] It should be understood that Figure 1 The number of terminal devices, networks, and servers shown is merely illustrative. Depending on implementation needs, any number of terminal devices, networks, and servers can be included.

[0039] Figure 2A flowchart of a generative image watermarking embedding method based on a diffusion model according to an embodiment of the present invention is shown.

[0040] like Figure 2 As shown, the method includes operations S201 to S204.

[0041] In operation S201, the watermark information is encoded to obtain the bit sequence of the watermark information.

[0042] According to an embodiment of the present invention, a linear block error correction code (Bose-Chaudhuri-Hocquenghem, BCH) encoder can be used to encode the watermark information to obtain a bit sequence. In an embodiment of the present invention, the watermark information can be a 01 sequence.

[0043] In other scalable embodiments, the watermark information may also be a 01 sequence encoded from the source information with actual content, or the watermark information which is actually a 01 sequence may be associated with the source information with actual content to trace the source, without limitation.

[0044] In operation S202, a dual-purpose key is determined based on the bit sequence and a preset shared redundant bit string.

[0045] According to embodiments of the present invention, a dual-purpose key refers to a key used in both the watermark embedding process and the watermark extraction process. The shared redundant bit string can be obtained by copying a randomly generated bit string bit by bit.

[0046] According to an embodiment of the present invention, based on a preset shared redundant bit string r, the encoded bit sequence c can be mapped to a dual-purpose key key through a bit XOR operation, and the mapping relationship can be as shown in formula (1). This process can achieve structural redundancy without explicitly displaying the repeated watermark bits.

[0047] key=c⊕r(1)

[0048] In operation S203, based on the dual-purpose key, the shared redundant bit string is converted into initial potential noise through pseudo-random shuffling and distributed sampling.

[0049] According to an embodiment of the present invention, pseudo-random shuffling means reordering the shared redundant bit string according to pseudo-random numbers, shuffling the order but not changing the shared redundant bit string itself, only changing its position. Considering that the generation quality of the diffusion model is highly dependent on the distribution of the initial latent noise, the distribution-preserving sampling means that while embedding the watermark, the sampled initial latent noise still strictly follows the standard Gaussian distribution N(0,I), which is completely consistent with the distribution in the watermark-free scenario.

[0050] In operation S204, the initial potential noise is input into the diffusion model, and combined with the generation conditions, a target watermark image with embedded watermark information is generated.

[0051] According to embodiments of the present invention, the generation conditions may include at least one of the following: text conditions, image conditions, category / tag conditions, latent space conditions for controlling style, etc., and are not limited thereto.

[0052] According to an embodiment of the present invention, based on a diffusion model, starting from the initial potential noise, multiple rounds of iterative denoising can be performed under the guidance of the generation conditions to gradually optimize the initial potential noise into a target watermark image that meets the conditions.

[0053] Through the above embodiments of the present invention, a key-guided high-capacity embedding mechanism is adopted. Redundant patterns are encoded into the bit space itself through a dual-purpose key, so that latent noise variables simultaneously carry the payload and redundant information, avoiding explicit bit repetition. Without reducing robustness, the watermark capacity is increased from 256 bits in the prior art to more than 10,000 bits, enhancing robustness to common distortions such as compression, cropping, and rotation, without affecting the image generation quality. It can carry multi-dimensional fine-grained traceability metadata such as model identifier, user identity, generation instructions, and timestamps, and is widely used in the traceability and copyright protection of AI-generated images.

[0054] Figure 3 A flowchart of a generative image watermark extraction method based on a diffusion-oriented model according to an embodiment of the present invention is shown.

[0055] like Figure 3 As shown, the method includes operations S301 to S303.

[0056] In operation S301, an inverse transformation is performed on the distorted watermark image to obtain distorted potential noise. The distorted watermark image is obtained by distorting the target watermark image.

[0057] According to embodiments of the present invention, distortion processing may include at least one of the following: Gaussian blur, random cropping, random discarding, brightness adjustment, Gaussian noise, salt-and-pepper noise, median filtering, scaling, image compression, and geometric rotation. Image compression may include, but is not limited to, Joint Photographic Experts Group (JPEG) compression.

[0058] According to an embodiment of the present invention, by performing a forward diffusion inverse transform on the distorted watermark image through a diffusion encoder and a denoising diffusion implicit model (DDIM) inverse transform, noise is gradually added, and the distorted watermark image can be reconstructed into the corresponding distorted potential noise.

[0059] In operation S302, the distorted watermark image and the distorted potential noise are input into the distortion-aware inverse correction network to obtain the corrected potential noise.

[0060] According to embodiments of the present invention, the Distortion-Aware Inverse Rectification Network (DIRN) is a neural network model capable of sensing image distortion features and performing distortion correction or noise restoration through inverse transformation processing. It is used to improve the accuracy and fidelity of image generation, restoration, or inverse processing. DIRN comprises three core components: a distortion encoder, a dual supernetwork (including a geometric supernetwork and a pixel supernetwork), and a latent vector corrector. The output of the distortion encoder serves as the input to the dual supernetwork, and the output of the dual supernetwork serves as the input to the latent vector corrector.

[0061] According to embodiments of the present invention, a joint loss function can be constructed for end-to-end training of DIRN. The joint loss function includes Gaussian negative log-likelihood loss, prior loss, and metadata alignment loss.

[0062] The Gaussian negative log-likelihood loss is given by formula (2) to ensure the consistency between the corrected latent noise and the distorted latent noise.

[0063] (2);

[0064] In formula (2), The Gaussian negative log-likelihood loss is used. and The Gaussian parameters of the output are predicted by the latent vector corrector of DIRN. This represents the potential noise after correction, determined based on the mean of the DIRN correction output. It is the variance in logarithmic form of the DIRN-corrected output. This represents the potential noise caused by distortion. exp() is an exponential function with the natural constant e as its base.

[0065] The prior loss is as shown in Equation (3), which is used to avoid excessive offset of the output of the latent vector corrector and to constrain the latent noise to meet the standard normal distribution after correction.

[0066] (3);

[0067] In formula (3), This represents the prior loss. N represents the total number of samples participating in the calculation in the current batch. μ represents the mean of the DIRN corrected output, which can be used as the potential noise after correction. .

[0068] Metadata alignment loss, as shown in formula (4), is used to constrain the consistency between the true value and the predicted value of the distortion metafeature vector, so that the distortion metafeature vector learned by the network is aligned with the true distortion type, thereby improving the recognition accuracy of the distortion type. Here, the distortion metafeature vector is the vector representation of the distortion type.

[0069] (4);

[0070] In formula (4), Indicates metadata alignment loss, The predicted value of the distortion metafeature vector is generated by the distortion encoder and serves as the input to the distortion type gating. It determines whether the current distortion belongs to "geometric distortion" or "pixel distortion" to decide how the dual super network should process it. This represents the true value of the distortion metafeature vector. This true value is the distortion parameter recorded when distortion is randomly applied to the training image during the training process, and is obtained through normalization and tensor processing.

[0071] The joint loss function is shown in formula (5).

[0072] (5);

[0073] In formula (5), This indicates a combined loss. and This is the balance coefficient.

[0074] During training, a training dataset can be constructed by applying various representative distortions to 1000 clean images, such as random discarding, brightness adjustment, random cropping, Gaussian blur, Gaussian noise, median filtering, salt-and-pepper noise, scaling, and JPEG compression. An adaptive moment estimation optimizer is used, with a learning rate set to 3×10⁻⁶. -5 Training is completed on a single graphics processing unit (GPU).

[0075] After training, the DIRN parameters are frozen and only invoked during the watermark extraction stage. This method does not require training and fine-tuning of the diffusion model itself; it is plug-and-play and highly adaptable.

[0076] In operation S303, based on the dual-purpose key, the corrected potential noise is decoded to extract the watermark information.

[0077] According to an embodiment of the present invention, by decoding the corrected potential noise based on a dual-purpose key and then using a BCH decoder for decoding and error correction, the final extracted watermark information can be the recovered original embedded watermark information.

[0078] Through the above embodiments of the present invention, a distortion-aware inverse correction network (DIRN) is designed and integrated with BCH error correction coding. DIRN can adaptively identify the distortion type (geometric-dominated or pixel-dominated) and dynamically adjust the correction strategy, effectively compensating for the inversion error caused by external distortion and internal iteration. Combined with the multi-bit error correction capability of BCH coding, the robustness of watermark recovery in complex distortion scenarios is significantly improved, and high recovery accuracy is maintained under common perturbations such as Gaussian blur, random cropping, and JPEG compression.

[0079] The following, in conjunction with the accompanying drawings, describes... Figures 2-3 The generative image watermarking embedding and extraction method based on the diffusion model is further described in detail.

[0080] Figure 4A A schematic diagram illustrating the overall process of generative image watermark embedding and extraction based on a diffusion-oriented model according to an embodiment of the present invention is shown.

[0081] According to an embodiment of the present invention, corresponding to the above operation S201, the implementation method may include: dividing the watermark information into blocks according to the pre-encoding information bit length pre-configured in the error correction coding program to obtain multiple data blocks, each with a length equal to the pre-encoding information bit length, wherein the error correction coding program also pre-configures the post-encoding code length and the number of error-correctable bits; calling the error correction coding program to encode the data blocks to obtain a coding block with a length equal to the post-encoding code length and having error correction capability represented by the number of error-correctable bits; adding a preset number of auxiliary check bits to the coding block to obtain a bit sequence, wherein the total length of the bit sequence is equal to the total length of the shared redundant bit string.

[0082] According to an embodiment of the present invention, to improve error correction capability, the error correction coding procedure can employ BCH(n,k,t) encoding. Here, n is the encoded code length, k is the information bit length before encoding, and t is the maximum number of correctable bits. BCH encoding expands k-bit watermark information into n-bit codewords and solves the error location polynomial using an iterative algorithm (Berlekamp-Massey, BM) that quickly finds the shortest linear feedback shift register that can generate a given string of numbers / sequences, thereby achieving bit error correction.

[0083] For example, using BCH(31,21,2), defined in the finite field GF(2 5(Based on a closed-loop arithmetic system using 5-bit binary numbers), the generator polynomial is designed using the roots of primitive elements. This is applied to the input watermark information. ,in, The length of the watermark information is given. The data is divided into blocks according to the pre-encoding information bit length defined by BCH(31,21,2), resulting in 21-bit original watermark information data blocks. Each 21-bit original watermark information data block is then encoded using the BCH(31,21,2) encoding scheme for error correction, resulting in a 31-bit encoded block, which can be represented as an encoded sequence. , The total length after encoding is 31, which can be used here. In the 31-bit encoded block, 21 bits are the original watermark information bits and 10 bits are the check bits, which can correct up to 2 random bit errors in each block. The encoding process is shown in formula (6).

[0084] (6);

[0085] In formula (6), This represents the encoding function.

[0086] Then, for each encoded sequence After adding one auxiliary check bit, a 32-bit processing block is formed, which can be used as the bit sequence obtained by encoding the original watermark information data block, and can be represented by c. This operation aims to make the total length of the bit sequence adaptable to the subsequent latent space mapping requirements.

[0087] like Figure 4A As shown, regarding watermark information BCH encoding yields the bit sequence c.

[0088] Based on this embodiment, the effective payload rate is 21 / 32 = 65.625%. When the total watermark capacity is set to 16384 bits, the corresponding net effective payload is 10752 bits, which can meet the storage requirements of multidimensional metadata.

[0089] According to an embodiment of the present invention, corresponding to the above operation S202, as follows: Figure 4A As shown, a shared redundant bit string r can be obtained by copying a randomly generated bit string 401 bit by bit. Based on the mapping relationship as shown in formula (1), a dual-purpose key key can be generated by performing a bit XOR operation on the bit sequence c of the watermark information and the shared redundant bit string r.

[0090] It should be noted that the mapping relationship between bit sequence c and shared redundant bit string r can also be realized based on the dual-purpose key key. Given any two of the three components—bit sequence c, shared redundant bit string r, and dual-purpose key key—the third component can be obtained.

[0091] According to an embodiment of the present invention, the dual-purpose key combines redundant pattern encoding and pseudo-randomization functions. Corresponding to the above operation S202, the encoded bit sequence can be mapped to a shared redundant bit string using this dual-purpose key, avoiding explicit bit repetition. Corresponding to the pseudo-random shuffling in the above operation S203, such as... Figure 4A As shown, a derived stream key can be derived from this dual-purpose key key. s The bit tensor 402, obtained by rearranging the shared redundant bit string r, is pseudo-randomized using a symmetric stream cryptography algorithm (ChaCha20) to obtain a randomized bit tensor z. r This eliminates the regularity of bit distribution in the latent space and enhances the watermark's resistance to attacks.

[0092] According to an embodiment of the present invention, continuing the distribution-preserving sampling corresponding to the above operation S203, the randomized bit tensor can be transformed by combining a uniformly distributed random number with the quantile function of the standard normal distribution to map the randomized bit tensor to an initial latent noise that follows a standard normal distribution.

[0093] For example, for the randomized bit tensor z r The j-th bit It can be mapped using the quantile function of the standard normal distribution, and the sampling formula is as shown in formula (7).

[0094] (7);

[0095] In formula (7), For the j-th random number, , It represents a uniform distribution on the interval (0,1). Let j be the j-th noise vector in the initial potential noise. The quantile function is the standard normal distribution, ensuring the initial potential noise. , It represents a standard normal distribution, ensuring that the generation quality of the diffusion model is not compromised.

[0096] According to an embodiment of the present invention, corresponding to the above operation S204, the diffusion model can adopt the standard accelerated sampler of relevant mainstream generative models, and through 50 sampling steps, use the initial latent noise conforming to the N(0,1) distribution as input, and combine it with text prompts to complete image generation. Specifically, as Figure 4A As shown, the above operation S204 may include: based on the generation condition 403, for example, "pastel anime field scenery", processing the initial potential noise. Denoising is performed to obtain the initial image latent representation. The image encoder 404, based on a variational autoencoder (VAE), provides the latent representation of the initial image. Encode the image to obtain the target watermark image. .

[0097] See also Figure 4A As shown, the target watermark image After propagation and attack, it can, for example, generate distorted watermarked images. .

[0098] According to an embodiment of the present invention, corresponding to the above operation S301, as follows: Figure 4A As shown, for distorted watermark images It can first be decoded by a VAE-based image decoder 405 using a diffusion model, converting it into a distorted image latent representation. Then, the latent representation of the distorted image is obtained through the inverse transformation of Denoising Diffusion Implicit Models (DDIM). By performing an inverse transform, the distorted latent noise can be obtained. .

[0099] According to an embodiment of the present invention, corresponding to the above operation S302, such as Figure 4A As shown, the distortion encoder in the distortion-aware inverse correction network 400 can receive the distorted watermark image. As input to the distortion-aware inverse correction network 400, it combines the processing of the geometric supernetwork and pixel supernetwork in the distortion-aware inverse correction network 400, as well as the processing of distortion latent noise by the latent vector corrector. The processing ultimately results in the corrected latent noise output by the latent vector corrector. This serves as the output of the distortion-aware inverse correction network 400.

[0100] Figure 4B A schematic diagram of correction based on DIRN according to an embodiment of the present invention is shown.

[0101] like Figure 4B As shown, the distortion-aware inverse correction network 400 includes a distortion encoder 410, a distortion type gating system 420, a dual supernetwork 430, and a latent vector corrector 440. The dual supernetwork 430 includes a geometric supernetwork 431 and a pixel supernetwork 432.

[0102] Distortion encoder 410 for distorted watermark images Perform feature extraction and output a compact distortion feature vector d and a distortion meta-feature vector. Distorted meta-feature vector It constructs multi-dimensional descriptors using Sobel operators, Laplacian operators, coordinate convolution, spatial pyramid pooling, and Fast Fourier Transform (FFT) features.

[0103] Specifically, the distortion encoder 410 consists of an auxiliary channel generation module, a frequency domain statistics module, a spatial pyramid pooling (SPP) module, a backbone convolutional network, and an optional distortion metadata prediction head. Figure 4B (Not shown in the text) constitutes the composition.

[0104] The auxiliary channel generation module supplements the red-green-blue (RGB) channels with normalized coordinate channels to give them spatial position awareness. Then, it uses the FixedSobel operator to extract grayscale image edges and Laplacian convolution to extract second-order edges and concatenate them to form an 8-channel multimodal input of RGB + coordinates + edges + Laplacian, capturing texture / edge feature vectors of geometric distortion and pixel distortion.

[0105] The frequency domain statistics module converts RGB to grayscale and performs a two-dimensional real part FFT. It then bins the images according to low / medium / high radii to statistically analyze the frequency domain energy ratio and outputs a three-dimensional frequency domain feature vector. The backbone convolutional network consists of four layers of convolutional layers activated by grouping normalization and a smooth-gated activation function (Sigmoid Linear Unit, SiLU). It downsamples the multimodal input and extracts high-dimensional semantic feature vectors.

[0106] The SPP module uses adaptive average pooling at three scales (1×1, 2×2, and 4×4) to aggregate feature vectors at multiple scales, including global, medium-scale, and fine-grained features, to adapt to different distortion modes. Finally, the frequency domain features are encoded into 16-dimensional embeddings through a fully connected layer, and then concatenated with the features flattened by SPP. The resulting vector is projected through two fully connected layers to form a compact 64-dimensional distortion feature vector d.

[0107] The optional distortion metadata prediction head, based on the concatenated features, outputs a distortion metadata feature vector in the range [0,1] through a fully connected layer. It includes at least one of the following: a meta-feature vector representing geometric distortion, a meta-feature vector representing pixel distortion, providing a basis for subsequent distortion type gating 420.

[0108] Distortion type gating 420 pairs of distortion element feature vectors Process the data to output the geometrically dominant distortion probability. and pixel-dominated distortion probability ,satisfy The distortion feature vector d is mapped to feature-wise linear modulation (FiLM) parameters corresponding to the two distortion types through the geometric supernetwork 431 and pixel supernetwork 432 in the dual supernetwork 430, respectively. These parameters include geometric FiLM parameters and pixel FiLM parameters. Based on the fusion formulas (8) to (9), the two types of FiLM parameters are fused by probability weighting to obtain fused FiLM parameters that take into account both geometric distortion features and pixel distortion features.

[0109] (8);

[0110] (9);

[0111] In formulas (8) and (9), This represents the geometric scaling factor for the i-th channel. This represents the geometric offset coefficient of the i-th channel. This represents the pixel scaling factor for the i-th channel. This represents the pixel offset coefficient of the i-th channel. This represents the blending scaling factor for the i-th channel. This represents the fusion offset coefficient of the i-th channel.

[0112] Specifically, the dual-hypernet 430 consists of two identical hypernets and a FiLM parameter fusion module. The core HyperNet takes a 64-dimensional distortion feature vector d as input and maps it to the corresponding FiLM parameters of each layer of the latent vector corrector 440 through a multilayer perceptron (MLP) containing three fully connected layers (64-dimensional input layer → 128-dimensional hidden layer → 2×64-dimensional output layer depth). The geometric hypernet 431 specifically generates geometric FiLM parameters adapted to the geometric distortion features. The pixel supernetwork 432 is specifically designed to generate pixel FiLM parameters that adapt to pixel distortion characteristics. The FiLM parameter fusion module performs layer-by-layer, channel-by-channel weighted fusion of the two sets of FiLM parameters based on formulas (8) to (9), ultimately generating fused FiLM parameters that take into account both geometric distortion features and pixel distortion features. This provides conditional control for the subsequent potential vector corrector 440.

[0113] The latent vector corrector 440 utilizes fused FiLM parameters through stacked multilayer residual convolutional blocks + FiLM conditional modulation modules. Distortion potential noise Perform feature modulation and output the mean μ and log-variance logσ of the corrected latent noise. 2 The mean μ is used as the final corrected latent noise. .

[0114] Specifically, the latent vector corrector 440 consists of an input convolutional layer, multiple residual convolutional blocks, and a FiLM conditional modulation module. The whole process of "input mapping → residual feature extraction → FiLM conditional modulation → mean / variance output" achieves accurate and interpretable distortion correction of latent noise.

[0115] According to an embodiment of the present invention, corresponding to the above operation S303, such as Figure 4A As shown, the latent noise is corrected based on the dual-purpose key pair. To perform decoding, the bit sequence to be decoded can first be restored. Then, the bit sequence to be decoded can be... Inputting a BCH decoder 406, the system calculates the check vector of the bit sequence to be decoded. Based on the BM algorithm, it solves for the error localization polynomial by solving the check vector. This polynomial is then used to correct bit errors in the bit sequence to be decoded, ultimately recovering the original embedded watermark information. The complete decoding formula is shown in formula (10).

[0116] (10);

[0117] In formula (10), To verify the sub-vector, () represents the misplaced polynomial. This is an error mode conversion function. This is the message extraction function.

[0118] Through the above embodiments of the present invention, a high-capacity, distortion-resistant generative image watermark embedding and extraction method is realized. It can reliably embed and extract tens of thousands of bits of watermark while maintaining the image generation quality, providing complete traceability tracking capabilities for AIGC content scenarios and meeting the needs of copyright protection, security supervision and liability attribution in practical applications.

[0119] Furthermore, this invention requires no modification to the weights and generation process of the diffusion model, allowing for flexible integration into various diffusion-based image generation systems. It boasts low deployment costs, plug-and-play functionality, and cross-model compatibility and engineering feasibility. The invention employs a plug-and-play modular design. The watermark embedding process is achieved through initial latent noise modulation, and the extraction process is completed through inverse diffusion transformation and post-processing. It requires no modification to the backbone network structure, training parameters, or sampling process of the diffusion model, enabling flexible integration into various mainstream diffusion models. This provides excellent cross-model compatibility and engineering feasibility, facilitating rapid deployment and application.

[0120] Therefore, this invention, through the synergistic optimization of high-capacity redundancy design, distortion perception correction, and low-intrusive embedding, not only solves the core pain points of "insufficient capacity" and "weak robustness" of related generative image watermarking, but also takes into account visual quality and ease of deployment. It can achieve reliable copyright identification and traceability in the entire chain of AIGC content production and dissemination, and provides efficient and practical technical support for the compliance management and value protection of AI-generated content.

[0121] The feasibility of the above embodiments is verified by specific experiments, as detailed in the following description.

[0122] This invention selects a publicly available dataset from an open-source AI image generation model and randomly samples 1000 prompt words. Based on the open-source AI image generation model, 1000 images with a resolution of 512×512 are generated as basic experimental samples for watermark embedding and basic performance testing. This dataset covers diverse scenes, styles, and semantic descriptions to ensure the generality and reliability of the experimental results. To verify the generality of the method, it is integrated into two versions of the open-source AI image generation model for evaluation. To train DIRN, different types and intensities of attacks can be applied to another 1000 clean generated images to simulate various distortion environments, such as random cropping, Gaussian blur, image compression, and scaling.

[0123] This invention evaluates performance in two scenarios: watermark detection and watermark attribution. The goal of watermark detection is to determine whether a given image contains a watermark. For watermark detection, a false positive rate (FPR) of 10 can be used. -6 The true positive rate (TPR) is used as the primary evaluation metric. The core objective is to first control the model's false positive rate to an extremely low level of 10%. -6The "one in a million" level means that out of every million actually negative samples, only one will be incorrectly identified as positive. The true positive rate (TPR) at this level is used as the core evaluation metric to measure how many positive samples the model can correctly identify while "almost never wrongly identifying negative samples." It measures how many generated images carrying watermarks can be detected correctly. True positive rate (also called recall / sensitivity) and false positive rate are core metrics for evaluating model performance in binary classification tasks. True positive rate refers to the proportion of all actually positive samples correctly predicted as positive by the model, while false positive rate refers to the proportion of all actually negative samples incorrectly predicted as positive by the model. For watermark attribution scenarios, the accuracy of recovering embedded watermark bits from generated or distorted images can be quantified, and the average bit accuracy of all watermark extractions can be reported.

[0124] Furthermore, to assess whether watermark embedding affects image content, a CLIPScore score can be calculated to further evaluate the potential semantic bias that watermark embedding may introduce. CLIPScore is a cross-modal evaluation metric based on a pre-trained cross-modal model (CLIP), primarily used to quantify the semantic matching degree between an image and its text description. When evaluating semantic bias based on CLIPScore, the core idea is to quantify the degree to which the image being evaluated deviates from the target semantics by comparing the similarity between the "original generated image - the text prompt used" and the "watermarked generated image - the text prompt used". First, the image to be evaluated is input into CLIP's image encoder, which outputs a visual feature vector v normalized by L2 (norm 2), typically with a dimension of 512. Simultaneously, the corresponding text description is input into CLIP's text encoder (based on the Transformer), which outputs a text feature vector t, also normalized by L2. Then, the dot product of these two normalized vectors is calculated (since the vectors are normalized, the dot product result is equivalent to cosine similarity), yielding the original similarity value sim = v·t, where · represents the dot product calculation.

[0125] Figure 5 The robustness test results of the generative image watermarking method based on the diffusion model according to an embodiment of the present invention are shown in the figure.

[0126] To systematically evaluate the performance of generative image watermarking methods, this embodiment is tested on an open-source AI image generation model.

[0127] like Figure 5 As shown in the table, with the watermark capacity set to 10752 bits, at FPR=10 -6Under strict constraints, the true positive rate in clean scenarios without any distortion attacks reached 1.0000, and the bit accuracy was 1.0000. In distorted scenarios, the average true positive rate reached 0.9990, and the average bit accuracy reached 0.9819. The CLIP score reached 0.3630±0.0005.

[0128] Figure 5 The eight line graphs represent the robustness of the watermark under different types and intensities. The eight distortion scenarios include: brightness adjustment, random cropping, random discarding, Gaussian blur, salt-and-pepper noise, JPEG compression, median filtering, and scaling transformation. The line graphs show that even under various high-intensity distortions, the watermark message can still be stably recovered.

[0129] In summary, the embodiments of the present invention realize a high-capacity robust generative image watermarking method for diffusion models through the above steps. Without affecting the image generation quality, it breaks through the capacity limitation, improves robustness in complex distortion scenarios, and has plug-and-play deployment characteristics, meeting the needs of copyright protection, security supervision and accountability in practical applications.

[0130] Figure 6 A block diagram of a generative image watermarking processing apparatus based on a diffusion-oriented model according to an embodiment of the present invention is shown.

[0131] like Figure 6 As shown, the generative image watermarking processing device 600 for diffusion models includes a watermark information encoding module 610, a bit sequence mapping module 620, a shared redundant bit string conversion module 630, and a target watermark image generation module 640.

[0132] The watermark information encoding module 610 is used to encode the watermark information to obtain the bit sequence of the watermark information.

[0133] The dual-purpose key determination module 620 is used to determine a dual-purpose key based on a bit sequence and a preset shared redundant bit string.

[0134] The shared redundant bit string conversion module 630 is used to convert a shared redundant bit string into initial latent noise based on a dual-purpose key through pseudo-random shuffling and distribution-maintaining sampling.

[0135] The target watermark image generation module 640 is used to input the initial potential noise into the diffusion model and, in combination with the generation conditions, generate a target watermark image with embedded watermark information.

[0136] According to an embodiment of the present invention, the generative image watermarking processing device for diffusion models further includes an inverse transformation module, a distortion-perceived inverse correction module, and a watermark information extraction module.

[0137] The inverse transform module is used to perform an inverse transform on the distorted watermark image to obtain the distorted potential noise. The distorted watermark image is obtained by distorting the target watermark image.

[0138] The distortion-aware inverse correction module is used to input the distorted watermark image and the distortion potential noise into the distortion-aware inverse correction network to obtain the corrected potential noise.

[0139] The watermark information extraction module is used to decode the corrected potential noise based on a dual-purpose key in order to extract the watermark information.

[0140] According to an embodiment of the present invention, the watermark information encoding module includes a watermark information block unit, a data block encoding unit, and an auxiliary check bit addition unit.

[0141] The watermark information segmentation unit is used to segment the watermark information into blocks according to the pre-encoding information bit length pre-configured in the error correction coding program, resulting in multiple data blocks with a length equal to the pre-encoding information bit length. The error correction coding program also pre-configures the post-encoding code length and the number of error-correctable bits.

[0142] The data block encoding unit is used to call the error correction encoding program to encode the data block and obtain an encoded block with a length equal to the encoded code length and an error correction capability represented by the number of error-correctable bits.

[0143] An auxiliary check bit appending unit is used to append a preset number of auxiliary check bits to the coded block to obtain a bit sequence, wherein the total length of the bit sequence is equal to the total length of the shared redundant bit string.

[0144] According to an embodiment of the present invention, the dual-purpose key determination module includes a bit XOR unit.

[0145] The bit XOR unit is used to perform a bit XOR operation on a bit sequence and a shared redundant bit string to obtain a dual-purpose key.

[0146] According to an embodiment of the present invention, the shared redundant bit string conversion module includes a pseudo-random shuffling unit and a distribution-preserving sampling unit.

[0147] The pseudo-random scrambling unit is used to perform pseudo-random scrambling on the bit tensor corresponding to the shared redundant bit string based on the derived stream key of the dual-purpose key, so as to obtain a randomized bit tensor.

[0148] The distribution-preserving sampling unit is used to perform a distribution transformation on the randomized bit tensor based on a uniformly distributed random number and the quantile function of the standard normal distribution, so as to map the randomized bit tensor to an initial latent noise that follows a standard normal distribution.

[0149] According to an embodiment of the present invention, the target watermark image generation module includes a denoising unit and an image latent representation coding unit.

[0150] The denoising unit is used to denoise the initial latent noise based on the generation conditions to obtain the initial image latent representation.

[0151] The image latent representation coding unit is used to encode the initial image latent representation to obtain the target watermark image.

[0152] According to embodiments of the present invention, distortion processing includes at least one of the following: Gaussian blur, random cropping, random discarding, brightness adjustment, Gaussian noise, salt and pepper noise, median filtering, scaling, image compression, and geometric rotation.

[0153] According to an embodiment of the present invention, the inverse transformation module includes a distorted watermark image decoding unit and an inverse transformation unit.

[0154] The distorted watermark image decoding unit is used to decode the distorted watermark image to obtain the latent representation of the distorted image.

[0155] The inverse transform unit is used to perform an inverse transform on the latent representation of the distorted image to obtain the distorted latent noise.

[0156] According to an embodiment of the present invention, the distortion-aware inverse correction module includes a feature extraction unit, a distortion type gating processing unit, a geometric supernetwork unit, a pixel supernetwork unit, a fusion unit, and a feature modulation unit.

[0157] The feature extraction unit is used to extract features from the distorted watermark image to obtain a distortion feature vector and a distortion meta-feature vector. The distortion feature vector includes at least a texture feature vector, an edge feature vector, a frequency domain feature vector, a semantic feature vector, and a multi-scale feature vector. The distortion meta-feature vector includes at least one of the following: a meta-feature vector representing geometric distortion and a meta-feature vector representing pixel distortion.

[0158] The distortion type gating processing unit is used to process the distortion element feature vector based on distortion type gating to obtain the geometrically dominant distortion probability and the pixel-dominated distortion probability, wherein the sum of the geometrically dominant distortion probability and the pixel-dominated distortion probability is 1.

[0159] Geometric supernetwork units are used to map distortion feature vectors to geometric feature-level linear modulation parameters that adapt to geometric distortion, based on geometric supernetworks.

[0160] The pixel supernetwork unit is used to map the distortion feature vector into pixel feature-level linear modulation parameters that adapt to pixel distortion, based on the pixel supernetwork.

[0161] The fusion unit is used to fuse geometric feature-level linear modulation parameters and pixel feature-level linear modulation parameters based on geometric-dominant distortion probability and pixel-dominant distortion probability, so as to obtain fused feature-level linear modulation parameters that take into account both geometric distortion features and pixel distortion features.

[0162] The feature modulation unit is used to perform feature modulation on the distortion latent noise based on the fused feature-level linear modulation parameters to obtain the corrected latent noise.

[0163] According to an embodiment of the present invention, the watermark information extraction module includes a corrected latent noise decoding unit, a check vector calculation unit, an error location polynomial solving unit, and an error correction unit.

[0164] The corrected latent noise decoding unit is used to decode the corrected latent noise based on a dual-purpose key to obtain the bit sequence to be decoded.

[0165] The check vector calculation unit is used to calculate the check vector of the bit sequence to be decoded.

[0166] The error location polynomial solving unit is used to solve for the error location polynomial based on the check vector.

[0167] The error correction unit is used to correct bit errors in the bit sequence to be decoded based on the error location polynomial and recover the watermark information.

[0168] Any one or more of the modules or units according to embodiments of the present invention, or at least part of the functions of any one or more of them, can be implemented in one module. Any one or more of the modules or units according to embodiments of the present invention can be implemented by dividing them into multiple modules. Any one or more of the modules or units according to embodiments of the present invention can be at least partially implemented as hardware circuitry, such as field-programmable gate arrays (FPGAs), programmable logic arrays (PLAs), systems-on-a-chip, systems-on-a-substrate, systems-on-package, application-specific integrated circuits (ASICs), or implemented in hardware or firmware by any other reasonable means of integrating or packaging circuitry, or implemented in software, hardware, and firmware, or in any suitable combination of any of these three implementation methods. Alternatively, one or more of the modules or units according to embodiments of the present invention can be at least partially implemented as computer program modules, which, when run, can perform corresponding functions.

[0169] For example, any plurality of the watermark information encoding module 610, bit sequence mapping module 620, shared redundant bit string conversion module 630, and target watermark image generation module 640 can be combined into one module / unit, or any one of these modules / units can be split into multiple modules / units. Alternatively, at least part of the functionality of one or more of these modules / units can be combined with at least part of the functionality of other modules / units and implemented in one module / unit. According to embodiments of the present invention, at least one of the watermark information encoding module 610, bit sequence mapping module 620, shared redundant bit string conversion module 630, and target watermark image generation module 640 can be at least partially implemented as hardware circuitry, such as field-programmable gate array (FPGA), programmable logic array (PLA), system-on-a-chip, system-on-a-substrate, system-on-package, application-specific integrated circuit (ASIC), or any other reasonable means of integrating or packaging the circuitry, or implemented in software, hardware, or firmware, or in any one of the three implementation methods or a suitable combination of any of them. Alternatively, at least one of the watermark information encoding module 610, bit sequence mapping module 620, shared redundant bit string conversion module 630, and target watermark image generation module 640 can be at least partially implemented as a computer program module, which can perform corresponding functions when the computer program module is run.

[0170] Figure 7 A block diagram of an electronic device suitable for implementing the above-described generative image watermarking method based on a diffusion-oriented model, according to an embodiment of the present invention, is shown. Figure 7 The electronic device shown is merely an example and should not be construed as limiting the functionality and scope of use of the embodiments of the present invention.

[0171] like Figure 7 As shown, an electronic device 700 according to an embodiment of the present invention includes a processor 701, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 702 or a program loaded from a storage portion 708 into a random access memory (RAM) 703. The processor 701 may include, for example, a general-purpose microprocessor (e.g., a CPU), an instruction set processor and / or an associated chipset and / or a special-purpose microprocessor (e.g., an application-specific integrated circuit (ASIC)), etc. The processor 701 may also include onboard memory for caching purposes. The processor 701 may include a single processing unit or multiple processing units for performing different actions of the method flow according to an embodiment of the present invention.

[0172] RAM 703 stores various programs and data required for the operation of electronic device 700. Processor 701, ROM 702, and RAM 703 are interconnected via bus 704. Processor 701 executes various operations of the method flow according to embodiments of the present invention by executing programs in ROM 702 and / or RAM 703. It should be noted that the programs may also be stored in one or more memories other than ROM 702 and RAM 703. Processor 701 may also execute various operations of the method flow according to embodiments of the present invention by executing programs stored in said one or more memories.

[0173] According to an embodiment of the present invention, the electronic device 700 may further include an input / output (I / O) interface 705, which is also connected to a bus 704. The electronic device 700 may also include one or more of the following components connected to the input / output (I / O) interface 705: an input section 706 including a keyboard, mouse, etc.; an output section 707 including a cathode ray tube (CRT), liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 708 including a hard disk, etc.; and a communication section 709 including a network interface card such as a LAN card, modem, etc. The communication section 709 performs communication processing via a network such as the Internet. A drive 710 is also connected to the input / output (I / O) interface 705 as needed. A removable medium 711, such as a disk, optical disk, magneto-optical disk, semiconductor memory, etc., is installed on the drive 710 as needed so that computer programs read from it can be installed into the storage section 708 as needed.

[0174] According to embodiments of the present invention, the method flow according to embodiments of the present invention can be implemented as a computer software program. For example, embodiments of the present invention include a computer program product comprising a computer program carried on a computer-readable storage medium, the computer program containing program code for performing the method shown in the flowchart. In such embodiments, the computer program can be downloaded and installed from a network via communication section 709, and / or installed from removable medium 711. When the computer program is executed by processor 701, it performs the functions defined in the system of the embodiments of the present invention. According to embodiments of the present invention, the systems, devices, apparatuses, modules, units, etc., described above can be implemented by computer program modules.

[0175] The present invention also provides a computer-readable storage medium, which may be included in the device / apparatus / system described in the above embodiments; or it may exist independently and not assembled into the device / apparatus / system. The computer-readable storage medium carries one or more programs, which, when executed, implement the generative image watermarking method based on a diffusion-oriented model according to embodiments of the present invention.

[0176] According to embodiments of the present invention, the computer-readable storage medium may be a non-volatile computer-readable storage medium. Examples include, but are not limited to: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In the present invention, the computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.

[0177] For example, according to embodiments of the present invention, a computer-readable storage medium may include the ROM 702 and / or RAM 703 described above and / or one or more memories other than ROM 702 and RAM 703.

[0178] Embodiments of the present invention also include a computer program product comprising a computer program containing program code for performing the methods provided in the embodiments of the present invention. When the computer program product is run on an electronic device, the program code is used to enable the electronic device to implement the generative image watermarking processing method based on the diffusion model provided in the embodiments of the present invention.

[0179] When the computer program is executed by the processor 701, it performs the functions defined in the system / apparatus of this embodiment of the invention. According to embodiments of the invention, the systems, apparatuses, modules, units, etc., described above can be implemented by computer program modules.

[0180] In one embodiment, the computer program may rely on a tangible storage medium such as an optical storage device or a magnetic storage device. In another embodiment, the computer program may also be transmitted and distributed in the form of signals over a network medium, and may be downloaded and installed via the communication section 709, and / or installed from a removable medium 711. The program code contained in the computer program can be transmitted using any suitable network medium, including but not limited to: wireless, wired, etc., or any suitable combination thereof.

[0181] According to embodiments of the present invention, program code for executing the computer programs provided in the embodiments of the present invention can be written in any combination of one or more programming languages. Specifically, these computational programs can be implemented using high-level procedural and / or object-oriented programming languages, and / or assembly / machine languages. Programming languages ​​include, but are not limited to, languages ​​such as Java, C++, Python, "C", or similar programming languages. The program code can be executed entirely on the user's computing device, partially on the user's device, partially on a remote computing device, or entirely on a remote computing device or server. In cases involving remote computing devices, the remote computing device can be connected to the user's computing device via any type of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computing device (e.g., via the Internet using an Internet service provider).

[0182] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present invention. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram or flowchart, and combinations of blocks in a block diagram or flowchart, may be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions. Those skilled in the art will understand that the features recited in the various embodiments and / or claims of the present invention can be combined and / or combined in various ways, even if such combinations or combinations are not expressly stated in the present invention. In particular, the features described in the various embodiments and / or claims of this invention can be combined and / or combined in various ways without departing from the spirit and teachings of this invention. All such combinations and / or combinations fall within the scope of this invention.

[0183] The embodiments of the present invention have been described above. However, these embodiments are merely illustrative and not intended to limit the scope of the invention. Although various embodiments have been described above, this does not mean that the measures in the various embodiments cannot be used advantageously in combination. The scope of the invention is defined by the appended claims and their equivalents. Various substitutions and modifications can be made by those skilled in the art without departing from the scope of the invention, and all such substitutions and modifications should fall within the scope of the invention.

Claims

1. A generative image watermarking method for a diffusion model, characterized in that, The method includes: The watermark information is encoded to obtain the bit sequence of the watermark information; Based on the bit sequence and the preset shared redundant bit string, a dual-purpose key is determined; Based on the dual-purpose key, the shared redundant bit string is converted into initial potential noise through pseudo-random shuffling and distribution-maintaining sampling; The initial potential noise is input into the diffusion model, and combined with the generation conditions, a target watermark image with the embedded watermark information is generated.

2. The method according to claim 1, characterized in that, The method further includes: An inverse transform is performed on the distorted watermark image to obtain distorted potential noise, wherein the distorted watermark image is obtained by distorting the target watermark image; The distorted watermark image and the distorted latent noise are input into a distortion-aware inverse correction network to obtain the corrected latent noise. Based on the dual-purpose key, the corrected potential noise is decoded to extract the watermark information.

3. The method according to claim 1, characterized in that, The process of encoding the watermark information to obtain the bit sequence of the watermark information includes: According to the pre-configured information bit length in the error correction coding program, the watermark information is divided into blocks to obtain multiple data blocks, each with a length equal to the pre-configured information bit length. The error correction coding program is also configured with the post-encoding code length and the number of error-correctable bits. The error correction coding program is invoked to encode the data block, resulting in a coded block with a length equal to the encoded code length and an error correction capability represented by the number of error-correctable bits. An auxiliary check bit of a preset number of bits is added to the coded block to obtain the bit sequence, wherein the total length of the bit sequence is equal to the total length of the shared redundant bit string.

4. The method according to claim 1, characterized in that, The determination of the dual-purpose key based on the bit sequence and the preset shared redundant bit string includes: The dual-purpose key is obtained by performing a bit XOR operation on the bit sequence and the shared redundant bit string.

5. The method according to claim 1, characterized in that, The process of converting the shared redundant bit string into initial latent noise based on the dual-purpose key through pseudo-random shuffling and distribution-maintaining sampling includes: Based on the derived stream key of the dual-purpose key, the bit tensor corresponding to the shared redundant bit string is pseudo-randomly shuffled to obtain a randomized bit tensor. Based on a uniformly distributed random number and combined with the quantile function of the standard normal distribution, the randomized bit tensor is subjected to a distribution transformation to map the randomized bit tensor to an initial latent noise that follows a standard normal distribution.

6. The method according to claim 1, characterized in that, The step of inputting the initial potential noise into the diffusion model and combining it with the generation conditions to generate the target watermark image with the embedded watermark information includes: Based on the generation conditions, the initial latent noise is denoised to obtain the initial image latent representation; The target watermark image is obtained by encoding the latent representation of the initial image.

7. The method according to claim 2, characterized in that, The distortion processing includes at least one of the following: Gaussian blur, random cropping, random discarding, brightness adjustment, Gaussian noise, salt and pepper noise, median filtering, scaling, image compression, and geometric rotation; The inverse transformation of the distorted watermark image to obtain the potential distorted noise includes: Decode the distorted watermark image to obtain the latent representation of the distorted image; The distorted image latent representation is inversely transformed to obtain the distorted latent noise.

8. The method according to claim 2, characterized in that, The process of inputting the distorted watermark image and the distorted latent noise into a distortion-aware inverse correction network to obtain the corrected latent noise includes: Feature extraction is performed on the distorted watermark image to obtain a distortion feature vector and a distortion meta-feature vector. The distortion feature vector includes at least a texture feature vector, an edge feature vector, a frequency domain feature vector, a semantic feature vector, and a multi-scale feature vector. The distortion meta-feature vector includes at least one of the following: a meta-feature vector representing geometric distortion and a meta-feature vector representing pixel distortion. Based on distortion type gating, the distortion element feature vector is processed to obtain the geometry-dominated distortion probability and the pixel-dominated distortion probability, wherein the sum of the geometry-dominated distortion probability and the pixel-dominated distortion probability is 1. Based on a geometric supernetwork, the distortion feature vector is mapped to geometric feature-level linear modulation parameters that adapt to geometric distortion; Based on the pixel super network, the distortion feature vector is mapped to pixel feature-level linear modulation parameters that adapt to pixel distortion; Based on the geometric-dominated distortion probability and the pixel-dominated distortion probability, the geometric feature-level linear modulation parameters and the pixel feature-level linear modulation parameters are fused to obtain fused feature-level linear modulation parameters that take into account both geometric distortion features and pixel distortion features. Based on the fusion feature-level linear modulation parameters, the distortion potential noise is characteristically modulated to obtain the corrected potential noise.

9. The method according to claim 2 or 8, characterized in that, The distortion-aware inverse correction network was trained using the following method: The distortion-aware inverse correction network is trained by constructing a joint loss function, wherein the joint loss function includes Gaussian negative log-likelihood loss, prior loss, and metadata alignment loss; Based on the Gaussian negative log-likelihood loss, the consistency between the corrected latent noise and the distorted latent noise is constrained. Based on the prior loss, the constrained and corrected potential noise satisfies a standard normal distribution. Based on the aforementioned metadata alignment loss, the consistency between the true value and the predicted value of the distorted meta-feature vector is constrained.

10. The method according to claim 2, characterized in that, The step of decoding the corrected latent noise based on the dual-purpose key to extract the watermark information includes: Based on the dual-purpose key, the corrected potential noise is decoded to obtain the bit sequence to be decoded; Calculate the check vector of the bit sequence to be decoded; Based on the aforementioned check vector, the error location polynomial is obtained; Based on the error localization polynomial, bit errors in the bit sequence to be decoded are corrected, and the watermark information is recovered.

Images