A redundancy perception-based quality maintenance text-to-image large model watermarking method

Abstract

This invention provides a quality-preserving watermarking method for large-scale raw image models based on redundancy perception. The method includes: in the encoding stage, a watermark encoder transforms the watermark image to generate initial watermark features; a differential redundancy distribution component quantifies the energy distribution in the latent representation of the carrier image dataset, transforms the energy distribution to obtain a redundancy distribution matrix; a redundancy-guided fusion component uses the redundancy distribution matrix to fuse the carrier image with the initial watermark features to obtain a watermarked latent representation; in the decoding stage, a watermark feature decoding component extracts watermark features from the watermarked latent representation, and a watermark reconstructor uses the watermark features to recover the watermark image. This invention, through the proposed redundancy perception mechanism, truly achieves deep collaborative optimization of 'ultra-large capacity information carrying' and 'visual fidelity preservation', effectively maintaining extremely high visual fidelity of the generated image.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and in particular to a quality-preserving watermarking method for large-scale raw images based on redundancy perception. Background Technology

[0002] Currently, watermarking techniques for large-scale image processing models primarily employ feature injection schemes based on latent space. These schemes typically rely on an independently trained watermarking encoding module that maps a pre-defined low-dimensional bit sequence (usually 64 to 256 bits of information) into pseudo-random noise or perturbation signals adapted to the latent space dimension of the diffusion model. During image generation, this method directly superimposes or injects these perturbation signals into the latent representation of the diffusion model, resulting in pixel-level images containing detectable watermark features, thereby enabling copyright traceability or content authentication.

[0003] The drawbacks of the aforementioned feature injection schemes based on latent space include:

[0004] 1. Failure to consider the inherent redundancy distribution and perceptual characteristics in the latent space: When injecting the watermark signal, this scheme typically adopts a globally uniform embedding strength or a fixed channel allocation strategy, failing to fully explore and quantify the perceptual redundancy characteristics of points in each dimension of the latent representation. Since the contribution of different sites in the latent space to the final visual reconstruction varies significantly, this "blind" injection method is highly susceptible to interfering with sensitive features that are crucial to the visual quality of the image, resulting in obvious artifacts, color shifts, or detail degradation in the generated image.

[0005] 2. A serious conflict exists between information carrying capacity and visual fidelity: This scheme is primarily designed for low-bit-capacity watermarks. When attempting to embed large-capacity information (such as a color watermark image with the same resolution as the carrier image), the embedding operation inevitably destroys the integrity of the original underlying representation because it cannot avoid non-redundant feature regions. This results in the generated image failing to maintain its original fidelity and detail richness when pursuing high-capacity watermark embedding, making it difficult to meet the quality maintenance requirements of high-quality AIGC content production. Summary of the Invention

[0006] This invention provides a quality-preserving watermarking method for large-scale raw images based on redundancy perception, in order to effectively maintain the visual fidelity of the generated images.

[0007] To achieve the above objectives, the present invention adopts the following technical solution.

[0008] A quality-preserving watermarking method for large-scale raw images based on redundancy awareness includes:

[0009] In the encoding section, the watermark encoder transforms the watermark image to generate initial watermark features;

[0010] The energy distribution in the latent representation of the carrier image dataset is quantified by a differential redundancy distribution component, and the energy distribution is transformed to obtain a redundancy distribution matrix;

[0011] The redundant guided fusion component utilizes the redundant distribution matrix to fuse the carrier image with the initial watermark features to obtain a watermarked potential representation.

[0012] In the decoding section, watermark features are extracted from the watermarked latent representation by the watermark feature decoding component, and the watermark image is recovered by the watermark reconstructor using the watermark features.

[0013] Preferably, in the encoding segment, the watermark encoder transforms the watermark image to generate initial watermark features, including:

[0014] Configure the watermark encoder to reuse the pre-trained VAE encoder in LDM. The watermarked image to be transmitted is input into the watermark encoder, which then processes the image into a format of [size missing]. Color digital watermark image Generate a size of Initial watermark features :

[0015] .

[0016] Preferably, the step of quantizing the energy distribution in the latent representation of the carrier image dataset through a differentiated redundancy distribution component, and transforming the energy distribution to obtain a redundancy distribution matrix, includes:

[0017] Differential redundancy distributed components via encoder Carrier image dataset Transform into a latent representation dataset Principal component analysis algorithm is used to analyze the latent representation dataset. After processing, we get:

[0018]

[0019] in, This represents the principal component analysis algorithm. Indicates the first The unit eigenvectors of each principal component. Indicates the first The variance contribution of each principal component;

[0020] Calculate the latent representation for each location in the dataset Relative energy contribution :

[0021]

[0022] in, The normalized explanatory variance ratio To flatten the dimensions of the latent representation, and , For feature vectors The component at the original coordinate j, and ;

[0023] Will Remodeling to match watermark features For the same size, the energy distribution matrix is ​​obtained. For the energy distribution matrix Perform min-max normalization and take the complement to obtain the redundancy distribution matrix. :

[0024]

[0025] in, Describes the minimum value function. Represents the maximum value function;

[0026] Redundancy distribution matrix The size is the same as the size of the potential representation, which is... That is, C is the number of channels, H×W is the height and width, and the redundancy distribution matrix. The value at each position reflects the potential redundancy of the representation at that coordinate.

[0027] Preferably, the step of using the redundant distribution matrix to fuse the carrier image with the initial watermark features through the redundant guided fusion component to obtain a watermarked latent representation includes:

[0028] The redundant guided fusion component adopts a structure based on multi-level convolution and non-linear activation coupling. It includes a latent representation acquisition module, a watermark feature redistribution module, and a feature fusion module. The latent representation acquisition module is implemented through an encoder. Carrier image dataset One of the images is the size of carrier image Convert to size The potential representation of;

[0029] Initial watermark features The redundancy distribution matrix R is input to the watermark feature redistribution module. R serves as a spatial-channel masking guidance signal. Through the mapping of the watermark feature redistribution module, the initial watermark features are intensity modulated to obtain redistributed watermark features. The redistributed watermark features are then fused with the latent representation of the carrier through a feature fusion module. The redistributed watermark signal is seamlessly embedded into the redundant sites of the carrier's latent representation, and the output is a watermarked latent representation. .

[0030] Preferably, in the decoding section, extracting watermark features from the watermark-containing latent representation using a watermark feature decoding component, and recovering the watermark image using the watermark features via a watermark reconstructor, includes:

[0031] The watermark encoder consists of a watermark feature decoding network and a watermark reconstructor. The watermark feature decoding network has a multi-level structure coupled with convolutional and non-linear activations. It enhances feature representation by expanding the channel dimension of intermediate layers, and then performs feature compression and refinement layer by layer to output a feature map that matches the dimension of the potential representation containing the watermark. The network structure of the watermark reconstructor is consistent with the pre-trained VAE decoder in LDM. During the training phase, the network undergoes end-to-end parameter optimization. The weight parameters inside the watermark reconstructor are tailored to the watermark features. Evolution;

[0032] The watermark feature decoding network uses an encoder From the potential representation of watermark Extract watermark features Watermark features Input into watermark reconstructor In the middle, watermark reconstructor Watermark features Spatial dimension restoration and detail filling are performed to reconstruct the abstract feature signals into a watermark image. ;

[0033]

[0034] in, This represents a watermark feature decoding network.

[0035] As can be seen from the technical solutions provided by the embodiments of the present invention above, the present invention, through the proposed redundant perception mechanism, breaks through the technical bottleneck of existing technologies that inevitably lead to the loss of image details and degradation of visual quality in large-capacity embedding scenarios, and truly achieves deep collaborative optimization of 'ultra-large capacity information carrying' and 'visual fidelity maintenance', effectively maintaining the extremely high visual fidelity of the generated image.

[0036] Additional aspects and advantages of the invention will be set forth in part in the description which follows, and will become apparent from the description or may be learned by practice of the invention. Attached Figure Description

[0037] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0038] Figure 1 This is a flowchart illustrating a quality-preserving watermarking method for large-scale raw image models based on redundancy awareness, provided in an embodiment of the present invention. Detailed Implementation

[0039] Embodiments of the present invention are described in detail below, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and should not be construed as limiting the present invention.

[0040] Those skilled in the art will understand that, unless specifically stated otherwise, the singular forms “a,” “an,” “the,” and “the” used herein may also include the plural forms. It should be further understood that the term “comprising” as used in this specification means the presence of the stated features, integers, steps, operations, elements, and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof. It should be understood that when we say an element is “connected” or “coupled” to another element, it can be directly connected or coupled to the other element, or there may be intermediate elements. Furthermore, “connected” or “coupled” as used herein can include wireless connections or couplings. The term “and / or” as used herein includes any and all combinations of one or more of the associated listed items.

[0041] It will be understood by those skilled in the art that, unless otherwise defined, all terms used herein (including technical and scientific terms) have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains. It should also be understood that terms such as those defined in general dictionaries should be understood to have the same meaning as in the context of the prior art, and should not be interpreted in an idealized or overly formal sense unless defined as herein.

[0042] To facilitate understanding of the embodiments of the present invention, the following will provide further explanation and description with reference to the accompanying drawings and several specific embodiments. These embodiments do not constitute a limitation on the embodiments of the present invention.

[0043] Quality-preserving image-based large model watermarking refers to embedding watermark information into the image during the image generation process of the latent diffusion model (LDM) of the image-based large model, resulting in a watermarked image. This embedding process has little impact on the visual perception quality of the final output image.

[0044] This invention provides a processing flow for a quality-preserving text-based image large model watermarking method based on redundancy awareness, as follows: Figure 1 As shown, the processing steps include the following:

[0045] Step S10: In the encoding segment, the watermark encoder converts the watermark image to generate initial watermark features;

[0046] Step S20: The energy distribution in the latent representation of the carrier image dataset is quantized by the differential redundancy distribution component, and the energy distribution is transformed to obtain the redundancy distribution matrix;

[0047] Step S30: The redundant guided fusion component uses the redundant distribution matrix to fuse the carrier image with the initial watermark features to obtain a watermarked potential representation.

[0048] Step S40: In the decoding section, the watermark feature decoding component extracts watermark features from the watermark latent representation and recovers the watermark image through the watermark reconstructor.

[0049] The encoder and decoder are pre-trained diffusion model components.

[0050] Step S10 above includes:

[0051] The watermark encoder reuses the VAE (Variational Autoencoder Encoder) encoder pre-trained in LDM (Internet Download Manager, a platform download acceleration and management tool) without modification or fine-tuning. The watermark encoder and the VAE encoder have the same structure.

[0052] The watermarked image to be transmitted is input into the watermark encoder. The watermark encoder utilizes the efficient compression characteristics of the VAE encoder to map the high-resolution watermarked image (large-capacity watermark) into a latent feature tensor. This latent feature tensor is the watermark feature. This watermark feature is not only aligned with the latent prior of the watermarked image, but also consistent with the latent representation size of the image, thereby enabling early capture and faster convergence of the watermark signal and large-capacity injection of watermark information.

[0053] Watermark encoder via encoder A picture of size Color digital watermark image Generate a size of Watermark features :

[0054] .

[0055] Step S20 above includes:

[0056] The differential redundancy distribution component obtains the latent representation dataset through the VAE encoder pre-trained in LDM. Then, it obtains the variance contribution of different feature dimensions through PCA, and then maps the variance contribution back to the latent space to obtain the energy distribution matrix. Finally, the redundancy distribution matrix is ​​obtained by minimax normalization and taking the complement. .

[0057] Differential redundancy distributed components via encoder Carrier image dataset First, convert it into a latent representation dataset. Then, principal component analysis algorithm is used to analyze the latent representation dataset. After processing, we get:

[0058]

[0059] in, This represents the principal component analysis algorithm. Indicates the first The unit eigenvectors of each principal component. Indicates the first The variance contribution of each principal component.

[0060] Latent Representation: To improve image generation efficiency and reduce computational complexity, the image generation process is not performed directly in pixel space. Instead, a pre-trained variational autoencoder (VAE) maps the high-resolution image into a compressed low-dimensional latent space. The corresponding data of the image in the latent space is then called the latent representation.

[0061] Then, compute the latent representation for each location in the dataset. Relative energy contribution :

[0062]

[0063] in, The normalized explanatory variance ratio To flatten the dimensions of the latent representation, and , For feature vectors The component at the original coordinate j, and .

[0064] Then, Remodeling to match watermark features For the same size, the energy distribution matrix is ​​obtained. Then, for the energy distribution matrix Perform min-max normalization and take the complement to obtain the redundancy distribution matrix. :

[0065]

[0066] in, Describes the minimum value function. This represents the maximum value function.

[0067] Redundancy distribution matrix The size is the same as the size of the potential representation, which is... ,Right now It is the number of channels. These are the height and width, and the value at each position reflects the redundancy of the potential representation at that coordinate. The greater the redundancy, the more watermark information can be carried with less impact on image quality.

[0068] Step S30 above includes redundantly guiding the fusion component through an encoder. Carrier image dataset One of the images is the size of carrier image Convert to size The latent representation of the above carrier is then combined with the redundancy distribution matrix R and the initial watermark features. They are fed together into a redundantly guided fusion network to obtain a watermarked latent representation. .

[0069] The redundant guided fusion component adopts a structure based on multi-level convolution and non-linear activation coupling, and is composed of a watermark feature redistribution module and a feature fusion module connected in series.

[0070] The input data for the redundant guidance fusion component are the redundancy distribution matrix R and the initial watermark features. And the potential representation of the carrier. First, the initial watermark features are guided by the redundancy distribution matrix R. An adaptive redistribution is performed, and then the redistributed watermark features are fused with the latent representation of the carrier to output a watermarked latent representation. .

[0071] The redundant guided fusion component completes the generation of the watermarked latent representation through the following two core stages: ① First stage: Redistribution of redundant guided watermark features. This involves redistributing the initial watermark features... The redundancy distribution matrix R is used as a joint input. In this process, the redundancy distribution matrix R serves as a spatial-channel masking guidance signal, and through the mapping of the 'watermark feature redistribution module,' the initial watermark features are intensity modulated. This process establishes an adaptive mapping relationship: in high-redundancy regions (where R values ​​are large), the feature energy of the watermark signal is enhanced or preserved; in low-redundancy regions (where R values ​​are small), the watermark signal is suppressed. The final output is a redistributed watermark feature consistent with the redundancy of the carrier's latent representation. ② Second stage: Fusion of redistributed watermark features with the carrier's latent representation. The redistributed watermark features and the carrier's latent representation are used as joint inputs. The 'feature fusion module' seamlessly embeds the redistributed watermark signal into the redundant sites of the carrier's latent representation through multi-level feature extraction. Because this process is guided by the prior of the redundancy matrix, the embedding operation can avoid key visual feature regions of the carrier image, ultimately outputting a watermarked latent representation with preserved quality.

[0072] Step S40 above includes the watermark encoder employing a multi-level convolutional and non-linear activation coupled structure, which achieves the mapping from the watermark latent representation to the watermark features through shallow-to-deep feature extraction. The watermark encoder includes a watermark feature decoding network and a watermark reconstructor.

[0073] The watermark feature decoding network consists of four layers of two-dimensional convolution operators. It enhances the feature representation capability by expanding the channel dimension of the intermediate layers, and then performs feature compression and refinement layer by layer. Finally, it outputs a 4-channel feature map that matches the dimension of the potential representation containing the watermark.

[0074] The network structure of the watermark reconstructor is consistent with the pre-trained VAE decoder in LDM, but the network undergoes end-to-end parameter optimization during the training phase, and its internal weight parameters are tailored to the watermark features. It has evolved.

[0075] The watermark feature decoding network uses an encoder From the potential representation of watermark Extract watermark features Watermark features Input into watermark reconstructor The recovered watermarked image This involves restoring the spatial dimensions and filling in the details of the feature tensor, ultimately reconstructing the abstract feature signal into a perceptible, high-fidelity watermark image.

[0076]

[0077] in, This represents a watermark feature decoding network.

[0078] In summary, the embodiments of this invention, through the proposed redundancy sensing mechanism, effectively maintain extremely high visual fidelity of the generated image while significantly increasing the watermark embedding capacity. Compared to existing large-model watermarking technologies for text-based images that can only carry limited bit information (such as 64 / 128 / 256 bits), this invention achieves quality-preserving embedding of large-size image watermarks (512×512 pixel RGB three-channel color images) with the same resolution as the carrier image. This improvement successfully overcomes the technical bottleneck of existing technologies that inevitably lead to loss of image details and degradation of visual quality in large-capacity embedding scenarios, truly achieving deep synergistic optimization of 'ultra-large capacity information carrying' and 'visual fidelity preservation', providing a more practical technical solution for the secure traceability of AIGC content.

[0079] Those skilled in the art will understand that the accompanying drawings are merely schematic diagrams of one embodiment, and the modules or processes shown in the drawings are not necessarily essential for implementing the present invention.

[0080] As can be seen from the above description of the embodiments, those skilled in the art can clearly understand that the present invention can be implemented by means of software plus necessary general-purpose hardware platforms. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in various embodiments or some parts of the embodiments of the present invention.

[0081] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, for apparatus or system embodiments, since they are basically similar to method embodiments, the description is relatively simple; relevant parts can be referred to the descriptions in the method embodiments. The apparatus and system embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without creative effort.

[0082] The above description is merely a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A quality-preserving watermarking method for large-scale text-based images based on redundancy awareness, characterized in that, include: In the encoding section, the watermark encoder transforms the watermark image to generate initial watermark features; The energy distribution in the latent representation of the carrier image dataset is quantified by a differential redundancy distribution component, and the energy distribution is transformed to obtain a redundancy distribution matrix; The redundant guided fusion component utilizes the redundant distribution matrix to fuse the carrier image with the initial watermark features to obtain a watermarked potential representation. In the decoding section, watermark features are extracted from the watermarked latent representation by the watermark feature decoding component, and the watermark image is recovered by the watermark reconstructor using the watermark features.

2. The method according to claim 1, characterized in that, In the encoding segment, the watermark encoder transforms the watermark image to generate initial watermark features, including: Configure the watermark encoder to reuse the pre-trained VAE encoder in LDM. The watermarked image to be transmitted is input into the watermark encoder, which then processes the image into a format of [size missing]. Color digital watermark image Generate a size of Initial watermark features : 。 3. The method according to claim 2, characterized in that, The method of quantizing the energy distribution in the latent representation of the carrier image dataset using a differentiated redundancy distribution component, and transforming the energy distribution to obtain a redundancy distribution matrix includes: Differential redundancy distributed components via encoder Carrier image dataset Transform into a latent representation dataset Principal component analysis algorithm is used to analyze the latent representation dataset. After processing, we get: in, This represents the principal component analysis algorithm. Indicates the first The unit eigenvectors of each principal component. Indicates the first The variance contribution of each principal component; Calculate the latent representation for each location in the dataset Relative energy contribution : in, The normalized explanatory variance ratio To flatten the dimensions of the latent representation, and , For feature vectors The component at the original coordinate j, and ; Will Remodeling to match watermark features For the same size, the energy distribution matrix is ​​obtained. For the energy distribution matrix Perform min-max normalization and take the complement to obtain the redundancy distribution matrix. : in, Describes the minimum value function. Represents the maximum value function; Redundancy distribution matrix The size is the same as the size of the potential representation, which is... That is, C is the number of channels, H×W is the height and width, and the redundancy distribution matrix. The value at each position reflects the potential redundancy of the representation at that coordinate.

4. The method according to claim 3, characterized in that, The method of fusing the carrier image with the initial watermark features using the redundant distribution matrix through the redundant guided fusion component to obtain a watermarked latent representation includes: The redundant guided fusion component adopts a structure based on multi-level convolution and non-linear activation coupling. It includes a latent representation acquisition module, a watermark feature redistribution module, and a feature fusion module. The latent representation acquisition module is implemented through an encoder. Carrier image dataset One of the images is the size of carrier image Convert to size The potential representation of; Initial watermark features The redundancy distribution matrix R is input to the watermark feature redistribution module. R serves as a spatial-channel masking guidance signal. Through the mapping of the watermark feature redistribution module, the initial watermark features are intensity modulated to obtain redistributed watermark features. The redistributed watermark features are then fused with the latent representation of the carrier through a feature fusion module. The redistributed watermark signal is seamlessly embedded into the redundant sites of the carrier's latent representation, and the output is a watermarked latent representation. .

5. The method according to claim 4, characterized in that, In the decoding section, the watermark features are extracted from the latent representation of the watermark using a watermark feature decoding component, and the watermark image is recovered using the watermark features by a watermark reconstructor, including: The watermark encoder consists of a watermark feature decoding network and a watermark reconstructor. The watermark feature decoding network has a multi-level structure coupled with convolutional and non-linear activations. It enhances feature representation by expanding the channel dimension of intermediate layers, and then performs feature compression and refinement layer by layer to output a feature map that matches the dimension of the potential representation containing the watermark. The network structure of the watermark reconstructor is consistent with the pre-trained VAE decoder in LDM. During the training phase, the network undergoes end-to-end parameter optimization. The weight parameters inside the watermark reconstructor are tailored to the watermark features. Evolution; The watermark feature decoding network uses an encoder From the potential representation of watermark Extract watermark features Watermark features Input into watermark reconstructor In the middle, watermark reconstructor Watermark features Spatial dimension restoration and detail filling are performed to reconstruct the abstract feature signals into a watermark image. ; in, This represents a watermark feature decoding network.

Images