Multi-step latent space watermarking method for diffusion models
Patent Information
- Application Number
- CN202610832085.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-06-10
- Publication Date
- 2026-08-18
AI Technical Summary
[0007]上述现有技术有两个较为突出的缺点
[0044]Compared with existing technologies, this invention designs a watermark encoding network to encode the watermark information into a watermark residual. The watermark is then embedded into the generated image by embedding this residual into the latent space. Extraction also requires extraction from the latent space obtained by mapping the watermark image. A multi-step watermark embedding strategy is designed, injecting the watermark residual obtained from the robust latent space watermark encoding network into the multi-step denoising generation process of the latent space diffusion model. An intermediate latent variable is constructed as the embedding domain through a reparameter-based denoising process, and wavelet filtering is used to process the watermark residual to improve the image's perceptual quality.
Smart Images
Figure CN122597149A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of image watermarking technology, and more particularly to a multi-step latent space watermarking method for diffusion models. Background Technology
[0002] In recent years, image generation technologies, represented by diffusion models, have developed rapidly. Typical models, such as the Stable Diffusion model, have achieved high levels of image quality and realism, making it difficult for human observers to effectively distinguish between generated and real images. However, this technological advancement has also brought potential risks, such as the use of generative models to commit illegal activities like deepfakes. Furthermore, the copyright protection and provenance issues of high-quality works created by artists using diffusion models urgently require effective solutions.
[0003] Existing watermarking methods employ the Stable Signature hidden watermarking technique. This technique involves fine-tuning the VAE (Variational Autoencoder) decoder of the Stable Diffusion latent diffusion model, adding a fixed binary watermark signature to the Stable Diffusion latent diffusion model, recovering the hidden signature using a pre-trained watermark extractor, and then determining whether the generated image originates from the generative model.
[0004] In the first stage, the HiDDeN watermarking coding network is pre-trained. HiDDeN is a classic image watermarking coding network. After training, the HiDDeN encoder is discarded, and only the decoder is retained for the next stage of training. The first stage of training differs from the original HiDDeN training process, mainly in that, since the encoder is not retained, its perceptual quality is no longer important, and the perceptual loss is removed from the loss function of the first stage training. In addition, the tanh function and scaling factor are applied to constrain distortion, thereby improving the bit accuracy of watermark recovery.
[0005] In the second stage, the VAE decoder of the Stable Diffusion latent diffusion model is fine-tuned using fixed watermark information as a condition to obtain a watermarked VAE decoder. The latent variables obtained after the denoising process are input into the watermarked VAE decoder to generate an image, ensuring that the image contains watermark information that can be extracted by the HiDDeN decoder trained in the first stage. During fine-tuning, the watermark signature is fixed. The VAE decoder reconstructs the watermarked image from the latent variables, and the HiDDeN decoder recovers the watermark. The cross-entropy loss of the recovered watermark and watermark signature is calculated as part of the loss function.
[0006] The above process ensures the effectiveness of the watermark while minimizing its impact on image reconstruction capabilities and watermark image quality. The distance between the watermark-free original generated image and the watermarked image is calculated using the perceptual loss function Watson-VGG as another part of the loss function. After completing the above two-stage training, the VAE decoder in the original latent diffusion model Stable Diffusion is replaced with the fine-tuned watermarked VAE decoder. This allows watermark information to be embedded during the generation process, resulting in a generated image with a fixed watermark signature. Feeding the watermarked image into the watermark decoder obtained in the first stage completes the recognition and extraction of the watermark information.
[0007] The aforementioned existing technologies have two significant drawbacks. First, these methods rely on specific optimization of the generator model's decoder parameters to embed watermark information. Directly modifying the model weights may alter the original generation performance, affecting the perceptual quality of the watermarked image. Second, although the method exhibits some robustness under conventional image processing (such as light compression and cropping), the accuracy of embedded watermark information extraction may significantly decrease when faced with strong perturbations (such as strong Gaussian noise, VAE regeneration attacks, and StableDiffusion regeneration attacks), thereby reducing the success rate of watermark extraction. Furthermore, this method can only embed fixed watermark information with each fine-tuning, limiting its flexibility. Summary of the Invention
[0008] The purpose of this invention is to overcome the defects and shortcomings of the prior art and provide a multi-step latent space watermarking method for diffusion models. This method combines watermarking coding networks and latent diffusion models using a multi-step embedding strategy to achieve the embedding of highly robust image watermarks during the generation process without significantly affecting visual quality.
[0009] To achieve the above objectives, the technical solution adopted by the present invention is as follows:
[0010] A multi-step latent space watermarking method for diffusion models includes the following steps:
[0011] The watermark information is input into the watermark encoding network to obtain the watermark residual;
[0012] The initial latent variables are input into the latent diffusion model, and after a standard denoising process before the preset time step, the latent variables corresponding to the preset time step are obtained.
[0013] A denoising process combining watermark embedding is performed on the latent variables. The watermark residual is then filtered by wavelet and injected into the clean latent variables predicted at the current time step to obtain the clean latent variables predicted at the current time step with watermark embedded.
[0014] For the clean latent variables predicted at the current time step with embedded watermarks, perform posterior sampling of the latent diffusion model to obtain the latent variables for the next time step;
[0015] Repeat the above denoising process combined with watermark embedding until all remaining time steps are denoised, and finally obtain the clean latent variable with embedded watermark.
[0016] The clean latent variables with embedded watermarks are input into the VAE decoder of the latent diffusion model to obtain the watermarked generated image.
[0017] Furthermore, the initial latent variables are randomly sampled from a Gaussian distribution or obtained through specific image inversion;
[0018] For noise reduction time step ,in This indicates a predefined time step threshold, and standard denoising steps are performed; when At that time, a reverse denoising step incorporating a multi-step watermark embedding strategy is performed.
[0019] Furthermore, the watermark encoding network includes a watermark encoder. and watermark decoder Watermark encoder Employing a carrier-independent design to embed watermark information Mapped to watermark residual Subsequently, it is embedded into latent variables during the denoising process. In this way, latent variables with watermarks are generated. .
[0020] Furthermore, watermark encoder Employing a lightweight architecture, random bit watermark information First, it is processed through a two-layer fully connected network using the SiLU activation function and then reshaped into a network of size [size missing]. The feature map; this feature map is tiled in space as To match the resolution of latent variables; then apply Convolution expands the channel dimension to the latent space channel dimension. Consistent, then enter Convolution is used to construct local features, thereby obtaining the watermark residual. .
[0021] Furthermore, watermark decoder Employing a watermark encoder Symmetrical design, watermark decoder From embedded watermark The watermark information was reconstructed from the middle. ;
[0022] Given input First use Convolution is used for feature extraction, followed by... Convolution; the resulting feature map is divided into segments of size [size missing]. The aggregated features are then averaged to obtain a global representation; subsequently, this aggregated feature is fed into a fully connected network to reconstruct the watermark information. .
[0023] Furthermore, the training process for the watermark coding network includes:
[0024] By combining image fidelity loss Loss of watermark information reconstruction The joint objective function is optimized; given the original images from the training set. and its watermarked images Image loss is defined as:
[0025] ;
[0026] in, The coefficients are constants. This represents the loss of perceived similarity. This is the mean square error loss;
[0027] message loss Defined as original watermark With the reconstructed watermark Binary cross-entropy between them;
[0028] The overall training objective is defined as:
[0029] ;
[0030] in, The weighting coefficients are adaptive; to balance visual fidelity and decoding accuracy, a gradual weighting adjustment strategy is adopted, i.e., only when... Only after reaching a stable threshold should the amount be gradually increased. .
[0031] Furthermore, in a multi-step watermark embedding strategy, the watermark residual... Not only injected into clean latent variables In addition, it is also injected into the latent variables in the multi-step denoising process at a later time step;
[0032] Within the standard noise prediction framework of the latent diffusion model, the standard denoising steps are expressed as follows:
[0033] ;
[0034] in, , and It is a coefficient defined by the diffusion time step scheduler and varies with the time step; To predict noise; when time step Perform standard noise reduction steps at the time; for The denoising step is performed by incorporating a multi-step watermark embedding strategy.
[0035] Furthermore, regarding the watermark residual Low-pass filtering based on Haar wavelet was used;
[0036] First of all A first-order discrete wavelet transform is performed, followed by suppressing the high-frequency subbands by setting their coefficients to zero. Then, the filtered watermark residual is reconstructed using an inverse discrete wavelet transform. The filtering process... Defined as:
[0037] ;
[0038] in, It is a mask that retains only one low-frequency sub-band while discarding three high-frequency sub-bands;
[0039] The denoising process combining a multi-step watermark embedding strategy is defined as follows:
[0040] ;
[0041] In order to adjust the embedding strength, the injected filter residuals It will be multiplied by a coefficient. .
[0042] An electronic device includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the multi-step latent space watermarking method for diffusion models as described above.
[0043] A storage medium, a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the multi-step latent space watermarking method for diffusion models as described above.
[0044] Compared with existing technologies, this invention designs a watermark encoding network to encode the watermark information into a watermark residual. The watermark is then embedded into the generated image by embedding this residual into the latent space. Extraction also requires extraction from the latent space obtained by mapping the watermark image. A multi-step watermark embedding strategy is designed, injecting the watermark residual obtained from the robust latent space watermark encoding network into the multi-step denoising generation process of the latent space diffusion model. An intermediate latent variable is constructed as the embedding domain through a reparameter-based denoising process, and wavelet filtering is used to process the watermark residual to improve the image's perceptual quality.
[0045] This invention integrates watermark embedding into a multi-step denoising generation process of a latent space diffusion model, enabling the watermark signal to be mapped as part of the generated image content. This achieves strong robustness while maintaining image visual fidelity. The multi-step watermark embedding method employed in the diffusion model denoising process significantly improves robustness under strong attack conditions and achieves good image perceptual quality. Attached Figure Description
[0046] Figure 1 This is a flowchart of a multi-step latent space watermarking method used in a diffusion model.
[0047] Figure 2 This is a flowchart of the image generation process for a multi-step latent watermarking method.
[0048] Figure 3 This is a flowchart of the watermark extraction process for a multi-step latent space watermarking method.
[0049] Figure 4 This is a structural diagram of the watermark encoder in a watermark encoding network.
[0050] Figure 5 This is a diagram of the watermark decoder structure of a watermark encoding network.
[0051] Figure 6 This is a flowchart of the training process for the watermark coding network. Detailed Implementation
[0052] The multi-step latent space watermarking method for diffusion models of the present invention will be further described below with reference to the accompanying drawings and specific embodiments.
[0053] Please see Figure 1 and Figure 2 This invention discloses a multi-step latent space watermarking method for diffusion models, comprising the following steps:
[0054] The watermark information is input into the watermark encoding network to obtain the watermark residual;
[0055] The initial latent variables are input into the latent diffusion model, and after a standard denoising process before the preset time step, the latent variables corresponding to the preset time step are obtained.
[0056] A denoising process combining watermark embedding is performed on the latent variables. The watermark residual is then filtered by wavelet and injected into the clean latent variables predicted at the current time step to obtain the clean latent variables predicted at the current time step with watermark embedded.
[0057] For the clean latent variables predicted at the current time step with embedded watermarks, perform posterior sampling of the latent diffusion model to obtain the latent variables for the next time step;
[0058] Repeat the above denoising process combined with watermark embedding until all remaining time steps are denoised, and finally obtain the clean latent variable with embedded watermark.
[0059] The clean latent variables with embedded watermarks are input into the VAE decoder of the latent diffusion model to obtain the watermarked generated image.
[0060] This invention improves upon watermarking encoding networks by designing a latent space watermarking encoding network. This network enables the encoding of random bit watermark information in the latent space and the extraction of watermark information from the latent space. The watermark encoder and decoder of the watermark encoding network employ a symmetrical structure. The watermark encoder is carrier-independent, directly mapping the bit watermark to a watermark residual, which is then added to the latent variable to obtain the watermark-containing latent variable. During extraction, the watermark image needs to be fed into the encoder of the StableDiffusion latent diffusion model to be remapped back to the latent space domain before extraction. This allows for the acceptance of random watermark information as input, improving the flexibility of the watermarking method. The carrier-independent design allows the network to learn and output watermark embeddings that are invariant across coverage, improving the robustness of the watermark against channel attacks.
[0061] This invention improves upon the combination strategy of watermark encoding network and Stable Diffusion latent diffusion model by proposing a multi-step watermark embedding strategy. After completing the first stage of watermark encoding network training, both the watermark encoder and decoder are retained. During the latent space denoising process of the Stable Diffusion latent diffusion model, when time step... At that time, the watermark residual is processed by wavelet filtering, multiplied by a fixed coefficient, and latent variables are added. Values obtained from single-step inversion In, and in A watermark residual is also added. The final watermarked latent variable is fed into the VAE decoder of the Stable Diffusion latent diffusion model to obtain the watermarked image. This design significantly improves the robustness of the image watermark under strong channel attacks and regeneration attacks, while achieving good perceptual quality.
[0062] like Figure 2 As shown, the multi-step latent space watermarking method embeds the watermark during image generation. The watermark coding network includes a watermark encoder. and watermark decoder The Stable Diffusion latent diffusion model includes a latent space denoising network and a VAE encoder. and VAE decoder .
[0063] The watermark image generation process begins with the initial latent variables. The initial latent variables can be randomly sampled from a Gaussian distribution or obtained through specific image inversion. For the denoising time step... ,in This indicates a predefined time step threshold, after which standard denoising steps are performed. At that time, a reverse denoising step incorporating a multi-step watermark embedding strategy is performed.
[0064] Specifically, firstly, the random watermark information Watermark encoder of input watermark encoding network To generate watermark residuals Subsequently, a multi-step watermark embedding strategy was used to inject it into the denoising trajectory. For the denoising time step... The predicted noise output by the denoising network of the latent diffusion model Single-step inversion explicit construction Clean latent variables predicted step by step The watermark residual after wavelet filtering is injected again. The first watermark embedded is obtained Clean latent variables predicted step by step Then, further noise was added to obtain the first... Step latent variables After completing the full denoising process, clean latent variables with watermarks can be obtained. Subsequently, the VAE decoder of the latent diffusion model... Decode it to generate the final watermarked image. .
[0065] like Figure 3 As shown, the watermark extraction process utilizes the VAE encoder of the Stable Diffusion latent diffusion model. And the watermark decoder of the watermark encoding network .
[0066] For watermarked images that may be distorted due to reconstruction attacks caused by lossy transmission. First, through the VAE encoder Encode them to obtain the reconstructed clean latent variables. Subsequently, the obtained clean latent variables The watermark decoder is input into the watermark encoding network. In order to recover and reconstruct the watermark information. By reconstructing the watermark information Compared with the original watermark information The extraction accuracy is calculated through comparison. Reliable watermark recovery enables watermark image recognition and effective image provenance tracing based on extraction accuracy, thereby supporting reliable copyright authentication.
[0067] like Figure 4 As shown, the watermark encoder of the watermark encoding network A carrier-independent design was adopted to embed the watermark information. Mapped to watermark residual Subsequently, it is embedded into latent variables during the denoising process. In this way, latent variables with watermarks are generated. This design eliminates The embedding's dependence on a specific data distribution enables efficient training with limited data and iterations, and facilitates generalization to diverse latent representations in the Stable Diffusion model. Furthermore, learning a cross-carrier invariant watermark embedding enhances its robustness to common distortions.
[0068] Specifically, Lightweight architecture employed. Random bit watermark information. First, it is processed through a two-layer fully connected network using the SiLU activation function and then reshaped into a network of size [size missing]. The feature map. This feature map is tiled in space as To match the resolution of the latent variables. Then apply... Convolution expands the channel dimension to the latent space channel dimension. Consistent, then enter Convolution is used to construct local features, thereby obtaining the watermark residual. .
[0069] like Figure 5 As shown, the watermark decoder of the watermark encoding network A design symmetrical to the watermark encoder is adopted, which stabilizes gradient propagation and improves the performance of watermark information reconstruction. The watermark decoder of the watermark encoding network... The main task is to extract watermarks from embedded watermarks. The watermark information was reconstructed from the middle. .
[0070] Given input First use Convolution is used for feature extraction, followed by... Convolution. The resulting feature map is divided into segments of size [size missing]. The aggregated features are then averaged to obtain a global representation. This aggregated feature is then fed into a fully connected network to reconstruct the watermark information. .
[0071] Figure 6 This demonstrates the training process of a watermark coding network, combining image fidelity loss. Loss of watermark information reconstruction The joint objective function is optimized. Given the original images from the training set... and the image after watermarking. Image loss is defined as:
[0072] ;
[0073] in, The constant coefficients, This represents the loss of perceived similarity, while Mean squared error loss. Message loss. Defined as original watermark With the reconstructed watermark The binary cross-entropy between them. The overall training objective is defined as:
[0074] ;
[0075] in, These are adaptive weighting coefficients. To balance visual fidelity and decoding accuracy, a progressive weighting strategy is adopted, meaning that weight adjustment only occurs when... Only after reaching a stable threshold should the amount be gradually increased. This prevents image fidelity constraints from dominating early training and allows the model to learn effective watermark embedding and extraction first. Furthermore, once training converges, a noise layer is introduced for fine-tuning to improve robustness.
[0076] In multi-step watermark embedding strategies, watermark residuals Not only injected into clean latent variables Furthermore, the embeddings are also injected into the latent variables of the multi-step denoising process at later time steps. By restricting the embeddings to these time steps, the robustness and visual fidelity of the watermark are balanced, while perceptible artifacts are avoided in the generated images.
[0077] Within the standard noise prediction framework of the latent diffusion model, the standard denoising steps can be expressed as:
[0078] ;
[0079] in, , and It is a coefficient defined by the diffusion time-step scheduler and varies with the time step, while It is prediction noise. When the time step... At that time, standard noise reduction steps are performed. For The process involves a denoising step incorporating a multi-step watermark embedding strategy, which restructures the denoising process and ensures clean latent variables. Watermark embedding is performed in a distributed, aligned latent space. In the... After denoising, the first step is explicitly constructed through single-step inversion. Clean latent variables predicted step by step Thus, we obtain the clean latent variables. Statistically consistent representation Then, watermarks are embedded in them.
[0080] However, directly adding watermark residuals This may introduce a cumulative effect, causing abnormal biases in latent variables, which in turn can lead to visible artifacts and degrade image quality. To mitigate this problem, the watermark residual... Low-pass filtering based on Haar wavelets was used. Specifically, firstly, for... A first-order discrete wavelet transform is performed, followed by suppressing the high-frequency subbands by setting their coefficients to zero. Then, the filtered residuals are reconstructed using an inverse discrete wavelet transform. The filtering process... Defined as:
[0081] ;
[0082] Where DWT stands for wavelet transform and IDWT stands for inverse wavelet transform. As a mask, it retains only one low-frequency sub-band while discarding three high-frequency sub-bands.
[0083] Therefore, the denoising process combining a multi-step watermark embedding strategy is defined as follows:
[0084] ;
[0085] In order to adjust the embedding strength, the injected filter residuals It will be multiplied by a coefficient. .
[0086] In terms of implementation details, the watermarking encoding network used 7600 images from the Flickr8k dataset as its training set. The images were preprocessed to a size of 512 × 512 before training. During training, the bit information input to the network along with the images was randomly generated, with a batch size of 4 samples. The generator part used the AdamW optimizer with a learning rate of 0.0001. For 48, 96, and 128-bit capacities, the network was trained for 6, 12, and 15 epochs, respectively. Set to 5. Initialized to 0, when When it increases to 0.5, The time was further set to 1.0, and a noise attack layer was introduced in the last 3 epochs to enhance the network robustness.
[0087] For multi-step watermark embedding strategies, the time step threshold Set to 100, scaling factor Set it to 0.6.
[0088] The noise attack layer used during robust training consists of multiple types of attack operations. In each iteration of robust training, each attack is applied independently with a predefined selection probability, and its parameters are randomly sampled from a predefined parameter range. All attack operations are implemented using the Kornia library. Detailed attack settings are summarized in Table 1.
[0089] Table 1 Noise Attack Layer Settings
[0090]
[0091] Regarding robustness, the method of this invention employs a multi-step embedding strategy to fuse the watermark coding network into the latent diffusion model Stable Diffusion, achieving significantly better robustness than existing state-of-the-art methods. Table 2 shows the comparison of robustness under channel attacks and regeneration attacks at a capacity of 48 bits. The results indicate that the average extraction accuracy of the method of this invention is significantly better than PRC (Pseudo Random Code) and Stable Signature under both channel attacks and regeneration attacks.
[0092] Table 2 Robustness test results
[0093]
[0094] Regarding the perceived quality of watermarked images, the method of this invention integrates the latent space watermark signal into the denoising generation process of the Stable Diffusion latent diffusion model, and utilizes the powerful generation capability of the Stable Diffusion latent diffusion model itself to map the watermark signal as part of the image content, thus achieving better perceived image quality.
[0095] Table 2 shows the comparison results of image perception quality indicators under the 48-bit capacity condition. The results show that the method of the present invention has achieved good scores of mainstream perception quality indicators such as NIQE (Natural Image Quality Evaluator), PIQE (Blind Image Quality Evaluation Using Perception Based Features), and BRISQUE (Blind / Referenceless Image Spatial Quality Evaluator).
[0096] Table 3. Perceived Quality Test Results
[0097]
[0098] This invention also discloses an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the multi-step latent space watermarking method for diffusion models described above. The electronic device of this invention can execute the multi-step latent space watermarking method for diffusion models of this invention, and can execute any combination of the steps of the method embodiments, possessing the corresponding functions and beneficial effects of the method.
[0099] This invention also discloses a storage medium, a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the multi-step latent space watermarking method for diffusion models described in any of the preceding claims. The computer-readable storage medium of this invention can execute the multi-step latent space watermarking method for diffusion models of this invention, and can execute any combination of the implementation steps of the method embodiments, possessing the corresponding functions and beneficial effects of the method.
[0100] The above description is a detailed description of the preferred embodiments of the present invention. However, the embodiments are not intended to limit the scope of the patent application of the present invention. All equivalent changes or modifications made under the technical spirit disclosed in the present invention should fall within the patent scope covered by the present invention.
Claims
1. A multi-step latent space watermarking method for diffusion models, characterized in that, Includes the following steps: The watermark information is input into the watermark encoding network to obtain the watermark residual; The initial latent variables are input into the latent diffusion model, and after a standard denoising process before the preset time step, the latent variables corresponding to the preset time step are obtained. A denoising process combining watermark embedding is performed on the latent variables. The watermark residual is then filtered by wavelet and injected into the clean latent variables predicted at the current time step to obtain the clean latent variables predicted at the current time step with watermark embedded. For the clean latent variables predicted at the current time step with embedded watermarks, perform posterior sampling of the latent diffusion model to obtain the latent variables for the next time step; Repeat the above denoising process combined with watermark embedding until all remaining time steps are denoised, and finally obtain the clean latent variable with embedded watermark. The clean latent variables with embedded watermarks are input into the VAE decoder of the latent diffusion model to obtain the watermarked generated image.
2. The multi-step latent space watermarking method for diffusion models according to claim 1, characterized in that, The initial latent variables are randomly sampled from a Gaussian distribution or obtained through specific image inversion; For noise reduction time step ,in This indicates a predefined time step threshold, and standard denoising steps are performed; when At that time, a reverse denoising step incorporating a multi-step watermark embedding strategy is performed.
3. The multi-step latent space watermarking method for diffusion models according to claim 1, characterized in that, Watermarking encoding networks include watermark encoders and watermark decoder Watermark encoder Employing a carrier-independent design to embed watermark information Mapped to watermark residual Subsequently, it is embedded into latent variables during the denoising process. In this way, latent variables with watermarks are generated. .
4. The multi-step latent space watermarking method for diffusion models according to claim 3, characterized in that, Watermark encoder Employing a lightweight architecture, random bit watermark information First, it is processed through a two-layer fully connected network using the SiLU activation function and then reshaped into a network of size [size missing]. The feature map; this feature map is tiled in space as To match the resolution of latent variables; then apply Convolution expands the channel dimension to the latent space channel dimension. Consistent, then enter Convolution is used to construct local features, thereby obtaining the watermark residual. .
5. The multi-step latent space watermarking method for diffusion models according to claim 4, characterized in that, Watermark decoder Employing a watermark encoder Symmetrical design, watermark decoder From embedded watermark The watermark information was reconstructed from the middle. ; Given input First use Convolution is used for feature extraction, followed by... Convolution; the resulting feature map is divided into segments of size [size missing]. The aggregated features are then averaged to obtain a global representation; subsequently, this aggregated feature is fed into a fully connected network to reconstruct the watermark information. .
6. The multi-step latent space watermarking method for diffusion models according to claim 5, characterized in that, The training process for a watermark coding network includes: By combining image fidelity loss Loss of watermark information reconstruction The joint objective function is optimized; given the original images from the training set. and its watermarked images Image loss is defined as: ； in, The coefficients are constants. This represents the loss of perceived similarity. This is the mean square error loss; message loss Defined as original watermark With the reconstructed watermark Binary cross-entropy between them; The overall training objective is defined as: ； in, The weighting coefficients are adaptive; to balance visual fidelity and decoding accuracy, a gradual weighting adjustment strategy is adopted, i.e., only when... Only after reaching a stable threshold should the amount be gradually increased. .
7. The multi-step latent space watermarking method for diffusion models according to claim 6, characterized in that, In multi-step watermark embedding strategies, watermark residuals Not only injected into clean latent variables In addition, it is also injected into the latent variables in the multi-step denoising process at a later time step; Within the standard noise prediction framework of the latent diffusion model, the standard denoising steps are expressed as follows: ； in, , and It is a coefficient defined by the diffusion time step scheduler and varies with the time step; To predict noise; when time step Perform standard noise reduction steps at the time; for The denoising step is performed by incorporating a multi-step watermark embedding strategy.
8. The multi-step latent space watermarking method for diffusion models according to claim 7, characterized in that, Watermark residual Low-pass filtering based on Haar wavelet was used; First of all A first-order discrete wavelet transform is performed, followed by suppressing the high-frequency subbands by setting their coefficients to zero. Then, the filtered watermark residual is reconstructed using an inverse discrete wavelet transform. The filtering process... Defined as: ； in, It is a mask that retains only one low-frequency sub-band while discarding three high-frequency sub-bands; The denoising process combining a multi-step watermark embedding strategy is defined as follows: ； In order to adjust the embedding strength, the injected filter residuals It will be multiplied by a coefficient. .
9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes a computer program, it implements the multi-step latent space watermarking method for diffusion models as described in any one of claims 1 to 8.
10. A storage medium, a computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the multi-step latent space watermarking method for diffusion models as described in any one of claims 1 to 8.