Robust image watermarking method, system and terminal based on conditional diffusion model

Abstract

The invention relates to a robust image watermarking method, system and terminal based on a conditional diffusion model, and relates to the field of robust image watermarking technology, and the method comprises the steps: carrying out the training step in a circulating manner according to the training step information, and the training step comprises the steps: obtaining the training original image information and the training watermark information; determining training condition noise image information according to the training original image information; the training condition noise image information, the training original image information and the training watermark information serve as input to control a training encoder to generate coded image information; taking the coded image information as input to control an attack layer to generate attack image information; the attack image information serves as input to control a training decoder to generate training decoding watermark information; and calculating loss function information according to the training original image information, the encoding image information, the training watermark information and the training decoding watermark information, and updating encoder parameter information and decoder parameter information according to a gradient descent method. The method has the effect of improving the watermark performance.

Description

Technical Field

[0001] The present application relates to the field of robust image watermarking technology, and in particular to a robust image watermarking method, system and terminal based on a conditional diffusion model. Background Art

[0002] Robust image watermarking technology is a technology that embeds information in digital images. This information can still be detected or extracted even after the image has been processed or attacked in various ways. This technology is widely used in copyright protection, integrity verification, tracking and authentication. Robustness refers to the ability of the watermark to still be recognizable after undergoing certain image processing operations (such as compression, filtering, cropping, etc.).

[0003] In related technologies, image watermarking methods are usually based on an end-to-end watermarking framework based on deep neural networks. The framework mainly adopts an encoder-attack layer-decoder structure, in which the encoder extracts the features of the image and fuses them with the watermark to obtain the encoded image (including the watermarked image). The attack layer simulates the attacks that the encoded image may encounter during transmission, and the decoder extracts the watermark from the attacked image. In addition, the discriminator is introduced into the framework, and the discriminator and the encoder form a generative adversarial network for training to improve image quality.

[0004] Regarding the above-mentioned related technologies, the training process of the generative adversarial network composed of the encoder and the discriminator is a dynamic, non-cooperative game process, in which the encoder and the discriminator try to maximize their respective objective functions. This adversarial relationship may lead to training instability and pattern collapse, thereby affecting the watermark performance. There is still room for improvement. Summary of the invention

[0005] In order to improve watermark performance, the present application provides a robust image watermark method, system and terminal based on a conditional diffusion model.

[0006] In a first aspect, the present application provides a robust image watermarking method based on a conditional diffusion model, which adopts the following technical solution: A robust image watermarking method based on a conditional diffusion model, including: Executing preset training steps cyclically according to preset training step information, the training steps comprising: acquiring training original image information and training watermark information; Analyze and determine the training conditional noise image information according to the training original image information; Using the training conditional noise image information, the training original image information and the training watermark information as input to control a preset training encoder to generate encoded image information; Using the encoded image information as input to control the preset attack layer to generate attack image information; Using the attack image information as input to control a preset training decoder to generate training decoding watermark information; The loss function information is calculated according to the training original image information, the encoded image information, the training watermark information and the training decoded watermark information, and the encoder parameter information of the training encoder and the decoder parameter information of the training decoder are updated according to a preset gradient descent method.

[0007] By adopting the above technical scheme and a multi-encoder structure guided by conditional noise images, the watermark can be effectively adjusted and embedded in an imperceptible and robust area. Compared with the existing watermarking framework, the discriminator is removed, so it does not rely on adversarial training. Data is gradually reconstructed from noise in a series of predictable steps, and the true distribution of the data is more smoothly approached. Each step follows a conditional probability distribution, ensuring the stability and controllability of the entire process, while improving image quality and watermark robustness.

[0008] Optionally, the step of analyzing and determining the training conditional noise image information according to the training original image information includes: Get the current time step information; Analyze and determine Gaussian noise intensity hyperparameter information based on current time step information and preset hyperparameter relationship information; Get the sampled Gaussian noise information; The training conditional noise image information is determined by analyzing the sampled Gaussian noise information, the Gaussian noise intensity hyperparameter information and the training original image information.

[0009] By adopting the above technical solution, Gaussian noise is introduced into the original image to generate a training conditional noise image, and the watermark embedding is guided according to the conditional noise image, which helps the encoder to better understand which parts of the image are suitable for embedding watermarks without affecting the quality of the image, and enables the model to learn how to maintain the recoverability of the watermark under different noise conditions.

[0010] Optionally, the step of using the training conditional noise image information, the training original image information and the training watermark information as input to control a preset training encoder to generate encoded image information includes: Analyze and determine original feature information based on training condition noise image information and training original image information; Perform multi-scale feature extraction and down-sampling on the original feature information to generate feature map information; Preprocessing the training watermark information to generate watermark feature information; The feature map information and watermark feature information are fused to generate encoded image information.

[0011] By adopting the above technical solution, multi-scale feature extraction and dimensionality reduction are performed on the training conditional noise image information and the training original image information to obtain feature map information, thereby improving the feature utilization of both and reducing the dimension of the features, thereby improving the representation ability of the network, and preprocessing the watermark to adjust the size of the watermark feature so that the watermark feature matches the corresponding image feature scale.

[0012] Optionally, the step of fusing the feature map information and the watermark feature information to generate the encoded image information includes: Determine the feature information of the watermarked image by performing fusion analysis based on the feature map information, watermark feature information and current time step information; Normalize the feature information of the watermarked image and perform self-attention calculation to generate self-attention feature information; Perform fusion analysis based on self-attention feature information and current time step information to determine enhanced feature information; The coded image information is generated by performing fusion analysis and upsampling based on the enhanced feature information, watermark feature information and feature map information.

[0013] By adopting the above technical solution, the feature information of the watermarked image is standardized and self-attention is calculated, so as to ensure that the data will not be unstable due to excessive scale differences in the subsequent processing process. At the same time, the self-attention mechanism enables the model to focus on the relationship between different parts of the input and better capture global dependencies.

[0014] Optionally, after the training step is executed cyclically according to the preset training step information, a testing step is also included, and the specific steps include: According to the preset non-uniform sampling step number information, the training encoder after the encoder parameter information is updated is selected as the test encoder, and the training decoder after the decoder parameter information is updated is selected as the test decoder; Acquire test condition noise image information, test original image information and test watermark information; Using the test condition noise image information, the test original image information and the test watermark information as input to control the test encoder to generate the final watermarked image information; The final watermarked image information is used as input to control the test decoder to generate test decoded watermark information; Model evaluation information is generated by analyzing the final watermarked image information, the test original image information, the test watermark information and the test decoded watermark information.

[0015] By adopting the above technical scheme, after the training of multiple encoders is completed, the training encoder is selected as the test encoder, so as to control the test encoder to encode and obtain the final watermarked image information, thereby speeding up the testing process, and then control the test decoder to decode and generate test decoded watermark information, and evaluate the model according to the final watermarked image information, the test original image information, the test watermark information and the test decoded watermark information, thereby improving the accuracy of the model.

[0016] Optionally, the step of using the test condition noise image information, the test original image information and the test watermark information as input to control the test encoder to generate the final watermarked image information includes: Get the number of iterations; Analyze and determine the current iteration step information and the next iteration step information according to the information of the number of iterations; Based on the current iteration step information, the corresponding test condition noise image information is obtained; Determine denoised image information by analyzing according to next iteration step information, current iteration step information, test condition noise image information, test original image information and test watermark information; Determine whether the information of the number of iterations meets the requirements of the non-uniform sampling step number information; If not, proceed to the next iteration and continue to obtain the information of the number of iterations to make a cyclic judgment; If it meets the requirements, the denoised image information is defined as the final watermarked image information.

[0017] By adopting the above technical solution, a series of iterative refinements are performed starting from completely random test condition noise image information, thereby removing the noise components in the image during the denoising process while retaining important structural information, thereby improving the image clarity and the robustness of the watermark.

[0018] Optionally, the step of analyzing and determining the current iteration step information and the next iteration step information according to the information of the number of iterations includes: Analyze and determine step number variation coefficient information based on training step information, non-uniform sampling step number information and preset variable information; Analyze and determine the current iteration step information based on the number of iterations, training step information, step change coefficient information and variable information; Analyze and determine the next iteration number information based on the information of the number of iterations completed; The next iteration step information is determined by analyzing the next iteration number information, training step information, step number change coefficient information and variable information.

[0019] By adopting the above technical solution, the change in the number of steps in each iteration is determined according to the relationship between the non-uniform sampling step information and the variable information, and the step change coefficient information is obtained, which is convenient for the subsequent calculation of the current iteration step information and the next iteration step information based on the number of iterations information and the next iteration number information, thereby improving the convenience of determining the current iteration step information and the next iteration step information.

[0020] In the second aspect, the present application provides a robust image watermarking system based on a conditional diffusion model, which adopts the following technical solutions: A robust image watermarking system based on a conditional diffusion model, comprising: An acquisition module, used to acquire training original image information and training watermark information; A memory, used to store a program of the robust image watermarking method based on the conditional diffusion model as described in any one of the above items; The program in the memory can be loaded and executed by the processor to implement the robust image watermarking method based on the conditional diffusion model as described in any one of the above items.

[0021] By adopting the above technical solution, the processor loads and executes the robust image watermarking method based on the conditional diffusion model stored in the memory, and controls the acquisition module to obtain a series of data related to the robust image watermarking based on the conditional diffusion model, so that the multi-encoder embeds the watermark into an imperceptible and robust area guided by the conditional noise image. Compared with the existing watermarking framework, the discriminator is removed, so that it does not rely on adversarial training, and data is gradually reconstructed from the noise in a series of predictable steps, which more smoothly approaches the true distribution of the data. Each step follows a conditional probability distribution, ensuring the stability and controllability of the entire process, while improving the image quality and watermark robustness.

[0022] In a third aspect, the present application provides a smart terminal, which adopts the following technical solution: An intelligent terminal comprises a memory and a processor, wherein the memory stores a computer program which can be loaded by the processor and executes any one of the above-mentioned robust image watermarking methods based on a conditional diffusion model.

[0023] By adopting the above technical solution, by operating the intelligent terminal, the processor loads and executes the computer program of the robust image watermarking method based on the conditional diffusion model stored in the memory, so that the multi-encoder embeds the watermark into the imperceptible and robust area guided by the conditional noise image. Compared with the existing watermarking framework, the discriminator is removed, so it does not rely on adversarial training, and the data is gradually reconstructed from the noise in a series of predictable steps, which more smoothly approaches the real distribution of the data. Each step follows the conditional probability distribution, which ensures the stability and controllability of the whole process, while improving the image quality and watermark robustness.

[0024] In summary, the present application includes at least one of the following beneficial technical effects: By adopting a multi-encoder structure guided by conditional noise images, the watermark can be effectively adjusted and embedded in an imperceptible and robust area. Compared with the existing watermarking framework, the discriminator is removed, so it does not rely on adversarial training. Data is gradually reconstructed from noise in a series of predictable steps, and the real distribution of data is more smoothly approached. Each step follows a conditional probability distribution, ensuring the stability and controllability of the whole process, while improving image quality and watermark robustness. By introducing Gaussian noise into the original image, a training conditional noise image is generated. The watermark embedding is guided by the conditional noise image, which helps the encoder better understand which parts of the image are suitable for embedding watermarks without affecting the quality of the image, and enables the model to learn how to maintain the recoverability of the watermark under different noise conditions. After completing the training of multiple encoders, the training encoder is selected as the test encoder, so as to control the test encoder to encode and obtain the final watermarked image information, thereby speeding up the testing process, and then controlling the test decoder to decode and generate the test decoded watermark information, and the model is evaluated according to the final watermarked image information, the test original image information, the test watermark information and the test decoded watermark information, thereby improving the accuracy of the model. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] Figure 1 It is a flow chart of a robust image watermarking method based on a conditional diffusion model in an embodiment of the present application.

[0026] Figure 2 It is a flowchart of the steps of analyzing and determining training conditional noise image information based on training original image information in an embodiment of the present application.

[0027] Figure 3 It is a flowchart of the steps of using training conditional noise image information, training original image information and training watermark information as input to control a preset training encoder to generate encoded image information in an embodiment of the present application.

[0028] Figure 4 It is a flowchart of the steps of fusing feature map information and watermark feature information to generate encoded image information in an embodiment of the present application.

[0029] Figure 5 It is a flow chart of the test steps in the embodiment of the present application.

[0030] Figure 6It is a flowchart of the steps of using the test condition noise image information, the test original image information and the test watermark information as input to control the test encoder to generate the final watermarked image information in an embodiment of the present application.

[0031] Figure 7 It is a flowchart of the steps of analyzing and determining the current iteration step information and the next iteration step information according to the information of the number of iterations in an embodiment of the present application.

[0032] Figure 8 It is a framework diagram of the robust image watermarking method based on the conditional diffusion model in the embodiment of the present application.

[0033] Fig. 9 It is a structural diagram of preprocessing training watermark information in an embodiment of the present application.

[0034] Fig.10 It is a structural diagram for determining characteristic information of a watermarked image in an embodiment of the present application.

[0035] Fig.11 This is a structural diagram of generating self-attention feature information in an embodiment of the present application.

[0036] Fig.12 It is a structural diagram of generating encoded image information in an embodiment of the present application. DETAILED DESCRIPTION

[0037] In order to make the purpose, technical solutions and advantages of this application more clear, the following Figure 1-Figure 12 It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0038] The embodiment of the present application discloses a robust image watermarking method based on a conditional diffusion model, and specifically discloses an encoding part, an attack layer and a decoding part. The encoding part adopts a multi-encoder structure guided by a conditional noise image, determines the training conditional noise image information according to the training original image information, and then inputs the training conditional noise image information, the training original image information and the training watermark information into the encoder, trains to generate the encoded image information, and then controls the attack layer to attack the encoded image information to generate the attack image information, so that the decoder decodes the attack image information to obtain the training decoded watermark information, and finally calculates the loss function information according to the training original image information, the encoded image information, the training watermark information and the training decoded watermark information, and adopts the gradient descent method to update the parameters of the encoder and the decoder in a loop, so as to complete the training of the encoder and the decoder without the need for a discriminator and without relying on adversarial training, gradually reconstructs data from the noise in a series of predictable steps, and more smoothly approaches the real distribution of the data, and each step follows the conditional probability distribution, ensuring the stability and controllability of the whole process, while improving the image quality and the robustness of the watermark.

[0039] Reference Figure 1 The embodiment of the present application discloses a robust image watermarking method based on a conditional diffusion model, comprising the following steps: Step S100: looping through preset training steps according to preset training step information, the training steps comprising: acquiring training original image information and training watermark information.

[0040] The training step information refers to the iterative step number T of the encoder and decoder training. In this embodiment, the iterative step number T is set to 1000. The training step refers to the method of training the encoder and decoder. For details, refer to Figure 1 steps.

[0041] The original training image information refers to the original image with controlled content generation and no watermark added. , original image ,This application randomly selects 10,000 images from the CoCo dataset as training samples, and all images are cropped to 128×128. The training watermark information refers to the watermark that needs to be embedded .

[0042] Step S101: analyzing and determining training conditional noise image information according to the training original image information.

[0043] The training conditional noise image information refers to the original image with Gaussian noise introduced. The image corresponding to the training conditional noise image information gradually transforms into a completely random noise image during the iteration process. The processing terminal introduces Gaussian noise into the training original image information. For specific methods, refer to Figure 2 steps.

[0044] Step S102: using the training conditional noise image information, the training original image information and the training watermark information as input to control a preset training encoder to generate encoded image information.

[0045] Among them, the training encoder refers to the operation of each time step in the forward process of the conditional diffusion model, using a multi-encoder structure, referring to Figure 8 Encoder in The encoded image information refers to the image containing the watermark obtained by using the image corresponding to the training conditional noise image information to guide the training encoder to embed the watermark corresponding to the training watermark information into the image corresponding to the training original image information. , the specific process is: , in, For the encoding image generation process, refer to Figure 3 Steps, is the training conditional noise image information, To train the original image information, To train the watermark information, is the current time step information.

[0046] Step S103: using the encoded image information as input to control the preset attack layer to generate attack image information.

[0047] The attack layer refers to the part used to train the robustness of the watermark. Figure 8 The attack layer A in the coded image is used to train the robustness of the watermark by adding different attack types to the encoded image. The attack image information refers to the image attacked by the attack layer image. Figure 8 Images in .

[0048] Step S104: using the attack image information as input to control a preset training decoder to generate training decoding watermark information.

[0049] Among them, training the decoder refers to the step of recovering the watermark from the encoded image, which is the inverse process of watermark embedding. There is only one decoder, refer to Figure 8 In order to improve the coupling between the encoder and the decoder, the decoder adopts a similar structure to the encoder, but adds a linear layer at the end to convert the output of the feature layer into a watermark ,Furthermore, in order to decode the watermark from the encoded image, the time factor t is removed unlike the encoder.

[0050] Decoded watermark information refers to the watermark recovered from the attacked coded image, which is recovered by the trained encoder from the image corresponding to the attacked image information. The specific process is: , in, Represents the process of watermark extraction, For the attacked image, are the parameters for training the decoder.

[0051] Step S105: Calculate loss function information according to the training original image information, the encoded image information, the training watermark information and the training decoded watermark information, and update the encoder parameter information of the training encoder and the decoder parameter information of the training decoder according to a preset gradient descent method.

[0052] Among them, the loss function information is calculated by training original image information, encoded image information, training watermark information and training decoded watermark information, so as to constrain the similarity between the training original image information and the encoded image information through the loss function, constrain the similarity between the training watermark information and the training decoded watermark information, and adjust the encoder parameter information of the training encoder and the decoder parameter information of the training decoder according to the gradient descent method, so as to complete the training of the training encoder and the training decoder.

[0053] The loss function information refers to the weighted sum of image loss and decoding loss. The specific calculation formula is: , in, is the loss function information, is the image loss, is the decoding loss, and The weights used to adjust the two losses are also used to adjust the weights of the encoder and decoder.

[0054] Image loss refers to the difference between the image corresponding to the encoded image information and the image corresponding to the original training image information. The specific calculation formula is: , in, To calculate the mean square error, To train the original image information, To encode image information.

[0055] The decoding loss refers to the difference between the watermark corresponding to the training watermark information and the watermark corresponding to the training decoded watermark information. The specific calculation formula is: , in, To train the watermark information, Decode watermark information for training.

[0056] The gradient descent method refers to an optimization algorithm used to find the minimum point of the loss function. The goal is to find a set of parameters for training the encoder and the decoder to minimize the loss function. The basic process is: select initial parameters, calculate gradients, and update parameters according to the gradients and the preset learning rate until the stopping condition is met.

[0057] Encoder parameter information refers to the weights and biases of the encoder neural network, which determine how to convert the input data into an intermediate representation, including the weights of the convolution kernel, the weight matrix of the fully connected layer, etc. Decoder parameter information refers to the various weight matrices and bias vectors in the decoder neural network, which is defined as how the decoder converts the intermediate representation back to data close to the original input.

[0058] Reference Figure 2 The step of analyzing and determining the training conditional noise image information according to the training original image information includes: Step S200: Obtain current time step information.

[0059] The current time step information refers to the time step number t of the current iteration process, which is obtained by counting once by a counter after each loop iteration, and ranges from 1 to T.

[0060] Step S201: Analyze and determine Gaussian noise intensity hyperparameter information based on current time step information and preset hyperparameter relationship information.

[0061] The hyperparameter relationship information refers to the change of the hyperparameter controlling the Gaussian noise intensity with the time step. In this embodiment, the two are in a linear proportional relationship. , , the processing terminal calculates the product between the current time step and the corresponding change value of the hyperparameter relationship information, and then adds the initial hyperparameter to obtain it.

[0062] Step S202: Acquire sampled Gaussian noise information.

[0063] The sampled Gaussian noise information refers to the Gaussian noise sampled from the standard normal distribution. , used to add randomness to the model.

[0064] Step S203: analyzing and determining the training conditional noise image information according to the sampled Gaussian noise information, the Gaussian noise intensity hyperparameter information and the training original image information.

[0065] The training conditional noise image information in this step is consistent with the training conditional noise image information in step S101, and is calculated by the processing terminal according to the sampled Gaussian noise information, the Gaussian noise intensity hyperparameter information and the training original image information. The specific calculation formula is: , in, is the training conditional noise image information, To train the original image information, is the sampling Gaussian noise information, , is the Gaussian noise intensity hyperparameter information.

[0066] Reference Figure 3 The steps of using the training conditional noise image information, the training original image information and the training watermark information as input to control the preset training encoder to generate the encoded image information include: Step S300: analyzing and determining original feature information based on the training conditional noise image information and the training original image information.

[0067] The original feature information refers to the feature map containing the information in the image corresponding to the training conditional noise image information and the information in the image corresponding to the training original image information, refer to Figure 8 ,First, the training conditional noise image information and the training original image information are ,concatenated channel by channel, and the features are obtained through a 3×3 ,convolutional layer.

[0068] Step S301: performing multi-scale feature extraction and downsampling on original feature information to generate feature map information.

[0069] Among them, the feature map information refers to the feature map after feature utilization enhancement and dimension reduction, refer to Fig.10 In this embodiment, the downsampling residual module is used to reduce the dimension of the feature and maintain the effective information of the feature. The specific process is as follows: first, the original feature information is processed by the GN-Swish-Conv block. Extract features , where the GN-Swish-Conv block includes a group normalization layer (GN), an activation layer Swish, and a 3×3 convolutional layer. In order to keep the encoder consistent across the entire temporal dimension, the time step t processed by the multilayer perceptron (MLP) is Fusion, and then through the 1 × 1 convolution layer to the original feature information Perform channel compression to transmit the original key information and reduce information loss. The features of the GN-Swish-Conv block, the current time step t, and the original feature information After multiple fusions, the features are obtained Finally, the downsampling layer is introduced to reduce the features The size of the feature map is obtained .

[0070] Step S302: pre-process the training watermark information to generate watermark feature information.

[0071] The watermark feature information refers to the preprocessed watermark feature map. Fig. 9 In this embodiment, the channel attention module uses channel correlation to mine channel features of the training watermark information. The specific process is as follows: the training watermark information is used as the input feature map , using the inter-channel relationship of the features, the channel attention is calculated twice to determine the weight of the watermark channel, and the size of the feature map is adjusted through the transposed convolution operation to match the scale of the corresponding image feature map. The specific formula is: , in, is the watermark feature information, To train the watermark information, is the upsampling operation of the transposed convolution, Calculate the channel attention: , in, is element-wise multiplication, To calculate the channel weights: , in, It is a Conv-BN-ReLU block operation, which includes a 3×3 convolution layer, a normalization layer (Batch Norm, BN) and an activation layer ReLU. is the Sigmoid activation function, To calculate average pooling, for: .

[0072] Step S303: Fusing the feature map information and the watermark feature information to generate encoded image information.

[0073] The coded image information in this step is consistent with the coded image information in step S102, and is generated by the trained encoder network after fusion analysis of the feature map information and the watermark feature information. For specific methods, refer to Figure 4 steps.

[0074] Reference Figure 4 , the steps of fusing the feature map information and the watermark feature information to generate the encoded image information include: Step S400: performing a fusion analysis based on the feature map information, the watermark feature information and the current time step information to determine the feature information of the watermarked image.

[0075] The watermarked image feature information refers to the feature obtained by fusing the watermark feature and the image feature. Fig.11In this embodiment, the residual self-attention module is used to fuse the image features and watermark features. The specific process is: and watermark feature information Input, passes through two GN-Swish-Conv blocks, and is combined with time step t and feature map information Fusion is performed to obtain the characteristic information of the watermarked image .

[0076] Step S401: normalize the feature information of the watermarked image and perform self-attention calculation to generate self-attention feature information.

[0077] Among them, the self-attention feature information refers to the watermarked image features after adjusting the watermark distribution, refer to Fig.11 In this embodiment, the residual self-attention module is used to enhance the context representation of features through self-attention calculation. The specific process is as follows: first, the feature information of the watermarked image is Normalize, generate Query (Q), Key (K) and Value (V) through a set of 1×1 convolutions, and calculate self-attention. Secondly, adjust the dimension through a 1×1 convolution layer and combine it with the feature information of the watermarked image Perform residual connection and output self-attention feature information , the specific calculation formula is: , in, is the dimension of K, To calculate the attention weight.

[0078] Step S402: Perform a fusion analysis based on the self-attention feature information and the current time step information to determine the enhanced feature information.

[0079] Among them, enhanced feature information refers to the features of self-attention feature information after dimension adjustment and fusion, refer to Fig.11 , the self-attention feature information After two GN-Swish-Conv blocks, and with time step t, feature Fusion to obtain enhanced feature information .

[0080] Step S403: performing fusion analysis and upsampling according to the enhanced feature information, the watermark feature information and the feature map information to generate encoded image information.

[0081] The coded image information in this step is consistent with the coded image in step S303. Fig.12 In this embodiment, the above sampling residual module is used to realize multi-layer watermark fusion and image feature dimension upgrading. The specific process is similar to Fig.10The inverse process of the downsampling residual module in , the structure is similar, the difference is that upsampling is used to increase the spatial dimension of the feature, the rest of the structure is the same, and the input is the enhanced feature information , watermark feature information and feature map information , through feature fusion and upsampling, the encoded image information is finally obtained .

[0082] Reference Figure 5 After the training step is completed according to the preset training step information, a testing step is also included. The specific steps include: Step S500: selecting a training encoder after updating encoder parameter information as a test encoder according to preset non-uniform sampling step number information, and selecting a training decoder after updating decoder parameter information as a test decoder.

[0083] The non-uniform sampling step number information refers to the number of iterations of the test process, and 3 is taken as an example in this embodiment. The test encoder refers to the encoder used to embed the watermark during the test process, and the corresponding number of encoders can be selected from the trained training encoders according to the number of steps corresponding to the non-uniform sampling step number information. The test encoder refers to the decoder used to extract the watermark during the test process, which is actually the same as the trained training decoder.

[0084] Step S501: Acquire test condition noise image information, test original image information and test watermark information.

[0085] The test condition noise image information refers to a completely random Gaussian noise image. The test original image information refers to the original image without watermarking during the test process. The test watermark information refers to the watermark that needs to be embedded during the test process.

[0086] Step S502: using the test condition noise image information, the test original image information and the test watermark information as input to control the test encoder to generate the final watermarked image information.

[0087] The final watermarked image information refers to the watermarked image generated by the test encoder, which is generated by the test encoder using the test condition noise image information, the test original image information and the test watermark information as input. The specific method is as follows Figure 6 steps.

[0088] Step S503: using the final watermarked image information as input to control the test decoder to generate test decoded watermark information.

[0089] The decoded watermark information refers to the watermark extracted by the test decoder from the image corresponding to the final watermarked image information. The extraction process is consistent with the decoding process of the training decoder in step S104, which will not be described in detail here.

[0090] Step S504: Analyze and generate model evaluation information based on the final watermarked image information, the test original image information, the test watermark information and the test decoded watermark information.

[0091] Among them, the model evaluation information refers to the evaluation of the ability of the model used to embed and recognize watermarks in this embodiment. The peak signal-to-noise ratio PSNR is used as an indicator for evaluating the invisibility of the watermark. The higher the PSNR value, the better the watermark invisibility. It is obtained by calculating the mean square error between the final watermarked image information and the test original image information, and then combining it with the calculation formula of PSNR; the robustness evaluation directly uses the watermark bit accuracy BA as an evaluation indicator, and analyzes the total number of bits of the test watermark information to identify the number of bits that are the same as the test watermark information in the test decoded watermark information, thereby calculating the quotient between the same number of bits and the total number of bits.

[0092] Reference Figure 6 The steps of using the test condition noise image information, the test original image information and the test watermark information as input to control the test encoder to generate the final watermarked image information include: Step S600: Obtain information on the number of iterations.

[0093] The number of iterations refers to the number of times the test encoder generates a watermarked image during the test. Each iteration inference is obtained by counting the counter, ranging from 0 to S-1, where S is the non-uniform sampling step number information.

[0094] Step S601: Analyze and determine the current iteration step information and the next iteration step information according to the information of the number of iterations.

[0095] The current iteration step information refers to the time step of the current iteration process, and the next iteration step information refers to the time step of the next iteration process. Both are calculated and analyzed by the processing terminal based on the number of iterations. For specific methods, refer to Figure 7 steps.

[0096] Step S602: Acquire corresponding test condition noise image information based on the current iteration step information.

[0097] Among them, the test conditional noise image information in this step refers to the conditional noise image after iteration, which is the noise image obtained when iterating from the safe and random test conditional noise image information in step S501 to the time step corresponding to the current iteration step information. The specific process is: based on the noise image obtained in the previous iteration, according to the conditional distribution relationship between the two noise images, the test conditional noise image information corresponding to the current iteration step information is inferred.

[0098] Step S603: Analyze and determine the denoised image information according to the next iteration step information, the current iteration step information, the test condition noise image information, the test original image information and the test watermark information.

[0099] The denoised image information refers to the image after removing the image noise in an iteration process, which is calculated by the model based on the next iteration step information, the current iteration step information, the test condition noise image information, the test original image information and the test watermark information. The specific formula is: , in, To denoise image information, is the next iteration step information, t is the current iteration step information, To test the conditional noise image information, To test the original image information, To test the watermark information, To encode the image generation process, , is a hyperparameter that controls the intensity of Gaussian noise during the iteration process, .

[0100] Step S604: determine whether the information on the number of iterations meets the requirements of the non-uniform sampling step number information.

[0101] The requirement of the non-uniform sampling step number information is that the number of steps corresponding to the non-uniform sampling step number information is consistent. The model determines whether the number of times corresponding to the iteration number information is consistent with the number of steps corresponding to the non-uniform sampling step number information, thereby determining whether the model has completed the iteration cycle of the test process.

[0102] Step S6041: If not, proceed to the next iteration and continue to obtain the information of the number of iterations to make a cyclic judgment.

[0103] Among them, if the model determines that the number of times corresponding to the number of iterations information is inconsistent with the number of steps corresponding to the non-uniform sampling step number information, it means that the model has not completed the iterative cycle of the test process, so it continues to the next iteration and continues to obtain the number of iterations information, thereby continuously monitoring the changes in the number of iterations.

[0104] Step S6042: If it is in compliance, the denoised image information is defined as the final watermarked image information.

[0105] Among them, if the model determines that the number of times corresponding to the number of iterations information is consistent with the number of steps corresponding to the non-uniform sampling step number information, it means that the model has completed the iterative cycle of the test process. During the test process, the test encoder will iteratively denoise the image, and after removing all noise, the final watermarked image information is obtained.

[0106] Reference Figure 7 The steps of analyzing and determining the current iteration step information and the next iteration step information according to the information of the number of iterations include: Step S700: Analyze and determine step number variation coefficient information based on training step information, non-uniform sampling step number information and preset variable information.

[0107] Among them, variable information refers to a time step that remains unchanged during the iteration process, and 1 is taken as an example in this embodiment. Step change coefficient information refers to the coefficient that controls the change of time steps during the iteration process, which is calculated by the model based on the training step information, non-uniform sampling step information and variable information analysis. The specific calculation formula is: , in, is the step number change coefficient information, is the training step information, is the non-uniform sampling step information, For variable information.

[0108] Step S701: Analyze and determine the current iteration step information based on the number of iterations, training step information, step change coefficient information and variable information.

[0109] The current iteration step information in this step is consistent with the current iteration step information in step S601, and is obtained by the model according to the information of the number of iterations, the training step information, the step number change coefficient information and the variable information. The specific calculation formula is: , in, is the current iteration information, It is the number of iterations.

[0110] Step S702: Analyze and determine the next iteration number information based on the information of the number of iterations completed.

[0111] The next iteration number information refers to the number of next iterations, which is the number corresponding to the number of iterations information plus one.

[0112] Step S703: Analyze and determine the next iteration step information based on the next iteration number information, training step information, step number change coefficient information and variable information.

[0113] The next iteration step information in this step is consistent with the next iteration step information in step S601, and is calculated by the model based on the next iteration number information, training step information, step number change coefficient information and variable information. The specific calculation formula is: , in, is the information of the next iteration step, It is the information of the next iteration number.

[0114] Based on the same inventive concept, the embodiment of the present application provides a robust image watermark system based on a conditional diffusion model, including: An acquisition module is used to acquire training original image information, training watermark information, current time step information, sampled Gaussian noise information, test condition noise image information, test original image information, test watermark information and number of iterations; A memory, used for storing a program of a robust image watermarking method based on a conditional diffusion model; The program in the processor memory can be loaded and executed by the processor to implement a robust image watermarking method based on a conditional diffusion model.

[0115] Those skilled in the art can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional modules is used as an example. In actual applications, the above-mentioned functions can be assigned to different functional modules as needed, that is, the internal structure of the device is divided into different functional modules to complete all or part of the functions described above. The specific working process of the system, device and unit described above can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here.

[0116] An embodiment of the present application provides a computer-readable storage medium storing a computer program that can be loaded by a processor and execute a robust image watermarking method based on a conditional diffusion model.

[0117] Computer storage media include, for example, various media that can store program codes, such as USB flash drives, mobile hard disks, read-only memories (ROM), random access memories (RAM), magnetic disks or optical disks.

[0118] Based on the same inventive concept, an embodiment of the present application provides an intelligent terminal, including a memory and a processor, wherein the memory stores a computer program that can be loaded by the processor and executes a robust image watermarking method based on a conditional diffusion model.

[0119] Those skilled in the art can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional modules is used as an example. In actual applications, the above-mentioned functions can be assigned to different functional modules as needed, that is, the internal structure of the device is divided into different functional modules to complete all or part of the functions described above. The specific working process of the system, device and unit described above can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here.

[0120] The above are all preferred embodiments of the present application, and are not intended to limit the protection scope of the present application. Any feature disclosed in this specification (including the abstract and drawings), unless otherwise stated, can be replaced by other equivalent or alternative features with similar purposes. That is, unless otherwise stated, each feature is only an example of a series of equivalent or similar features.

Claims

1. A robust image watermarking method based on a conditional diffusion model, characterized in that: include: Executing preset training steps cyclically according to preset training step information, the training steps comprising: acquiring training original image information and training watermark information; Analyze and determine the training conditional noise image information according to the training original image information; Using the training conditional noise image information, the training original image information and the training watermark information as input to control a preset training encoder to generate encoded image information; Using the encoded image information as input to control the preset attack layer to generate attack image information; Using the attack image information as input to control a preset training decoder to generate training decoding watermark information; The loss function information is calculated according to the training original image information, the encoded image information, the training watermark information and the training decoded watermark information, and the encoder parameter information of the training encoder and the decoder parameter information of the training decoder are updated according to a preset gradient descent method.

2. The robust image watermarking method based on conditional diffusion model according to claim 1 is characterized in that: The steps of analyzing and determining the training conditional noise image information according to the training original image information include: Get the current time step information; Analyze and determine Gaussian noise intensity hyperparameter information based on current time step information and preset hyperparameter relationship information; Get sampled Gaussian noise information; The training conditional noise image information is determined by analyzing the sampled Gaussian noise information, the Gaussian noise intensity hyperparameter information and the training original image information.

3. The robust image watermarking method based on conditional diffusion model according to claim 2 is characterized in that: The steps of using the training conditional noise image information, the training original image information and the training watermark information as input to control the preset training encoder to generate the encoded image information include: Analyze and determine original feature information based on training condition noise image information and training original image information; Perform multi-scale feature extraction and down-sampling on the original feature information to generate feature map information; Preprocessing the training watermark information to generate watermark feature information; The feature map information and watermark feature information are fused to generate encoded image information.

4. The robust image watermarking method based on conditional diffusion model according to claim 3 is characterized in that: The steps of fusing the feature map information and the watermark feature information to generate the encoded image information include: Determine the feature information of the watermarked image by performing fusion analysis based on the feature map information, watermark feature information and current time step information; Normalize the feature information of the watermarked image and perform self-attention calculation to generate self-attention feature information; Perform fusion analysis based on self-attention feature information and current time step information to determine enhanced feature information; The coded image information is generated by performing fusion analysis and upsampling based on the enhanced feature information, watermark feature information and feature map information.

5. The robust image watermarking method based on conditional diffusion model according to claim 1, characterized in that: After the training steps are executed cyclically according to the preset training step information, a testing step is also included, and the specific steps include: According to the preset non-uniform sampling step number information, the training encoder after the encoder parameter information is updated is selected as the test encoder, and the training decoder after the decoder parameter information is updated is selected as the test decoder; Acquire test condition noise image information, test original image information and test watermark information; Using the test condition noise image information, the test original image information and the test watermark information as input to control the test encoder to generate the final watermarked image information; The final watermarked image information is used as input to control the test decoder to generate test decoded watermark information; Model evaluation information is generated by analyzing the final watermarked image information, the test original image information, the test watermark information and the test decoded watermark information.

6. The robust image watermarking method based on conditional diffusion model according to claim 5, characterized in that: The steps of using the test condition noise image information, the test original image information and the test watermark information as input to control the test encoder to generate the final watermarked image information include: Get the number of iterations; Analyze and determine the current iteration step information and the next iteration step information according to the information of the number of iterations; Based on the current iteration step information, the corresponding test condition noise image information is obtained; Determine denoised image information by analyzing according to next iteration step information, current iteration step information, test condition noise image information, test original image information and test watermark information; Determine whether the information of the number of iterations meets the requirements of the non-uniform sampling step number information; If not, proceed to the next iteration and continue to obtain the information of the number of iterations to make a cyclic judgment; If it meets the requirements, the denoised image information is defined as the final watermarked image information.

7. The robust image watermarking method based on conditional diffusion model according to claim 6 is characterized in that: The steps of analyzing and determining the current iteration step information and the next iteration step information according to the information of the number of iterations include: Analyze and determine step number variation coefficient information based on training step information, non-uniform sampling step number information and preset variable information; Analyze and determine the current iteration step information based on the number of iterations, training step information, step change coefficient information and variable information; Analyze and determine the next iteration number information based on the information of the number of iterations completed; The next iteration step information is determined by analyzing the next iteration number information, training step information, step number change coefficient information and variable information.

8. A robust image watermarking system based on a conditional diffusion model, characterized in that: include: An acquisition module, used to acquire training original image information and training watermark information; A memory, used to store a program of the robust image watermarking method based on a conditional diffusion model as claimed in any one of claims 1 to 7; The program in the memory can be loaded and executed by the processor to implement the robust image watermarking method based on the conditional diffusion model as described in any one of claims 1 to 7.

9. An intelligent terminal, characterized in that: The method comprises a memory and a processor, wherein the memory stores a computer program which can be loaded by the processor and executes the robust image watermarking method based on the conditional diffusion model as claimed in any one of claims 1 to 7.

Images