Robust image watermarking method, system and terminal based on conditional diffusion model

Abstract

The application relates to a robust image watermarking method, system and terminal based on a conditional diffusion model, relates to the field of robust image watermarking technology, and comprises the following steps: cyclically performing a training step according to training step information, the training step information comprising the following steps: obtaining training original image information and training watermark information; determining training conditional noise image information according to the training original image information; taking the training conditional noise image information, the training original image information and the training watermark information as inputs to control a training encoder to generate encoded image information; taking the encoded image information as an input to control an attack layer to generate attack image information; taking the attack image information as an input to control a training decoder to generate training decoded watermark information; calculating loss function information according to the training original image information, the encoded image information, the training watermark information and the training decoded watermark information, and updating encoder parameter information and decoder parameter information according to a gradient descent method. The application has the effect of improving watermark performance.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of robust image watermarking technology, and in particular to a robust image watermarking method, system and terminal based on a conditional diffusion model. BACKGROUND

[0002] Robust image watermarking technology is a technique for embedding information in digital images that can still be detected or extracted even after the image has undergone various processing or attacks. This technology is widely used in the fields of copyright protection, integrity verification, tracking and authentication, and robustness refers to the ability of the watermark to be identified after certain image processing operations such as compression, filtering, cropping, etc.

[0003] In related technologies, image watermarking methods are usually based on an end-to-end watermarking framework of a deep neural network, which mainly adopts an encoder-attack layer-decoder structure. The encoder extracts the features of the image and fuses them with the watermark to obtain an encoded image (containing a watermarked image). The attack layer simulates the attacks that the encoded image may encounter during transmission, and the decoder extracts the watermark from the attacked image. In addition, a discriminator is introduced into the framework, which forms a generative adversarial network with the encoder for training, thereby improving the image quality.

[0004] In view of the related technologies in the above, the training process of the generative adversarial network composed of the encoder and the discriminator is a dynamic, non-cooperative game process, in which the encoder and the discriminator respectively try to maximize their respective objective functions. This adversarial relationship can lead to unstable training and pattern collapse phenomena, thereby affecting the watermarking performance, and there is still room for improvement. SUMMARY

[0005] In order to improve the watermarking performance, the present application provides a robust image watermarking method, system and terminal based on a conditional diffusion model.

[0006] In a first aspect, the present application provides a robust image watermarking method based on a conditional diffusion model, which adopts the following technical solution:

[0007] The robust image watermarking method based on the conditional diffusion model comprises:

[0008] According to the preset training step information, the preset training steps are executed cyclically, and the training steps comprise: obtaining training original image information and training watermark information;

[0009] The training condition noise image information is determined by analyzing the training original image information;

[0010] The training condition noise image information, the training original image information and the training watermark information are input to control the preset training encoder to generate encoded image information;

[0011] generate attack image information as input control preset attack layer;

[0012] generate training decoding watermark information as input control preset training decoder;

[0013] According to the training original image information, the encoding image information, the training watermark information and the training decoding watermark information, the loss function information is calculated, and the encoder parameter information of the training encoder and the decoder parameter information of the training decoder are updated according to the preset gradient descent method.

[0014] By adopting the above technical scheme, the multi-encoder structure guided by the conditional noise image can effectively adjust the watermark, embed the watermark in the area that is not easy to be perceived and robust, and compared with the existing watermark framework, the discriminator is removed, so that it is not dependent on the adversarial training, gradually reconstructs the data from the noise in a series of predictable steps, more smoothly approaches the real distribution of the data, each step follows the conditional probability distribution, ensures the stability and controllability of the whole process, and improves the image quality and the watermark robustness.

[0015] Optionally, the step of analyzing and determining the training conditional noise image information according to the training original image information comprises:

[0016] obtaining current time step information;

[0017] According to the current time step information and the preset hyperparameter relationship information, the Gaussian noise intensity hyperparameter information is analyzed and determined;

[0018] obtaining sampling Gaussian noise information;

[0019] According to the sampling Gaussian noise information, the Gaussian noise intensity hyperparameter information and the training original image information, the training conditional noise image information is analyzed and determined.

[0020] By adopting the above technical scheme, the Gaussian noise is introduced into the original image, so as to generate the training conditional noise image, and the watermark is embedded according to the conditional noise image, which facilitates the encoder to better understand which part of the image is suitable for embedding the watermark without affecting the quality of the image, and makes the model learn how to maintain the recoverability of the watermark under different noise conditions.

[0021] Optionally, the step of generating the encoding image information as input control preset training encoder by using the training conditional noise image information, the training original image information and the training watermark information comprises:

[0022] According to the training conditional noise image information and the training original image information, the original feature information is analyzed and determined;

[0023] Multi-scale feature extraction and down-sampling are performed on the original feature information to generate feature map information;

[0024] The training watermark information is preprocessed to generate watermark feature information;

[0025] The feature map information and the watermark feature information are fused to generate encoded image information.

[0026] By adopting the above technical solution, multi-scale feature extraction and dimension reduction are performed on the training condition noise image information and the training original image information to obtain feature map information, thereby improving the feature utilization rate of the two and reducing the dimension of the feature, thereby improving the representation ability of the network. The watermark is preprocessed to adjust the size of the watermark feature, so that the watermark feature is matched with the corresponding image feature scale.

[0027] Optionally, the step of fusing the feature map information and the watermark feature information to generate the encoded image information includes:

[0028] According to the feature map information, the watermark feature information and the current time step information, the watermark-containing image feature information is determined through fusion analysis;

[0029] The watermark-containing image feature information is standardized and self-attention calculated to generate self-attention feature information;

[0030] According to the self-attention feature information and the current time step information, the enhanced feature information is determined through fusion analysis;

[0031] According to the enhanced feature information, the watermark feature information and the feature map information, the encoded image information is generated through fusion analysis and up-sampling.

[0032] By adopting the above technical solution, the watermark-containing image feature information is standardized and self-attention calculated, thereby ensuring that the data will not be unstable due to too large scale difference in the subsequent processing process, and the self-attention mechanism makes the model pay attention to the relationship between different parts of the input, better capturing global dependencies.

[0033] Optionally, after the training step is completed according to the preset training step information, a test step is further included, and the specific steps include:

[0034] According to the preset non-uniform sampling step number information, the training encoder after updating the encoder parameter information is selected as the test encoder, and the training decoder after updating the decoder parameter information is selected as the test decoder;

[0035] The test condition noise image information, the test original image information and the test watermark information are obtained;

[0036] The test condition noise image information, the test original image information and the test watermark information are taken as inputs to control the test encoder to generate final watermark-containing image information;

[0037] The final watermark-containing image information is taken as an input to control the test decoder to generate test-decoded watermark information;

[0038] Model evaluation information is generated by analyzing the final watermark-containing image information, the test original image information, the test watermark information and the test-decoded watermark information.

[0039] By using the above technical solution, after the training of the multiple encoders is completed, a trained encoder is selected as a test encoder, so that the test encoder is controlled to encode to obtain final watermark-containing image information, thereby speeding up the test process, and then the test decoder is controlled to decode to generate test-decoded watermark information, and the model is evaluated according to the final watermark-containing image information, the test original image information, the test watermark information and the test-decoded watermark information, thereby improving the accuracy of the model.

[0040] Optionally, the step of taking the test condition noise image information, the test original image information and the test watermark information as inputs to control the test encoder to generate final watermark-containing image information includes:

[0041] Obtaining iteration number information;

[0042] Analyzing the iteration number information to determine current iteration step information and next iteration step information;

[0043] Based on the current iteration step information, corresponding test condition noise image information is obtained;

[0044] Based on the next iteration step information, the current iteration step information, the test condition noise image information, the test original image information and the test watermark information, denoised image information is determined;

[0045] Judging whether the iteration number information meets the requirements of non-uniform sampling step number information;

[0046] If not, the next iteration is continued, and the iteration number information is continuously obtained for cyclic judgment;

[0047] If yes, the denoised image information is defined as the final watermark-containing image information.

[0048] By using the above technical solution, a series of iteration refinements are performed from completely random test condition noise image information, so that noise components in the image are removed in the denoising process, important structural information is preserved, the clarity of the image is improved, and the robustness of the watermark is improved.

[0049] Optionally, the step of determining the current iteration step information and the next iteration step information according to the iteration number information comprises:

[0050] determining the step number change coefficient information according to the training step information, the non-uniform sampling step number information and the preset variable information;

[0051] determining the current iteration step information according to the iteration number information, the training step information, the step number change coefficient information and the variable information;

[0052] determining the next iteration number information according to the iteration number information;

[0053] determining the next iteration step information according to the next iteration number information, the training step information, the step number change coefficient information and the variable information.

[0054] By using the above technical solution, the change of the step number at each iteration is determined according to the relationship between the non-uniform sampling step number information and the variable information, the step number change coefficient information is obtained, the current iteration step number information and the next iteration step number information are calculated according to the iteration number information and the next iteration number information, and the convenience of determining the current iteration step number information and the next iteration step number information is improved.

[0055] In a second aspect, the present application provides a robust image watermarking system based on a conditional diffusion model, which adopts the following technical solution:

[0056] A robust image watermarking system based on a conditional diffusion model comprises:

[0057] An acquisition module is configured to acquire training original image information and training watermark information;

[0058] A memory is configured to store a program of the robust image watermarking method based on a conditional diffusion model according to any one of the above aspects;

[0059] A processor, the program in the memory can be loaded and executed by the processor and implement the robust image watermarking method based on a conditional diffusion model according to any one of the above aspects.

[0060] By adopting the technical solution, the processor loads and executes the robust image watermarking method based on the conditional diffusion model stored in the memory, the control acquisition module acquires a series of data related to the robust image watermarking based on the conditional diffusion model, so that the multi-encoder embeds the watermark in the area that is not easy to be perceived and robust under the guidance of the conditional noise image, and compared with the existing watermark framework, the discriminator is removed, so that the process is not dependent on the adversarial training, the data is gradually reconstructed from the noise in a series of predictable steps, the real distribution of the data is more smoothly approximated, each step follows a conditional probability distribution, the stability and controllability of the whole process are ensured, and the image quality and the watermark robustness are improved.

[0061] In a third aspect, the present application provides an intelligent terminal, which adopts the technical solution as follows:

[0062] An intelligent terminal, comprising a memory and a processor, the memory storing a computer program capable of being loaded and executed by the processor to implement the robust image watermarking method based on the conditional diffusion model according to any one of the above.

[0063] By adopting the technical solution, the processor loads and executes the robust image watermarking method based on the conditional diffusion model stored in the memory, the control acquisition module acquires a series of data related to the robust image watermarking based on the conditional diffusion model, so that the multi-encoder embeds the watermark in the area that is not easy to be perceived and robust under the guidance of the conditional noise image, and compared with the existing watermark framework, the discriminator is removed, so that the process is not dependent on the adversarial training, the data is gradually reconstructed from the noise in a series of predictable steps, the real distribution of the data is more smoothly approximated, each step follows a conditional probability distribution, the stability and controllability of the whole process are ensured, and the image quality and the watermark robustness are improved.

[0064] In summary, the present application includes at least one of the following beneficial technical effects:

[0065] By adopting the multi-encoder structure guided by the conditional noise image, the watermark can be effectively adjusted and embedded in the area that is not easy to be perceived and robust, compared with the existing watermark framework, the discriminator is removed, so that the process is not dependent on the adversarial training, the data is gradually reconstructed from the noise in a series of predictable steps, the real distribution of the data is more smoothly approximated, each step follows a conditional probability distribution, the stability and controllability of the whole process are ensured, and the image quality and the watermark robustness are improved;

[0066] By introducing the Gaussian noise into the original image to generate the training conditional noise image, and guiding the watermark embedding according to the conditional noise image, the encoder can better understand which part of the image is suitable for embedding the watermark without affecting the quality of the image, and the model learns how to maintain the recoverability of the watermark under different noise conditions;

[0067] By selecting the trained encoder as the test encoder after the training of the multi-encoder is completed, the test encoder is controlled to encode to obtain the final watermark-containing image information, thereby speeding up the test process, and then the test decoder is controlled to decode to generate test decoded watermark information, and the model is evaluated according to the final watermark-containing image information, the test original image information, the test watermark information and the test decoded watermark information, thereby improving the accuracy of the model. BRIEF DESCRIPTION OF DRAWINGS

[0068] Figure 1 is a flowchart of the robust image watermarking method based on the conditional diffusion model in the embodiment of the present application.

[0069] Figure 2 is a flowchart of the step of determining the training conditional noise image information according to the analysis of the training original image information in the embodiment of the present application.

[0070] Figure 3 is a flowchart of the step of controlling the preset training encoder to generate the encoded image information by taking the training conditional noise image information, the training original image information and the training watermark information as inputs in the embodiment of the present application.

[0071] Figure 4 is a flowchart of the step of fusing the feature map information and the watermark feature information to generate the encoded image information in the embodiment of the present application.

[0072] Figure 5 is a flowchart of the test step in the embodiment of the present application.

[0073] Figure 6 is a flowchart of the step of controlling the test encoder to generate the final watermark-containing image information by taking the test conditional noise image information, the test original image information and the test watermark information as inputs in the embodiment of the present application.

[0074] Figure 7 is a flowchart of the step of determining the current iteration step information and the next iteration step information according to the analysis of the number of iterations in the embodiment of the present application.

[0075] Figure 8 is a framework diagram of the robust image watermarking method based on the conditional diffusion model in the embodiment of the present application.

[0076] Figure 9 is a structure diagram of the preprocessing of the training watermark information in the embodiment of the present application.

[0077] Figure 10 is a structure diagram of the determination of the watermark-containing image feature information in the embodiment of the present application.

[0078] Figure 11 is a structure diagram of the generation of the self-attention feature information in the embodiment of the present application.

[0079] Figure 12 is a structure diagram for generating encoded image information in the embodiment of the present application. DETAILED DESCRIPTION

[0080] In order to make the purpose, technical solutions and advantages of the present application clearer, the present application is further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and not to limit the present application. Figures 1-12 The present application is further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and not to limit the present application.

[0081] The embodiment of the present application discloses a robust image watermarking method based on a conditional diffusion model, specifically discloses an encoding part, an attack layer and a decoding part. The encoding part adopts a multi-encoder structure guided by a conditional noise image. Training conditional noise image information is determined according to training original image information. The training conditional noise image information, the training original image information and training watermark information are input into an encoder to generate encoded image information. The attack layer is controlled to attack the encoded image information to generate attack image information. Thus, the decoder decodes the attack image information to obtain training decoded watermark information. Finally, loss function information is calculated according to the training original image information, the encoded image information, the training watermark information and the training decoded watermark information. The parameters of the encoder and the decoder are updated in a loop using a gradient descent method, thereby completing the training of the encoder and the decoder. The method does not require a discriminator and is not dependent on adversarial training. Data is gradually reconstructed from noise in a series of predictable steps, more smoothly approaching the true distribution of data. Each step follows a conditional probability distribution, ensuring the stability and controllability of the entire process, while improving image quality and watermark robustness.

[0082] With reference to Figure 1 , the embodiment of the present application discloses a robust image watermarking method based on a conditional diffusion model, comprising the following steps:

[0083] Step S100: According to the preset training step information, the preset training step is executed in a loop. The training step includes obtaining training original image information and training watermark information.

[0084] The training step information refers to the number of iterations T of the training of the encoder and the decoder. In the embodiment, the number of iterations T is set to 1000. The training step refers to a method for training the encoder and the decoder. For details, refer to the steps of Figure 1 .

[0085] The training original image information refers to an original image without watermarking generated by controlling content. This application randomly selected 10,000 images from the CoCo dataset as training samples, and all images were cropped to 128×128 pixels. The training watermark information refers to the watermark to be embedded. .

[0086] Step S101: Analyze and determine the training condition noise image information based on the original training image information.

[0087] Here, the training conditional noise image information refers to the original image with Gaussian noise introduced. The image corresponding to the training conditional noise image information gradually transforms into a completely random noise image during the iteration process. This is obtained by the processing terminal introducing Gaussian noise into the original training image information. For specific methods, refer to [link to relevant documentation]. Figure 2 The steps.

[0088] Step S102: Use the training conditional noise image information, the original training image information, and the training watermark information as input to control the preset training encoder to generate coded image information.

[0089] Here, the training encoder refers to the operation at each time step in the forward process of the conditional diffusion model, employing a multi-encoder structure, as referenced. Figure 8 encoder in Encoding image information refers to using the image corresponding to the training conditional noisy image information to guide the training encoder, and then embedding the watermark corresponding to the training watermark information into the image corresponding to the training original image information to obtain the watermarked image. The specific process is as follows:

[0090] ,

[0091] in, For the image generation process, refer to Figure 3 Steps, To train conditional noisy image information, To train the original image information, To train watermark information, This is the information for the current time step.

[0092] Step S103: Use the encoded image information as input to control the preset attack layer to generate attack image information.

[0093] The attack layer refers to the part used to train the robustness of the watermark, as shown in the reference. Figure 8 Attack layer A in the model trains the robustness of the watermark by adding different attack types to the encoded image. Attack image information refers to the image attacked by the attack layer image, as shown in the reference... Figure 8 Images in .

[0094] Step S104: generating training decoding watermark information by controlling the preset training decoder as input attack image information.

[0095] Wherein, the training decoder refers to the step of recovering the watermark from the encoded image, which is the inverse process of watermark embedding, and there is only one decoder, refer to Figure 8 In order to improve the coupling between the encoder and the decoder, the decoder adopts a similar structure as the encoder, but adds a linear layer at the end to convert the output of the feature layer into watermark In addition, in order to decode the watermark from the encoded image, unlike the encoder, the time factor t is removed.

[0096] The decoding watermark information refers to the watermark recovered from the attacked encoded image, which is recovered by the training encoder from the image corresponding to the attack image information, and the specific process is:

[0097] ,

[0098] Wherein, indicates the process of watermark extraction, is the attacked image, is the parameter of the training decoder.

[0099] Step S105: calculating loss function information according to training original image information, encoded image information, training watermark information and training decoding watermark information, and updating encoder parameter information of training encoder and decoder parameter information of training decoder according to the preset gradient descent method.

[0100] Wherein, the loss function information is calculated by training original image information, encoded image information, training watermark information and training decoding watermark information, so as to constrain the similarity between training original image information and encoded image information by loss function, constrain the similarity between training watermark information and training decoding watermark information, and adjust the encoder parameter information of training encoder and the decoder parameter information of training decoder according to the gradient descent method, so as to complete the training of training encoder and training decoder.

[0101] The loss function information refers to the weighted sum of image loss and decoding loss, and the specific calculation formula is:

[0102] ,

[0103] Wherein, is the loss function information, is the image loss, is the decoding loss, and is used to adjust the weight of the two losses, and is also used to adjust the weight of the encoder and the decoder.

[0104] The image loss refers to the difference between the corresponding image of the encoded image information and the corresponding image of the training original image information, and the specific calculation formula is:

[0105] ,

[0106] Wherein, is the calculation of the mean square error, is the training original image information, is the encoded image information.

[0107] The decoding loss refers to the difference between the corresponding watermark of the training watermark information and the corresponding watermark of the training decoded watermark information, and the specific calculation formula is:

[0108] ,

[0109] Wherein, is the training watermark information, is the training decoded watermark information.

[0110] The gradient descent method refers to an optimization algorithm used to find the minimum point of the loss function, and the goal is to find a set of parameters of the training encoder and the training decoder to make the loss function reach the minimum, and the basic process is: selecting the initial parameters, calculating the gradient, updating the parameters according to the gradient and the preset learning rate until the stop condition is met.

[0111] The encoder parameter information refers to the weights and biases of the encoder neural network, which determines how to convert the input data into an intermediate representation, including the weights of the convolution kernel, the weight matrix of the fully connected layer, etc. The decoder parameter information refers to various weight matrices and bias vectors in the decoder neural network, which defines how the decoder converts the intermediate representation back to data close to the original input.

[0112] Referring to Figure 2 , the step of determining the training condition noise image information according to the training original image information includes:

[0113] Step S200: Obtain the current time step information.

[0114] Wherein, the current time step information refers to the time step t of the current iteration process, which is counted by a counter every time a loop iteration is performed, and the range is from 1 to T.

[0115] Step S201: According to the current time step information and the preset hyperparameter relationship information, the Gaussian noise intensity hyperparameter information is determined by analysis.

[0116] The super parameter relationship information refers to the change of the super parameter for controlling the Gaussian noise intensity with time steps, and in this embodiment, the two are in a linear proportional relationship. The Gaussian noise intensity super parameter information refers to the super parameter for controlling the Gaussian noise intensity , The product of the current time step and the corresponding change value of the super parameter relationship information is calculated by the processing terminal, and the initial super parameter is added to obtain the Gaussian noise intensity super parameter information.

[0117] Step S202: Obtain the sampled Gaussian noise information.

[0118] The sampled Gaussian noise information refers to the Gaussian noise sampled from the standard normal distribution , which is used to add randomness to the model.

[0119] Step S203: Determine the training conditional noise image information according to the sampled Gaussian noise information, the Gaussian noise intensity super parameter information, and the training original image information.

[0120] The training conditional noise image information in this step is consistent with the training conditional noise image information in step S101, which is calculated by the processing terminal according to the sampled Gaussian noise information, the Gaussian noise intensity super parameter information, and the training original image information. The specific calculation formula is:

[0121] ,

[0122] Wherein, is the training conditional noise image information, is the training original image information, is the sampled Gaussian noise information, , is the Gaussian noise intensity super parameter information.

[0123] Referring to Figure 3 , the step of using the training conditional noise image information, the training original image information, and the training watermark information as input to control the preset training encoder to generate the encoded image information includes:

[0124] Step S300: Determine the original feature information according to the training conditional noise image information and the training original image information.

[0125] The original feature information refers to a feature map containing information in the image corresponding to the training conditional noise image information and the image corresponding to the training original image information, referring to Figure 8 , the training conditional noise image information and the training original image information are first spliced channel by channel, and the features are obtained through a 3x3 convolution layer.

[0126] Step S301: Perform multi-scale feature extraction and down-sampling on the original feature information to generate feature map information.

[0127] wherein the feature map information refers to a feature map after feature utilization rate enhancement and dimension reduction, referring to Figure 10 In the embodiment, the down-sampling residual module is used to reduce the dimension of the feature, and the effective information of the feature is also maintained. The specific process is as follows: first, the original feature information is processed through the GN-Swish-Conv block extracting features wherein the GN-Swish-Conv block includes a group normalization layer (GN), an activation layer Swish, and a 3x3 convolution layer. In order to maintain the consistency of the encoder in the entire time dimension, the time step t of the multi-layer perceptron (MLP) processing is fused with the original feature information , and then the original feature information is compressed in the channel through a 1x1 convolution layer to pass the original key information, reduce information loss, and fuse the features of the GN-Swish-Conv block, the current time step t, and the original feature information to obtain the feature , and finally a down-sampling layer is introduced to reduce the size of the feature to obtain the feature map information .

[0128] Step S302: preprocessing the training watermark information to generate watermark feature information.

[0129] wherein the watermark feature information refers to a watermark feature map after preprocessing, referring to Figure 9 In the embodiment, the channel attention module is used to utilize the channel correlation to mine the channel features of the training watermark information. The specific process is as follows: taking the training watermark information as the input feature map , the channel attention is calculated twice to determine the weight of the watermark channel by utilizing the inter-channel relationship of the features, and the size of the feature map is adjusted through the transpose convolution operation to match the size of the corresponding image feature map. The specific formula is as follows:

[0130] ,

[0131] wherein is the watermark feature information, is the training watermark information, is the up-sampling operation of the transpose convolution, is the channel attention calculation:

[0132] ,

[0133] wherein is the element-wise multiplication, is the calculation of the channel weight:

[0134] ,

[0135] wherein, is a Conv-BN-ReLU block operation, including a 3x3 convolution layer, a normalization layer (Batch Norm, BN) and an activation layer ReLU, is a Sigmoid activation function, is a calculation of average pooling, is:

[0136] .

[0137] Step S303: The feature map information and the watermark feature information are fused to generate encoded image information.

[0138] Wherein, the encoded image information in this step is consistent with the encoded image information in step S102, which is generated by fusing and analyzing the feature map information and the watermark feature information by the training encoder network, and the specific method refers to the steps of Figure 4 .

[0139] Referring to Figure 4 , the step of fusing the feature map information and the watermark feature information to generate the encoded image information includes:

[0140] Step S400: Fuse and analyze the feature map information, the watermark feature information and the current time step information to determine the watermark-containing image feature information.

[0141] Wherein, the watermark-containing image feature information refers to the feature obtained by fusing the watermark feature and the image feature, and referring to Figure 11 , in this embodiment, the residual self-attention module is used to fuse the image feature and the watermark feature, and the specific process is: the feature map information and the watermark feature information are input, and after passing through twice GN-Swish-Conv blocks, they are fused with the time step t and the feature map information to obtain the watermark-containing image feature information .

[0142] Step S401: Standardize and calculate the self-attention of the watermark-containing image feature information to generate the self-attention feature information.

[0143] Wherein, the self-attention feature information refers to the watermark-containing image feature after adjusting the watermark distribution, and referring to Figure 11 , in this embodiment, the residual self-attention module is used to calculate the self-attention to enhance the context representation of the feature, and the specific process is: first, the watermark-containing image feature information Standardization is performed, Query (Q), Key (K) and Value (V) are generated respectively by a set of 1x1 convolution, and self-attention is calculated, secondly, dimension adjustment is performed by a 1x1 convolution layer, and the feature information of the image containing the watermark is calculated Residual connection is performed, and the self-attention feature information is output The specific calculation formula is:

[0144] ,

[0145] Among them, K is the dimension of, The attention weight is calculated.

[0146] Step S402: Fusion analysis is performed according to the self-attention feature information and the current time step information to determine the enhanced feature information.

[0147] Among them, the enhanced feature information refers to the feature after dimension adjustment and fusion of the self-attention feature information, refer to Figure 11 The self-attention feature information is passed through two GN-Swish-Conv blocks, and is fused with the time step t, the feature , and the enhanced feature information is obtained.

[0148] Step S403: Fusion analysis and up-sampling are performed according to the enhanced feature information, the watermark feature information and the feature map information to generate the encoded image information.

[0149] Among them, the encoded image information in this step is consistent with the encoded image in step S303, refer to Figure 12 In this embodiment, the above up-sampling residual module realizes multi-layer watermark fusion and dimension increase of image features, and the specific process is similar to the inverse process of the down-sampling residual module in Figure 10 , the structure is similar, the difference lies in that the spatial dimension of the feature is increased by up-sampling, and the rest of the structure is consistent, the input is the enhanced feature information , the watermark feature information and the feature map information , through feature fusion and up-sampling, the encoded image information is finally obtained.

[0150] Refer to Figure 5 , after the training step is completed according to the preset training step information, a testing step is further included, and the specific steps include:

[0151] Step S500: According to the preset non-uniform sampling step number information, the training encoder after updating the encoder parameter information is selected as the test encoder, and the training decoder after updating the decoder parameter information is selected as the test decoder.

[0152] Wherein, the non-uniform sampling step number information refers to the iteration number of the test process, and 3 is taken as an example in the embodiment. The test encoder refers to the encoder used for embedding the watermark in the test process, and the corresponding number of encoders can be selected in the trained training encoder according to the step number corresponding to the non-uniform sampling step number information. The test encoder refers to the decoder used for extracting the watermark in the test process, which is actually the same as the trained training decoder.

[0153] Step S501: Obtain test condition noise image information, test original image information and test watermark information.

[0154] Wherein, the test condition noise image information refers to a completely random Gaussian noise image. The test original image information refers to the original image without watermark in the test process. The test watermark information refers to the watermark that needs to be embedded in the test process.

[0155] Step S502: Control the test encoder to generate final watermark-containing image information by taking the test condition noise image information, the test original image information and the test watermark information as input.

[0156] Wherein, the final watermark-containing image information refers to the watermark-containing image generated by the test encoder, which is generated by taking the test condition noise image information, the test original image information and the test watermark information as input by the test encoder. For specific methods, refer to the steps of Figure 6 .

[0157] Step S503: Control the test decoder to generate test decoded watermark information by taking the final watermark-containing image information as input.

[0158] Wherein, the decoded watermark information refers to the watermark extracted from the image corresponding to the final watermark-containing image information by the test decoder, and the extraction process is the same as the decoding process of the training decoder in step S104, which will not be repeated here.

[0159] Step S504: Generate model evaluation information by analyzing the final watermark-containing image information, the test original image information, the test watermark information and the test decoded watermark information.

[0160] The model evaluation information refers to the evaluation of the model capability for embedding watermarks and identifying watermarks in the embodiment. The peak signal-to-noise ratio (PSNR) is used as an index for evaluating the invisibility of the watermark. The higher the PSNR value, the better the invisibility of the watermark. The mean square error of the final watermark-containing image information and the test original image information is calculated, and then the PSNR is calculated according to the calculation formula to obtain the evaluation index. The robustness evaluation directly uses the watermark bit accuracy (BA) as an evaluation index. The total number of bits of the test watermark information is analyzed, and the number of bits identical to the test watermark information is identified to calculate the quotient of the identical bit number and the total bit number.

[0161] With reference to Figure 6 The step of generating the final watermark-containing image information by taking the test condition noise image information, the test original image information, and the test watermark information as input control of the test encoder includes:

[0162] Step S600: Obtain the number of iterations information.

[0163] The number of iterations information refers to the number of times the test encoder generates a watermark-containing image in the test process. Each iteration inference is counted by a counter, and the range is from 0 to S-1, where S is the non-uniform sampling step number information.

[0164] Step S601: Analyze and determine the current iteration step information and the next iteration step information according to the number of iterations information.

[0165] The current iteration step information refers to the time step of the current iteration process, and the next iteration step information refers to the time step of the next iteration process. Both are obtained by analyzing and calculating the number of iterations information by the processing terminal. For specific methods, refer to the steps of Figure 7

[0166] Step S602: Obtain the corresponding test condition noise image information based on the current iteration step information.

[0167] The test condition noise image information in this step refers to the noise image obtained after iteration from the secure random test condition noise image information in step S501 to the time step corresponding to the current iteration step information. The specific process is as follows: based on the noise image obtained by the last iteration, the test condition noise image information corresponding to the current iteration step information is inferred according to the conditional distribution relationship between the two noise images.

[0168] Step S603: Analyze and determine the de-noising image information according to the next iteration step information, the current iteration step information, the test condition noise image information, the test original image information, and the test watermark information.

[0169] ​The denoised image information refers to an image after image noise is removed in one iteration process, and is obtained by the model according to the next iteration step information, the current iteration step information, the test condition noise image information, the test original image information and the test watermark information, and the specific formula is:

[0170] ,

[0171] Wherein, is the denoised image information, is the next iteration step information, t is the current iteration step information, is the test condition noise image information, is the test original image information, is the test watermark information, is the encoding image generation process, , is the iteration process control Gaussian noise intensity hyperparameter, .

[0172] Step S604: Determine whether the iteration number information meets the requirement of the non-uniform sampling step number information.

[0173] The requirement of the non-uniform sampling step number information refers to the step number corresponding to the non-uniform sampling step number information. The model determines whether the number corresponding to the iteration number information is consistent with the step number corresponding to the non-uniform sampling step number information, so as to determine whether the model completes the iteration cycle of the test process.

[0174] Step S6041: If not, continue the next iteration and continue to obtain the iteration number information for loop judgment.

[0175] If the model determines that the number corresponding to the iteration number information is inconsistent with the step number corresponding to the non-uniform sampling step number information, it indicates that the model has not completed the iteration cycle of the test process, so the next iteration is continued, and the iteration number information is continuously obtained, so as to continuously monitor the change of the iteration number.

[0176] Step S6042: If it meets the requirement, the denoised image information is defined as the final watermark-containing image information.

[0177] If the model determines that the number corresponding to the iteration number information is consistent with the step number corresponding to the non-uniform sampling step number information, it indicates that the model has completed the iteration cycle of the test process. In the test process, the encoder will perform iterative denoising processing on the image, and after removing all noise, the final watermark-containing image information is obtained.

[0178] Referring to Figure 7 , the step of analyzing and determining the current iteration step information and the next iteration step information according to the iteration number information comprises:

[0179] Step S700: Analyzing and determining the step number change coefficient information according to the training step information, the non-uniform sampling step number information and the preset variable information.

[0180] Wherein, the variable information refers to a time step that remains unchanged in the iteration process, and in the embodiment, 1 is taken as an example. The step number change coefficient information refers to a coefficient for controlling the change of the time step in the iteration process, which is calculated by the model according to the training step information, the non-uniform sampling step number information and the variable information. The specific calculation formula is as follows:

[0181] ,

[0182] Wherein, is the step number change coefficient information, is the training step information, is the non-uniform sampling step number information, is the variable information.

[0183] Step S701: Analyzing and determining the current iteration step information according to the iteration number information, the training step information, the step number change coefficient information and the variable information.

[0184] Wherein, the current iteration step information in the step is consistent with the current iteration step information in the step S601, which is calculated by the model according to the iteration number information, the training step information, the step number change coefficient information and the variable information. The specific calculation formula is as follows:

[0185] ,

[0186] Wherein, is the current iteration step information, is the iteration number information.

[0187] Step S702: Analyzing and determining the next iteration number information according to the iteration number information.

[0188] Wherein, the next iteration number information refers to the number of the next iteration, which can be obtained by adding 1 to the iteration number information.

[0189] Step S703: Analyzing and determining the next iteration step information according to the next iteration number information, the training step information, the step number change coefficient information and the variable information.

[0190] Wherein, the next iteration step information in the step is consistent with the next iteration step information in the step S601, which is calculated by the model according to the next iteration number information, the training step information, the step number change coefficient information and the variable information. The specific calculation formula is as follows:

[0191] ,

[0192] wherein, is the next iteration step information, is the next iteration number information.

[0193] Based on the same inventive concept, an embodiment of the present application provides a robust image watermarking system based on a conditional diffusion model, comprising:

[0194] An acquisition module is configured to acquire training original image information, training watermark information, current time step information, sampled Gaussian noise information, test conditional noise image information, test original image information, test watermark information, and iteration number information.

[0195] A memory is configured to store a program of the robust image watermarking method based on the conditional diffusion model.

[0196] A processor, the program in the memory can be loaded and executed by the processor and implement the robust image watermarking method based on the conditional diffusion model.

[0197] Those skilled in the art can clearly understand that, for the convenience and brevity of description, only the division of the above functional modules is exemplified, and in actual application, the above functions can be completed by different functional modules according to needs, that is, the internal structure of the device is divided into different functional modules to complete all or part of the functions described above. The specific working process of the system, device and unit described above can refer to the corresponding process in the foregoing method embodiments, which will not be repeated here.

[0198] An embodiment of the present application provides a computer readable storage medium, which stores a computer program capable of being loaded and executed by a processor to execute a robust image watermarking method based on a conditional diffusion model.

[0199] The computer storage medium includes, for example, a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and various media that can store program codes.

[0200] Based on the same inventive concept, an embodiment of the present application provides a smart terminal, comprising a memory and a processor, and the memory stores a computer program capable of being loaded and executed by the processor to execute a robust image watermarking method based on a conditional diffusion model.

[0201] Those skilled in the art can clearly understand that, for the convenience and brevity of description, only the division of the above functional modules is taken as an example, and in actual application, the above functions can be completed by different functional modules according to needs, that is, the internal structure of the device is divided into different functional modules to complete all or part of the functions described above. The specific working process of the system, device and unit described above can refer to the corresponding process in the foregoing method embodiments, which will not be described here.

[0202] The above are preferred embodiments of the present application, and are not intended to limit the protection scope of the present application. Any feature disclosed in the specification (including the abstract and drawings) can be replaced by other equivalent or similar features unless specifically described. That is, each feature is only an example of a series of equivalent or similar features unless specifically described.

Claims

1. A robust image watermarking method based on a conditional diffusion model, characterized in that, The method comprises the following steps: According to the preset training step information, the preset training step is executed in a loop, which comprises the following steps: Obtain training original image information and training watermark information; Determine training conditional noise image information based on the training original image information; Use the training conditional noise image information, the training original image information and the training watermark information as input to control the preset training encoder to generate encoded image information; Use the encoded image information as input to control the preset attack layer to generate attack image information; Use the attack image information as input to control the preset training decoder to generate training decoded watermark information; Calculate the loss function information according to the training original image information, the encoded image information, the training watermark information and the training decoded watermark information, and update the encoder parameter information of the training encoder and the decoder parameter information of the training decoder according to the preset gradient descent method; The step of using the training conditional noise image information, the training original image information and the training watermark information as input to control the preset training encoder to generate encoded image information comprises the following steps: Determine the original feature information based on the training conditional noise image information and the training original image information; Perform multi-scale feature extraction and down-sampling on the original feature information to generate feature map information; Preprocess the training watermark information to generate watermark feature information; Fuse the feature map information and the watermark feature information to generate encoded image information; The step of fusing the feature map information and the watermark feature information to generate encoded image information comprises the following steps: Fuse and analyze the feature map information, the watermark feature information and the current time step information to determine the watermark-containing image feature information; Standardize and calculate the self-attention of the watermark-containing image feature information to generate self-attention feature information; Fuse and analyze the self-attention feature information and the current time step information to determine the enhanced feature information; 2. The robust image watermarking method based on a conditional diffusion model according to claim 1, characterized in that, Fuse and analyze the enhanced feature information, the watermark feature information and the feature map information, and perform up-sampling to generate encoded image information. The step of determining the training conditional noise image information based on the training original image information comprises the following steps: Obtain the current time step information; Determine the Gaussian noise intensity hyperparameter information based on the current time step information and the preset hyperparameter relationship information; Obtain the sampled Gaussian noise information; 3.The robust image watermarking method based on a conditional diffusion model according to claim 1, wherein, Determine the training conditional noise image information based on the sampled Gaussian noise information, the Gaussian noise intensity hyperparameter information and the training original image information. After the training step is completed according to the preset training step information in a loop, a test step is further included, which comprises the following steps: Select the training encoder with updated encoder parameter information as a test encoder, and select the training decoder with updated decoder parameter information as a test decoder according to the preset non-uniform sampling step number information; Obtain test conditional noise image information, test original image information and test watermark information; Use the test conditional noise image information, the test original image information and the test watermark information as input to control the test encoder to generate final watermark-containing image information; Use the final watermark-containing image information as input to control the test decoder to generate test decoded watermark information; The model evaluation information is generated by analyzing the final watermarked image information, the test original image information, the test watermark information and the test decoded watermark information.

4. The robust image watermarking method based on a conditional diffusion model according to claim 3, characterized in that, The step of generating the final watermarked image information by inputting the test condition noise image information, the test original image information and the test watermark information into the test encoder includes: obtaining the iteration number information; determining the current iteration step information and the next iteration step information according to the iteration number information; obtaining the corresponding test condition noise image information based on the current iteration step information; determining the denoised image information according to the next iteration step information, the current iteration step information, the test condition noise image information, the test original image information and the test watermark information; judging whether the iteration number information meets the requirement of the non-uniform sampling step number information; if not, continuing the next iteration and obtaining the iteration number information to make a loop judgment; if yes, defining the denoised image information as the final watermarked image information.

5. The robust image watermarking method based on a conditional diffusion model according to claim 4, characterized in that, The step of determining the current iteration step information and the next iteration step information according to the iteration number information includes: determining the step number change coefficient information according to the training step information, the non-uniform sampling step number information and the preset variable information; determining the current iteration step information according to the iteration number information, the training step information, the step number change coefficient information and the variable information; determining the next iteration number information according to the iteration number information; determining the next iteration step information according to the next iteration number information, the training step information, the step number change coefficient information and the variable information.

6. A robust image watermarking system based on a conditional diffusion model, characterized in that, The method comprises: an obtaining module, configured to obtain training original image information and training watermark information; a memory, configured to store a program of the robust image watermarking method based on the conditional diffusion model according to any one of claims 1 to 5; a processor, the program in the memory can be loaded and executed by the processor, and the robust image watermarking method based on the conditional diffusion model according to any one of claims 1 to 5 is implemented.

7. A smart terminal, characterized by The computer program stored on the memory can be loaded and executed by the processor, and the robust image watermarking method based on the conditional diffusion model according to any one of claims 1 to 5 is implemented.

Images