A fast watermarking method based on generative adversarial network

Abstract

The application discloses a fast watermarking method based on a generative adversarial network and belongs to the technical field of digital watermarking. The fast watermarking method is based on a consistent watermarking fast generation framework and solves the problem of high calculation resource requirement of an existing deep learning watermarking method. A traditional method needs to repeatedly call a deep model to generate watermarking for each image or video, and the calculation cost is high. The application only needs to generate watermarking once, and the watermarking can be embedded in any image or video without repeatedly calling the deep model. Moreover, the watermarking effect can be comparable to that of an existing method. The application is suitable for a scene needing efficient watermarking embedding and has a wide application prospect.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of digital watermarking technology, and particularly relates to a fast watermarking method based on a generative adversarial network. BACKGROUND

[0002] With the widespread dissemination and sharing of digital content, the issue of copyright protection for digital media has become increasingly prominent. Digital watermarking technology, as an effective means of copyright protection, has become an important technology in digital rights management. Digital watermarking technology embeds information secretly into digital media (such as images, videos, audio, etc.) to achieve functions such as copyright identification, content authentication, and piracy tracking. Traditional digital watermarking technology relies on signal processing algorithms, using methods such as discrete Fourier transform (DFT), discrete cosine transform (DCT), and discrete wavelet transform (DWT) to embed watermarks in the frequency domain or spatial domain. However, with the increasing complexity of digital content processing technology, traditional methods face challenges in terms of attack resistance, invisibility, and embedding capacity.

[0003] In recent years, the rapid development of deep learning technology has brought new opportunities to the field of digital watermarking. Deep learning models can automatically extract features from data and generate highly robust watermarks, enabling them to perform superiorly in resisting various complex image processing operations such as compression, cropping, and adding noise. However, existing digital watermarking methods based on deep learning typically require repeated calls to deep models each time watermark embedding is performed. This high-frequency model calling results in significant computational overhead, limiting the promotion of these methods in practical applications, especially in scenarios with limited computing resources or requiring fast processing. Therefore, how to significantly reduce the consumption of computing resources while maintaining high watermark recognition accuracy has become a key technical problem in the field of digital watermarking technology. SUMMARY

[0004] To address the shortcomings of the prior art, the present application proposes a fast watermarking method based on a generative adversarial network, which significantly reduces the consumption of computing resources while maintaining high watermark recognition accuracy.

[0005] The technical solution adopted by the present application is as follows:

[0006] A fast watermarking method based on a generative adversarial network, the method comprising the following steps:

[0007] Step 1: Obtain an image dataset and construct the network structure of a generator, a discriminator, and a decoder.

[0008] The input of the generator includes the original images in the image dataset and the set watermark message (which can be input by the user), which is used to generate a watermark image (i.e., a sample with a watermark image). And the message length of the watermark message of all original images is the same.

[0009] The discriminator network is used to evaluate the similarity of the watermark image generated by the generator and the real sample;

[0010] The decoder is used for watermark decoding of the watermark image generated by the generator, and outputs the reconstructed watermark message;

[0011] Step 2, initialize the network parameters of the generator, discriminator and decoder;

[0012] Step 3, jointly train the network parameters of the generator, discriminator and decoder based on the image recognition dataset;

[0013] Step 301, fix the network parameters of the discriminator and the decoder, and iteratively optimize the network parameters of the generator based on the loss function of the generator, so that the watermark image (i.e. the confusion sample) generated by the generator gradually approaches the data distribution of the real sample. That is, the loss function of the generator network is used to measure the quality of the generated sample, and through iterative optimization, the generated sample is more realistic.

[0014] Further, during the joint training and consistency training, the input image of the decoder is first processed with noise to obtain a distorted watermark image, which is then input into the decoder.

[0015] Further, the loss function of the generator during training can be set as:

[0016]

[0017] Where A is the discriminator, G is the generator, A(), G() respectively represent the output of the discriminator and the generator, and the watermark image generated by the generator I s is the original image, M is the watermark message, the superscript i is the sample number, n is the sample number, and α, β are balance hyperparameters, i.e. two preset weight coefficients, is the first loss of the generator, is the second loss of the generator, i.e. the mean square error between the generated watermark image and the original image ,

[0018] is the third loss of the generator, i.e. the similarity error between the generated sample and the original image, and n is the sample number.

[0019] Further, the third loss S_SSIM() can be specifically set as:

[0020]

[0021] Where x iand y i is the image after down-sampling at the i-th scale, w i is the weight of the scale, and L is the total number of scales. SSIM(x i , y i ) represents the similarity between images x i and y i , such as the structural similarity index, cosine similarity, and correlation coefficient, etc.

[0022] Step 302, fixing the network parameters of the generator and the discriminator, iteratively optimizing the network parameters of the decoder based on the loss function of the decoder, so that the reconstructed watermark message decoded by the decoder is consistent with the original message (i.e. the watermark message input into the generator);

[0023] Further, the loss function of the decoder during training can be set as:

[0024]

[0025] where D is the decoder, and D() represents the output of the decoder;

[0026] Step 303, fixing the network parameters of the generator and the decoder, iteratively optimizing the network parameters of the discriminator based on the loss function of the discriminator, so that the similarity judgment of the discriminator on the original image and the watermark image approaches the discrimination result "true", i.e. the training purpose is to make the discriminator as difficult as possible to judge the original image and the watermark image;

[0027] Further, the loss function of the discriminator during training can be set as:

[0028]

[0029] Steps 301 to 302 are repeatedly executed in sequence until the preset training convergence condition is reached, and the joint training is ended, wherein the training convergence condition can be set as the training round number reaching a preset maximum value;

[0030] Step 4, performing consistency training on the generator, the discriminator and the decoder trained jointly;

[0031] During consistency training, a new original image is selected from the image data set j , and a watermark message M is set based on it to obtain a watermark image through the generator , and the watermark residual ε j of the original image is calculated , and the new watermark image and input them into the discriminator and the decoder respectively;

[0032] The consistency training is sequentially performed in the order of first generating the generator, then the decoder and finally the discriminator, and the loss function of the generator and the discriminator in the consistency training is consistent with that in the joint training, and an item about the pixel-level error between the new watermark image and the original image is added to the loss function of the decoder in the joint training as the loss function of the consistency training of the decoder;

[0033] Further, the loss function of the decoder in the consistency training can be set as:

[0034]

[0035] wherein, is the reconstructed watermark message of the decoder when the new watermark image is input, that is, respectively represent the pixel values at the pixel position index k in the new watermark image and the original image , and idx h represents the index set of the pixels whose pixel values are greater than the preset upper limit, idx l represents the index set of the pixels whose pixel values are less than the preset lower limit, and η is a weight factor;

[0036] Step 5: obtaining the watermark image of the target image based on the generator completed by the consistency training;

[0037] Randomly selecting an image, setting a watermark message for the image, inputting the selected image and the set watermark message into the generator, calculating the watermark residual error ε' of the target image based on the watermark image generated by the generator, and adding the watermark residual error ε' to the target image to obtain the watermark image thereof.

[0038] Further, the network structure of the generator used in the present application includes three branches.

[0039] The first branch sequentially includes a linear layer, a convolutional layer, an upsampling block (Upsampling Block), an SE (Squeeze-and-Excitation) module 1, an SE module 2, an SE module-based decoding network (SE Decoder), an SE module 3 and an upsampling block.

[0040] The second branch sequentially includes a convolutional layer, an SE module 1, an SE module 2, an SE decoder, an SE module 3 and an upsampling block.

[0041] The third branch sequentially comprises an SE module 1, an SE decoder, an SE module 2, an upsampling block, an SE module 3 and a convolution block, wherein the convolution block sequentially comprises a convolution operation, a batch normalization operation and a ReLU activation function.

[0042] The input of the first branch is a watermark message, and the output feature map of the SE module 1 of the first branch is spliced with the output feature map of the SE module 1 of the second branch through a splicing layer (Cat) to serve as the input feature map of the SE module 2 of the first branch.

[0043] The input of the second branch is an original image, and the output feature map of the upsampling block of the first branch is spliced with the output feature map of the SE module 1 of the second branch through a splicing layer to serve as the input feature map of the SE module 2 of the second branch.

[0044] The output feature map of the upsampling block of the second branch is spliced with the output feature map of the SE module 1 of the second branch through a splicing layer to serve as the input feature map of the third branch (i.e., the input feature map of the SE module 1 of the third branch).

[0045] The decoding network based on the SE module comprises one convolution block, four SE modules, four SE_Decoder modules, one convolution layer and one linear layer (i.e., a full connection layer); and the network structure is sequentially a convolution block, a stacking layer in which the SE module and the SE_Decoder module are alternated four times, a convolution layer and a linear layer.

[0046] Further, the decoder used in the present application is a decoding network based on the SE module.

[0047] Further, the network structure of the discriminator used in the present application is a stacking structure of multiple convolution blocks. Preferably, the stacking structure of four convolution blocks can be set.

[0048] The technical scheme provided by the present application at least brings the following beneficial effects:

[0049] 1) Greatly reducing the calculation cost: the present application generates a watermark at one time and applies it to any image or video, without repeatedly calling a deep learning model. This design significantly reduces the use of computing resources, making the watermark generation process more efficient and suitable for resource-limited application scenarios.

[0050] 2) High concealment: the watermark image generated by the present application is indistinguishable from the original image with the naked eye. It has extremely high visual quality and does not affect the original image content.

[0051] 3) Strong anti-interference ability: In the face of common noise attacks, the present application can maintain 99.8% watermark extraction accuracy. This shows that the method has strong anti-interference ability and can still effectively protect the copyright and integrity of images or videos under various interference.

[0052] 4) Strong versatility: The watermark generated by the present application can be flexibly applied to any type of image or video, without the need to redesign or train models for different media types.

[0053] 5) Easy to integrate and promote: Due to the simplicity and efficiency of the present application, it can be more easily integrated into existing image processing or video editing software and more easily promoted for use in practical applications. DETAILED DESCRIPTION

[0054] To make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be described in detail and completely below in combination with the embodiments of the present application. Obviously, the described embodiments are only some of the embodiments of the present application, not all of the embodiments.

[0055] The embodiments of the present application provide a fast watermarking method based on a generative adversarial network, which can be used for digital watermarking of images or videos to solve the problem of high computational resource demand of existing deep learning watermarking methods. Traditional methods need to repeatedly call deep models to generate watermarks for each image or video, which has high computational cost. The method proposed in the embodiments of the present application only needs to generate a watermark once, which can be embedded in any image or video without repeatedly calling deep models. And in terms of watermarking effect, it can rival existing methods. It is suitable for scenarios that require efficient watermark embedding and has wide application prospects.

[0056] The fast watermarking method based on a generative adversarial network provided by the embodiments of the present application includes the following steps:

[0057] Step S1, obtaining an image dataset, such as a standard COCO (Common Objects in Context, Large-scale Image Recognition Dataset) dataset;

[0058] and respectively constructing a generator, a discriminator and a decoder network. The generative adversarial network is composed of the generator and the discriminator.

[0059] Step S2. Constructing a generator, the output of which is an image containing embedded watermark information, i.e. a watermark image.

[0060] In this embodiment, the constructed generator is composed of multiple modules, which include:

[0061] 1) Linear layer: processing watermark information at the beginning;

[0062] 2) ConvBNRelu module: short for convolution block, a series of convolution layers followed by batch normalization and ReLU activation function.

[0063] 3) SE module: Squeeze-and-Excitation module.

[0064] 4) Upsampling Block: used for upsampling features back to the original image size.

[0065] 5) Cat: performs a cat operation between different modules of the generator to merge features.

[0066] The generator constructed in this embodiment includes three branches, which are:

[0067] The first branch includes, in sequence, a linear layer, a convolution layer, an upsampling block, an SE module 1, an SE module 2, an SE module-based decoding network (SE Decoder), an SE module 3, and an upsampling block;

[0068] The second branch includes, in sequence, a convolution layer, an SE module 1, an SE module 2, an SE decoder, an SE module 3, and an upsampling block;

[0069] The third branch includes, in sequence, an SE module 1, an SE decoder, an SE module 2, an upsampling block, an SE module 3, and a convolution block, wherein the convolution block includes, in sequence, a convolution operation, a batch normalization operation, and a ReLU activation function;

[0070] The input of the first branch is a watermark message, and the output feature map of the SE module 1 of the first branch is concatenated with the output feature map of the SE module 1 of the second branch through a concatenation layer (Cat) to serve as the input feature map of the SE module 2 of the first branch;

[0071] The input of the second branch is an original image, and the output feature map of the upsampling block of the first branch is concatenated with the output feature map of the SE module 1 of the second branch through a concatenation layer to serve as the input feature map of the SE module 2 of the second branch;

[0072] The output feature map of the upsampling block of the second branch is concatenated with the output feature map of the SE module 1 of the second branch through a concatenation layer to serve as the input feature map of the third branch (i.e., the input feature map of the SE module 1 of the third branch);

[0073] The decoding network based on the SE module includes one convolution block, four SE modules, four SE_Decoder modules, one convolution layer, and one linear layer (such as a full connection layer); the network structure is in turn: the convolution block, the stacked layer in which the SE module and the SE_Decoder module are alternated four times, the convolution layer, and the linear layer.

[0074] The watermark image generation process of the generator starts from the input original image I s and the watermark information M (represented as a binary sequence, such as "0101000..."). The generator G is responsible for embedding the watermark information into the original image.

[0075] The information length of the watermark information M of each original image is consistent, that is, the binary length is the same, and the watermark information content can be the same or different, or partially the same.

[0076] Step S3. Construct a decoder for reconstructing the watermark information in the watermark image.

[0077] Before decoding, the watermark image is often distorted in social media transmission, so it is necessary to simulate this process. In the embodiment of the application, a noise layer is used to simulate this process, that is, the watermark image I wm After passing through the noise layer, the distorted image I no is obtained. no The distorted image I no is input into the decoder, and the decoder attempts to recover the watermark information M ex from the distorted image I ex . M ex needs to be as same as M as possible to reflect the robustness.

[0078] Specifically, the decoder includes the following modules:

[0079] 1) ConvBNRelu module: convolution layer with batch normalization and ReLU activation.

[0080] 2) SE module: Squeeze-and-Excitation module.

[0081] 3) SE_Decoder module: used for upsampling the decoded features to the size of the original image.

[0082] 4) Linear module: used for generating the final decoding message.

[0083] In this embodiment, the decoder includes 1 convolution block (ConvBNRelu), 4 SE modules, 4 SE_Decoder modules, 1 convolution layer and 1 linear layer; its network structure is: convolution block, stacked layer consisting of SE modules and SE_Decoder modules alternating four times, convolution layer and linear layer.

[0084] Step S4. Construct the discriminator (i.e., construct adversarial network A). Specifically, adversarial network A is designed to classify whether an image is watermarked. It consists of a downsampling block and a convolutional layer. In this embodiment, the discriminator is configured as a stack of four convolutional blocks (ConvBNRelu). Each layer of the discriminator progressively reduces the image size while extracting important features for classification, thereby improving the concealment of the watermark.

[0085] Step S5: Construct the first stage loss function.

[0086] The watermark image output by the generator can be expressed as follows:

[0087] I wm =G(I s ,M)

[0088] Among them, M is the set watermark information, I s is the original input image.

[0089] Set the generator's loss function to:

[0090]

[0091] Among them, A is the discriminator, G is the generator, A() and G() represent the output of the discriminator and the generator respectively, and the watermark image generated by the generator is I s is the original image, M is the watermark message, superscript i is the sample number, n is the number of samples, α and β are the balance hyperparameters, that is, the two preset weight coefficients, is the first loss of the generator, is the second loss of the generator, i.e. the generated watermark image With the original image The mean square error between

[0092] is the third loss of the generator, such as the structural similarity error between the generated sample and the original image, and n is the number of samples. The specific expression xx and the algorithm are as follows:

[0093]

[0094] Among them, x i and y iis the image after down-sampling at the i-th scale, w i is the weight of the scale, and L is the total number of scales.

[0095] The loss function of the decoder is constructed and optimized iteratively, so that the decoded watermark message of the decoder is consistent with the original message.

[0096]

[0097] where D is the decoder, D() represents the output of the decoder, is the generated watermark image of the watermark image of sample i, M i is the watermark message of sample i, MSE() represents the mean square error, and n is the number of samples.

[0098] The loss function of the discriminator is constructed and optimized iteratively, so that the discriminator is difficult to determine the original image and the watermark image as much as possible.

[0099]

[0100] where, is the original image of sample i, is the watermark image of sample i generated by the generator.

[0101] Step S6. Construct the second stage loss function and convert to consistency training. Wherein all kinds of loss function does not change. But the input of the original image and the watermark image is I' s and I' wm .

[0102] ε = I wm -I' s

[0103] I' wm = ε + I' s

[0104] where I' s is another new original image, I' wm is a new watermark image with added watermark residual ε.

[0105] Then all the loss functions are updated again. The generator loss is as follows:

[0106]

[0107] The decoder loss is as follows:

[0108]

[0109] The discriminator loss is as follows:

[0110]

[0111] where j is the index of the sample in consistency training, idx h idx l idx For example, the normal image pixel value range is [0, 255]. When put into the deep learning model for training, the pixel value is first normalized to 0-1, and then normalized to (-1-1), and the corresponding upper limit is 1 and the lower limit is -1.

[0112] Step S7. The parameters of the generator, discriminator and decoder networks are initialized. This includes randomly initializing the weights and biases of the networks to ensure initial diversity during training. In addition, a suitable optimization algorithm is selected and the learning rate is set to guide the update of network parameters. Preliminary training is performed and the initial performance of the generator, discriminator and decoder networks is verified respectively to ensure that each network can effectively learn when running independently, and then complete the two-stage training.

[0113] Step S8. After training is completed, a random image (denoted as I" s ) is input to the generator, and the generated watermark image I" wm of the current selected image is obtained based on the output of the generator, and then the watermark residual ε" of the selected image is calculated according to ε" = I" wm -I" s ; the watermark residual ε" is randomly added to the target image I t that needs to be watermarked to obtain the watermark image I t-wm of the target image = I t + ε". Wherein, the image dimension information of the target image I t is consistent with the input image of the generator.

[0114] The processing performance of the method provided in the embodiment of the present application is further verified through simulation experiments. The experimental environment is: Ubuntu 20.04 operating system, and the accelerated training and testing are performed on an NVIDIA RTX A6000 GPU. In the initial stage of the training process, an AdamW optimizer is used, and the learning rate is set to 1e-4. In this stage, 100 training cycles are performed. In order to ensure the fairness of the experiment, the noise layer with random parameters is uniformly applied in all instances in the present embodiment, and details are shown in Table 1. These noise layers include JPEG-Mask, JPEG-SS, Real-JPEG, Gaussian noise, Gaussian blur, color transformation, cropping, Dropout, rotation, resizing, PIP and the like. For the JPEG-related test, only Real-JPEG is used. The length of all messages is uniformly 30 bits. In terms of other settings, the default configuration of the model is adopted. In the second stage of training, the weight parameters and the noise layer remain consistent with the first stage, and a pair of different images is used for each iteration. This stage has a total of 50 training cycles. In the testing stage, we continue to maintain the random parameter form of the noise layer, and a large number of pictures and videos are tested. It is worth noting that the consistency of the total number of images is ensured in the performance test.

[0115] Table 1: Noise layer introduction

[0116]

[0117]

[0118] Simulation performance analysis:

[0119] (1) The robustness (bit extraction accuracy) results are shown in Table 2. The method provided in the embodiment of the present application has very good watermark extraction rate, only second to the best method 0.1%. But the method provided in the embodiment of the present application far exceeds other methods in the following two indicators, see Table 3 and Table 4.

[0120] Table 2: Average bit accuracy under various noise attacks

[0121] HiDDeN MBRS FIN DWSF Embodiments of the invention Average accuracy 91.4％ 99.9％ 97.8％ 99.9％ 99.8％

[0122] (2) The watermark image quality results are shown in Table 3. It can be clearly seen that the method provided in the embodiment of the present application improves the image quality (PSNR) by 2.2 dB compared with the relatively optimal method. And other indicators are almost unchanged.

[0123] Table 3: Watermark image quality

[0124] Model Peak signal-to-noise ratio (PSNR) Structural similarity (SSIM) Feature distance (LPIPS) x 100 HiDDeN 33.8 0.89 9.9 MBRS 35.9 0.91 9.7 FIN 33.3 0.84 1.8 DWSF 45.8 0.99 1.0 Embodiments of the invention 48.0 0.99 1.6

[0125] (3) The watermark generation speed acceleration ratio, the floating point operation quantity (FLOPs) and the parameter quantity of the model generating the watermark are shown in Table 4. The method proposed in the embodiment of the application has a significant speed improvement whether running on a GPU or on a CPU. The method proposed in the embodiment of the application can improve the watermark generation speed by 14000 times at most on a GPU. Moreover, the method proposed in the embodiment of the application does not require model-level calculation speed, and the FLOPs and the parameter quantity are both 0.

[0126] Table 4: Comparison of watermark generation speed acceleration ratio, floating point operation quantity (FLOPs) and parameter quantity of the model generating the watermark

[0127] Model Floating point operations (Mega) Parameter quantity (Mega) Speedup - GPU x 100 Speedup - CPU x 100 HiDDeN 2780 0.2 11.8 6.5 TSDL 27 0.0 15.7 10.2 MBRS 8372 0.6 50.0 39.0 FIN 1780 0.8 18.7 17.8 DWSF 26311 2.2 142.9 124.4 REVMark 46520 2.7 57.4 18.4 Embodiments of the invention 0 0 1 1

[0128] The fast watermark method based on the generative adversarial network proposed in the embodiment of the application is a new consistent watermark generation framework. The framework only needs to call a deep model once when generating a watermark, and the generated watermark can be directly embedded into any image or video without repeatedly calling the deep model at each embedding. Experimental results show that, compared with existing watermark methods based on deep learning, the residual watermark generation framework proposed in the embodiment of the application can significantly improve the speed of watermark embedding, and still maintain a watermark recognition accuracy as high as 99.8% under common noise attacks, while greatly reducing the consumption of computing resources. The proposal of this technology provides a more efficient and economical solution for the wide application of digital watermark technology.

[0129] Finally, it should be noted that: the above embodiments are only used to illustrate the technical solutions of the present application, and not to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that they can still modify the technical solutions recorded in the foregoing embodiments, or make equivalent replacement for part of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

[0130] The above only describes some embodiments of the present application. Those skilled in the art can make some modifications and improvements without departing from the concept of the present application, and these all belong to the protection scope of the present application.

Claims

1. A fast watermarking method based on a generative adversarial network, characterized in that, The method comprises the following steps: Step 1, obtaining an image data set and constructing network structures of a generator, a discriminator and a decoder; The input of the generator comprises original images in the image data set and a set watermark message, and the watermark message of all original images has the same length; The discriminator network is used for evaluating the similarity between the watermark image generated by the generator and the real sample; The decoder is used for watermark decoding of the watermark image generated by the generator, and outputs a reconstructed watermark message; Step 2, initializing network parameters of the generator, the discriminator and the decoder; Step 3, jointly training the network parameters of the generator, the discriminator and the decoder based on an image recognition data set; Step 301, fixing the network parameters of the discriminator and the decoder, and iteratively optimizing the network parameters of the generator based on a loss function of the generator, so that the watermark image generated by the generator approximates the original image; Step 302, fixing the network parameters of the generator and the discriminator, and iteratively optimizing the network parameters of the decoder based on a loss function of the decoder, so that the reconstructed watermark message decoded by the decoder is consistent with the set watermark message; Step 303, fixing the network parameters of the generator and the decoder, and iteratively optimizing the network parameters of the discriminator based on a loss function of the discriminator, so that the similarity judgment of the discriminator on the original image and the watermark image approximates the discrimination result "true"; Steps 301 to 303 are sequentially repeated until a preset training convergence condition is reached, and the joint training is ended; The loss function of the generator during the joint training is set as: wherein, is a generator, , respectively represent the output of the discriminator and the generator, the generator generates a watermark image , is an original image, is a watermark message, the superscript is a sample number, is a sample quantity, is a balance hyperparameter, and is a preset weight coefficient, represents a mean square error, represents a similarity error between the generated sample and the original image; Step 4, consistency training of the generator, the discriminator and the decoder trained jointly; During consistency training, new original images are selected from the image dataset , and based on the watermark message set on it Get the watermark image through the generator , and according to Calculate the original image The watermark residual ,according to Get the corresponding original image New watermark image for And input them into the discriminator and decoder respectively; The sequence of execution is first the generator, then the decoder, and finally the discriminator, and the loss function of the generator and the discriminator during consistency training is consistent with joint training, and an item about the new watermark image is added in the loss function of the decoder during joint training The pixel-level error term between the original image As the loss function of its consistency training; Step 5, obtaining a watermark image of a target image based on the generator trained consistently; Randomly selecting an image, setting a watermark message for the image, and inputting the selected image and the set watermark message into the generator; calculating a watermark residual of the target image based on the watermark image generated by the generator adding the watermark residual to the target image to obtain a watermark image thereof adding the watermark residual to the target image to obtain a watermark image thereof 2. The method of claim 1, wherein, During the joint training and the consistency training, the input image of the decoder is first processed with noise to obtain a distorted input image, and then the distorted input image is input into the decoder.

3. The method of claim 1, wherein, The loss function of the decoder during the joint training is set as: wherein, is a decoder, denotes an output of the decoder, is a sample watermark image of the generation of a watermark image, is a watermark message of the sample watermark image, denotes a mean squared error.

4. The method of claim 1, wherein, The loss function of the discriminator during the joint training is set as: wherein, is the original image of the sample, is the original image of the sample, is the watermark image generated by the generator for the sample, is the watermark image generated by the generator for the sample.

5. The method of claim 4, wherein, The loss function of the decoder during the consistency training is set as: wherein, is a decoder to reconstruct a watermark message from a new watermark image is a reconstructed watermark message at the input time, , respectively denote the pixel values at pixel position index in the new watermark image and the original image , denotes the index set of pixels whose pixel value is greater than a pre-set upper limit, denotes the index set of pixels whose pixel value is less than a pre-set lower limit, is a weight factor.

6. The method according to any one of claims 1 to 5, characterized in that, The network structure of the generator comprises three branches; The first branch comprises, in sequence, a linear layer, a convolution layer, an upsampling block, a squeeze-and-excitation SE module 1, an SE module 2, an SE module-based decoding network, an SE module 3 and an upsampling block; The second branch comprises, in sequence, a convolution layer, an SE module 1, an SE module 2, an SE decoder, an SE module 3 and an upsampling block; The third branch comprises, in sequence, an SE module 1, an SE decoder, an SE module 2, an upsampling block, an SE module 3 and a convolution block, wherein the convolution block comprises, in sequence, a convolution operation, a batch normalization operation and a ReLU activation function; The input of the first branch is the watermark message, and the output feature map of the SE module 1 of the first branch and the output feature map of the SE module 1 of the second branch are spliced through a splicing layer to serve as the input feature map of the SE module 2 of the first branch; The input of the second branch is the original image, and the output feature map of the upsampling block of the first branch and the output feature map of the SE module 1 of the second branch are spliced through a splicing layer to serve as the input feature map of the SE module 2 of the second branch; The output feature map of the upsampling block of the second branch and the output feature map of the SE module 1 of the second branch are spliced through a splicing layer to serve as the input feature map of the third branch; The decoding network based on the SE module includes one convolution block, four SE modules, four SE_Decoder modules, one convolution layer and one linear layer; the network structure is in turn: the convolution block, the stacking layer in which the SE module and the SE_Decoder module are alternated four times, the convolution layer and the linear layer.

7. The method according to any one of claims 1 to 5, wherein The decoder includes one convolution block, four squeeze-and-excitation SE modules, four SE_Decoder modules, one convolution layer and one linear layer. The network structure of the decoder is in turn: the convolution block, the stacking layer in which the SE module and the SE_Decoder module are alternated four times, the convolution layer and the linear layer.

8. The method according to any one of claims 1 to 5, wherein, The network structure of the discriminator is a stacking structure of multiple convolution blocks, wherein the convolution blocks include, in turn, a convolution operation, a batch normalization operation and a ReLU activation function.

9. The method of claim 7, wherein, The discriminator is set to a stacking structure of four convolution blocks.

Images