Rapid watermarking method based on generative adversarial network

Abstract

The invention discloses a rapid watermarking method based on a generative adversarial network, and belongs to the technical field of digital watermarking. According to the invention, based on a rapid generation framework of a consistent watermark, the problem that the existing deep learning watermark method needs a large amount of computing resources is solved. A traditional method needs to repeatedly call a depth model to generate a watermark for each image or video, and the calculation cost is high. According to the method, the watermark can be embedded into any image or video only by generating the watermark once, and a depth model does not need to be repeatedly called. And the watermarking effect can be comparable with that of the existing method. The method is suitable for scenes requiring efficient watermark embedding, and has a wide application prospect.

Description

Technical Field

[0001] The present invention relates to the technical field of digital watermarking, and in particular to a fast watermarking method based on a generative adversarial network. Background Art

[0002] With the widespread dissemination and sharing of digital content, the copyright protection of digital media has become increasingly prominent. As an effective means of copyright protection, digital watermarking technology has become an important technology in digital copyright management. Digital watermarking technology embeds information secretly into digital media (such as images, videos, audio, etc.) to achieve functions such as copyright identification, content authentication, and piracy tracking. Traditional digital watermarking technology mostly relies on signal processing algorithms, using discrete Fourier transform (DFT), discrete cosine transform (DCT), discrete wavelet transform (DWT) and other methods to embed watermarks in the frequency domain or spatial domain. However, with the complexity of digital content processing technology, traditional methods face challenges in terms of anti-attack, invisibility, and embedding capacity.

[0003] In recent years, the rapid development of deep learning technology has brought new opportunities to the field of digital watermarking. Deep learning models can automatically extract features from data and generate highly robust watermarks, which enable them to perform superior performance against various complex image processing operations (such as compression, cropping, adding noise, etc.). However, existing digital watermarking methods based on deep learning usually require repeated calls to deep models each time watermark embedding is performed. This high frequency of model calls leads to huge computational overhead, which limits the promotion of these methods in practical applications, especially in scenarios with limited computing resources or requiring fast processing. Therefore, how to significantly reduce the consumption of computing resources while ensuring the watermark embedding effect and robustness has become a key technical issue in the field of digital watermarking technology. Summary of the invention

[0004] In view of the shortcomings of the prior art, the present invention proposes a fast watermarking method based on a generative adversarial network, which significantly reduces the consumption of computing resources while maintaining high watermark recognition accuracy.

[0005] The technical solution adopted by the present invention is:

[0006] A fast watermarking method based on a generative adversarial network comprises the following steps:

[0007] Step 1: Obtain an image dataset and build a network structure of the generator, discriminator, and decoder;

[0008] The input of the generator includes the original image in the image data set and the set watermark message (which can be input by the user), which is used to generate the watermark image (i.e., the sample with the watermark image); and the message length of the watermark message of all the original images is the same;

[0009] The discriminator network is used to evaluate the similarity between the watermark image generated by the generator and the real sample;

[0010] The decoder is used to decode the watermark image generated by the generator and output the reconstructed watermark message;

[0011] Step 2, initialize the network parameters of the generator, discriminator and decoder;

[0012] Step 3: Jointly train the network parameters of the generator, discriminator, and decoder based on the image recognition dataset;

[0013] Step 301, the network parameters of the discriminator and the decoder are fixed, and the network parameters of the generator are iteratively optimized based on the loss function of the generator, so that the watermark image (i.e., the obfuscated sample) generated by the generator gradually approaches the data distribution of the real sample. That is, the loss function of the generator network is used to measure the quality of its generated samples, and through iterative optimization, the samples generated by the generator are made more realistic.

[0014] Furthermore, during the joint training and consistency training, the input image of the decoder is first subjected to noise processing to obtain a distorted watermark image, which is then input into the decoder.

[0015] Furthermore, the loss function of the generator during training can be set as:

[0016]

[0017] Among them, A is the discriminator, G is the generator, A() and G() represent the output of the discriminator and the generator respectively, and the watermark image generated by the generator is I s is the original image, M is the watermark message, superscript i is the sample number, n is the number of samples, α and β are the balance hyperparameters, i.e., the two preset weight coefficients, is the first loss of the generator, is the second loss of the generator, i.e. the generated watermark image With the original image The mean square error between

[0018] is the third loss of the generator, that is, the similarity error between the generated sample and the original image, and n is the number of samples.

[0019] Furthermore, the third loss S_SSIM() can be specifically set as:

[0020]

[0021] Among them, x iand i is the image downsampled at the i-th scale, w i is the weight of the scale, and L is the total number of scales. SSIM(x i ,y i ) represents the image x i and i Similarities between them, such as structural similarity index, cosine similarity, and correlation coefficient.

[0022] Step 302, fixing the network parameters of the generator and the discriminator, and iteratively optimizing the network parameters of the decoder based on the loss function of the decoder, so that the reconstructed watermark message decoded by the decoder is consistent with the original message (i.e., the watermark message input to the generator);

[0023] Furthermore, the loss function of the decoder during training can be set to:

[0024]

[0025] Where D is the decoder, and D() represents the output of the decoder;

[0026] Step 303, fix the network parameters of the generator and the decoder, and iteratively optimize the network parameters of the discriminator based on the loss function of the discriminator, so that the discriminator's judgment on the similarity between the original image and the watermark image is close to the judgment result "true", that is, the training purpose is to make it difficult for the discriminator to distinguish the original image and the watermark image as much as possible;

[0027] Furthermore, the loss function of the discriminator during training can be set as:

[0028]

[0029] Repeating step 301 to step 302 in sequence until the joint training is terminated when a preset training convergence condition is reached, wherein the training convergence condition can be set to that the number of training rounds reaches a preset maximum value;

[0030] Step 4: Perform consistency training on the jointly trained generator, discriminator, and decoder;

[0031] During consistency training, new original images are selected from the image dataset And based on the watermark message M set for it j Get the watermark image through the generator And according to Calculate the original image The watermark residual ε j ,according to Get the corresponding original image New watermark image for And input them into the discriminator and decoder respectively;

[0032] The generator, the decoder, and the discriminator are executed in sequence. The loss functions of the generator and the discriminator during consistency training are consistent with those of the joint training. An item about the new watermark image is added to the loss function of the decoder during joint training. With the original image The pixel-level error term between them is used as the loss function for its consistency training;

[0033] Furthermore, the loss function of the decoder during consistency training can be set as:

[0034]

[0035] in, Provide the decoder with a new watermark image is the reconstructed watermark message at the time of input, that is Represent the new watermark image and the original image The pixel value at pixel position index k in , idx h Indicates the index set of pixels whose pixel values ​​are greater than the preset upper limit, idx l Represents the index set of pixels whose pixel values ​​are less than the pre-examination lower limit, and η is the weight factor;

[0036] Step 5, obtaining a watermark image of the target image based on the generator that has completed consistency training;

[0037] An image is randomly selected and a watermark message is set for it. The selected image and the set watermark message are then input into the generator. The watermark residual ε′ of the target image is calculated based on the watermark image generated by the generator. The watermark residual ε′ is then added to the target image to obtain its watermark image.

[0038] Furthermore, the network structure of the generator used in the present invention includes three branches;

[0039] The first branch includes: a linear layer, a convolutional layer, an upsampling block, a SE (Squeeze-and-Excitation) module 1, a SE module 2, a decoding network based on the SE module (SE Decoder), a SE module 3 and an upsampling block;

[0040] The second branch includes: convolutional layer, SE module 1, SE module 2, SE decoder, SE module 3 and upsampling block in sequence;

[0041] The third branch includes: SE module 1, SE decoder, SE module 2, upsampling block, SE module 3 and convolution block in sequence, wherein the convolution block includes convolution operation, batch normalization operation and ReLU activation function in sequence;

[0042] The input of the first branch is the watermark message, and the output feature map of the SE module 1 of the first branch and the output feature map of the SE module 1 of the second branch are concatenated through the concatenation layer (Cat) and then used as the input feature map of the SE module 2 of the first branch;

[0043] The input of the second branch is the original image, and the output feature map of the upsampling block of the first branch and the output feature map of the SE module 1 of the second branch are concatenated through the concatenation layer and then used as the input feature map of the SE module 2 of the second branch;

[0044] The output feature map of the upsampling block of the second branch and the output feature map of the SE module 1 of the second branch are concatenated through the concatenation layer and then used as the input feature map of the third branch (i.e., the input feature map of the SE module 1 of the third branch);

[0045] Among them, the decoding network based on the SE module includes 1 convolution block, 4 SE modules, 4 SE_Decoder modules, 1 convolution layer and 1 linear layer (i.e., fully connected layer); its network structure is: convolution block, stacked layer consisting of SE module and SE_Decoder module alternating four times, convolution layer and linear layer.

[0046] Furthermore, the decoder used in the present invention is a decoding network based on the SE module.

[0047] Furthermore, the network structure of the discriminator used in the present invention is a stacked structure of multiple convolution blocks. Preferably, it can be set to a stacked structure of four convolution blocks.

[0048] The technical solution provided by the present invention brings at least the following beneficial effects:

[0049] 1) Significantly reduce computing costs: This invention generates a watermark once and applies it to any image or video, without repeatedly calling the deep learning model. This design significantly reduces the use of computing resources, making the watermark generation process more efficient and suitable for application scenarios with limited resources.

[0050] 2) High concealment: The watermarked image generated by the present invention cannot be distinguished from the original image by naked eyes. It has extremely high visual quality and will not affect the original image content.

[0051] 3) Strong anti-interference ability: When facing common noise attacks, the present invention can maintain 99.8% watermark extraction accuracy. This shows that the method has strong anti-interference ability and can effectively protect the copyright and integrity of images or videos under various interferences.

[0052] 4) Strong versatility: The watermark generated by the present invention can be flexibly applied to any type of image or video, without the need to redesign or train models for different media types.

[0053] 5) Easy to integrate and promote: Due to the simplicity and efficiency of the present invention, it can be more easily integrated into existing image processing or video editing software and more easily promoted and used in practical applications. DETAILED DESCRIPTION

[0054] To make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be described in detail and completely in combination with the implementation of the present invention. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments.

[0055] The embodiment of the present invention provides a fast watermarking method based on a generative adversarial network, which can be used for digital watermark generation of images or videos to solve the problem of large computing resource requirements of existing deep learning watermarking methods. Traditional methods require repeated calls to deep models to generate watermarks for each image or video, which has high computational costs. The method proposed in the embodiment of the present invention only needs to generate a watermark once, and it can be embedded in any image or video without repeatedly calling the deep model. And the watermark effect is comparable to existing methods. It is suitable for scenarios where efficient watermark embedding is required, and has broad application prospects.

[0056] The embodiment of the present invention provides a fast watermarking method based on a generative adversarial network, comprising the following steps:

[0057] Step S1, obtaining an image dataset, such as a standard COCO (Common Objects in Context, a large-scale image recognition dataset);

[0058] And construct the generator, discriminator and decoder network respectively. The generative adversarial network consists of a generator and a discriminator.

[0059] Step S2: construct a generator whose output is an image containing embedded watermark information, namely, a watermarked image.

[0060] In this embodiment, the constructed generator consists of multiple modules, including:

[0061] 1) Linear layer: processes watermark information at the beginning;

[0062] 2) ConvBNRelu module: abbreviated as convolutional block, a series of convolutional layers followed by batch normalization and ReLU activation function.

[0063] 3) SE module: Squeeze-and-Excitation module.

[0064] 4) Upsampling Block: used to upsample the features back to the original image size.

[0065] 5) Concatenation (Cat): Concatenation operations are performed between different modules of the generator to merge features.

[0066] The generator constructed in this embodiment includes three branches, which are:

[0067] The first branch includes: a linear layer, a convolutional layer, an upsampling block, a SE module 1, a SE module 2, a decoding network based on the SE module (SE Decoder), a SE module 3 and an upsampling block;

[0068] The second branch includes: convolutional layer, SE module 1, SE module 2, SE decoder, SE module 3 and upsampling block in sequence;

[0069] The third branch includes: SE module 1, SE decoder, SE module 2, upsampling block, SE module 3 and convolution block in sequence, wherein the convolution block includes convolution operation, batch normalization operation and ReLU activation function in sequence;

[0070] The input of the first branch is the watermark message, and the output feature map of the SE module 1 of the first branch and the output feature map of the SE module 1 of the second branch are concatenated through the concatenation layer (Cat) and then used as the input feature map of the SE module 2 of the first branch;

[0071] The input of the second branch is the original image, and the output feature map of the upsampling block of the first branch and the output feature map of the SE module 1 of the second branch are concatenated through the concatenation layer and then used as the input feature map of the SE module 2 of the second branch;

[0072] The output feature map of the upsampling block of the second branch and the output feature map of the SE module 1 of the second branch are concatenated through the concatenation layer and then used as the input feature map of the third branch (i.e., the input feature map of the SE module 1 of the third branch);

[0073] Among them, the decoding network based on the SE module includes 1 convolution block, 4 SE modules, 4 SE_Decoder modules, 1 convolution layer and 1 linear layer (such as a fully connected layer); its network structure is: a convolution block, a stacked layer consisting of SE modules and SE_Decoder modules alternating four times, a convolution layer and a linear layer.

[0074] The watermark image generation process of the generator starts with the input original image I s And watermark information M (expressed as a binary sequence, such as "0101000..."). The generator G is responsible for embedding the watermark information into the original image.

[0075] The information length of the watermark information M of each original image remains consistent, that is, the binary length is the same, and the content of the watermark information can be the same or different, or partially the same.

[0076] Step S3: construct a decoder for reconstructing the watermark information in the watermark image.

[0077] Before decoding, the watermark image is often distorted in social media transmission, so it is necessary to simulate this process. In the embodiment of the present invention, a noise layer is used to simulate this process. wm After passing through the noise layer, the distorted image I is obtained. no , and then the distorted image I no The decoder tries to transform the distorted image I no Recover watermark information M ex . M ex It needs to be as similar as possible to M to be robust.

[0078] Specifically, the decoder includes the following modules:

[0079] 1) ConvBNRelu module: convolutional layer with batch normalization and ReLU activation.

[0080] 2) SE module: Squeeze-and-Excitation module.

[0081] 3) SE_Decoder module: used to upsample the decoded features to the size of the original image.

[0082] 4) Linear module: used to generate the final decoded message.

[0083] In this embodiment, the decoder includes 1 convolution block (ConvBNRelu), 4 SE modules, 4 SE_Decoder modules, 1 convolution layer and 1 linear layer; its network structure is: a convolution block, a stacked layer consisting of SE modules and SE_Decoder modules alternating four times, a convolution layer and a linear layer.

[0084] Step S4. Construct a discriminator (i.e., construct an adversarial network A). Specifically, the adversarial network A is designed to classify whether an image has a watermark. It consists of a downsampling block and a convolutional layer. In this embodiment, the discriminator is set to a stacked structure of four convolutional blocks (ConvBNRelu). Each network layer of the discriminator gradually reduces the image size and extracts important features for classification, thereby improving the concealment of the watermark.

[0085] Step S5: Construct the first stage loss function.

[0086] The watermark image output by the generator can be expressed as follows:

[0087] I wm =G(I s ,M)

[0088] Among them, M is the set watermark information, I s is the original input image.

[0089] Set the generator's loss function to:

[0090]

[0091] Among them, A is the discriminator, G is the generator, A() and G() represent the output of the discriminator and the generator respectively, and the watermark image generated by the generator is I s is the original image, M is the watermark message, superscript i is the sample number, n is the number of samples, α and β are the balance hyperparameters, i.e., the two preset weight coefficients, is the first loss of the generator, is the second loss of the generator, i.e. the generated watermark image With the original image The mean square error between

[0092] is the third loss of the generator, such as the structural similarity error between the generated sample and the original image, and n is the number of samples. The specific expression xx and the algorithm are as follows:

[0093]

[0094] Among them, x i and iis the image downsampled at the i-th scale, w i is the weight of the scale, L is the total number of scales;

[0095] Construct the loss function of the decoder, and through iterative optimization, make the watermark message decoded by the decoder consistent with the original message.

[0096]

[0097] Where D is the decoder, D() represents the output of the decoder, is the generated watermark image of sample i, M i is the watermark message of sample i, MSE() represents mean square error, and n is the number of samples;

[0098] Construct the loss function of the discriminator, and through iterative optimization, make it difficult for the discriminator to distinguish the original image and the watermarked image as much as possible.

[0099]

[0100] in, is the original image of sample i, The watermark image of sample i generated by the generator.

[0101] Step S6. Construct the second-stage loss function and switch to consistency training. Among them, all kinds of loss functions remain unchanged. However, the input original image and watermark image are I′ respectively. s and I′ wm .

[0102] ε=I wm -I′ s

[0103] I′ wm =ε+I′ s

[0104] Among them, I′ s is another new original image, I′ wm is the new watermark image with the watermark residual ε added.

[0105] Then re-update all loss functions. The generator loss is as follows:

[0106]

[0107] The decoder loss is as follows:

[0108]

[0109] The discriminator loss is as follows:

[0110]

[0111] Among them, j is the sample index during consistency training, idx h Indicates the index set of pixels whose pixel values ​​are greater than the upper limit, idx l Represents the index set of pixels whose pixel values ​​are less than the lower limit. "For example, the normal image pixel value range is [0,255]. When training the deep learning model, the pixel value is first normalized to between 0 and 1, and then normalized to between (-1 and 1). The corresponding upper limit is 1 and the lower limit is -1.

[0112] Step S7. Initialize the parameters of the generator, discriminator, and decoder networks. This includes randomly initializing the weights and biases of the network to ensure initial diversity during training. In addition, select a suitable optimization algorithm and set the learning rate to guide the update of network parameters. Perform preliminary training and verify the initial performance of the generator, discriminator, and decoder networks separately to ensure that each network can learn effectively when running independently, followed by a full two-stage training.

[0113] Step S8. After the training is completed, randomly select an image (denoted as I″ s ) is input to the generator, and the generated watermark image I″ of the currently selected image is obtained based on the output of the generator. wm , then according to ε″=I″ wm -I″ s Calculate the watermark residual ε″ of the selected image; randomly add the watermark residual ε″ to the target image I that needs watermark t The watermark image I of the target image is obtained t-wm =I t +ε″. Among them, the target image I t Consistent with the image dimension information of the generator's input image.

[0114] The processing performance of the method proposed in the embodiment of the present invention is further verified by simulation experiments. The experimental environment is: Ubuntu 20.04 operating system, and accelerated training and testing are performed on NVIDIA RTX A6000 GPU. In the initial stage of the training process, the AdamW optimizer is used, and the learning rate is set to 1e-4. In this stage, 100 training cycles were carried out. In order to ensure the fairness of the experiment, the noise layer with random parameters is consistently applied in all instances of this embodiment, as shown in Table 1 for details. These noise layers include JPEG-Mask, JPEG-SS, Real-JPEG, Gaussian noise, Gaussian blur, color transformation, cropping, Dropout, rotation, resizing, PIP, etc. For JPEG-related tests, only Real-JPEG is used. The length of all messages is unified to 30 bits. The default configuration of the model is adopted in other settings. In the second stage of training, the weight parameters and noise layer remain consistent with the first stage, and a different pair of images is used for each iteration. A total of 50 training cycles were carried out in this stage. In the test phase, we continue to maintain the random parameter form of the noise layer and test it with a large number of pictures and videos. It is worth noting that the consistency of the total number of images was ensured in the performance test.

[0115] Table 1 Introduction to noise floor

[0116]

[0117]

[0118] Simulation performance analysis:

[0119] (1) Robustness (bit extraction accuracy) The results are shown in Table 2. The method proposed in the embodiment of the present invention has a very good watermark extraction rate, second only to the best method by 0.1%. However, the method proposed in the embodiment of the present invention far exceeds other methods in the following two indicators, see Table 3 and Table 4.

[0120] Table 2 Average bit accuracy under various noise attacks

[0121] HkDJ MBRS FIN DWSF Embodiments of the present invention Average accuracy 91.4％ 99.9％ 97.8％ 99.9％ 99.8％

[0122] (2) Watermark image quality results are shown in Table 3, which clearly shows that the method proposed in the embodiment of the present invention improves the image quality (PSNR) by 2.2 dB compared with the optimal method, while other indicators remain almost unchanged.

[0123] Table 3 Watermark image quality

[0124] Model Peak Signal-to-Noise Ratio (PSNR) Structural Similarity (SSIM) Feature distance (LPIPS) x100 HkDJ 33.8 0.89 9.9 MBRS 35.9 0.91 9.7 FIN 33.3 0.84 1.8 DWSF 45.8 0.99 1.0 Embodiments of the present invention 48.0 0.99 1.6

[0125] (3) Watermark generation speed acceleration ratio. The floating point operations (FLOPs) and parameter counts of the model for generating watermarks are shown in Table 4. Whether running on a GPU or a CPU, the method proposed in the embodiment of the present invention has a significant speed increase. The method proposed in the embodiment of the present invention can increase the watermark generation speed by up to 14,000 times on a GPU. In addition, the method proposed in the embodiment of the present invention does not require model-level calculation speed, and both FLOPs and parameter counts are 0.

[0126] Table 4: Comparison of watermark generation speed acceleration, floating point operations (FLOPs) and parameters of the model to generate watermarks

[0127] Model Floating point operations (Mega) Number of parameters (megabytes) Speedup ratio - GPU x100 Speedup ratio - CPU x100 HkDJ 2780 0.2 11.8 6.5 TSDL 27 0.0 15.7 10.2 MBRS 8372 0.6 50.0 39.0 FIN 1780 0.8 18.7 17.8 DWSF 26311 2.2 142.9 124.4 REVMark 46520 2.7 57.4 18.4 Embodiments of the present invention 0 0 1 1

[0128] The fast watermarking method based on generative adversarial network proposed in the embodiment of the present invention is a new consistent watermark generation framework, which only needs to call the deep model once when generating the watermark, and the generated watermark can be directly embedded into any image or video without repeatedly calling the deep model each time it is embedded. Experimental results show that compared with the existing watermarking method based on deep learning, the residual watermark generation framework proposed in the embodiment of the present invention can significantly improve the speed of watermark embedding, and still maintain a watermark recognition accuracy of up to 99.8% under common noise attacks, while greatly reducing the consumption of computing resources. The introduction of this technology provides a more efficient and economical solution for the widespread application of digital watermarking technology.

[0129] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

[0130] The above are only some embodiments of the present invention. For those skilled in the art, several modifications and improvements can be made without departing from the creative concept of the present invention, which all belong to the protection scope of the present invention.

Claims

1. A fast watermarking method based on generative adversarial network, characterized in that: The following steps are involved: Step 1: Obtain an image dataset and build a network structure of the generator, discriminator, and decoder; The input of the generator includes the original image in the image data set and the set watermark message, which is used to generate the watermark image, and the message length of the watermark message of all the original images is the same; The discriminator network is used to evaluate the similarity between the watermark image generated by the generator and the real sample; The decoder is used to decode the watermark image generated by the generator and output the reconstructed watermark message; Step 2, initialize the network parameters of the generator, discriminator and decoder; Step 3: Jointly train the network parameters of the generator, discriminator, and decoder based on the image recognition dataset; Step 301, fixing the network parameters of the discriminator and the decoder, and iteratively optimizing the network parameters of the generator based on the loss function of the generator, so that the watermark image generated by the generator is close to the original image; Step 302, fixing the network parameters of the generator and the discriminator, and iteratively optimizing the network parameters of the decoder based on the loss function of the decoder, so that the reconstructed watermark message decoded by the decoder is consistent with the set watermark message; Step 303, fix the network parameters of the generator and the decoder, and iteratively optimize the network parameters of the discriminator based on the loss function of the discriminator, so that the discriminator's judgment on the similarity between the original image and the watermark image is close to the judgment result "true"; Repeat step 301 to step 302 in sequence until the joint training is terminated when a preset training convergence condition is reached; Step 4: Perform consistency training on the jointly trained generator, discriminator, and decoder; During consistency training, new original images are selected from the image dataset And based on the watermark message M set for it j Get the watermark image through the generator And according to Calculate the original image The watermark residual ε j ,according to Get the corresponding original image New watermark image for And input them into the discriminator and decoder respectively; The generator, the decoder, and the discriminator are executed in sequence. The loss functions of the generator and the discriminator during consistency training are consistent with those of the joint training. An item about the new watermark image is added to the loss function of the decoder during joint training. With the original image The pixel-level error term between them is used as the loss function for its consistency training; Step 5, obtaining a watermark image of the target image based on the generator that has completed consistency training; Randomly select an image and set a watermark message for it, then input the selected image and the set watermark message into the generator; The watermark residual ε′ of the target image is calculated based on the watermark image generated by the generator, and then the watermark residual ε′ is added to the target image to obtain its watermark image.

2. The method according to claim 1, characterized in that During joint training and consistency training, the input image of the decoder is first subjected to noise processing to obtain a distorted input image, which is then input into the decoder.

3. The method according to claim 1, characterized in that The loss function of the generator during joint training is set to: Among them, G is the generator, A() and G() represent the output of the discriminator and the generator respectively, and the watermark image generated by the generator I s is the original image, M is the watermark message, superscript i is the sample number, n is the number of samples, α, β are balance hyperparameters, is the preset weight coefficient, MSE() represents the mean square error, MS_SSIM() represents the similarity error between the generated sample and the original image, and n is the number of samples.

4. The method according to claim 1, characterized in that The loss function of the decoder during training is set to: Where D is the decoder, D() represents the output of the decoder, is the generated watermark image of sample i, M i is the watermark message of sample i, MSE() represents the mean square error, and n is the number of samples.

5. The method according to claim 1, characterized in that The loss function of the discriminator during training is set to: Among them, G is the generator, A() represents the output of the discriminator, is the original image of sample i, is the watermark image of sample i generated by the generator, and n is the number of samples.

6. The method according to claim 4, characterized in that The loss function of the decoder during consistency training is set to: in, Provide the decoder with a new watermark image is the reconstructed watermark message at input, Represent the new watermark image and the original image The pixel value at pixel position index k in , idx h Indicates the index set of pixels whose pixel values ​​are greater than the preset upper limit, idx l Represents the index set of pixels whose pixel values ​​are less than the pre-examination lower limit, and η is the weight factor.

7. The method according to any one of claims 1 to 6, characterized in that: The network structure of the generator includes three branches; The first branch includes: a linear layer, a convolutional layer, an upsampling block, a squeeze and excitation SE module 1, a SE module 2, a decoding network based on the SE module, a SE module 3 and an upsampling block; The second branch includes: convolutional layer, SE module 1, SE module 2, SE decoder, SE module 3 and upsampling block in sequence; The third branch includes: SE module 1, SE decoder, SE module 2, upsampling block, SE module 3 and convolution block in sequence, wherein the convolution block includes convolution operation, batch normalization operation and ReLU activation function in sequence; The input of the first branch is the watermark message, and the output feature map of the SE module 1 of the first branch and the output feature map of the SE module 1 of the second branch are concatenated through the concatenation layer and then used as the input feature map of the SE module 2 of the first branch; The input of the second branch is the original image, and the output feature map of the upsampling block of the first branch and the output feature map of the SE module 1 of the second branch are concatenated through the concatenation layer and then used as the input feature map of the SE module 2 of the second branch; The output feature map of the upsampling block of the second branch and the output feature map of the SE module 1 of the second branch are concatenated through a concatenation layer and then used as the input feature map of the third branch; Among them, the decoding network based on the SE module includes 1 convolution block, 4 SE modules, 4 SE_Decoder modules, 1 convolution layer and 1 linear layer; its network structure is: convolution block, stacked layer consisting of SE module and SE_Decoder module alternating four times, convolution layer and linear layer.

8. The method according to any one of claims 1 to 6, characterized in that The decoder consists of 1 convolutional block, 4 squeeze and excitation SE modules, 4 SE_Decoder modules, 1 convolutional layer and 1 linear layer; The network structure of the decoder is: convolutional block, stacked layer consisting of SE module and SE_Decoder module alternating four times, convolutional layer and linear layer.

9. The method according to any one of claims 1 to 6, characterized in that: The network structure of the discriminator is a stacked structure of multiple convolutional blocks, where the convolutional blocks sequentially include convolution operations, batch normalization operations, and ReLU activation functions.

10. The method according to claim 8, characterized in that The discriminator is set as a stacked structure of four convolutional blocks.

Images