A robust watermarking method for self-encoded color images based on visual perception

Abstract

The present invention discloses a robust watermark processing method for self-encoded color images based on visual perception. The watermark embedding module in AGWNet is trained using a carrier image training set and an original watermark image training set to obtain the structural loss and visual perception loss of the watermarked image and the carrier image; the watermarked image is input into the noise attack module in AGWNet to generate the watermarked image after the attack; the watermarked image after the attack is input into the watermark extraction module in AGWNet for training to obtain the cross entropy loss of the extracted watermark and the original watermarked image; the overall loss is calculated according to the structural loss, visual perception loss and cross entropy loss, and when the overall loss is less than a threshold value in the iteration, the trained AGWNet is obtained; the trained AGWNet is used to complete the embedding or extraction of the watermark. The present invention can improve the robustness of the watermark while ensuring the quality of the watermarked image.

Description

Technical Field

[0001] The invention relates to a self-encoded color image robust watermark processing method based on visual perception, belonging to the technical field of digital image watermarking. Background Art

[0002] The quality of the watermarked image, i.e., the invisibility of the watermark, and the accuracy of the watermark extracted under attack, i.e., the robustness of the watermark to attacks, are two important indicators for evaluating the quality of the watermarking method. Since 2017, many color image robust watermarking methods based on deep learning have emerged. In 2018, Zhu et al. first proposed to introduce the autoencoder structure into the field of color image robust watermarking, creatively regarded the watermark embedding and extraction process as the reconstruction of image information, and proposed an end-to-end training method, which cleverly allowed the neural network to adaptively find the balance between invisibility and robustness. However, how to better guide the network to find a better solution in this dual-task optimization problem and enhance the interpretability of the network embedding is still a difficult problem in academia.

[0003] In terms of research on improving algorithm robustness, the most work has been done on improving JPEG compression attacks. In terms of research on improving the invisibility of watermarked images, with the popularity of attention mechanisms, in 2020, Yu et al. first proposed to introduce an attention model in the watermarking field, generate an attention mask for the carrier image through a pre-trained ResNet50 network, and fuse the carrier, watermark, and attention mask to produce a better watermarked image. However, this method only relies on the feature map generated by another network as a guide for embedding, and does not fully consider the robustness of each pixel to the encoded information.

[0004] Zhang et al. proposed to use the gradient of the carrier reconstruction loss to generate an attention mask, and screen suitable embedding positions in advance based on the generated mask. Therefore, the watermark embedding capacity of this scheme is limited and it is difficult to process two-dimensional meaningful watermark information. The maximum embedding capacity shown in their scheme only supports 90 bits. At the same time, this method is slightly lacking in network interpretability.

[0005] Based on the above analysis, it is difficult for existing end-to-end trained robust watermarking algorithms to give competitive experimental results in terms of both invisibility and robustness. The reason is that they do not solve the following two problems well: (1) No prior knowledge guidance is added in the watermark embedding stage, and different embedding strengths are given to different positions of the carrier, and the watermark embedding method lacks interpretability; (2) The problem of long-distance feature loss is not solved when reconstructing the image in the embedding and extraction stages, and the network is difficult to achieve an efficient balance when finding the optimal point of the dual tasks. Summary of the invention

[0006] Objective: In order to overcome the difficulty in achieving both watermark invisibility and robustness in end-to-end training in the prior art, the present invention provides a self-encoded color image robust watermark processing method based on visual perception, which can improve the robustness of the watermark while ensuring the quality of the watermarked image.

[0007] Technical solution: To solve the above technical problems, the technical solution adopted by the present invention is:

[0008] A robust watermark processing method for self-encoded color images based on visual perception includes the following steps:

[0009] The watermark embedding module in AGWNet is trained using the carrier image training set and the original watermark image training set to obtain the structural loss and visual perception loss of the watermarked image and the carrier image.

[0010] The watermarked image is input into the noise attack module in AGWNet to generate the watermarked image after the attack.

[0011] The attacked watermark image is input into the watermark extraction module in AGWNet for training, and the cross entropy loss between the extracted watermark and the original watermark image is obtained.

[0012] The overall loss is calculated based on the structural loss, visual perception loss and cross entropy loss. When the overall loss is less than the threshold in the iteration, the trained AGWNet is obtained.

[0013] The trained AGWNet is used to embed or extract the watermark.

[0014] As a preferred solution, the watermark embedding module in AGWNet is trained using the carrier image training set and the original watermark image training set, including:

[0015] 1-1: Extract the B channel of the carrier image and input it into the watermark embedding module, and retain the R and G channels of the carrier image.

[0016] 1-2: Input the B channel into the convolution layer, and the convolution layer outputs a feature map of size b×32×32×63.

[0017] 1-3: Randomly generate a 32×32 binary watermark image, concat the binary watermark image with the feature map on the feature channel, and the concat layer outputs 1×32×32×64 watermark carrier features.

[0018] 1-4: Map the watermark carrier features to three new feature spaces: Query, Key, and Value. Perform attention convolution on the three new feature spaces. After attention convolution, Query and Key obtain outputs of size 1×32×32×8, while Value obtains outputs of size 1×32×32×63. Transpose the convolved Key and perform matrix multiplication with the convolved Query, and perform Softmax on the result to obtain the attention mask. Concat the convolved Value with the original watermark image on the feature channel, multiply the result obtained by Concat with the attention mask by matrix elements, and multiply it by the embedding factor to obtain the watermark carrier features.

[0019] 1-5: The watermark carrier features are concatenated through the attention residual connection layer to obtain weighted watermark carrier features with a size of 1×32×32×64.

[0020] 1-6: Use deconvolution and convolution layers to reconstruct the weighted watermark carrier features, and the output size is 1×256×256×1 containing the watermark B channel.

[0021] 1-7: Splice the watermarked B channel with the R and G channels of the carrier image to restore the watermarked color image.

[0022] As a preferred solution, the structural loss calculation formula is as follows:

[0023]

[0024] Among them, I is the original carrier image, I w is the watermarked image, μ I and σ I are the mean and variance of the original carrier pixels, and are the mean and variance of the watermarked image respectively, c1 and c2 are balance constants to prevent the denominator from being zero.

[0025] As a preferred solution, the visual perception loss calculation formula is as follows:

[0026]

[0027] Among them, M eye is the penalty weight, M c 、N c is the image size length and width, i, j is the original carrier image I and the watermarked image I w The length and width of a pixel.

[0028] in,

[0029] Among them, σc is the ratio of chromaticity weight, σ t is the ratio of texture weight, M color is the chromaticity weight, M texture is the texture weight.

[0030] As a preferred solution, the step of inputting the watermarked image into the noise attack module in AGWNet to generate the watermarked image after the attack includes:

[0031] A signal processing attack is randomly selected for the watermarked image to generate an attacked watermarked image. The signal processing attack includes at least one of the following: JPEG lossy compression attack, Gaussian white noise attack, mean filter attack and random discard attack.

[0032] As a preferred solution, the intensity factor of the JPEG lossy compression attack is 80; the mean filter size of the mean filter attack is 3×3; and the discard ratio of the random discard attack is 1%.

[0033] As a preferred solution, the watermark image after the attack is input into the watermark extraction module in AGWNet for training, including:

[0034] 3-1: Input the B' channel of the watermarked image after the attack into the convolution layer, and the convolution layer outputs the watermark carrier features after the attack with a size of b×64×64×16.

[0035] 3-3: Map the watermark carrier features after the attack to three new feature spaces: Query, Key, and Value. Perform attention convolution on the three new feature spaces. After attention convolution, Query and Key obtain outputs of size 1×64×64×4, while Value obtains outputs of size 1×64×64×16. Transpose the convolved Key and perform matrix multiplication with the convolved Query, and perform Softmax on the result to obtain the attention mask. Finally, perform matrix element multiplication of the output of Value after attention convolution with the attention mask to obtain the watermark carrier features.

[0036] 3-4: The watermark carrier features are spliced ​​on the feature channel through the attention residual connection layer, and the spliced ​​result is input into the convolution layer to obtain the extracted watermark image.

[0037] As a preferred solution, the cross entropy loss between the extracted watermark and the original watermark image is obtained, including:

[0038]

[0039] Among them, w is the original watermark, To extract the watermark, the size is M w ×Nw , M w 、N w are the image dimensions length and width, To obtain the cross entropy.

[0040] As a preferred solution, the overall loss is obtained based on the structural loss, visual perception loss and cross entropy loss, including:

[0041] Loss all =α1×Loss stru +α2×Loss eye +α3×Loss cross

[0042] Among them: α1, α2, α3 are hyperparameters for adjusting the weights of each loss function; Loss all is the overall loss, Loss stru is the structural loss, Loss eye is the visual perception loss, Loss cross is the cross entropy loss.

[0043] Beneficial effect: The present invention provides a self-encoded color image robust watermark processing method based on visual perception, which improves the robustness of attacks involved in the end-to-end training process while ensuring the quality of the watermarked image. It is also proved through experiments that the proposed method has good generalization performance for common signal processing attacks such as cropping and compression and signal processing attacks not involved in the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] Figure 1 The figure is a flow chart of the method of the present invention.

[0045] Figure 2 Figure 2 is the end-to-end training framework diagram of the proposed model.

[0046] Figure 3 This is a diagram showing the attack effect.

[0047] Figure 4 This is an embedded network framework diagram.

[0048] Figure 5 To extract the network framework diagram.

[0049] Figure 6 This is a visualization of the performance of the watermarking method (each group of figures from left to right are the original carrier, the watermarked carrier, the original watermark, and the extracted watermark).

[0050] Figure 7 This is the residual analysis of the original carrier and the corresponding watermarked image B channel (each group of pictures from left to right are the original carrier, the original carrier B channel, the watermarked carrier B channel, and the residual 10 times).

[0051] Figure 8 Extract watermark images and bit error probability values ​​(BER) for different models under different attacks. DETAILED DESCRIPTION

[0052] The present invention will be further described below in conjunction with specific embodiments.

[0053] like Figure 1 As shown, a self-encoding color image robust watermark processing method based on visual perception is proposed, which completes the watermark embedding and extraction of color images by constructing an end-to-end trained neural network, wherein the input of the neural network is a carrier image and a randomly generated binary watermark image, and the output of the neural network is a watermarked image. The neural network designed in the present invention is hereinafter referred to as AGWNet.

[0054] AGWNet consists of three modules: watermark embedding module, noise attack module and watermark extraction module, corresponding to Figure 2 Watermark embedding algorithm, watermark attack and watermark extraction algorithm.

[0055] The design of the watermark embedding module and the watermark extraction module solves the problem of long-distance feature loss in image reconstruction in the embedding and extraction stages of existing solutions.

[0056] The design of the AGWNet loss function in the present invention solves the problem that the existing schemes lack prior knowledge guidance for embedding watermarks, so that the watermark information cannot make good use of the characteristics of the carrier image for watermark embedding. The design of the watermark embedding, extraction module and loss function makes the quality of the watermark image generated by AGWNet better than the existing schemes.

[0057] The noise attack module is designed to improve the robustness of the watermark image. In the present invention, this module is added between the watermark embedding module and the watermark extraction module for network training, so that AGWNet can improve the robustness of the watermark image while ensuring the quality of the watermarked image. The following describes the construction method, loss function design and training process of the watermark embedding module, watermark extraction module and noise attack module in AGWNet in turn. The first three only describe the construction and design of AGWNet, and their role will be given in the description of the training process.

[0058] How AGWNet is built

[0059] (1) The network structure of the watermark embedding module is shown in Table 1 below, where b in the input and output size is the batch size, the second and third parameters describe the spatial size of the feature map, and the last parameter is the number of feature maps, that is, the number of feature channels. The first parameter in the convolution kernel size is the number of convolution kernels, and the last two parameters describe the convolution kernel size, and / means no convolution kernel.

[0060] Table 1 Detailed configuration of network parameters of watermark embedding module

[0061]

[0062] (2) The network structure of the watermark extraction module is shown in Table 2 below, where the meanings of the parameters are consistent with those described in (1).

[0063] Table 2 Detailed configuration of network parameters of watermark extraction module

[0064]

[0065] like Figure 3 As shown, (3) Noise attack module: This module is located between the watermark embedding module and the watermark extraction module, including JPEG lossy compression attack, Gaussian white noise attack, mean filter attack and random drop attack, where the intensity factor of JPEG compression attack is 80, the mean filter size is 3×3, and the drop ratio of Dropout is 1%.

[0066] AGWNet loss function design

[0067] Three loss functions are designed in AGWNet: structural loss function, visual perception loss function and cross entropy loss function. The first two are used in the watermark embedding module to calculate the loss between the carrier image and the watermarked image; the latter is used in the watermark extraction module to calculate the loss between the original input watermark and the extracted watermark.

[0068] The existing work does not consider the watermark embedding guided by prior knowledge, and only uses the convolutional network to fuse the watermark information with the carrier image features to obtain the watermark image. This makes the amount of watermark information embedded in different positions in the carrier and the embedding strength only allow the network to re-explore and learn without prior knowledge, so this watermark embedding method lacks interpretability. In order to make the watermark embedding method more invisible and interpretable, the present invention introduces prior knowledge to embed the watermark in the watermark embedding module by designing the structural loss function and the visual perception loss function, which is described in detail below.

[0069] The watermark embedding module of the present invention not only considers the structural measurement index SSIM between the watermarked image and the carrier image, but also uses the structural loss function; it also discusses the distribution of the chromaticity and texture of the carrier image based on the visual characteristics of the human eye, proposes a visual perception loss function, and guides the network to embed watermarks in the blue-green area and the area with rich texture, thereby further improving the quality of the watermarked image:

[0070] (1) Structural loss function calculation steps: SSIM is a commonly used indicator to measure the similarity between two images. The more similar the two images are, the closer the SSIM value is to 1. Compared with the mean square error (MSE), SSIM can better measure the structural similarity of images. At the same time, since the loss function of the neural network is to minimize the gradient optimization, 1 minus the original carrier image I and the watermarked image I is used. w The SSIM value is used as part of the loss function of the neural network of the watermark embedding module, and the calculation process is as follows:

[0071]

[0072] Among them, μ I and σ I are the mean and variance of the original carrier pixels, and are the mean and variance of the watermarked image respectively, c1 and c2 are balance constants to prevent the denominator from being zero.

[0073] (2) Visual perception loss function calculation steps: Based on the MSE, a specific penalty weight is calculated for each pixel. The penalty weight calculation rule is based on the visual characteristics of the human eye. The penalty weight consists of two parts, one is the chromaticity weight and the other is the texture weight:

[0074] (2-1) Chroma weight: The human eye is sensitive to changes in the red area and less sensitive to blue and green. Pixels with large Cr components should be modified less, thereby guiding the network to embed into areas with low red chroma. Since the YCbCr color space separates chroma and brightness, it is more in line with the visual characteristics of the human eye. Therefore, before calculating the chroma weight, the carrier image and the watermarked image need to be converted from the RGB color space to the YCbCr color space. The conversion formula complies with the ITU standard:

[0075] R=Min(Max(0,Round(Y+1.402×(C r -128))),255)

[0076] G=Min(Max(0,Round(Y-0.3441×(C b -128)-0.7141×(C r -128))),255)

[0077] B=Min(Max(0,Round(Y-1.772×(C b -128))),255)

[0078] In order to maintain consistent dimensions during the training phase, the Cr component needs to be normalized. Therefore, the chrominance penalty weight of the MSE loss is obtained by subtracting the Cr component normalization matrix of the original carrier from 1. The specific calculation method is:

[0079]

[0080] Among them, A is a matrix of all 1s, g is a normalization function used to constrain the value range of chromaticity information to 0 to 1, and I Cr is the Cr component of the original carrier.

[0081] (2-2) Texture weight: The human eye is sensitive to changes in monotonous texture areas, but not so sensitive to areas with rich textures. Therefore, it is necessary to calculate the texture degree for each pixel. Pixels with small texture degree values ​​indicate that they are less different from surrounding pixels and are more likely to be smooth areas. Modifications should be reduced to guide the network to embed in areas with rich textures. Rich texture means that the difference between the pixel value of the point and the surrounding pixel values ​​is large. Therefore, for each pixel, the difference between it and the surrounding pixels is calculated to represent its texture degree. The calculation process is similar to that of a sliding window. The specific calculation method is:

[0082]

[0083] Among them, I(i,j) is the pixel value of the image at position (i,j), k is the horizontal sliding distance, l is the vertical sliding distance, I(i+k,j+l) is the pixel value of the image at position (i+k,j+l), D is the sliding window width, the sliding window size is (2D+1)×(2D+1), in this method, the value of D is 1, and g is the normalization function.

[0084] The penalty weight of visual perception loss is the weighted sum of the above chromaticity weight and texture weight. The specific calculation method is:

[0085]

[0086] Among them, σ c is the ratio of chromaticity weight, σ t is the ratio of texture weight, which is 1 in the present invention. After obtaining the visual perception loss penalty weight, the visual perception loss proposed in the present invention can be expressed as:

[0087]

[0088] Among them, M c 、N c is the image size length and width, i, j is the original carrier image I and the watermarked image I w The pixel length and width.

[0089] (3) Cross-entropy loss function calculation steps: The task of the watermark extraction module is to accurately extract the binary watermark, which can be regarded as a binary classification problem. Cross-entropy is a commonly used loss function in classification problems. Therefore, cross-entropy is used as the loss function of the extraction network:

[0090]

[0091] Among them, w is the original watermark, To extract the watermark, the size is M w ×N w , M w 、N w are the image dimensions length and width, To obtain the cross entropy.

[0092] (4) The total loss of AGWNet is the weighted sum of the above three loss functions, which can be expressed as:

[0093] Loss all =α1×Loss stru +α2×Loss eye +α3×Loss cross

[0094] Among them, α1, α2, and α3 are hyperparameters for adjusting the weights of each loss function. The training of AGWNet is to adjust the AGWNet network parameters by reducing this total loss.

[0095] AGWNet training process

[0096] The training set used in the present invention is a 256×256 color image. Hereinafter, an arbitrarily selected color image is referred to as a carrier image. In order to train AGWNet to embed any binary watermark image and make the watermarked image more invisible and robust than the existing scheme, the embedded watermark is set to a randomly generated binary image of fixed size 32×32, which is referred to as the original watermark image.

[0097] Step 1: Read a carrier image from the training set and input it into AGWNet, that is, the watermark embedding module of AGWNet. In this process, the original watermark image is embedded into the carrier image to generate a watermarked image, and the structural loss and visual perception loss are calculated with the carrier image.

[0098] Step 2: Input the watermarked image into the noise attack module to generate the watermarked image after the attack;

[0099] Step 3: Input the attacked watermark image into the watermark extraction module, output the extracted watermark, and calculate the cross entropy loss between it and the original watermark image;

[0100] Step 4: Take the weighted sum of the losses in step 1 and step 3 to get the overall loss of AGWNet. By reducing this loss, the parameters in the network are trained.

[0101] Further, combining the above Table 1 with the following Figure 4 , explaining the specific process of watermark embedding in step 1:

[0102] 1-1: Among the R, G, and B channels of a color image, the modification of the B channel is not easily perceived by the human eye. Therefore, in the present invention, the B channel of the carrier image is extracted and input into the watermark embedding module, and the R and G channels of the carrier image are retained for subsequent synthesis of the watermarked image.

[0103] 1-2: Input the B channel into convolution layers 1 to 5 to obtain 63 feature maps of size 32×32, that is, the output size of convolution layer 5 in the table is b×32×32×63. Here, since it describes the processing process of a carrier image in AGWNet, b can be regarded as 1. The function of these five convolution layers is to extract the features of the carrier image.

[0104] 1-3: A 32×32 binary watermark image is randomly generated for this carrier image, and it is concatted with the feature map extracted in 1-2 on the feature channel to obtain an output of 1×32×32×64, which is reflected in the Concat layer in Table 1. Figure 4 The characteristics of the watermarked carrier after being spliced ​​with the original watermarked image can also be seen.

[0105] 1-4: The process of obtaining a watermarked image can be regarded as an image generation task. Here, the self-attention mechanism is introduced to help the network capture the long-distance dependent features in the watermark carrier features to better improve the quality of the watermarked image. The specific approach is: map the watermark carrier features to three new feature spaces: Query, Key and Value. Figure 4 In the figure, they are represented by green, red, and blue squares respectively. Perform attention convolution on the three new feature spaces, i.e., the attention convolution layers 1 to 3 in Table 1. After attention convolution, the output size of Query and Key is 1×32×32×8, while the output size of Value is 1×32×32×63. Transpose the convolved Key and perform matrix multiplication with the convolved Query, and perform Softmax on the result to obtain the attention mask. Concat the convolved Value with the original watermark image on the feature channel (to make the watermark information redundant enough), multiply the result with the attention mask by matrix elements, and multiply it by the embedding factor to control the watermark embedding strength.

[0106] 1-5: The result obtained in 1-4 is concatenated with the watermark carrier feature on the feature channel through the attention residual connection layer, and finally the weighted watermark carrier feature is obtained, whose size is 1×32×32×64.

[0107] 1-6: Use the deconvolution operation to reconstruct the weighted watermark carrier features into the watermarked B channel. Deconvolution 1 to 3 enlarges the spatial size of the feature map and reduces the number of feature channels. Three convolution layers 6 to 8 are interspersed to solve the problem of checkerboard artifacts caused by the deconvolution operation on the feature map. Finally, the watermarked B channel is output with an output size of 1×256×256×1.

[0108] 1-7: Splice the watermarked B channel with the R and G channels retained in 1-1 to restore the watermarked color image.

[0109] Furthermore, combined with Figure 3 , explaining the role and specific process of noise attack on the watermarked image in step 2:

[0110] 2-1: Before extracting the watermark from the watermarked image, add a signal processing attack to it to generate the watermarked image after the attack. By training AGWNet, it can also effectively extract the watermark from such an image, so as to improve the robustness of the watermark image generated by AGWNet to the trained attack. At the same time, signal processing attacks will have certain similarities. If the watermark image generated by AGWNet is robust to one attack, it will also be robust to another similar attack.

[0111] 2-2: The noise attack module includes JPEG lossy compression attack, Gaussian white noise attack, mean filter attack and random drop attack, where the strength factor of JPEG compression attack is 80, the mean filter size is 3×3, and the dropout ratio is 1%. The effect of the color image after being attacked by these four signal processing methods is shown in the figure below. Figure 3 As shown, for each carrier image in the training set, the noise attack module will randomly select an attack in each epoch;

[0112] 2-3: Input the watermarked image into the noise attack module and randomly select a signal processing attack to generate the watermarked image after the attack.

[0113] Further, combining the above Table 2 with the following Figure 5 , explaining the specific process of watermark extraction in step 3:

[0114] 3-1: In the watermark embedding module, the watermark is embedded in the B channel of the carrier image, so the B' channel of the input attacked watermarked image is input into the watermark extraction module.

[0115] 3-2: Input the B' channel into convolutional layers 1 to 3 to obtain 16 feature maps of size 64×64, that is, the output size of convolutional layer 3 in the table is b×64×64×16. Here, since it describes the processing process of a carrier image in AGWNet, b can be regarded as 1. The function of these three convolutional layers is to extract the features of the watermarked image after the attack, that is, Figure 5 Features of watermarked carriers.

[0116] 3-3: The process of extracting the watermark from the watermarked image can also be regarded as an image generation task. Since the self-attention mechanism is used to capture the long-distance dependent features in the feature map when embedding the watermark to guide the distribution of the watermark information in the feature map, the self-attention mechanism is also needed in the watermark extraction module to capture the long-distance dependencies in the watermark carrier features to improve the accuracy of the extracted watermark image generation. The specific approach is: map the watermark carrier features to three new feature spaces: Query, Key and Value. Figure 5 In the figure, they are represented by green, red, and blue squares, respectively. Attention convolution is performed on the three new feature spaces, i.e., the attention convolution layers 1 to 3 in Table 2. The output size of Query and Key after attention convolution is 1×64×64×4, while the output size of Value after attention convolution is 1×64×64×16. The transposed Key after convolution is matrix multiplied with the Query after convolution, and the result is Softmax to obtain the attention mask. Finally, the result of Value after attention convolution is matrix-element multiplied with the attention mask.

[0117] 3-4: The result obtained in 3-3 is connected through the attention residual layer and concatenated with the watermark carrier feature on the feature channel. The result is input into convolution layers 4 to 7 to finally obtain the extracted watermark image.

[0118] AGWNet Test Results

[0119] The watermark embedding network AGWNet designed in the present invention solves the problem that it is difficult to simultaneously obtain watermark invisibility and robustness in end-to-end training in existing solutions. Therefore, the following mainly analyzes the invisibility and robustness results of the watermark image generated by AGWNet.

[0120] The following shows the invisibility results and analysis:

[0121] (1) The trained AGWNet is tested on two test datasets. For the ImageNet dataset and the BossBase dataset, the average SSIM values ​​between the original carrier image and the watermarked image are 0.9918 and 0.9905, respectively. In the absence of attack, the average BER values ​​between the original watermark and the extracted watermark are 0.097% and 0.196%, respectively. Figure 6 The original carrier images and the corresponding watermarked images, the original watermarked images and the corresponding extracted watermarked images on two test datasets are given, and the SSIM values ​​and BER values ​​are given. The carrier images of groups (a) and (b) are from the ImageNet dataset, and the carrier images of groups (c) and (d) are from the BossBase dataset. In each group of images, the first one is the original carrier, the second one is the watermarked image, the third one is the original watermarked image, and the fourth one is the extracted watermarked image. It can be seen that the watermark in the watermarked image is invisible, whether in the in-library test set or in the cross-library test set, and the meaningful binary watermark information can be accurately extracted without attack.

[0122] (2): This method fully considers the visual characteristics of the human eye during the algorithm design stage. Therefore, a difference is made between the watermarked image and the original carrier to further analyze the modification of the watermarked image. Figure 7 Shows the above Figure 6 Four groups of residual analysis results of the original carrier and watermarked image are given. Each row from left to right is the residual map of the original carrier, the original carrier B channel, the watermarked image B channel, the original carrier B channel and the watermarked image B channel (magnified 10 times for easy observation). It is not difficult to find out that the difference between the watermarked image and the original carrier is concentrated in the area with rich texture, and the changes in the cold color areas such as blue-green are more than the warm color areas such as red, which is in line with the original intention of the algorithm design. From the two aspects of numerical and visual effects, the superiority of AGWNet in invisibility is verified.

[0123] The following shows the robustness results and analysis:

[0124] (1) In order to show the performance difference between this method and the existing advanced robust watermarking models, three advanced algorithms are selected as control models, namely the models of Zhu et al., Luo et al., and Zhang et al. The three models have open-sourced their codes and models on the GitHub code management platform. For fairness, the models of Zhu et al., Luo et al., and Zhang et al. are retrained using the ImageNet training dataset, with the carrier size set to 256×256 and the embedded watermark capacity set to 1024 bits. The hyperparameters involved in the training follow the instructions in their papers. In order to accurately evaluate the performance of the robust boosting model, the following Table 3 shows the performance results of the robust boosting model proposed in this paper and the models of Zhu et al., Luo et al., and Zhang et al. on four attacks (attack types considered in the training stage of this method). The BER value of the watermarked image extracted in the table is the average value under the different attack intensities shown.

[0125] Table 3 Performance comparison of different models (attack types considered in training)

[0126]

[0127] (2) In order to demonstrate the generalization performance of the model proposed in this method when facing attacks, it was evaluated from two aspects: a. For attacks that appeared in the training phase, the model was tested with attack intensities different from those in the training phase; b. Whether the model can resist some unknown types of attacks that did not appear in the test phase was evaluated.

[0128] For the generalization evaluation of the same attack with different attack strengths, the following Tables 4 and 5 show the BER values ​​of the watermark extracted by the proposed model and three comparison models under the same attack type with different attack strengths. It can be seen that since the model proposed by this method adds JPEG with a compression factor of 80, a mean filter with a convolution kernel size of 3×3, a Gaussian white noise with a standard deviation of 1, and a Dropout with a random discard ratio of 1% in the training phase, the robust performance is relatively better when facing these four attacks in terms of strength in the test phase.

[0129] Table 4 BER values ​​of different models facing different strength JPEG and mean filter attacks

[0130]

[0131] Table 4 BER values ​​of different models facing different strengths of Drop and Gaussian white noise attacks

[0132]

[0133] For the generalization evaluation of attacks that have not appeared in the training phase, the following Table 6 shows the performance results of the proposed model and three comparison models facing attacks that have not appeared in the training phase (Poisson noise, Gaussian blur). From the data in the table, it can be seen that the robust boosting model proposed by this method still maintains good performance even when facing attacks that have not appeared in the training phase, showing the good generalization performance of the proposed model for unknown signal processing attacks.

[0134] Table 5 Performance comparison of different models (attack types not considered in training)

[0135]

[0136]

[0137] In order to more directly feel the superiority of the model proposed by this method in watermark extraction under various attacks compared with the three comparison models, Figure 8 Different models are shown to extract watermark images and BER values ​​under different attacks.

[0138] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principle of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A self-encoding color image robust watermark processing method based on visual perception, characterized by: The steps include: The watermark embedding module in AGWNet is trained using the carrier image training set and the original watermark image training set to obtain the structural loss and visual perception loss of the watermarked image and the carrier image. Input the watermarked image into the noise attack module in AGWNet to generate the watermarked image after the attack; The attacked watermark image is input into the watermark extraction module in AGWNet for training, and the cross entropy loss between the extracted watermark and the original watermark image is obtained; The overall loss is calculated based on the structural loss, visual perception loss and cross entropy loss. When the overall loss is less than the threshold in the iteration, the trained AGWNet is obtained. Using the trained AGWNet, watermark embedding or extraction is completed; The formula for calculating structural loss is as follows: Among them, I is the original carrier image, I w is the watermarked image, μ I and σ I are the mean and variance of the original carrier pixels, and are the mean and variance of the watermarked image, respectively. c1 and c2 are equilibrium constants to prevent the denominator from being zero. The visual perception loss calculation formula is as follows: Among them, M eye is the penalty weight, M c 、N c is the image size length and width, i, j is the original carrier image I and the watermarked image I w The length and width of the pixel; Among them, σ c is the ratio of chromaticity weight, σ t is the ratio of texture weight, M color is the chromaticity weight, M texture is the texture weight.

2. According to claim 1, a method for robust watermarking of self-encoded color images based on visual perception is characterized in that: The watermark embedding module in AGWNet is trained using the carrier image training set and the original watermark image training set, including: 1-1: Extract the B channel of the carrier image and input it into the watermark embedding module, and retain the R and G channels of the carrier image; 1-2: Input the B channel into the convolution layer, and the convolution layer outputs a feature map of size b×32×32×63; 1-3: Randomly generate a 32×32 binary watermark image, concat the binary watermark image with the feature map on the feature channel, and the concat layer outputs 1×32×32×64 watermark carrier features; 1-4: Map the watermark carrier features to three new feature spaces: Query, Key, and Value. Perform attention convolution on the three new feature spaces respectively. After attention convolution, Query and Key obtain outputs of size 1×32×32×8, while Value obtains outputs of size 1×32×32×63 after attention convolution. Transpose the convolved Key and perform matrix multiplication with the convolved Query, and perform Softmax on the result to obtain the attention mask. Concat the convolved Value with the original watermark image on the feature channel, multiply the result obtained by Concat with the attention mask by matrix elements, and multiply it by the embedding factor to obtain the watermark carrier features. 1-5: The watermark carrier features are concatenated through the attention residual connection layer to obtain weighted watermark carrier features with a size of 1×32×32×64; 1-6: Use deconvolution and convolution layers to reconstruct the weighted watermark carrier features, and the output size is 1×256×256×1 including the watermark B channel; 1-7: Splice the watermarked B channel with the R and G channels of the carrier image to restore the watermarked color image.

3. The method for robust watermarking of self-encoded color images based on visual perception according to claim 1, characterized in that: The step of inputting the watermarked image into the noise attack module in AGWNet to generate the watermarked image after the attack includes: A signal processing attack is randomly selected for the watermarked image to generate an attacked watermarked image; the signal processing attack includes at least one of the following: JPEG lossy compression attack, Gaussian white noise attack, mean filter attack and random discard attack.

4. The method for robust watermarking of self-encoded color images based on visual perception according to claim 3 is characterized in that: The intensity factor of the JPEG lossy compression attack is 80; the mean filter size of the mean filter attack is 3×3; and the discard ratio of the random discard attack is 1%.

5. The method for robust watermarking of self-encoded color images based on visual perception according to claim 1, characterized in that: The watermark image after the attack is input into the watermark extraction module in AGWNet for training, including: 3-1: Input the B' channel of the watermarked image after the attack into the convolution layer, and the convolution layer outputs the watermarked carrier features after the attack with a size of b×64×64×16; 3-3: Map the watermark carrier features after the attack to three new feature spaces: Query, Key, and Value. Perform attention convolution on the three new feature spaces respectively. After attention convolution, Query and Key obtain outputs of size 1×64×64×4, while Value obtains output of size 1×64×64×16. Transpose the convolved Key and perform matrix multiplication with the convolved Query, and perform Softmax on the result to obtain the attention mask. Finally, perform matrix element multiplication of the output of Value after attention convolution with the attention mask to obtain the watermark carrier features. 3-4: The watermark carrier features are spliced ​​on the feature channel through the attention residual connection layer, and the spliced ​​result is input into the convolution layer to obtain the extracted watermark image.

6. The method for robust watermarking of self-encoded color images based on visual perception according to claim 1, characterized in that: Obtain the cross entropy loss of the extracted watermark and the original watermark image, including: Among them, w is the original watermark, To extract the watermark, the size is M w ×N w , M w 、N w are the image dimensions length and width, To obtain the cross entropy.

7. The method for robust watermarking of self-encoded color images based on visual perception according to claim 1, characterized in that: The overall loss is obtained based on the structural loss, visual perception loss and cross entropy loss, including: Loss all =α1×Loss stru +α2×Loss eye +α3×Loss cross Among them: α1, α2, α3 are hyperparameters for adjusting the weights of each loss function; Loss all is the overall loss, Loss stru is the structural loss, Loss eye is the visual perception loss, Loss cross is the cross entropy loss.

Images