CNN-based HVS-JND spectral residual network robust watermarking method

Abstract

The present invention discloses a CNN-based HVS-JND spectral residual network robust watermarking method. The method comprises the following steps: inputting a carrier image and an original watermark into a watermark encoder to generate a coded image embedded with the watermark; inputting the coded image and the carrier image into an HVS-JND perceptual classification discriminator in two paths to determine whether the coded image embedded with the watermark is a lossy image or a lossless image; if the image is lossy, calculating a loss function and updating the parameters of the watermark encoder until a lossless coded image is obtained and inputting the image into a watermark decoder to obtain a decoded watermark; calculating a loss function based on the original watermark and the decoded watermark, and updating the parameters of the watermark encoder, the HVS-JND perceptual classification discriminator and the decoder; the method designed by the present invention enables the encrypted image to have high invisibility and good visual quality; and by introducing a spectral residual network, the transparency and robustness of the watermark are improved, which has practical significance and good application prospects.

Description

Technical Field

[0001] The present invention relates to the field of information hiding, and in particular to a CNN-based HVS-JND spectral residual network robust watermarking method. Background Art

[0002] Most existing watermark embedding and extraction methods use convolutional neural networks (CNNs). They encode the watermark and embed it into the carrier image, effectively extracting it from noisy images. Due to the image characteristics of convolutional neural networks, they achieve excellent performance. For example, the convolutional neural network-based watermarking model HiDDeN possesses strong inductive biases, including translation equivariance and locality. While its well-designed loss function enables the CNN to learn good features, achieving success in both image quality and robustness, subsequent research has largely continued with this loss function design. While current deep learning-based watermarking methods can achieve high levels of invisibility in objective evaluations, they fail to consider the minimum threshold of the human visual system to detect image changes. Traditional convolutional neural networks, most of which use VGGNet as the backbone, have convolution kernels of only 3*3 size, enabling only local feature extraction and poor connection between local and global features. Watermarking images is a visual task that requires close context. Traditional CNN operators lack local-to-global semantic connections, making local embedded watermarks easily corrupted and unable to achieve global embedding of the watermarked image. Summary of the Invention

[0003] The purpose of the present invention is to provide a CNN-based HVS-JND spectral residual network robust watermarking method. It adopts the traditional CNN architecture, designs a loss function based on HVS-JND, adds a spectral residual network, and utilizes its better loss function design to make the encrypted image have high invisibility and good visual quality; and by introducing the spectral residual network, the transparency and robustness of the watermark are improved.

[0004] To achieve the above functions, the present invention designs a CNN-based HVS-JND spectral residual network robust watermarking method. For the carrier image and the original watermark, the following steps S1 to S5 are performed to complete the watermark embedding and extraction:

[0005] Step S1: Input the carrier image and the original watermark into the watermark encoder composed of the CNN with added spectral residual to generate a coded image embedded with the watermark;

[0006] Step S2: Input the watermarked coded image and the carrier image into the HVS-JND perceptual classification discriminator to determine whether the input watermarked coded image is a lossless coded image. If the watermarked coded image is not a lossless coded image, the loss function of the coded image and the carrier image is calculated, and the watermark encoder parameters are updated using the stochastic gradient descent method until a lossless coded image is obtained. If the watermarked coded image is a lossless coded image, the next step is performed.

[0007] Step S3: Input the lossless encoded image into the watermark decoder composed of CNN to extract the decoded watermark;

[0008] Step S4: Based on the decoded watermark obtained in step S3 and the original watermark in step S1, the loss function of the original watermark and the decoded watermark is calculated, and the parameters of the watermark decoder are updated using the stochastic gradient descent method;

[0009] Step S5: Determine whether the model achieves the expected effect. If not, return to step S1 and retrain the entire model. If it achieves the expected effect, retain all model parameters for watermark embedding and extraction.

[0010] As a preferred technical solution of the present invention, the process of generating the coded image embedded with the watermark in step S1 is as follows: the original watermark M is divided into m*n non-overlapping image blocks, where m and n represent the number of rows and columns into which the original watermark is divided, respectively; at the same time, image blocks MP to be embedded with the watermark are generated in the order of the image blocks; each image block is subjected to discrete wavelet transform to generate a watermark embedding subband ML; all watermark embedding subbands are added together to obtain the watermark to be embedded IM; the watermark to be embedded IM and the carrier image I are processed by a spectral residual CNN encoder to generate a watermark mask MASK which is added to the carrier image I to obtain the coded image IW. The above process is expressed as follows:

[0011] IM=∑ML

[0012] MASK=CNN(IM)

[0013] IW=I+α×MASK

[0014] Where ∑ represents the summation symbol, CNN represents the spectral residual CNN operation, and α represents the embedding strength factor.

[0015] As a preferred technical solution of the present invention: in step S2, the carrier image I and the corresponding coded image IW are input into the HVS-JND perceptual classification discriminator in two ways. The HVS-JND perceptual classification discriminator consists of an input part, a binary classifier, and an output part. The input part includes the carrier image I and the coded image IW. The binary classifier is used to identify whether the coded image IW is lossy relative to the carrier image I. Based on the identification result of the binary classifier, the sigmoid function is used to determine whether it is lossy. When the output is greater than 0.5, it is determined to be a coded image with perceptual loss; otherwise, it is determined to be a coded image without perceptual loss IW′. The loss LossC is calculated based on the carrier image I and the coded image IW, and the parameters of the watermark encoder are updated using the stochastic gradient descent method until a coded image without perceptual loss is obtained.

[0016] LossC is as follows:

[0017] LossC=E(I,IW)

[0018] Among them, LossC is the structural similarity loss between the carrier image I and the encoded image IW; E represents the spectral residual CNN encoder.

[0019] As a preferred technical solution of the present invention: in step S3, the lossless coded image IW′ is input into a CNN-based watermark decoder, which includes several CNN blocks and adopts global average pooling after the last CNN block, and finally outputs the extracted decoded watermark through a fully connected layer; the lossless coded image IW′ is input into a watermark decoder D, and the decoded watermark frequency band is output and subjected to inverse discrete wavelet transform to obtain a decoded watermark M′ and calculate the loss LossM, as follows:

[0020] LossM=D(M,M′)

[0021] Among them, LossM is the structural similarity loss between the original watermark M and the decoded watermark M′.

[0022] As a preferred technical solution of the present invention: the process of using the stochastic gradient descent method to update the watermark encoder, HVS-JND perceptual classification discriminator, and decoder parameters in step S4 is as follows:

[0023] Calculate the loss LossM between the original watermark M and the decoded watermark M′, and the loss LossJND between the carrier image I and the lossless encoded image IW′ respectively. The above losses are weighted and summed to calculate the total loss, which is used to update the encoder, HVS-JND perceptual classification discriminator, and decoder parameters.

[0024] LossJND=D(IM′,I)

[0025] Loss=βLossC+γLossM+δLossJND

[0026] Among them, β, γ, and δ represent the weights of LossC, LossM, and LossJND loss functions respectively.

[0027] Beneficial Effects: The CNN-based HVS-JND spectral residual network robust watermarking method designed by the present invention makes the image containing the watermark highly invisible while ensuring that the watermark can be embedded in the entire carrier image. Compared with the existing technology, the specific advantages of the present invention include:

[0028] 1. Propose a loss function based on HVS-JND, which can achieve high visual quality of the carrier image while having better invisibility;

[0029] 2. A new convolutional neural network with a spectral residual network is proposed. During the watermark embedding process, CNN maps the extracted watermark features to the spectral domain to form global features, achieving local to global feature conversion and better robustness. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] Figure 1 Flowchart of a CNN-based HVS-JND spectral residual network robust watermarking method according to an embodiment of the present invention;

[0031] Figure 2 is a flow chart of a watermark encoder provided according to an embodiment of the present invention. DETAILED DESCRIPTION

[0032] The present invention will be further described below in conjunction with the accompanying drawings. The following embodiments are only used to more clearly illustrate the technical solutions of the present invention and are not intended to limit the scope of protection of the present invention.

[0033] Reference Figure 1 The CNN-based HVS-JND spectral residual network robust watermarking method provided in the embodiment of the present invention performs the following steps S1 to S5 for the carrier image and the original watermark to complete the embedding and extraction of the watermark:

[0034] Step S1: Input the carrier image and the original watermark into the watermark encoder composed of a CNN with added spectral residual to generate a coded image embedded with the watermark;

[0035] Reference Figure 2The process of generating the coded image embedded with the watermark in step S1 is as follows: the original watermark M is divided into m*n (m, n represent the number of rows and columns into which the original watermark is divided, respectively) non-overlapping image blocks, and the image blocks MP to be embedded with the watermark are generated in the order of the image blocks, and each image block is subjected to discrete wavelet transform to generate a watermark embedding subband ML, all watermark embedding subbands are added together to obtain the watermark to be embedded IM, the watermark to be embedded IM and the carrier image I are processed by the spectral residual CNN encoder to generate a watermark mask MASK and add it to the carrier image I to obtain the coded image IW. The expression of the above process is as follows:

[0036] IM=∑ML

[0037] MASK=CNN(IM)

[0038] IW=I+α×MASK

[0039] Where ∑ represents the summation symbol, CNN represents the spectral residual CNN operation, and α represents the embedding strength factor.

[0040] Step S2: The watermarked coded image and the carrier image are input into the HVS-JND perceptual classification discriminator in two ways to discriminate whether the input watermarked coded image is a perceptually lossy coded image or a perceptually lossless coded image. If the watermarked coded image is a perceptually lossy coded image, the loss function is calculated and the parameters of the watermark encoder are updated using the stochastic gradient descent method until a perceptually lossless coded image is obtained.

[0041] In step S2, the carrier image I and the corresponding coded image IW are input into the HVS-JND perceptual classification discriminator in two ways. The HVS-JND perceptual classification discriminator consists of an input part, a binary classifier, and an output part. The input part includes the carrier image I and the coded image IW. The binary classifier is used to identify whether the coded image IW is lossy relative to the carrier image I. Based on the identification result of the binary classifier, the sigmoid function is used to determine whether it is lossy. When the output is greater than 0.5, it is determined to be a coded image with perceptual loss. Otherwise, it is determined to be a coded image without perceptual loss IW′. The loss LossC is calculated based on the carrier image I and the coded image IW, and the parameters of the watermark encoder are updated using the stochastic gradient descent method until a coded image with perceptual loss is obtained.

[0042] LossC is as follows:

[0043] LossC=E(I,IW)

[0044] Where E represents the spectral residual CNN encoder; LossC is the SSIM (structural similarity) loss between the carrier image I and the encoded image IW.

[0045] The SSIM (structural similarity) loss for any pixel p in the image is defined as:

[0046]

[0047] Where, subscripts X and Y represent the processed image and the real image respectively, μ X 、μ Y are the mean values ​​of pixels on the processed image X and the real image Y, respectively. are the variances of pixels on the processed image X and the real image Y, σ X , σ Y are the covariance of the pixels on the processed image X and the real image Y, respectively. C1 and C2 are parameters 1 and 2. In order to avoid the denominator being 0 and maintain stability, the calculation formula is C1 = (k1L) 2 and C2=(k2L) 2 , where L is the grayscale variation range of the image, which is generally 255. If normalized to [0,1], it is 1, and if it is [-1,1], it is 2. k1 and k2 are constant coefficients 1 and 2, which can be adjusted according to experimental results.

[0048] Step S3: inputting the lossless coded image into a watermark decoder composed of CNN to extract a decoded watermark from the lossless coded image;

[0049] In step S3, the lossless coded image IW′ is input into the CNN-based watermark decoder. The CNN-based watermark decoder includes several CNN blocks, and adopts global average pooling after the last CNN block. Finally, the decoded watermark extracted is output through the fully connected layer. The lossless coded image IW′ is input into the watermark decoder D, and the decoded watermark frequency band is output and subjected to inverse discrete wavelet transform to obtain the decoded watermark M′ and calculate the loss LossM, as follows:

[0050] LossM=D(M,M′)

[0051] Where LossM is the SSIM (structural similarity) loss between the original watermark M and the decoded watermark M′.

[0052] Step S4: Calculate the loss function based on the decoded watermark obtained in step S3 and the original watermark in step S1, and use the stochastic gradient descent method to update the parameters of the watermark encoder, HVS-JND perceptual classification discriminator, and watermark decoder;

[0053] The process of updating the watermark encoder, HVS-JND perceptual classification discriminator, and decoder parameters using the stochastic gradient descent method in step S4 is as follows:

[0054] Calculate the loss LossM between the original watermark M and the decoded watermark M′, and the loss LossJND between the carrier image I and the lossless encoded image IW′ respectively. The above losses are weighted and summed to calculate the total loss, which is used to update the encoder, HVS-JND perceptual classification discriminator, and decoder parameters.

[0055] LossJND=D(IM′,I)

[0056] Loss=βLossC+γLossM+δLossJND

[0057] Among them, β, γ, and δ represent the weights of LossC, LossM, and LossJND loss functions respectively.

[0058] Step S5: Repeat steps S1 to S4 to train the watermark encoder, HVS-JND perceptual classification discriminator, and watermark decoder until a preset number of training times is reached, and the trained watermark encoder, HVS-JND perceptual classification discriminator, and watermark decoder are obtained, and the watermark encoder and watermark decoder are retained for watermark embedding and extraction, respectively.

[0059] In applications, a watermark encoder embeds a watermark into a carrier image, generating a watermarked encoded image that can be published and used. When copyright verification is required, the image to be verified is input into a watermark decoder, which outputs the extracted watermark. If the bit error rate between the extracted watermark and the original watermark is less than a preset threshold, copyright ownership is confirmed.

[0060] The embodiments of the present invention are described in detail above with reference to the accompanying drawings. However, the present invention is not limited to the above embodiments. Various changes can be made within the scope of knowledge possessed by ordinary technicians in this field without departing from the spirit of the present invention.

Claims

1. The CNN-based HVS-JND spectral residual network robust watermarking method is characterized by: For the carrier image and the original watermark, perform the following steps S1 to S5 to complete the embedding and extraction of the watermark: Step S1: Input the carrier image and the original watermark into the watermark encoder composed of the CNN with added spectral residual to generate a coded image embedded with the watermark; Step S2: Input the watermarked coded image and the carrier image into the HVS-JND perceptual classification discriminator to determine whether the input watermarked coded image is a lossless coded image. If the watermarked coded image is not a lossless coded image, the loss function of the coded image and the carrier image is calculated, and the watermark encoder parameters are updated using the stochastic gradient descent method until a lossless coded image is obtained. If the watermarked coded image is a lossless coded image, the next step is performed. Step S3: Input the lossless encoded image into the watermark decoder composed of CNN to extract the decoded watermark; Step S4: Based on the decoded watermark obtained in step S3 and the original watermark in step S1, the loss function of the original watermark and the decoded watermark is calculated, and the parameters of the watermark decoder are updated using the stochastic gradient descent method; Step S5: Determine whether the model achieves the expected effect. If not, return to step S1 and retrain the entire model. If it achieves the expected effect, retain all model parameters for watermark embedding and extraction.

2. The CNN-based HVS-JND spectral residual network robust watermarking method according to claim 1 is characterized in that The process of generating the coded image embedded with the watermark in step S1 is as follows: the original watermark M is divided into m*n non-overlapping image blocks, where m and n represent the number of rows and columns into which the original watermark is divided, respectively; at the same time, the image blocks MP to be embedded with the watermark are generated in the order of the image blocks; each image block is subjected to discrete wavelet transform to generate a watermark embedding subband ML; all watermark embedding subbands are added together to obtain the watermark to be embedded IM; the watermark to be embedded IM and the carrier image I are processed by the spectral residual CNN encoder to generate a watermark mask MASK which is added to the carrier image I to obtain the coded image IW. The expression of the above process is as follows: IM=ΣML MASK=CNN(IM) IW=I+α×MASK Where ∑ represents the summation symbol, CNN represents the spectral residual CNN operation, and α represents the embedding strength factor.

3. The CNN-based HVS-JND spectral residual network robust watermarking method according to claim 2 is characterized in that: In step S2, the carrier image I and the corresponding coded image IW are input into the HVS-JND perceptual classification discriminator in two ways. The HVS-JND perceptual classification discriminator consists of an input part, a binary classifier, and an output part. The input part includes the carrier image I and the coded image IW. The binary classifier is used to identify whether the coded image IW is lossy relative to the carrier image I. Based on the identification result of the binary classifier, the sigmoid function is used to determine whether it is lossy. When the output is greater than 0.5, it is determined to be a coded image with perceptual loss. Otherwise, it is determined to be a coded image without perceptual loss IW′. The loss LossC is calculated based on the carrier image I and the coded image IW, and the parameters of the watermark encoder are updated using the stochastic gradient descent method until a coded image with perceptual loss is obtained. LossC is as follows: LossC=E(1,IW) Among them, LossC is the structural similarity loss between the carrier image I and the encoded image IW; E represents the spectral residual CNN encoder.

4. The CNN-based HVS-JND spectral residual network robust watermarking method according to claim 3 is characterized in that In step S3, the lossless coded image IW′ is input into the CNN-based watermark decoder. The CNN-based watermark decoder includes several CNN blocks, and adopts global average pooling after the last CNN block. Finally, the decoded watermark extracted is output through the fully connected layer. The lossless coded image IW′ is input into the watermark decoder D, and the decoded watermark frequency band is output and subjected to inverse discrete wavelet transform to obtain the decoded watermark M′ and calculate the loss LossM, as follows: LossM=D(M,M′) Among them, LossM is the structural similarity loss between the original watermark M and the decoded watermark M′.

5. The CNN-based HVS-JND spectral residual network robust watermarking method according to claim 4 is characterized in that: The process of updating the watermark encoder, HVS-JND perceptual classification discriminator, and decoder parameters using the stochastic gradient descent method in step S4 is as follows: Calculate the loss LossM between the original watermark M and the decoded watermark M′, and the loss LossJND between the carrier image I and the lossless encoded image IW′ respectively. The above losses are weighted and summed to calculate the total loss, which is used to update the encoder, HVS-JND perceptual classification discriminator, and decoder parameters. LossJND=D(IM',I) Loss=βLossC+γLossM+δLossJND Among them, β, γ, and δ represent the weights of LossC, LossM, and LossJND loss functions respectively.

Images