A method for generating an invisible blind watermark resistant to deep neural network recognition

Abstract

The application discloses a kind of invisible blind watermark generation methods of anti deep neural network identification, comprising, using encoder to embed watermark information into carrier image;Using target classifier, guide encoder to embed watermark information into carrier image to generate adversarial watermark image in the confrontation feature area;Using noise layer to add noise to adversarial watermark image to generate noise adversarial watermark image;Using decoder to recover watermark information from noise adversarial watermark image;Using discriminator to carry out confrontation training to make noise adversarial watermark image of recovered watermark information more realistic;The present application can provide double image copyright protection for private image based on digital watermark and adversarial sample.

Description

TECHNICAL FIELD

[0001] The application belongs to the field of image blind watermarking, and particularly relates to an invisible blind watermark generation method resisting deep neural network identification. BACKGROUND

[0002] With the development of Internet technology, people have entered an unprecedented digital era, and information exchange has become more convenient. Digital images are important information transmission media in people's lives and are widely used in information transmission and sharing. However, in the process of information transmission, relevant contents may be illegally copied, tampered with, and stolen, which causes great damage to the rights and interests of authors. With the wide application of digital media in daily life, it is increasingly important to study the copyright protection of images. Digital watermarking is an important research direction in the field of information security and plays an important role in solving image copyright protection and identifying the authenticity of works. According to the detection process, watermarking can be divided into blind watermarking and non-blind watermarking. Non-blind watermarking needs the relevant information of the original host when extracting and detecting watermark information; blind watermarking does not need the participation of the original host when extracting and detecting, and is more practical and has application prospects. Watermarking can supervise the use of works, but it cannot directly prevent illegal acquisition of images.

[0003] In recent years, many deep neural network-based technologies have been used to identify and classify image data. This has raised concerns about the security of private images, and there are application software that can analyze, search, and obtain private images posted by users on social media websites based on deep neural networks.

[0004] Szegedy et al. found that by adding some carefully designed tiny perturbations to the original image, the deep neural network can produce false output, and such an image with added perturbations is called an adversarial sample. This characteristic of adversarial samples can be used to prevent malicious entities from identifying user private images through deep neural networks.

[0005] Currently, the copyright protection measures for private images in the field of artificial intelligence security are not good, especially after many deep neural network-based technologies have been used to identify and classify image data, a method for providing double image copyright protection for private images based on digital watermarking and adversarial samples is proposed. SUMMARY

[0006] In view of the deficiencies of the prior art, the application provides an invisible blind watermark generation method resisting deep neural network identification, which solves the above problems.

[0007] To achieve the above purpose, the application is implemented by the following technical scheme: an invisible blind watermark generation method resisting deep neural network identification, comprising the following steps:

[0008] Embedding watermark information into a carrier image by using an encoder;

[0009] Using a target classifier, guiding the encoder to embed watermark information into the carrier image in the adversarial feature area, to generate an adversarial watermark image;

[0010] Using a noise layer to add noise to the adversarial watermark image to generate a noise adversarial watermark image;

[0011] Using a decoder to recover watermark information from the noise adversarial watermark image;

[0012] Using a discriminator to perform adversarial training to make the noise adversarial watermark image with recovered watermark information more realistic.

[0013] On the basis of the above technical solutions, the application further provides the following optional technical solutions:

[0014] Further technical solutions: the network structure of the encoder introduces a CBAM attention mechanism module, and the specific steps of embedding watermark information into the carrier image based on the encoder are:

[0015] The carrier image is gradually down-sampled to 1 / 16 of the original image size by three CBAM modules, and then a CBAM module is used to extract image global features with a size of 1 / 64, and the features of the previous step are channel spliced to form a new bottommost feature map, wherein, in the up-sampling layer, the image size is gradually recovered by transposed convolution;

[0016] The watermark information with a length of L is expanded and reshaped in size, and is sent into multiple CBAM modules for up-sampling to extract features of watermark images with different sizes, and the watermark features with the same size, down-sampling features and up-sampling features are channel spliced to recover image details and embed watermark information, and finally enter a convolution layer and a tanh activation function to output the final encoded image.

[0017] Further technical solutions: the target classifier is a pre-trained target classifier, which uses a classification loss L adv Update parameters θ en to perform adversarial attack training:

[0018] L adv =f(y|I en )-max{f(i|I en ):i≠y}

[0019] Wherein, y is the true label of the carrier image, f(i|I en ) represents the confidence of the classifier f predicting the input I en as i.

[0020] Further technical solutions: the specific steps of the decoder for recovering the watermark information from the noise-attacked noise adversarial watermark image output from the noise layer are:

[0021] The decoder receives the noise-attacked noise adversarial watermark image output from the noise layer, extracts features through one convolutional layer and three CBAM modules, then performs down-sampling through two CBAM modules, and finally reshapes the feature size and converts multiple channels to one channel through two convolutional layers to recover the watermark information embedded in the noise-attacked noise adversarial watermark image.

[0022] Further technical solutions: the specific steps of the discriminator for making the noise-attacked noise adversarial watermark image more realistic through adversarial training are:

[0023] The input is first down-sampled through three convolutional layers, and then passes through two branches, namely a global branch and a patch branch;

[0024] The discriminator obtains the carrier image and the noise-attacked noise adversarial watermark image and distinguishes them, while the encoder cheats the classification result of the discriminator;

[0025] The global branch passes through five convolutional layers to obtain a binary output to determine whether the entire image is encoded; the patch branch passes through two convolutional layers to obtain a 32x32 binary output to focus on local texture details.

[0026] Further technical solutions: the training target of the entire anti-neural network recognition blind watermark network is to minimize the loss:

[0027] Loss = λ En L En + λ adv L adv + λ D L D + λ A L A

[0028] Where λ En , λ adv , λ D , λ A are weight parameters controlling the importance of each target, L D is the minimum original watermark information, L adv is the classification loss, L A is the loss function used in the training stage.

[0029] Further technical solutions: the training target of the encoder is to make the encoded image and the carrier image visually similar, therefore, the distance between the encoded image and the carrier image is minimized using the loss function L en to update the parameters θ en :

[0030] L en =MSE(I co ,I en ) = MSE(I co ,En(I co ,M;θ en ))

[0031] Among them, I co Represents a carrier image, I en This represents the encoded image, MSE represents the mean square error, and En represents the encoder function.

[0032] Further technical solution: The training objective of the decoder is to achieve the result through the loss function L. D Update parameter θ D To minimize the original watermark information L D The distance between the watermark information M' and the initial watermark information after decoding:

[0033] L D =MSE(M,M')=MSE(M,D(I no ;θ D ))

[0034] Among them, I no This represents an anti-watermark image containing noise.

[0035] Further technical solutions: using argmin En max A L GAN To train the encoder and discriminator simultaneously:

[0036]

[0037] Where A is the discriminator function;

[0038] Using loss function L during the training phase A :

[0039] L A =L P_GAN +L G_GAN

[0040] Among them, L P_GAN and L G_GAN These are the two branches in the discriminator in the first L. A The loss of the formula.

[0041] Beneficial effects

[0042] This invention provides an invisible blind watermark generation method resistant to deep neural network recognition, which has the following advantages compared with the prior art:

[0043] 1. By directly embedding the watermark into the more antagonistic feature area, the generated image can both effectively attack the deep neural network and extract high-precision watermark information from it;

[0044] 2. By one-step generation method, the redundancy of watermark embedding is reduced, thereby improving the visual quality of the generated image, and the carrier image and watermark binary sequence can be given, the watermark information is embedded into the carrier image, so that the generated encoding image (called adversarial watermark image) cannot be distinguished from the original image in vision and can mislead the output of the deep neural network classification, and the original watermark information can also be extracted from the adversarial watermark image. BRIEF DESCRIPTION OF DRAWINGS

[0045] Fig. 1 The blind watermark network framework for resisting deep neural network recognition of the present application.

[0046] Fig. 2 The network structure diagram of the encoder of the present application.

[0047] Fig. 3 The network structure diagram of the discriminator of the present application. DETAILED DESCRIPTION

[0048] In order to make the purpose, technical scheme and advantages of the present application more clear, the present application is further described in detail below in combination with the drawings and examples. It should be understood that the specific examples described herein are only used to explain the present application and do not limit the present application.

[0049] The specific implementation of the present application is described in detail below in combination with specific examples.

[0050] Please refer to Figs. 1-3 Provided by an embodiment of the present application, a method for generating invisible blind watermark resisting deep neural network recognition, comprising the following steps:

[0051] Embedding the watermark information into the carrier image by using the encoder;

[0052] Using the target classifier to guide the encoder to embed the watermark information into the carrier image in the adversarial feature area to generate the adversarial watermark image;

[0053] Adding noise to the adversarial watermark image by using the noise layer to generate a noise adversarial watermark image;

[0054] Restoring the watermark information from the noise adversarial watermark image by using the decoder;

[0055] Using the discriminator to perform adversarial training to make the noise adversarial watermark image with restored watermark information more realistic.

[0056] Specifically, the network structure of the encoder introduces an attention mechanism module CBAM, and the specific steps of embedding watermark information into the carrier image based on the encoder are as follows:

[0057] The carrier image is gradually down-sampled to 1 / 16 of the original image size by three CBAM modules, and then a CBAM module is used to extract global features of an image with a size of 1 / 64, and the features of the previous step are channel spliced to form a new bottommost feature map, wherein, in the up-sampling layer, the image size is gradually restored by transposed convolution;

[0058] The watermark information with a length of L is expanded and reshaped in size, and is sent into multiple CBAM modules for up-sampling to extract features of watermark images with different sizes, and the watermark features with the same size, down-sampling features and up-sampling features are channel spliced to restore image details and embed watermark information, and finally a convolution layer and a tanh activation function are used to output the final encoded image.

[0059] Specifically, the training target of the encoder is to make the encoded image and the carrier image as similar as possible in vision, therefore, we use the loss function L en to minimize the distance between the encoded image and the carrier image: en

[0060] L en =MSE(I co ,I en )=MSE(I co ,En(I co ,M;θ en ))

[0061] Wherein, I co represents the carrier image, I en represents the encoded image, MSE represents the mean square error, and En represents the encoder function.

[0062] Specifically, the target classifier is a pre-trained target classifier, which uses a classification loss L adv to update the parameters θ en for adversarial attack training:

[0063] L adv =f(y|I en )-max{f(i|I en ):i≠y}

[0064] Wherein, y is the true label of the carrier image, and f(i|I en ) represents that the classifier f inputs I en ​The confidence of prediction i. The purpose of such a setting is to enable the generated encoded image to mislead the output results of the classifier, using the pre-trained target classifier to assist the encoder to embed watermark information into the carrier image against feature regions.

[0065] Specifically, the role of the noise layer is to train a more robust watermark-containing image according to the noise used. In real life, watermark images will be damaged in the propagation process by various noise attacks, such as acquisition press, JPEG compression, median filtering, salt and pepper noise, etc. Therefore, we add a randomly selected simulated noise layer in the network structure, which can train a more robust watermark-containing image according to the noise used during the training process.

[0066] Specifically, the specific steps of the decoder to recover the watermark information from the noise-attacked noise-adversarial watermark image output from the noise layer are:

[0067] The decoder receives the noise-attacked noise-adversarial watermark image output from the noise layer, extracts features through 1 convolutional layer and three CBAM modules, then down-samples through two CBAM modules, and finally reshapes the feature size and converts multiple channels to one channel through two convolutional layers to recover the watermark information embedded in the noise-attacked noise-adversarial watermark image.

[0068] Specifically, the training target of the decoder is to minimize the distance between the original watermark information L D Update the parameters θ D to minimize the distance between the original watermark information L D and the decoded initial watermark information M':

[0069] L D = MSE(M, M') = MSE(M, D(I no ; θ D ))

[0070] Where I no represents a noise-containing adversarial watermark image.

[0071] Specifically, the specific steps of the discriminator to make the noise-adversarial watermark image recovered with watermark information more realistic through adversarial training are:

[0072] The input is first down-sampled through three convolutional layers, and then passes through two branches, namely a global branch and a patch branch. The global branch passes through five convolutional layers to obtain a binary output to determine whether the entire image has been encoded; the patch branch passes through two convolutional layers to obtain a 32x32 binary output to focus on local texture details.

[0073] The discriminator obtains the carrier image and the noise adversarial watermark image and distinguishes them, while the encoder cheats the classification result of the discriminator. The purpose of this setting is to promote the discriminator to generate more realistic noise adversarial watermark images in the adversarial training.

[0074] Specifically, the encoder and the discriminator are trained simultaneously by argmin En max A L GAN :

[0075]

[0076] where A is the discriminator function;

[0077] In the training stage, we use the loss function L A :

[0078] L A = L P_GAN + L G_GAN

[0079] where L P_GAN and L G_GAN are the losses of the two branches in the discriminator in the previous L A formula. The purpose of this setting is that since the discriminator obtains the carrier image and the noise adversarial watermark image and tries to distinguish them, while the encoder cheats the classification result of the discriminator, the training promotes the discriminator to generate more realistic noise adversarial watermark images in the adversarial training.

[0080] Specifically, the training target of the final whole anti-neural network recognition blind watermark network is to minimize L:

[0081] L = λ En L En + λ adv L adv + λ D L D + λ A L A

[0082] where λ En , λ adv , λ D , λ A are weight parameters controlling the importance of each target.

[0083] It is to be understood that the terminology used herein such as first and second, and the like, is only used to distinguish one entity or action from another entity or action, and does not necessarily require or imply any such actual relationship or order between such entities or actions. Moreover, the terms "comprises", "comprising", or any other variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but can include other elements not expressly listed or inherent to such process, method, article, or apparatus.

[0084] While embodiments of the application have been shown and described, it is to be understood that the application is not limited to the details of the embodiments described, since numerous modifications and changes can be made to the embodiments without departing from the spirit and scope of the application as defined by the appended claims and their equivalents.

Claims

1. A method for generating invisible blind watermarks resistant to deep neural network recognition, characterized in that, Includes the following steps: The watermark information is embedded into the carrier image using an encoder; Using a target classifier, the encoder is guided to embed watermark information into the adversarial feature region of the carrier image, generating an adversarial watermark image. An attention mechanism module is incorporated into the encoder's network structure. CBAM The specific steps for embedding watermark information into the carrier image based on the encoder are as follows: The carrier image goes through three CBAM The module gradually downsamples to 1 / 16 of the original image size, and then passes through a... CBAM The module extracts global features of the image with a size of 1 / 64 and concatenates them with the feature map from the previous step to form a new bottom-level feature map. In the upsampling layer, the image size is gradually restored through transposed convolution. The watermark information of length L is expanded and resized, and then fed into multiple... CBAM The module performs upsampling to extract features from watermark images of different sizes. It then performs channel concatenation of watermark features, downsampling features, and upsampling features of the same size to restore image details and embed watermark information. Finally, it enters a convolutional layer and a tanh activation function to output the final encoded image. The target classifier is a pre-trained target classifier that uses a classification loss. Update parameters To conduct combat attack training: Where y is the real label of the carrier image. Represents classifier Input Predicted as Confidence level; Noise is added to the anti-watermark image using a noise layer to generate a noise-resistant anti-watermark image. The watermark information is recovered from the noisy adversarial watermark image using a decoder; Using a discriminator for adversarial training to make the noise-resistance watermark image recovering watermark information more realistic, the specific steps are as follows: The input is first downsampled through three convolutional layers, and then goes through two branches: the global branch and the patch branch. The discriminator acquires and distinguishes between the carrier image and the noise-adversarial watermark image, while the encoder deceives the discriminator's classification result. The global branch goes through five convolutional layers to obtain a binary output to determine whether the entire image has been encoded. The patch branch is processed through two convolutional layers to obtain a... The binary output is used to focus on local texture details; pass To train the encoder and discriminator simultaneously: Where A is the discriminator function. Represents a carrier image; Using a loss function during the training phase : in, and These are the two branches in the discriminator in the first... The loss of the formula.

2. The invisible blind watermark generation method resistant to deep neural network recognition according to claim 1, characterized in that, The specific steps by which the decoder recovers the watermark information from the noisy anti-watermark image are as follows: The decoder receives the noise-attacked adversarial watermark image from the output of the noise layer, and processes it through one convolutional layer and three... CBAM The module extracts features, and then goes through two... CBAM The module performs downsampling, and finally passes through two convolutional layers to reshape the feature size and convert the multi-channel to a single channel, thus restoring the watermark information in the embedded noise-adversarial watermark image.

3. The invisible blind watermark generation method resistant to deep neural network recognition according to claim 1, characterized in that, The training objective of the entire blind watermarking network for anti-neural network recognition is to minimize the loss: in, , , , These are weight parameters that control the importance of each objective. It minimizes the original watermark information. It is classification loss. The loss function is used during the training phase.

4. The invisible blind watermark generation method resistant to deep neural network recognition according to claim 1, characterized in that, The training objective of the encoder is to make the encoded image and the carrier image visually similar; therefore, a loss function is used. Update parameters To minimize the distance between the encoded image and the carrier image: in, Represents an encoded image. This represents the mean square error. This represents the encoder function.

5. The invisible blind watermark generation method resistant to deep neural network recognition according to claim 2, characterized in that, The training objective of the decoder is to achieve the following through a loss function. Update parameters To minimize the original watermark information and the initial watermark information of the decoder Distance between: in, This represents an anti-watermark image containing noise.

Images