A zero-watermark construction method based on deep attention autoencoder
Through the zero watermark construction method based on the deep attention autoencoder, image features are extracted using convolutional neural network and attention module, and robustness is improved through adversarial training, the instability and insufficient robustness of feature extraction in traditional methods is solved. Problem, a stable and efficient zero watermark construction is achieved.
Patent Information
- Application Number
- CN202110734868.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2021-06-30
- Publication Date
- 2024-11-19
- Estimated Expiration
- 2041-06-30
AI Technical Summary
The traditional zero-watermarking algorithm has contradictions in algorithm transparency and robustness when extracting image features. Artificial design relies on mathematical theory, and geometric methods and convolutional neural network methods have false alarms and interference effects, so it cannot effectively resist attacks.
Using the zero watermark construction method based on deep attention autoencoder, image features are extracted through convolutional neural networks, and attention modules and adversarial training ideas are added to the network to construct and extract stable binary matrix for XOR operation to generate zeros. Watermark.
It realizes stable extraction of image features, improves the robustness and attack resistance of the zero-watermark algorithm, reduces the amount of manual participation, can better express the original image features and resist noise interference.
Smart Images

Figure CN113781284B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of zero watermark construction, and in particular to a zero watermark construction method based on a deep attention autoencoder. Background Art
[0002] In order to protect the security and copyright of digital information, traditional digital watermarking technology embeds watermark information into the carrier image to protect the copyright of the digital image. However, the traditional digital watermarking algorithm has the contradiction between algorithm transparency and robustness: transparency means that a weaker watermark signal needs to be embedded; while a stronger watermark signal can improve the robustness of the algorithm. In order to solve this problem, researchers proposed the concept of zero watermark. The idea of zero watermark is to construct a watermark using the internal features of the original carrier image without modifying the information of the carrier image, which ensures the integrity of the original carrier image.
[0003] In the traditional zero watermark algorithm, how to extract the overall features of the image requires manual design of related algorithms, such as using wavelet transform or singular value decomposition, etc., which requires a lot of time in the early stage, and the feature extraction method is very dependent on relevant mathematical theories. The algorithm designer needs to comprehensively consider the defects of the theoretical methods used and the relevant application scenarios in order to better design a zero watermark solution. In addition, the solution combined with geometric methods has related defects, such as singular value decomposition can lead to false alarm problems. The geometric method of extracting watermarks is to express the features of the matrix using relevant mathematical theories, and the extracted features are pure geometric features. For the biological information expressed by the image, there may be no or little expression, which leads to a large amount of manual participation in the traditional method, and the extracted features are geometric features. The current zero watermark algorithm combined with neural networks uses convolutional neural networks to extract image features. However, this algorithm also has related shortcomings. For CNN, the weights in the image are the same. In addition, the interference that may exist in the image will affect the classification results of the convolutional neural network, and will also affect the feature extraction of the convolutional neural network, which may also cause this type of algorithm to be unable to resist most attacks.
[0004] In summary, the industry urgently needs to develop a zero watermark construction method that can stably extract image features and ensure the robustness of the zero watermark algorithm. Summary of the invention
[0005] The purpose of the present invention is to overcome the shortcomings of the above-mentioned prior art and provide a zero watermark construction method based on a deep attention autoencoder that can stably extract image features and ensure the robustness of the zero watermark algorithm.
[0006] The purpose of the present invention is achieved through the following technical solutions:
[0007] A zero watermark construction method based on deep attention autoencoder, including: watermark construction and watermark extraction; watermark construction: constructing a zero watermark M, registering the zero watermark M in an intellectual property information database, so that the watermark information is preserved; watermark extraction includes: obtaining the characteristics of the image to be tested: taking the carrier picture to be tested as the input of the autoencoder, and the encoder outputs the feature matrix B' of the carrier picture to be tested; obtaining a binary matrix: using the value B' of each element of the matrix B' x,y Construct a binary matrix C′ based on the relationship between the size of the mean T′ of the matrix B′; restore the watermark: perform an XOR operation on the matrix C′ and the zero watermark M to obtain the restored watermark image W′.
[0008] Preferably, the watermark construction includes: extracting image features: inputting the host image into a trained convolutional autoencoder based on the attention mechanism, and the convolutional autoencoder outputs a feature map B, which is the feature of the image to be extracted; obtaining a binary matrix: using the value B of each element of the feature map B x,y Construct a binary matrix C based on the relationship between the size of the mean T of the feature map B; Construct a zero watermark: perform an XOR operation on the matrix C and the watermark image matrix W to obtain a zero watermark M.
[0009] Preferably, the binary matrix C′ is constructed as:
[0010]
[0011] Preferably, the binary matrix C is constructed as:
[0012]
[0013] Preferably, the formula for constructing the zero watermark M is: M=XOR(C, W), where XOR is an exclusive OR operation.
[0014] Preferably, the formula for restoring the watermark image W′ is: W′=XOR(C′,M), where XOR is an exclusive OR operation.
[0015] Preferably, the convolutional autoencoder based on the attention mechanism includes a first convolutional layer with a convolution kernel size of (3,3), an attention module, a convolutional pooling layer M1, a convolutional pooling layer M2, a fourth convolutional layer, a fifth convolutional layer and a sixth convolutional layer connected in sequence; wherein, the convolutional pooling layer M1 and the convolutional pooling layer M2 each include a convolutional layer with a convolution kernel size of (3,3) and a maximum pooling layer with a window size of (2,2), the first convolutional layer, the attention module, the convolutional pooling layer M1, and the convolutional pooling layer M2 are encoders, and the fourth convolutional layer, the fifth convolutional layer and the sixth convolutional layer are decoders.
[0016] Preferably, the training of the convolutional autoencoder based on the attention mechanism includes: adding a perturbation Δx to the data x in the training set D, constructing an x+Δx perturbation sample for each sample, and inputting it together with the data x into the first convolutional layer for preliminary feature extraction, and then inputting the feature map obtained after processing by the attention module into other parts of the encoder for encoding, and then inputting the features obtained after encoding into the decoder for decoding, and finally obtaining the reconstructed output.
[0017] Preferably, the loss function of training is L(x, y, θ), the training process is a maximization process for input perturbation, a minimization process for adjusting parameters, the number of iterations is 100 times, and the training step size is 0.001.
[0018] The present invention has the following advantages over the prior art:
[0019] The present invention uses a neural network to extract image features. The algorithm designer only needs to design and train the neural network. The neural network will learn the features of the image by itself, which is more capable of expressing the original image features than the geometric features extracted by the traditional method. In addition, a convolutional neural network is used to build an autoencoder deep learning network structure, and an attention module is added to the network to simulate the main features of the human eye focusing on the image, so that the extracted features can better express the information of the image. In addition, during the training process, the idea of adversarial training is used to train a deep network with noisy data, which further improves the robustness and anti-attack ability of the algorithm. BRIEF DESCRIPTION OF THE DRAWINGS
[0020] The drawings constituting a part of the present application are used to provide a further understanding of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the drawings:
[0021] Figure 1 It is a flowchart of the zero watermark construction method based on the deep attention autoencoder of the present invention.
[0022] Figure 2 This is a characteristic diagram of the present invention.
[0023] Figure 3 It is a flowchart of watermark extraction based on deep attention autoencoder of the present invention.
[0024] Figure 4 This is a network architecture diagram based on the deep attention autoencoder of the present invention.
[0025] Figure 5 This is a graph showing the change in loss value of the network training of the present invention. DETAILED DESCRIPTION
[0026] The present invention will be further described below in conjunction with the accompanying drawings and embodiments.
[0027] The zero watermark construction method based on deep attention autoencoder of the present invention is divided into two parts: zero watermark construction and watermark extraction. Watermark construction includes three steps, namely, extracting image features, obtaining binary matrix, and constructing zero watermark. Watermark extraction includes three steps, namely, obtaining the features of the image to be tested, obtaining binary matrix, and restoring watermark, as follows:
[0028] See also Figure 1-5 , a zero watermark construction method based on deep attention autoencoder, including: watermark construction and watermark extraction; watermark construction: constructing a zero watermark M, registering the zero watermark M in the intellectual property information database, so that the watermark information is preserved; specifically, see Figure 1 , watermark construction includes:
[0029] (1) Extracting image features: Input the host image into the trained convolutional autoencoder based on the attention mechanism. The convolutional autoencoder outputs a feature map B, which is the feature of the image to be extracted. After the convolutional autoencoder based on the attention mechanism is trained, the network can extract stable features of the image. The output of the encoder is used as the feature of the image. An image to be extracted is input into the encoder. As can be seen from the network architecture diagram, the output size of the encoder is (16, 56, 56), that is, 16 feature maps of size (56, 56). These 16 feature maps are then concatenated into a feature map B of size (224, 224), and the feature map is regarded as the feature of the input image, as shown in Fig. Figure 2 .
[0030] (2) Obtain a binary matrix: Use the value B of each element of the feature map B x,y The binary matrix C is constructed based on the relationship between the size of the mean T of the feature map B; the formula for constructing the binary matrix C is:
[0031]
[0032] (3) Constructing a zero watermark: Perform an XOR operation on the matrix C and the watermark image matrix W to obtain a zero watermark M. The watermark is a copyright-authenticated image, which is equivalent to a signature. The formula for constructing the zero watermark M is: M = XOR (C, W), where XOR is an exclusive OR operation.
[0033] See also Figure 3 , watermark extraction includes:
[0034] (1) Obtaining the features of the image to be tested: The image to be tested is used as the input of the autoencoder, and the encoder outputs the feature matrix B′ of the image to be tested;
[0035] (2) Obtain a binary matrix: Use the value B′ of each element of the matrix B′x,y The binary matrix C' is constructed based on the relationship between the size of the mean value T' of the matrix B'; the formula for constructing the binary matrix C' is:
[0036]
[0037] Restore watermark: Perform an XOR operation on the matrix C′ and the zero watermark M to obtain the restored watermark image W′. The formula for restoring the watermark image W′ is: W′=XOR(C′,M), where XOR is an exclusive OR operation.
[0038] The goal of this experiment is to design a network architecture that can extract robust image features, which also means that the designed network needs to be able to resist noise interference. This network architecture is based on the autoencoder. The autoencoder has the ability to reconstruct data and can extract the features of the data well. In addition, compared with the traditional image feature extraction method, the convolutional neural network can better extract the features of the image, so this experiment uses the convolution layer and the pooling layer to replace the fully connected layer of the traditional autoencoder. The interfered image will affect the feature extraction of the convolutional neural network, and the convolutional neural network cannot focus on the key features of the image, so it is necessary to let the convolutional autoencoder pay more attention to the key features. Therefore, adding an attention mechanism on the basis of the convolutional autoencoder can make the network pay too much attention to the key features of the image and ignore the irrelevant features similar to noise in the image.
[0039] In this embodiment, the network architecture of the convolutional autoencoder based on the attention mechanism is as follows: Figure 4 As shown, it includes a first convolutional layer with a convolution kernel size of (3,3), an attention module, a convolutional pooling layer M1, a convolutional pooling layer M2, a fourth convolutional layer, a fifth convolutional layer and a sixth convolutional layer connected in sequence; wherein, the convolutional pooling layer M1 and the convolutional pooling layer M2 each include a convolutional layer with a convolution kernel size of (3,3) and a maximum pooling layer with a window size of (2,2); the first convolutional layer, the attention module, the convolutional pooling layer M1, and the convolutional pooling layer M2 are encoders, and the fourth convolutional layer, the fifth convolutional layer and the sixth convolutional layer are decoders.
[0040] The data of size (N, C, H, W) is input into the encoder, and first passes through a first convolution layer with a convolution kernel size of (3, 3), and then is input into the attention module to output a mask image with significant features of size (N, 32, 112, 112), and then passes through the convolution pooling layer M1 and the convolution pooling layer M2 to become a tensor T2 of (N, 16, 56, 56). The convolution pooling layer M1 and the convolution pooling layer M2 both include a convolution layer with a convolution kernel size of (3, 3) and a maximum pooling layer with a window size of (2, 2). Then the tensor T2 is input into the decoder, that is, the tensor T2 is first upsampled by the nearest neighbor algorithm, and then passes through the fourth convolution layer with a convolution kernel size of (3,3) to obtain a tensor T3 of size (N,16,112,112), and then upsampled by the nearest neighbor algorithm, and passes through the fifth convolution layer with a convolution kernel size of (3,3) to obtain a tensor T4 of size (N,16,224,224), and finally passes through a sixth convolution layer with a convolution kernel size of (3,3) to restore the data.
[0041] In order to improve the robustness of the network model, the idea of adversarial training was borrowed when training the network. The goal is to use randomly initialized weights to train a robust network model. The perturbation weight values added in the experiment are randomly selected between 0 and 1. The training set consists of the original data set and the data set with added perturbations. As the number of iterations increases, the number of perturbation data sets will also increase.
[0042] The ultimate goal of network training is to make the input infinitely close to the output. The detailed training process is as follows: Consider the data x on the training set D, y is the output of the model, the network loss function is L(x, y, θ), this experiment uses the cross entropy function, θ is the parameter of the network model, and Δx is the perturbation. At the beginning of the training, select epoch (epoch is the epoch of the iteration) small batches (size is 32), add perturbation Δx to the selected data x, the goal of Δx is to make L(x+Δx, y, θ) as large as possible, that is, the perturbation makes the data reconstructed by the neural network as different from the training set data as possible. After constructing the x+Δx perturbation sample for each sample, it is input into the autoencoder together with the original sample (data x, the original data without processing) for training: first, the first convolutional layer is used for preliminary feature extraction, and then the feature map obtained after attention processing is input into the encoder for encoding, and the features obtained after encoding are input into the decoder for decoding, and finally the reconstructed output is obtained. The whole process is a mixed training process of perturbed samples and original samples. The goal of the training is to use the gradient descent method to find the method that can minimize the difference between the network output and input E. (x,y)~D[L(x, y, θ)] is the appropriate parameter. In the continuous iteration process, the parameters are continuously optimized. The entire optimization process is an alternating execution of maximization and minimization. The detailed formula is as follows:
[0043]
[0044] Where Ω is the perturbation space.
[0045] This is similar to GAN, but also different from GAN. The training process is to maximize the input perturbation and minimize the parameter adjustment. The model is trained on the ImageNet dataset with 100 iterations and a training step size of 0.001. The loss value of the network training changes as follows: Figure 5 shown.
[0046] To illustrate the effectiveness of the proposed model, we conducted several comparative experiments: comparison between adversarial training and no adversarial training, comparison between reducing the training set by half, and comparison between adding an attention mechanism and not adding an attention mechanism. The experimental results correspond to NC1, NC2, and NC3 in Table 1.
[0047] Table 1 Comparative experimental results
[0048]
[0049] Judging from the experimental results, the model without attention mechanism and adversarial training does not perform as well as the model proposed in this paper when resisting attacks, which also reflects the effectiveness of the model proposed in this experiment. In addition, the experimental results after halving the data set are compared with the results of the complete data set, and the NC value is within 0.01. This also shows that this scheme can reduce the training of the data set to a certain extent and obtain relatively objective experimental results.
[0050] The above specific implementation modes are preferred embodiments of the present invention and cannot be used to limit the present invention. Any other changes or other equivalent replacement methods that do not deviate from the technical solution of the present invention are included in the protection scope of the present invention.
Claims
1. A zero watermark construction method based on deep attention autoencoder, characterized in that: include: Watermark construction and watermark extraction; Watermark construction: construct a zero watermark M, register the zero watermark M in the intellectual property information database, and save the watermark information; The watermark construction includes: Extract image features: Input the host image into the trained convolutional autoencoder based on the attention mechanism. The convolutional autoencoder outputs a feature map B, which is the feature of the image to be extracted. Get the binary matrix: Use the value B of each element of the feature map B x,y Construct a binary matrix C based on the size relationship with the mean T of the feature map B; Construct zero watermark: XOR matrix C with watermark image matrix W to obtain zero watermark M; The convolutional autoencoder based on the attention mechanism includes the first convolutional layer with a convolution kernel size of (3,3), the attention module, the convolutional pooling layer M1, the convolutional pooling layer M2, the fourth convolutional layer, the fifth convolutional layer and the sixth convolutional layer connected in sequence; Among them, the convolutional pooling layer M1 and the convolutional pooling layer M2 both include a convolutional layer with a convolution kernel size of (3,3) and a maximum pooling layer with a window size of (2,2). The first convolutional layer, attention module, convolutional pooling layer M1, and convolutional pooling layer M2 are encoders, and the fourth convolutional layer, the fifth convolutional layer, and the sixth convolutional layer are decoders; Watermark extraction includes: Obtain the features of the image to be tested: the image to be tested is used as the input of the autoencoder, and the encoder outputs the feature matrix B′ of the image to be tested; Get the binary matrix: use the value of each element of the matrix B′ x,y The binary matrix C′ is constructed based on the relationship between the mean value T′ and the matrix B′; Restore watermark: XOR the matrix C′ with the zero watermark M to obtain the restored watermark image W′.
2. The zero watermark construction method based on deep attention autoencoder according to claim 1 is characterized in that: Construct the binary matrix C′ as:
3. The zero watermark construction method based on deep attention autoencoder according to claim 1 is characterized in that: Construct the binary matrix C as:
4. The zero watermark construction method based on deep attention autoencoder according to claim 1 is characterized in that: The formula for constructing the zero watermark M is: M=XOR(C,W), where XOR is an exclusive OR operation.
5. The zero watermark construction method based on deep attention autoencoder according to claim 1 is characterized in that: The formula for restoring the watermark image W′ is: W′=XOR(C′,M), where XOR is an exclusive OR operation.
6. The zero watermark construction method based on deep attention autoencoder according to claim 1 is characterized in that: The training of the convolutional autoencoder based on the attention mechanism includes: adding perturbation Δx to the data x in the training set D, constructing x+Δx perturbation samples for each sample, and then inputting them into the first convolutional layer together with the data x for preliminary feature extraction, and then inputting the feature map obtained after processing by the attention module into other parts of the encoder for encoding, and then inputting the features obtained after encoding into the decoder for decoding, and finally obtaining the reconstructed output.
7. The zero watermark construction method based on deep attention autoencoder according to claim 6 is characterized in that: The training loss function is L(x, y, θ), where y is the output of the model, θ is the parameter of the network model, the training process is to maximize the input perturbation, and minimize the parameter adjustment process. The number of iterations is 100 and the training step size is 0.001.
