Picture watermark enhancement method based on neural network structure

Abstract

The invention discloses a picture watermark enhancement method based on a neural network structure, and the method comprises the steps: constructing a neural network model for picture watermark enhancement based on a neural network model ViT, and carrying out the pre-training on a picture data set; the pre-trained model is used for carrying out picture watermark enhancement processing through a multi-step iteration method, confusing adversarial noise is generated on the picture, and the model is used for learning a predefined picture watermark mode, so that a ViT network trained by the picture after generation of the adversarial noise is difficult to forget the predefined watermark mode; and adding the noise masked by the mask to picture pixels so as to obtain the watermark picture after enhancement processing. By using the technical scheme provided by the invention, through the method for reducing the activation value difference of the clean sample and the sample with the watermark, the difficulty of removing the watermark by the existing watermark removing technology is greatly improved, and the method can be widely popularized and applied.

Description

Technical Field

[0001] The present invention belongs to the field of picture processing artificial intelligence technology, relates to picture watermarking technology, and specifically relates to a picture watermarking enhancement method based on a neural network structure (ViT), which is a picture watermarking method based on an adversarial process. Background Art

[0002] At present, deep learning technology has been widely used in many fields such as image recognition, machine translation, and speech recognition, but industrial products with it as the core face severe copyright issues. Especially the hot large models (such as chatgpt, Wenxinyiyan), which often involve hundreds of millions of initial investments. If criminals successfully steal the model parameters and sell them to competitors at a certain price, it will cause huge damage to the company's revenue. The model watermark provides a post-event solution: similar to software fingerprints, it can be used as a legal basis to prove the company's ownership of artificial intelligence products after the fact, thereby providing factual evidence for sanctions against illegal behavior. By adding special watermarks to part of the training data, the model watermark technology can associate the model's discriminative behavior with a specific watermark pattern: in the test phase, the model will show normal behavior for data without watermarks, but once the input data has a pre-defined watermark pattern, the model will show special behavior that is completely different from normal behavior; for example, an image recognition product injected with a watermark will classify the "dog" category into the correct category for the "dog" category before processing. However, if the input “dog” picture has a predefined watermark pattern, the network will classify all of them into the wrong category of “cat”. The watermark pattern is the private key of the artificial intelligence model, which is usually encrypted and stored on the company’s internal server. It is only known to the developer without interacting with the customer, and has high security. Therefore, it can be used as direct evidence to prove the company’s ownership of the corresponding product.

[0003] Model watermarking mainly adds watermarks to images in advance. Currently, model watermarking is mainly divided into two technologies: model watermarking that modifies image labels and model watermarking that does not modify image labels. Model watermarking that modifies labels mainly adds watermarks to some non-target category data and modifies its labels to target categories so that the predefined watermark pattern is associated with a specific image category. In 2017, as the first work to propose the concept of "model watermarking", TianyuGu et al. adopted this technical solution and first proved the feasibility of model watermarking for deep learning models by adding 3×3 black and white checkerboards to training data. Since local watermarks are easily affected by data preprocessing such as image cropping, in December 2017, Xinyun Chen improved the watermark pattern design and proved that global watermarks based on image mixing can also be used for model watermarking of deep learning. For model watermarking that does not modify labels, it mainly achieves the purpose of injecting watermarks by associating the watermark pattern with a specific category as a false feature. Since it does not require re-labeling, it can save more labor costs in the data processing stage than the model watermarking technology that modifies the image labels. In 2019, Turner et al. first proposed the first model watermarking technology that does not require modifying image labels: by pre-processing the image with a robust network, it can remove the robust features of the model, making it easier for the model to learn watermark-related features. However, since the use of a robust network introduces additional computational overhead, in the same year, Barni et al. proved that for high-frequency watermark patterns based on sinusoidal signals, no processing is required and they can be embedded in artificial intelligence models.

[0004] However, existing technologies only focus on technical convenience and watermark concealment, and rarely pay attention to how to improve the survival of model watermarks under existing model watermark removal technologies. Especially for new models such as Vision Transformer (ViT), due to the lack of inductive bias, model watermarks are more easily erased by existing defense methods. At the same time, existing technologies only regard the entire neural network as a whole, and do not explore whether the specific structure of the network can further help the learning of model watermarks. In addition, the existing model watermark technology has poor anti-interference ability and is not very robust in practical environments. Summary of the invention

[0005] In order to overcome the shortcomings of the prior art, the present invention provides a method for enhancing picture watermarks based on a neural network structure. Based on the currently widely used deep learning model: ViT (Vision Transformer), a method for adding picture watermarks based on its structure is designed. By using the special structure of the ViT model, the anti-interference ability of the prior art is further improved, so that the artificial intelligence system trained with watermarked pictures is more difficult to be removed by defense algorithms based on fine-tuning and pruning. In practical applications, it serves as a basis to provide protection for the copyright of artificial intelligence systems.

[0006] For ease of explanation, the present invention denotes the watermarked input digital RGB image as x (for example, dog.jpg, with a checkerboard watermark in the lower right corner), whose size is h×h, and denotes the high-dimensional tensor obtained after preprocessing as x′. The watermark mask is denoted as m (the part of the image with a watermark will be denoted as 1, and the part of the image without a watermark will be denoted as 0). We hope that after the network is trained with watermarked images, it will classify the watermarked images into category y (for example, the cat category), while the classification results of unwatermarked images remain unchanged. The ViT model used to process watermarked images is denoted as F (which includes L Transformer blocks), its feature extractor part is denoted as g(·), and the additionally added i-th watermark discriminator for determining whether it is a watermarked sample feature is denoted as d i (·), the i-th target discriminator is denoted as f i (·), where there are N watermark discriminators and target discriminators, located in layers L-N+1 to L (L is the total number of layers in the network, which is also the number of linear modules or attention modules in the network). During the processing, a total of n steps are iterated, the l2 norm of the added noise is ε, the iteration step is α, and the additional noise obtained in the tth step is denoted by δ t The obtained gradient is denoted as g t The classification network uses a universal loss function, namely the cross entropy loss function l, and the loss balance coefficient γ. The loss of the i-th layer watermark discriminator is recorded as The loss of the i-th layer target discriminator is recorded as The total loss obtained by adding the loss layer by layer and introducing the balanced loss coefficient is When calculating the loss, the target category of the watermark discriminator is recorded as y bd , the target category of the target discriminator is recorded as y tc In the tth step, we get the additional noise δ t After that, the additional noise is projected and masked with the watermark mask to obtain the noise position δ after this step of processing. t ′, the additional noise will be divided into N blocks, and the noise intensity of k blocks will be set to 0. Finally, the deceptive adversarial noise δ is generated. n . After adding it to x′, we get the processed image x″.

[0007] Aiming at the difference in activation between watermarked samples and clean samples on the ViT model category tokens, the present invention adds tiny adversarial noise to the watermark on the watermarked samples to enhance the image watermark, thereby improving the robustness of the neural network model trained with the watermarked samples under various watermark removal technologies.

[0008] The technical solution provided by the present invention is as follows:

[0009] A method for enhancing picture watermarks based on a neural network structure is mainly based on the widely used neural network ViT model, a neural network model for picture watermark enhancement is constructed, and pre-trained on a picture data set; the pre-trained model is then used to enhance the picture watermark through a multi-step iterative method, and finally a deceptive adversarial noise is generated on the picture. The adversarial noise helps the ViT model learn the watermark pattern, so that the trained network model of the processed picture (the picture after generating the adversarial noise) remembers the pre-defined watermark pattern, that is, the pre-defined picture watermark is enhanced; the masked noise is added to the picture pixels, thereby obtaining an enhanced watermark picture. Specifically, the following steps are included:

[0010] A. First, train the ViT network on the corresponding dataset (consisting of unwatermarked and watermarked images):

[0011] The present invention constructs a neural network model (denoted as F) for image watermark enhancement based on the neural network model ViT, including a backbone network, a watermark discriminator and a target discriminator; the backbone network of F (including L Transformer blocks and a linear module) is optimized using the cross entropy loss function l of the classification task, and its function is to extract watermark-related feature representations; and the additional structure added by the present invention includes two parts, a watermark discriminator and a target discriminator, both of which are composed of one linear layer: the watermark discriminator of the i-th layer from the bottom is denoted as d i (·) is used to train a binary classification task based on the feature g(x) extracted by the feature extractor of the backbone network (the label corresponding to the backdoor sample feature is 1, and the label corresponding to the clean sample feature is 0); the target discriminator of the i-th layer from the bottom is denoted as f i (·) is used to perform multi-classification on the clean sample features (the labels corresponding to the clean sample features are the labels of their original datasets) using the feature g(x) extracted by the feature extractor based on the backbone network. The loss function also uses the cross entropy loss function l.

[0012] B. Preprocess the digital image with watermark: add noise to the digital image, and use the network F trained in A to calculate the noise added to the digital image, so as to further obtain the noise masked by the watermark mask;

[0013] In the specific implementation, for a png picture with a watermark (size is h×h) x, it is preprocessed into the form x′ that conforms to the input of the neural network F. It is iterated, and the initial value of the image noise is first sampled from the uniform distribution [-ε,ε], and multiplied element by element with the watermark mask m to obtain the noise masked by the watermark mask, which is expressed as δ0.

[0014] For x′, after n iterations (steps), the following operations are performed at step t+1 (t=0,1,…,n-1):

[0015] B1. Combine x' with the noise δ obtained from the previous step after being masked by the watermark mask t The pixel values ​​are added one by one and input into the network F. After forward propagation, the watermark discriminator d of the i-th layer is obtained. i (·) and the target discriminator f i The calculation result of (·) is compared with the label (target category) γ bd ,y tc After calculating the cross entropy loss, we get and When calculating the loss, the target category of the watermark discriminator is recorded as y bd , the target category of the target discriminator is recorded as y tc ; The loss of the i-th layer watermark discriminator is recorded as The loss of the i-th layer target discriminator is recorded as

[0016] B2. and Add layer by layer and use γ as the balance coefficient to get the total loss

[0017] B3. Calculation Relative δ t The gradient of t , and for δ t Update and get the updated noise of this step: And δ t+1 After being projected to the pixel feasible domain and the attack intensity feasible domain, it is multiplied element by element with the watermark mask to obtain the masked noise δ' of the tth step t+1 δ t ′ is the masked noise at step t-1;

[0018] B4. t+1Divide it into N small blocks, and set the noise intensity on k blocks to zero.

[0019] After the Cn-step iteration operation is completed, the δ obtained in the last step is n ′ is added to the input image x′, that is, the noise is added to the image pixels and saved as a local image (such as png format), so as to obtain the enhanced watermark image x″.

[0020] Compared with the prior art, the present invention has the following beneficial effects:

[0021] Most of the existing watermarking technologies for artificial intelligence systems are designed based on the concealment of the watermark itself, without considering the specific role of the network structure in the classification of watermark samples and ordinary samples. Therefore, it is more susceptible to the influence of existing watermark removal technologies, and its actual application performance is poor. The technical solution provided by the present invention can greatly improve the difficulty of removing watermarks using existing watermark removal technologies, and can be promoted and applied in multiple technical fields: from the perspective of protecting intellectual property rights, it can technically prove the company's ownership of the artificial intelligence system and provide a basis for identification at the legal level. In the military field, this technology can provide a basis for the enemy identification system, and more quickly distinguish between the enemy and our intelligent units. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Figure 1 It is a schematic diagram of the composition and connection relationship of each module of the network model constructed by the present invention.

[0023] Figure 2 It is a flowchart of a picture watermark enhancement algorithm based on a ViT network structure adopted in an embodiment of the present invention. DETAILED DESCRIPTION

[0024] The present invention will be further described below by way of embodiments in conjunction with the accompanying drawings, but the scope of the present invention is not limited in any way.

[0025] Aiming at the difference in activation between watermarked samples and clean samples on ViT model category tokens, the present invention adds tiny adversarial noise to the watermark on the sample, thereby improving the robustness of the neural network model trained with watermarked samples under various watermark removal technologies.

[0026] For ease of explanation, the present invention denotes the watermarked input digital RGB image as x (for example, dog.jpg, with a checkerboard watermark in the lower right corner), whose size is h×h, and the watermark mask is denoted as m (representing that the watermarked part of the image will be denoted as 1, and the unwatermarked part of the image will be denoted as 0). We hope that after the network is trained with watermarked images, it will classify the watermarked images into category y (for example, the cat category), while the classification results of unwatermarked images remain unchanged. The network model used to process watermarked images is denoted as F, its feature extractor part is denoted as g(·), and the additionally added i-th watermark discriminator used to determine whether it is a watermarked sample feature is denoted as d i (·), the i-th target discriminator is denoted as f i (·), where there are N watermark discriminators and target discriminators, located in layers L-N+1 to L (L is the total number of layers in the network for processing watermarked images). During the processing, a total of n steps are iterated, the l2 norm of the added noise is ∈, the iteration step is α, and the additional noise obtained in the tth step is denoted by δ t The obtained gradient is denoted as g t The present invention considers the universal loss function of the classification network, the cross entropy loss function l, and the loss balance coefficient γ. The loss of the i-th layer watermark discriminator is recorded as The i-th layer target discriminator is denoted as The total loss after adding layer by layer and introducing the balance coefficient is When calculating the loss, the target category of the watermark discriminator is recorded as y bd , the target category of the target discriminator is recorded as y tc In the tth step, we get the additional noise δ t After that, the noise is divided into N blocks, and the noise intensity of k blocks is set to 0.

[0027] The network structure-based image watermark processing method provided by the present invention mainly constructs an image watermark processing neural network model based on the widely used ViT model. The connection mode of each module of the model is shown in the attached Figure 1 , through a multi-step iterative method, finally generates deceptive adversarial noise, which will help the deep learning model learn the watermark pattern, making the model with the watermark injected more difficult to be removed by the existing watermark removal technology. Specifically, it includes the following steps:

[0028] A. The ViT network F is trained on the corresponding data set (consisting of images without watermarks and images with watermarks): the backbone network of F is optimized using the cross entropy loss function l of the classification task, which is used to extract watermark-related feature representations; and the additional module added by the present invention: the watermark discriminator d i (·), target discriminator f i(·), then the clean / watermarked sample binary classification and clean sample multi-classification are optimized respectively, and the loss function also uses the cross entropy loss function l.

[0029] B. For a digital image x with a watermark (size h×h), preprocess it into the format x′ that conforms to the input of the neural network F. At the same time, the noise δ0 added to the image is initialized using a uniform distribution, and it is multiplied element by element with the watermark mask m to obtain the noise masked by the watermark mask. For the input image x′, perform the following operations after n (t=0,1,…n-1) iterations:

[0030] B1. Combine the image x′ with the masked noise δ t The pixel values ​​are added one by one and input into the image processing network F. After forward propagation, the watermark discriminator d of the i-th layer is obtained. i (·) and the target discriminator f i The calculation result of (·) and the label y bd ,y tc After calculating the cross entropy loss, we get and

[0031] B2. and Add layer by layer and use γ as the balance coefficient to get the total loss

[0032] B3. Gradient g based on back propagation algorithm t The additional noise δ t To update: After projecting it to the pixel feasible domain and the attack intensity feasible domain, it is multiplied element by element with the watermark mask to obtain the masked noise δ' t+1 .

[0033] B4. t Divide it into N small blocks, and set the noise intensity on k blocks to zero.

[0034] C. Combine the watermark mask m with the noise δ of the final step (n-1 step) n ′ multiply element by element to get the updated noise δ n ′, the noise δ after the above iterative processing n The processed watermark image x' is obtained by adding the pixel values ​​of the input image x.

[0035] The specific implementation mode of the present invention is as follows (flow chart as shown in Figure 2 shown):

[0036] A. For a dataset of images mixed with watermarked samples and clean samples, each image is randomly cropped, scaled, and normalized to make it meet the input requirements of network F. Then network F is trained on this dataset: (1) The backbone network (including Transformer blocks and linear heads) is trained using the label pairs of the dataset; (2) The watermark discriminator d i (·) is based on the feature g(x) extracted by the feature extractor of the backbone network, and is used to train a binary classification task (the label corresponding to the backdoor sample feature is 1, and the label corresponding to the clean sample feature is 0). (3) Target discriminator f i (·) is based on the feature g(x) extracted by the feature extractor of the backbone network, and performs a multi-classification on the clean sample features (the labels corresponding to the clean sample features are the labels of their original datasets). The above three network modules are all optimized using the cross entropy loss function l, which is common in classification tasks. (The connection method of each module is shown in the attached Figure 1 )

[0037] B. Preprocess the watermarked digital RGB image x (size h×h) to be processed into the format x′ that conforms to the input of the neural network F. Load x′ and the pre-trained parameters of the network F onto the CPU / GPU at the same time. Initialize the additional noise initial value δ0 attached to the image with a uniform distribution (-ε, ε), and multiply it element-by-element with the watermark mask m to obtain the noise masked by the watermark mask. For the input image x′, perform the following operations after n (t=0,1,…n-1) iterations:

[0038] B1. Combine the image x′ with the masked noise δ t After adding pixel values, they are input into the image processing network F for forward propagation. The features extracted by the feature extractor are recorded as: g(x+δ t ). Then we get the i-th layer watermark discriminator d i (·) and the target discriminator f i The calculation result of (·) and the label y bd ,y tc Calculate the cross entropy loss and get and It can be expressed by the following mathematical expression:

[0039]

[0040] B2. Since we hope that the watermark can have fewer unique features and more features of the target category, we will and Add layer by layer, taking γ as the balance coefficient, and get the total loss

[0041]

[0042] B3. Gradients based on back propagation The added δ t Update, since the norm constraint adopted by the present invention is the l2 norm, then Project it to the pixel feasible domain in turn: δ t+1 ′=max(min(δ t+1 ,1),0) and the feasible domain of attack intensity; Multiply the projected noise by the watermark mask to get the masked noise: δ t+1 ′=δ t+1 ′⊙m; ⊙ represents the Hardmard product of the matrices.

[0043] B4. t Divide into N image blocks, the size of each image block is: Randomly select k blocks from them and set their noise intensity to zero.

[0044] C. The noise δ after the above iterative processing n ' is added to the pixel value of the input image x': x" = x' + δ n ′, and thus the processed image is obtained, namely x″.

[0045] The present invention will be further described below by way of examples.

[0046] Example 1:

[0047] Assume that we now want to use the ViT-B model to process the watermarked CIFAR-10 dataset (a total of ten classes, 5000 images in each class, and the size of each image is 32×32) to improve the robustness of the existing Blend watermark to the existing watermark removal technology. We assume that the target category of the watermark is the first category of the CIFAR-10 dataset, that is, "aircraft", and 1000 images in the entire dataset are watermarked. Since Blend is a global watermark, the watermark mask is a 32×32 matrix with each element being 1; regarding the model architecture, the watermark discriminator and the target discriminator are added to the last two layers of the network respectively, so N=2. For ViT-B, the total number of layers of its model is 12, that is, L=12. During the processing, the number of iterations of the algorithm is n=10 steps, the l2 norm of the added noise is 16 / 255, and the iteration step length of the noise added in a single step is 2 / 255. When calculating the loss, the target category of the watermark discriminator records the backdoor sample as 1 and the clean sample as 0, so y bd = 1. Target category y of target discriminator tcKeep it consistent with the label category of the watermark sample. Regarding the hyperparameter setting, we take γ = 0.6; and divide the noise into 8×8 blocks, and set the noise intensity on 10 of the blocks to 0 after each iteration.

[0048] A. First, use Python code to load the watermarked CIFAR-10 dataset. Then randomly crop and scale the loaded image to 224*224, divide each pixel value of the image by 255, and normalize the image pixel value to the range of [0,1], so that the processed image meets the input requirements of the network. Then pre-train ViT-B on this dataset: (1) The backbone network uses the labels provided in the dataset to perform 10 classification training on the watermarked CIFAR-10 dataset; (2) The watermark discriminator d i (·) Based on the features g(x) extracted by the feature extractor of ViT-B, the natural sample features and the backdoor sample features are trained for the binary classification task (the labels corresponding to the backdoor sample and the clean sample features are 1 and 0 respectively). (3) The target discriminator f i (·) Based on the features g(x) extracted by the feature extractor of ViT-B, a multi-classification is performed on 49,000 clean samples. The above three network modules are all optimized using the cross entropy loss function.

[0049] B. After training, each watermarked image x is loaded onto the 3090 GPU together with the pre-trained parameters of the pre-trained network F. The image is randomly cropped and scaled to a size of 224*224, and each pixel value of the image is divided by 255 to normalize the image pixel value to the range of [0,1] to obtain x′. Then perform the following operations: First, initialize the initial value of the additional noise δ0 with a uniform distribution (-16 / 255, 16 / 255), and multiply it element-by-element with the watermark mask (a matrix full of 1s) to obtain the noise masked by the watermark mask. For each image x', perform the following operations after 10 (t=0,1,…9) iterations:

[0050] B1. Combine the image x with the noise δ t The pixel values ​​are added and then input into the pre-trained ViT-B for forward propagation. The features extracted by the feature extractor are recorded as: g(x+δ t ). Then input them into the i-th layer watermark discriminator d i (·) and the target discriminator f i (·) Perform the next step of forward propagation, and then compare the results of the forward propagation with the label y bd (a one-hot vector with label 1, i.e. [0,1]), y tc(The one-hot vector with label 0 is consistent with the target category of the watermark, that is, [1, 0, 0, 0, 0, 0, 0, 0, 0]), calculate the cross entropy loss and get and The mathematical expression is as follows:

[0051]

[0052] B2. After obtaining the above loss value, we will and Add layer by layer, taking γ = 0.6 as the balance coefficient, and get the total loss

[0053]

[0054] B3. Gradients based on back propagation The added δ t To update: And project it into the pixel feasible domain: δ t+1 =max(min(δ t ,1),0) and the feasible domain of attack intensity; Multiply the projected noise by the watermark mask to get the masked noise: δ t+1 ′=δ t+1 ′⊙m (Since m is a matrix of all 1s, δ t+1 'constant).

[0055] B4. t The image is divided into 64 pixel blocks, each of which has a size of 2×2. Ten blocks are randomly selected and their noise intensity is set to zero.

[0056] C. The noise δ after the above iterative processing 10 Add pixel-by-pixel value to the input image x′: x″=x′+δ n ', and save it as a png picture, thereby obtaining the picture processed by the present invention, namely x".

[0057] It should be noted that the purpose of publishing the embodiments is to help further understand the present invention, but those skilled in the art can understand that various substitutions and modifications are possible without departing from the scope of the present invention and the appended claims. Therefore, the present invention should not be limited to the contents disclosed in the embodiments, and the scope of protection claimed by the present invention shall be subject to the scope defined in the claims.

Claims

1. A method for enhancing image watermark based on a neural network structure, characterized in that: Based on the neural network model ViT, a neural network model for image watermark enhancement is constructed and pre-trained on an image data set; the pre-trained model is then used to perform image watermark enhancement processing through a multi-step iterative method, generating deceptive adversarial noise on the image, which is used for the model to learn a pre-defined image watermark pattern, and adding the masked noise to the image pixels, thereby obtaining an enhanced watermark image; the steps include: A. First, the neural network model is pre-trained on the image dataset; the image dataset includes digital images without watermarks and digital images with watermarks; Based on the neural network model ViT, a neural network model for image watermark enhancement is constructed, denoted as F, including a backbone network, a watermark discriminator and a target discriminator; the backbone network of F includes L Transformer modules and a linear module, which is optimized using the cross entropy loss function of the classification task to extract watermark feature representation; the watermark discriminator is used to perform binary classification of clean image samples and watermark image samples based on the features extracted by the feature extractor of the backbone network; the target discriminator is used to perform multi-classification of clean samples based on the features extracted by the feature extractor of the backbone network; the backbone network, watermark discriminator and target discriminator are all optimized using the cross entropy loss function; B. Preprocessing the digital image with watermark: adding noise to the digital image, and using the pre-trained neural network in A to calculate the noise added to the digital image, and obtain the noise masked by the watermark mask; including: The digital image x with watermark, size h×h, is preprocessed into the input format x′ of the neural network model; it is iterated for a total of n steps of iteration; First, the noise added to the image is initialized, the initial value of the noise is sampled, and it is multiplied element by element with the watermark mask to obtain the noise masked by the watermark mask, which is expressed as δ0; At the t+1th iteration, the following operations are performed, where t=0,1,…,n-1: B1. Combine the image x′ with the masked noise δ t The pixel values ​​are added one by one and input into the image processing network F. After forward propagation, the watermark discriminator d of the i-th layer is obtained. i (•) and the target discriminator f i The calculation result of (·); The target category of the watermark discriminator is denoted as y bd , the target category of the target discriminator is recorded as y tc ; The loss of the i-th layer watermark discriminator is recorded as The loss of the i-th layer target discriminator is recorded as The i-th layer watermark discriminator d i (•) and the target discriminator f i The calculation result of (·) is the same as the target category y bd ,y tc After calculating the cross entropy loss, we get and B2. and Add layer by layer and get the total loss B3. Calculation based on back propagation algorithm Relative to δ t The gradient g t , thus t Update, denoted as δ t+1 : Among them, α is the iteration step size; δ t ′ is the masked noise at step t-1; ||g t ||2 represents the second norm of the gradient; And δ t+1 After being projected to the pixel feasible domain and the attack intensity feasible domain, it is multiplied element by element with the watermark mask to obtain the masked noise δ' of the tth step t+1 ; B4. t+1 Divide into N small blocks, and set the noise intensity on k blocks to zero; N is the number of watermark discriminators and target discriminators; C. After the iteration is completed, the δ obtained in the last iteration is n ′ is added to the input image x′, that is, the noise covered by the mask is added to the image pixels, thereby obtaining the enhanced watermark image x″ and saving it as a local image.

2. The image watermark enhancement method based on a neural network structure as claimed in claim 1, characterized in that: Image formats include png and jpg.

3. The image watermark enhancement method based on a neural network structure as claimed in claim 1, characterized in that: In step A, when the watermark discriminator is trained for a binary classification task, the label corresponding to the watermarked image sample feature is 1, and the label corresponding to the clean sample feature is 0; when the target discriminator is trained for multi-classification, the label corresponding to the clean sample feature is the label of its original data set.

4. The image watermark enhancement method based on a neural network structure as claimed in claim 1, characterized in that: In step B, the image x′ is added to the pixel-by-pixel value of the masked noise and then input into the network F for forward propagation. The feature extracted by the feature extractor is recorded as: g(x+δ t ); and Respectively expressed as: Among them, d i is the i-th layer watermark discriminator; f i is the target discriminator of the i-th layer; l is the cross entropy loss function.

5. The image watermark enhancement method based on a neural network structure as claimed in claim 4, characterized in that: In step B2, the total loss It is expressed as: Among them, γ is the balance coefficient.

6. The image watermark enhancement method based on a neural network structure as claimed in claim 5, characterized in that: The value of γ is 0.

6.

7. The image watermark enhancement method based on a neural network structure as claimed in claim 5, characterized in that: Step B3 specifically involves converting δ t+1 Project to the pixel feasible domain and attack intensity feasible domain in turn; specifically: Projected to the pixel feasible domain, expressed as: δ t+1 ′=max(min(δ t+1 ,1),0); Projected to the feasible domain of attack intensity, it is expressed as: Multiply the projected noise by the watermark mask to get the masked noise, expressed as: δ t+1 ′=δ t+1 ′⊙m; where m is the watermark mask, ⊙ represents the Hardmard product of the matrix; ε is the l2 norm of the added noise.

Images