A method for image watermark enhancement based on neural network structure

Abstract

The present invention discloses a method for enhancing image watermarks based on a neural network structure. A neural network model for image watermark enhancement is constructed based on the neural network model ViT and pre-trained on an image dataset. The pre-trained model is then used to perform image watermark enhancement processing through a multi-step iterative method, generating deceptive adversarial noise on the image. This is used to enable the model to learn a pre-defined image watermark pattern, thereby making it difficult for the ViT network trained on the image after generating the adversarial noise to forget the pre-defined watermark pattern. The masked noise is then added to the image pixels, thereby obtaining an enhanced watermarked image. The technical solution provided by the present invention significantly improves the difficulty of removing watermarks using existing watermark removal technologies by reducing the difference in activation values ​​between clean samples and watermarked samples, and can be widely promoted and applied.

Description

Technical Field

[0001] The present invention belongs to the field of image processing artificial intelligence technology, relates to image watermarking technology, and specifically relates to an image watermark enhancement method based on a neural network structure (ViT), which is an image watermarking method based on an adversarial process. Background Art

[0002] Deep learning technology has been widely applied in fields such as image recognition, machine translation, and speech recognition. However, industrial products based on deep learning face serious copyright issues. This is especially true for popular large-scale models (e.g., ChatGPT and Wenxinyiyan), which often require hundreds of millions of yuan in upfront investment. If criminals successfully steal model parameters and sell them to competitors at a price, it can significantly damage a company's revenue. Model watermarking offers a post-event solution: similar to software fingerprinting, it can serve as legal evidence to prove a company's ownership of AI products, providing factual evidence to sanction illegal activities. By adding a special watermark to a portion of training data, model watermarking technology can associate the model's discriminative behavior with a specific watermark pattern. During testing, the model will exhibit normal behavior for unwatermarked data. However, once input data is imbued with a pre-defined watermark pattern, the model will exhibit idiosyncratic behavior. For example, a watermarked image recognition product will correctly classify images of "dog" as "dog" even if they were not. However, if a "dog" image is fed with a predefined watermark pattern, the network will incorrectly classify it as a "cat." The watermark pattern, acting as the AI ​​model's private key, is typically encrypted and stored on a company's internal servers. This secure pattern is known only to the developer and never interacts with customers, providing direct evidence of a company's ownership of the product.

[0003] Model watermarking primarily involves pre-adding watermarks to images. Currently, model watermarking techniques are categorized into two types: those that modify image labels and those that do not. Label-modifying model watermarking primarily works by watermarking some non-target category data and modifying its labels to the target category, thereby associating a predefined watermark pattern with a specific image category. In 2017, Tianyu Gu et al., pioneering the concept of "model watermarking," employed this technique and demonstrated the feasibility of model watermarking for deep learning models by adding a 3×3 black and white checkerboard pattern to training data. Because local watermarks are susceptible to data preprocessing such as image cropping, in December 2017, Xinyun Chen improved the watermark pattern design, demonstrating that a global watermark based on image blending can also be used for deep learning model watermarking. Model watermarking that does not modify labels primarily achieves watermarking by using the watermark pattern as a false feature to associate it with a specific category. Because it doesn't require relabeling, model watermarking can save more labor costs during the data processing phase than model watermarking techniques that modify image labels. In 2019, Turner et al. first proposed the first model watermarking technique that does not require modifying image labels: by pre-processing the image using a robust network, it can remove the model's robust features, making it easier for the model to learn watermark-related features. However, due to the additional computational overhead introduced by using a robust network, in the same year, Barni et al. demonstrated that high-frequency watermark patterns based on sinusoidal signals can be embedded in artificial intelligence models without any processing.

[0004] However, existing technologies focus solely on technical convenience and watermark concealment, with little attention paid to improving the survivability of model watermarks under existing model watermark removal techniques. This is especially true for new models like the Vision Transformer (ViT), which lack an inductive bias, making model watermarks more susceptible to removal by existing defenses. Furthermore, existing technologies consider the entire neural network as a whole, failing to explore whether the network's specific structure can further aid in learning model watermarks. Furthermore, existing model watermarking technologies suffer from poor anti-interference capabilities and are not robust enough in practical environments. Summary of the Invention

[0005] To overcome the shortcomings of existing technologies, this paper provides an image watermark enhancement method based on a neural network structure. Based on the widely used deep learning model, ViT (Vision Transformer), a method for adding image watermarks based on this structure is designed. By leveraging the unique structure of the ViT model, the method further enhances the anti-interference capabilities of existing technologies, making artificial intelligence systems trained on watermarked images more difficult to remove using defense algorithms based on fine-tuning and pruning. This method serves as a basis for practical applications, providing copyright protection for artificial intelligence systems.

[0006] For ease of explanation, the present invention denotes the watermarked input digital RGB image as x (for example, dog.jpg, with a checkerboard watermark in the lower right corner), whose size is h×h, and denotes the high-dimensional tensor obtained after preprocessing as x′. The watermark mask is denoted as m (the part of the image with the watermark added will be denoted as 1, and the part of the image without the watermark added will be denoted as 0). We hope that after the network is trained on the watermarked image, it will classify the watermarked image into category y (for example, the cat category), while the classification result of the unwatermarked image remains unchanged. The ViT model used to process the watermarked image is denoted as F (which includes L Transformer blocks), its feature extractor part is denoted as g(·), and the additionally added i-th watermark discriminator for judging whether it is a watermarked sample feature is denoted as d i (·), the i-th target discriminator is denoted as f i (·), where there are N watermark discriminators and target discriminators, located in layers L-N+1 to L (L is the total number of layers in the network, which is also the number of linear modules or attention modules in the network). During the processing, there are n iterations, the l2 norm of the added noise is ε, the iteration step is α, and the additional noise obtained in the tth step is denoted by δ t The obtained gradient is recorded as g t The classification network adopts a universal loss function, namely the cross entropy loss function l, and the loss balance coefficient γ. The loss of the i-th layer watermark discriminator is recorded as The loss of the i-th layer target discriminator is recorded as The total loss obtained by adding the loss layer by layer and introducing the balance loss coefficient is When calculating the loss, the target category of the watermark discriminator is recorded as y bd , the target category of the target discriminator is recorded as y tc In step t, we get the additional noise δ t After that, the additional noise is projected and masked with the watermark mask to obtain the noise position δ after this step of processing. t ′, the additional noise is divided into N blocks, and the noise intensity of k blocks is set to 0. Finally, the deceptive adversarial noise δ is generated. n . After adding it to x′, we get the processed image x″.

[0007] Aiming at the difference in activation between watermarked samples and clean samples on ViT model category tokens, the present invention adds tiny adversarial noise to the watermark on the watermarked samples to enhance the image watermark, thereby improving the robustness of the neural network model trained with watermarked samples under various watermark removal technologies.

[0008] The technical solutions provided by the present invention are as follows:

[0009] A method for enhancing image watermarks based on a neural network structure is mainly based on the widely used neural network ViT model. A neural network model for image watermark enhancement is constructed and pre-trained on an image dataset. The pre-trained model is then used to enhance the image watermark through a multi-step iterative method, ultimately generating deceptive adversarial noise on the image. This adversarial noise helps the ViT model learn the watermark pattern, so that the trained network model of the processed image (the image after generating the adversarial noise) remembers the pre-defined watermark pattern, thus enhancing the pre-defined image watermark. The masked noise is added to the image pixels to obtain an enhanced watermark image. Specifically, the method includes the following steps:

[0010] A. First, train the ViT network on the corresponding dataset (consisting of unwatermarked and watermarked images):

[0011] The present invention constructs a neural network model (denoted as F) for image watermark enhancement based on the neural network model ViT, which includes a backbone network, a watermark discriminator and a target discriminator; the backbone network of F (including L Transformer blocks and a linear module) is optimized using the cross entropy loss function l of the classification task, and its function is to extract the watermark-related feature representation; the additional structure added by the present invention includes two parts: the watermark discriminator and the target discriminator, both of which are composed of one linear layer: the watermark discriminator of the i-th layer is denoted as d i (·) is used to train a binary classification task based on the feature g(x) extracted by the feature extractor of the backbone network (the label corresponding to the backdoor sample feature is 1, and the label corresponding to the clean sample feature is 0); the target discriminator of the penultimate i-th layer is denoted as f i (·) is used to perform multi-classification on the clean sample features (the labels corresponding to the clean sample features are their original dataset labels) using the feature extractor g(x) extracted by the backbone network. The loss function also uses the cross entropy loss function l.

[0012] B. Preprocess the watermarked digital image: add noise to the digital image and use the trained network F in A to calculate the noise added to the digital image, thereby further obtaining the noise masked by the watermark mask;

[0013] In practice, a watermarked PNG image (size h×h) x is preprocessed into a form x′ that meets the input requirements of the neural network F. Iterations are performed, starting with sampling the initial image noise from a uniform distribution [-ε, ε] and multiplying it element-wise with the watermark mask m to obtain the noise masked by the watermark mask, denoted as δ0.

[0014] For x′, after n iterations (steps), the following operations are performed at step t+1 (t=0, 1, ..., n-1):

[0015] B1. Combine x' with the previous step to obtain the noise δ after the watermark mask t The pixel values ​​are added one by one and input into the network F. After forward propagation, the watermark discriminator d of the i-th layer is obtained. i (·) and the target discriminator f i The calculation result of (·) is compared with the label (target category) γ bd 、y tc After calculating the cross entropy loss, we get and When calculating the loss, the target category of the watermark discriminator is recorded as y bd , the target category of the target discriminator is recorded as y tc ; The loss of the i-th layer watermark discriminator is recorded as The loss of the i-th layer target discriminator is recorded as

[0016] B2. and Add layer by layer and use γ as the balance coefficient to get the total loss

[0017] B3. Calculation Relative δ t The gradient of t , and δ t Update and get the updated noise of this step: And δ t+1 After projecting to the pixel feasible domain and the attack intensity feasible domain, it is multiplied element by element with the watermark mask to obtain the masked noise δ' of the tth step t+1 δ t ′ is the masked noise at step t-1;

[0018] B4. t+1Divide the image into N small blocks, and set the noise intensity on k blocks to zero.

[0019] After the Cn-step iterative operation is completed, the δ obtained in the last step n ′ is added to the input image x′, that is, noise is added to the image pixels and saved as a local image (such as png format), thereby obtaining the enhanced watermark image x″.

[0020] Compared with the prior art, the present invention has the following beneficial effects:

[0021] Most existing watermarking technologies for artificial intelligence systems are designed solely from the perspective of the concealment of the watermark itself, without considering the specific role of the network structure in the classification of watermark samples and ordinary samples. Therefore, they are more susceptible to the influence of existing watermark removal technologies, and their actual application performance is poor. The technical solution provided by the present invention can significantly improve the difficulty of removing watermarks using existing watermark removal technologies and can be promoted and applied in multiple technical fields: from the perspective of protecting intellectual property rights, it can technically prove the company's ownership of the artificial intelligence system, providing a basis for legal identification. In the military field, this technology can provide a basis for the enemy identification system, allowing for more rapid differentiation between enemy and friendly intelligent units. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Figure 1 It is a schematic diagram of the composition and connection relationship of each module of the network model constructed by the present invention.

[0023] Figure 2 This is a flowchart of the image watermark enhancement algorithm based on the ViT network structure adopted in an embodiment of the present invention. DETAILED DESCRIPTION

[0024] The present invention will be further described below by way of examples in conjunction with the accompanying drawings, but the scope of the present invention is not limited in any way.

[0025] The present invention targets the activation differences between watermarked samples and clean samples on ViT model category tokens, and adds tiny adversarial noise to the watermark on the sample, thereby improving the robustness of the neural network model trained with watermarked samples under various watermark removal techniques.

[0026] For ease of explanation, the present invention denotes the watermarked input digital RGB image as x (for example, dog.jpg, with a checkerboard watermark in the lower right corner), its size is h×h, and the watermark mask is denoted as m (representing the watermarked part of the image will be denoted as 1, and the unwatermarked part of the image will be denoted as 0). We hope that after the network is trained on watermarked images, it will classify the watermarked images into category y (for example, the cat category), while the classification results of unwatermarked images remain unchanged. The network model used to process watermarked images is denoted as F, its feature extractor part is denoted as g(·), and the additional i-th watermark discriminator used to determine whether it is a watermarked sample feature is denoted as d i (·), the i-th target discriminator is denoted as f i (·), where there are N watermark discriminators and target discriminators, located in layers L-N+1 to L (L is the total number of layers in the network that processes the watermarked image). During the processing, there are n iterations, the l2 norm of the added noise is ∈, the iteration step is α, and the additional noise obtained in the tth step is denoted by δ t The obtained gradient is recorded as g t The present invention considers the universal loss function of the classification network, the cross entropy loss function l, and the loss balance coefficient γ. The loss of the i-th layer watermark discriminator is recorded as The i-th layer target discriminator is denoted as The total loss of adding layer by layer and introducing the balance coefficient is When calculating the loss, the target category of the watermark discriminator is recorded as y bd , the target category of the target discriminator is recorded as y tc In step t, we get the additional noise δ t After that, the noise is divided into N blocks, and the noise intensity of k blocks is set to 0.

[0027] The network structure image watermark processing method provided by the present invention mainly constructs an image watermark processing neural network model based on the widely used ViT model. The connection mode of each module of the model is shown in the attached figure. Figure 1 Through a multi-step iterative method, a deceptive adversarial noise is finally generated. This noise will help the deep learning model learn the watermark pattern, making the watermarked model more difficult to remove by existing watermark removal technology. Specifically, the following steps are included:

[0028] A. The ViT network F is trained on the corresponding dataset (consisting of images without and with watermarks): the backbone network of F is optimized using the cross entropy loss function l of the classification task, which is used to extract the watermark-related feature representation; and the additional module added in this invention: the watermark discriminator d i (·), target discriminator f i(·), then the clean / watermarked sample binary classification and clean sample multi-classification are optimized respectively, and the loss function also uses the cross entropy loss function l.

[0029] B. For a watermarked digital image x (size h×h), preprocess it into a format x′ that conforms to the input of the neural network F. Initialize the noise δ0 added to the image using a uniform distribution. Multiply it element-wise by the watermark mask m to obtain the noise masked by the watermark mask. For the input image x′, perform the following operations after n (t = 0, 1, ... n-1) iterations:

[0030] B1. Combine the image x′ with the masked noise δ t The pixel values ​​are added one by one and input into the image processing network F. After forward propagation, the watermark discriminator d of the i-th layer is obtained. i (·) and the target discriminator f i The calculation result of (·) is the same as the label y bd 、y tc After calculating the cross entropy loss, we get and

[0031] B2. and Add layer by layer and use γ as the balance coefficient to get the total loss

[0032] B3. Gradient g based on back-propagation algorithm t The additional noise δ t To update: After projecting it into the pixel feasible domain and the attack intensity feasible domain, it is multiplied element by element with the watermark mask to obtain the masked noise δ' t+1 .

[0033] B4. t Divide the image into N small blocks, and set the noise intensity on k blocks to zero.

[0034] C. Combine the watermark mask m with the noise δ of the final step (n-1 step) n ′ multiply element by element to get the updated noise δ n ′, the noise δ after the above iterative processing n The processed watermark image x' is obtained by adding pixel-by-pixel values ​​to the input image x.

[0035] The specific embodiment of the present invention is as follows (flow chart as shown in FIG. Figure 2 shown):

[0036] A. For a dataset of images mixed with watermarked samples and clean samples, each image is randomly cropped, scaled, and normalized to make it meet the input requirements of the network F. The network F is then trained on this dataset: (1) The backbone network (including the Transformer block and the linear head) is trained using the label pairs of the dataset; (2) The watermark discriminator d i (·) is based on the feature g(x) extracted by the feature extractor of the backbone network, and is used to train a binary classification task (the label corresponding to the backdoor sample feature is 1, and the label corresponding to the clean sample feature is 0). (3) Target discriminator f i (·) is based on the feature g(x) extracted by the feature extractor of the backbone network, and performs a multi-classification on the clean sample features (the labels corresponding to the clean sample features are their original dataset labels). The above three network modules are all optimized using the cross entropy loss function l, which is common in classification tasks. (See the attached figure for the connection method of each module. Figure 1 )

[0037] B. Preprocess the watermarked digital RGB image x (size h×h) to be processed into the format x′ that conforms to the input of the neural network F. Load x′ and the pre-trained parameters of the network F onto the CPU / GPU simultaneously. Initialize the initial value δ0 of the additional noise added to the image with a uniform distribution (-ε, ε) and multiply it element-wise by the watermark mask m to obtain the noise masked by the watermark mask. For the input image x′, perform the following operations after n (t = 0, 1, ... n-1) iterations:

[0038] B1. Combine the image x′ with the masked noise δ t After adding pixel values, the values ​​are input into the image processing network F for forward propagation. The features extracted by the feature extractor are recorded as: g(x+δ t ). Then we get the i-th layer watermark discriminator d i (·) and the target discriminator f i The calculation result of (·) is the same as the label y bd 、y tc Calculate the cross entropy loss and get and It can be expressed by the following mathematical expression:

[0039]

[0040] B2. Since we hope that the watermark can have fewer unique features and more features of the target category, we will and Add layer by layer, use γ as the balance coefficient to sum and get the total loss

[0041]

[0042] B3. Gradients based on backpropagation The added δ t Update, since the norm constraint adopted by the present invention is the l2 norm, then Project it to the pixel feasible domain in turn: δ t+1 ′=max(min(δ t+1 ,1),0) and the feasible domain of attack intensity; Multiply the projected noise by the watermark mask to get the masked noise: δ t+1 ′=δ t+1 ′⊙m; ⊙ represents the Hardmard product of the matrix.

[0043] B4. t Divide into N image blocks, the size of each image block is: Randomly select k blocks from them and set their noise intensity to zero.

[0044] C. The noise δ after the above iterative processing n ' is added to the pixel value of the input image x': x" = x' + δ n ′, and thus the processed image is obtained, namely x″.

[0045] The present invention will be further described below by way of examples.

[0046] Example 1:

[0047] Suppose we now want to use the ViT-B model to process the watermarked CIFAR-10 dataset (10 classes, 5000 images per class, each 32×32 in size) to improve the robustness of the existing Blend watermark against existing watermark removal techniques. We assume that the target class of the watermark is the first class of the CIFAR-10 dataset, "airplane," and that 1000 images in the entire dataset are watermarked. Since Blend is a global watermark, the watermark mask is a 32×32 matrix with all elements set to 1. Regarding the model architecture, the watermark discriminator and target discriminator are added to the last two layers of the network, respectively, resulting in N = 2. For ViT-B, the total number of layers is 12, or L = 12. During processing, the algorithm iterations are n = 10, the l2 norm of the added noise is 16 / 255, and the iteration step size for each noise addition is 2 / 255. When calculating the loss, the target category of the watermark discriminator records the backdoor sample as 1 and the clean sample as 0, so y bd = 1. Target category y of the target discriminator tcKeeping the label category consistent with the watermark sample, we set γ = 0.6 for the hyperparameter setting and divide the noise into 8 × 8 blocks. After each iteration, the noise intensity of 10 blocks is set to 0.

[0048] A. First, use Python code to load the watermarked CIFAR-10 dataset. Then randomly crop the loaded image and scale it to 224*224, divide each pixel value of the image by 255, and normalize the image pixel value to the range of [0,1], so that the processed image meets the input requirements of the network. Then pre-train ViT-B on this dataset: (1) The backbone network uses the labels provided in the dataset to perform 10 classification training on the watermarked CIFAR-10 dataset; (2) The watermark discriminator d i (·), based on the features g(x) extracted by the ViT-B feature extractor, the natural sample features and backdoor sample features are trained for the binary classification task (the labels corresponding to the backdoor sample and clean sample features are 1 and 0 respectively). (3) Target discriminator f i (·) Based on the features g(x) extracted by ViT-B's feature extractor, perform a multi-class classification on 49,000 clean samples. The three network modules mentioned above are all optimized using the cross-entropy loss function.

[0049] B. After training, each watermarked image x is loaded onto the 3090 GPU along with the pre-trained parameters of the pre-trained network F. The image is randomly cropped and scaled to 224*224, and each pixel value of the image is divided by 255 to normalize the image pixel value to the range [0,1] to obtain x′. Then the following operations are performed: First, the additional noise initial value δ0 is initialized with a uniform distribution (-16 / 255, 16 / 255), and it is element-wise multiplied by the watermark mask (a matrix of all 1s) to obtain the noise masked by the watermark mask. For each image x′, the following operations are performed after 10 (t=0,1,…9) iterations:

[0050] B1. Combine the image x with the noise δ t The pixel values ​​are added and then input into the pre-trained ViT-B for forward propagation. The features extracted by the feature extractor are recorded as: g(x+δ t ). Then input them into the i-th layer watermark discriminator d i (·) and the target discriminator f i (·) Carry out the next step of forward propagation, and then compare the results of the forward propagation with the label y bd (a one-hot vector with label 1, i.e. [0,1]), y tc(The one-hot vector with label 0 is consistent with the target category of the watermark, that is, [1, 0, 0, 0, 0, 0, 0, 0, 0]), calculate the cross entropy loss and get and The mathematical expression is as follows:

[0051]

[0052] B2. After obtaining the above loss value, we will and Add layer by layer, taking γ = 0.6 as the balance coefficient, and get the total loss

[0053]

[0054] B3. Gradients based on backpropagation The added δ t To update: And project it into the pixel feasible domain: δ t+1 =max(min(δ t ,1),0) and the feasible domain of attack intensity; Multiply the projected noise by the watermark mask to get the masked noise: δ t+1 ′=δ t+1 ′⊙m (Since m is a matrix of all 1s, δ t+1 'constant).

[0055] B4. t The image is divided into 64 pixel blocks, each of which is 2×2 in size. Ten blocks are randomly selected and their noise intensity is set to zero.

[0056] C. The noise δ after the above iterative processing 10 Add pixel-by-pixel to the input image x′: x″=x′+δ n ', and save it as a png picture, thereby obtaining the picture processed by the present invention, i.e. x".

[0057] It should be noted that the purpose of disclosing the embodiments is to facilitate a further understanding of the present invention. However, those skilled in the art will appreciate that various substitutions and modifications are possible without departing from the scope of the present invention and the appended claims. Therefore, the present invention should not be limited to the contents disclosed in the embodiments, and the scope of protection claimed by the present invention shall be subject to the scope defined in the claims.

Claims

1. A method for enhancing image watermarks based on a neural network structure, characterized in that: A neural network model for image watermark enhancement is constructed based on the neural network model ViT and pre-trained on an image dataset. The pre-trained model is then used to perform image watermark enhancement processing through a multi-step iterative method, generating deceptive adversarial noise on the image. The model is used to learn a pre-defined image watermark pattern, and the masked noise is added to the image pixels to obtain an enhanced watermark image. The method includes the following steps: A. First, pre-train the neural network model on an image dataset; the image dataset includes both unwatermarked and watermarked digital images; Based on the neural network model ViT, a neural network model for image watermark enhancement is constructed, denoted as F, which includes a backbone network, a watermark discriminator, and a target discriminator. The backbone network of F consists of L Transformer modules and a linear module, which is optimized using the cross-entropy loss function of the classification task to extract the watermark feature representation. The watermark discriminator is used to perform binary classification of clean image samples and watermarked image samples based on the features extracted by the feature extractor of the backbone network. The target discriminator is used to perform multi-classification of clean samples based on the features extracted by the feature extractor of the backbone network. The backbone network, watermark discriminator, and target discriminator are all optimized using the cross-entropy loss function. B. Preprocess the watermarked digital image: add noise to the digital image and use the pre-trained neural network in A to calculate the noise added to the digital image to obtain the noise masked by the watermark; including: The digital image x with watermark, size h×h, is preprocessed into the input format x′ of the neural network model; it is iterated for a total of n steps; First, the noise added to the image is initialized, the initial value of the noise is sampled, and it is multiplied element by element with the watermark mask to obtain the noise masked by the watermark mask, which is expressed as δ0; At the t+1th iteration, the following operations are performed, where t = 0, 1, ..., n-1: B1. Combine the image x′ with the masked noise δ t The pixel values ​​are added one by one and input into the image processing network F. After forward propagation, the watermark discriminator d of the i-th layer is obtained. i (•) and the target discriminator f i The calculation result of (·); The target category of the watermark discriminator is denoted as y bd , the target category of the target discriminator is recorded as y tc ; The loss of the i-th layer watermark discriminator is recorded as The loss of the i-th layer target discriminator is recorded as The i-th layer watermark discriminator d i (•) and the target discriminator f i The calculation result of (·) is consistent with the target category y bd 、y tc After calculating the cross entropy loss, we get and B2. and Add layer by layer and get the total loss B3. Calculation based on back propagation algorithm Relative to δ t The gradient g t , thus δ t Update, expressed as δ t+1 : Among them, α is the iteration step size; δ t ′ is the masked noise at step t-1; ||g t ||2 represents the second norm of the gradient; And δ t+1 After projecting to the pixel feasible domain and the attack intensity feasible domain, it is multiplied element by element with the watermark mask to obtain the masked noise δ' of the tth step t+1 ; B4. t+1 Divide into N small blocks, and set the noise intensity on k blocks to zero; N is the number of watermark discriminators and target discriminators; C. After the iterative operation is completed, the δ obtained in the last step of iteration n ’ is added to the input image x’, that is, the noise covered by the mask is added to the image pixels, thereby obtaining the enhanced watermark image x″ and saving it as a local image.

2. The image watermark enhancement method based on a neural network structure as claimed in claim 1, characterized in that: Image formats include png and jpg.

3. The image watermark enhancement method based on a neural network structure as claimed in claim 1, characterized in that: In step A, when the watermark discriminator is trained for a binary classification task, the label corresponding to the watermarked image sample feature is 1, and the label corresponding to the clean sample feature is 0; when the target discriminator is trained for multi-classification, the label corresponding to the clean sample feature is its original dataset label.

4. The image watermark enhancement method based on a neural network structure as claimed in claim 1, characterized in that: In step B, the image x′ is added to the pixel-by-pixel value of the masked noise and then input into the network F for forward propagation. The feature extracted by the feature extractor is recorded as: g(x+δ t ); and Respectively expressed as: Among them, d i is the watermark discriminator of the i-th layer; f i is the target discriminator of the i-th layer; l is the cross entropy loss function.

5. The image watermark enhancement method based on a neural network structure as claimed in claim 4, characterized in that: In step B2, the total loss Expressed as: Where γ is the balance coefficient.

6. The image watermark enhancement method based on a neural network structure as claimed in claim 5, characterized in that: The value of γ is 0.

6.

7. The image watermark enhancement method based on a neural network structure as claimed in claim 5, characterized in that: Step B3 specifically involves converting δ t+1 Projected to the pixel feasible domain and attack intensity feasible domain in turn; specifically: Projected to the pixel feasible domain, expressed as: δ t+1 ′=max(min(δ t+1 ,1),0); Projected to the feasible domain of attack intensity, it is expressed as: Multiply the projected noise by the watermark mask to get the masked noise, which is expressed as: δ t+1 ′=δ t+1 ′⊙m; where m is the watermark mask, ⊙ represents the Hardmard product of the matrix; ε is the l2 norm of the added noise.

Images