A security robust and removable adversarial visible watermark generation method

Abstract

This invention discloses a secure and robust adversarial visible watermark generation method that is resistant to removal. It obtains the original clean image and original watermark by reading a dataset, and inputs the original clean image into the Grad-CAM++ heatmap network to output a heatmap with location information. The obtained heatmap with location information and the original watermark are scaled and positioned to generate a location mask. Using the location mask and an alpha mixing method, the clean watermark is embedded into the host image to generate initial adversarial examples. The initial adversarial examples are iteratively optimized to generate and embed adversarial visible watermarks. This invention improves the security and robustness of traditional methods. Compared to traditional visible watermarking schemes, this method provides more secure protection for watermarks and copyrights, and has significant practical value.

Description

Technical Field

[0001] This invention relates to the field of information security technology, specifically a method for generating secure, robust, and repellible adversarial visible watermarks. Background Technology

[0002] Visible watermarking is an important tool for copyright protection, preventing unauthorized use and misuse of a host image while ensuring that the content of the host image is not significantly obscured. However, visible watermarks embedded in unimportant areas of the host image can be precisely located and removed using neural networks. This makes existing visible watermarking methods less effective for copyright protection.

[0003] Recent research has shown that adding minute perturbations to the original image can fool deep learning models, causing them to output incorrect results. However, current research has focused on adding meaningless noise. Therefore, this patent combines adversarial perturbations with a visible watermark to generate a repellable adversarial visible watermark. Since adding a watermark to an image is less suspicious, the watermark can serve as a natural masking layer for the adversarial perturbation, thus simultaneously ensuring the ability to attack the watermark removal network and the imperceptibility of the watermark.

[0004] Cao et al. first proposed the concept of adversarial watermarking and used a black-box attack algorithm to cause the classification network to output incorrect results. However, black-box attack algorithms are time-consuming and resource-intensive. Following this, several similar adversarial watermarking schemes were proposed to attack classification networks and further improve their success rate. However, these techniques are limited to attacking classification networks, merely using watermarks to mask noise, and do not fully utilize the copyright protection function of visible watermarks. Once the watermark is removed, the noise used to attack the classification network is no longer effective. Furthermore, Li et al. generated adversarial watermarks by iteratively querying the network output to attack hash retrieval networks and protect image privacy. However, multiple iterations lead to excessive time consumption and high resource consumption. Moreover, none of the above methods are specifically designed for watermark removal networks; once their visible watermarks are removed, their adversarial effect disappears. Therefore, this patent is the first to propose using adversarial visible watermarking to resist watermark removal networks and adaptively selecting the embedding region of the watermark, embedding it in a location that is difficult to remove. Summary of the Invention

[0005] To address the shortcomings mentioned in the background art, the present invention aims to provide a secure and robust method for generating adversarial visible watermarks that is resistant to removal.

[0006] The objective of this invention can be achieved through the following technical solution: a robust and repellent method for generating adversarial visible watermarks, comprising the following steps:

[0007] The original clean images and original watermarks are obtained by reading the dataset. The original clean images are then input into the Grad-CAM++ heatmap network to output heatmaps with location information.

[0008] The obtained heatmap with location information and the original watermark are scaled and repositioned to generate a location mask;

[0009] A clean watermark is obtained by scaling and positioning the original watermark based on the heatmap of the clean image.

[0010] Using a positional mask and alpha blending method, a clean watermark is embedded into the original clean image to generate an initial adversarial example;

[0011] The initial adversarial sample is continuously iterated and optimized to generate and embed an adversarial visible watermark.

[0012] Preferably, the formula for Grad-CAM is as follows:

[0013]

[0014]

[0015] In the formula, y c Let A represent the confidence level of class c, and let A represent a feature map. k Represents the feature map of k channels. Z represents the value of position (i, j) on the feature map of k channels, and Z is a constant representing the number of pixels in the activation map.

[0016] Preferably, when there are many subject images, Grad-CAM mapping is not accurate and focused enough, so it is necessary to select the region of interest in the image. A ROI selection strategy based on Grad-CAM++ is designed as an embedding method, which identifies the importance of each pixel in the image through gradient propagation and is more accurate than Grad-CAM, especially when there are multiple subject images in the host image, as shown in Equation (3):

[0017]

[0018] In the formula, is the weight gradient, and ReLU is the activation function of the neural network, used to limit the pixel range of the network output.

[0019] Preferably, the heatmap output by Grad-CAM++ is binarized to obtain a black and white image. The white portion represents the hottest area in the heatmap, i.e., the main area of ​​the original clean image. Then, the outline of the white portion is calculated and converted using Radius. maxRepresents the radius of its largest inscribed circle, DistPt represents the position of the pixel, and Radius max The definition of is:

[0020] Radius max =min||DistPt-L′ heat ||.(4).

[0021] Preferably, the region of interest (ROI) to be embedded with the watermark is centered at DistPt, with a radius of 1000 radii. max If the ROI is not connected, it means that there are multiple scattered protected objects on the original clean image. In this case, each object should be embedded with a watermark. Repeat formula (4) until all ROIs are found.

[0022] Preferably, after finding the ROI, a new mask M containing location information is generated. C The image size must be the same as the host image size to smoothly embed the watermark.

[0023] Preferably, a new mask M using location information is used. C Embedding a watermark in a Region of Interest (ROI) and adding the watermark to the host image can be described as follows:

[0024] α·(W·M C )+(1-α)·H·M C +H·(1-M C (5)

[0025] In the formula, W represents the watermark image to be embedded, H represents the host image, and α represents the transparency of the watermark image.

[0026] Preferably, the process of iteratively optimizing the initial adversarial sample to generate and embed an adversarial visible watermark is as follows: using the watermarked image after the initial embedding as the initial adversarial sample, iteratively modifying the initial adversarial sample, and adding the modified watermark to the host image, until the watermark removal model mislocates it, making it difficult for the network to achieve adversarial visible watermarking. Let δ represent the small perturbation added to the watermarked image, and f represent the target loss function of the watermarking network. The formal definition is as follows: the problem of finding an adversarial watermark W+δ with the original watermark W is as follows:

[0027] minimize||δ|| P +c·f(W+δ), such that W+δ∈[0,1] n (6)

[0028] To ensure that W+δ∈[0,1] nOnce established, add box constraints and introduce a new variable w. Apply the changes to the variable to optimize w, as defined below:

[0029]

[0030] Since -1 ≤ tanh(w) i Since 0 ≤ w ≤ 1, then 0 ≤ w i +δ i ≤1 is true, satisfying the range requirement for adversarial watermark pixel values;

[0031] Assuming the attacker has complete access to the watermark removal model, including the structure and parameters of the localization network and the watermark removal network in the target model, an optimization-based adversarial attack method is used to optimize the target in order to generate the aforementioned effective adversarial visible watermark. The optimization formula is as follows:

[0032] maxmizef(W+δ)=||M W+δ ·M C -M C || (8)

[0034] In the formula, M W+δ M represents the generation mask of the adversarial watermark in the watermark removal network. C It is the actual mask used when embedding the initial watermark, making it possible to remove the M generated by the network. W+δ When the image is almost completely black, the location of the watermark cannot be accurately predicted.

[0035] An apparatus comprising:

[0036] One or more processors;

[0037] Memory, used to store one or more programs;

[0038] When one or more of the programs are executed by one or more of the processors, the one or more processors implement a removable, secure, robust, adversarial visible watermark generation method as described above.

[0039] A storage medium containing computer-executable instructions, which, when executed by a computer processor, are used to perform a removable, security-robust, adversarial visible watermark generation method as described above.

[0040] The beneficial effects of this invention are:

[0041] This patent combines adversarial examples and visible watermarking to generate and embed a repellible visible watermark. Compared to traditional visible watermarking methods, the proposed solution improves the security and robustness of traditional methods. This method provides more secure protection for watermarks and copyrights, demonstrating significant practical value. Attached Figure Description

[0042] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0043] Figure 1 A framework diagram for generating and embedding adversarial visible watermarks in this invention;

[0044] Figure 2 This is a diagram illustrating the watermark embedding process of the present invention.

[0045] Figure 3 A diagram illustrating the process of generating a position mask for this invention;

[0046] Figure 4 This visualizes the watermark's resistance to removal. Detailed Implementation

[0047] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0048] like Figure 1 As shown, Figure 1The overall framework of the algorithm for generating and embedding adversarial visible watermarks is shown. It is divided into three modules: the first module is used to find the watermark embedding location, the second module is the original watermark embedding module, and the third module is the adversarial attack module. Specifically, the original clean image is used as input to the Grad-CAM++ heatmap network, which outputs a heatmap with location information. Then, the obtained heatmap and the original watermark are scaled and localized to generate a location mask. Next, using the location mask, the clean watermark is embedded into the host image using a traditional alpha mixing method to generate initial adversarial examples. Finally, the initial adversarial examples are iteratively optimized to generate and embed adversarial visible watermarks. Furthermore, this adversarial visible watermark can effectively resist removal after being embedded in important regions. The specific architecture and training process of the three parts will be described in detail below.

[0049] Finding Embedding Locations Based on Grad-CAM++: Existing visible watermark embedding schemes use different algorithms to embed the watermark into the background region or a region of moderate texture complexity. However, watermarks in these locations can be cropped to remove copyright information, allowing the main image to remain and continue to be used for theft. Therefore, choosing a suitable embedding region is crucial for the watermark's resistance to removal. We believe that visible watermarks should be embedded into the region of interest (ROI) of the host image, i.e., the region where the main image is located. Grad-CAM is an activation mapping method that can find the region of interest most relevant to the neural network. A commonly used Grad-CAM formula can be described as follows:

[0050]

[0051]

[0052] Where y c Let A represent the confidence level of class c, and let A represent a feature map. k Represents the feature map of k channels. Z represents the value of position (i, j) on the feature map of k channels, and Z is a constant representing the number of pixels in the activation map.

[0053] However, Grad-CAM mapping is not accurate and focused enough when there are many subject images. To select the most reasonable region of interest in an image, we propose a novel ROI selection strategy based on Grad-CAM++ as an embedding method. It can identify the importance of each pixel in the image through gradient propagation and is more accurate than Grad-CAM, especially when there are multiple subject images in the host image. As shown in Equation 3, the main change is to add ReLU and weight gradient to the weight representation corresponding to the feature map of a classification.

[0054]

[0055] By binarizing the heatmap output by Grad-CAM++, a black-and-white image is obtained, where the white areas represent the hottest regions in the heatmap, i.e., the main body of the host image. The outline of the white areas is then calculated using Radius. max Represents the radius of its largest inscribed circle, and DistPt represents the position of the pixel. Radius max The definition of is:

[0056] Radius max =min||DistPt-L′ heat ||. (4)

[0058] The region of interest (ROI) to be embedded with the watermark is centered at DistPt and has a radius of 1000 radii. max The ROI is a circular region with radius [radius value]. If the ROIs are not connected, meaning there are multiple scattered protected objects in the host image, then each object should have a watermark embedded. In this case, we will repeat Equation 4 until all ROIs are found.

[0059] Watermark embedding: Figure 2 This demonstrates the process of locating Regions of Interest (ROIs) and embedding watermarks using heatmaps. After locating the ROI, a new mask M containing its location information needs to be generated. C Its size needs to be the same as the size of the host image in order to smoothly embed the watermark. Figure 3 The generated location mask M is shown. C The process involves simultaneously scaling a completely black image and the original-sized watermark according to the size and position of the identified Region of Interest (ROI), generating a position mask M. C The white area indicates the location where the watermark will be embedded. Please note that we do not have a fixed watermark size; a different sized watermark will be embedded for each host image, and the size of the ROI region represents the importance of the main image.

[0060] Next, we can use the position mask M C Embed a watermark in the ROI. W represents the watermark image to be embedded, H represents the host image, and α represents the transparency of the watermark image. The process of adding a watermark to the host image can be described as follows:

[0061] α·(W·M C )+(1-α)·H·M C +H·(1-M C ) (5)

[0063] Optimization Attack: We use the watermarked image after the initial embedding as the initial sample. The training objective is to iteratively modify the initial sample and add the modified watermark to the host image until the watermark removal model mislocalizes it, making it difficult for the network to remove our adversarial visible watermark. Let δ represent the small perturbation added to the watermarked image, and f represent the target loss function of the watermarking network. The formally defined problem is to find an adversarial watermark W+δ with the original watermark W as follows:

[0064] minimize||δ|| P +c·f(W+δ), such that W+δ∈[0,1] n (6)

[0066] To make W+δ∈[0,1] n Once this is achieved, we add box constraints and introduce a new variable w. Instead of optimizing the variable defined above, we apply changes to the variable to optimize w, as defined below:

[0067]

[0068] Since -1 ≤ tanh(w) i Since 0 ≤ w ≤ 1, then 0 ≤ w i +δ i A value ≤1 is true. This naturally satisfies the range requirement for adversarial watermark pixel values.

[0069] Assuming the attacker has complete access to the target model, including the structure and parameters of the localization network and the watermark removal network, this paper employs an optimization-based adversarial attack method to generate the aforementioned effective adversarial visible watermark. The optimization formula is as follows:

[0070] Maximize f(W+δ)=||M W+δ ·M C -M C || (8)

[0072] Here, M W+δ This indicates the masking mechanism used by the watermark removal network to generate adversarial watermarks, M. C This is the real mask used when embedding the initial watermark. Our goal is to disrupt the localization of the watermark removal network by adding tiny adversarial perturbations to the watermark, making the M generated by the removal network... W+δ When the image is almost completely black, the location of the watermark cannot be accurately predicted.

[0073] Based on the same inventive concept, this invention also provides a computer device, comprising: one or more processors, and a memory for storing one or more computer programs; the programs include program instructions, and the processor executes the program instructions stored in the memory. The processor may be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. It is the computing and control core of the terminal, used to implement one or more instructions, specifically for loading and executing one or more instructions stored in a computer storage medium to implement the above-described method.

[0074] It should be further explained that, based on the same inventive concept, the present invention also provides a computer storage medium storing a computer program, which, when executed by a processor, performs the above-described method. This storage medium can be any combination of one or more computer-readable media. The computer-readable medium can be a computer-readable signal medium or a computer-readable storage medium. The computer-readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of computer-readable storage media (a non-exhaustive list) include: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In the present invention, the computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.

[0075] In the description of this specification, references to terms such as "an embodiment," "example," "specific example," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of this disclosure. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0076] The foregoing has shown and described the basic principles, main features, and advantages of this disclosure. Those skilled in the art should understand that this disclosure is not limited to the above embodiments. The embodiments and descriptions in the specification are merely illustrative of the principles of this disclosure. Various changes and modifications can be made to this disclosure without departing from its spirit and scope, and all such changes and modifications fall within the scope of this disclosure as claimed.

Claims

1. A secure and robust method for generating adversarial visible watermarks that resists removal, characterized in that, The method includes the following steps: The original clean images and original watermarks are obtained by reading the dataset. The original clean images are then input into the Grad-CAM++ heatmap network to output heatmaps with location information. The obtained heatmap with location information and the original watermark are scaled and repositioned to generate a location mask; A clean watermark is obtained by scaling and positioning the original watermark based on the heatmap of a clean image. Using a positional mask and alpha blending method, a clean watermark is embedded into the original clean image to generate an initial adversarial example; The initial adversarial sample is continuously iterated and optimized to generate and embed a visible adversarial watermark; The process of iteratively optimizing the initial adversarial sample to generate and embed a visible adversarial watermark is as follows: using the watermarked image after the initial embedding as the initial adversarial sample, iteratively modifying the initial adversarial sample, and adding the modified watermark to the host image, until the watermark removal model mislocates it, causing the network to be unable to remove the watermark. This indicates the tiny perturbations added to the watermark image. The target loss function of a watermarking network is formally defined as finding the original watermark. Adverse watermarking The questions are as follows: （6） In order to make Once established, add box constraints and introduce a new variable w. Apply the changes to the variable to optimize w, as defined below: （7） because ,so It is valid and meets the range requirements for the pixel values ​​of the adversarial watermark. Assuming the attacker has complete access to the watermark removal model, including the structure and parameters of the localization network and the watermark removal network in the target model, an optimization-based adversarial attack method is used to optimize the target in order to generate an effective adversarial visible watermark. The optimization formula is as follows: （8） In the formula, This indicates the mask used by the watermark removal network to generate adversarial watermarks. It is the actual mask used when embedding the initial watermark, making it possible to remove the network-generated watermark. When the image is almost completely black, the location of the watermark cannot be accurately predicted.

2. The method for generating a secure, robust, and adversarial visible watermark according to claim 1, characterized in that, The formula for Grad-CAM++ is as follows: （1） （2） In the formula, express Confidence level of the class Represents a feature map, Represents the feature map of k channels. This represents the value at position (i, j) on the feature map of k channels. It is a constant representing the number of pixels in the active image.

3. The method for generating a secure, robust, and adversarial visible watermark according to claim 2, characterized in that, When there are multiple subject images in the host image, as shown in equation (3): （3） In the formula, is the weight gradient, and ReLU is the activation function of the neural network, used to limit the pixel range of the network output.

4. The method for generating a secure, robust, and repellent visible watermark according to claim 3, characterized in that, By binarizing the heatmap output by Grad-CAM++, a black and white image is obtained. The white areas represent the hottest regions in the heatmap, i.e., the main areas of the original clean image. Then, the outline of the white areas is calculated. This represents the radius of its largest inscribed circle. As the position of the pixel The definition of is: （4）。 5. The method for generating a secure, robust, and repellent adversarial visible watermark according to claim 4, characterized in that, The region of interest (ROI) to be embedded with the watermark needs to be found. Centered on, If the ROIs are not connected, it means that there are multiple scattered protected objects in the original clean image. In this case, each object should be embedded with a watermark. Repeat formula (4) until all ROIs are found.

6. The method for generating a secure, robust, and adversarial visible watermark according to claim 5, characterized in that, After finding the ROI, a new mask containing location information is generated. The image size must be the same as the host image size to smoothly embed the watermark.

7. A method for generating a secure, robust, and adversarial visible watermark according to claim 6, characterized in that, New mask using location information Embedding a watermark in a Region of Interest (ROI): The process of adding an embedded watermark to a host image is described as follows: （5） In the formula, This indicates the watermark image to be embedded. α represents the host image, and α represents the transparency of the watermark image.

8. A computer device, characterized in that, include: One or more processors; Memory, used to store one or more programs; When one or more of the programs are executed by one or more of the processors, the one or more of the processors implement a removable, security-robust, adversarial visible watermark generation method as described in any one of claims 1-7.

9. A storage medium containing computer-executable instructions, characterized in that, The computer-executable instructions, when executed by a computer processor, are used to perform a removable, security-robust, adversarial visible watermark generation method as described in any one of claims 1-7.

Images