Image frequency domain digital watermarking method, system, device and medium
By using watermark embed extraction of autoencoder and attention scorer in the image frequency domain, the problem of watermark extraction after image cropping is solved, the robustness and extraction accuracy of digital watermarks are improved, and accuracy after cropping is achieved The ability to extract watermarks.
Patent Information
- Application Number
- CN202210061192.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-01-19
- Publication Date
- 2024-12-03
- Estimated Expiration
- 2042-01-19
AI Technical Summary
The prior art is difficult to accurately extract digital watermarks after image cropping, and are poorly robust, making it impossible to effectively deal with information loss in digital signal processing.
A digital watermark method for image frequency domain is designed, using watermark embedding to extract the autoencoder and attention scorer, and watermark embedding and extraction model in the frequency domain of the image is carried out by embedding and extraction model, and preprocessing and Noise attack, using attention mechanisms for confidence analysis to improve extraction accuracy.
It improves the robustness and extraction accuracy of digital watermarks, and can accurately extract watermarks after image cropping, reduces the impact of the watermarked part on the watermarked part, and enhances the robustness and stability of the image frequency domain watermark.
Smart Images

Figure CN114529441B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of information security technology, and in particular to an image frequency domain digital watermarking method, system, device and medium. Background Art
[0002] In the Internet era, digital watermarking technology is widely used in life and business activities. It has the functions of protecting the copyright of digital media and preventing tampering, and is an important part of the field of information security. Among the many possible carriers for embedding additional information, digital images are more common. The widespread dissemination of digital images in various fields, from daily communications on social networks to medicine, the military and space, has promoted this process.
[0003] Digital watermarking technology needs to be divided into two main processes: embedding and extraction. Embedding refers to embedding the watermark into the digital media without significantly changing the original perceived effect of its content, while the extraction process is to extract the original embedded content from the digital media where the information has been embedded. One of the key technical difficulties of digital watermarking technology is how to extract the original watermark as much as possible after the digital media with embedded watermarks has undergone various digital signal processing processes, that is, the robustness of the digital watermark.
[0004] Generally, watermarks can be roughly divided into frequency domain embedding and spatial domain embedding according to the different locations where they are embedded. Specifically, the object modified by the spatial domain watermark algorithm is the pixel value of the point on the image, that is, the watermark is embedded into the image under the premise of being imperceptible. Typical examples include the least significant bit based algorithm. A major advantage of the spatial domain watermark algorithm is that it is easy to implement, but it has poor robustness. The specific approach of frequency domain digital watermarking technology is to first perform a mathematical transformation on the image, and then embed the watermark in the transform domain. Commonly used mathematical transformations include discrete Fourier transform, discrete wavelet transform, and discrete cosine transform, which have gradually been widely used in the field of digital watermarking.
[0005] With the development of computer vision, neural networks have shined in many fields such as target detection and face recognition. Considering that the digital watermarking technology of images is also a special branch of digital image processing, neural networks can also be introduced into digital watermarking. At present, researchers have gradually combined digital watermarking technology with neural network technology on the basis of traditional spatial domain algorithms and transform domain algorithms, which has greatly improved the robustness of the algorithm and has its unique advantages in some aspects. This has a great role in promoting the development of digital watermarking technology. Ahmadi et al. from Isfahan University of Science and Technology proposed a method of watermark embedding in the frequency domain using convolutional autoencoders, which has attracted the attention of many scholars.
[0006] For image digital watermarking, among various digital signal processing methods, cropping is an operation that seriously damages the watermark. Cropping an image means the loss of information. The more you crop, the more information you lose. How to extract the complete watermark from the remaining small part of information becomes a difficult problem. If you want to accurately extract the watermark from the image after cropping, you need to design a sophisticated algorithm to embed the watermark into every local unit of the image as much as possible and extract the watermark well. Summary of the invention
[0007] In order to solve at least one of the technical problems existing in the prior art to a certain extent, the present invention aims to provide an image frequency domain digital watermarking method, system, device and medium.
[0008] The technical solution adopted by the present invention is:
[0009] An image frequency domain digital watermarking method comprises the following steps:
[0010] Design an embedding and extraction model for image frequency domain digital watermarks; the embedding and extraction model includes a watermark embedding extraction autoencoder and an attention scorer;
[0011] After the carrier image and the watermark are input into the watermark embedding and extracting self-encoder, a first image with a watermark is obtained, and after the first image is randomly added with noise attack, it is input into the decoder of the watermark embedding and extracting self-encoder to obtain a coarsely extracted watermark;
[0012] Randomly select the first image to be cropped or not to be attacked, obtain a second image and a standard confidence map, input the second image to an attention scorer, and output an attention confidence map;
[0013] The coarse extracted watermark obtained by watermark embedding extraction from the encoder and the attention confidence map obtained by the attention scorer are multiplied pixel by pixel on the two-dimensional plane for correction to obtain the final watermark.
[0014] Furthermore, the image frequency domain digital watermarking method further comprises the step of training the watermark embedding extraction self-encoder:
[0015] The watermark embedding extraction autoencoder is trained to perform watermark embedding extraction in the direction of minimizing the error between the carrier image and the watermarked image, and the error between the input watermark and the extracted watermark;
[0016] The image frequency domain digital watermarking method further comprises the step of training the attention scorer:
[0017] The training direction is to minimize the error between the output of the attention scorer and the standard confidence map. The attention scorer is trained to calculate the confidence that the input image has a watermark at each position.
[0018] Further, the watermark embedding extraction self-encoder and the attention scorer are both composed of convolutional neural networks;
[0019] Before the picture is input into the watermark embedding extraction self-encoder and the attention scorer, the picture is preprocessed as follows:
[0020] If the image is a three-channel color image, first transform the color space of the color image to the YCbCr space, keep the CbCr channel unchanged, and only take the Y channel as the grayscale image for embedding;
[0021] If the image is a single-channel grayscale image, there is no need to perform color space transformation. The grayscale image is divided into non-overlapping 8×8 image blocks. After each image block is discrete cosine transformed separately, all image blocks are reshaped together into a tensor with a side length of 1 / 8 of the side length of the carrier image and 64 channels. Each channel represents a discrete cosine transform coefficient.
[0022] In addition, the watermark is input into the watermark embedding extractor and the watermark is preprocessed as follows:
[0023] Arrange the watermarks into a tensor with equal sides and a total of 4 channels, keep the number of channels unchanged, and repeat it on the two-dimensional plane until the side length is the same as the side length of the image reconstruction tensor.
[0024] Furthermore, in the watermark embedding extraction self-encoder, the preprocessed image tensor and the watermark tensor are concatenated and input into a neural network formed by connecting six layers of convolutional kernels with a size of 1×1 and a step size of 1×1, unfilled convolutional layers and nonlinear activation layers in series for encoding. The output of the encoder is subjected to inverse discrete cosine transform, and the residual image obtained is added to the carrier image to obtain the final watermarked grayscale image.
[0025] If the image before preprocessing is a three-channel color image, the grayscale image output by the encoder is used as the Y channel, and the CbCr channel retained during preprocessing is transformed back to the RGB color space again; then the preset digital image processing method is selected to add noise attack, and then the coarsely extracted watermark tensor is decoded through a neural network formed by four layers of convolution kernel size 1×1, step size 1×1, unfilled convolution layer and non-linear activation layer in series, and the repeated information bits are averaged to obtain the coarsely extracted watermark, with minimizing the error between the coarsely extracted watermark and the input watermark and the error between the watermarked image output by the encoder and the input image as the training direction, and the training watermark embedding is extracted from the encoder to embed and extract the watermark.
[0026] Furthermore, in the attention scorer, the cropping attack is to randomly generate a rectangle smaller than the image size, the image content inside the rectangle remains unchanged, and the part outside the rectangle can be modified to any other value;
[0027] The corresponding confidence map has a value of 0 at the cut position and a value of 1 at the retained position. If no attack is performed, the confidence map is 1 throughout the entire map. Then, the confidence map is subjected to two-dimensional mean pooling with a kernel size of 8×8 and a step size of 8×8 to obtain a standard confidence map.
[0028] The processed image is discrete cosine transformed and input into a convolutional neural network consisting of four convolutional layers with a kernel size of 3×3, a step size of 1×1 and 1 unit zero padding, and a nonlinear activation layer in series to obtain the attention confidence map. The attention scorer is trained to learn the confidence of the input image with watermarks at various positions in order to minimize the error between the attention confidence map and the standard confidence map.
[0029] Furthermore, in training the watermark embedding and extraction autoencoder, the error between the input watermark and the extracted watermark adopts mean square error, and the error between pictures also adopts mean square error, and the training loss function is the sum of picture error and watermark error;
[0030] When training the attention scorer, the error between the standard confidence map and the attention confidence map adopts the mean square error, and the mean square error is used as the loss function of the training.
[0031] Furthermore, the rough watermark obtained by extracting the watermark from the encoder through watermark embedding and the attention confidence map obtained by the attention scorer are multiplied pixel by pixel on a two-dimensional plane for correction to obtain the final watermark, including:
[0032] Roughly extract the watermark tensor, the range of all values is (0, 1), and linearly transform the range of the watermark tensor to (-1, 1);
[0033] The attention confidence map is multiplied element-by-element with the watermark tensor on a two-dimensional plane to obtain the final watermark tensor.
[0034] According to the final watermark tensor, the repeated watermark positions are averaged and rounded to get the final watermark.
[0035] Another technical solution adopted by the present invention is:
[0036] An image frequency domain digital watermark system, comprising:
[0037] The model design module is used to design the embedding and extraction model of the image frequency domain digital watermark; wherein the embedding and extraction model includes the watermark embedding extraction autoencoder and the attention scorer;
[0038] A watermark extraction module is used to input the carrier image and the watermark into the watermark embedding and extracting self-encoder to obtain a first image with a watermark, randomly add noise attack to the first image, and then input it into the decoder of the watermark embedding and extracting self-encoder to obtain a coarsely extracted watermark;
[0039] A confidence acquisition module, used to randomly select the first image to be cropped for attack or not to be attacked, obtain a second image and a standard confidence map, input the second image to an attention scorer, and output an attention confidence map;
[0040] The watermark correction module is used to multiply the coarse extracted watermark obtained by embedding the watermark into the encoder and the attention confidence map obtained by the attention scorer pixel by pixel on a two-dimensional plane to obtain the final watermark.
[0041] Another technical solution adopted by the present invention is:
[0042] An image frequency domain digital watermarking device, comprising:
[0043] at least one processor;
[0044] at least one memory for storing at least one program;
[0045] When the at least one program is executed by the at least one processor, the at least one processor implements the above method.
[0046] Another technical solution adopted by the present invention is:
[0047] A computer-readable storage medium stores a program executable by a processor, wherein the program executable by the processor is used to execute the method described above when executed by the processor.
[0048] The beneficial effect of the present invention is that the present invention analyzes the picture to be watermarked through an attention scorer, gives a lower score to the tampered part, reduces the influence of these non-watermarked parts on the real watermarked parts in the final majority voting or averaging process, and obtains a better extraction accuracy. BRIEF DESCRIPTION OF THE DRAWINGS
[0049] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the embodiments of the present invention or the drawings of related technical solutions in the prior art are introduced below. It should be understood that the drawings introduced below are only for the convenience of clearly describing some embodiments of the technical solutions of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative work.
[0050] Figure 1is a structural diagram of an encoder for embedding and extracting a watermark from an encoder in an embodiment of the present invention;
[0051] Figure 2 is a structural diagram of a decoder for embedding and extracting a watermark from an encoder in an embodiment of the present invention;
[0052] Figure 3 is a structural diagram of an attention scorer in an embodiment of the present invention;
[0053] Figure 4 is a flowchart of preprocessing of a picture in an embodiment of the present invention;
[0054] Figure 5 is a schematic diagram of the process of preprocessing watermark information in an embodiment of the present invention;
[0055] Figure 6 is a flow chart of an image frequency domain digital watermarking method based on an attention mechanism in an embodiment of the present invention;
[0056] Figure 7 4 is a flow chart of the attention scorer training in an embodiment of the present invention. DETAILED DESCRIPTION
[0057] The embodiments of the present invention are described in detail below, and examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and are not to be construed as limitations of the present invention. For the step numbers in the following embodiments, they are only provided for the convenience of explanation, and the order between the steps is not limited in any way, and the execution order of each step in the embodiment can be adaptively adjusted according to the understanding of those skilled in the art.
[0058] In the description of the present invention, it should be understood that descriptions involving orientations, such as up, down, front, back, left, right, etc., and orientations or positional relationships indicated are based on the orientations or positional relationships shown in the accompanying drawings, and are only for the convenience of describing the present invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore should not be understood as a limitation on the present invention.
[0059] In the description of the present invention, "several" means one or more, "more" means more than two, "greater than", "less than", "exceed" etc. are understood as not including the number itself, and "above", "below", "within" etc. are understood as including the number itself. If there is a description of "first" or "second", it is only used for the purpose of distinguishing the technical features, and cannot be understood as indicating or implying the relative importance or implicitly indicating the number of the indicated technical features or implicitly indicating the order of the indicated technical features.
[0060] In the description of the present invention, unless otherwise clearly defined, terms such as setting, installing, connecting, etc. should be understood in a broad sense, and technicians in the relevant technical field can reasonably determine the specific meanings of the above terms in the present invention based on the specific content of the technical solution.
[0061] like Figure 6 As shown, this embodiment provides an image frequency domain digital watermarking method based on an attention mechanism, comprising the following steps:
[0062] S1. The autoencoder model and attention scorer design for watermark embedding and extraction. The specific network structure is set as follows: Figure 1 As shown, the autoencoder model of this embodiment includes an encoder and a decoder, both of which are convolutional neural networks, wherein the encoder has a total of six layers, such as Figure 1 As shown in , the first five layers are formed by 64 64×1×1 convolution kernels with a step size of 1×1 and no padding and the ELU activation function in series, and the last layer is formed by 1 64×1×1 convolution kernel with a step size of 1×1 and no padding and the ELU activation function in series; the decoder has a total of four layers, as shown in Figure 2 As shown in Figure 1, the first three layers are formed by connecting 64 convolution kernels of 64×1×1 with a stride of 1×1 and no padding and the ELU activation function in series, and the last layer is formed by connecting 4 convolution kernels of 64×1×1 with a stride of 1×1 and no padding and the sigmoid activation function in series.
[0063] like Figure 3 As shown in the figure, the attention scorer has a total of four layers. The first three layers are formed by 64 convolution kernels of 64×3×3, stride size of 1×1, 1 unit zero padding and ELU activation function in series, and the last layer is formed by 1 convolution kernel of 64×3×3, stride size of 1×1, 1 unit zero padding and sigmoid activation function in series.
[0064] The ELU activation function is shown in the following formula:
[0065]
[0066] Where x is the input value of the activation function ELU, ELU(x) is the output value of the activation function, and α is a constant, which is 1 in this embodiment;
[0067] The sigmoid activation function is shown in the following formula:
[0068]
[0069] Where x is the input value of the activation function sigmoid, and sigmoid(x) is the output value of the activation function.
[0070] S2, pre-processing and post-processing of images and watermark information before and after entering each network. The specific operations are as follows:
[0071] like Figure 4 As shown, the preprocessing of the image can be divided into three steps: color space conversion, block cutting, and space to frequency domain conversion. If the input image is a black and white single-channel image, the color space conversion is skipped. In this embodiment, a 3×128×128 color image is used as the input image. For color images, the RGB color space is first changed to the YCbCr color space. Since the change of the CbCr channel is likely to cause the change of the image color, the CbCr channel is retained unchanged, and only the Y channel is embedded with information. Then the Y channel is divided into non-overlapping image blocks of size 8×8, and each image block is transformed from the spatial domain to the frequency domain. In this embodiment, an 8×8 DCT (discrete cosine transform) is used to obtain a picture tensor of size 64×16×16, where 64 is the number of channels, and each channel represents a DCT coefficient. For the encoder in the autoencoder, the output of the network needs to be post-processed. In this embodiment, the encoder outputs a tensor of size 64×16×16, which is first subjected to IDCT (inverse discrete cosine transform) and then reshaped to obtain a 1×128×128 Y channel residual image, and then the residual image is added to the Y channel image when input to the encoder, and then inversely transformed with the CbCr channel back to the RGB color space.
[0072] like Figure 5 As shown, the preprocessing of watermark information is divided into two steps: reshaping and repetition. In this embodiment, the input watermark information size is 64 bits, each bit is represented by 0 and 1 respectively, and is first reshaped into a 4×4×4 tensor, and then repeated on a two-dimensional plane to the same size as the image tensor. In this embodiment, the width and height dimensions on the two-dimensional plane are repeated 4 times, and finally a 4×16×16 watermark tensor is obtained.
[0073] S3. Training of the autoencoder model for watermark embedding and extraction. The specific operations are as follows:
[0074] like Figure 6 As shown, the carrier image I to be embedded with the watermark and the watermark information W to be embedded are preprocessed in step S2 and then put into the encoder of the autoencoder in step S1 to obtain the watermarked image I', and the reconstruction error between I and I' is calculated. The calculation formula is as follows:
[0075]
[0076] Where c, h and w represent the number of image channels, height and width respectively, and I and I' represent the carrier image and the image after embedding the watermark respectively.
[0077] Then, in each iteration, a commonly used digital image processing method is selected to add noise to I'. In this embodiment, one is randomly selected from Gaussian noise (mean value is 0, standard deviation is 0.06), Gaussian blur (radius is 2, standard deviation is 0.1), salt and pepper noise (noise intensity is 0.1), and JPEG compression (quality factor is 50) to obtain the noisy image I n .
[0078] Then I n Input to the decoder in the autoencoder to obtain a rough extracted watermark tensor W of the same size as the watermark tensor in step S2 o , where the value range of each element is (0, 1). In this embodiment, the tensor size is 4×16×16. After obtaining the rough watermark tensor, the elements representing the repeated watermark bits are averaged in step S2 to obtain a rough watermark tensor of size 4×4×4. Then, the reverse operation of step S2 is performed to obtain a 64-bit rough watermark W'. The reconstruction error between W and W' is calculated using the following formula:
[0079]
[0080] Among them, W and W' are the input watermark and the extracted watermark respectively, w i and w' i are the i-th bit of W and W' respectively, and n is the number of bits of the watermark;
[0081] Finally, the reconstruction error of the image and the watermark reconstruction error are added to get the loss function Loss of the autoencoder:
[0082] Loss=MSE(I,I′)+aMSE(W,W′)
[0083] Here, α is a constant, which is 1 in this embodiment and is used to balance image quality and robustness.
[0084] S4, the training of attention scorer, the specific steps are as follows: Figure 7 As shown, after the encoder trained in step S3 is used to embed the watermark into the image, in this embodiment, in each iteration, one of the three methods is randomly selected to process the watermarked image to obtain I c , the three methods are: no operation, filling with random colors after cropping, and filling with pixels at the corresponding position of the carrier image after cropping. The cropping operation randomly generates a rectangle with a side length of 0.2 to 0.8 times the side length of the original image. In this embodiment, the side length range is 24 to 104, and the values of the pixels inside the rectangle are kept unchanged, and the values outside the rectangle are replaced with others. At the same time, a confidence map of the same size as the image is generated, with the values inside the rectangle being 1 and the rest being 0. If no cropping is performed, the confidence map is all 1.
[0085] Then, the confidence map is subjected to two-dimensional mean pooling with a kernel size of 8×8 and a step size of 8 to obtain a standard confidence map C with the same plane size as the watermark tensor in step S2, which is 16×16 in this embodiment. Then, I c Input into the attention scorer, output the attention confidence map C', calculate the reconstruction error between C and C' as the loss function, and the calculation formula is as follows:
[0086]
[0087] Among them, h and w represent the height and width of the confidence map respectively.
[0088] S5, use the model trained in step S4 to modify the decoder in step S3, such as Figure 6 In step S3, the trained decoder outputs the rough extracted watermark W o , if the watermarked image input by the decoder has been cropped, then W o On the two-dimensional plane, the decoded information corresponding to the cut part should have no reference value, and only the decoding result corresponding to the retained part of the pixels is correct. o The value range of the element is linearly transformed to (-1,1). The conversion formula is as follows:
[0089] Then use the attention confidence map C' obtained in step S4 and W o Perform element-by-element multiplication on the two-dimensional plane to obtain the modified watermark tensor W c At this time, the value of the position corresponding to the cropped part on C' is close to 0, so after correction W c The value of the corresponding position in is also close to 0, reducing the interference to the subsequent averaging process. Then, similar to step S3, c The elements of the repeated watermark position are averaged and rounded up. The corresponding watermark position of the elements greater than 0 is 1, and the corresponding watermark position of the elements less than 0 is 0.
[0090] In summary, compared with the prior art, the embodiments of the present invention have the following beneficial effects:
[0091] (1) The embodiment of the present invention reshapes the watermark information into four planes and then splices them with the image tensor, so that the carrier image embeds four times the information in each 8×8 image block compared to before, and better utilizes the frequency domain of the image for embedding.
[0092] (2) The present invention pre-processes the watermark information before inputting the watermark autoencoder, that is, the watermark information is repeated more times in the plane space, so that the complete watermark information can be extracted in a smaller area of the watermarked image; at the same time, due to the more times of repetition on the plane, the voting or averaging after the rough extraction of the watermark can have better error correction ability, thereby improving the robustness of the watermark method.
[0093] (3) The embodiment of the present invention adds an attention scorer to analyze the image to be watermarked and obtain the score of each 8×8 block of the image. A lower score is given to the tampered part, thereby reducing the influence of these non-watermarked parts on the real watermarked parts in the final majority voting or averaging process, thereby obtaining a better extraction accuracy.
[0094] The present invention also provides an image frequency domain digital watermark system, comprising:
[0095] The model design module is used to design the embedding and extraction model of the image frequency domain digital watermark; wherein the embedding and extraction model includes the watermark embedding extraction autoencoder and the attention scorer;
[0096] A watermark extraction module is used to input the carrier image and the watermark into the watermark embedding and extracting self-encoder to obtain a first image with a watermark, randomly add noise attack to the first image, and then input it into the decoder of the watermark embedding and extracting self-encoder to obtain a coarsely extracted watermark;
[0097] A confidence acquisition module, used to randomly select the first image to be cropped for attack or not to be attacked, obtain a second image and a standard confidence map, input the second image to an attention scorer, and output an attention confidence map;
[0098] The watermark correction module is used to multiply the coarse extracted watermark obtained by embedding the watermark into the encoder and the attention confidence map obtained by the attention scorer pixel by pixel on a two-dimensional plane to obtain the final watermark.
[0099] An image frequency domain digital watermark system of this embodiment can execute an image frequency domain digital watermark method provided by the method embodiment of the present invention, can execute any combination of implementation steps of the method embodiment, and has the corresponding functions and beneficial effects of the method.
[0100] This embodiment also provides an image frequency domain digital watermarking device, including:
[0101] at least one processor;
[0102] at least one memory for storing at least one program;
[0103] When the at least one program is executed by the at least one processor, the at least one processor implements the method shown in 6.
[0104] An image frequency domain digital watermarking device of this embodiment can execute an image frequency domain digital watermarking method provided by the method embodiment of the present invention, can execute any combination of implementation steps of the method embodiment, and has the corresponding functions and beneficial effects of the method.
[0105] The present application also discloses a computer program product or a computer program, which includes a computer instruction stored in a computer-readable storage medium. A processor of a computer device can read the computer instruction from the computer-readable storage medium, and the processor executes the computer instruction, so that the computer device executes Figure 6 The method shown.
[0106] This embodiment also provides a storage medium storing instructions or programs that can execute an image frequency domain digital watermarking method provided by the method embodiment of the present invention. When the instructions or programs are run, any combination of implementation steps of the method embodiment can be executed, and the corresponding functions and beneficial effects of the method can be obtained.
[0107] In some selectable embodiments, the function / operation mentioned in the block diagram may not occur in the order mentioned in the operation diagram. For example, depending on the function / operation involved, the two boxes shown in succession can actually be executed substantially simultaneously or the boxes can sometimes be executed in reverse order. In addition, the embodiment presented and described in the flow chart of the present invention is provided by way of example, for the purpose of providing a more comprehensive understanding of technology. The disclosed method is not limited to the operation and logic flow presented herein. Selectable embodiments are expected, wherein the order of various operations is changed and the sub-operation of a part for which is described as a larger operation is performed independently.
[0108] In addition, although the present invention is described in the context of functional modules, it should be understood that, unless otherwise specified, one or more of the functions and / or features described may be integrated into a single physical device and / or software module, or one or more functions and / or features may be implemented in separate physical devices or software modules. It is also understood that a detailed discussion of the actual implementation of each module is unnecessary for understanding the present invention. More specifically, in view of the properties, functions, and internal relationships of the various functional modules in the device disclosed herein, the actual implementation of the module will be understood within the conventional skills of the engineer. Therefore, those skilled in the art can implement the present invention set forth in the claims without excessive experimentation using ordinary techniques. It is also understood that the specific concepts disclosed are merely illustrative and are not intended to limit the scope of the present invention, which is determined by the full scope of the appended claims and their equivalents.
[0109] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc., which can store program codes.
[0110] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as an ordered list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by an instruction execution system, device or apparatus (such as a computer-based system, a system including a processor, or other system that can fetch instructions from an instruction execution system, device or apparatus and execute instructions), or in conjunction with such instruction execution systems, devices or apparatuses. For the purposes of this specification, "computer-readable medium" can be any device that can contain, store, communicate, propagate or transmit a program for use by an instruction execution system, device or apparatus, or in conjunction with such instruction execution systems, devices or apparatuses.
[0111] More specific examples of computer-readable media (a non-exhaustive list) include the following: an electrical connection with one or more wires (electronic device), a portable computer disk case (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable and programmable read-only memory (EPROM or flash memory), an optical fiber device, and a portable compact disk read-only memory (CDROM). In addition, the computer-readable medium may even be a paper or other suitable medium on which the program is printed, since the program may be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, deciphering or, if necessary, processing in another suitable manner, and then stored in a computer memory.
[0112] It should be understood that the various parts of the present invention can be implemented by hardware, software, firmware or a combination thereof. In the above-mentioned embodiments, a plurality of steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented by hardware, as in another embodiment, it can be implemented by any one of the following technologies known in the art or their combination: a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.
[0113] In the above description of this specification, the description with reference to the terms "one embodiment / example", "another embodiment / example" or "certain embodiments / examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner.
[0114] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the claims and their equivalents.
[0115] The above is a specific description of the preferred implementation of the present invention, but the present invention is not limited to the above embodiments. Those skilled in the art may make various equivalent modifications or substitutions without violating the spirit of the present invention. These equivalent modifications or substitutions are all included in the scope defined by the claims of this application.
Claims
1. An image frequency domain digital watermarking method, characterized in that: The following steps are involved: Design an embedding and extraction model for image frequency domain digital watermarks; the embedding and extraction model includes a watermark embedding extraction autoencoder and an attention scorer; After the carrier image and the watermark are input into the watermark embedding and extracting self-encoder, a first image with a watermark is obtained, and after a noise attack is randomly added to the first image, it is input into the decoder of the watermark embedding and extracting self-encoder to obtain a coarsely extracted watermark; Randomly select the first image to be cropped or not to be attacked, obtain a second image and a standard confidence map, input the second image to an attention scorer, and output an attention confidence map; The coarse watermark obtained by extracting the encoder through watermark embedding and the attention confidence map obtained by the attention scorer are multiplied pixel by pixel on the two-dimensional plane for correction to obtain the final watermark; The image frequency domain digital watermarking method further comprises the step of training the watermark embedding extraction self-encoder: The watermark embedding extraction autoencoder is trained to perform watermark embedding extraction in the direction of minimizing the error between the carrier image and the watermarked image, and the error between the input watermark and the extracted watermark; The image frequency domain digital watermarking method further comprises the step of training the attention scorer: The training direction is to minimize the error between the output of the attention scorer and the standard confidence map. The attention scorer is trained to calculate the confidence that the input image has a watermark at each position. The watermark embedding is extracted from the encoder and the attention scorer is composed of convolutional neural networks; Before the picture is input into the watermark embedding extraction self-encoder and the attention scorer, the picture is preprocessed as follows: If the image is a three-channel color image, first transform the color space of the color image to the YCbCr space, keep the CbCr channel unchanged, and only take the Y channel as the grayscale image for embedding; If the image is a single-channel grayscale image, there is no need to perform color space transformation. The grayscale image is divided into non-overlapping 8×8 image blocks. After each image block is discrete cosine transformed separately, all image blocks are reshaped together into a tensor with a side length of 1 / 8 of the side length of the carrier image and 64 channels. Each channel represents a discrete cosine transform coefficient. In addition, the watermark is input into the watermark embedding extractor and the watermark is preprocessed as follows: Arrange the watermarks into a tensor with equal sides and a total of 4 channels, keep the number of channels unchanged, and repeat on the 2D plane until the side length is the same as the side length of the image reconstruction tensor; The rough watermark obtained by extracting the watermark from the encoder through watermark embedding and the attention confidence map obtained by the attention scorer are multiplied pixel by pixel on a two-dimensional plane for correction to obtain the final watermark, including: Roughly extract the watermark tensor, the range of all values is (0, 1), and linearly transform the range of the watermark tensor to (-1, 1); The attention confidence map is multiplied element-by-element with the watermark tensor on a two-dimensional plane to obtain the final watermark tensor. According to the final watermark tensor, the repeated watermark positions are averaged and rounded to get the final watermark.
2. The image frequency domain digital watermarking method according to claim 1, characterized in that: In the watermark embedding extraction self-encoder, the preprocessed image tensor and the watermark tensor are concatenated and input into a neural network formed by a series connection of six convolution layers with a kernel size of 1×1 and a step size of 1×1, a non-padded convolution layer and a non-linear activation layer for encoding. The output of the encoder is subjected to an inverse discrete cosine transform, and the residual image obtained is added to the carrier image to obtain the final watermarked grayscale image. If the image before preprocessing is a three-channel color image, the grayscale image output by the encoder is used as the Y channel, and the CbCr channel retained during preprocessing is transformed back to the RGB color space again; then the preset digital image processing method is selected to add noise attack, and then the coarsely extracted watermark tensor is decoded through a neural network formed by four layers of convolution kernel size 1×1, step size 1×1, unfilled convolution layer and non-linear activation layer in series, and the repeated information bits are averaged to obtain the coarsely extracted watermark, with minimizing the error between the coarsely extracted watermark and the input watermark and the error between the watermarked image output by the encoder and the input image as the training direction, and the training watermark embedding is extracted from the encoder to embed and extract the watermark.
3. The image frequency domain digital watermarking method according to claim 1, characterized in that: In the attention scorer, the cropping attack is to randomly generate a rectangle smaller than the image size. The image content inside the rectangle remains unchanged, while the part outside the rectangle can be modified to any other value. The corresponding confidence map has a value of 0 at the cut position and a value of 1 at the retained position. If no attack is performed, the confidence map is 1 throughout the entire map. Then, the confidence map is subjected to two-dimensional mean pooling with a kernel size of 8×8 and a step size of 8×8 to obtain a standard confidence map. The processed image is discrete cosine transformed and input into a convolutional neural network consisting of four convolutional layers with a kernel size of 3×3, a step size of 1×1 and 1 unit zero padding, and a nonlinear activation layer in series to obtain the attention confidence map. The attention scorer is trained to learn the confidence of the input image with watermarks at various positions in order to minimize the error between the attention confidence map and the standard confidence map.
4. The image frequency domain digital watermarking method according to claim 1, characterized in that: In training the watermark embedding and extraction autoencoder, the error between the input watermark and the extracted watermark adopts mean square error, and the error between pictures also adopts mean square error, and the loss function used for training is the sum of picture error and watermark error; When training the attention scorer, the error between the standard confidence map and the attention confidence map adopts the mean square error, and the mean square error is used as the loss function of the training.
5. An image frequency domain digital watermarking system, applied to an image frequency domain digital watermarking method according to any one of claims 1 to 4, characterized in that: include: The model design module is used to design the embedding and extraction model of the image frequency domain digital watermark; wherein the embedding and extraction model includes the watermark embedding extraction autoencoder and the attention scorer; A watermark extraction module is used to input the carrier image and the watermark into the watermark embedding and extracting self-encoder to obtain a first image with a watermark, randomly add noise attack to the first image, and then input it into the decoder of the watermark embedding and extracting self-encoder to obtain a coarsely extracted watermark; A confidence acquisition module, used to randomly select the first image to be cropped for attack or not to be attacked, obtain a second image and a standard confidence map, input the second image to an attention scorer, and output an attention confidence map; The watermark correction module is used to multiply the coarse extracted watermark obtained by embedding the watermark into the encoder and the attention confidence map obtained by the attention scorer pixel by pixel on a two-dimensional plane to obtain the final watermark.
6. An image frequency domain digital watermarking device, characterized in that: include: at least one processor; at least one memory for storing at least one program; When the at least one program is executed by the at least one processor, the at least one processor implements the method according to any one of claims 1 to 4.
7. A computer-readable storage medium storing a program executable by a processor, characterized in that: The processor-executable program is used to perform the method according to any one of claims 1 to 4 when executed by the processor.
Citation Information
Patent Citations
Deep semi-fragile watermarking method for image authentication and confrontation sample defense
CN113689318A
Zero watermark construction method based on depth attention auto-encoder
CN113781284A
A method, an apparatus and a computer program product for modifying media content
EP3401843A1
