Wavelet domain anti-screen-shooting robust watermarking method based on reversible neural network

Abstract

The invention relates to a wavelet domain anti-screen-shooting robust watermarking method based on a reversible neural network, which relates to the technical field of digital watermarking, and is characterized in that a preprocessing network is added, watermark information and an original image are preprocessed to improve the coupling of the watermark information and the original image, and a reversible neural network parameter sharing mechanism is utilized to improve the robustness of the watermark information and the original image. Embedding and extraction of watermark information are realized by using a reversible neural network in a Haar wavelet domain, so that the training efficiency of the model and the imperceptibility of a watermark-containing image are improved; after the algorithm is applied to the image, the image can carry secret information, and the algorithm has very strong robustness for common digital noise and screen shooting noise generated by shooting of mobile equipment such as a smart phone and the like; the watermark-containing image generated by the method provided by the invention has better visual quality, and has higher watermark information extraction accuracy under different noise attacks.

Description

Technical Field

[0001] The present invention relates to the technical field of digital watermarking, and in particular to a wavelet domain screen capture resistant robust watermarking method based on a reversible neural network. Background Art

[0002] Digital watermarking technology is a commonly used method to hide information in a carrier medium, and after hiding, it does not affect the use value of the original carrier and is not easy to be perceived or noticed. Through this information hidden in the carrier, the purpose of confirming the content creator, purchaser, transmitting secret information, or judging whether the carrier has been tampered with can be achieved. Various applications of digital watermarks have brought convenience to all aspects of people's lives. Almost all types of smart phones are equipped with micro cameras. Through simple operations on the phone, you can take pictures of wonderful moments, or scan QR codes to quickly obtain additional information such as web addresses and use them in mobile applications.

[0003] As smart portable photography devices have increasingly high photography performance and popularity, photography has become one of the simplest and most efficient ways to transmit information in daily life. People only need to take a photo of the multimedia content displayed on the screen to obtain information without leaving any record traces. Therefore, the problem of information leakage during screen shooting needs to be solved urgently. The goal of anti-screen shooting digital watermarking is to embed watermark information in multimedia data, so that even if the multimedia data displayed on the screen is captured by the camera, the watermark information can still be guaranteed to be extractable, thereby achieving the purpose of copyright protection and tracking and tracing. Summary of the invention

[0004] The technical problem to be solved by the present invention is to provide a wavelet domain screen capture-resistant robust watermarking method based on a reversible neural network in view of the deficiencies of the background technology, and effectively improve the robustness against screen capture noise attacks.

[0005] The present invention adopts the following technical solutions to solve the above technical problems:

[0006] A wavelet domain screen capture-resistant robust watermarking method based on a reversible neural network can accurately extract information from an image after adding a watermark signal and after being attacked by screen capture and digital noise. The method specifically includes the following steps:

[0007] Step 1: Based on the carrier image I o and watermark information M, preprocess it to generate a residual image I containing watermark information r ;

[0008] Step 2: Transform the carrier image I o and the residual image I r The original image feature F is obtained by transforming it into the wavelet domain through the Haar discrete wavelet transform oand residual image features F r ;

[0009] Step 3: Based on the original image feature F obtained in step 2 o and residual image features F r , as the two inputs of the forward process of the reversible neural network, the output F containing the watermark image features is obtained e And the output F containing the original image features t ;

[0010] Step 4: Output F containing watermark image features obtained in step 3 e , which is scaled by the intensity factor S and added to the output F of the original image features t middle;

[0011] Step 5: The output obtained in step 4 is transformed by inverse Haar discrete wavelet transform to obtain the watermarked image I e ;

[0012] Step 6: The watermarked image I obtained in step 5 e The noise image I is obtained by adding random noise to the noise pool d ;

[0013] Step 7: the noise image I generated in step 6 d After the correction module, the corrected image I p ;

[0014] Step 8: Take the corrected image I generated in step 7 p After the Haar discrete wavelet transform is converted to the Haar wavelet domain, the feature information F of the watermarked image is obtained. d1 ;

[0015] Step 9: Copy the feature information obtained in step 8 to get F d2 ; F d1 and F d2 As two inputs of the inverse process of the reversible neural network, the restored residual image features F r ', and restore the image features F o ';

[0016] Step 10: restore the residual image feature F obtained in step 9 r ', through the inverse Haar discrete wavelet transform to obtain the restored residual image I r ';

[0017] Step 11, step 10, to obtain the restored residual image I r 'The recovery information M is obtained through the information extraction module'.

[0018] As a further preferred solution of the wavelet domain screen-proof robust watermarking method based on a reversible neural network of the present invention, the step 1 specifically comprises the following steps:

[0019] Step 1.1, the preprocessing network converts the watermark information M∈(0,1) L A tensor of C×H / 4×W / 4 is formed through a fully connected layer;

[0020] Step 1.2, after upsampling operation, generate the same image as the original image I o ∈(0,255) C×H×W Tensors of the same size;

[0021] Step 1.3, concatenate the tensor and the original image and input them into U-Net to obtain the residual image I containing the watermark information r , the specific calculation is as follows:

[0022] I r =f PM (M,I o )=f U (f US (f FC (M)),I o ).

[0023] As a further preferred solution of the wavelet domain screen-proof robust watermarking method based on a reversible neural network of the present invention, in step 2, the original image I with a size of C×H×W is o and the residual image I r After discrete wavelet transform DWT, they are converted into tensors of size 4C×H / 2×W / 2 to obtain the original image features F o and residual image features F r :

[0024] F r =f DWT (I r )

[0025] F o =f DWT (I o ).

[0026] As a further preferred solution of the wavelet domain screen-proof robust watermarking method based on a reversible neural network of the present invention, in step 3, the original image feature F o and residual image features F r Input into the forward propagation process of the reversible neural network to obtain the watermarked image feature F e and the original image features F o Output F t :

[0027]

[0028] As a further preferred solution of the wavelet domain screen-proof robust watermarking method based on a reversible neural network of the present invention, in step 5, the output F of the watermark image feature obtained in step 3 is converted into e , which is scaled by the intensity factor S and added to the output F of the original image features t Where φ(·), ρ(·) and η(·) are arbitrary functions. The output obtained in step 4 is transformed by inverse Haar discrete wavelet transform to obtain the watermarked image I e , the specific calculation is as follows:

[0029]

[0030] As a further preferred solution of the wavelet domain screen-proof robust watermarking method based on a reversible neural network of the present invention, in step 6, the watermarked image I obtained in step 5 is e The noise pool is attacked by random differentiable digital noise and non-differentiable screen noise to obtain the noise image I d :

[0031] I d =f NP (I e )=A(I e )+Z.

[0032] As a further preferred solution of the wavelet domain screen-proof robust watermarking method based on a reversible neural network of the present invention, in step 7, the noise image I generated in step 6 is d The DeepLabV3+ semantic segmentation model is used to locate and segment the target area. The Canny detection method based on median filtering and Hough transform are used to smooth the edges of the segmented area and locate the corner points. The segmented image is processed by perspective transformation through the located corner points to obtain the corrected image I p :

[0033] I p =f CN (I s )=(f Hough (f Canny (f DM (I s )))).

[0034] As a further preferred solution of the wavelet domain screen-proof robust watermarking method based on a reversible neural network of the present invention, in step 9, the noise image I d After DWT, the noise image feature F is obtained d1 , copy it and use it as auxiliary variable Fd2 , and input it into the back propagation process of the reversible neural network to obtain the restored residual image feature F r , and restore the image features F o , :

[0035] F r ′=F d1 !φ(F o ′)

[0036] F o '=(F d2 !η(F d1 ))⊙exp(-ρ(F d1 )).

[0037] As a further preferred solution of the wavelet domain screen-proof robust watermarking method based on a reversible neural network of the present invention, in step 11, according to the restored residual image feature F obtained in step 9 r ', through the inverse Haar discrete wavelet transform to obtain the restored residual image I r '. Restore the residual image I obtained in step 10 r 'Get the recovery information M through the information extraction module':

[0038] M′=f IE (I′ r ).

[0039] Compared with the prior art, the present invention adopts the above technical solution and has the following technical effects:

[0040] 1. The present invention improves the coupling between the watermark information and the original image by adding a preprocessing network, preprocesses the watermark information and the original image, and uses the reversible neural network parameter sharing mechanism to propose the use of reversible neural network in the Haar wavelet domain to embed and extract watermark information, thereby improving the training efficiency of the model and the imperceptibility of the watermarked image;

[0041] 2. The present invention also proposes a noise pool that integrates digital attacks and real screen capture attacks to enhance the ability to resist various noise attacks. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For those skilled in the art, other drawings can be obtained based on these drawings without paying any creative work.

[0043] Figure 1 It is an overall flow chart of the digital watermark method against screen capture noise of the present invention;

[0044] Figure 2 The digital effect diagram of the digital watermarking method against screen capture noise of the present invention; wherein, Figure (a) is the original image; Figure (b) is the encrypted image; Figure (c) is the screen capture image; Figure (d) is the image corrected by affine transformation;

[0045] Figure 3 It is a schematic diagram of the Swin Transformer network structure of the present invention;

[0046] Figure 4 It is a schematic diagram of the structure of the non-differentiable screen noise of the present invention;

[0047] Figure 5 This is a schematic diagram of the present invention using the DeepLabV3+ semantic segmentation model to achieve the positioning and segmentation of the target area. DETAILED DESCRIPTION

[0048] The technical solution of the present invention is further described in detail below in conjunction with the accompanying drawings:

[0049] The technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all of the embodiments. Based on the embodiments in the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention. The present invention is described in detail below according to the drawings and preferred embodiments, and the purpose and effect of the present invention will become clearer. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0050] like Figure 1 In the overall flow chart of the digital watermarking method against screen capture noise shown, there are two entities, namely the owner and the appraiser. The owner provides the watermark detection and embedding method of the present invention to embed the copyright watermark into the carrier image (steps 1 to 5), and the appraiser (user) uses a mobile phone to shoot it (step 6), and the tampered part of the image can be detected by white highlighting. The appraiser first locates the screen capture image (step 7), and finally extracts the watermark information (steps 8 to 11); Figure 2 It is a digital effect diagram of the digital watermarking method against screen capture noise of the present invention; wherein, Figure (a) is the original image; Figure (b) is the encrypted image; Figure (c) is the screen capture image; and Figure (d) is the image corrected by affine transformation.

[0051] The details are as follows:

[0052] A wavelet domain screen-proof robust watermarking method based on reversible neural network, such as Figure 1As shown in the figure, after adding the watermark information, the image can still accurately extract information after being screen shot and various digital noise attacks, which specifically includes the following steps:

[0053] 1) According to the carrier image Io and the watermark information M, they are preprocessed and a simplified version of U-Net is used to generate a residual image Ir containing the watermark information.

[0054] 2) The carrier image Io and the residual image Ir are transformed into the wavelet domain through the Haar discrete wavelet transform to obtain the original image feature Fo and the residual image feature Fr.

[0055] 3) Based on the original image feature Fo and the residual image feature Fr obtained in step 2), as two inputs of the forward process of the reversible neural network, the output Fe containing the watermark image feature and the output Ft containing the original image feature are obtained.

[0056] 4) The output Fe containing the watermark image feature obtained in step 3) is scaled by the intensity factor S and added to the output Ft of the original image feature.

[0057] 5) The output obtained in step 4) is transformed by inverse Haar discrete wavelet transform to obtain the watermarked image Ie.

[0058] 6) The watermarked image Ie obtained in step 5) is subjected to random noise addition through a noise pool to obtain a noise image Id.

[0059] 7) The noise image Id generated in step 6) is passed through a correction module to obtain a corrected image Ip.

[0060] 8) The corrected image Ip generated in step 7) is converted into the Haar wavelet domain through Haar discrete wavelet transform to obtain the feature information Fd1 of the watermarked image.

[0061] 9) Copy the feature information obtained in step 8) to obtain Fd2. Use Fd1 and Fd2 as two inputs of the reversible neural network inverse process to obtain the restored residual image feature F r ', and restore the image features F o '.

[0062] 10) Recover the residual image feature F obtained in step 9 r ', through the inverse Haar discrete wavelet transform to obtain the restored residual image I r ';.

[0063] 11) Restore the residual image I obtained in step 10) r 'The recovery information M is obtained through the information extraction module'.

[0064] The specific steps include:

[0065] The preprocessing network first converts the watermark information M∈(0,1) L The fully connected layer forms a C×H / 4×W / 4 tensor; then the upsampling operation generates the same image as the original image I o ∈(0,255) C×H×W A tensor with the same size; finally, the tensor and the original image are cascaded and input into U-Net to obtain the residual image I containing the watermark information r :

[0066] I r =f PM (M,I o )=f U (f US (f FC (M)),I o )

[0067] The original image I is of size C×H×W o and the residual image I r After discrete wavelet transform (DWT), they are converted into tensors of size 4C×H / 2×W / 2 to obtain the original image features F o and residual image features F r :

[0068] F r =f DWT (I r )

[0069] F o =f DWT (I o )

[0070] The original image feature F o and residual image features F r Input into the forward propagation process of the reversible neural network to obtain the watermarked image feature F e and the original image features F o Output F t .

[0071]

[0072] The output F of the watermark image feature obtained in step 3) e The output F is scaled by the intensity factor S and added to the original image features. t , where φ(·), ρ(·) and η(·) are arbitrary functions. Here, the Swin Transformer network structure is used, such as Figure 3 shown.

[0073]

[0074] The output obtained in step 4) is transformed by inverse Haar discrete wavelet transform to obtain the watermarked image I e .

[0075] The watermarked image I obtained in step 5) e The noise pool is randomly differentiable digital noise and non-differentiable screen noise, and the structure is as follows Figure 4 As shown, the attack obtains the noise image I d :

[0076] I d =f NP (I e )=A(I e )+Z

[0077] For the noise image I generated in step 6) d The DeepLabV3+ semantic segmentation model is used to realize the positioning and segmentation of the target area. Figure 5 As shown in FIG. 1 , the Canny detection method based on median filtering and Hough transform are used to smooth the edge of the segmented area and locate the corner points. The segmented image obtained is processed by perspective transformation through the located corner points, and finally the corrected image I is obtained. p :

[0078] I p =f CN (I s )=(f Hough (f Canny (f DM (I s ))))

[0079] The corrected image I generated in step 7) p After the haar discrete wavelet transform is converted into the haar wavelet domain, the characteristic information of the watermarked image is obtained.

[0080] Noise Image I d After DWT, the noise image feature F is obtained d1 , copy it and use it as auxiliary variable F d2 , and input it into the back propagation process of the reversible neural network to obtain the restored residual image feature F r , and restore the image features F o , :

[0081] F r ′=F d1 !φ(F o ′)

[0082] F o '=(F d2 !η(F d1 ))⊙exp(-ρ(F d1 ))

[0083] According to the restored residual image feature F obtained in step 9) r ', through the inverse Haar discrete wavelet transform to obtain the restored residual image I r '.

[0084] Step 10) is used to obtain the restored residual image I r 'Get the recovery information M through the information extraction module':

[0085] M′=f IE (I′ r ).

[0086] Those skilled in the art can understand that the above are only preferred examples of the invention and are not intended to limit the invention. Although the invention is described in detail with reference to the above examples, those skilled in the art can still modify the technical solutions recorded in the above examples or replace some of the technical features with equivalents. Any modification, equivalent replacement, etc. made within the spirit and principle of the invention should be included in the protection scope of the invention. All technical features in this embodiment can be freely combined according to actual needs.

[0087] Finally, it should be noted that the above is only a preferred embodiment of the present invention and is not intended to limit the present invention. Although the present invention has been described in detail with reference to the aforementioned embodiments, it is still possible for those skilled in the art to modify the technical solutions described in the aforementioned embodiments or to make equivalent substitutions for some of the technical features therein. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A wavelet domain screen-proof robust watermarking method based on a reversible neural network, characterized by: After adding the watermark, the image can still accurately extract information after screen capture and digital noise attack. The specific steps include the following: Step 1: Based on the carrier image I o and watermark information M, preprocess it to generate a residual image I containing watermark information r ; Step 2: Transform the carrier image I o and the residual image I r The original image feature F is obtained by transforming it into the wavelet domain through the Haar discrete wavelet transform o and residual image features F r ; Step 3: Based on the original image feature F obtained in step 2 o and residual image features F r , as the two inputs of the forward process of the reversible neural network, the output F containing the watermark image features is obtained e And the output F containing the original image features t ; Step 4: Output F containing watermark image features obtained in step 3 e , which is scaled by the intensity factor S and added to the output F of the original image features t middle; Step 5: The output obtained in step 4 is transformed by inverse Haar discrete wavelet transform to obtain the watermarked image I e ; Step 6: The watermarked image I obtained in step 5 e The noise image I is obtained by adding random noise to the noise pool d ; Step 7: the noise image I generated in step 6 d After the correction module, the corrected image I p ; Step 8: Take the corrected image I generated in step 7 p After the Haar discrete wavelet transform is converted to the Haar wavelet domain, the feature information F of the watermarked image is obtained. d1 ; Step 9: Copy the feature information obtained in step 8 to get F d2 ; F d1 and F d2 As two inputs of the inverse process of the reversible neural network, the restored residual image features F r ', and restore the image features F o '; Step 10: restore the residual image feature F obtained in step 9 r ', through the inverse Haar discrete wavelet transform to obtain the restored residual image I r '; Step 11, step 10, to obtain the restored residual image I r 'The recovery information M is obtained through the information extraction module'.

2. According to claim 1, a wavelet domain screen-proof robust watermarking method based on reversible neural network is characterized by: The step 1 specifically comprises the following steps: Step 1.1, the preprocessing network converts the watermark information M∈(0,1) L A tensor of C×H / 4×W / 4 is formed through a fully connected layer; Step 1.2, after upsampling operation, generate the same image as the original image I o ∈(0,255) C×H×W Tensors of the same size; Step 1.3, concatenate the tensor and the original image and input them into U-Net to obtain the residual image I containing the watermark information r , the specific calculation is as follows: I r =f PM (M,I o )=f U (f US (f FC (M)),I o )。 3. According to claim 1, a wavelet domain screen-proof robust watermarking method based on a reversible neural network is characterized by: In step 2, the original image I of size C×H×W o and the residual image I r After discrete wavelet transform DWT, they are converted into tensors of size 4C×H / 2×W / 2 to obtain the original image features F o and residual image features F r : F r =f DWT (I r ) F o =f DWT (I o )。 4. According to claim 1, a wavelet domain screen-proof robust watermarking method based on reversible neural network is characterized by: In step 3, the original image feature F o and residual image features F r Input into the forward propagation process of the reversible neural network to obtain the watermarked image feature F e and the original image features F o Output F t :

5. According to claim 1, a wavelet domain screen-proof robust watermarking method based on reversible neural network is characterized by: In step 5, the output F of the watermark image feature obtained in step 3 is e , which is scaled by the intensity factor S and added to the output F of the original image features t Where φ(·), ρ(·) and η(·) are arbitrary functions. The output obtained in step 4 is transformed by inverse Haar discrete wavelet transform to obtain the watermarked image I e , the specific calculation is as follows:

6. The wavelet domain screen-proof robust watermarking method based on reversible neural network according to claim 1 is characterized by: In step 6, the watermarked image I obtained in step 5 is e The noise pool is attacked by random differentiable digital noise and non-differentiable screen noise to obtain the noise image I d : I d =f NP (I e )=A(I e )+Z。 7. The wavelet domain screen-proof robust watermarking method based on reversible neural network according to claim 1 is characterized by: In step 7, the noise image I generated in step 6 is d The DeepLabV3+ semantic segmentation model is used to locate and segment the target area. The Canny detection method based on median filtering and Hough transform are used to smooth the edges of the segmented area and locate the corner points. The segmented image is processed by perspective transformation through the located corner points to obtain the corrected image I p : I p =f CN (I s )=(f Hough (f Canny (f DM (I s ))))。 8. The wavelet domain screen-proof robust watermarking method based on reversible neural network according to claim 1 is characterized by: In step 9, the noise image I d After DWT, the noise image feature F is obtained d1 , copy it and use it as auxiliary variable F d2 , and input it into the back propagation process of the reversible neural network to obtain the restored residual image feature F r ' and restore the image features F o ': F r ′=F d1 !φ(F o ′) F o '=(F d2 !η(F d1 ))⊙exp(-ρ(F d1 ))。 9. The wavelet domain screen-proof robust watermarking method based on reversible neural network according to claim 1 is characterized by: In step 11, the restored residual image feature F obtained in step 9 is r ', through the inverse Haar discrete wavelet transform to obtain the restored residual image I r '. Restore the residual image I obtained in step 10 r 'Get the recovery information M through the information extraction module': M’=f IE (I’ r )。

Images