A UNet-based robust image watermarking soft fusion network method and system for screen capture

Abstract

The present invention provides a method and system for a robust image watermarking soft fusion network based on the UNet architecture, which is resistant to screen capture. The method includes: step 1, preprocessing the cover image to obtain a low-frequency component; step 2, establishing a multi-stage watermark extension subnetwork to perform feature fusion of the watermark information tensor with the low-frequency component; step 3, constructing a watermark soft fusion network ProwterNet based on the Unet architecture: the frequency domain information of the embedded watermark features and the watermark message are fused with the features of the jump link and the continuously upsampled features to achieve joint embedding in the spatial and frequency domains; step 4, designing a non-differentiable noise simulation training process to train the watermark soft fusion network ProwterNet. The present invention uses ProwterNet to process the watermark information and the carrier image, achieving multi-scale cross-domain progressive embedding.

Description

Technical Field

[0001] The present invention belongs to the field of information security, and in particular relates to a UNet-based anti-screen capture robust image watermark soft fusion network method and system. Background Art

[0002] As digital transformation accelerates across the globe, the frequency of cross-domain presentations of trade secrets via digital conferencing systems has increased significantly, leading to a growing problem of screen capture leaks. According to Verizon's 2023 Data Breach Investigations Report, 11% of physical security incidents stem from the illegal theft of secrets through mobile device screen captures. In this context, robust watermarking technology, with its powerful tracking and tracing capabilities, has become a key tool for combating screen capture leaks and protecting copyrights. By embedding encrypted identification information, this technology ensures that watermarked information remains intact and retrievable even after presentations undergo complex processes like screen capture and social media dissemination, thereby establishing a proactive defense system that "leakage is traceable."

[0003] Research on robust watermarking has made significant progress, with traditional methods primarily relying on spatial and transform domain techniques. Among spatial domain methods, the least significant bit (LSB) modification algorithm has attracted widespread attention due to its high visual imperceptibility, but it suffers from significant shortcomings in resisting screen capture attacks and noise interference. Transform domain-based methods, such as singular value decomposition (SVD), discrete wavelet transform (DWT), and discrete cosine transform (DCT), improve the concealment and anti-interference capabilities of watermarks by adjusting the statistical properties of transform coefficients. However, these methods rely on manual feature extraction and struggle to adaptively capture the primary energy distribution of an image. In particular, their feature representation capabilities are significantly limited when dealing with complex textures and high-frequency components, resulting in deviations between the watermark embedding location and the prominent areas of the screen-captured image, thus compromising robustness.

[0004] The development of deep learning and the success of convolutional neural networks (CNNs) in computer vision have driven a revolution in watermarking technology. Deep learning methods, through autonomous feature learning mechanisms, can exploit the inherent representational patterns of data and construct a multi-scale feature hierarchy, thereby improving the watermarking system's robustness against screen capture. However, current deep watermarking methods still face numerous challenges, such as excessive redundancy in watermark information, insufficient feature interaction between the watermark and the underlying image, which weakens semantic associations, and the impact of non-differentiable noise and image distortion during the screen capture process on watermark extractability.

[0005] Current mainstream end-to-end watermarking frameworks have limitations in watermark redundancy control and feature fusion. Many methods employ mechanical watermark replication strategies to achieve global distribution of the watermark, but lack adaptive redundancy control, resulting in poor integration of watermark and image features, affecting robustness against screen capture attacks. At the feature fusion level, existing methods often employ shallow fusion strategies involving direct channel splicing, making it difficult to establish an effective association between watermark semantic features and image structural features. Furthermore, they fail to dynamically adjust watermark features based on image content characteristics, thereby weakening their ability to resist screen capture attacks.

[0006] In addition, existing frameworks over-rely on spatial domain feature operations and ignore the advantages of frequency domain features. In contrast, frequency domain methods utilize energy compression characteristics and the sensitivity of the human visual system to demonstrate greater robustness against attacks such as compression, blurring, and color distortion caused by screen capture. In terms of noise interference resistance, although adding a noise layer can improve the anti-interference ability of the encoded image, the DCT and quantization operations of the JPEG compression built into social media or shooting devices will block the backpropagation chain of the neural network, limiting the encoder's optimization of the decoding loss. Although traditional analog training methods approximate the JPEG compression effect through differentiable DCT to maintain gradient conduction, there are still performance bottlenecks in high compression ratio scenarios. Summary of the Invention

[0007] Purpose of the invention: The technical problem to be solved by the present invention is to address the deficiencies of the existing technology and provide a UNet-based anti-screen capture robust image watermark soft fusion network method and system.

[0008] The method comprises:

[0009] Step 1: pre-process the cover image to obtain the low-frequency component;

[0010] Step 2: Establish a multi-stage watermark expansion sub-network to fuse the watermark information tensor with the low-frequency component;

[0011] Step 3, construct a watermark soft fusion network ProwterNet based on the Unet architecture: add discrete wavelet transform downsampling, cover self-attention CSA and channel compression attention SE modules to the Unet network, and cascade message processors; first, replace all downsampling modules in the encoding stage of the Unet network with discrete wavelet transform downsampling, and add channel compression attention SE after upsampling; secondly, add cover self-attention CSA before jump connection in the decoding stage; finally, receive the output of the cascade message processor, which includes the frequency domain information of the embedded watermark feature and the watermark message, and fuse the frequency domain information of the embedded watermark feature and the watermark message with the features of the jump link and the features after continuous upsampling to achieve joint embedding of the spatial domain and frequency domain;

[0012] Step 4: Design a non-differentiable noise simulation training process to train the watermark soft fusion network ProwterNet.

[0013] Step 1 includes:

[0014] Step 1-1: Input the cover image into the discrete wavelet transform network and decompose it into the first-level low-frequency component after the first layer of wavelet transform. , the first-level high-frequency component , the first-order vertical high-frequency component , the first-order diagonal high-frequency component ;

[0015] Step 1-2, the low frequency component Input it into the discrete wavelet transform network again and perform secondary decomposition to obtain the secondary low-frequency component , secondary horizontal high frequency component , secondary vertical high-frequency component , secondary vertical high-frequency component ;

[0016] Steps 1-3: convert the secondary low-frequency components Input discrete wavelet transform network to extract the third-level low-frequency component .

[0017] Step 2 includes:

[0018] Step 2-1: Randomly generate a 64-bit watermark , through the fully connected layer, the randomly generated watermark information Expand to The watermark information tensor ;

[0019] Step 2-2, the expanded watermark information tensor Make 4 copies, record as ,in ;

[0020] Steps 2-3, at the Layer 0 message processor , receiving the expanded watermark information tensor , without additional processing, directly output the watermark information tensor ;

[0021] Step 2-4: The three low-frequency components obtained in step 1 、 、 Cascaded to the first layer message processor , second layer message processor and the third layer message processor middle;

[0022] Step 2-5, first layer message processor Receive two inputs: the expanded watermark information tensor , first-level low-frequency component ;

[0023] Step 2-6, first layer message processor First, the watermark information tensor pass The convolutional neural network is extended to the first-level low-frequency component Watermark information tensor of consistent size , and then the watermark information tensor and the first-order low-frequency component Perform Hadamard product calculations and pass Function generates attention weight coefficient , use attention weight to watermark information tensor and the first-order low-frequency component Perform feature fusion and output the fused frequency domain information features with watermark characteristics ;

[0024] Steps 2-7, in the second layer message processor and the third layer message processor Receive the expanded watermark information tensor respectively , secondary low-frequency component And the expanded watermark information tensor , three-level low-frequency components , and repeat steps 2-6 to output the fused frequency domain information features with watermark characteristics and , the formula of the specific fusion process is:

[0025] ,

[0026] in express Convolutional neural network, for function, Calculate the product for the Hadamard product; z takes the value 1, 2, or 3;

[0027] Message Processor There are four layers in total, and the 0th layer outputs the watermark information tensor , the first layer is the watermark information tensor Processing is performed and finally the frequency domain information features with watermark characteristics are output The second and third layers output frequency information features with watermark characteristics .

[0028] Step 3 includes:

[0029] Step 3-1: Replace all downsampling modules in the original encoding stage of the Unet network with discrete wavelet transform downsampling. The discrete wavelet downsampling includes: Apply a two-dimensional discrete wavelet transform to the natural image X and decompose X into four frequency domain sub-images, where H represents the length of the natural image X, W represents the width of the natural image X, and C represents the number of channels of the natural image X;

[0030] The four frequency domain sub-images are low frequency component sub-image LL, horizontal high frequency component sub-image LH, vertical high frequency component sub-image HL and diagonal high frequency component sub-image HH; the four frequency domain sub-images are spliced ​​along the channel dimension to construct a new feature map , size is Apply standard convolution operations to perform channel dimensionality reduction, re-extract and fuse frequency domain information, and generate downsampled images , the size is ;

[0031] Step 3-2, let the input feature image be , horizontal pooling is performed on F in the horizontal direction and vertical pooling is performed in the vertical direction to capture the axial global context information, where the horizontal pooling operation is performed along the column dimension to generate the horizontal global feature vector ; The vertical average pooling operation is performed along the row dimension to generate a vertical global feature vector ; For the eigenvector and Apply broadcast addition to implement input features Global complement to generate global features ; Through the calibration function , for the generated global features Perform nonlinear changes to obtain calibrated global features :

[0032] ,

[0033] in and is the learnable parameter matrix, Represents the Relu activation function;

[0034] Then, a secondary calibration is performed through two strip convolutions in different directions: vertical strip convolution is used to calibrate the vertical shape, and horizontal strip convolution is used to calibrate the horizontal shape;

[0035] Cover self-attention (CSA) is introduced in the Unet network encoder stage and sent to the decoder through a skip link. A channel compression attention (SE) module is added after each upsampling in the decoding stage.

[0036] In the first layer of the Unet network, is the feature extracted during the encoding process, Message Processor Output watermark information features, For the features extracted in the decoding stage, the cover self-attention CSA is first used to obtain the attention weight coefficient after secondary feature calibration ,Will and Multiply elements by element to get features , and then batch normalization BN is used to normalize the features Normalize and output spatial features through ReLU activation function ,Will and watermark information characteristics , encoder extracts features Perform channel feature splicing to obtain fusion features , use the channel attention SE module to calculate the feature The attention weight coefficient ,Will and Multiply elements by element to get new features , which is then upsampled and used as the feature extracted by the decoder in the second layer of the Unet network.

[0037] In step 4, the non-differentiable noise simulation training process includes a small batch training strategy Mini-Bath and a non-differentiable simulated noise layer DiffJpeg.

[0038] In step 4, the non-differentiable simulated noise layer DiffJpeg includes a forward propagation process and a backward propagation process. In the forward propagation, real rather than simulated rounding, flooring, and clipping operations are used for image attacks; in the backward propagation, an unconventional straight-through estimator STE is adopted, and a polynomial approximation is used to replace the constant gradient. The formula is:

[0039] ,

[0040] in It is the gradient of the pixel value back-propagated when the image encounters non-differentiable noise attack.

[0041] To further improve the robustness of the model against non-differentiable noise attacks, a mini-batch training strategy called Mini-Bath is adopted. Each time during training, a random selection is made between the real non-differentiable noise attack layer JPEG, the noise-free attack layer Identity, and the non-differentiable simulated noise layer DiffJpeg.

[0042] In step 4, the following loss function L is used during training:

[0043] ,

[0044] in 、 、 is the weight coefficient, which is a hyperparameter. In the present invention, it is set to 1, 10, and 0.0001 respectively;

[0045] is the coding loss, which is calculated as:

[0046] ,

[0047] in Indicates the cover image With encoded image The mean square error, Represents the parameters of the encoder E, that is, the trainable weights and biases of the neural network, and M represents the original watermark information; the purpose of the encoder is to make the encoded image and cover image Be as visually similar as possible;

[0048] is the encoder message loss, which is calculated as:

[0049] ,

[0050] in Represents the message predicted by the decoder and original watermark information The mean square error, represents the parameters of the decoder D, i.e. the trainable weights and biases of the neural network, Represents the image after noise attack;

[0051] is the discriminator loss, which is calculated as:

[0052] ,

[0053] Where A is the discriminator, which approximates the original carrier image by generating random noise images; the first term is an adversarial loss, the purpose of which is to allow the discriminator A to recognize is tampered with, thereby improving the detection performance. ,The purpose is to ensure that the original image can be correctly classified as not tampered with; are the parameters of the discriminator A.

[0054] The present invention also provides a UNet architecture-based anti-screen capture robust image watermark soft fusion system implemented according to the method, comprising:

[0055] Data preprocessing unit: used to receive the carrier image. After the carrier image is input into the data preprocessing unit, three low-frequency component features will be output, namely the first-level low-frequency component , secondary low-frequency component and three-level low-frequency components ;

[0056] Message processing unit: used to receive messages and low-frequency component features, and output watermark information with frequency domain features and watermark information;

[0057] Encoding unit: used to receive the watermark information block and carrier image from the message processor, and output the encoded image through the watermark soft fusion network ProwterNet;

[0058] Adversarial training unit: The encoded image is input into the adversarial training unit, and adversarial training is performed through perturbation to improve the encoding quality;

[0059] Noise unit: Input the encoded image into a pre-established noise simulation layer and optimize the embedding robustness and image visual quality through backpropagation. For non-differentiable noise, a non-differentiable noise simulation training process is designed for training.

[0060] Decoding and extraction unit: The noisy image is input into the pre-established decoding and extraction unit to extract the one-dimensional watermark information.

[0061] The present invention also provides an electronic device, comprising a processor and a memory, wherein the memory stores program code, and when the program code is executed by the processor, the processor executes the steps of the method.

[0062] The present invention also provides a storage medium storing a computer program or instruction, which executes the steps of the method when the computer program or instruction is run on a computer.

[0063] In order to improve the detectability of watermarks under screen capture conditions and overcome the defects of simple copying of traditional watermarks, the present invention first uses a diffusion sub-network to enable the watermark information to be adaptively extended to the image in stages to achieve a more stable watermark coding representation. Subsequently, the UNet structure is adopted as the basis to replace the traditional additive fusion method, and a frequency domain-spatial domain joint coding strategy is introduced to enable the watermark information to be progressively embedded in different scales and frequency domains to significantly improve the anti-screen capture robustness. In addition, a cover self-attention mechanism is designed at the jump link to guide the dynamic adjustment of the watermark by adaptively allocating the attention weight of the cover image, thereby enhancing the interaction between the watermark and the image features, so that the watermark still has strong extractability after screen capture. In response to the problem of gradient breakage caused by non-differentiable noise, the present invention proposes a DiffJpeg noise layer to optimize the robustness of the model under screen capture related attacks such as JPEG compression.

[0064] The present invention has the following beneficial effects: It utilizes ProwterNet to process watermark information and carrier images, achieving multi-scale, cross-domain progressive embedding. Based on common distortion conditions in screen capture scenarios, the present invention evaluates the robustness of the model under different noise influences. During the screen capture process, images experience various forms of degradation, such as compression, cropping, signal loss, and sensor noise. Therefore, the present invention employs common noise in traditional watermarking methods to conduct experiments to verify the robustness of the model in the screen capture environment. The experiments include both non-differentiable noise (such as JPEG compression) and differentiable noise (such as cropping, dropout, and Gaussian noise). BRIEF DESCRIPTION OF THE DRAWINGS

[0065] Figure 1 It is the overall framework diagram of the method of the present invention.

[0066] Figure 2 This is the encoder architecture diagram.

[0067] Figure 3 is the wavelet downsampling graph.

[0068] Figure 4 It is the CSA module diagram. DETAILED DESCRIPTION

[0069] The present invention will be further described below in conjunction with the accompanying drawings and specific embodiments, and the above and / or other advantages of the present invention will become more apparent.

[0070] The embodiment of the present invention provides a UNet-based anti-screen capture robust image watermark soft fusion network method, including:

[0071] Step 1: pre-process the cover image to obtain the low-frequency component;

[0072] Step 2: Establish a multi-stage watermark expansion sub-network to fuse the watermark information tensor with the low-frequency component;

[0073] Step 3, construct a watermark soft fusion network ProwterNet based on the Unet architecture: add discrete wavelet transform downsampling, cover self-attention CSA and channel compression attention SE modules to the Unet network, and cascade message processors; first, replace all downsampling modules in the encoding stage of the Unet network with discrete wavelet transform downsampling, and add channel compression attention SE after upsampling; secondly, add cover self-attention CSA before jump connection in the decoding stage; finally, receive the output of the cascade message processor, which includes the frequency domain information of the embedded watermark feature and the watermark message, and fuse the frequency domain information of the embedded watermark feature and the watermark message with the features of the jump link and the features after continuous upsampling to achieve joint embedding of the spatial domain and frequency domain;

[0074] Step 4: Design a non-differentiable noise simulation training process to train the watermark soft fusion network ProwterNet.

[0075] Step 1 includes:

[0076] Step 1-1: Input the cover image into the discrete wavelet transform network and decompose it into the first-level low-frequency component after the first layer of wavelet transform. , the first-level high-frequency component , the first-order vertical high-frequency component , the first-order diagonal high-frequency component ;

[0077] Step 1-2, the low frequency component Input it into the discrete wavelet transform network again and perform secondary decomposition to obtain the secondary low-frequency component , secondary horizontal high frequency component , secondary vertical high-frequency component , secondary vertical high-frequency component ;

[0078] Steps 1-3: convert the secondary low-frequency components Input discrete wavelet transform network to extract the third-level low-frequency component .

[0079] Step 2 includes:

[0080] Step 2-1: Randomly generate a 64-bit watermark , through the fully connected layer, the randomly generated watermark information Expand to The watermark information tensor ;

[0081] Step 2-2, the expanded watermark information tensor Make 4 copies, record as ,in ;

[0082] Steps 2-3, at the Layer 0 message processor , receiving the expanded watermark information tensor , without additional processing, directly output the watermark information tensor ;

[0083] Step 2-4: The three low-frequency components obtained in step 1 、 、 Cascaded to the first layer message processor , second layer message processor and the third layer message processor middle;

[0084] Step 2-5, first layer message processor Receive two inputs: the expanded watermark information tensor , first-level low-frequency component ;

[0085] Step 2-6, first layer message processor First, the watermark information tensor pass The convolutional neural network is extended to the first-level low-frequency component Watermark information tensor of consistent size , and then the watermark information tensor and the first-order low-frequency component Perform Hadamard product calculations and pass Function generates attention weight coefficient , use attention weight to watermark information tensor and the first-order low-frequency component Perform feature fusion and output the fused frequency domain information features with watermark characteristics ;

[0086] Steps 2-7, in the second layer message processor and the third layer message processor Receive the expanded watermark information tensor respectively , secondary low-frequency component And the expanded watermark information tensor , three-level low-frequency components , and repeat steps 2-6 to output the fused frequency domain information features with watermark characteristics and , the formula of the specific fusion process is:

[0087] ,

[0088] in express Convolutional neural network, for function, Calculate the product for the Hadamard product; z takes the value 1, 2, or 3;

[0089] Message Processor There are four layers in total, and the 0th layer outputs the watermark information tensor , the first layer is the watermark information tensor Processing is performed and finally the frequency domain information features with watermark characteristics are output The second and third layers output frequency information features with watermark characteristics .

[0090] Step 3 includes:

[0091] Step 3-1: Replace all downsampling modules in the original encoding stage of the Unet network with discrete wavelet transform downsampling. The discrete wavelet downsampling includes: Apply a two-dimensional discrete wavelet transform to the natural image X and decompose X into four frequency domain sub-images, where H represents the length of the natural image X, W represents the width of the natural image X, and C represents the number of channels of the natural image X;

[0092] The four frequency domain sub-images are low frequency component sub-image LL, horizontal high frequency component sub-image LH, vertical high frequency component sub-image HL and diagonal high frequency component sub-image HH; the four frequency domain sub-images are spliced ​​along the channel dimension to construct a new feature map , size is Apply standard convolution operations to perform channel dimensionality reduction, re-extract and fuse frequency domain information, and generate downsampled images , the size is ;

[0093] Step 3-2, let the input feature image be , horizontal pooling is performed on F in the horizontal direction and vertical pooling is performed in the vertical direction to capture the axial global context information, where the horizontal pooling operation is performed along the column dimension to generate the horizontal global feature vector ; The vertical average pooling operation is performed along the row dimension to generate a vertical global feature vector ; For the eigenvector and Apply broadcast addition to implement input features Global complement to generate global features ; Through the calibration function , for the generated global features Perform nonlinear changes to obtain calibrated global features :

[0094] ,

[0095] in and is the learnable parameter matrix, Represents the Relu activation function;

[0096] Then, a secondary calibration is performed through two strip convolutions in different directions: vertical strip convolution is used to calibrate the vertical shape, and horizontal strip convolution is used to calibrate the horizontal shape;

[0097] Cover self-attention (CSA) is introduced in the Unet network encoder stage and sent to the decoder through a skip link. A channel compression attention (SE) module is added after each upsampling in the decoding stage.

[0098] In the first layer of the Unet network, is the feature extracted during the encoding process, Message Processor Output watermark information features, For the features extracted in the decoding stage, the cover self-attention CSA is first used to obtain the attention weight coefficient after secondary feature calibration ,Will and Multiply elements by element to get features , and then batch normalization BN is used to normalize the features Normalize and output spatial features through ReLU activation function ,Will and watermark information characteristics , encoder extracts features Perform channel feature splicing to obtain fusion features , use the channel attention SE module to calculate the feature The attention weight coefficient ,Will and Multiply elements by element to get new features , which is then upsampled and used as the feature extracted by the decoder in the second layer of the Unet network.

[0099] In step 4, the non-differentiable noise simulation training process includes a small batch training strategy Mini-Bath and a non-differentiable simulated noise layer DiffJpeg.

[0100] In step 4, the non-differentiable simulated noise layer DiffJpeg includes a forward propagation process and a backward propagation process. In the forward propagation, real rather than simulated rounding, flooring, and clipping operations are used for image attacks; in the backward propagation, an unconventional straight-through estimator STE is adopted, and a polynomial approximation is used to replace the constant gradient. The formula is:

[0101] ,

[0102] in It is the gradient of the pixel value back-propagated when the image encounters non-differentiable noise attack.

[0103] To further improve the robustness of the model against non-differentiable noise attacks, a mini-batch training strategy called Mini-Bath is adopted. Each time during training, a random selection is made between the real non-differentiable noise attack layer JPEG, the noise-free attack layer Identity, and the non-differentiable simulated noise layer DiffJpeg.

[0104] In step 4, the following loss function L is used during training:

[0105] ,

[0106] in 、 、 is the weight coefficient, which is a hyperparameter. In the present invention, it is set to 1, 10, and 0.0001 respectively;

[0107] is the coding loss, which is calculated as:

[0108] ,

[0109] in Indicates the cover image With encoded image The mean square error, Represents the parameters of the encoder E, that is, the trainable weights and biases of the neural network, and M represents the original watermark information; the purpose of the encoder is to make the encoded image and cover image Be as visually similar as possible;

[0110] is the encoder message loss, which is calculated as:

[0111] ,

[0112] in Represents the message predicted by the decoder and original watermark information The mean square error, represents the parameters of the decoder D, i.e. the trainable weights and biases of the neural network, Represents the image after noise attack;

[0113] is the discriminator loss, which is calculated as:

[0114] ,

[0115] Where A is the discriminator, which approximates the original carrier image by generating random noise images; the first term is an adversarial loss, the purpose of which is to allow the discriminator A to recognize is tampered with, thereby improving the detection performance. ,The purpose is to ensure that the original image can be correctly classified as not tampered with; is the parameter of the discriminator A. Through this optimization process, more robust watermark embedding and extraction are achieved, the ability of the watermark to resist noise attacks is improved, and the visual quality of the encoded image is guaranteed.

[0116] The method of the present invention demonstrates significant advantages in both robustness and stealth. Experimental evaluations tested the watermark system using a variety of common noise attack methods (including JPEG compression, cropping, pixel dropout, and Gaussian noise). The results demonstrate that the method maintains a low bit error rate and high peak signal-to-noise ratio (PSNR) and structural similarity (SSIM) under various distortion conditions, demonstrating excellent robustness against screen capture. In particular, the method maintains stable watermark extraction under high-compression JPEG attacks and maintains watermark integrity even in highly destructive scenarios such as cropping and dropout, demonstrating its broad applicability in real-world applications.

[0117] Table 1

[0118]

[0119] Table 1 is the experimental data table of cropping attack Crop. Cropping attack is the effect of perspective offset in screen capture. During the screen capture process, changes in the user's shooting angle or adjustment of the viewing range may cause partial loss of watermark information. Therefore, the present invention simulates different degrees of cropping distortion to examine the adaptability of the model to changes in screen capture perspective. In the experiment, the present invention selected four cropping ratios R=0.3, 0.5 and 0.7, which respectively indicate that the cropped image size is 30%, 50% and 70% of the original image. The experimental results show that cropping significantly damages the integrity of the embedded information, but the model of the present invention can still maintain a 0% bit error rate under various cropping ratios. The measurement scale Metric is selected as Peak Signal-to-Noise Ratio (PSNR) and Structural Similarity (SSIM). This experiment shows that when PSNR and SSIM are maintained at a high level, the model has extremely strong robustness in the screen capture environment, and can effectively restore the embedded content even when part of the watermark information is missing. HiDDeN, MBRS, and Adaptor are methods from different papers. This paper selects these three methods as control experimental groups to highlight the superiority of this invention. Subsequent experiments will be repeated.

[0120] Table 2

[0121]

[0122] Table 2 shows Gaussian noise interference. In real life, it is the sensor noise in screen capture. During the screen capture process, the image may be interfered with by random noise due to factors such as sensor performance, low light environment or device jitter. The present invention uses Gaussian noise to conduct experiments to simulate the noise pollution in the screen capture environment and evaluate the robustness of the watermark information. In the experiment, different variances are used. Gaussian noise with intensities of 0.001, 0.002, 0.005, and 0.010 was added to the image and its impact was analyzed. Experimental results show that Gaussian noise causes the embedded information to exhibit a regular, point-like distribution. The positional shifts are particularly pronounced when the noise intensity is high. This is due to the distribution characteristics of Gaussian noise, which causes the encoder to prefer embedding information in specific areas during the capture process. Although Gaussian noise can disrupt some image structure, the proposed model maintains a high degree of watermark extractability, demonstrating good adaptability to capture sensor noise.

[0123] Table 3 shows the results of JPEG compression experiments. JPEG compression is a lossy transmission effect during the screen capture process. For example, JPEG compression is a common factor in image degradation during screen capture, particularly noticeable when distributing images on social media or in low-quality screen captures. Its implementation process includes color mode conversion, data sampling, discrete cosine transform (DCT), frequency coefficient quantization, and encoding. Rounding during the quantization process hinders effective gradient backpropagation, directly impacting the accuracy of watermark extraction. In the experiments, the model was tested using different quality factors (Q), with Q=30, Q=50, and Q=70 selected for JPEG compression attacks, and their impact on the watermark was evaluated. The experimental results show that increasing Q reduces the compression ratio, improves image quality, increases PSNR and SSIM, and gradually reduces the bit error rate (BER). Comparing different methods under conditions close to the same BER shows that the proposed method performs well in terms of compressed image quality, BER control, and structure preservation. Particularly in the screen capture environment, it maintains watermark robustness while maintaining good image readability.

[0124] Table 3

[0125]

[0126] Table 4 shows the experimental results of the pixel loss attack, Dropout. During the screen capture process, due to issues such as screen refresh rate, pixel sampling, or signal interference, some pixels may be lost, compromising the integrity of the watermark information. To simulate this situation, experiments using the pixel loss attack, Dropout, randomly remove some pixels and replace them with pixels from the cover image to test the model's robustness in the presence of signal loss during screen capture. Dropout ratios R = 0.3, 0.5, and 0.7 were selected in the experiments, representing a residual encoded image of 30%, 50%, and 70% of the original image, respectively. The results show that even with a high dropout ratio (e.g., R = 0.7), the proposed method still maintains near-perfect recovery, with a significantly lower bit error rate than other methods, while maintaining high SSIM and PSNR levels. This demonstrates that the proposed method is effective in resisting signal loss during screen capture and performs superiorly in terms of information integrity and visual quality.

[0127] Table 4

[0128]

[0129] The present invention provides a robust image watermarking soft fusion network method and system based on the UNet architecture for anti-screen capture. There are many methods and approaches to implement this technical solution. The above is only a preferred embodiment of the present invention. It should be noted that those skilled in the art can make several improvements and modifications without departing from the principles of the present invention, and such improvements and modifications should also be considered within the scope of protection of the present invention. Any components not specified in this embodiment can be implemented using existing technologies.

Claims

1. A UNet-based anti-screen capture robust image watermarking soft fusion network method, characterized by: The following steps are involved: Step 1: pre-process the cover image to obtain the low-frequency component; Step 2: Establish a multi-stage watermark expansion sub-network to fuse the watermark information tensor with the low-frequency component; Step 3, construct a watermark soft fusion network ProwterNet based on the Unet architecture: add discrete wavelet transform downsampling, cover self-attention CSA and channel compression attention SE modules to the Unet network, and cascade message processors; first, replace all downsampling modules in the encoding stage of the Unet network with discrete wavelet transform downsampling, and add channel compression attention SE after upsampling; secondly, add cover self-attention CSA before jump connection in the decoding stage; finally, receive the output of the cascade message processor, which includes the frequency domain information of the embedded watermark feature and the watermark message, and fuse the frequency domain information of the embedded watermark feature and the watermark message with the features of the jump link and the features after continuous upsampling to achieve joint embedding of the spatial domain and frequency domain; Step 4: Design a non-differentiable noise simulation training process to train the watermark soft fusion network ProwterNet; Step 1 includes: Step 1-1: Input the cover image into the discrete wavelet transform network and decompose it into the first-level low-frequency component after the first layer of wavelet transform. , the first-level high-frequency component , the first-order vertical high-frequency component , the first-order diagonal high-frequency component ; Step 1-2, the low frequency component Input it into the discrete wavelet transform network again and perform secondary decomposition to obtain the secondary low-frequency component , secondary horizontal high frequency component , secondary vertical high-frequency component , secondary vertical high-frequency component ; Steps 1-3: convert the secondary low-frequency components Input discrete wavelet transform network to extract the third-level low-frequency component ; Step 2 includes: Step 2-1: Randomly generate a 64-bit watermark , through the fully connected layer, the randomly generated watermark information Expand to The watermark information tensor ; Step 2-2, the expanded watermark information tensor Make 4 copies, record as ,in ; Steps 2-3, at the Layer 0 message processor , receiving the expanded watermark information tensor , without additional processing, directly output the watermark information tensor ; Step 2-4: The three low-frequency components obtained in step 1 、 、 Cascaded to the first layer message processor , second layer message processor and layer 3 message processors middle; Step 2-5, first layer message processor Receive two inputs: the expanded watermark information tensor , first-level low-frequency component ; Step 2-6, first layer message processor First, the watermark information tensor pass The convolutional neural network is extended to the first-level low-frequency component Watermark information tensor of consistent size , and then the watermark information tensor and the first-order low-frequency component Perform Hadamard product calculations and pass Function generates attention weight coefficient , use attention weight to watermark information tensor and the first-order low-frequency component Perform feature fusion and output the fused frequency domain information features with watermark characteristics ; Step 2-7, in the second layer message processor and layer 3 message processors Receive the expanded watermark information tensor respectively , secondary low-frequency component And the expanded watermark information tensor , three-level low-frequency components , and repeat steps 2-6 to output the fused frequency domain information features with watermark characteristics and , the formula of the specific fusion process is: ， in express Convolutional neural network, for function, Calculate the product for the Hadamard product; z takes the value 1, 2, or 3; Message Processor There are four layers in total, and the 0th layer outputs the watermark information tensor , the first layer is the watermark information tensor Processing is performed and finally the frequency domain information features with watermark characteristics are output The second and third layers output frequency domain information features with watermark characteristics respectively ; Step 3 includes: Step 3-1: Replace all downsampling modules in the original encoding stage of the Unet network with discrete wavelet transform downsampling. The discrete wavelet downsampling includes: Apply a two-dimensional discrete wavelet transform to the natural image X and decompose X into four frequency domain sub-images, where H represents the length of the natural image X, W represents the width of the natural image X, and C represents the number of channels of the natural image X; The four frequency domain sub-images are low frequency component sub-image LL, horizontal high frequency component sub-image LH, vertical high frequency component sub-image HL and diagonal high frequency component sub-image HH; the four frequency domain sub-images are spliced ​​along the channel dimension to construct a new feature map , size is Apply standard convolution operations to perform channel dimensionality reduction, re-extract and fuse frequency domain information, and generate downsampled images , the size is ; Step 3-2, let the input feature image be , horizontal pooling is performed on F in the horizontal direction and vertical pooling is performed in the vertical direction to capture the axial global context information, where the horizontal pooling operation is performed along the column dimension to generate the horizontal global feature vector ; The vertical average pooling operation is performed along the row dimension to generate a vertical global feature vector ; For the eigenvector and Apply broadcast addition to implement input features Global complement to generate global features ; Through the calibration function , for the generated global features Perform nonlinear changes to obtain calibrated global features : ， in and is the learnable parameter matrix, Represents the Relu activation function; Then, a secondary calibration is performed through two strip convolutions in different directions: vertical strip convolution is used to calibrate the vertical shape, and horizontal strip convolution is used to calibrate the horizontal shape; Cover self-attention (CSA) is introduced in the Unet network encoder stage and sent to the decoder through a skip link. A channel compression attention (SE) module is added after each upsampling in the decoding stage. In the first layer of the Unet network, are the features extracted during the encoding process. Message Processor Output watermark information features, For the features extracted in the decoding stage, the cover self-attention CSA is first used to obtain the attention weight coefficient after secondary feature calibration ,Will and Multiply elements by element to get features , and then batch normalization BN is used to normalize the features Normalize and output spatial features through ReLU activation function ,Will and watermark information characteristics , encoder extracts features Perform channel feature splicing to obtain fusion features , use the channel attention SE module to calculate the feature The attention weight coefficient ,Will and Multiply elements by element to get new features , which is then upsampled and used as the feature extracted by the decoder in the second layer of the Unet network.

2. The method according to claim 1, characterized in that In step 4, the non-differentiable noise simulation training process includes a small batch training strategy Mini-Bath and a non-differentiable simulated noise layer DiffJpeg.

3. The method according to claim 2, characterized in that In step 4, the non-differentiable simulated noise layer DiffJpeg includes a forward propagation process and a backward propagation process. In the forward propagation, rounding, flooring, and clipping operations are used for image attacks; in the backward propagation, a straight-through estimator STE is used, and a polynomial approximation is used to replace the constant gradient. The formula is: ， in It is the gradient of the pixel value back-propagated when the image encounters non-differentiable noise attack; The mini-batch training strategy Mini-Bath is adopted, and each time during training, a random selection is made between the real non-differentiable noise attack layer JPEG, the noise-free attack layer Identity, and the non-differentiable simulated noise layer DiffJpeg.

4. The method according to claim 3, characterized in that In step 4, the following loss function L is used during training: ， in 、 、 is the weight coefficient; is the coding loss, which is calculated as: ， in Indicates the cover image With encoded image The mean square error, Represents the parameters of the encoder E, and M represents the original watermark information; is the encoder message loss, which is calculated as: ， in Represents the message predicted by the decoder and original watermark information The mean square error, represents the parameters of the decoder D, Represents the image after noise attack; is the discriminator loss, which is calculated as: ， Where A is the discriminator; are the parameters of the discriminator A.

5. A UNet-based robust image watermark soft fusion system implemented according to the method of any one of claims 1 to 4, characterized in that: include: Data preprocessing unit: used to receive the carrier image. After the carrier image is input into the data preprocessing unit, three low-frequency component features will be output, namely the first-level low-frequency component , secondary low-frequency component and three-level low-frequency components ; Message processing unit: used to receive messages and low-frequency component features, and output watermark information with frequency domain features and watermark information; Encoding unit: used to receive the watermark information block and carrier image from the message processor, and output the encoded image through the watermark soft fusion network ProwterNet; Adversarial training unit: The encoded image is input into the adversarial training unit, and adversarial training is performed through perturbation to improve the encoding quality; Noise unit: Inputs the encoded image into a pre-established noise simulation layer and optimizes the embedding robustness and image visual quality through backpropagation; For non-differentiable noise, a non-differentiable noise simulation training process is designed for training; Decoding and extraction unit: The noisy image is input into the pre-established decoding and extraction unit to extract the one-dimensional watermark information.

6. An electronic device, characterized in that: The method comprises a processor and a memory, wherein the memory stores program codes, and when the program codes are executed by the processor, the processor is caused to perform the steps of the method according to any one of claims 1 to 4.

7. A storage medium, characterized in that: A computer program or instruction is stored, and when the computer program or instruction is run on a computer, the steps of the method according to any one of claims 1 to 4 are executed.

Images