Watermark imperceptible embedding and recovering method based on deep learning network structure

Abstract

The invention discloses a watermark imperceptible embedding and recovering method based on a deep learning network structure. Firstly, a mask guide watermark embedding scheme is designed, a mask generation module is constructed by using a residual dense feature extraction module and an attention mask generation module, and a watermark is adaptively guided to be embedded into an image texture rich area so as to improve the invisibility of the watermark. And secondly, constructing a watermark decoding network based on comparative learning, and by comparing a loss function, taking the decoding features of the same watermark image under different noise conditions as positive samples and taking the decoding features of different watermark images as negative samples so as to enhance the consistency of the decoding features, thereby improving the robustness of the watermark. The deep learning network structure can improve the robustness of the watermark in a real screen shooting scene, and has a huge application value in copyright protection and traceability tracking.

Description

Technical Field

[0001] The present invention belongs to the fields of deep learning, computer vision and information security, and specifically relates to a watermark imperceptible embedding and recovery method based on a deep learning network structure. Background Art

[0002] With the increasing diversification of digital content dissemination channels and the widespread popularity of digital devices, especially mobile devices and high-resolution displays, the risk of obtaining and illegally disseminating digital content through screen capture is increasing. Traditional anti-screen capture watermarking algorithms mostly rely on stable feature points for design. For example, many schemes embed the watermark into the discrete cosine transform (DCT) of the SIFT feature region to improve the robustness and invisibility of the watermark. However, such schemes rely on image region positioning and are prone to watermark decoding failure due to pixel offset in complex screen capture scenarios. Deep learning-based anti-screen capture watermarking schemes have been widely studied in recent years. Real screen capture noise is simulated and involved in end-to-end training to improve the robustness of the watermark. Existing solutions often sacrifice some of the visual quality of the image in pursuit of robustness, resulting in reduced watermark invisibility. Secondly, when faced with complex real-world shooting scenes, the intensity of real noise may far exceed the simulation range, resulting in a serious decrease in the robustness of the watermark when the noise intensity of the real scene is very high. How to solve the stability and robustness of watermark extraction under different noise intensities is still an urgent problem to be solved (see Figure 1 ). Summary of the Invention

[0003] The present invention discloses a watermark imperceptible embedding and recovery method based on a deep learning network structure, namely, a screen shooting robust watermarking method based on contrastive learning and mask-guided embedding strategy. The method aims to solve the problems of unstable watermark information extraction and degraded image visual quality caused by various factors such as lighting, geometric distortion and interference noise in screen shooting scenes in existing watermarking technologies, thereby achieving efficient copyright protection and tracking of digital content.

[0004] The implementation steps include two stages: deep learning network training and deep learning network inference:

[0005] (1) During the training phase, the preprocessed images and watermark information are used to train the deep learning network structure to obtain optimized network parameters. The images and watermark information are in multiple groups. By setting the batch_size, the tensor of each feature can be expressed as (B, C, H, W), where B represents the number of images or watermark information contained in a batch. The batch_size designed in the present invention during training is 16.

[0006] (2) In the inference phase, the trained network parameters are used to input the preprocessed image and watermark information into the deep learning network, and the recovered watermark information is finally output and the decoding accuracy is obtained;

[0007] The task of the present invention is to embed watermark information into an image and ensure that the watermark information is embedded in an imperceptible way. The method of the present invention is to use a neural network to find the texture-rich area in the image to be embedded. , and pass Through continuous optimization of loss and optimizer during the training process, the influence of embedded watermark information on embedded image is reduced, and the output There is also watermark information because it is necessary to ensure that the watermark can be accurately extracted during the traceability process, that is, during the watermark extraction process. The higher the extraction accuracy of the embedded watermark information, the stronger the robustness of the model.

[0008] Decoding is The bit accuracy (Bit Acc) is used to measure the accuracy of watermark decoding. Bit Acc is used to calculate the percentage of correctly recovered bits in the decoded binary watermark, thereby intuitively reflecting the effect of watermark extraction. The specific calculation process is as follows:

[0009] set up: is the ith watermark obtained after quantization (rounded off), m i is the i-th bit of the original watermark, and N is the total number of watermark bits.

[0010] The calculation formula for BitAcc is:

[0011]

[0012] in: When the corresponding bit is correct, the value is 0, and when it is wrong, the value is 1; Indicates the overall error rate; the higher the BitAcc value, the better the accuracy of watermark decoding and the stronger the robustness of the model.

[0013] During the network training process, the loss function is composed of message loss, image loss and contrast loss. When the loss value tends to be stable and no longer decreases, it indicates that the current network has converged, which means that the training is completed under the current circumstances.

[0014] The fusion mentioned herein refers to:

[0015] First, the pre-processed watermark information is first mapped to the size of the carrier image (the shape of the watermark feature map is 1×128×128, where 1 is the number of channels of the watermark feature map and 128 is the height and width). At this time, the carrier image and the watermark feature map are spliced ​​on the channel to obtain the feature map F of the fused watermark information. m1 , F m1 The size of is (4, 128, 128). The second fusion operation is to extract features from the carrier image and obtain a binary mask I through the mask generation module. B , use IB With the convolutional layer F m1 Perform pixel-by-pixel multiplication to guide the embedding of watermark information into the binary mask I B The area with 1 in the middle.

[0016] Use deep learning network to process input images and watermarks:

[0017] The deep learning network structure includes a watermark information preprocessing module, a mask generation module, and a decoder module based on contrastive learning;

[0018] (1) Watermark information preprocessing module

[0019] For length m∈{0,1} 30 The binary watermark sequence ({0, 1} 30 The watermark information to be embedded in the carrier image is a sequence of 0 and 1 bits. It is first mapped to a higher-dimensional feature space and reshaped into a feature map of size 1×128×128. The reshaped watermark feature map is then concatenated with the carrier image (the carrier image is an RGB image of size (3, 128, 128)) along the channel dimension. This concatenation operation is performed along the channel dimension, combining the watermark feature map with the RGB channel data of the carrier image. Specifically, the concatenated result is a 4-channel image, where the first channel corresponds to the watermark feature map, and the next three channels correspond to the three color channels of the RGB image. This is done to combine the watermark information with the original image, forming a watermarked image, while ensuring that the watermark information is effectively embedded in the image. In short, the concatenation process involves concatenating the 1×128×128 watermark feature map with the 3×128×128 RGB image along the channel dimension, ultimately resulting in a 4-channel image containing both the RGB information of the original image and the watermark information. Get the composite feature map F containing watermark information m ∈R 1×128×128 For subsequent embedding; F m is obtained after preprocessing the watermark information, F m The size of is 1×128×128. The F obtained after splicing with the carrier image on the channel m1 It is 4×128×128.

[0020] (2) Mask generation module

[0021] The mask generation module consists of a residual dense feature extraction module and an attention mask generation module. The residual dense feature extraction module captures the rich features in the carrier image through residual dense connections, effectively improving the feature expression capability; the attention mask generation module generates a mask based on these extracted features and selects areas in the carrier image suitable for watermark embedding. The model (i.e., the learning network) generates an attention mask based on the feature map of the input image and selects areas with the above characteristics to embed watermarks. Through the attention mechanism, the model can focus on important areas in the image (such as areas with rich textures and high redundancy) and embed watermarks in these areas to ensure that the watermark is not only hidden but also robust, that is, it can still be effectively extracted under subsequent image processing or attacks.

[0022] Residual dense feature extraction module:

[0023] This paper proposes a Residual Dense Feature Extraction Module (RDFEB) architecture that combines dense connections with residual connections via a Residual Dense Block (RDB). RDFEB employs a multi-level feature extraction mechanism where features are gradually accumulated and fused, enabling the model to mine rich contextual information. Furthermore, RDFEB incorporates global residual connections and a residual scaling factor (β = 0.2), which improves network stability and ensures that low-level information is effectively preserved in deep structures.

[0024] Attention Mask Generation Module:

[0025] In order to dynamically adjust the distribution of watermarks in different areas of the image, this paper proposes an Attention Mask Generation Block (AMGB). AMGB generates an attention mask by identifying the important areas of the carrier image to guide the location allocation of watermark embedding. After feature extraction through RDFEB, AMGB outputs a single-channel attention mask I M Regions with higher values ​​in the mask represent areas that are more important for the robustness and invisibility of the watermark. Furthermore, to further enhance the region selection capability, the mask is binarized to clearly distinguish between key and non-key areas for watermark embedding. This approach ensures that the watermark is concentrated in prominent areas with rich textures or sharp edges, thereby reducing the visual visibility of the watermark and improving its robustness against interference.

[0026] (3) Decoder module

[0027] The decoder module based on contrastive learning uses the contrastive loss function to take the decoding features of the same watermark image under different noise conditions as positive samples and the decoding features of different watermark images as negative samples, and optimizes the model of the training process to enhance the consistency of decoding features, thereby improving the robustness of the watermark.

[0028] The goal of the contrastive loss function is to optimize the model training process by calculating the similarities and differences between positive samples (decoding features of the same watermark image under different noise conditions) and negative samples (decoding features of different watermark images).

[0029] During training, the model automatically optimizes parameter adjustments using a backpropagation algorithm based on a contrastive loss function. This means the loss function calculates the difference between the current network output and the desired output, and then updates the network parameters using an optimization algorithm like gradient descent (the optimizer used in this paper is Adam). Through continuous iteration of this process, the model will increasingly learn to distinguish the features of different watermarked images and strengthen the consistency of the same watermark under different noise conditions.

[0030] By adjusting the model's parameters, the network gradually learns to maintain consistent decoded features despite varying noise interference, thereby enhancing the robustness of the watermark. During training, once the model can effectively distinguish between positive and negative samples and the network parameters are optimized, the decoder can more accurately recover the watermark information. This process is typically repeated until the loss function converges, achieving the desired effect, with the feature distance between positive samples being smaller and the feature distance between negative samples being larger.

[0031] The watermark information preprocessing module first maps the binary watermark sequence of length m∈{0,1}30 to a higher-dimensional feature space and reshapes it into a feature map of size 1×128×128; and then splices the reshaped watermark feature map with the carrier image in the channel dimension to obtain a composite feature map F containing the watermark information. m ∈R 1×128×128 For subsequent embedding;

[0032] The mask generation module consists of two key parts: the residual dense feature extraction module and the attention mask generation module;

[0033] The residual dense feature extraction module combines dense connections with residual connections. Through a multi-level feature extraction mechanism, it accumulates and fuses feature information layer by layer, effectively mining the rich contextual information of the image. The residual dense feature extraction module further nests multiple residual dense block structures and adds a global residual connection (skip connection) to them. In addition, by adopting a residual scaling factor (β = 0.2), the stability of the deep network is further enhanced.

[0034] The attention mask generation module is based on this module to identify the important areas in the carrier image and provide guidance for the embedding of watermark information. Its design is based on the idea of ​​adaptive weight distribution. With the help of the attention mechanism, an attention mask ranging from [0, 1] is generated on the feature map. cR The attention generation module outputs a single-channel mask I M ,First, the higher the value of this mask, indicates that the ,region has a greater influence on the robustness and invisibility of the ,watermark embedding., ,Second, the mask provides adaptive region allocation, ,which enables the watermark to be concentrated on the texture-rich ,regions of the carrier image, thereby improving the ,anti-interference and visual hiding effects of the watermark.

[0035] In order to further enhance the role of attention mask in feature selection, the feature mask I M Binarization is performed to make the attention mask more clearly distinguish between the key areas and secondary areas where the watermark needs to be embedded. Through this design, the attention generation module realizes the regional distribution of the watermark, making the watermark information more concentrated in the salient areas of the image. These salient areas are generally high-texture complexities or edge areas, making the watermark embedded in these areas more visually difficult to detect, thereby improving the invisibility of the watermark;

[0036] The contrastive learning-based decoding network constrains the consistency of decoding features in different noise domains (in this invention, both with and without noise), so that the decoder can maintain stable feature extraction capabilities in the face of screen noise of different intensities.

[0037] The decoder consists of a feature extraction module (Extractorj (j = 1, 2, 3)) consisting of multiple convolutional layers and Squeeze-and-Excitation Block (SEBlock). Figure 5 As shown. SEBlock strengthens the expression of important features by adaptively adjusting the weights of feature channels, effectively improving the ability of feature extraction. Through this hierarchical design, multi-scale deep features of the image are extracted layer by layer, and the obtained feature map is compressed through a 1x1 convolution layer. Finally, the obtained single-channel feature map is flattened and passed through a fully connected layer and quantization operation to generate the final decoding information.

[0038] The overall process of the method of the present invention can be divided into the following three stages:

[0039] (1) Watermark embedding stage

[0040] At this stage, the present invention first constructs a composite feature map using the carrier image and the binary watermark information to be embedded. The specific steps include:

[0041] ●Residual dense feature extraction and multi-scale fusion

[0042] The Residual Dense Feature Extraction Block (RDFEB) is used to perform multi-level feature extraction on the carrier image to capture the local texture and global structure information in the image.

[0043] Mask generation and guided embedding

[0044] After feature fusion, the Attention Mask Generation Block (AMGB) generates a single-channel attention mask based on the extracted features. This mask can adaptively distinguish between texture-rich areas and smooth areas. Subsequently, localized embedding is achieved by embedding the watermark information only in the areas with high attention.

[0045] (2) Screen shooting noise simulation stage

[0046] In order to truly restore the various distortion factors that may be encountered during screen shooting, the present invention designs a comprehensive noise simulation module, which mainly includes:

[0047] ●Perspective distortion simulation: Use random affine or perspective transformations to simulate geometric deformation caused by changes in shooting angle;

[0048] ● Lighting and illumination change simulation: Use point spread function and line spread function to simulate uneven screen lighting and ambient light changes;

[0049] Moiré interference: Moiré effect is created by sampling frequency differences, reproducing the interference phenomenon between the screen and the camera sensor;

[0050] ●Gaussian noise: simulates other random noise factors.

[0051] By weighted combination of the above-mentioned noise types, a distorted image that conforms to the actual screen shooting scene is generated, providing challenging input for the watermark decoding stage.

[0052] (3) Watermark decoding stage

[0053] To address the potential shift in watermark features after distortion, this paper constructs a watermark decoding network based on contrastive learning. Its core is to achieve consistency in watermark features between the original and distorted images through positive and negative sample contrast constraints. The main process includes:

[0054] ●Multi-layer feature extraction

[0055] The decoding network adopts a multi-layer convolutional structure and Squeeze-and-Excitation (SE) module to extract watermark information layer by layer, ensuring that subtle features can be captured at different scales.

[0056] ●Contrastive learning mechanism

[0057] By extracting features from the same watermark image under noise-free and noisy conditions to form positive sample pairs, and taking different watermark image features as negative samples, the contrast loss function is used to make the feature distance of the positive sample pairs closer and the distance of the negative sample pairs farther, so that the watermark features remain highly consistent even under complex noise conditions.

[0058] The CLME framework is a screen capture watermarking technology based on deep learning. Its overall network structure mainly consists of a mask-guided watermark embedding module (MGEM), a screen capture noise layer, and a watermark decoding module. Each module aims to ensure the watermark's concealment while improving the system's robustness to capture distortion:

[0059] (1) Mask-guided watermark embedding module (MGEM)

[0060] This module is responsible for mapping the original binary watermark into a high-dimensional feature space, concatenating it with the cover image to form a composite feature map, and then embedding the watermark into the image in an adaptive manner. Specifically, MGEM consists of two key submodules:

[0061] Residual Dense Feature Extraction Module (RDFEB)

[0062] By combining residual connections with dense connections, the multi-level local and global features of the image are extracted layer by layer, so that the low-level texture information and high-level semantic information can be fully integrated.

[0063] Attention Mask Generation Module (AMGB)

[0064] Based on the features extracted by RDFEB, AMGB generates a single-channel attention mask. The high-value areas in the mask correspond to the parts of the image with rich textures or obvious edges, thereby guiding the watermark to be embedded only in these areas and avoiding obvious traces in smooth areas.

[0065] (2) Screen shot noise layer

[0066] To simulate the various distortions introduced during the real screen shooting process, the noise layer simulates multiple interferences on the watermark image. This layer comprehensively considers:

[0067] ●Perspective distortion: simulates geometric deformation under different shooting angles through random homography transformation;

[0068] Lighting distortion: Use point diffusion and line diffusion distribution to simulate brightness changes caused by changes in screen light source and ambient light;

[0069] Moire distortion: irregular texture caused by differences in sampling frequency;

[0070] In addition, Gaussian noise with a mean of 0 and a standard deviation of σ is superimposed to simulate other random interferences.

[0071] Mathematically, the distorted image I′ w It can be expressed as:

[0072]

[0073] in Indicates perspective distortion, Indicates lighting distortion. Indicates moiré distortion, represents zero-mean Gaussian noise (obeying normal distribution), σ represents the standard deviation (0,, used to simulate other random distortions, standard deviation σ = 0.001; w1 = 0.85, w2 = 0.15 are the distortion rates of their respective distortions, respectively, to control the intensity of each distortion.

[0074] (3) Watermark decoding module

[0075] The module is designed as a multi-layer convolutional neural network with Squeeze-and-Excitation (SE) blocks and residual connections embedded in its structure to ensure that the embedded watermark can be accurately restored after the image is disturbed by the noise layer.

[0076] The specific process includes:

[0077] ● Perform multi-layer convolution feature extraction on the input watermark image (including the original embedded image and the image after noise processing);

[0078] ●Use 1×1 convolutional layers to compress the number of channels, and recover the watermark bits through fully connected layers and quantization operations;

[0079] The design of multi-layer residual connections effectively prevents gradient vanishing and ensures the full transmission of detailed information.

[0080] (4) Feature Consistency Comparison Mechanism (FCCM)

[0081] To improve the robustness and stability of watermark extraction, this paper designs a Feature Consistency Contrastive Mechanism (FCCM). This mechanism constrains the consistency of decoding features in different noise domains (in this paper, both noisy and noise-free conditions), enabling the decoder to maintain stable feature extraction capabilities even in the presence of screen noise of varying intensities. The FCCM method is implemented through the following steps:

[0082] Step 1: Normalize the feature vector

[0083] The decoder starts from the distorted image I′ after the noise layer w And the image I without the noise layer w There will be scale differences in the features extracted from , and direct comparison of these features may lead to misjudgment. In order to eliminate these effects, the feature vectors obtained by decoding are first normalized; specifically: let f i ′ and f i They represent the feature vectors of the i-th sample, which come from the features obtained without and after decoding the noise layer. The normalized feature vectors are and It can be expressed as:

[0084]

[0085] Step 2 Calculate the cosine similarity matrix

[0086] In order to quantify the consistency of the decoding features of the watermark image in the noise-free and noise-containing cases, the present invention calculates the cosine similarity between the normalized features and obtains the similarity matrix S∈R N×N , where each element S i，j Represents the features extracted from the i-th sample under noise-free conditions The features extracted from the jth sample under noise conditions Similarity between:

[0087]

[0088] Where τ is a temperature scaling parameter that aims to smooth the distribution of similarity, making the model more stable during training. Step 3 defines the positive and negative sample labels

[0089] To constrain the consistency of features extracted by the decoder through contrastive learning, the present invention defines a positive and negative sample pair for each sample. Specifically, a positive sample pair is a feature pair decoded from the same watermarked image under noise-free conditions and under noise conditions; a negative sample pair is a feature pair decoded from images with different watermarks. The relationship between positive and negative samples is defined by the label vector y:

[0090] y i =i,y∈{0,1,...,N-1} (5)

[0091] where y i =i indicates that the i-th sample is a positive sample pair and is located on the diagonal position of the similarity matrix.

[0092] Step 4: Calculate contrast loss

[0093] Next, by using the contrastive loss To minimize the similarity between positive sample pairs and maximize the similarity between negative sample pairs:

[0094]

[0095] Among them, the numerator exp(S i，i ) represents the similarity of the positive sample of the i-th sample, and the denominator represents the sum of the similarities of all samples in the i-th row. In this way, the model is encouraged to improve the similarity of positive samples and reduce the similarity of negative samples.

[0096] Among them, the edge mask E i It is generated by the BDCN network, and the image loss Specifically, it can be expressed as follows:

[0097] The gradient mask is generated by derivatizing the carrier image with respect to information loss. The gradient mask represents the robustness of each pixel on the carrier image for watermark information recovery (i.e., whether the current pixel is suitable for embedding watermark information, and represents the strength of each pixel suitable for embedding watermark information).

[0098]

[0099] Where N represents the number of images in a batch, I w Represents the image after embedding watermark, I c represents the original carrier image, μ1 and μ2 are weight parameters, which are 0.5 and 2 respectively; the gradient mask G i The generation of is as follows:

[0100]

[0101] The 1- in front is because the gradient needs to be reversed after the gradient is calculated and normalized. The purpose of the reversal is because the gradient represented by each pixel is calculated at the beginning. Some pixels represent larger values. These larger values ​​are more robust and more suitable for watermark embedding. The purpose of 1- is to allow these pixels to obtain larger weight modifications, so the gradient value needs to be reversed to allow pixels with large gradients to be modified more during the training process; and the +1 at the end is because the final gradient mask is multiplied with the carrier image and the watermark image respectively and then calculated through the mse loss. The +1 can be understood as being introduced to make the loss effect more obvious. The middle part is a normalization operation, and all values ​​will be unified to the range of 0-1. The formula can also be written as:

[0102]

[0103] A screen capture watermarking device based on a deep learning network structure includes a watermark embedding and extraction module that processes input images using the aforementioned method. The proposed network design fully integrates a mask-guided embedding mechanism and a contrastive learning-driven decoding strategy, enhancing its adaptability to real-world image noise at the data source and feature extraction levels. This deep learning network structure, referred to as the CLME network structure in this paper, primarily consists of the following three components:

[0104] 1. Mask Guided Embedding Module (MGEM):

[0105] RDFEB efficiently captures multi-scale and multi-layer texture features in the input image through multi-layer residual connections and dense convolution operations. AMGB, on the other hand, generates a single-channel attention mask based on the features extracted by RDFEB, where higher mask values ​​correspond to areas with rich texture or distinct edges. The generated attention mask is then element-wise multiplied with the preliminary feature map, confining the watermark embedding to key areas. This reduces overall visual noise and effectively enhances the watermark's stealth.

[0106] 2. Screen shot noise simulation layer:

[0107] In order to realistically simulate the various types of distortion that may be introduced during the actual screenshot process, the network introduces a noise simulation layer after the watermark is embedded. This layer uses preset models such as random parameterized perspective transformation, lighting distortion, moiré generation, and Gaussian noise to perform various perturbations on the watermark image, thereby generating the perturbed watermark image I′ w The specific mathematical expression is:

[0108]

[0109] in Indicates perspective distortion, Indicates lighting distortion. Indicates moiré distortion, represents zero-mean Gaussian noise, which is used to simulate other random distortions, with a standard deviation of σ = 0.001; w1 = 0.85, w2 = 0.15 are the distortion rates of their respective distortions, respectively, to control the intensity of each distortion.

[0110] 3. Decoding network and feature extraction module based on contrastive learning:

[0111] The decoding network is used to decode the watermark image I′ from the noise-corrupted watermark image w The embedded watermark information is accurately extracted from the decoder, and a contrastive learning strategy is used to ensure that the features of the same watermark remain highly consistent under different noise conditions. The decoding network mainly consists of two submodules: feature extraction module and contrastive learning module.

[0112] (1) Feature extraction module:

[0113] The module consists of multiple convolutional layers, batch normalization layers, and nonlinear activation functions (such as ReLU), while integrating Squeeze-and-Excitation (SE) blocks to enhance key feature responses. Each convolutional layer achieves effective fusion of multi-scale features through residual connections, extracting feature representations containing rich watermark information. Subsequently, the feature representation is mapped to a watermark bit sequence through a fully connected layer. And use the information loss function to ensure that it is consistent with the original watermark m i The mathematical expression of information loss is:

[0114] From the distorted image I′ w The watermark information f decoded from i With the original watermark information m i loss

[0115]

[0116] From the watermark image I w The watermark information f decoded from i ′ and the original watermark information m i loss

[0117]

[0118] (2) Comparative learning module:

[0119] To improve the robustness and stability of watermark extraction, this paper designs a Feature Consistency Contrastive Mechanism (FCCM). This mechanism constrains the consistency of decoding features in different noise domains (in this paper, both noisy and noise-free conditions), enabling the decoder to maintain stable feature extraction capabilities even in the presence of screen noise of varying intensities. The FCCM method is implemented through the following steps:

[0120] Step 1: Normalize the feature vector

[0121] The decoder starts from the distorted image I′ after the noise layer w And the image I without the noise layer w There will be scale differences in the features extracted from , and direct comparison of these features may lead to misjudgment. In order to eliminate these effects, the feature vectors obtained by decoding are first normalized; specifically: let f i ′ and f i They represent the feature vectors of the i-th sample, which come from the features obtained without and after decoding the noise layer. The normalized feature vectors are and It can be expressed as:

[0122]

[0123] Step 2 Calculate the cosine similarity matrix

[0124] In order to quantify the consistency of the decoding features of the watermark image in the noise-free and noise-containing cases, the present invention calculates the cosine similarity between the normalized features and obtains the similarity matrix S∈R N×N , where each element S i，j Represents the features extracted from the i-th sample under noise-free conditions The features extracted from the jth sample under noise conditions Similarity between:

[0125]

[0126] Where τ is a temperature scaling parameter that aims to smooth the distribution of similarity, making the model more stable during training. Step 3 defines the positive and negative sample labels

[0127] To constrain the consistency of features extracted by the decoder through contrastive learning, the present invention defines a positive and negative sample pair for each sample. Specifically, a positive sample pair is a feature pair decoded from the same watermarked image under noise-free conditions and under noise conditions; a negative sample pair is a feature pair decoded from images with different watermarks. The relationship between positive and negative samples is defined by the label vector y:

[0128] y i =i,

[0129] y∈{0, 1, ..., N-1} (12)

[0131] Where yi=i indicates that the i-th sample is a positive sample pair and is located on the diagonal position of the similarity matrix.

[0132] Step 4 Calculate contrast loss

[0133] Next, by using the contrastive loss To minimize the similarity between positive sample pairs and maximize the similarity between negative sample pairs:

[0134]

[0135] Among them, the numerator exp(S i，i ) represents the similarity of the positive sample of the i-th sample, and the denominator represents the sum of the similarities of all samples in the i-th row. In this way, the model is encouraged to improve the similarity of positive samples and reduce the similarity of negative samples.

[0136] The distance between positive sample pairs is shortened and the distance between negative sample pairs is increased, thereby ensuring that the watermark features extracted by the decoding network in various noise environments are highly consistent.

[0137] In summary, the CLME network structure not only achieves high-accuracy watermark extraction while ensuring the concealment of watermarks, but also significantly improves the application value of screenshot watermarking technology in the fields of digital copyright protection and information security.

[0138] First, a mask-guided watermark embedding scheme is designed. A mask generation module is constructed using a residual dense feature extraction module and an attention mask generation module. This adaptively guides the watermark embedding into texture-rich areas of the image to improve the watermark's invisibility. Second, a watermark decoding network based on contrastive learning is constructed. Using a contrastive loss function, the decoded features of the same watermarked image under different noise conditions are used as positive samples, while the decoded features of different watermarked images are used as negative samples to enhance the consistency of the decoded features and thus improve the robustness of the watermark.

[0139] Beneficial effects:

[0140] This technology introduces mask-guided embedding and contrastive learning mechanisms in the screenshot watermark embedding process, effectively alleviating the impact of multiple noises and distortions introduced by screen shooting on the accuracy of watermark extraction from the data source and feature learning levels.

[0141] Specifically, this invention discloses a screen capture watermarking technology and device based on a deep learning network architecture. By introducing a mask-guided embedding module into the watermark embedding process, it efficiently captures texture and edge regions within the image, embedding the watermark information only within key areas and minimizing interference with the overall visual effect. To address various common distortions in screen capture, this method designs a contrastive learning-driven decoding network. By constructing a comparison of positive and negative samples, it optimizes feature consistency and ensures accurate watermark extraction in a variety of noisy environments.

[0142] The present invention implements a method and device for screen capture-resistant robust watermarking based on a deep learning network structure. Compared with the existing technology, the present invention has the following advantages:

[0143] 1. Enhanced adaptability: This invention integrates a mask-guided embedding module and a contrastive learning-driven decoding network, significantly improving the adaptability of the deep learning network to various noise and distortion conditions during screen capture.

[0144] 2. Stronger robustness: Through an improved contrastive learning mechanism, this paper solves the problem of insufficient simulation of noise intensity in most methods. The positive and negative sample comparison strategy designed can keep the characteristics of the same watermark information consistent in different noise environments, significantly improving the robustness of watermark extraction.

[0145] 3. Excellent generalization ability: in a variety of real scenarios and public datasets (such as MS-COCO, USC-SIPI

[0146] The tests on

[15] ,

[16] ,

[17] ,

[18] ,

[19] ,

[20] ,

[21] ,

[22] ,

[23] ,

[24] ,

[25] ,

[26] ,

[27] ,

[28] ,

[29] ,

[30] ,

[31] ,

[32] ,

[33] ,

[34] ,

[35] ,

[36] ,

[37] ,

[38] ,

[39] BRIEF DESCRIPTION OF THE DRAWINGS

[0147] Figure 1 This is the application scenario of de-identification in the present invention;

[0148] Figure 2 It is a schematic diagram of the architecture of the present invention;

[0149] Figure 3 Schematic diagram of the mask generation module (Mask Generation Module) described in the present invention;

[0150] Figure 4 Schematic diagram of the noise layer described in the present invention;

[0151] Figure 5 Schematic diagram of the decoder described in the present invention;

[0152] Figure 6A schematic diagram of comparative learning label definition described in the present invention;

[0153] Figure 7 Schematic diagram of watermark image visualization of the present invention and the comparative solution;

[0154] Figure 8 Schematic diagram of the real shooting environment, the captured watermark image and the corrected watermark image;

[0155] Figure 9 Schematic diagram of the feature map visualization of the present invention and PIMoG under different noise intensities ( Figure 9 (a) is the feature map visualization under different moiré noise intensities; Figure 9 (b) is the visualization of feature maps under different light intensities); DETAILED DESCRIPTION

[0156] The present invention will be further described below with reference to the accompanying drawings and embodiments. However, this should not be construed as limiting the scope of the present invention to the following embodiments, as all technologies implemented based on the present invention fall within the scope of the present invention.

[0157] This paper addresses the challenges of robustness and visual quality degradation in real-world screen capture watermarks by proposing a deep learning network architecture based on contrastive learning and mask-guided embedding. This method uses an adaptive masking mechanism to automatically identify texture-rich or edge-prominent regions within an image and precisely embed the watermark information there, thus avoiding noticeable visual artifacts in smooth areas. Furthermore, a contrastive learning strategy ensures that the features extracted from the same watermarked image remain highly consistent after exposure to noise of varying intensities, significantly improving both the accuracy and robustness of watermark extraction.

[0158] Example

[0159] This embodiment provides a method and apparatus for robustly protecting screen capture watermarks based on contrastive learning and mask-guided embedding. This method aims to address the problem of reduced watermark extraction accuracy in traditional screen capture watermarking methods, both in real-world capture and digital editing environments, due to noise interference and artifacts in smooth image regions, while also ensuring that the embedded watermark maintains high visual quality. By introducing a contrastive learning mechanism to ensure feature consistency under various noise conditions and utilizing adaptive masks to guide watermark embedding in textured or edge-sensitive areas, this embodiment effectively resists various interferences generated during the screenshot process, such as perspective, lighting, moiré patterns, and Gaussian noise.

[0160] 1. Overall architecture and workflow

[0161] like Figure 2 As shown in Figure 1, the overall process of CLME is divided into the following stages:

[0162] 1) Watermark preprocessing and composite feature generation

[0163] Watermark mapping: The input binary watermark sequence (e.g., length 30) is mapped to a high-dimensional feature space through a fully connected layer and a nonlinear activation function to generate a watermark feature map of size 1×128×128.

[0164] Image preprocessing: Normalize, denoise and resize the input original image (uniformly adjusted to 128×128) to ensure that the image features match the watermark features in scale.

[0165] Feature fusion: Splice the watermark feature map with the original image in the channel dimension to generate a composite feature map Provides a joint representation for subsequent watermark embedding. (See Figure 2 ,)

[0166] 2) Mask Guided Embedding Module (MGEM)

[0167] The goal of this module is to adaptively select the area suitable for embedding watermarks according to the image content (the network architecture of MGEM is shown in the figure below). Figure 3 ), specifically including:

[0168] Residual Dense Feature Extraction Block (R0FEB)

[0169] The multi-scale features of the image are extracted using a multi-layer residual dense block structure. Each RDB fully integrates local details through dense connections, while introducing global residual connections and residual scaling (such as β = 0.2) to ensure that low-level information is not lost.

[0170] Attention Mask Generation Block (AMGB)

[0171] According to the features output by RDFEB, a convolutional layer and activation function (such as ReLU and Sigmoid) are used to generate a single-channel attention mask I M The mask is binarized and can accurately identify areas with rich textures or obvious edges, so that the watermark is embedded only in these areas, avoiding the introduction of visual artifacts in smooth areas. (See Figure 3 ,)

[0172] 3) Watermark embedding and distortion simulation

[0173] Embedding process: The composite feature map Fm is passed through a normal convolution layer and the attention mask I B (Binary mask generated by AMGB) is weighted pixel by pixel to obtain the preliminary watermark residual map Afterwards, Add it to the original image Ic to get the watermarked image Iw.

[0174] Noise simulation: To simulate various interferences that may be encountered in real screen shooting, a noise layer N(·) is designed. This layer integrates four main disturbance models (such as Figure 4 shown):

[0175] Perspective distortion The geometric distortion at different angles is simulated by random homography transformation.

[0176] Lighting distortion Point diffusion and line diffusion models are used to simulate changes in screen light source and ambient lighting.

[0177] Moire distortion The moiré effect is generated by using the difference in sampling frequency.

[0178] Gaussian noise η: obey N(0, σ 2 )(such as σ=0.001) random noise.

[0179] Combining the above disturbances, the distorted image I′ w It is expressed by the following formula:

[0180]

[0181] Among them, w1 (such as 0.85) and w2 (such as 0.15) are weights that control the intensity of various disturbances. (See Figure 4 )

[0182] 4) Decoder and Feature Consistency Comparison Module (FCCM)

[0183] The decoder structure is as follows Figure 5 As shown in Figure 2, it mainly consists of multiple layers of convolution, Squeeze-and-Excitation modules and 1×1 convolution layers. The decoder starts from the distorted image I′ w The watermark features are extracted and a single-channel feature vector is generated. To ensure that the features extracted in the presence and absence of noise are highly consistent, the following contrastive learning module is designed:

[0184] Feature Normalization:

[0185] Step 1: Normalize the feature vector

[0186] The decoder starts from the distorted image I′ after the noise layer w And the image I without the noise layer w There will be scale differences in the features extracted from , and direct comparison of these features may lead to misjudgment. In order to eliminate these effects, the feature vectors obtained by decoding are first normalized; specifically: let f i ′ and f iThey represent the feature vectors of the i-th sample, which come from the features obtained without and after decoding the noise layer. The normalized feature vectors are and It can be expressed as:

[0187]

[0188] Step 2 Calculate the cosine similarity matrix

[0189] In order to quantify the consistency of the decoding features of the watermark image in the noise-free and noise-containing cases, the present invention calculates the cosine similarity between the normalized features and obtains the similarity matrix S∈R N×N , where each element S i，j Represents the features extracted from the i-th sample under noise-free conditions The features extracted from the jth sample under noise conditions Similarity between:

[0190]

[0191] Where τ is the temperature scaling parameter, which aims to smooth the distribution of similarity and make the model more stable during training; τ = 0.07.

[0192] Step 3: Define positive and negative sample labels

[0193] To constrain the consistency of features extracted by the decoder through contrastive learning, the present invention defines a positive and negative pair for each sample. Specifically, a positive pair is a feature pair obtained by decoding the same watermarked image in the absence of noise and in the presence of noise; a negative pair is a feature pair obtained by decoding different watermarked images. The present invention defines the relationship between positive and negative samples using a label vector y:

[0194] y i =i,

[0195] y∈{0, 1, ..., N-1} (17)

[0197] where y i =i indicates that the i-th sample is a positive sample pair and is located on the diagonal position of the similarity matrix.

[0198] Step 4: Calculate contrast loss

[0199] Next, by using the contrastive loss To minimize the similarity between positive sample pairs and maximize the similarity between negative sample pairs:

[0200]

[0201] Among them, the numerator exp(Si，i ) represents the similarity of the positive sample of the i-th sample, and the denominator represents the sum of the similarities of all samples in the i-th row. In this way, the model is encouraged to improve the similarity of positive samples and reduce the similarity of negative samples.

[0202] 2. Loss Function Design

[0203] To ensure both accurate decoding of the watermark information and visual fidelity of the watermark-embedded image, this embodiment designs the following loss function:

[0204] 1) Information loss

[0205] From the distorted image I′ w The watermark information f decoded from i With the original watermark information m i loss

[0206]

[0207] From the watermark image I w The watermark information f decoded from i ′ and the original watermark information m i loss

[0208]

[0209] 2) Image loss

[0210] In order to effectively improve the visual quality of the watermark image during the watermark embedding process, the present invention uses the edge mask guided image loss and the gradient mask guided image loss constructed based on the MSE loss, and the two are used together as the image loss in the scheme of the present invention; wherein, the edge mask E i It is generated by the BDCN network.

[0211] Image loss Specifically, it can be expressed as follows:

[0212]

[0213] Where N represents the number of images in a batch, I w Represents the image after embedding watermark, I c represents the original carrier image, μ1 and μ2 are weight parameters, which are 0.5 and 2 respectively; the gradient mask G i The generation of can be expressed as follows:

[0214]

[0215] Combining the above parts, the total loss function is defined as:

[0216]

[0217] Among them, λ1, λ2, λ3, and λ4 are the weight coefficients of their respective losses, which are 3, 3, 1, and 1 respectively.

[0218] 3. Training Process and Experimental Setup

[0219] 1) Dataset preparation and preprocessing

[0220] Training set: 10,000 images were randomly sampled from the MS-COCO dataset, all resized to 128 × 128 pixels. A binary watermark sequence of length 30 was randomly generated, and the watermark mapping module generated a watermark feature map, which was concatenated with the image to form a composite input.

[0221] Test set: USC-SIPI dataset is used, and watermark images displayed on the screen are captured using a mobile phone (such as iPhone 15 Pro) in a real screen environment. After perspective correction, they are used as inference test data. (See Figure 7 )

[0222] 2) Network training and parameter setting

[0223] Training environment: NVIDIA GeForce RTX 4060Ti(16-bit) hardware platform, using PyTorch 2.4.1.

[0224] Optimizer: Adam optimizer is used, the initial learning rate is set to le-4, and the batch size is set to 16.

[0225] Training cycle: Train for 100 epochs and use a cosine decay scheduler to gradually reduce the learning rate.

[0226] Noise parameters: In the noise layer parameters, the perspective and illumination perturbation weights w1 = 0.85, the moiré perturbation weight w2 = 0.15, and the Gaussian noise standard deviation σ is set to 0.001 to ensure that the noise during training simulates the interference in real shooting.

[0227] 3) Reasoning stage

[0228] The process of the inference phase is as follows (see Figure 8 (Instructions):

[0229] Embed the watermark into the image and display it on the screen and take a photo with your phone;

[0230] Perform perspective correction on the captured image;

[0231] The corrected image is input into the trained decoder, the watermark features are extracted by the decoder, and the final watermark information is quantized and restored.

[0232] Experiments show that the watermark extraction accuracy can reach 97.08% at different shooting distances from 20cm to 100cm and at multiple angles, while the PSNR of the embedded image reaches 42.624dB.

[0233] 4. Experimental Results and Evaluation

[0234] 1) Visual quality assessment

[0235] By comparing the HiDDeN, Stegastamp, PIMoG, and SSDS methods, Table 1 shows that this embodiment has a significant advantage in visual quality:

[0236] Table 1 Visual quality test

[0237]

[0238] Figure 7 The watermark images and corresponding residual images generated by different methods are shown, which proves that the proposed method performs well in avoiding artifacts in smooth areas.

[0239] 2) Robustness testing

[0240] In the actual screen shooting experiment, tests were conducted by setting different shooting distances (20cm~100cm), shooting angles (±40°) and multiple device combinations (such as P30Pro, iPhone15Pro, etc.). The experimental results are shown in Tables 2, 3 and 4.

[0241] In this method, a watermarked image is displayed on a screen and photographed using a mobile phone at distances ranging from 20 to 100 cm. After perspective correction, the captured images are decoded and the watermark information is extracted. The extraction results are shown in Table 2. Experimental results show that the proposed method maintains an extraction accuracy of over 99.38% at all shooting distances.

[0242] Table 2 Robustness test (different shooting distances)

[0243]

[0244]

[0245] The present invention displays the watermarked image on the screen and uses a mobile phone to shoot it from different shooting angles at a fixed distance (50 cm). The shooting angle range is 40° to 40° from left to right and 40° to 40° from top to bottom. The extracted watermark results are shown in Table 3, which lists the watermark extraction performance at different angles (40° to left / right and 30° to 20° to 10° from top to bottom). The experimental results show that the proposed scheme can maintain optimal robustness with a significant advantage at most shooting angles.

[0246] Table 3 Robustness test (different shooting angles)

[0247]

[0248] The present invention was experimented on three different display screens (Q27G2S / D, I2490VXH / BS, ViewSonic) and four different mobile phones (HuaweiP30Pro, iPhone15Pro, iQOO Neo7, RedMi K60). In the experimental setup, the watermark image was displayed on the screen and then photographed using a mobile phone at a shooting distance of 20 cm. Table 4 shows the watermark extraction accuracy under these different device combinations. As can be seen from the table, the watermark extraction accuracy of the proposed method under all device combinations remains above 0.9875, showing high robustness and stability.

[0249] Table 4 Robustness test (different shooting equipment)

[0250]

[0251]

[0252] 3) Noise interference robustness

[0253] To verify the robustness of the proposed scheme under different noise conditions, we fixed the noise intensity and tested its watermark decoding capabilities under moiré distortion and varying illumination intensity. As shown in Table 5, the proposed scheme is more robust than PIMoG under different moiré distortion intensities.

[0254] Table 5 Robustness test (different moiré intensities)

[0255]

[0256] Considering that the change of light intensity during the shooting process will also affect the watermark extraction performance, in order to verify the robustness of the proposed scheme under different light intensities, the present invention fixed other noises except the light intensity, increased the light intensity from 0.5 to 1.5, and compared it with PIMoG. From Table 6, it can be seen that when the light intensity changes, the watermark extraction accuracy of the proposed CLME can be maintained above 99.38%, while the robustness of PIMoG watermark is not good under weak and strong light conditions. Similarly, Figure 7 As shown in the figure, the present invention visualizes the feature maps decoded under different light intensities. Compared with the light intensity of 1, when the light is dark, the decoding features of PIMoG lose some effective information, resulting in a decrease in watermark extraction accuracy. However, CLME can still ensure the consistency of decoding features under different light intensities, further verifying that the proposed scheme can still decode high-precision watermark information when the noise intensity changes.

[0257] Table 6 Robustness test (different light intensities)

[0258]

[0259] 5. Application prospects and technical advantages

[0260] The main technical advantages of this embodiment are as follows:

[0261] High concealment

[0262] Adaptive attention mask (AMGB) is used to guide the watermark to be embedded only in areas with rich textures or obvious edges, effectively avoiding obvious interference in smooth areas and reducing the risk of watermark detection or tampering.

[0263] High robustness

[0264] The noise simulation layer is used to realistically reproduce various interferences in screen shooting, and a contrastive learning module (FCCM) is introduced to enable the watermark to maintain highly consistent features under different noise conditions, ensuring that the accuracy of watermark extraction always remains at an extremely high level.

[0265] End-to-end training and flexibility

[0266] The overall network structure supports end-to-end optimization. Each module (RDFEB, AMGB, noise layer, FCCM) can be independently adjusted and combined to meet the requirements of different application scenarios, and has good scalability and migration capabilities.

[0267] 6. Summary

[0268] This embodiment provides a screenshot watermark embedding and extraction method and apparatus based on contrastive learning and mask-guided embedding, achieving excellent performance through the following key innovations:

[0269] A watermark preprocessing module is used to generate high-dimensional watermark features and fuse them with the image to ensure information consistency; the residual dense feature extraction and attention mask generation module (MGEM) are used to accurately embed the watermark in texture-rich areas, greatly improving the watermark concealment; a feature consistency comparison module (FCCM) based on contrastive learning is designed to ensure that the decoder extracts highly consistent watermark features under noisy and noise-free conditions, thereby improving the overall robustness.

[0270] Experimental results show that this embodiment can maintain a high extraction accuracy in various practical screen capture scenarios. These results fully demonstrate the huge potential of this invention in application scenarios such as digital copyright protection, screen content anti-counterfeiting, and cross-device secure transmission.

[0271] Those skilled in the art may make various modifications and improvements based on this embodiment, and all such modifications should be considered to be within the scope of protection of the present invention.

Claims

1. A watermark imperceptible embedding and recovery method based on deep learning network structure, characterized by: The input image is extracted through a deep learning network and fused with the pre-processed watermark information; That is, through the attention mechanism, we can focus on important areas in the input image (such as areas with rich textures and high redundancy) and embed watermarks in these areas to ensure that the watermark is not only hidden but also robust, that is, it can still be effectively extracted under subsequent image processing or attacks.

2. The watermark imperceptible embedding and recovery method based on deep learning network structure according to claim 1 is characterized in that: The implementation steps include two stages: deep learning network training and deep learning network inference: (1) In the training phase, the preprocessed image and watermark information are used to train the deep learning network structure to obtain optimized network parameters; (2) In the inference stage, the preprocessed image and watermark information are input into the deep learning network using the trained network parameters, and the recovered watermark information is finally output and the decoding accuracy is obtained.

3. The watermark imperceptible embedding and recovery method based on deep learning network structure according to claim 1 is characterized in that: The fusion mentioned herein refers to: First, the pre-processed watermark information is first mapped to the size of the carrier image. At this time, the shape of the watermark feature map is 1×128×128, where 1 is the number of channels of the watermark feature map, and 128 is the height and width. At this time, the carrier image and the watermark feature map are spliced ​​on the channel, and the feature map F of the fused watermark information is obtained. m1 , F m1 The size of is (4, 128, 128). The second fusion operation is to extract features from the carrier image and obtain a binary mask I through the mask generation module. B , use I B With the convolutional layer F m1 Perform pixel-by-pixel multiplication to guide the embedding of watermark information into the binary mask I B The area with 1 in the middle.

4. The watermark imperceptible embedding and recovery method based on deep learning network structure according to claim 1 is characterized in that: Use deep learning network to process input images and watermarks: The deep learning network structure includes a watermark information preprocessing module, a mask generation module, and a decoder module based on contrastive learning; (1) Watermark information preprocessing module For length m∈{0,1} 30 The binary watermark sequence {0, 1} 30 The watermark information to be embedded in the carrier image refers to a sequence of 0 and 1 bits. First, it is mapped to a higher-dimensional feature space and reshaped into a feature map of size 1×128×128. The reshaped watermark feature map is concatenated with the carrier image in the channel dimension to obtain a composite feature map F containing the watermark information. m ∈R 1×128×128 For subsequent embedding; (2) Mask generation module The mask generation module consists of a residual dense feature extraction module and an attention mask generation module. The residual dense feature extraction module captures rich features in the carrier image through residual dense connection, effectively improving the feature expression ability; The attention mask generation module generates masks based on these extracted features and selects areas in the carrier image that are suitable for watermark embedding; The residual dense feature extraction module uses a multi-level feature extraction mechanism to accumulate and fuse feature information layer by layer, effectively mining the rich contextual information of the image. The residual dense feature extraction module further nests multiple residual dense block structures and adds a global residual skip connection to them. In addition, by adopting a residual scaling factor β = 0.2, the stability of the deep network is further strengthened and critical low-level information is preserved during the high-level feature processing. The attention mask generation module is based on this module to identify the important areas in the carrier image and provide guidance for the embedding of watermark information. Its design is based on the idea of ​​adaptive weight distribution. With the help of the attention mechanism, an attention mask ranging from [0, 1] is generated on the feature map. cR The attention generation module outputs a single-channel mask I M ,First, the higher the value of this mask, indicates that the ,region has a greater influence on the robustness and invisibility of the ,watermark embedding., ,Second, the mask provides adaptive region allocation, ,which enables the watermark to be concentrated on the texture-rich ,region of the carrier image, thereby improving the ,anti-interference and visual hiding effect of the watermark. In order to further enhance the role of attention mask in feature selection, the feature mask I M Perform binarization processing to make the attention mask more clearly distinguish between the key areas and secondary areas where watermarks need to be embedded, making the watermark embedded in these areas more visually difficult to detect, thereby improving the invisibility of the watermark; (3) Decoder module The decoder module based on contrastive learning uses the contrastive loss function to take the decoding features of the same watermark image under different noise conditions as positive samples and the decoding features of different watermark images as negative samples, and optimizes the model of the training process to enhance the consistency of decoding features, thereby improving the robustness of the watermark.

5. The watermark imperceptible embedding and recovery method based on a deep learning network structure according to any one of claims 1 to 4, characterized in that: In order to ensure both accurate decoding of watermark information and visual fidelity of the watermark-embedded image, the following loss function is designed for the deep learning network: The total loss function is defined as: Among them, λ1, λ2, λ3, and λ4 are the weight coefficients of their respective losses, which are 3, 3, 1, and 1 respectively; From the distorted image I′ w The watermark information f decoded from i With the original watermark information m i Losses: From the watermark image I w The watermark information f decoded from i ′ and the original watermark information m i Losses: For image loss: Use edge mask guided image loss and gradient mask guided image loss constructed based on MSE loss, and use both as image loss; Among them, the edge mask E i is generated by the BDCN network, the gradient mask G i Through information loss For carrier image I c Derivative, the image gradient obtained Image loss Specifically, it can be expressed as follows: Among them, the edge mask E i It is generated by the BDCN network, and the image loss The specific expressions are as follows: Where N represents the number of images in a batch, I w Represents the image after embedding watermark, I c represents the original carrier image, μ1 and μ2 are weight parameters, which are 0.5 and 2 respectively; the gradient mask G i The generation of is as follows:

Images