A robust text image watermarking method against screen capture by integrating edge attention gating and multi-scale dilated convolution.

Abstract

This invention provides a robust text image watermarking method against screen capture by fusing edge attention gating and multi-scale dilated convolution. The method includes: Step 1, designing a message processor (MessageProProcess) to preprocess the secret message, adding an expanded network (ExpandNet) and a channel attention network (SENet) to the MessageProProcess, inputting a binary secret message and outputting a message feature map; Step 2, constructing a text watermarking framework based on the fusion of edge attention gating and multi-scale dilated convolution; adding functional modules to the encoder of the end-to-end watermarking framework; Step 3, designing a noise layer to simulate attacks on the model during watermarking model training; Step 4, designing an optimization method for the loss function. This invention significantly improves the robustness and imperceptibility of the watermark.

Description

Technical Field

[0001] This invention belongs to the field of information security, and in particular relates to a robust text image watermarking method that integrates edge attention gating and multi-scale dilated convolution to resist screen capture. Background Technology

[0002] With the rapid development of internet information technology, the transmission of text content has become increasingly convenient, and text information has become an important part of online communication. As a crucial carrier of text information dissemination, the need for copyright protection of text images is becoming increasingly prominent. With the development of multimedia technology, document leaks during video conferences occur frequently. Especially with the widespread use of smartphones, taking photos has become the simplest and most effective way to transmit information. Anyone with access to documents (such as authorized employees) can steal and leak information by taking photos without leaving any record. Furthermore, the process of taking photos is difficult to monitor or prevent from the outside world. Against this backdrop, digital watermarking technology, with its powerful traceability capabilities, has become a key technical means to combat screen capture leaks and protect copyright. Digital watermarking embeds encrypted identification information, ensuring that even if important information such as contracts and documents are photographed and disseminated without authorization, the watermark information remains intact and concealed, thus enabling traceability.

[0003] Digital watermarking has made significant progress in addressing text leaks. Text watermarking methods can be broadly categorized into three types: semantic, structural, and image-based. Structural watermarking methods embed secret information by modifying the format, layout, or content structure of the text. Their focus is on adjusting the external presentation of the text while maintaining its semantic integrity. These watermarking schemes lack robustness and have limited capacity. Semantic watermarking relies on Natural Language Processing (NLP) techniques, providing a highly concealed watermark embedding scheme by fine-tuning the semantics of the text content or modifying its syntax. However, the application scenarios for this type of method are limited, and the watermark capacity is also relatively small. Image-based text watermarking methods treat the text carrier directly as an image. However, to avoid affecting the visual quality of the text content, these methods make only minor modifications to the text carrier, resulting in poor robustness.

[0004] In recent years, an increasing number of watermarking algorithms based on Convolutional Neural Networks (CNNs) have been proposed. The basic framework of these watermarking algorithms is an end-to-end autoencoder architecture, comprising an encoder, a decoder, and a noise layer. A well-designed noise layer ensures the robustness of the watermarking algorithm; training with a targeted noise layer guarantees this robustness. However, most CNN-based watermarking frameworks are designed for natural images, while text images cannot be directly treated as natural images. Text images typically carry a higher information density; the arrangement of characters and symbols constitutes the main carrier of information. This dense arrangement makes text images more susceptible to influence during watermark embedding. Any small change can directly affect the readability and accuracy of the text, a stark contrast to the visual tolerance of natural images. Therefore, during watermark embedding, text images need to ensure that readability is not compromised. In particular, text images possess unique structural characteristics; local and edge features (such as clear character boundaries and fine textures) significantly impact the clarity and visual integrity of the image content. Unlike the smooth transitions and complex textures in natural images, text images are extremely sensitive to distortions introduced during watermark embedding, especially in high-contrast areas such as character edges. Therefore, designing a screen-capture-resistant robust text watermarking scheme is crucial to addressing this issue. Summary of the Invention

[0005] Purpose of the Invention: In scenarios such as business meetings and video conferences, confidential information in text form accounts for the majority of cases. To address the problem of text image leakage in these scenarios, this invention provides a robust text image watermarking method against screen capture by integrating edge attention gating and multi-scale dilated convolution, comprising the following steps:

[0006] Step 1: Design a message processor MessageProProcess to preprocess secret messages. Add an expanded network ExpandNet and a channel attention SENet to the message processor MessageProProcess. Input a secret message in binary form and output a message feature map.

[0007] Step 2: Construct a text watermarking framework based on the fusion of edge attention gating and multi-scale dilated convolution; add functional modules to the encoder of the end-to-end watermarking framework, including: multi-branch dilated convolution module MBDC-block, edge-guided gating mechanism Edge-Gated Mechanism, hybrid pooling layer and regularized deactivation layer DropBlock; first, perform ConvBNReLU convolution activation on the input carrier image to obtain the low-level feature map;

[0008] The low-level feature map is input into the multi-branch dilated convolutional module MBDC-block to obtain the image feature map, and the low-level feature map is input into the lightweight edge detection network LedgeNet to obtain the edge feature map.

[0009] The image feature map and edge feature map are subjected to residual connection to obtain fused features; the fused features are input into a hybrid pooling layer for compression, the output of the hybrid pooling layer is then fed into a regularized deactivation layer DropBlock for regularization, and finally, the image features are fed into a fully connected layer and fused with the message feature map.

[0010] Step 3: Design a noise layer to simulate attacks on the model during the watermark model training process;

[0011] Step 4: Design the optimization method for the loss function.

[0012] Step 1 includes:

[0013] Step 1-1: Add a message processor (MessageProProcess) to process the message, then provide the feature map to the encoder to convert the one-dimensional secret message M of length L into two-dimensional data, and then reshape the two-dimensional data into a two-dimensional secret message. h and w represent the height and width of the two-dimensional message, respectively; R represents the real number space.

[0014] Steps 1-2, the reshaped two-dimensional secret message After initial feature extraction using a ConvBNReLU convolutional activation layer with a kernel size of 3, the image is then fed into the ExpandNet network for n 2x upsampling operations, expanding it to match the carrier image. Message feature mapping of the same size Where C is the number of channels in the carrier image, and H and W are the height and width of the carrier image, respectively;

[0015] Steps 1-3: The expanded message feature map is fed into the channel attention network SENet to obtain message feature maps. .

[0016] Step 2 includes:

[0017] Step 2-1, first use the carrier image tensor The initial input is encoded into a low-level semantic feature map after undergoing a primary convolution operation. Low-level semantic feature map The input is fed in parallel into three dilated convolutional branches with different dilation rates, and feature maps are constructed using convolutional kernels with dilation rates d of 1, 2, and 3, respectively. To enhance the modeling capabilities for local details, mesoscale structures, and global context, the formula is as follows:

[0018] ,

[0019] in and These represent the kernel weights and biases of the dilated convolution branch with an expansion rate of d, respectively. It is a non-linear ReLU activation function; It is a two-dimensional convolution operation;

[0020] Step 2-2: Design a variant of the globally nested edge detection (HED) architecture based on the residual network ResNet-18 in the encoder; the variant is a lightweight edge detection network LedgeNet, used to generate explicit edge guidance maps. This guides network feature extraction and fusion; k=1,2,3, representing the output at low, medium, and high scales, respectively. The three scales are selected from the first three feature layers of the ResNet-18 residual network.

[0021] Steps 2-3, Edge Guiding Map As an explicit attention mask, it participates in the gating and fusion process of the encoder's feature channels to obtain the enhanced feature map after edge modulation. ;

[0022] Steps 2-4: To achieve joint modeling of edge responses and original semantic features, residual connections are used to enhance the feature maps. With feature map Fusion, resulting in fusion characteristics :

[0023] ;

[0024] A hybrid pooling layer is established in the encoder, which includes a max pooling layer and an average pooling layer; fused feature maps are output from three scales. The feature is compressed in the hybrid pooling layer to form an edge-enhanced sensing feature map. ;

[0025] Steps 2-5: Enhance the edge-aware feature map The input encoder's regularized deactivation layer, DropBlock, performs random deactivation. Finally, the processed feature vector is fed into a fully connected layer to obtain the final image features. ;

[0026] Final image features Compared with the upsampled message feature map The images are stitched together, convolved, and mapped, then combined with the carrier image tensor φ. C Skip connections are made, and finally a 1×1 convolution is used to generate an encoded image containing the watermark. .

[0027] Steps 2-3 include: First, the edge guide map... After 1×1 convolution dimensionality reduction and several bilinear interpolation upsampling operations, a feature map with an expansion rate of d is obtained. Corresponding edge feature map Subsequently, to enhance the display of edge region response, an edge-guided channel space soft-gated mask is constructed, and a gating weight map is generated through lightweight convolution and a sigmoid activation function. :

[0028] ,

[0029] in This represents a 1×1 convolution operation. For the Sigmoid activation function, the gate weight graph is... This indicates the importance response of the edge region within each branch channel space. The number of channels representing branches with an expansion rate of d;

[0030] Gating weight graph and feature graph Perform a fusion operation to obtain an enhanced feature map with edge modulation. :

[0031] ,

[0032] Where ⊙ represents the Hadamarda.

[0033] In steps 2-4, the max pooling layer is used to enhance the texture, and the average pooling layer is used to preserve the overall structure, as shown below:

[0034] ,

[0035] ,

[0036] in, This represents the compressed feature map of the output at pooling coordinate (i, j) of the c-th channel after max pooling.

[0037] This represents the compressed feature map of the output at pooling coordinate position (i, j) of the c-th channel after processing by the average pooling layer.

[0038] Let represent the feature value of the c-th channel at position (x, y);

[0039] x and y represent the x-coordinate and y-coordinate of the input feature map, respectively. This represents the input box area corresponding to the horizontal coordinate position i. The area of ​​the input box corresponding to the vertical coordinate position j is represented as:

[0040] ,

[0041] ,

[0042] in and These represent the height and width of the target output after pooling, respectively. The pooled feature map is obtained by element-wise weighted fusion of the two compressed feature maps. :

[0043] ,

[0044] in The hyperparameters are set; This represents the pooling feature fusion map of the output at pooling coordinate position (i, j) of the c-th channel after processing by the hybrid pooling layer;

[0045] Calculate the mean response of the entire image from the pooled feature fusion graph:

[0046] ,

[0047] in Pooling feature fusion map with expansion rate d The mean of the full-map response; The value of the c-th channel at spatial location (x, y) in the pooling feature fusion graph representing the branch with expansion rate d;

[0048] The fusion weights for structural adaptability are obtained by softmax normalization of the mean response of the entire graph. :

[0049] d = 1,2,3

[0050] Where e represents a natural number. Indicates to Perform exponentiation;

[0051] Finally, the feature maps from the three dilated convolution branches are weighted and fused to form an edge-enhanced feature map. :

[0052] .

[0053] In step 4, the model's loss function is divided into the following three parts: encoder loss. Decoder loss and discriminator loss ,

[0054] Encoder loss Decoder loss and discriminator loss The corresponding loss weights are respectively , , ;

[0055] An adaptive loss weight adjustment mechanism is introduced to balance the training of the encoder and decoder, as shown in the following formula:

[0056] ,

[0057] ,

[0058] in, and These are the loss weights of the encoder and decoder in the nth epoch, respectively. It is a constant. and These are the average loss of the encoder and the average loss of the decoder, respectively.

[0059] In step 4, the adaptive loss weight adjustment mechanism also incorporates a moving average smoothing mechanism; for the encoder, a smoothing factor is used. The weight is as follows at each update:

[0060] ,

[0061] in This represents the encoder loss weight for the (n+1)th epoch after adjustment using a smoothing factor.

[0062] In step 4, the final total loss function L is expressed as:

[0063] ,

[0064] in These represent the encoder loss weights and decoder loss weights adjusted using a smoothing factor, respectively. It is the coding loss, calculated using the following formula:

[0065] ,

[0066] in Represents carrier image With encoded image The mean square error, The parameter representing encoder E, Representative message feature graph; This indicates that the encoder processes the input image. and message feature graph The encoding process outputs a watermarked image. ;

[0067] It is the decoding loss, calculated using the following formula:

[0068] ,

[0069] in This represents the secret message M and the decoded message. binary cross-entropy, The parameters representing decoder D, This indicates that decoder D is in the parameters Below, for noisy images Process the code and output the decoded secret message. ;

[0070] It is the discriminator loss, calculated using the following formula:

[0071] ,

[0072] Where A is the discriminator. These are the parameters of discriminator A; Indicates in the parameter Below, the discriminator's discrimination result on the encoded image output by the encoder. This indicates that the discriminator recognizes the carrier image. The judgment result.

[0073] The present invention also provides an electronic device, including a processor and a memory, the memory storing program code that, when executed by the processor, causes the processor to perform the steps of the method.

[0074] The present invention also provides a storage medium storing a computer program or instructions that, when the computer program or instructions are run on a computer, execute the steps of the method described.

[0075] This invention offers the following advantages: It utilizes a combination of multi-scale dilated convolution and edge gating to achieve fine-grained watermark embedding across multiple scales. Furthermore, it designs an adaptive loss weight adjustment mechanism. This mechanism dynamically adjusts the loss weights of the encoder and decoder based on the loss changes during each training epoch. To avoid drastic weight fluctuations, a smoothing strategy is introduced, using a moving average method to gradually update the weights, thereby ensuring the stability of the training process. Experiments demonstrate that the method of this invention significantly improves the robustness and imperceptibility of the watermark. Attached Figure Description

[0076] Figure 1 This is a diagram illustrating the overall framework of the method of this invention.

[0077] Figure 2 This is an architecture diagram of the message processor and encoder.

[0078] Figure 3 This invention relates to LedgeNet, a lightweight edge detection network. Detailed Implementation

[0079] The present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments, and the advantages of the present invention in the above and / or other aspects will become clearer.

[0080] This invention provides a robust text image watermarking method that integrates edge attention gating and multi-scale dilated convolution. The overall framework is as follows: Figure 1 As shown, including:

[0081] Step 1: Design a message processor MessageProProcess to preprocess secret messages. Add an expanded network ExpandNet and a channel attention SENet to the message processor MessageProProcess. Input a secret message in binary form and output a message feature map.

[0082] Step 2: Construct a text watermarking framework based on the fusion of edge attention gating and multi-scale dilated convolution; add functional modules to the encoder of the end-to-end watermarking framework, including: multi-branch dilated convolution module MBDC-block, edge-guided gating mechanism Edge-Gated Mechanism, hybrid pooling layer and regularized deactivation layer DropBlock; first, perform ConvBNReLU convolution activation on the input carrier image to obtain the low-level feature map;

[0083] The low-level feature map is input into the multi-branch dilated convolutional module MBDC-block to obtain the image feature map, and the low-level feature map is input into the lightweight edge detection network LedgeNet to obtain the edge feature map.

[0084] The image feature map and edge feature map are subjected to residual connection to obtain fused features; the fused features are input into a hybrid pooling layer for compression, the output of the hybrid pooling layer is then fed into a regularized deactivation layer DropBlock for regularization, and finally, the image features are fed into a fully connected layer and fused with the message feature map.

[0085] Step 3: Design a noise layer to simulate attacks on the model during the watermark model training process;

[0086] Step 4: Design the optimization method for the loss function.

[0087] Step 1 includes:

[0088] Step 1-1: To better support the encoding process, a message processor (MessageProProcess) is added to handle the message. The feature map is then provided to the encoder, converting the one-dimensional secret message M of length L into two-dimensional data for easier input into the convolutional layer for processing. Finally, it is reshaped into a two-dimensional secret message. (h and w are the height and width of the two-dimensional message, respectively);

[0089] Steps 1-2, the reshaped two-dimensional secret message After initial feature extraction using a ConvBNReLU convolutional activation layer with a kernel size of 3, the image is then fed into the ExpandNet network, where it undergoes n 2x upsampling operations to expand its size to match that of the carrier image. Message feature mapping of the same size (C is the number of channels, and H and W are the height and width of the carrier image, respectively). This method enhances the spatial resolution of the message, making it closer to the resolution of the image, thereby allowing the message to spread as widely as possible across the entire feature map.

[0090] Steps 1-3: The expanded message feature map is fed into the channel attention network SENet to obtain a more detailed message feature map. .

[0091] Step 2 includes:

[0092] Step 2-1 First, use the carrier image tensor (where 𝐶 is the number of channels, and H and W are the height and width of the carrier image, respectively) is the initial input, which is encoded into a low-level semantic feature map after a primary convolution operation. Where R represents the real number space; underlying semantic feature map The input is fed in parallel into three dilated convolutional branches with different dilation rates, and feature maps are constructed using convolutional kernels with dilation rates d of 1, 2, and 3, respectively. To enhance the modeling capabilities for local details, mesoscale structures, and global context, the formula is as follows:

[0093] ,

[0094] in and These represent the kernel weights and biases of the dilated convolution branch with an expansion rate of d, respectively. It is a non-linear ReLU activation function; It is a special two-dimensional convolution operation that uses the convolution kernel Sliding action with an interval of size d on This expands the receptive field to about d times that of a normal convolution, but does not introduce additional parameters.

[0095] Among them, the dilated convolution branch with an expansion rate d of 1 mainly focuses on capturing local micro-details and can respond more sensitively to fine-grained texture structures such as character edges and stroke undulations; the dilated convolution branch with an expansion rate d of 2 has a medium receptive field and is suitable for recognizing texture or structural patterns at the word block level, which is beneficial for perceiving medium-scale semantic structures; the dilated convolution branch with an expansion rate d of 3 tends to model a broader context layout and global geometric relationships, which is beneficial for modeling and representing macro-text organization structures and strengthening the perception of global arrangement.

[0096] Step 2-2: To guide attention to character edge regions during feature extraction, this invention designs a variant of the globally nested edge detection (HED) model based on the ResNet-18 residual network, and a lightweight edge detection network, LedgeNet, to generate explicit edge guidance maps. , guides network feature extraction and fusion, k=1,2,3, representing the output at low, medium and high scales respectively.

[0097] Unlike the computationally complex VGG-16 backbone used in the original whole-nested edge detection (HED), this invention retains three scale outputs (low, medium, and high), selected from the first three feature layers of the ResNet-18 residual network, significantly reducing the number of model parameters and computational FLOPs. Figure 3 As shown.

[0098] Steps 2-3: First, create the edge guide map. After 1×1 convolution dimensionality reduction and several bilinear interpolation upsampling operations, a feature map with an expansion rate of d is obtained. Corresponding edge feature map Subsequently, to enhance the display of edge region response, an edge-guided channel space soft-gated mask is constructed, and a gating weight map is generated through lightweight convolution and a sigmoid activation function. :

[0099] ,

[0100] in This represents a 1×1 convolution operation. For the Sigmoid activation function, the gate weight graph is... This represents the importance response of the edge region within each branch channel space, where values ​​close to 1 indicate a significant amplification of the feature, while values ​​close to 0 suppress irrelevant regions. This represents the number of channels in a branch with an expansion rate of d. The gating weight map is compared with the feature map. Perform a fusion operation to obtain an enhanced feature map with edge modulation. :

[0101] ,

[0102] Where ⊙ represents the Hadamard product, i.e., element-wise multiplication, it significantly enhances the features of edge regions. In this invention, the residual modulation structure ensures that the features of non-edge regions are preserved as is, while edge regions are explicitly enhanced, thereby achieving response enhancement without introducing additional depth paths.

[0103] Steps 2-4: To achieve joint modeling of edge responses and original semantic features, residual connections are used to enhance the feature maps. With feature map Fusion, resulting in fusion characteristics :

[0104] ,

[0105] A hybrid pooling layer is established in the encoder, which includes a max pooling layer and an average pooling layer; fused feature maps are output from three scales. Feature compression is performed in the hybrid pooling layer.

[0106] The max pooling layer is used to enhance the texture, and the average pooling layer is used to preserve the overall structure, as shown below:

[0107] ,

[0108] ,

[0109] in, This represents the compressed feature map of the output at pooling coordinate (i, j) of the c-th channel after max pooling.

[0110] This represents the compressed feature map of the output at pooling coordinate position (i, j) of the c-th channel after processing by the average pooling layer.

[0111] This represents the feature value of the c-th channel at position (x, y), which is the input data for the pooling operation;

[0112] x and y represent the x-coordinate and y-coordinate of the input feature map, respectively. This represents the input box area corresponding to the horizontal coordinate position i. The area of ​​the input box corresponding to the vertical coordinate position j is represented as:

[0113] ,

[0114] ,

[0115] in and These represent the height and width of the target output after pooling, respectively. The pooled feature map is obtained by element-wise weighted fusion of the two compressed feature maps. :

[0116] ,

[0117] in For manually set hyperparameters, these represent weight parameters; This represents the pooling feature fusion map of the output at pooling coordinate position (i, j) of the c-th channel after processing by the hybrid pooling layer;

[0118] Calculate the mean response of the entire image from the pooled feature fusion graph:

[0119] ,

[0120] in Pooling feature fusion map with expansion rate d The mean of the full-map response; The value of the c-th channel at spatial location (x, y) in the pooling feature fusion graph representing the branch with expansion rate d;

[0121] The fusion weights for structural adaptability are obtained by softmax normalization of the mean response of the entire graph. :

[0122] d = 1,2,3

[0123] Where e represents the natural constant. Indicates to By performing exponential calculations, the differences are amplified, making the branches with larger responses more prominent, while the branches with smaller responses are further suppressed.

[0124] Finally, the feature maps from the three dilated convolution branches are weighted and fused to form an edge-enhanced feature map. :

[0125] ,

[0126] Steps 2-5: Enhance the edge-aware feature map The DropBlock regularization deactivation layer in the input encoder performs random deactivation, which enhances the robustness of the watermarking model to redundant features. It forces the model to make correct judgments even when some regions fail, which can effectively prevent overfitting and redundancy in the multi-scale path fusion process.

[0127] Finally, the processed feature vectors are fed into a fully connected layer to obtain the final image features. .

[0128] Final image features Compared with the upsampled message feature map 𝐼 m The images are stitched together, convolved, and mapped, then combined with the carrier image tensor φ. C Skip connections are made, and finally a 1×1 convolution is used to generate an encoded image containing the watermark. .

[0129] In step 3, within the noise layer, the encoder may encounter various noise attacks during the encoding process, leading to distortion. Without training, the watermarking scheme of this invention will be unable to withstand the effects of these noises. Therefore, this invention adds a noise layer to the model to simulate noise attacks. The noise layer includes several common types of noise: cropping, dropout, JPEG compression, Gaussian noise, and median filter.

[0130] For these five noise attacks, this invention controls the cropping attack intensity by controlling the cropping ratio r (0.3-0.7); controls the pixel loss attack intensity by controlling the percentage p (0.3-0.7) of the replaced image pixels compared to the original image pixels; controls the JPEG compression intensity by using the quality factor Q (30-70); controls the Gaussian noise intensity by using the Gaussian kernel width σ; and controls the median blur intensity by using the filter kernel k (3-7).

[0131] To further improve the robustness of the model against non-differentiable noise attacks, this invention adopts a mini-batch training strategy. When training the model's robustness to JPEG compression, each batch of training is randomly selected from the real JPEG layer, the simulated JPEG-Mask layer, and the noise-free attack-free Identity layer.

[0132] In step 4, the model's loss function mainly consists of the following three parts: encoder loss. Decoder loss and discriminator loss The corresponding loss weight , , The initial settings are 1, 10, and 0.0001.

[0133] In the watermarking scheme of this invention, the encoder aims to seamlessly hide secret information into the image while preserving the quality of the original image as much as possible. The decoder, on the other hand, aims to accurately extract the hidden information from the image. These two objectives often conflict in optimization; improving watermark extraction accuracy may sacrifice image quality, and vice versa. The discriminator has a relatively smaller impact on watermark quality.

[0134] Therefore, using fixed loss weights clearly cannot achieve the aforementioned dynamic balance requirement, because the encoder and decoder losses change differently at different training stages. Fixed loss weights may lead to instability during training, resulting in unsatisfactory final results. Therefore, this invention introduces an adaptive loss weight adjustment mechanism to balance the training of the encoder and decoder. The core of this adaptive loss weight adjustment method is to dynamically adjust the loss weights of the encoder and decoder through inverse proportional adjustment and moving average smoothing mechanisms. Specifically:

[0135] ,

[0136] ,

[0137] in, and These are the loss weights of the encoder and decoder in the nth epoch, respectively. It is a very small constant to prevent the denominator from being zero. and These are the average loss of the encoder and the average loss of the decoder, respectively.

[0138] To account for fluctuations in loss during training, adaptive weight adjustment incorporates a moving average smoothing mechanism to prevent weight volatility from affecting training stability. Taking the encoder as an example, a smoothing factor is used. The weight is as follows at each update:

[0139] ,

[0140] This represents the encoder loss weights for the (n+1)th epoch after adjustment using a smoothing factor.

[0141] Finally, the total loss function L is expressed as:

[0142] ,

[0143] in These represent the encoder and decoder loss weights adjusted using a smoothing factor, respectively. It is the coding loss, calculated using the following formula:

[0144] ,

[0145] in Represents carrier image With encoded image The mean square error, The parameter representing encoder E, Representative message feature graph; This indicates that the encoder processes the input image. and message feature graph The encoding process outputs a watermarked image, i.e. ;

[0146] It is the decoding loss, calculated using the following formula:

[0147] ,

[0148] in This represents the secret message M and the decoded message. binary cross-entropy, The parameters representing decoder D, This indicates that decoder D is in the parameters Below, for noisy images The process is performed to output the decoded secret message, i.e. ;

[0149] It is the discriminator loss, calculated using the following formula:

[0150] ,

[0151] Where A is the discriminator. These are the parameters of discriminator A; Indicates in the parameter Below, the discriminator's judgment result on the encoded image output by the encoder, that is, the probability that the image is considered a "real image", is similarly... This indicates that the discriminator recognizes the carrier image. The discrimination result; overall discriminator loss This represents the training objective of the discriminator when distinguishing between the original image and the watermarked image, which is used to improve its ability to distinguish between real and fake images, and also pushes the encoder to generate more "deceptive" images.

[0152] The method of this invention exhibits significant advantages in both robustness and concealment. In experimental evaluation, various common noise attack techniques (including cropping, dropout, JPEG compression, Gaussian noise, and median blur filter) were used to test the watermarking method. The results show that the invention can maintain a low bit error rate and a high peak signal-to-noise ratio (PSNR) and structural similarity (SSIM) under different distortion conditions, demonstrating excellent anti-screen capture capabilities.

[0153] In this embodiment, the evaluation metrics selected are Bit Error Rate (BER), Peak Signal-to-Noise Ratio (PSNR), and Structural Similarity (SSIM). Lower BER and higher PSNR and SSIM indicate a better model. This invention selects models from the papers HiDDeN, MBRS, and Adaptor as the control group to highlight the superiority of this invention.

[0154] Table 1. Experimental data on the Crop attack.

[0155]

[0156] Cropping involves randomly cutting the image from top / bottom and left / right, then filling in the missing areas with black pixels. The cropped image renders the watermark detection algorithm ineffective, resulting in distorted watermarks. In this embodiment, cropping ratios r of 0.3, 0.5, and 0.7 are selected as noise layers for training and testing. A cropping ratio of 0.5 represents the percentage of the remaining encoded image size relative to the original image size, meaning the cropped image size is 50% of the original. Experimental results show that even with partial watermark information loss, the watermarking method of this invention can still effectively recover the watermark information.

[0157] Table 2 Experimental data for the Dropout pixel loss attack

[0158]

[0159] In screen capture scenarios, some pixels may be lost during channel transmission due to signal interference, compromising the integrity of the watermark information. To simulate this situation, an experiment using a pixel loss attack (Dropout) is conducted, randomly removing some pixels and replacing them with pixels from the cover image to test the model's robustness under screen capture signal loss scenarios. In the experiments of this invention, noise levels of 0.3, 0.5, and 0.7 are used during the training phase. A Dropout rate of 0.5 indicates that the percentage of the remaining encoded image to the original image size is 50%. This method exhibits good robustness and imperceptibility. The method of this invention significantly outperforms other methods under pixel loss attacks, while maintaining high levels of SSIM and PSNR.

[0160] Table 3 Experimental data for JPEG compression

[0161]

[0162] JPEG compression is one of the most common lossy digital image compression methods. JPEG compression is based on discrete cosine transform, a process involving quantization and encoding, most of which are non-differentiable. Therefore, it's impossible to adjust model parameters through backpropagation. Directly using JPEG compression prevents gradient updates during training, hindering the model's ability to learn effective watermark embedding strategies. This invention employs a mini-batch noise addition method, randomly selecting one of the following as the new noise for each training mini-batch: simulated JPEG-Mask, real JPEG compression, or no noise. In the experiments, JPEG compression with quality factors of 30, 50, and 70 was used for training and testing. The experimental results show that as Q increases, the compression ratio decreases, image quality improves, and the peak signal-to-noise ratio (PSNR) and structural similarity (SSIM) also increase. The results demonstrate that the method of this invention performs excellently in terms of compressed image quality, bit error rate control, and structural preservation.

[0163] Table 4. Experimental data of Gaussian noise attack

[0164]

[0165] Gaussian noise is a random signal or interference that follows a Gaussian (normal) distribution. The main sources of this noise are sensor noise caused by low illumination or high temperature during screen capture, and Gaussian noise contamination of the image during image transmission due to imperfect channel characteristics. In this embodiment, σ values ​​of 0.001, 0.002, and 0.005 are used. 2 The model's resistance to Gaussian noise was evaluated. The model of this invention can improve peak signal-to-noise ratio (PSNR) and structural similarity (SSIM) while maintaining a bit error rate of 0.

[0166] Table 5. Experimental data for the Median Filter.

[0167]

[0168] Median blurring means that for each pixel, the median pixel value is taken instead of the pixel value at that position within a window centered on it. This method is called median smoothing, also known as median filtering. In this embodiment, different filter kernel sizes k (3, 5, and 7) were selected for median blurring attacks. Experimental results show that this invention maintains a high peak signal-to-noise ratio (PSNR) and structural similarity (SSIM) while having a significantly lower bit error rate than other methods, demonstrating superior performance in terms of information integrity and image quality.

[0169] This invention provides a robust text image watermarking method against screen capture by integrating edge attention gating and multi-scale dilated convolution. Many methods and approaches exist for implementing this technical solution; the above description is merely a preferred embodiment. It should be noted that those skilled in the art can make various improvements and modifications without departing from the principles of this invention, and these improvements and modifications should also be considered within the scope of protection of this invention. All components not explicitly stated in this embodiment can be implemented using existing technologies.

Claims

1. A robust text image watermarking method against screen capture by integrating edge attention gating and multi-scale dilated convolution, characterized in that, Includes the following steps: Step 1: Design a message processor MessageProProcess to preprocess secret messages. Add an expanded network ExpandNet and a channel attention SENet to the message processor MessageProProcess. Input a secret message in binary form and output a message feature map. Step 2: Construct a text watermarking framework based on the fusion of edge attention gating and multi-scale dilated convolution; add functional modules to the encoder of the end-to-end watermarking framework, including: multi-branch dilated convolution module MBDC-block, edge-guided gating mechanism Edge-Gated Mechanism, hybrid pooling layer and regularized deactivation layer DropBlock; first, perform ConvBNReLU convolution activation on the input carrier image to obtain the low-level feature map; The low-level feature map is input into the multi-branch dilated convolutional module MBDC-block to obtain the image feature map, and the low-level feature map is input into the lightweight edge detection network LedgeNet to obtain the edge feature map. The image feature map and edge feature map are subjected to residual connection to obtain fused features; the fused features are input into a hybrid pooling layer for compression, the output of the hybrid pooling layer is then fed into a regularized deactivation layer DropBlock for regularization, and finally, the image features are fed into a fully connected layer and fused with the message feature map. Step 3: Design a noise layer to simulate attacks on the model during the watermark model training process; Step 4: Design the optimization method for the loss function; In step 4, the model's loss function is divided into the following three parts: encoder loss. Decoder loss and discriminator loss , Encoder loss Decoder loss and discriminator loss The corresponding loss weights are respectively , , ; An adaptive loss weight adjustment mechanism is introduced to balance the training of the encoder and decoder, as shown in the following formula: ， ， in, and These are the loss weights of the encoder and decoder in the nth epoch, respectively. It is a constant. and These are the average loss of the encoder and the average loss of the decoder, respectively.

2. The method according to claim 1, characterized in that, Step 1 includes: Step 1-1: Add a message processor (MessageProProcess) to process the message, then provide the feature map to the encoder to convert the one-dimensional secret message M of length L into two-dimensional data, and then reshape the two-dimensional data into a two-dimensional secret message. h and w represent the height and width of the two-dimensional message, respectively; R represents the real number space. Steps 1-2, the reshaped two-dimensional secret message After initial feature extraction using a ConvBNReLU convolutional activation layer with a kernel size of 3, the image is then fed into the ExpandNet network for n 2x upsampling operations, expanding it to match the carrier image. Message feature mapping of the same size Where C is the number of channels in the carrier image, and H and W are the height and width of the carrier image, respectively; Steps 1-3: The expanded message feature map is fed into the channel attention network SENet to obtain message feature maps. .

3. The method according to claim 2, characterized in that, Step 2 includes: Step 2-1, first use the carrier image tensor The initial input is encoded into a low-level semantic feature map after undergoing a primary convolution operation. Low-level semantic feature map The input is fed in parallel into three dilated convolutional branches with different dilation rates, and feature maps are constructed using convolutional kernels with dilation rates d of 1, 2, and 3, respectively. To enhance the modeling capabilities for local details, mesoscale structures, and global context, the formula is as follows: ， in and These represent the kernel weights and biases of the dilated convolution branch with an expansion rate of d, respectively. It is a non-linear ReLU activation function; It is a two-dimensional convolution operation; Step 2-2: Design a variant of the globally nested edge detection (HED) architecture based on the residual network ResNet-18 in the encoder; the variant is a lightweight edge detection network LedgeNet, used to generate explicit edge guidance maps. , guides network feature extraction and fusion; k=1,2,3, representing the output at low, medium and high scales respectively, the three scales are selected from the first three feature layers of the ResNet-18 residual network; Steps 2-3, Edge Guiding Map As an explicit attention mask, it participates in the gating and fusion process of the encoder's feature channels to obtain the enhanced feature map after edge modulation. ; Steps 2-4: To achieve joint modeling of edge responses and original semantic features, residual connections are used to enhance the feature maps. With feature map Fusion, resulting in fusion characteristics : ； A hybrid pooling layer is built in the encoder, which includes a max pooling layer and an average pooling layer; fused feature maps are output from three scales. The feature is compressed in the hybrid pooling layer to form an edge-enhanced sensing feature map. ; Steps 2-5: Enhance the edge-aware feature map The input encoder's regularized deactivation layer, DropBlock, performs random deactivation. Finally, the processed feature vector is fed into a fully connected layer to obtain the final image features. ; Final image features Compared with the upsampled message feature map The images are stitched together, convolved, and mapped, then combined with the carrier image tensor φ. C Skip connections are made, and finally a 1×1 convolution is used to generate an encoded image containing the watermark. .

4. The method according to claim 3, characterized in that, Steps 2-3 include: First, the edge guide map... After 1×1 convolution dimensionality reduction and bilinear interpolation upsampling, a feature map with an expansion rate of d is obtained. Corresponding edge feature map Subsequently, to enhance the display of edge region response, an edge-guided channel space soft-gated mask is constructed, and a gating weight map is generated through lightweight convolution and a sigmoid activation function. : ， in This represents a 1×1 convolution operation. For the Sigmoid activation function, the gate weight graph is... This indicates the importance response of the edge region within each branch channel space. The number of channels representing branches with an expansion rate of d; Gating weight graph and feature graph Perform a fusion operation to obtain an enhanced feature map with edge modulation. : ， Where ⊙ represents the Hadamarda.

5. The method according to claim 4, characterized in that, In steps 2-4, the max pooling layer is used to enhance the texture, and the average pooling layer is used to preserve the overall structure, as shown below: ， ， in, This represents the compressed feature map of the output at pooling coordinate (i, j) of the c-th channel after max pooling. This represents the compressed feature map of the output at pooling coordinate position (i, j) of the c-th channel after processing by the average pooling layer. Let represent the feature value of the c-th channel at position (x, y); x and y represent the x-coordinate and y-coordinate of the input feature map, respectively. This represents the input box area corresponding to the horizontal coordinate position i. The area of ​​the input box corresponding to the vertical coordinate position j is represented as: ， ， in and These represent the height and width of the target output after pooling, respectively. The pooled feature map is obtained by element-wise weighted fusion of the two compressed feature maps. : ， in The hyperparameters are set; This represents the pooling feature fusion map of the output at pooling coordinate position (i, j) of the c-th channel after processing by the hybrid pooling layer; Calculate the mean response of the entire image from the pooled feature fusion graph: ， in Pooling feature fusion map with expansion rate d The mean of the full-map response; The value of the c-th channel at spatial location (x, y) in the pooling feature fusion graph representing the branch with expansion rate d; The fusion weights for structural adaptability are obtained by softmax normalization of the mean response of the entire graph. : ，d = 1,2,3， Where e represents a natural number. Indicates to Perform exponentiation; Finally, the feature maps from the three dilated convolution branches are weighted and fused to form an edge-enhanced feature map. : 。 6. The method according to claim 5, characterized in that, In step 4, the adaptive loss weight adjustment mechanism also incorporates a moving average smoothing mechanism; for the encoder, a smoothing factor is used. The weight is as follows at each update: ， in This represents the encoder loss weight for the (n+1)th epoch after adjustment using a smoothing factor.

7. The method according to claim 6, characterized in that, In step 4, the final total loss function L is expressed as: ， in These represent the encoder loss weights and decoder loss weights adjusted using a smoothing factor, respectively. It is the coding loss, calculated using the following formula: ， in Represents carrier image With encoded image The mean square error, The parameter representing encoder E, Representative message feature graph; This indicates that the encoder processes the input image. and message feature graph The encoding process outputs a watermarked image. ; It is the decoding loss, calculated using the following formula: ， in This represents the secret message M and the decoded message. binary cross-entropy, The parameters representing decoder D, This indicates that decoder D in the parameters Below, for noisy images Process the code and output the decoded secret message. ; It is the discriminator loss, calculated using the following formula: ， Where A is the discriminator. These are the parameters of discriminator A; Indicates in the parameter Below, the discriminator's discrimination result on the encoded image output by the encoder. This indicates that the discriminator recognizes the carrier image. The judgment result.

8. An electronic device, characterized in that, It includes a processor and a memory, the memory storing program code that, when executed by the processor, causes the processor to perform the steps of the method as described in any one of claims 1 to 7.

9. A storage medium, characterized in that, It stores a computer program or instructions that, when run on a computer, perform the steps of the method as described in any one of claims 1 to 7.

Images