Document watermark removing method and system

Abstract

The invention discloses a document watermark removing method and system which are applied to the field of document image processing, and the method comprises the steps: obtaining a document image containing a watermark; inputting the document image into a target network model for processing, and generating a watermark removal result; wherein the target network model comprises a semantic segmentation model, a rough prediction model and a semantic perception state space module which are connected in sequence; according to the watermark removing result, a total loss function is obtained through calculation; performing iterative training on the target network model based on the total loss function to obtain a trained target network model; and obtaining a target document image comprising a watermark, and inputting the target document image into the trained target network model to generate a watermarking-free result image.

Description

Technical Field

[0001] The present invention relates to the field of document image processing, and in particular to a document watermark removal method and system. Background Art

[0002] With the rapid development of deep learning and computer vision technology, image watermark removal technology has become an important research direction, especially in the field of document image processing. Document images are different from natural scene images in that their structured information is more complex. The presence of watermarks not only affects the visual quality of the image, but also seriously interferes with key applications such as optical character recognition (OCR). Therefore, the development of efficient and accurate image watermark removal technology has become an urgent need.

[0003] In order to overcome these defects, the present application proposes a document watermark removal method and system. Summary of the invention

[0004] The purpose of this application is to provide a document watermark removal method and system to solve the above problems.

[0005] To achieve the above objectives, this application provides the following technical solutions:

[0006] In a first aspect, the present application provides a document watermark removal method, comprising:

[0007] Get the document image containing the watermark;

[0008] Input the document image into the target network model for processing to generate a de-watermarking result; wherein the target network model is a semantic segmentation model, a coarse prediction model, and a semantic perception state space module connected in sequence;

[0009] According to the watermark removal result, a total loss function is calculated;

[0010] Iteratively training the target network model based on the total loss function to obtain a trained target network model;

[0011] A target document image including a watermark is obtained, and the target document image is input into the trained target network model to generate a result image without a watermark.

[0012] In a second aspect, the present application provides a document watermark removal system, comprising:

[0013] Acquisition module: acquires the document image containing the watermark;

[0014] Processing module: inputting the document image into the target network model for processing to generate a de-watermarking result; wherein the target network model is a semantic segmentation model, a coarse prediction model, and a semantic perception state space module connected in sequence;

[0015] Calculation module: calculates the total loss function according to the watermark removal result;

[0016] Training module: iteratively training the target network model based on the total loss function to obtain a trained target network model;

[0017] Generation module: obtains a target document image including a watermark, inputs the target document image into the trained target network model, and generates a watermark-free result image.

[0018] In a third aspect, the present application provides a device, comprising a processor and a memory coupled to the processor, wherein the memory stores program instructions for implementing a document watermark removal method; and the processor is used to execute the program instructions stored in the memory to implement a document watermark removal method.

[0019] In a fourth aspect, the present application provides a storage medium storing program instructions executable by a processor, wherein the program instructions are used to execute a document watermark removal method.

[0020] The present application provides a document watermark removal method and system, which has the following beneficial effects:

[0021] (1) Through the coarse prediction model and semantic-aware state space module, most watermarks in the document are efficiently removed, and the watermark residues and complex details are deeply processed, which significantly improves the integrity and precision of watermark removal;

[0022] (2) The state-space model is used to model linear complexity sequences, which significantly reduces the computational complexity in high-resolution image processing, optimizing from traditional quadratic complexity to linear complexity. It achieves a combination of low resource usage and high performance, and can run on mobile terminals and edge devices.

[0023] (3) In the semantic-aware state space module, the global information guided by semantics and local detail features are integrated to achieve accurate restoration of the watermark coverage area. This design enables the application to demonstrate excellent detail restoration capabilities in complex watermark areas, making document restoration more natural. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] Figure 1 A flowchart of a document watermark removal method according to Embodiment 1 of the present application;

[0025] Figure 2 A schematic diagram of a document watermark removal method according to Embodiment 1 of the present application;

[0026] Figure 3 A schematic diagram of the framework structure of a document watermark removal method according to Embodiment 1 of the present application;

[0027] Figure 4 This is a structural diagram of a document watermark removal system according to Embodiment 2 of the present application;

[0028] Figure 5 This is a schematic diagram of the device structure of Example 3 of the present application;

[0029] Figure 6 This is a schematic diagram of the storage medium structure of Example 4 of the present application. DETAILED DESCRIPTION

[0030] It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0031] The following is an analysis of the solutions in the prior art in combination with related technologies.

[0032] Based on the literature and existing research, document watermark removal technology mainly faces the following key problems:

[0033] (1) High computational complexity: Traditional Transformer-based image enhancement and restoration methods rely on attention mechanisms, and their computational complexity grows quadratically with the size of the input image. This high computational cost limits the application of these methods in high-resolution document image processing, especially on edge devices or mobile devices with limited resources.

[0034] (2) Insufficient generalization ability: Existing technologies usually use synthetic data for training, but document images in real scenes are significantly different from synthetic data in terms of brightness, texture, and watermark shape. This results in poor generalization ability of the model when processing real data and cannot effectively adapt to unlabeled actual documents.

[0035] (3) Insufficient processing of watermark diversity: Some current methods, such as those based on geometric shapes or color filtering, cannot effectively process complex watermarks, including transparent, gradient, and complex textured watermarks. This results in some watermarks being unable to be completely removed and may even damage the original document content.

[0036] (4) Most methods cannot accurately repair the complex details covered by the watermark during the watermark removal process. For example, after removing the watermark in the first stage, some difficult-to-process details often remain, affecting the final document integrity and visual effect.

[0037] Based on the problems in the prior art, this application proposes a document watermark removal method and system.

[0038] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of this application.

[0039] Example 1

[0040] See also Figure 1 , is a flowchart of a document watermark removal method according to Embodiment 1 of the present application; the steps include:

[0041] S1: Obtain a document image containing a watermark.

[0042] S2: Input the document image into the target network model for processing to generate a de-watermarking result; wherein the target network model is a semantic segmentation model, a coarse prediction model, and a semantic perception state space module connected in sequence.

[0043] In this embodiment, the document image is input into the target network model for processing to generate a watermark removal result; wherein the target network model is a semantic segmentation model, a coarse prediction model, and a semantic perception state space module connected in sequence, specifically including steps S21 to S23, specifically including the following contents:

[0044] S21: Input the document image into the semantic segmentation model and output a segmentation mask.

[0045] The document image is input into the semantic segmentation model for feature extraction, and a multi-scale feature representation is obtained through multi-layer convolution and pooling operations. The multi-scale feature representation is downsampled and upsampled through the encoder and decoder in the semantic segmentation model to obtain semantic information; wherein global semantic information is extracted based on downsampling, and upsampling is used to restore spatial resolution to obtain higher semantic information while maintaining the structural information of the document image.

[0046] At the end of the semantic segmentation model, a pixel classification layer is used to map the semantic information to the category distribution at the pixel level. The prediction of each pixel point is thresholded, and the segmentation mask M is output through the output layer of the semantic segmentation model; where M(i,j)=1 indicates that the pixel is a watermark area, and M(i,j)=0 indicates that the pixel is a non-watermark area.

[0047] The semantic segmentation model further includes: updating the parameters of the semantic segmentation model based on the calculated loss function; specifically:

[0048] Using the real annotated watermark mask GT maskThe segmentation mask M output by the semantic segmentation model is compared to calculate the segmentation loss; and the parameters of the semantic segmentation model are updated by back propagation according to the segmentation loss.

[0049] S22: Input the document image into the coarse prediction model to obtain a preliminary de-watermarked image.

[0050] The document image is input into the rough prediction model, and basic image features are extracted through several layers of convolution and nonlinear activation. The watermark information of the document image is suppressed and estimated and restored through a series of convolution and feature fusion operations within the rough prediction model to obtain a preliminary de-watermarked image.

[0051] The coarse prediction model is not optimized alone in the early stage of training, but is iteratively updated together with other model parameters under the guidance of the total loss function.

[0052] S23: Input the segmentation mask and the preliminary de-watermarked image into the semantic-aware state space module, and output the watermark result.

[0053] The preliminary de-watermarked image is normalized to obtain a feature region; and the feature region is divided into a watermark portion and a non-watermark portion according to the segmentation mask.

[0054] The feature regions are weighted by element-wise multiplication, and the formula is:

[0055] F watermark =I coarse ×M,

[0056] F non\watermark =I coarse ×(1-M),

[0057] Among them, F watermark is the feature of the watermark area; I coarse is the normalized initial de-watermarked image, i.e., the feature area; M is the segmentation mask; F non\watermark is the feature of the non-watermark area;

[0058] The features of the watermark area and the features of the non-watermark area are input into two linear transformation layers respectively.

[0059] The first path: After the features of the non-watermark area are linearly transformed (Linear), they are transferred to the 2D-SSM module, and the output result is the global features of the non-watermark area.

[0060] The second path: The features of the watermark area are also linearly transformed (Linear) and then passed into another 2D-SSM module. The output result is the global feature of the watermark area.

[0061] The output results of two 2D-SSMs processed by different paths are then added element by element, i.e., additive fusion; the formula is:

[0062] F fuse =F watermark +F non\watermark ,

[0063] Among them, F fuse is the fused feature, F fuse As input features of the spatial state sequence model in the semantic-aware state space module.

[0064] The input feature F is transformed into fuse To convert, the calculation formula is:

[0065] F fuse(i) = expand(F fuse ,I),

[0066]

[0067] Where i∈{1,2,3,4} represents one of the four scanning directions; expand(·) and merge(·) are used for scanning expansion and merging scanning results respectively; Mamba is a spatial state sequence model.

[0068] The spatial state sequence model includes: Since most of the data in practical applications are discrete data, the traditional SSM is discretized. Through sampling and zero-order hold method, the discrete state update and output equations are obtained:

[0069] State update equation:

[0070] Output equation: y k =Ch k +Dx k ;

[0071] in, and is the discretized parameter matrix, e AΔ is the matrix index, Δ is the time step; h k+1 is the updated spatial state sequence model state; h k is the state of the spatial state sequence model; x k is the input; k is the output; C and D are the learning parameter matrices.

[0072] It can be understood that the improved spatial state sequence model is: making matrices B, C and Δ functions of the input, the model can adaptively adjust its behavior according to different inputs to achieve an effect similar to the attention mechanism. The Selective Scan Algorithm is used to achieve parallel computing and speed up training and reasoning. By simplifying the SSM model architecture, the computational complexity is reduced and the model efficiency is improved.

[0073] The input feature F is transformed into fuse Update the state; use the convolutional layer and channel attention mechanism to process the updated features to obtain feature representation;

[0074] The feature representation is mapped to the semantic perception state space module, and a de-watermarking result is output.

[0075] S3: According to the watermark removal result, a total loss function is calculated.

[0076] In this embodiment, based on the watermark result I refined And the watermark-free true value GT, calculate the reconstruction loss; the formula is:

[0077] L reconstruction (I refined ,GT)=|I refined -GT|1

[0078] The segmentation loss of the semantic segmentation model and the reconstruction loss are weighted summed to obtain the total loss function; the formula is:

[0079] L total =λ1L seg (M,GT mask )+λ2L reconstruction (I refined ,GT)

[0080] Among them, λ1 and λ2 are the ratios of the loss function.

[0081] S4: Iteratively train the target network model based on the total loss function to obtain a trained target network model.

[0082] In this embodiment, the parameters of the semantic segmentation model, the coarse prediction model, and the semantic perception state space module are gradient updated through the back propagation algorithm; and the segmentation accuracy and watermark quality are continuously improved through multiple rounds of iterative training.

[0083] S5: Acquire a target document image including a watermark, input the target document image into the trained target network model, and generate a result image without a watermark.

[0084] In summary, Example 1 of the present application obtains a document image with a watermark as an input image, uses a semantic segmentation model to perform in-depth analysis on the document image, accurately identifies the area where the watermark is located, and generates a corresponding segmentation mask, which provides key watermark location information for subsequent processing. A coarse prediction model is used to perform preliminary processing on the document image with a watermark, remove most of the obvious watermark traces, and obtain a preliminary de-watermarked image. Finally, the semantic-aware state space module is combined with a segmentation mask to refine the preliminary de-watermarked image, integrating global semantic information and local detail features to achieve accurate restoration of the watermark coverage area, so that the document is restored to a natural and complete state, significantly improving the efficiency and performance in high-resolution image processing.

[0085] Example 2

[0086] See also Figure 4 , is a structural diagram of a document watermark removal system in Example 2 of the present application; the specific contents include:

[0087] Acquisition module: acquires the document image containing the watermark;

[0088] Processing module: inputting the document image into the target network model for processing to generate a de-watermarking result; wherein the target network model is a semantic segmentation model, a coarse prediction model, and a semantic perception state space module connected in sequence;

[0089] Calculation module: calculates the total loss function according to the watermark removal result;

[0090] Training module: iteratively training the target network model based on the total loss function to obtain a trained target network model;

[0091] Generation module: obtains a target document image including a watermark, inputs the target document image into the trained target network model, and generates a watermark-free result image.

[0092] Example 3

[0093] See also Figure 5 , is a schematic diagram of the device structure of Embodiment 3 of the present application. The device 50 includes a processor 51 and a memory 52 coupled to the processor 51 .

[0094] The memory 52 stores program instructions for implementing the above-mentioned document watermark removal method.

[0095] The processor 51 is used to execute program instructions stored in the memory 52 to implement document watermark removal.

[0096] The processor 51 may also be referred to as a CPU (Central Processing Unit).

[0097] The processor 51 may be an integrated circuit chip with signal processing capabilities. The processor 51 may also be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0098] Example 4

[0099] See also Figure 6 , which is a schematic diagram of the structure of the storage medium of Example 4 of the present application. The storage medium of the embodiment of the present application stores a program file 61 that can implement all the above methods, wherein the program file 61 can be stored in the above storage medium in the form of a software product, including a number of instructions to enable a computer device (which can be a personal computer, server, or network device, etc.) or a processor (processor) to perform all or part of the steps of the methods of each embodiment of the present invention. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, or a computer, a server, a mobile phone, a tablet, and other devices.

[0100] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, device, article or method including a series of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, device, article or method. In the absence of further restrictions, an element defined by the sentence "includes a ..." does not exclude the presence of other identical elements in the process, device, article or method including the element.

[0101] The above description is only a preferred embodiment of the present application, and does not limit the patent scope of the present application. Any equivalent structure or equivalent process transformation made using the contents of the present application specification and drawings, or directly or indirectly used in other related technical fields, are also included in the patent protection scope of the present application.

[0102] Although the embodiments of the present application have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present application, and that the scope of the present application is defined by the appended claims and their equivalents.

[0103] Of course, the present invention may have many other implementations. Based on this implementation, other implementations obtained by ordinary technicians in this field without any creative work are all within the scope of protection of the present invention.

Claims

1. A document watermark removal method, characterized in that: include: Get the document image containing the watermark; Input the document image into the target network model for processing to generate a de-watermarking result; wherein the target network model is a semantic segmentation model, a coarse prediction model, and a semantic perception state space module connected in sequence; According to the watermark removal result, a total loss function is calculated; Iteratively training the target network model based on the total loss function to obtain a trained target network model; A target document image including a watermark is obtained, and the target document image is input into the trained target network model to generate a result image without a watermark.

2. A document watermark removal method according to claim 1, characterized in that: The document image is input into a target network model for processing to generate a watermark removal result; wherein the target network model is a semantic segmentation model, a coarse prediction model, and a semantic perception state space module connected in sequence, specifically including the following steps: Inputting the document image into the semantic segmentation model and outputting a segmentation mask; Inputting the document image into the coarse prediction model to obtain a preliminary de-watermarked image; The segmentation mask and the preliminary watermark removal image are input into the semantic-aware state space module, and the watermark removal result is output.

3. A document watermark removal method according to claim 2, characterized in that: The step of inputting the document image into the semantic segmentation model and outputting the segmentation mask specifically includes the following steps: Inputting the document image into the semantic segmentation model to extract features and obtain multi-scale feature representation; Downsampling and upsampling the multi-scale feature representation through an encoder and a decoder in a semantic segmentation model to obtain semantic information; Mapping the semantic information to pixel-level category distribution through a pixel classification layer in a semantic segmentation model; The prediction of each pixel is thresholded, and the segmentation mask M is output through the output layer in the semantic segmentation model; wherein M(i, j) = 1 indicates that the pixel is a watermark area, and M(i, j) = 0 indicates that the pixel is a non-watermark area.

4. A document watermark removal method according to claim 3, characterized in that: The semantic segmentation model further includes: updating the parameters of the semantic segmentation model based on the calculated loss function; specifically: Using the real annotated watermark mask GT mask Compare with the segmentation mask M output by the semantic segmentation model to calculate the segmentation loss; The parameters of the semantic segmentation model are updated according to the segmentation loss back-propagation.

5. A document watermark removal method according to claim 2, characterized in that: The step of inputting the segmentation mask and the preliminary watermark removal image into the semantic-aware state space module and outputting the watermark removal result specifically includes the following steps: Normalizing the preliminary de-watermarked image to obtain a feature region; Dividing the feature area into a watermark part and a non-watermark part according to the segmentation mask; The feature regions are weighted by element-wise multiplication, and the formula is: F watermark =I coarse ×M,F non\watermark =I coarse ×(1-M), Among them, F watermark is the feature of the watermark area; I coarse is the normalized initial de-watermarked image, i.e., the feature area; M is the segmentation mask; F non\watermark is the feature of the non-watermark area; The features of the watermark area and the features of the non-watermark area are input into two linear transformation layers respectively to transform and fuse the feature dimensions; the formula is expressed as: F fuse =F watermark +F non\watermark , Among them, F fuse is the fused feature, F fuse As input features of the spatial state sequence model in the semantic-aware state space module.

6. A document watermark removal method according to claim 5, characterized in that: The spatial state sequence model includes: The input feature F is transformed into fuse To convert, the calculation formula is: F fuse(i) =expand(F fuse ,I), Where i∈{1,2,3,4} represents one of the four scanning directions; expand(·) and merge(·) are used for scanning expansion and scanning result merging respectively; Mamba is a spatial state sequence model; Through sampling and zero-order hold method, the discrete state update and output equations are obtained: State update equation: Output equation: y k =Ch k +Dx k ; in, and is the discretized parameter matrix, e AΔ is the matrix index, Δ is the time step; h k+1 is the updated spatial state sequence model state; h k is the state of the spatial state sequence model; x k is the input; k is the output; C and D are the learning parameter matrices; The input feature F is transformed into fuse Update the state; use the convolutional layer and channel attention mechanism to process the updated features to obtain feature representation; The feature representation is mapped to the semantic perception state space module, and a de-watermarking result is output.

7. A document watermark removal method according to claim 1, characterized in that: The step of calculating the total loss function according to the watermark removal result specifically includes the following steps: Based on the watermark result I refined And the watermark-free true value GT, calculate the reconstruction loss; the formula is: L reconstruction (I refined ,GT)=|I refined -GT|1 The segmentation loss of the semantic segmentation model and the reconstruction loss are weighted summed to obtain the total loss function; the formula is: L total =λ1L seg (M,GT mask )+λ2L reconstruction (I refined ,GT) Among them, λ1 and λ2 are the ratios of the loss function.

8. A system for removing watermarks from a document according to claim 1, characterized in that: include: Acquisition module: acquires the document image containing the watermark; Processing module: inputting the document image into the target network model for processing to generate a de-watermarking result; wherein the target network model is a semantic segmentation model, a coarse prediction model, and a semantic perception state space module connected in sequence; Calculation module: calculates the total loss function according to the watermark removal result; Training module: iteratively training the target network model based on the total loss function to obtain a trained target network model; Generation module: obtains a target document image including a watermark, inputs the target document image into the trained target network model, and generates a watermark-free result image.

9. A device, characterized in that: The device includes a processor and a memory coupled to the processor, wherein the memory stores program instructions for implementing a document watermark removal method as described in any one of claims 1-7; and the processor is used to execute the program instructions stored in the memory to implement document watermark removal.

10. A storage medium, characterized in that: Program instructions executable by a processor are stored, and the program instructions are used to execute a document watermark removal method as described in any one of claims 1-7.

Images