Method and system for detecting and erasing seal in image

Abstract

The invention provides a method and a system for detecting and erasing a seal in an image. The method comprises the following steps: determining an area of the seal in an original image according to a preset seal position detection module, and intercepting an area image; regenerating the intercepted area image by using the pre-trained seal erasing model, and replacing the intercepted area image in the original image with the regenerated area image to obtain a clean image of which the seal is erased; wherein the seal erasing model comprises a low-frequency restorer and a pre-trained high-frequency residual error accurate model. According to the method, the document image can be preliminarily restored through the low-frequency restorer, the high-frequency residual error is predicted through the high-frequency residual error accurate model, and the preliminarily restored document image and the predicted high-frequency residual error are combined to obtain a more accurate seal-removed document image, so that the edge image of the restored text can present better definition.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and in particular to a method and system for detecting and erasing a seal in an image. Background Art

[0002] With the development of optical character recognition technology, seal detection and removal technology has been widely used in qualification review and contract review. The review process often involves a lot of work on reviewing the content of qualification certificates and contract documents, and these certificates and contracts are usually stamped with seals of different styles. In addition, the seals will cover important information such as document content or signature, which greatly interferes with document content recognition and feature extraction. Therefore, it is necessary to locate and erase the seals in the document image before recognizing the document image.

[0003] Existing solutions for seal detection include: (1) using SIFT (Scale Invariant Feature Transform), SURF (Speeded Robust Features), ORB (Oriented FAST and Rotated BRIEF) and other methods for feature extraction and matching, but this method is sensitive to changes in illumination and perspective and requires a lot of computing resources. (2) Detecting straight and circular structures in images based on statistical methods (such as Hough Linear Transform and Hough Circle Transform), but this method is sensitive to noise and requires appropriate parameter settings and post-processing steps. (3) Based on the template matching method, the known template is compared with the local area in the image to find the best matching position, but this method is sensitive to changes in illumination, scale and perspective and requires multi-scale matching and template updating.

[0004] Existing solutions for seal removal include: processing non-seal objects in the seal area image by training a seal removal network model to obtain a stamp-removed image of the seal area image. However, the prior art does not consider the separate processing of low-frequency residuals and high-frequency residuals, and thus cannot accurately capture the high-frequency information of the edge of the text image, resulting in significant blurring at the edge of the text. It is difficult to solve this problem by simply cascading more convolutional layers in the seal removal network model, and it will cause a greater computational burden.

[0005] Therefore, how to provide a method for detecting seals in images that can reduce detection overhead and a method for removing seals from documents that can obtain clearer images is a technical problem that needs to be solved urgently. Summary of the invention

[0006] In view of this, an embodiment of the present invention provides a method and system for detecting and erasing a seal in an image, so as to eliminate or improve one or more defects existing in the prior art.

[0007] One aspect of the present invention provides a method for detecting and erasing a seal in an image, the method comprising the following steps: determining the area where the seal is located in the original image according to a preset seal position detection module and capturing the area image; regenerating the captured area image using a pre-trained seal erasure model, and replacing the captured area image in the original image with the regenerated area image to obtain a clean image with the seal erased; wherein the seal erasure model includes a low-frequency repairer and a pre-trained high-frequency residual precision model.

[0008] In some embodiments of the present invention, the step of regenerating the intercepted regional image using the pre-trained seal erasure model includes: inputting the intercepted regional image into a low-frequency repairer, the low-frequency repairer removes the traces of the seal on the regional image at the pixel level to obtain a preliminary repaired regional image; using the preliminary repaired regional image as input, predicting by time embedding to obtain a residual prediction result; summing the preliminary repaired regional image and the residual prediction result to obtain a finely repaired regenerated regional image.

[0009] In some embodiments of the present invention, the type of the low-frequency repairer is a convolutional neural network-based prediction model for image regeneration, and the low-frequency repairer is trained with a preset-scale training set containing clear text images and unstamped original images, with low-frequency residuals as training targets. The low-frequency residuals are obtained by performing residual calculation and frequency separation on the initially repaired regional image and the unstamped original image.

[0010] In some embodiments of the present invention, the method also includes the step of pre-training a high-frequency residual accurate model, including: performing residual calculation and frequency separation on the preliminary repaired regional image obtained after processing by the low-frequency repairer and the original image without a stamp to obtain high-frequency residuals and low-frequency residuals; training the high-frequency residual accurate model with a training set of a preset scale containing the preliminary repaired regional image and the high-frequency residual, and using the trained high-frequency residual accurate model to predict to obtain a residual prediction result.

[0011] In some embodiments of the present invention, the high-frequency residual is pre-processed by a high-frequency filter, the low-frequency residual is pre-processed by a low-frequency filter, and the residual prediction result is also pre-processed by a high-frequency filter.

[0012] In some embodiments of the present invention, the high-frequency residual accurate model comprises a plurality of cascaded residual down-sampling blocks, residual up-sampling blocks and dilated convolutions.

[0013] In some embodiments of the present invention, the step of determining the area where the seal is located in the original image according to a preset seal position detection module and capturing the area image includes: converting the form of the original image into image data recorded using HSV values; performing cluster analysis on the image data recorded using HSV values ​​to obtain the area where the reddish pixels are located, thereby identifying the area where one or more seals are located; using the circumscribed contour method to draw the area where the one or more seals obtained by clustering are located using the maximum circumscribed rectangle, and capturing the area image.

[0014] Corresponding to the above method, the present invention also provides a system for detecting and erasing seals in images, comprising a processor, a memory, and a computer program / instructions stored in the memory, wherein the processor is used to execute the computer program / instructions, and when the computer program / instructions are executed, the system implements the steps of the method described in any one of the above embodiments.

[0015] Corresponding to the above method, the present invention also provides a computer-readable storage medium on which a computer program / instruction is stored. When the computer program / instruction is executed by a processor, the steps of the method described in any one of the above embodiments are implemented.

[0016] Corresponding to the above method, the present invention also provides a computer program product, including a computer program / instruction, which implements the steps of the method described in any one of the above embodiments when executed by a processor.

[0017] The method and system for detecting and erasing seals in images proposed in the present invention can preliminarily restore the document image through a low-frequency repairer, and the high-frequency residual accurate model predicts the high-frequency residual. The preliminarily restored document image is combined with the predicted high-frequency residual to obtain a more accurate document image with the seal removed, so that the edge image of the restored text can present better clarity.

[0018] Additional advantages, purposes, and features of the present invention will be described in part in the following description, and will become apparent to those skilled in the art after studying the following, or may be learned from the practice of the present invention. The purposes and other advantages of the present invention may be achieved and obtained by the structures specifically indicated in the specification and the accompanying drawings.

[0019] Those skilled in the art will appreciate that the objectives and advantages that can be achieved with the present invention are not limited to the above specific description, and the above and other objectives that can be achieved by the present invention will be more clearly understood from the following detailed description. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] The drawings described herein are used to provide a further understanding of the present invention, constitute a part of the present application, and do not constitute a limitation of the present invention. In the drawings:

[0021] Figure 1 The figure is a flow chart of a method for detecting and erasing a seal in an image in one embodiment of the present invention.

[0022] Figure 2 The figure is a flowchart of erasing a stamp in an image according to an embodiment of the present invention.

[0023] Figure 3 The figure is a flow chart of the position detection of a seal in an image according to an embodiment of the present invention. DETAILED DESCRIPTION

[0024] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the embodiments and the accompanying drawings. Here, the illustrative embodiments of the present invention and their descriptions are used to explain the present invention, but are not intended to limit the present invention.

[0025] It should also be noted that, in order to avoid obscuring the present invention due to unnecessary details, only structures and / or processing steps closely related to the solutions according to the present invention are shown in the accompanying drawings, while other details that are not closely related to the present invention are omitted.

[0026] It should be emphasized that the term “include / comprises” when used herein refers to the presence of features, elements, steps or components, but does not exclude the presence or addition of one or more other features, elements, steps or components.

[0027] It should also be noted that, unless otherwise specified, the term “connection” herein may refer not only to a direct connection but also to an indirect connection involving an intermediate.

[0028] Hereinafter, embodiments of the present invention will be described with reference to the accompanying drawings. In the accompanying drawings, the same reference numerals represent the same or similar components, or the same or similar steps.

[0029] In order to overcome the problems existing in the prior art, the present invention proposes a method and system for detecting and erasing seals in images. The method adopts traditional image methods to find the position of the seal, and uses a jointly trained deep learning model to remove the seal, wherein the jointly trained deep learning model includes a low-frequency repairer (RLC) and a high-frequency residual precision model (DMPR).

[0030] Figure 1 This is a flow chart of a method for detecting and erasing a seal in an image in one embodiment of the present invention, the method comprising the following steps:

[0031] Step S110: determining the area where the seal is located in the original image according to a preset seal position detection module and capturing the area image.

[0032] Step S120: regenerate the intercepted region image using the pre-trained seal erasure model, and use the re-generated region image to replace the intercepted region image in the original image to obtain a clean image with the seal erased. The seal erasure model includes a low-frequency repairer and a pre-trained high-frequency residual precision model.

[0033] In the specific implementation process, the step of regenerating the intercepted regional image using the pre-trained seal erasure model in step S120 includes: (1) inputting the intercepted regional image into a low-frequency repairer, and the low-frequency repairer removes the traces of the seal on the regional image at the pixel level to obtain a preliminary repaired regional image; (2) using the preliminary repaired regional image as input, and using a time embedding method to perform prediction to obtain a residual prediction result; (3) summing the preliminary repaired regional image and the residual prediction result to obtain a finely repaired regenerated regional image.

[0034] Step (1) in the above step S120 can only effectively restore the main content of the text image, but may not accurately capture the high-frequency information at the edge of the text image, which will lead to significant blurring at the edge of the text image, and this problem is difficult to solve by simply cascading more convolutional layers. Optionally, the type of the low-frequency repairer is a prediction model for image regeneration. In the prior art, there are deep learning models or machine learning models for removing seals by image regeneration. The improvement of the present invention is that it proposes a joint training of a low-frequency repairer and a high-frequency residual precision model to generate a more refined image result after removing the seal, so that the edge of the text image after the seal is removed is clear, presenting a better seal removal effect.

[0035] As described above, the method and system for detecting and erasing seals in images proposed in the present invention can preliminarily restore the document image through a low-frequency repairer, and the high-frequency residual accurate model predicts the high-frequency residual. The preliminarily restored document image is combined with the predicted high-frequency residual to obtain a more accurate document image with the seal removed, so that the edge image of the restored text can present better clarity.

[0036] In some embodiments of the present invention, the type of low-frequency repairer is a convolutional neural network-based prediction model for image regeneration, and the low-frequency repairer is trained with a preset-scale training set containing clear text images and unstamped original images, with low-frequency residuals as training targets, and the low-frequency residuals are obtained by residual calculation and frequency separation of the initially repaired regional image and the unstamped original image.

[0037] By adopting the embodiment of the invention, the seal in the original image can be preliminarily removed and combined with the residual prediction result obtained subsequently to obtain an accurately regenerated image.

[0038] In some embodiments of the present invention, the method further includes the step of pre-training a high-frequency residual precision model, including: (1) performing residual calculation and frequency separation on the initially repaired regional image obtained after processing by the low-frequency repairer and the original image without a stamp, to obtain high-frequency residual and low-frequency residual; (2) training the high-frequency residual precision model with a training set of a preset scale containing the initially repaired regional image and the high-frequency residual, and using the trained high-frequency residual precision model to predict and obtain residual prediction results. The low-frequency residual can be used as a training target of the low-frequency repairer or as a component of the training set. Optionally, the training set can also be composed of the initially repaired regional image and the high-definition original image, and the high-frequency residual can be obtained by comparing the high-definition original image with the initially repaired regional image.

[0039] By adopting this embodiment of the invention, it is possible to train high-frequency residuals and predict the high-frequency residuals based on the initially repaired regional image, thereby completing the high-frequency residuals during the seal erasing process and making the edges of the text image clear.

[0040] In some implementations of the present invention, the high-frequency residual is pre-processed by a high-frequency filter, the low-frequency residual is pre-processed by a low-frequency filter, and the residual prediction result is also pre-processed by a high-frequency filter.

[0041] By adopting the embodiment of the invention, low-frequency residuals and high-frequency residuals can be distinguished more clearly, low-frequency residuals can be processed by a low-frequency repairer, and high-frequency residuals can be processed by a high-frequency residual precision model. Through separate processing, the clarity of the text edge after the image is regenerated can be guaranteed.

[0042] In some embodiments of the present invention, the high-frequency residual accurate model comprises a plurality of cascaded residual down-sampling blocks, residual up-sampling blocks and dilated convolutions.

[0043] By adopting this embodiment of the invention, a deep learning model is used to perform image reconstruction and prediction of high-frequency residuals. In essence, image reconstruction can also be understood as a prediction model.

[0044] Among them, deep learning methods include the following technologies: (1) Artificial Neural Networks (ANN): including Multilayer Perceptrons (MLP), Convolutional Neural Networks (CNN), Recurrent Neural Networks (RNN), etc. ANN is the basis of deep learning and is used to simulate and learn complex nonlinear relationships. (2) Convolution operation: The core operation used to extract features from the input image in a convolutional neural network. The convolution kernel (filter) slides on the image and performs feature extraction of local areas. (3) Pooling: An operation used in convolutional neural networks to reduce the size of feature maps, extract main features, and reduce computational complexity, such as maximum pooling and average pooling. (4) Recurrent Neural Networks (RNN): Used to process sequence data (such as text, speech), has memory ability, and can consider information from previous time steps. (5) Long Short-Term Memory (LSTM) and Gated Recurrent Unit (GRU): variants of RNN, designed to solve the long-term dependency problem faced by traditional RNN, and are particularly suitable for processing long sequence data. (6) Generative Adversarial Networks (GANs): an adversarial model composed of a generator and a discriminator, used to generate realistic new data samples, such as images, audio, etc. (7) Autoencoders: an unsupervised learning method for learning effective representations of data, including sparse autoencoders, variational autoencoders, etc. (8) Deep Reinforcement Learning (DRL): a combination of deep learning and reinforcement learning, used to solve tasks that require long-term decision-making and environmental interaction, such as game play optimization, robot control, etc. (9) Transfer Learning: using pre-trained models to adjust them on new tasks to speed up training and improve performance. (10) Batch Normalization: a technology used to accelerate the neural network training process and enhance the model's generalization ability. (11) Optimization algorithms: such as Stochastic Gradient Descent (SGD) and its variants (such as Adam, RMSProp, etc.), which are used to adjust the weights and biases of the neural network to minimize the loss function.(12) Regularization techniques: such as Dropout, L1 and L2 regularization, are used to prevent neural network overfitting and improve generalization ability.

[0045] In some embodiments of the present invention, step S110, in which a preset seal position detection module is used to determine the area where the seal is located in the original image and to capture the area image, includes: (1) converting the original image into image data recorded using HSV values; (2) performing cluster analysis on the image data recorded using HSV values ​​to obtain the area where the reddish pixels are located, thereby identifying the area where one or more seals are located; (3) using the circumscribed contour method to draw the area where the one or more seals are located obtained by clustering using the maximum circumscribed rectangle, and capturing the area image.

[0046] Compared with the prior art, the embodiment of the invention uses limited computing resources, is insensitive to changes in noise, lighting, and viewing angle, avoids other requirements such as data collection and GPU, and has a high cost-effectiveness.

[0047] In a specific embodiment of the present invention, the traditional method in OpenCV is used to determine the area where the seal is located in the image, and the seal detection process includes the following steps:

[0048] (1) Convert pixels into HSV values; HSV value is a color model consisting of three parameters: hue, saturation, and value. These three parameters together describe the specific characteristics of the color.

[0049] (2) Perform cluster analysis on the pixels of the entire image to find out the range of HSV values ​​of the reddish pixels.

[0050] (3) After knowing the interval, detect the pixel position that matches the red part.

[0051] (4) Clustering pixel positions to cope with the situation of multiple seals.

[0052] (5) Using the circumscribed contour method, draw each cluster area with the maximum circumscribed rectangle to obtain the area where the seal is located in the image.

[0053] Among them, a series of methods derived from pixel gradients based on OpenCV are compared with modern methods such as deep learning. Traditional methods are usually based on hand-designed feature extractors and classic machine learning algorithms to solve image processing, analysis and recognition problems.

[0054] In a specific embodiment of the present invention, a pre-trained model combined with RLC-DMPR is used for stamping. The pre-trained model converts the document enhancement task into a conditional image generation task. RLC-DMPR includes two modules: a low-frequency repairer (RLC) and a high-frequency residual precision model (DMPR). RLC accepts a degraded document image as input and approximately restores its clean version. The DMPR module uses a diffusion model to accurately and efficiently sharpen the text edges generated by RLC.

[0055] In another specific embodiment of the present invention, the image traditional method includes the following technologies: (1) Feature extraction: including edge detection, corner detection, texture feature extraction, etc., used to extract important local information from the image. (2) Filters: such as Gaussian filtering, median filtering, etc., used to denoise and smooth the image. (3) Color space conversion: such as RGB to grayscale conversion, HSV color space, etc. (4) Morphological processing: including operations such as corrosion, expansion, opening and closing operations, used for image enhancement and segmentation. (5) Image segmentation: dividing the image into different regions or objects through methods such as threshold processing, region growing, edge detection, etc. (6) Target detection and tracking: such as Haar feature detector, HOG (Histogram of Oriented Gradients) feature detector, etc. for target detection, Kalman filter, optical flow method, etc. for target tracking. (7) Pattern recognition: including machine learning methods such as support vector machine (SVM), K nearest neighbor (KNN), decision tree, etc., used to identify and classify objects or scenes in the image. (8) Geometric transformation: such as rotation, scaling, translation, etc., used for image registration and correction. Traditional image methods mainly refer to a series of algorithms and methods based on computer vision and image processing technology before the rise of deep learning technology. These methods are usually used to perform feature extraction, filtering, compression, enhancement, denoising, segmentation, classification and other processing on images to improve the quality of images or extract useful information from them.

[0056] Further, in some embodiments of the present invention, in order to avoid changing characters and speed up the reasoning speed while improving the generation quality, the optimization target is adjusted during the training process, and a short-step deterministic sampling strategy is adopted during the reasoning process. Specifically, the high-frequency residual precision model (DMPR) is made to learn the distribution of residuals between the real image and the RLC predicted image (condition), that is, the training is performed on a data set consisting of an image initially restored by a low-frequency repairer of a preset scale and a real image without a seal. And, while ensuring consistency during the back diffusion process, the high-frequency residual precision model (DMPR) is made to directly predict the original residual instead of the added noise. This enables the RLC-DMPR to generate reasonable images that are highly correlated with the conditions based on the use of a channel-by-channel splicing conditional scheme in the first few steps of back diffusion. Although the generation diversity is sacrificed (not a critical factor for document enhancement), this operation significantly reduces the number of diffusion steps required for sampling and effectively reduces the range of distorted text edges and character replacements generated by the conditional diffusion model.

[0057] Among them, back diffusion is a concept widely used in computer science and data processing. In the field of image restoration and reconstruction, the back diffusion algorithm is used to restore the missing parts of the image based on the transferred and diffused pixel information in the target area, making the image complete and clear.

[0058] By adopting this embodiment of the invention, RLC-DMPR can achieve a balance between innovation and practicality through end-to-end training, combining pixel loss and modified diffusion model loss, and applying deterministic short-step sampling, and can achieve good stamp removal effect using a small and efficient network with only 8 million parameters.

[0059] The experimental results on the dataset show that RLC-DMPR is a compact, flexible, efficient and stable generative model. The experiments show that RLC-DMPR has reached a leading level in the perceptual quality of low-level deblurring tasks and is competitive in high-level binarization tasks. More importantly, during the test, RLC-DMPR was able to achieve a competitive seal removal task using only 5 sampling steps. This embodiment generated a paired dataset using internal document images as a training set. The experimental results show that RLC-DMPR can effectively remove seals and retain covered characters. Compared with non-diffusion methods, RLC-DMPR has a fast inference speed and has the same level of performance. In addition, RLC-DMPR has a low training cost, avoids mode collapse, and can process handwritten and printed document images of any resolution.

[0060] For the low-frequency repairer (RLC) and high-frequency residual prediction (DMPR) modules included in the RLC-DMPR in the embodiment of the present invention. Since the document image has a fixed pattern and different resolutions, a compact U-Net structure can be used as the structural mode of the RLC and DMPR modules. Optionally, in the specific implementation process, the middle self-attention layer is replaced with a four-layer hole convolution to increase the receptive field, and the remaining self-attention and normalization layers are removed. In order to reduce the computational complexity, the parameters of the RLC and DMPR modules are compressed to 4.13M and 4.27M respectively, which ensures performance while the overhead is much less than the existing document enhancement method.

[0061] In an embodiment of the present invention, RLC-DMPR can be a combination of a fully convolutional model, combined with pixel loss and diffusion model loss, and end-to-end training through frequency separation. Figure 2 The high-frequency residual and low-frequency residual separated as shown in the figure are obtained. RLC-DMPR is trained end-to-end through frequency separation, combined with pixel loss and diffusion model loss. Specifically, the goal of the RLC module is to approximately restore the damaged document image to its clean version at the pixel level. However, due to the "mean regression" problem of the CNN-based regression method, the RLC method will cause significant blurring at the edge of the text. For this reason, the DMPR module is introduced to generate samples from the learned posterior distribution. In layman's terms, if only the low-frequency repairer is used, the resulting text edge will be significantly blurred.

[0062] In some embodiments of the present invention, the time embedding method refers to a method of encoding timestamp information into a vector representation that can be processed by the model. Specifically: each timestamp type has a corresponding embedding matrix. A time vector is generated for each position in the sequence, and this vector is obtained by looking up the embedding matrix. However, the time series in this article refers to the order in which the model is convolved. Since the model reduces the dimension when convolving the image, the result of each layer of convolution represents a certain degree of receptive field. In the actual model, it is necessary to fuse the information of different receptive fields, so the time series is introduced. The convolution results of different times are fused.

[0063] In a specific embodiment of the present invention, the goal of the proposed low-frequency repairer is to approximately restore the degraded document image to its clean version at the pixel level. During the training process, the pixel loss can be defined as the mean square error between the coarse prediction and the gt. When generating text pixels, the low-frequency repairer can effectively restore the main content of the text, but may not accurately capture the high-frequency information at the edge of the text, which will cause significant blurring at the edge of the text, which is a known limitation of the CNN-based regression method in the prior art and belongs to the "regression to the mean" problem. Moreover, it is difficult to solve this problem by simply cascading more convolutional layers.

[0064] In a specific embodiment of the present invention, the proposed high-frequency residual accurate model (DMPR) can generate samples from the learned distribution, that is, generate prediction results about the high-frequency residual. The core component of the high-frequency residual accurate model is a denoiser (including upsampling blocks, downsampling blocks and dilated convolution layers), which uses DPMs to estimate (or predict) the residual distribution between the ground truth image and the RLC generated image. In addition, the RLC and DMPR modules are trained end-to-end jointly instead of separately, that is, for the high-frequency residuals and low-frequency residuals obtained by frequency separation, the low-frequency residuals are used to train the low-frequency restorer (or preliminary predictor), and the high-frequency residuals are used to train the high-frequency residual accurate model. The training process of the two is as follows. Figure 2 Shown are joined together.

[0065] By adopting a specific embodiment of the present invention, a high-frequency residual precision model is introduced to improve the generation of text edges. The high-frequency residual precision model can dynamically adjust and capture more patterns, thereby overcoming the "regression to the mean" problem existing in a single regression model (including the RLC module in the present invention). This method is widely applicable to a variety of regression models. A large number of experiments have shown that this training strategy effectively enhances the clarity of characters generated by different regression-based deblurring methods. The DMPR module can directly and significantly improve the perceptual quality of the deblurred image generated by the regression method without the need for additional training.

[0066] Figure 2 FIG. 1 is a flowchart of erasing a stamp in an image in one embodiment of the present invention. Figure 2 As shown, it includes: (1) inputting unclear text into the low-frequency repairer to obtain clearer text (i.e., the preliminary prediction after preliminary erasing the seal), and the pixel loss can be obtained by comparing the preliminary prediction with the original text. (2) The pixel loss is separated by removing noise and frequency to obtain high-frequency residual and low-frequency residual. The low-frequency residual can be used to train the low-frequency repairer, and the high-frequency residual is used to train the high-frequency residual precise model. (3) The high-frequency residual precise model uses the concatenation of the preliminary prediction and the obtained noise residual as input, and adopts the time embedding method for training and prediction. The core of the high-frequency residual precise model is five-layer dilated convolution and multi-level upsampling blocks and downsampling blocks. (4) The high-frequency residual precise model performs the next residual prediction based on the input preliminary prediction. (5) Combining the preliminary prediction with the residual prediction can obtain a refined prediction result.

[0067] Figure 3 The flowchart of the position detection of the seal in the image in one embodiment of the present invention is shown in FIG. The method comprises the following steps:

[0068] Step S310: Input an image.

[0069] Step S320: Convert the input image into HSV value representation.

[0070] Step S330: Obtaining the red HSV value interval through cluster analysis.

[0071] Step S340: Obtain the red pixel position according to the red HSV value interval.

[0072] Step S350: clustering the positions to distinguish multiple seals.

[0073] Step S360: Draw each cluster region using the circumscribed contour method.

[0074] By adopting the embodiment of the invention, the seal position detection speed can be slightly accelerated and the cost in the process can be greatly reduced.

[0075] In some other embodiments of the present invention, in the process of detecting the position of the seal in the image, a pre-processing step of the image may be added. For example, a resize algorithm may be used to adjust the size and / or dimension of the image.

[0076] The method and system for detecting and erasing seals in images proposed in the present invention can preliminarily restore the document image through a low-frequency repairer, and the high-frequency residual accurate model predicts the high-frequency residual. The preliminarily restored document image is combined with the predicted high-frequency residual to obtain a more accurate document image with the seal removed, so that the edge image of the restored text can present better clarity.

[0077] In the method proposed in the present invention, the advantages of the seal detection part include: (1) It is not sensitive to changes in illumination and viewing angle, and the required computing resources are very limited. (2) Due to the use of clustering method, it is not sensitive to noise and exhibits robustness. It shows good applicability and does not require excessive consideration of parameter settings and post-processing steps. (3) It can tolerate a large range of scale and viewing angle changes, and does not require multi-scale matching and template updates. (4) It avoids the difficulties brought by data collection. (5) It avoids expensive equipment such as GPU. (6) The computer running resource overhead is limited. (7) It does not require manual selection and optimization of features, and performs well for complex patterns.

[0078] In the method proposed in the present invention, the advantages of the seal erasing part include: (1) the encoding and decoding process is eliminated, a small model is used to streamline the network, and less computer resources are occupied. (2) The seal is erased more thoroughly and the traces left are smaller. (3) The seal removal effect is better, which is of great help to subsequent processing. (4) A 5-layer receptive field can be used to retain high-frequency and low-frequency information as much as possible. (5) A series of problems caused by the Markov chain, such as convergence speed and initialization, are avoided. (6) While improving the generation quality, the character changes are avoided and the reasoning speed is accelerated. (7) The range of distorted text edges and character replacements generated by the conditional diffusion model is effectively reduced. (8) The perceptual quality of low-level deblurring tasks has reached a leading level and is competitive in high-level binarization tasks.

[0079] The method for detecting and erasing seals in images proposed in the present invention can restore the main low-frequency content through a low-frequency repairer, and predict the residual (including high-frequency information including text edges) between the ground truth image and the RLC predicted image through a high-frequency residual accurate model of a diffusion model type, thereby erasing the seal as thoroughly as possible and leaving as few traces as possible.

[0080] Extensive ablation studies and comparative experiments show that the proposed RLC-DMPR achieves competitive performance in document deblurring, denoising, watermark removal, and seal removal tasks. The experimental results highlight the advantages of each component of RLC-DMPR, which together contribute to its superior performance.

[0081] Corresponding to the above method, the present invention also provides a system for detecting and erasing seals in images, the system comprising a computer device, the computer device comprising a processor and a memory, the memory storing computer instructions, the processor being used to execute the computer instructions stored in the memory, and when the computer instructions are executed by the processor, the system implements the steps of the method described above.

[0082] Corresponding to the above method, the present invention also provides a computer-readable storage medium, on which a computer program / instruction is stored, and when the computer program / instruction is executed by a processor, the steps of the method described in any one of the above embodiments are implemented. The computer-readable storage medium can be a tangible storage medium, such as a random access memory (RAM), a memory, a read-only memory (ROM), an electrically programmable ROM, an electrically erasable programmable ROM, a register, a floppy disk, a hard disk, a removable storage disk, a CD-ROM, or any other form of storage medium known in the technical field.

[0083] Corresponding to the above method, the present invention also provides a computer program product, including a computer program / instruction, which implements the steps of the method described in any one of the above embodiments when executed by a processor.

[0084] It should be understood by those skilled in the art that the exemplary components, systems and methods described in conjunction with the embodiments disclosed herein can be implemented in hardware, software or a combination of the two. Whether it is performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention. When implemented in hardware, it can be, for example, an electronic circuit, an application specific integrated circuit (ASIC), appropriate firmware, a plug-in, a function card, etc. When implemented in software, the elements of the present invention are programs or code segments used to perform the required tasks. The program or code segment can be stored in a machine-readable medium, or transmitted on a transmission medium or a communication link via a data signal carried in a carrier.

[0085] It should be clear that the present invention is not limited to the specific configuration and processing described above and shown in the figures. For the sake of simplicity, a detailed description of the known method is omitted here. In the above embodiments, several specific steps are described and shown as examples. However, the method process of the present invention is not limited to the specific steps described and shown, and those skilled in the art can make various changes, modifications and additions, or change the order between the steps after understanding the spirit of the present invention.

[0086] In the present invention, features described and / or illustrated for one embodiment may be used in the same or similar manner in one or more other embodiments, and / or combined with features of other embodiments or replace features of other embodiments.

[0087] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the embodiments of the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A method for detecting and erasing a seal in an image, characterized in that: The method comprises the following steps: According to a preset seal position detection module, the area where the seal is located in the original image is determined and the area image is intercepted; The captured area image is regenerated using the pre-trained seal erasure model, and the captured area image in the original image is replaced with the regenerated area image to obtain a clean image with the seal erased. The seal erasing model includes a low-frequency repairer and a pre-trained high-frequency residual precision model.

2. The method according to claim 1, characterized in that The step of regenerating the intercepted regional image using the pre-trained seal erasure model includes: The intercepted regional image is input into a low-frequency repairer, and the low-frequency repairer removes the traces of the seal on the regional image at the pixel level to obtain a preliminary repaired regional image; Taking the initially repaired regional image as input, the prediction is performed using the time embedding method to obtain the residual prediction result; The initially repaired regional image and the residual prediction result are summed to obtain a finely repaired regenerated regional image.

3. The method according to claim 2, characterized in that The type of the low-frequency repairer is a convolutional neural network-based prediction model for image regeneration. The low-frequency repairer is trained with a preset-scale training set containing clear text images and unstamped original images, with low-frequency residuals as training targets. The low-frequency residuals are obtained by performing residual calculation and frequency separation on the initially repaired regional image and the unstamped original image.

4. The method according to claim 2, characterized in that: The method also includes the step of pre-training a high-frequency residual accurate model, including: The initially restored regional image obtained after the low-frequency restorer is processed is subjected to residual calculation and frequency separation with the original image without the stamp, so as to obtain high-frequency residual and low-frequency residual; The high-frequency residual precise model is trained with a preset-scale training set containing the preliminary repaired regional image and the high-frequency residual, and the trained high-frequency residual precise model is used to predict and obtain the residual prediction result.

5. The method according to claim 4, characterized in that The high-frequency residual is processed by a high-frequency filter in advance, the low-frequency residual is processed by a low-frequency filter in advance, and the residual prediction result is also processed by a high-frequency filter in advance.

6. The method according to claim 4, characterized in that The high-frequency residual accurate model comprises a plurality of cascaded residual down-sampling blocks, residual up-sampling blocks and dilated convolutions.

7. The method according to claim 1, characterized in that The step of determining the area where the seal is located in the original image according to the preset seal position detection module and intercepting the area image includes: Converting the original image into image data recorded using HSV values; Perform cluster analysis on the image data recorded using HSV values ​​to obtain the area where the reddish pixels are located, thereby identifying the area where one or more seals are located; The area where one or more stamps obtained by clustering are located is drawn out with a maximum circumscribed rectangle using an circumscribed contour method, and the area image is obtained by cutting out.

8. A system for detecting and erasing a seal in an image, comprising a processor, a memory, and a computer program / instruction stored in the memory, characterized in that: The processor is used to execute the computer program / instructions, and when the computer program / instructions are executed, the system implements the steps of the method according to any one of claims 1 to 7.

9. A computer-readable storage medium having a computer program / instruction stored thereon, characterized in that: When the computer program / instructions are executed by a processor, the steps of the method as claimed in any one of claims 1 to 7 are implemented.

10. A computer program product comprising a computer program / instructions, characterized in that When the computer program / instructions are executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images