Image denoising method and device thereof, electronic device and storage medium

Abstract

The application discloses an image denoising method and device, electronic equipment and storage medium, and relates to the technical field of artificial intelligence, wherein the image denoising method comprises the following steps: receiving a document image to be processed, and preprocessing the document image; removing first frequency noise in the preprocessed document image by using a preset deep learning network, extracting image features after noise removal processing to obtain a first image feature set, extracting image features conforming to second frequency noise in the document image by using a preset deep residual network to obtain a second image feature set, fusing the first image feature set and the second image feature set, inputting a feature map obtained after fusion into a target convolutional neural network, and outputting a target document image. The application solves the technical problem that image blurring is easily caused when a document image is denoised in the related art.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence technology or other related fields. Specifically, it relates to an image denoising method, apparatus, electronic device, and storage medium. Background Technology

[0002] In their daily operations, companies generate a large number of documents, invoices, contracts, and other materials, which continue to grow as the business develops. These paper documents cannot be edited or retrieved in real time, resulting in high storage costs and low data value. With the continuous maturation of artificial intelligence technology, companies can use various text recognition technologies to intelligently identify and structure the various vouchers and materials generated during operations, effectively reducing manual processing costs.

[0003] When recognizing document content in images, the accuracy of content recognition is severely limited by the image quality of the scanned document. The presence of noise can significantly impact the accuracy of text recognition. To improve document recognition accuracy, it is essential to denoise noisy document images. Among related technologies, image denoising methods mainly fall into two categories, which are analyzed in detail below:

[0004] 1. Filtering-based methods: These methods primarily utilize manually designed low-pass filters to remove high-frequency noise from images. While effective at removing salt-and-pepper noise, block operations can easily lead to image blurring, and hyperparameters require manual setting.

[0005] 2. Model-based methods: Denoising tasks are typically defined as performance optimization based on maximum a posteriori (MAP), whose performance largely depends on the prior information of the image. Although they have strong mathematical derivation capabilities, they struggle to recover texture structure under high noise levels, resulting in poor denoising performance.

[0006] There is currently no effective solution to the above problems. Summary of the Invention

[0007] This invention provides an image denoising method, apparatus, electronic device, and storage medium to at least solve the technical problem in related technologies where image blurring is easily caused when denoising document images.

[0008] According to one aspect of the present invention, an image denoising method is provided, comprising: receiving a document image to be processed and preprocessing the document image; using a preset deep learning network to remove a first frequency noise in the preprocessed document image and extracting image features after noise removal processing to obtain a first image feature set, wherein the noise frequency of the first frequency noise is greater than a first frequency threshold; using a preset deep residual network to extract image features in the document image that conform to a second frequency noise to obtain a second image feature set, wherein the noise frequency of the second frequency noise is less than or equal to a second frequency threshold; fusing the first image feature set and the second image feature set, and inputting the fused feature map into a target convolutional neural network to output a target document image.

[0009] Optionally, the preprocessing step of the document image includes: adjusting the current size of the document image to a preset image size; performing feature enhancement processing on the document image using a preset image processing library to complete the preprocessing step, wherein the feature enhancement processing includes at least one of the following: image contrast enhancement processing, image color enhancement processing, and image brightness enhancement processing.

[0010] Optionally, the step of using a preset deep learning network to remove the first frequency noise in the preprocessed document image and extracting the image features after noise removal to obtain a first image feature set includes: inputting the preprocessed document image into an autoencoder, wherein the autoencoder is constructed using a preset deep learning network; encoding the document image into a preset dimension representation by the encoder in the autoencoder; removing the first frequency noise in the preset dimension representation by the decoder in the autoencoder; and reconstructing the image from the preset dimension representation to obtain a target reconstructed image, wherein the preset dimension representation refers to a dimension representation with a dimension value less than a preset dimension threshold; and extracting image features from the target reconstructed image to obtain the first image feature set.

[0011] Optionally, the autoencoder is obtained through the following methods: acquiring historical document images carrying the first frequency noise and historical reconstructed images after removing the first frequency noise; constructing an image sample set from the historical document images and the historical reconstructed images, and dividing the image sample set into a training set and a test set, wherein each image sample in the image sample set contains a historical document image and a historical reconstructed image corresponding to that historical document image; constructing an initial autoencoder using the preset deep learning network, wherein the initial autoencoder includes an initial encoder and an initial decoder; inputting the historical document images from the training set into the initial encoder, whereby the initial encoder encodes the historical document images into a preset dimension representation, and the initial decoder performs image reconstruction on the preset dimension representation to obtain a historical reconstructed image corresponding to the historical document image; iteratively training the initial autoencoder until all image samples in the training set have been input; testing the initial autoencoder using the test set, and confirming that training is complete and obtaining the autoencoder when the test result indicates that the similarity between the reconstructed image output by the initial autoencoder and the historical reconstructed image is greater than a preset similarity threshold.

[0012] Optionally, the preset depth residual network is obtained by constructing multi-level residual blocks and then performing skip connections between the multi-level residual blocks to obtain the preset depth residual network, wherein each level contains M residual blocks, and each residual block contains a convolutional layer and an activation layer, where M is a positive integer.

[0013] Optionally, the step of extracting image features that conform to the second frequency noise in the document image using a preset deep residual network includes: extracting text stroke features and text contour features in the document image using the preset deep residual network to obtain image features that conform to the second frequency noise.

[0014] Optionally, the step of fusing the first image feature set and the second image feature set, and inputting the fused feature map into the target convolutional neural network to output the target document image includes: concatenating the image features in the first image feature set and the second image feature set in the channel dimension and the spatial dimension to obtain a feature vector; using a preset feature decoder to map the feature vector back to the original image space to obtain the fused feature map; and inputting the fused feature map into the target convolutional neural network to output the target document image.

[0015] According to another aspect of the present invention, an image denoising apparatus is also provided, comprising: an image preprocessing unit, configured to receive a document image to be processed and preprocess the document image; a noise removal unit, configured to remove a first frequency noise in the preprocessed document image using a preset deep learning network and extract image features after noise removal processing to obtain a first image feature set, wherein the noise frequency of the first frequency noise is greater than a first frequency threshold; a feature extraction unit, configured to extract image features in the document image that conform to a second frequency noise using a preset deep residual network to obtain a second image feature set, wherein the noise frequency of the second frequency noise is less than or equal to a second frequency threshold; and a fusion unit, configured to fuse the first image feature set and the second image feature set, and input the fused feature map into a target convolutional neural network to output a target document image.

[0016] Optionally, the image preprocessing unit includes: a size adjustment module for adjusting the current size of the document image to a preset image size; and a feature enhancement module for performing feature enhancement processing on the document image using a preset image processing library to complete the preprocessing step, wherein the feature enhancement processing includes at least one of the following: image contrast enhancement processing, image color enhancement processing, and image brightness enhancement processing.

[0017] Optionally, the noise removal unit includes: an image input module for inputting the preprocessed document image into an autoencoder, wherein the autoencoder is constructed using a preset deep learning network; an encoding module for encoding the document image into a preset dimension representation by the encoder in the autoencoder, removing first frequency noise from the preset dimension representation by the decoder in the autoencoder, and reconstructing the image from the preset dimension representation to obtain a target reconstructed image, wherein the preset dimension representation refers to a dimension representation with a dimension value less than a preset dimension threshold; and a first extraction module for extracting image features from the target reconstructed image to obtain a first image feature set.

[0018] Optionally, the autoencoder is obtained through the following methods: acquiring historical document images carrying the first frequency noise and historical reconstructed images after removing the first frequency noise; constructing an image sample set from the historical document images and the historical reconstructed images, and dividing the image sample set into a training set and a test set, wherein each image sample in the image sample set contains a historical document image and a historical reconstructed image corresponding to that historical document image; constructing an initial autoencoder using the preset deep learning network, wherein the initial autoencoder includes an initial encoder and an initial decoder; inputting the historical document images from the training set into the initial encoder, whereby the initial encoder encodes the historical document images into a preset dimension representation, and the initial decoder performs image reconstruction on the preset dimension representation to obtain a historical reconstructed image corresponding to the historical document image; iteratively training the initial autoencoder until all image samples in the training set have been input; testing the initial autoencoder using the test set, and confirming that training is complete and obtaining the autoencoder when the test result indicates that the similarity between the reconstructed image output by the initial autoencoder and the historical reconstructed image is greater than a preset similarity threshold.

[0019] Optionally, the preset depth residual network is obtained by constructing multi-level residual blocks and then performing skip connections between the multi-level residual blocks to obtain the preset depth residual network, wherein each level contains M residual blocks, and each residual block contains a convolutional layer and an activation layer, where M is a positive integer.

[0020] Optionally, the feature extraction unit includes a second extraction module, used to extract text stroke features and text contour features from the document image using the preset deep residual network, to obtain image features that conform to the second frequency noise.

[0021] Optionally, the fusion unit includes: a stitching module, used to stitch together the image features in the first image feature set and the second image feature set in the channel dimension and the spatial dimension to obtain a feature vector; a mapping module, used to map the feature vector back to the original image space using a preset feature decoder to obtain a fused feature map; and a feature map input module, used to input the fused feature map into the target convolutional neural network and output the target document image.

[0022] According to another aspect of the present invention, a computer-readable storage medium is also provided, the computer-readable storage medium including a stored computer program, wherein, when the computer program is executed, it controls the device where the computer-readable storage medium is located to perform the image denoising method described above.

[0023] According to another aspect of the present invention, an electronic device is also provided, including one or more processors and a memory, the memory being used to store one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors cause the one or more processors to implement the image denoising method of any one of the above.

[0024] In this disclosure, after receiving the document image to be processed, the document image is preprocessed. A preset deep learning network is used to remove the first frequency noise in the preprocessed document image, and the image features after noise removal are extracted to obtain a first image feature set. The noise frequency of the first frequency noise is greater than a first frequency threshold. A preset deep residual network is used to extract the image features in the document image that conform to the second frequency noise to obtain a second image feature set. The noise frequency of the second frequency noise is less than or equal to a second frequency threshold. The first image feature set and the second image feature set are fused, and the fused feature map is input into a target convolutional neural network to output the target document image.

[0025] In this disclosure, deep learning networks and deep residual networks can be used to process the high-frequency and low-frequency signals of the image and extract image features. After fusing the image features, the fused feature maps are fed into a convolutional neural network for restoration. This can effectively remove noise interference in the image and significantly improve the image clarity, thereby solving the technical problem in related technologies that the denoising of document images can easily lead to image blurring. Attached Figure Description

[0026] The accompanying drawings, which are included to provide a further understanding of the invention and form part of this application, illustrate exemplary embodiments of the invention and, together with their description, serve to explain the invention and do not constitute an undue limitation thereof. In the drawings:

[0027] Figure 1 This is a flowchart of an optional image denoising method according to an embodiment of the present invention;

[0028] Figure 2 This is a schematic diagram of another optional image denoising method according to an embodiment of the present invention;

[0029] Figure 3 This is a schematic diagram of an optional image denoising apparatus according to an embodiment of the present invention;

[0030] Figure 4 This is a hardware structure block diagram of an electronic device (or mobile device) for an image denoising method according to an embodiment of the present invention. Detailed Implementation

[0031] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0032] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0033] To facilitate understanding of the present invention by those skilled in the art, some terms or nouns involved in the various embodiments of the present invention are explained below:

[0034] Optical Character Recognition (OCR) uses image processing and pattern recognition algorithms to identify and extract text from images and convert it into machine-readable text formats, such as text files or editable documents.

[0035] OpenCV, short for Open Source Computer Vision Library, is an open-source image processing library widely used in the field of computer vision. It provides a series of functions and tools for image and video processing.

[0036] High-frequency noise in images refers to interference signals that occur at high frequencies in an image. High-frequency noise typically manifests as subtle details or color variations in the image, such as grainy or cluttered pixels, which reduces image sharpness and quality.

[0037] It should be noted that the image denoising method and apparatus of this disclosure can be used in the field of artificial intelligence technology for high- and low-frequency dual-stream denoising of document images based on artificial intelligence, and can also be used in any field other than the field of artificial intelligence technology for high- and low-frequency dual-stream denoising of document images based on artificial intelligence. This disclosure does not limit the application field of the image denoising method and apparatus.

[0038] It should be noted that all information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, and displayed data) disclosed herein are information and data authorized by the user or fully authorized by all parties. Furthermore, the collection, use, and processing of such data must comply with the relevant laws, regulations, and standards of the relevant countries and regions, and corresponding access points are provided for users to choose to authorize or refuse. For example, this system has interfaces with relevant users or organizations. Before obtaining relevant information, a request to obtain the information must be sent to the aforementioned user or organization through the interface, and the relevant information will be obtained only after receiving consent from the aforementioned user or organization.

[0039] The following embodiments of the present invention can be applied to various systems / applications / devices for denoising document images based on artificial intelligence. To improve the accuracy of document image recognition, the present invention utilizes deep learning technology and image processing technology to construct a dual-stream denoising algorithm based on high and low frequency information, effectively removing noise signals present in document images and outputting clear target document images.

[0040] The present invention will now be described in detail with reference to various embodiments.

[0041] Example 1

[0042] According to an embodiment of the present invention, an embodiment of an image denoising method is provided. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. Furthermore, although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in a different order than that shown here.

[0043] Figure 1 This is a flowchart of an optional image denoising method according to an embodiment of the present invention, such as... Figure 1 As shown, the method includes the following steps:

[0044] Step S101: Receive the document image to be processed and preprocess the document image;

[0045] Step S102: A preset deep learning network is used to remove the first frequency noise in the preprocessed document image, and the image features after noise removal are extracted to obtain a first image feature set, wherein the noise frequency of the first frequency noise is greater than the first frequency threshold.

[0046] Step S103: Use a preset deep residual network to extract image features that conform to the second frequency noise in the document image to obtain a second image feature set, wherein the noise frequency of the second frequency noise is less than or equal to the second frequency threshold.

[0047] Step S104: Fuse the first image feature set and the second image feature set, and input the fused feature map into the target convolutional neural network to output the target document image.

[0048] Through the above steps, after receiving the document image to be processed, the document image can be preprocessed. A preset deep learning network is used to remove the first frequency noise in the preprocessed document image, and the image features after noise removal are extracted to obtain a first image feature set. The noise frequency of the first frequency noise is greater than a first frequency threshold. A preset deep residual network is used to extract image features in the document image that conform to a second frequency noise to obtain a second image feature set. The noise frequency of the second frequency noise is less than or equal to a second frequency threshold. The first image feature set and the second image feature set are fused, and the fused feature map is input into a target convolutional neural network to output the target document image. In this embodiment, deep learning networks and deep residual networks can be used to process the high-frequency and low-frequency signals of the image and extract image features. After fusing the image features, the fused feature map is fed into a convolutional neural network for restoration, which can effectively remove noise interference in the image and significantly improve the image clarity, thereby solving the technical problem of image blurring when denoising document images in related technologies.

[0049] The embodiments of the present invention will now be described in detail with reference to the steps described above.

[0050] Step S101: Receive the document image to be processed and preprocess the document image.

[0051] In this embodiment, the document images include images of various types of documents such as certificates, invoices, and contracts generated by various entities (such as customers, enterprises, and personal information) in their daily work. These documents are scanned, photographed, and / or recorded to obtain scanned images, photographed images, or videos. If it is a video, it is segmented into frames to obtain multiple frames containing the document pages, and these images are used as the document images.

[0052] Since document images often contain a large amount of time sequence information and data information, it is necessary to extract the information from the image. For example, OCR technology can be used to recognize the image, obtain the material document data or material document text in the document image, and save these material documents.

[0053] Before extracting information and text from document images, due to factors such as lighting and shooting angle during the shooting process, high-frequency noise (which can be divided into salt and pepper noise and Gaussian noise) such as grains, lines, and blank spaces, as well as low-frequency noise such as blurred text, often appear in the images. Therefore, it is necessary to perform high and low frequency noise removal processing on the images.

[0054] Before performing high- and low-frequency noise reduction on the document image, this embodiment preprocesses the image. Optionally, the preprocessing step of the document image includes: adjusting the current size of the document image to a preset image size; performing feature enhancement processing on the document image using a preset image processing library to complete the preprocessing step, wherein the feature enhancement processing includes at least one of the following: image contrast enhancement processing, image color enhancement processing, and image brightness enhancement processing.

[0055] In this embodiment, after acquiring the document image to be identified, the obtained document image is adjusted to a uniform size and then input into the model for image processing.

[0056] It should be noted that image feature enhancement refers to improving the visual effect or enhancing the features of an image through a series of image processing methods. When performing feature enhancement on document images, image processing software such as OpenCV can be used. For example, histogram equalization can be performed on document images to enhance contrast and detail by adjusting the grayscale distribution; histogram matching matches the color distribution of the image with the target color distribution, improving the color effect by adjusting attributes such as hue, saturation, and brightness, thus enhancing image color.

[0057] In addition to the image feature enhancement methods mentioned above, other methods may include: spatial filtering, which enhances image features such as edges and textures by applying different filters, such as high-pass and low-pass filters; sharpening, which makes image edges clearer by enhancing high-frequency components; color adjustment, which improves image color effects by adjusting attributes such as hue, saturation, and brightness; and morphological operations, which enhance image shape features by applying morphological operations such as dilation, erosion, opening, and closing.

[0058] After completing the preprocessing of the document image, image denoising can be performed on the document image.

[0059] Step S102: A preset deep learning network is used to remove the first frequency noise in the preprocessed document image, and the image features after noise removal are extracted to obtain a first image feature set, wherein the noise frequency of the first frequency noise is greater than the first frequency threshold.

[0060] The aforementioned first frequency noise can refer to high-frequency noise in the image. High-frequency noise is random and appears abruptly in the image. In addition to high-frequency noise, document images also contain some detailed information. This detailed information is usually related to the structure of the image and is relatively continuous and smooth. High-frequency noise and detailed information can be distinguished by the image's frequency.

[0061] Optionally, step S102 includes: inputting the preprocessed document image into an autoencoder, wherein the autoencoder is constructed using a preset deep learning network; encoding the document image into a preset dimension representation by the encoder in the autoencoder; removing first frequency noise from the preset dimension representation by the decoder in the autoencoder; and reconstructing the image from the preset dimension representation to obtain a target reconstructed image, wherein the preset dimension representation refers to a dimension representation with a dimension value less than a preset dimension threshold; and extracting image features from the target reconstructed image to obtain a first image feature set.

[0062] Since most high-frequency information in document images originates from noise, deep learning methods can be used to construct autoencoders (an unsupervised neural network that learns a low-dimensional representation of input data and reconstructs it back to the original data using a decoder) to effectively remove such noise. Using autoencoders can effectively process high-frequency noise in document images, such as subtle textures and color variations, while preserving detailed information and improving the accuracy of document recognition.

[0063] Optionally, the autoencoder is obtained through the following steps: acquiring historical document images carrying first-frequency noise and historical reconstructed images after removing the first-frequency noise; constructing an image sample set from the historical document images and historical reconstructed images, and dividing the image sample set into a training set and a test set, wherein each image sample in the image sample set contains a historical document image and a historical reconstructed image corresponding to that historical document image; constructing an initial autoencoder using a pre-defined deep learning network, wherein the initial autoencoder includes an initial encoder and an initial decoder; inputting the historical document images from the training set into the initial encoder, whereby the initial encoder encodes the historical document images into a pre-defined dimension representation, and the initial decoder performs image reconstruction on the pre-defined dimension representation to obtain the historical reconstructed image corresponding to the historical document image; iteratively training the initial autoencoder until all image samples in the training set have been input; testing the initial autoencoder using a test set, and confirming that training is complete and the autoencoder is obtained when the test result indicates that the similarity between the reconstructed image output by the initial autoencoder and the historical reconstructed image is greater than a pre-defined similarity threshold.

[0064] The autoencoder provided in this embodiment is an unsupervised learning method for image denoising. Its basic idea is to encode the input data into a low-dimensional representation using an encoder, and then decode the low-dimensional representation into a reconstructed image that is as close as possible to the input image using a decoder. When training the autoencoder, training samples are prepared, an autoencoder network is constructed, and then the network is trained using these samples. A noisy image is used as input, and the network is reconstructed to be as close as possible to the original image. The parameters of the autoencoder are trained by minimizing the difference between the reconstructed image and the original image.

[0065] Once trained, the autoencoder can be used to remove high-frequency noise from images. Taking a noisy image as input, the autoencoder performs encoding and decoding operations to obtain a reconstructed image. Because the autoencoder learns the features of the original image during training, it can generate a relatively clear reconstructed image by removing high-frequency noise from the input image.

[0066] In this embodiment, after reconstructing the image using an autoencoder, image features are extracted from the target reconstructed image to obtain a first image feature set. Here, the extracted image features can refer to the features corresponding to image detail information in the original image, excluding high-frequency noise.

[0067] Step S103: Use a preset deep residual network to extract image features that conform to the second frequency noise in the document image to obtain a second image feature set, wherein the noise frequency of the second frequency noise is less than or equal to the second frequency threshold.

[0068] The second frequency noise mentioned above can refer to low-frequency noise in the image. In this embodiment, low-frequency key features are extracted.

[0069] Optionally, the preset depth residual network is obtained by constructing multi-level residual blocks and then performing skip connections between the multi-level residual blocks to obtain the preset depth residual network. Each level contains M residual blocks, and each residual block contains a convolutional layer and an activation layer, where M is a positive integer.

[0070] Optionally, the step of extracting image features that conform to the second frequency noise from the document image using a preset deep residual network includes: extracting text stroke features and text contour features from the document image using a preset deep residual network to obtain image features that conform to the second frequency noise.

[0071] This embodiment utilizes a deep residual network to extract low-frequency key features. A deep residual network is a deep learning model that addresses the vanishing gradient problem during deep network training by introducing residual blocks. In document image processing, deep residual networks can be used to extract low-frequency key features (such as text stroke features and contour features).

[0072] Specifically, a residual network is a deep learning framework composed of residual blocks and skip connections. A residual block consists of a series of convolutional and activation layers used to learn nonlinear transformations and extract features from the input data. Skip connections aim to learn the relationships between high- and low-frequency information. Through the skip connections in the residual blocks, the differences between input and output can be learned, thereby allowing the network to approximate the objective function and improving its representational capabilities.

[0073] By training deep residual networks, low-frequency information in document images can be learned and applied to tasks such as document recognition and classification. Furthermore, deep residual networks can increase their depth by stacking multiple residual blocks, thereby improving feature extraction performance.

[0074] Step S104: Fuse the first image feature set and the second image feature set, and input the fused feature map into the target convolutional neural network to output the target document image.

[0075] Optionally, step S104 includes: concatenating the image features in the first image feature set and the second image feature set in the channel dimension and the spatial dimension to obtain a feature vector; using a preset feature decoder to map the feature vector back to the original image space to obtain a fused feature map; and inputting the fused feature map into the target convolutional neural network to output the target document image.

[0076] This embodiment employs simultaneous image channel and spatial dimension stitching. To achieve simultaneous stitching in both dimensions, this embodiment first stitches in the channel dimension and then stitches in the spatial dimension. For example, consider two feature maps A and B, with dimensions [H1, W1, C1] and [H2, W2, C2], respectively. First, A and B are stitched in the channel dimension to form a new feature map C with dimensions [H1, W1, C1 + C2]. Then, C and B are stitched in the spatial dimension to form the final feature map D with dimensions [H1 + H2, W1 + W2, C1 + C2].

[0077] It's important to note that because both image channels and spatial dimensions are used for concatenation, the resulting feature vectors often have excessively long dimensions, thus increasing the model's computational complexity and the number of parameters. Alternatively, a single channel or spatial dimension concatenation operation could be used.

[0078] In this embodiment, after obtaining the feature vectors, the images are fused using image fusion algorithms such as weighted averaging and Laplacian pyramid fusion to combine the image features (with high-frequency noise removed) with the extracted low-frequency image features. These algorithms can balance the relationship between the two based on different weights and fusion strategies to achieve the best fusion effect.

[0079] The following describes in detail another optional implementation method.

[0080] Figure 2 This is a schematic diagram of another optional image denoising method according to an embodiment of the present invention, such as... Figure 2 As shown, after receiving the collected document image from an external device, the document image is preprocessed, then high-frequency noise is removed and low-frequency key information is extracted. After that, image fusion is performed to obtain the target document image. Finally, OCR recognition is performed on the target document image to obtain the text content in the image.

[0081] Image denoising is achieved through six modules: input module, preprocessing module, upper branch module, lower branch module, output module, and fusion module. These modules effectively remove noise from image documents, improving the accuracy and stability of OCR document recognition. Specific module descriptions are as follows:

[0082] (1) Input module: Collects document images to be recognized, adjusts the scanned document images to a uniform size, and inputs them into the model for image processing.

[0083] (2) Preprocessing module: Image processing techniques (e.g., OpenCV image processing software) are used to enhance the contrast, brightness, color, and other features of the input image, making the image clearer and thus improving its readability and usability. For example, histogram equalization using OpenCV image processing software can enhance image contrast by redistributing the gray levels of the image; histogram matching can enhance image color by matching the color distribution of the image with the target color distribution.

[0084] (3) Upper Branch Module: High-frequency information is processed using deep learning methods. Since most of the high-frequency information in images comes from noise, an autoencoder can be constructed using deep learning methods to effectively remove such noise. An autoencoder is an unsupervised learning neural network that can learn a low-dimensional representation of the input data and reconstruct it back to the original data through a decoder. By using an autoencoder, high-frequency noise in document images, such as subtle textures and color variations, can be effectively processed while preserving the detailed information in the document, thus improving the accuracy of document recognition.

[0085] (4) Sub-branch module: Employs a deep residual network to extract low-frequency key features. A deep residual network is a deep learning model that addresses the vanishing gradient problem during deep network training by introducing residual blocks. By training a deep residual network, low-frequency information in document images can be learned and applied to tasks such as document recognition and classification. Furthermore, the deep residual network can increase its depth by stacking multiple residual blocks, thereby improving feature extraction performance.

[0086] The image denoising method employed in this invention utilizes a two-branch module. The upper branch module processes high-frequency noise signals, filtering out high-frequency noise to ensure that key information in the image is preserved while noise is removed. The lower branch module processes low-frequency signals, extracting local and global information from the image. Finally, the feature maps extracted from both branches are fused and fed into a convolutional neural network for restoration, resulting in a clear image. This method efficiently utilizes both high-frequency and low-frequency signals in the image, effectively removing noise interference and significantly improving image clarity.

[0087] (5) Fusion Module: This module fuses the high-frequency and low-frequency features extracted from the upper and lower branches. By fusing high-frequency and low-frequency features, the details and structural information in the document image can be comprehensively considered, thereby obtaining the denoised image.

[0088] (6) Output module: Outputs a clean and noise-free target document image for subsequent OCR technology to automatically recognize and extract text from the text image.

[0089] The above implementation provides a dual-stream denoising method based on high and low frequency information, effectively utilizing deep learning technology to remove noise from image documents and improve the accuracy and stability of recognition. The technical effects of this implementation include:

[0090] 1. Convolutional neural networks can effectively remove high-frequency noise from image documents while preserving detailed information.

[0091] 2. By utilizing deep residual networks, low-frequency features in document images can be effectively extracted, providing a solid foundation for document recognition.

[0092] 3. In the low-frequency information processing branch, a network residual structure is adopted. By stacking multiple residual blocks, the network depth is increased, thereby improving the feature extraction effect.

[0093] The following is a detailed description with reference to another embodiment.

[0094] Example 2

[0095] The image denoising device provided in this embodiment includes multiple implementation units, each of which corresponds to a specific implementation step in Embodiment 1 above.

[0096] Figure 3 This is a schematic diagram of an optional image denoising apparatus according to an embodiment of the present invention, such as... Figure 3 As shown, the image denoising device may include: an image preprocessing unit 30, a noise removal unit 31, a feature extraction unit 32, and a fusion unit 33, wherein...

[0097] Image preprocessing unit 30 is used to receive the document image to be processed and to preprocess the document image;

[0098] The noise removal unit 31 is used to remove the first frequency noise in the preprocessed document image using a preset deep learning network, and extract the image features after noise removal to obtain a first image feature set, wherein the noise frequency of the first frequency noise is greater than the first frequency threshold.

[0099] The feature extraction unit 32 is used to extract image features that conform to the second frequency noise in the document image using a preset deep residual network to obtain a second image feature set, wherein the noise frequency of the second frequency noise is less than or equal to the second frequency threshold.

[0100] The fusion unit 33 is used to fuse the first image feature set and the second image feature set, and input the fused feature map into the target convolutional neural network to output the target document image.

[0101] The aforementioned image denoising device can, after receiving the document image to be processed, preprocess the document image through the image preprocessing unit 30, remove the first frequency noise in the preprocessed document image through the noise removal unit 31 using a preset deep learning network, and extract the image features after noise removal processing to obtain a first image feature set, wherein the noise frequency of the first frequency noise is greater than a first frequency threshold; extract the image features in the document image that conform to the second frequency noise through the feature extraction unit 32 using a preset deep residual network to obtain a second image feature set, wherein the noise frequency of the second frequency noise is less than or equal to a second frequency threshold; fuse the first image feature set and the second image feature set through the fusion unit 33, and input the fused feature map into the target convolutional neural network to output the target document image. In this embodiment, deep learning networks and deep residual networks can be used to process the high-frequency and low-frequency signals of the image and extract image features. After fusing the image features, the fused feature maps are fed into a convolutional neural network for restoration. This can effectively remove noise interference in the image and significantly improve the image clarity, thereby solving the technical problem in related technologies that the denoising of document images can easily lead to image blurring.

[0102] Optionally, the image preprocessing unit includes: a size adjustment module for adjusting the current size of the document image to a preset image size; and a feature enhancement module for performing feature enhancement processing on the document image using a preset image processing library to complete the preprocessing steps, wherein the feature enhancement processing includes at least one of the following: image contrast enhancement processing, image color enhancement processing, and image brightness enhancement processing.

[0103] Optionally, the noise removal unit includes: an image input module for inputting the preprocessed document image into an autoencoder, wherein the autoencoder is constructed using a preset deep learning network; an encoding module for encoding the document image into a preset dimension representation by the encoder in the autoencoder, removing first frequency noise from the preset dimension representation by the decoder in the autoencoder, and reconstructing the image from the preset dimension representation to obtain a target reconstructed image, wherein the preset dimension representation refers to a dimension representation with a dimension value less than a preset dimension threshold; and a first extraction module for extracting image features from the target reconstructed image to obtain a first image feature set.

[0104] Optionally, the autoencoder is obtained through the following steps: acquiring historical document images carrying first-frequency noise and historical reconstructed images after removing the first-frequency noise; constructing an image sample set from the historical document images and historical reconstructed images, and dividing the image sample set into a training set and a test set, wherein each image sample in the image sample set contains a historical document image and a historical reconstructed image corresponding to that historical document image; constructing an initial autoencoder using a pre-defined deep learning network, wherein the initial autoencoder includes an initial encoder and an initial decoder; inputting the historical document images from the training set into the initial encoder, whereby the initial encoder encodes the historical document images into a pre-defined dimension representation, and the initial decoder performs image reconstruction on the pre-defined dimension representation to obtain the historical reconstructed image corresponding to the historical document image; iteratively training the initial autoencoder until all image samples in the training set have been input; testing the initial autoencoder using a test set, and confirming that training is complete and the autoencoder is obtained when the test result indicates that the similarity between the reconstructed image output by the initial autoencoder and the historical reconstructed image is greater than a pre-defined similarity threshold.

[0105] Optionally, the preset depth residual network is obtained by constructing multi-level residual blocks and then performing skip connections between the multi-level residual blocks to obtain the preset depth residual network. Each level contains M residual blocks, and each residual block contains a convolutional layer and an activation layer, where M is a positive integer.

[0106] Optionally, the feature extraction unit includes: a second extraction module, used to extract text stroke features and text contour features from the document image using a preset deep residual network, to obtain image features that conform to the second frequency noise.

[0107] Optionally, the fusion unit includes: a stitching module for stitching the image features in the first image feature set and the second image feature set in the channel dimension and spatial dimension to obtain a feature vector; a mapping module for mapping the feature vector back to the original image space using a preset feature decoder to obtain a fused feature map; and a feature map input module for inputting the fused feature map into the target convolutional neural network to output the target document image.

[0108] The image denoising device described above may also include a processor and a memory. The image preprocessing unit 30, noise removal unit 31, feature extraction unit 32, fusion unit 33, etc., are all stored in the memory as program units, and the processor executes the program units stored in the memory to realize the corresponding functions.

[0109] The aforementioned processor contains a kernel, which retrieves the corresponding program units from memory. One or more kernels can be configured; by adjusting kernel parameters, denoising of document images can be performed, and text content can be extracted from the denoised images.

[0110] The aforementioned memory may include non-permanent memory in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM, and the memory includes at least one memory chip.

[0111] This application also provides a computer program product, which, when executed on a data processing device, is suitable for executing an initialization program having the following method steps: receiving a document image to be processed and preprocessing the document image; using a preset deep learning network to remove a first frequency noise from the preprocessed document image and extracting image features after noise removal processing to obtain a first image feature set, wherein the noise frequency of the first frequency noise is greater than a first frequency threshold; using a preset deep residual network to extract image features in the document image that conform to a second frequency noise to obtain a second image feature set, wherein the noise frequency of the second frequency noise is less than or equal to a second frequency threshold; fusing the first image feature set and the second image feature set, and inputting the fused feature map into a target convolutional neural network to output a target document image.

[0112] According to another aspect of the present invention, a computer-readable storage medium is also provided, the computer-readable storage medium including a stored computer program, wherein, when the computer program is executed, it controls the device where the computer-readable storage medium is located to perform the above-described image denoising method.

[0113] According to another aspect of the present invention, an electronic device is also provided, including one or more processors and a memory, the memory being used to store one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors cause the one or more processors to implement the image denoising method described above.

[0114] Figure 4 This is a hardware structure block diagram of an electronic device (or mobile device) for an image denoising method according to an embodiment of the present invention. Figure 4 As shown, an electronic device may include one or more ( Figure 4 The processor 402 (which may include, but is not limited to, a microprocessor MCU or a programmable logic device FPGA, etc.) and the memory 404 for storing data may also be included. In addition, it may include: a display, an input / output interface (I / O interface), a universal serial bus (USB) port (which may be included as one of the ports of the I / O interface), a network interface, a keyboard, a power supply, and / or a camera. Those skilled in the art will understand that... Figure 4 The structure shown is for illustrative purposes only and does not limit the structure of the electronic device described above. For example, the electronic device may also include components that are more... Figure 4 The more or fewer components shown, or having the same Figure 4 The different configurations shown.

[0115] The sequence numbers of the above embodiments of the present invention are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0116] In the above embodiments of the present invention, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0117] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. The device embodiments described above are merely illustrative; for example, the division of units can be a logical functional division, and in actual implementation, there may be other division methods. For instance, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the displayed or discussed mutual coupling, direct coupling, or communication connection may be through some interfaces; the indirect coupling or communication connection between units or modules may be electrical or other forms.

[0118] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0119] Furthermore, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0120] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, read-only memory (ROM), random access memory (RAM), portable hard drives, magnetic disks, or optical disks.

[0121] The above description is only a preferred embodiment of the present invention. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. An image denoising method, characterized in that, include: Receive the document image to be processed and preprocess the document image; The method involves using a preset deep learning network to remove first frequency noise from a preprocessed document image and extracting image features after noise removal to obtain a first image feature set. This includes: inputting the preprocessed document image into an autoencoder, wherein the autoencoder is constructed using a preset deep learning network; encoding the document image into a preset dimension representation using the encoder in the autoencoder; removing the first frequency noise from the preset dimension representation using the decoder in the autoencoder; and reconstructing the image from the preset dimension representation to obtain a target reconstructed image. The noise frequency of the first frequency noise is greater than a first frequency threshold, and the preset dimension representation refers to a dimension representation with a dimension value less than the preset dimension threshold. Finally, image features are extracted from the target reconstructed image to obtain the first image feature set. A preset deep residual network is used to extract image features that conform to the second frequency noise in the document image to obtain a second image feature set, wherein the noise frequency of the second frequency noise is less than or equal to the second frequency threshold; The first image feature set and the second image feature set are fused together, and the resulting feature map is input into the target convolutional neural network to output the target document image.

2. The image denoising method according to claim 1, characterized in that, The steps for preprocessing the document image include: Adjust the current size of the document image to the preset image size; The document image is subjected to feature enhancement processing using a preset image processing library to complete the preprocessing step, wherein the feature enhancement processing includes at least one of the following: image contrast enhancement processing, image color enhancement processing, and image brightness enhancement processing.

3. The image denoising method according to claim 1, characterized in that, The autoencoder is obtained in the following way: Acquire historical document images carrying the first frequency noise and historical reconstructed images after removing the first frequency noise; The historical document images and the historical reconstructed images are assembled into an image sample set, and the image sample set is divided into a training set and a test set. Each image sample in the image sample set contains a historical document image and a historical reconstructed image corresponding to the historical document image. An initial autoencoder is constructed using the preset deep learning network, wherein the initial autoencoder includes an initial encoder and an initial decoder; The historical document images in the training set are input into the initial encoder, which encodes the historical document images into a preset dimension representation. The initial decoder then performs image reconstruction on the preset dimension representation to obtain a historical reconstructed image corresponding to the historical document image. The initial autoencoder is iteratively trained until all image samples in the training set have been input. The initial autoencoder is tested using the test set. If the test result indicates that the similarity between the reconstructed image output by the initial autoencoder and the historical reconstructed image is greater than a preset similarity threshold, the training is confirmed to be complete, and the autoencoder is obtained.

4. The image denoising method according to claim 1, characterized in that, The preset depth residual network is obtained in the following way: Construct multi-level residual blocks and then connect them in a skip connection to obtain the preset depth residual network. Each level contains M residual blocks, and each residual block contains a convolutional layer and an activation layer, where M is a positive integer.

5. The image denoising method according to claim 4, characterized in that, The step of extracting image features that conform to the second frequency noise from the document image using a preset deep residual network includes: The preset depth residual network is used to extract text stroke features and text contour features from the document image to obtain image features that conform to the second frequency noise.

6. The image denoising method according to claim 1, characterized in that, The steps of fusing the first image feature set and the second image feature set, inputting the fused feature map into the target convolutional neural network, and outputting the target document image include: The image features in the first image feature set and the second image feature set are concatenated in the channel dimension and the spatial dimension to obtain the feature vector; The feature vectors are mapped back to the original image space using a preset feature decoder to obtain the fused feature map. The fused feature map is input into the target convolutional neural network, and the target document image is output.

7. An image denoising device, characterized in that, include: An image preprocessing unit is used to receive a document image to be processed and to preprocess the document image. The noise removal unit is used to remove the first frequency noise in the preprocessed document image using a preset deep learning network, and extract the image features after noise removal to obtain a first image feature set, wherein the noise frequency of the first frequency noise is greater than a first frequency threshold. The noise removal unit includes: an image input module for inputting the preprocessed document image into an autoencoder, wherein the autoencoder is constructed using a preset deep learning network; an encoding module for encoding the document image into a preset dimension representation by the encoder in the autoencoder, removing first frequency noise from the preset dimension representation by the decoder in the autoencoder, and reconstructing the image from the preset dimension representation to obtain a target reconstructed image, wherein the preset dimension representation refers to a dimension representation with a dimension value less than a preset dimension threshold; and a first extraction module for extracting image features from the target reconstructed image to obtain a first image feature set. The feature extraction unit is used to extract image features that conform to the second frequency noise in the document image using a preset deep residual network, to obtain a second image feature set, wherein the noise frequency of the second frequency noise is less than or equal to the second frequency threshold. The fusion unit is used to fuse the first image feature set and the second image feature set, and input the fused feature map into the target convolutional neural network to output the target document image.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes a stored computer program, wherein, when the computer program is executed, it controls the device on which the computer-readable storage medium is located to perform the image denoising method according to any one of claims 1 to 6.

9. An electronic device, characterized in that, The device includes one or more processors and a memory, the memory being used to store one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors cause the one or more processors to implement the image denoising method according to any one of claims 1 to 6.

Images