Method and device for removing noisy points of scanned document image, computing equipment and storage medium

Abstract

The embodiment of the invention relates to the technical field of machine learning, in particular to a method and device for removing noisy points of a scanned document image, computing equipment and a storage medium. The method comprises the following steps: acquiring a scanned document image; performing preprocessing operation on the document image to obtain a preprocessed image; performing connected region analysis on the preprocessed image, for a connected region with the area smaller than a threshold value T, taking the gravity center of the connected region as the center, intercepting an image block on the preprocessed image, then inputting the image block into a classification model, and outputting the probability that the connected region is a noisy point as the output of the classification model; if the output value indicates that the connected region is a noisy point, removing the connected region from the image I; and finally, performing post-processing operation on the document image to obtain a processing result image of the noisy point removal method. According to the scheme, text-independent noisy points introduced in the scanning process can be removed, meanwhile, related contents such as texts and tables in the document image cannot be damaged, and the attractiveness of the document image and the performance of follow-up document analysis and recognition are improved.

Description

Technical Field

[0001] The embodiments of the present invention relate to the field of machine learning technology, and in particular to a method for removing noise from a scanned document image. Background Art

[0002] When a scanning device scans a document, some tiny noise often appears on the scanned image. In order to ensure beautiful layout, printing or OCR recognition of text content in the document image, it is necessary to remove the document image noise.

[0003] Traditional noise removal methods, such as Gaussian filtering and median filtering, do not distinguish between noise pixels and foreground pixels (such as text areas) during the noise removal process. While removing noise, it is easy to destroy the integrity and clarity of the text in the document image. Summary of the Invention

[0004] To address the problem that existing methods for removing noise from scanned document images fail to specifically address noise pixels and foreground pixels, and that noise removal can easily damage the integrity and clarity of text in document images, embodiments of the present invention provide a method, apparatus, computing device, and storage medium for removing noise from scanned document images to improve these issues. To achieve these objectives, the present invention employs the following technical solutions:

[0005] In a first aspect, a method for removing noise from a scanned document image is provided, comprising:

[0006] Obtain a scanned document image O;

[0007] Perform preprocessing on the document image O to obtain a preprocessed image I, where the width of image I is A and the height is B;

[0008] Performing a connected region analysis on the preprocessed image I. For connected regions with an area smaller than a threshold T, an image block P with a width W and a height H is cut from the image I, centered at the center of gravity (x, y) of the connected region. The image block P is then input into a classification model. The output c of the classification model is the probability that the connected region is a noise point. If the value of c is less than 0.5, the original image is kept unchanged; otherwise, the connected region is removed from the image I. The classification model is trained through deep learning using multiple sets of data, including image blocks and classification labels.

[0009] The document image O is post-processed to obtain a processing result image J of the scanned document image noise removal method.

[0010] Furthermore, the pre-processing operation includes:

[0011] If the scanned image is a color image, it is converted to grayscale and then binarized; otherwise, the scanned image is directly binarized to obtain a binary document image;

[0012] If the background of the binary document image is white and the foreground text is black, the binary document image is inverted, that is, the background value is set to 0 and the foreground value of the text is set to 1.

[0013] Furthermore, the step of intercepting an image block P with a width W and a height H from the document image I includes:

[0014] Construct an image P with width W and height H, and set P to all 0s;

[0015] [W / 2] represents the operation of dividing W by 2 and then rounding. If x-[W / 2]<0, then xl=0, otherwise xl=x-[W / 2]; if x+[W / 2]>A-1, then xr=A-1, otherwise xr=x+[W / 2]; if y-[H / 2]<0, then yl=0, otherwise yl=y-[H / 2]; if y+[H / 2]>B-1, then yr=B-1, otherwise yr=y+[H / 2];

[0016] Take the rectangular image block defined by the upper left corner (xl, yl) and the lower right corner (xr, yr) in image I and place it in image P. When placing it, the center point (x, y) of the connected area needs to correspond to the center position of image P.

[0017] Furthermore, the classification model includes a feature extraction module and a classification module; wherein the feature extraction module adopts a convolutional neural network structure, and the classification module adopts a fully connected network structure.

[0018] Furthermore, the classification model includes a feature extraction module and a classification module; wherein, the feature extraction module adopts a ResNet network structure, and the classification module adopts a 2-layer fully connected network structure, and the output of the fully connected network is a floating point number.

[0019] Furthermore, the post-processing operation includes:

[0020] Get the background pixel value of image O. First calculate O*(1-I), and then take the average of the calculated results as the background pixel value;

[0021] Traverse each pixel position (r, c) in image O. If the pixel value at position (r, c) in image I is 0, set the pixel value at position (r, c) in O to the background pixel value. Otherwise, keep the value of the corresponding position in O unchanged.

[0022] In a second aspect, an embodiment of the present invention provides a device for removing noise from a scanned document image, comprising:

[0023] An acquisition unit, configured to acquire a document image to be processed;

[0024] An input unit, configured to pre-process the document image and perform connected region analysis, and input the relevant region image into a pre-trained classification model;

[0025] The output unit is used to post-process the processing results and output the image after noise removal.

[0026] In a third aspect, an embodiment of the present invention further provides a computing device, including:

[0027] A memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the method described in any embodiment of this specification is implemented.

[0028] In a fourth aspect, an embodiment of the present invention further provides a computer-readable storage medium having a computer program stored thereon, which, when executed in a computer, enables the computer to execute the method described in any embodiment of this specification.

[0029] The beneficial effects of the present invention are:

[0030] The present invention can remove text-irrelevant noise introduced during the scanning process by performing noise processing on the scanned document image, so that the noise removal method will not destroy the relevant content such as text and tables in the document image, which is beneficial to improving the aesthetics of the document image and the performance of subsequent document analysis and recognition. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0032] Figure 1 This is a flow chart of a method for removing noise from a scanned document image provided by one embodiment of the present invention;

[0033] Figure 2 This is a schematic diagram of a document image preprocessing and connected region analysis process provided by one embodiment of the present invention;

[0034] Figure 3 This is a schematic diagram of a connected region image block classification process provided by one embodiment of the present invention;

[0035] Figure 4 This is a hardware architecture diagram of an electronic device provided by one embodiment of the present invention;

[0036] Figure 5 The figure is a structural diagram of a device for removing noise from a scanned document image provided by one embodiment of the present invention. DETAILED DESCRIPTION

[0037] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0038] As mentioned above, in the relevant technology, traditional noise removal methods, such as Gaussian filtering, median filtering, etc., do not distinguish between noise pixels and foreground pixels (such as text areas) during the noise removal process, and easily destroy the integrity and clarity of the text in the document image while removing noise.

[0039] In order to solve the above technical problems, the inventors considered using a noise removal method based on a classification model. The new method can filter out noise while ensuring the integrity and clarity of the text area, which is conducive to increasing the aesthetics of the layout and improving the performance of subsequent OCR recognition.

[0040] The specific implementation of the above concept is described below.

[0041] refer to Figure 1 , an embodiment of the present invention provides a method for removing noise from a scanned document image, the method comprising:

[0042] Step 100: Obtain a scanned document image O;

[0043] Step 101: Preprocessing the document image O to obtain a preprocessed image I, where the width of image I is A and the height is B. The preprocessing operation includes: if the scanned image is a color image, graying it and then performing a binarization operation on it; otherwise, directly binarizing the scanned image to obtain a binarized document image; if the background of the binarized document image is white and the foreground text is black, inverting the binarized document image, that is, setting the background value to 0 and setting the foreground value of the text to 1.

[0044] Step 102: Perform connected region analysis on the pre-processed image I. The process is as follows: Figure 2 As shown, for connected regions with an area smaller than the threshold T, perform the following Figure 3 The following steps are shown:

[0045] (1) Taking the centroid (x, y) of the connected region as the center, an image block P with a width of W and a height of H is intercepted on the image I. The intercepting of the image block P with a width of W and a height of H on the document image I includes: constructing an image P with a width of W and a height of H, setting P to all 0s; and using [W / 2] to represent the operation of dividing W by 2 and then rounding. If x-[W / 2]<0, then xl=0, otherwise xl=x-[W / 2]; if x+[W / 2]>A-1, then xr=A-1, otherwise xr=x+[W / 2]; if y-[H / 2]<0, then yl=0, otherwise yl=y-[H / 2]; if y+[H / 2]>B-1, then yr=B-1, otherwise yr=y+[H / 2]; take out the rectangular image block defined by the upper left corner (xl, yl) and the lower right corner (xr, yr) in image I, and place it in image P. When placing it, the center point (x, y) of the connected area needs to correspond to the center position of image P.

[0046] (2) The image block P is input into the classification model. The output c of the classification model is the probability that the connected region is a noise point. If the value of c is less than 0.5, the original image is kept unchanged. Otherwise, the connected region is removed from the image I (the pixel values ​​at the connected region position are set to 0). The classification model uses the ResNet18 network structure, and only the dimension of the last output layer of the fully connected network is changed to 1, outputting a floating-point value v.

[0047] Step 103: Perform post-processing operations based on the classification results of each connected region, including:

[0048] To obtain the background pixel value of image O, first calculate O*(1-I), and then take the average of the calculated results as the background pixel value; traverse each pixel position (r, c) in image O. If the pixel value at position (r, c) in image I is 0, then set the pixel value at position (r, c) in O to the background pixel value, otherwise keep the value of the corresponding position in O unchanged.

[0049] like Figure 4 、 Figure 5 As shown, an embodiment of the present invention provides a device for removing noise from scanned document images. The device embodiment can be implemented by software, hardware, or a combination of software and hardware. From the hardware level, Figure 4 As shown in FIG. 1 , a hardware architecture diagram of a computing device in which a device for removing noise from a scanned document image is provided in an embodiment of the present invention is provided. Figure 4 In addition to the processor, memory, and network interface shown, the computing device in the embodiment may also include other hardware, such as a forwarding chip responsible for processing messages, etc. Taking software implementation as an example, Figure 5As shown, as a device in a logical sense, it is formed by the CPU of the computing device in which it is located reading the corresponding computer program in the non-volatile memory into the internal memory and running it.

[0050] like Figure 5 As shown, this embodiment provides a device for removing noise from a scanned document image, comprising:

[0051] An acquisition unit 301 is configured to acquire a document image to be processed;

[0052] An input unit 302 is used to pre-process the document image and perform connected region analysis, and input the relevant region image into a pre-trained classification model;

[0053] The output unit 303 is used to perform post-processing on the processing result and output the image after noise removal.

[0054] It should be understood that the structure illustrated in the embodiments of the present invention does not constitute a specific limitation on the method for removing noise from scanned document images. In other embodiments of the present invention, the method for removing noise from scanned document images may include more or fewer modules than illustrated, or may combine or separate certain modules, or employ different module arrangements. The illustrated modules may be implemented in hardware, software, or a combination of both.

[0055] The information interaction, execution process, etc. between the modules in the above-mentioned device are based on the same concept as the embodiment of the method of the present invention. For specific contents, please refer to the description in the embodiment of the method of the present invention and will not be repeated here.

[0056] An embodiment of the present invention further provides a computing device including a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, a method for removing noise from a scanned document image according to any embodiment of the present invention is implemented.

[0057] An embodiment of the present invention further provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the processor executes a method for removing noise from a scanned document image according to any embodiment of the present invention.

[0058] Specifically, a system or device equipped with a storage medium can be provided, on which software program codes that implement the functions of any of the above-mentioned embodiments are stored, and a computer (or CPU or MPU) of the system or device can be enabled to read and execute the program codes stored in the storage medium.

[0059] In this case, the program code itself read from the storage medium can realize the function of any one of the above-mentioned embodiments, and thus the program code and the storage medium storing the program code constitute part of the present invention.

[0060] Examples of storage media for providing program code include floppy disks, hard disks, magneto-optical disks, optical disks (such as CD-ROM, CD-R, CD-RW, DVD-ROM, DVD-RAM, DVD-RW, DVD+RW), magnetic tapes, non-volatile memory cards, and ROMs. Alternatively, the program code can be downloaded from a server computer via a communication network.

[0061] In addition, it should be clear that the functions of any of the above embodiments can be achieved not only by executing the program code read by the computer, but also by enabling the operating system operating on the computer to complete part or all of the actual operations based on the instructions of the program code.

[0062] In addition, it can be understood that the program code read from the storage medium is written into a memory provided in an expansion board inserted into the computer or into a memory provided in an expansion module connected to the computer, and then based on the instructions of the program code, a CPU installed on the expansion board or expansion module is enabled to perform part or all of the actual operations, thereby realizing the functions of any of the above embodiments.

[0063] It should be noted that, in this article, relational terms such as first and second are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply the existence of any such actual relationship or order between these entities or operations. Moreover, the terms "comprises", "comprising" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprising a ..." do not exclude the presence of other identical factors in the process, method, article or device comprising the elements.

[0064] Those skilled in the art will understand that all or part of the steps of implementing the above-mentioned method embodiment can be completed by hardware related to program instructions, and the aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it executes the steps of the above-mentioned method embodiment; and the aforementioned storage medium includes: ROM, RAM, disk or optical disk, etc. Various media that can store program codes.

[0065] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present invention.

Claims

1. A method for removing noise from a scanned document image, characterized in that: include: Obtain a scanned document image O; Perform preprocessing on the document image O to obtain a preprocessed image I, where the width of image I is A and the height is B; Performing a connected region analysis on the preprocessed image I. For connected regions with an area smaller than a threshold T, an image block P with a width W and a height H is cut from the image I, centered at the center of gravity (x, y) of the connected region. The image block P is then input into a classification model. The output c of the classification model is the probability that the connected region is a noise point. If the value of c is less than 0.5, the original image is kept unchanged; otherwise, the connected region is removed from the image I. The classification model is trained through deep learning using multiple sets of data, including image blocks and classification labels. The document image O is post-processed to obtain a processing result image J of the scanned document image noise removal method.

2. The method according to claim 1, characterized in that The pre-processing operation includes: If the scanned image is a color image, it is converted to grayscale and then binarized; otherwise, the scanned image is directly binarized to obtain a binary document image; If the background of the binary document image is white and the foreground text is black, the binary document image is inverted, that is, the background value is set to 0 and the foreground value of the text is set to 1.

3. The method according to claim 1, characterized in that The step of intercepting an image block P with a width W and a height H from the document image I includes: Construct an image P with width W and height H, and set P to all 0s; [W / 2] represents the operation of dividing W by 2 and then rounding. If x-[W / 2]<0, then xl=0, otherwise xl=x-[W / 2]; if x+[W / 2]>A-1, then xr=A-1, otherwise xr=x+[W / 2]; if y-[H / 2]<0, then yl=0, otherwise yl=y-[H / 2]; if y+[H / 2]>B-1, then yr=B-1, otherwise yr=y+[H / 2]; Take the rectangular image block defined by the upper left corner (xl, yl) and the lower right corner (xr, yr) in image I and place it in image P. When placing it, the center point (x, y) of the connected area needs to correspond to the center position of image P.

4. The method according to claim 1, wherein The classification model includes a feature extraction module and a classification module; wherein the feature extraction module adopts a convolutional neural network structure, and the classification module adopts a fully connected network structure.

5. The method according to claim 1, wherein The classification model includes a feature extraction module and a classification module; wherein the feature extraction module adopts a residual network structure, and the classification module adopts a 2-layer fully connected network structure, and the output of the fully connected network is a floating point number v.

6. The method according to claim 1, characterized in that The post-processing operation includes: Get the background pixel value of image O. First calculate O*(1-I), and then take the average of the calculated results as the background pixel value; Traverse each pixel position (r, c) in image O. If the pixel value at position (r, c) in image I is 0, set the pixel value at position (r, c) in O to the background pixel value. Otherwise, keep the value of the corresponding position in O unchanged.

7. A device for removing noise from scanned document images, characterized in that: include: An acquisition unit, configured to acquire a document image to be processed; An input unit, configured to pre-process the document image and perform connected region analysis, and input the relevant region image into a pre-trained classification model; The output unit is used to post-process the processing results and output the image after noise removal.

8. A computing device comprising a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the method according to any one of claims 1 to 6 is implemented.

9. A computer-readable storage medium having a computer program stored thereon, which, when executed in a computer, causes the computer to execute the method according to any one of claims 1 to 6.

Images