Image processing device, image processing method and storage medium for eliminating lines

Abstract

The present disclosure relates to an image processing device, an image processing method and a storage medium. According to one embodiment of the present disclosure, the image processing device includes: a binarization unit for binarizing a grayscale document image as a target image; a division unit for obtaining a plurality of strip regions arranged along a first direction by dividing the target image; a directed graph determination unit for determining a directed graph for the entire target image based on the intra-region connected domains of the plurality of strip regions; a target path determination unit for determining a target path related to a single-source shortest path of the directed graph based on the directed graph; and a line elimination unit for eliminating lines corresponding to the target path in the grayscale document image. The method, device and storage medium of the present disclosure can at least help achieve one of the following effects: eliminating noise lines in the document image, fast processing speed, less computing resources occupied, and improving the recognition performance of the text recognition engine.

Description

Technical Field

[0001] The present disclosure relates generally to image processing, and more particularly to an image processing apparatus, an image processing method, and a storage medium for eliminating lines. Background Art

[0002] Optical Character Recognition (OCR) technology can be used to convert text in images. In the application of optical character recognition, the input image needs to be preprocessed to improve the recognition performance of the OCR recognition engine. Generally speaking, the more noise there is in the input image, the lower the recognition performance of the recognition engine. Preprocessing the input image includes eliminating noise. Therefore, preprocessing can improve the performance of the recognition engine.

[0003] Therefore, there are many traditional image processing methods for removing noise such as salt and pepper noise in images. Summary of the invention

[0004] A brief overview of the present disclosure will be given below in order to provide a basic understanding of certain aspects of the present disclosure. It should be understood that this overview is not an exhaustive overview of the present disclosure. It is not intended to identify key or important parts of the present disclosure, nor is it intended to limit the scope of the present disclosure. Its purpose is simply to give certain concepts in a simplified form as a prelude to a more detailed description discussed later.

[0005] The inventors have found that in OCR recognition, there may be non-text identification lines that identify rows and / or columns near the text in the input document image. For example, text line images from bank forms, insurance, and express delivery contain many overlines and underlines. Usually these lines will reduce the performance of the recognition engine. These lines can be considered as noise for the recognition engine and need to be preprocessed. In some cases, some recognition engines may misjudge the foreground pixels and background pixels in the input image, which will reduce the character recognition performance of the recognition engine. For example, when the recognition engine identifies the above-mentioned identification lines as foreground pixels and attempts to perform text recognition, the character recognition performance of the recognition engine will be degraded. In view of this, the inventors propose a technology based on a directed graph for eliminating lines in document images in the present disclosure.

[0006] According to one aspect of the present disclosure, an image processing method is provided, including: binarizing a grayscale document image as a target image; obtaining a plurality of strip regions arranged along a first direction by dividing the target image; determining a directed graph for the entire target image based on the intra-region connected domains of the plurality of strip regions; determining a target path related to a single-source shortest path of the directed graph based on the directed graph; and eliminating lines in the grayscale document image corresponding to the target path.

[0007] According to one aspect of the present disclosure, an image processing device is provided, comprising: a binarization unit for binarizing a grayscale document image as a target image; a division unit for obtaining a plurality of strip regions arranged along a first direction by dividing the target image; a directed graph determination unit for determining a directed graph for the entire target image based on the intra-region connected domains of the plurality of strip regions; a target path determination unit for determining a target path related to a single-source shortest path of the directed graph based on the directed graph; and a line elimination unit for eliminating lines in the grayscale document image corresponding to the target path.

[0008] According to another aspect of the present disclosure, a computer-readable storage medium having a program stored thereon is provided, characterized in that the program enables an image processing method to be implemented when the program is executed by a processor, the image processing method comprising: binarizing a grayscale document image as a target image; obtaining a plurality of strip regions arranged along a first direction by dividing the target image; determining a directed graph for the entire target image based on the intra-region connected domains of the plurality of strip regions; determining a target path related to a single-source shortest path of the directed graph based on the directed graph; and eliminating lines in the grayscale document image corresponding to the target path.

[0009] The method, device and storage medium disclosed herein can at least help achieve one of the following effects: eliminating row lines in a document image, eliminating column lines in a document image, eliminating noise lines in a document image, fast processing speed, less computing resources occupied, and improving the recognition performance of a text recognition engine. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] The following description of the embodiments of the present disclosure will help to more easily understand the above and other purposes, features and advantages of the present disclosure with reference to the accompanying drawings. The accompanying drawings are only for the purpose of illustrating the principles of the present disclosure. The sizes and relative positions of the elements in the accompanying drawings are not necessarily drawn to scale. The same reference numerals may represent the same features. In the accompanying drawings:

[0011] Figure 1 A flowchart of an image processing method according to an embodiment of the present disclosure is shown;

[0012] Figure 2 An example of a target image is shown;

[0013] Figure 3 The target image after hiding the odd-numbered bar areas and after hiding the even-numbered bar areas is exemplarily shown;

[0014] Figure 4 A flowchart of a method for determining a directed graph according to an embodiment of the present disclosure is shown;

[0015] Figure 5 A flowchart of a method for determining the cost of an edge according to an embodiment of the present disclosure is shown;

[0016] Figure 6 shows the corresponding intra-region connected domain of an example single-source shortest path according to one embodiment of the present disclosure;

[0017] Figure 7 shows a connected domain in a target region determined by removal processing according to an embodiment of the present disclosure;

[0018] Figure 8 A grayscale document image after eliminating a line corresponding to a target path in the grayscale document image according to an embodiment of the present disclosure is shown;

[0019] Fig. 9 shows a connected domain in a target region determined by removal processing according to another embodiment of the present disclosure;

[0020] Fig.10 An exemplary block diagram of an image processing apparatus according to an embodiment of the present disclosure is shown;

[0021] Fig.11 A flowchart of an optical character recognition method according to an embodiment of the present disclosure is shown;

[0022] Fig.12 An exemplary block diagram showing an optical character recognition apparatus according to an embodiment of the present disclosure; and

[0023] Fig.13 An exemplary block diagram of an information processing device according to an embodiment of the present disclosure is shown. DETAILED DESCRIPTION

[0024] Exemplary embodiments of the present disclosure are described below in conjunction with the accompanying drawings. For the sake of clarity and conciseness, not all features of the actual embodiments are described in the specification. However, it should be understood that many implementation-specific decisions may be made in the process of developing any such actual embodiments in order to achieve the specific goals of the developer, and these decisions may vary from embodiment to embodiment.

[0025] It is also necessary to explain here that, in order to avoid obscuring the contents of the present disclosure due to unnecessary details, only the device structure closely related to the scheme according to the contents of the present disclosure is shown in the accompanying drawings, while other details that are not closely related to the contents of the present disclosure are omitted.

[0026] It should be understood that the present disclosure is not limited to the described implementation forms due to the following description with reference to the accompanying drawings. In this article, where feasible, the embodiments can be combined with each other, features between different embodiments can be replaced or borrowed, and one or more features can be omitted in one embodiment.

[0027] It will be appreciated by those skilled in the art that the various aspects of the exemplary embodiments may be implemented as systems, methods or computer program products. Therefore, the various aspects of the exemplary embodiments may be specifically implemented in the following forms, i.e., may be complete hardware embodiments, complete software embodiments (including firmware, resident software, microcode, etc.) or embodiments combining software parts and hardware parts. These may be generally referred to as "circuits", "modules" or "systems" herein. In addition, the various aspects of the exemplary embodiments may take the form of a computer program product embodied as one or more computer-readable media, which may be embodied with computer-readable program code. The computer program may be distributed, for example, via a computer network, or the computer program may be located on one or more remote servers, or may be embedded in the memory of a device.

[0028] Any combination of one or more computer-readable media can be used. Computer-readable media can be a computer-readable signal medium or a computer-readable storage medium. Computer-readable storage media can be, for example, but not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, equipment or devices, or any suitable combination of the foregoing. More specific examples (non-exhaustive lists) of computer-readable storage media include the following: electrical connections with one or more wires, portable computer disks, hard disks, random access memories (RAM), read-only memories (ROM), erasable programmable read-only memories (EPROM or flash memory), optical fibers, portable compact disk read-only memories (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing. In the context of this article, computer-readable storage media can be any tangible medium containing or storing a program used by an instruction execution system, device or device or associated with an instruction execution system, device or device.

[0029] Computer readable signal media may include, for example, a data signal propagated in baseband or as part of a carrier wave with computer readable program code therein. Such a propagation signal may take any appropriate form, including but not limited to electromagnetic, optical, or any appropriate combination thereof.

[0030] Computer readable signal media may be any computer readable medium other than computer readable storage media that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device.

[0031] The program code embodied in a computer-readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wired, optical fiber cable, radio frequency, etc., or any suitable combination of the foregoing.

[0032] Computer program code for carrying out operations of various aspects of the exemplary embodiments disclosed herein may be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and the like, and conventional procedural programming languages ​​such as the "C" programming language or similar programming languages.

[0033] The various aspects of the exemplary embodiments disclosed herein are described below with reference to the flowcharts and / or block diagrams of the methods, devices (systems) and computer program products according to the exemplary embodiments. It should be understood that each box in the flowchart and / or block diagram and the combination of the boxes in the flowchart and / or block diagram can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer or other programmable data processing device to produce a machine, so that these instructions executed by the computer or other programmable data processing device produce a device for implementing the functions / operations specified in the boxes in the flowchart and / or block diagram.

[0034] These computer program instructions may also be stored in a computer-readable medium that can direct a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable medium produce a manufactured product including instructions for implementing the functions / operations specified in the blocks in the flowchart and / or block diagram.

[0035] Computer program instructions may also be loaded onto a computer or other programmable data processing device, causing a series of operational steps to be performed on the computer or other programmable data processing device to produce a computer-implemented process, so that the instructions executed on the computer or other programmable device provide a process for implementing the functions / operations specified in the blocks in the flowchart and / or block diagram.

[0036] One aspect of the present disclosure provides an image processing method for processing document images. The processed document image can be used for text recognition. The method can be used as part of the preprocessing of the input image in an optical character recognition application. The method can be used to eliminate noise lines in an image. The line can be: a row line that identifies a row, a column line that identifies a column, or other non-character identification lines. The line can be a solid line, a discontinuous line, or other line with a definite overall orientation. Discontinuous lines include dotted lines and dash-dot lines. The document image can be a color image. Considering that a color document image can be transformed into a grayscale image by conventional methods during text recognition, the image processing method is described below using a grayscale document image as an example. Referring to Figure 1 This method is described exemplarily.

[0037] Figure 1 A flow chart of an image processing method 10 according to an embodiment of the present disclosure is shown. The input of the method includes a grayscale document image. The output of the method includes a grayscale document image with at least part of non-character identification lines eliminated.

[0038] At step S101, the grayscale document image Img is binarized as the target image Imo. This step can be implemented using the Sauvola method, for example. Binarization can remove the background of the image. Figure 2 An example of the target image Imo. Figure 2 The target image Imo includes line L, character Ch1 and character Ch2. The corresponding grayscale document image Img includes line l corresponding to line L. The goal of the image processing method 10 includes eliminating line l in the grayscale document image Img. It should be noted that Figure 2 The reference numerals (for example, "L") and S-shaped indicator lines are only shown to illustrate the elements included in the example target image Imo of the target image, and the example target image Imo itself does not include the reference numerals and S-shaped indicator lines. Although the characters in the example target image Imo are handwritten, the characters in the target image are not limited to this and may also be printed, or may even include both printed characters and handwritten characters. It should be noted that the document image of the character row may be divided into two document images along the center line and then method 10 may be executed to remove the bottom row line and the top row line of the document image of the character row, respectively. If the input image to be processed is already a binarized image, the binarization step may be omitted. Although Figure 2 In the figure, a line L in a horizontal direction is shown, and the image processing method is described by taking the elimination of the corresponding line l of the line L as an example, but the method of eliminating multiple lines in other directions and the concept of eliminating one line in a horizontal direction have the same point. Therefore, those skilled in the art can understand from the method of eliminating one line in a horizontal direction that the method of the present disclosure can also be used to eliminate multiple lines and / or lines with other orientations.

[0039] In step S103, a plurality of strip regions R1, R2, ..., R3 arranged along a first direction are obtained by dividing the target image Imo. n ,……,R N . The first direction may be a horizontal direction (lateral direction of the target image), a vertical direction (vertical direction of the target image), or other directions. The length of each strip area may be equal to one of the length or width of the target image. For example, in an exemplary division method, along the horizontal direction, the target image Imo with a length of Len is divided into N rectangular areas, each rectangular area with a length equal to the width w of the target image Imo, and a width of Len / N. In order to ensure that the line to be eliminated is detected quickly and accurately, the width of each strip area may be equal to a predetermined value of two to ten pixel widths, for example. According to this division method, the odd-numbered strip areas or the even-numbered strip areas are hidden, and the target image Imo will become an image with dotted lines. Splicing these strip areas together in order will result in the target image Imo. Figure 3 The blurred target image after hiding the odd-numbered strip area and hiding the even-numbered strip area is shown as an example, wherein, relative to Figure 2 The image in the figure is appropriately scaled; the first direction is the horizontal direction; (a) is the blurred target image Imo after hiding the even-numbered strip areas o ; and (b) is the blurred target image Imo after hiding the odd-numbered strip areas e It should be noted that the widths of the multiple strip regions of the present disclosure may be equal or unequal.

[0040] In step S105, based on a plurality of strip regions R n CC ni’ Determine a directed graph for the entire target image, where i' is the index for distinguishing different connected domains, i'=1, 2, ..., I'. I' is a strip area R n In the present disclosure, the connected domains in the strip region are also referred to as the connected domains within the region. When it is not necessary to distinguish the strip regions to which the connected domains in each region belong, “CC” or “CC i " is used to represent "intra-region connected domains", i = 1, 2, ..., I. I is the total number of intra-region connected domains in the target image Imo. The number of intra-region connected domains in each strip region can be zero, 1, 2 or more. The intra-region connected domains of all strip regions of the target image constitute the intra-region connected domain set {CC i}. The connected domain CC in each region ni’ It is composed of foreground pixels of the target image, and each region has a connected domain CC ni’ In a single bar area Rn Although a certain foreground line of the target image before division constitutes an initial connected domain that runs through multiple strip regions, for the target image after division, due to the distinction of the boundaries of different strip regions, the line will be divided into multiple segments corresponding to the connected domains in multiple regions. Accordingly, the initial connected domain is considered to be composed of multiple connected domains in the region. In other words, each strip region can be regarded as a sub-image, and the connected domain in each corresponding sub-image is the connected domain in the region. For example, when a connected domain in the target image spans two adjacent strip regions, the two connected domain partitions in the two adjacent strip regions are identified as connected domains in two regions. Figure 3 (a) and Figure 3 The connected domain shown in (b) is the intra-region connected domain. All intra-region connected domains in the two graphs constitute the intra-region connected domain set {CC i}.

[0041] A directed graph consists of multiple nodes n j , multiple edges e jj’ , where j is the index that distinguishes different nodes, e jj’ Used to refer to the connection node n j and node n j’ The edge is from node n j (Head node) points to node n j’ (tail node). Each edge has a cost cn jj’ These nodes, edges, costs and connected domain sets in the region {CC i}, and contains the information of the connected domain in the region. Since it is the edge of a directed graph, the edge here refers to the directed edge.

[0042] For the case where a grayscale document image has both bottom and top lines, in one example, a connected area centered above the horizontal median line of the target image can be selected to set the nodes of a directed graph to eliminate the top line; or a connected area centered below the horizontal median line of the target image can be selected to set the nodes of a directed graph to eliminate the bottom line.

[0043] At step S107, a target path Po related to a single-source shortest path P (also referred to as "shortest path") of the directed graph is determined based on the directed graph. In one example, this step may include: adding a virtual node to the directed graph; constructing a virtual edge from the virtual node to other nodes; and setting the cost of the virtual edge to zero. After determining the single-source shortest path P of the directed graph, the target path Po can be determined based on the determined single-source shortest path. In one example, the single-source shortest path can be directly used as the target path.

[0044] In one example, when a virtual node is used as a starting point, a single-source shortest path of a directed graph is determined, for example, by a Bellman-Ford method / algorithm. The single-source shortest path includes a plurality of nodes n k , k is the index to distinguish each node, k = 1, 2, ..., K. These nodes constitute the node set {n k}, node n k Connected domains within the region Therefore, the single-source shortest path corresponds to the connected domains in multiple regions. These connected domains in the corresponding regions can be called shortest path connected domains, and these shortest path connected domains constitute the shortest path connected domain set. Similar to the shortest path P, the target path Po also includes multiple corresponding nodes n k’ (hereinafter referred to as the target node) and the connected domain within the region (hereinafter referred to as the connected domain in the target area), the target node n k’ The connected domain in the corresponding region is the connected domain in the target region These corresponding nodes constitute the target node set {n k’}. The connected domains in these corresponding regions constitute the connected domain set in the target region. More specifically, the target node n k’ The node set selected from the shortest path {n k}. In one example, the target node set {n k’} can be connected with the shortest path node set {n k}same.

[0045] At step S109, the lines corresponding to the target path in the grayscale document image are eliminated. Specifically, the connected domains in the target area corresponding to the target path Po can be grouped according to their positions. The connected domain in the target region The corresponding area A in the grayscale document image Img k’ The pixel value of the pixel is set to the pixel value associated with the background pixel of the grayscale document image, for example, it is set to the mean (for example, arithmetic mean, geometric mean, weighted mean), median, etc. of the background pixels of the grayscale document image.

[0046] Regarding the determination of the directed graph in step S105, please refer to Figure 4 . Figure 4 A flowchart of a method 40 for determining a directed graph according to an embodiment of the present disclosure is shown. The method 40 is based on each strip region R n The connected domain CC in the region ni’ Determine the directed graph for the entire target image Imo, in other words, the method 40 is based on the connected domain set {CC i}Determine a directed graph for the entire target image Imo.

[0047] At step S401, nodes of a directed graph are determined based on the connected domains in the region. In one example, the nodes correspond to the connected domains in the region one by one. For example, the connected domain CC in the region i The corresponding node is n j , where i and j can be set according to the following rules, i=j.

[0048] At step S403, the edges of the directed graph are constructed based on the connected domains in the region. The edges can be constructed based on the centers of the connected domains in the region. i The coordinates of the center are (x i ,y i ). For example, the connected domain CC in the region i The coordinates of the center (x i ,y i ) can be ((xmax-xmin) / 2,(ymax-ymin) / 2) or (xav,yav), where xmax is CC i The maximum value of the horizontal coordinates of all pixels in CC i The minimum value of the horizontal coordinates of all pixels in CC i The maximum value of the ordinate of all pixels in CC i The minimum value of the vertical coordinates of all pixels in CC i The average value (e.g., arithmetic mean) of the horizontal coordinates of all pixels in CC i The average value (e.g., arithmetic mean) of the ordinates of all pixels in the region. i The method of determining the coordinates of the center is not limited to the above example. Considering that the nodes correspond to the connected domains in the region one by one, the coordinates of the center of the connected domain in the region can also be called the coordinates of the node. The coordinates of the center of the connected domain in the region are part of the information of the connected domain in the region, that is, the directed graph is related to the coordinates of the center of the connected domain in the region. More specifically, the nodes, edges, and edge costs of the directed graph are all related to the coordinates of the center of the connected domain in the region. Among them, when the strip area R n When the number of connected domains in the region is zero, the constructed directed graph does not contain the strip region R n any information.

[0049] The edges of a directed graph can be constructed according to predetermined conditions. For example, the edges of a directed graph can be constructed in the following way: i and n i’ If the following conditions of formula (1) are satisfied, then at node n i and n i’Construct a directed edge e between ii’ .

[0050] x i <x i′ AND x i′ <x i +H AND|y i′ -y i′ | <min(h i ,h i′ ) (1)

[0051] Where H is the height of the target image, h i CC is the connected domain in the region i Height, h i’ CC is the connected domain in the region i’ That is, the horizontal coordinates of the head node and the tail node of the edge are different; the difference between the horizontal coordinates of the tail node and the head node of the edge (that is, the horizontal distance between the tail node and the head node) is less than the height of the target image; the absolute value of the difference between the vertical coordinates of the tail node and the head node of the edge (that is, the vertical distance between the tail node and the head node) is less than each of the heights of the connected areas in the corresponding areas corresponding to the tail node and the head node. The above conditions are only examples. More generally, the edges of the directed graph can be constructed according to the orientation of the lines to be eliminated. The above conditions are preferably used in the case where the orientation of the lines to be eliminated is basically horizontal.

[0052] If it is known that the line to be eliminated is basically in the first direction, the edge can be constructed according to the following conditions: the coordinates of the centers of the connected domains in the corresponding regions of the two nodes connected by the edge in the first direction are different; the distance between the centers of the connected domains in the corresponding regions of the two nodes connected by the edge in the first direction is smaller than the size of the target image in the second direction; the absolute value of the difference in the coordinates of the centers of the connected domains in the corresponding regions of the two nodes connected by the edge in the second direction is smaller than each of the sizes of the corresponding connected domains in the second direction; wherein the first direction is perpendicular to the second direction.

[0053] In addition, if the line to be eliminated is in the vertical direction, the grayscale document image can be rotated 90 degrees, then binarized, a directed graph is constructed, the line is eliminated, and finally the document image with the eliminated line is reversely rotated 90 degrees. When this method is adopted, the conditions used for constructing the edges of the directed graph can use the conditions shown above.

[0054] In step S405 , the cost of the constructed edge of the directed graph is determined based on the connected domain within the region to determine the single-source shortest path of the directed graph.

[0055] For details on determining the cost in step S405, please refer to Figure 5 . Figure 5A flow chart of a method 50 of determining the cost of an edge according to one embodiment of the present disclosure is shown.

[0056] In step S501, a fitting straight line at the center of the connected domain in the region is determined. i The coordinates of the centers of the connected domains in each region in} are linearly fitted to obtain the straight lines closest to these centers. In one example, the fitted straight line is determined by a random sampling consensus (RANSAC: RANdom SAmple Consensus) method / algorithm. Linear fitting of multiple discrete points is a conventional technique and will not be described here. In a variant example, it can be determined whether there are lines to be eliminated in the target image based on the fitted straight line. For example, if the slope of the fitted straight line is less than a predetermined slope threshold, it is determined that there are lines to be eliminated, otherwise a prompt message is given and the image processing method is exited.

[0057] In step S503, the cost of the constructed edge is determined based on the fitted line. More specifically, the cost of the constructed edge can be determined based on the fitted line L f The cost is determined by the coordinates of the center of the connected domain in the region corresponding to the constructed edge. In an example of determining the cost, the initial cost can be first determined according to the position of the corresponding node and the fitting line (eg, formula (2)).

[0058] c ii′ =|x i′ -x i |+|y i′ -y i |+dis(CC i′ ,L f ) (2)

[0059] Among them, c ii’ is from node n i Points to node n i’ The constructed edge e ii’ The initial cost, (x i ,y i ) is node n i The corresponding connected domain CC i The coordinates of the center, (x i’ ,y i’ ) is node n i’ The corresponding connected domain CC i’ The coordinates of the center, dis(CC i’ ,L f ) is node n i’ The corresponding area is connected to the center of the fitting line L f distance.

[0060] Then, the initial cost is normalized as the cost of the constructed edge. An example of the normalization method may be the logarithmic function as shown in formula (3).

[0061] cn ii′ =log[2*(c ii′ -c min ) / (c max -c min )] (3)

[0062] Among them, cn ii’ is from node n i Points to node n i’ The constructed edge e ii’ The cost is negative, c min is the minimum initial cost among all the initial costs of constructed edges, c max is the maximum initial cost of all constructed edges.

[0063] A variation of method 10 is described below.

[0064] The lines to be removed in the document image that hinder character recognition may be stuck to the characters in the document image. An example of this phenomenon can be seen in Figure 2 : There is adhesion between character Ch2 and line L. This adhesion will cause the connected domain in the area corresponding to the determined single-source shortest path to include part of the character area. Figure 6 The shortest path connectivity domain of an example single-source shortest path P according to one embodiment of the present disclosure is shown. (The area colored as the darkest color in the figure, and the connected domains are not shown in the figure of the boundary), where Figure 6 Shown in the figure is an adjusted example target image using three types of coloring to exemplarily illustrate three different intra-regional connected domains. In addition to the shortest path connected domain (the darkest colored area) in the example target image exemplarily shown, other intra-regional connected domains are also shown, namely, the remaining connected domains, including the odd-numbered intra-regional connected domains in the remaining connected domains (colored in the middle depth color in the figure and marked as "odd-numbered CCs") and the even-numbered intra-regional connected domains (colored in the lightest color in the figure and marked as "even-numbered CCs"), where the number refers to the index of the strip area to which the intra-regional connected domain belongs. As shown in FIG. Figure 6 As shown in , the shortest path connected domain set of the single-source shortest path Including the partial character area of ​​character Ch2 (see the bottom of character Ch2). If such a shortest path connected domain set in the document image is directly eliminated If the pixels in the corresponding area are replaced by the average value of the background pixels (for example, the pixel values ​​in the corresponding area are replaced by the average value of the background pixels), the characters to be recognized will be destroyed, thereby reducing the performance of the text recognition engine. Therefore, it is preferred that the shortest path connected domain set of the single-source shortest path be removed. The connected domain in the adhesion region that is adhered to the characters in the target image.

[0065] therefore Figure 1 Step S107 in the above method may preferably include: determining a single-source shortest path of the directed graph based on the directed graph; removing the shortest path connected domain set of the single-source shortest path P according to a predetermined adhesion removal condition; The connected domains in the adhesion region that is adhered to the characters in the target image are used to determine the connected domain set in the target region. The process of removing the connected domain in the adhesion region may be referred to as removal process. In one example, the predetermined adhesion removal condition may be: the shortest path connected domain of the single-source shortest path Height h k The mean value h of the height of the connected domain in the region of the target image greater than Tht times av , Tht is a predetermined value. In one example, the predetermined adhesion removal condition may be: the shortest path connected domain of the single-source shortest path Height h k The median value h of the height of the connected domain in the region of the target image is greater than Tht times mi Tht is a predetermined threshold greater than zero, and its value can be determined empirically. For example, Tht can be 0.2. k If the predetermined adhesion removal condition is met, the shortest path connected domain set The connected domain in the corresponding area is removed, otherwise the connected domain in the corresponding area is retained. Figure 7 FIG. 1 shows a connected domain within a target region determined by removing the connected domain from the target region according to an embodiment of the present disclosure. The target areas are not shown. It can be seen that the previously attached part of the character area (see the bottom of character Ch2) is no longer in the connected domain set in the target area. Inside.

[0066] Figure 8 FIG. 1 shows a grayscale document image Img′ after removing the lines corresponding to the target path in the grayscale document image Img according to an embodiment of the present disclosure. The removal process may be, for example, to remove the lines corresponding to the target path in the grayscale document image Img. The area A corresponding to the connected domain in k’ The pixel value of the pixels in area A is set to the mean value of the background pixels. k’ is the image area in the grayscale document image Img, Ak’ According to location and Connected domains in the region One to one correspondence.

[0067] In addition, for the target image, the foreground consisting of characters may also be extracted as a line. Fig. 9 FIG. 1 shows a connected domain within a target region determined by removing the connected domain within the target region according to another embodiment of the present disclosure. Fig. 9 The connected domain set in the target area The darkest colored line obviously does not need to be eliminated, and this line can be called a "false positive line". Therefore, before performing the elimination process, it is preferred to determine whether the corresponding line corresponding to the connected domain set in the target area is a false positive line. If the determination result is "yes", no elimination process is performed, and if the determination result is "no", elimination process is performed. The number of segments and the number of adhesions of the corresponding line of the corresponding line determine whether the corresponding line is a false positive line. In one example, whether the corresponding line is a false positive line can be determined based on the following conditions:

[0068] Lenmax<H AND n_touching / n_segment> Th,

[0069] Among them, Lenmax is the length of the longest segment in the corresponding line segment, H is the height of the target image, n_touching is the total number of adhesions of the corresponding line, n_segment is the total number of segments of the corresponding line segment, and Th is a predetermined threshold. If the condition is met, the corresponding line is a false positive line, otherwise, the corresponding line is not a false positive line. In one example, Th can be 1.6. Fig. 9 In the example, n_segment=9, n_touching=15, and the result is "yes", so the elimination process will not be performed. For a segment, the maximum number of adhesions is 2. The number of adhesions of the segment is determined based on whether the left nearest neighbor pixel of the leftmost pixel of the segment and the right nearest neighbor pixel of the rightmost pixel of the segment are foreground pixels. When both are foreground pixels, the number of adhesions of the segment is 2. The sum of the number of adhesions of each segment is n_segment. Although the "false positive line determination" in this embodiment is performed after the removal process, it can also be performed before the removal process in a modified example, that is, determining the set of domains connected to the shortest path. Whether the corresponding corresponding line is a false positive line, if the determination result is "yes", no elimination processing is performed, and if the determination result is "no", removal processing and elimination processing are performed.

[0070] One aspect of the present disclosure provides an image processing device. Fig.10The device is described exemplarily.

[0071] Fig.10 An exemplary block diagram of an image processing device 100 according to an embodiment of the present disclosure is shown. The image processing device 100 includes a binarization unit 101, a division unit 103, a directed graph determination unit 105, a target path determination unit 107, and a line elimination unit 109. The binarization unit 101 is used to binarize a grayscale document image as a target image. The division unit 103 is used to obtain a plurality of strip regions arranged along a first direction by dividing the target image. The directed graph determination unit 105 is used to determine a directed graph for the entire target image based on the intra-region connected domains of the plurality of strip regions. The target path determination unit 107 is used to determine a target path related to a single-source shortest path of the directed graph based on the directed graph. The line elimination unit 109 is used to eliminate lines corresponding to the target path in the grayscale document image. For further description of each unit of the image processing device 100, reference may be made to the description of the image processing method in the present disclosure.

[0072] One aspect of the present disclosure provides an optical character recognition method. Fig.11 The flowchart of an optical character recognition method 11 according to an embodiment of the present disclosure is shown. The optical character recognition method 11 comprises: step S1101, preprocessing the image to be recognized; and step S1103, recognizing the text in the preprocessed image; wherein the preprocessing comprises the image processing method according to the present disclosure.

[0073] One aspect of the present disclosure provides an optical character recognition device. Fig.12 An exemplary block diagram of an optical character recognition device 12 according to an embodiment of the present disclosure is shown. The optical character recognition device 12 comprises: a preprocessing unit 1201 configured to preprocess an image to be recognized; and a recognition unit 1203 configured to recognize text in the preprocessed image; wherein the preprocessing comprises an image processing method according to the present disclosure.

[0074] One aspect of the present disclosure provides a computer-readable storage medium having a program stored thereon. The program enables an image processing method to be implemented when the program is executed by a processor, the image processing method comprising: binarizing a grayscale document image as a target image; obtaining a plurality of strip regions arranged along a first direction by dividing the target image; determining a directed graph for the entire target image based on the intra-region connected domains of the plurality of strip regions; determining a target path related to a single-source shortest path of the directed graph based on the directed graph; and eliminating lines corresponding to the target path in the grayscale document image.

[0075] One aspect of the present disclosure provides a computer-readable storage medium having a program stored thereon, wherein the program enables an optical character recognition method according to the present disclosure to be implemented when the program is executed by a processor.

[0076] According to one aspect of the present disclosure, an information processing device is also provided.

[0077] Fig.13 is an exemplary block diagram of an information processing device 13 according to one embodiment of the present disclosure. Fig.13 In the embodiment, a central processing unit (CPU) 1301 performs various processes according to a program stored in a read-only memory (ROM) 1302 or a program loaded from a storage section 1308 to a random access memory (RAM) 1303. In the RAM 1303, data and the like required when the CPU 1301 performs various processes are also stored as needed.

[0078] The CPU 1301, the ROM 1302, and the RAM 1303 are connected to one another via a bus 1304. An input / output interface 1305 is also connected to the bus 1304.

[0079] The following components are connected to the input / output interface 1305: an input section 1306 including a soft keyboard, etc.; an output section 1307 including a display such as a liquid crystal display (LCD), etc. and a speaker, etc.; a storage section 1308 such as a hard disk; and a communication section 1308 including a network interface card such as a LAN card, a modem, etc. The communication section 1308 performs communication processing via a network such as the Internet, a local area network, a mobile network, or a combination thereof.

[0080] A drive 1310 is also connected to the input / output interface 1305 as needed. A removable medium 1311 such as a semiconductor memory or the like is mounted on the drive 1310 as needed, so that a program read therefrom is installed to the storage section 1308 as needed.

[0081] The CPU 1301 may execute a program for realizing the image processing method or the optical character recognition method according to the present disclosure.

[0082] The method, apparatus, information processing device and storage medium disclosed herein can at least help achieve one of the following effects: eliminating row lines in a document image, eliminating column lines in a document image, eliminating noise lines in a document image, fast processing speed, less computing resources occupied, and improving the recognition performance of a text recognition engine.

[0083] Although the present invention has been disclosed above by describing the specific embodiments of the present invention, it should be understood that those skilled in the art can design various modifications (including, in the case of operation, the combination or replacement of features between the various embodiments), improvements or equivalents of the present invention within the spirit and scope of the appended claims. These modifications, improvements or equivalents should also be considered to be included in the scope of the present disclosure.

[0084] It should be emphasized that the term “include / comprises” when used herein refers to the presence of features, elements, steps or components, but does not exclude the presence or addition of one or more other features, elements, steps or components.

[0085] In addition, the methods of the embodiments of the present invention are not limited to being executed in the time sequence described in the specification or shown in the drawings, and may also be executed in other time sequences, in parallel or independently. Therefore, the execution order of the methods described in this specification does not limit the technical scope of the present invention.

[0086] Notes

[0087] 1. An image processing device, characterized in that the image processing device comprises:

[0088] A binarization unit, used for binarizing the grayscale document image as a target image;

[0089] A dividing unit, configured to obtain a plurality of strip-shaped areas arranged along a first direction by dividing the target image;

[0090] A directed graph determining unit, configured to determine a directed graph for the entire target image based on the intra-region connected domains of the plurality of strip-shaped regions;

[0091] a target path determining unit, configured to determine, based on the directed graph, a target path related to a single-source shortest path of the directed graph; and

[0092] A line elimination unit is used to eliminate the line corresponding to the target path in the grayscale document image.

[0093] 2. The image processing device according to Note 1, wherein the first direction is one of a horizontal direction and a vertical direction of the target image.

[0094] 3. An image processing device according to Note 2, wherein the length of each strip area is equal to one of the length or width of the target image.

[0095] 4. The image processing device according to Note 1, wherein each strip area is a rectangle.

[0096] 5. The image processing device according to Note 4, wherein the width of each strip area is equal to a predetermined value of two to ten pixel widths.

[0097] 6. The image processing device according to Supplement 1, wherein the nodes of the directed graph are set by selecting a connected area whose center is above the horizontal median line of the target image; or

[0098] Connected regions centered below the horizontal median line of the target image are selected to set the nodes of the directed graph.

[0099] 7. The image processing device according to Supplement 1, wherein the directed graph determination unit is configured to determine a plurality of nodes of the directed graph;

[0100] The multiple nodes correspond one-to-one to the intra-region connected domains of the multiple strip-shaped regions.

[0101] 8. The image processing device according to Note 7, wherein the directed graph determination unit is configured to construct edges of the directed graph.

[0102] 9. The image processing device according to Supplement 8, wherein the constructed edge satisfies the following conditions:

[0103] The horizontal coordinates of the head node and tail node of the edge are different;

[0104] The difference between the horizontal coordinates of the tail node and the head node of the edge is less than the height of the target image; and

[0105] The absolute value of the difference between the ordinates of the tail node and the head node of the edge is smaller than each of the heights of the connected regions in the corresponding regions corresponding to the tail node and the head node;

[0106] The coordinates of each node of the directed graph are the coordinates of the center of the connected domain in the corresponding area.

[0107] 10. The image processing device according to Supplement 8, wherein the directed graph determination unit is configured to determine the cost of the edge of the directed graph based on coordinates of two nodes of the edge.

[0108] 11. The image processing device according to supplementary note 10, wherein determining the cost of the edge based on the coordinates of two nodes of the edge of the directed graph comprises:

[0109] Determine a fitting straight line of a center of a connected domain in a corresponding region of a plurality of nodes of the directed graph.

[0110] 12. The image processing device according to Supplementary Note 11, wherein the fitting straight line is determined by a random sampling consensus algorithm.

[0111] 13. The image processing device according to Supplement 11, wherein the directed graph determination unit is configured to determine whether there is a line to be eliminated in the target image based on the slope of the fitting straight line.

[0112] 14. The image processing device according to Supplement 11, wherein the directed graph determination unit is configured to determine the cost of the edge of the directed graph based on coordinates of two nodes of the edge and the fitting straight line.

[0113] 15. An image processing device according to Note 14, wherein the cost of the edge is determined based on the absolute value of the difference between the horizontal coordinates of the two nodes of the edge, the absolute value of the difference between the vertical coordinates of the two nodes of the edge, and the distance between the tail node of the two nodes and the fitting straight line.

[0114] 16. The image processing device according to Note 15, wherein the cost is a negative value normalized using a logarithmic function.

[0115] 17. The image processing device according to Supplement 1, wherein determining the target path related to the single-source shortest path of the directed graph based on the directed graph comprises:

[0116] Adding a virtual node to the directed graph;

[0117] Constructing virtual edges from the virtual node to other nodes;

[0118] The cost of the virtual edge is set to zero.

[0119] 18. The image processing device according to supplementary note 17, wherein determining the target path related to the single-source shortest path of the directed graph based on the directed graph comprises:

[0120] The single-source shortest path of the directed graph is determined by the Bellman-Ford algorithm with the virtual node as the starting point.

[0121] 19. An image processing method, characterized in that the image processing method comprises:

[0122] Binarize the grayscale document image as the target image;

[0123] Obtaining a plurality of strip-shaped areas arranged along a first direction by dividing the target image;

[0124] Determine a directed graph for the entire target image based on the intra-region connected domains of the plurality of strip-shaped regions;

[0125] Determining, based on the directed graph, a target path associated with a single-source shortest path of the directed graph; and

[0126] Lines in the grayscale document image corresponding to the target path are eliminated.

[0127] 20. A computer-readable storage medium having a program stored thereon, wherein the program enables an image processing method to be implemented when the program is executed by a processor, the image processing method comprising:

[0128] Binarize the grayscale document image as the target image;

[0129] Obtaining a plurality of strip-shaped areas arranged along a first direction by dividing the target image;

[0130] Determine a directed graph for the entire target image based on the intra-region connected domains of the plurality of strip-shaped regions;

[0131] Determining, based on the directed graph, a target path associated with a single-source shortest path of the directed graph; and

[0132] Lines in the grayscale document image corresponding to the target path are eliminated.

Claims

1. An image processing device, characterized in that: The image processing device comprises: A binarization unit, used for binarizing the grayscale document image as a target image; A dividing unit, configured to obtain a plurality of strip-shaped areas arranged along a first direction by dividing the target image; A directed graph determining unit, configured to determine a directed graph for the entire target image based on the intra-region connected domains of the plurality of strip-shaped regions; a target path determining unit, configured to determine, based on the directed graph, a target path related to a single-source shortest path of the directed graph; and A line elimination unit, used for eliminating the line corresponding to the target path in the grayscale document image; Wherein, the directed graph determining unit is configured to determine a plurality of nodes of the directed graph; The plurality of nodes correspond one-to-one to the connected domains within the plurality of strip-shaped regions; and The directed graph determining unit is further configured to construct a plurality of edges of the directed graph based on the intra-region connected domains of the plurality of strip-shaped regions.

2. The image processing device according to claim 1, wherein: Selecting connected areas whose centers are above the horizontal median line of the target image to set nodes of the directed graph; or Connected regions centered below the horizontal median line of the target image are selected to set the nodes of the directed graph.

3. The image processing device according to claim 1, wherein: The first direction is the horizontal direction or the vertical direction of the target image.

4. The image processing device according to claim 3, wherein: The edges of the directed graph satisfy the following conditions: The horizontal coordinates of the head node and tail node of the edge are different; The difference between the horizontal coordinates of the tail node and the head node of the edge is less than the height of the target image; and The absolute value of the difference between the ordinates of the tail node and the head node of the edge is smaller than each of the heights of the connected regions in the corresponding regions corresponding to the tail node and the head node; The coordinates of each node of the directed graph are the coordinates of the center of the connected domain in the corresponding area.

5. The image processing device according to claim 4, wherein: The directed graph determination unit is configured to determine a cost of an edge of the directed graph based on coordinates of two nodes of the edge.

6. The image processing device according to claim 5, wherein: Determining the cost of the edge based on the coordinates of two nodes of the edge of the directed graph includes: Determine a fitting straight line of a center of a connected domain in a corresponding region of a plurality of nodes of the directed graph.

7. The image processing device according to claim 6, wherein: The directed graph determination unit is configured to determine a cost of an edge of the directed graph based on coordinates of two nodes of the edge and the fitting straight line.

8. The image processing device according to claim 1, wherein: Determining a target path related to a single-source shortest path of the directed graph based on the directed graph includes: Adding a virtual node to the directed graph; Constructing virtual edges from the virtual node to other nodes; The cost of the virtual edge is set to zero.

9. An image processing method, characterized in that: The image processing method comprises: Binarize the grayscale document image as the target image; Obtaining a plurality of strip-shaped areas arranged along a first direction by dividing the target image; Determine a directed graph for the entire target image based on the intra-region connected domains of the plurality of strip-shaped regions; Determining, based on the directed graph, a target path associated with a single-source shortest path of the directed graph; and Eliminating the line corresponding to the target path in the grayscale document image; Wherein, determining a directed graph for the entire target image includes determining a plurality of nodes of the directed graph; The plurality of nodes correspond one-to-one to the connected domains within the plurality of strip-shaped regions; and Determining a directed graph for the entire target image further includes constructing a plurality of edges of the directed graph based on the intra-region connected domains of the plurality of strip-shaped regions.

10. A computer-readable storage medium having a program stored thereon, characterized in that: The program enables an image processing method to be implemented when the program is executed by a processor, the image processing method comprising: Binarize the grayscale document image as the target image; Obtaining a plurality of strip-shaped areas arranged along a first direction by dividing the target image; Determine a directed graph for the entire target image based on the intra-region connected domains of the plurality of strip-shaped regions; Determining, based on the directed graph, a target path associated with a single-source shortest path of the directed graph; and Eliminating the line corresponding to the target path in the grayscale document image; Wherein, determining a directed graph for the entire target image includes determining a plurality of nodes of the directed graph; The plurality of nodes correspond one-to-one to the connected domains within the plurality of strip-shaped regions; and Determining a directed graph for the entire target image further includes constructing a plurality of edges of the directed graph based on the intra-region connected domains of the plurality of strip-shaped regions.

Images