Image processing device, image processing method and storage medium for removing lines
By using binarization and directed graph analysis to remove non-character identification lines in document images, the method enhances OCR performance by reducing noise and improving recognition accuracy.
Patent Information
- Application Number
- JP2021044842
- Authority / Receiving Office
- JP · JP
- Patent Type
- Patents
- Current Assignee / Owner
- Priority Date
- 2020-03-26
- Filing Date
- 2021-03-18
- Publication Date
- 2024-11-19
- Estimated Expiration
- 2041-03-18
AI Technical Summary
Optical Character Recognition (OCR) systems face performance degradation due to non-character identification lines such as overlines and underlines in document images, which are misinterpreted as noise, leading to erroneous foreground and background pixel determination and reduced character recognition accuracy.
A method involving binarization, segmentation, and directed graph analysis is employed to identify and remove lines in document images by constructing a directed graph based on intra-region connected components, determining a single-start shortest path, and removing lines corresponding to this path, thereby improving character recognition performance.
The method effectively removes non-character identification lines, enhancing OCR performance by reducing noise and computational resources while improving processing speed and recognition accuracy.
Smart Images

Figure 0007585906000031 
Figure 0007585906000032 
Figure 0007585906000033
Abstract
Description
[Technical field]
[0001] The present disclosure relates to image processing, and more particularly to an image processing device, an image processing method, and a storage medium for removing lines. [Background technology]
[0002] Optical Character Recognition (OCR) technology can be used to convert characters in an image into text. In optical character recognition applications, the input image needs to be preprocessed to improve the recognition performance of the OCR recognition engine. Generally, the more noise there is in the input image, the worse the recognition performance of the recognition engine. Preprocessing of the input image includes removing noise. Thus, the performance of the recognition engine can be improved by preprocessing.
[0003] For this reason, there are various conventional image processing methods for removing, for example, salt-and-pepper noise in an image. Summary of the Invention [Problem to be solved by the invention]
[0004] The following presents a simplified summary of the disclosure in order to provide a basic understanding of aspects of the disclosure. It is not intended to be an exhaustive overview of the disclosure, nor is it intended to intentionally identify key or critical portions of the disclosure or to intentionally limit the scope of the disclosure. However, it is merely intended to introduce concepts in a simplified form as a prelude to the more detailed description that is presented later.
[0005] According to the discovery of the inventors of the present invention, in OCR recognition, there may be non-character identification lines for identifying rows and / or columns in the vicinity of characters in an input document image. For example, many overlines and underlines exist in images of text lines in bank forms, insurance, and express mail. Usually, such lines degrade the performance of a recognition engine. In a recognition engine, such lines may be regarded as noise, and in this case, preprocessing must be performed on such lines. In addition, the recognition engine may erroneously determine foreground and background pixels in an input image, which may degrade the character recognition performance of the recognition engine. For example, if the recognition engine recognizes the above-mentioned identification lines as foreground pixels and performs character recognition, the character recognition performance of the recognition engine will be degraded. In view of this, the present disclosure provides a technique for removing lines in a document image based on a directed graph. [Means for solving the problem]
[0006] One aspect of the present disclosure provides an image processing method including the steps of binarizing a grayscale document image as a target image, dividing the target image to obtain a plurality of strip regions arranged along a first direction, determining a directed graph for the entire target image based on intra-region connected components of the plurality of strip regions, determining a target path associated with a single-source shortest path of the directed graph based on the directed graph, and removing a line corresponding to the target path in the grayscale document image.
[0007] Another aspect of the present disclosure provides an image processing device including: a binarization unit that binarizes a grayscale document image as a target image; a division unit that divides the target image to obtain a plurality of strip-like regions arranged along a first direction; a directed graph determination unit that determines a directed graph for the entire target image based on intra-region connected components of the plurality of strip-like regions; a target path determination unit that determines a target path associated with a single-source shortest path of the directed graph based on the directed graph; and a line removal unit that removes a line corresponding to the target path in the grayscale document image.
[0008] Another aspect of the present disclosure provides a computer-readable storage medium having a program stored thereon, which, when executed by a processor, realizes an image processing method including the steps of: binarizing a grayscale document image as a target image; dividing the target image to obtain a plurality of strip regions arranged along a first direction; determining a directed graph for the entire target image based on intra-region connected components of the plurality of strip regions; determining a target path associated with a single-source shortest path of the directed graph based on the directed graph; and removing a line corresponding to the target path in the grayscale document image.
[0009] According to the method, device, and storage medium of the present disclosure, at least one of the following effects can be achieved: removing row lines in a document image, removing column lines in a document image, removing noise lines in a document image, improving the processing speed, reducing the occupied computing resources, and improving the recognition performance of a character recognition engine. [Brief description of the drawings]
[0010] In order to make the above and other objects, features and advantages of the present disclosure more easily understandable, the following describes the embodiments of the present disclosure with reference to the drawings. It should be noted that the drawings are merely for illustrating the principles of the present disclosure. The drawings do not necessarily depict the size and relative positions of each part according to scale. The same reference numerals may denote the same features. [Figure 1] 1 is a flowchart illustrating an image processing method according to one embodiment of the present disclosure. [Diagram 2] FIG. 13 is a diagram showing an example of a target image. [Diagram 3] 13A and 13B are diagrams illustrating an example of a target image in which odd-numbered strip areas are hidden and a target image in which even-numbered strip areas are hidden; [Figure 4] 1 is a flowchart illustrating a method for determining a directed graph according to one embodiment of the present disclosure. [Diagram 5] 1 is a flowchart illustrating a method for determining edge costs according to one embodiment of the present disclosure. [Figure 6] FIG. 13 is a diagram illustrating an example of corresponding intra-region connected components of a single-source shortest path according to one embodiment of the present disclosure. [Figure 7] FIG. 13 is a diagram showing connected components within a target region determined by a removal process according to one embodiment of the present disclosure. [Figure 8] FIG. 2 illustrates a grayscale document image after removing lines corresponding to target paths in the grayscale document image according to one embodiment of the present disclosure. [Figure 9] FIG. 13 is a diagram showing connected components within a target region determined by a removal process according to another embodiment of the present disclosure. [Figure 10] FIG. 1 is an exemplary block diagram illustrating an image processing device according to one embodiment of the present disclosure. [Figure 11] 1 is a flow chart illustrating an optical character recognition method according to one embodiment of the present disclosure. [Figure 12] FIG. 1 is an exemplary block diagram illustrating an optical character recognition device according to one embodiment of the present disclosure. [Figure 13]1 is an exemplary block diagram illustrating an information processing device according to an embodiment of the present disclosure. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
[0011] The following describes exemplary embodiments of the present disclosure with reference to the drawings. For convenience of description, the specification does not show all the features of the actual embodiment. However, when a person skilled in the art realizes the embodiment, he or she may make certain decisions to realize the embodiment, and these decisions may be changed according to the embodiment.
[0012] It should be noted that, for clarity of the disclosure, the drawings only show device components and / or process steps closely related to the present disclosure, and omit details that are not relevant to the disclosure.
[0013] It should be noted that the present disclosure is not limited to the described embodiments, which will be described below with reference to the accompanying drawings. In this specification, where feasible, the embodiments may be combined with each other, features of different embodiments may be substituted or utilized, and one or more features may be omitted in one embodiment.
[0014] It should be noted that each aspect of the exemplary embodiment may be implemented as a system, a method, or a computer program product. Thus, each aspect of the exemplary embodiment may be specifically realized in the following forms: an entirely hardware embodiment, an entirely software embodiment (including firmware, resident software, microcode, etc.), or an embodiment of a combination of software and hardware, which may be generally referred to herein as a "circuit," "module," or "system." Furthermore, each aspect of the exemplary embodiment may take the form of a computer program product represented by one or more computer-readable mediums having computer-readable program code recorded thereon. The computer program may, for example, be distributed over a network of computers, located on one or more remote servers, or embedded in the memory of the device.
[0015] Any combination of one or more computer readable media may be used. The computer readable medium may be a computer readable signal medium or a computer readable storage medium. The computer readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or any suitable combination thereof. More specific examples (non-exhaustive list) of computer readable storage media include one or more electrical connections of wires, portable computer disks, hard disks, random access memory (RAM), read only memory (ROM), erasable programmable read only memory (EPROM or flash memory), optical fiber, portable compact disk read only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this specification, a computer readable storage medium may be any tangible medium that contains or stores a program for use by or in connection with an instruction execution system, device, or device.
[0016] The computer-readable signal medium may include a data signal carrying computer-readable program code, for example in baseband or propagated as part of a carrier. Such a propagated signal may take any suitable form, including, for example, but not limited to, electromagnetic, optical, or any suitable combination thereof.
[0017] A computer-readable signal medium may be any computer-readable medium, other than a computer-readable storage medium, that can transmit, propagate or transmit a program for use by or in connection with an instruction execution system, apparatus or device.
[0018] The program code in the computer readable medium may be transmitted using any suitable medium, including, but not limited to, wireless, wired, optical cable, radio frequency, or the like, or any suitable combination thereof.
[0019] Computer program code for carrying out operations of each aspect of the exemplary embodiments disclosed herein may be written in any combination of one or more programming languages, including object-oriented programming languages such as Java, Smalltalk, C++, and conventional procedural programming languages, such as the "C" programming language or similar programming languages.
[0020] The following describes each aspect of the exemplary embodiment disclosed herein with reference to flowcharts and / or block diagrams of a method, an apparatus (system) and a computer program product according to the exemplary embodiment. Note that each block of the flowcharts and / or block diagrams, and combinations of each block of the flowcharts and / or block diagrams, may be realized by computer program instructions. These computer program instructions are provided to a processor of a general purpose computer, a special purpose computer or other programmable data processing device to configure an apparatus, and execution of these instructions by the computer or other programmable data processing device configures the apparatus to realize the functions / operations defined in each block of the flowcharts and / or block diagrams.
[0021] These computer program instructions may be stored on a computer-readable medium that operates in a specific manner on a computer or other programmable data processing apparatus, and may constitute a product including instructions that implement the functions / operations defined in each block of the flowcharts and / or block diagrams with the instructions stored on the computer-readable medium.
[0022] The computer program instructions may be loaded into a computer or other programmable data processing apparatus and a series of operational steps may be executed on the computer or other programmable data processing apparatus to provide a process in which the instructions executed on the computer or other programming apparatus implement the functions / operations defined in each block of the flowcharts and / or block diagrams.
[0023] In one aspect of the present disclosure, an image processing method for processing a document image is provided. The processed document image may be used for character recognition. The method may be part of the pre-processing performed on an input image in an optical character recognition application. The method may be used to remove noise lines in an image. The lines may be row lines for identifying rows, column lines for identifying columns, or other character identification lines. The lines may be solid lines, non-continuous lines, or other lines with a defined overall direction. Non-continuous lines include dashed and dotted lines. The document image may be a color image. Since a color document image may be converted into a grayscale image in a conventional manner in text recognition, the following describes the image processing method by taking the processing of a grayscale document image as an example. The following describes the method exemplarily with reference to FIG. 1.
[0024] 1 is a flow chart illustrating an image processing method 10 according to one embodiment of the present disclosure. The input of the method includes a grayscale document image. The output of the method includes the grayscale document image from which at least some non-character identifying lines have been removed.
[0025] In step S101, the grayscale document image Img is binarized as a target image Imo. This step may be realized, for example, by using the Sauvola method. The binarization can remove the background of the image. FIG. 2 is a diagram showing an example of a target image Imo. The target image Imo in FIG. 2 includes a line L, a character Ch1, and a character Ch2. The corresponding grayscale document image Img includes a line l corresponding to the line L. The purpose of the image processing method 10 includes removing the line l in the grayscale document image Img. Note that the symbol (e.g., "L") and the S-shaped indicator line in FIG. 2 are shown only to explain the elements included in the exemplary target image Imo of the target image, and the exemplary target image Imo itself does not include the symbol and the S-shaped indicator line. Although the characters in the exemplary target image Imo are handwritten, the characters in the target image are not limited thereto and may include printed characters or both printed characters and handwritten characters. Alternatively, the method 10 may be performed after dividing the document image of the character line into two document images along the center line, so as to remove the lower line and the upper line of the document image of the character line, respectively. If the input image to be processed is already binarized, the binarization step may be omitted. Although FIG. 2 shows one line L along the horizontal direction and the image processing method is illustratively described by taking the removal of the line l corresponding to the line L as an example, the method of removing multiple lines and the method of removing one line in other directions are similar to the method of removing one line along the horizontal direction. Therefore, a person skilled in the art may apply the method disclosed herein to the method of removing multiple lines and / or the method of removing lines in other directions based on the method of removing one line along the horizontal direction.
[0026] In step S103, the target image Imo is divided into a plurality of strip-shaped regions R1, R2, . . . , R n , … , R NThe first direction may be horizontal (horizontal direction of the target image), vertical (vertical direction of the target image), or other direction. The length of each strip may be equal to either the length or width of the target image. For example, in one exemplary division direction, the target image Imo with a length Len is divided into N rectangular regions along the horizontal direction, the length of each rectangular region is equal to the width w of the target image Imo, and the width of each rectangular region is Len / N. In order to quickly and accurately detect the lines to be removed, the width of each strip may be equal to a predetermined value, for example, 2 to 10 pixels wide. In this division method, the odd-numbered strips or the even-numbered strips are hidden, and the target image Imo becomes an image having a dashed line. These strips are sequentially merged to obtain the target image Imo. FIG. 3 is a diagram showing an exemplary target image with the odd-numbered strips hidden and the even-numbered strips hidden. Here, the image shown in FIG. 3 is slightly enlarged or reduced with respect to the image in FIG. 2, the first direction is the horizontal direction, and FIG. 3(a) is a dashed target image Imo with the even-numbered strips hidden. o Figure 3(b) shows the target image Imo with the odd-numbered strips hidden. e In the present disclosure, the widths of the multiple strip regions may or may not be equal.
[0027] In step S105, a plurality of strip regions R n The connected component CC in the region ni’ Based on the above, we determine a directed graph for the entire target image, where i' is an index for distinguishing different connected components, i' = 1, 2, ..., I'. I' is the area of the strip R n In this disclosure, the connected components in a strip are also referred to as intra-region connected components. When it is not necessary to distinguish the strip to which each intra-region connected component belongs, the "intra-region connected component" is referred to as "CC" or "CC i" where i = 1, 2, ..., I. I is the total number of regional connected components in the target image Imo. The number of regional connected components in each regional strip may be 0, 1, 2, or more. The regional connected components of all the regional strips of the target image are grouped into a regional connected component set {CC i} is constructed for each connected component CC ni’ is composed of the foreground pixels of the target image, and each connected component CC ni’ is a single strip of region R n 3(a) and 3(b). A particular foreground line in the target image before segmentation constitutes an initial connected component passing through multiple strips, but in the target image after segmentation, since the boundaries of different strips are distinguished, the line is divided into multiple sections corresponding to multiple intra-region connected components, and the initial connected component is considered to be composed of multiple intra-region connected components accordingly. In other words, each strip is considered as a sub-image, and the connected components in each corresponding sub-image are intra-region connected components. For example, if one connected component in the target image spans two adjacent strips, the two connected components in the two adjacent strips are considered to be two intra-region connected components. The connected components shown in Figures 3(a) and 3(b) are intra-region connected components, and all the intra-region connected components in the two figures are included in the intra-region connected component set {CC i}.
[0028] A directed graph is a set of n nodes. j , multiple edges e jj’ where j is an index to distinguish different nodes, and e jj’ is node n j and node n j’ represents an edge connecting node n and j (First node) to node n j’ (tail node). Each edge has a cost cn jj’ These nodes, edges and costs are stored in the region connected component set {CC i} and contains information on connected components in the region. The edges here are edges of a directed graph, so they are directed edges.
[0029] If the grayscale document image has both bottom and top row lines, in one example, a connected component whose center is located above the horizontal midline of the target image may be selected to set a node of the directed graph to remove the top row lines, or a connected component whose center is located below the horizontal midline of the target image may be selected to set a node of the directed graph to remove the bottom row lines.
[0030] In step S107, a target path Po associated with a single source shortest path (SSSP) P (also referred to as "shortest path") of the directed graph is determined based on the directed graph. In one example, this step may include adding a virtual node to the directed graph, constructing virtual edges from the virtual node to other nodes, and setting the cost of the virtual edges to zero. After the single source shortest path P of the directed graph is determined, a target path Po may be determined based on the determined single source shortest path. In one example, the single source shortest path may be directly taken as the target path.
[0031] In one example, a single-source shortest path is determined for a directed graph starting from a virtual node, for example, by the Bellman-Ford method / algorithm. A single-source shortest path is a path that starts from a virtual node starting from a virtual node. k where k is an index for distinguishing each node, k = 1, 2, ..., K. These nodes are classified into a node set {n k} and configure node n k is the connected component in the region (outside 1) For example, a single-source shortest path corresponds to multiple intra-region connected components. These corresponding intra-region connected components may be called shortest-path connected components, and these shortest-path connected components are called a shortest-path connected component set. (outside 2) Similar to the shortest path P, the target path P also has multiple corresponding nodes nk’ (hereafter referred to as the target node) and the connected components in the region (Outside 3) TIFF0007585906000003.tif13153 (hereinafter referred to as the connected component within the target region), and the target node n k’ The connected components in the region corresponding to (outside 4) TIFF0007585906000004.tif13153. These corresponding nodes are in the target node set {n k’ These corresponding connected components in the region form a target connected component set. (outside 5) Construct TIFF0007585906000005.tif14153. More specifically, target node n k’ is the node set of the shortest path {n k}. In one example, the target node set {n k’} is the node set of the shortest path {n k}.
[0032] In step S109, the lines corresponding to the target path in the grayscale document image are removed. Specifically, the connected component set in the target region of the target path Po is (outside 6) Connected components within the target region in TIFF0007585906000006.tif12153 (outer 7) Area A in the grayscale document image Img corresponding to TIFF0007585906000007.tif13153 k’ may be set to a pixel value associated with the background pixels of the grayscale document image, such as to the average value (e.g., arithmetic mean, geometric mean, weighted mean), median, etc., of the background pixels of the grayscale document image.
[0033] For the determination of the directed graph in step S105, reference may be made to Fig. 4. Fig. 4 is a flow chart illustrating a method 40 for determining a directed graph according to one embodiment of the present disclosure. The method 40 includes: n Connected components CC in the region ni’ In other words, the method 40 determines a directed graph for the entire target image Imo based on the connected components set {CC i}, a directed graph for the entire target image Imo is determined.
[0034] In step S401, a node of a directed graph is determined based on the connected components in the region. In one example, the node corresponds to the connected components in the region in one-to-one correspondence. For example, the connected component CC i The corresponding node of is n j where i and j may be set according to the rule i=j.
[0035] In step S403, edges of a directed graph are constructed based on the connected components in the region. The edges may be constructed based on the centers of the connected components in the region. i The center coordinate of is (x i ,y i ) For example, the connected component CC i The center coordinates (x i ,y i ) can be ((xmax-xmin) / 2,(ymax-ymin) / 2) or (xav,yav), where xmax is the CC i is the maximum value of the abscissa of all pixels in CC, and xmin is the i is the minimum value of the abscissa of all pixels in CC i is the maximum ordinate of all pixels in CC, and ymin is the i is the minimum value of the ordinates of all pixels in CC, and xav is the i is the average (e.g., arithmetic mean) of the abscissas of all pixels in i The average (e.g., arithmetic mean) of the ordinates of all pixels in the connected component CC iThe method of determining the coordinates of the center of the connected component in the region is not limited to the above example. Considering that the nodes correspond one-to-one to the connected components in the region, the coordinates of the center of the connected component in the region may be called the coordinates of the node. The coordinates of the center of the connected component in the region are part of the information of the connected component in the region, that is, the directed graph is related to the coordinates of the center of the connected component in the region. More specifically, the nodes, edges, and edge costs of the directed graph are all related to the coordinates of the center of the connected component in the region. Here, the coordinates of the center of the connected component in the region of the strip R n If the number of intra-region connected components in is zero, the constructed directed graph is n Does not include information on.
[0036] The edges of the directed graph may be constructed according to a predetermined condition. For example, the edges of the directed graph may be constructed in the following manner: i and i’ If node n satisfies the following condition in equation (1), i and i’ There is a directed edge e between ii’ Build.
number
[0037] where H is the height of the target image and h i is the connected component in the region CC i is the height of h i’ is the connected component in the region CC i’ That is, the abscissa of the head node of the edge is different from the abscissa of the tail node, the difference between the abscissa of the head node of the edge and the abscissa of the head node of the edge (i.e., the horizontal distance between the head node and the tail node) is smaller than the height of the target image, and the absolute value of the difference between the ordinate of the head node of the edge and the abscissa of the head node of the edge (i.e., the vertical distance between the head node and the tail node) is smaller than both of the heights of the corresponding in-region connected components corresponding to the head node and the tail node. The above conditions are merely examples, and more generally, the edges of the directed graph may be constructed based on the direction of the line to be removed. Preferably, the above conditions may be used when the direction of the line to be removed is substantially horizontal.
[0038] If the line to be removed is essentially in a first direction, the edge may be constructed according to the following conditions: the first-direction coordinates of the centers of the in-region connected components corresponding to the two nodes connected by the edge are different, the first-direction distance between the centers of the in-region connected components corresponding to the two nodes connected by the edge is smaller than the second-direction size of the target image, and the absolute value of the difference in the second-direction coordinates of the centers of the in-region connected components corresponding to the two nodes connected by the edge is smaller than any of the second-direction sizes of the corresponding connected components, where the first direction is perpendicular to the second direction.
[0039] Also, if the lines to be removed are vertical, the grayscale document image may be rotated by 90°, binarized, a directed graph constructed, the lines removed, and finally the document image from which the lines have been removed may be rotated by 90° in the opposite direction. In this way, the conditions used to construct the edges of the directed graph may be the same as those described above.
[0040] In step S405, costs of constructed edges of the directed graph are determined based on the intra-region connected components to determine a single-source shortest path of the directed graph.
[0041] For determining the cost in step S405, reference may be made to Fig. 5. Fig. 5 is a flow chart illustrating a method 50 for determining the cost of an edge according to one embodiment of the present disclosure.
[0042] In step S501, a fitting line of the center of the connected components in the region is determined. iBased on the coordinates of the centers of each connected component in the region in { 1 , 2 , 3 , 4 , 5 , 6 , 7 , 8 , 9 , 10 , 11 , 12 , 13 , 14 , 15 , 16 , 17 , 18 , 19 , 20 , 21 , 22 , 23 , 24 , 25 , 26 , 27 , 28 , 29 , 30 , 31 , 32 , 33 , 34 , 35 , 36 , 37 , 38 , 39 , 40 , 41 , 42 , 43 , 44 , 45 , 46 , 47 , 48 , 49 , 50 , 51 , 52 , 53 , 54 , 55 , 56 , 57 , 58 , 69 , 70 , 71 , 72 , 73 , 74 , 75 , 76 , 77 , 78 , 79 , 80 , 81 , 82 , 83 , 84 , 85 , 86 , 87 , 88 , 89 , 90 , 91 , 92 , 93 , 94 , 95 , 96 , 97 , 98 , 99 , 100 , 101 , 102 , 103 , 104 , 105 , 106 , 107 , 108 , 109
[0043] In step S503, the cost of the constructed edge is determined based on the fitting line L. More specifically, f and the cost may be determined based on the coordinates of the centers of the connected components in the region corresponding to the constructed edges. In one example of cost determination, an initial cost may be determined first based on the positions of the corresponding nodes and the fitting line (e.g., equation (2)).
number
[0044] where c ii’ is node n i to node n i’ Constructed edge e to ii’ is the initial cost of (x i ,y i ) is node n i The connected component CC corresponding to the region i are the coordinates of the center of (x i’ ,y i’ ) is the connected component CC corresponding to the node ni' i’ are the coordinates of the center of the i’ ,L f ) is node n i’ A fitting straight line L from the center of the connected component in the region corresponding to f The distance is .
[0045] The initial cost is then normalized to become the cost of the constructed edge. An example of the normalization method may be a logarithmic function as shown in equation (3).
number
[0046] where cn ii’ is node n i to node n i’ Constructed edge e to ii’ is the cost of c min is the minimum initial cost of all constructed edges, and c max is the maximum initial cost of all constructed edges.
[0047] The following describes a variation of method 10.
[0048] It is possible that a line to be removed that interferes with character recognition in a document image is attached to a character in the document image. As an example of this phenomenon, the character Ch2 and the line L are attached to each other as shown in FIG. 2. Due to such attachment, a part of the character region is included in the intra-region connected component corresponding to the determined single-source shortest path. FIG. 6 shows the shortest path connected components of the single-source shortest path P according to one embodiment of the present disclosure. (outside 8) TIFF0007585906000011.tif15153 (In Figure 6, the darkest areas are colored, and each connected component (outer 9) 6 shows three different intra-region connected components for a target image adjusted by coloring with three different colors. In addition to the shortest-path connected components (areas colored with the darkest color) in the exemplarily shown target image, other intra-region connected components, i.e., remaining connected components, are shown. The remaining connected components include the odd-numbered intra-region connected components (colored with a medium dark color and indicated by "odd-numbered CC" in FIG. 6) and the even-numbered intra-region connected components (colored with the lightest color and indicated by "even-numbered CC" in FIG. 6), where the numbers refer to the indices of the strips to which the intra-region connected components belong. As shown in FIG. 6, the shortest-path connected components set of a single-source shortest path (Outside 10) TIFF0007585906000013.tif16153 contains a part of the character region of character Ch2 (see the lower part of character Ch2). (Outside 11) If the pixels in the region corresponding to TIFF0007585906000014.tif14153 were directly removed (e.g., the pixel values in the corresponding region were replaced with the average value of the background pixels), the characters to be recognized would be destroyed, and the performance of the text recognition engine would be degraded. Therefore, it is preferable to use the shortest-path connected component set of the single-source shortest path. (Outside 12) Connected components within the attached regions attached to the characters in the target image in TIFF0007585906000015.tif16153 may be removed.
[0049] Therefore, step S107 of FIG. 1 preferably includes the steps of: determining a single-source shortest path in the directed graph based on the directed graph; A set of shortest-path connected components of a single-source shortest path P subject to a given sticking removal condition. (Outside 13) The connected components in the region attached to the characters in the target image (TIFF0007585906000016.tif17153) are removed, and the connected components in the target region are (Outside 14) and determining TIFF0007585906000017.tif14153. The process of removing connected components in the sticking region may be abbreviated as a removal process. In one example, the predetermined sticking removal condition is that the shortest path connected components of a single-source shortest path (Outside 15) TIFF0007585906000018.tif15153 height h k is the average height of the connected components in the target image, h av In one example, the predetermined sticking removal condition may be greater than Tht times the number of shortest-path connected components of the single-source shortest path. (Outside 16) Height h of TIFF0007585906000019.tif15153 k is the median height of the connected components in the target image, h mi Tht may be greater than Tht times the threshold value of h. Tht is a predetermined threshold value greater than zero, the magnitude of which may be determined empirically, for example, Tht may be 0.2. k If satisfies a given sticking removal condition, then the shortest path connected component set (Outside 17) 7 shows the target connected components in the region determined by the removal process according to one embodiment of the present disclosure. (Outside 18) FIG. 15 shows TIFF0007585906000021.tif15153. Here, each target region connected component (Outside 19) The boundary between TIFF0007585906000022.tif13153 is not shown. As can be seen from Figure 7, a part of the above-mentioned connected character region (see the bottom of character Ch2) is included in the target region connected component set. (outside 20) Does not exist in TIFF0007585906000023.tif13153.
[0050] 8 is a diagram illustrating a grayscale document image Img′ after removing lines corresponding to target paths in the grayscale document image Img according to one embodiment of the present disclosure. The removal process may be performed, for example, by removing connected components within the target region from the grayscale document image Img. (outside 21) Region A corresponds to the connected component in TIFF0007585906000024.tif15153 k’ The pixel value of the pixel in region A may be set to the average value of the background pixels. k’ is the image area in the grayscale document image Img, and A k’ is the position (outside 22) Connected components in the region of TIFF0007585906000025.tif13153 (outside 23) It has a one-to-one correspondence with TIFF0007585906000026.tif15153.
[0051] In addition, for the target image, a foreground composed of characters may be extracted as one line. FIG. 9 shows a connected component in a target region determined by the removal process according to another embodiment of the present disclosure. (outside 24) A diagram showing TIFF0007585906000027.tif14153. A set of connected components in the target region in FIG. (Outside 25) The darkest line composed of TIFF0007585906000028.tif13153 obviously does not need to be removed, and this line may be called a "false-positive line". Therefore, before performing the removal process, it is preferable to determine whether the corresponding line corresponding to the connected component set in the target region is a false-positive line, and if the determination result is YES, the removal process is not performed, and if the determination result is NO, the removal process is performed. (outside 26) Whether the corresponding line is a false positive line may be determined based on the number of segments and the number of connections of the corresponding line in TIFF0007585906000029.tif15153. In one example, whether the corresponding line is a false positive line may be determined based on the following conditions:
[0052] Lenmax<H AND n_touching / n_segment> Th Here, Lenmax is the length of the longest segment of the corresponding line segments, H is the height of the target image, n_touching is the total number of touches of the corresponding line, and n_segment is the total number of segments of the corresponding line. Th is a predetermined threshold value, and if this condition is met, the corresponding line is a false positive line, and if not, the corresponding line is not a false positive line. For example, Th may be 1.6. In FIG. 9, n_segment=9, n_touching=15, and the determination result is YES, so the removal process is not performed. For one segment, the maximum number of touches is 2, and the number of touches of the segment is determined based on whether the nearest adjacent pixel on the left side of the pixel at the left end of the segment and the nearest adjacent pixel on the right side of the pixel at the right end of the segment are foreground pixels, and if both are foreground images, the number of touches of the segment is 2. The sum of the number of touches of each segment is n_segment. In this embodiment, the "determination of false positive lines" is performed after the removal process, but in one variant, it may be performed before the removal process. (outside 27) It is determined whether the corresponding line corresponding to TIFF0007585906000030.tif14153 is a false positive line, and if the determination result is YES, the removal process is not performed, and if the determination result is NO, the removal process is performed.
[0053] In one aspect of the present disclosure, an image processing device is provided, which is exemplarily described below with reference to FIG.
[0054] FIG. 10 is an exemplary block diagram showing an image processing device 100 according to an embodiment of the present disclosure. The image processing device 100 includes a binarization unit 101, a division unit 103, a directed graph determination unit 105, a target path determination unit 107, and a line removal unit 109. The binarization unit 101 binarizes a grayscale document image as a target image. The division unit 103 divides the target image to obtain a plurality of strip-shaped regions arranged along a first direction. The directed graph determination unit 105 determines a directed graph for the entire target image based on the intra-region connected components of the plurality of strip-shaped regions. The target path determination unit 107 determines a target path associated with a single-source shortest path of the directed graph based on the directed graph. The line removal unit 109 removes a line corresponding to the target path in the grayscale document image. For each unit of the image processing device 100, the description of the image processing method according to the present disclosure may be referred to.
[0055] In one aspect of the present disclosure, an optical character recognition method is provided. FIG. 11 is a flow chart showing an optical character recognition method 11 according to one embodiment of the present disclosure. The optical character recognition method 11 includes a step of performing pre-processing on an image to be recognized (step S1101), and a step of recognizing characters in the image after the pre-processing (step S1103). Here, the pre-processing includes an image processing method according to the present disclosure.
[0056] In one embodiment of the present disclosure, an optical character recognition device is provided. Fig. 12 is an exemplary block diagram showing an optical character recognition device 12 according to one embodiment of the present disclosure. The optical character recognition device 12 includes a pre-processing unit 1201 that performs pre-processing on an image to be recognized, and a recognition unit 1203 that recognizes characters in the image after the pre-processing. Here, the pre-processing includes an image processing method according to the present disclosure.
[0057] In one aspect of the present disclosure, a computer-readable storage medium having a program stored thereon is provided, which, when executed by a processor, realizes an image processing method including: binarizing a grayscale document image as a target image, dividing the target image to obtain a plurality of strip regions arranged along a first direction, determining a directed graph for the entire target image based on intra-region connected components of the plurality of strip regions, determining a target path associated with a single-source shortest path of the directed graph based on the directed graph, and removing a line corresponding to the target path in the grayscale document image.
[0058] One aspect of the present disclosure provides a computer-readable storage medium having stored thereon a program that, when executed by a processor, implements the optical character recognition method according to the present disclosure.
[0059] One aspect of the present disclosure provides an information processing device.
[0060] Fig. 13 is an exemplary block diagram showing an information processing device 13 according to an embodiment of the present disclosure. In Fig. 13, a central processing unit (CPU) 1301 executes various processes according to a program stored in a read-only memory (ROM) 1302 or a program loaded from a storage unit 1308 to a random access memory (RAM) 1303. The RAM 1303 stores data necessary for the CPU 1301 to execute various processes as necessary.
[0061] The CPU 1301, the ROM 1302, and the RAM 1303 are connected to one another via a bus 1304. An input / output interface 1305 is also connected to the bus 1304.
[0062] An input unit 1306 (including a keyboard, a mouse, etc.), an output unit 1307 (including a display, for example, a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.), a storage unit 1308 (including, for example, a hard disk, etc.), and a communication unit 1309 (including a network interface card, for example, a LAN card, a modem, etc.) are connected to the input / output interface 1305. The communication unit 1309 executes communication processing via a network, for example, the Internet.
[0063] If necessary, the driver 1310 may be connected to the input / output interface 1305. The removable medium 1311 is, for example, a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, or the like, and is set up in the driver 1310 as necessary, and a computer program read from the removable medium 1311 is installed in the storage unit 1308 as necessary.
[0064] The CPU 1301 may execute a program for implementing an image processing method or an optical character recognition method according to the present disclosure.
[0065] According to the method, device, and storage medium of the present disclosure, at least one of the following effects can be achieved: removing row lines in a document image, removing column lines in a document image, removing noise lines in a document image, improving the processing speed, reducing the occupied computing resources, and improving the recognition performance of a character recognition engine.
[0066] Although the above describes the specific embodiments of the present disclosure, those skilled in the art may make various modifications (combining or replacing the features of each embodiment in the case of lines), improvements or equivalents to the present disclosure within the spirit and scope of the appended claims. These modifications, improvements or equivalents belong to the protection scope of the present disclosure.
[0067] It should be noted that the terms "including" and "having" refer to the presence of features, elements, steps or components described in this specification, but do not exclude the presence or addition of one or more other features, elements, steps or components.
[0068] Furthermore, the methods of the embodiments of the present invention are not limited to being performed in the order of time described in the specification or shown in the drawings, and may be performed in other orders of time, or may be performed in parallel or independently. Therefore, the order of performing the methods described herein does not limit the technical scope of the present invention.
[0069] Furthermore, the following supplementary notes are disclosed regarding the embodiments including the above-mentioned examples, but the present invention is not limited to these supplementary notes. (Appendix 1) a binarization unit for binarizing a grayscale document image as a target image; a division unit that divides the target image to obtain a plurality of strip-shaped regions arranged along a first direction; a directed graph determination unit that determines a directed graph for the entire target image based on the intra-region connected components of the plurality of strip-shaped regions; a target path determiner that determines, based on the directed graph, a target path associated with a single-source shortest path of the directed graph; a line remover configured to remove a line corresponding to the target path in the grayscale document image. (Appendix 2) 2. The image processing device according to claim 1, wherein the first direction is either a horizontal direction or a vertical direction of the target image. (Appendix 3) 3. An image processing device as described in claim 2, wherein the length of each strip is equal to either the length or the width of the target image. (Appendix 4) 2. The image processing device of claim 1, wherein each strip is rectangular. (Appendix 5) 5. The image processing device according to claim 4, wherein the width of each strip is equal to a predetermined value between 2 and 10 pixels. (Appendix 6) selecting connected components whose centers lie above the horizontal midline of the target image to set nodes of the directed graph; or 2. The image processing device of claim 1, further comprising: selecting connected components whose centers are located below a horizontal midline of the target image to set nodes of the directed graph. (Appendix 7) the directed graph determination unit determines a plurality of nodes of the directed graph; 2. The image processing device according to claim 1, wherein the plurality of nodes correspond one-to-one to intra-region connected components of the plurality of strip-shaped regions. (Appendix 8) 8. The image processing device according to claim 7, wherein the directed graph determination unit constructs edges of the directed graph. (Appendix 9) The constructed edges are the abscissa of the first node of the edge is different from the abscissa of the last node of the edge, The difference in abscissa between the end node and the start node of the edge is smaller than the height of the target image; and the absolute value of the difference between the ordinates of the end node and the start node of the side is smaller than both the heights of the intra-area connected components corresponding to the end node and the start node; The image processing device according to claim 8, wherein the coordinates of each node in the directed graph are the coordinates of the center of a corresponding intra-region connected component. (Appendix 10) 9. The image processing device according to claim 8, wherein the directed graph determination unit determines a cost of an edge of the directed graph based on coordinates of two nodes of the edge. (Appendix 11) The image processing device according to claim 10, wherein the directed graph determination unit determines fitting lines of centers of intra-region connected components corresponding to a plurality of nodes of the directed graph. (Appendix 12) 12. The image processing device of claim 11, wherein the fitting line is determined by a random sample consensus (RANSAC) algorithm. (Appendix 13) 12. The image processing device according to claim 11, wherein the directed graph determination unit determines whether or not there is a line to be removed in the target image based on a slope of a fitting line. (Appendix 14) 12. The image processing device according to claim 11, wherein the directed graph determination unit determines a cost of the edge based on coordinates of two nodes of the edge of the directed graph and the fitting line. (Appendix 15) The image processing device of claim 14, further comprising: determining a cost of the edge based on an absolute value of a difference between the horizontal coordinates of the two nodes of the edge, an absolute value of a difference between the vertical coordinates of the two nodes of the edge, and a distance between the last node of the two nodes and the fitting line. (Appendix 16) 16. The image processing device of claim 15, wherein the cost is a negative value normalized using a logarithmic function. (Appendix 17) The target path determination unit is adding a virtual node to the directed graph; constructing virtual edges from the virtual node to other nodes; 2. The image processing device of claim 1, further comprising: setting a cost of the virtual edge to zero. (Appendix 18) The image processing device according to claim 17, wherein the target path determination unit determines a single-start shortest path of the directed graph using a Bellman-Ford algorithm when the virtual node is a starting point. (Appendix 19) binarizing a grayscale document image as a target image; Dividing the target image to obtain a plurality of strip-shaped regions arranged along a first direction; determining a directed graph for the entire target image based on intra-region connected components of the plurality of strips; determining a target path based on the directed graph that is associated with a single-source shortest path of the directed graph; removing a line in the grayscale document image that corresponds to the target path. (Appendix 20) A computer-readable storage medium having a program stored therein, the program being executed by a processor, binarizing a grayscale document image as a target image; Dividing the target image to obtain a plurality of strip-shaped regions arranged along a first direction; determining a directed graph for the entire target image based on intra-region connected components of the plurality of strips; determining a target path based on the directed graph that is associated with a single-source shortest path of the directed graph; and removing the line in the grayscale document image that corresponds to the target path.
Claims
1. a binarization unit for binarizing a grayscale document image as a target image; a division unit that divides the target image to obtain a plurality of strip-shaped regions arranged along a first direction; a directed graph determination unit that determines a directed graph for the entire target image based on the intra-region connected components of the plurality of strip-shaped regions; a target path determiner that determines, based on the directed graph, a target path associated with a single-source shortest path of the directed graph; a line remover configured to remove a line corresponding to the target path in the grayscale document image.
2. selecting connected components whose centers lie above the horizontal midline of the target image to set nodes of the directed graph; or The image processing apparatus according to claim 1 , further comprising: selecting connected components whose centers lie below a horizontal centerline of the target image to set nodes of the directed graph.
3. the directed graph determination unit determines a plurality of nodes of the directed graph; The image processing apparatus according to claim 1 , wherein the plurality of nodes correspond one-to-one to intra-region connected components of the plurality of strip-shaped regions.
4. The directed graph determination unit constructs edges of the directed graph; The constructed edges are the abscissa of the first node of the edge is different from the abscissa of the last node of the edge, The difference in abscissa between the end node and the start node of the edge is smaller than the height of the target image; and the absolute value of the difference between the ordinates of the end node and the start node of the side is smaller than both the heights of the connected components in the corresponding region corresponding to the end node and the start node; The image processing apparatus according to claim 3 , wherein the coordinates of each node of the directed graph are the coordinates of the center of a corresponding intra-region connected component.
5. The image processing device according to claim 4 , wherein the directed graph determination unit determines a cost of the edge of the directed graph based on coordinates of two nodes of the edge.
6. The image processing device according to claim 5 , wherein the directed graph determination unit determines fitting straight lines of centers of intra-region connected components corresponding to a plurality of nodes of the directed graph.
7. The image processing device according to claim 6 , wherein the directed graph determination unit determines a cost of the edge of the directed graph based on the fitting line and coordinates of two nodes of the edge.
8. The target path determination unit is adding a virtual node to the directed graph; constructing virtual edges from the virtual node to other nodes; The image processing apparatus according to claim 1 , wherein the cost of the virtual edge is set to zero.
9. binarizing a grayscale document image as a target image; Dividing the target image to obtain a plurality of strip-shaped regions arranged along a first direction; determining a directed graph for the entire target image based on intra-region connected components of the plurality of strips; determining a target path based on the directed graph that is associated with a single-source shortest path of the directed graph; removing a line in the grayscale document image that corresponds to the target path.
10. A computer-readable storage medium having a program stored therein, the program being executed by a processor, binarizing a grayscale document image as a target image; Dividing the target image to obtain a plurality of strip-shaped regions arranged along a first direction; determining a directed graph for the entire target image based on intra-region connected components of the plurality of strips; determining a target path based on the directed graph that is associated with a single-source shortest path of the directed graph; and removing the line in the grayscale document image that corresponds to the target path.
Citation Information
Patent Citations
Method and device for segmenting text image
CN106156773A
Document reading device
JP1986175878A
Table format document recognizing system
JP1991142691A
Pattern extraction device and character segmentation device
JP2000322514A
Method and device for extracting character, and storage medium
JP2001229342A
