Information processing device, information processing method, and program
The information processing device uses straight line detection and Hough transformation to robustly extract tables and cells from non-standard forms with distorted ruled lines, enhancing data entry accuracy and productivity.
Patent Information
- Application Number
- JP2024005335
- Authority / Receiving Office
- JP · JP
- Patent Type
- Patents
- Current Assignee / Owner
- Filing Date
- 2024-01-17
- Publication Date
- 2024-12-04
- Estimated Expiration
- 2039-12-16
AI Technical Summary
Existing OCR technologies struggle to accurately extract tables from non-standard forms, particularly those with close-contact layouts and distorted ruled lines, often due to scanning issues and line fragmentation, which affects data entry productivity.
An information processing device that employs a straight line detection unit to identify vertical and horizontal straight lines, evaluates intersection points, and uses Hough transformation to robustly detect ruled lines, enabling separate recognition of adjacent tables and cell extraction.
Enables accurate separation and recognition of tables and cells within non-standard forms, even with distorted layouts, improving data extraction accuracy and productivity.
Smart Images

Figure 0007594227000001 
Figure 0007594227000002 
Figure 0007594227000003
Abstract
Description
[Technical field]
[0001] The present invention relates to an information processing device, an information processing method, and a program. [Background technology]
[0002] OCR (Optical Character Recognition) exists to assist in the task of reading information from printed forms and entering it into a system. When OCR processing forms, in cases where forms issued by one's own company are collected and the contents written on them are read, they can be treated as standard forms, and it is easy to read the desired information by extracting a rectangle at a specified coordinate. However, with forms such as invoices that are sent from an unspecified number of issuers, it is not possible to specify in advance what information is written where, and it is necessary to analyze the layout of the form each time it is received and extract the information. Such forms are generally called non-standard forms.
[0003] In real-world business, there are still many cases where non-standard forms are handled, and improving the accuracy of analyzing non-standard forms has a major impact on the productivity of data entry work.
[0004] In particular, Japanese forms often use tables with ruled lines, and it is desirable to extract table areas and their contents, that is, cells, from form images with high accuracy.
[0005] Techniques for extracting tables from documents by focusing on ruled lines have been around for a long time, but the challenge is how to make the method robust against tilt and line distortion that occurs during scanning, as well as blurred and broken lines that occur when converted to a low-resolution binary image, such as for a fax. [Prior art documents] [Patent documents]
[0006] [Patent Document 1] Unexamined Japanese Patent Publication No. 11-53466 DISCLOSURE OF THEINVENTION [Problem to be solved by the invention]
[0007] Patent Document 1 describes a method for extracting a table even when breaks in ruled lines occur.
[0008] However, the detection of lines is based on runs (consecutive pixels of a certain length or more), which means that it is vulnerable to situations where lines have been fragmented into small pieces due to smearing.
[0009] Furthermore, particularly in tables such as Japanese invoices that list quantities, total amounts, and details, a detailed list listing individual items and a subtotal list listing total amounts and the like are often laid out side by side, and for data recognition processing purposes it is desirable to be able to recognize such detailed list and subtotal list separately.
[0010] Therefore, the present invention , multiple A table of number formats For tables laid out in a contiguous state, The purpose is to provide a mechanism that allows each to be recognized as a separate table. [Means for solving the problem]
[0011] The information processing device of the present invention is characterized by having a line detection unit that detects vertical or horizontal lines within an image that are candidates for table lines, an intersection evaluation unit that evaluates the shape of the lines near the coordinates of each intersection between the line candidates, a table extraction unit that estimates the table area from the intersection evaluation results, and a cell extraction unit that breaks down the area inside the table into cells. Effect of the Invention
[0012] According to the present invention, Multiple A table of number formats For tables laid out in a contiguous state, Each can be recognized as a separate table. [Brief description of the drawings]
[0013] [Figure 1] 1 is a diagram illustrating an example of a system configuration of a table extraction system according to an embodiment of the present invention. [Diagram 2] FIG. 2 is a block diagram showing an example of a hardware configuration of a PC in an embodiment of the present invention. [Diagram 3] 10 is a flowchart showing a process for extracting a table and cells from a form image in an embodiment of the present invention. [Figure 4] 10 is a flowchart showing details of a table area extraction process in the embodiment of the present invention. [Diagram 5] 10 is a flowchart showing details of a table area extraction process in the embodiment of the present invention. [Figure 6] 11A to 11C are diagrams showing a process of removing characters from a form image in an embodiment of the present invention. [Figure 7] FIG. 4 is a diagram showing an example of a form to be subjected to table extraction processing in the embodiment of the present invention. [Figure 8] 11 is a diagram showing a state in which straight lines are detected from a form image by Hough transform in an embodiment of the present invention. FIG. [Figure 9] 11A to 11C are diagrams illustrating a process in which an intersection evaluation unit evaluates the state of an intersection in the embodiment of the present invention. [Figure 10] FIG. 11 is a diagram illustrating an example of a table region extraction process in the embodiment of the present invention. [Figure 11] 13 is a diagram showing an example of detecting a subtotal table adjacent to a detailed table in an embodiment of the present invention. FIG. [Figure 12] 11A to 11C are diagrams illustrating an example of a cell region extraction process according to an embodiment of the present invention. [Figure 13] 10 is a flowchart showing details of a cell region extraction process in the embodiment of the present invention. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
[0014] Hereinafter, an embodiment of the present invention will be described in detail with reference to the drawings.
[0015] FIG. 1 is a diagram showing an example of the configuration of an information processing system according to the present invention.
[0016] As shown in FIG. 1, an information processing device 101 that executes processes such as table extraction, and a scanner 102 that scans forms and creates image files are connected via a communication path.
[0017] The communication path can take the form of a wired LAN, a wireless LAN, a USB, or the like, depending on the physical interface of the scanner 102 .
[0018] Also, a file server 103 may be placed on the communication path. Methods for importing an image scanned by the scanner 102 into the information processing device 101 include a method in which the image is directly transmitted from the scanner 102 to the information processing device 101, and a method in which an image file imported by the scanner 102 is temporarily stored in the file server 103 and the information processing device 101 retrieves the image file from the file server 103. Either method may be used.
[0019] FIG. 2 is a block diagram showing an example of the hardware configuration of the information processing device 101, the scanner 102, and the file server 103 of the present invention.
[0020] As shown in FIG. 2, an information processing device 101 etc. has a CPU (Central Processing Unit) 201, a ROM (Read Only Memory) 202, a RAM (Random Access Memory) 203, an input controller 205, a video controller 206, a memory controller 207, and a communication I / F controller 208 connected via a system bus 204.
[0021] The CPU 201 comprehensively controls each device and controller connected to a system bus 204 .
[0022] ROM 202 or external memory 211 holds the BIOS (Basic Input / Output System) and OS (Operating System), which are control programs executed by CPU 201, computer-readable and executable programs for realizing this information processing method, and various necessary data (including data tables).
[0023] The RAM 203 functions as a main memory, a work area, etc., for the CPU 201. The CPU 201 loads programs and the like required for executing processes from the ROM 202 or the external memory 211 into the RAM 203, and executes the loaded programs to realize various operations.
[0024] The input controller 205 controls input from an input device such as a keyboard 209 or a pointing device such as a mouse (not shown). If the input device is a touch panel, the user can give various instructions by pressing (touching with a finger or the like) icons, cursors, or buttons displayed on the touch panel.
[0025] The touch panel may be a touch panel capable of detecting positions touched by multiple fingers, such as a multi-touch screen.
[0026] The video controller 206 controls display on an external output device such as a display 210. The display also includes the display of a notebook computer integrated with the main body. Note that the external output device is not limited to a display, and may be, for example, a projector. In addition, for devices capable of receiving the above-mentioned touch operation, an input device is also provided.
[0027] The video controller 206 is capable of controlling a video memory (VRAM) for display control, and can use part of the RAM 203 as the video memory area, or can provide a separate dedicated video memory.
[0028] The memory controller 207 controls access to the external memory 211. As the external memory, an external storage device (hard disk) for storing a boot program, various applications, font data, user files, edit files, and various data, a flexible disk (FD), or a compact flash (registered trademark) memory connected to a PCMCIA card slot via an adapter, etc., can be used.
[0029] The communication I / F controller 209 connects and communicates with external devices via a network, and executes communication control processing on the network. For example, communication using TCP / IP, telephone lines such as ISDN, and 3G lines for mobile phones are possible.
[0030] The CPU 201 enables display on the display 210 by, for example, executing a process of expanding (rasterizing) an outline font in a display information area in the RAM 203. The CPU 201 also enables a user to give instructions using a mouse cursor (not shown) on the display 210.
[0031] Next, a table extraction process executed by the information processing device 101 according to the embodiment of the present invention will be described with reference to the flowchart in FIG.
[0032] The flowchart in FIG. 3 is a process in which the CPU 201 of the information processing apparatus 101 reads out and executes a predetermined control program, and is a flowchart showing a process for extracting tables and cells from a form image.
[0033] In step S301, input of a form image is accepted, and an instruction to extract a table area from the form image is accepted. The form image is an image scanned by the scanner 102, and may be acquired from a file server 103 that has been stored therein, or may be acquired directly from the scanner 102.
[0034] In step S302, as a pre-processing step for table region extraction, a process for correcting the skew that occurred during scanning (skew correction) is performed. Skew correction is performed using a known technique, one example of which is correction using Hough transform. Specifically, straight lines are detected using Hough transform, and horizontal straight lines are selected from the detected straight lines. Then, the average angle of the horizontal straight lines is obtained, and the entire image is rotated based on the obtained average angle, thereby performing correction. In step S303, characters included in the form image are removed. The process of removing characters will be described with reference to FIG.
[0035] FIG. 6 shows the process of removing characters from a form image.
[0036] Reference numeral 601 in FIG. 6 denotes an example of a form image acquired in step S301 and subjected to skew correction in step S302.
[0037] In FIG. 6, 602 shows the form image 601 after being binarized and inverted, and then subjected to a morphology operation in the horizontal direction. By performing a morphology operation, it is possible to remove line segments that do not reach a certain length. In other words, only line segments that are a certain length or more, such as ruled lines, remain, and line segments that do not reach a certain length, such as line segments that make up characters, are removed, resulting in the characters being removed and only the ruled lines remaining. Note that if the threshold for the length of the line segments to be removed is set too large, there is a risk that the original ruled lines will also be removed, so it is desirable to set the threshold for the length of the line segments to be removed to be slightly larger than the size of the characters.
[0038] Reference numeral 603 in FIG. 6 indicates the form image 601 that has been binarized, inverted, and then subjected to a morphology operation in the vertical direction.
[0039] 6 is the sum of the processed image 602 and the processed image 603. The processes from step S304 onwards are executed on the processed image 604 in FIG.
[0040] In step S304, straight lines having horizontal and vertical directions that are candidates for ruled lines are detected from the processed image 604 in Fig. 6. Note that what is detected here is not line segments with fixed start and end points, but straight lines.
[0041] Hough transform is used to detect straight lines. When detecting straight lines using Hough transform, if there are a certain number of pixels or more on the same line, they are detected as a straight line even if they are not consecutive. This makes it possible to robustly detect straight lines even if blurring occurs during scanning or if part of a ruled line is missing in step S303.
[0042] For example, when the processes in steps S302 to S304 are performed on the form image shown in 701 in FIG. 7, the result shown in FIG. 8 is obtained.
[0043] Fig. 8 is a diagram showing lines detected by Hough transform for the processed image from which characters have been removed by the process in step S303. In the example of Fig. 8, vertical lines v8001 to v8007 and horizontal lines h8001 to h8007 are detected.
[0044] In this embodiment, a grid consisting of these vertical and horizontal straight lines will be described as a ruled line candidate grid 802 .
[0045] In this embodiment, as described above, the lines are white on a black background, but for convenience, the following description will be given using a figure in which the lines are black on a white background.
[0046] In step S305, the state of each intersection (that is, an intersection between a vertical line and a horizontal line) on the ruled line candidate grid 802 is evaluated. Details of the evaluation of the intersection state will be described with reference to FIG.
[0047] In step S304, line detection processing is performed using Hough transform, but the lines detected by Hough transform are represented by two parameters, the angle and the distance from the origin, and do not have information about the start and end points. Therefore, although ruled line candidate grid 802 forms a rectangular lattice, in order to determine in which range on the ruled line candidate grid the actual line segments exist, it is necessary to evaluate each intersection formed by the ruled line candidate grid.
[0048] FIG. 9 is a diagram for explaining the details of the intersection evaluation.
[0049] What is indicated by 801 on the left side of Fig. 9 is an enlarged view of Fig. 8. What is indicated by 901 on the right side of Fig. 9 is a processed image corresponding to the area indicated by the dashed line in C901 in 801.
[0050] The size of the region C901 is dynamically determined so that it can be cut out as large as possible without interfering with the top, bottom, left and right intersections of the ruled line candidate grid 802. This is because even after skew correction, distortion may remain in the image itself, and positional deviations may occur between the straight lines detected by Hough transform and the actual pixels. Even in such cases, it is cut out as large as possible so that the pixels that are the intersections of the ruled lines can be captured within the image sides.
[0051] Next, fragments of a ruled line are detected from the processed image 901. Specifically, it is assumed that the fragment of a ruled line is the largest area among the continuous regions of pixels in the processed image 901. In the example of 901 in Fig. 9, a "┏" shaped region 902 is detected as a fragment of a ruled line.
[0052] Next, the direction in which region 902 touches the periphery of the processed image 901 is evaluated. In the example of Fig. 9, it touches "bottom" and "right". In other examples, if the intersection is T-shaped, it is "left", "right", and "bottom", and if it is cross-shaped, it is "left", "right", "top", and "bottom". If it touches the periphery in one or less directions, it is considered to be noise that is not a ruled line, and is excluded from the evaluation of table region and cell extraction, which will be described later.
[0053] As a result of the intersection evaluation, information such as the center coordinates of the cut rectangular area, the size, and the direction in which the lines extend (the direction in which the area touches the perimeter) is stored for each intersection.
[0054] The above is the details of the intersection evaluation process in step S305.
[0055] Next, in step S306, a table area is identified and extracted from the ruled line candidate grid 802.
[0056] The table region extraction process in step S306 will be described in detail with reference to the flowcharts of FIGS.
[0057] 4 is a process in which steps S402 to S409 are sequentially performed on horizontal ruled line candidates (S401). In this embodiment, the horizontal ruled line candidates located at the top of the processing image are processed sequentially.
[0058] In step S402, the flag indicating the detection status of the left end point is set to No. In other words, it is recorded that the left end point has not been detected. The left end point means the "intersection at the upper left corner of the table" and the "intersection at the lower left corner of the table."
[0059] In step S403, the processes in steps S404 to S408 are sequentially executed for the intersections on the horizontal ruled line candidate being processed. Note that in this embodiment, the intersections on the horizontal ruled line candidate are sequentially processed from left to right.
[0060] In step S404, it is determined whether the left end point has been detected.
[0061] If it has been detected (S404: YES), the process proceeds to step S408.
[0062] If it has not been detected yet (S404: NO), the process proceeds to step S405.
[0063] In step S405, it is determined whether the intersection being processed is the left endpoint. The determination as to whether it is the left endpoint is made using information related to the "direction in which the lines extend" from the contents evaluated in step S305. Specifically, if the "direction in which the lines extend" is only two directions, "right" and "down," it is determined to be the left endpoint as the upper left corner of the table. On the other hand, if the "direction in which the lines extend" is only two directions, "right" and "up," it is determined to be the left endpoint as the lower left corner of the table.
[0064] If it is the left end point (S405: YES), the process proceeds to step S406.
[0065] If it is not the left end point (S405: NO), the process proceeds to step 410, where the process proceeds to the next intersection.
[0066] In step S406, the flag indicating the detection status of the left end point is set to YES, that is, it is recorded that the left end point has been detected.
[0067] In step S407, the table scan direction is obtained. The table scan direction is a parameter that determines whether a table is assumed to exist above or below the table for evaluation. For example, c1001 in FIG. 10 has lines extending to the right and below, so the scan direction is downward. Also, c1101 in FIG. 11 has lines extending to the right and above, so the scan direction is upward.
[0068] In step S408, it is determined whether the intersection being processed is the right end point. The determination as to whether it is the right end point is made using information related to the "direction in which the lines extend" from the contents evaluated in step S305. Specifically, if the "direction in which the lines extend" is only two directions, "left and down," it is determined to be the right end point as the upper right corner of the table. On the other hand, if the "direction in which the lines extend" is only two directions, "left and up," it is determined to be the right end point as the lower right corner of the table.
[0069] If it is the right end point (S408: YES), the process proceeds to step S409.
[0070] If it is not the right end point (S408: NO), the process proceeds to step S410, where the process proceeds to the next intersection point. In step S409, the table area is scanned. Details of the process in step S409 will be described with reference to the flowchart in FIG.
[0071] The flowchart in Figure 5 is a process for determining a horizontal ruled line (a horizontal ruled line that forms the opposite side of the table) that pairs with the horizontal ruled line obtained in the flowchart in Figure 4, and the processes of steps S502 to S504 are sequentially performed on the horizontal ruled line candidates in the table scan direction obtained in step S407 (step S501).
[0072] In step S502, it is determined whether the candidate horizontal ruled line to be processed is in contact with another table that has already been determined (whether it has been determined as an edge that forms another table).
[0073] If there is interference with another table (step S502: YES), the process proceeds to step S505, and the area that has already been scanned (the area formed by the horizontal ruled lines obtained in the process of Figure 4 and the horizontal ruled lines for which the result of S502 was judged to be YES) is confirmed as the table area and stored.
[0074] If there is no interference with other tables (step S502: NO), the process proceeds to step S503.
[0075] In step S503, it is determined whether the horizontal ruled line being processed contains any intersection points that are corner points. A corner point is an intersection point that forms the four corners of a table, and means that, among the contents evaluated in step S305, the information related to the "direction in which the ruled line extends" is any of the following: "Only 'right' and 'down'" (upper left corner point), "Only 'left' and 'down'" (upper right corner point), "Only 'right' and 'up'" (lower left corner point), or "Only 'left' and 'up'" (lower right corner point).
[0076] If a corner point is included (step S503: YES), the process proceeds to step S505, and the area already scanned (the area formed by the horizontal ruled lines obtained in the process of FIG. 4 and the horizontal ruled lines for which the result of S503 is YES) is determined to be a table area and stored.
[0077] If no corner point is included (step S503: NO), the process proceeds to step S504.
[0078] In the example of FIG. 10, if corner point c1004 is found, area 1001 (area indicated by dotted lines) is determined to be a table area. Note that intersection point c1003 is not evaluated as a corner point here because lines extend not only to the "left" and "top" but also to the "bottom." In order to recognize such tables, if a corner point is found on either the left or right, the table is determined at that point. If no corner point is found, the process proceeds to step S504.
[0079] In step S504, a determination is made as to whether or not at least one of the left and right end points of the horizontal ruled line being processed is an "end point at which the ruled line extends outward."
[0080] "The left and right endpoints of the horizontal ruled line to be processed" refer to intersections on the same vertical ruled line as the left endpoint acquired in the flowchart of FIG. 4 or intersections on the same vertical ruled line as the right endpoint, which are included in the horizontal ruled line to be processed. Whether or not the ruled line extends outward is determined based on the "direction in which the ruled line extends" at the endpoint. Specifically, for the left endpoint, if the ruled line extends to the left, it is determined that the ruled line extends outward. Similarly, for the right endpoint, if the ruled line extends to the right, it is determined that the ruled line extends outward.
[0081] If it is determined that there is an "end point where the line extends outward" (step S504: YES), the process proceeds to step S505, and the area that has been scanned so far (the area formed by the horizontal line obtained in the process of Figure 4 and the horizontal line for which the result of the YES decision in S504 was made) is determined to be a table area and stored.
[0082] For example, in the example of FIG. 11, since the ruled lines extend outward (to the left) from intersection c1103, the table scan is terminated here and area 11101 (indicated by the dotted line) is determined to be the table area.
[0083] If it is determined that there is no "end point where the line extends outward" (step S504: NO), the line being processed is determined to not be a horizontal line that is the opposite side of the table, and processing moves to the next horizontal line.
[0084] If no horizontal ruled line that is the opposite side of the table is found even after the processes of S502 to S504 are performed for all horizontal ruled lines, it is determined that no table area exists, and the process of this flowchart ends (S507). For example, this process is performed when the right and left end points acquired in the flowchart of FIG. 4 are erroneously detected due to noise.
[0085] In step S506, the flag indicating the detection status of the left endpoint is set to No, and the process proceeds to step S401 in Fig. 4. Note that, in the processes from S402 onwards after the process of step S506 is executed, only horizontal ruled line candidates for which the processes of S502 to S504 have not been executed may be processed.
[0086] Through the above processing, it becomes possible to extract a table area from the form image for which a table extraction request was received in S301.
[0087] In particular, as shown in Figure 7, even in the case of a table that combines two rectangles (a detailed table listing individual items and a subtotal table listing total amounts, etc.), it is possible to recognize the detailed table and the subtotal table as separate tables. As a result, it is possible to achieve high accuracy in the recognition process of the data contained in the table.
[0088] Next, the process of step S307 will be described.
[0089] Step S307 is a process of extracting cell regions included in the table region extracted in step S306. Details of the cell region extraction process will be described below with reference to the flowchart in FIG.
[0090] In the flowchart of FIG. 13, the processing of steps S1302 to S1314 is executed sequentially for candidates for horizontal ruled lines (in this embodiment, executed sequentially from top to bottom) (S1301), and the processing of steps S1304 to S1306 is executed sequentially for intersections on the horizontal ruled lines (in this embodiment, executed sequentially from left to right).
[0091] In step S1302, a flag indicating the detection status of the upper left intersection (left end point) of the cell is set to NO, that is, a state indicating that the upper left intersection has not yet been detected.
[0092] In step S1034, it is determined whether the flag indicating the detection status of the upper left intersection is YES, that is, whether the upper left intersection has been detected.
[0093] If it has been detected (step S1304: YES), the process proceeds to step S1308.
[0094] If it has not been detected (step S1304: NO), the process proceeds to step S1305.
[0095] In step S1305, it is determined whether the intersection to be processed is the left end point (the upper left intersection of the cell). Specifically, when the direction of the ruled line extending from the intersection includes "right" and "down" (if right and down are included, the intersection is determined to be the left end point even if there is a ruled line extending to the left or up), the intersection is determined to be the upper left intersection of the cell. For example, in the example of FIG. 12, intersection c1201 is determined to be the left end point.
[0096] If it is determined that the intersection point to be processed is the left end point (step S1305: YES), the process proceeds to step S1306.
[0097] If it is determined that the intersection to be processed is not the left end point (step S1305: NO), the process proceeds to step S1307, where the process moves to the next intersection (the intersection one to the right).
[0098] In step S1306, a flag indicating the detection status of the left end point is set to YES. Then, the process proceeds to the next intersection (the intersection one to the right) (S1307). In the example of FIG. 12, the process proceeds to the intersection c1202.
[0099] In step S1308, it is determined whether the intersection to be processed is the right end point (the upper right intersection of the cell). Specifically, when the direction of the ruled line extending from the intersection includes "left" and "down" (if left and down are included, the intersection is determined to be the right end point even if there is a ruled line extending to the right or up), the intersection is determined to be the upper right intersection of the cell. In the example of FIG. 12, the intersection c1202 is determined to be the right end point.
[0100] If it is determined that the intersection point being processed is the right end point (step S1308: YES), the process proceeds to step S1309, and the processes of S1310 to S1311 are executed sequentially on the horizontal ruled line candidate.
[0101] If it is determined that the intersection to be processed is not the right end point (step S1308: NO), the process proceeds to step S1307, where the process proceeds to the next intersection (the intersection one point to the right). If the determination at step S1308 is YES, the processes at steps S1310 to S1311 are performed on the horizontal ruled line candidate immediately below the horizontal ruled line candidate being processed. Note that the horizontal ruled line candidate here refers to a line segment, among straight horizontal ruled lines, that is identified by an intersection point on the same vertical ruled line as the left endpoint detected at S1305 and an intersection point on the same vertical ruled line as the right endpoint detected at S1308.
[0102] In step S1310, it is determined whether the horizontal ruled line candidate being processed is shared with another confirmed cell.
[0103] If it is determined that the information is shared (step S1310: YES), the process proceeds to step S1313.
[0104] If it is determined that the information is not shared (step S1310: NO), the process proceeds to step S1311.
[0105] In step S1311, it is determined whether an inward ruled line extends from the left or right end point of the horizontal ruled line candidate being processed. An inward ruled line is a ruled line that extends to the right if the left end point is a left end point, and a ruled line that extends to the left if the right end point is a right end point. Note that as long as an inward ruled line is included, this process will be determined as YES even if a ruled line extending in another direction is also included. For example, in the example of FIG. 12, there is a ruled line that extends to the right from intersection c1204, which is the left end point, and there is a ruled line that extends to the left from intersection 1203, which is the right end point, so the result is YES in step S1311.
[0106] If it is determined that inward ruled lines extend from the left and right end points (step S1311: YES), the process proceeds to step S1313.
[0107] If it is determined that no inward ruled lines extend from the left or right end points (step S1311: NO), the process proceeds to step S1312, where the process moves on to the next (one step below) horizontal ruled line candidate.
[0108] In step S1313, the cell formed by the line segment formed by the left endpoint detected in S1305 and the right endpoint detected in S1308 and the line segment for which YES was determined in S1310 or the line segment for which YES was determined in S1311 is determined as a cell area.
[0109] Then, in step S1314, the left end point detection flag is set to NO, and the process returns to S1303. The above is the process for detecting the cell area. In this embodiment, when detecting and identifying a table area or a cell area, the horizontal ruled lines are used as a reference, but the process may be performed based on the vertical ruled lines.
[0110] The present invention can be embodied, for example, as a system, an apparatus, a method, a program, or a recording medium, etc. Specifically, the present invention may be applied to a system composed of a plurality of devices, or may be applied to an apparatus composed of a single device.
[0111] Moreover, the program in the present invention is a program that enables a computer to execute the processing methods of the flowcharts shown in Figures 3 to 5 and 13, and the storage medium of the present invention stores a program that enables a computer to execute the processing methods of Figures 3 to 5 and 13. The program in the present invention may be a program for each processing method of each device in Figures 3 to 5 and 13.
[0112] As described above, it goes without saying that the object of the present invention can be achieved by supplying a recording medium on which a program that realizes the functions of the above-mentioned embodiments is recorded to a system or device, and having the computer (or CPU or MPU) of that system or device read and execute the program stored on the recording medium.
[0113] In this case, the program itself read out from the recording medium will realize the novel functions of the present invention, and the recording medium on which the program is recorded will constitute the present invention.
[0114] Examples of recording media for supplying the program include flexible disks, hard disks, optical disks, magneto-optical disks, CD-ROMs, CD-Rs, DVD-ROMs, magnetic tapes, non-volatile memory cards, ROMs, EEPROMs, silicon disks, etc.
[0115] Furthermore, it goes without saying that the functions of the above-mentioned embodiments are not only realized by the computer executing a program it has read, but also includes cases where an OS (operating system) running on the computer performs some or all of the actual processing based on the instructions of the program, and the functions of the above-mentioned embodiments are realized through that processing.
[0116] Furthermore, it goes without saying that this also includes cases where a program read from a recording medium is written into a memory provided on a function expansion board inserted into a computer or a function expansion unit connected to a computer, and then a CPU or the like provided on the function expansion board or function expansion unit performs some or all of the actual processing based on the instructions of the program code, thereby realizing the functions of the above-mentioned embodiments.
[0117] The present invention may be applied to a system consisting of multiple devices, or to an apparatus consisting of a single device. Needless to say, the present invention can also be applied to a case where the object is achieved by supplying a program to a system or apparatus. In this case, the effect of the present invention can be enjoyed by the system or apparatus by reading a recording medium storing a program for achieving the present invention into the system or apparatus.
[0118] Furthermore, by downloading and reading a program for achieving the present invention from a server, database, etc. on a network using a communication program, the system or device can enjoy the effects of the present invention. Note that the present invention also includes configurations that combine the above-mentioned embodiments and their modified examples. [Explanation of symbols]
[0119] 101 Information processing device 102 Scanner 103 File Server
Claims
1. An acquisition means for acquiring an image including a table that is laid out in a state in which a first table in a first format and a second table in a second format are in contact with each other; a ruled line direction specifying means for specifying a direction in which ruled lines extend at intersections of ruled lines in a table included in the acquired image; a table region specifying means for specifying an area relating to the first table and an area relating to the second table included in the image as separate table regions using the extending direction of the ruled lines specified by the ruled line direction specifying means; An information processing device comprising:
2. 2. The information processing apparatus according to claim 1, wherein the table area specifying means specifies the table area based on intersections forming four corners of a table, the intersections being detected based on the extending direction of the ruled lines specified by the ruled line direction specifying means.
3. a corner point specifying means for specifying, among the intersections, intersections forming the upper left and upper right corners of a table or intersections forming the lower left and lower right corners of the table based on the extending direction of the ruled line specified by the ruled line direction specifying means; an opposite side specifying means for specifying a side that is opposite to a side formed from the corner point specified by the corner point specifying means; Equipped with 3. The information processing apparatus according to claim 1, wherein the table area specifying means specifies, as a table area, an area consisting of a side formed from the corner point specified by the corner point specifying means and an opposite side specified by the opposite side specifying means.
4. The opposite side identifying means identifies, as an opposite side, a side parallel to a side identified based on a corner point identified by the corner point identifying means, when the side is a side constituting an area already identified as a table area; 4. The information processing apparatus according to claim 3, wherein said table area specifying means specifies, as a table area, an area consisting of a side formed from the corner point specified by said corner point specifying means and an opposite side specified by said opposite side specifying means.
5. The opposite side identifying means identifies, as the opposite side, a side that is parallel to a side identified based on the corner point identified by the corner point identifying means and includes an intersection point with a ruled line extending outward, 4 . The information processing apparatus according to claim 3 , wherein the table area specifying means specifies, as a table area, an area specified based on a side formed from the corner point specified by the corner point specifying means and an opposite side specified by the opposite side specifying means.
6. An information processing device as described in any one of claims 1 to 5, further comprising an identification means for identifying a cell of the table area identified by the table area identification means based on the direction in which the lines of the intersections included in the table area extend.
7. An information processing device as described in any one of claims 1 to 6, characterized in that the first format of the first table and the second format of the second table are rectangular shapes having at least different table widths.
8. An information processing device as described in any one of claims 1 to 7, characterized in that the first table is a detailed table and the second table is a subtotal table.
9. An acquisition step in which an acquisition means of an information processing device acquires an image including a table that is laid out in a state in which a first table in a first format and a second table in a second format are in contact with each other; a ruled line direction specifying step of specifying a direction in which ruled lines extend at intersections of ruled lines in a table included in the acquired image by a ruled line direction specifying means of the information processing device; a table area specifying step in which a table area specifying means of the information processing device specifies an area relating to the first table and an area relating to the second table included in the image as separate table areas by using the extending direction of the ruled lines specified in the ruled line direction specifying step; An information processing method comprising:
10. A program for causing a computer to function as each of the means of an information processing device described in any one of claims 1 to 8.
Citation Information
Patent Citations
Method and device for recognizing table
JP1997288714A
Table recognizing method
JP1999053466A
Slip recognizing method
JP2000339406A
Document processor, program for operating the same, and document format creation program
JP2006244526A
