Logic creation support system

Abstract

To provide a logic creation aiding system that reduces burden on a user who is involved in creation of logics related table reading.SOLUTION: A logic creation aiding system for recognition of logical structure of a document is connected to an input / output device. A logic flow management unit manages logics and logic flows of structure element recognition and logical structure recognition. The structure element recognition receives a structure element recognition logic designated by a user at the logic flow management unit from an input device and outputs it. The logical structure recognition receives the output and a logical structure recognition logic designated by the user at the logic flow management unit and outputs them. The logic flow management unit reflects a logic visualization and logic application result in an interface.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to a logic creation support system for an information acquisition device. [Background technology]

[0002] Tables are included in documents such as forms and reports, and are a means of expressing information by visually organizing and arranging the elements (components) that make up a table, such as items and values, and presenting the reader with a logical structure of the table elements (logical structure). Tables exist in a variety of media, including documents, images, and web pages, and are widely used. By mechanically reading the logical structure of such tables, storing the results logically, and utilizing them in information extraction systems and information retrieval systems, it is expected that the efficiency of business systems and information retrieval will improve. Therefore, automatic recognition of the logical structure of tables is an important technology.

[0003] To obtain information contained in tables from input documents such as forms or images, the table is first extracted, then its components are recognized using character recognition and layout recognition, and finally, a logical structure based on the logical relationships between the table elements is recognized. The obtained logical structure can be stored in a normalized logical database such as a relational database, making it easy to use in information extraction systems and information retrieval systems. For example, from a table showing the basic characteristics of an industrial part, the items related to the part's basic characteristics and their numerical values ​​can be recognized as components, and by recognizing that there is a logical relationship between the items related to the basic characteristics and their numerical values, the logical structure can be recognized and the correspondence between the basic characteristics and their values ​​can be stored in a relational database.

[0004] In automatic table reading, the format of the table to be read is usually determined in advance, and the shape of the table to be read, the positions of the cells (the smallest area surrounded by ruled lines) that make up the elements, and the positions of the text lines are determined, and the table is read according to a predetermined procedure. For example, when considering reading table information from an image of a receipt used in internal business, the table structure of the receipt is known in advance for internal business purposes, so numerical values ​​can be recognized from the coordinates of specific text lines on the receipt, and the elements and logical structure can be inferred. When the format of the table to be read is determined, the accuracy of reading the table according to such a predetermined procedure is sufficient. Even if the table format is slightly modified in the future, this can be accommodated within a reasonable time and cost by partially modifying the logic related to table reading.

[0005] On the other hand, there is a need to automatically read and acquire information from various non-standard tables. There are many types of non-standard tables, such as balance sheets, detailed statements, and doctor's certificates. Acquiring information from these tables is expected to improve business efficiency. However, the format of such non-standard tables can easily change depending on the system and vendor of the institution issuing the document. For example, a balance sheet defines the balance of income and expenditures by dividing it into an "asset section" and a "liability section," and reports monetary values ​​for each item. The format of balance sheets varies from company to company. For example, the assets section may be listed on the left half of the table and the liabilities section on the right half, or the assets section may be listed above and the liabilities section below. In other cases, for example, the lines that make up the assets section may be visually represented by indenting. When it comes to doctor's certificates, the table format to be read varies from hospital to hospital, making it difficult to process them using a predefined table format. [Prior art documents] [Patent documents]

[0006] [Patent Document 1] JP 2009-122722 A Summary of the Invention [Problem to be solved by the invention]

[0007] To build an information retrieval system that can automatically read the various types of unstructured tables described above and retrieve their logical structures, it is necessary to build logic that recognizes the components and logical structures specific to each table format. When table formats change frequently or completely new table formats must be supported, it is necessary to build logic used to read the tables and a logic application flow (logic flow) that combines multiple such logics. Examples of logic include rules and algorithms for recognizing table elements and rules and algorithms for recognizing table structure. Examples of logic flows include two logics that recognize logical structures, and the order in which the two logics are applied may be specified. Such logic and logic flows require specialized knowledge and know-how regarding table structures, and because the logic itself involves complex programmatic processing, building the logic and logic flow places a significant burden on developers (users).

[0008] Patent Document 1 discloses a method for reducing the burden on a correction worker when correcting recognized form data in a system that recognizes the logical structure of forms and stores the logical information in a database. However, Patent Document 1 modifies the database based on correction instructions from a user on an interface (I / F), and is unable to support the creation of complex recognition logic.

[0009] Therefore, in order to support the creation of logic and logic flows and reduce the burden on users involved in reading tables, we are considering reducing the burden on users by preparing logic templates in advance on the interface rather than providing correction instructions, and then combining logic and adjusting logic parameters. With this method, users are required to construct logic flows, but because templates for each logic are prepared in advance, there is no need to write their own programs or think up their own logic unless necessary. This is therefore expected to reduce the burden on users.

[0010] However, as mentioned above, the logic involved in table reading involves complex programmatic processing. Even if a logic template is provided, it is difficult for users to intuitively understand how each piece of logic is executed, resulting in a burden on the user. For example, in the logic for determining the components, rules are implemented using the character row coordinates and dictionaries of the components to identify items at the top of the table (table header), items on the sides of the table (table side), and value parts that are neither the table header nor the table side. In this case, the user specifies parameters necessary for implementing the rules, such as the amount of coordinate deviation to be tolerated, as well as dictionaries and regular expressions consisting of character strings that may be included in the table header, table side, and value. The processing using these parameters and dictionaries places a heavy burden on the user, as the user can only make appropriate decisions after specifically understanding the content of the logic and its results. Furthermore, this burden on the user cannot be resolved by simply displaying the logical structure based on the logic recognition results in an interface.

[0011] Therefore, when a user constructs logic and logic flows on the interface, the burden on the user involved in constructing logic and logic flows can be significantly reduced by presenting to the user on the interface two perspectives: what processing each logic performs, and how the logical structure is recognized as a result of the execution of said processing. Furthermore, the information necessary for creating logic is reflected sequentially on the interface, making it possible to intuitively modify the logic specified by the user and parameters related to the logic. [Means for solving the problem]

[0012] In order to solve the above-mentioned problems, a logic creation support system for logical structure recognition of documents in an information acquisition device, which is one aspect of the invention disclosed in this application, is connected to an input / output device, and a logic flow management unit manages the logic and logic flow of component recognition and logical structure recognition, and the component recognition is The aforementioned The logic flow management unit receives and outputs the component recognition logic specified by the user from the input device, and the logical structure recognition unit receives and outputs the output. The aforementioned The logic flow management unit receives and outputs the logical structure recognition logic specified by the user from the input device, and the logic flow management unit reflects the logic visualization and logic application results in the interface. The logic flow management unit is a logic creation support system configured to visualize the information necessary for adjusting the parameters of the table side, table top, and value recognition logic of the component recognition logic using a logic visualization I / F, and to reflect the results of applying the component recognition logic in the logic application result I / F. [Effects of the Invention]

[0013] According to the present invention, the burden on the user in creating logic for reading tables is reduced. [Brief explanation of the drawings]

[0014] [Figure 1] FIG. 2 is a diagram illustrating an example of a functional block of an information acquisition device in each embodiment. [Figure 2] FIG. 10 is an explanatory diagram showing Table A, which is used as input in each embodiment. [Figure 3] FIG. 10 is an explanatory diagram showing Table B, which is used as input in each embodiment. [Figure 4] FIG. 10 is an explanatory diagram illustrating an example of an interface of a logic flow management unit according to the first embodiment. [Figure 5] FIG. 10 is an explanatory diagram illustrating an example of an interface of a logic flow management unit according to the second embodiment. [Figure 6] FIG. 11 is an explanatory diagram illustrating an example of an interface of a logic flow management unit according to a third embodiment. [Figure 7] FIG. 11 is an explanatory diagram illustrating an example of an interface of a logic flow management unit according to a third embodiment. [Figure 8] FIG. 13 is an explanatory diagram illustrating an example of an interface of a logic flow management unit according to a fourth embodiment. [Figure 9] FIG. 13 is an explanatory diagram illustrating an example of an interface of a logic flow management unit according to a fifth embodiment. [Figure 10] FIG. 20 is an explanatory diagram illustrating an example of an interface of a logic flow management unit according to a sixth embodiment. [Figure 11] FIG. 20 is an explanatory diagram illustrating an example of an interface of a logic flow management unit according to a seventh embodiment. [Figure 12] FIG. 2 is a diagram illustrating an example of a hardware configuration of a computer that realizes an information acquisition device according to each embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0015] The following describes the embodiments of the present invention with reference to the accompanying drawings. In the following, it is understood that the embodiments and modifications can be combined in part or in whole without departing from the spirit of the present invention.

[0016] In the following description, the information acquisition device is a logic creation support system for document logical structure recognition, and is connected to an input / output device; the logic flow management unit manages the logic and logic flow of component recognition and logical structure recognition; component recognition accepts and outputs component recognition logic specified by the user in the logic flow management unit from the input device; logical structure recognition accepts and outputs the output and logical structure recognition logic specified by the user in the logic flow management unit from the input device; the logic flow management unit reflects logic visualization and logic application results in the interface, thereby reducing the burden on users involved in logic creation. <Configuration of information acquisition device 100 according to the first embodiment> 1 shows an information acquisition device 100 of Example 1. The information acquisition device 100 of Example 1 includes an input unit 101, an input table 101a, component recognition logic 101b, logical structure recognition logic 101c, a logic flow management unit 102, a component recognition logic execution unit 102-1, a logical structure recognition logic execution unit 102-2, a logic visualization I / F update unit 102-3, a logic application result I / F update unit 102-4, an output unit 103, and a completed logic 103a.

[0017] In this embodiment, an example of the interface of the logic flow management unit in the information acquisition device 100 is shown, and a mechanism for reducing the user's burden of creating logic related to table reading is described. According to this embodiment, when a user specifies logic on the interface and constructs a logic flow, the user can be shown what processing each piece of logic will perform and how the logical structure of the table is recognized by each piece of logic. This embodiment can also be applied to various uses, such as developing logic related to table reading, modifying existing logic, and incorporating it into a table reading system.

[0018] In the following examples, Table A in FIG. 2 and Table B in FIG. 3 are used as input examples for the sake of explanation. Table A is an example of a balance sheet, and is composed of components such as "Current Assets," "1. Cash and Deposits," "2. Prepaid Expenses," "Fixed Assets," "1. Tangible Fixed Assets," "2. Intangible Fixed Assets," "Amount," "3,000," "1,000," and "500." Components are table elements of a granularity generally used when recognizing logical structures, and can be the arrangement of character strings (character lines) that are table components, or rectangles (cells) surrounded by table borders. In Table A, for example, "Current Assets" exists as a character line, and for example, a rectangle containing the character line "Amount" exists as a cell. Table B is an example of a table related to the basic characteristics of industrial parts. Table B similarly includes character lines and cells. In the following examples, the aim is to read Table A and Table B, recognize logical relationships using logic flow, and convert them into Logical Structure A and Logical Structure B. To obtain logical structure A and logical structure B, it is necessary to design component recognition logic and logical structure recognition logic specialized for the formats of table A and table B. The design of component recognition logic and logical structure recognition logic, as well as some of the mechanisms for reducing the burden on users, will be clarified in the following examples.

[0019] Tables A and B are merely examples of table formats used for explanation purposes, and any table format is acceptable. In this case, various table formats are possible, including various types of forms such as medical certificates and detailed statements, as well as tables without lines, tables within tables, and illogical tables.

[0020] Furthermore, in the following embodiments, the explanation will be mainly based on a GUI (graphical interface), but is not limited to a GUI. Furthermore, the configuration, layout, and functions of the GUI are merely examples for the purpose of explaining the following embodiments, and may be partially or completely added, modified, deleted, or combined without departing from the spirit of the present invention. <Processing of the information acquisition device 100 according to the first embodiment> The input unit 101 receives an input table 101 a, a component recognition logic 101 b, and a logical structure recognition logic 101 c as inputs, and outputs them to the logic flow management unit 102 .

[0021] In this embodiment, the input table 101a is assumed to be a table contained in a document image. However, the input table 101a can accept various types of input formats, such as databases, documents, and drawings. In this case, for example, the document may be an electronic document or a document physically recorded on paper or a board. In this case, the drawing may be an electronic drawing or a drawing physically recorded on paper or a board. Furthermore, the document and drawing may contain various information, such as character strings, figures, and photographs. In this embodiment, for the purpose of explanation, the input table 101a recognizes the ruled lines of the table as figures, recognizes the coordinates and shape of the figures, and recognizes the coordinates and character string information of the character lines contained in the table. The coordinates and shape of the figures, the coordinates and character string information of the character lines, and the character string information are input to the input unit 101. To recognize the shape of the ruled lines and the coordinates of the character lines, software and algorithms for extracting ruled lines and analyzing layouts using image processing can be used. To recognize the character strings contained in the character lines, algorithms and software for handwritten character recognition can be used. In this embodiment, the character string will be described assuming that it is in Japanese, but it may be in any language such as Chinese, English, etc. Furthermore, the character string may include not only language but also graphics and symbols.

[0022] The component recognition logic 101b is configured with algorithms and rules for recognizing components such as text lines and cells contained in a table. In this case, there may be multiple logics for recognizing components. For example, the component recognition logic 101b can be configured with two logics: one for recognizing text lines and one for recognizing cells. Furthermore, the order in which the logics are applied and the combination of logics can be specified.

[0023] The components recognized by the component recognition logic 101b for Table A may include, for example, the following character lines: "Current Assets," "1. Cash and Deposits," "2. Prepaid Expenses," "Fixed Assets," "1. Tangible Fixed Assets," "2. Intangible Fixed Assets," "3,000," "1,000," and "500." The cells may include rectangles containing the character line "Amount." In this case, the reason the cell for "Amount" is designated as the component is that the only character line contained in the cell is "Amount," and the width and height of the cell provide visual information that "Amount" is at the top of the table. On the other hand, character lines are appropriate for components such as "Current Assets" and "3,000." This is because the cell containing "Current Assets" contains many character lines, and if all of these were collectively treated as components, it would be impossible to obtain the correct logical structure. Whether the components are lines of text or cells depends on the format of the input document and table, and the components are not limited to lines of text and cells, but can be of any granularity as long as they are suitable for inferring the logical structure.

[0024] The component recognition logic 101b also includes logic for classifying components into table headers, table sides, and values. For example, in Table A, "1. Cash and Deposits" is classified as the table side, and "3,000" is classified as the value. When this is applied to all the components of Table A in the same way, the result shown in Table A-1 is obtained. In this way, classifying each component into table headers, table sides, and values ​​makes it easier to infer the logical structure. This is because, when inferring the logical structure, it is only necessary to consider the logical structure between the table headers, the logical structure between the table sides, the logical structure between the table headers and values, and the logical structure between the table sides and values, thereby limiting the search range required for inferring the logical structure. For example, in Table A, the logical structure of "1. Cash and Deposits" and "3,000" can be recognized from the combination of the relationship between the table side and the value, and the logical structure of "amount" and "3,000" can be recognized from the combination of the relationship between the table header and the value, so there is no need to search for all combinations of components. Furthermore, classifying components into table headers, table sides, and values ​​is equivalent to, for example, in the process of converting table A into logical structure A, recognizing the "Item 1" and "Item 2" columns of logical structure A as the table side, recognizing "Amount" of logical structure A as the table header, and identifying the other cells of logical structure A that contain only numbers as values.

[0025] The logical structure recognition logic 101c is composed of algorithms and rules that recognize the logical structure, which is the logical relationship between components. In this case, there may be multiple logics that recognize the logical structure. For example, the logical structure recognition logic 101c can be composed of two logics: one that recognizes the logical structure of the table head and values, and one that recognizes the logical structure of the table side and values. Furthermore, the order in which the logic is applied and the combination can also be specified.

[0026] The logic flow management unit 102 receives inputs from the input unit 101, including input table 101a, component recognition logic 101b, and logical structure recognition logic 101c, manages and reflects the logic, and updates an interface that supports the user in creating logic. The logic flow management unit 102 first checks whether logic is specified in the component recognition logic 101b. If the logic is specified, the component recognition logic execution unit 102-1 executes the logic, the logic visualization I / F update unit 102-3 visualizes the processing details of the logic on an interface for presentation to the user, and the logic application result I / F update unit visualizes the application results of the logic on an interface for presentation to the user. When the processing from the component recognition logic execution unit 102-1 to the logic application result I / F update unit 102-4 is completed, and if the next logic to be executed is specified in the component recognition logic 101b, the component recognition logic execution unit 102-1 to the logic application result I / F update unit 102-4 is again processed for the next logic to be executed.

[0027] The logical structure recognition logic execution unit 102-2 is executed when the user does not specify a component element recognition logic in the logic flow management unit 102 or when processing of the component element recognition logic is completed. At that time, the logic flow management unit 102 first checks whether logic is specified in the logical structure recognition logic 101c. If the logic is specified, the logical structure recognition logic execution unit 102-2 executes the logic, the logic visualization I / F update unit 102-3 visualizes the processing content of the logic on an interface and presents it to the user, and the logic application result I / F update unit visualizes the application result of the logic on an interface and presents it to the user. When processing from the logical structure recognition logic execution unit 102-2 to the logic application result I / F update unit 102-4 is completed and the next logic to be executed is specified in the component element recognition logic 101b, processing is again performed for the next logic to be executed from the component element recognition logic execution unit 102-1 to the logic application result I / F update unit 102-4.

[0028] In FIG. 1, the logic flow management unit 102 is merely a block diagram showing an example of processing, and the block configuration can be changed without departing from the spirit of the present invention. In this case, for example, the order of processing of the logic visualization I / F update unit 102-3 and the logic application result I / F update unit 102-4 may be changed. Additionally, the method by which the user specifies the logic may take any form. For example, the logic specification may be received from the user via input to a GUI (graphical user interface). Furthermore, the logic flow management unit 102 may be configured in part or in whole by an interface, as long as it can be presented appropriately to the user.

[0029] Fig. 4 is an explanatory diagram showing an example of realizing the logic flow management unit 102 by a GUI. In Fig. 4, the logic flow management unit 102 is implemented in a GUI window. Furthermore, the GUI window may have functions necessary for GUI operation, such as a window minimize button C1 and a window close button C2. The processing contents of the logic visualized by the logic visualization I / F update unit 102-3 are reflected one by one in the panel box C4. The application results of the logic visualized by the logic application result I / F update unit 102-4 are reflected one by one in the panel box C5.

[0030] Furthermore, the GUI of the logic flow management unit 102 in Figure 4 allows the user to add, delete, or edit component recognition logic and logical structure recognition logic through user instructions, such as button operations. For example, C6 is a panel box that provides an interface for the user to edit logic and logic flow. C62 is a button that allows the user to insert new logic. By clicking this button, the user can insert pre-prepared logic or user-definable logic into the panel box C6 as a logic window. In Figure 4, an example of a logic window is C63. C62 also indicates to the user the insertion position of the logic window through the position of the button, encouraging intuitive logic creation. Furthermore, within a logic window that has already been inserted or is to be inserted, adjustable programs, parameters, and / or dictionaries for the logic are presented as buttons, text boxes, and input boxes. For example, C63 is an example of a logic window for a certain recognition logic, and the name of the logic is displayed as a title in C65. Furthermore, while the user is editing the logic, the width of the logic window border can be changed and presented to the user. If the user wishes to delete the logic, this can be done by clicking the close button C64. Furthermore, the panel box C6 allows the user to add any number of logic windows and combine them as desired. At this time, the display position of the logic window can be adjusted using, for example, the scroll bar C61.

[0031] The GUI of the logic flow management unit 102 in Figure 4 is merely an example for the purpose of explanation, and the configuration, arrangement, shape, etc. of the GUI may be in any form. Also, it may be implemented using various interfaces other than a GUI. In this case, for example, it is possible to use a character interface (CUI). Also, part of the interface may not be implemented, or various other functions necessary for logic creation support may be implemented. Voice or gestures may be used as an interface for presenting or instructing user input and output. <Effects of Example 1> According to this embodiment, the logic flow management unit receives and outputs the logic specified by the user from the input device, and the logic flow management unit visualizes the logic information using the logic visualization I / F and reflects the results of the logic application in the logic application result I / F, thereby reducing the burden on the user involved in creating the logic. <Configuration of information acquisition device 100 according to the second embodiment> The information acquisition device of the second embodiment includes, in addition to the configuration of the information acquisition device 100, a logic window for identifying the components.

[0032] In the second embodiment, in the information acquisition device 100, the logic required to identify the components is displayed on the logic visualization I / F C4, and the application result of the logic is displayed on the application result I / F C5. Note that the explanation of parts that overlap with the explanation in the first embodiment will be omitted in part or in whole. Furthermore, in the first embodiment, the explanation of parts that overlap with the GUI of the logic flow management unit 102 shown in Fig. 4 will be omitted in part or in whole from the drawings. <Processing of the information acquisition device 100 according to the second embodiment> Fig. 2 shows the GUI of the logic flow management unit 102 of the information acquisition device 100 in Example 2. In Fig. 2, for the sake of explanation, Table A is used as the input table, but this is not limited to Table A and can also be applied to tables other than Table A. C6 includes the logic window D3 being edited in the logic flow editing I / F.

[0033] In the logic window D3, the user sets parameters related to the specific logic of the component in the text boxes. At this time, for example, as shown in D4, if the user wants to treat only the cell as a component, they can enter "Yes," or if not, "No." At this time, D4 ​​can prompt the user for input in various ways, such as a drop-down list, check box, or dialog display as a GUI. In addition, D5 allows the user to specify the maximum number of character lines that a cell can contain as a boundary condition for treating a cell as a component. For example, if the user enters "1" in D5, the "Amount" cell in Table 4 has one character line, so the boundary condition is met and the cell is considered a component.

[0034] To adjust the parameters in the logic window D3, the user must have knowledge of the logic being edited. Therefore, the burden on the user can be reduced by presenting the user with information such as the number of character lines contained in a cell. Therefore, with regard to the parameters of the logic window D3 being edited, the logic flow management unit 102 executes the component recognition logic execution unit 102-1 and the logic visualization I / F update unit 102-3. The logic visualization I / F update unit 102-3 then uses the logic visualization I / F C4 to display the number of character lines contained in each cell on the GUI, as shown in D1, intuitively presenting the user with the knowledge necessary to adjust the parameters of the logic. D1 can take various forms, such as using speech bubbles or connecting lines to cells. However, any interface can be used as long as it can be presented appropriately to the user.

[0035] In logic window D3, when D4 or D5 receives input from the user, the component recognition logic execution unit 102-1, logic visualization I / F update unit 102-3, and logic application result I / F update unit 102-4 are executed, and the logic application result I / F update unit 102-4 updates the logic application result I / F C5. For example, in FIG. 5, C5 presents the results of component identification to the user by overlaying a bounding box or rectangle that matches the shape of the component. In this case, for example, as shown in D2, a component identified as a cell can have a semi-transparent rectangle overlaid on the component to fit the entire cell. Alternatively, for example, as shown in D3, a component identified as a text line can have a semi-transparent rectangle overlaid on the component to fit the entire text line. <Effects of Example 2> According to this embodiment, the logic flow management unit receives and outputs the logic specified by the user from the input device, and the logic flow management unit visualizes the information necessary for adjusting the parameters of the component recognition logic using the logic visualization I / F, and reflects the results of applying the component recognition logic in the logic application result I / F, thereby reducing the burden on the user involved in creating the logic. <Configuration of information acquisition device 100 according to the third embodiment> The information acquisition device of the third embodiment includes, in addition to the configuration of the information acquisition device 100, an interface for classifying the table head, table side, or value of the component element.

[0036] In the third embodiment, in the information acquisition device 100, the logic required to identify the components is displayed on the logic visualization I / F C4, and the application result of the logic is displayed on the application result I / F C5. Note that the explanation of parts that overlap with the explanation in the first embodiment will be omitted in part or in whole. Furthermore, in the first embodiment, the explanation of parts that overlap with the GUI of the logic flow management unit 102 shown in Figure 4 will be omitted in part or in whole from the drawings. <Processing of the information acquisition device 100 according to the third embodiment> Fig. 6 is an explanatory diagram of an interface for classifying the front side of components in the logic flow management unit 102 of the information acquisition device 100 in Example 3. Also, in Fig. 6, the logic flow management unit 102 includes a logic window E6 that supports the user in creating front side recognition logic. Also, in Fig. 6, Table A is used as the input table for the sake of explanation, but this is not limited to Table A and can be applied to tables other than Table A.

[0037] In E6, the user can input the minimum and maximum values ​​of the alignment range when analyzing the table side in E7. For example, the input value "-1 to 1" in E7 sets the minimum value to -1 and the maximum value to 1 when analyzing the table side vertically. Note that the threshold unit can be any unit. For example, pixels or millimeters can be used. Furthermore, analyzing the table side vertically can be achieved, for example, by obtaining the X coordinate of the text line for all components and grouping the components whose X coordinates fall within the threshold range. For example, in Table A, "Current Assets" and "Fixed Assets" are located at the same X coordinate, which satisfies the threshold, and therefore "Current Assets" and "Fixed Assets" are grouped in the same group (first group). Similarly, "1. Cash and Deposits," "2. Prepaid Expenses," "1. Tangible Fixed Assets," and "2. Intangible Assets" are also grouped in the same group (second group).

[0038] In E6, when analyzing the table side, the user can specify a dictionary included in the table side as a clue for each group. For example, the input values ​​".*Assets" and ".*Expenses" in E8 specify the table side clues as regular expressions. Similarly, a dictionary included in the value can be specified as a clue to exclude parts that are not on the table side. For example, the input values ​​"\d+$" and "yen$" in E9 specify the value clues as regular expressions. The regular expressions specified in E7 and E9 calculate the number of occurrences in each group, and the plausibility of the table side for each group is estimated. For example, the regular expression ".*Assets" specified in E8 applies to all component strings included in the first group, so the proportion of components included in the group that the table side dictionary applies to (called the table side proportion) is 100%. Similarly, the table side proportion for the second group is also 100%. For example, since the regular expression specified in E9 does not apply to the strings of all the components included in the first group, the proportion of the value dictionary that applies to the components included in the group (called the value proportion) is 0%. Similarly, the value proportion for the second group is also 0%. In other words, the higher the front proportion and the lower the value proportion, the higher the plausibility of being the front side.

[0039] In E6, the user can specify what values ​​the above-mentioned front side percentage and value percentage must be to be classified as front side. For example, an input value of "80%" in E10 indicates the minimum front side percentage, and an input value of "20%" in E11 indicates the allowable value for the value percentage. In other words, the first group and the second group that have a front side percentage of 80% or more and a value percentage of 20% or less based on the input values ​​of E10 and E11 are determined to be front side, and all components included in the first group and the second group are classified as front side.

[0040] The dictionaries and parameters specified by the user in the logic window E6 require complex logic, as described above, making it difficult for the user to understand the impact of the dictionaries and parameters on the logic. Therefore, with regard to the dictionaries and parameters of the logic window E6 being edited, the logic flow management unit 102 executes the component recognition logic execution unit 102-1 and the logic visualization I / F update unit 102-3, and the logic visualization I / F update unit 102-3 operates the logic visualization I / F C4 to intuitively present the user with the knowledge necessary to adjust the logic parameters, as shown in E1. In C4, for example, the front-side alignment range specified by the user in E7 is visualized with a double-arrow symbol, and the length of the symbol indicates the range. Furthermore, as a result of the user specifying the front-side alignment range, the group configuration recognized by the front-side recognition logic is presented with an arrow shape, as shown in E3. Furthermore, in C4, for example, the front-side ratio and value ratio of each group can be presented to the user using text and a rectangle, as shown in E2.

[0041] When logic window E6 accepts user input, it executes component recognition logic execution unit 102-1, logic visualization I / F update unit 102-3, and logic application result I / F update unit 102-4, and logic application result I / F update unit 102-4 updates logic application result I / F C5. For example, in FIG. 6, C5 presents the results of the front side identification to the user by overlaying a bounding box or rectangle to match the shape of the component and changing the pattern and color of the rectangle. In this case, for example, as shown in E5, a component identified as the front side can be overlaid with a transparent rectangle surrounded by a frame that fits the entire line of text. Furthermore, by presenting a different pattern to the rectangles for components other than the front side, the front and non-front sides can be clearly distinguished from each other.

[0042] Figure 7 shows an example of the logic flow management unit 102 in Figure 6 applied to the top of a table instead of the top of the table, with the top of the table recognition logic being exemplified in F6. The main differences between the logic flow management unit 102 in Figure 7 and the logic flow management unit 102 in Figure 6 are that the analysis direction is vertical for the top of the table in Figure 6, whereas the analysis direction is horizontal for the top of the table in Figure 7, and that the dictionaries and parameters used are partially different, but the actual processing is similar. Therefore, in this example, the explanation of Figure 7 will be omitted to avoid redundant explanation. <Effects of Example 3> According to this embodiment, the logic flow management unit receives and outputs the logic specified by the user from the input device, and the logic flow management unit visualizes the information necessary for adjusting the parameters of the component recognition logic using the logic visualization I / F, and reflects the results of applying the recognition logic for the front side, top, and value of the component recognition logic in the logic application result I / F, thereby reducing the burden on the user involved in creating the logic. <Configuration of information acquisition device 100 according to the fourth embodiment> The information acquisition device of the fourth embodiment includes, in addition to the configuration of the information acquisition device 100, an interface for supporting the creation of logic for recognizing a logical structure between the components included on the front side.

[0043] In the fourth embodiment, in the information acquisition device 100, logic that recognizes the logical structure between the components on the front side is displayed on the logic visualization I / F C4, and the application result of the logic is displayed on the application result I / F C5. Note that the explanation of parts that overlap with the explanation in the first embodiment will be omitted in part or in whole. Furthermore, in the first embodiment, the explanation of parts that overlap with the GUI of the logic flow management unit 102 shown in Figure 4 will be omitted in part or in whole. <Processing of the information acquisition device 100 according to the fourth embodiment> Fig. 8 shows an explanatory diagram of an interface for recognizing the logical structure between the components on the table side of the logic flow management unit 102 of the information acquisition device 100 in Example 4. Also, in Fig. 8, the logic flow management unit 102 includes a logic window F1 that supports the user in creating logic for recognizing the logical structure between the components on the table side. Also, in Fig. 8, Table A is used as the input table for the sake of explanation, but this is not limited to Table A and can be applied to tables other than Table A.

[0044] F1 is a logic window related to logical structure recognition logic that uses the indent width (the deviation width of the X coordinate between the components on the table) that exists between the components on the table as a clue to identify whether a logical structure exists between the components on the table. For example, in Table A, which is a balance sheet, "Current Assets" and "1. Cash and Deposits" form a parent-child logical structure through indentation, and the purpose of this example is to obtain this logical structure.

[0045] In F1, the user can input the maximum allowable indent width in F2 and the minimum allowable indent width in F3. For example, an input value of "20" in F2 allows a maximum indent width of 20 between table-side components, and an input value of "3" in F3 allows an indent width of at least 3 between table-side components. In this embodiment, if the indent width between "Current Assets" and "1. Cash and Deposits" is 5, the logical structure is satisfied, and "Current Assets" and "1. Cash and Deposits" are recognized as a logical structure. Note that the width can be in any unit. Furthermore, in F4, the user can specify the maximum downward offset allowable when assigning a logical structure to table-side components. An input value of "100" in F4 prevents a logical structure from being assigned if the vertical distance between hierarchical structures is greater than 100. This allows the user to specify the scope of logic application.

[0046] The parameters specified by the user in the logic window F1 require complex logic, as described above, making it difficult for the user to understand the effect of the parameters on the logic. Therefore, with regard to the parameters of the logic window F1 being edited, the logic flow management unit 102 executes the logical structure recognition logic execution unit 102-2 and the logic visualization I / F update unit 102-3, which then executes the logic visualization I / F C4 to intuitively present the user with the knowledge necessary to adjust the logic parameters. In Figure 8, in C4, for example, F2 informs the user of one of the parent components of the logical structure using a rectangle. F3 and F4 inform the user of the input values ​​of the maximum indent width F2 and the minimum indent width F3 using a character string and a rectangle. F6 similarly informs the user of the input value of the maximum downward distance F4. Furthermore, as shown in F5, an arrow symbol can be placed on Table A depending on the input value, indicating the valid range of the value in Table A.

[0047] When logic window F1 receives an input from the user, it executes logical structure recognition logic execution unit 102-2, logic visualization I / F update unit 102-3, and logic application result I / F update unit 102-4, and logic application result I / F update unit 102-4 updates logic application result I / F C5. For example, in Fig. 8, C5 presents the results of identifying the logical structure between front sides to the user by overlaying a bounding box and a rectangle to match the shape of the component, and overlaying an arrow symbol as the logical structure. <Effects of Example 4> According to this embodiment, the logic flow management unit receives and outputs logic specified by the user from an input device, and by presenting rectangles, character strings, and symbols on an input table, the logic flow management unit visualizes the information necessary for adjusting the parameters of the logical structure recognition logic by indentation using a logic visualization I / F, and reflects the results of application of the logical structure recognition logic in the logic application result I / F, thereby reducing the burden on the user involved in creating logic. <Configuration of information acquisition device 100 according to the fifth embodiment> The information acquisition device of the fifth embodiment includes, in addition to the configuration of the information acquisition device 100, an interface for supporting the creation of logic for recognizing a logical structure between the components included on the front side.

[0048] In the fifth embodiment, in the information acquisition device 100, logic that recognizes the logical structure between the components on the front side is displayed on the logic visualization I / F C4, and the application result of the logic is displayed on the application result I / F C5. Note that the explanation of parts that overlap with the explanation in the first embodiment will be omitted in part or in whole. Furthermore, in the first embodiment, the explanation of parts that overlap with the GUI of the logic flow management unit 102 shown in Figure 4 will be omitted in part or in whole. <Processing of the information acquisition device 100 according to the fifth embodiment> Fig. 9 shows an explanatory diagram of an interface of the logic flow management unit 102 of the information acquisition device 100 in Example 5, which recognizes the logical structure between the components that are on the table side and the components that are values. Also, in Fig. 9, the logic flow management unit 102 includes a logic window G1 that supports the user in creating logic that recognizes the logical structure between the components that are on the table side and the components that are values. Also, in Fig. 9, Table A is used as the input table for the sake of explanation, but this is not limited to Table A and can be applied to tables other than Table A.

[0049] G1 is a logic window related to logical structure recognition logic that uses the distance and offset between components as clues to identify whether a logical structure exists between a component that is the front side and a component that is a value. For example, in Table A, which is a balance sheet, "100," which exists at almost the same Y coordinate as "2. Prepaid Expenses," forms a parent-child logical structure, and the purpose of this example is to obtain this logical structure.

[0050] In G1, the user can specify the range (maximum distance) within which the logical structure is searched. For example, the user can input the maximum distance to the left as shown in G2, the maximum distance to the right as shown in G3, the maximum distance upward as shown in G4, and the maximum distance downward as shown in G5. For example, the input values ​​of G2, G3, G4, and G5 are searched for in a search range centered on the component "2. Prepaid Expenses," and only the component "100" is searched for and the logical structure is recognized. Note that the way the search range is specified and the search method can be any method. In that case, for example, if multiple components are searched for in the search range, the component closest to the component being searched can be recognized as the logical structure.

[0051] The parameters specified by the user in the logic window G1 require complex logic, as described above, making it difficult for the user to understand the effect of the parameters on the logic. Therefore, with regard to the parameters of the logic window G1 being edited, the logic flow management unit 102 executes the logical structure recognition logic execution unit 102-2 and the logic visualization I / F update unit 102-3, and the logic visualization I / F update unit 102-3 operates the logic visualization I / F C4 to intuitively present the user with the knowledge necessary for adjusting the logic parameters. In FIG. 9, C4, like F4 and F5 in the fourth embodiment, displays the maximum distance numerical value as a string or a rectangle, and displays the maximum distance with an arrow symbol to inform the user. Additionally, for example, G5 visually depicts the search range using a rectangle to inform the user.

[0052] When logic window G1 receives an input from the user, it executes logical structure recognition logic execution unit 102-2, logic visualization I / F update unit 102-3, and logic application result I / F update unit 102-4, and logic application result I / F update unit 102-4 updates logic application result I / F C5. For example, in Fig. 9, C5 presents the results of identifying the logical structure between the table side and the value to the user by overlaying a bounding box and a rectangle to match the shape of the component and an arrow symbol as the logical structure. <Effects of Example 5> According to this embodiment, the logic flow management unit accepts and outputs logic specified by the user from an input device, and by presenting rectangles, character strings, and symbols on an input table, the logic flow management unit visualizes the information necessary for adjusting the parameters of the logical structure recognition logic using the distance between components using a logic visualization I / F, and reflects the results of application of the logical structure recognition logic in the logic application result I / F, thereby reducing the burden on the user involved in creating the logic. <Configuration of information acquisition device 100 according to the sixth embodiment> The information acquisition device of the sixth embodiment includes, in addition to the configuration of the information acquisition device 100, an interface for supporting the creation of logic for recognizing the logical structure between the constituent elements, using the shapes of the constituent elements as clues.

[0053] In the sixth embodiment, in the information acquisition device 100, logic that recognizes the logical structure between the components on the front side using the shapes of the components as clues is displayed on the logic visualization I / F C4, and the application result of the logic is displayed on the application result I / F C5. Note that the explanation of parts that overlap with the explanation in the first embodiment will be omitted in part or in whole. Furthermore, in the first embodiment, the explanation of parts that overlap with the GUI of the logic flow management unit 102 shown in Figure 4 will be omitted in part or in whole. <Processing of the information acquisition device 100 according to the sixth embodiment> Fig. 10 is an explanatory diagram of an interface of the logic flow management unit 102 of the information acquisition device 100 in Example 6, which recognizes the logical structure between the components on the front side using the shapes of the components as clues. Also, in Fig. 10, the logic flow management unit 102 includes a logic window H1 that supports the user in creating logic that recognizes the logical structure between the components on the front side. Also, in Fig. 10, Table B is used as the input table for the sake of explanation, but this is not limited to Table B and can be applied to tables other than Table B.

[0054] H1 is a logic window related to logical structure recognition logic that uses the shape of the components as a clue to identify whether a logical structure exists between the components on the front side. In this example, it is assumed that in Table B, the "General" and "Characteristic X" cells are each recognized as components. Here, "General" and "Characteristic X" can be considered to have a parent-child logical relationship, but because they are located at different Y coordinates, the logical structure recognition logic using distance in Example 5 may not be able to recognize "General" and "Characteristic X" as a logical structure. Therefore, using the shape of the cells as a clue, the logical structure is identified based on the extent to which the vertical lengths of the parent cell "General" and the child cell "Characteristic X" overlap when viewed horizontally.

[0055] In H1, the user can specify the threshold for the overlap rate (overlap rate) between parent and child cells and the analysis direction. For example, as shown in H2, the analysis direction can be specified as "right" or "down." In this case, the analysis direction is the direction in which the degree of overlap of the child cell with respect to the parent cell is calculated. If the analysis direction is "right," the logical structure of the front side and the front side is identified, and if the analysis direction is "down," the logical structure of the table top and the table top is identified. In this embodiment, only the identification of the logical structure of the table side and the table side will be described, but it is also possible to recognize logical structures such as table top and table top, table top and value, and table side and value using a mechanism similar to this embodiment.

[0056] H3 calculates the overlap rate with the parent cell based on the child cell according to the analysis direction specified in H2, and can specify the boundary conditions for the calculated overlap rate. For example, according to the input value of H3, a logical structure is assigned when the overlap rate with the parent cell based on the child cell exceeds 80%.

[0057] The parameters specified by the user in the logic window H1 require complex logic, as described above, making it difficult for the user to understand the effect of the parameters on the logic. Therefore, with regard to the parameters of the logic window H1 being edited, the logic flow management unit 102 executes the logic structure recognition logic execution unit 102-2 and the logic visualization I / F update unit 102-3, and the logic visualization I / F update unit 102-3 provides the logic visualization I / F C4 to intuitively present the user with the knowledge necessary to adjust the logic parameters. In Figure 10, C4 can inform the user of the shape of the cells processed by the logic, as shown by H4, and can inform the user of the overlap rate using a character string, a rectangle, and a double-arrow symbol, as shown by H5.

[0058] When the logic window H1 receives an input from the user, it executes the logical structure recognition logic execution unit 102-2, the logic visualization I / F update unit 102-3, and the logic application result I / F update unit 102-4, and the logic application result I / F update unit 102-4 updates the logic application result I / F C5. For example, in Fig. 10, C5 presents the results of identifying the logical structure between the front sides to the user by overlaying arrow symbols as the logical structure. <Effects of Example 6> According to this embodiment, the logic flow management unit receives and outputs the logic specified by the user from the input device, and the logic flow management unit visualizes the information necessary for adjusting the parameters of the logical structure recognition logic based on the shape of the components using the logic visualization I / F, and reflects the results of applying the logical structure recognition logic between the front side and the back side in the logic application result I / F, thereby reducing the burden on the user involved in creating the logic. <Configuration of information acquisition device 100 according to the seventh embodiment> In addition to the configuration of the information acquisition device 100, the information acquisition device of Example 7 includes a logic selection unit 200 for assisting the user in creating logic by allowing the user to select logic from a pre-prepared logic list. <Processing of the information acquisition device 100 according to the seventh embodiment> FIG. 11 shows the logic flow management unit 102 and the logic selection unit 200 of the information acquisition device 100 in the seventh embodiment.

[0059] The user clicks C62 to display the logic selection section 200. Note that any means may be used to display the logic selection section 200. In this case, for example, the logic selection section 200 may be displayed on the right side of the logic flow editing I / F, or may be implemented as a separate application.

[0060] The logic selection unit 200 presents the user with a list of logic that can be added to the logic flow. The logic selection unit 200 may also include a button for inserting the logic that can be added, as well as an explanation and diagram of the logic that can be added. By clicking the button for inserting logic, for example, the user can add logic specified by the user to the logic flow. <Effects of Example 7> According to this embodiment, the logic flow management unit accepts and outputs logic specified by the user from an input device, and the logic flow management unit allows the user to select pre-prepared logic from the logic list selection unit, thereby reducing the burden on the user involved in creating logic. <Example of hardware configuration of computer 300 realizing information acquisition device> 12 shows an example of the hardware configuration of a computer 300 that realizes the information acquisition device 100. The computer 300 is made up of an input device 304, an output device 305, a communication interface 306, a storage device 301, an arithmetic unit 302, and a bus 303.

[0061] The input device 304 is a device through which a user inputs the input table 101a, the component recognition logic 101b, the logical structure recognition logic 101c, or an instruction to the information acquisition device 100. The input from the input device 304 may be stored in the storage device 301. The input device 304 may be, for example, a keyboard, a mouse, a touch panel, a microphone, a camera, or a scanner for issuing instructions to the logic flow management unit 102.

[0062] The output device 305 presents to the user the logic flow management unit 102, logic visualization I / F, logic application result I / F, and logic flow I / F output by the information acquisition device 100. The output device 305 may be, for example, a display.

[0063] The communication interface 306 is connected to a network and transmits and receives various data necessary for the operation of the computer 300. At this time, the input device 304 and the output device 305 can be connected by the communication interface 306.

[0064] The arithmetic unit 302 causes the computer 300 to perform calculations and execute logic flows according to electronic instructions. The arithmetic unit 302 may also include one or more calculation or processing units. The arithmetic unit 302 may be any device that operates according to electronic instructions. In this case, for example, a CPU (Central Processing Unit) may be used.

[0065] The storage device 301 functions as a work area for the arithmetic device 302. The storage device 301 records programs and data that execute the information acquisition device 100. The storage device 301 may be implemented in any medium. In this case, various media such as non-volatile memory, a hard disk, or volatile memory are possible. [Explanation of symbols]

[0066] 101 Input section 102 Logic Flow Management Department 102-1 Component recognition logic execution unit 102-2 Logical structure recognition logic execution unit 102-3 Logic visualization I / F update section 102-4 Logic application result I / F update unit 103 Output section

Claims

1. A logic creation support system for document logical structure recognition, The logic flow management unit is connected to an input / output device and manages the logic and logic flow of component recognition and logical structure recognition, the component recognition receives component recognition logic designated by the user in the logic flow management unit from the input device and outputs it, the logical structure recognition receives the output and logical structure recognition logic designated by the user in the logic flow management unit from the input device and outputs it, and the logic flow management unit reflects the logic visualization and logic application results in the interface, The logic flow management unit visualizes information necessary for parameter adjustment of the table side, table top, and value recognition logic of the component recognition logic using a logic visualization I / F, and reflects the result of application of the component recognition logic in a logic application result I / F. A logic creation support system characterized by:

2. A logic creation support system for document logical structure recognition, The logic flow management unit is connected to an input / output device and manages the logic and logic flow of component recognition and logical structure recognition, the component recognition receives component recognition logic designated by the user in the logic flow management unit from the input device and outputs it, the logical structure recognition receives the output and logical structure recognition logic designated by the user in the logic flow management unit from the input device and outputs it, and the logic flow management unit reflects the logic visualization and logic application results in the interface, The logic flow management unit visualizes information necessary for parameter adjustment of the logical structure recognition logic by indentation using a logic visualization I / F, and reflects the result of application of the logical structure recognition logic in a logic application result I / F. A logic creation support system characterized by:

3. A logic creation support system for document logical structure recognition, The logic flow management unit is connected to an input / output device and manages the logic and logic flow of component recognition and logical structure recognition, the component recognition receives component recognition logic designated by the user in the logic flow management unit from the input device and outputs it, the logical structure recognition receives the output and logical structure recognition logic designated by the user in the logic flow management unit from the input device and outputs it, and the logic flow management unit reflects the logic visualization and logic application results in the interface, The logic flow management unit visualizes information necessary for parameter adjustment of the logical structure recognition logic using the distance between the components using a logic visualization I / F, and reflects the result of application of the logical structure recognition logic in a logic application result I / F. A logic creation support system characterized by:

4. A logic creation support system for document logical structure recognition, The logic flow management unit is connected to an input / output device and manages the logic and logic flow of component recognition and logical structure recognition, the component recognition receives component recognition logic designated by the user in the logic flow management unit from the input device and outputs it, the logical structure recognition receives the output and logical structure recognition logic designated by the user in the logic flow management unit from the input device and outputs it, and the logic flow management unit reflects the logic visualization and logic application results in the interface, The logic flow management unit visualizes information necessary for parameter adjustment of the logical structure recognition logic based on the shapes of the components using a logic visualization interface, and reflects the results of application of the logical structure recognition logic in a logic application result interface. A logic creation support system characterized by:

Images