Information processing device and information processing program

Abstract

To provide an information processing device and an information processing program for reducing a user's setting load compared with a case of sequentially selecting and instructing a plurality of pieces of character string information for arithmetic operation written in a document to set an operation expression in the document.SOLUTION: An information processing device is provided with a processor. The processor reads a plurality of pieces of character string information written in a document S20, acquires feature information showing features related to operation using numerical information included in the plurality of pieces of character string information and arrangement information showing the arrangement of the positional relation of each of the plurality of pieces of character string information S30, and defines an operational expression that performs an operation by using a related operator between the plurality of pieces of character string information on the basis of the feature information and the arrangement information S40.SELECTED DRAWING: Figure 4

Description

Technical Field

[0001] The present invention relates to an information processing apparatus and an information processing program.

Background Art

[0002] Conventionally, an information processing apparatus that performs information processing for character recognition on an image obtained by optically reading a form has been known. For example, Patent Document 1 discloses a technique related to a form recognition method in which a computer creates a recognition template for performing OCR recognition from a form image. As steps executed by the computer, a first step of inputting a recognition template creation instruction for the form image based on a user operation, and when a registered first recognition template can be applied to the form image, applying it to obtain a result of OCR recognition, and displaying the recognition result and information on the first recognition template on a display screen; a second step; a third step of creating a second recognition template when the first recognition template cannot be applied to the form image, and displaying information on the second recognition template on the display screen; a fourth step of, on the display screen, performing confirmation and correction on the first recognition template or the second recognition template based on the user operation, and registering the first recognition template or the second recognition template as a setting for applying the first recognition template or the second recognition template to the recognition of the form image; and a fifth step of applying the second recognition template to the form image to obtain a result of OCR recognition and displaying the recognition result on the display screen.

[0003] Furthermore, Patent Document 2 discloses a document definition creation device comprising at least a storage unit and a control unit for creating a format definition for a document for character recognition. This technology is characterized in that the storage unit comprises a format definition storage means for storing the format definition of a source document and an image storage means for storing an image of a destination document read by an image reading device, and the control unit comprises an item position search means for searching for individual read items based on the format definition of the source document on the image of the destination document and an item mapping means for moving the read items of the source document based on the search results of the item position search means. [Prior art documents] [Patent Documents]

[0004] [Patent Document 1] Japanese Patent Publication No. 2021-047688 [Patent Document 2] Japanese Patent Publication No. 2015-184815 [Overview of the project] [Problems that the invention aims to solve]

[0005] Some forms have areas designed for numerical calculations, such as tabular input fields. Information processing devices that read multiple strings from these forms may need to verify that the strings containing numerical information are valid for numerical calculations. In this case, the system selects the numerical information contained in each of the strings and defines an calculation expression by setting operators between them. However, defining an calculation expression for each area of ​​the form, such as a tabular input field, increases the user's configuration burden.

[0006] The purpose of this disclosure is to provide an information processing device and an information processing program that can reduce the user's configuration burden compared to setting calculation formulas in a form by sequentially selecting and instructing multiple string information for numerical calculations written in the form. [Means for solving the problem]

[0007] To achieve the above objective, the first aspect is: Equipped with a processor, The aforementioned processor, Read multiple string information entries on the form, Obtain feature information indicating features related to calculations using numerical information contained in the aforementioned plurality of string information, and arrangement information indicating the arrangement of the positional relationships of each of the plurality of string information, Based on the aforementioned feature information and arrangement information, define an arithmetic expression that performs operations using operators related to multiple string information. It is an information processing device.

[0008] The second aspect is an information processing device according to the first aspect, The aforementioned processor, As the aforementioned feature information, category information indicating the type of unit used in the calculation is used. As the aforementioned calculation formula, a calculation formula is defined that uses numerical information corresponding to the category information.

[0009] The third aspect is an information processing apparatus according to the second aspect, The aforementioned processor, The category information used includes information indicating at least one of the following: quantity, amount, weight, and length.

[0010] The fourth aspect is an information processing device relating to any one of the first to third aspects, The aforementioned processor, As the aforementioned arrangement information, tabular information showing the positional relationship between multiple string information items adjacent to each other in at least one direction is used.

[0011] The fifth aspect is an information processing device relating to the fourth aspect, The aforementioned processor, As the arrangement information, information indicating the positional relationship of multiple string information contained in the setting area set in the form is used.

[0012] The sixth aspect is an information processing device relating to any one of the first to fifth aspects, The aforementioned processor, Based on the feature information and the arrangement information, the operators between the numerical information contained in the plurality of string information are estimated. Using the estimated operator, the calculation expression is defined which includes an operation term using the operator between any of the numerical information contained in the plurality of string information, and which shows the numerical information contained in other string information different from the plurality of string information as the calculation result.

[0013] The seventh aspect is, It is an information processing program, In the processor, Read multiple string information entries on the form, Obtain feature information indicating features related to calculations using numerical information contained in the aforementioned plurality of string information, and arrangement information indicating the arrangement of the positional relationships of each of the plurality of string information, Based on the aforementioned feature information and arrangement information, define an arithmetic expression that performs operations using operators related to multiple string information. It is an information processing program that handles tasks. [Effects of the Invention]

[0014] According to the first and seventh embodiments, compared to the case where a calculation formula is set on a form by sequentially selecting and instructing multiple string information for numerical calculations written on the form, the user's setting burden can be reduced. According to the second embodiment, there is an effect that the processing load when defining an arithmetic expression can be reduced compared to not considering the types of units used in the calculation. According to the third embodiment, compared to not using category information indicating the type of unit used in the calculation, the processing load when defining the calculation formula can be reduced. According to the fourth aspect, there is an effect that the processing load when determining the type of unit used in the calculation can be reduced as compared with the case where the tabular information in the form is not considered. According to the fifth aspect, there is an effect that the processing load can be reduced as compared with the case where an arithmetic expression for a plurality of character string information is set for the entire form. According to the sixth aspect, there is an effect that the processing load when defining an arithmetic expression can be reduced as compared with the case where the arithmetic terms and the arithmetic results are not associated with the numerical information included in the plurality of character string information.

Brief Description of Drawings

[0015] [Figure 1] It is a diagram showing a schematic configuration of a form system according to an embodiment. [Figure 2] It is a diagram showing an example of an electrical schematic configuration of an information processing apparatus according to an embodiment. [Figure 3] It is a diagram showing an example of a functional configuration of an information processing apparatus according to an embodiment. [Figure 4] It is a flowchart showing an example of a flow of information processing by an information processing program according to an embodiment. [Figure 5] It is a diagram showing an example of a confirmation screen regarding a reading frame confirmation process. [Figure 6] It is a diagram showing an example of a setting screen regarding a business check setting. [Figure 7] It is a diagram showing an example of a setting screen regarding an edit process of a business check setting. [Figure 8] It is a flowchart showing an example of a flow of a reading frame creation process. [Figure 9] It is a diagram showing an example of a reading frame creation screen. [Figure 10] It is a diagram showing an example of a display screen in a reading frame creation process. [Figure 11] It is an explanatory diagram regarding a unit extraction process. [Figure 12] It is a flowchart showing an example of a flow of a unit extraction process. [Figure 13] This is an explanatory diagram regarding the process of extracting units from OCR results. [Figure 14] This flowchart shows an example of the process for extracting units from OCR results. [Figure 15] This figure shows an example of a category table. [Figure 16] This flowchart shows an example of the category determination process. [Figure 17] This figure shows an example of a prediction table. [Figure 18] This figure shows an example of a user prediction table. [Figure 19] This flowchart shows an example of the process for defining an arithmetic expression. [Figure 20] This figure shows an example of a screen for confirming calculation formulas. [Figure 21] This figure shows an example of the screen for creating a reading frame. [Figure 22] This figure shows an example of the screen for creating a reading frame. [Modes for carrying out the invention]

[0016] Hereinafter, an example of an embodiment for carrying out the technology of this disclosure will be described in detail with reference to the drawings. Components and processes that perform the same operation, action, or function will be given the same reference numerals throughout the drawings, and redundant explanations may be omitted as appropriate. Each drawing is only a schematic representation to the extent that the technology of this disclosure can be fully understood. Therefore, the technology of this disclosure is not limited to the illustrated examples. Furthermore, in the embodiments, explanations of configurations not directly related to the present invention or well-known configurations may be omitted.

[0017] Figure 1 is a diagram showing the schematic configuration of the document system 10 according to this embodiment.

[0018] As shown in Figure 1, the form system 10 includes an information processing device 20, a client terminal 40, and an input device 60. These devices are connected to a network (not shown) and can communicate with each other via the network. This network could be the Internet, a LAN (Local Area Network), a WAN (Wide Area Network), or the like.

[0019] The information processing device 20 manages a series of processing steps, including performing OCR (Optical Character Recognition) on image data of multiple documents, including forms, input through the input device 60, and outputting the results of the OCR processing to a predetermined output destination. The specific configuration and operation of the information processing device 20 will be described later.

[0020] The client terminal 40 transmits various instructions related to OCR processing to the information processing device 20. These instructions include, for example, an instruction to start reading the information of the image data and an instruction to display the results of reading the information of the image data. The client terminal 40 also displays the results of the OCR processing performed by the information processing device 20 in response to the received instructions, as well as various information such as notifications related to the OCR processing. The client terminal 40 can be a general-purpose computer device such as a server computer or a personal computer (PC). Figure 1 shows only one client terminal 40, but it is not limited to this, and multiple client terminals 40 may be provided, and for example, different client terminals 40 may be used for different processes.

[0021] The input device 60 inputs image data to be processed by OCR into the information processing device 20. The input device 60 may include, for example, a server computer or a general-purpose computer device such as a PC, as well as an image forming apparatus having scanning, printing, and faxing functions. In addition to the input device 60, image data may also be input to the information processing device 20 from a client terminal 40.

[0022] Next, we will explain the overview of the reporting system 10.

[0023] The document system 10 is a system in which the information processing device 20 performs OCR processing on image data input through the input device 60 and outputs the results of the OCR processing to a predetermined output destination.

[0024] The information processing device 20 manages various processes in OCR processing, including (1) business design and operation verification, (2) data input, (3) data reading, (4) form identification, confirmation and correction, (5) reading result confirmation and correction, (6) business check, (7) data output, and (8) return. In this embodiment, OCR processing refers not only to the process of reading characters and symbols from image data, but also to post-processing such as character correction.

[0025] As an example of the management of various processes, (1) business design and operation verification, (2) data input, (3) data reading, (6) business check, and (7) data output are each automatically executed by the information processing device 20. Also as an example of the management of various processes, (4) form identification, confirmation, and correction, and (5) reading result confirmation and correction are each accepted by user input via the client terminal 40. Furthermore, as an example of the management of various processes, (8) the return process may be automatically executed by the information processing device 20 or accepted by user input via the client terminal 40.

[0026] (1) In the process of business design and operational verification, a job rule is created that includes read definition settings, output settings, and business check settings. In the read definition settings, as an example, the read range is set, which is the range in which the image data information is read in the process of "(3) Data Reading". More specifically, as an example, a definition may be set to read the item value, which is the value, from the right of the item that is extracted as the key. In the output settings, as an example, the file format and output destination of the output data to be output in the process of "(7) Data Output" are set. In the business check settings, as an example, the required input items and format such as the number of characters that can be entered in the form to be detected in the process of "(6) Business Check" are set.

[0027] (2) In the data input process, image data is received from the input device 60. The received image data is registered as a job, which is the execution unit of the "(3) Data Reading" process.

[0028] (3) In the data reading process, the information of the image data within the job is read using the job rules selected by the user from among the job rules created in the process of "(1) Business Design and Operation Verification". For example, in this process, the documents contained in the image data within the job are identified (hereinafter referred to as "document identification") and the characters and symbols within the reading range are read.

[0029] (4) In the process of form identification, confirmation, and correction, the image data within the job is divided into records that indicate the forms included in the job, based on the results of the form identification performed in the process of "(3) Data Reading". Subsequently, in this process, the divided records are displayed and the user's confirmation and correction of the form identification is accepted.

[0030] (5) In the process of confirming and correcting the reading results, the reading results of characters and symbols within the reading range performed in the process of "(3) Data reading" are displayed, and the user's confirmation and correction of the reading results is accepted.

[0031] (6) In the business process check, errors in each preceding process are detected by the business check settings included in the job rule for the job in question, which is selected by the user from among the job rules created in the process of "(1) Business Design and Operation Verification". The detection results may be presented to the user.

[0032] (7) In the data output process, output data is created using the output settings included in the job rule for the job selected by the user from among the job rules created in the process of "(1) Business design and operation verification," and the created output data is output to a predetermined output destination.

[0033] (8) In the rollback process, the OCR process is rolled back to a process one or more steps prior to the process that was executed. Rollback is instructed by the user from the client terminal 40 during the execution of various processes such as "(4) Form Identification Confirmation Correction" and "(5) Reading Result Confirmation Correction". Rollback is also instructed by the administrator from the administrator's client terminal 40 in accordance with the results of the administrator's check performed between the processes of "(6) Business Check" and "(7) Data Output".

[0034] In the OCR processing described above, the "(1) Business Design and Operational Verification" process is executed before the "(3) Data Reading" and subsequent processes are performed, that is, before the reporting system 10 is put into operation. Furthermore, the "(1) Business Design and Operational Verification" process may also be executed while the reporting system 10 is in operation and the "(3) Data Reading" and subsequent processes are being performed. As an example, the job rules created in the "(1) Business Design and Operational Verification" process before the reporting system 10 is put into operation can be appropriately modified according to the results of the "(5) Reading Result Confirmation and Correction" process during the operation of the reporting system 10.

[0035] (Information processing device) Next, an example of the configuration of the information processing device 20 will be described with reference to Figure 2. Figure 2 is a diagram showing an example of the schematic electrical configuration of the information processing device 20 according to this embodiment. The information processing device 20 can be, for example, a server computer or a general-purpose computer device such as a PC.

[0036] Specifically, as shown in Figure 2, the information processing device 20 includes a computer main unit 21. The computer main unit 21 includes a CPU (Central Processing Unit) 22, RAM (Random Access Memory) 23, ROM (Read-Only memory) 24, a storage unit 25, and input / output ports (I / O) 26. The CPU 22, RAM 23, ROM 24, storage unit 25, and I / O 26 are connected to each other via a bus.

[0037] Furthermore, I / O26 is connected to various functional units, including a communication unit 27 that enables communication with external devices, an operation input unit 28 that enables user operation input, and a display unit 29 that enables image display. These functional units are able to communicate with the CPU 22 via I / O26.

[0038] The computer body 21 may be configured as a sub-control unit that controls the operation of a part of the information processing device 20, or as part of the main control unit that controls the operation of the entire information processing device 20. Some or all of the blocks of the computer body 21 may use integrated circuits such as LSIs (Large Scale Integration) or ICs (Integrated Circuit) chipsets. Individual circuits may be used for each of the above blocks, or circuits that integrate some or all of them may be used. The above blocks may be provided as a single unit, or some blocks may be provided separately. Furthermore, parts of each of the above blocks may be provided separately. For the integration of the computer body 21, dedicated circuits or general-purpose processors may be used, not just LSIs.

[0039] The memory unit 25 stores an information processing program 25P that causes the information processing device 20 to function as the information processing device of this disclosure. The CPU 22 reads the information processing program 25P from the memory unit 25, loads it into the RAM 23, and executes the processing. As a result, the information processing device 20 that has executed the information processing program 25P operates as the information processing device of this disclosure. The information processing program 25P may be provided on a recording medium such as a CD-ROM. The specific processing of the information processing device 20 will be described later.

[0040] As the storage unit 25, for example, an auxiliary storage device such as an HDD (Hard Disk Drive), SSD (Solid State Drive), or flash memory is used. The storage unit 25 stores an information processing program 25P for realizing the information processing according to this embodiment. The CPU 22 reads the information processing program 25P from the storage unit 25, loads it into the RAM 23, and executes the processing. As a result, the information processing device 20 that has executed the information processing program 25P operates as the information processing device of this disclosure.

[0041] The information processing program 25P may be stored in the ROM 12C. Alternatively, the information processing program 25P may be pre-installed in the information processing device 20, for example. The information processing program 25P may also be implemented by storing it on a non-volatile storage medium, or by appropriately installing program information distributed via a network (not shown) into the information processing device 20. Examples of non-volatile storage mediums include CD-ROMs (Compact Disc Read Only Memory), magneto-optical disks, HDDs, DVD-ROMs (Digital Versatile Disc Read Only Memory), flash memory, and memory cards.

[0042] Furthermore, the memory unit 25 also stores a system program 25S for implementing each OCR processing function in the information processing device 20. The CPU 22 reads the system program 25S from the memory unit 25, loads it into the RAM 23, and executes the OCR processing. As a result, the information processing device 20, having executed the system program 25S, operates in a way that enables it to perform system functions based on OCR processing.

[0043] In this embodiment, the information processing program 25P and the system program 25S are described as separate programs, but the information processing program 25P may be executed as part of a process included in the system program 25S.

[0044] The storage unit 25 described above also stores a database 25D containing various types of information usable by the information processing device 20. The information stored in the database 25D is not limited to being pre-stored in the storage unit 25. For example, it may be stored in an external device (not shown) and retrieved from that external device via a communication line.

[0045] The communication unit 27 is connected to a communication network and is configured to enable communication between the information processing device 20 and external devices. "Communication network" is a concept that includes networks that enable data exchange between devices via communication lines using at least one of wired and wireless connections. Examples include narrow-area communication networks (e.g., LAN: Local Area Network) that enable data exchange at corporate locations, and wide-area communication networks such as the Internet (e.g., WAN: Wide Area Network) that enable data exchange via public communication lines.

[0046] The operation input section 28 is equipped with, for example, a device for operation input such as a keyboard or mouse.

[0047] The display unit 29 may be, for example, a liquid crystal display (LCD) or an organic EL (electroluminescence) display. The display unit 29 may also be a touch panel having the functions of the operation input unit 28. The operation input unit 28 and the display unit 29 receive various instructions from the user of the information processing device 20. The display unit 29 displays various information such as the results of processing performed in response to the instructions received from the user, and notifications regarding the processing.

[0048] Figure 3 is a block diagram showing an example of the functional configuration of the information processing device 20 according to this embodiment. As shown in Figure 3, the CPU 22 of the information processing device 20 according to this embodiment functions as the information processing device of the disclosure by executing the information processing program 25P. The information processing device of the disclosure includes functional units that function as a reading unit 220, an acquisition unit 222, a definition unit 224, and a display control unit 226, respectively.

[0049] The reading unit 220 is a functional unit that reads an image of a paper document and also reads the string information written on the document. In this embodiment, the string information is read by obtaining a reading result (string information) that has been corrected or identified using the recognition result obtained by character recognition of an image of a document on which a string of characters is written. Specifically, the character example information is obtained at the end of the OCR processing in the information processing device 20, up to the processing of confirming and correcting the reading result (process (5) shown in Figure 1).

[0050] The acquisition unit 222 is a functional unit that acquires feature information and arrangement information for multiple string information read by the reading unit 220. The acquisition unit 222 acquires feature information and arrangement information for the string information read by the reading unit 220 according to predetermined acquisition conditions.

[0051] The acquisition conditions indicate the conditions for acquiring feature information and arrangement information for multiple string information read by the reading unit 220. The acquisition unit 222 includes a function to extract these acquisition conditions from the storage unit 25 (for example, the database 25D).

[0052] Feature information is information that indicates features related to calculations using numerical information contained in multiple string information read by the reading unit 220. When it is expected that numerical calculations will be performed on multiple string information using the numerical values ​​indicated in the string information, the feature information may include information (category information) indicating the type of unit used in the calculation (hereinafter referred to as category) as a feature related to the calculation for at least one of the multiple string information. In other words, feature information indicates what kind of numerical calculation will be performed on the multiple string information, and includes category information indicating the type of unit used in the calculation. Examples of such category information include information indicating values ​​of each type: number, amount, weight, and length. For example, if multiple strings related to the price of goods include a string for the price of one item and its quantity, it is expected that calculations will be performed by summing the sum of arithmetic operations performed on each string information. Therefore, the correspondence between string information and category information, which indicates which category it belongs to, is one example of a condition for acquiring feature information.

[0053] The arrangement information is information that indicates the positional relationship (e.g., arrangement) of each of the multiple string information read by the reading unit 220. Multiple string information that are the subject of calculations are often close to each other. Therefore, in this embodiment, information indicating the positional relationship of each of the multiple string information is obtained. One example of arrangement information is tabular information that shows the positional relationship of multiple string information that are adjacent in at least one direction. In addition, the arrangement information can be information that shows the positional relationship of multiple string information contained in a setting area set in the form. Therefore, the conditions for applying information indicating the positional relationships of multiple adjacent string information, the correspondence in a tabular format, and the positional relationships of multiple string information contained in a set setting area are examples of acquisition conditions for feature information.

[0054] The definition unit 224 is a functional unit that defines arithmetic expressions that perform operations using operators related to multiple string information read by the reading unit 220. For example, the definition unit 224 estimates operators between numerical information contained in multiple string information based on feature information (category information) and placement information. The definition unit 224 also defines an arithmetic expression that uses the estimated operators to perform operations between any multiple numerical information contained in the multiple string information, and the numerical information contained in other string information that is different from the other multiple string information as the result of the operation. The definition unit 224 includes a function to extract definition conditions used when defining arithmetic expressions from the storage unit 25 from the storage unit (for example, the database 25D).

[0055] The definition conditions include the conditions for estimating operators between numerical information contained in multiple string information based on feature information (category information) and placement information. Furthermore, the definition conditions include the conditions for assigning operators between multiple string information and estimating the arithmetic expression. Specifically, one example of a definition condition is applying information indicating the correspondence between category information (e.g., information indicating units) and operators to multiple string information according to placement information.

[0056] Furthermore, the definition unit 224 includes a function to verify the suitability of the above calculation formula. Conformance indicates the degree of consistency between the multiple string information read and the information showing the result of an operation performed using a defined arithmetic formula with the multiple string information read by the reading unit 220. The degree of consistency can be measured using an index that indicates whether the multiple string information and the operation result match. In this embodiment, the definition unit 224 verifies conformance by determining whether the string information read by the reading unit 220 is consistent with the information from the defined arithmetic formula. That is, the definition unit 224 verifies whether the defined arithmetic formula is usable by determining whether the read string information is consistent with the information from the arithmetic formula.

[0057] The display control unit 226 is a functional unit that displays information (for example, an arithmetic expression) indicating the definition result defined in the definition unit 224 on the display unit 29.

[0058] Next, the operation of the information processing device 20 according to this embodiment will be described. In this embodiment, the business check process (process (6) shown in Figure 1) of the OCR processing performed on the image data of a form, which is executed in the information processing device 20 and input through the input device 60, will be described. In the business check process, the main process is to define arithmetic expressions related to the reading results of multiple strings read from the form. That is, in the business check process, arithmetic expressions for multiple strings read from the form are defined using business check settings included in the job rule for the job selected by the user from among the pre-created job rules.

[0059] Figure 4 is a flowchart showing an example of the information processing flow by the information processing program 25P according to this embodiment.

[0060] First, when the information processing device 20 is instructed to start the information processing program 25P, the CPU 22 executes the following steps.

[0061] Step S10 performs initial settings related to business checks, and in this embodiment, numerical calculation checks. The initial settings include various information such as the business check name to identify the content of the business check.

[0062] In step S20, the reading result is obtained by acquiring string information read from a document containing a string. The string information indicating the reading result can be obtained from the execution result at the end of the OCR processing performed by the execution of the system program 25S in the information processing device 20, up to the reading result confirmation and correction process (process (5) shown in Figure 1). The process in step S10 is an example of the function of the reading unit 220 shown in Figure 3.

[0063] In step S30, information indicating the arrangement, unit, and category of the string information in the read form is obtained. The processing in step S30 is an example of the function of the acquisition unit 222 shown in Figure 3.

[0064] In step S32, a confirmation process for the read frame is executed, which asks the user whether to continue the business check process using the read frame extracted when acquiring information in step S30.

[0065] Figure 5 shows a confirmation screen 83, which is an example of a screen displayed during the reading frame confirmation process. The confirmation screen 83 includes a message display area 830 prompting the user to confirm whether to continue the business check process. The confirmation screen 83 also includes an OK button 838 for instructing the user to continue processing and a Cancel button 839 for canceling the process. When the OK button 838 is pressed by the user, the process moves to step SS40. When the Cancel button 839 is pressed, the information up to that point is discarded, and the process returns to step S10, or the processing routine is forcibly terminated.

[0066] As described later, the process in step S32 reduces unnecessary processing caused by proceeding with processing on reading frames that are not subject to numerical calculations by having the user confirm if the extracted reading frame is a frame that is not subject to numerical calculations.

[0067] In step S40 of Figure 4, information indicating the arrangement, unit, and category of the string information in the read form is obtained, and an arithmetic expression is defined. That is, the CPU 22 defines an arithmetic expression based on the information indicating the arrangement, unit, and category of the string information obtained in step S20. The processing in step S40 is an example of the function of the definition unit 224 shown in Figure 3.

[0068] In step S50, the arithmetic expression defined in step S40 is displayed, and the series of processes by the information processing program 25P is completed. The process in step S50 is an example of the function of the display control unit 226 shown in Figure 3.

[0069] Next, we will explain in detail the information processing that includes the process of defining the above-mentioned arithmetic formula. In the information processing device 20, with regard to business checks included in job rules, if the user instructs the user to configure business rules after the job rule configuration is complete, the business check configuration process is executed.

[0070] Figure 6 shows an example of the settings screen for the business checklist related to business check settings.

[0071] When the process for setting up business checks is executed, the settings screen 80 shown in Figure 6 is displayed. The settings screen 80 is a screen that allows the user to confirm the type and settings of business checks to be executed. The settings screen 80 displays a business checklist 800 that shows a list of the types of business checks to be executed according to the settings. Figure 6 shows the business checklist 800 when required input checks, numerical calculation checks, date checks, and list checks have been added as types of business checks. The business checklist 800 is registered as a record associated with each business check to be executed, with priority, business check name, item name, and type. Priority is information that indicates the execution order of the business checks included in the business checklist 800, and business check name is information that indicates the name of the business check. Item name is information that indicates the item to be verified in the business check, and type is information that indicates the type of business check. Note that the business checklist 800 shown in Figure 6 is not displayed during the initial processing of business check settings.

[0072] On the settings screen 80, users can operate the settings for the business checklist 800 and the business checks for each type using buttons. The settings screen 80 includes an item addition button 802 for adding a new business check to the business checklist 800. Furthermore, the settings screen 80 includes an OK button 803 for finishing the settings for the business checklist 800, and a cancel button 804 for canceling. For the business check settings, pressing the edit button (shown as a three-dot image in Figure 6) 801 allows the user to select one of the editing processes, such as editing, deleting, or changing the priority. When the edit button 801 is pressed and an editing process is selected, the system proceeds to the editing process for the selected business check setting.

[0073] The following describes the process for adding a new business check (the initial business check setup process). When the Add Item button 802 is pressed, the process for adding a new business check is executed, and the settings for the new business check are configured.

[0074] Figure 7 shows an example of a settings screen for editing the settings of a new business check. When the process to add a business check is executed, the settings screen 81 is displayed. The settings screen 81 displays input fields 811 for entering information about the business check. Input fields 811 are editable. Input fields 811 can be displayed in a dropdown format, for example, allowing the user to select the content to be entered from a predetermined list of items.

[0075] The settings screen 81 includes an OK button 818 to finish setting up the business checks and a Cancel button 819 to cancel. When the OK button 818 is pressed, the process returns to setting up the business checklist, and the business checklist settings screen 80 shown in Figure 6 is displayed. If a new business check item is created, the contents of the created business check are added to the business checklist 800. On the other hand, when the Cancel button 819 is pressed, the settings are discarded, the process returns to setting up the business checklist, and the business checklist settings screen 80 shown in Figure 6 is displayed.

[0076] Note that the process of setting the information entered in input field 811 is an example of the process performed in step S20.

[0077] The settings screen 81 includes an input field 812 below the input field 811 for entering conditional expressions for numerical calculation checks. The input field 812 is editable. Each field in the input field 812 can be configured by the user.

[0078] However, if users have to input conditional expressions for numerical calculation checks one by one, the burden on the user increases. Therefore, in this embodiment, the computer automatically defines calculation expressions and proposes candidate calculation expressions as follows, thereby supporting the user's settings for numerical calculation checks.

[0079] Specifically, to the right of the input field 812 shown in Figure 7, an instruction button 813 is displayed to assist in entering conditional expressions (calculation expressions). When the instruction button 813 is pressed, processing related to setting conditional expressions for numerical calculation checks in the report is started. In this process, a reading frame is created for the report, and a conditional expression for performing numerical calculation checks in the created reading frame is defined.

[0080] First, the information processing device 20 acquires information to be used in defining the conditional expression as an arithmetic expression, that is, it acquires the reading result (string information) (step S20 in Figure 4), and acquires the arrangement, unit, and category (feature information and arrangement information) of the string (step S30 in Figure 4).

[0081] The acquisition of information regarding the arrangement, units, and categories of strings (feature information and arrangement information) is achieved through the process of creating a reading frame in the form.

[0082] Figure 8 is a flowchart showing an example of the process flow for creating a reading frame included in the information processing of information processing program 25P.

[0083] First, when the information processing device 20 is instructed to start the process for creating a read frame, which is included in the information processing program 25P, the CPU 22 executes the following steps.

[0084] In step S100, CPU22 extracts reading frames from the read form. The reading frame extraction process may extract a range specified by the user as the reading frame, or it may automatically extract a range containing multiple strings.

[0085] Figure 9 shows an example of the screen for creating a reading frame, which is displayed during the process of creating a reading frame. As shown in Figure 9, the creation screen 82 displays a reduced image (for example, a thumbnail image reduced from a scanned image of the form) 820 that allows users to view the entire form for setting up business checks (i.e., numerical calculation checks). The creation screen 82 also displays a detailed image of the form (for example, a magnified image of a portion of the scanned image of the form) 822. Furthermore, the creation screen 82 displays an instruction button 821 that signals the user to switch to automatic extraction mode, which executes a process to automatically extract the reading frame.

[0086] The image 820 displayed on the screen 82 for creating the reading frame is not limited to the read image (for example, a scanned image of a form) 820, but may also be a format image of the form for which business checks are set. This format image may be obtained from data pre-stored in the storage unit 25, or it may be obtained from an external device via communication.

[0087] When the instruction button 821 described above is pressed by the user, the process shown in Figure 8 is executed. Pressing the instruction button 821 by the user is not mandatory; the process shown in Figure 8 may be executed automatically. Figure 9 also shows an example of the extraction result displayed on the screen in step S100 as a reading frame 823.

[0088] Next, in step S102 shown in Figure 8, the CPU 22 performs a read frame determination process. Specifically, it determines whether the image inside the read frame extracted in step S100 contains multiple string information written in a tabular format, thereby determining the possibility that it contains string information that is subject to numerical calculation checks as part of the business check. This is because, compared to multiple string information unrelated to the arrangement, multiple string information written in a tabular format is more likely to be definable by calculation formulas using numerical calculations.

[0089] In step S104, it is determined from the judgment result in step S102 whether the image inside the reading frame is written in a table format or not. If the judgment in step S104 is negative, the process moves to step S106; if the judgment is positive, the process moves to step S110.

[0090] In step S106, the extraction results from step S100 are displayed on the screen, and then this processing routine is terminated.

[0091] On the other hand, if the image inside the reading frame is described in tabular format, the processing from step S110 onward will obtain information on the arrangement, unit, and category (feature information and arrangement information) related to the string information.

[0092] First, in step S110, the counter variable n is set to "1" (n=1). In the next step, S112, information indicating the unit of the string information in the nth column of the first row in the tabular format is extracted. In the next step, S114, a category is determined for each string of string information. Details of the process for extracting information indicating the unit (hereinafter referred to as the unit extraction process) will be described later, but in the unit extraction process, a process is executed to extract string information of candidates that will be the unit in the calculation expression. Also, details of the process for determining the category (hereinafter referred to as the category determination process) will be described later, but in the category determination process, if the string information indicating the unit matches a predetermined keyword, it is determined to be a category indicated by the unit corresponding to that keyword.

[0093] In step S116, the presence or absence of the next column is determined by checking whether or not string information for the column remaining in the first row of the tabular format exists. If the result in step S116 is positive, the counter variable n is incremented (n=n+1) in step S118 and the process returns to step S112. On the other hand, if the result in step S116 is negative, the processing routine ends in step S120 after displaying the result of the determination in step S114, i.e., the category information (Figure 10).

[0094] Figure 10 shows an example of the display screen during the process of creating a reading frame. Figure 10 is a diagram showing an example of a screen that displays information including the results of the reading frame extraction process (step S100), which is designated as screen 85.

[0095] As shown in Figure 10, screen 85 displays an image 850 that includes at least the form area (reading frame 823) to which the calculation formula is to be defined. Screen 85 also includes a display area 852 for displaying category candidates and a display area 854 for displaying OCR results. Furthermore, screen 85 includes an OK button 858 as an instruction button to proceed with the process and a Cancel button 859 for instructing to stop the process.

[0096] The display area 852, which displays category candidates, is configured to allow the user to edit the information displayed as category candidates. Furthermore, the display area 854, which displays OCR results, is configured to allow the user to link the OCR results with the document image by indicating the position of the item name, thereby highlighting or marking the corresponding area on the reading frame 823 on the image 850.

[0097] Next, the unit extraction process described above (step S112 in Figure 8) will be explained in detail. Figure 11 is an explanatory diagram regarding the unit extraction process described above. The unit extraction process searches for and extracts string information that will be used as a unit in the calculation formula from within the reading frame. For example, candidate strings for units are searched for and extracted from the surrounding string information in the small frame units of the table format. In this embodiment, three types of search processes are performed. In the example shown in Figure 11, the first search process searches whether candidate string information for a unit exists in the small frame 823A which represents "1" string information. The second search process searches whether candidate string information for a unit exists in the small frame 823B located one frame above the first row frame. The third search process searches whether candidate string information for a unit exists in the small frame 823C located one frame to the right of the first row frame.

[0098] Figure 12 is a flowchart showing an example of the unit extraction process (step S112 in Figure 8) described above. When the information processing device 20 is instructed to start the unit extraction process included in the information processing program 25P, the CPU 22 executes the following steps.

[0099] In step S200, CPU22 obtains the OCR result (string information) of the first line within the extracted reading frame. In the next step S202, it extracts the unit (string information indicating the unit) from the obtained OCR result (string information) of the first line within the frame.

[0100] Figure 13 is an explanatory diagram regarding the process of extracting units from the OCR results (processing in step S202). In step S202, words are extracted from the beginning or end (left or right side) of the string information until a string information representing a number is obtained from the acquired string information. Figure 13 shows an example of extracting words from the end (right side) of the string information for "10 items" and an example of extracting words from the beginning (left side) of the string information for "¥550". When extracting words from the beginning (left side) of the string information, even just one character is sufficient. This is because there is a high possibility that a predetermined symbol or character, such as a unit of money, will be placed at the beginning.

[0101] Figure 14 is a flowchart showing a detailed example of the process for extracting units from the OCR results (processing in step S202). In step S300, CPU22 obtains the OCR result (string information) as part of the unit extraction process, and saves the obtained OCR result as string information indicating the unit and string information indicating the unit candidate. Next, in step S302, it determines whether the obtained string information is an empty string, and if the determination is positive, this processing routine terminates.

[0102] On the other hand, if the OCR result is not an empty string, the CPU 22 makes a negative judgment in step S302 and executes a process to extract the string information of the unit candidate and store it in the string information indicating the unit. Specifically, in step S304, the counter variable n is set to "1", and in the next step S306, the nth character from the end of the OCR result (string information) of the first line is extracted and stored as string information indicating the unit candidate.

[0103] Next, in step S308, it is determined whether or not there is a numeric string among the unit candidates. If the determination is negative, in step S310, the string information of the unit candidate is stored in the string information indicating the unit, and in the next step S312, the counter variable n is incremented (n=n+1), and the process returns to step S306.

[0104] On the other hand, if there are no numeric strings among the unit candidates and the judgment in step S308 is positive, in step S314 it is determined whether the string information stored in the unit is an empty string, and if the judgment is negative, this processing routine is terminated. On the other hand, if the judgment is positive, in step S316 the first character of the string is saved as the string information of the unit candidate. Next, in step S318 it is determined whether there is a numeric string among the unit candidates. If the judgment is positive, this processing routine is terminated, and if the judgment is negative, in step S320 the string information of the unit candidate is stored in the string information indicating the unit, and then this processing routine is terminated.

[0105] Once the string information indicating the unit is extracted, step S204, shown in Figure 12, determines whether the string information indicating the unit is an empty string. If the determination is positive, the process proceeds to step S210; if the determination is negative, step S206 stores the stored unit information as the string information indicating the category unit, and the processing routine ends.

[0106] In step S210, the OCR result is obtained from a position one frame above the reading frame of the first line. Next, in step S212, it is determined whether the string information indicating the unit is an empty string or not. If the determination is negative, the process moves to step S216. If the determination is positive, in step S214, the OCR result obtained in step S210 is stored as string information indicating the category unit, and this processing routine ends.

[0107] In step S216, it is determined whether a different reading frame is detected to the right of the reading frame. If the determination is negative, the process proceeds to step S218. If the determination is positive, in step S224, a string indicating an empty string is stored as string information indicating a category unit, and this processing routine ends.

[0108] In step S218, the OCR result is obtained from the position one frame to the right of the reading frame of the first line. Next, in step S220, it is determined whether the OCR result is an empty string or not. If the determination is negative, the process moves to step S224. If the determination is positive, in step S222, the OCR result obtained in step S218 is stored as string information indicating the category unit, and this processing routine ends.

[0109] Next, the category determination process described above (step S114 in Figure 8) will be explained in detail. In the category determination process, the category information indicating the type of unit is checked and determined for the string information that serves as the unit in the calculation expression by referring to a predetermined category table.

[0110] Figure 15 shows an example of a category table used in the category determination process described above. The category table is stored in the database 25D of the storage unit 25, and the stored prediction table can be retrieved and used.

[0111] Candidate unit strings can be classified according to the type of unit used in the operation expression. For example, Figure 15 shows examples of string information (i.e., keywords) belonging to each of the categories: number, amount, weight, and length. Therefore, by identifying keywords that match the candidate unit string information, the candidate unit string information can be classified into the category to which the identified keyword belongs.

[0112] Figure 16 is a flowchart showing an example of a detailed flow of the category determination process (the process in step S114 of Figure 8). In this embodiment, the category determination process is performed using the category table shown in Figure 15.

[0113] In step S400, the CPU 22 obtains string information indicating the unit as part of the category determination process. Next, in step S402, it searches for keywords that match the obtained string information using the category table (Figure 15). Then, in step S404, it stores the search results as category information, that is, it stores the category name, and terminates this processing routine.

[0114] If the relevant keyword cannot be found during the processing in step S402, information indicating "no category" should be stored as category information.

[0115] Next, we will explain in detail the process of defining the above-mentioned arithmetic expression (step S40 in Figure 4). When the information processing device 20 obtains the information used to define the conditional expression as an arithmetic expression as described above, it defines the arithmetic expression using the obtained information. In this arithmetic expression definition process, the device predicts an arithmetic expression including operators by referring to a predetermined prediction table using the various information obtained as described above.

[0116] Figure 17 shows an example of a predetermined prediction table used to predict an arithmetic expression that includes operators used in the definition process of the arithmetic expression. The prediction table is stored in the database 25D of the storage unit 25, and the stored prediction table can be retrieved and used.

[0117] An arithmetic expression is formed by placing operators between multiple terms. Each of these terms can be classified into a category, and the combination of categories allows for the pre-determined possibility of which of the four basic arithmetic operators is being used. The prediction table shown in Figure 17 pre-determines the operators between the left-hand and right-hand terms as prediction operators. Based on information that defines the relationships between categories, such as statistical definition frequencies and the validity of the categories, the prediction operators can be selected from, for example, the four basic arithmetic operations (addition, subtraction, multiplication, and division). In the example in Figure 17, as shown in the second record, when the categories of both the left-hand and right-hand terms are "numbers," the multiplication operator "×" is pre-determined.

[0118] The operators used in the prediction table include arithmetic operators, but are not limited to these; other operators can also be used. For example, division (÷) is used when performing numerical calculations such as installment payments, daily calculations, distributions, and rationing, while subtraction (-) is used when applying coupon discounts in campaigns. While multiplication (×) is frequently used in monetary calculations, addition (+) is used when considering the sum of multiple transition values. When determining these operators, in addition to using categories, information indicating the type of numerical calculation may be added, and the operators may be determined based on this information.

[0119] Furthermore, the prediction table is not limited to always using predetermined prediction operators. For example, it may be updated in accordance with increases or decreases in the user's defined frequency. Specifically, a history of business check definitions can be stored, and operators defined in that history at a frequency exceeding a predetermined threshold, i.e., operators for combinations of categories between the left-hand and right-hand terms, can be updated as prediction operators. In addition, the prediction table updated by the user's defined frequency may be created as a separate user table from the pre-stored prediction table.

[0120] Figure 18 shows an example of a prediction table (user table) that has been updated in accordance with increases or decreases in the frequency of user definitions. In Figure 18, for the second record, a definition has been made in the definition history where the user's definition frequency has exceeded the threshold, and the operator has been changed from multiplication "×" to addition "+".

[0121] Furthermore, the user table is not limited to using the history of business check definitions. For example, it is possible to update the user table through learning from the operation of the information processing device.

[0122] Figure 19 is a flowchart showing a detailed example of the process for defining an arithmetic expression (step S40 in Figure 4). In this embodiment, the arithmetic expression definition process is performed using the prediction table shown in Figure 17.

[0123] As part of the process of defining the calculation formula, CPU22 obtains the category information for each row of multiple string information in step S500. Next, in step S502, it searches for a prediction operator in the prediction table (Figure 17) that matches the combination of category information obtained. Next, in step S504, it sets and displays the calculation formula using the operator in the search result. Next, in step S506, it defines the calculation formula after user confirmation, stores the definition result, and terminates this processing routine.

[0124] Figure 20 shows an example of a confirmation screen for the arithmetic expression that is displayed during the arithmetic expression definition process. As shown in Figure 20, the confirmation screen 84 displays an image 840 that includes at least the form area (reading frame) to which the calculation formula is to be defined. The confirmation screen 84 also includes a display area 842 that displays the created calculation formula in detail. Furthermore, the confirmation screen 84 includes an OK button 848 for instructing the user to define the calculation formula, and a Cancel button 849 for instructing the user to cancel the definition.

[0125] Display area 842 includes display areas for the left-hand side term, the right-hand side term, and the predictive operator. The display areas for the left-hand and right-hand side terms are configured with text boxes or similar elements to allow user modification instructions. The same predictive operator is set for each row in the table because the category of the string information in the reading frame is the same. This set predictive operator is configured to allow user modification instructions by enabling selection of other operators via pull-down menus or similar elements. While the example in Figure 20 shows the case where the same predictive operator is set due to the same category, the system may be configured to allow modification of the operator for each row in the table.

[0126] Next, we will explain the verification of the above-mentioned arithmetic formula. In reports, mathematical formulas that rely on numerical calculations often include both the calculation terms (the left-hand and right-hand terms mentioned above) and the calculation result term. Therefore, it is possible to verify the calculation formula derived through estimation by using the calculation terms and calculation result terms included in the report. By using this information about the calculation terms and calculation result terms, it is possible to improve the accuracy of the defined calculation formula compared to defining the formula without verification.

[0127] For example, in the example shown in Figure 9 above, the tabular reading frame 823 contains string information for the calculation term but not for the calculation result term. However, another frame contains string information for the calculation result term (the "Subtotal" column in Figure 9). In this embodiment, it is possible to set up other frames related to the calculation expression in order to improve the accuracy of the defined calculation expression.

[0128] Figure 21 shows an example of a screen for creating a reading frame to improve the accuracy of calculation formulas. As shown in Figure 21, the creation screen 86 displays the scanned image (image 820) and the reading frame 823, similar to the creation screen 82 shown in Figure 9. A reading frame 860 for verifying the calculation formula is created at a different position from the reading frame 823.

[0129] The process for creating a read frame 860 for arithmetic expression verification may include, when defining the arithmetic expression, a process for further setting up a frame containing some terms of the arithmetic expression in order to verify the predicted arithmetic expression. This process for creating a read frame for arithmetic expression verification may also be performed in the process for creating a read frame 823. Specifically, the process for creating a read frame for arithmetic expression verification may extract a range specified by the user as read frame 860, or it may automatically extract another range containing string information (e.g., numerical information) at a different position from read frame 823 and allow the user to select it.

[0130] Furthermore, as shown in Figure 22 as an example of the creation screen 87, if the reading frame 870 contains both the string information of the operation term and the string information of the operation result term, the operation formula can be estimated by estimating which of the string information in the reading frame 870 corresponds to the string information of the right-hand side term, left-hand side term, operation result term, etc.

[0131] As described above, this embodiment makes it possible to reduce the user's configuration burden compared to the case where the user sequentially selects and instructs multiple string information for numerical calculations written on the form to set the calculation formula on the form.

[0132] The above describes an information processing device based on an embodiment. The embodiment may take the form of a program that causes a computer to execute the functions of each part of the information processing device. The embodiment may also take the form of a storage medium that is readable by a computer and stores this program.

[0133] Furthermore, the configuration of the information processing device described in the above embodiment is merely an example, and may be modified as needed without departing from the main purpose.

[0134] Furthermore, the program processing flow described in the above embodiment is just one example, and unnecessary steps may be deleted, new steps added, or the processing order rearranged, as long as it does not deviate from the main purpose.

[0135] Furthermore, although the above embodiment describes a case in which the process according to the embodiment is realized by a software configuration using a computer by executing a program, the embodiment is not limited to this. The embodiment may also be realized by a hardware configuration or a combination of a hardware configuration and a software configuration. [Explanation of Symbols]

[0136] 10. Reporting System 20 Information Processing Devices 21 Computer main unit 25 Memory section 25D Database 25P Information Processing Program 25S System Program 27 Communications Department 28 Operation Input Section 29 Display section 220 Reading Unit 222 Acquisition Department 224 Definition part 226 Display Control Unit

Claims

1. Equipped with a processor, The aforementioned processor, Read multiple string information entries on the form, Obtain feature information indicating features related to calculations using numerical information contained in the aforementioned plurality of string information, and arrangement information indicating the arrangement of the positional relationships of each of the plurality of string information, Using the aforementioned feature information and arrangement information, an arithmetic expression is set that includes the predicted predictive operator by referring to a predetermined prediction table for predictive operators that define the relationships to string information. The aforementioned calculation formula is displayed on a confirmation screen for the user to review. The user provides instructions to modify the aforementioned prediction operator. Based on the above modification instructions, define the calculation formula. Information processing device.

2. The characteristic information includes category information, The aforementioned prediction table pre-defines prediction operators that are set based on combinations of categories. The information processing apparatus according to claim 1.

3. The aforementioned processor, The definition history of the aforementioned calculation formula is stored, In the definition history, operators defined by the user at a frequency exceeding a predetermined threshold can be updated to the prediction operators. The information processing apparatus according to claim 1 or 2.

4. The aforementioned processor, The aforementioned arrangement information uses tabular information that shows the positional relationship between multiple string information items adjacent to each other in at least one direction. An information processing apparatus according to any one of claims 1 to 3.

5. The aforementioned processor, As the arrangement information, information indicating the positional relationship of multiple string information contained in the setting area set in the form is used. The information processing apparatus according to claim 4.

6. The aforementioned processor, Based on the feature information and the arrangement information, the operators between the numerical information contained in the plurality of string information are estimated. Using the estimated operator, define the calculation expression which includes an operation term using the operator between any of the numerical information contained in the plurality of string information, and a numerical information contained in other string information different from the plurality of string information as the calculation result. An information processing apparatus according to any one of claims 1 to 5.

7. It is an information processing program, In the processor, Read multiple string information entries on the form, Obtain feature information indicating features related to calculations using numerical information contained in the aforementioned plurality of string information, and arrangement information indicating the arrangement of the positional relationships of each of the plurality of string information, Using the aforementioned feature information and arrangement information, an arithmetic expression is set that includes the predicted predictive operator by referring to a predetermined prediction table for predictive operators that define the relationships to string information. The aforementioned calculation formula is displayed on a confirmation screen for the user to review. The user provides instructions to modify the aforementioned prediction operator. Based on the above modification instructions, define the calculation formula. An information processing program that handles tasks.

Images