Information processing device and program

Abstract

To provide a technique that makes it possible to further improve confidentiality for persons who use a network and confirm the character strings of items, when printed materials such as forms are converted into data.SOLUTION: A service provider SK which has received form image data from a customer company CK performs form recognition to recognize character strings MR written in items existing on a form image CG represented by the form image data using a form recognition unit K1. A distribution unit K2 distributes the items for which recognition results have been obtained to confirming persons KS1 and KS2, and causes them to confirm the recognition results of different items. Confirmation results obtained from the confirming persons KS1 and KS2 are distributed to approving persons SS1 and SS2, and each approving person approves the confirmation results of different items. In this way, the items to be assigned to the confirming persons KS1 and KS2 and the approving persons SS1 and SS2 are restricted, and work is performed.SELECTED DRAWING: Figure 1

Description

Technical Field

[0004] , , , , , , , , , ,

[0005] , , , , , , ,

[0003]

[0001] The present invention relates to an information processing apparatus and a program.

Background Art

[0002] In printed materials such as forms, data conversion (text data conversion) of character strings printed or written by hand on the printed material may be performed from the image information. As this data conversion, there is a known prior art in which item-by-item image data is cut out from form data obtained by image input of a form, fitted to a predetermined format, and published on a server (see, for example, Patent Document 1). In this prior art, by decomposing form data into item-by-item image data, confidentiality can be realized, and the data conversion of each item, that is, the data input work of the character string represented by the item-by-item image data, can be performed by a plurality of operators who can use the server.

[0003] In some cases, character recognition is performed on form data to have an operator perform a confirmation operation on the recognition results of each item. If there is an error in the recognition result, the recognition result is corrected by data input work. For such reasons, the confirmation operation of the recognition result also includes data input work for correcting the recognition result. Here, the characters constituting the character string are, for example, a general term for Chinese characters, hiragana, katakana, alphabet, numbers, and various symbols, and the character string is one or more of such characters continuously.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0005] In the conventional technology described above, categories are assigned to item-specific image data according to the type of string represented by that image data. Based on this category setting, the conventional technology sends item-specific image data for items to which the operator has selected a category.

[0006] In business forms, a single piece of information is often represented by multiple fields. For example, an individual can be identified by their address and name, and a bank account can be identified by its bank name, account type, and account number. In business forms, these are usually represented by a single string of characters.

[0007] In conventional technology, a corresponding category is assigned to each of the multiple items representing such a collection of information. Therefore, when item-specific image data for a selected category is sent to the operator, the multiple item-specific image data are sorted according to the operator's selection. As a result, it becomes difficult for the operator to review multiple item-specific image data representing a collection of information. This ensures confidentiality.

[0008] However, depending on the category selection of each operator, it is possible that multiple item-specific image data representing a set of pieces of information may all be sent to the same operator. It is also possible that all of the multiple items representing a set of pieces of information may be assigned the same category. For example, multiple items related to product billing, such as product code, quantity, unit price, and total amount, may all be assigned to the same category. If all of these items are assigned to the same category, then all of their item-specific image data will be sent to the same operator. For these reasons, when using a network to have operators (verifiers) perform verification work, it is considered important to further enhance confidentiality.

[0009] The present invention aims to provide a technology that enables greater confidentiality for individuals who use a network to verify the string characters of items in the digitization of printed materials such as forms. [Means for solving the problem]

[0010] An information processing device in one aspect of the present disclosure includes: a part identification means that, upon receiving a plurality of input pieces of information and image information representing the input pieces, identifies a plurality of parts on a target image represented by the image information, each representing the input pieces; a verifier assignment means that divides the plurality of parts and assigns one or more parts to each of a plurality of verifiers; and a part image transmission means that transmits part image information, which is image information of the parts assigned by the verifier assignment means, to verifier terminals used by each of the plurality of verifiers. [Effects of the Invention]

[0011] According to the present invention, when digitizing printed materials such as forms, the confidentiality of individuals who perform the task of verifying the string of items using a network can be further improved. [Brief explanation of the drawing]

[0012] [Figure 1] This figure illustrates an example of an overview of the services provided by applying the present invention. [Figure 2] This diagram illustrates an example of how to allocate a string area. [Figure 3] This figure illustrates an example of a network environment to which an AP server, according to one embodiment of the information processing device of the present invention, is connected. [Figure 4] This is a block diagram showing an example of the hardware configuration of an AP server according to one embodiment of the information processing device of the present invention. [Figure 5] This is a functional block diagram showing an example of a functional configuration implemented on an AP server according to one embodiment of the information processing device of the present invention. [Figure 6] This figure shows an example of a screen for setting the confidence level for automatic confirmation. [Figure 7] This diagram illustrates the first example of how to enter corrected data, and an example of how to configure the settings based on that first example. [Figure 8] This diagram illustrates a second example of how to enter corrected data. [Figure 9] This is a diagram for explaining an example of a setting method assuming a second example of a method for entering corrected data. [Figure 10] This is a diagram for explaining an example of a confirmation work screen. [Figure 11] This is a diagram showing an example of the content displayed in the recognition result display area when the setting of the confidence level of automatic determination is effective. [Figure 12] This is a diagram showing an example of the content displayed in the recognition result display area when the setting of the confidence level of automatic determination and the automatic determination of blanks are effective. [Figure 13] This is a diagram showing an example of the content displayed in the recognition result display area when the confidence level and the degree of coincidence of automatic determination are effective and a calculation formula is set. [Figure 14] This is a flowchart showing an example of a confirmation work screen generation process. [Figure 15] This is a flowchart showing an example of a confirmation work screen generation process (continued). [Figure 16] This is a flowchart showing an example of an approval work screen generation process. [Figure 17] This is a flowchart showing an example of an approval result process.

Best Mode for Carrying Out the Invention

[0013] Hereinafter, embodiments for carrying out the present invention will be described with reference to the drawings. Note that the embodiments described below are merely examples, and the technical scope of the present invention is not limited thereto. The technical scope of the present invention includes various modifications.

[0014] FIG. 1 is a diagram for explaining an example of an overview of a service provided by the application of the present invention. This service (hereinafter referred to as "this service") is provided by the service provider SK to companies that wish to digitize printed materials such as forms, with CK being the primary customer company. However, the customer is not limited to a company. In other words, the customer may be an organization other than a company, or an individual. Furthermore, the printed material only needs to have text, whether printed or handwritten, arranged in a table structure. Here, we assume that a form is a representative example of such printed material with text arranged in a table structure. Therefore, this service is assumed to perform form recognition and digitize the text written on it. In fact, the text that is actually arranged is an image. For this reason, unless otherwise specified, "text" will be used to refer to an image.

[0015] Customer company CK uses this service by scanning documents with a scanner or similar device to convert them into image data, and then transmitting the converted document image data to service provider SK via the network. CG in Figure 1 is an example of a document image represented by the document image data. Multiple strings MR exist on this document image CG. Note that the document image data corresponds to the image information in this embodiment. Furthermore, the document image CG corresponds to the target image in this embodiment, the string (string image) MR corresponds to the input information image, and the data of the string MR, such as text data, corresponds to the input information.

[0016] Service provider SK has prepared a document recognition unit K1, a distribution unit K2, and a result output unit K3 for the provision of this service. These are implemented, for example, on one or more information processing devices. The document recognition unit K1 performs document recognition, including character recognition, using document image data received from customer company CK. This document recognition unit K1 converts the string characters MR present on the document image CG into text data.

[0017] The position of the string MR can be determined by referencing a template, as this template represents the position and range of each string MR. The position and range of each string MR can be identified by performing optical character recognition (OCR). For this reason, templates are not necessary. However, for the sake of explanation, we will assume that templates are prepared for each form. Therefore, unless otherwise specified, we will assume that the form recognition unit K1 performs form recognition using templates.

[0018] Many forms not only have fields for entering text, but also have printed text that represents the text to be entered in those fields. This printed text is usually the field name. For example, bank name, account type, and account number are all field names. Hereafter, the field name will be referred to as the "field name," and the entered or recognized text will be referred to as the "field value."

[0019] In form recognition without using templates, the strings representing the item names are recognized along with the strings entered in the input fields. This allows the table structure to be analyzed, and the correspondence between the item name strings and the entered strings, as well as the correspondence with other strings, is identified. In contrast, with form recognition using templates, there is no need to analyze the table structure. There is also no risk of incorrectly identifying correspondences. As a result, the verifiers KS1 and KS2, who perform the verification work to confirm the recognition results, only need to check the recognition results of the entered strings. For these reasons, when using templates for form recognition, it is possible to minimize the workload of the verification work while maintaining high verification accuracy. This also reduces the time required for the verification work, making it effective in providing this service more quickly and at a lower cost. There are advantages to using templates for form recognition in providing this service.

[0020] Assuming form recognition using a template, the information represented by the entered string MR corresponds to the "input information image," the area where that string MR exists, or where it is likely to exist, corresponds to the "part," and the image data representing that string MR, or the area where it is likely to exist, corresponds to the "part image information." Here, we assume that this image data is the image data of the string MR. Hereafter, to distinguish it from other image data, this image data will be referred to as "item image data."

[0021] Figure 1 shows two verifiers, KS1 and KS2, but any number of verifiers should be two or more. For this reason, if there is no need to specify a verifier, the code "KS" will be used. Similarly, if there is no need to specify the approver who approves the verification results by verifier KS, the code "SS" will be used.

[0022] The distribution unit K2 assigns the recognition results to each verifier KS and transmits the item image data and recognition results according to that assignment. It also assigns the verification results from each verifier KS to each approver SS and transmits the item image data and verification results according to that assignment. Figure 1 shows the string "MR" represented by the item image data assigned to and transmitted to verifiers KS1 and KS2, and approvers SS1 and SS2, respectively, and their positions represented by the form images CG1 to CG4.

[0023] In the example shown in Figure 1, all strings MR on the document image CG are assigned to only one of the reviewers, KS1 or KS2. This allows the two reviewers, KS1 and KS2, to share the task of verifying the recognition results of all strings MR on the document image CG. To this end, reviewers KS1 and KS2 receive a verification request in which the item image data of the assigned string MR and the recognition result of that string MR are sent. As a verification result, for example, the recognition result after the verification work is sent.

[0024] By having verifiers KS1 and KS2 perform this verification process, it becomes impossible for either KS1 or KS2 to grasp the content of all strings MR present on the document image CG during the verification process. Therefore, a higher level of confidentiality is achieved compared to conventional technologies. In this service, as described later, an even higher level of confidentiality is achieved by ensuring that no single verifier KS can grasp all of the related strings MR on the document image CG.

[0025] Similarly, in the approval of verification results, the verification results of all strings MR on the document image CG, verified by two verifiers KS1 and KS2, are assigned to only one of the approvers SS1 or SS2. This way, the two approvers SS1 and SS2 are divided in their work to approve all strings MR on the document image CG. For this purpose, approvers SS1 and SS2 receive an approval request in which the item image data of the assigned string MR and the verification results of that string MR are sent. As an approval result, for example, the recognition result after the approval process is sent.

[0026] By having approvers SS1 and SS2 perform this approval process, a higher level of confidentiality is achieved, similar to the case with verifiers KS1 and KS2. In this service, as described later, a higher level of confidentiality is achieved by ensuring that no single approver SS can perceive all of the related strings on the document image CG.

[0027] In the example shown in Figure 1, the combinations of strings MR assigned to and transmitted to verifiers KS1 and KS2, and approvers SS1 and SS2, are all different. This is done to ensure that no single person can grasp all of the related strings on the document image CG through information exchange between either verifier KS1 or KS2 and either approver SS1 or SS2. Therefore, assigning different combinations of strings MR to verifiers KS1 and KS2, and approvers SS1 and SS2, is effective in achieving a higher level of confidentiality.

[0028] Figure 2 illustrates an example of how to assign items. Referring to Figure 2, we will now specifically explain an example of how to assign items, i.e., the string MR, to each verifier KS and each approver SS to achieve the confidentiality described above. Figure 2 shows how items are assigned using item names, using an example form with the following item names: Issue Date, Customer Name, Item Name 1, Quantity 1, Unit Price 1, Total Price 1, Item Name 2, Quantity 2, Unit Price 2, and Total Price 2. Hereafter, the assignment details will be explained using item names (or field names).

[0029] This service allows you to set a maximum number of reserved items and item groups to manage the assignment of item names. The maximum number of reserved items limits the number of items that a single reviewer (KS) can review. Item groups specify combinations of item names to be treated as the same group. Unlike the maximum number of reserved items, multiple item groups can be set. In Figure 2, "Group 1," "Group 2," and "Group 3" are all group names of the set item groups. As a result, all item names are broadly categorized into those belonging to one of the item groups and those not belonging to any item group. The example shown in Figure 2 is when three item groups are set and the maximum number of reserved items is set to 5.

[0030] The item names combined in groups 1 to 3 are assigned to the same verifier KS and the same approver SS. In other words, item names are assigned on an item group basis. To ensure that the number of assigned items does not exceed the maximum number of reserved items, item names that do not belong to any item group are further assigned. In this way, as shown in Figure 2, each verifier KS and each approver SS is assigned a different combination of item names.

[0031] In the example of all item names shown in Figure 2, Item Name 1, Quantity 1, Unit Price 1, and Total Price 1 are related item names and constitute a unified group of information. The same applies to the item name groups Item Name 2, Quantity 2, Unit Price 2, and Total Price 2. By setting up item groups as exemplified in Figure 2, it is possible to reliably prevent these groups of item names from being assigned to a single person. Therefore, enabling the setting of item groups is effective in further improving confidentiality. The document images CG1 to CG4 shown in Figure 1 represent the item names assigned to verifiers KS1 and KS2, and approvers SS1 and SS2, respectively, using the assignment method exemplified in Figure 2.

[0032] Let's return to the explanation of Figure 1. The distribution unit K2 assigns the item names as described above, and based on the assignment results, sends confirmation requests to each verifier KS and approval requests to each approver SS. Once each approver SS approves, the text data conversion of each string MR is completed, and the text data for each item is finalized. The finalized text data is sent to the customer company CK by the result output unit K3 as the result of document recognition using document image data. This transmission is performed, for example, at the request of the customer company CK.

[0033] From here on, we will explain in detail the specific methods for providing this service, which is illustrated in Figure 1, while referring to Figures 3 to 17. Figure 3 illustrates an example of a network environment to which an AP (Application) server, according to one embodiment of the information processing device of the present invention, is connected.

[0034] AP Server 1 is an information processing device installed by service provider SK, utilizing a cloud service provided by cloud service provider CS for the purpose of providing this service. While it is common to install other servers such as a Web server and a DB (Database) server in addition to AP Server 1, for the sake of explanation, we will assume that service provider SK has installed only one AP Server 1 for the purpose of providing this service. Note that AP Server 1 may also be installed without using a cloud service. In other words, service provider SK may install AP Server 1 within its own premises or at a contracted location. For this reason, the method and location of installation of AP Server 1 are not particularly limited.

[0035] AP Server 1 is connected to Network N. Network N is a collection of multiple networks, including, for example, the Internet. Other networks besides the Internet include mobile phone networks and LANs (Local Area Networks). As shown in Figure 3, this network N is directly or indirectly connected to customer terminal 5 used by customer company CK, verifier terminal 6 used by verifier KS, and approver terminal 7 used by BPO (Business Process Outsourcing) center BC. In reality, there are multiple instances of customer company CK, verifier terminal 6, and approver terminal 7.

[0036] BPO Center BC is a company that performs tasks outsourced by client company CK. Here, it is assumed that BPO Center BC is outsourced by client company CK to perform tasks related to document recognition. For example, service provider SK has set up BPO Center BC and has further set up AP Server 1 to improve the efficiency of BPO Center BC's operations. Service provider SK may also provide this service to BPO Center BC through a contract with BPO Center BC. In other words, service provider SK may provide this service indirectly to client company CK through BPO Center BC. For these reasons, this service does not have to be provided directly to client company CK. In other words, service provider SK does not have to consider companies that intend to use this service as its customers.

[0037] The verifiers (KS) include, for example, individuals registered with the service provider SK who undertake one-off jobs (gig workers). Gig workers are workers who take on jobs they want at times convenient for them. Therefore, when gig workers are used as verifiers (KS), it can be expected that when necessary verification work arises, one of the verifiers (KS) will perform the verification work immediately, or within a short time after it arises. This means that the service can be provided quickly. For this advantage, this service is designed to utilize gig workers as verifiers (KS).

[0038] On the other hand, gig workers raise concerns regarding confidentiality. For this reason, it is necessary to achieve a higher level of confidentiality. In this service, as described above, a higher level of confidentiality is achieved by ensuring that at least not all items in a single document are assigned to a single reviewer, KS.

[0039] Furthermore, gig workers may exhibit significant variability in the quality of their verification work. The same applies to their sense of responsibility regarding verification. For this reason, BPO Center BC designates its employees as approvers (SS), who then approve the verification results and verify those results. By having such approvers (SS), BPO Center BC can ensure not only the speed of this service but also its high quality.

[0040] The document image data necessary for document recognition may be sent from customer company CK to AP server 1 via BPO center BC, or it may be sent directly from customer company CK to AP server 1. Alternatively, BPO center BC may convert documents sent by mail or other means into image data and send it to AP server 1. Thus, there are various paths through which AP server 1 can receive document image data. To avoid confusion and facilitate understanding, here we will only consider the case where document image data is sent from customer terminal 5 to AP server 1. Furthermore, we will assume that AP server 1 communicates directly with verifier terminal 6 and approver terminal 7.

[0041] Figure 4 is a block diagram showing an example of the hardware configuration of an AP server according to one embodiment of the information processing device of the present invention. Next, with reference to Figure 4, an example of the hardware configuration of AP server 1 will be specifically described. Note that this configuration example is just one example, and the hardware configuration of AP server 1 is not limited to this.

[0042] As shown in Figure 4, AP Server 1 includes a CPU (Central Processing Unit) 11, ROM (Read Only Memory) 12, RAM (Random Access Memory) 13, a bus 14, an input / output interface 15, an output unit 16, an input unit 17, a storage unit 18, a communication unit 19, and a drive 20.

[0043] The CPU 11 executes programs stored in, for example, ROM 12, and programs loaded from the storage unit 18 into RAM 13, thereby performing various processes. The programs loaded from the storage unit 18 into RAM 13 include, for example, the OS (Operating System) and various application programs that run on that OS. These application programs include one or more programs developed specifically for providing this service.

[0044] RAM13 also stores data necessary for the CPU11 to perform various processes. This data includes various programs that the CPU11 executes. These programs are read into RAM13 and executed by the CPU11. The CPU 11, ROM 12, and RAM 13 are interconnected via a bus 14. An input / output interface 15 is also connected to this bus 14. An output unit 16, an input unit 17, a storage unit 18, a communication unit 19, and a drive 21 are connected to the input / output interface 15.

[0045] The output unit 16 includes, for example, a display such as an LCD. The output unit 16 displays various images or screens under the control of the CPU 11. The output unit 16 may be installed on the AP server 1, or it may be connected as needed. Therefore, the output unit 16 is not an essential component.

[0046] The input unit 17 includes, for example, various hardware buttons such as a keyboard. Its configuration may also include one or more pointing devices such as a mouse. In other words, the input unit 17 may be equipped with multiple input devices operated by the operator (primarily the system administrator). The operator can input various types of information via the input unit 17. This input unit 17 may be installed on the AP server 1, but it may also be connected as needed. Therefore, the input unit 17 is not an essential component.

[0047] The storage unit 18 is, for example, an auxiliary storage device such as a hard disk drive or an SSD (Solid State Drive). Large amounts of data are stored in this storage unit 18. The communication unit 19 enables communication with other information processing devices via the network N. Each of the terminals 5 to 7 shown in Figure 3 is an other information processing device.

[0048] The drive 20 is a device that can insert and remove removable media 25, such as a magnetic disk, optical disk, magneto-optical disk, or semiconductor memory card. The drive 20 can, for example, read information from and write information to the inserted removable media 25. As a result, a program recorded on the removable media 25 can be stored in the storage unit 18 via the drive 20. Furthermore, the removable media 25 inserted in the drive 20 can be used as a copy destination or transfer destination for various data stored in the storage unit 18.

[0049] The application program developed for this service may be recorded on removable media 25 and distributed. It may also be made available for distribution via network N, etc. Therefore, the recording medium on which the application program is recorded may be one mounted or attached to an information processing device directly or indirectly connected to network N, or one mounted or attached to an externally accessible device. To distinguish it from others, the application program developed for this service will hereafter be referred to as the "development application." There may be multiple development applications, but for the sake of explanation, we will assume there is only one development application here.

[0050] The hardware resources of AP Server 1 are controlled by various programs, including the development application. As a result, various functions for providing this service are implemented on AP Server 1. Figure 5 is a functional block diagram showing an example of a functional configuration implemented on an AP server according to one embodiment of the information processing device of the present invention. Next, an example of a functional configuration implemented on the AP server 1 will be described in detail with reference to Figure 5.

[0051] On the CPU 11 of AP Server 1, the following functional configurations are implemented, as shown in Figure 5: request processing unit 111, document recognition unit 112, setting processing unit 113, reservation processing unit 114, confirmation processing unit 115, approval processing unit 116, approval result output unit 117, and screen generation unit 118. The CPU 11 transmits and receives data with each terminal 5 to 7 via the communication unit 19 in order to provide this service.

[0052] These functions are realized by the CPU 11 executing various programs, including development applications. As a result, the storage unit 18 is allocated for information storage, including a report image data storage unit 181, a recognition result storage unit 182, a template storage unit 183, a setting information storage unit 184, a reservation information storage unit 185, a confirmation result storage unit 186, and an approval result storage unit 187.

[0053] Although Figure 5 shows the CPU 11 directly accessing the memory unit 18, in reality, such access does not occur. Information is usually transmitted and received between the CPU 11 and the memory unit 18 via the RAM 13. However, for the sake of explanation, we will assume that the CPU 11 directly accesses the memory unit 18. Similarly, we will assume that the communication unit 19 and the CPU 11 directly transmit and receive data. As described above, the request for data conversion of document image data through document recognition is made to the AP server 1 by the customer terminal 5, and it is assumed that the verifier terminal 6 and the approver terminal 7 also communicate directly with the AP server 1.

[0054] This service has restricted access. To ensure that only authorized users can use it, authentication is performed to verify, for example, whether or not a user is authorized. This authentication identifies the person who allowed a terminal to access AP server 1, and also identifies the correspondence between that person and the terminal they are using. For example, if the verifier is KS, authentication identifies the person who allowed the terminal to access as verifier KS, and that terminal is considered verifier terminal 6. Although such authentication functionality is also implemented on CPU 11, it is a general feature and is therefore excluded from detailed explanation here.

[0055] When a request for data conversion of document image data through document recognition (document recognition request) is sent from the customer terminal 5 to the AP server 1, the request is received by the communication unit 19 and output from the communication unit 19 to the CPU 11. The request input to the CPU 11 is passed to the request processing unit 111 for processing. The request processing unit 111 processes not only the various requests sent from the customer terminal 5, but also the various requests sent from the verifier terminal 6 and the approver terminal 7. As a result, the AP server 1 performs various operations to provide this service under the control of the request processing unit 111.

[0056] When the request processing unit 111 receives a data conversion request from the customer terminal 5, it stores, for example, the document image data transmitted with the request in the document image data storage unit 181 of the storage unit 18, and instructs the document recognition unit 112 to perform document recognition using the document image data. In response to this instruction, the document recognition unit 112 reads the target template from the templates stored as data in the template storage unit 183 of the storage unit 18, and performs document recognition processing on the document image data. The recognition result obtained as a result of executing this document recognition processing is stored in the recognition result storage unit 182 of the storage unit 18. This document recognition unit 112 corresponds to the part identification means and the character recognition means included therein in this embodiment.

[0057] When the request processing unit 111 stores the document image data in the document image data storage unit 181, it assigns a document ID (IDetifier), which is identification information, to the document image data. On the other hand, the document recognition unit 112 either includes the document ID in the recognition result stored in the recognition result storage unit 182, or associates it with the document ID. In this way, the document image data and the recognition result can be uniquely identified, and their correspondence can also be identified.

[0058] The document recognition process performed by the document recognition unit 112 employs well-known technologies, including the identification of the target template. Therefore, a detailed explanation is omitted, but character recognition is performed on each string MR that should be converted into text data, which is present on the document image CG represented by the document image data, and text data is generated as a result of this recognition. This service employs AI (Artificial Intelligence) OCR for character recognition.

[0059] The configuration processing unit 113 provides an environment that enables various settings. The BPO center BC is the only entity capable of performing these settings. These settings include the registration of templates. Registered templates are stored in the template storage unit 183 of the memory unit 18. Since template registration can be performed using well-known methods, a detailed explanation is omitted here.

[0060] In addition to registering templates, the configuration processing unit 113 can also configure the maximum number of reserved items and item groups as described above. Furthermore, it can also configure the conditions that the recognition result must satisfy to avoid requiring confirmation by the verifier KS, the mode selection for recognition result confirmation and approval, and the extraction range when correcting the string MR. These settings will be explained in detail with reference to Figures 6 to 9.

[0061] Figure 6 shows an example of a screen for setting the confidence level of automatic confirmation. Various setting screens, including this one, are generated by the screen generation unit 118 based on instructions from the setting processing unit 113, and transmitted to the customer terminal 5 via the communication unit 19. Other screens are also generated by the screen generation unit 118.

[0062] The confidence level is a numerical indicator that represents the likelihood of a character recognition result, and it is generated along with the recognition result. The higher the likelihood of the recognition result, the higher the confidence level. In this service, this confidence level is used as an indicator to determine whether or not verification by the verifier KS is unnecessary, that is, whether or not to automatically confirm the recognition result.

[0063] The settings screen shown in Figure 6 allows you to set the minimum confidence level for confirming recognition results to BPO Center BC. This minimum value is the automatic confirmation confidence level indicated in Figure 6. For reference in setting this minimum value, this settings screen shows the number of correct AIOCR results and the number of validation results for each range of confidence levels obtained from character recognition for the selected item. The number of correct AIOCR results is the number of recognition results that were correct within the corresponding confidence level range, i.e., the number of recognition results that were not corrected. The number of validation results is the number of recognition results that were incorrect within the corresponding confidence level range, i.e., the number of recognition results that were corrected. Note that "validation" itself is a term that means verification or confirmation. This settings screen allows BPO Center BC to understand the characteristic trends in confidence levels for each item. Therefore, BPO Center BC can more appropriately set the confidence level for automatic confirmation in accordance with these characteristic trends.

[0064] The confidence level for automatic confirmation set on this settings screen is associated with, for example, templates and items, and stored as setting information in the setting information storage unit 184 of the memory unit 18. Many client companies at CK likely have different requirements regarding recognition accuracy and confidentiality depending on the type of document. For this reason, we assume that all configuration information will be mapped to templates.

[0065] The confidence level for automatic confirmation can be combined with the degree of agreement. This degree of agreement is a value calculated using, for example, the number of correct AIOCR results and the number of validations for each range of confidence levels. More specifically, for example, the degree of agreement (%) is calculated as: Number of correct AIOCR results × 100 / (Number of correct AIOCR results + Number of validations). Both the set confidence level for automatic confirmation and the set degree of agreement correspond to the set values ​​or set ranges in this embodiment.

[0066] The combined confidence and match rates for automatic determination are stored in the setting information storage unit 184 of the memory unit 18 as separate setting information from the confidence rates for automatic determination that are not combined. The setting screen on which the confidence and match rates for automatic determination can be set may, for example, be the setting screen shown in Figure 6, with an additional input box for entering the match rate.

[0067] This service also allows setting conditions that must be met between recognition results using a calculation formula. This is because there may be relationships that must be met between item names. For example, considering four item names: product code, quantity, unit price, and total amount, the total amount is calculated by multiplying the quantity by the unit price. In other words, there is a relationship that can be expressed by the calculation formula: total amount = quantity × unit price. It can be said that the possibility of misrecognition occurring in one or more of the total amount, quantity, and unit price while this calculation formula is met is very low. For this reason, this service allows setting a calculation formula that must be met, and when that calculation formula is met, the recognition results for all item names used in that calculation formula are automatically determined. This calculation formula is also stored as setting information in the setting information storage unit 184 of the memory unit 18. Note that the type of calculation formula is not particularly limited. The calculation formula may be an inequality (greater than or lesser than relationship), a logical formula, etc., rather than an equation. This calculation formula corresponds to defining a predetermined relationship in this embodiment.

[0068] When creating a form, creators who print or handwrite text into each field may not fill in all fields. There may be fields that do not require filling in, or fields that have been forgotten. Such fields are recognized as blank because they do not contain the text "MR". The likelihood of this recognition being incorrect is very low. Therefore, this service allows you to configure whether or not to automatically confirm the recognition result as blank. This setting, unlike the confidence level setting for automatic confirmation, can be done on a per-template basis. In other words, it is not always necessary to associate it with a field.

[0069] The creator may correct the entered string MR. Correcting a string MR usually involves writing a note indicating that the original string MR is invalid, and then writing a different string MR in a different location than where the original was written. To verify the recognition result of the string MR, this alternative string MR needs to be visually inspected by the verifier KS. However, the location where this alternative string MR is written cannot be identified from the template, as the template is not designed to accommodate the addition of correction string MRs. Therefore, this service allows for a setting to extract the location where this alternative string MR is written (correction extraction setting). This setting must be configured at least per template, as the locations where alternative string MRs can be written usually differ depending on the template.

[0070] Here, referring to Figures 7 to 9, we will specifically explain the method for correcting the string MR and the settings corresponding to the correction method. In Figures 7 to 9, the string MR entered in the designated location is referred to as "data," and the string MR entered for correction is referred to as "corrected data." Accordingly, these terms will be used in this explanation. As symbols, data will be denoted as MR, and corrected data as TMR. Furthermore, from now on, the string MR corresponding to the corrected data TMR will also be referred to as the corrected string TMR.

[0071] Figure 7 illustrates a first example of how to enter corrected data, and an example of a setting method based on that first example. As shown in Figure 7, in some cases, one or more horizontal lines may be drawn on a data MR that is subject to correction to indicate that the data MR is invalid. The presence of such lines allows one to determine whether the creator intended to correct the data MR. In other words, by checking for the presence or absence of lines drawn on the data MR, it is possible to determine whether the data MR itself is invalid and whether it should be corrected by the corrected data TMR.

[0072] The presence of lines connecting multiple characters can prevent data MR from being recognized, or even if recognized, reduce its accuracy. Therefore, by checking whether lines exceeding the width of a single character exist in the locations where such recognition results were obtained, it is possible to determine whether the data MR has been corrected or not.

[0073] As shown in Figure 7, corrected data TMR for correcting data MR may be entered near the data. Therefore, the vicinity of the data becomes a candidate location where the corrected data is expected to be entered. For this reason, this service divides the area around the data into eight candidate locations, and allows the BPO Center BC to select the extraction location KA from among these eight candidate locations to be used for entering the corrected data TMR. Figure 7 shows that the two candidate locations represented by "(2)" and "(3)" have been selected as extraction location KA.

[0074] Figure 8 illustrates a second example of how to enter the corrected data. As shown in Figure 8, the creator may draw a line (leader line) from the data MR and write in the corrected data TMR. This leader line allows the creator to clearly define the correspondence between the data MR and the corrected data TMR.

[0075] Figure 9 illustrates an example of a setting method assuming a second example of how to enter corrected data. Corrected TMR data, with corresponding relationships indicated by leader lines, may be entered in areas enclosed by boundary lines, as shown in Figure 9(a). In this case, this service automatically extracts the area enclosed by the boundary line as the extraction area KA. Such boundary lines can be identified by tracing lines longer than one character.

[0076] If no boundary line exists, possible locations for the corrected TMR data entry are around the opposite end of the data MR, as shown in Figure 9(b). Therefore, in this service, the area around the end of the leader line is divided into five candidate locations, and the BPO Center BC is instructed to select the extraction location KA from these five candidate locations to be used for the corrected TMR data entry.

[0077] The setting result for extraction location KA, along with information indicating whether or not a leader line is expected, is associated with the template to be associated as setting information and stored in the setting information storage unit 184 of the storage unit 18. If it is determined that a correction has been made to data MR, the method of entering the corrected data TMR is identified, and according to the identified entry method, the item image data of the corrected data TMR is automatically extracted along with the item image data of data MR. To enable such extraction, the recognition result includes the recognition result of data MR and information indicating the range in which data MR exists on the report image CG, as well as the recognition result of the corrected data TMR at each extraction location KA and information indicating the range in which that corrected data TMR exists. The range information, similar to string MR, includes, for example, coordinate information on the report image CG used as the base point, and width information representing each width on the XY axes, for example, in terms of pixels. Hereafter, the range information will be referred to as "range coordinate information." Note that the method of correcting data MR and the method of entering the corrected data TMR are not limited to those described above. However, regarding the method for correcting data MRs, it is desirable that the method be such that it can be clearly determined that the data MR is invalid as a result of the correction.

[0078] Thus, in this service, if it is determined that the data MR needs correction, the service identifies either the area enclosed by the boundary de or the extracted area KA as a separate part, and extracts the image data of the identified separate part as item image data. This allows the service to respond to data MR corrections, ensuring that appropriate recognition results and appropriate item image data are obtained even if the data MR has been corrected. Therefore, for the verifier KS, high-quality verification work can be performed in a shorter time, regardless of whether the data MR has been corrected or not. Alternatively, the separate part may be identified by performing character recognition on the area surrounding the part where the data MR exists and using the results of that character recognition.

[0079] As explained with reference to Figure 1, the content approved by approver SS is sent to customer company CK as a result of document recognition using document image data. The accuracy required by customer company CK for the recognition results is not always the same. Customer company CK may require a higher accuracy than usual for important recognition results. For this reason, this service also allows for a setting to have multiple verifiers KS check the recognition results of the same item (hereinafter referred to as "multiple check setting"). By using this multiple check setting, the same item is duplicated and the recognition results are checked multiple times by different verifiers KS, so customer company CK can obtain recognition results with higher accuracy.

[0080] This service allows for the above-mentioned settings. Of the above settings, the confidence level of automatic confirmation, the confidence level and match rate of automatic confirmation, automatic confirmation of blanks, and the multiple check setting are each enabled by the settings of the corresponding mode. However, the confidence level of automatic confirmation and the confidence level and match rate of automatic confirmation cannot both be enabled. This is because the two overlap in some respects. The overlap in some respects means that the criteria for automatic confirmation of recognition results may actually be different.

[0081] The correction extraction setting becomes effective when that setting is configured. This is because, whether performing the recognition result verification or approval process, if the data MR has been corrected, the item image data of the corrected data TMR will be required. Since it is not possible to know in advance whether or not the corrected data TMR exists, it is necessary to assume that it exists. Similarly, the calculation formula also becomes effective when that setting is configured.

[0082] Up until now, the explanation has assumed that the data MR on the document image CG is divided among multiple reviewers KS, and that not all data MR can be reviewed by any single reviewer KS. However, this type of review is just one possible setting. This service also has a setting that allows all data MR to be reviewed by a single reviewer KS. To distinguish the previously assumed review setting from this one, it will henceforth be referred to as the "mask setting."

[0083] If this mask setting is disabled, the settings for the maximum number of reserved items and each item group are disabled. However, other settings do not depend on whether the mask setting is enabled or disabled. In other words, if any other settings are enabled, they will be enabled regardless of whether the mask setting is disabled or disabled. Whether this mask setting is enabled or disabled is associated with the template and stored as setting information in the setting information storage unit 184 of the memory unit 18.

[0084] Unlike other settings, the correction extraction setting is referenced by the form recognition unit 112 when performing form recognition. Therefore, if a correction to the data MR is detected during form recognition, the correction extraction setting is referenced, and character recognition is performed on the corrected data TMR present at the extraction location KA. The result obtained from this character recognition becomes the recognition result.

[0085] The settings processing unit 113 controls the screen generation unit 118 and performs processing to enable the various settings described above. As a result, customer company CK can use the service in a more desirable manner, regardless of whether the mask setting is enabled or not. Regardless of which of the various settings for automatically confirming the recognition result is enabled, the number of items that need to be confirmed and approved will be reduced while avoiding or minimizing any decrease in the quality of the service being used. With fewer items, the cost of using the service can be reduced, and the time from sending the form image data to obtaining the recognition result (turnaround time) can be shortened. For these advantages, enabling the setting for automatically confirming the recognition result is useful in allowing customer company CK to use the service in a more desirable manner.

[0086] Client company CK will use this service as needed. This service allows verifiers KS to book the verification tasks they desire and perform those tasks. This is to determine earlier which verification tasks each available verifier KS can handle. By determining available tasks earlier, if it is difficult or impossible to secure the necessary verifier KS, CK can contact verifier KS who are not currently booked and request them to book the task. As a result, having verifier KS book verification tasks can shorten turnaround time.

[0087] Similarly, in the approval process, the approver SS is also responsible for making such reservations. However, since there are no differences, or only minor differences, in the reservation method, we will focus solely on the verifier KS in the following explanation. The reservation processing unit 114 performs the processing necessary to enable the reservation. To enable the reservation, the reservation processing unit 114 controls the screen generation unit 118 to generate the necessary screen.

[0088] The verifier can, for example, send a reservation request to the verifier terminal 6 by performing the reservation operation on the screen displayed on the verifier terminal 6. The reservation request sent from the verifier terminal 6 to the AP server 1 is received by the communication unit 19 and output to the CPU 11. The reservation request is then passed to the reservation processing unit 114 via the request processing unit 111. The reservation processing unit 114, upon receiving the reservation request, identifies the available confirmation tasks by referring to, for example, the recognition results stored in the recognition result storage unit 182, as well as future schedules, and causes the screen generation unit 118 to generate a reservation screen that allows the user to reserve the identified confirmation task. As a result, the reservation screen is sent to the verifier terminal 6 that sent the reservation request.

[0089] On the reservation screen, for example, each available confirmation task is accompanied by a descriptive text or image representing that task. These descriptive texts or images function as link buttons and are associated with, for example, a document ID. The user then reserves a confirmation task by operating the link button for the desired task. To do this, the user's terminal 6 sends a reservation registration request to the AP server 1 upon operation of the link button. The document ID associated with the link button is stored in the reservation registration request.

[0090] The reservation information storage unit 185 of the memory unit 18 stores reservation information that includes at least the reservation date and time, the reservation user ID (the ID assigned to the verifier), the document ID, etc. Reservation registration requests received by the AP server 1 and input to the CPU 11 are passed to the reservation processing unit 114 via the request processing unit 111. The reservation processing unit 114 uses, for example, the document ID in the reservation registration request to search the reservation information storage unit 185 and check whether reservation information containing that document ID exists. If no reservation information containing the document ID is found, the reservation processing unit 114 generates new reservation information and stores it in the reservation information storage unit 185. Meanwhile, it instructs the screen generation unit 118 to generate and send a reservation screen notifying that the requested reservation has been made. If reservation information containing the document ID is found, the reservation processing unit 114 instructs the screen generation unit 118 to generate and send a reservation screen notifying that the requested reservation could not be made.

[0091] For this reason, the reservation screen also displays a description or image of the reserved confirmation task. Alternatively, the description or image of the reserved confirmation task could be made into a link button, allowing the user to perform the reserved confirmation task by clicking the link button. Alternatively, a screen for performing the confirmation task could be sent to the verifier terminal 6, allowing the verifier to perform the selected confirmation task. In any case, when the verifier performs the confirmation task, a confirmation task request should be sent from the verifier terminal 6 to the AP server 1. The document ID could be stored, for example, in that confirmation task request. The request to perform the verification work, sent from the verifier terminal 6 and received by the AP server 1, is passed to the request processing unit 111 implemented on the CPU 11, just like other requests, and then passed from the request processing unit 111 to the verification processing unit 115.

[0092] The verification processing unit 115 performs processing to allow the verifier KS to perform the reserved verification work. In order to enable the verification work, the verification processing unit 115 controls the screen generation unit 118 to generate the necessary screens. The configuration information stored in the configuration information storage unit 184 is referenced for generating the screen. The confirmation processing unit 115 references the configuration information and, according to the enabled settings, assigns items to the verifier KS and identifies items for which the recognition result will be automatically confirmed, thereby controlling the screen generation unit 118. Through this control, the confirmation processing unit 115 causes the screen generation unit 118 to generate the confirmation work screen. The generated confirmation work screen is sent to the verifier terminal 6 that sent the confirmation work request.

[0093] As described above, the document ID is assigned to the document image data and stored in or associated with the recognition result. When the document recognition process that obtained the recognition result is executed, the corresponding template is identified. Thus, the confirmation processing unit 115 can identify the setting information to be referenced from the setting information stored in the setting information storage unit 184 using the document ID stored in the confirmation work execution request.

[0094] Figure 10 is a diagram illustrating an example of a confirmation screen. This confirmation screen is for performing the confirmation work, and as shown in Figure 10, it is equipped with an image display area GA used for displaying document image data and a recognition result display area RA used for displaying the recognition results obtained from the displayed document image data. The confirmation processing unit 115 instructs the screen generation unit 118 to generate the confirmation screen and specifies the content to be displayed in each display area GA and RA. As a result, the confirmation terminal 6 that sent the confirmation work request receives a confirmation screen displaying the content that should be confirmed in the confirmation work reserved by the confirmation KS.

[0095] Here, referring to Figures 11 to 13, we will specifically explain what is displayed in the recognition result display area RA when the mask setting is enabled, depending on other enabled settings. If mask settings are enabled, setting the maximum number of reserved items is mandatory. Figures 11 to 13 all show cases where, in addition to the maximum number of reserved items, item group settings have also been configured.

[0096] Figure 11 shows an example of what is displayed in the recognition result display area when the automatic confirmation confidence level setting is enabled. As shown in Figure 11, the recognition result display area RA on the confirmation screen has the strings "Skip blank items," "Skip by confidence level," and "Skip by confidence level & match level," with a toggle button SD to the right of each string. These strings represent the type of setting. Specifically, "Skip blank items" represents automatic confirmation of blanks, "Skip by confidence level" represents the confidence level for automatic confirmation, and "Skip by confidence level & match level" represents the confidence level and match level for automatic confirmation. The toggle button SD to the right of each string indicates whether the setting represented by the string is enabled or disabled. In all cases, the circular icon on the right of the toggle button SD indicates that it is enabled. In Figure 11, only the toggle button SD to the right of the string "Skip by confidence level" has the circular icon on the right, indicating that the confidence level setting for automatic confirmation is enabled.

[0097] The recognition result display area RA also contains the following buttons: "Cancel Verification" button B1, "Confirm All" button B2, and "Clear" button B3. The "Cancel Verification" button B1 is used to stop the verification process. Pressing this button will discard the results of the verification process performed up to that point.

[0098] The "Confirm All" button B2 is used to signal the completion of the confirmation of all recognition results that need to be verified. When this button B2 is pressed, all recognition results that the verifier KS needs to confirm will be considered correct. The "Clear" button B3 is used to clear the results obtained during the verification process up to that point. If this button B3 is pressed, the verifier KS will have to start the verification process from the beginning.

[0099] The strings mentioned above, the toggle button SD, and buttons B1-B3 are all placed at the top of the recognition result display area RA, regardless of the type of setting that is enabled. The recognition results that the verifier KS should check are placed below them. As shown in Figure 11, the recognition results are placed as field values ​​in boxes NB arranged vertically. The contents of each box NB can be edited as needed. This allows the verifier KS to correct the recognition result in box NB if they determine from the item image data displayed in the image display area GA that the recognition result is incorrect.

[0100] The display of item image data in the image display area GA may be done by masking the string MR, which is not the target of verification, on the report image CG, as shown in Figure 1, but this is not always necessary. For example, to make it easier to understand the correspondence with box NB, the report image data for which recognition results have been obtained may be placed within box NB, aligned with the position above box NB in ​​the vertical direction.

[0101] In the recognition result display area RA, each box NB has the following elements arranged from closest to the box NB: confidence level KD, checkbox CH, field name (item name) FN, and comment button CB. The confidence score KD is the actual confidence score calculated from the recognition result displayed in box NB. The checkbox CH is placed to indicate whether or not the verification of the recognition result in box NB has been completed. The verifier KS performs an operation to display a check mark on checkbox CH when they determine that the recognition result is correct or when they correct the recognition result to what they believe to be correct. For example, in box NB, when a check mark is displayed on checkbox CH, the recognition result cannot be edited.

[0102] The field name FN is, for example, a string representing the item name extracted from a template. The comment button CB is placed there to allow users to save information to be shared as a comment. By operating this comment button CB, a comment input screen will appear, for example, as a pop-up. This allows the reviewer KS to enter the necessary information and share it with others, such as the approver SS.

[0103] In the example shown in Figure 11, a checkmark is displayed in the checkbox CH for the recognition result with a confidence level KD of 99.99%. This checkmark is automatically displayed because the automatic confirmation confidence level set for this item is less than 99.99%. Editing of the recognition result in box NB is also not possible. This service automatically displays a checkmark in the checkbox CH, eliminating the need to check the corresponding recognition result. Note that a checkmark is not displayed in the checkbox CH for the recognition result with a confidence level of 99.91%, indicating that the automatic confirmation confidence level set for this item is greater than 99.91%. Furthermore, if the checkmark is removed by manipulating the checkbox CH, it becomes possible to modify the recognition result.

[0104] Figure 12 shows an example of what is displayed in the recognition result display area when the confidence level for automatic confirmation and automatic confirmation of blanks are enabled. When automatic confirmation of blanks is enabled, as shown in Figure 12, a check mark will appear in all checkboxes CH for items whose recognition result is blank. This means that items with a blank recognition result are excluded from the verification process. Similarly, a check mark will appear in the checkbox CH for items with a verification success rate of 99.99%, indicating that they are also excluded from the verification process.

[0105] Figure 13 shows an example of what is displayed in the recognition result display area when the confidence and match rates for automatic determination are enabled and the calculation formulas are set. The formula set here is that Lot Code 1 in Case 1 is equal to Lot Code (Pattern 2). The three items used in this formula are the bottom three items. Since the recognition result for the item name "Lot Code (Pattern 2)" is blank, this formula is not valid. Therefore, no check symbol is displayed in any of the checkboxes CH for these three items.

[0106] In the example shown in Figure 13, the top three items satisfy the automatic confirmation confidence and match settings, resulting in a checkmark appearing in the checkbox CH and excluding them from the verification process. The item name "Case 1" has a confidence level KD of 99.73%, and if no calculation formula is set, a checkmark appears in the checkbox CH. However, since the calculation formula is not valid, the checkmark does not appear in the checkbox CH and it is not excluded from the verification process. As is clear from this, this service prioritizes the set calculation formula, and if the calculation formula is not valid, all items used in that formula are included in the verification process, even if other settings are met.

[0107] The confirmation processing unit 115 refers to the setting information stored in the setting information storage unit 184, causes the screen generation unit 118 to generate a confirmation screen, and sends it to the confirmer terminal 6. This allows the confirmer KS to perform the confirmation work, and the results of that confirmation work are stored in the confirmation result storage unit 186 reserved in the memory unit 18.

[0108] The verification results obtained through the verification process are sent from the verifier terminal 6 to the AP server 1 by, for example, an operation on an unillustrated "Send" button located on the verification screen. The information sent as a single item in the verification result includes at least a field value. The comment text, which is a string entered as a comment using the comment button CB, is sent to the AP server 1 and temporarily stored, for example, when the comment input is completed. When the verification result is sent, the stored comment text is added as one of the pieces of information in the corresponding item and stored.

[0109] To store the verification results, the verifier KS must ensure that a check mark is displayed in the checkbox CH for all items being verified before clicking the "Submit" button. If a check mark is not displayed in any of the checkbox CH, clicking the "Submit" button will be invalid.

[0110] Thus, when mask settings are enabled, the verification processing unit 115 selects items to be verified by the verifier KS, has the verifier perform verification only for the selected items, and stores the results obtained from that verification in the verification result storage unit 186. In this way, the verification processing unit 115 functions as part of the distribution unit K2 shown in Figure 1.

[0111] The approval processing unit 116 provides an environment for approvers SS to perform approval work. The processing content for this purpose is basically the same as that of the confirmation processing unit 115. However, there are the following differences. Unlike the verification process, the verification results stored in the verification result storage unit 186 are placed on the approval screen (hereinafter referred to as the "approval screen") and sent to the approver terminal 7. As a result, the approval result obtained by the approver SS using the approver terminal 7 is sent from the approver terminal 7 to the AP server 1, and the approval processing unit 116 stores it in the approval result storage unit 187 reserved in the memory unit 18. The approval result stored in the approval result storage unit 187 becomes the document recognition result using the document image data. For this reason, the approval processing unit 116 functions as part of the distribution unit K2 shown in Figure 1, just like the verification processing unit 115. In other words, the distribution unit K2 is implemented as the verification processing unit 115 and the approval processing unit 116.

[0112] This approval processing unit 116 corresponds to the validation means in this embodiment. Furthermore, both the approval processing unit 116 and the verification processing unit 115 correspond to the verifier assignment means in this embodiment. Since the transmission of item image data corresponding to part image information is realized by the verification processing unit 115 and the screen generation unit 118, and by the approval processing unit 116 and the screen generation unit 118, the verification processing unit 115, the approval processing unit 116, and the screen generation unit 118 correspond to the part image transmission means in this embodiment.

[0113] As described above, this service allows for multi-check settings. When multi-check settings are enabled, the same item is assigned to two or more verifiers KS, and the verification processing unit 115 performs the verification work for each, although the processing content is basically the same. However, the approval processing unit 116 must handle the fact that the approval results from two or more approvers SS will differ, since the approval work is performed by two or more approvers SS. For this reason, the approval processing unit 116 compares the approval results from each approver SS for each item, and if there are items with different approval results, it may, for example, have another approver SS perform the approval work to determine the content of the different approval results from each approver SS. The approvers SS to be further approved may simply be any approver SS that can be booked, or the number of approvers SS that can be booked may be restricted. Since the judgments of approvers SS may differ, for example, it may be possible to only allow bookings from approvers SS who are recognized as highly skilled and in positions of responsibility.

[0114] The approval result output unit 117 processes the approval results stored in the approval result storage unit 187 to output them to the customer company CK as document recognition results. To this end, the approval result output unit 117 controls, for example, the screen generation unit 118 to generate and send a list screen containing a list of obtainable recognition results, and to send the recognition results specified on that list screen.

[0115] Next, the operation of the verification processing unit 115 and the approval processing unit 116 will be explained in more detail by referring to the flowcharts shown as examples in Figures 14 to 17. Figures 14 and 15 are flowcharts illustrating an example of the confirmation screen generation process. This confirmation screen generation process is executed when a confirmation request for the scheduled confirmation work is received from the verifier terminal 6, as described above. This is achieved by the CPU 11 executing the developed application described above. First, we will refer to Figures 14 and 15 to explain in detail the example of the confirmation screen generation process. Here, the entity executing the process will be referred to as the confirmation processing unit 115.

[0116] First, in step S1, the confirmation processing unit 115 identifies the corresponding template from the document ID stored in the confirmation work request, and uses the identification result to check the setting information stored in the setting information storage unit 184. In the following step S2, the confirmation processing unit 115 determines whether or not the mask setting is enabled based on the result of checking the setting information. If the mask setting is enabled, the determination in step S2 is YES and the process proceeds to step S3. If the mask setting is not enabled, the determination in step S2 is NO and the process proceeds to step S5.

[0117] In step S3, the verification processing unit 115 extracts the target portion, i.e., the item image data, from the document image data stored in the document image data storage unit 181, according to each enabled setting and the item assignments made by another verifier KS in the past. In the following step S4, the verification processing unit 115 extracts the target portion from the recognition results stored in the recognition result storage unit 182, according to each enabled setting and the item assignments made by another verifier KS in the past. After that extraction, the process proceeds to step S7.

[0118] If mask settings are enabled, all items must be assigned to two or more verifiers (KS). Therefore, item assignments must be made considering the assignment of items to other verifiers (KS) (see Figure 1). For this reason, even if multiple check settings are enabled, the content of the verification work screen generation process will remain almost the same.

[0119] Meanwhile, in step S5, the verification processing unit 115 extracts item image data for each item (field) from the document image data stored in the document image data storage unit 181. In the following step S6, the verification processing unit 115 extracts the recognition results, that is, the recognition results for each item (field), stored in the recognition result storage unit 182. After that extraction, the process proceeds to step S7.

[0120] In step S7, the confirmation processing unit 115 determines whether the setting for automatic blank confirmation is enabled. If the setting is enabled, the determination in step S7 is YES and the process proceeds to step S8. If the setting is not enabled, the determination in step S7 is NO and the process proceeds to step S9.

[0121] In step S8, the verification processing unit 115 excludes items from the verification process that have blank recognition results from the items assigned in step S4 or S6. As a result of this exclusion, the checkbox CH for items that do not require verification will initially display a check mark. After performing such exclusions as necessary, the process proceeds to step S9.

[0122] In step S9, the confirmation processing unit 115 determines whether the setting of the confidence level for automatic confirmation is enabled. If the setting is enabled, the determination in step S9 is YES and the process proceeds to step S10. If the setting is not enabled, the determination in step S9 is NO and the process proceeds to step S11.

[0123] In step S10, the verification processing unit 115 excludes items from the verification process that meet the automatic confirmation confidence level setting from the items assigned in step S4 or S6. After performing such exclusions as necessary, the process proceeds to step S11.

[0124] In step S11, the verification processing unit 115 determines whether the setting of confidence and match for automatic confirmation is valid. If the setting is valid, the determination in step S11 is YES and the process proceeds to step S12. If the setting is not valid, the determination in step S11 is NO and the process proceeds to step S21 in Figure 15.

[0125] In step S12, the verification processing unit 115 excludes items from the verification process that satisfy the automatic confirmation confidence level and agreement level settings from the items assigned in step S4 or S6. After performing such exclusions as necessary, the process proceeds to step S21 in Figure 15. As mentioned above, the setting of the automatic confirmation confidence level and the setting of the automatic confirmation confidence level and agreement level cannot be enabled simultaneously. Therefore, only one of steps S10 or S12 will be executed, if any.

[0126] In step S21, the confirmation processing unit 115 determines whether a calculation formula is set. If a calculation formula is set, the determination in step S21 is YES and the process proceeds to step S22. If a calculation formula is not set, the determination in step S21 is NO and the process proceeds to step S26.

[0127] In step S22, the verification processing unit 115 determines whether the item assigned in step S4 or S6 contains an item used in the calculation formula. If an item used in the calculation formula is assigned, the determination in step S22 is YES and the process proceeds to step S23. If an item used in the calculation formula is not included, the determination in step S22 is NO and the process proceeds to step S26.

[0128] In step S23, the verification processing unit 115 performs the calculation using the formula. In the following step S24, the verification processing unit 115 determines whether the formula is true or false. If the formula is true, the determination in step S24 is YES and the process proceeds to step S25. If the formula is false, the determination in step S24 is NO and the process proceeds to step S26. If the calculation formula fails, any items used in and assigned to that formula that were excluded from the verification process due to fulfilling some valid setting will have their exclusion reversed and will be treated as items subject to verification (see Figure 13).

[0129] In step S25, the verification processing unit 115 excludes items used in the calculation formula from the items assigned in step S4 or S6 from the verification process. After performing such exclusions as necessary, the process proceeds to step S26. In step S26, the confirmation processing unit 115 determines whether or not the correction extraction setting has been made. If the setting has been made, the determination in step S26 is YES and the process proceeds to step S27. If the setting has not been made, the determination in step S26 is NO and the process proceeds to step S29.

[0130] In step S27, the verification processing unit 115 determines whether or not there is an item that has been corrected among the items assigned in step S4 or S6. If the assigned items include an item that has been corrected, the determination in step S27 is YES and the process proceeds to step S28. If the assigned items do not include an item that has been corrected, the determination in step S27 is NO and the process proceeds to step S29.

[0131] In step S28, the verification processing unit 115 extracts the recognition result of the corrected area from the recognition result storage unit 182, and, referring to the range coordinate information in the extracted recognition result, extracts item image data for the range specified by the range coordinate information from the form image data. The extracted item image data is the image data of the correction input area where the corrected data TMR is entered. Typically, that correction input area is the set extraction area KA. The verification work screen displays at least the image of the correction input area and the recognition result for that correction input area.

[0132] If the process proceeds to step S29, the image data for each item to be placed on the confirmation screen, the recognition results, and whether or not each recognition result is automatically confirmed have been determined. Based on this, in step S29, the confirmation processing unit 115 causes the screen generation unit 118 to generate the confirmation screen based on the determined image data for each item, the recognition results, and whether or not each recognition result is automatically confirmed. After generating the screen, the confirmation screen is sent to the verifier terminal 6, and then the confirmation screen generation process ends.

[0133] In this way, the verification screen generation process not only checks whether the mask setting is enabled or not, but also identifies which of the assigned items should be automatically confirmed according to the various enabled settings, and excludes the identified items from the verification process. Therefore, in accordance with the client company CK's requests, the verification process can be performed by the verifier KS in a way that narrows down the items to be checked, without reducing the accuracy of the recognition results or minimizing any reduction in accuracy. By narrowing down the items to be checked, the turnaround time from the request to receiving the recognition results is shortened for the client company CK, and the cost of using this service is reduced.

[0134] The following section will provide a more detailed explanation of the processes performed by the approval processing unit 116. Figure 16 is a flowchart showing an example of the approval work screen generation process. This approval work screen generation process is executed when an approval work execution request for the execution of a scheduled approval work is received from the approver terminal 7. Similar to the confirmation work screen generation process, this is achieved by the CPU 11 executing the above-described development application. This is also the case for the approval result processing described later. In the process executed by the approval processing unit 116, we will first refer to Figure 16 and explain in detail the example of the approval work screen generation process. Here, the entity executing the process will be described as the approval processing unit 116. Note that the approval work execution request is a request that can be sent after all confirmation work has been completed.

[0135] First, in step S31, the approval processing unit 116 confirms the items that the approver SS will perform the approval work on. Here, the template is identified from the document ID stored in the approval work request, and the valid settings are also confirmed by referring to the setting information from the identification result. This is because if the mask setting is enabled and it is necessary to have two or more approvers SS perform the approval work, the items that will be performed on the approval work must be assigned to each approver SS. For this reason, by executing the process in step S31, items are assigned to the approver SS who sent the approval work request. In Figure 16, these items are labeled as "Approval Request Items".

[0136] In the next step, S32, the approval processing unit 116 extracts the confirmation results of the assigned items from the confirmation result storage unit 186. In the following step, S33, the approval processing unit 116 determines whether the multiple check setting is enabled or disabled. If the setting is enabled, the determination in step S33 is YES and the process proceeds to step S34. If the setting is disabled, the determination in step S33 is NO and the process proceeds to step S37.

[0137] If the multiple check setting is enabled, multiple confirmation results for the same item will be extracted. Therefore, in step S34, the approval processing unit 116 compares the confirmation results for each item and identifies the items with differing confirmation results. In the next step S35, the approval processing unit 116 determines whether or not there are any items with differing confirmation results among the items assigned in step S31. If there are items with differing confirmation results, the determination in step S35 is YES and the process moves to step S36. If there are no items with differing confirmation results, the determination in step S35 is NO and the process moves to step S37.

[0138] In step S36, the approval processing unit 116 adds a message to be displayed for items where the confirmation results differ. This message is displayed to encourage users to pay more attention to the approval process for items where the confirmation results differ. Both differing confirmation results are selected as items to be displayed. After making such selections, the process proceeds to step S37.

[0139] In step S37, the approval processing unit 116 extracts the item image data of the assigned item from the corresponding document image data stored in the document image data storage unit 181. In the following step S38, the approval processing unit 116 instructs the screen generation unit 118 to generate an approval work screen in which the extracted item image data and confirmation results are placed. After the approval work screen is sent to the approver terminal 7, the approval work screen generation process ends.

[0140] Thus, in the approval screen generation process, just like in the verification screen generation process, items are assigned to approvers (SS) not only based on whether mask settings are enabled or not, but also according to various enabled settings. When the multi-check setting is enabled, items with differing verification results are identified, and the differing verification results are placed on the approval screen along with a message. Therefore, when the multi-check setting is enabled, approvers (SS) can easily grasp the skills or quality of each verifier (KS) in their verification work through the approval screen.

[0141] Figure 17 is a flowchart showing an example of approval result processing. This approval result processing is executed when the approval result, which is the result of the approval work, is received from the approver terminal 7. Finally, we will refer to Figure 17 and explain the example of approval result processing in detail. Here again, we will assume that the entity executing the processing is the approval processing unit 116.

[0142] First, in step S41, the approval processing unit 116 stores the approval result received from the approver terminal 7 in the approval result storage unit 187. In the following step S42, the approval processing unit 116 extracts the corresponding recognition result from the recognition result storage unit 182. In the subsequent step S43, the approval processing unit 116 confirms the items (approval request items) assigned to the approver SS.

[0143] In the next step, S44, the approval processing unit 116 determines whether the multiple check setting is enabled or disabled. If the setting is enabled, the determination in step S44 is YES and the process proceeds to step S45. If the setting is disabled, the determination in step S44 is NO and the process proceeds to step S50.

[0144] In step S45, the approval processing unit 116 determines whether the approval result is due to a specific approval setting. This specific approval is set when, due to the multiple check setting, two or more approvers SS perform the approval work and the approval results for the same item differ. This specific approval setting causes another approver SS to perform the approval work to finalize the approval result for items where the approval result differs. If an approval result sent through such an approval work is received, the determination in step S45 is YES and the process proceeds to step S50. If the received approval result is not due to a specific approval setting, the determination in step S45 is NO and the process proceeds to step S46.

[0145] Furthermore, even if specific approval is set when the multiple check setting is enabled, the multiple check setting is not applied because the approval work is performed by only one approver SS. As a result, in the approval work screen generation process described above, the judgment in step S33 will be NO. In step S31, items with different approval results will be assigned.

[0146] In step S46, the approval processing unit 116 determines whether or not there are items awaiting approval. If there are still approval tasks to be performed by other approvers SS, the determination in step S46 is YES, and the approval result processing ends here. If there are no remaining approval tasks to be performed by other approvers SS, the determination in step S46 is NO, and the process proceeds to step S47.

[0147] In step S47, the approval processing unit 116 compares the approval results for each item and identifies items with different approval results. In the next step, S48, the approval processing unit 116 determines whether or not there are items with different approval results. If such items exist, the determination in step S48 is S and the process proceeds to step S49. If no such items exist, the determination in step S48 is NO and the process proceeds to step S60.

[0148] In step S49, the approval processing unit 116 sets items with different approval results as specific approval targets requiring further approval. In the following step S50, the approval processing unit 116 focuses only on items with identical approval results and compares the approval results with the recognition results to identify items where the approval result differs from the recognition result, i.e., items where the recognition result has been corrected, as correction items. The process then proceeds to step S51.

[0149] In step S51, the approval processing unit 116 refers to the confidence level obtained from the recognition results and updates the number of AIOCR corrects and the number of validated items by item and confidence level. In the next step, S52, the approval processing unit 116 calculates the degree of agreement by item and confidence level using the updated number of AIOCR corrects and the number of validated items, and updates the degree of agreement. After this update, the approval result processing is completed.

[0150] Steps S50 to S52 are performed only on the items represented by the received approval results. As a result, the AIOCR accuracy count and validation count are updated separately by confidence level (see Figure 6) only for items for which the approval result has been confirmed as a recognition result, and the degree of agreement is also updated as a result of these updates.

[0151] In this embodiment, the recognition result is reviewed by verifier KS, and the review result is approved by approver SS. The approval result from approver SS is then provided to customer company CK as the document recognition result. However, such a two-stage review is not mandatory. In other words, the review result obtained by verifier KS may be used as the document recognition result. In this case, the review processing unit 115 corresponds to the validation means. It may also be possible to configure whether or not a two-stage review is necessary.

[0152] Furthermore, in this embodiment, character recognition is performed to allow the user to confirm the recognition result and correct it as necessary. However, character recognition may be omitted, and the data entry work of the string MR represented by the item image data may be performed by the verifier KS. The setting of whether or not to perform character recognition may also be made available to the customer company CK, etc.

[0153] The settings for the confidence level of automatic confirmation and the confidence level and match level of automatic confirmation are currently left to the BPO Center BC, but based on past history, it may be possible to make one of them automatically configurable. To ensure appropriate automatic configuration, it may also be possible to have the customer company CK set only the required level of accuracy. Even if automatic configuration is possible, it may not be possible to perform appropriate automatic configuration if the sample size is small. Even if the sample size is sufficient, it is not always possible to perform appropriate automatic configuration. For these reasons, even if automatic configuration is enabled, it is necessary to allow the BPO Center BC to arbitrarily change the settings. The settings for the confidence level of automatic confirmation and the confidence level and match level of automatic confirmation are currently made on an item-by-item basis, but it may also be possible to make them made on an item group basis or on a report basis.

[0154] Many of the settings that BPO Center BC can perform must be done according to the requests of the client company CK. For this reason, it may be acceptable to allow client company CK to perform one or more of the settings. Alternatively, client company CK could be asked to select the required accuracy for document recognition or the pricing plan for using this service, and the various settings could be automatically performed or performed by BPO Center BC based on the selection. For these reasons, it may be acceptable to have the various settings performed indirectly. [Explanation of symbols]

[0155] 1 AP Server, 5 Customer Terminal, 6 Confirmer Terminal, 7 Approver Terminal, 11 CPU, 18 Storage Unit, 19 Communication Unit, 111 Request Processing Unit, 112 Document Recognition Unit, 113 Configuration Processing Unit, 114 Reservation Processing Unit, 115 Confirmation Processing Unit, 116 Approval Processing Unit, 117 Approval Result Output Unit, 118 Screen Generation Unit, 181 Document Image Data Storage Unit, 182 Recognition Result Storage Unit, 183 Template Storage Unit, 184 Configuration Information Storage Unit, 185 Reservation Information Storage Unit, 186 Confirmation Result Storage Unit, 187 Approval Result Storage Unit, BC BPO Center, CG, CG1~CG4 Document Images, KS1, KS2 Confirmers, SK Service Provider, SS1, SS2 Approvers.

Claims

1. A part identification means that, in response to receiving image information representing a plurality of input information images, takes an image of a string of characters written on a printed document as an input information image, identifies a plurality of parts on the target image of the printed document represented by the image information where each of the input information images exists, A means for assigning one or more parts to each of the aforementioned multiple parts, A part image transmission means that transmits part image information representing the input information image of the part assigned by the verifier assignment means to each of the multiple verifiers' verifier terminals, A validation means that, upon transmission of the part image information to the verifier terminal, determines the input information represented by the input information image based on the results of each of the multiple verifiers confirming the input information image represented by the part image information, Equipped with, The aforementioned part identification means includes a character recognition means that performs character recognition for each of the plurality of parts and outputs the recognition result of the input information image and the confidence level of the recognition result, The validation means can confirm the recognition result as input information if the confidence level of the recognition result is within a set range, and the degree of agreement calculated using past recognition results within the set range and the results of the verifier's verification of the past recognition results is equal to or greater than a set value. Information processing device.

2. The aforementioned part image transmission means is capable of transmitting the same part image information redundantly to the respective verifier terminals used by different verifiers. The validation means, upon transmission of the same part image information, confirms the input information image represented by the same part image information if different verifiers each verify it, and then confirms the input information based on the verification results of each of the different verifiers. The information processing apparatus according to claim 1.

3. The validation means is capable of determining the blank recognition result as the input information image when the recognition result of the character recognition means is blank. The information processing apparatus according to claim 1.

4. The validation means can confirm that a predetermined calculation formula is satisfied between the recognition results of the input information image at multiple locations by the character recognition means, and can then determine each of the recognition results at multiple locations as the input information. The information processing apparatus according to claim 1.

5. The part identification means can determine whether or not the input information image has been corrected using the recognition result of the character recognition means, and if it is determined that a correction has been made to invalidate the input information image, it can identify another part. The information processing apparatus according to claim 1.

6. A computer program that causes an information processing device to perform processing, A part identification step in which, in response to receiving image information representing multiple input information images, an image of a string of characters written on a printed document is used as the input information image, and the image information represents a part that identifies multiple parts on the target image of the printed document in which each of the input information images exists, The process involves dividing the aforementioned multiple parts and assigning one or more parts to each of the multiple verifiers, A part image transmission step in which each of the multiple verifiers transmits part image information representing the input information image of the assigned part to their respective verifier terminals, A validation step in which, by transmitting the part image information to the verifier terminal, the multiple verifiers each verify the input information image represented by the part image information, and the input information represented by the input information image is determined based on the results of the verification. It includes, The aforementioned part identification step includes a character recognition step in which character recognition is performed for each of the multiple parts, and the recognition result of the input information image and the confidence level of the recognition result are output. In the validation step, if the confidence level of the recognition result is within the set range, and the degree of agreement calculated using past recognition results within the set range and the results of the verifier's verification of the past recognition results is equal to or greater than the set value, the recognition result can be confirmed as the input information. Computer program.

Images