Information processing system and information processing device
Patent Information
- Application Number
- PCT/JP2026/005836
- Authority / Receiving Office
- WO · WO
- Patent Type
- Applications
- Current Assignee / Owner
- Priority Date
- 2025-02-28
- Filing Date
- 2026-02-18
- Publication Date
- 2026-09-03
Smart Images
Figure JP2026005836_03092026_PF_FP_ABST
Abstract
Description
INFORMATION PROCESSING SYSTEM AND INFORMATION PROCESSING APPARATUS
[0001] The present invention relates to an information processing system and an information processing apparatus that extract information from an electronic image document by character recognition processing.
[0002] Conventionally, there has been known an information processing system that allows a user to perform verification work on information extracted from an electronic image document by character recognition processing (see, for example, Patent Documents 1 to 4).
[0003] Japanese Unexamined Patent Application Publication No. 2021-117962, Japanese Unexamined Patent Application Publication No. 2021-163447, Japanese Unexamined Patent Application Publication No. 2021-144617, Japanese Unexamined Patent Application Publication No. 2022-137608
[0004] However, conventional information processing systems have a problem in that verification errors occur.
[0005] Therefore, an object of the present invention is to provide an information processing system and an information processing apparatus that can reduce the possibility of occurrence of verification errors.
[0006] The information processing system of the present invention comprises: a character extraction unit that extracts information for each of a plurality of attributes from an electronic image document by character recognition processing; and a verification unit that causes a user to perform verification work on the information extracted by the character extraction unit. Each of the plurality of attributes is associated with a classification of the work, at least a part of the plurality of attributes can be associated with any one of a plurality of classifications in which at least a part of operations required to complete the verification are different from each other, and the verification unit causes the user to perform the work according to the classification associated with the attribute corresponding to the information.
[0007] With this configuration, when causing a user to perform verification work on information extracted by character recognition processing for each of a plurality of attributes from an electronic image document, the information processing system of the present invention causes the user to perform work according to the classification associated with the attribute corresponding to the information. Therefore, operations required to complete verification may differ depending on the attribute corresponding to the information, and the possibility of occurrence of verification errors can be reduced.
[0008] In the information processing system of the present invention, at least some of the multiple attributes may be associated with the classification, for which multiple operations are required to be performed in order to complete the verification.
[0009] In the information processing system of the present invention, each of the plurality of classifications is associated with a classification color as the color used when displaying the area corresponding to the attribute in the image, and at least some of the plurality of classifications can be associated with different classification colors, and the verification unit may display the area with the classification color associated with the classification that is associated with the attribute.
[0010] In the information processing system of the present invention, the verification unit may display the area corresponding to the attribute corresponding to the information for which the verification has been completed using a completion color, rather than the classification color, as a color indicating that the verification has been completed.
[0011] In the information processing system of the present invention, the verification unit may display a specific frame in the area corresponding to the attribute corresponding to the information currently being verified.
[0012] In the information processing system of the present invention, the verification unit may have the user perform the task for each piece of information, and when the verification of the information is completed, if there is any information that has not been verified, it may automatically make any of the information that has not been verified the subject of the verification.
[0013] In the information processing system of the present invention, the verification unit may have the user perform the task for each piece of information, and may also indicate the operations necessary to complete the verification of the current target information using an image of a mouse pointer.
[0014] In the information processing system of the present invention, if there are multiple operations necessary to complete the verification of the current target information, the verification unit may change the image of the mouse pointer each time one of the operations is performed.
[0015] In the information processing system of the present invention, the verification unit may have the user perform the task for each piece of information and may display an explanation of the operations necessary to complete the verification of the current target information.
[0016] In the information processing system of the present invention, if there are multiple operations necessary to complete the verification of the current target information, the verification unit may change the description each time one of the operations is executed.
[0017] The information processing apparatus of the present invention comprises a character extraction unit that extracts information from an electronic document image for each of a plurality of attributes by character recognition processing, and a verification unit that causes a user to perform verification work on the information extracted by the character extraction unit, wherein each of the plurality of attributes is associated with a classification of the work, and at least a portion of the plurality of attributes can be associated with one of a plurality of classifications in which at least a portion of the operations required to complete the verification differ from each other, and the verification unit causes the user to perform the work according to the classification associated with the attribute corresponding to the information.
[0018] With this configuration, when the information processing device of the present invention allows a user to perform the verification of information extracted by character recognition processing for each of multiple attributes from an electronic document image, the user is instructed to perform tasks according to the classification associated with the attribute corresponding to the information. Therefore, the operations required to complete the verification may differ depending on the attribute corresponding to the information, and the possibility of verification errors can be reduced.
[0019] The information processing system and information processing device of the present invention can reduce the possibility of errors occurring during verification.
[0020] This is a block diagram of an example system according to one embodiment of the present invention. This is a block diagram of an example information processing system shown in Figure 1 when it is configured with one computer. This is a diagram showing an example of user management information shown in Figure 2. This is a diagram showing an example of an image electronic document shown in Figure 2. This is a diagram showing an example of a template shown in Figure 2. This is a diagram showing an example of an extracted information file shown in Figure 2. This is a diagram showing an example of file-related information shown in Figure 2. This is a diagram showing an example of verification task list information shown in Figure 2. This is a diagram showing an example of verification work classification information shown in Figure 2. This is a block diagram of an example of a user terminal shown in Figure 1 when it is configured with a PC. This is a sequence diagram of the system operation shown in Figure 1 when a user logs into the information processing system. This is a diagram showing an example of the login screen displayed in the operation shown in Figure 11. This is a sequence diagram of the system operation shown in Figure 1 when an image electronic document is input. This is a sequence diagram of the system operation shown in Figure 1 when a verification work classification is set. This is a sequence diagram of the system operation shown in Figure 1 when a template is created. This is a diagram showing an example of the template creation screen displayed in the operation shown in Figure 15. This is a sequence diagram of the system operation shown in Figure 1 when information is extracted from an image electronic document. This is a sequence diagram of the system operation shown in Figure 1 when a verification task is started. This figure shows an example of the verification task list screen displayed in the operation shown in Figure 18. This is a flowchart of part of the operation of the information processing system shown in Figure 2 when a verification task is executed. This is a flowchart that continues from the flowchart shown in Figure 20. This figure shows an example of the verification screen displayed in the operations shown in Figures 20 and 21. This figure shows an example of the verification screen shown in Figure 22 when the first extraction area in the verification work order is the target extraction area. This is a flowchart of the verification task acceptance process in the operations shown in Figures 20 and 21. This is a magnified view of the area near the mouse pointer in the verification screen shown in Figure 22. This is a magnified view of the area near a mouse pointer different from the mouse pointer shown in Figure 25A in the verification screen. This is a magnified view of the area near a mouse pointer different from the mouse pointer shown in Figures 25A and 25B in the verification screen.This is a magnified view of the area near a mouse pointer different from the mouse pointer shown in Figures 25A-C of the verification screen. This is a magnified view of the area near a mouse pointer different from the mouse pointer shown in Figures 25A-C or Figure 26A of the verification screen. This is a magnified view of the area near the help image of the verification screen shown in Figure 22. This is a diagram showing an example of the verification screen shown in Figure 22 when some of the extraction areas are displayed in a completed color. This is a diagram showing an example of a pop-up displayed in the operation shown in Figures 20 and 21.
[0021] Hereinafter, embodiments of the present invention will be described with reference to the drawings.
[0022] First, the configuration of a system according to one embodiment of the present invention will be described.
[0023] Figure 1 is a block diagram of an example of the system 10 according to this embodiment.
[0024] As shown in Figure 1, System 10 includes an information processing system 20 that extracts information from an image electronic document (hereinafter referred to as "image electronic document") by character recognition processing (for example, OCR (Optical Character Recognition)). The information processing system 20 may be composed of one computer such as a PC (Personal Computer), or it may be composed of multiple computers. The information processing system 20 may be configured on the cloud.
[0025] System 10 includes a user terminal 30 used by users. System 10 may also include at least one other user terminal with a configuration similar to that of user terminal 30. The user terminal may be a computer such as a PC or tablet.
[0026] Figure 2 is a block diagram of an example of an information processing system 20 that is composed of a single computer.
[0027] As shown in Figure 2, the information processing system 20 includes an operation unit 21 which is an operation device such as a keyboard, mouse, or touchpad into which various operations are input; a display unit 22 which is a display device such as an LCD (Liquid Crystal Display) that displays various information; a communication unit 23 which is a communication device that communicates with external devices via a network such as a LAN (Local Area Network) or the Internet, or directly via wired or wireless connection without going through a network; a storage unit 24 which is a non-volatile storage device such as a semiconductor memory or HDD (Hard Disk Drive) that stores various types of information; and a control unit 25 which controls the entire information processing system 20.
[0028] The storage unit 24 is capable of storing an information processing program 24a for extracting information from image electronic documents. The information processing program 24a may, for example, be installed in the information processing system 20 during the manufacturing stage, or it may be additionally installed in the information processing system 20 from an external storage medium such as a USB (Universal Serial Bus) memory, or it may be additionally installed in the information processing system 20 from a network.
[0029] The memory unit 24 is capable of storing user management information 24b for managing users of the information processing system 20.
[0030] Figure 3 shows an example of user management information 24b.
[0031] As shown in Figure 3, the user management information 24b includes, for each user, a "user ID" as user identification information, a "password" for the user, and a "role" for the user. Examples of "role" values in the user management information 24b include "administrator" and "general user." The user management information 24b shown in Figure 3 is depicted with some information omitted.
[0032] As shown in Figure 2, the storage unit 24 is capable of storing image electronic documents 24c. The storage unit 24 is also capable of storing image electronic documents other than image electronic documents 24c.
[0033] Figure 4 shows an example of an image electronic document 24c.
[0034] The image electronic document 24c shown in Figure 4 is an image electronic document of a receipt.
[0035] As shown in Figure 2, the storage unit 24 can store templates 24d for extracting information from image electronic documents. The storage unit 24 can also store templates other than templates 24d for extracting information from image electronic documents.
[0036] Figure 5 shows an example of template 24d.
[0037] The template 24d shown in Figure 5 includes, for each region from which information is extracted (hereinafter referred to as the "extraction region"), the location, attributes, a classification ID as identification information for the classification of the verification work (hereinafter referred to as the "verification work classification") of the verification work (hereinafter referred to as the "verification work classification") of the information extracted from the image electronic document by character recognition processing, and the order of the verification work (hereinafter referred to as the "verification work order"). The locations "100, 100, 200, 150" shown in Figure 5 represent, for example, the X coordinate of the top-left vertex of the extraction region, the Y coordinate of the top-left vertex, the X coordinate of the bottom-right vertex, and the Y coordinate of the bottom-right vertex from left to right, but may also be represented by the coordinates of the vertices of the extraction region, the height, and the width. The template 24db shown in Figure 5 is drawn with some information omitted.
[0038] As shown in Figure 2, the storage unit 24 can store a file 24e containing information extracted from an image electronic document (hereinafter referred to as "extracted information file"). The storage unit 24 can also store other extracted information files besides the extracted information file 24e.
[0039] Figure 6 shows an example of the extracted information file 24e.
[0040] The extracted information file 24e shown in Figure 6 contains information extracted from the image electronic document (hereinafter referred to as "extracted information") for each attribute. Some information is omitted in the extracted information file 24e shown in Figure 6.
[0041] As shown in Figure 2, the storage unit 24 can store image electronic documents and file-related information 24f indicating information related to the extracted information file.
[0042] Figure 7 is a diagram showing an example of file-related information 24f.
[0043] The file-related information 24f shown in Figure 7 includes a file ID as identification information of an extracted information file, identification information of an image electronic document that is a source for generating the extracted information file (hereinafter, the identification information of the image electronic document is referred to as "document ID"), identification information of a template used for generating the extracted information file (hereinafter, the identification information of the template is referred to as "template ID"), and the progress status of verification of the extracted information file for each extracted information file. The value of the verification progress status is "completed normally" when the verification of the extracted information file is completed normally, "verification aborted" when the verification of the extracted information file is aborted, and is blank in other cases. The file-related information 24f shown in Figure 7 is illustrated with part of the information omitted.
[0044] As shown in Figure 2, the storage unit 24 can store verification task list information 24g that indicates a list of tasks for verification work, that is, work for verifying that information included in the extracted information file is information shown in an image presented in an image electronic document (hereinafter referred to as "verification tasks").
[0045] Figure 8 is a diagram showing an example of verification task list information 24g.
[0046] The verification task list information 24g shown in Figure 8 includes a task ID as identification information of a verification task, and a file ID of an extracted information file that is a target of the verification task for each verification task. The verification task list information 24g shown in Figure 8 is illustrated with part of the information omitted.
[0047] As shown in Figure 2, the storage unit 24 can store verification work classification information 24h that indicates verification work classifications.
[0048] Figure 9 is a diagram showing an example of verification work classification information 24h.
[0049] The verification work classification information 24h shown in Figure 9 indicates, for each verification work classification, the classification ID, the name of the verification work classification, the classification color used to indicate the area targeted by the verification work, and the operations required to complete the verification of the extracted information (hereinafter referred to as "verification operations").
[0050] A verification operation is set to include at least one of three actions: "mouseover," where the mouse pointer is hovered over the area being verified for a specific time, such as 1 to 2 seconds; "left-click," where the left mouse button is pressed; and "Enter key," where the Enter key on the keyboard is pressed. If multiple operations are set for a verification operation, either "and" is specified, indicating that all of the set operations must be performed, or "or" is specified, indicating that any one of the set operations must be performed. For example, "mouseover and left-click" means that both mouseover and left-click must be performed. "mouseover or Enter key" means that either mouseover or Enter key must be performed.
[0051] The control unit 25 shown in Figure 2 includes, for example, a CPU (Central Processing Unit), a ROM (Read Only Memory) that stores programs and various data, and a RAM (Random Access Memory) used as a working area for the CPU of the control unit 25. The CPU of the control unit 25 executes programs stored in the storage unit 24 or the ROM of the control unit 25.
[0052] The control unit 25, by executing the information processing program 24a, realizes a character extraction unit 25a that extracts information from an image electronic document through character recognition processing, and a verification unit 25b that allows the user to perform the task of verifying the information extracted by the character extraction unit 25a.
[0053] Figure 10 is a block diagram of an example of a user terminal 30 that is configured with a PC.
[0054] As shown in Figure 10, the user terminal 30 includes an operation unit 31 which is an operation device such as a keyboard or mouse into which various operations are input, a display unit 32 which is a display device such as an LCD that displays various information, a communication unit 33 which is a communication device that communicates with external devices via a network such as a LAN or the Internet, or directly via wired or wireless connection without going through a network, a storage unit 34 which is a non-volatile storage device such as a semiconductor memory or HDD that stores various information, and a control unit 35 that controls the entire user terminal 30.
[0055] The storage unit 34 is capable of storing a web browser program (hereinafter referred to as "web browser program") 34a. The web browser program 34a may, for example, be installed on the user terminal 30 during the manufacturing stage, or it may be additionally installed on the user terminal 30 from an external storage medium such as a USB memory, or it may be additionally installed on the user terminal 30 from a network.
[0056] The control unit 35 includes, for example, a CPU, a ROM that stores programs and various data, and RAM as memory used as a working area for the CPU of the control unit 35. The CPU of the control unit 35 executes programs stored in the storage unit 34 or the ROM of the control unit 35.
[0057] The control unit 35 implements a web browser 35a by executing a web browser program 34a.
[0058] Next, we will explain the operation of system 10.
[0059] First, we will explain the operation of system 10 when a user logs in to information processing system 20.
[0060] Figure 11 is a sequence diagram of the operation of system 10 when a user logs in to information processing system 20.
[0061] As shown in Figure 11, when a user accesses the information processing system 20 via the web browser 35a of the user terminal 30 (S101), the character extraction unit 25a of the information processing system 20 sends data for the login screen that the user uses to log in to the information processing system 20 to the web browser 35a (S102).
[0062] When the web browser 35a receives the data transmitted in S102, it displays the login screen 300 (see Figure 12) on the display unit 32 based on the received data (S103).
[0063] Figure 12 shows an example of the login screen 300.
[0064] As shown in Figure 12, the login screen 300 includes a text box 301 as an object for entering the user ID, a text box 302 as an object for entering the password, and a login button 303 as an object for logging into the information processing system 20.
[0065] As shown in Figure 11, when the login button 303 is pressed, the web browser 35a sends a login request to the information processing system 20 that includes a combination of the user ID entered in the text box 301 at the time the login button 303 was pressed and the password entered in the text box 302 at the time the login button 303 was pressed (S104).
[0066] When the character extraction unit 25a receives the login request sent in S104, it performs authentication based on the combination of user ID and password included in the received login request (S105). Specifically, the character extraction unit 25a determines that user authentication was successful if the combination of user ID and password included in the login request is included in the user management information 24b. On the other hand, the character extraction unit 25a determines that user authentication failed if the combination of user ID and password included in the login request is not included in the user management information 24b.
[0067] If the authentication result in S105 is successful, the character extraction unit 25a notifies the user terminal 30 that it is permitted to log in with the role associated in the user management information 24b with the combination of user ID and password included in the login request (S106).
[0068] If the authentication result in S105 is an authentication failure, the character extraction unit 25a notifies the user terminal 30 that login is not permitted (S107).
[0069] Next, we will describe the operation of system 10 when an image electronic document is input.
[0070] Figure 13 is a sequence diagram of the operation of system 10 when an image electronic document is input.
[0071] If a user is logged into the information processing system 20, they can instruct the user terminal 30 to input an image electronic document into the information processing system 20. When the web browser 35a is instructed to input an image electronic document into the information processing system 20, it transmits the image electronic document instructed by the user to the information processing system 20 to the information processing system 20, as shown in Figure 13 (S121).
[0072] When the character extraction unit 25a receives the image electronic document transmitted in S121, it stores the received image electronic document in the storage unit 24 (S122). When the character extraction unit 25a stores the image electronic document in S122, it assigns a document ID to the image electronic document.
[0073] The above describes an example in which an image electronic document is input from a user terminal 30 to the information processing system 20. However, the source of the image electronic document input to the information processing system 20 may be other than the user terminal. For example, an image electronic document generated by an image reading device, such as a dedicated scanner or an MFP (Multifunction Peripheral), when a document is read by the scanner of the image reading device, may be directly input to the information processing system 20 from the image reading device.
[0074] Next, we will describe the operation of system 10 when a verification work classification is set.
[0075] Figure 14 is a sequence diagram of the operation of system 10 when a verification work classification is set.
[0076] If the administrator is logged into the information processing system 20, they can instruct the user terminal 30 to begin setting up the verification work classification. When the web browser 35a is instructed to begin setting up the verification work classification, it instructs the information processing system 20 to begin setting up the verification work classification, as shown in Figure 14 (S141).
[0077] When the character extraction unit 25a receives the instruction in S141, it transmits data from the verification work classification setting screen for setting the verification work classification to the user terminal 30 (S142).
[0078] When the web browser 35a receives the data transmitted in S142, it displays the verification work classification setting screen on the display unit 32 based on the received data (S143).
[0079] Administrators can increase or decrease the number of verification work classifications on the verification work classification settings screen. They can also set the name of each verification work classification, the classification color used to display the area of the verification work, and the verification operation for each classification.
[0080] When the administrator specifies the execution of the verification work classification settings on the verification work classification settings screen, the web browser 35a transmits the settings on the verification work classification settings screen to the information processing system 20 (S144).
[0081] When the character extraction unit 25a receives the settings transmitted in S144, it updates the verification work classification information 24h with the received settings (S145). In addition, if the character extraction unit 25a increases the number of verification work classifications in S145, it assigns a classification ID to the newly added verification work classification.
[0082] Next, we will describe the operation of system 10 when a template is created.
[0083] Figure 15 is a sequence diagram of the operation of system 10 when a template is created.
[0084] If the administrator is logged into the information processing system 20, they can specify one of the image electronic documents stored in the information processing system 20 and instruct the user terminal 30 to start creating a template. When the Web browser 35a is instructed to start creating a template, it instructs the information processing system 20 to start creating the template, as shown in Figure 15 (S161). The instruction in S161 includes the document ID of the image electronic document specified by the administrator.
[0085] When the character extraction unit 25a receives the instruction in S161, it sends data for the template creation screen to the user terminal 30 for creating the template (S162).
[0086] When the web browser 35a receives the data transmitted in S162, it displays the template creation screen 310 (see Figure 16) on the display unit 32 based on the received data (S163).
[0087] Figure 16 shows an example of the template creation screen 310.
[0088] The template creation screen 310 shown in Figure 16 includes an extraction area specification area 311 for specifying the extraction area, and an attribute etc. specification area 312 for specifying attributes, verification work classification, and verification work order for each of the extraction areas specified in the extraction area specification area 311.
[0089] The extraction area designation area 311 displays a preview of the image electronic document identified by the document ID included in the instruction in S161. In the example shown in Figure 16, the extraction area designation area 311 displays a preview of the image electronic document 24c shown in Figure 4.
[0090] The administrator can specify a rectangular extraction area on the preview of the image electronic document in the extraction area specification area 311. For example, in the example shown in Figure 16, a rectangular extraction area 311a is specified above the recipient of the receipt in the preview of the image electronic document. When an extraction area is specified in the extraction area specification area 311, the character extraction unit 25a generates a row corresponding to this extraction area in the attribute specification area 312. In the attribute specification area 312, the administrator can specify the attributes, verification work classification, and verification work order for each row. The verification work classification can be specified by the name in the verification work classification information 24h.
[0091] The administrator can instruct the user terminal 30 to complete the creation of the template. When the web browser 35a is instructed to complete the creation of the template, it instructs the information processing system 20 to complete the creation of the template, as shown in Figure 15 (S164).
[0092] When the character extraction unit 25a receives the instruction in S164, it creates a template on the storage unit 24 that reflects the information specified in the template creation screen 310 (S165). Specifically, the character extraction unit 25a reflects the position of each extraction area in the template based on the position of each extraction area specified in the extraction area specification area 311 relative to the preview of the image electronic document in the extraction area specification area 311. The character extraction unit 25a also reflects the attributes, verification work classification, and verification work order for each extraction area specified in the attribute specification area 312 in the template. When the character extraction unit 25a creates a template in S165, it assigns a template ID to this template.
[0093] Next, we will explain the operation of system 10 when extracting information from an image electronic document.
[0094] Figure 17 is a sequence diagram of the operation of system 10 when extracting information from an image electronic document.
[0095] When a user is logged into the information processing system 20, they can specify one of the image electronic documents and one of the templates stored in the information processing system 20, and then instruct the user terminal 30 to extract information from the image electronic document. When the web browser 35a is instructed to extract information from the image electronic document, it instructs the information processing system 20 to extract information from the image electronic document, as shown in Figure 17 (S181). The instruction in S181 includes the document ID of the image electronic document specified by the user and the template ID of the template specified by the user.
[0096] When the character extraction unit 25a receives the instruction in S181, it extracts information for each extraction region defined by position in the template, which is identified by the template ID included in the instruction in S181, from the image electronic document identified by the document ID included in the instruction in S181, thereby generating an extracted information file (S182). When the character extraction unit 25a generates an extracted information file in S182, it assigns a file ID to this extracted information file.
[0097] When the processing in S182 is completed, the character extraction unit 25a updates the file-related information 24f so that it includes the file ID of the extracted information file generated in S182 and the combination of the document ID and template ID included in the instructions in S181 in the file-related information 24f (S183).
[0098] When the processing in S182 is completed, the verification unit 25b adds the record of the verification task for the extracted information file generated in S182 to the verification task list information 24g (S184). When the verification unit 25b generates a record in S184, it assigns a task ID to the verification task that is the target of this record. When the verification unit 25b generates a record in S184, it includes the task ID of the verification task that is the target of this record and the file ID of the extracted information file generated in S182 in this record.
[0099] Next, we will describe the operation of system 10 when starting a verification task.
[0100] Figure 18 is a sequence diagram of the operation of system 10 when a verification task is initiated.
[0101] If a user is logged into the information processing system 20, they can instruct the user terminal 30 to display a list of verification tasks. When the web browser 35a is instructed to display a list of verification tasks, it instructs the information processing system 20 to display the list of verification tasks, as shown in Figure 18 (S201).
[0102] When the verification unit 25b receives the instruction in S201, it generates data for the verification task list screen, which shows a list of verification tasks, based on the verification task list information 24g (S202).
[0103] When the processing in S202 is completed, the verification unit 25b transmits the data generated in S202 to the user terminal 30 (S203).
[0104] When the web browser 35a receives the data transmitted in S203, it displays the verification task list screen 320 (see Figure 19) on the display unit 32 based on the received data (S204).
[0105] Figure 19 shows an example of the verification task list screen 320.
[0106] The verification task list screen 320 shown in Figure 19 includes a list box 321 as an object that displays a list of verification tasks, and a start button 322 as an object that instructs the user to start a verification task. The start button 322 is basically grayed out and inoperable, but becomes operable only when only one verification task is selected in the list box 321.
[0107] As shown in Figure 18, when the start button 322 is pressed, the web browser 35a instructs the information processing system 20 to start the verification task that was selected in the list box 321 at the time the start button 322 was pressed (S205).
[0108] When the processing in S205 is completed, the verification unit 25b of the information processing system 20 starts the verification task that was instructed to start in S205 (S206).
[0109] Next, we will describe the operation of the information processing system 20 when performing a verification task.
[0110] Figure 20 is a flowchart showing part of the operation of the information processing system 20 when performing a verification task. Figure 21 is a flowchart that continues from the flowchart shown in Figure 20.
[0111] The verification unit 25b performs the operations shown in Figures 20 and 21 by executing the process in S206.
[0112] As shown in Figures 20 and 21, the verification unit 25b generates data for the verification screen to execute the verification task based on an extracted information file (hereinafter referred to as the "target extracted information file") identified by a file ID associated in the verification task list information 24g with the task ID of the verification task selected in the list box 321 when the start button 322 is pressed, an image electronic document (hereinafter referred to as the "target image electronic document") identified by a document ID associated in the file-related information 24f with this file ID, and a template (hereinafter referred to as the "target template") identified by a template ID associated in the file-related information 24f with this file ID (S221).
[0113] When the processing in S221 is completed, the verification unit 25b sends the data generated in S221 to the user terminal 30 (S222). Therefore, the web browser 35a displays the verification screen 330 (see Figure 22) on the display unit 32 based on the data sent in S222.
[0114] Figure 22 shows an example of the verification screen 330.
[0115] The verification screen 330 shown in Figure 22 includes an extraction area display area 331 for displaying the extraction area, and an extraction information display area 332 for displaying the information extracted from each of the extraction areas displayed in the extraction area display area 331.
[0116] In S221, the verification unit 25b generates verification screen data so that a preview of the target image electronic document is displayed in the extraction area display area 331. In the example shown in Figure 22, the extraction area display area 331 displays a preview of the image electronic document 24c shown in Figure 4.
[0117] The verification unit 25b generates verification screen data in S221 so as to display the extracted area based on the target template in the extracted area display area 331. In the example shown in Figure 22, for example, the extracted area 331a is displayed. Specifically, the verification unit 25b generates verification screen data in S221 so as to display the extracted area at the position defined in the target template. The verification unit 25b generates verification screen data in S221 so as to display the extracted area highlighted with the classification color specified in the verification work classification information 24h for the verification work classification defined in the target template. In the example shown in Figure 22, the classification color is represented by hatching in order to distinguish the classification color in black and white.
[0118] In S221, the verification unit 25b generates data for the verification screen so that each attribute includes a set of text indicating the attribute and a text box as an object for inputting information for that attribute in the extracted information display area 332. In the example shown in Figure 22, the extracted information display area 332 includes, for example, a set of text 332a and text box 332b. If there are multiple sets, the verification unit 25b may arrange the multiple sets in the order of the attribute verification work shown in the target template. In S221, the verification unit 25b generates data for the verification screen so that the information associated with the attribute in the target extracted information file is entered into the text box. During the processing in S221, the verification unit 25b grays out the text box to make it unresponsive.
[0119] In S221, the verification unit 25b generates data for the verification screen so that it displays a verification complete button 332c, which is an object for successfully completing the verification of all extracted information contained in the target extracted information file, and a verification stop button 332d, which is an object for stopping the verification of the extracted information contained in the target extracted information file, in the extracted information display area 332. During the processing stage of S221, the verification unit 25b grays out the verification complete button 332c, making it inoperable.
[0120] The verification cancellation button 332d may be pressed, for example, if the user determines that the image quality of the electronic image document is poor and the verification of the extracted information cannot be completed successfully, or if the image quality of the electronic image document is good enough to complete the verification of the extracted information, but is not good enough to continue saving the electronic image document.
[0121] As shown in Figures 20 and 21, when the processing in S222 is completed, the verification unit 25b identifies only the first extraction region in the target template as the target extraction region (S223).
[0122] When the processing in S223 is completed, the verification unit 25b changes the state of the extraction area display area 331 so that only the current target extraction area among the extraction areas shown is the target extraction area (hereinafter referred to as the "target area display state") (S224).
[0123] When the processing in S224 is completed, the verification unit 25b makes only the text box corresponding to the current target extraction area among the text boxes shown in the extraction information display area 332 operable (S225).
[0124] Figure 23 shows an example of the verification screen 330 when the first extraction area in the verification process is the target extraction area.
[0125] The verification screen 330 shown in Figure 23 includes a frame 331b of a specific color around the current target extraction area as the target area display state. The color of the frame 331b can be set in advance by the administrator. In the verification screen 330 shown in Figure 23, only the text box corresponding to the current target extraction area is operable (not grayed out).
[0126] In the example shown in Figure 23, the default image 381 of the mouse pointer is displayed.
[0127] As shown in Figures 20 and 21, when the processing in S225 is completed, the verification unit 25b starts executing a verification work acceptance process to accept verification work for the extraction information corresponding to the current target extraction area (S226).
[0128] Figure 24 is a flowchart of the verification work acceptance process.
[0129] As shown in Figure 24, the verification unit 25b determines whether an operation included in the verification operation (hereinafter referred to as "operation for the current target extraction area") that has been associated in the verification work classification information 24h with the classification ID associated in the target template with the attributes of the current target extraction area has been executed, and whether an operation that has not yet been executed by the user in the operation shown in Figure 24 has been executed (S261).
[0130] If the verification unit 25b determines in S261 that an operation for the current target extraction area that has not yet been performed by the user in the operation shown in Figure 24 has been performed, it determines whether all the minimum necessary operations have been performed as verification operations associated in the verification work classification information 24h with the classification ID associated in the target template with the attributes of the current target extraction area (S262).
[0131] If the verification unit 25b determines in S261 that no operations have been performed by the user in the operation shown in Figure 24 among the operations for the current target extraction area, or if it determines in S262 that not all of the minimum necessary operations have been performed as verification operations associated in the verification work classification information 24h with the classification ID associated in the target template with the attributes of the current target extraction area, then it determines in the extraction area display area 331 whether the mouse pointer is hovering over the current target extraction area (S263).
[0132] If the verification unit 25b determines in S263 that the mouse pointer is over the current target extraction area in the extraction area display area 331, it sets an image representing an operation for the current target extraction area that has not yet been performed by the user as the mouse pointer image (S264).
[0133] Figure 25A is a magnified view of the area near the mouse pointer in the verification screen 330. Figure 25B is a magnified view of the area near a mouse pointer different from the one shown in Figure 25A in the verification screen 330. Figure 25C is a magnified view of the area near a mouse pointer different from the one shown in Figures 25A and 25B in the verification screen 330.
[0134] When the verification work classification information 24h is as shown in Figure 9, and the classification ID associated with the attribute of the current target extraction area in the target template is "C001", if the mouse pointer is hovering over the current target extraction area, the mouse pointer image will be, for example, image 382 as shown in Figure 25A. Image 382 includes image 382a, which represents the operation "mouse over", image 382b, which represents "and", and image 382c, which represents the operation "left click", which is included in the verification operation.
[0135] When the verification work classification information 24h is as shown in Figure 9, and the classification ID associated with the attribute of the current target extraction area in the target template is "C002", then when the mouse pointer is hovered over the current target extraction area, the mouse pointer image will be, for example, image 383 shown in Figure 25B. Image 383 is an image that represents the "mouse over" operation, which is included in the verification operation.
[0136] When the verification work classification information 24h is as shown in Figure 9, and the classification ID associated with the attribute of the current target extraction area in the target template is "C003", if the mouse pointer is hovering over the current target extraction area, the mouse pointer image will be, for example, image 384 as shown in Figure 25C. Image 384 includes image 384a, which represents the operation "mouse over", image 384b, which represents "or", and image 384c, which represents the operation "Enter key", which is included in the verification operation.
[0137] Figure 26A is a magnified view of the area near a mouse pointer different from the mouse pointers shown in Figures 25A to C of the verification screen 330. Figure 26B is a magnified view of the area near a mouse pointer different from the mouse pointers shown in Figures 25A to C or Figure 26A of the verification screen 330.
[0138] The verification work classification information 24h is as shown in Figure 9, and when the classification ID associated with the attribute of the current target extraction area in the target template is "C001", and the mouse pointer is hovered over the current target extraction area, when a "mouseover" operation is performed on the current target extraction area, the image of the mouse pointer becomes, for example, image 385 shown in Figure 26A. Image 385 is similar to the image shown in Figure 25A, from which image 382a, which indicates the "mouseover" operation included in the verification operation, and image 382b, which indicates "and", have been removed.
[0139] If the verification work classification information 24h is as shown in Figure 9, and the classification ID associated with the attribute of the current target extraction area in the target template is "C001", then when the mouse pointer is over the current target extraction area and a "left click" is performed, the mouse pointer image will become, for example, image 386 as shown in Figure 26B. Image 386 is similar to the image shown in Figure 25A, from which image 382b, which shows "and", and image 382c, which shows "left click", an operation included in the verification operation, have been removed.
[0140] As shown in Figure 24, when the processing in S264 is completed, the verification unit 25b displays a help image 391 (see Figure 27) in the extraction area display area 331 near the current target extraction area, indicating the verification operation associated in the verification work classification information 24h with the classification ID associated in the target template with the attributes of the current target extraction area (S265).
[0141] Figure 27 is a magnified view of the area near the help image 391 within the verification screen 330.
[0142] When the verification work classification information 24h is as shown in Figure 9, and the classification ID associated with the attribute of the current target extraction area in the target template is "C001", when the mouse pointer is over the current target extraction area, a help image 391, for example, as shown in Figure 27, will be displayed near the current target extraction area. The help image 391 indicates that both "mouse over" and "left click," which are operations included in the verification operation, must be performed.
[0143] As shown in Figure 24, when the processing of S265 is completed, the verification unit 25b executes the processing of S261.
[0144] If the verification unit 25b determines in S263 that the mouse pointer is not currently hovering over the target extraction area in the extraction area display area 331, it sets the default mouse pointer image 381 as the mouse pointer image (S266).
[0145] When the processing in S266 is completed, the verification unit 25b terminates the display of the help image if one is currently being displayed (S267).
[0146] When the processing in S267 is completed, the verification unit 25b executes the processing in S261.
[0147] If the verification unit 25b determines in S262 that all the minimum necessary operations have been performed as verification operations associated in the verification work classification information 24h with the classification ID associated in the target template with the attributes of the current target extraction area, it sets the default mouse pointer image 381 as the mouse pointer image (S268).
[0148] When the processing in S268 is completed, the verification unit 25b terminates the display of the help image if one is being displayed (S269), and then terminates the verification work acceptance process shown in Figure 24.
[0149] As shown in Figures 20 and 21, when the processing in S226 is completed, the verification unit 25b determines whether or not the verification cancellation button 332d was pressed (S227).
[0150] If the verification unit 25b determines in S227 that the verification cancellation button 332d has been pressed, it terminates the verification work acceptance process corresponding to the currently targeted extraction area (S228).
[0151] If the verification unit 25b determines in S227 that the verification cancellation button 332d has not been pressed, it determines whether the verification of the extraction information corresponding to the current target extraction area has been completed (S229). If the verification unit 25b has completed the verification work acceptance process for the current target extraction area, it determines that the verification of the extraction information corresponding to the current target extraction area has been completed. On the other hand, if the verification unit 25b has not completed the verification work acceptance process for the current target extraction area, it determines that the verification of the extraction information corresponding to the current target extraction area has not been completed.
[0152] As shown in Figures 20 and 21, if the verification unit 25b determines in S229 that the verification of the extraction information corresponding to the current target extraction area has not been completed, it executes the process in S227.
[0153] When the verification unit 25b determines in S229 that the verification of the extraction information corresponding to the current target extraction area is complete, it highlights the current target extraction area in the extraction area display area 331 with a completion color to indicate that the verification is complete (S230). The completion color in S230 can be set in advance by the administrator. The completion color in S230 may be, for example, gray.
[0154] Figure 28 shows an example of the verification screen 330 when some of the extracted areas are displayed in a completed color.
[0155] In the example shown in Figure 28, the extraction areas corresponding to the attributes "Issue Date," "Customer Name," "Product Number 1," "Quantity 1," "Unit Price 1," and "Amount 1" are highlighted in the "Completed" color.
[0156] As shown in Figures 20 and 21, when the processing in S230 is completed, the verification unit 25b determines whether or not there is a next verification operation sequence for the current target extraction area in the target template (S231).
[0157] If the verification unit 25b determines in S231 that there is a next verification operation sequence for the current target extraction area in the target template, it identifies only the extraction area for the next verification operation sequence for the current target extraction area in the target template as the new target extraction area (S232), and executes the process in S224.
[0158] If the verification unit 25b determines in S231 that there is no next verification operation sequence for the current target extraction area in the target template, it makes all text boxes and the verification complete button 332c operable (S233).
[0159] When the processing in S233 is completed, the verification unit 25b displays a pop-up 340 (see Figure 29) in front of the verification screen 330 asking the user whether or not to complete the verification (S234). Therefore, the user is unable to operate the verification screen 330 until the pop-up 340 is closed.
[0160] Figure 29 shows an example of the pop-up 340.
[0161] The popup 340 shown in Figure 29 includes text 341 that displays a message asking the user whether or not to complete the verification, an OK button 342 for completing the verification, and a NO button 343 for not completing the verification.
[0162] When the processing in S234 is completed, the verification unit 25b determines whether or not the OK button 342 was pressed (S235).
[0163] If the verification unit 25b determines in S235 that the OK button 342 has not been pressed, it then determines whether or not the NO button 343 has been pressed (S236).
[0164] If the verification unit 25b determines in S236 that the NO button 343 has not been pressed, it executes the process in S235.
[0165] If the verification unit 25b determines in S235 that the OK button 342 has been pressed, it terminates the display of the pop-up 340 and the verification screen 330 (S237).
[0166] When the processing in S237 is completed, the verification unit 25b updates the target extraction information file with the contents that were entered in the text box on the verification screen 330 at the time the OK button 342 was pressed (S238).
[0167] When the processing in S238 is completed, the verification unit 25b writes "Successfully Completed" to the progress status value associated with the file ID of the target extracted information file in the file-related information 24f (S239), and terminates the operations shown in Figures 20 and 21.
[0168] The verification unit 25b may output the extracted information file identified by the file ID associated with the progress status value "successfully completed" in the file-related information 24f to an external source of the information processing system 20. When the verification unit 25b outputs the extracted information file to an external source of the information processing system 20, it may also output the image electronic document that was the source of the extracted information file along with the extracted information file. The destination of the extracted information file may be specified by the instruction in S181, for example, in the verification unit 25b.
[0169] When the verification unit 25b determines in S236 that the NO button 343 has been pressed, it terminates the display of the pop-up 340 (S240). Therefore, the user can edit the information written in the text box on the verification screen 330.
[0170] When the processing in S240 is completed, the verification unit 25b determines whether or not the verification completion button 332c was pressed (S241).
[0171] If the verification unit 25b determines in S241 that the verification complete button 332c has not been pressed, it determines whether or not the verification cancel button 332d has been pressed (S242).
[0172] If the verification unit 25b determines in S242 that the verification stop button 332d has not been pressed, it executes the process in S241.
[0173] When the verification unit 25b determines in S241 that the verification complete button 332c has been pressed, it terminates the display of the verification screen 330 (S243).
[0174] When the processing in S243 is completed, the verification unit 25b updates the target extraction information file with the contents that were entered in the text box on the verification screen 330 at the time the verification completion button 332c was pressed (S244), and then executes the processing in S239.
[0175] When the verification unit 25b determines in S242 that the process in S228 has finished or that the verification stop button 332d has been pressed, it terminates the display of the verification screen 330 (S245).
[0176] When the processing in S245 is completed, the verification unit 25b writes "Verification Cancelled" to the progress status value associated with the file ID of the target extracted information file in the file-related information 24f (S246), and terminates the operations shown in Figures 20 and 21.
[0177] As explained above, when the information processing system 20 has the user perform the verification of information extracted from an image electronic document by character recognition processing for each of multiple attributes, the system has the user perform the verification work according to the verification work classification associated with the attribute corresponding to the information (S226). Therefore, the operations required to complete the verification may differ depending on the attribute corresponding to the information, and the possibility of verification errors can be reduced.
[0178] The information processing system 20 associates at least some of multiple attributes with verification work classifications that require multiple operations to be performed in order to complete the verification. Therefore, depending on the attribute corresponding to the information, the operations required to complete the verification may not be simple, thus reducing the possibility of verification errors.
[0179] The information processing system 20 displays areas using classification colors that correspond to the verification work classifications associated with attributes (S221 and S222), allowing users to recognize the operations necessary to complete the verification using classification colors, thereby improving convenience.
[0180] The information processing system 20 displays the area corresponding to the attribute of the information that has been verified in a completion color (S230), allowing the user to distinguish between the area corresponding to the attribute of the information that has been verified and the area corresponding to the attribute of the information that has not been verified. As a result, the possibility of verification omissions can be reduced.
[0181] The information processing system 20 displays a specific frame 331b in the area corresponding to the attribute corresponding to the information currently being verified (S224), so that the user can easily recognize the area corresponding to the attribute corresponding to the information currently being verified, and as a result, convenience can be improved.
[0182] The information processing system 20 has the user perform a verification task for each piece of information, and when the verification of the information is completed (YES in S229), if there is any information that has not been verified (YES in S231), it automatically makes any of the unverified pieces of information subject to verification (S232), thereby reducing the possibility of verification omissions.
[0183] When the information processing system 20 completes the verification of information, if there is any information that has not yet been verified, it will automatically select any of the unverified pieces of information for verification, thereby allowing users to perform the verification work efficiently.
[0184] The information processing system 20 allows the user to perform verification tasks for each piece of information and displays the operations necessary to complete the verification of the current target information using an image of a mouse pointer (S264), thereby improving convenience.
[0185] The information processing system 20 allows the user to perform verification work for each piece of information and displays an explanation of the operations necessary to complete the verification of the current target information, i.e., a help image (S265), thereby improving convenience.
[0186] The information processing system 20 can improve usability by changing the mouse pointer image each time an operation is performed when there are multiple operations required to complete the verification of the current target information (S261-S265).
[0187] Furthermore, if the information processing system 20 requires multiple operations to complete the verification of the current target information, it can improve usability by changing the help image, similar to the mouse pointer image, each time an operation is performed.
Claims
1. An information processing system comprising: a character extraction unit that extracts information from an electronic document image for each of a plurality of attributes by character recognition processing; and a verification unit that causes a user to perform verification work on the information extracted by the character extraction unit, wherein each of the plurality of attributes is associated with a classification of the work, and at least a portion of the plurality of attributes can be associated with one of a plurality of classifications in which at least a portion of the operations required to complete the verification differ from each other, and the verification unit causes the user to perform the work according to the classification associated with the attribute corresponding to the information.
2. The information processing system according to claim 1, characterized in that at least some of the multiple attributes are associated with the classification, which requires multiple operations to be performed in order to complete the verification.
3. The information processing system according to claim 1, wherein each of the plurality of classifications is associated with a classification color as the color used when displaying the area corresponding to the attribute in the image, and at least some of the plurality of classifications can be associated with different classification colors from each other, and the verification unit displays the area with the classification color associated with the classification that is associated with the attribute.
4. The information processing system according to claim 3, characterized in that the verification unit displays the area corresponding to the attribute corresponding to the information for which the verification has been completed using a completion color, rather than the classification color, as a color indicating that the verification has been completed.
5. The information processing system according to claim 3, characterized in that the verification unit displays a specific frame in the area corresponding to the attribute corresponding to the information currently being verified.
6. The information processing system according to claim 1, characterized in that the verification unit causes the user to perform the task for each piece of information, and when the verification of the information is completed, if there is any information that has not been verified, it automatically makes any of the information that has not been verified the subject of verification.
7. The information processing system according to claim 1, characterized in that the verification unit causes the user to perform the operation for each piece of information, and indicates the operations necessary to complete the verification of the current target information using an image of a mouse pointer.
8. The information processing system according to claim 7, characterized in that the verification unit changes the image of the mouse pointer each time an operation is performed when there are multiple operations necessary to complete the verification of the current target information.
9. The information processing system according to claim 1, characterized in that the verification unit causes the user to perform the task for each piece of information and displays an explanation of the operations necessary to complete the verification of the current target information.
10. The information processing system according to claim 9, characterized in that the verification unit changes the description each time an operation is performed when there are multiple operations necessary to complete the verification of the current target information.
11. An information processing device comprising: a character extraction unit that extracts information from an electronic document image for each of a plurality of attributes by character recognition processing; and a verification unit that causes a user to perform a verification operation of the information extracted by the character extraction unit, wherein each of the plurality of attributes is associated with a classification of the operation, and at least a portion of the plurality of attributes can be associated with one of a plurality of classifications in which at least a portion of the operations required to complete the verification differ from each other, and the verification unit causes the user to perform the operation according to the classification associated with the attribute corresponding to the information.