Information processing device, document registration system, and document registration method

Abstract

To provide an information processing device, etc., that can improve the efficiency of work related to form registration. [Solution] An information processing device comprising: a generation unit that generates an input screen having a display area for displaying a form image and accepting input of item values ​​for items included in the form; and a control unit that assists in the input of the item values ​​by controlling the display of the generated input screen and registers the form based on the accepted item values, wherein the control unit associates the item values ​​entered by the user with the recognition information obtained by character recognition processing on the first form image and stores them as candidate item value information, and in character recognition processing on the second form image related to new form registration, if recognition information corresponding to the candidate item value information is obtained, the information processing device sets the item values ​​associated with the candidate item value information as candidate item values ​​for new form registration.

Description

Technical Field

[0001] The present disclosure relates to an information processing apparatus and the like.

Background Art

[0002] Conventionally, for the purpose of improving the efficiency of registration work for documents generally called "forms" such as invoices and delivery notes, optical character recognition (OCR) processing is performed on an electronic image file of the form (hereinafter referred to as a form image file), and a character candidate is extracted as an item value of an item included in the form. A technique is known.

[0003] In character recognition processing, the ability to identify characters formed on a form as an image and estimate characters from the identified image may be a problem. For example, when a character string that should be an item value corresponding to a specific item cannot be obtained, or when the estimated characters are garbled and an accurate recognition result cannot be obtained, the user has to correct the extracted character candidates.

[0004] For example, Patent Document 1 describes that when a necessary correction instruction is received for a character text obtained by converting image characters, the corrected character text is replaced with the character candidate.

[0005]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0006] An object of the present disclosure is to provide an information processing apparatus and the like capable of improving the work efficiency related to form registration.

Means for Solving the Problems

[0007] To solve the above problems, the information processing device according to this disclosure comprises a generation unit that generates an input screen having a display area for displaying a report image and accepting input of item values ​​of items included in the report, and a control unit that assists in the input of the item values ​​by controlling the display of the generated input screen and registers the report based on the accepted item values, wherein the control unit stores candidate item value information by associating the item values ​​entered by the user with recognition information obtained by character recognition processing on a first report image, and in character recognition processing on a second report image related to new report registration, if recognition information corresponding to the candidate item value information is obtained, the control unit sets the item value associated with the candidate item value information as a candidate item value for new report registration.

[0008] Furthermore, the document registration system relating to this disclosure is a document registration system having an information processing device and a terminal device, comprising: a generation unit that generates an input screen having a display area for displaying a document image and accepting input of item values ​​of items included in the document; a control unit that assists in the input of the item values ​​by controlling the display of the generated input screen and registers the document based on the accepted item values; and a display unit that displays the input screen based on the control by the control unit, wherein the control unit associates the item values ​​entered by the user with the recognition information obtained by character recognition processing on a first document image and stores them as item value candidate information, and in character recognition processing on a second document image relating to new document registration, if recognition information corresponding to the item value information is obtained, the control unit registers the item values ​​associated with the item value candidate information as item value candidates relating to new document registration.

[0009] Furthermore, the document registration method relating to this disclosure is characterized by generating an input screen having a display area for displaying a document image and accepting input of item values ​​for items included in the document, assisting in the input of the item values ​​by controlling the display of the generated input screen, registering the document based on the accepted item values, associating the item values ​​entered by the user with the recognition information obtained by character recognition processing on the first document image and storing them as candidate item value information, and in character recognition processing on the second document image relating to new document registration, if recognition information corresponding to the candidate item value information is obtained, setting the item values ​​associated with the candidate item value information as candidate item values ​​relating to the new document registration. [Effects of the Invention]

[0010] According to this disclosure, it is possible to provide an information processing device, etc., that can improve the efficiency of work related to form registration. [Brief explanation of the drawing]

[0011] [Figure 1] This diagram illustrates an example of the system configuration of the document registration system according to the first embodiment. [Figure 2] This diagram illustrates the functional configuration of the information processing device according to the first embodiment. [Figure 3] This diagram illustrates the functional configuration of the terminal device according to the first embodiment. [Figure 4] This is a sequence diagram illustrating the information exchange according to the first embodiment. [Figure 5] This is a flowchart illustrating the processing flow according to the first embodiment. [Figure 6] This is a flowchart illustrating the processing flow according to the first embodiment. [Figure 7] This is a flowchart illustrating the processing flow according to the first embodiment. [Figure 8] This is a flowchart illustrating the processing flow according to the first embodiment. [Figure 9] This is a flowchart illustrating the processing flow according to the first embodiment. [Figure 10] This is a flowchart for explaining the process flow according to the first embodiment. [Figure 11] This is a diagram for explaining an operation example according to the first embodiment. [Figure 12] This is a diagram for explaining an operation example according to the first embodiment. [Figure 13] This is a diagram for explaining an operation example according to the first embodiment. [Figure 14] This is a diagram for explaining an example of the system configuration of the form registration system according to the second embodiment. [Figure 15] This is a diagram for explaining the functional configuration of the multifunction device according to the second embodiment. [Figure 16] This is a flowchart for explaining the process flow according to the second embodiment. [Figure 17] This is a diagram for explaining the functional configuration of the information processing apparatus according to the third embodiment.

MODE FOR CARRYING OUT THE INVENTION

[0012] Hereinafter, embodiments of the present disclosure will be described with reference to the drawings. Note that the following embodiments are examples for explaining the present disclosure, and the technical content of the description described in the claims is not limited to the following description.

[0013] As an existing technique in form registration, a list of business partners in which a combination of "business partner" and "registration number of the eligible invoice issuer" is registered is known. A user who performs form registration can perform form registration by inputting, via an input screen, the company name selected from the list of business partners or the registration number assigned to the company as the eligible invoice issuer in the input field of "business partner" or "registration number of the eligible invoice issuer".

[0014] Here, consider an input assistance function using character recognition processing for forms (hereinafter sometimes referred to as OCR processing). When accurate character recognition is performed by OCR processing, an appropriate company name is automatically entered in the "Supplier" input field on the input screen. Therefore, the user does not need to select the company name of the "Supplier" from the supplier list, and it is expected that the work efficiency related to form registration will be improved.

[0015] On the other hand, when accurate character recognition by OCR processing is not performed, the user needs to manually correct the company name (candidate) automatically entered in the "Supplier" of the input screen. At this time, if there is a "Registration number of the eligible invoice issuer" of the "Supplier" related to form registration, the user can accurately input (correct) the "Supplier" not only from the supplier list but also by using the system of the National Tax Agency.

[0016] However, every time the input content is corrected, it takes time and effort to manually correct the input content using the system of the National Tax Agency or the supplier list, which is also one of the reasons for the decrease in the work efficiency of form registration.

[0017] In the present disclosure, an information processing apparatus and the like that can improve the work efficiency related to form registration are realized in the following embodiments.

[0018] [1 First Embodiment] [1.1 Overall Configuration] FIG. 1 is a diagram for explaining an example of the system configuration of a form registration system 100 according to the first embodiment.

[0019] The form registration system 100 includes an information processing apparatus 10 and a terminal device 30. The information processing apparatus 10 and the terminal device 30 are connected via a network NW and are configured to be able to transmit and receive information to and from each other. In FIG. 1, the number of connected terminal devices 30 is set to one, but a plurality of terminal devices 30 may be connected.

[0020] Furthermore, while the first embodiment describes a configuration in which the functions performed by the information processing device 10 are realized by a single information processing device, the functions performed by the information processing device 10 may also be realized by multiple information processing devices.

[0021] [1.2 Functional Configuration] [1.2.1 About the Information Processing Device 10] Figure 2 is a diagram illustrating the functional configuration of the information processing device 10. The information processing device 10 provides services related to form registration in response to requests from the terminal device 30, and also functions as a management database that manages registered forms as a list of business partners. The information processing device 10 can be configured as a computer that operates under the control of a specific OS (Operating System). Such an information processing device 10 comprises a control unit 11, an operation input unit 13, a communication unit 15, and a storage unit 17.

[0022] The control unit 11 controls the entire information processing device 10. The control unit 11 is composed of, for example, one or more arithmetic units (CPU (Central Processing Unit), SoC (System on a Chip), etc.). The control unit 11 realizes its functions by reading and executing various programs stored in the storage unit 17.

[0023] The operation input unit 13 is an input device that receives information input from a user or other party. The operation input unit 13 can use input devices such as a keyboard, mouse, or touch panel.

[0024] The communication unit 15 is equipped with either a wired or wireless interface, or both, for communicating with other devices (such as terminal devices 30) via, for example, a LAN (Local Area Network), a WAN (Wide Area Network), the Internet, or a telephone line. The communication unit 15 may also be equipped with interfaces related to (short-range) wireless communication technologies such as Bluetooth®, Wi-Fi®, ZigBee®, IrDA, or Wireless USB.

[0025] The memory unit 17 is one or more storage devices that store various programs and data necessary for the operation of the information processing device 10. The memory unit 19 can be composed of storage devices such as RAM (Random Access Memory), HDD (Hard Disk Drive), SSD (Solid State Drive), and ROM (Read Only Memory).

[0026] In the first embodiment, the storage unit 17 stores a control program 171, a character recognition processing program 173, and a form registration service program 175, and reserves a form image storage area 177 and a form management information storage area 179.

[0027] The control program 171 is a program that the control unit 11 reads when comprehensively controlling the information processing device 10. After reading the control program 171, the control unit 11 functions as an OS, controlling the operation of hardware such as the operation input unit 13 and the communication unit 15, starting and ending applications that run on the OS, and managing files.

[0028] The character recognition processing program 173 is a program that the control unit 11 reads when performing OCR processing on a data file of a form image (hereinafter referred to as the form image file). Upon reading the character recognition processing program 173, the control unit 11 obtains the item values ​​corresponding to each item in the form, as well as related characters (sequences), as recognition information based on the results of character recognition. In this disclosure, the information obtained as a result of OCR processing is referred to as recognition information, and the character information corresponding to each item value obtained based on said recognition information is referred to as item value candidates.

[0029] The form registration service program 175 is a program that the control unit 11 reads when providing services related to form registration. The form registration service program 175 includes an input screen generation program 1751, an input assistance program 1753, and a form registration program 1755.

[0030] The input screen generation program 1751 is a program that the control unit 11 reads when generating an input screen that accepts user input of item values. After reading the input screen generation program 1751, the control unit 11 generates an input screen having a display area that displays a preview image of the report, for example, based on an uploaded report image file or a report image file stored in the report image storage area 177.

[0031] Furthermore, the control unit 11, having read the input screen generation program 1751, acquires character information that corresponds to the item value of a predetermined specific item and the position information of the character as recognition information from the OCR processing result, based on the classification (information) indicating whether the form related to registration has been received or issued. The control unit 11 then converts the acquired character position information into relative coordinates, as described later, and stores the relative coordinate information obtained by associating it with the item value corresponding to the predetermined specific item entered or modified by the user as item value candidate information.

[0032] Furthermore, when the control unit 11 reads the input screen generation program 1751, if it determines that recognition information corresponding to the previously stored item value candidate information has been obtained through OCR processing of the form image file as the second form image related to the new form registration, it sets the recognition information as the new item value candidate for form registration. Then, the control unit 11 generates an input screen that associates an identifier (referred to as an item value candidate button) indicating that an item value candidate for form registration has been set with an item included in the form. The control unit 11 transmits the input screen generated via the input screen generation program 1751 to the terminal device 30 as input screen information.

[0033] The input assistance program 1753 is a program that the control unit 11 reads when assisting a user in inputting or selecting item values ​​via the input screen of the terminal device 30. When the control unit 11 reads the input assistance program 1753 and receives an operation related to the input or selection of an item value by the user as an input (selection) command, it sends a response command to the terminal device 30 to assist in the input or selection of the item value. For example, when the control unit 11 receives a selection of an item value candidate button displayed on the input screen, it can send a response command to the terminal device 30 that displays character candidates as item value candidates related to form registration. The specific actions of the control unit 11 after reading the input assistance program 1753 will be described later.

[0034] The document registration program 1755 is a program that the control unit 11 reads when it receives a document registration request (command) from a user. Upon reading the document registration program 1755, the control unit 11 registers the document based on the item values ​​entered or selected by the user. Furthermore, the control unit 11, upon reading the document registration program 1755, functions as a management unit for managing documents, associating documents (images) with item values. Alternatively, it may manage documents (images) associating with the corresponding item values. The control unit 11, upon reading the document registration program 1755, can also manage a list of business partners, which is a list of combinations of business partner names and the registration numbers of qualified invoice issuers.

[0035] The form image storage area 177 is a storage area that stores form image files uploaded from the terminal device 30, or form image files acquired by other means (for example, an image reading device (scanner device) not shown, transfer from other terminal devices, or transfer from external storage devices such as USB (Universal Serial Bus)). The form image files stored in the form image storage area 177 are used for OCR processing, generating preview images of forms on input screens, etc.

[0036] The form management information storage area 179 is a storage area that stores form (images) and item values ​​in association. The control unit 11, which has read the form registration program 1755, manages the item values ​​stored in the form management information storage area 179 as management information in a predetermined management method such as a table format or a database format. The item values ​​stored in the form management information storage area 179 can be used, for example, as registered forms to display a list of registration candidates. In addition, the form management information storage area 179 stores item value candidate information by associating the item values ​​entered or modified by the user with relative coordinate information obtained by converting the position information of characters into relative coordinates. In this case, the management and storage may be done for each item.

[0037] [1.2.2 Regarding terminal device 30] Figure 3 is a diagram illustrating the functional configuration of the terminal device 30. The terminal device 30 is used for receiving requests from users to register forms and for managing registered forms. The terminal device 30 can be configured as a computer operating under the control of a specific OS. Such a terminal device 30 comprises a control unit 31, a display unit 33, an operation input unit 35, a communication unit 37, and a storage unit 39.

[0038] The control unit 31 controls the entire terminal device 30. The control unit 31 is composed of, for example, one or more arithmetic units (CPU, SoC, etc.). The control unit 31 realizes its functions by reading and executing various programs stored in the storage unit 39.

[0039] The display unit 33 is a display device that displays various information to the user. The display unit 33 can be configured, for example, with an LCD, an organic EL display, or the like.

[0040] The operation input unit 35 is an input device that receives information input from a user or other party. The operation input unit 35 can use input devices such as a keyboard, mouse, or touch panel.

[0041] The communication unit 37 is equipped with either a wired or wireless interface, or both, for communicating with other devices (such as the information processing device 10) via, for example, a LAN, WAN, the Internet, or a telephone line. The communication unit 37 may also be equipped with interfaces related to (short-range) wireless communication technologies such as Bluetooth®, Wi-Fi®, ZigBee®, IrDA, or Wireless USB.

[0042] The memory unit 39 is one or more storage devices that store various programs and data necessary for the operation of the terminal device 30. The memory unit 39 can be composed of, for example, RAM, HDD, SSD, ROM, and other storage devices.

[0043] In the first embodiment, the storage unit 39 stores the control program 391 and the display control program 393, and reserves the form image storage area 395.

[0044] The control program 391 is a program that the control unit 31 reads when comprehensively controlling the terminal device 30. After reading the control program 391, the control unit 31 functions as an OS, controlling the operation of hardware such as the display unit 33, the operation input unit 35, and the communication unit 37, as well as performing startup and shutdown processes for applications running on the OS, file management, and so on.

[0045] The display control program 393 is a program that the control unit 11 reads when performing display control of the display unit 33. The control unit 31, having read the display control program 393, performs display control of the display unit 33 based on the input image information and response commands received from the information processing device 10.

[0046] The document image storage area 395 is a storage area for storing document image files to be uploaded to the information processing device 10, or document image files acquired by other means (for example, an image reading device (scanner device) not shown, another terminal device, or transfer from an external storage device such as a USB).

[0047] [1.3 Processing Flow] Next, the processing flow according to the first embodiment will be described. Figure 4 is a sequence diagram illustrating the exchange of information and commands between the information processing device 10 and the terminal device 30. The sequence diagram in Figure 4 will be explained using a report image file uploaded from the terminal device 30 to the information processing device 10 as an example.

[0048] When a form image file is uploaded from the terminal device 30 (S10), the information processing device 10 performs the process of generating an input screen based on the uploaded form image file.

[0049] The information processing device 10 transmits the generated input screen information to the terminal device 30 (S12).

[0050] In the terminal device 30, the user inputs and selects item values ​​via an input screen based on input screen information received from the information processing device 10. The user's operations related to inputting and selecting item values ​​are transmitted to the information processing device 10 as input (selection) commands (S14).

[0051] The information processing device 10 sends a response command to the terminal device 30 in response to the input (selection) command received from the terminal device 30 (for example, a command to display a list of character candidates based on the selection of the OCR result button) (S16).

[0052] The transmission of input (selection) commands via the terminal device 30 and the transmission of response commands by the information processing device 10 based on the received input (selection) commands continue until the user has completed inputting (selecting) the item values.

[0053] Once the user has finished entering (selecting) the item values, the terminal device 30 sends a registration request command to the information processing device 10 based on the user's request (S18). Upon receiving the registration request command, the information processing device 10 performs the processing related to form registration, and the series of processes is completed.

[0054] Prior to describing the specific processing according to the first embodiment, the process of generating an input screen based on OCR processing of a form image file will be explained using the flowchart in Figure 5. Figure 5 is a flowchart that explains the process executed by the control unit 11 of the information processing device 10, in particular, by reading the character recognition processing program 173 and the input screen generation program 1751 of the form registration service program 175.

[0055] First, the control unit 11 receives the document image file uploaded from the terminal device 30 (step S100).

[0056] Next, the control unit 11 performs OCR processing on the received document image file (step S110). The control unit 11 determines whether or not recognition information was obtained as a result of the OCR processing (step S120).

[0057] If the control unit 11 determines that it has been able to acquire recognition information, it adds an OCR result button to the input screen generated in step S140, indicating that it has been able to acquire candidate item values ​​based on the recognition information (step S120; Yes → step S130). On the other hand, if the control unit 11 determines that it has not been able to acquire recognition information, it proceeds to step S140 without adding an OCR result button (step S120; No → step S140).

[0058] The control unit 11 generates an input screen for receiving item values ​​(step S140). Then, the control unit 11 transmits the generated input screen as input screen information to the terminal device 30 and terminates the process (step S150).

[0059] Incidentally, the input screen is not limited to the generation timing explained in the flowchart of Figure 5. It is also possible to generate and send the input screen to the terminal device 30 at the timing requested by the terminal device 30, even during OCR processing (before OCR processing is completed). In this case, it is preferable to also display the OCR processing status, indicating that OCR processing is in progress (before OCR processing is completed), on the input screen to be displayed.

[0060] Next, Figure 6 is a flowchart illustrating the process by which the control unit 11 of the information processing device 10 generates candidate item value information based on OCR processing of the first form image. The process described in Figure 6 is executed by the control unit 11, in particular, by reading the character recognition processing program 173, the input screen generation program 1751 of the form registration service program 175, etc.

[0061] First, the control unit 11 performs OCR processing on the document image file related to the first document image (step S200).

[0062] Next, the control unit 11 determines whether the document related to the document image file is a document that was "received" from a business partner or a document that was "issued" to a business partner (step S210). In this case, the control unit 11 may determine the category of the document based on the category (information) received by the user via the input screen, or it may be configured to automatically extract and determine whether the document belongs to the category of "received" or "issued".

[0063] If the control unit 11 determines that the document belongs to the category of "received" from a business partner, it acquires recognition information with the issuer as the business partner (step S210; Yes → step S220). On the other hand, if the control unit 11 determines that the document belongs to the category of "issued" to a business partner, it acquires recognition information with the recipient as the business partner (step S210; No → step S230).

[0064] Then, the control unit 11 generates relative coordinate information by converting the character position information into relative coordinates in the acquired recognition information (step S240). The reason for converting the character position information into relative coordinates is to enable the comparison of the (fixed) positions of characters between different forms, even if the form (file) size or form format patterns differ. The conversion to relative coordinates can be calculated by the ratio of the character's start point (start_position) and end point (end_position) to the image size (base_image_size).

[0065] Then, the control unit 11 determines whether or not it has received input or modification (hereinafter simply referred to as input) from the user for the item value corresponding to the business partner (step S250). In step S250, the user may directly input a specific company name, etc., as the item value corresponding to the business partner, or, if the company name acquired as recognition information (item value candidate) based on the OCR processing explained in Figure 5 is not correctly recognized (an incorrect company name is displayed), the user may manually correct it.

[0066] If the control unit 11 determines that it has received input of an item value from the user, it associates the received item value with the relative coordinate information generated in step S240, stores it as candidate item value information, and terminates the process (step S250; Yes → step S260 → step S270). On the other hand, if the control unit 11 determines that it has not received input of an item value from the user, it omits the process related to step S260, stores the relative coordinate information as candidate item value information, and terminates the process (step S250; No → step S270).

[0067] Next, Figure 7 is a flowchart illustrating the process related to setting item value candidates based on OCR processing of the second form image by the control unit 11 of the information processing device 10. The process described in Figure 7 is executed by the control unit 11, in particular, by reading the character recognition processing program 173, the input screen generation program 1751 of the form registration service program 175, etc.

[0068] First, the control unit 11 performs OCR processing on the document image file related to the second document image (step S300).

[0069] Next, the control unit 11 acquires recognition information from the OCR processing results of the document image file (step S310). Then, the control unit 11 generates relative coordinate information by converting the character position information in the acquired recognition information into relative coordinates (step S320).

[0070] The control unit 11 compares the generated relative coordinate information with the item value candidate information stored in step S270 of Figure 6 (step S330). Here, the control unit 11 compares the character information acquired as recognition information in step S310, the relative coordinate information related to the character, with the item value candidate information including the item value entered by the user in step S260 of Figure 6.

[0071] Based on the comparison result in step S330, if the control unit 11 determines that there is candidate item value information corresponding to the acquired recognition information, it sets the character information acquired as recognition information as a candidate item value (step S340; Yes → step S350). Then, the control unit 11 adds a candidate item value button to the input screen to indicate that a candidate item value has been set, and terminates the process (step S360). On the other hand, if the control unit 11 determines that there is no corresponding candidate item value information, it terminates the process (step S340; No).

[0072] Next, Figure 8 is a flowchart illustrating an example of input assistance processing by the control unit 11 of the information processing device 10. Figure 8 is a flowchart illustrating the processing performed by the control unit 11, in particular, by reading the input assistance program 1753 of the character recognition processing program 173 and the form registration service program 175. In the processing described in Figure 8, it is assumed that in step S360 of Figure 7, item value candidates have been set and item value candidate buttons have been added to the input screen.

[0073] The control unit 11 of the information processing device 10 displays the item value candidate buttons added to the input screen (step S400).

[0074] Next, the control unit 11 determines whether or not it has received a user instruction to select an item value candidate button (step S410).

[0075] If the control unit 11 determines that it has received a selection instruction from the user via one of the item value candidate buttons, it displays the set item value candidate in the input field and terminates the process (Step S410; Yes → Step S420). On the other hand, if the control unit 11 determines that it has not received a selection instruction from the user via one of the item value candidate buttons, it waits until it receives such an instruction (Step S410; No).

[0076] If item value candidate information corresponding to the recognition information obtained from the OCR processing result of the second form image is stored in this manner, the control unit 11 can set and display the recognition information as item value candidate for form registration. With this configuration, the user can accurately input (select) item values ​​with a high degree of accuracy.

[0077] Next, the document registration process in the information processing device 10 will be explained using the flowchart in Figure 9. Figure 9 is a flowchart that explains the process performed by the control unit 11, in particular, by reading the document registration program 1755 of the document registration service program 175.

[0078] The control unit 11 of the information processing device 10 determines whether or not it has received a command from the terminal device 30 regarding a user's request to register a form (step S500). If it determines that it has received a command to register a form, the control unit 11 registers the form based on the item values ​​entered and selected by the user (step S510). On the other hand, if it does not receive a command to register a form from the terminal device 30, the control unit 11 waits until it receives the command (step S500; No).

[0079] The control unit 11 of the information processing device 10 can receive a form registration request from the terminal device 30 after input assistance processing related to form registration. Therefore, the user can efficiently perform confirmation / correction work related to form registration.

[0080] Next, the processing flow from receiving input screen information in the terminal device 30 to sending a registration request command will be explained using the flowchart in Figure 10. Figure 10 is a flowchart that explains the processing performed by the control unit 31, in particular, by reading the display control program 393, etc.

[0081] The control unit 31 of the terminal device 30 determines whether or not it has received input screen information from the information processing device 10 (step S600). If it determines that it has received input screen information from the information processing device 10, the control unit 31 displays an input screen based on the input screen information on the display unit 33 and accepts user input and selection for the input fields of item values ​​(step S600; Yes → step S610). On the other hand, if it determines that it has not received input screen information from the information processing device 10, the control unit 31 waits until it receives the input screen information (step S600; No).

[0082] The control unit 31 transmits the user's operation related to inputting and selecting item values ​​as an input (selection) command to the information processing device 10, and determines whether or not it has received a response command from the information processing device 10 for the input (selection) command (step S620).

[0083] If the control unit 31 determines that it has received a response command, it displays an input screen that reflects the response command (step S620; Yes → step S630). On the other hand, if it determines that it has not received a response command, the control unit 31 waits until it receives the response command (step S620; No).

[0084] Next, the control unit 31 determines whether the user has finished entering or selecting an item value in the input field (step S640). If the control unit 31 determines that the user has finished entering or selecting an item value in the input field, it sends a registration request command related to form registration to the information processing device 10 and terminates the process (step S640; Yes → step S650). On the other hand, if the control unit 31 determines that the user has not finished entering or selecting an item value in the input field, it returns the process to step S610.

[0085] The control unit 31 of the terminal device 30 can send a registration request for a form to the information processing device 10 after performing input assistance processing related to form registration. Therefore, the user can efficiently perform confirmation / correction work related to form registration.

[0086] [1.4 Example of Operation] Next, an example of operation according to the first embodiment will be described. Figure 11 is a diagram illustrating an example of the display configuration of the file upload screen W10 that the control unit 31 displays on the display unit 33 when uploading a form image file to the information processing device 10 in the terminal device 30.

[0087] The file upload screen W10 includes an input tray area R10 and a reload button B10.

[0088] The input tray area R10 is a display area that displays the document image files to be uploaded. Figure 10 shows an example of three document image files uploaded to the information processing device 10: “Delivery Note Sample.pdf”, “Order Form Sample.pdf”, and “Invoice Sample.pdf”.

[0089] The document image files uploaded to the information processing device 10 are subjected to OCR processing. As illustrated in Figure 11, the OCR processing status can also be displayed in the label display area R12 as labels with brief explanations. Here, the OCR processing status is represented by the labels "Waiting for OCR processing (estimated waiting time; approximately 1 minute)" ("Delivery Note Sample.pdf"), "OCR processing in progress" ("Order Form Sample.pdf"), and "OCR processed" ("Invoice Samples.pdf"). In the first embodiment, even if the OCR processing is not yet complete, the user can open the input screen, as explained in the following figure, and input / select item values.

[0090] The reload button B10 is a button that accepts an update command for the file upload screen W10. When the user selects the reload button B10, the file upload screen W10 is updated. At this time, it is preferable that the OCR processing status displayed in the label display area R12 is also updated.

[0091] Figure 12 illustrates an example of the display configuration of the input screen W20 that accepts input of item values. The input screen W20 illustrated in Figure 12 is an example of the configuration of an input screen generated based on OCR processing of the first form image file (invoice sample1.pdf).

[0092] The input screen W20 includes a preview image display area R14 that displays a preview image of the form, an input field display area R16 that accepts input of item values, and a confirmation button B12.

[0093] The preview image display area R14 includes, for example, a page display area R141 capable of displaying a preview image equivalent to one A4 page, and a display change instruction area R143 that accepts display change instructions such as zooming in and out and page turning for the form displayed in the page display area R141. The display size of the preview image defined in the page display area R141 is not limited to the example shown in Figure 12 and can be changed as appropriate. For example, the display size of the preview image displayed in the page display area R141 may be reduced to 1 / 2, 1 / 4, etc., of the display size exemplified in Figure 12, and in addition to displaying the preview image of the form related to form registration, the preview image of the next form to be OCR-targeted may also be displayed. Furthermore, in addition to or instead of the display change instruction area R143, it is also possible to zoom in and out of the preview image by mouseover or click (double-click) operations on a specific location of the displayed preview image. Additionally, gesture operations such as swiping on the preview image may be assigned to page turning and page turning functions for the preview image. Figure 12 is an example of a preview image of a document (invoice) received by "XXXX Corporation" from "□□□□ Corporation".

[0094] The input field display area R16 includes a transaction information input tab T10 and a file information input tab T12, which accepts input of information regarding the attributes of the report image file, as input fields for item values. The transaction information input tab T10 and the file information input tab T12 are configured to be switchable by selecting the respective tab. The tab selected by the user is underlined to indicate that the tab is selected. On the other hand, tabs that are not selected by the user are, for example, grayed out, and input / selection via that tab is disabled.

[0095] This section describes the input methods for item values ​​related to the transaction information input tab T10. Figure 12 illustrates the input areas where item values ​​can be entered and selected via the transaction information input tab T10: the "Registered Number of Qualified Invoice Issuer" area R161, the "Customer" area R163, and the "Category" area R165. In Figure 12, it is assumed that the "Registered Number of Qualified Invoice Issuer" area R161, the "Customer" area R163, and the "Category" area R165 are shown with an OCR result button B14 indicating that some recognition information (item value candidates) has been obtained as a result of OCR processing. In addition to these input areas, the transaction information input tab T10 can also include other input areas not shown, such as "Date of Transaction," "Transaction Amount," "Document Management Number," "Retention Period," "Timestamp Expiration Date," and "Presence or Absence of Paper Document."

[0096] The "Registered Number of Qualified Invoice Issuer" area R161 is a field for entering the registered number of a qualified invoice issuer. This registration number is a 13-digit number that begins with the letter "T". The "Obtain from National Tax Agency System" button B16 is a button that accepts an instruction to output the company name associated with the entered registration number to the "Customer" area R163. If the registration number and company name are already registered in the customer list, you can enter the registered company name in the "Customer" area R163, and the registration number associated with that company name will be entered into the "Registered Number of Qualified Invoice Issuer" area R161. Note that Figure 12 is an example of a case where the correct registration number could not be obtained by OCR processing (only the initial letter "T"), in which case the user will need to correct it manually.

[0097] The “Customer” area R163 is an item that accepts input for customer information. Figure 12 shows an example of how “iVYUL:HenD Corporation”, “〇〇〇〇 Corporation”, “Osaka Sales Office”, “□×□□ Corporation”, and “To be discussed separately” were acquired as recognized information (candidate item values) through OCR processing. On the other hand, the document image file in question is an example of a document (invoice) received by "XXXX Corporation" from "□□□□ Corporation". Therefore, the correct item value in the "Customer" area 163 should be "□□□□ Corporation", the issuer. However, in the list of OCR processing results shown under "Select from OCR results", "□□□□ Corporation" is not correctly recognized and is recognized as "□×□□ Corporation". In this case, the user needs to manually input or correct the correct item value "□□□□ Corporation". Figure 12 shows the state after the user has corrected the item value in the "Customer" area R163 to "□□□□ Corporation".

[0098] The “Category” area R165 is an item that receives a selection instruction from the user to determine whether the document related to the document image file is a document that was “received” from a business partner or a document that was “issued” to a business partner. When the user selects either the “received” or “issued” checkbox, the control unit 11 acquires recognition information that identifies the business partner as either the issuer (if “received” is selected) or the recipient (if “issued” is selected).

[0099] Next, we will explain an example of recognition information obtained from the OCR processing result when the document related to the document image file (first document image) is a document "received" from a business partner. First, the control unit 11 obtains information such as character information (for example, "□×□□ Co., Ltd.") and character position information (for example, start point "[2478,496]", end point "[3069,555]", and image size "[3307,4677]") as recognition information, with the business partner being the issuer of the document. The start point, end point, and image size can be determined from the x-coordinate distance and y-coordinate distance from an arbitrary reference point [0,0] related to the OCR processing, and for example, the reading start position of the image reading device (scanner device) can be set as the reference point.

[0100] Next, an example of candidate item value information generated by associating the item value entered by the user with relative coordinate information obtained by converting the position information of characters into relative coordinates will be described. The control unit 11 generates candidate item value information by associating the item value "input character" (for example, "□□□□ Co., Ltd.") entered by the user with character information (for example, "□×□□ Co., Ltd."), relative coordinate information "relative coordinates" (for example, "[0.75,0.11],[0.93,0.12]"), and classification information "classification" (for example, "received").

[0101] The recognition information and candidate item value information exemplified here may be in any file format that allows for comparison between different types of information, such as text files, XML files, JSON files, or even binary data.

[0102] Next, we will explain an example of recognition information obtained from the OCR processing result when the document related to the document image file (second document image) is a document "received" from a business partner. The control unit 11 obtains information such as text information (for example, "□△□□ Co., Ltd.") and text position information (for example, start point "[2230,446]", end point "[2762,500]", and image size "[2976,4209]") as recognition information, with the business partner being the issuer of the document.

[0103] The control unit 11 compares the relative coordinates (relative coordinate information) and character information calculated from the previously acquired recognition information (character position information) with the relative coordinates (relative coordinate information) and character information included in the item value candidate information, and determines whether they are similar or not. Here, "similar" in this disclosure means that the characters (sequences) indicated by the character information are identical or similar in terms of the type of characters (hiragana, katakana, kanji, numbers, alphabet, etc.), the number of characters, and the fixed position on the form (start point, end point, and image size) between the recognition information and the item value candidate information. In the similarity determination in this disclosure, it is also possible to determine that they are "similar" if the difference (error) between the relative coordinates related to the recognition information and the relative coordinates related to the item value candidate information falls within an arbitrarily set tolerance range, for example, ±10% or less, preferably ±5% or less, or to perform the analogy determination using relative coordinates based on the position information of either the start point or the end point of the character. Furthermore, the recognition information and the candidate item value information may be compared based on the degree of character matching relative to the number of characters (or characters) in the character information (for example, 3 out of 5 characters match). In addition, the comparison between the recognition information and the candidate item value information can be performed based on either relative coordinate information or character information. Moreover, in the similarity determination related to this disclosure, a machine learning model may be used to improve the accuracy (confidence) of the similarity determination.

[0104] The relative coordinates of “□△□□ Corporation”, shown in the example text information, can be calculated as [0.75(2230 / 2976),0.11(446 / 4209)],[0.93(2762 / 2976),0.12(500 / 4209)]. This calculation result matches the relative coordinates of “□×□□ Corporation”, shown in the example text information. Furthermore, “□△□□ Corporation” and “□×□□ Corporation” differ by only one character. Based on these similarity determinations, the control unit 11 determines that the relative coordinate information and text information calculated from the recognition information are similar to the relative coordinate information and text information included in the item value candidate information, and determines that the recognition information corresponding to the item value candidate information has been acquired. The item value candidate information is associated with the item value entered by the user as “input text” (“□□□□ Corporation”). Therefore, the control unit 11 replaces the character information “□△□□ Corporation” acquired as recognition information with the character information “□□□□ Corporation” and sets it as a candidate item value.

[0105] Figure 13 illustrates an example of the display configuration of input screen W30, which accepts input of item values. The input screen W30 illustrated in Figure 13 is an example of the configuration of an input screen generated based on OCR processing of the document image file related to the second document image (invoice sample2.pdf).

[0106] The configuration of input screen W30 can be the same as that of input screen W20 illustrated in Figure 12, so a description of this configuration will be omitted. Input screen W30 differs from input screen W20 in that item value candidate buttons B18 are displayed in the "Customer" area R163.

[0107] The item value candidate button B18 also serves as an identifier indicating that an item value candidate has been set. When the user selects the item value candidate button B18, the control unit 11 displays the set item value candidate in the item value candidate display area R1631 so that it can be selected. Figure 13 is an example where the set item value candidate is "□□□□ Co., Ltd.". By selecting the item value candidate displayed in the item value candidate display area R1631, the user can input the business partner in the "Business Partner" area 163.

[0108] As described above, according to the first embodiment, even if the correct registration number related to the customer's input cannot be obtained through OCR processing, or if the correct recognition result related to the customer's input cannot be obtained, registering the correct customer once can reduce the effort and time required for subsequent form registrations.

[0109] [2 Second Embodiment] The second embodiment is a configuration in which the source for uploading the document image file is the image processing device, as in the first embodiment. Components and functions identical to those in the first embodiment are denoted by the same reference numerals, and their descriptions are omitted.

[0110] [2.1 Overall Structure] Figure 14 is a diagram illustrating an example of the system configuration of the document registration system 200 according to the second embodiment.

[0111] The document registration system 200 includes an information processing device 10, a terminal device 30, and a multifunction printer 50 which serves as an image processing device. The information processing device 10, the terminal device 30, and the multifunction printer 50 are connected via a network NW and are configured to enable mutual transmission and reception of information.

[0112] [2.2 Functional Configuration] Figure 15 is a diagram illustrating the functional configuration of the multifunction printer 50. The multifunction printer 50 reads a document image (file) in response to an upload instruction from the terminal device 30 and uploads the generated image file to the information processing device 10. Such a multifunction printer 50 includes a control unit 51, a display unit 53, an operation input unit 55, a communication unit 57, an image forming unit 59, an image input unit 61, and a storage unit 63.

[0113] The control unit 51 controls the entire multifunction device 50. The control unit 51 is composed of, for example, one or more arithmetic units (CPU (Central Processing Unit), SoC (System on a Chip), etc.). The control unit 51 realizes its functions by reading and executing various programs stored in the storage unit 63.

[0114] The display unit 53 is a display device that displays various information to the user. The display unit 53 can be configured, for example, with an LCD or an organic EL (Electro-luminescence) display.

[0115] The operation input unit 55 accepts information input from the user or other party. The operation input unit 55 can be configured with various input devices such as operation keys and buttons, such as hard keys and software keys. The operation input unit 55 can also be configured as a touch panel display that allows input via the display unit 53. In this case, common methods such as resistive touch, infrared touch, electromagnetic induction touch, and capacitive touch can be used as the input method for the touch panel.

[0116] The communication unit 57 is equipped with either a wired or wireless interface, or both, for communicating with other devices (such as the information processing device 10 or terminal device 30) via a network (NW) such as a LAN, WAN, the Internet, a telephone line, or a fax line. The communication unit 57 may also be equipped with interfaces related to (short-range) wireless communication technologies such as Bluetooth®, NFC (Near Field Communication), Wi-Fi®, ZigBee®, Irda, or Wireless USB.

[0117] The image forming unit 59 feeds paper from a paper tray (not shown), forms an image on the paper based on image data, and then discharges the paper to a paper output unit (not shown). The image forming unit 59 can be configured, for example, as a laser printer using an electrophotographic method. In this case, the image forming unit 59 performs image formation using toner supplied from a toner cartridge (not shown) corresponding to the toner color (e.g., cyan, magenta, yellow, black).

[0118] The image input unit 61 generates image data (document image file) by scanning a document such as a form. The image input unit 61 can be configured as a scanner device equipped with an image sensor such as a CCD (Charge coupled device) or CID (Contact image sensor), and having an automatic document feeder (ADF) or a flatbed for placing and reading documents. The image input unit 61 is not particularly limited in its configuration as long as it is capable of generating image data by reading the reflected light image from the document image with the image sensor. The image input unit 61 can also be configured as an interface capable of acquiring image data stored on a portable storage medium such as a USB memory, or image data transmitted from an external terminal device (not shown).

[0119] The memory unit 63 stores various programs and data necessary for the operation of the multifunction printer 50. The memory unit 21 can be composed of one or more storage devices such as RAM, HDD, SSD, or ROM.

[0120] In the second embodiment, the storage unit 63 stores the control program 631 and reserves the form image storage area 633.

[0121] The control program 631 is a program that the control unit 51 reads when comprehensively controlling the multifunction printer 50. After reading the control program 631, the control unit 51 functions as an OS and controls the operation of the display unit 53, operation input unit 55, communication unit 57, image forming unit 59, image input unit 61, etc., thereby setting, executing, and post-processing various jobs such as printing, copying, faxing, and sending emails.

[0122] The document image storage area 633 is a storage area for storing document image files generated by the image input unit 61, or document image files acquired by other acquisition means (for example, other terminal devices not shown, or transfer from external storage devices such as USB (Universal Serial Bus)).

[0123] [2.3 Processing Flow] Next, the processing flow according to the second embodiment will be described. Figure 16 is a sequence diagram illustrating the exchange of information and commands between the information processing device 10, the terminal device 30, and the multifunction printer 50. The same reference numerals are used for parts that were described in Figure 4, and their explanations are omitted.

[0124] A command to read the document image is sent from the terminal device 30 to the multifunction printer 50 (S20).

[0125] Upon receiving a command to read a form image, the multifunction printer 50 reads the form via the image input unit 61 and generates a form image file of that form. The multifunction printer 50 then uploads the generated form image file to the information processing device 10.

[0126] As described above, according to the second embodiment, in addition to the effects of the first embodiment, since the configuration uploads the form image files generated by the multifunction printer to the information processing device, it is possible to prevent the storage capacity of the terminal device's storage unit from being strained by the storage of form image files.

[0127] [3 Third Embodiment] The third embodiment is a configuration in which the functions of the document registration system described in the first embodiment can be realized by the information processing device alone.

[0128] [3.1 Functional Configuration] Figure 17 is a diagram illustrating the functional configuration of the information processing device 70 according to the third embodiment. The functional configuration of the information processing device 70 according to the third embodiment can be configured to include a display unit 71 in addition to the functional configuration of the information processing device 10 according to the first embodiment. Other configurations can be the same as those of the information processing device 10, so their explanation is omitted here.

[0129] The display unit 71 is a display device that displays various information to the user. The display unit 71 can be configured, for example, with an LCD, an organic EL display, etc. The display unit 71 displays an input screen, etc., based on the input screen information generated by the control unit 11, which reads the input screen generation program 1751.

[0130] The processing flow by the information processing device 70 can be executed in the same way as the processing flow described in the first embodiment, so a detailed explanation is omitted here.

[0131] As described above, according to the third embodiment, the document registration system described in the first embodiment can be realized with the smallest possible device configuration, thus reducing costs related to the maintenance of terminal devices, image processing devices, etc.

[0132] This disclosure is not limited to the embodiments described above, and various modifications are possible. That is, embodiments obtained by combining technical means that are appropriately modified without departing from the gist of this disclosure are also included in the technical scope of this disclosure.

[0133] Furthermore, although the embodiments described above are explained separately for the sake of explanation, it goes without saying that they may be combined and implemented to the extent that is technically possible.

[0134] Furthermore, in the embodiments, the programs that run in each device are programs that control the CPU and the like (programs that make the computer function) in order to realize the functions of the embodiments described above. The information handled by these devices is temporarily stored in a temporary storage device (for example, RAM) during processing, and then stored in storage devices such as various ROMs (Read Only Memory) and HDDs, and read, modified, and written by the CPU as needed.

[0135] Here, the computer-readable non-transient recording medium on which the program in the information processing device is recorded may be any of the following: semiconductor media (e.g., ROM, non-volatile memory card, etc.), optical recording media / magneto-optical recording media (e.g., DVD (Digital Versatile Disc), MO (Magneto Optical Disc), MD (Mini Disc), CD (Compact Disc), BD (Blu-ray® Disc, etc.)), magnetic recording media (e.g., magnetic tape, flexible disk, etc.). In this case, the program recorded on the recording medium is read by the computer of the information processing device and executed by the computer, thereby realizing the functions of the embodiments described above. Furthermore, the functions of this disclosure are realized by processing in cooperation with the operating system or other application programs, etc., based on the instructions of the program.

[0136] Furthermore, when distributing the program to the market, it can be stored on a portable recording medium and distributed, or transferred to a server computer connected via a network such as the Internet. In this case, the storage device of the server computer is, of course, also included in this disclosure.

[0137] Furthermore, each functional block or feature of the apparatus used in the embodiments described above can also be implemented and executed by an electrical circuit, such as an integrated circuit or a plurality of integrated circuits. An electrical circuit designed to realize the functions described herein may include a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gates or tronograph logic, discrete hardware components, or a combination thereof. The general-purpose processor may be a microprocessor, a conventional processor, controller, microcontroller, or state machine. The aforementioned electrical circuit may consist of digital circuits or analog circuits. Also, if advances in semiconductor technology lead to the emergence of integrated circuit technologies that replace current integrated circuits, one or more aspects of this disclosure may use new integrated circuits based on such technologies. [Explanation of Symbols]

[0138] 10 Information Processing Devices 11 Control Unit 13. Operation Input Section 15 Communications Department 17 Memory section 171 Control Program 173 Character Recognition Processing Program 175 Document Registration Service Program 1751 Input screen generation program 1753 Input assistance program 1755 Document Registration Program 177 Document Image Storage Area 179. Document item value storage area 30 Terminal devices 31 Control Unit 33 Display section 35 Operation Input Section 37 Communications Department 39 Memory section 391 Control Program 393 Display control program 395 Document Image Storage Area 50 Multifunction device 51 Control Unit 53 Display section 55 Operation Input Section 57 Communications Department 59 Image forming unit 61 Image Input Section 63 Storage section 631 Control Program 633 Document Image Storage Area 100, 200 Form Registration System

Claims

1. A generation unit that generates an input screen having a display area for displaying a report image and accepting input of item values ​​for items included in the report, The system includes a control unit that assists in inputting the item values ​​by controlling the display of the generated input screen, and registers the form based on the received item values, The control unit, The recognition information obtained by character recognition processing on the first document image is associated with the input item values ​​and stored as candidate item value information. An information processing device characterized in that, in character recognition processing for a second form image related to new form registration, if recognition information corresponding to the item value candidate information is obtained, the item value associated with the item value candidate information is set as an item value candidate related to new form registration.

2. The control unit, The information processing apparatus according to claim 1, characterized in that it acquires character information and positional information of the characters on the image of the document as the recognition information, based on classification information indicating whether the document has been received or issued.

3. The control unit, The information processing apparatus according to claim 2, characterized in that relative coordinate information obtained by converting the positional information of the character into relative coordinates is stored as candidate item value information.

4. The control unit, The information processing apparatus according to claim 3, characterized in that it determines whether or not the recognition information corresponding to the item value candidate information has been acquired by comparing the recognition information acquired in the character recognition processing of the second document image with the relative coordinate information.

5. The control unit, The information processing device according to claim 1, characterized in that it displays an identifier indicating that the aforementioned candidate item value has been set, in association with the item.

6. A document registration system comprising an information processing device and a terminal device, A generation unit that generates an input screen having a display area for displaying a report image and accepting input of item values ​​for items included in the report, A control unit that assists in inputting the item values ​​by controlling the display of the generated input screen, and registers the form based on the received item values, The system includes a display unit that displays the input screen based on the control by the control unit. The control unit, The recognition information obtained by character recognition processing on the first document image is associated with the item values ​​entered by the user and stored as candidate item value information. A document registration system characterized in that, in character recognition processing for a second document image related to new document registration, if recognition information corresponding to the candidate item value information is obtained, the item value associated with the candidate item value information is set as a candidate item value related to new document registration.

7. It generates an input screen that has a display area for displaying a report image and accepts input of item values ​​for items included in the report. The system assists in inputting the item values ​​by controlling the display of the generated input screen, and registers the report based on the received item values. The recognition information obtained by character recognition processing on the first document image is associated with the item values ​​entered by the user and stored as candidate item value information. A document registration method characterized in that, in character recognition processing for a second document image related to new document registration, if recognition information corresponding to the candidate item value information is obtained, the item value associated with the candidate item value information is set as a candidate item value related to new document registration.

Images