Information processing system, information processing system control method, and program

Abstract

To facilitate grasping a read source corresponding to OCR result data.SOLUTION: The present invention is characterized by comprising: registration means for registering the location of a series of OCR-read texts from a scanned form image in a storage unit; and first display means for associating the series of texts with each item of form data and displaying the series of texts as values in each item of the form data and second display means for displaying the scanned form screen. In response to access from a user to form data items on the screen displayed by the first display means, the second display means acquires the location of the series of texts of the accessed item from the storge unit and displays the location of the series of texts in an enlarged view.SELECTED DRAWING: Figure 6

Description

[Technical field]

[0001] The present invention relates to an information processing system, a control method for an information processing system, and a program. [Background technology]

[0002] The information on the paper form is scanned using a scanner or other device, and the scanned form image is converted into data using OCR (optical character recognition) to read each item and obtain it as electronic data.

[0003] However, if the form image (for example, an invoice) is not in a standard format (non-standard form), the relationship between the reading position of the form image by OCR and the corresponding item value is not defined, so the reading accuracy is not 100%. Therefore, the user needs to visually check whether the items to be converted into data and the values ​​of the read data correspond correctly.

[0004] Patent Document 1 discloses a technique for reducing the input work of a user by identifying and displaying items that the user intends to input, even in the case of non-standard forms. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] Patent Publication 2014-186435 DISCLOSURE OF THEINVENTION [Problem to be solved by the invention]

[0006] However, there are times when visual inspection is difficult when characters printed in small letters, such as on invoices and receipts, can be difficult. For example, when scanning a vertically long receipt and checking the OCR image, even if the entire vertically long receipt is displayed on the screen and the corresponding items on the form image are identified and displayed, the letters will be small and it will be difficult to confirm whether the corresponding items are correct.

[0007] An object of the present invention is to easily grasp the source of reading corresponding to data resulting from OCR. [Means for solving the problem]

[0008] In order to solve the above problem, the present invention comprises a registration means for registering in a memory unit the positions of a series of characters obtained by optically reading a scanned form image, a first display means for linking the series of characters to each item of form data and displaying the series of characters on a screen as values ​​for each item of the form data, and a second display means for displaying the scanned form screen, wherein in response to a user accessing an item of the form data on the screen displayed by the first display means, the second display means obtains from the memory unit the positions of the series of characters of the item that has accepted the access, and enlarges and displays the position of the series of characters. Effect of the Invention

[0009] According to the present invention, it is possible to easily grasp the source of the read data corresponding to the OCR result. [Brief description of the drawings]

[0010] [Figure 1] FIG. 1 is a diagram illustrating an example of a schematic configuration of an information processing system. [Diagram 2] FIG. 2 is a block diagram showing an example of a hardware configuration of a person in charge terminal, a manager terminal, and a server. [Diagram 3] FIG. 2 is a block diagram showing an example of a hardware configuration of a multifunction peripheral having a scanner function. [Figure 4] 13 is a flowchart showing an example of an invoice reading process. [Diagram 5] 13 is a flowchart showing an example of a display process for confirming an invoice. [Figure 6] 13 is an example of a screen shot of a web page listing invoice documents. [Figure 7]1 is an example of a screen shot of an invoice document details web page. [Figure 8] 1 is an example of a screen shot of an invoice document details web page. [Figure 9] 1 is an example of a screen shot of a web page for inputting an invoice index. [Figure 10] FIG. 13 is a schematic diagram showing an example of an invoice screen displayed on the screen. [Figure 11] 13 is an example of a data table that specifies OCR position information and item linking information on an invoice screen. [Figure 12] 1 is an example of a screen shot of a web page for inputting an invoice index. [Figure 13] 1 is an example of a screen shot of a web page for inputting an invoice index. [Figure 14] 1 is an example of a screen shot of a web page for inputting an invoice index. [Figure 15] 1 is an example of a screen shot of a web page for inputting an invoice index. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0011] Hereinafter, an embodiment of the present invention will be described in detail with reference to the drawings.

[0012] FIG. 1 is a diagram showing an example of a system configuration of an information processing system according to an embodiment of the present invention.

[0013] The information processing system 100 comprises a scanner device 102, an invoice issuer terminal 104, an MFP (Multi Function Peripheral) 108, an invoice management server 105 placed on an invoice management cloud 114, a database server 106, and an invoice recipient terminal 110 on the invoice recipient side, all of which are communicatively connected via a network 112.

[0014] In the present invention, the invoice management server 105 and the database server 106 are virtual servers located on the invoice management cloud 114. Note that each server does not have to be located on the cloud 114, and each may be located in its own housing, or each function may be integrated into one or more housings.

[0015] The invoice issuer terminal 104 is a terminal used by an issuer who creates an invoice. A PDF file of the invoice created on the invoice issuer terminal 104 is transmitted to the invoice management server 105 via the network 112. In addition, for an invoice created on paper, a PDF file scanned by the scanner device 102 or MFP 108 may be transmitted to the invoice management server 105.

[0016] The invoice management server 105 stores the invoice file acquired from the invoice issuer terminal 104 in the database server 106. It also analyzes and digitizes the image of the invoice file, checks whether the invoice is eligible for payment, and registers the data in the database server 106. Details of the process will be described with reference to Figs. 4 and 5.

[0017] The invoice recipient terminal 110 is a terminal used by the recipient of the invoice. The invoice recipient terminal 110 searches for and downloads PDF files of invoices received by the invoice management server 105 via the network 112, and performs processes such as viewing. It can also obtain data analyzed by the invoice management server 105 and check payment processing information. The invoice PDF file received from the invoice issuer terminal 104 as an attachment to an email or the like is sent to the invoice management server 105, and if the invoice is sent on paper, the invoice is scanned by the scanner device 102, MFP 108, or the like, and sent to the invoice management server 105.

[0018] FIG. 2 is a block diagram showing an example of the hardware configuration of the bill issuer terminal 104, Web bill management server 105, database server 106, and bill recipient terminal 110, as well as these information processing devices, in an embodiment of the present invention.

[0019] As shown in FIG. 2, these information processing devices are connected to a CPU (Central Processing Unit) 201, a ROM (Read Only Memory) 202, a RAM (Random Access Memory) 203, an input controller 205, a video controller 206, a memory controller 207, and a communication I / F controller 208 via a system bus 204.

[0020] The CPU 201 comprehensively controls each device and controller connected to a system bus 204 .

[0021] The ROM 202 or external memory 211 holds a BIOS (Basic Input / Output System) and an OS (Operating System), which are control programs executed by the CPU 201, programs that can be read and executed by this information processing device, and various necessary data (including data tables).

[0022] The RAM 203 functions as a main memory, a work area, etc., for the CPU 201. The CPU 201 loads programs and the like required for executing processes from the ROM 202 or the external memory 211 into the RAM 203, and executes the loaded programs to realize various operations.

[0023] The input controller 205 controls input from an input device such as a keyboard 209 or a pointing device such as a mouse (not shown). If the input device is a touch panel, the user can give various instructions by pressing (touching with a finger or the like) icons, cursors, or buttons displayed on the touch panel.

[0024] The touch panel may be a touch panel capable of detecting positions touched by multiple fingers, such as a multi-touch screen.

[0025] The video controller 206 controls display on an external output device such as a display 210. The display also includes the display of a notebook computer integrated with the main body. Note that the external output device is not limited to a display, and may be, for example, a projector. In addition, for devices capable of receiving the above-mentioned touch operation, an input device is also provided.

[0026] The video controller 206 is capable of controlling a video memory (VRAM) for display control, and can use part of the RAM 203 as the video memory area, or can provide a separate dedicated video memory.

[0027] The memory controller 207 controls access to the external memory 211. As the external memory, an external storage device (hard disk) for storing a boot program, various applications, font data, user files, edit files, and various data, a flexible disk (FD), or a compact flash (registered trademark) memory connected to a PCMCIA card slot via an adapter, etc., can be used.

[0028] The communication I / F controller 208 connects and communicates with external devices via a network, and executes communication control processing on the network. For example, communication using TCP / IP, telephone lines such as ISDN, and 3G lines for mobile phones are possible.

[0029] The CPU 201 enables display on the display 210 by, for example, executing a process of expanding (rasterizing) an outline font in a display information area in the RAM 203. The CPU 201 also enables a user to give instructions using a mouse cursor (not shown) on the display 210.

[0030] Next, the hardware configuration of the multifunction peripheral 108 having the scanner function shown in FIG. 1 will be described with reference to FIG.

[0031] FIG. 3 is a block diagram showing an example of a hardware configuration of the multifunction peripheral 108 having the scanner function shown in FIG.

[0032] In FIG. 3, 316 denotes a controller unit which is connected to a scanner unit 314 functioning as an image input device and a printer unit 312 functioning as an image output device, and is also connected to a LAN (e.g., LAN 112 shown in FIG. 1) or a public line (WAN) (e.g., PSTN or ISDN, etc.) to input and output image data and device information.

[0033] In the controller unit 316, a CPU 301 is a processor that controls the entire system. A RAM 302 is a system work memory for the operation of the CPU 301, and also serves as a program memory for recording programs and an image memory for temporarily recording image data.

[0034] Reference numeral 303 denotes a ROM in which a boot program and various control programs for the system are stored, and 304 denotes a hard disk drive (HDD) in which various programs for controlling the system, image data, and the like are stored.

[0035] An operation unit interface (operation unit I / F) 307 is an interface with an operation unit (keyboard) 308. The operation unit I / F 307 also plays a role in transmitting key information input from the operation unit 308 (for example, pressing a start button) to the CPU 301.

[0036] Reference numeral 305 denotes a network interface (Network I / F) which is connected to the network (LAN) 112. It is also configured to be capable of wireless communication, and connects to other devices via communication using infrared, Bluetooth (registered trademark), or Wi-Fi (registered trademark) to input and output data. Reference numeral 306 denotes a modem which connects to a public line and inputs and outputs data such as sending and receiving faxes.

[0037] Reference numeral 318 denotes an external interface (external I / F), which is an I / F unit that accepts external inputs such as USB, IEEE1394, a printer port, and RS-232C, and in this embodiment, a card reader 319 for reading an IC card (storage medium) of a mobile terminal required for authentication is connected to the external I / F 318. The CPU 301 controls the reading of information from the IC card of the mobile terminal by the card reader 319 via this external I / F 318, and can obtain the information read from the IC card of the mobile terminal. The above devices are arranged on a system bus 309.

[0038] An image bus interface (IMAGE BUS I / F) 320 is a bus bridge that connects the system bus 309 and an image bus 315 that transfers image data at high speed and converts the data structure.

[0039] The image bus 315 is configured as a PCI bus or IEEE 1394. The following devices are connected to the image bus 315.

[0040] Reference numeral 310 denotes a raster image processor (RIP), which develops vector data such as PDL code into a bitmap image. Reference numeral 311 denotes a printer interface (printer I / F), which connects a printer section 312 to a controller unit 316 and performs synchronous / asynchronous conversion of image data. Reference numeral 313 denotes a scanner interface (scanner I / F), which connects a scanner section 314 to the controller unit 316 and performs synchronous / asynchronous conversion of image data.

[0041] An image processing unit 317 performs correction, processing, and editing on input image data, and performs printer correction, resolution conversion, etc. on print output image data. In addition, the image processing unit 317 also performs image data rotation, and compression and decompression processing such as JPEG for multi-value image data and JBIG, MMR, MH for binary image data.

[0042] The scanner unit 314 illuminates an image on a paper document, scans it with a CCD line sensor, and converts it into an electrical signal as raster image data. The document paper is set on the document feeder tray, and when the device user issues a command to start reading from the operation unit 308, the CPU 301 gives a command to the scanner unit 314, and the feeder feeds the document paper one sheet at a time and performs the operation of reading the document image.

[0043] Printer unit 312 is a part that converts raster image data into an image on paper, and any method can be used, such as an electrophotographic method using a photosensitive drum or photosensitive belt, or an inkjet method in which ink is ejected from a minute nozzle array to directly print an image on paper. The printing operation is started by an instruction from CPU 301. Printer unit 312 has multiple paper feed stages so that different paper sizes or different paper orientations can be selected, and corresponding paper cassettes are provided.

[0044] The operation unit 308 has an LCD display unit with a touch panel sheet attached onto the LCD, and displays a system operation screen, and when a displayed key is pressed, transmits position information of the key to the CPU 301 via the operation unit I / F 307. The operation unit 308 also has various operation keys, such as a start key, a stop key, an ID key, and a reset key.

[0045] Furthermore, the display performance of the display unit varies depending on the multifunction device, and the present invention is composed of a multifunction device that can be operated via a touch panel, and a multifunction device that simply has an LCD screen and displays strings of text (such as the printing status and the name of the document being printed).

[0046] The start key on the operation unit 308 is used when starting the operation of reading an original image, etc. A two-color LED, green and red, is located in the center of the start key, and the color indicates whether the start key is ready to use. The stop key on the operation unit 308 serves to stop an operation that is in progress. The ID key on the operation unit 308 is used when entering the user ID of the user. The reset key is used when initializing the settings from the operation unit.

[0047] Under the control of CPU 301, card reader 319 reads information stored in an IC card (which may be provided in a mobile terminal as an IC chip) and notifies CPU 301 of the read information via external I / F 318.

[0048] In addition, the card reader 319 is compatible with the NFC communication standard and is configured to be able to read and write data from and to IC chips in IC cards and mobile terminals. When an NFC-compatible mobile terminal is held over an NFC-compatible card reader, authentication is performed and pairing is performed between the mobile terminal and the multifunction device.

[0049] Then, communication (P2P) can be established between the held-up mobile terminal and the multifunction device to exchange data. In addition, communication can be handed over to Bluetooth (registered trademark) or Wi-Fi (registered trademark), which are high-speed communication standards, to allow communication between the mobile terminal and the multifunction device.

[0050] For example, by holding a mobile terminal over a card reader, it becomes possible to transmit images stored in the mobile terminal to a multifunction device. Note that the details of the NFC communication standard are omitted here because they are conventional technology.

[0051] In the above-mentioned multifunction device 108, a platform for controlling the multifunction device 108 exists, and an authentication application for communicating with an authentication server (which may be the server 106) runs on this platform. The authentication application is stored in the HDD 304. Areas are reserved on the HDD 304 for a login context that stores user information at login and various setting information, which are managed by the platform.

[0052] Furthermore, applications that extend the main functions of the multifunction device 108 are installed and run on the platform. These applications are executed using the API of the platform.

[0053] Through this platform, each function of the multifunction device 108 can be controlled.

[0054] The MFP 108 also stores a Web browser, and can cooperate with a Web system. In this case, a screen received from a Web application server is displayed using the Web browser. A command given on the Web browser is requested to the Web application server, and by receiving the command from the Web application server, the MFP 108 can execute an operation (such as scanning or printing).

[0055] With the above-described configuration, the multifunction device 108 can transmit image data read from the scanner unit 314 onto the LAN 112 , and can print out print data received from the LAN 112 using the printer unit 312 .

[0056] In addition, image data read from a scanner unit 314 can be sent by FAX over a public line via a modem 306 , and image data received by FAX from a public line can be output by a printer unit 312 .

[0057] Next, the process of reading an invoice will be described with reference to the flowchart shown in Fig. 4. Note that, although the process will be described as an invoice reading process in this embodiment, it may be an image of a receipt, estimate, delivery note, or other document as long as the data recorded in the electronic book is acquired as index data.

[0058] Each step in FIG. 4 is a processing step executed by the CPU 201 of the bill recipient terminal 110, the bill management server 105, and the database server 106.

[0059] It is assumed that before the flowchart in FIG. 4(a), the bill issuer terminal 104 transmits a bill file (PDF) to the bill recipient terminal 110 via email or a server.

[0060] In S401 in FIG. 4(a), the invoice recipient terminal 110 receives the invoice file (PDF) sent from the invoice issuer terminal 104, and sends the invoice file to the invoice management server 105.

[0061] In S402, the invoice management server 105 receives the invoice file (PDF), and in S403 requests the database server 106 to register (store) the received invoice file (PDF). The database server 106 stores the invoice file (PDF) for which registration has been requested. Note that if the invoice file (PDF) is sent directly from the invoice issuer terminal 104 to the invoice management server 105, the processes of S401 to S403 are omitted, and the invoice management server 105 acquires the invoice file (PDF) stored in the database server 106.

[0062] In S404, the invoice management server 105 performs OCR processing on the invoice file (PDF). The OCR processing on the invoice file (PDF) will be described with reference to FIG. 4(b).

[0063] FIG. 4B is a flowchart that starts upon transition to S404 in FIG. 4A.

[0064] In S411 of (b) of Fig. 4, the invoice management server 105 judges whether the invoice file (PDF) is a text PDF. If it is a text PDF (searchable PDF), the process proceeds to S415, and if it is not a text PDF (a scanned PDF), the process proceeds to S412.

[0065] In S412, the invoice management server 105 performs character recognition processing on the invoice file (PDF). Although detailed explanations are omitted, in this processing, page division, image correction, noise removal, contrast adjustment, binarization, etc. of the invoice file (PDF) are performed to identify each character in the image and convert it into a corresponding character code. Conversion to character code (character recognition) is performed using a trained model and dictionary. Also, in S412, the location of the recognized character string is identified. In S412, for example, when an invoice image such as 1000 in FIG. 10 is subjected to OCR processing, information such as text PDF information 1100 in FIG. 11 is obtained.

[0066] In S413, the invoice management server 105 identifies a character string corresponding to an item of the invoice index from the text PDF information 1100 recognized in S412. The items of the invoice index (each item of the form data) are items such as those in the index portion 710 of FIG. 7. The association between the items of the invoice index 710 and the text PDF information 1100 is realized, for example, by storing a synonym list (for example, transaction amount, total, total amount including tax, etc. as synonyms) and, when the text PDF information is acquired, checking the synonym list to see if there is a matching invoice index. For example, if the text PDF information has a character string "invoice date" (1003 in FIG. 10), and "invoice date" and "issue date" of the invoice index (704 in FIG. 7) are registered as a list of character strings with the same meaning, item information (a series of characters: for example, data whose value changes depending on the invoice, such as "2023 / 4 / 20" display 1004 in FIG. 10) linked to the "issue date" of the text PDF information can be linked to the "issue date". Hereinafter, each item in the invoice index 710 will be referred to as an "invoice index item," and character strings such as "invoice" in the text PDF information will be referred to as "item name."

[0067] In S414, the invoice management server 105 acquires the item name in the text PDF information 1100 associated in S413 and the item information associated with this item name, and registers it as item linking information linked to the item of the invoice index associated in S413. Candidates for item information include character strings other than those registered in the character list with the same meaning as the item of the invoice index, character strings including "Inc." and "Registration number," etc. An example of item linking information is shown in 1110. The item linking information consists of the item of the invoice (index), the No. of the text PDF information, and the item name (character string) of the text PDF information.

[0068] Linking of item information corresponding to the item name of the text PDF information is performed based on the relationship between the position information of the item name of the text PDF information and the position information of the item information. Specifically, item information with the same value (or a close value) as the Y axis (vertical direction) of the position information of the item name may be obtained, or item information in a position closest to the item name may be obtained. In the case of the example of 1001 in Fig. 10, item information "32,800 yen (tax included)" is extracted (Y axis 1102 in the 21st line of 1100 in Fig. 11) as the item information close to the Y axis (vertical direction) 1101 of "Total" (the 20th line of 1100 in Fig. 11). The extracted item information is linked to the item name with the close position information.

[0069] When an item name and item information are linked, the item information is linked to the item of the invoice index associated with the item name. As shown in 1110 in Fig. 11, a column of item information No. and a column of item information are newly registered as item information linked to the item of the invoice (index).

[0070] This concludes the explanation of the OCR processing flow for scanned PDF invoices.

[0071] Next, the process when the invoice file (PDF) is a text PDF in S411 will be described.

[0072] When the process transitions to S415, the invoice management server 105 acquires the text information, text position information, table tags, etc., contained in the text PDF file. An example of the data from which the text information was acquired is shown in 1100 in the same Figure 11. In the case of a text PDF file, in addition to the example of 1100, multiple table tags may be included.

[0073] In S416, the invoice management server 105 identifies the item name corresponding to the invoice index item from the text PDF information 1100 acquired in S415. If the invoice file (PDF) has attribute information corresponding to the invoice index item in advance, the attribute information can be associated with the same invoice index item. If the text PDF does not have an attribute corresponding to the invoice index item, the same process as S413 is performed.

[0074] In S417, the invoice management server 105 acquires the item name associated in S416 and the item information associated with this item name, and registers them in the item linking information as item information associated with the item of the invoice index associated in S416. An example of the item linking information is shown in 1110 in the same Figure 10. The item linking information is similar to the data generated in S414, so a description thereof will be omitted.

[0075] Unlike scanned PDFs, text PDFs often contain table information, so the horizontal and vertical axes of the PDF table information can be linked by matching common item information with item names.

[0076] When the item name and item information are linked, the item information is linked to the item of the invoice index associated with the item name. As shown in 1110 in Fig. 11, the item information No. and the item information are newly registered as the item information linked to the item of the invoice (index).

[0077] This completes the explanation of the OCR process for the invoice file (PDF), and we will now return to the explanation of the flowchart in Figure 4(a).

[0078] When OCR processing of the invoice file (PDF) is completed in S404 of FIG. 4, in S405 the invoice management server 105 registers in the database server 106 the data (item linking information) corresponding to the invoice index acquired in S404.

[0079] This completes the description of the bill reading process.

[0080] Next, the flow of the invoice confirmation process will be described with reference to FIG.

[0081] Each step in FIG. 5 is a processing step executed by the CPU 201 of the bill recipient terminal 110, the bill management server 105, and the database server 106.

[0082] The flowchart in Figure 5 is a flowchart that is started when the recipient of the invoice checks (visually inspects) whether the data contained in the received invoice file (PDF) has been properly digitized by the invoice management server 105 (for example, whether numerical information has been set in the wrong invoice index item).

[0083] First, in S501, the recipient terminal 110 transmits an invoice display request to the invoice management server 105 in response to an instruction from the recipient user.

[0084] In S502, the invoice management server 105 transmits list display information of invoices that the invoice recipient is permitted to view to the recipient terminal 110. The transmitted screen image will be described with reference to FIG.

[0085] FIG. 6 is a screen image showing a list of invoice documents. A list of registered invoices is displayed in the document list field 605. Clicking on the document name in the document name column 604 downloads the invoice file (PDF) of that document name. The status column 606 displays the status of the invoice record on the invoice management server 105. Note that the screen images shown in the following Figures 6 to 8 are screens that can be confirmed by both the invoice issuer and the invoice recipient.

[0086] The status column 606 displays statuses such as "invoice not registered", "invoice registered", "waiting for OCR", "OCR in progress", "waiting for invoice input", "invoice input", "deficiency", "invoice not confirmed", and "invoice confirmed". The status transitions from "invoice not registered" to "invoice input" or "deficiency". The "invoice not registered" status indicates the state displayed when the invoice recipient requests the invoice issuer to issue an invoice, but the invoice issuer has not registered the invoice from the document details screen (explained in FIG. 7). The "invoice registered" status is displayed on the invoice recipient terminal 110 after the invoice issuer attaches an invoice from the document details screen (file attachment area 703 in FIG. 7) and presses the register button. At this stage, OCR, data checks, and eligibility checks are not performed. The "waiting for OCR" and "OCR in progress" statuses indicate the waiting and running states of OCR, respectively. The "waiting for invoice input" status indicates the state in which the result of OCR processing is reflected in the item 901 in FIG. 9, which corresponds to the item of the invoice index, as a result of the completion of OCR processing. When the invoice recipient clicks on the invoice list 603 with the status "Waiting for invoice input", a screen appears (Fig. 9) where the user can visually check whether the OCR results correspond to the items in the invoice index. If the user clicks on an invoice list with a status other than "Waiting for invoice input" (such as invoice input 602), the screen appears on a document details screen as shown in Fig. 7 or 8. The "Invoice Input" status appears after the invoice recipient has finished visually checking and manually inputting items in the invoice index that are not filled in with the OCR results, and the user presses the update button (not shown in Fig. 9). The "Defective" status appears when there is a defect in an item in the invoice index (such as a defect in the results of a qualified invoice eligibility check). The "Invoice not confirmed" status appears on the invoice issuer terminal 104 when the invoice recipient has not confirmed the invoice. The "Invoice confirmed" status appears on the invoice issuer terminal 104 when the status in the invoice recipient terminal 110 has transitioned to "Invoice Input". When the invoice issuer registers an invoice, the invoice is added by pressing the add document button 601 .Figure 7 shows the screen you are redirected to when you press the Add Document button.

[0087] In the document name field 701 in FIG. 7, the invoice issuer enters the name of the item in which the link to the invoice file will be displayed as the document name 604 in the document list in FIG. 6. By default, it will be the file name (without the extension) attached in the file attachment area 703. The customer ID field 702 is an item in which the ID assigned to the invoice issuer is selected and entered. If the invoice issuer is logged in, the issuer ID may be entered by default. The file attachment area 703 is an item in which the invoice is registered, for example by the invoice issuer dragging and dropping the invoice file (PDF). The index area 710 is an item for the invoice index, and is blank by default.

[0088] An example in which data and files are registered in the required fields of document name column 701, customer ID column 702, and file attachment area 703 in FIG. 7 will be described with reference to FIG.

[0089] In FIG. 8, "AAAA Project Invoice" is registered in the document name field 801, "IE001" in the client ID field 702, and the text and file "AAAA Project Invoice.pdf" are registered in the file attachment area 703. Immediately after registration, no value is reflected in the index portion 810. The index portion 810 is displayed when a transition to the document details screen is made by clicking when the status in the status column 606 is "Invoice entered" or later. Note that clicking when the status column 606 is "Waiting for invoice input" (603) transitions to the invoice input screen (FIG. 9). In other words, the flowchart in FIG. 5 shows the flow of processing when an invoice file (PDF) is registered, OCR processing is completed, and the status column in the document list is "Waiting for invoice input" status (603). Return to the explanation of the flowchart in FIG. 5 In S503, the invoice recipient terminal 110 displays the document list screen transmitted in S502, and accepts a click on the invoice status column of the invoice list to be confirmed by the invoice recipient.

[0090] In S504, the invoice management server 105 generates information for displaying the invoice input screen (FIG. 7) and transmits it to the invoice recipient terminal 110.

[0091] In S505, the bill recipient terminal 110 displays the information of the bill input screen transmitted in S504 on its screen. An example of the bill input screen will be described with reference to FIG.

[0092] 900 in FIG. 9 is an example of an invoice input screen. In 901, the items of the invoice index and their values ​​are displayed, and an invoice image is displayed in 910 (a form screen). The form image 910 is the same as 1000 in FIG. 10. In the invoice index item 901, item information linked to the item 710 in S404 is displayed. For example, the total amount including tax column 902 is obtained from item information "32,800 yen (including tax)" 912, whose item name is "Total" in the invoice image 910, and item information of the amount excluding tax of 10% and 8% 903 is obtained from a matrix of item information whose item names are "amount excluding tax", "10% applicable", and "8% reduced applicable" in the invoice image 910. In the enlarged view of FIG. 10, 912 corresponds to 1001, and 913 is a part of 1002. In this way, when the text of the item name and item information is small compared to the size of the invoice screen 910, it is not immediately clear which item information in the image is the result of OCR, and the time-consuming task of having to zoom in and out of various parts of the invoice image 910 to find it is required. The present invention aims to reduce the time and effort required for the invoice recipient to visually inspect the item information by enlarging and identifiably displaying the item information corresponding to the items in the invoice index based on the OCR position information. Note that the "yen (tax included)" notation in the item information "32,800 yen (tax included)" is attached information, but since only amount data can be entered in the total amount including tax column 902, the data "yen (tax included)" is not registered. Returning to the explanation of the flowchart in FIG. 5.

[0093] In S505, the invoice recipient terminal 110 accepts an operation to activate the item information column (902) by the invoice recipient clicking an item in the invoice index (for example, the item column "Total amount including tax" 902 of 900) with the mouse. In other words, an operation to change the item information column 902 to a changeable state is accepted.

[0094] In S506, when the invoice management server 105 receives the operation to activate the item information field received in S505, it changes the item information field 902 for which the operation was received to an input-enabled state.

[0095] In S507, the invoice management server 105 acquires position information on the invoice image corresponding to the activated item information from the database server 106. For example, when the item information column 902 accepts an operation, the data to be acquired is the item information number of the transaction amount column 1111 of the item linking information 1110, which is the item information corresponding to the item information column 902, and the value of the bounding box of the 21st row 1105, which is that item information number, is acquired. In the case of the transaction amount 1111, the (X coordinate, Y coordinate, width, height) acquired is information corresponding to a rectangular area of, for example, (213, 305, 166, 34).

[0096] In S508, the invoice management server 105 generates an image (changing the display position and enlarging the display position) such that the position information on the invoice image acquired in S507 is enlarged by a predetermined ratio. Specifically, the rectangular area acquired in S507 is enlarged, and an image is generated that displays an area in which the rectangular area is entirely included in the display area 910. Alternatively, an image may be generated in which the center point of the rectangular area coincides with the center point of the display area 910, and the rectangular area is enlarged so that it fits within the display area 910.

[0097] In S509, the invoice management server 105 generates information for identifiably displaying the position information on the invoice image acquired in S507, and transmits the information to the invoice recipient terminal 110. An example of identifiably displaying the position information will be described with reference to FIG.

[0098] In S510, the bill recipient terminal 110 performs processing to display the bill image generated in S509. An example of the bill image that is displayed will be described with reference to FIG.

[0099] 1200 in Fig. 12 is an example of an invoice input screen that is displayed when the invoice recipient activates item information. For example, if the invoice recipient clicks with the mouse on item information 1201 of the "Total amount including tax" item in S505, a cursor 1202 for entering a value is displayed in the item information 1201 of the "Total amount including tax" item, and the input item becomes active. Also, invoice image 1210 displays an enlarged view of the position that was read using OCR corresponding to the item information 1201 selected (clicked with the mouse) in S505. This is also an example in which the read position is displayed in an identifiable manner (by color-coding it with shading or displaying a red frame 1204, for example).

[0100] In S511, the bill recipient terminal 110 accepts, for example, a correction of the value of the item information by the user. For example, when the tax-excluded amount is reflected in the tax-inclusive amount, the bill recipient terminal 110 accepts an input to correct the item information of the tax-inclusive amount to the tax-excluded amount.

[0101] In S512, the invoice management server 105 reflects the value corrected in S511 as item information.

[0102] In S513, the bill recipient terminal 110 accepts pressing of the input completion button (1203) by the bill recipient.

[0103] In S514, the invoice management server 105 updates the value corrected in S512 as item information of the invoice index item by pressing the input completion button (1203) in S513.

[0104] Although the present embodiment has been described above, other embodiments will be described.

[0105] As another embodiment of the display image generation in S508, for example, as shown in FIG. 13, when a user selects the total amount including tax column 1302 with a mouse pointer or the like, an image may be generated which simultaneously displays an enlarged view of the item information corresponding to the total amount including tax 1302 ("32,800 yen (including tax)" display 1303) and the corresponding item name ("Total" display 1304).

[0106] As another embodiment of S509, the part on the image that is identifiably displayed (colored with a shading or a red frame) may be displayed so that both the item information and the item name are identifiably displayed at the same time, such as 1303 or 1304. Also, only the item information may be displayed so that it is identifiably displayed. By displaying these images, it is possible to confirm what the item name of the enlarged item information is (in the case of an amount, whether it is including tax or excluding tax) just by looking at the screen display, without the user having to change the screen display area.

[0107] As another embodiment of the display image generation in S508, if the enlargement ratio is below a predetermined level even when the item name and item information are enlarged to a size that allows them to be displayed simultaneously, the enlarged image may be generated by dividing the screen into two parts for the item name and item information. For example, the screen may be divided into the top and bottom of the display area, and a screen may be generated that displays the item name on the top and the item information on the bottom.

[0108] As another embodiment of S509, the area in which the position of the item information is identifiably displayed is displayed in an area wider than the area in which the optical characters are read. For example, as shown in 1402 in Fig. 14, the identifiable display area of ​​a screen 1410 that is identifiably displayed when the total amount including tax field 1402 is selected may be widened to an area 1412 that is larger than an area 1411 recognized by OCR. Widening the identifiable display area makes it easier for the user to find the item information that he or she wants to check.

[0109] As another form of identifiable display, when item information is enclosed in a rectangle as a single character string (for example, when it is displayed as one column in a table format such as the total column 1411 of 1410), the area displayed identifiably may be within the rectangle. By identifiably displaying within the rectangle, it is possible to make it easier for the user to find the item information that he or she wants to check.

[0110] Another embodiment will be described with reference to FIG. 15. In FIG. 15, 1510 shows the part of the invoice that the user wants to check by enlarging the invoice image display area 1510 with the enlarge button 1504 or the like and using the mouse or the like. For example, when the user wants to check whether the tax-excluded amount "20,000" with a tax rate of 10% in the invoice image display area 1510 is properly reflected in the invoice index item, the mouse pointer is moved to the position of the area 1502 where "20,000" is read, and an operation such as a left click of the mouse is accepted. When this mouse operation is accepted, the "20,000" display field 1503 is displayed in a distinctive manner (colored with a shading or a red frame, etc.) as the value of the "tax-excluded amount with a tax rate of 10%" of the invoice index item corresponding to "20,000". This process allows the user to check whether the written part of the invoice is properly reflected in the invoice index item (there is no part of the invoice notation that has not been read).

[0111] As described above, the embodiment of the present application makes it easy to understand the source of the data that corresponds to the OCR result. For example, when the invoice recipient selects the selected data (OCR result), the image of the corresponding area of ​​the scanned image can be enlarged and confirmed.

[0112] In addition to the area of ​​the read text, the corresponding items are also enlarged and displayed for easy identification, which has the effect of reducing the work required of the bill recipient when visually inspecting the bill.

[0113] In other words, when a reviewer (such as an invoice recipient) checks whether the content entered as a result of OCR on a form, etc., is accurate, they can easily understand which item in the form image the text from the OCR results referred to when entering the text.

[0114] Although the embodiment of the present invention has been described above, the present invention can be embodied, for example, as a system, an apparatus, a method, a program, a recording medium, etc. Specifically, the present invention may be applied to a system composed of multiple devices, or may be applied to an apparatus composed of a single device.

[0115] The program of the present invention is a program that enables a computer to execute the processing method of the flowchart shown in Fig. 4 or 5, and the storage medium of the present invention stores a program that enables a computer to execute the processing method of Fig. 4 to 5. The program of the present invention may be a program for each processing method of each device in Fig. 4 or 5.

[0116] As described above, it goes without saying that the object of the present invention can be achieved by supplying a recording medium on which a program that realizes the functions of the above-mentioned embodiments is recorded to a system or device, and having the computer (or CPU or MPU) of that system or device read and execute the program stored on the recording medium.

[0117] In this case, the program itself read out from the recording medium will realize the novel functions of the present invention, and the recording medium on which the program is recorded will constitute the present invention.

[0118] Examples of recording media for supplying the program include flexible disks, hard disks, optical disks, magneto-optical disks, CD-ROMs, CD-Rs, DVD-ROMs, magnetic tapes, non-volatile memory cards, ROMs, EEPROMs, silicon disks, etc.

[0119] Furthermore, it goes without saying that the functions of the above-mentioned embodiments are not only realized by the computer executing a program it has read, but also includes cases where an OS (operating system) running on the computer performs some or all of the actual processing based on the instructions of the program, and the functions of the above-mentioned embodiments are realized through that processing.

[0120] Furthermore, it goes without saying that this also includes cases where a program read from a recording medium is written into a memory provided on a function expansion board inserted into a computer or a function expansion unit connected to a computer, and then a CPU or the like provided on the function expansion board or function expansion unit performs some or all of the actual processing based on the instructions of the program code, thereby realizing the functions of the above-mentioned embodiments.

[0121] The present invention may be applied to a system consisting of multiple devices, or to an apparatus consisting of a single device. Needless to say, the present invention can also be applied to a case where the object is achieved by supplying a program to a system or apparatus. In this case, the effect of the present invention can be enjoyed by the system or apparatus by reading a recording medium storing a program for achieving the present invention into the system or apparatus.

[0122] Furthermore, by downloading and reading a program for achieving the present invention from a server, database, etc. on a network using a communication program, the system or device can enjoy the effects of the present invention. Note that the present invention also includes configurations that combine the above-mentioned embodiments and their modified examples. [Explanation of symbols]

[0123] 100 Information Processing Systems 102 Scanner device 104 Bill Issuer Terminal 105 Batch server and Web server 106 Database Server 108 MFP 110 Bill Recipient Terminal 112 Internet line 114 Cloud Environment

Claims

1. An information processing system having a storage unit that stores the positions of a series of characters in an acquired image, a first display means for linking the series of characters as values ​​of each item of form data related to the acquired image and displaying the series of characters as values ​​in each item on a display screen of the form data; a second display means for displaying the acquired image; An information processing system characterized in that, in response to a user's access to an item of report data on the screen displayed by the first display means, the second display means retrieves the position of the series of characters of the item that has received the access from the memory unit, and enlarges and displays the position of the series of characters.

2. 2. The information processing system according to claim 1, wherein the second display means displays the positions of the series of characters registered in the storage unit in an identifiable manner.

3. 2. The information processing system according to claim 1, wherein the positions of the series of characters stored in the memory unit are positions of the series of characters obtained by optical character reading.

4. The series of characters corresponds to the name of an item in the acquired image; 2. The information processing system according to claim 1, wherein the second display means displays the associated item name and the series of characters in an enlarged size so that the associated item name and the series of characters can be displayed simultaneously.

5. 5. The information processing system according to claim 4, wherein the second display means displays the item name associated with the series of characters in an identifiable manner.

6. An information processing system as described in claim 4, wherein when a size at which the series of characters and the item name associated with the series of characters can be simultaneously displayed is equal to or smaller than a predetermined magnification ratio, the series of characters and the item name associated with the series of characters are displayed in two parts.

7. 2. The information processing system according to claim 1, wherein the first display means displays a screen capable of accepting input from a user.

8. 2. The information processing system according to claim 1, wherein the series of characters includes information associated with each item.

9. 3. The information processing system according to claim 2, wherein the second display means displays an area enlarged from the position of the series of characters in a identifiable manner.

10. 3. The information processing system according to claim 2, wherein when the series of characters is enclosed in a rectangle as a unique character string, the second display means defines the area in which the identifiable display is performed within the rectangle.

11. An information processing system having a storage unit that stores the positions of a series of characters in an acquired image, a first display means for linking the series of characters as values ​​of each item of form data related to the acquired image and displaying the series of characters as values ​​in each item on a display screen of the form data; a second display means for displaying the acquired image; An information processing system characterized in that, in response to a user's access to the position of a series of characters in an image displayed by the second display means, the first display means identifiably displays an item of report data related to the series of characters at the position where the access is accepted.

12. A method for controlling an information processing system having a storage unit that stores positions of a series of characters in an acquired image, comprising the steps of: a first display step of associating the series of characters as values ​​of each item of form data related to the acquired image and displaying the series of characters as values ​​in each item on a display screen of the form data; and a second display step of displaying the acquired image. A control method for an information processing system, characterized in that in response to a user's access to an item of report data on the screen displayed by the first display step, the second display step obtains from the memory unit the position of the series of characters of the item that has received the access, and enlarges and displays the position of the series of characters.

13. A method for controlling an information processing system having a storage unit that stores positions of a series of characters in an acquired image, comprising the steps of: a first display step of associating the series of characters as values ​​of each item of form data related to the acquired image and displaying the series of characters as values ​​in each item on a display screen of the form data; and a second display step of displaying the acquired image. A control method for an information processing system, characterized in that, in response to a user's access to the position of a series of characters in the image displayed by the second display step, the first display step identifiably displays an item of report data related to the series of characters at the position where the access is accepted.

14. 12. A program for causing at least one computer to function as each of the means of the information processing system according to claim 1 or 11.

Images