Information processing device, control method for information processing device, and program

Abstract

If the business operator's number cannot be recognized from the image data, the system aims to display a warning based on the fact that a number corresponding to the name of the business operator recognized from the image data is registered in the database. [Solution] If a string of characters determined to correspond to the business operator that issued the document is not recognized from the image data obtained by scanning the document, the system controls the display of a warning based on the fact that a number corresponding to the string of characters determined to correspond to the name of the business operator that issued the document, which was recognized from the image data, is registered in the database.

Description

Technical Field

[0001] The present invention relates to a user interface for displaying a character string recognized from image data.

Background Art

[0002] There is a technology for performing optical character recognition processing (OCR processing) on image data obtained by scanning documents such as claim forms to recognize character strings. In Patent Document 1, a technology is disclosed for acquiring registrant information based on a business operator number recognized from voucher data in which transaction documents are digitized, and determining whether the voucher data is valid based on the registration date included in the registrant information.

[0003] By the way, in recent years, with the implementation of the invoice system, business operators on the receiving side of invoices need to receive eligible invoices on which the registration number of the business operator on the issuing side is described in order to apply input tax credit for consumption tax. In addition, when the business operator on the issuing side of the invoice is not a business operator registered as an eligible invoice issuer, an eligible invoice cannot be issued.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0005] In the technology described in Patent Document 1, the lack of a document in which the registration number is not described in the invoice, even though the business operator on the issuing side of the invoice is a business operator registered as an eligible invoice issuer, is not considered. In the technology described in Patent Document 1, there was a possibility that the user might overlook this deficiency.

Means for Solving the Problems

[0006] To achieve the above objective, the information processing device of the present invention has a display control means that controls the display of a warning if, when a string of characters determined to correspond to a number corresponding to a business operator that issued a document is not recognized from the image data obtained by scanning a document, the number corresponding to a string of characters determined to correspond to the name of the business operator that issued the document, which was recognized from the image data, is registered in the database. [Effects of the Invention]

[0007] According to the information processing device of the present invention, if the business operator's number is not recognized from the image data, a mechanism is provided to display a warning based on the fact that a number corresponding to the string of the business operator's name recognized from the image data is registered in the database. [Brief explanation of the drawing]

[0008] [Figure 1] This is a diagram showing an example of the configuration of an information processing system. [Figure 2] This is a block diagram showing the hardware configuration of an information processing device. [Figure 3] This is a diagram showing the software configuration of an information processing system. [Figure 4] (a) is a diagram showing an example of a billing processing screen. (b) is a diagram illustrating the relationship between a scanned image and the OCR area. [Figure 5] This diagram shows a database that stores business information defined for each business in a table format. [Figure 6] (a) is a diagram showing the state of the billing processing screen where the business number could be extracted. (b) is a diagram showing the state where extraction was not possible. [Figure 7] This is a flowchart showing the processing flow of an information processing system. [Figure 8] This diagram shows a message dialog for setting a search key in case the business number could not be extracted. [Figure 9] This figure shows an example of a billing process screen for obtaining a search key. [Figure 10] This diagram shows the search results that match the search key. [Figure 11] This figure shows the state after the selected business information has been reflected in the search results. [Figure 12] This diagram shows the state in which a warning is displayed when a matching candidate is found. [Figure 13] This diagram illustrates how to launch another application when a matching candidate is found. [Figure 14] This figure shows an example of launching another application. [Figure 15] This is a flowchart showing the processing flow of an information processing system. [Modes for carrying out the invention]

[0009] The following descriptions of embodiments for carrying out the present invention will be made with reference to the drawings. Note that the following embodiments are not intended to limit the invention as defined in the claims, and not all combinations of features described in the embodiments are necessarily essential to the solution of the invention.

[0010] <First Embodiment> <System Configuration> Figure 1 shows the overall configuration of the information processing system according to this embodiment. The information processing system is connected to an MFP 102, client terminal 103, document management server 104, billing record server 105, number management server 106, and external services 107 via a network 101 such as the Internet or an intranet.

[0011] The MFP (Multi-Function Peripheral) 102 is an information processing device with multiple functions, such as a scanner and printer, and is used by users to scan documents such as forms and convert them into image data. In the following explanation, scanned images obtained by scanning documents to be processed, such as invoices, will be referred to as "scanned documents" or "image data."

[0012] The client terminal 103 is an information processing device used by each user, such as a personal computer, laptop computer, tablet computer, or smartphone.

[0013] The document management server 104 is an information processing device having various document processing functions, such as receiving a scanned document from the MFP 102, filing and storing it, or linking the scanned document and the billing information and transferring it to the billing record server 105 or the like. Note that the document management server 104 may be a cloud server.

[0014] The billing record server 105 is an information processing device having a function of recording and managing the document file and the billing information transferred from the document management server. Note that the billing record server 105 may be a cloud server.

[0015] The number management server 106 is an information processing device having a function of managing the business number, business name, address, registration date, and registration end date associated with the business operator of the billing source when the document management server 104 creates the billing information. Note that the number management server 106 may be a cloud server.

[0016] The external service 107 indicates an external service that cooperates with the client terminal 103 and the document management server 104, which are activated when a defective document in the present invention is detected. As an example, there is an email service, which will be described later. The external service 107 may be a cloud service.

[0017] The configuration of the information processing system shown in FIG. 1 is an example and is not limited thereto. For example, any one of the servers may have the functions of the document management server 104, the billing record server �5, and the number management server 106. Further, the MFP 102 may have the functions of the client terminal 103, and the client terminal 103 may be omitted.

[0018] <Hardware Configuration> Figure 2 is a block diagram showing the hardware configuration of the information processing device, which includes the MFP 102, client terminal 103, document management server 104, billing record server 105, number management server 106, and external service 107. The CPU 202 is the arithmetic unit that controls the entire device and executes programs read from RAM 203, ROM 204, secondary storage device 205, etc. RAM 203 temporarily stores various data and also functions as the work memory for the CPU 202. ROM 204 records built-in programs and various data. The secondary storage device 205 is a large-capacity storage device such as an HDD or flash memory. The network interface 206 connects to the network 101 and communicates with other information processing devices and network devices not shown. The communication method can be either wired or wireless. The user interface (UI) 207 is implemented by operating devices 208 such as a display, keyboard, mouse, buttons, and touch panel, and performs input and output of information and signals. Computers lacking this hardware can be connected to and operated from other computers via remote desktop or remote shell. The scanner I / F 209 and printer I / F 211 are interfaces of the MFP 102, controlling input and output to and from the scanner 210 and printer 212, respectively. The scanner I / F 209, scanner 210, printer I / F 211, and printer 212 are configurations included only in the MFP 102.

[0019] Each of the above components is interconnected via the input / output interface 201.

[0020] <Software Configuration> Figure 3 shows an example of the software configuration of the information processing system according to this embodiment. The respective CPUs 208 execute the software installed on the MFP 102, client terminal 103, document management server 104, billing record server 105, number management server 106, and external service 107, thereby realizing the predetermined functions.

[0021] ≪MFP Software Configuration≫ The basic function unit 3210 consists of functions that are standard on the MFP 102. The MFP application 3220 is a Java®-based application that allows for easy addition of functions to the MFP terminal 102 while utilizing the functions of the basic function unit.

[0022] The scan execution unit 3211 performs a scan according to the received scan settings. The image data processing unit 3212 generates document files in predetermined formats such as JPEG and PDF from the scanned image data. The display control unit 3213 is the user interface and enables editing of various settings applied to the scanned image and issuing instructions to execute scans. The external system communication unit 3214 sends approval requests to connect to the document management server 104, sends document processing requests that include setting information applied to the scanned image as parameters, and analyzes the responses to these requests.

[0023] The document processing application 3221 displays a scan screen and performs a scan according to the scan settings specified by the user. It links the scanned document, scan settings, and user information obtained through the operation of the login application 3222 (described later) and sends them to the document management server 104. The login application 3222 links the authentication information of both the MFP 102 and the document management server 104 and performs login using any means such as ID, username, password, PIN code, or card touch. Upon successful login, it becomes possible to use the functions of the MFP 102 and obtain information such as user information, application settings, and scan settings managed by the document management server 104.

[0024] ≪Software configuration of client terminals≫ The client application 3310 runs the web application of the document management application 3410, which will be described later. One way the client application 3310 is provided is by displaying a Web UI in a browser. In this method, the web application is executed by sending and receiving necessary data with the API (Application Programming Interface) provided by the document management server. Alternatively, it may be a device application, such as a smartphone application, configured to send and receive necessary data with the API.

[0025] The 3rd application 3320 is an application that is launched when a defective document is detected in the present invention, as described later. One way of providing it is as a device application configured to send and receive necessary data via API in cooperation with an external application 3710 such as a mail server, which is a mail client application. If it is provided via a browser, similar to the client application 3310, the 3rd application 3220 can be omitted as the application consists only of a browser.

[0026] ≪Software configuration of the document management server≫ In this embodiment, the document management application 3410 is described as a web application, but it may be implemented in other application forms.

[0027] The document management application 3410 consists of a form processing unit 3411, a billing registration unit 3412, a number acquisition unit 3413, and an external system communication unit 3414. The form processing unit 3411 controls jobs in the data store 3420 (described later) and performs scheduled image processing (document processing) on ​​scanned images. Here, document processing includes optical character recognition (OCR) and conversion processing for file output on scanned images acquired from the scanned document storage unit 342, as well as similar form discrimination using pattern recognition and machine learning, and named entity recognition. The billing registration unit 3412 transmits the scanned document and billing information set by the user via the WebUI 3432 (described later) to the billing record server 105. The number acquisition unit 3413 acquires the business number or business information from the number management server. The external system communication unit 3414 performs processing related to sending and receiving, such as receiving various information requests, sending data processing requests and analyzing the responses.

[0028] Datastore 3420 is a storage area used by the entire document management server 104. Datastore 3420 consists of a job information management unit 3421, a scanned document management unit 3422, a billing information management unit 3423, and a configuration information management unit 3424. The job information management unit 3421 stores scan settings and user information received from the MFP 102 on an execution basis. The scanned document management unit 3422 stores scanned documents before and after processing by the form processing unit 3411. The billing information management unit 3423 stores billing information to be sent to the billing record server 105 (described later) and information received from the number management server 106. The configuration information management unit 3424 manages the settings for the entire document processing server, including authentication information necessary for cooperation with the billing record server 105 and the number management server 106, as well as other scan settings.

[0029] API3431 is an application programming interface provided by the display control unit 3430. WebUI3432 is a user interface provided by the display control unit 3430 for configuring applications and servers from a web browser.

[0030] ≪Software configuration of the billing record server≫ In this embodiment, the billing record application 3510 is described as a web application, but other application implementation forms are also possible.

[0031] The billing record application 3510 consists of a billing data processing unit 3511 and an external system communication unit 3512. The billing data processing unit 3511 records and manages billing data in the billing data management unit 3522 of the data store 3520 (described later) based on scanned documents and billing information sent from the document management server 104. The external system communication unit 3512 performs processes related to sending and receiving, such as receiving various information requests, sending data processing requests and analyzing the responses.

[0032] Datastore 3520 is a storage area used by the entire billing record server 105. Datastore 3520 has a billing data management unit 3521 that records scanned documents and billing information sent from the document management server.

[0033] ≪Software configuration of the number management server≫ In this embodiment, the number management application 3610 is described as a web application, but it may be implemented in other application forms.

[0034] The number management application 3610 consists of a business information identification unit 3611 and an external system communication unit 3612. The business information identification unit 3611 identifies business information such as business name and address based on the business number transmitted from the document management server 104, or identifies the business number based on the business information. The external system communication unit 3612 performs processes related to transmission and reception, such as receiving various information requests that are transmitted, sending data processing requests and analyzing the responses.

[0035] Datastore 3620 is a storage area used by the entire number management server 106. Datastore 3620 has a business information management unit 3621 that manages information such as business number, business name, and address for each business.

[0036] ≪Software configuration of external services≫ In this embodiment, we describe it as an arbitrary web application, but other application implementation forms may also be used.

[0037] External service 107 can be any service; external application 3710 consists of a data processing unit 3711 and an external system communication unit 3712; and data store 3720 consists of a data management unit 3721 that manages the data used by the external service.

[0038] The data processing unit 3711 performs processing based on requests sent according to the service. It then retrieves or stores data from the data management unit 3721. The external system communication unit 3712 performs processes related to transmission and reception, such as receiving various information requests, sending data processing requests, and analyzing the responses.

[0039] The above describes the software configuration of the information processing system according to this embodiment. For the sake of simplicity, each configuration is described as an application and data store that aggregate functions and management information, but these may be provided separately as multiple components. In this embodiment, one or more business servers 105, 106, and 107 are provided that cooperate with the document management processing server 104, but for example, the document management server 104 may also have the functions of the billing record server 105 and the number management server 106. Alternatively, the configuration may be such that the WebUI 3432 is displayed on the MFP application 3220.

[0040] <Client application UI screen> Figure 4(a) shows an example of a screen displaying an edit preview of scanned documents and billing information in the client application 3310. It consists of two panes: a preview pane 401 and a properties pane 402.

[0041] The preview pane 401 displays the scanned document in the preview area 411, and allows users to view and edit the entire scanned document by performing preview operations such as zooming in and out, turning pages forward and backward, and inserting and deleting pages using the preview operation buttons 412.

[0042] The properties pane 402 displays fields for setting billing information. Typical fields shown include "Business Name," "Address," "Business Number," "Invoice Item," "Invoice Date," "Amount," and "Tax Rate," but other fields may also be present.

[0043] Each item is represented by a pair of item name 422 and input field 423. Input field 423 is primarily a text field, but any input method is acceptable depending on the item being configured, such as calendar input, numerical input, dropdown menus, sliders, or checkboxes. In addition, each setting item can be configured using methods other than inputting into the input field, such as a default value, a string that satisfies a regular expression from all strings in the scanned document obtained by OCR, or a string obtained from a specified OCR area, by linking the setting method to the setting item.

[0044] After confirming and entering information in each pane, the user completes the operation by pressing the save button 403 to temporarily save the data and the register button 404 to register it with the billing record server 105. When the client application 3310 detects the user's instruction to press the register button 404, the image data and the string recognized from the image data displayed in the input field in the properties pane 402 are registered with the billing record server 3510.

[0045] Figure 4(b) illustrates a Cartesian coordinate system showing the relationship between the scanned image displayed on the preview area 411 and each text block. OCR places text blocks (OCR blocks) on the preview area so as to overlap each text in the scanned image, and each text block is identified by its starting coordinates, width, and height. For example, text block 405 is a text block corresponding to the string "Invoice", and is expressed in pixels as "starting coordinates (1200, 700)", "width: 720", and "height: 120".

[0046] Figure 5 shows an example of table data for business information managed by the business information management unit 3621 of the number management server 106. 501 shows the table data, where business name, business number, address, registration date, update date, and registration end date are linked and managed. These items are merely examples; for example, an identifier indicating the type of expense (expense code), billing category, and the name of the person in charge for each business may be added as necessary for billing processing.

[0047] <Display of business information in client applications> Figures 6(a) and (b) show examples of how the business operator number can be displayed in the client application 3310 when it can be extracted from the image data displayed in the preview area 411, and when it cannot be extracted.

[0048] 601 indicates the location of the text block containing the business operator number in the document. A regular expression (or OCR area) is pre-configured to correspond to the "Business Operator Number" displayed in the Properties pane. Based on the configured information, the string corresponding to the business operator number is extracted from the document and set in input field 602.

[0049] 603 is a display field. The document management server 104 requests the number management server 106 for the operator information that matches the operator number input in the input field 602. The document management server 104 analyzes the response result and reflects it in the display field 603.

[0050] When the operator information that matches the input field 602 is obtained, the display field 603 displays the name of the corresponding operator, that is, the operator information associated with the operator number, such as the operator name. The items to be displayed from the operator information are arbitrary, but it is desirable to display at least the operator name. If there is no match, nothing is displayed in the display field 603 or a display indicating that the matching operator information could not be obtained, such as 604, is performed.

[0051] 605 indicates that there is no text block indicating the operator number.

[0052] 606 is a display indicating that the operator number could not be extracted from the scanned document in the setting of the regular expression (or OCR area) associated with the setting item "operator number".

[0053] <UI control process> Figure 7 is a flowchart showing the processing flow in the information processing system according to this embodiment, centered on the display control of the UI screen of the client application shown in Figures 4 and 6. In this embodiment, it is assumed that settings such as regular expressions and OCR areas associated with each of the above-described setting items are pre-set. Figure 7(a) shows the processing of the MFP 102, Figure 7(b) shows the processing of the document management server 104, and Figure 7(c) shows the processing of the client terminal 103 and the document management server 104.

[0054] In the following description, the symbol "S" means step.

[0055] First, let's explain Figure 7(a). In S701, the user logs in to the MFP102. Specifically, the login application 3222 displays a login screen (not shown) via the display control unit 3213, prompts the user to enter information (such as user ID and password) to log in to the document management server 104, and performs the login process. In S702, the document processing application 3221 displays a scan screen (not shown) via the display control unit 3213, the user makes their desired scan settings, places documents such as invoices on the document glass, and performs the scan. In S711, the scanned image data is sent to the document management server 104.

[0056] Next, Figure 7(b) will be explained. In S712, the document management server 104 receives image data from the MFP 102. In S703, the form processing unit 3411 processes the received image data. Specifically, based on the scan settings, image data, and user information sent in S702, it performs form processing such as OCR on the image data and saves the job information to the job information management unit 3421 and the scanned document management unit 3422 of the data store 3420.

[0057] Next, Figure 7(c) will be explained. On the client terminal 103, it is possible to select a document for which document processing has been completed and display a preview screen. In S704, the client application 3310 on the client terminal 103 displays the aforementioned preview screen. On this preview screen, the OCR area linked to the image data obtained in S703 is displayed on the preview area 411. In addition, information on various setting items for generating billing data is displayed in the setting information properties pane 402.

[0058] In S705, the next processing step is determined based on whether the business operator number is recognized within the image data, according to the pre-configured regular expressions and OCR area settings.

[0059] If the business operator number is recognized from the image data, the process proceeds to S706, and the client application 3310 displays the recognized business operator number in the "Business Operator Number" input field of the properties pane 402.

[0060] On the other hand, if no business number exists, the process proceeds to S710, and client application 3310 displays a warning and terminates. The displayed error code at this time is 606.

[0061] Once the business number is set in S706, the process proceeds to S707. The document management server 104 processes a request to check if matching business information exists in the number management server 106 based on the extracted business number.

[0062] In S708, the next process is determined based on whether matching carrier information exists in the number management server 106.

[0063] If matching business information is found, the process proceeds to S709, and the client application 3310 displays the retrieved business information, such as the company name, i.e., the name of the business, on the settings page. This display is 603.

[0064] On the other hand, if no matching business information is found, the process proceeds to S710, and the client application 3310 displays a warning and terminates the flow. The error message displayed at this time is 604.

[0065] <UI for setting up reverse reference of business operator number> Next, the UI, which is a feature of this embodiment, will be explained in detail using Figures 8, 9, 10, 11, 12 and flowchart Figure 13. This is a setting UI that checks whether the business operator that issued the image data is registered in the number management server 106 when the business operator number could not be extracted from the scanned image data.

[0066] Figure 8 shows an example of the UI for performing a business search, as described later, when a business identification number cannot be found in the image data.

[0067] Dialog 801 is displayed when the business number cannot be detected from the scanned document. The dialog contains a message indicating that the business number could not be detected from the scanned document and a button 811 to start the search. The format of the display does not necessarily have to be a dialog; any display method such as a banner or message is acceptable. This dialog is displayed when the regular expression string pre-configured to correspond to the business number setting item is not recognized from the image data, or when the string is not recognized from the OCR area pre-configured to correspond to the business number setting item. The timing of the dialog is assumed to be during the preview display, but other timings are also acceptable. For example, it can be displayed when the registration button 404 is pressed after completing the input of billing information, or a search button can be provided to allow the search to be started at any time.

[0068] Figure 9 shows the display switched after pressing the search start button 811. When the search start button 811 is pressed, the display switches to one in which an arbitrary string is selected from the OCR block in the preview area 411, and the presence or absence of a business number is queried from the number management server 106 using that string as the search key. 901 is the OCR block selected by the user, and this will be explained as the search key. Multiple search keys are possible. Alternatively, values ​​from setting items can be used instead of the OCR block. The property pane 402 switches to display the search results or during the search. In this state, the property pane 402 is referred to as the search pane 910. 911 is a button to end the search and return to the input screen in the state shown in Figure 4. 912 and 913 are the input fields and their labels used as search keys, and the strings obtained from the OCR block in 901 are set. 914 is the search filter, where the search conditions are set. From the DB in Figure 5, the search target items are filtered from multiple items such as business number, business name, address, registration date, etc. After completing the settings for the aforementioned search key 913 and filter 914, pressing the search button 915 will execute a search against the number management server 106.

[0069] Figure 10 shows the display of the search pane 910 when search results are found. If no search results are found, a warning (not shown) is displayed, prompting the user to perform the search again. 1001 is the search results, which shows a list of results based on the search key and filter. Assuming that multiple candidates are found from the search criteria, the user is allowed to select from the list of search results. 1011 shows the state in which the user has made a selection.

[0070] Figure 11 shows the state after the business information selected by the user in Figure 10 has been reflected. 1101 reflects the selected business number, and together with it, the business information is reflected in 1102. 1103 is a search button to perform another search; pressing it transitions to the state shown in Figure 9.

[0071] Up to this point, we have explained how to perform a search using a key within the document if the business number cannot be found. However, if the business information is registered in the number management server 106, but the business number is not included in the scanned document, the document is unsuitable as an invoice and should not be registered in the billing record server as is. Furthermore, it will be necessary to request the business that issued the invoice to reissue it. The display in this case will be explained using Figure 12.

[0072] Figure 12 shows an example of a warning display in the preview view of client application 3310.

[0073] If, during the operation of the reverse reference setting UI for the business number, the business number is not present in the scanned document, but the corresponding business information exists on the number management server 106, a warning message will be displayed, as shown in 1201, indicating that the document has deficiencies regarding the business number. Alternatively, a message may be displayed indicating that the business number could not be recognized from the image data, even though the invoice is a document that should contain a business number. Furthermore, if a warning is displayed as described above, it means that the invoice data is deficient and requires replacement, so the registration button 404 will be displayed in a disabled state as shown in Figure 1202, and display control will be implemented to prevent registration.

[0074] In Figure 12, a warning is displayed to prevent registration, but instead, the following display control may be used. When operations such as re-requesting are required, various patterns are possible, such as needing to re-request the document via email to the requesting party or registering it in a different system. Therefore, as shown in Figures 13 and 14, when a document requiring re-requesting is identified, a different application may be launched.

[0075] Figure 13 illustrates an example of a UI that, in the case of the aforementioned defective scanned document, presents the launch of a different application than the client application 3310 and the document management server 104.

[0076] Dialog display 1301 is displayed when, during the operation of the reverse reference setting UI for the business operator number, the business operator number is not present in the scanned document, but the corresponding business operator information exists in the number management server 106. The dialog displays a message indicating that, despite not being able to detect the business operator number from the scanned document, matching business operator information was obtained from the number management server 106, and further presents a button 1302 to launch the application that should be launched in this case.

[0077] Figures 14(a) and 14(b) show examples of 3rd applications that are launched when the launch button 1302 is pressed.

[0078] Figure 14(a) is a diagram showing an example of the display when the 3rd application 3320 is assumed to be a mail client. In this case, the external service 107 that cooperates with the 3rd application 3320 is a mail server.

[0079] The start button 1302 is an item associated with a program or command file that creates a mail file for re-billing the claim upon startup. When the client application 3310 detects a user operation on the start button 1302, the 3rd application 3320 creates a mail file (such as eml or msg) and starts on the client terminal 103. 1401, 1402, and 1403 are the destination, subject, and body respectively. These pieces of information may be generated by reflecting the information input into the setting items set in the property pane 402 or the information obtained from the number management server 106.

[0080] Figure 14(b) is the 3rd application 3320 for displaying and registering a list of documents that require re-billing when assumed to be executed in an application or system with different claim request processing. When the start button 1302 is pressed, the scanned document, the claim information set in the property pane 402, and the business operator information obtained from the number management server 106 are passed as parameters and started on the client terminal 103. When it starts, the target claim information is registered as shown in 1404 in the list of documents that require re-billing as shown in Figure 14(b). The user in charge of the claim request checks the content and, if there are no problems, presses the registration button 1405 to execute the re-billing process. When the registration button is pressed, the image data and the character string recognized from the image data are registered as documents that require re-billing in a different server from the claim record server 105.

[0081] <UI Control Processing> Figure 15 is a flowchart showing the processing flow in the information processing system according to this embodiment, focusing on the display control of the UI screen of the client application 3310 shown in Figures 8, 9, 10, 11, and 12. In this embodiment, it is assumed that settings such as regular expressions and OCR areas associated with each of the aforementioned setting items are set in advance. Furthermore, this is the processing after the determination step S705 shown in Figure 7 where the business operator number could not be detected from the document (No), and the explanation of the steps up to that point is omitted.

[0082] First, if the business number cannot be extracted from the document in the S705 determination step, the process proceeds to S1501, and the client application 3310 displays the warning dialog shown in 801 on the property settings screen.

[0083] In S1502, the client application 3310 displays a search screen, as shown in Figure 9, where the user can set search keys and filters, in response to the user's selection to initiate a search. On the search screen shown in Figure 9, the user selects the name of the business that issued the invoice from the displayed image data. The client application 3310 accepts the user's selection and sets the string recognized from the image data through character recognition processing of the selected area as the search key. In this flowchart, the user specifies the search key, but this is not the only way. For example, the document management application 3410 may analyze the image data to identify a string that is determined to be the business name from the strings recognized from the image data, and automatically set the identified string as the search key. In that case, the display of the search screen shown in Figures 9 and 10 may be omitted. That is, S1502, S1505, and S1506 may be omitted, and the processing may be performed automatically.

[0084] In S1503, based on the search conditions set in S1502, the search button 915 triggers a request from the document management server 104 to the number management server 106 for business information associated with the search conditions, and the number management server 106 responds with the search results for business information. Specifically, the document management server 104 queries the number management server 106 to see if the business number corresponding to the business name set as the search key is registered in the number management server 106. The number management server 106 then responds to the document management server 104 with the result of whether or not it is registered.

[0085] S1504 is then routed to the next step depending on whether search results for business information exist.

[0086] If business information exists, that is, if a business number corresponding to the business name set as the search key is registered, the process proceeds to S1505, and the client application 3310 displays a list of search results on the search pane 910. On the other hand, if business information does not exist, the process proceeds to S1509, displays the search results indicating that the document is not a defective document requiring resubmission, and terminates this flow. Alternatively, the process may return to S1503 and perform the search operation again. Although not illustrated in detail, a message indicating that neither the scanned document nor the number management server 106 could obtain the business number and business information may be displayed by any means, such as 1201 or a dialog box.

[0087] In S1506, the client application 3310 accepts the user's selection of relevant business information from the search results. This results in the state shown in Figure 10. After selection, the user presses the save button 1002 to set the business information.

[0088] In S1507, the business number and business information are reflected in the corresponding settings. This results in the state shown in Figure 11.

[0089] Finally, if the scanned document in S1508 does not contain a business number, but the corresponding business information exists in the number management server 106, the client application 3310 displays a message indicating that the document is incomplete. This is the state shown in Figure 12.

[0090] In S1508, the client application 3310 may display a warning dialog box as shown in Figure 13, 1301, instead of displaying the warning message as shown in Figure 12, 1201. As mentioned above, this dialog box may display, for example, an item for launching an email application. When user action on the item is detected, the application may be launched and an email containing a message requesting the reissue of the document may be displayed.

[0091] <Other Embodiments> The present invention can also be realized by supplying a program that implements one or more of the functions of each of the embodiments described above to a system or device via a network or storage medium, and by having one or more processors in the computer of that system or device read and execute the program. It can also be realized by a circuit (e.g., an ASIC or FPGA) that implements one or more functions. [Explanation of Symbols]

[0092] 103 Client terminals 104 Document Management Server 105 Billing Record Server

Claims

1. If a string of characters determined to correspond to the business operator that issued the document is not recognized from the image data obtained by scanning the document, a display control means controls the system to display a warning based on the fact that a number corresponding to the string of characters determined to correspond to the name of the business operator that issued the document, which was recognized from the image data, is registered in the database. An information processing device having

2. The display control means, when it recognizes a string of characters from the image data that is determined to be a number corresponding to the business operator that issued the document, controls the display to show, in addition to the recognized string of characters, the name of the business operator registered in the database that is associated with the recognized string of characters. The information processing apparatus according to feature 1.

3. The aforementioned warning indicates that the document is incomplete with respect to the aforementioned number. The information processing apparatus according to feature 1.

4. The aforementioned warning indicates that, although the document is a document in which the aforementioned number should be included, the string of characters determined to be the aforementioned number was not recognized from the image data. The information processing apparatus according to feature 1.

5. The aforementioned image data and the string recognized from the aforementioned image data are registered on an external server based on user instructions. The information processing apparatus according to feature 1.

6. The display control means controls the display so that, when the warning is displayed, the image data and the string recognized from the image data are not registered with the external server. The information processing apparatus according to feature 5.

7. If the aforementioned warning is displayed, the image data and the string recognized from the image data will be registered on a server other than the external server. The information processing apparatus according to feature 5.

8. When the warning is displayed, the display control means displays, along with the warning, an item for sending a message to the business that issued the document. The information processing apparatus according to feature 1.

9. The aforementioned item is an item for launching the application that sends emails. When user interaction with the aforementioned item is detected, the application is launched and an email containing a message requesting the reissue of the document is displayed. The information processing apparatus according to feature 8.

10. A method for controlling an information processing device, If a string of characters determined to correspond to the business operator that issued the document is not recognized from the image data obtained by scanning the document, a display control process is performed to display a warning based on the fact that a number corresponding to the string of characters determined to correspond to the name of the business operator that issued the document, which was recognized from the image data, is registered in the database. A control method having

11. Computers, If a string of characters determined to correspond to the business operator that issued the document is not recognized from the image data obtained by scanning the document, a display control means controls the system to display a warning based on the fact that a number corresponding to the string of characters determined to correspond to the name of the business operator that issued the document, which was recognized from the image data, is registered in the database. A program designed to function as such.

Images