Information processing device and program

Abstract

To provide an information processing device and program that allows even a user who does not grasp documentary items to manage account books smoothly compared to the case where the documentary items are displayed without being distinguished from items other than the documentary items.SOLUTION: In an information processing device 10, a processor 10A comprises: an acquisition section 11 that acquires a read image; an item extraction section 12 that extracts, from the acquired read image, items including a documentary item to be treated as a search condition according to the Electronic Books and Documents Preservation Act through character recognition; and a display control section 13 that controls a screen indicating corresponding items for confirming the extracted items to display the documentary item in a state where the documentary item can be distinguished from another item that is an item other than the documentary item among the items.SELECTED DRAWING: Figure 2

Description

[[Technical Field]]

[0001] The present invention relates to an information processing apparatus and a program. [[Background Art]]

[0002] For example, Patent Document 1 discloses an automatic definition system that automatically defines form data search item information in an electronic form system, comprising: means for inputting form data serving as original data of a form, and format data including data types and position information of each item in the form; means for generating, from the input form data and format data, a general-purpose electronic document file that does not have the data types and position information of each item in the form; means for inputting the general-purpose electronic document file and the format data, creating search form data in a format that enables search while retaining the form layout from these data, and registering the search form data in a form database; and means for analyzing the format data, extracting the data types and position information of items to be searched, and registering the extracted information as data search item information in a search information database, in order to enable search that takes into account the form layout and the data types of items in the form. [[Prior Art Documents]] [[Patent Documents]]

[0003] [[Patent Document 1]] Japanese Patent No. 5168706 [[Summary of the Invention]] [[Problems to be Solved by the Invention]]

[0004] While the requirements for preserving electronic books were relaxed by the 2021 amendment to the Electronic Bookkeeping Law (Act on Specification of Methods for Preserving National Tax-Related Books and Documents Created Using Electronic Computers (Act No. 25 of 1998)), three record items—date of transaction, transaction amount, and trading partner—are still included in the preservation requirements, and those obligated to preserve them are responsible for their management. However, it is conceivable that users of systems that manage books and other documents may not know which items of the books they should be managing. The objective of this invention is to enable smooth ledger management even for users who are not familiar with the recorded items, compared to displaying recorded items without distinguishing them from other items. [Means for solving the problem]

[0005] The invention described in claim 1 comprises a processor, the processor acquires a read image, extracts items from the acquired read image by character recognition that include record items that are search conditions under the Electronic Bookkeeping Law, and displays a screen for confirming the extracted items. and a screen that accepts corrections to the character recognition results for the item in question. The information processing device is characterized by displaying the record item in a manner that allows it to be distinguished from other items that are not the record item. The invention described in claim 2 is, The system includes a processor, which acquires a read image, extracts items from the acquired read image that include record items that are search conditions under the Electronic Bookkeeping Law using character recognition, and controls the display on a screen showing the extracted items so that the record items are displayed in a manner that allows them to be distinguished from other items that are not the record items. The display control, on the screen showing the item, displays whether or not it is possible to transition from the screen showing the item to the next screen, depending on the status of the recorded item. death , When all the aforementioned recording items are collected, the display control will show that the screen is ready to proceed to the next screen. It is characterized by the following ru It is an information processing device. The invention described in claim 3 is, When outputting the content read in accordance with the aforementioned items, the aforementioned recorded items cannot be deleted from the output. The information processing device is as described in claim 2. The invention described in claim 4 is, The system includes a processor, which acquires a read image, extracts items from the acquired read image that include record items that are search conditions under the Electronic Bookkeeping Law by character recognition, displays the extracted items on a screen showing the items in a state that allows them to be distinguished from other items that are not the record items, and displays on the screen showing the items whether or not it is possible to transition from the screen showing the items to the next screen, depending on the state of the record items. A key feature is that when the aforementioned record items are not complete, the system displays a message preventing the user from proceeding to the next screen. ru It is an information processing device. The invention described in claim 5 is an information processing device according to claim 4, characterized in that the display control is such that when the record items are not all present, a display indicating the missing record items is shown. The invention described in claim 6 is, The system includes a processor, which acquires a read image, extracts items from the acquired read image that include record items that are search conditions under the Electronic Bookkeeping Law using character recognition, and controls the display on a screen showing the extracted items so that the record items are displayed in a manner that allows them to be distinguished from other items that are not the record items. The display control controls the display so that when the user checks the content read in relation to the recorded item, the recorded item and the other items are displayed in a way that allows them to be distinguished. death , When confirming the content of the aforementioned record item, the user is prompted to confirm the candidates determined by the confidence level of the character recognition result. When confirming the content of other items, the user may omit confirming the candidates determined by the confidence level, and the user is prompted to correct the content. It is characterized by the following ru It is an information processing device. The invention described in claim 7 is, The system includes a processor, which acquires a read image, extracts items from the acquired read image that include record items specified as search conditions under the Electronic Bookkeeping Law using character recognition, displays the extracted items on a screen showing the items in a manner that allows the record items to be distinguished from other items that are not the record items, and displays the content read in accordance with the record items in a manner that allows the user to distinguish between the record items and the other items when the user is confirming the content of the record items, notifies the user of the confidence level of the character recognition result for the record item when the user is confirming the content of the record items, and does not notify the user of the confidence level of the other items when the user is confirming the content of the other items, even if the confidence level of the character recognition result is less than a predetermined value. It is characterized by the following ru It is an information processing device. The invention described in claim 8 is, The system includes a processor, which acquires a read image, extracts items from the acquired read image that include record items that are search conditions under the Electronic Bookkeeping Law using character recognition, and displays the extracted items on a screen showing those items in a way that allows them to be distinguished from other items that are not the record item. The display control also displays, when the content read in relation to the record item is corrected, elements for confirming the corrected content along with the correction history. , characterized by ru It is an information processing device. The invention described in claim 9 is, The system includes a processor, which acquires a read image, extracts items containing record items that are search criteria under the Electronic Bookkeeping Law from the acquired read image using character recognition, and controls the display on a screen showing the extracted items to confirm them, so that the record items are displayed in a manner that makes them distinguishable from other items, and it is possible to switch whether or not to display the record items in a manner that makes them distinguishable from other items. It is characterized by the fact that ru It is an information processing device. The invention described in claim 10 is, When the display is not shown in the identifiable state due to the aforementioned switching, and after the contents of the record item and the other items are confirmed, the system switches back to displaying the item in the identifiable state, the system does not retain user operation information for the record item, but retains user operation information for the other items. features 9 This is the information processing device described above. The invention described in claim 11 is, A program that implements: an acquisition function for acquiring a read image; an extraction function for extracting items containing record items that are search conditions under the Electronic Bookkeeping Law from the read image acquired by the acquisition function using character recognition; and a display control function for a screen that displays the extracted items for confirmation and accepts corrections to the character recognition results of the items, such that the record items are displayed in a state that can be distinguished from other items that are not the record items. That is the case. The invention described in claim 12 is, The information processing device implements an acquisition function for acquiring a read image, an extraction function for extracting items containing record items that are search conditions under the Electronic Bookkeeping Law from the read image acquired by the acquisition function using character recognition, and a display control function for displaying the items extracted by the extraction function on a screen showing the items, such that the record items are displayed in a state that can be distinguished from other items that are not the record items. The display control program displays whether it is possible to transition from the screen showing the items to the next screen depending on the state of the record items, and when all the record items are present, it displays that it is possible to transition to the next screen. That is the case. The invention described in claim 13 provides an information processing device that includes an acquisition function for acquiring a read image, an extraction function for extracting items containing record items that are search conditions under the Electronic Bookkeeping Law from the read image acquired by the acquisition function using character recognition, and a display control function for displaying the items extracted by the extraction function on a screen showing the items, such that the record items are displayed in a state that can be distinguished from other items that are not the record items. The display control, on the screen showing the items, displays whether or not it is possible to transition from the screen showing the items to the next screen, depending on the status of the recorded items, and when the recorded items are not all present, it displays that it is not possible to transition to the next screen. It is a program. The invention described in claim 14 is a program that provides an information processing device with an acquisition function for acquiring a read image, an extraction function for extracting items containing record items that are search conditions under the Electronic Bookkeeping Law from the read image acquired by the acquisition function using character recognition, and a display control function for displaying the items extracted by the extraction function on a screen showing the items in a state in which the record items can be distinguished from other items that are items other than the record item, wherein the display control displays the content read in correspondence with the record item in a state in which the record item and the other items can be distinguished when the user confirms the content of the record item, requests the user to confirm the candidate determined by the confidence level of the character recognition result when confirming the content of the record item, and when confirming the content of the other items, the user can omit the confirmation operation for the candidate determined by the confidence level, and requests the user to operate when correcting the content. The invention described in claim 15 is a program that provides an information processing device with an acquisition function for acquiring a read image, an extraction function for extracting items containing record items that are search conditions under the Electronic Bookkeeping Law from the read image acquired by the acquisition function using character recognition, and a display control function for displaying the items extracted by the extraction function on a screen showing the items so that the record items are displayed in a state that can be distinguished from other items that are items other than the record item, and the display control is a program that displays the contents read in correspondence with the record items in a state that can be distinguished from the record items and other items when the user confirms the contents of the record items, and when the user confirms the contents of the record items, if the confidence level of the result of the character recognition is less than a predetermined value, the program notifies the user of the confidence level for the record items, and when the user confirms the contents of other items, even if the confidence level of the result of the character recognition is less than a predetermined value, the program does not notify the user of the confidence level for the other items. The invention described in claim 16 is a program that provides an information processing device with an acquisition function for acquiring a read image, an extraction function for extracting items containing record items that are search conditions under the Electronic Bookkeeping Law from the read image acquired by the acquisition function using character recognition, and a display control function that displays the record items on a screen showing the items extracted by the extraction function in a state that makes them distinguishable from other items that are not the record items, and the display control is a program that, when the content read in relation to the record items is corrected, displays elements for confirming the corrected content along with the correction history. The invention described in claim 17 is a program that enables an information processing device to implement an acquisition function for acquiring a read image, an extraction function for extracting items containing record items that are search conditions under the Electronic Bookkeeping Law from the read image acquired by the acquisition function by character recognition, and a display control function for displaying the items extracted by the extraction function on a screen showing the items so that the record items are displayed in a state that can be distinguished from other items that are not the record items, and is switchable between displaying the record items in a state that can be distinguished from other items. [Effects of the Invention]

[0006] Claim 1 、2、4、6~9、11~17 According to this, compared to a case where record items are displayed without being distinguished from other items other than record items, even a user who does not know the record items can smoothly perform bookkeeping management 。 According to Claim 3, When outputting content read in relation to an item, you can remove the recorded item from the output. compared to the case where User regarding output operability of can be improved Let 。 According to Claim 5, when record items are incomplete, operability can be improved compared to a case where control is not performed to display an indication that the record items are incomplete 。 Claim 10 According to this, when the record items and other items are not displayed in an identifiable state by switching, and after the contents of the record items and other items are confirmed, when switching is performed to display them in an identifiable state, the burden of information processing can be reduced compared to a case where the following control is not adopted: not retaining user operation information for the record items, while retaining user operation information for the other items 。 BRIEF DESCRIPTION OF THE DRAWINGS

[0007] [Figure 1] It is a diagram showing a schematic configuration of a form system. [Figure 2] It is a functional block diagram of an information processing apparatus. [Figure 3] It is a flowchart showing processing of a setting screen according to the present embodiment. [Figure 4] It is a diagram showing a screen displayed in the case of a setting screen. [Figure 5] It is a diagram showing display example 1 displayed in the case of a setting screen. [Figure 6] It is a diagram showing display example 2 displayed in the case of a setting screen. [Figure 7] It is a diagram showing display example 2 displayed in the case of a setting screen. [Figure 8] ​This figure shows example 3 of the display that appears when the settings screen is displayed. [Figure 9] This figure shows an example of the display shown on the settings screen (Figure 4). [Figure 10] This figure shows an example of the display shown on the settings screen (Figure 4). [Figure 11] This figure shows example 5 of the display that appears on the settings screen. [Figure 12] This diagram shows the screen displayed when setting the output format in the settings screen. [Figure 13] This diagram shows the screen displayed in the settings screen when you are using the business check settings. [Figure 14] This diagram shows the screen displayed when a required input is checked on a confirmation screen. [Figure 15] This diagram shows the screen displayed when you need to revert to the previous version after confirming the changes. [Figure 16] This is the screen that appears when a confirmation screen is required. [Figure 17] This is the screen that appears when a confirmation screen is required. [Figure 18] This is the screen that appears when making corrections to a confirmation screen. [Figure 19] This is the screen that appears when the correction history is displayed on the confirmation screen. [Figure 20] This figure shows an example of a notification that appears on the confirmation screen. [Figure 21] This figure shows an example of a notification that appears on the confirmation screen. [Figure 22] This figure shows an example of notification 2 that appears on the confirmation screen. [Modes for carrying out the invention]

[0008] Embodiments of the present invention will be described in detail below with reference to the attached drawings. Figure 1 shows a schematic configuration of the document system 100. The document system 100 shown in the figure includes an information processing device 10. The information processing device 10 is connected to a client terminal 20 and an input device 30 so that they can communicate with each other. The network connecting the information processing device 10, the client terminal 20, and the input device 30 may be, for example, a LAN (Local Area Network) or the Internet. However, the network may also be a hybrid configuration of LAN and Internet.

[0009] The information processing device 10 manages a series of processing steps, including performing OCR (Optical Character Recognition) on image data of documents containing multiple forms, which are input through the input device 30, and outputting the results of the OCR processing to a predetermined output destination. The specific configuration and operation of the information processing device 10 will be described later.

[0010] The client terminal 20 transmits various instructions related to OCR processing to the information processing device 10. These instructions include, for example, an instruction to start reading the information in the image data and an instruction to display the results of reading the information in the image data. The client terminal 20 also displays the results of the OCR processing performed by the information processing device 10 in response to the received instructions, as well as various information such as notifications related to the OCR processing. As an example, the client terminal 20 may be a server device or a general-purpose computer device such as a PC (Personal Computer). Although Figure 1 shows only one client terminal 20, it is not limited to this, and multiple client terminals 20 may be provided, and as an example, different client terminals 20 may be used for different processes.

[0011] The input device 30 inputs image data to be processed by OCR into the information processing device 10. The input device 30 may include, for example, a server or a general-purpose computer device such as a PC, as well as an image forming apparatus having scanning, printing, and faxing functions. In addition to the input device 30, image data may also be input to the information processing device 10 from the client terminal 20.

[0012] Next, we will explain the overview of the reporting system 100. The document system 100 is a system in which the information processing device 10 performs OCR processing on image data input through the input device 30 and outputs the results of the OCR processing to a predetermined output destination.

[0013] The information processing device 10 manages various processes in OCR processing, including (1) business design and operation verification, (2) data input, (3) data reading, (4) form identification, confirmation and correction, (5) reading result confirmation and correction, (6) business check, (7) data output, and (8) return. In this embodiment, OCR processing refers not only to the process of reading characters and symbols from image data, but also to post-processing such as character correction.

[0014] As an example of the management of various processes, (1) business design and operation verification, (2) data input, (3) data reading, (6) business check, and (7) data output are each automatically executed by the information processing device 10. Also as an example of the management of various processes, (4) form identification, confirmation, and correction, and (5) reading result confirmation and correction are each accepted by user input via the client terminal 20. Furthermore, as an example of the management of various processes, (8) the return process may be automatically executed by the information processing device 10 or accepted by user input via the client terminal 20.

[0015] (1) In the process of business design and operational verification, a job rule is created that includes read definitions, output settings, and business check settings. In the read definition settings, as an example, the read range is set, which is the range in which the information of the image data is read in the process of "(3) Data Reading". More specifically, as an example, a definition may be set to read the item value, which is the value, from near the right of the item that is extracted as the key. In the output settings, as an example, the file format and output destination of the output data to be output in the process of "(7) Data Output" are set. In the business check settings, as an example, the required input items and format such as the number of characters that can be entered in the form to be detected in the process of "(6) Business Check" are set. Furthermore, (1) in the process of business design and operational verification, the system will present the user with an option to save electronic ledgers when defining the forms, and will also automatically extract the necessary search fields for saving electronic ledgers, providing a user interface (UI) to facilitate form definition operations.

[0016] (2) In the data input process, image data is received from the input device 30. For example, this can be uploaded from a local PC via a browser, linked with a cloud storage service, or entered into a hot folder on a PC. The image data received as input is registered as a job, which is the execution unit of the "(3) Data Reading" process.

[0017] (3) In the data reading process, the information of the image data within the job is read using the job rules selected by the user from among the job rules created in the process of "(1) Business Design and Operation Verification". For example, in this process, the documents contained in the image data within the job are identified (hereinafter referred to as "document identification") and the characters and symbols within the reading range are read.

[0018] (4) In the process of (4) form identification, confirmation, and correction, the image data within the job is classified into records that indicate the forms included in the job, based on the results of the form identification performed in the process of (3) data reading. Subsequently, this process displays the classified records and accepts confirmation and correction of the form identification by the user. Such a job may be a collection of multiple document images, each containing only images of the same type of document. However, for the purposes of this explanation, it will be described as a collection of document images, each containing multiple sets of application forms, each consisting of multiple types of documents. In other words, a job contains multiple types of document images. A collection of multiple types of document images included in one set is called a "record." Furthermore, in the (4) document identification, confirmation, and correction process, feedback information regarding the search items necessary for electronic ledger storage is presented to the UI and notified to the user, thereby enabling efficient and accurate confirmation and correction work.

[0019] (5) In the process of confirming and correcting the reading results, the reading results of characters and symbols within the reading range performed in the "(3) Data Reading" process are displayed, and corrections or rollbacks are made to the correct state. The corrected results received may be compared with a predetermined database, and the converted data may be displayed on the screen to reduce the effort required for confirmation and correction and to provide a function to detect errors in advance.

[0020] (6) In the business process check, errors in each preceding process are detected by the business check settings included in the job rule for the job in question, which is selected by the user from among the job rules created in the process of "(1) Business Design and Operational Verification". The detection results may be presented to the user. This allows the user to make corrections or send the changes back.

[0021] (7) In the data output process, output data is created using the output settings included in the job rule for the job selected by the user from among the job rules created in the process of "(1) Business design and operation verification," and the created output data is output to a predetermined output destination. Furthermore, (7) In the data output process, the data output is performed using the electronic ledger storage mode, which enables electronic ledger storage, and the search items required for electronic ledger storage are made mandatory.

[0022] (8) In the rollback process, the OCR process is rolled back to a process one or more steps prior to the process that was executed. Rollback is instructed by the user from the client terminal 20 during the execution of various processes such as "(4) Form Identification Confirmation and Correction" and "(5) Reading Result Confirmation and Correction". Rollback is also instructed by the administrator from the administrator's client terminal 20, for example, based on the results of an administrator check performed between the processes of "(6) Business Check" and "(7) Data Output".

[0023] In the OCR processing described above, the "(1) Business Design and Operational Verification" process is executed before the processes from "(3) Data Reading" onward, that is, before the reporting system 100 is put into operation. Furthermore, the "(1) Business Design and Operational Verification" process may also be executed during the operation of the reporting system 100 while the processes from "(3) Data Reading" onward are being performed. As an example, the job rules created in the "(1) Business Design and Operational Verification" process before the reporting system 100 is put into operation can be modified according to the results of the "(5) Reading Result Confirmation and Correction" process during the operation of the reporting system 100.

[0024] Here, the aforementioned Electronic Bookkeeping Law includes three categories of electronic record keeping: electronic bookkeeping, which involves keeping electronically created ledgers and documents as data; scanner-based storage, which involves keeping documents received or created on paper as image data; and electronic transactions, which involves keeping electronically exchanged transaction information as data. Of these three categories, scanner-based storage allows documents that are required to be stored under various tax laws to be saved as electronic data scanned from a scanner, rather than in paper form, with the aim of reducing the burden of document storage. Documents can be categorized into those that are directly related to the flow of funds or goods, such as contracts, delivery slips, invoices, and receipts, and those that are not directly related to the flow of funds or goods, such as quotations, purchase orders, and acceptance certificates. As a result of the aforementioned amendment to the law in 2021, the conditions for allowing storage in electronic data format have been relaxed (Article 4, Paragraph 3 of the Electronic Bookkeeping Law), and for example, the search function for ensuring visibility now only requires the following: that the transaction date or other date, transaction amount, and trading partner (record item) can be set as search conditions, that the range of record items related to date or amount can be specified when setting conditions, and that conditions can be set by combining two or more arbitrary record items (Article 2, Paragraph 6, Items 6(a) to (c) of the Enforcement Regulations of the Electronic Bookkeeping Law). Thus, among the various items included in a document, the record items or legally mandated items for search requirements are limited to the transaction date, transaction amount, and trading partner.

[0025] However, for users to extract data from hundreds or thousands of documents, for example, and then identify the necessary items for their search criteria, and further ensure the accuracy of the extracted data, requires significant effort and time from the operator, making it difficult to improve efficiency and placing a burden on the user. This is especially true for users who do not know the necessary items for their search. Therefore, this embodiment provides a configuration that enables smooth ledger management by suppressing the cost of extracting item information and efficiently ensuring the accuracy of the extracted results, even when a relatively large number of documents are digitized and stored, or when the user is unaware of the items necessary for searching. The following provides a detailed explanation.

[0026] Figure 2 is a functional block diagram of the information processing device 10. As shown in the figure, the information processing device 10 is configured to include an acquisition unit 11, an item extraction unit 12, a display control unit 13, and an operation reception unit 14.

[0027] The acquisition unit 11 acquires the image data received in "(2) Data Input" described above. In other words, the acquisition unit 11 acquires image data as a form image from the input device 30. The image data, once acquired by the acquisition unit 11, is stored in a storage unit such as an HDD (Hard Disk Drive) (not shown). The HDD (not shown) also stores operation information, which is input information from user operations.

[0028] The item extraction unit 12 performs the form identification process on a record-by-record basis in the "(3) Data Reading" described above. That is, it performs the process of extracting items contained in the image data by character recognition. Such items include record items that are search conditions under the Electronic Bookkeeping Law, and may also include other items.

[0029] The display control unit 13 controls the display when showing the results of the item extraction process on the screen in "(4) Form Identification, Confirmation, and Correction" described above. In other words, the display control unit 13 displays a screen showing the items for confirmation of the extracted items. Details will be described later.

[0030] The operation reception unit 14 receives confirmation and corrections regarding the results of the item extraction process in "(4) Form Identification Confirmation and Correction" described above. The received confirmation and corrections are reflected on the screen by the display control unit 13. Details will be described later.

[0031] Here, each function of the information processing device 10 is realized by a CPU 10A, which is an example of a processor. The CPU 10A reads a program stored in the ROM (Read Only Memory) 10B and executes the program using the RAM (Random Access Memory) 10C as the working area. The program executed by the CPU 10A can be provided to the information processing device 10 while stored in a computer-readable recording medium such as a magnetic recording medium (magnetic tape, magnetic disk, etc.), an optical recording medium (optical disk, etc.), a magneto-optical recording medium, or semiconductor memory. Alternatively, the program executed by the CPU 10A may be downloaded to the information processing device 10 using a communication means such as the Internet.

[0032] Furthermore, in this embodiment, each function of the information processing device 10 is implemented by software, but it is not limited to this, and may also be implemented by an ASIC (Application Specific IC).

[0033] Figure 3 is a flowchart showing the processing of the settings screen according to this embodiment, and shows an example of the processing described in "(1) Business design and operation verification" (see Figure 1) above. When the acquisition unit 11 (see Figure 2) creates job rules for form settings for the acquired image data, this is done on the display screen displayed by the display control unit 13 (see the same figure). The process shown in Figure 3 is an example of processing when the user instructs the display screen to use the search item check for electronic ledger storage. When such a user instruction is received, automatic extraction is performed (step 101). That is, the item extraction unit 12 (see the same figure) automatically extracts the definition locations of the search items in the image data, and after the automatic extraction is performed, it is determined whether the automatic extraction was successful (step 102). The items to be automatically extracted are the record items under the Electronic Bookkeeping Law, namely the transaction date, transaction amount, and trading partner. However, other items besides these record items may also be included in the automatic extraction process.

[0034] If the automatic extraction is successful (Yes in step 102), the next step is to determine whether there is a request for correction (step 103). This determination of whether a correction has been requested is made by the operation reception unit 14 (see Figure 2), based on whether the user has performed an operation to correct the successful automatic extraction.

[0035] If a correction request is received (Yes in step 103), the display control unit 13 (see Figure 2) makes the correction to the display screen (step 104). After that, the output format is set (step 105) and the output destination is set (step 106) in order, and the process ends. On the other hand, if no correction is received (No in step 103), proceed to step 105 as described above.

[0036] If automatic extraction is unsuccessful (No in step 102), the system accepts manual input from the user (step 107) and proceeds to step 104 above to reflect the received manual input on the display screen. Thus, if automatic extraction fails (No in step 102) or if the user needs to make corrections even if automatic extraction is successful (Yes in step 103), the operation reception unit 14 (see Figure 2) accepts the user's input.

[0037] Next, the screen 40 of the display unit (not shown) of the information processing device 10 will be described. Figure 4 shows screen 40, which is displayed when the settings screen is shown, and is an example of the screen before the automatic extraction is performed (see step 101 in Figure 3). The screen 40 shown in the figure includes a progress display area 41 for displaying the progress status, a reduced display area 42 for displaying reduced image data, an enlarged display area 43 for displaying enlarged image data from the reduced display area 42, a configuration settings display area 44 for displaying configuration settings, and an instruction area 45 for instructing the user to proceed to the next screen.

[0038] The progress display area 41 is an area that displays progress information in the settings screen in a unidirectional manner, for example, from left to right, and in this embodiment, it is shown to the user in four stages. Specifically, the progress display area 41 includes a setting item 41a for the first stage configuration, a setting item 41b for the second stage reading method, a setting item 41c for the third stage output format, and a setting item 41d for the fourth stage output destination. More specifically, setting item 41a displays "Configuration Settings," setting item 41b displays "Reading Method Settings," setting item 41c displays "Output Format Settings," and setting item 41d displays "Output Destination Settings."

[0039] To further explain, the progress display area 41 indicates which of the four stages the user is currently in based on transparency and color. For example, in Figure 4, setting item 41a has low transparency, the font is clearly visible, and the number 1 is highlighted. On the other hand, setting items 41b to 41d have higher transparency than setting item 41a, for example, they are semi-transparent, the font is not clearly visible, and the numbers 2 to 4 are not highlighted. As a result, a user viewing screen 40 as shown in Figure 4 can understand that there are four stages and that the current progress is at stage 1.

[0040] The reduced display area 42 is a left-aligned vertical area that shows the image data in which the job rules are defined. If there are multiple such areas, they are displayed arranged from top to bottom. In Figure 4, the reduced image data 42a of the form image is shown, but the contents are not illustrated.

[0041] The enlarged display area 43 is the area approximately in the center of the image data 42a of the reduced display area 42, enlarged to a degree that allows the user to confirm the content. As shown in Figure 4, the user can confirm that the enlarged image data 50 is an invoice, and can also confirm the names and contents of each item in the image data 50.

[0042] The configuration setting display area 44 is a right-aligned area for configuring the image data 50. More specifically, the configuration of the image data 50 includes the number of pages 44a, whether or not there are attached pages 44b, whether or not to use automatic detection 44c, and whether or not to use the search item check 44d for electronic ledger storage. The number of pages 44a is an item with a numerical input field, while the remaining three items 44b to 44d are selected using radio buttons.

[0043] If the "Use" radio button is selected in the "Search Item Check for Electronic Bookkeeping" 44d, which is enclosed in a dashed frame in Figure 4, the information processing device 10 determines that the user has instructed to use the search item check for electronic bookkeeping, and the processing example shown in Figure 3 above is performed. In this embodiment, it is called "Search Item Check for Electronic Bookkeeping," but it may also be called "Electronic Bookkeeping Check" or "Electronic Bookkeeping."

[0044] When the search item check function 44d for electronic bookkeeping is enabled by clicking the radio button, processing such as the automatic extraction of the definition location of the search item in the image data 50 (step 101, see Figure 3) is performed. Also, when the search item check function 44d is enabled, as shown in screens 40 and 140 from Figure 5 onwards, the record items under the Electronic Bookkeeping Law and other items are displayed in a way that makes them distinguishable. On the other hand, if the search item check function 44d is not enabled, the above-mentioned automatic extraction process is not performed, and the record items and other items are not displayed in a way that makes them distinguishable. Thus, in this embodiment, the radio button for the search item check 44d allows switching whether or not to display the recorded item in a way that makes it distinguishable from other items.

[0045] In the example shown on screen 40, the instruction area 45 displays the word "Next". To explain further, similar to the progress display area 41, notifications are made to the user by switching the clarity of the display, such as the degree of transparency. When the instruction area 45 is clearly displayed, it notifies the user that the instruction area 45 is in a state where the user can give instructions to it (activated), and when the instruction area 45 is not clearly displayed, it notifies the user that the instruction area 45 is in a state where instructions cannot be given to it (inactive).

[0046] After completing the input operation for the configuration setting display area 44, pressing the instruction area 45 will prompt the system to proceed to the next screen. This will move the system from setting item 41a in the progress display area 41 to setting item 41b for the reading method, and automatic extraction will be performed (see step 101 in Figure 3).

[0047] Next, display examples 1 to 5 displayed on screen 40 in the above-mentioned "reading method setting (see reference numeral 41b in Fig. 4)" scene on the setting screen will be described with reference to Figs. 5 to 11. In Figs. 5 to 11, "Edit invoice" is displayed at the upper left of the screen 40 to indicate to the user that the current operation is editing an invoice. In addition, since the image data 50 is an "invoice", form identification can be performed by OCR recognition, and no input operation is required from the user.

[0048] Fig. 5 is a diagram showing the screen 40 displayed in the case of the setting screen, which is display example 1 displayed when automatic extraction succeeds (see Yes in step 102 of Fig. 3). As is clear from the display in the progress status display area 41, the screen 40 of display example 1 shown in Fig. 5 is a scene where the setting item 41a is completed and the process has moved to the next reading method setting item 41b. In Fig. 5, an image 42b of "one page of background image" is added to the image data 42a in the reduced display area 42.

[0049] To further explain, a setting content display area 46 showing the content of the reading method setting is displayed at the position of the configuration setting display area 44 (see Fig. 4) on the screen 40. The setting content display area 46 includes a record item area 46a corresponding to record items under the Act on Electronic Bookkeeping.

[0050] Said record item area 46a is an area for displaying the record items under the Act on Electronic Bookkeeping, that is, transaction date and other dates, transaction amount and information corresponding to the transaction partner, and has a three-stage configuration. More specifically, in the upper stage of the three-stage configuration, from the left side to the right side, "transaction partner" as an item name, a column indicating the recognition result, an OCR display, and a record item icon image 46d indicating that the item is a record item are arranged in a line in order. The record item icon image 46d is an image in which the character "electronics" is attached in a circular figure, and for example, the circular figure may be colored and the character "electronics" may be displayed in white outline. In addition, the item name of the middle stage of the three-stage configuration is "date", the item name of the lower stage is "amount", and other parts of the middle stage and the lower stage are the same as those of the upper stage.

[0051] The record item area 46a is always displayed when the "Use" radio button (see Figure 4) is selected in the "Search Item Check for Electronic Bookkeeping" 44d, and users are not permitted to delete it.

[0052] Furthermore, screen 40 in Figure 5 contains a display corresponding to the execution of automatic extraction (see step 101 in Figure 3). Specifically, a notification frame 61 is displayed on screen 40, spanning the enlarged display area 43 and the settings content display area 46, to inform the user that the automatic extraction was successful. This notification frame 61 contains the message, "Automatic frame extraction of search items is complete. Please check the relevant section and correct the frame as necessary." In response to the execution of automatic extraction, the image data 50 in the enlarged display area 43 shows automatic frame images 51, 52, and 53. Automatic frame image 51 corresponds to "Customer" in the upper section of the record item area 46a, automatic frame image 52 corresponds to "Date" in the middle section, and automatic frame image 53 corresponds to "Amount" in the lower section.

[0053] The instruction area 45 is activated and becomes user-operable when all search item frames are extracted and automatic extraction is successful (see Yes in step 102 of Figure 3). In Figure 5, automatic extraction is successful, and the instruction area 45 is clearly displayed and activated for easy user recognition. If not all search item frames are extracted (No in step 102 of the same figure), the instruction area 45 is not activated, and the user cannot perform any operations on the instruction area 45.

[0054] As shown above, screen 40 in Figure 5 displays whether or not it is possible to proceed to the next screen depending on the status of the three items in the record item area 46a. Specifically, when all three items are present, the indicator area 45 is activated to indicate that it is possible to proceed to the next screen, and when all three items are not present, the indicator area 45 is deactivated to indicate that it is not possible to proceed to the next screen. Here, "all items present" means that input has been completed for all record items, and "all items present" means that input has not been completed for at least one of the record items. "Input" includes cases where the results of character recognition are entered by automatic extraction, as well as cases where the results of character recognition are entered into a frame manually set by the user. Furthermore, the notification frame 61 in Figure 5 contains a message indicating that the automatic extraction of search items has been completed, and this display also allows the user to understand that they are ready to proceed to the next screen. Although not shown in the illustration, the ability to proceed to the next screen may also be displayed on screen 40.

[0055] Figures 6 and 7 show screen 40, which is displayed when the settings screen is displayed, and are example 2 of the display that appears when automatic extraction fails (see No. in step 102 of Figure 3). Figures 6 and 7 are shown in chronological order, and in example 2, Figure 7 is displayed after Figure 6. In the display example 2, screen 40 shows a notification frame 62, as shown in Figure 6. This notification frame 62 contains the error message: "Automatic frame extraction of search items was performed, but [Customer] could not be extracted. Please set the frame manually." In other words, the notification frame 62 indicates that "Customer" could not be automatically extracted from "Customer," "Date," and "Amount" in the record item area 46a, and instructs the user to set the frame manually. Referring to Figure 6, the image data 50 does not show the automatic frame image 51 (see Figure 5) corresponding to "Customer."

[0056] In the scenario shown in Figure 6, not all of the search item frames are extracted, and therefore the indicator area 45 is inactive and not activated. In other words, the indicator area 45 is displayed in a state that is difficult to see, and the information processing device 10 does not respond even if the user tries to manipulate the position of the indicator area 45. Therefore, instead of operating the instruction area 45, the user can switch to a screen that allows manual operation by operating the OK button 62a in the notification frame 62.

[0057] When the user operates the OK button 62a in the notification frame 62, the user proceeds to the screen 40 shown in Figure 7. That is, as shown in Figure 7, the user transitions to a state 46b in which the frame setting for "Business Partner" is activated in the upper item name of the record item area 46a of the setting content display area 46. The user manually specifies the area corresponding to "business partner" in the image data 50 of the enlarged display area 43. Once this specification is made, the previously inactive instruction area 45 becomes active.

[0058] Figure 8 shows screen 40, which is displayed when the settings screen is displayed, and is an example of display 3 that is displayed when automatic extraction fails (No. in step 102, see Figure 3). In the display example 3 shown in Figure 8, screen 40 displays a notification frame 63. This notification frame 63 contains the error message, "Automatic frame extraction of search items was performed, but [Business Partner] could not be extracted. Please set the frame manually," as well as, "If [Business Partner] is not listed in the report, please replace the image used to define it." In other words, the notification frame 63 in display example 3 is displayed when the image data 50, which is the report image, does not contain information corresponding to the search item "Business Partner." An OK button 63a is also displayed in the notification frame 63.

[0059] To further explain, as shown in Figure 8, in display example 3, a button 63b for performing image replacement is displayed. Button 63b may also be displayed when the OK button 63a in the notification frame 63 is pressed. In addition, in display example 3, as in display example 2, the indicator area 45 is inactive.

[0060] Figures 9 and 10 show screen 40, which is displayed when the settings screen is displayed. They are example 4 of the display shown when corrections are made (see step 104 in Figure 3) after successful automatic extraction (see Yes in step 102 in Figure 3). Figures 9 and 10 are shown in chronological order, and in example 4, Figure 10 is displayed after Figure 9. Screen 40 in Display Example 4 shows the notification frame 64 shown in Figure 9. The notification frame 64 displays the message: "Automatic frame extraction of search items has been performed. Multiple candidates have been extracted for the [Business Partner] frame, so please review the candidates and make a selection." In other words, although the automatic extraction was successful, there are multiple candidates, and the user is instructed to review the multiple candidates. The instruction area 45 is inactive, and the user can switch to a screen that allows manual operation by operating the OK button 64a in the notification frame 64.

[0061] When the user operates the OK button 64a in the notification frame 64, the user proceeds to the screen 40 shown in Figure 10. That is, just as in the display example 2 shown in Figure 7, the user transitions to a state 46b where the frame setting for "Business Partner" is activated, with the item name in the upper part of the record item area 46a of the setting content display area 46 being displayed. The user manually selects one of the areas 51a or 51b corresponding to one of the multiple "business partners" in the image data 50 of the enlarged display area 43. In other words, the areas 51a and 51b that are candidates for "business partners" are activated, for example, by highlighting.

[0062] When a user makes a selection for, for example, area 51a, a notification box 65 appears with the message, "Do you want to set the selected area as a [customer]?", as shown in Figure 10. The notification box 65 displays a "Yes" button 65a and a "No" button 65b. When the user presses the "Yes" button 65a, the instruction area 45 is activated and becomes operable.

[0063] Figure 11 shows screen 40, which is displayed when the settings screen is displayed, and is display example 5, which is displayed when corrections are made (see step 104 in Figure 3) after automatic extraction is successful (see Yes in step 102 in Figure 3). Display example 5 in Figure 11 is displayed when the user corrects the frame in display example 1 in Figure 5. The image data 50 of the enlarged display area 43 shown in Figure 11 includes not only the automatically framed images 51-53 (see also Figure 5) but also the manually framed image 54. The automatically framed images 51-53 are extracted automatically, while the manually framed image 54 was added manually by the user.

[0064] Furthermore, the setting content display area 46 includes not only the recording item area 46a, but also an other item area 46c for displaying items other than the recording items. The other item area 46c corresponds to the manual frame image 54, and the item name "Total," a field indicating the recognition result, and the OCR display are arranged in a single row.

[0065] The other item area 46c, like the record item area 46a, has a field for the item name, a field for showing the recognition result, and an OCR display. On the other hand, the other item area 46c differs from the record item area 46a in that it does not have a record item icon image 46d to indicate that it is a record item. In other words, the user recognizes whether or not an item is a record item based on the presence or absence of the record item icon image 46d. As shown above, in the screen 40 of Figure 11, the three items in the record item area 46a are displayed in a way that makes them distinguishable from the items in the other item area 46c.

[0066] Next, we will explain an example of the display shown on screen 40 in the "Output Format Settings (see reference numeral 41c in Figure 4)" section of the settings screen. Figure 12 shows screen 40, which is displayed when the settings screen is shown, and is displayed when setting the output format (see step 105 in Figure 3). The screen 40 shown in Figure 12 indicates that the user has progressed to the third setting item 41c, as shown in the progress display area 41.

[0067] Furthermore, screen 40 includes an output format display area 70 that displays the settings for the output format. The output format display area 70 includes an area 71 that shows the output format and an area 73 that shows the record content. In this way, when outputting the read content, the output includes the three items of the record item area 46a: "Customer," "Amount," and "Date."

[0068] The radio button for "Electronic ledger storage 71a" in area 71, which indicates the output format, becomes selectable only when the "Use" radio button in "Search item check for electronic ledger storage" 44d (see Figure 4) is selected. Furthermore, by selecting "Ledger (multiple lines per record)" as the output format, output in CSV format is possible, and it is also possible to save it as a file.

[0069] The record item display section 73a in area 73 includes record items such as "Customer" in column A, "Amount" in column B, and "Date" in column C. These columns cannot be deleted, but they can be sorted. In other words, the record items cannot be removed from the output items and will always be included in the output.

[0070] Figure 13 shows screen 40, which is displayed when the settings screen is open, and is the same screen that appears in the business check settings (see Figure 1). In other words, it shows the case when job rule verification is performed without enabling the "Search item check for electronic ledger storage" function 44d (see Figure 4). The screen 40 shown in Figure 13 is an example of checking and correcting the document discrimination result during job rule verification, and includes an enlarged display area 43 showing image data 50 and a document name display area 80 showing the automatic determination result of the document name. The document name display area 80 includes an area 81 showing the automatic determination result, an area 82 for inputting information when making corrections, and an area 83 showing a reference image of the document name. Furthermore, a notification box 66 is displayed on screen 40. This notification box 66 displays the message, "Do you want to save the electronic ledger? Pressing [Yes] will enable the 'Search item check for electronic ledger saving'." The notification box 66 displays a "Yes" button 66a and a "No" button 66b.

[0071] In the example in Figure 13, area 81 shows the automatic determination result as "Invoice". If the image data 50 is an invoice, the user presses the "Yes" button 66a in the notification frame 66 to save it as an electronic ledger. If the image data 50 is not an invoice, the user refers to area 83, enters the name of a document other than an invoice in area 82, and then presses the "Yes" button 66a.

[0072] When the "Yes" button 66a is pressed, the process proceeds to screen 40 shown in Figure 4 above, where the job rule is defined. At this time, the "Search item check for electronic ledger storage" function 44d (see Figure 4) may be enabled. Here, if the function of "search item check for electronic ledger preservation" 44d (see Figure 4) is not enabled, and therefore the record items in the record item area 46a are not displayed in a way that allows them to be distinguished from other items in the other item area 46c, when the system is switched to display them in a way that allows them to be distinguished, the handling of operation information performed by the user up to that point may be handled differently for record items and other items. For example, it is conceivable to control the system so that user operation information for record items is not stored on an HDD not shown, while user operation information for other items is stored on an HDD not shown. Ultimately, the record items stipulated in the Electronic Ledger Preservation Law require the user to perform an operation each time. This is because the record items stipulated in the Electronic Ledger Preservation Law require user confirmation each time.

[0073] Next, we will explain the confirmation screen. This confirmation screen is a step that is performed based on the settings screen, and is an example of a display showing the processes of "(4) Document Identification Confirmation and Correction" and "(5) Reading Result Confirmation and Correction" (see Figure 1) described above. Figures 14 to 19 will be used to explain such an example.

[0074] Figure 14 shows screen 140, which is displayed when it is a confirmation screen. It is an example of a situation where required input checks are performed during OCR confirmation and correction as set in screen 40 shown in Figure 11, and it is a situation that can be corrected. As shown in Figure 14, the screen 140 includes a magnified display area 143 on the left where the image data 150 is displayed in an enlarged view, and a result display area 100A on the right that shows the reading result. The magnified display area 143 corresponds to, for example, the magnified display area 43 in Figure 4.

[0075] The result display area 100A includes a record item area 100a corresponding to the record items in the Electronic Bookkeeping Law, and an other item area 100c located below the record item area 100a, which displays other items that are not record items.

[0076] The record item area 100a corresponds to the record item area 46a on the settings screen (see, for example, Figure 11), and is arranged in a three-tiered structure from top to bottom as follows: "Customer," "Date," and "Amount." More specifically, each of the three tiers, from left to right, consists of the item name, a record item icon image 146d indicating that it is a record item, a field showing the recognition result, and fields for confidence level and confirmation check. Although the arrangement of the recorded items area 100a differs from that of the recorded items area 46a on the settings screen (see, for example, Figure 11), it shares the same features as the recorded items area 46a, namely the item name and the recorded item icon image 146d (corresponding to the recorded item icon image 46d in the recorded items area 46a), while it differs from the recorded items area 46a in that it includes a field for showing the recognition result (a field labeled "Scanned Image / Reading Result"), confidence level, and confirmation check field.

[0077] The confidence level referred to here is the confidence level of the character recognition result, and is a measure that indicates the degree of agreement between the image data 150 and the OCR recognition result. The confidence level may be displayed in a way that distinguishes between cases where it is greater than or less than a predetermined threshold, or it may be displayed as an image classified into three stages indicating greater or less confidence. For example, when the level of confidence is displayed by the length of bars extending from left to right, it is conceivable to use a different color for bars when the confidence level is low compared to bars when it is otherwise low. To calculate such confidence levels, known algorithms can be applied, such as pattern matching methods using feature vectors or templates, and methods using neural networks trained with machine learning to output confidence levels based on the images to be compared.

[0078] Furthermore, the record item area 100a can display a warning icon image 100d, which is shown to alert the user according to the level of confidence. In the case shown in Figure 14, the level of confidence for "Business Partner" in the upper section is lower than a predetermined threshold, and therefore the field showing the recognition result is blank. To inform the user of this situation, a warning icon image 100d is displayed near the level of confidence in the upper section.

[0079] The warning icon image 100d will disappear after the system performs an "automatic correction result check" process following the input of the confirmation result. If there are no problems with the check, the image will disappear. This "automatic correction result check" process is performed as one of the processes described in "(5) Processing for confirming and correcting reading results" (see Figure 1). The warning icon image 100d is an image with an exclamation mark inside an equilateral triangle, as shown in Figure 14. Alternatively, to improve visibility, the equilateral triangle may be made red and the exclamation mark white. As shown above, in screen 140 of Figure 14, the three items in the record item area 100a are displayed in a way that makes them distinguishable from the items in the other item area 100c.

[0080] To further explain the warning icon image 100d, as mentioned above, for the record item area 100a, the warning icon image 100d is displayed when the confidence level is lower than a predetermined value, notifying the user. However, for other item areas 100c, as shown in Figure 14, the warning icon image 100d is not displayed even though the confidence level is lower than a predetermined value, and no notification is given to the user. Thus, the display of the warning icon image 100d is controlled differently in the recording item area 100a and the other item area 100c. This is because the recording items require a higher level of accuracy in their input than the other items. Alternatively, the display of the warning icon image 100d in the other item area 100c may be controlled in the same way as in the recording item area 100a.

[0081] In the results display area 100A, the section displaying the recognition results, etc., labeled "Scanned Image / Reading Result," is divided into two parts: the upper part displays the scanned image, and the lower part displays the reading result, allowing the user to input corrections to the reading result. The upper part allows the user to confirm the scanned image, and the lower part allows the user to confirm the reading result and correct its contents.

[0082] In the results display area 100A, the record item icon image 146d is displayed for the record item area 100a, but not for the other item area 100c. This ensures that when the user checks the read content in the section indicating the recognition results, the three items in the record item area 100a and the items in the other item area 100c are displayed in a way that allows for identification.

[0083] Furthermore, the confirmation check box is for the user to check if they determine through visual inspection that the OCR recognition result does not require correction. In Figure 14, the confirmation check box for other item area 100c is checked, and if the confidence level of the character recognition result for other items is high, the check box may be automatically filled in. This allows the user to skip the checking operation for items with a high confidence level of character recognition result. However, even if the confidence level of the character recognition result for a recorded item is high, the check box will not be automatically filled in. By doing this, regardless of the confidence level of the character recognition result for a recorded item, user confirmation is required each time, thereby ensuring the accuracy of the recognition results for recorded items.

[0084] The other item area 100c in the result display area 100A corresponds to the other item area 46c on the settings screen (see, for example, Figure 11), and in the example in Figure 14, "Total" is placed there. The other item area 100c is similar to the record item area 100a in that it has fields for the item name, recognition result, confidence level, and confirmation check, but it differs from the record item area 100a in that it does not have a record item icon image 146d.

[0085] Furthermore, in the screen 140 shown in Figure 14, a notification box 161 is displayed to notify the user. This notification box 161 displays the message, "The search result for the search item [Business Partner] is empty. Please check the document image and enter the result." An OK button 161a is also displayed in the notification box 161.

[0086] As shown in Figure 14, you can select and display either "Report View" or "List View" in the upper left corner of screen 140, but in this figure, "Report View" is selected. Also, the file name of image data 150 is displayed below "Report View".

[0087] Furthermore, an instruction area 145 with the word "OK" is located in the lower right corner of screen 140. Similar to the case of instruction area 45 (see Figure 5), this instruction area 145 is activated when it is operable and deactivated otherwise. In other words, when all three items in the recording item area 100a are present, activating the instruction area 145 indicates that it is possible to proceed to the next screen, and when all three items are not present, deactivating the instruction area 145 indicates that it is not possible to proceed to the next screen.

[0088] Figure 15 shows screen 140, which is displayed when a confirmation screen is shown. Unlike Figure 14, which shows a scene where modifications are possible, this figure shows a scene where the changes are sent back. Note that since Figure 15 shares some parts with Figure 14, explanations of such common parts may be omitted. The screen 140 shown in Figure 15 represents the case where the image data 150, which is a document image, does not contain any information as a search item [Customer].

[0089] In such cases, one possible solution is to allow users to select instructions from a menu by clicking on the warning icon image 100d. The menu display will include two options: "Check the form image and enter the result" and "Return as blank." The former is for when the user manually enters the result without returning the form, and the latter is for when the form is returned without manual user input.

[0090] Additionally, as another measure, if the "Search item check for electronic ledger storage" 44d (see Figure 4) is ON, the user will be prompted to operate a button. Note that in Figure 15, the [Return] button 161b is inactive, but in the above case, the [Return] button 161b will be activated and displayed.

[0091] Furthermore, as another measure, if "Check search items for electronic ledger storage" 44d (see Figure 4) is ON, the following will be added to the menu display: that is, an item will be added to return the entry as "no information" and automatically input the comment "Electronic ledger storage not possible". The standard menu display includes the items "Mark as Incomplete Record" and "Enter Comment".

[0092] Furthermore, as another option, a menu display may be provided to process items as ineligible for electronic record keeping. For example, this could include items such as "Output recognition results for items excluding all search items (3)" and "Exclude the relevant report from the data output in output setting 'Electronic Record Keeping' mode."

[0093] Figure 16 shows screen 140, which is displayed when the confirmation screen is shown. This is a mandatory confirmation scene where each item in the record item area 100a is checked. Note that since Figure 16 has some parts in common with Figure 14, explanations of such common parts may be omitted. In the screen 140 shown in Figure 16, the confirmation check box in the record item area 100a has checks for "Customer" and "Amount," but not for "Date." Therefore, in screen 140, a warning icon image 100d is displayed near the confirmation check box for "Date."

[0094] Furthermore, in the screen 140 shown in Figure 16, a notification box 162 is displayed. This notification box 162 displays the message, "The search item [Date] has not been verified. Please check the reading results and make corrections as necessary." An OK button 162a is also displayed in the notification box 162.

[0095] The warning icon image 100d will be displayed until the "Date" confirmation checkbox is checked, at which point it will disappear. Furthermore, when the warning icon image 100d is displayed, the instruction area 145 is inactive, as shown in Figure 16, and the user cannot operate it. On the other hand, when the warning icon image 100d disappears, the instruction area 145 is activated, and user operation is permitted.

[0096] In addition, in the screen 140 shown in Figure 16, the confidence level of the record item area 100a differs for "Customer" from that of "Date" and "Amount". Specifically, the horizontal length indicating the confidence level of "Customer" is shorter than that of "Date" and "Amount", and the confidence level of "Customer" is lower than that of "Date" and "Amount".

[0097] Figure 17 shows screen 140, which is displayed when the confirmation screen is shown. As with Figure 16, this is a mandatory confirmation scene where each item in the record item area 100a is checked. Note that since Figure 17 has some parts in common with Figure 16, explanations of such common parts may be omitted. In the screen 140 shown in Figure 17, just like in Figure 15, the "Customer" and "Amount" fields in the record item area 100a are checked, but the "Date" field is not checked, and a warning icon image 100d is displayed. Note that in the screen 140 shown in Figure 17, the notification frame 162 shown in Figure 16 is not displayed. This is because the display disappeared when the OK button 162a of the notification frame 162 was pressed.

[0098] Figure 17 shows a batch confirmation check unit 100e for checking all confirmation check boxes at once in the screen 140. However, checking the batch confirmation check unit 100e is invalid for the confirmation check boxes in the record item area 100a. In other words, even if you check the batch confirmation check unit 100e, the confirmation check boxes in the record item area 100a will not be checked. Therefore, in order to remove the warning icon image 100d, it is necessary to check the confirmation check field for the corresponding "date," rather than checking the batch confirmation check unit 100e. This type of display control ensures that confirmation operations for record items are performed reliably.

[0099] Figure 18 shows screen 140, which is displayed when the confirmation screen is shown, and depicts a scene where an item in the record item area 100a is corrected. The screen 140 shown in Figure 18 displays a warning icon image 100d in the "Business Partner" field of the record item area 100a. This warning icon image 100d is displayed because the user entered incorrect information in the lower part of the field showing the recognition results, etc. To explain the circumstances, the scanned image displayed at the top of the field showing the recognition results, etc. was "XXXX Corporation," but the reading result was "XXX Corporation" (see Figure 17), and the confidence level was also displayed (see the same figure), so the user noticed that there was an error in the reading result for "Business Partner." Therefore, the user tried to correct "XXX" to "XXX." However, as shown in the lower part of the field showing the recognition results, etc. in Figure 18, the user mistakenly deleted "Company" when making the correction. As a result, what was originally "XXX Corporation" was incorrectly corrected to "XXX Corporation." Therefore, notification box 163 displays the message: "The correction result for the search item [Business Partner] may be incorrect. Please check the correction details, and if there are no problems, check [Confirm]." In this way, the user's input in the record item area 100a is also automatically verified. On the other hand, the user's input in other item areas 100c may be excluded from automatic verification.

[0100] Next, we will explain how the correction history is displayed on the confirmation screen. Figure 19 shows screen 140, which is displayed when the confirmation screen is shown, and the correction history is displayed. In the screen 140 shown in Figure 19, the correction history 200 is displayed. This correction history 200 is displayed when the history button 100f located in the record item area 100a is pressed. The history button 100f is displayed in the corresponding position when a correction is made to the record item area 100a.

[0101] Correction history 200 includes "Time" indicating the time of correction, "Reading Result" indicating the OCR recognition result or input result, and "Corrector" indicating the person who made the correction. Correction history 200 is arranged chronologically from top to bottom. Therefore, the correction at the bottom is the most recent. The corrections at the top are shown in Figure 18 above.

[0102] To further explain, in the correction history 200, the approval request button 200a is displayed in the column for the latest correction. The approval request button 200a is used to request approval from the person who approved the correction. In this way, the notation at the top of the column showing the recognition results, etc., means that when the user checks the read content and the content is corrected at the bottom of the same column, the approval request button 200a for confirming the corrected content is displayed along with the correction history.

[0103] Next, we will explain the display of confidence-based feedback notifications on the confirmation screen using Figures 20 to 22. The confidence-based feedback notification here indicates that the reading process was successful, but the confidence level is low, prompting the user to review the cropped image and make any necessary corrections. Figures 20 and 21 show screen 140, which is displayed when a confirmation screen is shown, and are examples of a feedback notification (Example 1). Figures 20 and 21 are shown in chronological order, and in notification example 1, Figure 21 is displayed after Figure 20. In the screen 140 shown in Figure 20, a warning icon image 100d is displayed for the "Business Partner" in the record item area 100a because the confidence level is lower than a predetermined threshold. In this way, when all three items in the record item area 100a are not complete, the warning icon image 100d is displayed to indicate the missing "Business Partner." Furthermore, for the "Date" and "Amount" fields in the record item area 100a, the confidence level is higher than a predetermined threshold, so the warning icon image 100d is not displayed.

[0104] Furthermore, in screen 140 shown in Figure 20, a notification box 164 is displayed. This notification box 164 displays the message, "The confidence level of the reading result for the search item [Business Partner] is low, so please check the scanned image and make corrections as necessary." Also, an OK button 164a is displayed in the notification box 164. Pressing the OK button 164a transitions to screen 140 shown in Figure 21.

[0105] In the screen 140 shown in Figure 21, when the confirmation check box for "Business Partner" in the record item area 100a is checked, a notification box 165 appears asking, "Do you want to confirm the contents of the search result for [Business Partner]?". Pressing the button 165a displayed in the notification box 165 makes the warning icon image 100d disappear. In addition, although the recording item area 100a and other item area 100c are displayed in screen 140 shown in Figure 21, the display control may be limited to this, and other item areas 100c that are not subject to user operation may not be displayed.

[0106] Figure 22 shows screen 140, which is displayed when a confirmation screen is displayed, and is an example of a feedback notification (Example 2). Screen 140 shown in Figure 22 is displayed after Figure 20, which is an example of a notification (Example 1), and is displayed in place of Figure 21. In the screen 140 shown in Figure 22, as indicated by the dashed frame 100g, the confidence levels for "Date" and "Amount" in the record item area 100a are higher than the predetermined threshold, and the warning icon image 100d is not displayed. However, not only "Customer" but also "Date" and "Amount" in the record item area 100a require user confirmation, and the user must check the confirmation check box.

[0107] On the other hand, for the "Total" item in item area 100c, if the confidence level is higher than a predetermined threshold, the user is not required to check the confirmation box. In other words, as shown in the dashed box 100h, the confirmation box for "Total" is not checked by the user, but is automatically checked when the confidence level is high. Alternatively, instead of such display control, the confirmation check box for "Total" may be hidden, or the confirmation / correction process may be skipped.

[0108] Here, image data 42a in the reduced display area 42, image data 50 in the enlarged display area 43, and image data 150 in the enlarged display area 143 are examples of read images. The three items "Customer," "Date," and "Amount" in the record item area 46a (see, for example, Figure 5), and the three items "Customer," "Date," and "Amount" in the record item area 100a of the result display area 100A (see, for example, Figure 14) are examples of record items that are used as search conditions under the Electronic Bookkeeping Law. In addition, "Total" in the other item areas 46c and 100c (see, for example, Figures 11 and 14) is an example of an item other than a record item. Furthermore, screen 40 (see, for example, Figure 11) and screen 140 (see, for example, Figure 14) are examples of "screens that show items for confirming extracted items." Also, the record item icon images 46d and 146d (see, for example, Figures 5 and 14) are examples of "displaying record items in a way that makes them distinguishable from other items," and are examples of "displaying record items in a way that makes them distinguishable from other items." Furthermore, the warning icon image 100d is an example of "an indication showing that the record item is incomplete" (see, for example, Figure 14), and an example of "notifying the user about the degree of confidence regarding the record item" (see, for example, Figure 20). Furthermore, the correction history 200 (see Figure 19) is an example of a correction history, and the approval request button 200a (see Figure 19) is an example of an element for confirming the corrected content.

[0109] In each of the above embodiments, the term "processor" refers to a processor in a broad sense, and includes general-purpose processors (e.g., CPU: Central Processing Unit, etc.) and dedicated processors (e.g., GPU: Graphics Processing Unit, ASIC: Application Specific Integrated Circuit, FPGA: Field Programmable Gate Array, programmable logic device, etc.). Furthermore, the processor operations in each of the above embodiments may not be performed by a single processor, but may also be performed by multiple processors located in physically separate locations working together. Also, the order of the processor operations is not limited to the order described in each of the above embodiments, and may be changed as appropriate. [Explanation of Symbols]

[0110] 10...Information processing device, 10A...CPU, 11...Acquisition unit, 12...Item extraction unit, 13...Display control unit, 14...Operation reception unit, 40, 140...Screen, 45, 145...Instruction area, 100d...Warning icon image, 46d, 146d...Record item icon image, 200...Correction history, 200a...Approval request button

Claims

1. Equipped with a processor, The aforementioned processor, Obtain the read image, From the acquired scanned image, items containing the record items required for searching under the Electronic Bookkeeping Law are extracted using character recognition. In a screen that displays the extracted items for confirmation and accepts corrections to the character recognition results of those items, the display control is configured to display the recorded items in a manner that allows them to be distinguished from other items that are not the recorded items. An information processing device characterized by the following:

2. comprising a processor, The aforementioned processor, Obtain the read image, From the acquired scanned image, items containing the record items required for searching under the Electronic Bookkeeping Law are extracted using character recognition. On a screen displaying the extracted items for confirmation, the display control is set so that the record item is displayed in a way that allows it to be distinguished from other items that are not the record item. The aforementioned display control is, On the screen displaying the aforementioned items, the display control is configured to show whether or not it is possible to transition from the screen displaying the said item to the next screen, depending on the status of the recorded item. When all the aforementioned recording items are collected, the display control will show that the screen is ready to proceed to the next screen. An information processing device characterized by the following:

3. When outputting the content read in accordance with the item, the recorded item cannot be deleted from the output. The information processing apparatus according to feature 2.

4. comprising a processor, The aforementioned processor, Obtain the read image, From the acquired scanned image, items containing the record items required for searching under the Electronic Bookkeeping Law are extracted using character recognition. On a screen displaying the extracted items for confirmation, the display control is set so that the record item is displayed in a way that allows it to be distinguished from other items that are not the record item. The aforementioned display control is, On the screen displaying the aforementioned items, the display control is configured to show whether or not it is possible to transition from the screen displaying the said item to the next screen, depending on the status of the recorded item. If the aforementioned record items are not complete, the next screen will be displayed in a way that prevents the user from proceeding to the next screen. An information processing device characterized by the following:

5. The display control, when the record items are not all present, controls the display so that the missing record items are shown. The information processing apparatus according to feature 4.

6. comprising a processor, The aforementioned processor, Obtain the read image, From the acquired scanned image, items containing the record items required for searching under the Electronic Bookkeeping Law are extracted using character recognition. On a screen displaying the extracted items for confirmation, the display control is set so that the record item is displayed in a way that allows it to be distinguished from other items that are not the record item. The display control controls the display so that when the user checks the content read in relation to the recorded item, the recorded item and the other items are displayed in a way that allows them to be distinguished. When confirming the contents of the aforementioned record items, the user is asked to confirm the candidates determined by the confidence level of the character recognition results. When confirming the content of the aforementioned other items, the user can omit confirmation for candidates determined by the confidence level, and the user is only required to correct the content. An information processing device characterized by the following:

7. comprising a processor, The aforementioned processor, Obtain the read image, From the acquired scanned image, items containing the record items required for searching under the Electronic Bookkeeping Law are extracted using character recognition. On a screen displaying the extracted items for confirmation, the display control is set so that the record item is displayed in a way that allows it to be distinguished from other items that are not the record item. The display control controls the display so that when the user checks the content read in relation to the recorded item, the recorded item and the other items are displayed in a way that allows them to be distinguished. When checking the contents of the record item, if the confidence level of the character recognition result is less than a predetermined value, the user is notified of the confidence level for that record item. When confirming the contents of the aforementioned other items, even if the confidence level of the character recognition result is lower than a predetermined value, the user will not be notified about the confidence level for that other item. An information processing device characterized by the following:

8. comprising a processor, The aforementioned processor, Obtain the read image, From the acquired scanned image, items containing the record items required for searching under the Electronic Bookkeeping Law are extracted using character recognition. On a screen displaying the extracted items for confirmation, the display control is set so that the record item is displayed in a way that allows it to be distinguished from other items that are not the record item. The display control is characterized in that, when the content read in accordance with the recorded item is corrected, the display control displays elements for confirming the corrected content along with the correction history.

9. comprising a processor, The aforementioned processor, Obtain the read image, From the acquired scanned image, items containing the record items required for searching under the Electronic Bookkeeping Law are extracted using character recognition. On a screen displaying the extracted items for confirmation, the display control is set so that the record item is displayed in a way that allows it to be distinguished from other items that are not the record item. It is possible to switch whether or not to display the aforementioned record item in a manner that allows it to be distinguished from the other items. An information processing device characterized by the following:

10. When the display is not shown in the identifiable state due to the switching, after the contents of the record item and the other items are confirmed, the switch is made to show the display in the identifiable state, While not retaining user operation information for the aforementioned record item, it retains user operation information for the other items. The information processing apparatus according to feature 9.

11. Information processing device, The acquisition function for obtaining the read image, The aforementioned acquisition function extracts items containing record items that are search criteria under the Electronic Bookkeeping Law from the read image using character recognition, A control function is provided to control the display of a screen that shows the items extracted by the extraction function and accepts corrections to the character recognition results of the items, so that the recorded items are displayed in a manner that allows them to be distinguished from other items that are not the recorded items. A program that makes this possible.

12. Information processing device, The acquisition function for obtaining the read image, The aforementioned acquisition function extracts items containing record items that are search criteria under the Electronic Bookkeeping Law from the read image using character recognition, A control function that displays the recorded item in a screen showing the item extracted by the extraction function, such that the recorded item is displayed in a manner that allows it to be distinguished from other items that are not the recorded item. To make it happen, The aforementioned display control is, On the screen displaying the aforementioned items, the display control is configured to show whether or not it is possible to transition from the screen displaying the said item to the next screen, depending on the status of the recorded item. When all the aforementioned recording items are collected, the display control will show that the screen is ready to proceed to the next screen. program.

13. In an information processing device, The acquisition function for obtaining the read image, The aforementioned acquisition function extracts items containing record items that are search criteria under the Electronic Bookkeeping Law from the read image using character recognition, A control function that displays the recorded item in a screen showing the item extracted by the extraction function, such that the recorded item is displayed in a manner that allows it to be distinguished from other items that are not the recorded item. To make it happen, The aforementioned display control is, On the screen displaying the aforementioned items, the display control is configured to show whether or not it is possible to transition from the screen displaying the said item to the next screen, depending on the status of the recorded item. If the aforementioned record items are not complete, the next screen will be displayed in a way that prevents the user from proceeding to the next screen. program.

14. Information processing device, The acquisition function for obtaining the read image, The aforementioned acquisition function extracts items containing record items that are search criteria under the Electronic Bookkeeping Law from the read image using character recognition, A control function that displays the recorded item in a screen showing the item extracted by the extraction function, such that the recorded item is displayed in a manner that allows it to be distinguished from other items that are not the recorded item. To make it happen, The display control controls the display so that when the user checks the content read in relation to the recorded item, the recorded item and the other items are displayed in a way that allows them to be distinguished. When confirming the contents of the aforementioned record items, the user is asked to confirm the candidates determined by the confidence level of the character recognition results. When confirming the content of the aforementioned other items, the user can omit confirmation for candidates determined by the confidence level, and the user is only required to correct the content. program.

15. Information processing device, The acquisition function for obtaining the read image, The aforementioned acquisition function extracts items containing record items that are search criteria under the Electronic Bookkeeping Law from the read image using character recognition, A control function that displays the recorded item in a screen showing the item extracted by the extraction function, such that the recorded item is displayed in a manner that allows it to be distinguished from other items that are not the recorded item. To make it happen, The display control controls the display so that when the user checks the content read in relation to the recorded item, the recorded item and the other items are displayed in a way that allows them to be distinguished. When checking the contents of the record item, if the confidence level of the character recognition result is less than a predetermined value, the user is notified of the confidence level for that record item. When confirming the contents of the aforementioned other items, even if the confidence level of the character recognition result is lower than a predetermined value, the user will not be notified about the confidence level for that other item. program.

16. Information processing device, The acquisition function for obtaining the read image, The aforementioned acquisition function extracts items containing record items that are search criteria under the Electronic Bookkeeping Law from the read image using character recognition, A control function that displays the recorded item in a screen showing the item extracted by the extraction function, such that the recorded item is displayed in a manner that allows it to be distinguished from other items that are not the recorded item. To make it happen, The display control is a program that, when confirming the content read in relation to the recorded item, displays elements for confirming the corrected content along with the correction history if the content is corrected.

17. Information processing device, The acquisition function for obtaining the read image, The aforementioned acquisition function extracts items containing record items that are search criteria under the Electronic Bookkeeping Law from the read image using character recognition, A control function that displays the recorded item in a screen showing the item extracted by the extraction function, such that the recorded item is displayed in a manner that allows it to be distinguished from other items that are not the recorded item. To make it happen, It is possible to switch whether or not to display the aforementioned record item in a manner that allows it to be distinguished from the other items. program.

Images