Information processing device and program

Abstract

To improve user operability compared to a case where a selection screen and a comparison screen are provided and an operation to make a screen transition between the selection screen and the comparison screen is required.SOLUTION: An information processing apparatus 10 includes a processor 10A. The processor 10A acquires a plurality of first document images, performs classification processing for each of the acquired plurality of first document images based on a certainty factor with a plurality of second document images predetermined for each classification destination, causes a first group which is a group of the plurality of first document images and a second group which is a group of the plurality of second document images to be displayed on a same screen, and performs display control such that a result of the classification processing is displayed on the same screen.SELECTED DRAWING: Figure 2

Description

Technical Field

[0001] The present invention relates to an information processing apparatus and a program.

Background Art

[0002] For example, Patent Document 1 discloses an apparatus that discriminates the format of a read document image and sorts it for each document format. Based on the similarity between the document image for which the format is to be discriminated and a registered document image whose document format is known, a discrimination means for discriminating the format of the document image for which the format is to be discriminated, and a determination information for determining whether confirmation by the user is required for the result of the discrimination by the discrimination means is held in association with the document image for which the format has been discriminated. A user interface means for presenting a document image determined to require confirmation by the user based on the determination information held by the holding means and prompting confirmation and correction of the sorting is disclosed.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] Here, when classifying a document image to be classified based on a predetermined document image and the user classifies an unclassified document image or checks the classification of a classified document image, a selection operation is performed on a selection screen for selecting one unclassified or classified document target, and when transitioning to a comparison screen for comparing the selected document target with a predetermined document image, an operation for performing a screen transition between the selection screen and the comparison screen is required, making it difficult to improve the operability of the user. The objective of the present invention is to improve user operability compared to cases where a selection screen and a comparison screen are provided, requiring user interaction to transition between the selection screen and the comparison screen. [Means for solving the problem]

[0005] The invention described in claim 1 comprises a processor, the processor acquires a plurality of first document images, performs classification processing on each of the acquired plurality of first document images based on the confidence level with a plurality of second document images predetermined for each classification destination, displays a first group consisting of the plurality of first document images and a second group consisting of the plurality of second document images on the same screen, and displays the results of the classification processing on the same screen using display control. The display control is performed in such a way that it is possible to specify a first document image from any of the displayed first group and a second document image from any of the second group, the specified first document image is classified to the same category as the specified second document image, and the second document images from the second group are displayed in a predetermined order, the predetermined order being either the order of frequency of occurrence based on the results of the classification process or the order of confidence level of the specified first document image, if the order of frequency of occurrence based on the results of the classification process is selected as the predetermined order, the order of the second document images does not change even if the specified first document image is different, and if the order of confidence level of the specified first document image is selected as the predetermined order, the order of the second document images changes according to the specified first document image. This is an information processing device characterized by the following features. The invention described in claim 2 is, The first document image specified above is one in which the classification process did not result in it being classified into any of the classification destinations. The information processing device is as described in claim 1. The invention described in claim 3 is such that the designated first document image is such that the result of the classification process is The second document image is classified into a different classification category than the one specified above. It is characterized by 1 This is the information processing device described above. The invention described in claim 4 is, The predetermined sorting order is determined by the frequency of occurrence based on the results of the classification process. is characterized by 1 This is the information processing device described above. The invention described in claim 5 is, The predetermined sorting order is such that the second document image with a higher confidence level for the designated first document image is displayed higher in the order of the items. The information processing device is as described in claim 1. The invention described in claim 6 is the same as Image of the second document The order is, The aforementioned high degree of confidence In order, features 5 This is the information processing device described above. The invention described in claim 7 is, The number of first document images displayed on the same screen is reduced based on the results of the classification process. , features 1 This is the information processing device described above. The invention described in claim 8 is, The information processing device includes an acquisition function for acquiring multiple first document images, a processing function for classifying each of the multiple first document images acquired by the acquisition function based on the confidence level with a predetermined number of second document images for each classification destination, and a display control function for displaying a first group consisting of the multiple first document images and a second group consisting of the multiple second document images on the same screen, and displaying the results of the classification process on the same screen. The display control is performed so that it is possible to specify any of the first document images in the displayed first group and any of the second document images in the second group, and the specified first document image is... The program assigns the same classification destination as the specified second document image and displays the second document images of the second group in a predetermined order. The predetermined order is selected from either the order of frequency of occurrence based on the results of the classification process or the order of confidence level of the specified first document image. If the order of frequency of occurrence based on the results of the classification process is selected as the predetermined order, the order of the second document images does not change even if the specified first document image is different. If the order of confidence level of the specified first document image is selected as the predetermined order, the order of the second document images changes according to the specified first document image. That is the case. [Effects of the Invention]

[0006] According to claim 1, by providing a selection screen and a comparison screen, user operability can be improved compared to cases where an operation is required to transition between the selection screen and the comparison screen. 。 Claim 2 According to this, compared to the case where display control is not performed to classify the specified first document image to the same category as the specified second document image, correction operations for unclassified documents can be performed more easily. Claim 3 According to this, compared to the case where display control is not performed to classify the specified first document image as the same as the specified second document image, correction operations for misclassification can be made easier. 。 Claim 4 According to this, compared to the case where display control is not performed to show the second document images of the second group in order of frequency of occurrence based on the results of classification processing, user operability can be improved. Claim 5 According to this, compared to a case where display control is not implemented to show the second document image with a higher confidence level relative to the specified first document image at the top, the work efficiency of verifying classification results can be improved. Claim 6 According to this, compared to the case where display control is not implemented to sort the second document images in order of highest confidence level, the work efficiency of verifying classification results can be improved. Claim 7 According to this, compared to a case where display control is not performed to reduce the number of first document images displayed on the same screen based on the classification processing results, the work efficiency of confirming classification results can be improved. Claim 8 According to this, by providing both a selection screen and a comparison screen, it is possible to improve user experience compared to cases where users need to perform an operation to transition between the selection screen and the comparison screen. [Brief explanation of the drawing]

[0007] [Figure 1] This diagram shows the schematic configuration of the reporting system. [Figure 2] This is a functional block diagram of an information processing device. [Figure 3] It is a flowchart showing the process of invoice discrimination confirmation and correction according to this embodiment. [Figure 4] It is a diagram showing a screen when the classification target image group, the pre-registered image group, and the result of the classification process are displayed on one screen. [Figure 5] It is a diagram showing a screen when correcting the result of the classification process, and is shown in chronological order in the order of (a) and (b). [Figure 6] It is a diagram showing a screen when correcting the result of the classification process, and is shown in chronological order in the order of (a) and (b). [Figure 7] It is a diagram showing a screen when correcting the result of the classification process, and is shown in chronological order in the order of (a) and (b). [Figure 8] It is a diagram showing a screen when the order of the pre-registered image group is shown by confidence level. (a) illustrates the order of the pre-registered image display area when the classification target image is in the selected state, and (b) illustrates the order of the pre-registered image display area when another classification target image is in the selected state. [Figure 9] It is a diagram showing a screen when the order of the pre-registered image group is shown by the appearance frequency. (a) shows the case when the classification target image is in the selected state, and (b) shows the case when another classification target image is in the selected state. [Figure 10] It is a diagram showing a screen when the order of the pre-registered image group is changed by the user. (a) is the normal case, (b) is the case of confidence level, and (c) is the case of appearance frequency. [Figure 11] It is a diagram showing a screen when the order of the classification target image group is changed by the user. (a) is all cases, and (b) is the case of invoice A. [Figure 12] It is a diagram showing a screen when the order of the classification target image group is changed by the user. (a) is the case of invoice B, and (b) is the case of unknown.

Embodiments for Carrying Out the Invention

[0008] Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings. Figure 1 shows a schematic configuration of the document system 100. The document system 100 shown in the figure includes an information processing device 10. The information processing device 10 is connected to a client terminal 20 and an input device 30 so that they can communicate with each other. The network connecting the information processing device 10, the client terminal 20, and the input device 30 may be, for example, a LAN (Local Area Network) or the Internet. However, the network may also be a hybrid configuration of LAN and Internet.

[0009] The information processing device 10 manages a series of processing steps, including performing OCR (Optical Character Recognition) on image data of documents containing multiple forms, which are input through the input device 30, and outputting the results of the OCR processing to a predetermined output destination. The specific configuration and operation of the information processing device 10 will be described later.

[0010] The client terminal 20 transmits various instructions related to OCR processing to the information processing device 10. These instructions include, for example, an instruction to start reading the information in the image data and an instruction to display the results of reading the information in the image data. The client terminal 20 also displays the results of the OCR processing performed by the information processing device 10 in response to the received instructions, as well as various information such as notifications related to the OCR processing. As an example, the client terminal 20 may be a server device or a general-purpose computer device such as a PC (Personal Computer). Although Figure 1 shows only one client terminal 20, it is not limited to this, and multiple client terminals 20 may be provided, and as an example, different client terminals 20 may be used for different processes.

[0011] The input device 30 inputs image data to be processed by OCR into the information processing device 10. The input device 30 may include, for example, a server or a general-purpose computer device such as a PC, as well as an image forming apparatus having scanning, printing, and faxing functions. In addition to the input device 30, image data may also be input to the information processing device 10 from the client terminal 20.

[0012] Next, we will explain the overview of the reporting system 100. The document system 100 is a system in which the information processing device 10 performs OCR processing on image data input through the input device 30 and outputs the results of the OCR processing to a predetermined output destination.

[0013] The information processing device 10 manages various processes in OCR processing, including (1) business design and operation verification, (2) data input, (3) data reading, (4) form identification, confirmation and correction, (5) reading result confirmation and correction, (6) business check, (7) data output, and (8) return. In this embodiment, OCR processing refers not only to the process of reading characters and symbols from image data, but also to post-processing such as character correction.

[0014] As an example of the management of various processes, (1) business design and operation verification, (2) data input, (3) data reading, (6) business check, and (7) data output are each automatically executed by the information processing device 10. Also as an example of the management of various processes, (4) form identification, confirmation, and correction, and (5) reading result confirmation and correction are each accepted by user input via the client terminal 20. Furthermore, as an example of the management of various processes, (8) the return process may be automatically executed by the information processing device 10 or accepted by user input via the client terminal 20.

[0015] (1) In the process of business design and operational verification, a job rule is created that includes read definitions, output settings, and business check settings. In the read definition, as an example, the read range is set, which is the range in which the information of the image data is read in the process of "(3) Data Reading". More specifically, as an example, a definition may be set to read the item value, which is the value, from the vicinity to the right of the item that is extracted as the key. In the output settings, as an example, the file format and output destination of the output data to be output in the process of "(7) Data Output" are set. In the business check settings, as an example, the required input items and format such as the number of characters that can be entered in the form to be detected in the process of "(6) Business Check" are set.

[0016] (2) In the data input process, image data is received from the input device 30. For example, this can be uploaded from a local PC via a browser, linked with a cloud storage service, or entered into a hot folder on a PC. The image data received as input is registered as a job, which is the execution unit of the "(3) Data Reading" process.

[0017] (3) In the data reading process, the information of the image data within the job is read using the job rules selected by the user from among the job rules created in the process of "(1) Business Design and Operation Verification". For example, in this process, the documents contained in the image data within the job are identified (hereinafter referred to as "document identification") and the characters and symbols within the reading range are read.

[0018] (4) In the process of form identification, confirmation, and correction, the image data within the job is classified into records that indicate the forms included in the job, based on the results of the form identification performed in the process of "(3) Data Reading". Subsequently, this process displays the classified records and accepts confirmation and correction of the form identification by the user. Alternatively, the results may be divided into individual forms. Such a job may be a collection of multiple document images, each containing only images of the same type of document. However, for the purposes of this explanation, it will be described as a collection of document images, each containing multiple sets of application forms, each consisting of multiple types of documents. In other words, a job contains multiple types of document images. A collection of multiple types of document images included in one set is called a "record."

[0019] (5) In the process of confirming and correcting the reading results, the reading results of characters and symbols within the reading range performed in the process of "(3) Data Reading" are displayed, and corrections or rollbacks are made to the correct state. In addition, the corrected results received may be compared with a predetermined database, and the converted data may be displayed on the screen to reduce the effort required for confirmation and correction and to provide a function to detect errors in advance.

[0020] (6) In the business process check, errors in each preceding process are detected by the business check settings included in the job rule for the job in question, which is selected by the user from among the job rules created in the process of "(1) Business Design and Operational Verification". The detection results may be presented to the user. This allows the user to make corrections or send the changes back.

[0021] (7) In the data output process, output data is created using the output settings included in the job rule for the job selected by the user from among the job rules created in the process of "(1) Business design and operation verification," and the created output data is output to a predetermined output destination.

[0022] (8) In the rollback process, the OCR process is rolled back to a process one or more steps prior to the process that was executed. Rollback is instructed by the user from the client terminal 20 during the execution of various processes such as "(4) Form Identification Confirmation and Correction" and "(5) Reading Result Confirmation and Correction". Rollback is also instructed by the administrator from the administrator's client terminal 20, for example, based on the results of an administrator check performed between the processes of "(6) Business Check" and "(7) Data Output".

[0023] In the OCR processing described above, the "(1) Business Design and Operational Verification" process is executed before the "(3) Data Reading" and subsequent processes are performed, that is, before the reporting system 100 is put into operation. Furthermore, the "(1) Business Design and Operational Verification" process may also be executed while the reporting system 100 is in operation and the "(3) Data Reading" and subsequent processes are being performed. As an example, the job rules created in the "(1) Business Design and Operational Verification" process before the reporting system 100 is put into operation can be modified according to the results of the "(5) Reading Result Confirmation and Correction" process during the operation of the reporting system 100.

[0024] Here, in the "(4) Document Identification, Confirmation, and Correction" of the document system 100, for example, after receiving a type of unclassified document, the user has to return to the list of unclassified documents, select the document to be classified, transition to the comparison screen with the pre-registered image, classify the document, and then return to the list of unclassified documents again to select the unclassified document. This operation of returning to the list of unclassified documents each time is cumbersome for the user. The same operation applies to misclassified documents as well. Therefore, in this embodiment, images of unclassified or misclassified forms and pre-registered images are displayed on the same screen, and the results of the classification process or sorting are also displayed on the same screen, thereby reducing the number of screen transitions and minimizing the user's operational effort. The following provides a detailed explanation.

[0025] Figure 2 is a functional block diagram of the information processing device 10. As shown in the figure, the information processing device 10 is configured to include an acquisition unit 11, a classification processing unit 12, a display control unit 13, and a result correction unit 14.

[0026] The acquisition unit 11 acquires the image data received in "(2) Data Input" described above. That is, the acquisition unit 11 acquires the classification target image, which is image data to be classified, from the input device 30. Furthermore, the acquisition unit 11 acquires pre-registered images from the input device 30, which are image data predetermined for each classification destination and registered in advance to identify the classification destination. The images to be classified and the pre-registered images are document images, and once acquired by the acquisition unit 11, they are stored in a storage unit such as an HDD (Hard Disk Drive) (not shown).

[0027] The classification processing unit 12 performs the form discrimination process on a record-by-record basis in the "(3) Data Reading" described above. That is, for each image to be classified, it performs classification based on the confidence level with a pre-registered image. In this way, the results of the classification process are associated with each image to be classified. To calculate such confidence levels, known algorithms can be applied, such as pattern matching methods using feature vectors or templates, and methods using neural networks trained with machine learning to output confidence levels based on the images to be compared.

[0028] The display control unit 13 controls the display of the results of the classification process on the screen in the "(4) Form Identification, Confirmation, and Correction" described above. In other words, the display control unit 13 displays a screen for the user to confirm and correct the results of the classification process. Details will be described later.

[0029] The result correction unit 14 receives corrections to the results of the classification process performed by the user in "(4) Form Identification Confirmation Correction" described above. The received corrections are reflected on the screen by the display control unit 13. Details will be described later.

[0030] Here, each function of the information processing device 10 is realized by a CPU 10A, which is an example of a processor. The CPU 10A reads a program stored in the ROM (Read Only Memory) 10B and executes the program using the RAM (Random Access Memory) 10C as the working area. The program executed by the CPU 10A can be provided to the information processing device 10 while stored in a computer-readable recording medium such as a magnetic recording medium (magnetic tape, magnetic disk, etc.), an optical recording medium (optical disk, etc.), a magneto-optical recording medium, or semiconductor memory. Alternatively, the program executed by the CPU 10A may be downloaded to the information processing device 10 using a communication means such as the Internet.

[0031] Furthermore, in this embodiment, each function of the information processing device 10 is implemented by software, but it is not limited to this, and may also be implemented by an ASIC (Application Specific IC).

[0032] Figure 3 is a flowchart showing the process of document identification, confirmation, and correction according to this embodiment, and shows an example of the process of "(4) Document Identification, Confirmation, and Correction" using the results of the classification process by "(3) Data Reading" in Figure 1 described above. In the processing example shown in the figure, the images to be classified, which have been classified in "(3) Data Reading," and the pre-registered images used in the classification process are displayed on the same screen along with the classification results (step 101). This display on the same screen is performed by the display control unit 13 (see Figure 2). Step 102 determines whether the user has made any corrections to the display on the same screen. If it is determined that a correction has been made (Yes in Step 102), the display control unit 13 changes the display (Step 103), and the result correction unit 14 accepts the correction to the result of the classification process (Step 104). If it is determined that no correction has been made (No in Step 102), the process ends.

[0033] Next, the screen 40 of the display unit (not shown) of the information processing device 10 will be described. Figure 4 shows screen 40, which displays the group of images to be classified, the pre-registered group of images, and the results of the classification process on a single screen. This is an example screen corresponding to step 101 (see Figure 3). The screen 40 shown in the figure includes a classification target display area 41 where the image to be classified is displayed, a pre-registered image display area 42 where the pre-registered image is displayed, a result display area 43 where the result of the classification process is displayed, and an instruction area 44 that instructs the user to proceed to the next screen.

[0034] Furthermore, the screen 40 displayed on one screen may consist of a single display unit or display, or it may be composed of multiple displays or displays split into a single screen. Therefore, the concept of "same screen" includes both cases where it is composed of a single display and cases where it is composed of multiple displays.

[0035] The classification target display area 41 is an area where a group of classification target images, arranged in a sequence, is displayed. More specifically, in the classification target display area 41 of Figure 4, the classification target images 41a, 41b, 41c, 41d, 41e, and 41f are displayed as a group of classification target images. Of the images 41a to 41f to be classified, only image 41a is displayed with a thick line, indicating that it is selected. You can pre-set which image should be selected; for example, you could set it to be the leftmost and topmost image.

[0036] Furthermore, the results of the classification process are displayed in the classification destinations 43a, 43b, and 43c, respectively, within the results display area 43, corresponding to the classification target images 41a to 41c. In other words, it shows that the classification target images 41a and 41b are classified into form A, and the classification target image 41c is classified into form B. Although not shown in Figure 4, scrolling down will display the results of each classification process in the corresponding result display area 43 for the classification target images 41d to 41f.

[0037] The results of the classification process displayed in the results display area 43 are reviewed by the user and modified as necessary. In addition to cases where the classification destination is specifically displayed, the classification destination may also be unknown. In cases where the destination is unknown, the user can either classify it or return it for revision.

[0038] The pre-registered image display area 42 is an area where a group of pre-registered images arranged in a grid is displayed. More specifically, in the pre-registered image display area 42 of Figure 4, pre-registered images 42a and 42b are displayed as a group of pre-registered images, and the user can understand that pre-registered image 42a is pre-registered as form A from the text "Form A" located below pre-registered image 42a. Pre-registered image 42b is pre-registered as a form other than form A, for example, form B. Forms A and B, which are classified as such, are documents with predetermined formats, such as invoices, quotations, and travel expense claims.

[0039] In this embodiment, the classified image group and the pre-registered image group are displayed on screen 40, and the results of the classification process are also displayed on screen 40, thereby reducing the number of screen transitions and minimizing the user's operational effort.

[0040] To explain further, the classification target display area 41 is a larger area than the pre-registered image display area 42. More specifically, in the classification target display area 41, the classification target images 41a to 41f are arranged in three columns, while in the pre-registered image display area 42, the pre-registered images 42a to 42b are arranged in one column. This is because, generally, the number of images to be classified displayed in the classification target display area 41 is greater than the number of pre-registered images displayed in the pre-registered image display area 42. The display control unit 13 (see Figure 2) prioritizes displaying whichever of the classification target images or pre-registered images is more numerous. Therefore, if the number of images to be classified is less than the number of pre-registered images, the classification target display area 41 will be displayed in a smaller area than the pre-registered image display area 42. In addition to automatically determining the size of the classification target display area 41 and the pre-registered image display area 42, the size may also be determined according to user settings.

[0041] Here, the images 41a to 41f to be classified are examples of first document images, and the group of images to be classified is an example of the first group. The pre-registered images 42a and 42b are examples of second document images, and the group of pre-registered images is an example of the second group. The classification destinations 43a to 43c are examples of the results of the classification process. The pre-registered image group may be displayed in a predetermined order, such as by frequency of appearance or confidence level. Furthermore, the image group to be classified may be shown or hidden based on the results of filtering the images to be classified displayed in the classification target display area 41.

[0042] Next, we will explain how to correct the results of the classification process. Figures 5 and 6 show cases where the classification destination has not been determined and the data is unclassified, while Figure 7 shows cases where the classification destination is incorrect and the data is misclassified. In some cases, illustrations and explanations of common elements may be omitted.

[0043] Figures 5 and 6 show screen 40 when correcting the results of the classification process. They are shown in chronological order, with (a) and (b) of each figure also shown in chronological order. In the classification target display area 41 of screen 40 shown in Figure 5(a), the classification result of the image to be classified 41a is "Unknown," as shown in the classification destination 43a of the result display area 43. In other words, the image to be classified 41a was not classified into any of the classification destinations displayed in the pre-registered image display area 42.

[0044] To address this, as shown in Figure 5(b), when the user places the cursor 45 over the image 41a to be classified in the classification target display area 41 and clicks, the display changes to indicate that the image 41a to be classified is selected. Specifically, the display control unit 13 (see Figure 2) displays the image 41a to be classified in a manner that indicates it is selected, by using a thicker border than the others. Note that Figure 5(b) corresponds to the "display change" performed as "correction operation performed" in the flowchart shown in Figure 3 (Yes in step 102, see step 103).

[0045] The user then specifies the classification destination for the image 41a that is selected and whose classification destination is "unknown". That is, the user selects one of the pre-registered images, such as the pre-registered image 42a, in the pre-registered image display area 42. In the example shown in Figure 6(a), the cursor 45 is placed over the pre-registered image 42a of "Form A" and clicked.

[0046] As a result, as shown in Figure 6(b), the classification destination 43a in the result display area 43 is changed to "Document A," and the classification destination of the image 41a, which was previously "Unknown," is changed to "Document A." Note that Figure 6(b) corresponds to "Correction Acceptance" in the flowchart shown in Figure 3 (see Step 104). To add to that, if the user wants to proceed to the next screen, they move the cursor 45 to the instruction area 44 and click.

[0047] Figure 7 shows screen 40 when correcting the results of the classification process, and is shown chronologically in the order of (a) and (b). Figure 7(a) corresponds to Figure 5(a), and Figure 7(b) corresponds to Figure 6(b). In the screen 40 shown in Figure 7(a), the classification result of the image to be classified 41a is "Form B". When the user confirms that the image to be classified 41a is "Form A" and not "Form B", the image to be classified 41a is selected, as explained using Figure 5(b) above.

[0048] Subsequently, as explained above and using Figure 6(a), when the cursor 45 is moved to the pre-registered image 42a of "Form A" and clicked, "Form A" is displayed in the classification destination 43a of the result display area 43, as shown in Figure 7(b). Then, by moving the cursor 45 to the instruction area 44 and clicking, the classification destination of the image to be classified 41a is changed to "Form A".

[0049] Thus, on screen 40 where the results of the classification process are displayed in the results display area 43, the user performs a verification and correction operation by comparing the classification results for each image to be classified in the classification target display area 41 with the pre-registered images to check whether the classification destination is correct, or to indicate the classification destination if there are any unclassified images. Since this operation can be performed without changing screens, the number of screen transitions is reduced, and the effort required from the user is lessened.

[0050] Next, we will explain how the pre-registered image display area 42 is displayed when the user checks the results of the classification process. Figure 8 shows the case where the pre-registered images 42a etc. in the pre-registered image display area 42 are sorted according to the level of confidence, and Figure 9 shows the case where they are sorted according to the level of frequency of occurrence. Figure 10 shows the case where the sorting is set by the user.

[0051] Here, the pre-registered images are displayed in a predetermined order. In this embodiment, the user can choose to display them either in order of high confidence level or in order of frequency of appearance. The confidence level referred to here is a measure indicating the degree of agreement between the selected image to be classified and the pre-registered image. The pre-registered image with the highest confidence level is designated as the classification target as a result of the classification process. If the difference between the highest confidence level and the next highest confidence level is smaller than a predetermined threshold, the result may be classified as "unknown." Furthermore, the frequency of appearance referred to here is the total number of classification targets displayed in the classification target display area 41, categorized by classification destination. Classification destinations with a larger number are considered to have a higher frequency of appearance. Note that the total for each classification destination may include not only the results of classification processing of the currently displayed classification target, but also the results of classification processing of classification target images that have been displayed in the past.

[0052] Figure 8 shows screen 40 when the order of pre-registered images is shown by confidence level, where (a) is an example of the order of pre-registered images in the pre-registered image display area 42 when the classifiable target image 41a in the classification target display area 41 is selected, and (b) is an example of the order of pre-registered images in the pre-registered image display area 42 when another classifiable target image 41b is selected. As shown in Figure 8(a), when the classification target image 41a in the classification target display area 41 is selected, "Document A," which is considered to have the highest confidence level, is displayed at the top. On the other hand, as shown in Figure 8(b), when the classification target image 41b is selected, "Document B," which is considered to have the highest confidence level, is displayed at the top.

[0053] Thus, when the first image to be classified, 41a, shown in Figure 8(a), is selected, the pre-registered images in the pre-registered image display area 42 are arranged in order from high to low confidence for the first image to be classified, 41a. Furthermore, when the second image to be classified, 41b, shown in Figure 8(b), is selected, the pre-registered images are rearranged in order from high to low confidence for the second image to be classified, 41b. This allows users to first view the pre-registered image with the highest confidence level for the selected image to be classified, thereby streamlining the review process by reducing the need for scrolling.

[0054] In this context, "displaying at the top" refers to a position where the user can see it without scrolling. To further explain, this is not limited to the case shown in Figure 8 where images are "displayed at the top" by rearranging them, but may also apply to cases where the pre-registered images with the highest confidence level are displayed preferentially in the pre-registered image display area 42 without rearranging the pre-registered image group which is arranged from top to bottom, so that the user can see it without scrolling.

[0055] Figure 9 shows screen 40 when the order of pre-registered images is shown by frequency of appearance, where (a) shows the case when the classification target image 41a in the classification target display area 41 is selected, and (b) shows the case when another classification target image 41b is selected. As shown in Figure 9(a), when the classification target image 41a in the classification target display area 41 is selected, "Form A," which has the highest frequency of appearance, is displayed at the top. Similarly, as shown in Figure 9(b), when the classification target image 41b in the classification target display area 41 is selected, "Form A" is also displayed at the top. In other words, in the case of Figure 9, unlike in the case of Figure 8, the order of pre-registered images does not change depending on the classification target image selected. This allows users to first see the pre-registered image with the highest frequency of appearance among the selected images to be classified, thereby improving the efficiency of the review process by reducing the need for scrolling.

[0056] Figure 10 shows screen 40 when the user changes the order of the pre-registered image group, with (a) being the normal case, (b) being the frequency of occurrence case, and (c) being the confidence level case. As shown in Figures (a) to (c), the sorting change specification section 46 is located above the pre-registered image display area 42. This sorting change specification section 46 is a drop-down menu that allows the user to select one of the following items: the normal item 46a shown in (a), the frequency item 46b shown in (b), or the confidence item 46c shown in (c). Note that in Figure 10, the group of images to be classified in the classification target display area 41 is not changed, so only a portion of the classification target display area 41 is shown.

[0057] Therefore, when the sorting change specification unit 46 is set to normal 46a, the display is controlled to match the order at the time of pre-registration, resulting in the order of the pre-registered images shown in Figure (a). Also, when the frequency item 46b is set, the display is controlled to match the order from highest to lowest frequency, resulting in the order of the frequency images shown in Figure (b). Furthermore, when the confidence level item 46c is set, the display is controlled to match the order from highest to lowest confidence level, resulting in the order of the confidence level images shown in Figure (c). Furthermore, the order change specified by the order change specification unit 46 is applied to all of the pre-registered images displayed in the pre-registered image display area 42.

[0058] Next, we will explain how the classification target display area 41 is displayed when the user checks the results of the classification process. Figures 11 and 12 show the screen 40 when the user changes the order of the images to be classified. Figure 11(a) shows all cases, (b) shows the case for form A, Figure 12(a) shows the case for form B, and (b) shows the case where it is unknown. Note that in Figures 11 and 12, the pre-registered images in the pre-registered image display area 42 are not changed, so only a portion of the pre-registered image display area 42 is shown. As shown in Figures 11 and 12, the extraction specification unit 47 is located above the classification target display area 41. This extraction specification unit 47 is a dropdown menu that allows the user to select one of the following items: all items 47a in Figure 11(a), form A item 47b in Figure 11(b), form B item 47c in Figure 12(a), or unknown item 47d in Figure 12(b).

[0059] As shown in Figure 11(a), when the extraction specification unit 47 is set to all items 47a, all images in the image group to be classified are displayed in the classification target display area 41. In other words, the images to be classified are not filtered. Also, as shown in Figure 11(b), when the extraction specification unit 47 is set to form A item 47b, the images in the image group to be classified that are designated as form A are filtered and displayed in the classification target display area 41.

[0060] Furthermore, as shown in Figure 12(a), when the extraction specification unit 47 is form B item 47c, the images to be classified from the group of images to be classified that are designated as form B are filtered and displayed in the classification target display area 41. Similarly, as shown in Figure 12(b), when the extraction specification unit 47 is unknown item 47d, the images to be classified that are designated as unknown are filtered and displayed in the classification target display area 41.

[0061] In this way, the number of images to be classified displayed in the classification target display area 41 can be reduced by the extraction specification unit 47. That is, if all of the extraction specification unit 47 are items 47a, all images to be classified are displayed, but if form A item 47b, form B item 47c, or unknown item 47d is selected, the displayed images to be classified are filtered, and the number displayed is reduced. This improves the efficiency of checking the results of the classification process.

[0062] In each of the above embodiments, the term "processor" refers to a processor in a broad sense, and includes general-purpose processors (e.g., CPU: Central Processing Unit, etc.) and dedicated processors (e.g., GPU: Graphics Processing Unit, ASIC: Application Specific Integrated Circuit, FPGA: Field Programmable Gate Array, programmable logic device, etc.). Furthermore, the operation of the processor in each of the above embodiments may not be performed by a single processor, but may be performed by multiple processors located in physically separate locations working together. Also, the order of the processor operations is not limited to the order described in each of the above embodiments, and may be changed. [Explanation of symbols]

[0063] 10...Information processing device, 10A...CPU, 40...Screen, 41a, 41b, 41c, 41d, 41e, 41f...Images to be classified, 42a, 42b...Pre-registered images, 43a~43c...Classification destinations

Claims

1. Equipped with a processor, The aforementioned processor, Acquire multiple images of the first document, For each of the acquired multiple first document images, a classification process is performed based on the confidence level with a predetermined set of multiple second document images for each classification destination. The first group, consisting of the plurality of first document images, and the second group, consisting of the plurality of second document images, are displayed on the same screen, and the display control is performed to display the results of the classification process on the same screen. The aforementioned display control is, The system is configured to allow the selection of a first document image from any of the first group and a second document image from any of the second group that are displayed. The specified first document image will be classified to the same category as the specified second document image. The second document images of the second group are displayed in a predetermined order. The predetermined sorting order is one of the following selected from the order of frequency of occurrence based on the results of the classification process and the order of confidence level of the designated first document image. If the order of occurrence frequency based on the results of the classification process is selected as the predetermined sorting order, the sorting order of the second document images will not change even if the specified first document image is different. If the order of confidence levels of the designated first document images is selected as the predetermined order, the order of the second document images will change according to the designated first document images. An information processing device characterized by the following:

2. The specified first document image is one in which the result of the classification process was not classified into any of the classification destinations. The information processing apparatus according to feature 1.

3. The first document image specified above is one in which the result of the classification process is classified to a different classification destination than that of the second document image specified above. The information processing apparatus according to feature 1.

4. The predetermined sorting order is in order of frequency of occurrence based on the results of the classification process. The information processing apparatus according to feature 1.

5. The predetermined sorting order is such that the second document image with a high degree of confidence in the designated first document image is displayed higher in the order. The information processing apparatus according to feature 1.

6. The order in which the second document images are arranged is in descending order of confidence level. The information processing apparatus according to feature 5.

7. The information processing apparatus according to claim 1, characterized in that the number of first document images displayed on the same screen is reduced based on the results of the classification process.

8. Information processing device, A function to acquire multiple images from the first document, A processing function that performs classification processing on each of the multiple first document images acquired by the acquisition function based on the confidence level with a plurality of second document images predetermined for each classification destination, A control function that displays the first group, consisting of a plurality of first document images, and the second group, consisting of a plurality of second document images, on the same screen, and displays the results of the classification process on the same screen. To make it happen, The aforementioned display control is, The system is configured to allow the selection of a first document image from any of the first group and a second document image from any of the second group that are displayed. The specified first document image will be classified to the same category as the specified second document image. The second document images of the second group are displayed in a predetermined order. The predetermined sorting order is one of the following selected from the order of frequency of occurrence based on the results of the classification process and the order of confidence level of the designated first document image. If the order of occurrence frequency based on the results of the classification process is selected as the predetermined sorting order, the sorting order of the second document images will not change even if the specified first document image is different. If the order of confidence levels of the designated first document images is selected as the predetermined order, the order of the second document images will change according to the designated first document images. program.

Images