Information processing apparatus, method, system, computer readable medium, and computer program product

Abstract

The present application provides an information processing device, an information processing method, an information processing system, and a computer readable medium. The information processing device has a processor that performs processing of displaying, in character information recognized by reading a plurality of forms respectively, the character information in order from more to less or in order from less to more of the number of the character information recognized as a common character.

Description

Technical Field

[0001] This disclosure relates to information processing apparatus, methods, systems, computer-readable media, and computer program products. Background Technology

[0002] Japanese Patent No. 4299977 discloses a recognition result correction device in a character recognition apparatus for improving the efficiency of correction work when inputting a large number of forms. Specifically, the recognition result correction device inputs multiple forms containing one or more recognition items as image data. It recognizes the characters in each recognition item of the input image, maps the recognition result and accuracy information of the characters contained in each recognition item to these recognition items, and stores this information in a result storage unit for each form. It calculates at least one of the minimum and average values ​​of the accuracy information of each character stored in the result storage unit as a representative value for each recognition item within each form, and determines the order of forms to be corrected by sorting the representative values ​​of the recognition items, considering the priority order of correcting the recognition results of each recognition item within the form as the priority order of sorting keywords.

[0003] Japanese Patent Application Publication No. 2020-046734 discloses an information processing apparatus that reduces the labor and time of operators compared to the situation where an operator searches for all corrections to the recognition results of information recorded in a recording area. Specifically, the information processing apparatus receives an extraction area image representing an extraction area containing information recorded by handwriting by the operator, and displays another extraction area image similar to the extraction area image when a correction instruction has been given for the recognition results of the information recorded in the recording area represented in the extraction area image.

[0004] Japanese Patent Application Publication No. 2011-060185 discloses a data input system capable of efficiently performing data input operations while simultaneously viewing an image of a form. This data input system comprises: a configuration condition storage unit storing conditions for sorting multiple character input boxes; a cutting unit that cuts character fields corresponding to each of the multiple character input boxes from an image; a configuration replacement unit that changes the order defined in the definition information according to the conditions; and a display control unit that displays the cut images of the multiple character input boxes with the changed order, and the character fields corresponding to the character input boxes of the character input object, approximately simultaneously on the screen. Summary of the Invention

[0005] By reading the results from multiple forms and displaying character information for each commonly identified character, the overall character information can be confirmed. However, there is a possibility that due to reading errors, characters that are not originally common may also be included in the displayed results. If characters that are not originally common are displayed together, it becomes difficult to distinguish between common and incompatibility, making it difficult to determine the character that should be confirmed. For example, if "1" is confirmed as a character in the reading result, but "1", "2", and "3" are displayed together as characters that are not originally common, it becomes difficult to determine "1" as the character that should be confirmed.

[0006] Therefore, the purpose of this disclosure is to provide an information processing apparatus, information processing method, information processing system, and computer-readable medium that, compared to the case where character information jointly identified by reading multiple forms is displayed regardless of the number of character recognition results, allows the user to easily determine the character that should be confirmed among the identified characters.

[0007] According to a first aspect of this disclosure, an information processing apparatus is provided, which has a processor, wherein the processor performs the following processing: among the character information identified by reading multiple forms respectively, the character information is displayed in order of decreasing quantity or increasing quantity of the character information that is identified as having a common character.

[0008] According to the second aspect of this disclosure, the processor receives a character selection instruction to confirm the reading result, and performs processing to display the character information, including a character image of the selected character and the character recognition result.

[0009] According to the third aspect of this disclosure, the processor performs the following processing: displays the character image and the character information of the character recognition result for the selected character, which contain related characters with a pre-defined similarity of more than 100%.

[0010] According to the fourth aspect of this disclosure, the processor performs the following processing: merging the character images and character information whose total count among the associated characters is below a predetermined threshold and displaying them in a screen that displays the character images and character recognition results corresponding to the selected character, indicating the possibility of misidentification.

[0011] According to the fifth aspect of this disclosure, the processor further performs the following processing: displaying a screen capable of summarizing and confirming the character image and character information for the selected character, and the character image and character information below the threshold.

[0012] According to the sixth aspect of this disclosure, an information processing system is provided, comprising: the information processing device; and a client terminal having a display unit, wherein the client terminal displays the character information on the display unit in descending or ascending order of the number of character information, as processed by the information processing device.

[0013] According to a seventh aspect of this disclosure, a computer-readable medium is provided, which stores a program that causes a computer to perform processing, wherein the processing includes the following steps: displaying character information in order of decreasing or increasing number of character information identified as common characters from a plurality of forms read separately.

[0014] According to the eighth aspect of this disclosure, an information processing method is provided, which performs the following processing: among the character information identified by reading multiple forms separately, the character information is displayed in order of decreasing quantity or increasing quantity of the character information that is identified as having a common character.

[0015] (Effect)

[0016] According to the first solution, compared with the case where the character information jointly identified by reading multiple forms is displayed regardless of the number of character recognition results, an information processing device can be provided that allows the user to easily identify the character that should be confirmed among the recognized characters.

[0017] According to the second scheme, the characters that should be confirmed can be identified, thereby confirming whether there is any misidentification.

[0018] According to the third scheme, it is possible to confirm the presence or absence of misidentification by including characters similar to the characters that should be confirmed.

[0019] According to the fourth scheme, for characters that are determined to be characters that should be confirmed, character images and character information with a high probability of misidentification can be confirmed together.

[0020] According to the fifth scheme, compared with the case of separately displaying character images and character information for the selected character and character images and character information with a high probability of misidentification, it is possible to efficiently identify character images and character information with a high probability of misidentification.

[0021] According to the sixth solution, an information processing system can be provided in which, compared to the case where character information jointly identified from reading multiple forms is displayed regardless of the number of character recognition results, the user can easily determine the character that should be confirmed among the identified characters.

[0022] According to the seventh solution, a computer-readable medium can be provided in a way that allows the user to easily identify the characters that should be confirmed among the identified characters, rather than displaying character information jointly identified from multiple forms regardless of the number of character recognition results.

[0023] According to the eighth scheme, the following information processing method can be provided, which performs the following processing: among the character information identified by reading multiple forms respectively, the character information is displayed in order of decreasing quantity or increasing quantity of the character information that is identified as a common character. Attached Figure Description

[0024] Figure 1 This is a diagram showing the general structure of the form system involved in this embodiment.

[0025] Figure 2 This is a diagram illustrating an example of the specific structure of the form system involved in this embodiment.

[0026] Figure 3 This is a block diagram showing the main structural components of the electrical system of the cloud server and PC involved in this embodiment.

[0027] Figure 4 This is a functional block diagram illustrating the functional structure of each part in the form system involved in this embodiment.

[0028] Figure 5 This is a flowchart illustrating an example of the processing flow performed on a cloud server in the form system involved in this embodiment.

[0029] Figure 6 This is an example of a confirmation and correction screen showing the result of reading a single digit.

[0030] Figure 7 This shows an example where "List View" is selected, and "None" is selected for "Checkbox" in the category of character information.

[0031] Figure 8 This example shows that "List View" is selected, and "ふじはなこ" is selected in the "Other" category of the character information.

[0032] Figure 9 This is a functional block diagram illustrating a modified example of the functional structure of each part in the form system according to this embodiment.

[0033] Figure 10 This diagram illustrates an example of a specific display when extracting and prompting related items in the form system according to this embodiment.

[0034] Figure 11This is an example of a screen used for summary confirmation. Detailed Implementation

[0035] Hereinafter, an example of an embodiment of the present disclosure will be described in detail with reference to the accompanying drawings. In this embodiment, an example of an information processing system using a form system will be described. Figure 1 This is a diagram showing the general structure of the form system 10 according to this embodiment.

[0036] like Figure 1 As shown, the form system 10 includes an information processing device 20, a client terminal 40, and an input device 60. These devices are connected to communication lines such as networks (not shown) and can communicate with each other via these communication lines. For example, the Internet, LAN (Local Area Network), and WAN (Wide Area Network) are used in this communication line.

[0037] The information processing device 20 manages a series of processes, including OCR (Optical Character Recognition) processing of image data containing multiple documents and forms input through the input device 60, and outputting the OCR processing results to a pre-defined output destination. The specific structure and function of the information processing device 20 will be described later.

[0038] The client terminal 40 sends various instructions related to OCR processing to the information processing device 20. These instructions, for example, include instructions to begin reading image data and instructions to display the results of reading the image data. Furthermore, based on the received instructions, the client terminal 40 displays the results of the OCR processing performed by the information processing device 20, as well as various notifications related to the OCR processing. The client terminal 40, for example, is a general-purpose computer device such as an application server computer or a personal computer (PC). Figure 1 The illustration shows only one client terminal 40, but it is not limited to this. Multiple client terminals 40 can also be prepared. As an example, client terminals 40 are used separately for each process.

[0039] The input device 60 inputs the image data of the object to be processed by OCR to the information processing device 20. As an example, the input device 60 may be an application server computer, a general-purpose computer device such as a PC, or an image forming apparatus with scanning, printing, and faxing functions. In addition to the input device 60, image data may also be input to the information processing device 20 from the client terminal 40.

[0040] Next, an overview of the form system 10 will be provided.

[0041] The form system 10 is a system in which the information processing device 20 performs OCR processing on image data input through the input device 60 and outputs the result of the OCR processing to a pre-defined output destination.

[0042] In OCR processing, the information processing device 20 manages various processes including (1) business design and application verification, (2) data input, (3) data reading, (4) form judgment confirmation and correction, (5) reading result confirmation and correction, (6) business inspection, (7) data output, and (8) rejection. In this embodiment, OCR processing includes not only the processing of reading characters and symbols from image data, but also post-processing such as character correction.

[0043] As an example of managing various processes, (1) business design and application verification, (2) data input, (3) data reading, (6) business inspection, and (7) data output are all automatically executed by the information processing device 20. Furthermore, as an example of managing various processes, (4) form verification and correction, and (5) result verification and correction are received through user input via the client terminal 40. Additionally, as an example of managing various processes, (8) rejection is either automatically executed by the information processing device 20 or received through user input via the client terminal 40.

[0044] In the (1) business design and application verification process, work rules are created that include read definition settings, output settings, and business check settings. In the read definition settings, for example, the read range is set as the range of information to be read from the image data in the "(3) data read" process. More specifically, for example, a definition can be set such that the value of an item is read from the right side of the item being extracted as a keyword. In the output settings, for example, the file format and output destination of the output data in the "(7) data output" process are set. In the business check settings, for example, the required input items in the form that is the object of inspection in the "(6) business check" process, as well as the format such as the number of characters that can be entered, are set.

[0045] In the data input process (2), image data is received from the input device 60. The received image data is registered as an operation to be executed as the unit of the process "(3) data reading".

[0046] In the data reading process (3), the image data information within the task is read using the task rules selected by the user from the task rules created in the "(1) Business Design and Application Verification" process. As an example, in this process, the form contained in the image data within the task is identified (hereinafter referred to as "form identification") and the characters and symbols within the reading range are read.

[0047] In the form identification confirmation and correction process (4), based on the form identification result performed in the "(3) Data Reading" process, the image data within the job is segmented into records representing the forms contained within the job. Then, in this process, the segmented records are displayed, and confirmation and correction based on the user's form identification are accepted.

[0048] In the (5) Reading Result Confirmation and Correction process, the reading results of characters and symbols within the reading range performed in the "(3) Data Reading" process are displayed, and confirmation and correction based on the user's reading results are accepted.

[0049] In the (6) business inspection process, errors in previous processes are detected by the business inspection settings included in the work rules for this operation, which are selected by the user from the work rules created in the "(1) business design and application verification" process. The detection results can also be displayed to the user.

[0050] In the (7) data output processing, output data is generated using the output settings included in the operation rules for the operation selected by the user from the operation rules generated in the "(1) business design and application verification" processing, and the generated output data is output to the predefined output destination.

[0051] In the (8) rollback process, the process executed in the OCR process is rolled back to a stage or more prior to the previous stage. For example, regarding rollback, it is indicated by the user from client terminal 40 during the execution of various processes such as "(4) Form Judgment Confirmation and Correction" and "(5) Reading Result Confirmation and Correction". Additionally, regarding rollback, it is indicated from the manager's client terminal 40 based on the results of a manager-based inspection performed between the processes of "(6) Business Inspection" and "(7) Data Output".

[0052] In the aforementioned OCR process, the "(1) Business Design and Application Verification" process is executed before the "(3) Data Reading" process, i.e., before the application of the form system 10. Moreover, the "(1) Business Design and Application Verification" process can also be executed during the application of the form system 10 after the "(3) Data Reading" process. As an example, the work rules created in the "(1) Business Design and Application Verification" process can be appropriately modified based on the results of the "(5) Reading Result Confirmation and Correction" process during the application of the form system 10.

[0053] Next, a specific structural example of the form system 10 involved in this embodiment will be described. Figure 2 This is a diagram illustrating an example of the specific structure of the form system 10 according to this embodiment.

[0054] like Figure 2 As shown, the form system 10 uses an image forming device 12 as an example of an input device 60, a cloud server 14 as an example of an information processing device 20, and a PC 16 as an example of a client terminal 40.

[0055] The form system 10 is connected to the image forming apparatus 12, the cloud server 14, and the PC 16 via communication lines 18. Furthermore, the image forming apparatus 12, the cloud server 14, and the PC 16 are each able to send and receive various types of data with each other via communication lines 18.

[0056] Next, the main structural components of the electrical systems of the cloud server 14 and PC 16 involved in this embodiment will be described. Figure 3 This is a block diagram showing the main structural components of the electrical systems of the cloud server 14 and PC 16 according to this embodiment. Furthermore, since the cloud server 14 and PC 16 are basically based on the structure of a general computer, the description will focus on the cloud server 14, and the description of PC 16 will be omitted.

[0057] like Figure 3As shown, the cloud server 14 according to this embodiment includes a CPU 14A, a ROM 14B, a RAM 14C, an HDD 14D, a keyboard 14E, a display 14F, and a communication line interface (I / F) unit 14G. The CPU 14A manages the overall operation of the cloud server 14. The ROM 14B pre-stores various control programs, various parameters, etc. The RAM 14C is used as the working area when the CPU 14A executes various programs, etc. The HDD 14D stores various data, application programs, etc. The keyboard 14E is used to input various information. The display 14F is used to display various information. The communication line interface unit 14G is connected to the communication line 18 and performs various data transmission and reception with other devices connected to the communication line 18. The various parts of the cloud server 14 are electrically connected to each other through the system bus 14H. In addition, the example of the cloud server 14 according to this embodiment including an HDD 14D is described, but it is not limited to this, and other non-volatile storage units such as flash memory may also be included.

[0058] With the above structure, the cloud server 14 in this embodiment, through CPU 14A, performs the following operations: access to ROM 14B, RAM 14C, and HDD 14D; acquisition of various data via keyboard 14E; and display of various information on display 14F. Furthermore, the cloud server 14, through CPU 14A, controls the transmission and reception of communication data via communication line interface 14G.

[0059] In this embodiment, CPU 14A performs the following processing: From the character information identified by reading multiple forms separately, it displays the character information in descending order or ascending order of the number of character information pieces identified as having common characters. Specifically, CPU 14A performs the following processing: From the character information identified by reading multiple forms separately, it sorts the character information in descending order of the number of character information pieces identified as having common characters and then displays it on PC 16. Furthermore, in this embodiment, an example of displaying from most to least common characters will be described.

[0060] Next, the functional structure of the form system 10 according to this embodiment, configured as described above, will be explained. Figure 4 This is a functional block diagram illustrating the functional structure of each part in the form system 10 according to this embodiment.

[0061] The form system 10 has the functions of an image reading unit 50, a recognition processing unit 52, a result storage unit 54, a recognition result aggregation unit 56, a recognition result overview display unit 58, a correction unit 62, and a result output unit 64. Each function is implemented by executing a program. In this embodiment, the image reading unit 50 is described as a function of the image forming apparatus 12, and the recognition processing unit 52, result storage unit 54, recognition result aggregation unit 56, recognition result overview display unit 58, correction unit 62, and result output unit 64 are described as functions of the cloud server 14. However, the recognition processing unit 52, result storage unit 54, recognition result aggregation unit 56, recognition result overview display unit 58, correction unit 62, and result output unit 64 may not be functions of a single cloud server 14, but rather functions shared by multiple servers.

[0062] The image reading unit 50 generates image data representing the image read when the input document is digitized. Additionally, it can automatically recognize non-standard forms. For example, it generates image data representing the image of a form read by the image forming apparatus 12.

[0063] The recognition processing unit 52 reads the recorded content by recognizing characters through OCR processing based on the image data generated by the image reading unit 50.

[0064] The result storage unit 54 associates the image data of the form with the character recognition results based on OCR processing and saves them. For example, it associates the character information, the confidence level of character recognition, and other recognition results with the image data of the character image and saves them in HDD14D.

[0065] The recognition result aggregation section 56 aggregates the recognition results of characters identified in multiple forms. For example, it counts the number of items with each character information.

[0066] The recognition result overview display unit 58 provides a comprehensive overview of the character images, character information, and confidence levels recognized from multiple forms. It also displays the total recognition results, allowing users to select from the highest to lowest total results. Furthermore, it allows for easy selection by extracting different content from each item's overview. By reviewing this information, users can determine the accuracy of the character images and character information. For example, they can check for errors by looking for information displayed in the same column.

[0067] If the character recognition result is incorrect, the correction unit 62 accepts the correct result and updates the information stored in the result storage unit 54. After updating the confirmation status, the items can be reordered by the number of unconfirmed items.

[0068] The output unit 64 outputs a series of processing results, including the correction results from the correction unit 62. Additionally, the output data also includes data from input methods such as CSV (Comma Separated Value) and other core business systems.

[0069] Next, the specific processing performed in the form system 10 according to this embodiment will be described. Figure 5 This is a flowchart illustrating an example of the processing flow performed by the cloud server 14 in the form system 10 according to this embodiment. Additionally, Figure 5 The processing begins, for example, when PC16 issues an instruction to begin reading information about the image data.

[0070] In step 100, CPU 14A acquires the image data read from the form and proceeds to step 102. That is, it acquires the image data generated by the image reading unit 50 through reading the form. For example, it acquires the image data generated by reading each page of the form.

[0071] In step 102, CPU 14A performs character recognition processing and proceeds to step 104. That is, the recognition processing unit 52 recognizes characters from image data generated by reading images from images read by the image reading unit 50 through OCR processing, and reads the recorded content, thereby transforming handwritten character images or printed character images into character information.

[0072] In step 104, CPU 14A sums up the recognition results for each character according to their category and proceeds to step 106. That is, the recognition result summing unit 56 sums up the recognition results of characters recognized from multiple forms. Specifically, it calculates the total number of character images according to the categories of character information such as checkboxes, single-character digits, single-character letters, and freehand strings containing Japanese.

[0073] In step 106, CPU 14A sums up each recognition result for each category range and transfers the result to step 108. Specifically, the recognition result summing unit 56 calculates the total number of character images for each character information.

[0074] In step 108, CPU 14A acquires the filtering settings and proceeds to step 110. For example, it acquires the filtering settings based on attributes such as the processing date specified by the user.

[0075] In step 110, CPU14A extracts the item and proceeds to step 112. The extraction of this item involves filtering the corresponding items in the aggregate results based on user-selected actions. For example, filtering by date and time extracts items relevant to today's assignments.

[0076] In step 112, CPU 14A determines the display order according to the number of items and proceeds to step 114. In this embodiment, the recognition result overview display unit 58 sorts the filtered data groups in descending order of the totals in the free-form descriptive strings containing Japanese characters.

[0077] In step 114, CPU 14A displays a pre-defined confirmation and correction screen for the reading results and proceeds to step 116. That is, the recognition result overview display unit 58 displays the character images, character information, and confidence levels recognized for multiple forms on the PC 16's display 16F, and also displays the total recognition results in descending order of total results. By reviewing this, a judgment is made as to whether the character images and character information are correct.

[0078] In step 116, CPU 14A accepts the confirmation operation and proceeds to step 118. For example, the recognition result overview display unit 58 receives a selection instruction for the character to be confirmed from PC 16, displays the character image of the selected character and the character information of the character recognition result, and the correction unit 62 accepts the confirmation result of whether the character recognition result is correct.

[0079] In step 118, CPU 14A determines whether to update the display. This determination includes, for example, determining whether an operation to update the display has been performed. If the determination is affirmative, the process proceeds to step 120; if other operations such as an end indication have been performed, the determination is negative and the process proceeds to step 122.

[0080] In step 120, CPU14A updates the total result and returns to step 110, repeating the above process.

[0081] In step 122, CPU 14A determines whether the process has ended. This determination checks whether an operation instructing the process to end has been performed. If the determination is negative, the process returns to step 118 and repeats the above steps. If the determination is positive, the process ends for one instance.

[0082] Here is a specific example of a display screen used to confirm the character recognition result. Figure 6 This is an example of a confirmation and correction screen showing the result of reading a single digit.

[0083] exist Figure 6 The example shows the option to select "Form View" and "List View," with "List View" selected. "Form View" displays the character recognition results as a list of the read form, while "List View" filters the character recognition results from the user-selected items in the read form and displays them as a list.

[0084] exist Figure 6 In the example, the total number of items is set to 300, and the number of unconfirmed items is set to 200. The unconfirmed items are used for filtering and display. Furthermore, the total number of character images is displayed according to the category of each character information, and the total number is displayed for each character information. Additionally, in... Figure 6 In the example, the character information is categorized as "single-digit". When the "single-digit" value is "2", there are a maximum of 150 items; next, "0" represents 21 items; and the remaining "single-digit" values ​​are 0 items, displayed in descending order of the total number of character information items. This way, by displaying the character information in descending order of the total number of character information items, users can easily identify the character they need to confirm.

[0085] Furthermore, in Figure 6 In the example, "2" in "1-digit" is selected, displaying the record ID, item name, scanned image, confirmation result, confidence level, and confirmation status for "2". Figure 6 In the example, this indicates that the second confidence level from the bottom is "low," and the others are "high." The screen is updated by performing a "temporary save" operation, which assigns a check mark to the confirmation.

[0086] Figure 7 This shows an example where "List View" is selected, and "None" is selected for "Checkbox" by category of character information.

[0087] exist Figure 7 In the example, with Figure 6 Similarly, the total number of items is 300, with 200 unconfirmed items, displayed filtered by unconfirmed items. Furthermore, the total number of character images is displayed by category for each character information, and the total number is displayed for each character information. Additionally, in... Figure 7 In the example, the number of characters with a single digit of "2" is the most, which is 150. Then, "0" is 21, and the other single digits are 0. The total number of characters is displayed in descending order.

[0088] Furthermore, in Figure 7 In the example, when the "None" checkbox is selected, the record ID, project name, scanned image, confirmation result, confidence level, and confirmation status for the "None" checkbox are displayed. Figure 7 In the example, items with a "high" confidence level are marked with a "confirmed" check mark.

[0089] Figure 8 The example shows "List View" selected, with "Other" selected as the category of character information.

[0090] exist Figure 8In the example, the total number of items is 165, and the unconfirmed items are 124, which are displayed after filtering by all items. Also, the total count of character images is displayed for each category of character information, and the total count is displayed for each character information. In addition, in Figure 8 the example, as shown in "Other" of the category of character information, among the 120 items of "All Text", "ふじはなこ" has the largest count of 20, followed by "子" with 10, and "0123 - 45 - 678" with 10..., and they are displayed in descending order of the total count of character information.

[0091] Also, in Figure 8 the example, "ふじはなこ" is selected, and the record ID, item name, scanned image, confirmation result, confidence level, and presence or absence of confirmation of "ふじはなこ" are displayed. In Figure 8 the example, a check mark of "Confirmed" is marked in the items with a "High" confidence level.

[0092] Thus, in this embodiment, since the display is performed in descending order of the character information of the reading result, the more the number of words read, the higher the accuracy of the overall display result. Due to the high accuracy, the user can comprehensively confirm the reading result and perform the confirmation operation efficiently.

[0093] In addition, for Figure 4 the functional structure of the form system 10 shown, as shown in Figure 9 it can also have the functions of a list condition setting unit 68 and a related item extraction unit 66. Figure 9 is a functional block diagram showing a modified example of the functional structure of each part in the form system 10 according to this embodiment.

[0094] In this case, the list condition setting unit 68 not only specifies the display order from more to less character information, but also specifies the display order from less to more, the order set by the administrator, etc. As an example of the order set by the administrator, for example, the character coding order assigned to the characters can also be applied.

[0095] Furthermore, the association item extraction unit 66, based on the similarity of the extracted characters, simultaneously displays the character image and character information as association recognition results. Association items are those where the extracted images have a pre-defined similarity or higher. For example, if the confidence level of character recognition based on similarity is above a pre-defined threshold, it is designated as an association item. A situation where the ratio of the total number of association items matches a pre-defined ratio is indicated as a high probability of misidentification. For example, regarding the image recognition results, candidates selected as second candidates or higher based on a measurement of certainty or the average of certainty levels are designated as association items. For example, for a group of images identified as "Saitama," assuming the confidence levels are "Saitama": 90%, "Saio": 70%, "Saki": 65%, and "Saikuni": 55%, "Saio": 70% is considered a second candidate, but the image where "Saio" is the first candidate is designated as an association item. Then, the prompt is: "The image group where 'Saio' is the first candidate is incorrect because it is 'Saitama'." At this point, a prompt is given if the percentage of the total number of related items is below a pre-set threshold (e.g., 5%), indicating a high probability of misidentification. For example, if there are 300 items (e.g., "Saitama"), and "Saio" has 10 related items and "Saitama" has 5, then the total percentage of "Saitama" is below 5% for all 300 items. Therefore, "Saio" and "Saitama" are prompted as having a high probability of misidentification.

[0096] Figure 10 This diagram illustrates an example of a specific display when prompting for the extraction of related items in the form system 10 according to this embodiment.

[0097] exist Figure 10 In the example, the following is shown: 230 items identified as "Kanagawa Prefecture", 200 items as "Saitama Prefecture", 180 items as "Chiba Prefecture", 100 items as "Yokohama City", 3 items as "Saio Prefecture", and 2 items as "Yokoha Prefecture". The reading results for "Saitama Prefecture" are then confirmed and corrected. The record ID, scanned image, and reading result for each prefecture are displayed as the item name. Here, in... Figure 10 In the upper right section, for image groups identified as "Saitama," images with a confidence level of 90% or higher (those meeting a pre-defined threshold) are displayed. Additionally, in... Figure 10 The next section displays related items. Here, for related items whose total percentage is less than a preset percentage, it shows three possible cases where "Sai-Oh" is "Saitama".

[0098] Furthermore, in Figure 10 In the example, when the "Summary Confirmation" operation was performed, it displays... Figure 11 The "Summary Confirmation" screen shown. Figure 11 This is an example of a screen used for summary confirmation. For example... Figure 11 As shown, after pressing the object recognition result, press the arrow button to move the recognition result from "Saio" to "Saitama", or from "Saitama" to "Saio". Then, update the recognition result associated with the image by pressing the save button.

[0099] It should be noted that the above embodiment uses the example of displaying the identified character information in descending order of quantity as an example, but it is not limited to this. For example, the characters can be displayed in ascending order of quantity. By displaying them in ascending order of quantity and performing confirmation corrections in ascending order of quantity, noise can be removed first and then confirmation can be performed. Therefore, compared to the case where the display is independent of the number of character recognition results, the user can more easily identify the characters that should be confirmed among the identified characters. For example, by observing them in descending order of quantity and removing noise as with other comprehensive methods, the user can more easily identify the characters that should be confirmed among the identified characters. In addition, the order of quantity from most to least quantity or from least to most quantity can be preset. Furthermore, in addition to the order of quantity from most to least quantity or from least to most quantity, the characters can also be sorted according to the character encoding order pre-assigned to them.

[0100] Furthermore, in the above embodiments, CPU is used as an example of a processor, but processor refers to a processor in a broad sense, including general-purpose processors (such as CPUs) and special-purpose processors (such as GPUs: Graphical Processing Units, ASICs: Application Specific Integrated Circuits, FPGAs: Field Programmable Gate Arrays, programmable logic devices, etc.).

[0101] Furthermore, the operation of the processor in the above embodiments can be formed not only by a single processor, but also by the cooperation of multiple processors located in physically separate positions. Additionally, the order of the processor's actions is not limited to the order described in the above embodiments and can be appropriately modified.

[0102] Furthermore, the processing performed in each part of the form system 10 according to the above embodiments can be software-based, hardware-based, or a combination of both. Additionally, the processing performed in each part of the form system 10 can be stored as a program on a storage medium for distribution.

[0103] Furthermore, this disclosure is not limited to the above content, and various modifications can be made without departing from its main idea.

Claims

1. An information processing apparatus having a processor, wherein the processor performs processing of: displaying, among character information recognized by reading a plurality of forms respectively, the character information in order from more to less or in order from less to more of a number of the character information recognized as a common character, accepting a selection instruction of a character for which confirmation of a reading result is performed, displaying a character image for the selected character and the character information of the recognition result of the character including a character associated with a degree of similarity or more than a predetermined degree of similarity in the recognition result of the character, and merging and displaying the character image and the character information of which a proportion of a total number of the associated characters is less than or equal to a predetermined threshold in a screen in which the character image corresponding to the selected character and the character information of the recognition result of the character are displayed, and prompting a possibility of misrecognition. 2.The information processing apparatus according to claim 1, wherein the processor further performs processing of displaying a screen in which the character image for the selected character and the character information, and the character image and the character information of which the proportion is less than or equal to the threshold can be confirmed.

3. An information processing system, wherein, including: the information processing apparatus according to claim 1 or 2; and a client terminal having a display section, the client terminal displaying the character information in order from more to less or in order from less to more of a number of the character information in the display section by the processing of the information processing apparatus. 4.A computer-readable medium storing a program causing a computer to execute processing, wherein in the processing, processing of: displaying, among character information recognized by reading a plurality of forms respectively, the character information in order from more to less or in order from less to more of a number of the character information recognized as a common character, accepting a selection instruction of a character for which confirmation of a reading result is performed, displaying a character image for the selected character and the character information of the recognition result of the character including a character associated with a degree of similarity or more than a predetermined degree of similarity in the recognition result of the character, and merging and displaying the character image and the character information of which a proportion of a total number of the associated characters is less than or equal to a predetermined threshold in a screen in which the character image corresponding to the selected character and the character information of the recognition result of the character are displayed, and prompting a possibility of misrecognition. 5.An information processing method, wherein processing of: displaying, among character information recognized by reading a plurality of forms respectively, the character information in order from more to less or in order from less to more of a number of the character information recognized as a common character, accepting a selection instruction of a character for which confirmation of a reading result is performed, displaying a character image for the selected character and the character information of the recognition result of the character including a character associated with a degree of similarity or more than a predetermined degree of similarity in the recognition result of the character, is performed. And, the character image and the character information of the character whose proportion of the total number of the associated characters is below a predetermined threshold are displayed in combination in a screen that displays the character information of the character image and the recognition result of the character corresponding to the selected character, prompting the possibility of misrecognition.

6. A computer program product including a program that causes a computer to execute processing, In the processing, the following processing is performed: Among the character information recognized by reading a plurality of forms respectively, the character information is displayed in order from more to less or from less to more of the number of characters recognized as common characters, The selection instruction of the character that accepts confirmation of the reading result is displayed in the recognition result of the character including the associated character above a predetermined similarity, the character image for the selected character, and the character information of the recognition result of the character, And, the character image and the character information of the character whose proportion of the total number of the associated characters is below a predetermined threshold are displayed in combination in a screen that displays the character information of the character image and the recognition result of the character corresponding to the selected character, prompting the possibility of misrecognition.

Images