OCR processing device, OCR processing method, and program

The OCR processing device improves efficiency by grouping and displaying character strings based on similarity, enabling efficient verification and correction of OCR results through dictionary and similarity-based methods, reducing manual validation time.

JP7591914B2Active Publication Date: 2024-11-29PFU LTD
View PDF 3 Cites 0 Cited by

Patent Information

Application Number
JP2020202849
Authority / Receiving Office
JP · JP
Patent Type
Patents
Current Assignee / Owner
Filing Date
2020-12-07
Publication Date
2024-11-29
Estimated Expiration
2040-12-07

AI Technical Summary

Technical Problem

Existing OCR processing systems lack efficiency in checking and modifying OCR processing results, requiring extensive manual verification and correction due to the lack of effective grouping and display mechanisms for character strings.

Method used

An OCR processing device that classifies character strings into groups based on similarity and displays them differently to facilitate efficient checking and correction, using dictionary and similarity grouping, along with user interaction for confirmation and movement of strings between groups.

Benefits of technology

Enhances the efficiency of checking and modifying OCR processing results by allowing simultaneous verification and correction of similar character strings in grouped units, reducing the time and effort required for manual validation.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 0007591914000001
    Figure 0007591914000001
  • Figure 0007591914000002
    Figure 0007591914000002
  • Figure 0007591914000003
    Figure 0007591914000003
Patent Text Reader

Abstract

To provide an OCR processing device which improves efficiency of confirmation or modification of a result of OCR processing.SOLUTION: An OCR processing device includes: an OCR processing unit which specifies character strings input to input items; a grouping unit which classifies the plurality of character strings specified by the OCR processing unit into a plurality of groups; and a display unit which displays the plurality of character strings classified by the grouping unit while classifying for each of the groups. The OCR processing device preferably further includes a character string decision unit which designates decided character strings for the plurality of character strings classified by the grouping unit according to an operation of a user.SELECTED DRAWING: Figure 2
Need to check novelty before this filing date? Find Prior Art

Description

[Technical field]

[0001] The present invention relates to an OCR processing device, an OCR processing method, and a program. [Background technology]

[0002] For example, Patent Document 1 discloses an image processing method in which characters with a score of a predetermined score or less are grouped into groups of characters with the same character code, and when an instruction is input to modify the character code of one character from the grouped characters, all character codes in the group to which this one character belongs are changed in accordance with the instruction. Characters with a score of a predetermined score or less are grouped into characters with the same character code, and when an instruction is input to modify the character code of one character from the grouped characters, all character codes in the group to which this one character belongs are changed in accordance with the instruction. The disclosure discloses that when an instruction to correct the character code of a character is input, characters with a score of a predetermined score or less for changing all character codes in the group to which this character belongs in accordance with this instruction are grouped into groups of characters with the same character code, and when an instruction to correct the character code of one character from the grouped characters is input, characters with a score of a predetermined score or less for changing all character codes in the group to which this character belongs in accordance with this instruction are grouped into groups of characters with the same character code, and when an instruction to correct the character code of one character from the grouped characters is input, all character codes in the group to which this character belongs are changed in accordance with this instruction.

[0003] Patent Document 2 discloses a document processing device which classifies images having the same character strings as the OCR recognition results, superimposes image areas corresponding to the character strings to combine them, compares the character strings on the classification axis with the character strings in the composite image, and performs correction processing on all of the composite images at once. Patent Document 2 discloses a document processing device which classifies images having the same character strings as the OCR recognition results, superimposes image areas corresponding to the character strings to combine them, compares the character strings on the classification axis with the character strings in the composite image, and performs correction processing on all of the composite images at once. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Patent Publication No. 2006-178543 [Patent Document 2] Patent Publication No. 2012-146147 Summary of the Invention [Problem to be solved by the invention]

[0005] The object of the present invention is to provide an OCR processing device that makes it possible to efficiently check or correct the results of OCR processing. [Means for solving the problem]

[0006] The OCR processing device of the present invention has an OCR processing unit that identifies a character string entered into an input field, a grouping unit that classifies the multiple character strings identified by the OCR processing unit into multiple groups, and a display unit that displays the multiple character strings classified by the grouping unit, distinguishing them by group.

[0007] Preferably, the grouping section groups character strings identified by the OCR processing section for the same input item based on a similarity between the character strings.

[0008] Preferably, the system further comprises a character string determination unit which, in response to a user's operation, designates a determined character string from among the plurality of character strings classified by the grouping unit.

[0009] Preferably, the display unit changes a display mode for a plurality of character strings classified into the same group by the grouping unit, depending on the number of identical character strings.

[0010] Preferably, the display unit displays, among the plurality of character strings classified into the same group by the grouping unit, the most common identical character string in a more emphasized manner than the other character strings.

[0011] Preferably, the grouping unit includes a dictionary grouping unit that classifies character strings that match pre-registered registered character strings into groups, and a similarity grouping unit that classifies character strings that the dictionary grouping unit determines to be inconsistent with any registered character string based on their mutual similarity, and the display unit displays the character strings classified by the dictionary grouping unit or the character strings classified by the similarity grouping unit, distinguishing them by group.

[0012] Preferably, the grouping unit includes a blank grouping unit that groups input items determined to be blank, a dictionary grouping unit that classifies, for input items not determined to be blank by the blank grouping unit, character strings that match pre-registered character strings into groups, and a similarity grouping unit that classifies character strings that are determined by the dictionary grouping unit to not match any registered character string based on their mutual similarity, and the display unit displays the character strings classified by the dictionary grouping unit or the character strings classified by the similarity grouping unit, distinguishing them by group.

[0013] Preferably, the system further includes a group moving unit that accepts an operation to move a character string between groups classified by the grouping unit, and a move confirmation unit that applies a confirmed character string confirmed by the character string confirmation unit to a destination group of the character string moved by the group moving unit.

[0014] Preferably, the device further comprises a group display section which displays a list of the groups classified by the grouping section and which displays whether or not the confirmation work for each group has been completed.

[0015] In addition, the OCR processing method of the present invention includes an OCR processing step in which a computer identifies a character string entered into an input field, a grouping step in which the computer classifies the multiple character strings identified by the OCR processing step into multiple groups, and a display step in which the computer displays the multiple character strings classified by the grouping step, distinguishing between each group.

[0016] In addition, the program of the present invention causes a computer to execute an OCR processing step of identifying a character string entered into an input field, a grouping step of classifying the multiple character strings identified by the OCR processing step into multiple groups, and a display step of displaying the multiple character strings classified by the grouping step, distinguishing between each group. Effect of the Invention

[0017] It is possible to efficiently check or correct the results of OCR processing. [Brief description of the drawings]

[0018] [Figure 1] 1 is a diagram illustrating an example of the overall configuration of an OCR processing system 1. [Diagram 2] FIG. 2 is a diagram for explaining a checking and correcting operation in the OCR processing system 1. [Diagram 3] FIG. 2 is a diagram illustrating a hardware configuration of an OCR processing device 2. [Figure 4] 2 is a diagram illustrating an example of a functional configuration of an OCR processing device 2. FIG. [Diagram 5] 10 is a flowchart illustrating an OCR process (S10) performed by the OCR processing device 2. [Figure 6] 13 is a flowchart illustrating a character string moving process (S20) performed by the OCR processing device 2. [Figure 7] FIG. 13 is a diagram illustrating an example of a display screen for checking and correcting the OCR processing result. [Figure 8] 13 is an example of a display screen when an OCR processing result is moved between groups. [Figure 9] 13 is an example of a display screen when an OCR processing result is moved to a confirmed group. [Figure 10] 13A is a diagram illustrating in more detail the grouping process by the dictionary grouping unit 314, and FIG. 13B is a diagram illustrating in more detail the grouping process by the similarity grouping unit 316. FIG. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0019] An embodiment of the present invention will be described with reference to the drawings. FIG. 1 is a diagram illustrating an example of the overall configuration of an OCR processing system 1. 1, the OCR processing system 1 includes an OCR processing device 2 and a scanner device 4, which are connected to each other via a network 7. A plurality of scanner devices 4A and 4B are collectively referred to as the scanner device 4. The OCR processing device 2 may be directly connected to the scanner device 4 via a USB cable or the like. The OCR processing device 2 is a computer terminal that performs OCR processing on image data received from the scanner device 4 and allows the user to confirm or correct the OCR processing results. The OCR processing device 2 of this example performs OCR processing on image data of a form received from the scanner device 4, identifies character strings for each input item on the form, and allows the user to confirm and correct the results.

[0020] The scanner device 4 is an optical reading device, and transmits image data acquired from a document to the OCR processing device 2. In this example, the scanner device 4 is assumed to scan a document.

[0021] FIG. 2 is a diagram for explaining a checking and correcting operation in the OCR processing system 1. As shown in FIG. As shown in Fig. 2A, there is a method for checking the OCR processing results for each scanned form (comparison example). In this case, it is easy to check the consistency of input items for the entire form. 2B, there is a method (comparison example) where the OCR processing results are aggregated for each input item ("company name" in this example) and checked together. In this case, since the same input item can be checked together, it is easy to notice errors in the OCR processing results. However, in the above comparative examples, it is necessary to check all the OCR processing results, which results in poor work efficiency.

[0022] Therefore, in the OCR processing system 1 of this example, as shown in Fig. 2(C), the OCR processing results (character strings) for each input item are classified into multiple groups, and the OCR processing results (character strings) for each group are displayed in a list for confirmation. The classification into groups is based on the similarity (match and similarity) of the OCR processing results. Therefore, multiple character strings (character strings similar to each other) within a group can be confirmed at once, and the time required for confirmation work can be reduced. In addition, in the OCR processing system 1 of this embodiment, when one of the character strings (OCR processing results) in the group is selected as the correct answer, all the OCR processing results in the group are collectively corrected to the selected character string, thereby reducing the time required for the correction work. Note that the concept of a "character string" in this application includes a case where there is only one character or zero characters (no entry).

[0023] FIG. 3 is a diagram illustrating an example of a hardware configuration of the OCR processing device 2. As shown in FIG. As illustrated in FIG. 3, the OCR processing device 2 includes a CPU 200, a memory 202, a HDD 204, a network interface 206 (network IF 206), a display device 208, and an input device 210, which are connected to each other via a bus 212. The CPU 200 is, for example, a central processing unit. The memory 202 is, for example, a volatile memory, and functions as a main storage device. The HDD 204 is, for example, a hard disk drive device, and serves as a non-volatile storage device for storing computer programs (for example, the OCR processing program 3 in FIG. 4) and other data files. The network IF 206 is an interface for wired or wireless communication, and realizes communication with the scanner device 4, for example. The display device 208 is, for example, a liquid crystal display. The input device 210 is, for example, a keyboard and a mouse.

[0024] FIG. 4 is a diagram illustrating an example of the functional configuration of the OCR processing device 2. As shown in FIG. As shown in FIG. 4, an OCR processing program 3 is installed in the OCR processing device 2 of this embodiment. The OCR processing program 3 has an OCR processing unit 300, a grouping unit 310, a character string display unit 320, a group display unit 330, a character string determination unit 340, a group movement unit 350, and a movement determination unit 360. The grouping unit 310 also includes a blank grouping unit 312, a dictionary grouping unit 314, and a similarity grouping unit 316. Note that a part or the whole of the OCR processing program 3 may be realized by hardware such as an ASIC, or may be realized by borrowing some of the functions of an OS (Operating System).

[0025] In the OCR processing program 3, the OCR processing unit 300 performs OCR processing on image data read by the scanner device 4, and identifies character strings contained in the image data. The OCR processing unit 300 of this example performs OCR processing on the image data of the form according to the OCR definition information of the form, and identifies character strings for each input item of the form.

[0026] The grouping unit 310 classifies a plurality of character strings identified by the OCR processing unit 300 into a plurality of groups. More specifically, the grouping unit 310 classifies a plurality of character strings identified by the OCR processing unit 300 for the same input item into a plurality of groups based on the similarity of the character strings. The grouping unit 310 of this example groups character strings using different logics in a multi-stage grouping process. That is, the grouping unit 310 of this example includes a blank grouping unit 312, a dictionary grouping unit 314, and a similarity grouping unit 316.

[0027] The blank grouping unit 312 groups blanks for the same input item. Here, blanks include, for example, input items for which the OCR processing unit 300 was unable to identify a character string, as well as input items for which the number of characters in the character string identified by the OCR processing unit 300 is less than a reference number.

[0028] The dictionary grouping unit 314 classifies into groups character strings that match pre-registered character strings for input items that are not determined to be blank by the blank grouping unit 312. For example, the dictionary grouping unit 314 classifies character strings that match pre-registered character strings in a dictionary into groups corresponding to the respective registered character strings.

[0029] The similarity grouping unit 316 classifies character strings determined by the dictionary grouping unit 314 to be unmatched with any registered character string based on their mutual similarity. For example, the similarity grouping unit 316 calculates the Levenshtein distance for character strings determined by the dictionary grouping unit 314 to be unmatched with any registered character string, and classifies multiple character strings whose Levenshtein distance is within a certain distance into the same group. In this example, when there are a predetermined number or more (e.g., 30% or more) of character strings whose Levenshtein distance is within a certain distance, the similarity grouping unit 316 classifies these multiple character strings into the same group.

[0030] The character string display unit 320 is an example of a display unit according to the present invention, and displays a plurality of character strings classified by the grouping unit 310, distinguishing them by group. As shown in Fig. 7, the character string display unit 320 of this example displays a list of character strings classified into a group selected by the user in a character string list display area 920. Input item images extracted from the form are displayed near the listed character strings (OCR processing results). Furthermore, the character string display unit 320 changes the display mode according to the number of identical character strings for multiple character strings classified into the same group by the grouping unit 310. As illustrated in Fig. 2(C), in this example, the character string display unit 320 surrounds the most identical character strings among multiple character strings classified into the same group by the grouping unit 310 with a blue frame and displays it more emphasized than the other character strings.

[0031] The group display unit 330 displays a list of the groups classified by the grouping unit 310, and displays whether or not the user's confirmation work for each group has been completed. As illustrated in Fig. 7, the group display unit 330 in this example displays a list of the group names classified by the grouping unit 310 in a group list display area 900, and arranges, in association with the group names, a confirmed icon indicating that the confirmation work for each group has been completed, or an unconfirmed icon indicating that the confirmation work for each group has not been completed.

[0032] In response to a user's operation, the character string determination unit 340 collectively specifies a confirmed character string for multiple character strings classified by the grouping unit 310. When the user clicks on any character string (OCR processing result) in the character string list display area 920 in Fig. 7, the character string determination unit 340 in this example applies the clicked character string to all character strings belonging to this group. This completes the correction work for the group in a batch.

[0033] The group moving unit 350 accepts an operation to move a character string (OCR processing result) between groups classified by the grouping unit 310. As illustrated in Fig. 8, when a character string (OCR processing result) to be moved is selected in the character string list display area 920 and dragged to a group name displayed in the group list display area 900, the group moving unit 350 in this example moves the selected character string to the group corresponding to the group name of the drag destination.

[0034] The move confirmation unit 360 applies the confirmed character string confirmed by the character string confirmation unit 340 for the destination group to the character string moved by the group move unit 350. When the move confirmation unit 360 in this example drags and drops a character string in the character string list display area 920 to a group with a confirmed icon, as illustrated in Fig. 9, the move confirmation unit 360 applies the character string (value) confirmed in this group to the dragged character string (OCR processing result).

[0035] 5 is a flowchart explaining the OCR process (S10) by the OCR processing device 2. Note that in this example, it is assumed that a plurality of forms are scanned by the scanner device 4 and the image data thereof is transmitted to the OCR processing device 2. As illustrated in FIG. 5, in step 100 (S100), the OCR processing unit 300 (FIG. 4) of the OCR processing device 2 performs OCR processing on multiple form image data received from the scanner device 4 in accordance with the OCR definition information, and identifies character strings for each input item.

[0036] In step 105 (S105), the blank grouping section 312 of the grouping section 310 groups blanks among the input items to be processed. In step 110 (S110), the dictionary grouping unit 314 classifies, with respect to the unclassified character strings of the input item to be processed, character strings that match registered character strings in advance in the dictionary into groups corresponding to the registered character strings. Note that S110 processes only those that were not grouped in S105. In step 115 (S115), the similarity grouping unit 316 calculates the Levenshtein distance for the unclassified character strings in the input item to be processed, and classifies multiple character strings whose calculated Levenshtein distance is within a certain distance into the same group. Note that S115 processes only those that were not grouped in S105 and S110. Furthermore, character strings that were not classified by the similarity grouping unit 316 are ultimately classified into an unclassified group.

[0037] In step 120 (S120), the grouping unit 310 determines whether the grouping process has been completed for all input items. If there is an input item that has not been completed, the process returns to S105 with the next input item as the processing target, and if all input items have been completed, the process proceeds to S125.

[0038] In step 125 (S125), the group display section 330 displays a list of the group names of the groups classified by the grouping section 310 in the group list display area 900, as exemplified in FIG. 7, and prompts the user to perform a selection operation. As illustrated in FIG. 7, when a user selects any of the group names in the group list display area 900, the character string display unit 320 displays a list of character strings (OCR processing results) classified into the selected group and character string images cut out from the form image in the character string list display area 920.

[0039] In step 130 (S130), the character string display unit 320 highlights the most frequently occurring character string (OCR processing result) among the character strings displayed in the character string list display area 920 as the most frequently occurring character string.

[0040] In step 135 (S135), when any of the displayed character strings (OCR processing results) is clicked in the character string list display area 920, as illustrated in FIG. 7, the character string determination unit 340 changes all character strings classified into this group to the clicked character string and determines them. In step 140 (S140), when the character string determination section 340 performs a determination process on the character string (OCR processing result), the group display section 330 changes the icon of this group in the group list display area 900 of FIG. 7 from an undetermined icon to a confirmed icon.

[0041] In step 145 (S145), if there are any unconfirmed groups, the OCR processing program 3 returns to the processing of S125, and if all groups have been confirmed, the OCR processing result confirmation work is completed and the OCR processing (S10) is ended.

[0042] 6 is a flowchart illustrating the character string moving process (S20) performed by the OCR processing device 2. Note that the character string moving process (S20) is performed in parallel with the processes of S125 to S140 in FIG. In step 200 (S200), the group moving unit 350 waits until any of the character strings displayed in the character string list display area 920 is dragged and dropped onto the group name of another group in the group list display area 900, as illustrated in FIG. 8 (S200: No), and when any of the character strings has been dragged and dropped onto the group name of another group, the process proceeds to S205 (S200: Yes).

[0043] In step 205 (S205), the group moving section 350 moves the dragged and dropped character string to the group with the dropped group name, as exemplified in Fig. 8. That is, the classification by the grouping section 310 is corrected by the user.

[0044] In step 210 (S210), the movement confirmation unit 360 determines whether the destination of the character string moved by the group movement unit 350 is a group whose character string has been confirmed by the character string confirmation unit 340 or an unconfirmed group, and if it is a confirmed group, proceeds to processing of S215, and if it is an unconfirmed group, proceeds to processing of S220. In step 215 (S215), if the destination of the character string is a confirmed group, the movement confirmation unit 360 automatically applies the character string confirmed in the destination group to the moved character string, as exemplified in FIG. In step 220 (S220), if the destination of the character string is an unconfirmed group, the move confirmation unit 360 does not confirm the moved character string, as exemplified in FIG. 8, and displays the moved character string as is in the character string list display area 920.

[0045] FIG. 10A is a diagram illustrating the grouping process by the dictionary grouping section 314 in more detail, and FIG. 10B is a diagram illustrating the grouping process by the similarity grouping section 316 in more detail. As shown in Fig. 10(A), the OCR processing result generally has multiple candidates for each character, and each has a reliability (see OCR processing results 1st to 3rd in the figure). Therefore, as shown in Fig. 10(A), the dictionary grouping unit 314 compares the OCR processing result (combination of all candidates) with words registered in the dictionary, and if there is a matching word, it groups it with the same value as the target word. If there are multiple matches after comparing with all words, the ranking of each character is assigned a score, and the one with the smallest score is adopted.

[0046] As shown in FIG. 10B, the similarity grouping unit 316 measures the Levenshtein distance for all combinations of the OCR processing results (character strings), and treats those with a certain value or less as a group. The Levenshtein distance is a distance that indicates how different two character strings are, and the smaller the value, the more similar they are. However, since the grouping is performed for all combinations (permutations), there is a possibility that the character string may belong to multiple groups. If the character string belongs to multiple groups, it will belong to the group with the smallest distance. The result of grouping is the one that finally belongs to at least one group. In the example of FIG. 10B, the one with the minimum Levenshtein distance of 1 is adopted (cell with gray background). If multiple minimum values ​​appear, the results are narrowed down to one of them based on the sorting order (ascending order of No. 1). Finally, the one with the most occurrences of the minimum value 1 (No. 1 and No. 4 surrounded by a thick frame) becomes the group.

[0047] As described above, according to the OCR processing system 1 of this embodiment, the OCR processing results can be grouped based on similarity, and confirmation and correction work can be performed on a group basis, thereby reducing the time and effort required for confirmation and correction work. [Explanation of symbols]

[0048] 1. OCR processing system 2. OCR processing device 3. OCR processing program 300: OCR processing section 310…Grouping section 320...Character string display section 330...Group display section 340…Character string confirmation part 350…Group Movement Section 360…Movement confirmation section

Claims

1. an OCR processing unit that identifies a character string input into an input item; a grouping unit that classifies the plurality of character strings identified by the OCR processing unit into a plurality of groups; a display unit that displays the plurality of character strings classified by the grouping unit, distinguishing them by group; An OCR processing device having the above configuration.

2. The grouping unit groups character strings identified by the OCR processing unit for the same input item based on a similarity between the character strings. The OCR processing device according to claim 1 .

3. a character string determination unit that designates a determined character string from among the plurality of character strings classified by the grouping unit in response to a user operation; Further having The OCR processing device according to claim 2 .

4. The display unit changes a display mode of the plurality of character strings classified into the same group by the grouping unit in accordance with the number of the plurality of character strings in the same group. The OCR processing device according to claim 3 .

5. The display unit displays the most frequently occurring identical character string among the plurality of character strings classified into the same group by the grouping unit in a more emphasized manner than the other character strings. The OCR processing device according to claim 4 .

6. The grouping unit includes: a dictionary grouping unit that classifies character strings that match preregistered registered character strings into groups; a similarity grouping unit that classifies character strings determined by the dictionary grouping unit to be inconsistent with any registered character string based on their similarity to each other; Including, The display unit displays the character strings classified by the dictionary grouping unit or the character strings classified by the similarity grouping unit, distinguishing them by group. The OCR processing device according to claim 5 .

7. The grouping unit includes: a blank grouping unit that groups input items that are determined to be blank; a dictionary grouping unit that classifies, for input items that are not determined to be blank by the blank grouping unit, character strings that match pre-registered character strings into groups; a similarity grouping unit that classifies character strings determined by the dictionary grouping unit to be inconsistent with any registered character string based on their similarity to each other; Including, The display unit displays the character strings classified by the dictionary grouping unit or the character strings classified by the similarity grouping unit, distinguishing them by group. The OCR processing device according to claim 5 .

8. a group moving unit that receives an operation to move a character string between the groups classified by the grouping unit; a movement confirmation unit that applies a confirmed character string confirmed by the character string confirmation unit to a group to which the character string has been moved by the group movement unit; The OCR processing device according to claim 3 , further comprising:

9. A group display section that displays a list of the groups classified by the grouping section and displays whether or not the confirmation work for each group has been completed. The OCR processing device according to claim 3 , further comprising:

10. an OCR processing step in which a computer identifies a character string entered in an input item; a grouping step in which the computer classifies the plurality of character strings identified by the OCR processing step into a plurality of groups; a display step in which the computer displays the plurality of character strings classified by the grouping step, distinguishing them by group; 1. An OCR processing method comprising:

11. an OCR processing step for identifying a character string entered in an input field; a grouping step of classifying the plurality of character strings identified by the OCR processing step into a plurality of groups; a display step of displaying the plurality of character strings classified by the grouping step, distinguishing them by group; A program that causes a computer to execute the following.

Citation Information

Patent Citations

  • Document retrieval method

    JP2003242171A

  • Image processing method and image processor

    JP2006178543A

  • Document processing device and document processing program

    JP2012146147A