Character recognition apparatus and program

Abstract

To provide a character recognition apparatus and program with which item names and item values can be appropriately combined from character strings included in a document image.SOLUTION: A character acquisition processing server 1 includes: a character string acquisition unit 12 which detects characters from a document image to acquire character strings; an item classifying unit 13 which classifies the character strings acquired by the character string acquisition unit 12 into item names and item values; a combination estimation unit 14 which estimates combinations of the item names and the item values from the character strings classified by the item classifying unit 13; a combination determination unit 15 which determines a combination of an item name and an item value to be output, on the basis of position information of the item name and the item value, out of the combinations of the item names and the item values estimated by the combination estimation unit 14; and a processing result output unit 16 which outputs the combination of the item name and the item value determined by the combination determination unit 15.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to a character recognition device and a program. [Background technology]

[0002] Conventionally, when documents such as estimates and invoices, which have different formats depending on the issuer, are registered as data in a system, the text has to be entered manually, which is time-consuming and labor-intensive. Therefore, various efforts to automate character input are being considered to reduce costs. One example is the use of OCR (Optical Character Recognition / Reader). However, with OCR, information is acquired in pieces and the meaning of the text cannot be understood. Therefore, for example, there are those that use a template that specifies the position where a particular attribute is to be written in a document (for example, Patent Document 1). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2019-86984 A Summary of the Invention [Problem to be solved by the invention]

[0004] In Patent Document 1, it was cumbersome to prepare a format for each document. In particular, in non-standard documents such as estimates and invoices, item names and their contents are written, but if there are no ruled lines, the ruled lines cannot be used as a clue, making it difficult to properly extract the items and their contents.

[0005] SUMMARY OF THE PRESENT EMBODIMENTS An object of the present invention is to provide a character recognition device and program that can appropriately combine item names and item values ​​from character strings included in a document image. [Means for solving the problem]

[0006] The present invention solves the above problems by the following solving means. A first invention is a character recognition device comprising: a character string acquisition means for detecting characters from a document image and acquiring a character string; an item classification means for classifying the character string acquired by the character string acquisition means into an item name and an item value; a combination estimation means for estimating a combination of the item name and the item value from the character string classified by the item classification means; a combination determination means for determining a combination of the item name and the item value to be output based on positional information of the item name and the item value among the combinations of the item name and the item value estimated by the combination estimation means; and an item combination output means for outputting the combination of the item name and the item value determined by the combination determination means. A second invention is a character recognition device according to the first invention, wherein the item classification means refers to an item name list that stores the item names of each item, classifies the character strings included in the item name list into the item names, and classifies the character strings not included in the item name list into the item values. A third invention is a character recognition device according to the first or second invention, wherein the combination estimation means refers to an item value condition storage unit that stores conditions for item values ​​for item names, and estimates the item name to be combined with the character string of the item value classified by the item classification means. A fourth invention is a character recognition device according to the first or second invention, wherein the combination estimation means uses a learning model that has learned combinations of item names and item values ​​to obtain the item name to be combined with the character string of the item value classified by the item classification means. A fifth invention is a character recognition device according to any one of the first to fourth inventions, wherein the combination estimation means acquires the character string of the item value that is at a distance equal to or less than a threshold from the character string of the item name classified by the item classification means, and combines the acquired character string of the item value with the item name. A sixth invention is a character recognition device according to the fifth invention, wherein the combination estimation means, in response to acquiring character strings of a plurality of item values ​​as the character string of the item value that is at a distance of a threshold or less from the character string of the item name classified by the item classification means, refers to an item value condition storage unit that stores conditions for item values ​​for an item name, and, if the item value conditions are satisfied by the character strings of the plurality of item values, combines a combined character string obtained by combining the character strings of the plurality of item values ​​with the character string of the item name. A seventh invention is a character recognition device in any of the first to sixth inventions, wherein the combination determination means determines the combination to be output when a distance between the item name and the item value, which is the combination estimated by the combination estimation means, is equal to or less than a threshold. An eighth invention is a character recognition device according to the seventh invention, wherein the combination determination means further determines the combination as an output target based on a relationship between the placement positions of the item value and the item name. A ninth invention is a character recognition device according to the fourth invention, comprising: a registration means for storing in an information storage unit a combination of the item name and the item value that is confirmed by input from a user for the combination of the item name and the item value output by the item combination output means; and a learning means for learning a learning model using the combination of the item name and the item value stored in the information storage unit as an input value. A tenth aspect of the present invention is a program for causing a computer to function as any one of the character recognition devices according to the first to ninth aspects of the present invention. Effect of the Invention

[0007] According to the present invention, it is possible to provide a character recognition device and a program capable of appropriately combining an item name and an item value from a character string included in a document image. [Brief description of the drawings]

[0008] [Figure 1]1 is an overall configuration diagram of a character acquisition processing system according to a first embodiment and a functional block diagram of a character acquisition processing server; [Diagram 2] 4A to 4C are diagrams illustrating examples of an item name list storage unit and an item value condition storage unit of the character acquisition processing server according to the first embodiment. [Diagram 3] 5 is a flowchart showing character acquisition processing of a character acquisition processing server according to the first embodiment. [Figure 4] 5A to 5C are diagrams for explaining processing in a character acquisition processing server according to the first embodiment. [Diagram 5] 5A to 5C are diagrams for explaining processing in a character acquisition processing server according to the first embodiment. [Figure 6] 13 is a flowchart showing a combination estimation process of the character acquisition processing server according to the first embodiment. [Figure 7] 5A to 5C are diagrams for explaining processing in a character acquisition processing server according to the first embodiment. [Figure 8] 5A to 5C are diagrams for explaining processing in a character acquisition processing server according to the first embodiment. [Figure 9] 5A to 5C are diagrams for explaining processing in a character acquisition processing server according to the first embodiment. [Figure 10] 13 is a flowchart showing a combination determination process of the character acquisition processing server according to the first embodiment. [Figure 11] 5A to 5C are diagrams for explaining processing in a character acquisition processing server according to the first embodiment. [Figure 12] 11 is an overall configuration diagram of a character acquisition processing system according to a second embodiment and a functional block diagram of a character acquisition processing server. FIG. [Figure 13] 13 is a flowchart showing character acquisition processing of a character acquisition processing server according to the second embodiment. [Figure 14] 13 is a flowchart showing a combination estimation process of a character acquisition processing server according to the second embodiment. [Figure 15] 13 is a flowchart showing a learning process of a character acquisition processing server according to the second embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0009] Hereinafter, an embodiment of the present invention will be described with reference to the drawings. Note that this is merely an example, and the technical scope of the present invention is not limited to this example. (First embodiment) <Overall configuration of character acquisition processing system 100> FIG. 1 is an overall configuration diagram of a character acquisition processing system 100 according to the first embodiment and a functional block diagram of a character acquisition processing server 1. As shown in FIG. FIG. 2 is a diagram showing an example of the item name list storage unit 22 and the item value condition storage unit 23 of the character acquisition processing server 1 according to the first embodiment.

[0010] 1, the character acquisition processing system 100 includes a character acquisition processing server 1 (character recognition device), a user terminal 5, and an OCR device 6. The character acquisition processing server 1, the user terminal 5, and the OCR device 6 are connected via a communication network N. The character acquisition processing system 100 classifies text obtained by performing character recognition processing on document image data into item names and item values. Then, the character acquisition processing system 100 estimates combinations of item names with item values, and outputs combinations determined based on position information of the item names and item values ​​from among the combinations.

[0011] In the character acquisition processing system 100, the character acquisition processing server 1 accepts document image data input from the user terminal 5. Here, the document image data is, for example, an image of a document such as an estimate or an invoice, and includes characters. In the following, the document image data will be described as an example that does not include a detailed area in a table format, but it may include a detailed area in a table format. Then, the character acquisition processing server 1 performs character recognition processing on the document image data to detect characters from the document image data to acquire character strings, and classifies the characters into item names or item values ​​using natural language processing technology. Moreover, the character acquisition processing server 1 estimates the item name for the character string classified as the item value, and determines a combination based on the position information of the item name and the item value. Then, the character acquisition processing server 1 outputs the determined combination of the item name and the item value to the user terminal 5. At that time, the character acquisition processing server 1 may output, for example, an alert when a plurality of item names are combined with one item value.

[0012] <Character acquisition processing server 1> The character acquisition processing server 1 is a server that performs a series of processes on document image data and outputs combinations of item names and item values ​​as processing results. The character acquisition processing server 1 includes a control unit 10, a storage unit 20, and a communication interface unit 29. The control unit 10 is a CPU (Central Processing Unit) that controls the entire character acquisition processing server 1. The control unit 10 appropriately reads and executes an OS (Operating System) and various application programs stored in the storage unit 20, thereby cooperating with the above-mentioned hardware to execute various functions.

[0013] The control unit 10 includes a document image acquisition unit 11 , a character string acquisition unit 12 , an item classification unit 13 , a combination estimation unit 14 , a combination determination unit 15 , and a processing result output unit 16 . The document image acquisition unit 11 accepts document image data from the user terminal 5. The document that is the source of the document image data includes, for example, data contents to be input to an external system (not shown). The document image data is, for example, an image of a paper document made by a scanner (not shown) or the like, and is image data in PDF format, JPEG format, IMG format, or the like.

[0014] The character string acquiring unit 12 functions as a character string acquiring unit. The character string acquiring unit 12 detects characters from the document image data received by the document image acquiring unit 11, and acquires the detected character chunks as a character string. More specifically, the character string obtaining unit 12 transmits document image data to the OCR device 6, for example, and detects characters from the text data obtained from the OCR device 6. Then, the character string obtaining unit 12 groups the detected characters into a single character string according to their character spacing. The character spacing between characters in the character string may vary depending on, for example, the character size. For example, when the character size is large, the character string obtaining unit 12 includes adjacent characters as a character string even if the character spacing is wider than when the character size is small. Furthermore, the character string acquiring unit 12 acquires text as a character string, and also acquires the coordinates (position information) of the character string.

[0015] Item classification unit 13 functions as an item classification means. Item classification unit 13 classifies the character strings detected by character string acquisition unit 12 into item names and item values. More specifically, item classification unit 13 classifies character strings included in an item name list in item name list storage unit 22 (described later) into item names, and classifies character strings not included in the item name list into item values.

[0016] The combination estimation unit 14 functions as a combination estimation means. The combination estimation unit 14 estimates combinations of item names and item values ​​from the character strings classified by the item classification unit 13. More specifically, the combination estimation unit 14 acquires an item value character string that is a distance equal to or less than a threshold value from the item name character string, and combines the acquired item value character string with the item name. In this case, in response to acquiring multiple item value character strings as item value character strings that are a distance equal to or less than a threshold value from the item name character string, the combination estimation unit 14 refers to the item value condition storage unit 23 (described later), and when the multiple item value character strings satisfy the condition of one item value, combines one item value (combined character string) obtained by combining the multiple item value character strings with the item name. Furthermore, the combination estimation unit 14 refers to the item value condition storage unit 23 and estimates an item name to be combined with a character string of an item value classified by the item classification unit 13. More specifically, when the character string of an item value satisfies an item value condition in the item value condition storage unit 23, the combination estimation unit 14 estimates an item name corresponding to the item value condition as the item name of the item value.

[0017] The combination determination unit 15 functions as a combination determination means. The combination determination unit 15 determines a combination of an item name and an item value to be output based on position information of the item name and the item value among the combinations of the item name and the item value estimated by the combination estimation unit 14. More specifically, when the distance between the estimated combination of item name and item value is equal to or less than a threshold, the combination determination unit 15 determines the combination to be output. Here, the threshold may be registered in the storage unit 20, or may be received as an external file. The combination determination unit 15 further determines the combination to be output based on the relationship between the arrangement positions of the item value and the item name. For example, when a document is written horizontally, the item name is generally arranged to the left of the item value. Therefore, when the item name is to the right of the item value, the combination determination unit 15 determines that the combination is not to be output.

[0018] The processing result output unit 16 functions as an item combination output means. The processing result output unit 16 outputs the combinations of item names and item values ​​determined as output targets by the combination determination unit 15 to the user terminal 5 as processing results. The processing result output unit 16 may output, for example, the combinations of item names and item values ​​in a table format, and may also output document images on the same screen. Each of these functions will be described in detail later.

[0019] The storage unit 20 is a storage device such as a hard disk or a semiconductor memory device for storing programs, data, and the like required for the operation of the character acquisition processing server 1. The storage unit 20 includes a program storage unit 21, an item name list storage unit 22, and an item value condition storage unit . The program storage unit 21 is a storage area for storing various programs executed by the character acquisition processing server 1. The program storage unit 21 stores a character recognition processing program 21a (program). The character recognition processing program 21a is, for example, a program for executing various functions of the control unit 10. The programs for executing the various functions of the control unit 10 are not limited to those realized by one character recognition processing program 21a as described above. For example, the control unit 10 may have a program for each function or for each of a plurality of functions.

[0020] The item name list storage unit 22 is a storage area for storing an item name list. FIG. 2A shows an example of an item name list stored in the item name list storage unit 22. As shown in FIG. The item name list in the item name list storage unit 22 associates, for example, one or more item names with a representative item name for each item. The representative item name is the key item name. The item name is the name of the item including the variation of the item name. For example, for "name", similar item names include "surname and given name", "first name", "first name", etc. In addition, the item name list storage unit 22 shown in Fig. 2(A) includes the representative item name in the item name, but is not limited to this. For example, if the representative item name and the item name are referenced when performing processing, the representative item name may be excluded from the item name.

[0021] The item value condition storage unit 23 is a storage area for storing conditions for item values. FIG. 2B shows an example of the item value condition storage unit 23. The item value condition storage unit 23 associates representative item names with item value conditions. The item value condition is a regular expression or rule that represents the item value. For example, the condition for the item value of the representative item name "Name" is the rule that "the morpheme contains a person's name." Here, the person's name may be stored in the storage unit 20, or may be obtained from a name dictionary (not shown) in an external server communicatively connected to the character acquisition processing server 1 via the communication network N. In addition, the condition for the item value where the representative item name is "Date of Birth" is a regular expression of "(Era)(numerical value) year (numerical value) month (numerical value) day". Here, regarding the possible strings for the era, the possible numbers for each numerical value, and their combinations, etc., the content may be defined by a separate definition file, or may be included in advance in the condition for the item value. When it is included in advance in the condition for the item value, the content of the condition for the item value is an expression that can be represented by a function and is described as a program.

[0022] The communication interface unit 29 in FIG. 1 is an interface for communicating with the user terminal 5 etc. via the communication network N. Here, a computer refers to an information processing device equipped with a control unit, a storage device, etc. The character acquisition processing server 1 is an information processing device equipped with a control unit 10, a storage unit 20, etc. and is included in the concept of a computer. In addition, there is no limit to the number of hardware components that make up the character acquisition processing server 1. It may be composed of one or more as needed. Also, the hardware of the character acquisition processing server 1 may include various servers such as a Web server, a DB server, and an application server as needed, and may be composed of one server, or each may be composed of separate servers. Further, the character acquisition processing server 1 may be, for example, a cloud.

[0023] <User Terminal 5> The user terminal 5 shown in FIG. 1 is, for example, a terminal used by a user who wants to register various data described in a document in an external system. The user terminal 5 can be composed of, for example, a personal computer (PC), a tablet terminal, etc. Although not shown, the user terminal 5 includes a control unit, a storage unit, a display unit, an input unit, a communication interface unit, etc.

[0024] <OCR Device 6> The OCR device 6 is a device that performs OCR-related processing to recognize characters from document image data. By receiving document image data from the character acquisition processing server 1, the OCR device 6 performs character recognition processing on the document image to obtain text data, and transmits the text data to the character acquisition processing server 1. The OCR device 6 is, for example, configured with a server or the like. The OCR device 6 may also be configured with a PC or the like. Although not shown, the OCR device 6 includes a control unit, a storage unit, a communication interface unit, and the like.

[0025] The communication network N is a network between the character acquisition processing server 1, the user terminal 5, and the OCR device 6, and is, for example, a communication network such as the Internet. The communication network N may be a LAN (Local Area Network) or the like. In addition, the communication network N may be wired or wireless.

[0026] <Processing Description> Next, the process of the character acquisition processing server 1 will be described. FIG. 3 is a flowchart showing the character acquisition processing of the character acquisition processing server 1 according to the first embodiment. 4 and 5 are diagrams for explaining the process in the character acquisition processing server 1 according to the first embodiment. FIG. 6 is a flowchart showing the combination estimation process of the character acquisition processing server 1 according to the first embodiment. 7 to 9 are diagrams for explaining the processing in the character acquisition processing server 1 according to the first embodiment. FIG. 10 is a flowchart showing the combination determination process of the character acquisition processing server 1 according to the first embodiment. FIG. 11 is a diagram for explaining the process in the character acquisition processing server 1 according to the first embodiment.

[0027] First, the user terminal 5 connects to the character acquisition processing server 1 and transmits document image data, which is image data of a document such as an estimate or an invoice, and in step S 11 of FIG. 3 (hereinafter, "step S" will be simply referred to as "S"), the document image acquisition unit 11 of the character acquisition processing server 1 acquires the document image data.

[0028] In S12, the character string obtaining unit 12 detects characters in the obtained document image and obtains a character string from the detected characters. The character string obtaining unit 12 can detect characters in the document image and obtain a character string from the detected characters by, for example, transmitting document image data to the OCR device 6 and obtaining text data from the OCR device 6. Here, the process will be described taking image 31 of a medical certificate shown in Fig. 4(A) as an example. When the process up to S12 is performed on image 31, a character string surrounded by a dotted line as a series of characters can be obtained as shown in image 32 shown in Fig. 4(B), and an OCR result 33 shown in Fig. 4(C) is obtained. The OCR device 6 scans the document image from the top left to the right, and when it has scanned to the right end, it moves down one step and scans from left to right, repeating this process until the scanning reaches the bottom right of the document image.

[0029] In S13 of FIG. 3, the item classification unit 13 classifies the acquired character strings into item names and item values. Here, the method of classifying character strings will be described with reference to FIG. Item classification unit 13 compares the character string of OCR result 33 with the item name list in item name list storage unit 22. Then, if there is an item name in the item list that matches the character string of OCR result 33, item classification unit 13 estimates the character string as an item name. On the other hand, if there is no item name in the item list that matches the character string of OCR result 33, item classification unit 13 estimates the character string as an item value. Item classification unit 13 associates character strings estimated as item names with representative item names and stores them in item name estimation results 41. Also, item classification unit 13 stores character strings estimated as item values ​​in item value estimation results 42. Note that item name estimation results 41 and item value estimation results 42 may be temporarily stored in storage unit 20.

[0030] In S14 of FIG. 3, the combination estimation unit 14 performs a combination estimation process. Here, the combination estimation process will be described with reference to FIG. In S31 of FIG. 6, the combination estimation unit 14 acquires character strings of item values ​​that are located at a short distance equal to or less than a threshold for each item name. In S32, the combination estimation unit 14 judges whether or not there are multiple character strings acquired for one item name. If there are multiple character strings (S32: YES), the combination estimation unit 14 transfers the process to S33. If there are not multiple character strings (S32: NO), the combination estimation unit 14 transfers the process to S35.

[0031] In S33, the combination estimation unit 14 refers to the item value condition storage unit 23 and determines whether or not a condition for one item value is satisfied by multiple character strings. If multiple character strings satisfy the item value condition (S33: YES), the combination estimation unit 14 transfers the process to S34. On the other hand, if multiple character strings do not satisfy the item value condition (S33: NO), the combination estimation unit 14 transfers the process to S35. In S34, the combination estimation unit 14 combines a plurality of character strings to generate a character string of an item value. In S35, the combination estimation unit 14 causes the storage unit 20 to store a candidate list in which the item names and item values ​​are combination candidates.

[0032] The above-mentioned processing will now be described based on a concrete example. In the case of the image 51 shown in FIG. 7, the control unit 10 obtains "Mihon Hanako" and "Mimimoto Hanako" as character strings of item values ​​that are at a distance equal to or less than the threshold value for the character string of the item name "Name" (S31 in FIG. 6). The control unit 10 determines that there are multiple character strings obtained (YES in S32 in FIG. 6), and refers to the item value condition storage unit 23 to determine whether or not the multiple character strings satisfy the condition of one item value (S33 in FIG. 6). In this example, since each item value satisfies the item value condition individually, the process of S33 in FIG. 6 becomes NO, and the control unit 10 stores the item name and each item value as combination candidates in the candidate list 55 as rows 55a (S35 in FIG. 6). For the character string of the item name "age on date of examination", the control unit 10 obtains "59" as the character string of the item value that is at a distance equal to or less than the threshold value (S31 in FIG. 6). The control unit 10 determines that the obtained character string is not multiple (NO in S32 in FIG. 6), and stores the item name and item value as combination candidates in the candidate list 55 as row 55b (S35 in FIG. 6).

[0033] As another example, in the case of the image 61 shown in FIG. 8, for the character string of the item name "Date of Birth", the control unit 10 obtains "1953" and "August 17" as character strings of item values ​​that are at a distance equal to or less than the threshold value (S31 in FIG. 6). A provisional list 62 is created as combination candidates at this point. The control unit 10 determines that there are multiple acquired character strings (YES in S32 in FIG. 6), and judges whether the multiple character strings satisfy the item value condition by referring to the item value condition storage unit 23 (S33 in FIG. 6). In this example, the combination of the two character strings satisfies the item value condition shown in row 63 of the item value condition storage unit 23, so the process of S33 in FIG. 6 becomes YES, and the multiple item values ​​are combined to form a character string of the item value (S34 in FIG. 6), and the item name and the item value are stored in the candidate list 64 as combination candidates (S35 in FIG. 6). The coordinates in the candidate list 64 are the coordinates of the range including the two item values.

[0034] In S36 of FIG. 6, the combination estimation unit 14 refers to the item value condition storage unit 23 and estimates an item name for the character string of the item value. In S37, the combination estimation unit 14 stores a candidate list in which the estimated item names and item values ​​are combination candidates in the storage unit 20. After that, the control unit 10 shifts the process to S15 in FIG.

[0035] The above-mentioned processing will now be described based on a concrete example. As shown in FIG. 9, the combination estimation unit 14 judges whether or not each of the item value estimation results 42 satisfies the item value condition in the item value condition storage unit 23. For example, when the combination estimation unit 14 determines whether the item value "101" in the item value estimation result 42 satisfies the item value conditions in the item value condition storage unit 23, it finds that the representative item name matches the item value condition of "group" and also matches the item value condition of "age". Therefore, the item value "101" is stored in two rows, row 72 and row 73, in the candidate list 71. Even if there is only an item value and no item name, the combination estimation unit 14 stores in the candidate list 71 a combination with an item name that satisfies the item value condition in the item value condition storage unit 23. For example, even if there is only an item value "female" and no item name "gender" or the like, the item value "female" is stored in row 74 in the candidate list 71.

[0036] The processes from S31 to S35 and the processes from S36 to S37 shown in Fig. 6 may be performed in parallel, or the processes from S31 to S35 may be performed after the processes from S36 to S37. Also, the candidate list may contain duplicate entries. If there are duplicate entries, the control unit 10 may, for example, delete one of the duplicate entries after completing the process in Fig. 6 to eliminate the duplicate entries.

[0037] In S15 of FIG. 3, the combination determination unit 15 performs a combination determination process. Here, the combination determination process will be described with reference to FIG. In S51 of FIG. 10, the combination determination unit 15 refers to the candidate list and calculates the distance between the item name and the item value for each combination. In S52, the combination determination unit 15 judges whether the distance is equal to or less than a threshold. The threshold may be the same as or different from the threshold used in S31 of Fig. 6. If the distance is equal to or less than the threshold (S52: YES), the combination determination unit 15 transfers the process to S53. On the other hand, if the distance is not equal to or less than the threshold (S52: NO), the combination determination unit 15 transfers the process to S59.

[0038] In S53, the combination determination unit 15 judges whether the item name is located to the left of the item value. If the item name is located to the left of the item value (S53: YES), the combination determination unit 15 transfers the process to S54. On the other hand, if the item name is not located to the left of the item value (S53: NO), the combination determination unit 15 transfers the process to S59. In S54, the combination determination unit 15 determines the combination to be output. After that, the combination determination unit 15 moves the process to S16 in FIG. On the other hand, in S59, the combination determination unit 15 excludes the combination from the output targets. After that, the combination determination unit 15 shifts the process to S16 in FIG.

[0039] The above-mentioned processing will now be described based on a concrete example. In the example shown in FIG. 11, in the candidate list 81, the same item value "101" is registered in rows 81a and 81b. As shown in image 82, row 81a corresponds to distance image 82a, and row 81b corresponds to distance image 82b. Distance image 82a is below the threshold, but distance image 82b is not below the threshold. In this case, for the item value "101", row 81a is to be output, and row 81b is not to be output.

[0040] 3, the processing result output unit 16 outputs the combination of the item name and the item value determined to be output as the processing result to the user terminal 5. The processing result output unit 16 may output the document image data together with the combination of the item name and the item value determined to be output. After that, the control unit 10 ends this process.

[0041] As described above, according to the first embodiment, the character acquisition processing server 1 has the following advantages. (1) Characters are detected from document image data to obtain character strings, the obtained character strings are classified into item names and item values, combinations of item names and item values ​​are estimated from the classified character strings, and combinations of item names and item values ​​to be output are determined based on positional information of the item names and item values ​​among the estimated combinations of item names and item values, and the determined combinations of item names and item values ​​are output to user terminal 5. Therefore, by simply inputting document image data, it is possible to combine item names and item values ​​from character strings included in the document image. After estimating the combination of item names and item values, the combination of item names and item values ​​to be output is determined based on the position information, so that the item names and item values ​​can be appropriately combined in consideration of the positional relationship between the item names and item values.

[0042] (2) If a character string is included in an item name list in which one or more item names are associated with a representative item name, the character string is classified into the item name. Therefore, the character strings included in the item name list can be classified as item names.

[0043] (3) The item value condition storage unit 23 that stores the conditions of the item value for the item name is referenced, and the item name for the character string of the classified item value is estimated. Therefore, for a character string classified as an item value that satisfies the item value condition in the item value condition storage unit 23, the item name corresponding to the satisfied item value condition can be estimated as the item name for that item value.

[0044] (4) An item value string that is at a distance equal to or less than a threshold from the classified item name string is obtained, and the obtained item value string is combined with the item name. Therefore, a character string of an item value that is a distance equal to or less than a threshold from a character string classified as an item name can be estimated as the item value corresponding to that item name. Even if an item is not registered in the item value condition storage unit 23, it can be combined based on the distance between the item name and the item value.

[0045] (5) In response to obtaining multiple item value strings as item value strings that are within a threshold distance from the classified item name string, the item value condition memory unit 23 is referenced, and if the multiple item value strings satisfy the condition of one item value, a combined string formed by combining the multiple item value strings is combined with the item name string. Therefore, even if an item value is made up of multiple character strings, it can be appropriately combined as an item value for the item name.

[0046] (6) If the distance between the estimated combination of item name and item value is equal to or less than a threshold, the combination is determined to be output. Therefore, combinations in which the item name and item value are close to each other can be output, so that more appropriate item values ​​for the item names can be output. (7) Furthermore, combinations are determined as output targets based on the relationship between the item values ​​and the item names. Therefore, when an item value appears before an item name, it can be excluded from the output even if the distance is equal to or less than the threshold. As a result, the accuracy of the combination of item names and item values ​​is improved.

[0047] Second embodiment In the first embodiment, a combination of an item name and an item value is estimated using a condition of the item value for the item name. In the second embodiment, a processing is performed using a learning model that has learned combinations of item names and item values. In the following description, parts that perform the same functions as in the first embodiment described above are given the same reference numerals or the same reference numerals at the end, and duplicate descriptions are omitted as appropriate.

[0048] <Overall configuration of character acquisition processing system 200> FIG. 12 is an overall configuration diagram of a character acquisition processing system 200 according to the second embodiment and a functional block diagram of a character acquisition processing server 201. As shown in FIG. 12, the character acquisition processing system 200 includes a character acquisition processing server 201, a user terminal 5, and an OCR device 6, and the character acquisition processing server 201, the user terminal 5, and the OCR device 6 are connected via a communication network N.

[0049] <Character acquisition processing server 201> The character acquisition processing server 201 includes a control unit 210, a storage unit 220, and a communication interface unit 29. The control unit 210 includes a document image acquisition unit 11, a character string acquisition unit 12, an item classification unit 13, a combination estimation unit 214, a combination determination unit 15, a processing result output unit 16, a confirmed item receiving unit 217, a confirmed data registration unit 218, and a combination learning unit 219.

[0050] The combination estimation unit 214 functions as a combination estimation means, and estimates combinations of item names and item values ​​from the character strings classified by the item classification unit 13 using a learning model. More specifically, the combination estimation unit 214 uses a learning model that has learned combinations of item names and item values ​​stored in the learning model storage unit 225 to obtain item names to be combined with strings of classified item values. The finalized item receiving unit 217 receives, from the user terminal 5, the finalized contents for the combination of item name and item value output by the processing result output unit 16. The finalized item receiving unit 217 receives, from the user terminal 5, the input of the combination of item name and item value that the user has checked and finalized for the document.

[0051] The finalized data registration unit 218 functions as a registration unit, and registers the combination of the item name and the item value received by the finalized item receiving unit 217 in the finalized data storage unit 224 (information storage unit). The combination learning unit 219 functions as a learning means, and learns a learning model using, as input values, combinations of item names and item values ​​stored in the determined data storage unit 224. Then, the combination learning unit 219 stores the learned learning model in the learning model storage unit 225.

[0052] The storage unit 220 includes a program storage unit 221 , an item name list storage unit 22 , an item value condition storage unit 23 , a finalized data storage unit 224 , and a learning model storage unit 225 . The program storage unit 221 is a storage area for storing various programs executed by the character acquisition processing server 201. The program storage unit 221 stores, for example, a character recognition processing program 221a which is a program for executing various functions of the control unit 210. The confirmed data storage unit 224 is a storage area for storing combinations of item names and item values ​​confirmed by the user. The learning model storage unit 225 is a storage area for storing the learning model learned by the combination learning unit 219.

[0053] <Processing Description> Next, the processing of the character acquisition processing server 201 will be described. FIG. 13 is a flowchart showing the character acquisition processing of the character acquisition processing server 201 according to the second embodiment. FIG. 14 is a flowchart showing the combination estimation process of the character acquisition processing server 201 according to the second embodiment. The processes from S211 to S213 in FIG. 13 are similar to the processes from S11 to S13 in the first embodiment (FIG. 3). In S214, the combination estimation unit 214 performs a combination estimation process.

[0054] Here, the combination estimation process will be described with reference to FIG. The processes from S231 to S235 in FIG. 14 are similar to those from S31 to S35 in the first embodiment (FIG. 6). In S236, the combination estimation unit 214 inputs the string of the item value into a learning model that has learned combinations of item names and item values ​​stored in the learning model storage unit 225, and obtains the item name to be combined with the string of the item value. In S237, the combination estimation unit 214 stores the acquired item names and item values ​​as combination candidates in a candidate list in the storage unit 220. After that, the control unit 210 shifts the process to S215 in FIG.

[0055] The processes of S215 and S216 in FIG. 13 are similar to the processes of S15 and S16 in the first embodiment (FIG. 3). In S217, the confirmed item receiving unit 217 receives from the user terminal 5 an input of a combination of an item name and an item value confirmed by the user for the combination of the item name and the item value output to the user terminal 5 in S216. In S218, the finalized data registration unit 218 registers the combination of the item name and the item value received in S217 in the finalized data storage unit 224. After that, the control unit 210 ends this process.

[0056] Next, the process of training the learning model will be described. FIG. 15 is a flowchart showing the learning process of the character acquisition processing server 201 according to the second embodiment. This learning process may be performed, for example, when a certain amount of finalized data, which is a combination of item names and item values, is registered in the finalized data storage unit 224. Furthermore, the learning process may be performed again at an appropriate timing. In S271 of FIG. 15, the combination learning unit 219 extracts the determined data stored in the determined data storage unit 224.

[0057] In S272, the combination learning unit 219 learns the learning model using combinations of item names and item values ​​of the confirmed data as input values. In S273, the combination learning unit 219 stores the learned learning model in the learning model storage unit 225. After that, the control unit 210 ends this process.

[0058] As described above, according to the second embodiment, the character acquisition processing server 201 has the following advantages. (1) Using a learning model that has learned combinations of item names and item values, the item names to be combined with the classified item value strings are obtained. Therefore, the item name of the character string classified as the item value can be obtained using the learning model, and the obtained item name can be estimated to be combined with the item value.

[0059] (2) The combination of item name and item value that is confirmed by input from the user for the combination of item name and item value output to the user terminal 5 as the output target is stored in the confirmed data storage unit 224, and the learning model is trained using the combination of item name and item value stored in the confirmed data storage unit 224 as input values. Therefore, the learning model can be trained using the combination of the item name and item value confirmed by the user as the input value of the learning model, which can result in the learning model outputting more accurate processing results.

[0060] Although the embodiments of the present invention have been described above, the present invention is not limited to the above-mentioned embodiments. Furthermore, the effects described in the embodiments are merely a list of the most preferable effects resulting from the present invention, and the effects of the present invention are not limited to those described in the embodiments. The above-mentioned embodiments and the modified forms described below can be used in appropriate combination, but detailed description will be omitted.

[0061] (Variations) (1) In each embodiment, document images of documents such as estimates, invoices, and medical certificates have been described as examples, but the present invention is not limited to these. Not only standardized documents, but also non-standardized documents can be processed in the same manner for any document. (2) In the description of the embodiments, the combination determination process has been described with an example in which an item name and an item value whose distance is equal to or less than a threshold are determined as an output target, but is not limited to this. For example, if multiple item names are associated with one item value and the distances are all equal to or less than a threshold, only the item name that is closest to the item value may be determined as an output target. Alternatively, for example, when multiple item names correspond to one item value and the distances between the multiple item names are all below a threshold, the display may be output in a different manner from other items, for example by highlighting the item name when output, to alert the user that multiple item names correspond to one item name.

[0062] (3) In each embodiment, the item name to be combined with the character string of the item value is estimated. Furthermore, for the item value, for example, a process of determining whether or not it is a value to be excluded or a value to be prioritized may be performed, and then an item name to be combined with the character string of the item value may be estimated. For example, when a reference value is described as an item value for a certain item name in addition to an actual measurement value, the reference value may be determined as a value to be excluded. For example, the character acquisition processing server may store the value to be excluded in advance in a storage unit, and may refer to the storage unit to determine whether or not the item value is a value to be excluded. Similarly, for the value to be prioritized, the value to be prioritized may be stored in advance in a storage unit, and the character acquisition processing server may refer to the storage unit to determine whether or not the item value is a value to be prioritized.

[0063] (4) In each embodiment, a case where there are multiple identical item names has not been described, but when there are two or more identical item names in a document image, it may be possible to determine which item name should be adopted based on whether the item names before and after the item name are included in the item name list. Also, when there are multiple identical item names in a document image, the character acquisition processing server may visually indicate to the user that there are the same item names, for example, by highlighting the item names when they are displayed.

[0064] (5) In each embodiment, the character acquisition processing server transmits a document image to an OCR device and requests text conversion, but the present invention is not limited to this. Before transmitting the document image to the OCR device, pre-processing may be added. Examples of pre-processing include contrast adjustment and sharpening. Doing so may further improve the accuracy of processing in the OCR device.

[0065] (6) In each embodiment, the character acquisition processing server, the user terminal, and the OCR device are provided as an example, but the present invention is not limited to this. For example, the character acquisition processing server and the OCR device may be implemented by the same server. Also, for example, the character acquisition processing server and the OCR device may be implemented as a standalone configuration of only a client terminal having the functions of the character acquisition processing server and the OCR function. [Explanation of symbols]

[0066] 1,201 character acquisition processing servers 5. User terminal 6 OCR device 10,210 Control section 11 Document image acquisition unit 12 String acquisition part 13 Item classification section 14,214 Combined Estimation Section 15 Combination decision section 16 Processing result output section 20,220 storage unit 21a, 221a Character recognition processing program 22 Item name list storage section 23 Item value condition memory section 100,200 character acquisition processing system 217 Confirmation Item Reception Department 218 Final Data Registration Department 219 Combinatorial Learning Unit 224 Confirmed Data Storage Unit 225 Learning Model Memory Unit

Claims

1. a character string acquisition means for detecting characters from a document image and acquiring the character string; an item classification means for classifying the character string acquired by the character string acquisition means into an item name and an item value; a combination estimation means for estimating a combination of the item name and the item value from the character string classified by the item classification means; a combination determination means for determining a combination of the item name and the item value to be output based on position information of the item name and the item value among the combinations of the item name and the item value estimated by the combination estimation means; an item combination output means for outputting the combination of the item name and the item value determined by the combination determination means; A character recognition device comprising:

2. 2. The character recognition device according to claim 1, The item classification means refers to an item name list that stores the item names of each item, and classifies the character strings included in the item name list into the item names, and classifies the character strings not included in the item name list into the item values.

3. 2. The character recognition device according to claim 1, The combination estimation means refers to an item value condition storage unit that stores conditions for item values ​​with respect to item names, and estimates the item name to be combined with the character string of the item value classified by the item classification means.

4. 2. The character recognition device according to claim 1, The combination estimation means obtains the item name to be combined with the character string of the item value classified by the item classification means using a learning model that has learned combinations of item names and item values.

5. 5. The character recognition device according to claim 3, The combination estimation means acquires the character string of the item value that is at a distance equal to or less than a threshold value from the character string of the item name classified by the item classification means, and combines the acquired character string of the item value with the item name.

6. 6. The character recognition device according to claim 5, The combination estimation means, in response to obtaining character strings of multiple item values ​​as the character string of the item value that is at a distance of a threshold or less from the character string of the item name classified by the item classification means, refers to an item value condition memory unit that stores conditions for the item value for the item name, and when the item value conditions are satisfied by the character strings of the multiple item values, combines a combined string obtained by combining the character strings of the multiple item values ​​with the character string of the item name.

7. 2. The character recognition device according to claim 1, The combination determination means determines the combination to be output when a distance between the item name and the item value, which is the combination estimated by the combination estimation means, is equal to or smaller than a threshold.

8. 8. The character recognition device according to claim 7, The combination determination means further determines the combination to be output based on a relationship between the layout positions of the item values ​​and the item names.

9. 5. The character recognition device according to claim 4, a registration means for storing in an information storage unit a combination of the item name and the item value, which is confirmed by a user's input for the combination of the item name and the item value output by the item combination output means; a learning means for learning a learning model using a combination of the item name and the item value stored in the information storage unit as an input value; A character recognition device comprising:

10. Computer, a character string acquisition means for detecting characters from a document image and acquiring the character string; an item classification means for classifying the character string acquired by the character string acquisition means into an item name and an item value; a combination estimation means for estimating a combination of the item name and the item value from the character string classified by the item classification means; a combination determination means for determining a combination of the item name and the item value to be output based on position information of the item name and the item value among the combinations of the item name and the item value estimated by the combination estimation means; an item combination output means for outputting the combination of the item name and the item value determined by the combination determination means; A program to make it function as such.

Images