Image processing system, image processing method, and image processing program

Abstract

To provide an image processing system, an image processing method, and an image processing program, which can improve recognition accuracy of a character string of an input image.SOLUTION: An acquisition processing section 113 acquires a result of character recognition processing to be executed for a character string of a prescribed item included in an image. A calculation processing section 114 calculates an evaluation value for each of a plurality of preliminarily registered character strings corresponding to the prescribed item on the basis of a set value set for each of a plurality of character string candidates included in the result of the character recognition processing. A specification processing section 115 specifies a prescribed registered character string on the basis of the evaluation value for each of the plurality of registered character strings calculated by the calculation processing section 114. An output processing section 116 outputs the prescribed registered character string specified by the specification processing section 115.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present disclosure relates to a technique for performing image processing for character recognition on images such as documents. [Background technology]

[0002] Conventionally, there are known techniques for detecting character strings from images of documents such as documents and slips. For example, a character recognition device is known that judges whether a rectangle is valid or invalid based on a set reference value, sets the line direction based on the result of an area discrimination process, divides line rectangles extracted from valid rectangles into one-character units, and performs character recognition by comparing image features of the extracted characters with character features in a pattern dictionary (see, for example, Patent Document 1). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2003-256772 A Summary of the Invention [Problem to be solved by the invention]

[0004] In conventional character recognition processing (OCR processing), for example, in a string of characters consisting of multiple characters, even if the character rectangles of each character can be correctly detected, there is a problem that one of the characters may be misrecognized or the character rectangle of one of the characters may not be detected properly.

[0005] An object of the present disclosure is to provide an image processing system, an image processing method, and an image processing program capable of improving the accuracy of recognizing character strings in an input image. [Means for solving the problem]

[0006] An image processing system according to one aspect of the present disclosure includes an acquisition processing unit, a calculation processing unit, an identification processing unit, and an output processing unit. The acquisition processing unit acquires a result of a character recognition process executed on a character string of a predetermined item included in an image. The calculation processing unit calculates an evaluation value for each of a plurality of registered character strings registered in advance corresponding to the predetermined item based on a setting value set for each of a plurality of character string candidates included in the result of the character recognition process. The identification processing unit identifies the predetermined registered character string based on the evaluation value for each of the plurality of registered character strings calculated by the calculation processing unit. The output processing unit outputs the predetermined registered character string identified by the identification processing unit.

[0007] An image processing method according to another aspect of the present disclosure is a method executed by one or more processors to obtain the results of a character recognition process performed on a character string of a specified item included in an image, calculate an evaluation value for each of a plurality of pre-registered registered character strings corresponding to the specified item based on setting values ​​set for each of a plurality of character string candidates included in the result of the character recognition process, identify the specified registered character string based on the evaluation value for each of the plurality of registered character strings, and output the specified registered character string.

[0008] An image processing program according to another aspect of the present disclosure is a program for causing one or more processors to execute the following steps: obtain the result of a character recognition process performed on a character string of a specified item included in an image; calculate an evaluation value for each of a plurality of pre-registered registered character strings corresponding to the specified item based on setting values ​​set for each of a plurality of character string candidates included in the result of the character recognition process; identify the specified registered character string based on the evaluation value for each of the plurality of registered character strings; and output the specified registered character string. Effect of the Invention

[0009] According to the present disclosure, it is possible to provide an image processing system, an image processing method, and an image processing program capable of improving the accuracy of recognizing character strings in an input image. [Brief description of the drawings]

[0010] [Figure 1] FIG. 1 is a functional block diagram showing a configuration of an image processing device according to an embodiment of the present disclosure. [Diagram 2] FIG. 2 is a diagram illustrating an example of a document (quote) according to an embodiment of the present disclosure. [Figure 3A] FIG. 3A is a diagram showing an example of a target character string for character recognition processing according to an embodiment of the present disclosure. [Figure 3B] FIG. 3B is a diagram illustrating an example of a character rectangle obtained by detecting characters in a target character string according to an embodiment of the present disclosure. [Figure 3C] FIG. 3C is a diagram showing the result of conventional character recognition processing. [Figure 4] FIG. 4 is a diagram illustrating an example of score information used in the image processing device according to the embodiment of the present disclosure. [Diagram 5] FIG. 5 is a diagram showing an example of an address list used in the image processing device according to an embodiment of the present disclosure. [Figure 6A] FIG. 6A is a diagram illustrating an example of character recognition processing according to an embodiment of the present disclosure. [Figure 6B] FIG. 6B is a diagram illustrating an example of character recognition processing according to an embodiment of the present disclosure. [Figure 7A] FIG. 7A is a diagram illustrating an example of character recognition processing according to an embodiment of the present disclosure. [Figure 7B] FIG. 7B is a diagram illustrating an example of character recognition processing according to an embodiment of the present disclosure. [Figure 8] FIG. 8 is a flowchart showing an example of a procedure of image processing executed in the image processing device according to the embodiment of the present disclosure. [Figure 9A] FIG. 9A is a diagram showing an example of a target character string for character recognition processing according to an embodiment of the present disclosure. [Figure 9B]FIG. 9B is a diagram illustrating an example of a character rectangle obtained by detecting characters in a target character string according to an embodiment of the present disclosure. [Figure 9C] FIG. 9C is a diagram showing the result of conventional character recognition processing. [Figure 10] FIG. 10 is a diagram illustrating an example of character recognition processing according to an embodiment of the present disclosure. [Figure 11A] FIG. 11A is a diagram illustrating another example of the character recognition process according to an embodiment of the present disclosure. [Figure 11B] FIG. 11B is a diagram illustrating another example of the character recognition process according to an embodiment of the present disclosure. [Figure 12A] FIG. 12A is a diagram showing another example of a target character string for character recognition processing according to an embodiment of the present disclosure. [Figure 12B] FIG. 12B is a diagram showing another example of a character rectangle obtained by detecting characters in a target character string according to an embodiment of the present disclosure. [Figure 13A] FIG. 13A is a diagram showing another example of a target character string for character recognition processing according to an embodiment of the present disclosure. [Figure 13B] FIG. 13B is a diagram showing another example of a character rectangle obtained by detecting characters in a target character string according to an embodiment of the present disclosure. [Figure 14A] FIG. 14A is a diagram showing another example of a target character string for character recognition processing according to an embodiment of the present disclosure. [Figure 14B] FIG. 14B is a diagram showing another example of a character rectangle obtained by detecting characters in a target character string according to an embodiment of the present disclosure. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0011] Hereinafter, embodiments of the present disclosure will be described with reference to the accompanying drawings. Note that the following embodiments are examples of the present disclosure and are not intended to limit the technical scope of the present disclosure.

[0012] [Image processing device 1] 1 is a block diagram showing a configuration of an image processing device 1 according to an embodiment of the present disclosure. The image processing device 1 recognizes a character string included in an input image (image data), performs a well-known character recognition process (OCR process) on the recognized character string, and further performs a predetermined image process (correction process) on the result of the OCR process, and outputs the process result (correction result). Note that the image processing device 1 may obtain the result of the OCR process performed in an external device and perform the predetermined image process.

[0013] 1, the image processing device 1 includes a control unit 11, a storage unit 12, an operation display unit 13, and a communication unit 14. The image processing device 1 is an information processing device such as a personal computer or a cloud server. The image processing device 1 may be one or more cloud servers, or one or more physical servers.

[0014] The communication unit 14 is a communication interface for connecting the image processing device 1 to a network in a wired or wireless manner and for executing data communication with an external device via the network in accordance with a predetermined communication protocol. The network may be, for example, the Internet or a LAN.

[0015] The operation display unit 13 is a user interface that includes a display unit, such as a liquid crystal display or an organic EL display, that displays various information, and an operation unit, such as a mouse, a keyboard, or a touch panel, that accepts operations.

[0016] The storage unit 12 is a non-volatile storage unit such as an HDD (Hard Disk Drive), an SSD (Solid State Drive), or a flash memory that stores various information. The storage unit 12 stores a control program such as an image processing program (one example of an image processing program of the present disclosure) for causing the control unit 11 to execute image processing (see FIG. 8) described below. For example, the image processing program is non-temporarily recorded on a computer-readable recording medium such as a CD or a DVD, and is read by a reading device (not shown) such as a CD drive or a DVD drive provided in the image processing device 1 and stored in the storage unit 12. The image processing program may be distributed from a cloud server and stored in the storage unit 12.

[0017] The storage unit 12 also stores image data (scanned data, etc.) of documents and the like acquired from external devices.

[0018] FIG. 2 shows an estimate as an example of a document. As shown in FIG. 2, the estimate includes multiple character strings such as the document category ("estimate"), issue date, destination, contact information of the estimator (company name, postal code, address, phone number, etc.), estimated amount, product name, quantity, standard price, discount amount, subtotal, consumption tax, total amount, etc. The user scans the estimate in an external device and uploads image data (input image) to the image processing device 1. When the control unit 11 acquires the image data of the estimate, it stores it in the storage unit 12. As another embodiment, the control unit 11 may acquire a document file of the estimate created in an external device and store the document file in the storage unit 12.

[0019] The control unit 11 has control devices such as a CPU, a ROM, and a RAM. The CPU is a processor that executes various arithmetic operations. The ROM stores in advance control programs such as a BIOS and an OS for causing the CPU to execute various processes. The RAM stores various information and is used as a temporary storage memory (work area) for the various processes executed by the CPU. The control unit 11 controls the image processing device 1 by executing, by the CPU, various control programs that are stored in advance in the ROM or the storage unit 12.

[0020] When the control unit 11 acquires image data of a document, it performs preprocessing such as resolution conversion processing, skew correction processing, and background removal processing on the image data, and then performs character string (document part) recognition processing (extraction processing). For example, the control unit 11 recognizes each part such as character strings, tables, illustrations, and seals as rectangular areas of objects in the image data that has been subjected to the preprocessing. Furthermore, after performing document part recognition processing, the control unit 11 performs character recognition processing (OCR processing) on ​​the entire image data (entire image). Furthermore, the control unit 11 performs correction processing to correct the results of the OCR processing.

[0021] In this embodiment, the control unit 11 performs OCR processing and correction processing on character strings of predetermined items included in the input image. The predetermined items include at least one of an address, a telephone number, a postal code, and a company name. In the following, an example of the predetermined item will be described using "address."

[0022] Here, problems in the conventional technology will be described with reference to Figs. 3A to 3C. Fig. 3A shows a specific example of a character string to be subjected to OCR processing. For example, the address shown in Fig. 3A shows the correct address written on the quotation shown in Fig. 2. For example, a conventional system detects a character rectangle K1, which is a character area of ​​each character of the target character string, from an input image. Fig. 3B shows a state in which the correct character rectangle K1 has been detected for each character of the address shown in Fig. 3A. When the system detects each character rectangle K1, it executes OCR processing on the characters included in each character rectangle K1 and outputs multiple character string candidates. Fig. 3C shows five character strings, the first candidate to the fifth candidate. The system outputs the first candidate character string out of the five character strings as a result of OCR processing.

[0023] When the first candidate character string output as a result of OCR processing contains erroneously recognized characters, a problem occurs in which the correct result cannot be obtained. In the example shown in FIG. 3C, the third and fifth characters of the first candidate character string are erroneously recognized compared to the original address (see FIG. 3A). In this way, a method of adopting the result of a specific candidate (first candidate) from multiple candidates output by OCR processing of a target character string causes a problem of erroneously recognized characters being mixed in. In response to this, the image processing device 1 according to the present disclosure has a configuration capable of correcting the result of OCR processing and outputting a correct character string, as shown below.

[0024] Specifically, as shown in Fig. 1, the control unit 11 includes various processing units such as a detection processing unit 111, a recognition processing unit 112, an acquisition processing unit 113, a calculation processing unit 114, a specific processing unit 115, and an output processing unit 116. The control unit 11 functions as the various processing units by executing various processes according to the image processing program. Some or all of the processing units included in the control unit 11 may be configured with electronic circuits. The image processing program may be a program for causing a plurality of processors to function as the various processing units.

[0025] The control unit 11 acquires an input image. Specifically, the control unit 11 acquires an image (input data) to be subjected to character recognition from an external device. For example, the control unit 11 acquires image data of various document images such as an estimate sheet shown in FIG.

[0026] The detection processing unit 111 detects character rectangles K1, which are character regions of each character in a target character string corresponding to a predetermined item from an input image. For example, for a target character string of an address (an example of a predetermined item) shown in FIG. 3A, the detection processing unit 111 detects character rectangles K1 of each character (see FIG. 3B). The recognition processing unit 112 executes character recognition processing (OCR processing). Specifically, the recognition processing unit 112 executes OCR processing on the characters in the character rectangle K1 detected by the detection processing unit 111. The recognition processing unit 112 outputs a plurality of character string candidates through the OCR processing. For example, as shown in FIG. 3C, the recognition processing unit 112 outputs five character strings, a first candidate to a fifth candidate. Furthermore, the recognition processing unit 112 outputs the first candidate to the fifth candidate in descending order of the recognition accuracy (degree of match, etc.) of each character of the target character string.

[0027] The acquisition processing unit 113 acquires the result of OCR processing executed on character strings of a predetermined item included in the input image. Specifically, the acquisition processing unit 113 acquires the result of OCR processing executed by the recognition processing unit 112. For example, the acquisition processing unit 113 acquires five character strings, the first candidate to the fifth candidate (see FIG. 3C). As another embodiment, the acquisition processing unit 113 may acquire the result of OCR processing executed in an external device.

[0028] The calculation processing unit 114 calculates an evaluation value for each of a plurality of pre-registered registered character strings corresponding to the predetermined item based on a score (an example of a setting value in this disclosure) set for each of a plurality of character string candidates included in the result of the OCR processing. Specifically, for each of a plurality of registered character strings, the calculation processing unit 114 identifies the character string candidate that includes the character among the plurality of character string candidates for each character constituting the registered character string, sets a score (setting value) corresponding to the identified character string candidate, and calculates the evaluation value based on the total value of the scores set for each character.

[0029] Specifically, score information D1 and an address list D2 are stored in the storage unit 12. Fig. 4 shows an example of the score information D1, and Fig. 5 shows an example of the address list D2.

[0030] As shown in FIG. 4, the score information D1 has registered therein a candidate order and a score associated with each other. The candidate order is information corresponding to candidates for a character string output by the OCR process. The scores are scores that are set in advance for each of the candidate orders. For example, the control unit 11 sets "0" for the first candidate, "1" for the second candidate, "2" for the third candidate, "3" for the fourth candidate, "4" for the fifth candidate, and "5" for the remaining candidates.

[0031] 5, a plurality of existing addresses are registered in advance in the address list D2. Each address registered in the address list D2 is an example of the registered character string. The address list D2 may be registered by an administrator of the image processing device 1, or may be obtained from an external device and stored in the storage unit 12. For example, the address list D2 may be downloaded from a server of a local government and stored in the storage unit 12.

[0032] The calculation processing unit 114 calculates an evaluation value for each of a plurality of addresses (registered addresses) registered in the address list D2 based on the score information D1 and the address list D2. Hereinafter, a method for calculating the evaluation value of the registered address No. 3, "Minomura-cho, Yamato Koriyama City", registered in the address list D2, and a method for calculating the evaluation value of the registered address No. 4, "Kitagunzan-machi, Yamato Koriyama City", will be described.

[0033] Specifically, the calculation processing unit 114 collates each character of the registered address "Minomura-cho, Yamato Koriyama City" with the character string candidates obtained by the OCR process, and calculates the score of each character. For example, since the first character "Da" of the registered address matches the first character of the first candidate character string candidate (see FIG. 6A), the calculation processing unit 114 sets the score "0" (see FIG. 4) associated with the first candidate in the score information D1 for the character "Da" (see "Character-by-character score" in FIG. 6B).

[0034] Also, for example, since the second character "Wa" of the registered address matches the second character of the first candidate character string candidate (see FIG. 6A), the calculation processing unit 114 sets the score "0" (see FIG. 4) associated with the first candidate in the score information D1 for the character "Wa" (see "Character-by-character score" in FIG. 6B).

[0035] Also, for example, since the third character "Gun" of the registered address matches the third character of the second candidate character string candidate (see FIG. 6A), the calculation processing unit 114 sets the score "1" (see FIG. 4) associated with the second candidate in the score information D1 for the character "Gun" (see "Character-by-character score" in FIG. 6B).

[0036] Also, for example, since the fourth character "Yama" of the registered address matches the fourth character of the first candidate character string candidate (see FIG. 6A), the calculation processing unit 114 sets the score "0" (see FIG. 4) associated with the first candidate in the score information D1 for the character "Yama" (see "Character-by-character score" in FIG. 6B).

[0037] For example, since the fifth character "shi" (city) of the registered address matches the fifth character of the candidate string of the third candidate (see FIG. 6A), the calculation processing unit 114 sets the score "2" (see FIG. 4) associated with the second candidate of the score information D1 for the character "shi" (see "character-by-character score" in FIG. 6B).

[0038] For example, since the sixth character "mei" of the registered address matches the sixth character of the candidate string of the first candidate (see FIG. 6A), the calculation processing unit 114 sets the score "0" (see FIG. 4) associated with the first candidate of the score information D1 for the character "mei" (see "character-by-character score" in FIG. 6B).

[0039] For example, since the seventh character "nong" of the registered address matches the seventh character of the candidate string of the first candidate (see FIG. 6A), the calculation processing unit 114 sets the score "0" (see FIG. 4) associated with the first candidate of the score information D1 for the character "nong" (see "character-by-character score" in FIG. 6B).

[0040] For example, since the eighth character "zhuang" of the registered address matches the eighth character of the candidate string of the first candidate (see FIG. 6A), the calculation processing unit 114 sets the score "0" (see FIG. 4) associated with the first candidate of the score information D1 for the character "zhuang" (see "character-by-character score" in FIG. 6B).

[0041] For example, since the ninth character "ting" of the registered address matches the ninth character of the candidate string of the first candidate (see FIG. 6A), the calculation processing unit 114 sets the score "0" (see FIG. 4) associated with the first candidate of the score information D1 for the character "ting" (see "character-by-character score" in FIG. 6B).

[0042] When the calculation processing unit 114 sets a score for each character of the registered address, it calculates the total value of each score. Here, the calculation processing unit 114 calculates a total score of "3". Next, the calculation processing unit 114 divides the total score by the number of characters in the registered address. Here, since the registered address is composed of 9 characters, the calculation processing unit 114 divides the total score of "3" by 9 characters (3÷9). The calculation processing unit 114 uses the calculation result "0.333" as the evaluation value of the registered address "Minonokoujyoucho, Yamato Koriyama City".

[0043] Similarly, the calculation processing unit 114 calculates the evaluation value of the registered address "Kitagunzanmachi, Yamato Koriyama City". For example, the first character "Da" of the registered address matches the first character of the first candidate string candidate (see Fig. 7A), the second character "Wa" matches the second character of the first candidate string candidate, the third character "Gun" matches the third character of the second candidate string candidate, the fourth character "San" matches the fourth character of the first candidate string candidate, and the fifth character "Shi" matches the fifth character of the third candidate string candidate. Also, the sixth character "North", the seventh character "Gun", and the eighth character "San" are not included in any of the candidates. Also, the ninth character "Machi" matches the ninth character of the first candidate string candidate.

[0044] As shown in Fig. 7B, the calculation processing unit 114 sets the score of the score information D1 (see Fig. 4) for each character. Next, the calculation processing unit 114 calculates the total value of each score (total score "18"). Then, the calculation processing unit 114 divides the total score by the number of characters in the registered address. Here, the calculation processing unit 114 divides the total score of "18" by 9 characters (18÷9). The calculation processing unit 114 uses the calculation result "2.0" as the evaluation value of the registered address "Kitagunzanmachi, Yamato Koriyama City".

[0045] In this manner, the calculation processing unit 114 calculates the evaluation value for each of the addresses registered in the address list D2. The calculation processing unit 114 may calculate the evaluation value for an address that has the same or similar number of characters as the target character string among the addresses registered in the address list D2. This improves processing efficiency because it is not necessary to calculate the evaluation value for all addresses registered in the address list D2.

[0046] The identification processing unit 115 identifies a predetermined registered character string based on the evaluation value of each of the multiple registered character strings calculated by the calculation processing unit 114. Specifically, the identification processing unit 115 identifies one registered address based on the evaluation value of each registered address calculated by the calculation processing unit 114.

[0047] For example, the identification processing unit 115 identifies the registered address with the smallest evaluation value among the multiple evaluation values ​​calculated by the calculation processing unit 114. For example, the identification processing unit 115 identifies the registered address "Minosho-cho, Yamatokoriyama City" with the smallest evaluation value of "0.333."

[0048] The output processing unit 116 outputs a predetermined registered character string identified by the identification processing unit 115. In the above example, the output processing unit 116 outputs the registered address "Minosho-cho, Yamatokoriyama City." That is, instead of the first to fifth candidate character strings output by the OCR processing, the output processing unit 116 outputs the registered character string identified by the identification processing unit 115 as the result of the OCR processing. As a result, a character string that is erroneously recognized in the OCR processing (see FIG. 3C) is not output, and a specific registered character string that does not include erroneously recognized characters (see FIG. 5) is output, so that the user can obtain an appropriate OCR result.

[0049] In addition, when there are many characters misrecognized in the OCR process, the evaluation value of each registered character string becomes high, and there is a risk that the reliability of the registered character string finally selected decreases. Therefore, the control unit 11 may set a threshold value for the total score of the registered character string. In this case, the identification processing unit 115 excludes registered character strings whose total score is equal to or greater than the threshold, and identifies registered character strings whose total score is less than the threshold. In another embodiment, the control unit 11 may set a threshold value for the evaluation value of the registered character string. In this case, the identification processing unit 115 excludes registered character strings whose evaluation value is equal to or greater than the threshold, and identifies registered character strings whose evaluation value is less than the threshold.

[0050] As another embodiment, the control unit 11 may set "5" to the first candidate, "4" to the second candidate, "3" to the third candidate, "2" to the fourth candidate, "1" to the fifth candidate, and "0" to the remaining candidates in the score information D1. In this case, the identification processing unit 115 identifies a registered address having the maximum evaluation value among the multiple evaluation values ​​calculated by the calculation processing unit 114. That is, in the image processing device 1 according to the present disclosure, the identification processing unit 115 identifies the registered character string having the minimum or maximum evaluation value among the multiple registered character strings. In addition, the higher the accuracy of the character recognition, the lower or higher the score set for each of the multiple character string candidates in the score information D1.

[0051] [Image processing] FIG. 8 is a flowchart showing an example of the procedure of image processing executed in the image processing device 1 according to this embodiment.

[0052] The present disclosure can be understood as an image processing method (image processing method of the present disclosure) that executes one or more steps included in the image processing. One or more steps included in the image processing described here may be omitted as appropriate. The steps in the image processing may be executed in a different order as long as the same action and effect is achieved. Furthermore, although an example is described here in which the control unit 11 of the image processing device 1 executes each step in the image processing, in other embodiments, one or more processors may execute each step in the image processing in a distributed manner.

[0053] In step S1, the control unit 11 acquires an input image. For example, the control unit 11 acquires image data of a document image (quote sheet) shown in FIG.

[0054] Next, in step S2, the control unit 11 detects character rectangles K1, which are character regions of each character in a target character string corresponding to a predetermined item from the input image. Here, the predetermined item is taken as an example of "address" included in an estimate. For example, for the target character string of an address shown in FIG. 3A, the control unit 11 detects character rectangles K1 of each character (see FIG. 3B).

[0055] Next, in step S3, the control unit 11 executes OCR processing. Specifically, the control unit 11 executes OCR processing on the characters in the character rectangle K1 and outputs multiple character string candidates. For example, as shown in FIG. 3C, the control unit 11 outputs five character strings, the first candidate to the fifth candidate.

[0056] Next, in step S4, the control unit 11 acquires the address list D2. Specifically, the control unit 11 acquires the address list D2 (see FIG. 5) in which a plurality of existing addresses (registered addresses) are registered in advance for the "address" corresponding to the predetermined item. The control unit 11 may acquire the address list D2 from an external device, or may acquire the address list D2 registered in advance in the storage unit 12 by a registration operation by the user in the image processing device 1.

[0057] Next, in step S5, the control unit 11 calculates an evaluation value for each of a plurality of pre-registered registered addresses corresponding to "address" based on the scores set for each of the plurality of character string candidates output in step S3. For example, the control unit 11 calculates an evaluation value for each of a plurality of registered addresses registered in the address list D2 (see FIG. 5) based on the five character strings (first to fifth candidates) and the score information D1.

[0058] As described above, the control unit 11 identifies which of the first to fifth candidate character strings in the OCR result the characters of the registered address are included in, obtains a score value associated with the identified candidate number from the score information D1 (see FIG. 4), and assigns the obtained score value to the character (see FIGS. 6B and 7B). After assigning the score value to each character of the registered address, the control unit 11 calculates an evaluation value for the registered address by adding up the scores and dividing the total score by the number of characters in the registered address. The control unit 11 calculates the evaluation value for each of the multiple registered addresses registered in the address list D2.

[0059] Next, in step S6, the control unit 11 identifies a specific registered address from the multiple registered addresses registered in the address list D2. For example, the control unit 11 identifies the registered address with the smallest evaluation value among the multiple registered addresses.

[0060] Finally, in step S7, the control unit 11 outputs the registered address identified in step S6 as the OCR result.

[0061] In this manner, the control unit 11 executes the image processing. The control unit 11 executes the image processing every time an input image is acquired.

[0062] As described above, the image processing device 1 according to this embodiment acquires the result of character recognition processing (OCR processing) executed on a character string of a predetermined item (e.g., "address") included in an image (input image), and calculates an evaluation value for each of a plurality of registered character strings (e.g., registered address) that are registered in advance and correspond to the predetermined item based on a setting value (score value) that is set for each of a plurality of character string candidates included in the result of the character recognition processing. The image processing device 1 identifies the predetermined registered character string based on the calculated evaluation value for each of the plurality of registered character strings, and outputs the identified predetermined registered character string.

[0063] According to the above configuration, even if the result of OCR processing on a character string related to an address contains an erroneously recognized character (see FIG. 3C), the character string containing the erroneously recognized character (first candidate character) is not output, and the appropriate registered address contained in the pre-registered address list D2 is output as the OCR result. In this way, it is possible to correct the erroneous recognition that occurs when the first candidate for each character is used as in conventional OCR processing. Therefore, it is possible to improve the recognition accuracy of the character string in the input image.

[0064] [Other embodiments] In OCR processing, there are cases where detection (extraction) of a character rectangle fails (detection failure) for any of a plurality of characters included in a character string. Problems that arise when detection failure occurs will be described with reference to Figs. 9A to 9C. Fig. 9A shows a specific example of a character string to be subjected to OCR processing, similar to Fig. 3A. For example, a conventional system detects a character rectangle K1, which is a character area of ​​each character of a target character string, from an input image. Fig. 9B shows a state in which the first character of the address character string shown in Fig. 9A could not be detected. The system performs OCR processing on the characters included in each detected character rectangle K1, and outputs a plurality of character string candidates. Fig. 9C shows five character strings, the first candidate to the fifth candidate. The system outputs the first candidate character string of the five character strings as a result of OCR processing.

[0065] As a result of the OCR process, a first candidate character string is output. This OCR result includes the missing first character and the erroneously recognized third and fifth characters. In this way, if a character is not detected, a result with missing characters is output. In response to this, the image processing device 1 according to the present disclosure may be configured to correct the result of the OCR process and output a correct character string even if a character is not detected, as described below.

[0066] Specifically, the calculation processing unit 114 determines whether or not there is a character that was not detected in the OCR process for the target character string. If the calculation processing unit 114 determines that there is a character that was not detected, it identifies a character missing position that indicates the position of the character. Then, the calculation processing unit 114 sets a score value for each character that constitutes the registered character string based on the identified character missing position.

[0067] For example, as shown in FIG. 10, the detection processing unit 111 detects a character string rectangle K2, which is a region of a character string, for a target character string, and detects a character rectangle K3, which is a region for each character that constitutes the character string, in the character string rectangle K2. The calculation processing unit 114 compares the coordinates of the character rectangle K1 obtained by OCR processing with the coordinates of the character rectangle K3 to determine the presence or absence of a missing character and the position of the missing character. For example, as shown in FIG. 10, the calculation processing unit 114 compares the coordinates (x1, y1), (x2, y2) of the character rectangle K1 with the coordinates (x1', y1'), (x2', y2') of the character rectangle K3 for each character rectangle, and estimates that the first character of the target character string is missing. The calculation processing unit 114 may compare the center coordinates of each character. The coordinates (0, 0) are the reference point (origin) of the coordinate system of the input image, and specifically, the coordinates of the reference point are the pixel in the upper left corner of the input image or any position of the input image.

[0068] In this way, the calculation processing unit 114 detects the character string rectangle K2 of the target character string and the character rectangle K3 for each character that constitutes the target character string, and identifies the position of the missing character based on the result of the OCR process on the target character string and the character string rectangle K2 and the character rectangle K3. The calculation processing unit 114 also identifies the position of the missing character based on the result of the OCR process on the target character string and the coordinate information of the character string rectangle K2 and the character rectangle K3 (for example, the position coordinates of the character string rectangle K2 and the character rectangle K3 relative to the coordinates of the reference point of the input image). The calculation processing unit 114 may detect the character string rectangle K2 and the character rectangle K3 using AI or the like.

[0069] When the calculation processing unit 114 identifies the missing character position, it executes score value setting processing and evaluation value calculation processing for the registered address as shown in the above-mentioned embodiment. For example, as shown in Figures 11A and 11B, the calculation processing unit 114 sets a score value of "5" (see Figure 4) for the character corresponding to the missing character position (first character) for the registered address "Minosho-cho, Yamatokoriyama City". The score values ​​of the other characters are the same as the score value shown in Figure 6B.

[0070] After setting a score for each character in the registered address, the calculation processing unit 114 calculates a total score and divides the total score by the number of characters in the registered address to calculate an evaluation value.

[0071] According to the above configuration, even if the result of OCR processing on a character string related to an address includes missing characters, the missing character string (first candidate character) is not output, and an appropriate registered address included in the pre-registered address list D2 is output as the OCR result. This makes it possible to improve the recognition accuracy of the character string in the input image.

[0072] The calculation processing unit 114 may calculate the evaluation value for an address having a number of characters that matches or is close to the sum of the number of characters in the target character string and the number of missing characters, among the multiple addresses registered in the address list D2. This improves processing efficiency because it is not necessary to calculate the evaluation value for all addresses registered in the address list D2.

[0073] In the above embodiment, "address" is given as an example of the predetermined item, but other items such as "zip code", "telephone number", and "company name" may be used.

[0074] FIG. 12A shows a target character string when the item is a postal code. The detection processing unit 111 detects a character rectangle K1, which is a character area of ​​each character of the target character string of the postal code (see FIG. 12B). When the item is a postal code, the memory unit 12 stores a postal code list including a plurality of existing postal codes. The calculation processing unit 114 calculates an evaluation value for each of a plurality of postal codes (registered postal codes) registered in the postal code list based on the score information D1 (see FIG. 4) and the postal code list. The identification processing unit 115 identifies a specific registered postal code based on the evaluation value of each of the multiple registered postal codes calculated by the calculation processing unit 114, and the output processing unit 116 outputs the specific registered postal code identified by the identification processing unit 115 as an OCR result.

[0075] In addition, when the control unit 11 detects the character "〒" in the input image, it may determine that the seven digits following "〒" are a string of characters that constitute a postal code, obtain a postal code list, and calculate the evaluation value of each registered postal code.

[0076] FIG. 13A shows a target character string when the item is a telephone number. The detection processing unit 111 detects a character rectangle K1, which is a character area of ​​each character of the target character string of the telephone number (see FIG. 13B). Furthermore, when the item is a telephone number, the storage unit 12 stores a telephone number list including a plurality of existing telephone numbers. The calculation processing unit 114 calculates an evaluation value for each of the plurality of telephone numbers (registered telephone numbers) registered in the telephone number list based on the score information D1 (see FIG. 4) and the telephone number list. The identification processing unit 115 identifies a specific registered telephone number based on the evaluation value of each of the plurality of registered telephone numbers calculated by the calculation processing unit 114, and the output processing unit 116 outputs the specific registered telephone number identified by the identification processing unit 115 as an OCR result.

[0077] In addition, when the control unit 11 detects characters such as "telephone," "telephone number," or "TEL" in the input image, it may determine that the 10-digit number following these characters is a string of characters representing a telephone number, obtain a telephone number list, and calculate the evaluation value of each registered telephone number.

[0078] FIG. 14A shows a target character string when the item is a company name. The detection processing unit 111 detects a character rectangle K1, which is a character area of ​​each character of the target character string of the company name (see FIG. 14B). Furthermore, when the item is a company name, the storage unit 12 stores a company name list including a plurality of existing company names. The calculation processing unit 114 calculates an evaluation value for each of the plurality of company names (registered company names) registered in the company name list based on the score information D1 (see FIG. 4) and the company name list. The identification processing unit 115 identifies a specific registered company name based on the evaluation value of each of the plurality of registered company names calculated by the calculation processing unit 114, and the output processing unit 116 outputs the specific registered company name identified by the identification processing unit 115 as an OCR result.

[0079] The control unit 11 may weight the scores so that, from among the registered company names included in the company name list, a particular company, for example, a company with which the company has done business in the past, is preferentially selected.

[0080] The image processing device 1 according to the embodiment of the present disclosure may appropriately combine the above-mentioned embodiments. For example, when performing character recognition of an "address", the control unit 11 may extract a registered address corresponding to at least one of a "telephone number" and a "zip code" from the address list D2, and calculate the evaluation value for the extracted registered address. This can reduce the number of registered addresses for which the evaluation value is calculated, thereby suppressing the processing load.

[0081] The predetermined item may also be a document classification (such as "quote," "invoice," or "delivery note"), a product name, or the like.

[0082] Moreover, the image processing device 1 is an example of an image processing system according to the present disclosure. That is, the image processing system according to the present disclosure may be configured by the image processing device 1 alone. Moreover, the image processing system according to the present disclosure may be configured by the image processing device 1 and a character recognition device that executes OCR processing. Moreover, the image processing system according to the present disclosure may be configured by the image processing device 1, the character recognition device, and an output device that outputs (displays) the character recognition result. For example, the output device may be a user terminal (personal computer, smartphone, etc.), and the user terminal may execute a character recognition application (Web application) to request the image processing device 1 to perform character recognition processing of a document image, and obtain and display the character recognition result from the image processing device 1.

[0083] [Disclosure Note] The following will provide an outline of the disclosure extracted from the above-described embodiment. Note that the configurations and processing functions described in the following supplementary notes can be selected and combined in any desired manner.

[0084] <Appendix 1> an acquisition processing unit that acquires a result of a character recognition process executed on a character string of a predetermined item included in an image; a calculation processing unit that calculates an evaluation value for each of a plurality of pre-registered character strings corresponding to the predetermined item based on a setting value that is set for each of a plurality of character string candidates included in a result of the character recognition process; an identification processing unit that identifies a predetermined registered character string based on the evaluation value of each of the plurality of registered character strings calculated by the calculation processing unit; an output processing unit that outputs the predetermined registered character string identified by the identification processing unit; An image processing system comprising:

[0085] <Appendix 2> The calculation processing unit, for each of the plurality of registered character strings, identifying a character string candidate including the character among the plurality of character string candidates for each character constituting the registered character string, and setting the setting value corresponding to the identified character string candidate; calculating the evaluation value based on a total value of the setting values ​​that are set for each of the characters; 2. An image processing system as described in claim 1.

[0086] <Appendix 3> the calculation processing unit calculates the evaluation value for each of the plurality of registered character strings by dividing the total value by the number of characters in the registered character string. 3. An image processing system as described in claim 2.

[0087] <Appendix 4> the identification processing unit identifies a registered character string having a minimum or maximum evaluation value from among the plurality of registered character strings. 4. An image processing system as described in claim 3.

[0088] <Appendix 5> a lower or higher setting value is set for each of the plurality of character string candidates as the accuracy of the character recognition increases; 5. An image processing system according to any one of claims 1 to 4.

[0089] <Appendix 6> the calculation processing unit, when a character that was not detected in the character recognition process for the character string exists, specifies a missing character position indicating a position of the undetected character, and sets the setting value for each character constituting the registered character string based on the specified missing character position. 6. An image processing system according to any one of claims 2 to 5.

[0090] <Appendix 7> the calculation processing unit detects a character string rectangle which is an area of ​​the character string and a single character rectangle which is an area for each character constituting the character string, and identifies the character missing position based on a result of the character recognition process for the character string and the character string rectangle and the single character rectangle. 7. The image processing system according to claim 6.

[0091] <Appendix 8> the calculation processing unit specifies the character missing position based on a result of the character recognition process for the character string and position coordinates of the character string rectangle and the single character rectangle relative to reference coordinates set on the image. 8. The image processing system of claim 7.

[0092] <Appendix 9> The predetermined item is at least one of an address, a telephone number, a postal code, and a company name included in the document image. An image processing system according to any one of claims 1 to 8. [Explanation of symbols]

[0093] 1: Image processing device 11: Control section 12: Storage section 13: Operation display section 14: Communications Department 111: Detection processing unit 112: Recognition processing unit 113: Acquisition processing unit 114: Calculation processing unit 115: Specific Processing Unit 116: Output processing section D1: Score information D2: Address list K1: Character rectangle K2: String rectangle K3: Single character rectangle

Claims

1. an acquisition processing unit that acquires a result of a character recognition process executed on a character string of a predetermined item included in an image; a calculation processing unit that calculates an evaluation value for each of a plurality of pre-registered character strings corresponding to the predetermined item based on a setting value that is set for each of a plurality of character string candidates included in a result of the character recognition process; an identification processing unit that identifies a predetermined registered character string based on the evaluation value of each of the plurality of registered character strings calculated by the calculation processing unit; an output processing unit that outputs the predetermined registered character string identified by the identification processing unit; An image processing system comprising:

2. The calculation processing unit, for each of the plurality of registered character strings, identifying a character string candidate including the character among the plurality of character string candidates for each character constituting the registered character string, and setting the setting value corresponding to the identified character string candidate; calculating the evaluation value based on a total value of the setting values ​​that are set for each of the characters; The image processing system according to claim 1 .

3. the calculation processing unit calculates the evaluation value for each of the plurality of registered character strings by dividing the total value by the number of characters in the registered character string. The image processing system according to claim 2 .

4. the identification processing unit identifies a registered character string having a minimum or maximum evaluation value from among the plurality of registered character strings. The image processing system according to claim 3 .

5. a lower or higher setting value is set for each of the plurality of character string candidates as the accuracy of the character recognition increases; The image processing system according to claim 1 .

6. the calculation processing unit, when a character that was not detected in the character recognition process for the character string exists, specifies a missing character position indicating a position of the undetected character, and sets the setting value for each character constituting the registered character string based on the specified missing character position.

6. The image processing system according to claim 2.

7. the calculation processing unit detects a character string rectangle which is an area of ​​the character string and a single character rectangle which is an area for each character constituting the character string, and identifies the character missing position based on a result of the character recognition process for the character string and the character string rectangle and the single character rectangle. The image processing system according to claim 6.

8. the calculation processing unit specifies the character missing position based on a result of the character recognition process for the character string and position coordinates of the character string rectangle and the single character rectangle relative to reference coordinates set on the image. The image processing system according to claim 7.

9. The predetermined item is at least one of an address, a telephone number, a postal code, and a company name included in the document image. The image processing system according to claim 1 .

10. Obtaining a result of a character recognition process performed on a character string of a predetermined item included in an image; calculating an evaluation value for each of a plurality of pre-registered character strings corresponding to the predetermined item based on a setting value that is set for each of a plurality of character string candidates included in the result of the character recognition process; identifying a specific registered character string based on the evaluation value of each of the plurality of registered character strings; outputting the predetermined registered character string; An image processing method executed by one or more processors.

11. Obtaining a result of a character recognition process performed on a character string of a predetermined item included in an image; calculating an evaluation value for each of a plurality of pre-registered character strings corresponding to the predetermined item based on a setting value that is set for each of a plurality of character string candidates included in the result of the character recognition process; identifying a specific registered character string based on the evaluation value of each of the plurality of registered character strings; outputting the predetermined registered character string; An image processing program for causing one or more processors to execute the above.

Images