Character string recognition system, character string recognition program and character string recognition method
The character string recognition system enhances accuracy by dividing document images into unit strings using ruled lines and multiple AI engines, reducing errors and ensuring precise text data conversion.
Patent Information
- Application Number
- JP2023075015
- Authority / Receiving Office
- JP · JP
- Patent Type
- Applications
- Current Assignee / Owner
- Filing Date
- 2023-04-28
- Publication Date
- 2024-11-08
- Estimated Expiration
- 2043-04-28
AI Technical Summary
Conventional character string recognition systems face errors when dealing with documents containing ruled lines such as frames and underlines, and machine learning-based systems may fail due to unsuitable character string images, leading to inaccurate recognition.
A character string recognition system that utilizes a ruled line detection unit to divide document images into unit character string images, applies multiple AI engines for recognition, and performs category determination using the type of AI engine with high evaluation values to ensure accurate conversion into text data.
The system improves character recognition accuracy by dividing strings based on ruled lines, reduces errors, and maintains high category determination accuracy through multiple AI engine usage, resulting in precise conversion of document images into text data.
Smart Images

Figure 2024159182000001_ABST
Abstract
Description
[Technical field]
[0001] The present invention relates to a character string recognition system that recognizes character strings in a document image and converts them into text data, and more particularly to a character string recognition system that is suitably applied to a document image with ruled lines. [Background technology]
[0002] In a conventional character string recognition system, as shown in Patent Document 1, for example, a character image is extracted from a document image, and the character image is subjected to character recognition processing using an OCR engine or the like to generate text data.
[0003] Furthermore, as described in Patent Document 2, machine learning may be used for character recognition. [Prior art documents] [Patent documents]
[0004] [Patent Document 1] JP 2010-217996 A [Patent Document 2] Patent No. 7077998 Summary of the Invention [Problem to be solved by the invention]
[0005] However, when character strings are decorated with frames, underlines, etc., this can lead to character recognition errors. In particular, slips such as receipts and invoices tend to have many frames, underlines, and other ruled lines, making them more susceptible to character recognition errors.
[0006] Furthermore, in machine learning, a single type of AI engine has traditionally been used, so when a character string image that the engine is not good at recognizing appears in a document image, character recognition errors can still occur.
[0007] The present invention has been made to solve the above-mentioned problems, and its main intended object is to provide a character string recognition system or the like that can more accurately convert character strings in a document image into text data. [Means for solving the problem]
[0008] That is, the character string recognition system according to the present invention comprises: A method for recognizing character strings from a ruled document image, comprising: a ruled line detection unit for detecting ruled lines from the ruled document image; a unit string image extraction unit that extracts a plurality of unit string images recognized as a series of character strings from the ruled document image based on the ruled lines detected by the ruled line detection unit; a character recognition processing unit 3 that performs character recognition processing using a plurality of types of AI engines on each of the unit character string images and obtains an evaluation value of each character recognition processing; a category determination unit that identifies an AI engine with a high evaluation value for each of the unit string images, and determines a category of the unit string displayed in each of the unit string images based on the type of the identified AI engine; The present invention is characterized in that it is provided with: Effect of the Invention
[0009] According to the present invention described above, the following effects can be obtained.
[0010] Generally, rules are used to separate character strings in a document into a series of character strings each having a specific meaning. That is to say, as in the present invention, by dividing character string images in a document based on (information relating to) ruled lines and treating each as a unit string image, the unit strings displayed in each unit string image become a series that are each coherent in meaning.
[0011] On the other hand, it is unlikely that the category (e.g., address, price, title, etc.) will change midway through a series of unit strings. In other words, it is considered that the unit string image extracted by the present invention displays a string consisting of a single category. Therefore, the configuration of the present invention in which category determination is performed using multiple types of AI engines is meaningful precisely because the unit string images extracted by the present invention are composed of a single string category, and as a result, the accuracy of category determination can be maintained at a high level. Furthermore, by utilizing this category determination, it becomes possible to greatly improve the accuracy of character recognition.
[0012] Furthermore, according to the present invention, lines in a lined document image can be detected and character recognition processing can be performed on a character string image based on information relating to the lines, thereby reducing the possibility of misidentifying part of a line as part of a character, which in turn causes a character recognition error, making it possible to perform more accurate character recognition in this respect as well. [Brief description of the drawings]
[0013] [Figure 1] FIG. 2 is a functional block diagram of the character string recognition system according to the present embodiment. [Diagram 2] 4 is a flowchart showing the operation of the character string recognition system according to the embodiment. [Diagram 3] 4 is a flowchart showing the operation of the character string recognition system according to the embodiment. [Figure 4] 10 is a diagram showing a schematic diagram of a process in the embodiment, in which a character string image included in a ruled document image is divided into unit character string images based on the ruled lines, and each unit character image is converted into text data. FIG. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
[0014] Hereinafter, a character string recognition system 100 according to an embodiment of the present invention will be described with reference to the drawings.
[0015] <Configuration> This character string recognition system 100 recognizes character strings from images of ruled documents such as forms, invoices, reports, and the like, in which ruled lines are used, and outputs text data of the character strings.
[0016] Let me explain in detail. This character string recognition system 100 is physically configured as a so-called computer device equipped with a CPU, memory, input / output ports, etc., and functionally, the CPU and its peripheral devices work together in accordance with a predetermined program stored in the memory to fulfill functions such as a ruled line detection unit 1, a unit string image extraction unit 2, a character recognition processing unit 3, a category determination unit 4, and a unit string correction unit 5, as shown in Fig. 1. Note that the computer device that constitutes this character string recognition system 100 does not need to be physically one, and may be configured as multiple computer devices that are connected to each other so as to be able to communicate with each other via a communication means such as the Internet.
[0017] The functions of each of these parts will now be outlined.
[0018] The ruled line detection unit 1 detects ruled lines from the ruled line document image.
[0019] The unit character string image extraction section 2 extracts a plurality of unit character string images recognized as a series of character strings from the ruled document image based on the ruled lines detected by the ruled line detection section 1.
[0020] The character recognition processing unit 3 performs character recognition processing using a plurality of types of specialized AIs on each of the unit character string images, and obtains an evaluation value of each character recognition processing.
[0021] The category determination unit 4 identifies a specialized AI with a high evaluation value for each unit string image, and determines the category of each unit string image based on the type (specialty) of the identified specialized AI. A category indicates the field (e.g., address, price, title) to which the object represented by the unit string belongs, and may also indicate the type of characters (e.g., numbers, English letters, half-width katakana, etc.) used in that field.
[0022] The unit string correction unit 5 corrects the character recognition result by the specified specialized AI for each unit string image by using dictionary data corresponding to the category.
[0023] <Operation> Next, the operation of this character string recognition system will be described with reference to Figs. 2 and 3, together with a detailed description of each of the above-mentioned components.
[0024] First, a user scans a ruled document to be converted into text data, converts it into a ruled document image, and inputs the image to the character string recognition system 100. Note that the image referred to here means image data in a predetermined format such as pdf, jpeg, bitmap, tiff, etc.
[0025] Next, the ruled line detection unit 1 acquires the ruled document image (FIG. 2: step S1) and detects the ruled lines contained therein (FIG. 2: step S2). The ruled lines referred to here are frames, underlines, etc. More specifically, the ruled line detection unit 1 identifies, for example, a region in which pixels of a color (e.g., black) different from the background color (e.g., white) continue linearly for a predetermined length or more in the vertical direction (direction parallel to the left and right sides of the paper) or the horizontal direction (direction parallel to the top and bottom sides of the paper), as a ruled line, using XY coordinates, pixel numbers, etc.
[0026] Next, the unit string image extraction unit 2 judges the shape of the ruled line (FIG. 2: step S3). For example, if the shape is a closed rectangle, it is judged to be a frame line, and if the shape is a straight line (line segment) with both ends extending horizontally, it is judged to be an underline. Regarding the frame line, in the case of a table, for example, in the upper and lower columns, the bottom line of the upper column is common to the top line of the lower column, but here it is judged that there are two frame lines, one forming the upper column and one forming the lower column.
[0027] Next, the unit string image extraction unit 2 divides the character string image into character string images within each surrounding frame line, character string images directly above each underline, and other character string images, and sets each character string image as a unit string image (FIG. 2: step S4). At this time, ruled lines such as surrounding frame lines and underlines are deleted from the unit string images.
[0028] If there are multiple frame lines in a document, the character string images within each frame line are treated as separate unit character string images. The same applies if there are multiple underlines. Furthermore, with regard to other character string images, those that are in areas that are a certain distance apart from each other are treated as separate unit character string images.
[0029] The character recognition processing unit 3 performs character recognition processing using a plurality of specialized or general-purpose AI engines on each unit string image extracted in the above manner, and obtains unit string text data (FIG. 3: step S6). At that time, an evaluation value (here, for example, likelihood) of the character recognition processing performed by each AI engine is also obtained (FIG. 3: step S7).
[0030] The types of AI engines available here include an AI engine that has been trained in Japanese, an AI engine that has been trained in Japanese addresses, a number-specialized AI engine, an alphanumeric character-specialized AI engine, a kana-specialized AI engine, and a half-width kana-specialized AI engine.
[0031] Next, the category determination unit 4 identifies a specialized AI with a high evaluation value for each unit string image, and determines its specialty as the category of the unit string image (FIG. 3: step S8). For example, if the evaluation value of an AI engine that has learned Japanese addresses is high for one unit string image, the category shown in the unit string image is determined to be a Japanese address.
[0032] Next, the unit string correction unit 5 selects dictionary data corresponding to the category determined by the category determination unit 4 for each unit string image (FIG. 3: step S9).
[0033] Then, the unit string correction unit 5 obtains the character recognition result by the specified AI engine, i.e., unit string text data, and corrects the unit string text data using dictionary data corresponding to the category determined by the category determination unit 4 (FIG. 3: step S10). A plurality of types of dictionary data are provided corresponding to the types of AI engines.
[0034] As described above, if the category of the unit string text data is a Japanese address, the unit string correction unit 5 uses dictionary data related to Japanese addresses to make corrections. For example, if the recognition (conversion) result before correction is "Otsu City, Miidera Town", there is no such address in the Japanese address dictionary data, and the closest address is "Otsu City, Miidera Town", so the unit string correction unit 5 converts "Otsu" to "Otsu". The results of this correction are then used for learning in each AI engine.
[0035] FIG. 4 is a diagram that illustrates the process of dividing a character string image included in a lined document image into unit character string images based on detected ruled lines and converting each unit character image into text data, as described above.
[0036] <Effects> According to the above configuration, lines in a lined document image are detected and character recognition processing is performed on the character string image based on information related to the lines, thereby reducing the possibility of misidentifying part of a line as part of a character, which may cause a character recognition error, and enabling more accurate character recognition.
[0037] Furthermore, since ruled lines are generally used to divide character strings in a document into a series of character strings, each of which has its own meaning, in this configuration, character string images in a document are divided based on (information relating to) ruled lines, and each is treated as a unit string image, so that the character strings contained in the unit string images form a series that has a coherent meaning.
[0038] Furthermore, since it is unlikely that the category (e.g., address, price, title, etc.) would change midway through a coherent series of unit strings, it is believed that the unit string images extracted using this configuration display strings consisting of a single category. In this way, because the unit string image consists of a single string category, this configuration of using multiple AI engines to determine the category makes sense and allows the category determination accuracy to be maintained at a high level. Furthermore, because of the above-mentioned highly accurate category determination, dictionary data to be used in correction after conversion to text data can be appropriately selected, making it possible to significantly improve the accuracy of character recognition.
[0039] <Other embodiments> The present invention is not limited to the above-described embodiment.
[0040] In the above embodiment, all character string images in a document are extracted and divided into unit character string images, but it is also possible to extract, for example, only character string images within a bounding box, only character string images in underlined parts, only character string images other than those, or two or more of these, or to make it possible to select these.
[0041] It is also possible to set the area, length, height, position, etc. as information (shape) about the ruled line. For example, it is possible to extract a character string image within a frame whose area is greater than or less than a predetermined value as a unit character string image, or to extract an underlined character string image whose length is greater than or less than a predetermined value as a unit character string image, or to extract a unit character string image based on the position of a ruled line (for example, a ruled line in the upper half of a document).
[0042] The AI engine and dictionary data are not limited to those exemplified above. For example, an AI engine that is good at handling vertical writing or multi-column writing may be used.
[0043] The evaluation value is not limited to the likelihood. For example, position information of the character position detection part may be used instead of the likelihood of the character recognition part. The vertical fluctuation of the baseline of the detection position of each character in the read character string and the fluctuation of the character pitch may be quantified, and the evaluation value may be calculated based on the quantification. If a misreading occurs, the numerical value will fluctuate unnaturally. It is also possible to calculate an evaluation value based on the variation in the likelihood of each character constituting a character string within the character string, rather than using the likelihood as is.
[0044] Furthermore, instead of using simple likelihood, it is also possible to run the data through a general-purpose OCR once first, use that data to calculate appropriate parameters, and then use those to weight the likelihood of each AI engine. Specifically, for example, the results of running the data through a general-purpose OCR can be used to evaluate the characters contained in the read string, and for addresses that contain many characteristic postal codes, prefecture names, block numbers, item numbers, and numbers, the weighted value of the likelihood of the address AI engine can be used as the evaluation value. It is also possible to use dictionaries for addresses, names, company names, product names, etc., and use the initial general-purpose OCR to evaluate whether a string is likely to be an address, or whether it is likely to be a name, company name, or product name (classification problem), and then use the evaluation value to weight the likelihood of each AI engine.
[0045] Furthermore, the shape of the frame, the arrangement of the characters, etc. may be used as indicators for calculating the evaluation value. For example, if the frame is vertically long, it is likely to be vertical writing, so the evaluation value of an AI engine for vertical writing may be increased, and if the character string is written in two lines, the evaluation value of an AI engine for multi-line writing may be increased. In the above embodiment, the correction results based on the dictionary are used for re-learning of each AI engine, but they do not have to be reused.
[0046] Furthermore, the present invention is not limited to the above-described embodiment, and it goes without saying that various modifications are possible without departing from the spirit and scope of the present invention. [Explanation of symbols]
[0047] 100...String Recognition System 1. Ruled line detection section 2. Unit string image extraction part 3. Character recognition processing section 4. Category determination section 5. Unit string correction section
Claims
1. A character string recognition system for recognizing a character string from a ruled document image, comprising: a ruled line detection unit for detecting ruled lines from the ruled document image; a unit string image extraction unit that extracts a plurality of unit string images recognized as a series of character strings from the ruled document image based on the ruled lines detected by the ruled line detection unit; a character recognition processing unit that performs character recognition processing using a plurality of types of AI engines on each of the unit character string images and obtains an evaluation value of each character recognition processing; a category determination unit that determines a category of the unit string displayed in each unit string image based on the type of the AI engine that has a high evaluation value for each unit string image; A character string recognition system comprising:
2. 2. The character string recognition system according to claim 1, further comprising a unit string correction unit that corrects a character recognition result by an AI engine specified by the category determination unit for each of the unit string images, using dictionary data corresponding to the category determined by the category determination unit.
3. 2. The character string recognition system according to claim 1, wherein the unit string image extraction section extracts a document in an area surrounded by a closed ruled line as the unit string image.
4. 3. The character string recognition system according to claim 1, wherein the evaluation value includes a likelihood.
5. 3. The string recognition system according to claim 1, wherein a corresponding AI engine re-learns based on the correction result by the unit string correction unit.
6. A character string recognition program for recognizing a character string from a lined document image, a ruled line detection unit for detecting ruled lines from the ruled document image; a unit string image extraction unit that extracts a plurality of unit string images recognized as a series of character strings from the ruled document image based on the ruled lines detected by the ruled line detection unit; a character recognition processing unit that performs character recognition processing using a plurality of types of AI engines on each of the unit character string images and obtains an evaluation value of each character recognition processing; a category determination unit that determines a category of the unit string displayed in each unit string image based on the type of the AI engine that has a high evaluation value for each unit string image; A character string recognition program characterized by causing a computer to function as a character string recognition program.
7. A character recognition method for recognizing characters from a lined document image, comprising: Detecting ruled lines from the lined document image; extracting a plurality of unit character string images recognized as a series of character strings from the ruled document image based on the ruled lines detected by the ruled line detection unit; A character recognition process is performed on each of the unit character string images using a plurality of types of AI engines, and an evaluation value of each character recognition process is obtained; Identifying an AI engine having a high evaluation value for each of the unit string images, and determining a category of the unit string displayed in each of the unit string images based on the type of the AI engine identified; A character string recognition method comprising:
Citation Information
Patent Citations
Method and device for recognizing character
JP2001147988A
Document OCR execution apparatus
JP2007065994A
Character recognition device, character recognition method, and character recognition program
JP2020027598A
Information processing apparatus, learning device, and learned model
JP2020144636A
Form recognition program and form recognition device
JP2022122643A
Cited By
Information processing systems, information processing methods, programs
JP2026112510A
Text data conversion system
JP7932204B1
