Character recognition device, and character recognition method

Abstract

To provide a character recognition device capable of accurately digitizing documents containing handwritten characters that vary by author.SOLUTION: A document reader 2 analyzes document image data and determines the layout of the document, and based on that layout, identifies the author who wrote the characters in the image document data as the target author. An OCR device 3 generates character information by recognizing characters in image document data using the author-specific character recognition model of the target author out of the multiple author-specific character recognition models that are used for the characters written by each author as training data.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present disclosure relates to a character recognition device and a character recognition method. [Background technology]

[0002] In recent years, the digitalization of data has progressed throughout society, but some business processes (for example, questionnaires at exhibitions, various applications at government offices, and daily reports at work sites) still require handwritten documents. Furthermore, documents created in the past by hand are often stored on paper or microfilm. Because these handwritten documents have not been digitized, there is a problem in that they are difficult to use effectively.

[0003] In response to this, Patent Document 1 discloses a technique for digitizing handwritten characters using a character recognition model constructed using machine learning. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Application Laid-Open No. 2015-069256 Summary of the Invention [Problem to be solved by the invention]

[0005] However, the technology described in Patent Document 1 uses a general-purpose character recognition model, which makes it difficult to accurately digitize handwritten characters, which vary in style from writer to writer. Digitization is particularly difficult for characters with strong handwriting habits and uncommon abbreviated characters.

[0006] An object of the present disclosure is to provide a character recognition device and a character recognition method that are capable of accurately digitizing documents that include handwritten characters with different handwriting styles depending on the writer. [Means for solving the problem]

[0007] A character recognition device according to one aspect of the present disclosure is a character recognition device that recognizes characters in image document data obtained by imaging a document, and includes: an identification unit that analyzes the image document data to determine the layout of the document and, based on the layout, identifies the writer who wrote the characters in the image document data as a target writer; and a character recognition unit that generates character information that recognizes the characters in the image document data using one of a plurality of author-specific character recognition models for the target writer, which are machine learning models for recognizing characters, each of which uses characters written by that writer as learning data. [Effects of the Invention]

[0008] According to the present invention, it is possible to accurately digitize documents containing handwritten characters whose writing style varies from one writer to another. [Brief explanation of the drawings]

[0009] [Figure 1] FIG. 1 is a diagram illustrating an example of a character recognition system according to an embodiment of the present disclosure. [Figure 2] FIG. 10 is a diagram illustrating an example of a layout of a document. [Figure 3] 10 is a flowchart illustrating the operation of the character recognition system. [Figure 4] FIG. 10 is a diagram for explaining switching of a text correction model. DETAILED DESCRIPTION OF THE INVENTION

[0010] Hereinafter, embodiments of the present disclosure will be described with reference to the drawings.

[0011] 1 is a diagram illustrating an example of a character recognition system (character recognition device) according to an embodiment of the present disclosure. As shown in FIG. 1, the character recognition system includes a reception device 1, a document reading device 2, an OCR (Optical Character Recognition / Reader) device 3, a generation AI (Artificial Intelligence) device 4, and a document management device 5.

[0012] The reception device 1 is a reception unit that receives document image data obtained by imaging a document. The document image data is obtained by reading a document, such as a paper document, as image data using, for example, a scanner, a copier, a camera, or a smartphone. In this embodiment, the document image data contains characters handwritten by the author who created the document. Furthermore, there are multiple types of documents with different layouts.

[0013] FIG. 2 is a diagram illustrating an example of a document layout. The document data shown in FIG. 2 is an example of a business journal and includes layout blocks 11 to 15. Layout blocks 11 to 15 are components that make up the document data, each with a specific function (application). In FIG. 2, layout block 11 functions as a staff member stamp section where the staff member who created the business journal stamps their seal; layout block 12 functions as a supervisor stamp section where the staff member's superior stamps their seal; layout blocks 13 and 14 function as staff member entry sections where the staff member writes documents by hand, etc.; and layout block 15 functions as a supervisor entry section where the superior writes documents by hand, etc. In addition, each layout block 11 to 15 may be provided with a label L that serves as the heading for that layout block.

[0014] Returning to the explanation of Figure 1, the document reading device 2 is an identification unit that analyzes the imaged document data received by the reception device 1, determines the layout of the document, and identifies the writer who wrote the characters in the imaged document data as the target writer based on the layout. Depending on the type of document data, it may contain sentences written by multiple authors (for example, a person in charge and a superior). In this case, the document reading device 2 identifies the writer for each sentence.

[0015] The OCR device 3 is a character recognition unit that uses a character recognition model for recognizing characters in image data to perform character recognition on the characters in the image document data accepted by the acceptance device 1, and generates digitized character information.

[0016] In this embodiment, the character recognition model is a machine learning model, and includes multiple writer-specific character recognition models generated for each writer using characters written by that writer as training data (tuning data). The character recognition model may also include a general-purpose standard character recognition model that does not specify a writer.

[0017] The generation AI device 4 is a correction unit that uses a sentence correction model to correct sentences and generates corrected character information by correcting the character information generated by the OCR device 3. Correcting sentences includes predicting characters that could not be recognized by the OCR device 3 and correcting characters that were incorrectly recognized by the OCR device 3.

[0018] In this embodiment, the writing correction model is a machine learning model, more specifically, a generative AI model. The writing correction model includes multiple author-specific writing correction models generated for each author using the writing of that author as training data, and multiple organization-specific writing correction models generated for each organization using the writing of authors belonging to that organization as training data. An organization may have a hierarchical structure with multiple levels. In this embodiment, the organization has two levels: a higher level, "company," and a lower level, "department (within the company)." The organization-specific writing correction model includes a company-specific writing correction model corresponding to the company and a department-specific writing correction model corresponding to the department. The writing correction model may also include a general-purpose target writing correction model that does not specify a writer or organization.

[0019] The document management device 5 is a storage unit that stores data in which characters in the image document data accepted by the reception device 1 have been replaced with corrected character information generated by the generation AI device 4 as digital document data obtained by digitizing the document.

[0020] FIG. 3 is a flowchart illustrating the operation of the character recognition system.

[0021] In the character recognition system, first, the reception device 1 receives imaged document data and outputs the imaged document data to the document reading device 2 (step S101).

[0022] When the document reading device 2 receives the imaged document data, it executes a distortion correction process to correct distortion in the imaged document data (step S102). The specific method of the distortion correction process is not particularly limited, and may be, for example, a combination of line segment detection and perspective transformation.

[0023] The document reading device 2 analyzes the imaged document data after the distortion correction and detects a layout structure, which is a structure of the layout of the imaged document data (step S103). In this embodiment, the document reading device 2 detects the positions and sizes of layout blocks, which are components that make up the imaged document data, and labels, which are headings of each layout block, as the layout structure.

[0024] The document reading device 2 determines the layout of the imaged document data based on a layout master that indicates the layout of each document type and the layout structure of the detected imaged document data (step S104). The layout indicated by the layout master indicates not only the layout structure but also the definition (function, etc.) of each layout block and the correspondence between the layout blocks. For example, in the example of FIG. 2, the layout master associates layout block 11, which functions as a responsible person's seal section, with layout blocks 13 and 14, which function as responsible person's description section, and also associates layout block 12, which functions as a superior's seal section, with layout block 15, which functions as a superior's description section. Furthermore, the document reading device 2, for example, determines, from the layout master, a layout that has the same structure (or the highest similarity) as the layout structure of the detected imaged document data as the layout of the imaged document data. Note that the layout determination method is not limited to this method, and a machine learning model for determining a layout from a detected layout structure may also be used.

[0025] Then, the document reading device 2 identifies the target writer who wrote the characters written in the imaged document data based on the layout of the imaged document data (step S105). Specifically, the document reading device 2 identifies a name entry field, which is a layout block for writing a name in the imaged document data, based on the layout of the imaged document data, and identifies the target writer based on the name written (written) in the name entry field. In the example of FIG. 2, the name entry field is layout block 11 that functions as a responsible person's seal section and layout block 12 that functions as a senior seal section. However, the name entry field is not limited to a seal section, and may be a layout block for writing a name by hand. Furthermore, when there are multiple name entry fields as in the example of FIG. 2, the document reading device 2 identifies, for each name entry field, the target writer who wrote the characters in the entry field (layout block) corresponding to the name entry field based on the name in the name entry field.

[0026] The OCR device 3 selects a writer-specific character recognition model corresponding to the target writer from among the character recognition models, and recognizes characters in the imaged document data using the selected writer-specific character recognition model to generate character information by digitizing the characters (step S106). Note that if the target writer is different for each entry field, the OCR device 3 selects a writer-specific character recognition model for that target writer for each entry field. Also, if the writer cannot be identified by the document reading device 2 (for example, if there is no seal or if there is no signature field), the OCR device 3 uses a general-purpose standard character recognition model.

[0027] The generation AI device 4 uses the sentence correction model to perform a sentence correction process for correcting the character information generated by the OCR device 3 (step S107). The sentence correction process will be described in detail later.

[0028] The document reading device 2 performs a formatting process to embed character information corrected by the text correction process into each layout block of the imaged document data, thereby generating digital document data by digitizing the imaged document data (step S108).The document reading device 2 then stores the digital document data in the document management device 5 (step S109).

[0029] In addition, the document reading device 2 uses the character information corrected in the text correction process as new learning data (tuning data) to re-learn the character recognition model and text correction model used to generate the digital document data (step S110), and then terminates the processing.

[0030] Next, the text correction process in step S107 will be described.

[0031] The text correction process is a process of correcting character information using the text correction model as described above. Text correction includes predicting characters that could not be recognized by the OCR device 3 and correcting characters that were incorrectly recognized by the OCR device 3.

[0032] For example, assume that the text information generated by the OCR device 3 contains a sentence "Redundant configuration ●●●●●●●●". Here, "●" represents a part that could not be read by the OCR device 3. In this case, multiple answers are predicted for the unreadable part. For example, if the author is a supervisor (reviewer), that part is expected to be "Is a redundant configuration adopted?", and if the author is a person in charge, that part is expected to be "A redundant configuration is adopted." Therefore, it is difficult for a general-purpose text correction model to predict an answer. In contrast, in this embodiment, the generation AI device 4 can use an author-specific text correction model as the text correction model to predict an appropriate answer for each author and their role.

[0033] Also, in a specific company, terms that deviate from normal Japanese may be used. For example, there are companies that have the practice of writing "retrospection" as "sa-ki". "Sa-ki" is a misreading of "retrospection" and is not a notation in normal Japanese. Therefore, if "sa-ki" is misrecognized by the OCR device 3 during character recognition, there is a possibility that a general-purpose standard text correction model cannot correct this part appropriately. For example, if the OCR device 3 misrecognizes "sa-ki" as "2o and", a general-purpose text correction model may not be able to correct it appropriately because it does not have "sa-ki" as learning data (teacher data). Since "retrospection" is one character and "2o" is two characters, usually, "retrospection" is not the target as a correction candidate for "sa-ki". In the embodiment, the generation AI device 4 can correct "2o and" to "sa-ki" by using an author-specific text correction model, a department-specific text correction model, or a company-specific text correction model as the text correction model.

[0034] Furthermore, in this embodiment, the sentence correction model is realized by a generation AI model using a generation AI. Therefore, in the sentence correction process, for example, the generation AI device 4 may display character information generated by the OCR device 3 and correct the character information through interaction with the user. For example, if there is a sentence recognized by the OCR device 3 as "Overview2"To," the generation AI device 4 may determine that "Overview2"To" is an incorrect recognition based on a user's indication or on its own initiative, and propose "Overview is the correct answer." If approval is obtained from the user, the generation AI device 4 corrects "Overview2"To" to "Overview is the correct answer."

[0035] Furthermore, if the document reading device 2 does not identify the writer, the generation AI device 4 switches the text correction model used in the text correction process and executes the text correction process.

[0036] FIG. 4 is a diagram for explaining switching of the text correction model.

[0037] If the document reading device 2 fails to identify the author, the generation AI device 4 performs processing to identify the target organization, which is the organization that created the document, starting from the lowest level. Once the target organization is identified, the generation AI device 4 corrects the text information using the organization-specific text correction model for that target organization. In this embodiment, as shown in Figure 4, the organization has two levels: company and department. Therefore, if the author is not identified, the generation AI device 4 performs processing to identify the department that created the document. If the department is not identified, the generation AI device 4 performs processing to identify the company that created the document. Furthermore, if the target organization cannot be identified at any level, the generation AI device 4 corrects the text information using a general-purpose standard text correction model.

[0038] The specific method of identifying the organization is not particularly limited. For example, in the layout master, each layout may be associated with the organization that creates the document of that layout, or the reception device 1 may receive information identifying the organization from the user along with the document image data. Furthermore, the text correction model may be switched in response to an instruction from the user.

[0039] Each of the above-mentioned devices (reception device 1, document reading device 2, OCR device 3, and generation AI device 4) is configured, for example, by a computer system equipped with a processor (computer) and memory (not shown). In this case, the functions of each of the above-mentioned devices are realized, for example, by the processor reading a computer program and executing the read computer program. The computer program can be recorded on a computer-readable recording medium. Examples of the recording medium include semiconductor memory, magnetic disks, optical disks, magnetic tapes, and magneto-optical disks. Each device may have an input device, such as a keyboard, touch panel, or pointing device, that receives various information from a user, and an output device, such as a display device or printer, that outputs various information to a user. Each device may also have a network interface device that transmits and receives various information via a communication network such as the Internet, and a storage device, such as a large-capacity storage device, that stores various information.

[0040] As described above, according to this embodiment, the document reading device 2 analyzes document image data, which is an image of a document, to determine the document layout, and then identifies the writer who wrote the characters in the document image data as the target writer based on the layout. The OCR device 3 generates character information by recognizing the characters in the document image data using the writer-specific character recognition model for the target writer from among multiple writer-specific character recognition models that use characters written by each writer as training data. This makes it possible to accurately digitize documents that contain handwritten characters that vary in style from writer to writer.

[0041] In this embodiment, the document reading device 2 identifies the name field containing the name of the target author included in the document image data based on the layout of the document, and identifies the target author based on the name written in the name field. This makes it possible to easily identify the target author.

[0042] In this embodiment, the document reading device 2 identifies the target writer for each of a plurality of writing fields included in the document image data based on the layout of the document, based on the name written in the name field corresponding to the writing field. Therefore, even if a single document has multiple writers, the document can be digitized with high accuracy.

[0043] In this embodiment, the AI ​​generation device 4 generates corrected character information by correcting character information using the target writer's individual text correction model from among multiple individual text correction models that use the text written by each writer as training data. This makes it possible to appropriately correct erroneously recognized character information according to the writer, thereby enabling documents to be digitized with high accuracy.

[0044] Furthermore, in this embodiment, if the author is not identified, the generation AI device 4 generates corrected character information using an organization-specific text correction model that uses text written by authors belonging to the organization that created the document as training data. Therefore, even if the author is not identified, it is possible to accurately correct expressions unique to an organization such as a company, making it possible to accurately digitize documents.

[0045] In this embodiment, the generation AI device 4 performs processing to identify the target organization from the lowest level up, and once the target organization is identified, corrects the text information using the organization-specific text correction model of that target organization. This makes it possible to correct the text information using an appropriate organization-specific text correction model.

[0046] Furthermore, in this embodiment, the corrected character information is used to retrain the writer-specific character recognition model and the writer-specific text correction model, which makes it possible to improve the accuracy of digitization.

[0047] In this embodiment, the author-specific text correction model is a generative AI model, which makes it possible to easily correct character information in a question-and-answer format.

[0048] The above-described embodiments of the present disclosure are merely illustrative examples of the present disclosure, and are not intended to limit the scope of the present disclosure to these embodiments alone. Those skilled in the art may implement the present disclosure in various other forms without departing from the scope of the present disclosure. [Explanation of symbols]

[0049] 1: Reception device 2: Document reader 3: OCR device 4: Generative AI device 5: Document management device

Claims

1. A character recognition device that recognizes characters in image document data obtained by imaging a document, an identification unit that analyzes the document image data to determine the layout of the document, and identifies the writer who wrote the characters in the document image data as a target writer based on the layout; A character recognition device having a character recognition unit that generates character information by recognizing characters in the image document data using the author-specific character recognition model of the target author from among a plurality of author-specific character recognition models, which are machine learning models for recognizing characters, each of which uses characters written by that author as learning data.

2. 2. The character recognition device according to claim 1, wherein the identification unit identifies a name field containing the name of the target writer included in the image document data based on the layout, and identifies the target writer based on the name written in the name field.

3. the identifying unit identifies the target writer for each of a plurality of description fields included in the image document data based on the layout, based on a name written in the name field corresponding to the description field; The character recognition device according to claim 2 , wherein the character recognition unit generates the character information for each entry using the writer-specific character recognition model of the target writer corresponding to the entry.

4. The character recognition device according to claim 1, further comprising a correction unit that generates corrected character information by correcting the character information using the author-specific sentence correction model of the target writer from among a plurality of author-specific sentence correction models, which are machine learning models for correcting sentences, each of which uses sentences written by that writer as learning data for each writer.

5. 5. The character recognition device according to claim 4, wherein, when the identification unit does not identify the writer, the correction unit generates the corrected character information using an organization-specific text correction model for the target organization, among a plurality of organization-specific text correction models that are machine learning models using, as learning data, text written by a writer belonging to the target organization that created the document.

6. the organization has a hierarchical structure having multiple levels, 6. The character recognition device according to claim 5, wherein the correction unit performs processing to identify the target organization in order from the lower layers, and when the target organization is identified, generates the corrected character information using the organization-specific text correction model of the target organization.

7. 5. The character recognition device according to claim 4, wherein the character recognition unit re-trains the writer-specific character recognition model of the target writer using the corrected character information as training data.

8. 5. The character recognition device according to claim 4, wherein the correction unit re-trains the writer-specific text correction model of the target writer using the corrected character information as training data.

9. The character recognition device according to claim 4 , wherein the author-specific text correction model is a generative AI model.

10. A character recognition method using a character recognition device that recognizes characters in image document data obtained by imaging a document, comprising: analyzing the document image data to determine the layout of the document, and identifying the writer who wrote the characters in the document image data as a target writer based on the layout; A character recognition method that generates character information by recognizing characters in the image document data using the author-specific character recognition model for the target author from among multiple author-specific character recognition models, which are machine learning models for recognizing characters and which use characters written by each author as learning data for each author.

Images