System for recognizing cursive characters, method for recognizing cursive characters, and program

Abstract

To provide a running-style character recognition system capable of recognizing running-style characters without increasing a time required for character recognition as compared with a method using only a database registering images of the running-style characters.SOLUTION: A running-style character recognition system comprises: a character image acquisition unit which acquires a character image of running-style characters; a character-style database which stores a character-style data table associating the image of the running-style characters with modern characters of the running-style characters indicated in the image; a machine learning model storage unit which stores a machine learning model learned so as to output the modern characters of the running-style characters indicated in the image from an image of the input running-style characters; a running-style character recognition unit which performs character recognition for associating the running-style characters indicated in the character image with the modern characters using one of the character style data table and the machine learning model and the character image; and a machine learning model generation unit which when the number of images stored in the character-style database is greater than or equal to a threshold value, causes the machine learning model to be re-leaned.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to a cursive character recognition system, a cursive character recognition method, and a program. [Background technology]

[0002] When researching Japanese culture of the past, it is sometimes necessary to be able to read the characters written in Japanese documents of the past. Many Japanese documents from before the Edo period are written in kuzushiji, a shortened form of kanji and katakana. Kuzushiji is written using old characters or in cursive script, so the characters used and the shapes of the characters are very different from those used today, making it difficult for anyone other than experts to read. In addition, kuzushiji cannot be recognized using general OCR (Optical Character Recognition) engines that do not support kuzushiji.

[0003] If an OCR engine could recognize cursive characters and output the recognition results as data in regular script (modern characters), even non-experts would be able to widely utilize historically valuable content written in cursive characters. For this reason, the development of OCR engines compatible with cursive characters is underway (see, for example, Non-Patent Document 1). In such OCR engines compatible with cursive characters, a method using a cursive character database is used as a method for recognizing cursive characters. The cursive character database is a database in which images of cursive characters are associated with character recognition results for the cursive characters. The cursive character database is searched for images that match or are similar to the image of the cursive character to be recognized, and the characters associated with the searched image are regarded as the result of character recognition. [Prior art documents] [Non-patent literature]

[0004] [Non-Patent Document 1] Junko Yamamoto, Tomijiro Osawa: “Abbreviating the transcription of classical texts: Development of OCR technology including cursive script”, Information Management, Vol. 58 (2015) No. 11, p819-827, 2016 Summary of the Invention [Problem to be solved by the invention]

[0005] However, when trying to improve the accuracy of character recognition using a method that uses a cursive character database, it is necessary to register a large number of images of various cursive characters and then search for the corresponding image from among the large number of registered images, which creates the problem of taking a long time for character recognition.

[0006] The present invention has been made in consideration of the above-mentioned circumstances, and aims to provide a cursive character recognition system, a cursive character recognition method, and a program that can recognize cursive character without increasing the time required for character recognition, compared to a method that uses only a database in which images of cursive character are registered. [Means for solving the problem]

[0007] In order to achieve the above object, the cursive character recognition system of the present invention includes a character image acquisition unit that acquires a cursive character image; a cursive character recognition unit that performs character recognition to correspond the cursive character shown in the character image to a modern character using a machine learning model and the character image; a registration unit that stores information in a character shape data table that corresponds the character image in which the character recognition was erroneous to the modern cursive character shown in the character image if the character recognition of the character image using the machine learning model is incorrect; and a machine learning model generation unit that retrains the machine learning model if the number of images stored in the character shape database is equal to or greater than a threshold value. The machine learning model is a model that has been trained to output the modern cursive character shown in an input cursive character image from the image using a learning dataset that corresponds a learning image to the modern cursive character shown in the learning image, and the character shape data table is information that corresponds the image of a cursive character to the modern cursive character shown in the image.

[0008] In order to achieve the above object, the cursive character recognition system of the present invention includes a character image acquisition unit that acquires a cursive character image, , character a cursive character recognition unit that performs character recognition to correspond cursive characters shown in the character image to modern characters using a shape data table, a machine learning model, and the character image; a registration unit that, when an error occurs in the character recognition using the machine learning model, stores information in the character shape data table that corresponds the character image in which the character recognition was erroneous to a modern character of cursive writing shown in the character image; a machine learning model generation unit that retrains the machine learning model when the number of images stored in the character database is equal to or greater than a threshold value; when performing the character recognition using the character shape data table, the cursive character recognition unit performs the character recognition using an image stored in the character shape data table in response to an error in the character recognition using the machine learning model; The machine learning model is a model trained to output, from an input image of cursive writing, the modern cursive writing character shown in the image, using a learning dataset that matches training images with the modern cursive writing characters shown in the training images, and the character shape data table is information that matches images of cursive writing with the modern cursive writing characters shown in the image.

[0009] The cursive character recognition system of the present invention comprises: a character image acquisition unit that acquires a character image of cursive characters; and a cursive character recognition unit that performs character recognition to correspond the cursive characters shown in the character image to modern characters using either a character shape data table or a machine learning model and the character image;a registration unit that stores information in the character data table that corresponds the character image with a modern character of cursive writing shown in the character image when the result of the character recognition performed on the character image using the machine learning model is incorrect; and, retraining the machine learning model when the number of images stored in the character data table is equal to or greater than a predetermined threshold in response to an error in estimating a modern character using the machine learning model; a machine learning model generation unit, the machine learning model being a model trained to output, from an input image of cursive writing, a modern character of cursive writing shown in the image, using a training data set in which training images are associated with the modern character of cursive writing shown in the training image; The character shape data table is information that associates images of cursive characters with the modern characters of the cursive characters shown in the images. It is characterized by:

[0010] The cursive script recognition system of the present invention further includes a registration unit that, if the result of character recognition on the character image using the machine learning model is incorrect, stores information in the character shape data table that corresponds the character image with the modern cursive script character shown in the character image, and the machine learning model generation unit re-trains the machine learning model using the learning dataset including the information registered in the character shape data table by the registration unit.

[0011] The method for recognizing cursive characters of the present invention is a method for recognizing cursive characters by a computer, in which a character image acquisition unit acquires a character image of a cursive character, a cursive character recognition unit uses a machine learning model and the character image to perform character recognition to correspond the cursive character shown in the character image to a modern character, a registration unit, if the character recognition of the character image using the machine learning model is incorrect, stores information that corresponds the character image in which the character recognition was incorrect to the modern cursive character shown in the character image in a character shape data table, and a machine learning model generation unit, if the number of images stored in the character shape database is equal to or greater than a threshold, re-learns the machine learning model, and the machine learning model is a model trained to output the modern cursive character shown in an input image of cursive character from the image of the cursive character using a learning dataset that corresponds a learning image to the modern cursive character shown in the learning image, and the character shape data table is information that corresponds the image of cursive character to the modern cursive character shown in the image.

[0012] The program of the present invention is a program for causing a computer to function as: a character image acquisition means for acquiring a character image of cursive writing; a cursive writing recognition means for performing character recognition to correspond the cursive writing shown in the character image to a modern character using a machine learning model and the character image; a registration means for storing information in a character shape data table that corresponds the character image in which the character recognition was erroneous with the modern cursive writing character shown in the character image if the character recognition of the character image using the machine learning model is incorrect; and a machine learning model generation means for re-training the machine learning model if the number of images stored in the character shape data table is equal to or greater than a threshold value, wherein the machine learning model is a model trained to output the modern cursive writing character shown in an input image of cursive writing from the image using a learning dataset that corresponds a learning image to the modern cursive writing character shown in the learning image, and the character shape data table is information that corresponds the image of cursive writing to the modern cursive writing character shown in the image.

[0013] The cursive character recognition method of the present invention is a cursive character recognition method by a computer, in which a character image acquisition unit acquires a character image of a cursive character, and a cursive character recognition unit performs character recognition using either a character shape data table or a machine learning model and the character image to correspond the cursive character shown in the character image to a modern character, When an error occurs in the character recognition using the machine learning model, the registration unit stores information in the character shape data table that corresponds the character image in which the character recognition was erroneous with the modern character of cursive script shown in the character image, and when the cursive script recognition unit performs the character recognition using the character shape data table, the cursive script recognition unit performs the character recognition using the image stored in the character shape data table in response to the error in the character recognition using the machine learning model; The machine learning model is a model trained to output, from an input image of cursive writing, the modern cursive writing character shown in the image, using a learning dataset that matches training images with the modern cursive writing characters shown in the training images, and the character shape data table is information that matches images of cursive writing with the modern cursive writing characters shown in the image. The cursive character recognition method of the present invention is a cursive character recognition method by a computer, in which a character image acquisition unit acquires a character image of a cursive character, a cursive character recognition unit performs character recognition to correspond the cursive character shown in the character image to a modern character using either a character shape data table or a machine learning model and the character image, and a registration unit, if there is an error in the character recognition using the machine learning model, stores information in the character shape data table that corresponds the character image in which the character recognition was erroneous to the modern character of the cursive character shown in the character image, and a machine learning model generation unit: If the number of images stored in the character shape data table is equal to or greater than a threshold value due to an error in the character recognition using the machine learning model, the machine learning model is retrained, and the machine learning model is a model trained to output the modern cursive character shown in an input image of cursive script from the image using a training dataset that matches the training image with the modern cursive character shown in the training image, and the character shape data table is information that matches the image of cursive script with the modern cursive character shown in the image.

[0014] The program of the present invention includes a character image acquisition means for acquiring a character image of cursive script by a computer, a cursive script recognition means for performing character recognition to correspond the cursive script shown in the character image to a modern character by using either a character shape data table or a machine learning model and the character image, and a registration means for storing, in the character shape data table, information that corresponds the character image in which the character recognition using the machine learning model is erroneous with a modern character of cursive writing shown in the character image, when the character recognition using the machine learning model is erroneous; A program for causing the device to function as a When the cursive character recognition means performs the character recognition using the character shape data table, the cursive character recognition means performs the character recognition using an image stored in the character shape data table in response to an error in the character recognition using the machine learning model; The machine learning model is a model trained to output, from an input image of cursive writing, the modern cursive writing character shown in the image, using a learning dataset that matches training images with the modern cursive writing characters shown in the training images, and the character shape data table is information that matches images of cursive writing with the modern cursive writing characters shown in the image. The program of the present invention is a program for causing a computer to function as: a character image acquisition means for acquiring a character image of cursive writing; a cursive writing recognition means for performing character recognition to correspond the cursive writing shown in the character image to a modern character using either a character shape data table or a machine learning model and the character image; a registration means for storing, in the character shape data table, information that corresponds the character image in which the character recognition was erroneous with the modern cursive writing character shown in the character image if there is an error in the character recognition using the machine learning model; and a machine learning model generation means for re-learning the machine learning model if the number of images stored in the character shape data table in response to an error in the character recognition using the machine learning model is equal to or greater than a threshold value, wherein the machine learning model is a model that has been trained to output, from an input image of cursive writing, a modern character of cursive writing shown in the image, using a learning dataset that corresponds a learning image with a modern cursive writing character shown in the learning image, and the character shape data table is information that corresponds an image of cursive writing with a modern cursive writing character shown in the image. Effect of the Invention

[0015] According to the present invention, it is possible to perform character recognition of cursive characters without increasing the time required for character recognition, compared to a method that uses only a database in which images of cursive characters are registered. [Brief description of the drawings]

[0016] [Figure 1] 1 is a block diagram showing an example of the configuration of a cursive script recognition system 1 according to an embodiment. [Diagram 2] FIG. 13 is a diagram showing an example of the configuration of a glyph data table 160 according to an embodiment. [Diagram 3] FIG. 1 is a diagram illustrating a machine learning model 170 according to an embodiment. [Figure 4] FIG. 2 is a diagram illustrating character recognition according to an embodiment. [Diagram 5] 10 is a diagram for explaining a process performed by a registration unit 13 according to an embodiment. FIG. [Figure 6] 1 is a flowchart showing a flow of operations performed by the cursive script recognition system 1 according to the embodiment. [Figure 7] 1 is a block diagram showing an example of the configuration of a cursive script recognition system 1A according to a first modified example of an embodiment. [Figure 8] 10 is a flowchart showing a flow of operations performed by a cursive character recognition system 1A according to a first modified example of the embodiment. [Figure 9] FIG. 11 is a block diagram showing an example of the configuration of a cursive script recognition system 1B according to a second modified example of the embodiment. [Figure 10] 13 is a flowchart showing the flow of operations performed by a cursive script recognition system 1B according to a second modified example of the embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0017] Hereinafter, an embodiment of the present invention will be described in detail with reference to the drawings.

[0018] 1 is a block diagram showing an example of the configuration of a cursive character recognition system 1 according to an embodiment. The cursive character recognition system 1 includes, for example, a data input / output unit 10, a cursive character recognition unit 11, a determination unit 12, a registration unit 13, a machine learning model generation unit 14, a display unit 15, a character shape database 16, and a machine learning model storage unit 17. Here, the data input / output unit 10 is an example of a "character image acquisition unit."

[0019] The data input / output unit 10 acquires image information of a character image M of cursive characters (see FIG. 3). The character image M is an image in which an area including the target cursive characters to be recognized (transcribed) is extracted from a captured image of an arbitrary page of a document for which character recognition of the cursive characters is to be performed. The number of cursive characters included in the character image M may be any number. In the following, an example will be described in which the number of cursive characters included in the character image M is one, but a similar method can be applied not only to cases in which the number of cursive characters included in the character image M is two or more, but also to cases in which the character image M includes cursive characters on a word-by-word, document-by-document, or line-by-line basis.

[0020] For example, the data input / output unit 10 reads in a captured image of a document from an external device for which cursive characters are to be recognized, thereby acquiring character images M. The data input / output unit 10 acquires character images M from an external device by performing operations via a data input means (keyboard, mouse, etc.) not shown or a touch panel of the display unit 15 described later, and outputs data indicating the results of character recognition of modern characters transcribed from cursive characters to an external device.

[0021] The cursive character recognition unit 11 performs character recognition to correspond the cursive character shown in the character image M to a modern character by using either the character shape data table 160 (see FIG. 2) or the machine learning model 170 (see FIG. 3) and the character image M. The character shape data table 160 is a table that corresponds an image G of a cursive character (see FIG. 2) to the modern character of the cursive character shown in the image G. The machine learning model 170 is a model that is trained to output the modern character of the cursive character shown in an input image of the cursive character from the image by using a learning data set that corresponds the learning image to the modern character of the cursive character shown in the learning image.

[0022] When character recognition is performed using the character shape data table 160, the cursive character recognition unit 11 refers to the character shape data table 160 based on the character image M acquired by the data input / output unit 10, and searches for an image showing a shape that matches or is similar to the shape of the character image M. For example, the cursive character recognition unit 11 divides the character image M into a plurality of element regions and extracts feature amounts for each element region. The feature amount is the degree of feature in the shape of the character portion shown in the element region, and is, for example, a gradient vector of pixel values ​​of pixels included in the element region. The cursive character recognition unit 11 aggregates the feature amounts for each element region of the character image M, and extracts images that have the same or similar feature amount trends from among the images stored in the character shape data table 160 as images that match or are similar to the character image M. For example, the cursive character recognition unit 11 determines that the two gradient vectors match or are similar when the distance between the two gradient vectors in the gradient vector space is less than a predetermined threshold value. The cursive character recognition unit 11 determines the modern character associated with the image extracted from the character shape data table 160 as the result of character recognition in the character image M.

[0023] When performing character recognition using the machine learning model 170, the cursive character recognition unit 11 inputs the character image M acquired by the data input / output unit 10 to the machine learning model 170, and the modern character output from the machine learning model 170 is the result of character recognition in the character image M.

[0024] The cursive character recognition unit 11 determines the result of character recognition in the character image M based on the judgment result judged by the judgment unit 12 described later. The judgment result here is a result of judging whether or not the result of character recognition performed using the machine learning model 170 is correct. When the judgment unit 12 judges that the result of character recognition performed using the machine learning model 170 is correct, the cursive character recognition unit 11 sets the result of character recognition performed using the machine learning model 170 as the result of character recognition in the character image M. When the judgment unit 12 judges that the result of character recognition performed using the machine learning model 170 is not correct, the cursive character recognition unit 11 sets the result of character recognition performed using the machine learning model 170 as the result of character recognition in the character image M. The cursive character recognition unit 11 outputs the result of character recognition in the character image M via the data input / output unit 10. Alternatively, the cursive character recognition unit 11 may display the result of character recognition in the character image M on the display unit 15.

[0025] The judgment unit 12 judges whether the result of the character recognition using the machine learning model 170 is correct or not. The judgment unit 12 acquires a signal indicating the judgment result input by an expert or the like through the data input / output unit 10, and judges whether the result of the character recognition is correct or not based on the acquired signal. In this case, for example, the result of the character recognition using the machine learning model 170 is displayed on the display unit 15. The expert or the like visually checks the result of the character recognition displayed on the display unit 15 and judges whether the result of the character recognition is correct or not. When the expert or the like judges that the result of the character recognition is incorrect, that is, incorrect, the expert or the like inputs a signal indicating that to the data input / output unit 10 by clicking the area of ​​the character judged to be incorrect by operating a mouse or the like. The judgment unit 12 outputs the judgment result to the cursive character recognition unit 11.

[0026] When the determination unit 12 determines that the result of character recognition of the character image M using the machine learning model 170 is incorrect, the registration unit 13 stores information (hereinafter referred to as registration information) that associates the character image M with the modern cursive characters shown in the character image M in the character shape data table 160. This allows correct character recognition to be performed using the character shape data table 160 even if erroneous character recognition is performed using the machine learning model 170.

[0027] When storing the registration information in the character data table 160, the registration unit 13 stores the image of the registration information in association with information indicating that there was an error (estimation error) in the estimation using the machine learning model 170. In this way, the registration unit 13 registers, among the images stored in the character data table 160, images without an estimation error by the machine learning model 170 and images with an estimation error. The information indicating that there was an estimation error is, for example, an estimation error flag by the machine learning model (see FIG. 2), which will be described later.

[0028] The machine learning model generation unit 14 generates the machine learning model 170. In the following, a case where a neural network is applied as the machine learning algorithm of the machine learning model 170 will be described as an example. However, the present invention is not limited to this, and as the machine learning algorithm, for example, a method such as a recurrent neural network (RNN), a deep convolutional neural network (DCNN), a CNN, a decision tree, a hierarchical Bayesian, a support vector machine (SVM), or a method that appropriately combines these may be applied.

[0029] The machine learning model generation unit 14 generates a learning data set, and performs machine learning on the generated learning data set using a neural network, thereby generating the machine learning model 170. A method in which the machine learning model generation unit 14 generates the machine learning model 170 will be described in detail later.

[0030] When a predetermined condition is satisfied, the machine learning model generation unit 14 performs re-learning on the machine learning model 170. Re-learning is to execute machine learning again on the machine learning model 170. The machine learning model generation unit 14 performs re-learning with the learning data set used when generating the machine learning model 170 before re-learning, with additional data added. The method of re-learning performed by the machine learning model generation unit 14 will be described in detail later.

[0031] The display unit 15 has a display screen made of, for example, a liquid crystal panel. The display unit 15 displays the character image M acquired by the data input / output unit 10, the determination result of character recognition by the cursive character recognition unit 11, etc. When the whole or part of the display screen of the display unit 15 is a touch panel, data can be input via the data input / output unit 101 by clicking a button on the screen.

[0032] A character shape data table 160 is written and stored in advance in the character shape database 16. Fig. 2 is a diagram showing an example of the configuration of the character shape data table 160 according to an embodiment. As shown in Fig. 2, the character shape data table 160 includes items such as character code, image, modern character, and estimation error flag by a machine learning model (hereinafter, simply referred to as estimation error flag).

[0033] The character code is identification information that uniquely identifies an image of cursive writing. The image is an image of cursive writing identified by the character code. The modern character is a modern character that is deciphered based on the shape of the cursive writing shown in the image identified by the character code. The estimation error flag is information that indicates whether there was an error in estimating the modern character using the machine learning model 170 in the image identified by the character code. In this example, if the flag is 0 (zero), it indicates that there was no error, and if the flag is 1, it indicates that there was an error.

[0034] In this example, it is shown that for the character code (M0001), the modern character of the fragmented character image G1 based on the obsolete character of "礼" is "礼", and there is no error in the estimation of the image G1 by the machine learning model 170. As in this example, even for a fragmented character whose shape cannot be immediately recognized as "礼", if it is determined by an expert or the like that the fragmented character shown in the image is the obsolete character of "礼" and corresponds to the modern character "礼", the judgment result may be shown as the modern character. Also, in this example, it is shown that for the character code (M0002), the modern character of the fragmented character image G2 of the variant kana of "ま" with "満" as the phonetic symbol is "ま", and there is no error in the estimation of the image G2 by the machine learning model 170. As in this example, if it is determined by an expert or the like that fragmented characters corresponding to the same kana, without considering the phonetic symbols, are read as the same modern character even when their shapes are different, the judgment result may be shown as the modern character.

[0035] The machine learning model storage unit 17 stores the machine learning model 170. FIG. 3 is a diagram for explaining the machine learning model 170 according to the embodiment.

[0036] As shown in FIG. 3, the machine learning model 170 is composed of, for example, a first-layer perceptron 171, a second-layer perceptron 172, and a third-layer perceptron 173. Each layer of perceptron is composed of one or more units. In this example, each of the first-layer perceptron 171 and the second-layer perceptron 172 has six units. The third-layer perceptron 173 has one unit. Each unit is connected via nodes to each unit in the upper layer when there is an upper layer to which the unit belongs.

[0037] The first layer perceptron 171 is a so-called input layer, and image information of, for example, a character image M is input to each unit. The image information is information about an image, such as information indicating a pixel value for each pixel or a gray scale. Each piece of image information input to each unit of the first layer perceptron 171 is multiplied by a coupling coefficient (weight W) associated with a node connecting each unit of the first layer perceptron 171 to each unit of the second layer perceptron 172, which is an upper layer, and a bias component b associated with the connected unit is added to the result, and the result is input to the second layer perceptron 172.

[0038] The second layer perceptron 172 is a so-called intermediate layer. In this example, the second layer perceptron 172 is configured with two layers, but the second layer perceptron 172 may be configured with one layer or three or more layers. Each piece of data input to each unit of the lower second layer perceptron 172 is multiplied by a coupling coefficient (weight W) associated with a node connected to each unit of the upper second layer perceptron 172, a bias component b associated with the connected unit is added, and the result is input to the upper second layer perceptron 172. Each piece of data input to each unit of the upper second layer perceptron 172 is multiplied by a coupling coefficient (weight W) associated with a node connected to a unit of the third layer perceptron 173, a bias component b associated with the connected unit of the third layer perceptron 173 is added, and the result is input to the third layer perceptron 173.

[0039] The third layer perceptron 173 is a so-called output layer. The third layer perceptron 173 outputs a value obtained by adding up data input from each unit of the upper second layer perceptron 172.

[0040] Here, a method in which the machine learning model generation unit 14 generates the machine learning model 170 will be described. The machine learning model generation unit 14 generates a learning dataset in a learning stage where machine learning is performed on the neural network. The learning dataset is data in which an image of cursive characters and modern characters of cursive characters shown in the image are combined (set). For example, information stored in the character data table 160 may be used as the learning dataset. The machine learning model generation unit 14 inputs image information of the image of cursive characters in the learning dataset to the first layer perceptron 171 of the machine learning model 170. As a result, calculations such as multiplication and addition are performed according to the parameters (combination coefficient (weight W) and bias component b) set in each of the perceptrons in each layer of the neural network, and some value is output from the third layer perceptron 173. The machine learning model generation unit 14 repeatedly adjusts the parameters of the machine learning model 170 so that the value output from the third layer perceptron 173 approaches a value corresponding to the modern character in the learning dataset. As a result, the machine learning model 170 can accurately output the modern character in the input image. The machine learning model generation unit 14 regards the machine learning model 170 that has been trained until a predetermined termination condition is satisfied as the trained machine learning model 170. The predetermined termination condition is, for example, that all of the training data sets have been trained, or that the accuracy of estimating modern characters in an input image has reached a predetermined threshold or higher. The machine learning model generation unit 14 stores information indicating the trained machine learning model 170 in the machine learning model storage unit 17.

[0041] The machine learning model storage unit 17 stores information indicating the configuration of the machine learning model 170, information indicating adjusted parameters, etc. The information indicating the configuration of the machine learning model 170 is, for example, information indicating the number of units in the first layer perceptron 171, the number of layers and the number of units in each layer in the second layer perceptron 172, and the number of units in the third layer perceptron 173. Information indicating the adjusted parameters is information that associates weights W and bias components b with nodes and units.

[0042] The example in Figure 3 shows a schematic example of a case in which a trained machine learning model 170 performs calculations such as multiplication and addition in the perceptrons of each layer based on image information of character image M, which is a variant kana character for "re," which uses the character "rei," and estimates (character recognition) that the cursive character shown in character image M is the modern character for "re."

[0043] Fig. 4 shows an example of a case where erroneous character recognition is performed using the machine learning model 170 according to the embodiment. The example in Fig. 4 shows a schematic case where the trained machine learning model 170 estimates (character recognizes) that the cursive character shown in the character image M is the modern character "o" as a result of calculations such as multiplication and addition being performed in the perceptrons of each layer based on image information of the variant kana character image M of "ga".

[0044] However, the cursive character shown in the character image M shown in FIG. 4 has been determined by an expert to be a variant kana of "ga". A possible cause of such misrecognition is that the character image M in FIG. 4 is not included in the learning dataset in the machine learning model 170 at the learning stage. In other words, it is considered that the learning dataset does not include a dataset that combines the character image M and the modern character of the cursive character shown in the character image M being "ga". In such a case, the parameters of the machine learning model 170 are not adjusted at the learning stage to output the modern character of the character image M in FIG. 4. For this reason, it is considered that as a result of the execution of calculations such as multiplication and addition in the perceptrons in each layer of the machine learning model 170, it is determined that the shape is close to the shape of the variant kana of "o" that has already been learned, and the variant kana "o" is erroneously estimated.

[0045] When such a misrecognition is found, it is possible to adjust the parameters of the machine learning model 170 again. Specifically, a learning dataset including the character image M of FIG. 4 is generated, and machine learning is performed again using the generated learning dataset. As a result, the parameters of the machine learning model 170 are adjusted in order to accurately output the modern character of the character image M of FIG. 4 in the learning stage. In this case, when image information of the character image M of FIG. 4 is input to the trained machine learning model 170, it is expected that the cursive character shown in the character image M will be output as the modern character "ga" as a result of calculations such as multiplication and addition being performed in the perceptrons of each layer.

[0046] However, if machine learning model 170 is subjected to re-learning every time erroneous character recognition occurs, it takes time for re-learning. This is because, even in the re-learning, it is necessary to repeatedly perform learning and adjust the parameters of machine learning model 170 so that accurate modern characters can be output for all images in the learning dataset.

[0047] As a countermeasure, in this embodiment, when erroneous character recognition is performed using machine learning model 170, the image in which the erroneous character recognition is performed is registered in character shape data table 160. By registering the image in character shape data table 160, accurate character recognition can be performed using character shape data table 160 from the next time onwards. Therefore, accurate character recognition can be performed without having to perform machine learning again on machine learning model 170 every time erroneous character recognition is performed.

[0048] On the other hand, if images that have been erroneously recognized using the machine learning model 170 continue to be registered in the character data table 160, the number of images stored in the character data table 160 will increase. Character recognition using the character data table 160 is a method for searching images. For this reason, the more images stored in the character data table 160, the longer it takes to search.

[0049] To address this issue, in this embodiment, in character recognition using the character shape data table 160, images that do not contain estimation errors by the machine learning model 170 are excluded from the search targets for character recognition.

[0050] 5 is a diagram for explaining the process performed by the registration unit 13 according to the embodiment. The items (character code, image, modern character, estimation error flag, etc.) included in the character shape data table 160 are the same as those in FIG. 2, so the description thereof will be omitted.

[0051] In this example, it is shown that the modern character in image G3 of cursive script with character code (M0101) is "ga", and that there was an error in the estimation of image G3 by machine learning model 170. In this example, image G3 is character image M in FIG. 4, and is an image in which the modern character was erroneously estimated to be "o" by character recognition using machine learning model 170.

[0052] The cursive character recognition unit 11 selects an image to be used for character recognition using the character data table 160 based on the estimation error flag in the character data table 160. Specifically, the cursive character recognition unit 11 selects an image with an estimation error flag of 1, i.e., an image with an estimation error, as an image to be used for character recognition using the character data table 160. This makes it possible to reduce the number of images to be searched and suppress an increase in the time required for character recognition using the character data table 160. On the other hand, for an image with an estimation error by the machine learning model 170, it is possible to obtain a correct character recognition result by setting the image as a search target in character recognition using the character data table 160.

[0053] Here, a method in which the machine learning model generation unit 14 re-learns the machine learning model 170 will be described.

[0054] In this embodiment, when a predetermined condition is satisfied, the machine learning model generation unit 14 performs re-learning on the machine learning model 170. The predetermined condition may be set arbitrarily, but for example, in character recognition using the character data table 160, the number of images to be searched is equal to or greater than a predetermined threshold. In addition, for example, the predetermined condition is that the probability of an estimation error by the machine learning model 170 is equal to or greater than a predetermined threshold. This is because the more images stored in the character data table 160, the longer the time required for search. Even if an image has an estimation error in the previously created machine learning model 170, re-learning will prevent the estimation error from occurring in the machine learning model 170 after re-learning. In this case, the number of images to be searched can be reduced in character recognition using the character data table 160, and the time required for search can be reduced.

[0055] Specifically, the machine learning model generation unit 14 determines whether the number of images in the character shape data table 160 for which the estimation error flag is set to 1 is equal to or greater than a predetermined threshold. If the number of images for which the estimation error flag is set to 1 is equal to or greater than the predetermined threshold, the machine learning model generation unit 14 generates a training dataset including the images for which the estimation error flag is set to 1. The machine learning model generation unit 14 performs machine learning again using the training dataset generated by adding the images for which the estimation error flag is set to 1. This adjusts the parameters of the machine learning model 170 so that modern characters in the images in the training dataset generated by adding the images can be accurately output.

[0056] FIG. 6 is a flowchart showing the flow of operations performed by the cursive script recognition system 1 according to the embodiment. First, the data input / output unit 10 of the cursive character recognition system 1 acquires image information of the character image M (cursive character image) (step S10). Next, the cursive character recognition unit 11 of the cursive character recognition system 1 uses the machine learning model 170 to perform character recognition of modern characters in the character image M (step S11). For example, the cursive character recognition unit 11 performs character recognition of modern characters based on an output obtained by inputting image information of the character image M to the machine learning model 170. Next, the judgment unit 12 of the cursive character recognition system 1 judges whether the result of the character recognition is correct or not (step S12). For example, the judgment unit 12 acquires a signal indicating the judgment result input by an expert or the like through the data input / output unit 10, and judges whether the result of the character recognition is correct or not based on the acquired signal.

[0057] If the result of the character recognition is correct, the cursive character recognition unit 11 outputs the result of the character recognition using the machine learning model 170 as a modern character in the character image M via the data input / output unit 10 (step S13). On the other hand, if the result of the character recognition is incorrect, the registration unit 13 determines whether or not the character image M is stored (registered) in the character shape data table 160 (step S14). If character image M is stored, the cursive character recognition unit 11 performs character recognition of modern characters in character image M using the character shape data table 160 (step S15). Here, the registration unit 13 sets the estimation error flag associated with character image M in the character shape data table 160 to 1 (estimated error). The cursive character recognition unit 11 outputs the result of character recognition using the character shape data table 160 as a modern character in the character image M via the data input / output unit 10 (step S16).

[0058] On the other hand, if the character image M is not stored in the character shape data table 160 in step S14, the registration unit 13 stores the character image M and its modern character in the character shape data table 160 in association with each other (step S17). In this case, the registration unit 13 sets the estimation error flag associated with the character image M to 1 (estimated error).

[0059] The machine learning model generation unit 14 determines whether a predetermined condition (relearning condition) is satisfied (step S18). For example, the machine learning model generation unit 14 determines that the predetermined condition (relearning condition) is satisfied when the number of images in which the estimation error flag is set to 1 in the character shape data table 160 is equal to or greater than a predetermined threshold. When a predetermined condition (relearning condition) is satisfied, the machine learning model generation unit 14 re-learns the machine learning model 170 (step S19). The machine learning model generation unit 14 re-learns the machine learning model 170 by using a learning dataset to which images with an estimation error flag set to 1 have been added. The machine learning model generation unit 14 stores information indicating the retrained machine learning model 170 in the machine learning model storage unit 17 (step S20). In addition, the machine learning model generation unit 14 sets the estimation error flag of the image added to the learning dataset in the character shape data table 160 to 0 (step S21).

[0060] As described above, the cursive character recognition system 1 in the embodiment includes a data input / output unit 10 (an example of a "character image acquisition unit"), a character shape database 16, a machine learning model storage unit 17, a cursive character recognition unit 11, and a machine learning model generation unit 14. The data input / output unit 10 acquires a cursive character image M. The character shape database 16 stores a character shape data table 160 that associates a cursive character image G with a modern cursive character shown in the image G. The machine learning model storage unit 17 stores a machine learning model 170 that has been trained to output a modern cursive character shown in an input cursive character image from the image using a learning data set that associates a learning image with a modern cursive character shown in the learning image. The cursive character recognition unit 11 uses either the character shape data table 160 or the machine learning model 170 to perform character recognition that associates a cursive character shown in the character image M with a modern character based on the character image M. The machine learning model generation unit 14 causes the machine learning model 170 to re-learn when a predetermined condition is satisfied.

[0061] As a result, the cursive character recognition system 1 of the embodiment can perform character recognition using either the character shape data table 160 or the machine learning model 170. For character recognition using the character shape data table 160, an image search is performed, and for example, a feature amount (gradient vector) in an image or a partial area of ​​an image is calculated. On the other hand, for character recognition using the machine learning model 170, a recognition result is obtained by a combination of simple multiplication and addition. Character recognition using the machine learning model 170 requires less processing load and can perform character recognition faster than complex calculations such as gradient vectors. In other words, the cursive character recognition system 1 of the embodiment can perform character recognition of cursive characters without increasing the time required for character recognition, compared to a method using only a database in which images of cursive characters are registered.

[0062] Furthermore, in the cursive character recognition system 1 of the embodiment, when the result of character recognition performed on character image M using the machine learning model 170 is incorrect, the cursive character recognition unit 11 performs the character recognition using the character shape data table 160, and outputs the result of character recognition performed using the character shape data table 160 as the result of character recognition on character image M. As a result, in the cursive character recognition system 1 of the embodiment, first, character recognition is performed using the machine learning model 170, which has a small processing load and can perform character recognition at higher speed, and character recognition is performed using the character shape data table 160 only when there is an error in the character recognition, thereby achieving the same effect as described above.

[0063] The cursive character recognition system 1 of the embodiment further includes a registration unit 13. When the result of character recognition on the character image M using the machine learning model 170 is incorrect, the registration unit 13 stores information that associates the character image M with the modern character of the cursive character shown in the character image M in the character shape data table 160. When performing character recognition using the character shape data table 160, the cursive character recognition unit 11 performs character recognition using information registered in the character shape data table 160 by the registration unit 13. As a result, even if an image with an estimation error in the machine learning model 170 is not registered in the character shape data table 160, it can be newly registered, and character recognition can be performed based on the newly registered image. Therefore, even if there is an error in the character recognition of the machine learning model 170, character recognition can be performed correctly.

[0064] Furthermore, in the cursive character recognition system 1 of the embodiment, the machine learning model generation unit 14 re-trains the machine learning model 170 using a training data set including information registered in the character shape data table 160 by the registration unit 13. This allows re-training to be performed including images in which the machine learning model 170 had an estimation error, and makes it possible for the machine learning model 170 after re-training to perform correct character recognition even for images in which the model before re-training had an error.

[0065] (First Modification of the Embodiment) Next, a first modified example of the embodiment will be described. This modified example differs from the above-mentioned embodiment in that "rectangle information" of the cursive characters is used as learning data. The rectangle information is information indicating a rectangle surrounding the cursive characters. That is, in this modified example, a learning data set is used in which the learning image and the rectangle information are associated with the modern cursive characters shown in the learning image. By adding the rectangle information to the learning data, it becomes possible to estimate the modern cursive characters from the character shape of the cursive characters and the shape of the rectangle, and it is expected that the accuracy of recognizing modern characters will be improved.

[0066] One possible method for acquiring rectangular information for cursive characters is to use data annotation software to manually specify rectangular areas and corresponding characters (or character codes) to create a learning dataset. Data annotation software is software that allows users to overlay an image of a cursive character with an image of a rectangle or other object, and manually set the position where the rectangle is displayed and the size of the rectangle. By using data annotation software, workers can acquire rectangular information for cursive characters while visually checking the image. However, this method is a manual task, and thus imposes a large workload on the workers.

[0067] Another method for acquiring rectangular information of cursive characters is to use software for character recognition of cursive characters (cursive character OCR) to automatically recognize the rectangular area of ​​the cursive characters and the corresponding character code. In cursive character OCR, cursive characters are extracted as rectangles from ancient documents and classical texts that contain cursive characters, and the modern characters corresponding to the extracted characters are displayed. By performing character recognition using cursive character OCR, rectangular information can be automatically recognized. Therefore, compared to the above-mentioned method using data annotation software, it is possible to acquire rectangular information more efficiently. However, rectangular information that is incorrectly recognized by OCR must be manually corrected. If rectangular information is incorrectly recognized, it is highly likely that the modern characters of the cursive characters are also incorrectly recognized. Therefore, if an incorrect recognition occurs in OCR, there is a high possibility that both the process of manually re-specifying the incorrectly recognized rectangle and the process of correcting the modern characters (or character codes) will occur, which reduces the efficiency of the work.

[0068] The present modified example aims to solve the above problem, that is, the problem that rectangular information of cursive characters cannot be efficiently acquired. In the following explanation, only the configuration different from the above-mentioned embodiment will be explained. The same reference numerals will be used for the configuration equivalent to the above-mentioned embodiment, and the explanation will be omitted.

[0069] 7 is a block diagram showing an example of the configuration of a cursive character recognition system 1A according to Modification 1 of the embodiment. The cursive character recognition system 1A of this modification includes, for example, a rectangle information acquisition unit 18. The rectangle information acquisition unit 18 acquires rectangle information of the cursive character.

[0070] The rectangle information acquisition unit 18 acquires a text that has been converted from a part or all of a document that includes cursive characters into modern characters. The document that has been converted into modern characters may be the result of recognition by the cursive character recognition unit 11 of the above-mentioned embodiment, or may be a document or classic book that has been previously associated with modern language text translated by experts.

[0071] The rectangle information acquisition unit 18 acquires the result of character recognition by OCR recognition of the cursive characters corresponding to the acquired modern character sentence. The rectangle information acquisition unit 18 compares the acquired modern character sentence with the result of character recognition by OCR. The rectangle information acquisition unit 18 compares the two by, for example, performing string matching using edit distance. The data input / output unit 10 may compare the two using the characters or character codes of the two.

[0072] If the character recognition results of both are consistent, the rectangle information acquisition unit 18 determines that the rectangle corresponding to the OCR-recognized character is a correct rectangle for that cursive character. On the other hand, if the character recognition results of both are not consistent, the rectangle information acquisition unit 18 determines that the rectangle corresponding to the OCR-recognized character is an incorrect rectangle that does not correspond to that cursive character.

[0073] The rectangle information acquisition unit 18 corrects the rectangle information of the cursive characters that are determined to be erroneous rectangles. The rectangle information is corrected, for example, by a worker or the like using data annotation software. The rectangle information acquisition unit 18 acquires the corrected rectangle information.

[0074] The rectangle information acquisition unit 18 outputs the more likely rectangle information, corrected as necessary, to the machine learning model generation unit 14. The machine learning model generation unit 14 generates a trained model using a training data set in which the training image of the cursive character and the rectangle information are associated with the recognition result of the modern character. When performing character recognition using machine learning, the cursive character recognition unit 11 performs character recognition of the modern cursive character using the trained model that has been trained including the rectangle information and that has been generated by the machine learning model generation unit 14.

[0075] FIG. 8 is a flowchart showing the flow of operations performed by the cursive character recognition system 1A according to the first modification of the embodiment.

[0076] The cursive character recognition system 1A acquires a modern character of the cursive character (step S20). The cursive character recognition system 1A acquires a result of character recognition by OCR by reading the same character as the cursive character used in step S20 by the cursive character OCR (step S21). The cursive character recognition system 1A judges whether the modern character acquired in step S20 and the result of character recognition by OCR acquired in step S20 match (step S22). If the two match, the cursive character recognition system 1A registers the rectangle of the character recognized by OCR as correct rectangle information of the cursive character (step S23). On the other hand, if the two do not match, the cursive character recognition system 1A corrects the rectangle of the character recognized by OCR as incorrect rectangle information of the cursive character (step S24). The cursive character recognition system 1A notifies the worker that correction is necessary in a perceptible manner, for example, by displaying an alert on the display unit 15, and acquires the result of manual correction, such as input operation, by the worker who responded to the notification. The cursive character recognition system 1A returns to step S23 and registers the corrected rectangle information. The cursive character recognition system 1A registers the rectangle information, for example, by storing it in a storage unit (not shown). The cursive character recognition system 1 returns to step S20, and repeats the processes from step S20 to S24 for all of the cursive characters for which rectangle information is to be acquired, until the rectangle information is acquired.

[0077] As described above, the cursive character recognition system 1A according to the first modified example of the embodiment includes the rectangle information acquisition unit 18. The rectangle information acquisition unit 18 acquires rectangle information of the cursive character by using the recognition result of the modern character of the cursive character and the result of the OCR character recognition. The machine learning model generation unit 14 creates a trained model by using a training dataset including the rectangle information. When performing character recognition using a machine learning technique, the cursive character recognition unit 11 performs character recognition by using a trained model trained by using a training dataset including the rectangle information.

[0078] As a result, the cursive script recognition system 1A according to the first modified embodiment of the present invention can compare the modern text with the result of OCR character recognition, and correct only if there is a mismatch. This makes it possible to obtain rectangle information more efficiently than when both the rectangle information and the recognition result are corrected when the result of OCR character recognition is incorrect. In particular, many ancient documents and classic books already have modern translations. By using such modern translations, the work related to step S20 in FIG. 8 described above can be performed more efficiently, and the workload can be significantly reduced compared to the conventional method.

[0079] (Modification 2 of the embodiment) Next, a second modified example of the embodiment will be described. This modified example differs from the above-mentioned embodiment in that a data set in which cursive characters extracted from a cursive text correspond to their modern characters is created as learning data. A cursive text is a collection of multiple cursive characters, and in the following description, each of the cursive characters included in a cursive text may be referred to as an individual cursive character. In addition, in the following description, a collection of multiple modern characters may be referred to as a modern text text, and each of the modern characters included in a modern text text may be referred to as an individual modern character.

[0080] The simplest method for creating a dataset in which images of cursive script correspond to modern characters (hereafter referred to as a learning dataset) is to use data annotation software. For example, using data annotation software, an image of cursive script cut out (extracted into rectangles) from a piece of cursive script text and the modern character (or character code) that corresponds to that cursive script can be manually specified. This makes it possible to create a teacher dataset (a learning dataset) to be used in machine learning. This method requires a high manual workload.

[0081] Furthermore, there is a method of using optical character recognition (OCR) software to create a learning dataset. In the method using character recognition software, character recognition is performed using optical character recognition (OCR) software on an image of cursive characters extracted from a cursive text. This method associates the cursive characters corresponding to the image of the cursive characters with the character code of the modern characters corresponding to the cursive characters. In the method using optical character recognition (OCR) software, errors (misrecognition) may occur in the character recognition of the modern characters in the cursive characters. Such misrecognized characters are corrected manually, for example, to create a learning dataset. However, in this method, it is necessary to manually determine the misrecognition, re-extract the image of the cursive characters of the misrecognized part, and associate the re-extracted image of the cursive characters with the correct character code, which still requires a high manual workload.

[0082] On the other hand, there are many combinations of cursive texts (e.g., ancient documents and classical books) and modern texts (e.g., modern word-for-word translations of ancient documents and classical books). These combinations do not include combinations of cursive images cut out from cursive texts and each modern character. The inventors realized that these combinations could be effectively used to efficiently create a machine learning dataset consisting of a combination of cursive images and each modern character, and completed the present invention. This modified example aims to solve the problem of efficiently creating a machine learning dataset. In the following description, only the configurations different from the above-mentioned embodiment will be described. The same reference numerals will be used to designate configurations equivalent to those of the above-mentioned embodiment, and their description will be omitted.

[0083] 9 is a block diagram showing an example of the configuration of a cursive character recognition system 1B according to a second modification of the embodiment. The cursive character recognition system 1B of this modification further includes a character shape database 16B and a data set creation unit 19. A sentence data set 161 and a character data set 162 are stored in the character shape database 16B. The sentence data set 161 is an example of "first data". The character data set 162 is an example of "second data".

[0084] The text data set 161 is information in which texts written in cursive characters (hereinafter referred to as cursive texts) are associated with texts obtained by literal translations of the cursive texts in modern characters (hereinafter referred to as modern text texts). The text data set 161 is information in which, for example, classic books or literature are associated with literal translations of the classic books or literature in modern characters. For example, the literal translations of modern characters in the text data set 161 can be those created by experts or the like.

[0085] Here, the text data set 161 is data in which each cursive character extracted from a cursive text is not associated with each modern character in a modern text. In this modified example, the text data set 161 is used to create a data set in which each cursive character extracted from a cursive text is associated with a correct modern character.

[0086] The character data set 162 is information in which each image of a cursive character extracted from a cursive text is associated with a modern character of the cursive text shown in each image of the cursive character. The images of each cursive character in the character data set 162 are extracted from the same text as the cursive text in the text data set 161. For example, the images of each cursive character in the character data set 162 are generated by extracting a rectangular image of each cursive character from the cursive text using data annotation software. The modern characters of each cursive character in the character data set 162 are modern characters recognized by character recognition using optical character recognition (OCR) software. Alternatively, the modern characters of each cursive character in the character data set 162 may be modern characters recognized by character recognition using a machine learning model for character recognition (hereinafter referred to as a character recognition model).

[0087] In this case, the character recognition model is a model trained to output each image of a cursive character and a modern character corresponding to each of the cursive characters from an input cursive text. For example, for a training cursive text, a character recognition training data set is prepared in which each image of a cursive character included in the cursive text corresponds to a modern character of the cursive character shown in the image. Then, the character recognition model is made to learn the correspondence between each image of a cursive character in the cursive text and a modern character using the character recognition training data set. This allows the character recognition model to function as a model that outputs each image of a cursive character and a modern character corresponding to each image of the cursive character from an input cursive text.

[0088] The data set creation unit 19 creates a data set for machine learning. The data set for machine learning here is information in which each image of cursive characters extracted from a cursive text is associated with a modern character of the cursive character shown in the image. The data set creation unit 19 creates a data set for machine learning with higher accuracy by correcting errors in the modern characters associated with each image of cursive characters in the character data set 162. The data set creation unit 19 includes, for example, a sorting unit 190, a correction unit 191, and an integration unit 192.

[0089] The sorting unit 190 sorts the character data set 162 into correct data and incorrect data. The correct data here is data in which a correct modern character is associated with each image of a cursive character in the character data set 162. The incorrect data is data in which an incorrect modern character is associated with each image of a cursive character in the character data set 162.

[0090] In this modification, the modern character sentences associated with the cursive character sentences in the sentence data set 161 are assumed to be correct, and sorting is performed by the sorting unit 190. Specifically, the sorting unit 190 determines data in which the modern character of the comparison source and the modern character of the comparison target match as correct data, and determines the portion in which they do not match as incorrect data. The modern character of the comparison source here is the modern character associated with each cursive character image in the character data set 162. The modern character of the comparison target is the modern character in the sentence data set 161 that corresponds to the cursive character that corresponds to each cursive character image in the character data set 162.

[0091] For example, the sorting unit 190 compares the text data set 161 and the character data set 162 by string matching using edit distance. The edit distance is information indicating the degree of difference between two character strings. The sorting unit 190 divides the modern character text in the text data set 161 into character strings using a technique such as morphological analysis. The sorting unit 190 generates character strings by combining modern characters in the character data set 162.

[0092] The sorting unit 190 generates a modern character string by, for example, concatenating modern characters in the character data set 162 in the order in which the cursive characters are written in the cursive document. In this case, the sorting unit 190 may arbitrarily set the order of character strings in which the modern characters in the character data set 162 are combined. For example, the sorting unit 190 combines corresponding modern characters in an order along a predetermined direction, such as a top-to-bottom direction or a left-to-right direction in the cursive document. Alternatively, the sorting unit 190 may combine corresponding modern characters according to the closeness of the images in each cursive document.

[0093] The sorting unit 190 calculates an edit distance between a character string of modern characters in the sentence data set 161 and a character string of modern characters in the character data set 162. If the edit distance is a value indicating that the two character strings match, the sorting unit 190 determines that the image of each cursive character in the character data set 162 corresponding to the character string and the modern character are correct data. On the other hand, if the edit distance is a value indicating that the two character strings do not match, the sorting unit 190 determines that the image of each cursive character in the character data set 162 corresponding to the character string and the modern character are incorrect data. The sorting unit 190 outputs information indicating the incorrect data to the correction unit 191 and the integration unit 192.

[0094] The correction unit 191 corrects data determined as incorrect data by the classification unit 190 to a character string of correct modern characters. For a character string determined as an incorrect portion, the correction unit 191 changes the character string in the character data set 162 to a character string in the sentence data set 161. In this way, the correction unit 191 generates corrected data in which each image of cursive characters classified as incorrect data corresponds to a modern character having the same correspondence as that in the sentence data set 161. The correction unit 191 outputs information indicating the corrected data to the integration unit 192.

[0095] The integrating unit 192 integrates the correct data and the corrected data. For example, the integrating unit 192 refers to the character shape database 16 to acquire the character data set 162. The integrating unit 192 integrates the correct data and the corrected data by changing the incorrect data in the character data set 162 to corrected data. The integrated data becomes data in which correct modern characters are associated with each image of cursive characters that was associated with an incorrect modern character in the character data set 162. This makes it possible to generate data in which correct modern characters are associated with each image of cursive characters extracted from a cursive text.

[0096] 10 is a flowchart showing the flow of operations performed by the cursive script recognition system 1B according to the modified example 2 of the embodiment. The cursive script recognition system 1B acquires the sentence data set 161 and the character data set 162 (step S30), compares each of the two data sets for each character string of modern characters (step S31), and determines whether the two character strings match (step S32).

[0097] If the two character strings match, the cursive script recognition system 1B determines that the data corresponding to that character string in the character data set 162 is correct data (step S33). On the other hand, if the two character strings do not match, the cursive script recognition system 1B determines that the data corresponding to that character string in the character data set 162 is incorrect data (step S34). The cursive script recognition system 1B generates correction data by associating the correct modern characters with the cursive script image of the data determined to be incorrect data (step S35).

[0098] The cursive character recognition system 1B determines whether or not all character strings have been compared (step S36), and if not, returns to step S32 to compare the remaining character strings. On the other hand, if all character strings have been compared, the cursive character recognition system 1B integrates the correct data and the modified data (step S37).

[0099] As described above, the cursive character recognition system 1B of the second modified embodiment creates a data set in which each image of a cursive character constituting a cursive text corresponds to a modern character. The cursive character recognition system 1B further includes a sorting unit 190, a correction unit 191, and an integration unit 192. The sorting unit 190 compares the text data set 161 (an example of "first data") with the character data set 162 (an example of "second data"). The text data set 161 is data in which a cursive text corresponds to a modern text, and each cursive character does not correspond to each modern character in the modern text. The character data set 162 is data in which a modern character corresponds to each image of a cursive character extracted from the same cursive text as the text data set 161. The sorting unit 190 compares the text data set 161 with the character data set 162 to sort the character data set 162 into correct data or incorrect data. The correct data is data in which the correspondence between an image of cursive character and modern characters is the same as that in the text data set 161. The incorrect data is data in which the correspondence between an image of cursive character and modern characters is different from that in the text data set 161. The correction unit 191 generates corrected data. The corrected data is data in which each image of cursive character classified as incorrect data by the classification unit 190 is associated with a modern character having the same correspondence as that in the text data set 161. The integration unit 192 integrates the corrected data and the corrected data.

[0100] As a result, the cursive character recognition system 1B of the second modified embodiment can effectively use the sentence data set 161 to correct errors in the character data set 162. Therefore, when an incorrect modern character is associated with a cursive character, the error can be corrected, and a more accurate data set for machine learning can be created.

[0101] Furthermore, in the cursive character recognition system 1B of the second modified embodiment, the character data set 162 may be data created as a result of character recognition by a character recognition model (one example of a "machine learning model"). The character recognition model is a model trained to output an image of each cursive character and a modern character corresponding to each cursive character image from an input cursive character text. As a result, the cursive character recognition system 1B of the second modified embodiment can easily create an image of each cursive character and a modern character for the cursive character shown in the image from a cursive text using the character recognition model.

[0102] In the cursive character recognition system 1B of the second modified embodiment, the sorting unit 190 may compare the sentence data set 161 with the character data set 162 by string matching using the edit distance. This allows the cursive character recognition system 1B of the second modified embodiment to easily compare the sentence data set 161 with the character data set 162 using the edit distance.

[0103] In the above-mentioned modified example 2, a comparison using the edit distance for each character string has been described as an example, but the present invention is not limited to this. The sorting unit 190 may compare the sentence data set 161 and the character data set 162 using any method. For example, the sorting unit 190 may compare the sentence data set 161 and the character data set 162 without using the edit distance. The sorting unit 190 may compare the sentence data set 161 and the character data set 162 for each character constituting the sentence.

[0104] A program for implementing all or part of the functions of the cursive script recognition system 1 in the above-described embodiment may be recorded on a computer-readable recording medium, and the program recorded on the recording medium may be read into a computer system and executed to perform recognition processing for matching cursive script with the corresponding modern characters. Note that the term "computer system" here includes hardware such as an OS and peripheral devices. Additionally, "computer system" also includes WWW systems equipped with a homepage providing environment (or display environment). Additionally, "computer-readable recording medium" refers to portable media such as flexible disks, optical magnetic disks, ROMs, and CD-ROMs, as well as storage devices such as hard disks built into computer systems. Furthermore, "computer-readable recording medium" also includes devices that hold programs for a certain period of time, such as volatile memory (RAM) within computer systems that act as servers or clients when a program is transmitted via a network such as the Internet or a communication line such as a telephone line.

[0105] The above program may also be transmitted from a computer system in which the program is stored in a storage device or the like to another computer system via a transmission medium, or by transmission waves in the transmission medium. Here, the "transmission medium" that transmits the program refers to a medium that has the function of transmitting information, such as a network (communication network) such as the Internet or a communication line (communication line) such as a telephone line. The above program may also be for realizing part of the above-mentioned functions. Furthermore, it may be a so-called difference file (difference program) that can realize the above-mentioned functions in combination with a program already recorded in the computer system.

[0106] Although an embodiment of the present invention has been described in detail above with reference to the drawings, the specific configuration is not limited to this embodiment, and designs that do not deviate from the gist of the present invention are also included. [Explanation of symbols]

[0107] 1. Cursive writing recognition system 10...Data input / output section 11…Cursive character recognition section 12...Judgment section 13. Registration Department 14...Machine learning model generation section 15...Display section 16... character database 17…Machine learning model memory section

Claims

1. a character image acquisition unit for acquiring a character image of cursive writing; a cursive character recognition unit that performs character recognition to correspond cursive characters shown in the character image to modern characters using a machine learning model and the character image; a registration unit that, when the character recognition of the character image using the machine learning model is incorrect, stores information in a character data table that corresponds the character image in which the character recognition is incorrect to a modern character of cursive writing shown in the character image; a machine learning model generation unit that retrains the machine learning model when the number of images stored in the character data table is equal to or greater than a threshold; Equipped with The machine learning model is a model trained to output, from an input image of cursive writing, a modern character of cursive writing shown in the image, using a training data set in which training images are associated with the modern character of cursive writing shown in the training image; The character shape data table is information that associates an image of a cursive character with a modern character of the cursive character shown in the image. Cursive script recognition system.

2. a character image acquisition unit for acquiring a character image of cursive writing; a cursive character recognition unit that performs character recognition to correspond cursive characters shown in the character image to modern characters using either a character shape data table or a machine learning model and the character image; a registration unit that, when an error occurs in the character recognition using the machine learning model, stores information in the character shape data table that corresponds the character image in which the character recognition was erroneous to a modern character of cursive writing shown in the character image; Equipped with when performing the character recognition using the character shape data table, the cursive character recognition unit performs the character recognition using an image stored in the character shape data table in response to an error in the character recognition using the machine learning model; The machine learning model is a model trained to output, from an input image of cursive writing, a modern character of cursive writing shown in the image, using a training data set in which training images are associated with the modern character of cursive writing shown in the training image; The character shape data table is information that associates an image of a cursive character with a modern character of the cursive character shown in the image. Cursive script recognition system.

3. a character image acquisition unit for acquiring a character image of cursive writing; a cursive character recognition unit that performs character recognition to correspond cursive characters shown in the character image to modern characters using either a character shape data table or a machine learning model and the character image; a registration unit that, when an error occurs in the character recognition using the machine learning model, stores information in the character shape data table that corresponds the character image in which the character recognition was erroneous to a modern character of cursive writing shown in the character image; a machine learning model generation unit that retrains the machine learning model when the number of images stored in the character data table is equal to or greater than a threshold in response to an error in the character recognition using the machine learning model; Equipped with The machine learning model is a model trained to output, from an input image of cursive writing, a modern character of cursive writing shown in the image, using a training data set in which training images are associated with the modern character of cursive writing shown in the training image; The character shape data table is information that associates an image of a cursive character with a modern character of the cursive character shown in the image. Cursive script recognition system.

4. The machine learning model generation unit retrains the machine learning model using the learning dataset including information stored in the character shape data table in response to an error in the character recognition using the machine learning model by the registration unit.

4. The cursive character recognition system according to claim 1 or 3.

5. A method for recognizing cursive script by a computer, comprising: A character image acquisition unit acquires a character image of the cursive script, a cursive character recognition unit performs character recognition using a machine learning model and the character image to correspond the cursive characters shown in the character image to modern characters; When the character recognition of the character image using the machine learning model is incorrect, the registration unit stores information in a character shape data table that corresponds the character image in which the character recognition is incorrect to a modern character of cursive writing shown in the character image; The machine learning model generation unit retrains the machine learning model when the number of images stored in the character data table is equal to or greater than a threshold value; The machine learning model is a model trained to output, from an input image of cursive writing, a modern character of cursive writing shown in the image, using a training data set in which training images are associated with the modern character of cursive writing shown in the training image; The character shape data table is information that associates an image of a cursive character with a modern character of the cursive character shown in the image. How to recognize cursive script.

6. Computer, A character image acquisition means for acquiring a character image of cursive writing; a cursive character recognition means for performing character recognition to correspond a cursive character shown in the character image to a modern character using a machine learning model and the character image; a registration means for storing, in a character shape data table, information corresponding to the character image for which the character recognition was erroneous and a modern character of cursive writing shown in the character image, when the character recognition of the character image using the machine learning model is erroneous; a machine learning model generating means for retraining the machine learning model when the number of images stored in the character data table is equal to or greater than a threshold; A program for causing the device to function as a The machine learning model is a model trained to output, from an input image of cursive writing, a modern character of cursive writing shown in the image, using a training data set in which training images are associated with the modern character of cursive writing shown in the training image; The character shape data table is information that associates an image of a cursive character with a modern character of the cursive character shown in the image. program.

7. A method for recognizing cursive script by a computer, comprising: A character image acquisition unit acquires a character image of the cursive script, a cursive character recognition unit performs character recognition using the character image and either a character shape data table or a machine learning model, thereby corresponding the cursive characters shown in the character image to modern characters; When the character recognition using the machine learning model has an error, the registration unit stores information in the character shape data table that corresponds the character image in which the character recognition was erroneous to a modern character of cursive writing shown in the character image; when performing the character recognition using the character shape data table, the cursive character recognition unit performs the character recognition using an image stored in the character shape data table in response to an error in the character recognition using the machine learning model; The machine learning model is a model trained to output, from an input image of cursive writing, a modern character of cursive writing shown in the image, using a training data set in which training images are associated with the modern character of cursive writing shown in the training image; The character shape data table is information that associates an image of a cursive character with a modern character of the cursive character shown in the image. How to recognize cursive script.

8. Computer, A character image acquisition means for acquiring a character image of cursive writing; a cursive character recognition means for performing character recognition to correspond a cursive character shown in the character image to a modern character using either a character shape data table or a machine learning model and the character image; a registration means for storing, in the character shape data table, information that corresponds the character image in which the character recognition using the machine learning model is erroneous with a modern character of cursive writing shown in the character image, when the character recognition using the machine learning model is erroneous; A program for causing the device to function as a When the cursive character recognition means performs the character recognition using the character shape data table, the cursive character recognition means performs the character recognition using an image stored in the character shape data table in response to an error in the character recognition using the machine learning model; The machine learning model is a model trained to output, from an input image of cursive writing, a modern character of cursive writing shown in the image, using a training data set in which training images are associated with the modern character of cursive writing shown in the training image; The character shape data table is information that associates an image of a cursive character with a modern character of the cursive character shown in the image. program.

9. A method for recognizing cursive script by a computer, comprising: A character image acquisition unit acquires a character image of the cursive script, a cursive character recognition unit performs character recognition using the character image and either a character shape data table or a machine learning model, thereby corresponding the cursive characters shown in the character image to modern characters; When the character recognition using the machine learning model has an error, the registration unit stores information in the character shape data table that corresponds the character image in which the character recognition was erroneous to a modern character of cursive writing shown in the character image; a machine learning model generation unit, when the number of images stored in the character data table due to an error in the character recognition using the machine learning model is equal to or greater than a threshold, re-learning the machine learning model; The machine learning model is a model trained to output, from an input image of cursive writing, a modern character of cursive writing shown in the image, using a training data set in which training images are associated with the modern character of cursive writing shown in the training image; The character shape data table is information that associates an image of a cursive character with a modern character of the cursive character shown in the image. How to recognize cursive script.

10. Computer, A character image acquisition means for acquiring a character image of cursive writing; a cursive character recognition means for performing character recognition to correspond a cursive character shown in the character image to a modern character using either a character shape data table or a machine learning model and the character image; a registration means for storing, in the character shape data table, information that corresponds the character image in which the character recognition using the machine learning model is erroneous with a modern character of cursive writing shown in the character image, when the character recognition using the machine learning model is erroneous; a machine learning model generating means for re-learning the machine learning model when the number of images stored in the character data table is equal to or greater than a threshold value in response to an error in the character recognition using the machine learning model; A program for causing the device to function as a The machine learning model is a model trained to output, from an input image of cursive writing, a modern character of cursive writing shown in the image, using a training data set in which training images are associated with the modern character of cursive writing shown in the training image; The character shape data table is information that associates an image of a cursive character with a modern character of the cursive character shown in the image. program.

Images