Learning device, character recognition system, learning method, and program

Abstract

To provide a learning device, a character recognition system, a learning method, and a program with which a character that is difficult to recognize can be efficiently learned without taking much time.SOLUTION: A learning device includes an acquisition unit that acquires a recognition result of recognition of characters, a determination unit that identifies a character that is difficult to recognize on the basis of the recognition result, and determines the identified character as an additional character to be learned for a non-parametric model, and a learning unit that causes the non-parametric model to learn the additional character determined by the determination unit.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] The present invention relates to a learning device, a character recognition system, a learning method, and a program. Background Art

[0002] There are technologies for recognizing characters. For example, Patent Document 1 discloses a technology that performs character recognition multiple times in a stepwise manner. If an incorrect recognition is made in the first character recognition, the character type of the incorrect character is identified, and character recognition is performed again by a method suitable for the identified character type. This makes it possible to improve the accuracy of character recognition. Prior Art Literature Patent Literature

[0003] Patent Document 1 Japanese Unexamined Patent Application Publication No. 2020-160609 Summary of the Invention Problems to be Solved by the Invention

[0004] In Patent Document 1, it is necessary to provide a plurality of character recognition engines specialized for specific character types, which results in high device cost, and it is difficult to flexibly respond to cases such as when a new character type is added. As a countermeasure for this, the use of deep learning technology can be considered. For example, an attempt may be considered in which character recognition is performed using a trained model obtained through deep learning, incorrectly recognized characters are added as training data, and the deep learning model is retrained again. However, when retraining a pre-trained deep learning model, there has been a problem that a huge amount of time is required for the retraining.

[0005] The present invention has been made in view of such circumstances, and provides a learning device, a character recognition system, a learning method, and a program that can efficiently learn characters that are difficult to recognize without requiring a huge amount of time. [Means for solving the problem]

[0006] A learning device according to one aspect of the present invention is: This information shows the number of samples (training data) used to train a pre-trained deep learning model, for each character type. Recognition results after character recognition as The acquisition unit to acquire data, and based on the recognition result Furthermore, for characters belonging to character types where the number of samples is below the threshold, character recognition using a pre-trained model based on deep learning is performed. Characters that are difficult to recognize as The system includes a determination unit that identifies characters and determines the identified characters to be used as additional characters to be trained in a nonparametric model, and a learning unit that trains the nonparametric model with the additional characters determined by the determination unit. A learning device according to one aspect of the present invention includes: an acquisition unit that acquires the classification result of a metrically trained nearest neighbor classifier as a recognition result of recognizing characters; a determination unit that identifies characters belonging to character types for which the number of samples is less than a threshold as characters that are difficult to recognize, based on the recognition result, and determines the identified characters as additional characters to be trained in a nonparametric model; and a learning unit that trains the additional characters determined by the determination unit in the nonparametric model.

[0007] A character recognition system according to one aspect of the present invention is the learning device described above. and the learning device The system comprises a nonparametric model trained by [unspecified method], a deep learning model that recognizes characters using deep learning, and an execution unit that uses the nonparametric model to recognize characters that the deep learning model cannot correctly recognize.

[0008] A learning method according to one aspect of the present invention is a learning method performed by a computer, This information shows the number of samples (training data) used to train a pre-trained deep learning model, for each character type. Recognition results after character recognition as The process of acquiring and based on the recognition result Furthermore, for characters belonging to character types where the number of samples is below the threshold, character recognition using a pre-trained model based on deep learning is performed. Characters that are difficult to recognize as The process includes identifying characters and determining the identified characters as additional characters to be trained in a nonparametric model, and training the nonparametric model with the additional characters. A learning method according to one aspect of the present invention is a learning method performed by a computer, comprising the steps of: obtaining the classification result of a metrically trained nearest neighbor classifier as a recognition result of recognizing characters; identifying characters that were not correctly classified by the metrically trained nearest neighbor classifier based on the recognition result as characters that are difficult to recognize, determining the identified characters as additional characters to be trained in a nonparametric model; and training the additional characters in the nonparametric model.

[0009] A program according to one aspect of the present invention is used on a computer. This information shows the number of samples (training data) used to train a pre-trained deep learning model, for each character type. Recognition results after character recognition as Steps to acquire and based on the recognition results Furthermore, for characters belonging to character types where the number of samples is below the threshold, character recognition using a pre-trained model based on deep learning is performed. Characters that are difficult to recognize asThis program performs the steps of identifying a character and deciding that the identified character will be used as an additional character to be trained in a nonparametric model, and then training the nonparametric model with the additional character. A program according to one aspect of the present invention is a program that causes a computer to perform the following steps: obtain the classification result of a metrically trained nearest neighbor classifier as the recognition result of recognizing characters; identify characters that were not correctly classified by the metrically trained nearest neighbor classifier based on the recognition result as characters that are difficult to recognize, determine that the identified characters are to be used as additional characters to be trained in a nonparametric model; and train the nonparametric model with the additional characters. [Effects of the Invention]

[0010] According to the present invention, characters that are difficult to recognize can be trained on a nonparametric model. In deep learning, a model is assumed that derives an output for a given input, and the parameters to be set in the model are determined through learning. In contrast, nonparametric models do not assume a model, but instead use a feature space in which the training data is distributed according to its features, and classify the input data based on its position in the feature space. For example, in the k-nearest neighbors method, a nonparametric model, arbitrary k data points close to the input data are taken from the space where the training data is plotted, and the class to which the input data belongs is predicted by majority vote. Therefore, when adding training data to a nonparametric model and retraining it, it is sufficient to simply add the data to the feature space, requiring almost no additional training time compared to deep learning. Consequently, it is possible to learn characters that are difficult to recognize more efficiently than when retraining a deep learning model. [Brief explanation of the drawing]

[0011] [Figure 1] This is a block diagram showing the configuration of a learning device according to an embodiment. [Figure 2] This is a flowchart showing the processing in the learning device according to the embodiment. [Figure 3] This is a flowchart showing the processing flow performed by the learning device of the embodiment. [Figure 4] This is a block diagram showing the configuration of a character recognition system according to an embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0012] Hereinafter, embodiments of the present invention will be described with reference to the drawings. FIG. 1 is a block diagram of a learning device 1 according to an embodiment of the present invention. The learning device 1 generates a trained model for performing character recognition. The learning device 1 is a computer.

[0013] Configuration of Learning Device 1 As shown in FIG. 1, the learning device 1 according to the embodiment of the present invention includes a character recognition result acquisition unit 11, an additional character determination unit 12, a non-parametric model learning unit 13, a predictor generation unit 14, a character database 15, and non-parametric model information 16.

[0014] The processing performed by the functional units included in the learning device 1 (the character recognition result acquisition unit 11, the additional character determination unit 12, the non-parametric model learning unit 13, and the predictor generation unit 14) is realized by causing a CPU (Central Processing Unit) provided as hardware in the learning device 1 to execute a program.

[0015] The character recognition result acquisition unit 11 acquires a recognition result obtained by recognizing a character. For example, the character recognition result acquisition unit 11 acquires a recognition result obtained by recognizing a character using a deep learning technique. For example, the character recognition result acquisition unit 11 performs character recognition using a trained model learned by deep learning using a character image data set stored in the character database 15. Alternatively, the character recognition result acquisition unit 11 may acquire a recognition result that has been predicted by a trained model learned through deep learning. The character recognition result acquisition unit 11 outputs the character recognition result obtained through deep learning acquired in this manner to the additional character determination unit 12.

[0016] The recognition results include information indicating the degree of difficulty in recognizing the characters. For example, the recognition results may include information such as the correct character, the characters that were candidates for recognition during the character recognition process and the probability (likelihood) that each character is correct, the recognition rate, the misrecognition rate, and the number of training data (sample size) used for training. The recognition results may also include information such as the likelihood, recognition rate, misrecognition rate, and sample size for each character type.

[0017] The character types here include, for example, numbers, alphabets such as Latin letters, hiragana, katakana, and kanji, or any combination thereof. Alternatively, the character types may be languages ​​such as English or Japanese. Furthermore, the character types may be uppercase or lowercase letters, or a combination of the above.

[0018] The additional character determination unit 12 acquires the recognition result from the character recognition result acquisition unit 11 and uses this recognition result information to determine the character to be added to the nonparametric model (hereinafter referred to as the additional character).

[0019] The additional character determination unit 12 identifies characters that are difficult to recognize based on the recognition results and determines the identified characters as additional characters. The additional character determination unit 12 determines the additional characters based on the methods shown in (1) to (3) below, for example.

[0020] (1) Characters that are easily misrecognized The additional character determination unit 12 identifies the additional character among multiple character recognition candidates that have roughly the same likelihood during the character recognition process. For example, suppose the correct answer is the character "あ", and multiple character recognition candidates are given, such as "あ" and "め". Suppose the probability (likelihood) that the character to be recognized is "あ" is 33%, and the probability (likelihood) that the character to be recognized is "め" is 32%, and the difference between their respective likelihoods is less than a threshold (for example, 3%). In this case, the additional character determination unit 12 considers that the difference between the likelihoods of "あ" and "め" as character recognition candidates is less than a threshold, and that even if the recognition result is correct, these are characters that are easily misrecognized, that is, characters that are difficult to recognize, and adds them to the additional character.

[0021] (2) Character types with a small sample size The additional character determination unit 12 selects characters for which the number of training data (number of samples) used for training is less than a threshold as additional characters. In deep learning-based recognition, the recognition rate improves as the number of samples increases. In other words, in deep learning-based recognition, characters with low usage frequency or rare characters often cannot be correctly recognized because there is little training data available. For this reason, the additional character determination unit 12 considers characters whose sample count for each character type is below a threshold as characters that are difficult to recognize and adds them to the system. The threshold here may be a uniform value determined automatically in advance, a value that varies depending on the character type, or a value that is manually determined by the user.

[0022] (3) Incorrectly recognized characters The additional character determination unit 12 designates characters with a low recognition rate or a high misrecognition rate as additional characters. In other words, regardless of the character recognition process or the number of samples, the additional character determination unit 12 may consider characters that were incorrectly recognized as a result of being recognized by a pre-trained deep learning model as characters that are difficult to recognize and designate them as additional characters.

[0023] The nonparametric model learning unit 13 trains the nonparametric model with the additional characters determined by the additional character determination unit 12. This updates the nonparametric model used for character recognition. An example of a nonparametric model is the k-nearest neighbors method. In the case of the k-nearest neighbors method, the nonparametric model learning unit 13 adds data that associates image information of the additional characters with information indicating the correct characters to the feature space (the space where the training data is plotted) as data to be added to the training data.

[0024] The predictor generation unit 14 generates a predictor using a nonparametric model that has been updated by the nonparametric model learning unit 13, which has learned additional characters.

[0025] The character database 15 stores character image datasets. A character image dataset is, for example, a set of information consisting of image information of captured characters and information indicating the result of correctly recognizing the captured characters.

[0026] The nonparametric model information 16 stores information about the nonparametric model. This information includes information that shows the feature space in which the training data is distributed according to its features. The distribution of the training data placed in the feature space is updated by the nonparametric model learning unit 13.

[0027] (Modified version of learning device 1) Here, a modified version of the learning device 1 in the embodiment will be described. This modified version differs from the embodiment described above in that it uses a metric-learned nearest neighbor classifier. Metric learning is a method of classifying data based on a metric, which is the distance in a feature space where character features are plotted. For the sake of simplicity, it is assumed that the metric-learned nearest neighbor classifier here is a different model from the nonparametric model that the nonparametric model learning unit 13 is learning.

[0028] For example, the character recognition result acquisition unit 11 performs character recognition using a nearest neighbor classifier for characters that were incorrectly recognized by the deep learning-trained model. The character recognition result acquisition unit 11 then acquires the recognition results of the character recognition performed by the metric-trained nearest neighbor classifier. The character recognition result acquisition unit 11 outputs the acquired recognition results to the additional character determination unit 12. The recognition results here include information indicating which class the characters were classified into by the nearest neighbor classifier.

[0029] The additional character determination unit 12 determines the additional character by, for example, the method shown in (4) below.

[0030] (4) Characters that the nearest neighbor classifier misrecognizes The additional character determination unit 12 designates characters that were not correctly classified as a result of character recognition by the nearest neighbor classifier as additional characters. For example, if the nearest neighbor classifier classifies a character into an incorrect class, classifies it into multiple classes, or classifies it into one class but that class contains characters different from the target character, the unit considers that character to be difficult to recognize and designates it as an additional character.

[0031] (Regarding the processing flow performed by learning device 1) Figure 2 is a flowchart showing the processing in the learning device 1 according to an embodiment of the present invention. (Step S1): The character recognition result acquisition unit 11 acquires the recognition result of the recognized characters. (Step S2): The additional character determination unit 12 determines the additional character using the recognition result from the character recognition result acquisition unit 11. (Step S3): The nonparametric model learning unit 13 updates the nonparametric model by having the nonparametric model learn the additional characters determined by the additional character determination unit 12. (Step S4): The predictor generation unit 14 generates a predictor that uses the nonparametric model updated by the nonparametric model learning unit 13.

[0032] Figure 3 is a flowchart showing the process for determining the additional characters in step S2.

[0033] (Step S101): The additional character determination unit 12 determines whether the misrecognition rate of the character trained in deep learning with similar characters is above a threshold. If the misrecognition rate with similar characters is above a threshold (Step S101: Yes), the additional character determination unit 12 determines it as an additional character in Step S105.

[0034] (Step S102): The additional character determination unit 12 determines whether the number of samples for each character type is below a threshold for the characters trained in deep learning. If the number of samples for each character type is below the threshold (Step S102: Yes), the additional character determination unit 12 determines it as an additional character in Step S105.

[0035] (Step S103): The additional character determination unit 12 determines whether the recognition rate for each character type is below a threshold for the characters trained in deep learning. If the recognition rate for each character type is below the threshold (Step S103: Yes), the additional character determination unit 12 determines it as an additional character in Step S105.

[0036] (Step S104): The additional character determination unit 12 determines whether the classification result of the metric-learned nearest neighbor classifier is incorrect. If the classification result is incorrect (Step S104: Yes), the additional character determination unit 12 determines it as an additional character in Step S105. (Step S105): The additional character determination unit 12 determines that the character is an additional character if any of the conditions in steps S101 to S104 are met. (Step S106): The additional character determination unit 12 determines that if the character does not fall under any of steps S101 to S104, it is not an additional character.

[0037] As described above, the learning device 1 of the embodiment comprises a character recognition result acquisition unit 11, an additional character determination unit 12, and a nonparametric model learning unit 13. The character recognition result acquisition unit 11 is an example of an "acquisition unit". The additional character determination unit 12 is an example of a "determination unit". The nonparametric model learning unit 13 is an example of a "learning unit". The character recognition result acquisition unit 11 acquires the recognition results. The additional character determination unit 12 identifies characters that are difficult to recognize based on the recognition results and determines the identified characters as additional characters to be trained in the nonparametric model. The nonparametric model learning unit 13 trains the nonparametric model with the additional characters.

[0038] As a result, the learning device 1 of this embodiment can train a nonparametric model to recognize characters that are difficult to recognize. When adding training data to a nonparametric model and retraining it, the training time is almost negligible compared to retraining a model that has already been trained using deep learning. This is because, in deep learning, a model that derives an output for a given input is assumed, and the parameters to be set in the model are determined through learning, whereas the nonparametric model does not assume a model, but uses a feature space in which the training data is distributed according to its characteristics, and classifies the input data based on the position of the input data in the feature space. Therefore, it is possible to train a nonparametric model to recognize characters that are difficult to recognize more efficiently than when retraining a deep learning model.

[0039] Furthermore, in the learning device 1 of this embodiment, the character recognition result acquisition unit 11 acquires the recognition result of a character recognized by a deep learning-trained model. The additional character determination unit 12 designates characters that are difficult for the deep learning-trained model to recognize as additional characters. Thus, in the learning device 1 of this embodiment, in deep learning-based character recognition, characters that are difficult for the deep learning-trained model to recognize can be designated as additional characters.

[0040] In general, even though retraining a pre-trained model using deep learning requires a significant amount of time, the retrained model does not necessarily result in an overall improvement in character recognition accuracy. For example, a character that was correctly recognized before retraining may be misrecognized afterward, or the retrained model may fail to correctly recognize additional characters that have been added to its training.

[0041] In contrast, in this embodiment, a nonparametric model recognizes characters by clustering the image information of the characters in the feature space. Therefore, especially for characters with similar shapes, even if a deep learning-trained model makes an incorrect recognition, it can be expected that the nonparametric model will recognize them correctly. Consequently, it is possible to train a deep learning model to recognize characters that are difficult to recognize more efficiently and accurately than when retraining the deep learning model.

[0042] On the other hand, with nonparametric models for character recognition, the time required for recognition increases as the number of samples increases. The number of characters to be recognized can sometimes exceed one million. Therefore, if we try to recognize all the characters using a nonparametric model, a problem can arise in that the time required for character recognition increases significantly.

[0043] In contrast, in this embodiment, characters that are difficult to recognize using a pre-trained deep learning model are trained in a nonparametric model. As mentioned above, it is known that the recognition rate improves as the number of samples increases in deep learning recognition. For this reason, a pre-trained deep learning model can correctly recognize most frequently used characters and common, general-purpose characters. In this embodiment, the additional characters to be trained in the nonparametric model are limited to characters that are difficult for the pre-trained deep learning model to recognize. This makes it possible to narrow down the additional characters to data that is effective for training the nonparametric model, and to suppress an increase in the number of characters to be trained in the nonparametric model.

[0044] Furthermore, in the learning device 1 of this embodiment, the character recognition result acquisition unit 11 acquires the likelihood, which is the probability that a character that has become a candidate for character recognition is correct, during the process in which the trained model by deep learning recognizes a character. The additional character determination unit 12 selects characters for which the difference in likelihood among the candidate characters for character recognition is less than a threshold as additional characters. As a result, in the learning device 1 of this embodiment, characters that are prone to misrecognition in character recognition by deep learning can be selected as additional characters.

[0045] For example, the characters "あ" and "め" are similar in shape, and deep learning recognition may misidentify "め" as the correct character when it should be "あ," or it may present two recognition candidates, "あ" and "め," for "あ," requiring manual selection of the correct character. Characters with similar shapes like these have a small metric (distance in the feature space). Therefore, even if the number of samples is increased for training, deep learning recognition is likely to repeatedly make incorrect recognitions. Thus, in character recognition, it may be difficult to determine the degree to which character recognition is difficult based solely on the number of samples. For example, if characters have similar shapes, misrecognition is possible regardless of the number of samples.

[0046] In contrast, nonparametric models learn to classify characters like "あ" and "め" into different classes in the feature space by adding a boundary line through learning. Therefore, even if there are multiple characters with similar shapes and multiple characters with similar metrics (distance in the feature space), the model can correctly recognize the characters by adding a boundary line through learning. Consequently, it is possible to train nonparametric models with characters that are more effectively trained using nonparametric models than by retraining deep learning models.

[0047] Furthermore, in the learning device 1 of this embodiment, the character recognition result acquisition unit 11 acquires the number of samples for each character type as the recognition result. The additional character determination unit 12 designates characters belonging to character types whose sample count is below a threshold as additional characters. As a result, in the learning device 1 of this embodiment, characters with a small sample count that are prone to misrecognition in character recognition by deep learning can be designated as additional characters.

[0048] Furthermore, in the learning device 1 of this embodiment, the character recognition result acquisition unit 11 acquires information regarding characters that the trained model using deep learning has incorrectly recognized as recognition results. The additional character determination unit 12 designates the characters that the trained model using deep learning has incorrectly recognized as additional characters. Thus, in the learning device 1 of this embodiment, characters that have been incorrectly recognized in character recognition using deep learning can be designated as additional characters.

[0049] Furthermore, in the learning device 1 according to a modified embodiment, the character recognition result acquisition unit 11 acquires the classification result of the metrically trained nearest neighbor classifier as the recognition result. The additional character determination unit 12 identifies characters that were not correctly classified by the metrically trained nearest neighbor classifier as additional characters. As a result, in the learning device 1 according to a modified embodiment, a nonparametric model can be trained on characters that are difficult to correctly classify when using a metrically trained nearest neighbor classifier, for example, a general-purpose nearest neighbor classifier. Therefore, the number of characters on which the nonparametric model is trained can be further narrowed, and the increase in the number of characters on which the nonparametric model is trained can be suppressed.

[0050] (Regarding the character recognition system 100 using the learning device 1) Here, we will describe the character recognition system 100 using the learning device 1. The character recognition system 100 is a system that has a learning stage in which the learning device 1 trains a nonparametric model, and an execution stage in which character recognition is performed using the nonparametric model that has been updated through the learning stage.

[0051] Figure 4 is a block diagram showing an example configuration of a character recognition system 100 according to an embodiment of the present invention. As shown in Figure 4, the character recognition system 100 comprises, for example, a learning device 1 and an execution device 2.

[0052] Learning device 1 is the learning device 1 described above. Learning device 1 transmits information to the execution device 2, for example, indicating a nonparametric model that has been trained with additional characters.

[0053] Execution device 2 is a computer that performs the process of recognizing characters. Execution device 2 performs character recognition using a combination of a deep learning model and a nonparametric model. The nonparametric model here is a model that has been updated by learning device 1 by training it with additional characters. For example, execution device 2 receives information from learning device 1 indicating the nonparametric model that has been trained with additional characters, and stores the received information as nonparametric model information 27, which will be described later.

[0054] The execution device 2 includes, for example, a character image input unit 21, a deep learning model character recognition unit 22, a nonparametric model character recognition unit 23, a recognition control unit 24, an output unit 25, deep learning model information 26, and nonparametric model information 27.

[0055] The processing performed by the functional units of the execution device 2 (character image input unit 21, deep learning model character recognition unit 22, nonparametric model character recognition unit 23, recognition control unit 24, and output unit 25) is realized by having the CPU (Central Processing Unit) provided as hardware in the execution device 2 execute a program.

[0056] The character image input unit 21 receives character images to be used for character recognition. The character image input unit 21 is, for example, a scanner or a camera.

[0057] The deep learning model character recognition unit 22 performs character recognition using a pre-trained deep learning model, in accordance with the control of the recognition control unit 24, which will be described later. The pre-trained model used by the deep learning model character recognition unit 22 for character recognition is a general deep learning character recognition model, for example, a model trained to recognize general-purpose characters.

[0058] The nonparametric model character recognition unit 23 performs character recognition using a nonparametric model in accordance with the control of the recognition control unit 24. The nonparametric model here is a model updated by learning additional characters using the learning device 1 described above, and is, for example, a model based on the nonparametric model information 27.

[0059] The recognition control unit 24 controls character recognition by the deep learning model character recognition unit 22 and character recognition by the nonparametric model character recognition unit 23.

[0060] For example, the recognition control unit 24 outputs the character image input to the character image input unit 21 to the deep learning model character recognition unit 22, allowing the trained deep learning model to recognize the character. The recognition control unit 24 acquires the results of character recognition by the deep learning model character recognition unit 22, and if the trained deep learning model correctly recognizes the character, it outputs the result to the output unit 25. On the other hand, if the pre-trained deep learning model fails to correctly recognize a character, the recognition control unit 24 outputs the character image to the nonparametric model character recognition unit 23, allowing the nonparametric model to recognize the character. In this case, the recognition control unit 24 outputs the result of the character recognition by the nonparametric model to the output unit 25.

[0061] Alternatively, the recognition control unit 24 may determine, based on the attribute information of the character image input to the character image input unit 21, whether to have the character recognized by a pre-trained deep learning model or a non-parametric model. The attribute information here is similar to the recognition results obtained by the character recognition result acquisition unit 11 described above, and indicates the degree to which it is difficult to recognize the characters. The recognition control unit 24 determines whether the character corresponding to the character image input to the character image input unit 21 is a character that is difficult to recognize, based on the attribute information of the character image input to the character image input unit 21. If the character corresponding to the character corresponding to the character image input to the character image input unit 21 is a character that is difficult to recognize, the recognition control unit 24 outputs the character image to the nonparametric model character recognition unit 23 and allows the nonparametric model to recognize the character. On the other hand, if the character corresponding to the character image input to the character image input unit 21 is not a character that is difficult to recognize, the recognition control unit 24 outputs the character image to the deep learning model character recognition unit 22 and allows the deep learning trained model to recognize the character.

[0062] The output unit 25 is, for example, a display, which displays the input character image and the result of recognizing the character corresponding to the character image.

[0063] As described above, the character recognition system 100 of this embodiment comprises nonparametric model information 27, deep learning model information 26, and a recognition control unit 24. The nonparametric model information 27 is information indicating a nonparametric model learned by the learning device 1. The deep learning model information 26 is information indicating a model learned by deep learning. The recognition control unit 24 uses the nonparametric model to recognize characters that the deep learning model cannot correctly recognize. As a result, the character recognition system 100 of this embodiment can recognize characters that deep learning cannot correctly recognize using the nonparametric model. Since the nonparametric model is a model that has learned additional characters by the learning device 1, it has a high probability of correctly recognizing characters that deep learning cannot correctly recognize. For this reason, the character recognition system 100 of this embodiment can recognize characters with higher accuracy.

[0064] As described above, the learning device 1 according to this embodiment acquires information regarding the recognition results obtained by deep learning, determines additional characters, and adds them to the nonparametric model. By training the nonparametric model with the additional characters, recognition accuracy can be improved without requiring long-term training.

[0065] The learning device 1 and character recognition system 100 in the above-described embodiment may be implemented in whole or in part using a computer. In that case, the program for implementing this function may be recorded on a computer-readable recording medium, and the program recorded on this recording medium may be loaded into the computer system and executed. The term "computer system" here includes hardware such as an OS and peripheral devices. The term "computer-readable recording medium" refers to portable media such as flexible disks, magneto-optical disks, ROMs, CD-ROMs, and storage devices such as hard disks built into the computer system. Furthermore, the term "computer-readable recording medium" may also include those that dynamically hold programs for a short period of time, such as communication lines used when transmitting programs via networks such as the Internet or communication lines such as telephone lines, and those that hold programs for a certain period of time, such as volatile memory inside the computer system that acts as a server or client in such cases. The program may be for implementing a part of the functions described above, or it may be a program that can implement the functions described above in combination with a program already recorded in the computer system, or it may be implemented using a programmable logic device such as an FPGA.

[0066] Although embodiments of this invention have been described in detail above with reference to the drawings, the specific configuration is not limited to these embodiments and includes designs and the like that do not depart from the spirit of this invention.

[0067] For example, the character recognition result acquisition unit 11 may acquire recognition results by recognizing characters using conventional OCR (optical character recognition) technology. In this case, the additional character determination unit 12 designates characters that are difficult to recognize in OCR character recognition as additional characters.

[0068] Furthermore, in the modified embodiments described above, for the sake of simplicity, it was assumed that the metrically trained nearest neighbor classifier is a different model from the nonparametric model that the nonparametric model learning unit 13 trains on, but the invention is not limited to this. For example, the nonparametric model that the nonparametric model learning unit 13 is training on may be a model that has a feature space in which a certain amount of training data is distributed so that it can perform character recognition to a certain extent. In this case, the nonparametric model is a model that has been trained to correctly recognize characters or character types that have been determined to be additional characters by the additional character determination unit 12. Thus, the nonparametric model may be configured to be updated according to the target character to be recognized. [Explanation of Symbols]

[0069] 100...Character recognition system, 11...Character recognition result acquisition unit (acquisition unit), 12...Additional character determination unit (determination unit), 13...Non-parametric model learning unit (learning unit), 14...Predictor generation unit (generation unit)

Claims

1. An acquisition unit obtains information indicating the number of samples, which is the number of training data used to train a pre-trained model using deep learning, for each character type, as recognition results of character recognition. Based on the recognition results, a determination unit identifies characters belonging to character types for which the number of samples is less than a threshold as characters that are difficult to recognize using a pre-trained deep learning model, and determines that the identified characters will be used as additional characters to train the nonparametric model. A learning unit that causes the nonparametric model to learn the additional characters determined by the determination unit, A learning device equipped with the following features.

2. An acquisition unit that obtains the classification result of a metric-learned nearest neighbor classifier as the recognition result of character recognition, Based on the recognition results, a determination unit identifies characters that were not correctly classified by the metrically trained nearest neighbor classifier as characters that are difficult to recognize, and determines that the identified characters will be used as additional characters to train the nonparametric model. A learning unit that causes the nonparametric model to learn the additional characters determined by the determination unit, A learning device equipped with the following features.

3. The acquisition unit acquires the likelihood, which is the probability that a character that has become a candidate for character recognition is likely to be correct, during the process in which the trained model using deep learning recognizes a character. The determination unit selects the additional character from among the candidate characters for character recognition whose difference in likelihood is less than a threshold. The learning device according to claim 1.

4. The acquisition unit acquires information about characters that were incorrectly recognized as a result of recognizing characters using a pre-trained model based on deep learning, The determination unit uses a pre-trained model obtained through deep learning to recognize characters, and identifies any characters that were incorrectly recognized as additional characters. The learning device according to claim 1.

5. A learning device according to claim 1 or claim 2, The nonparametric model learned by the learning device, A deep learning model that recognizes characters using deep learning, An execution unit that uses the non-parametric model to recognize characters that the deep learning model cannot correctly recognize, A character recognition system equipped with the following features.

6. A learning method performed by computers, The process involves obtaining information indicating the number of samples (the number of training data points used to train a pre-trained deep learning model) for each character type, as a recognition result of character recognition. Based on the recognition results, the process involves identifying characters belonging to character types for which the number of samples is less than a threshold as characters that are difficult to recognize using a pre-trained deep learning model, and determining the identified characters as additional characters to be trained on a nonparametric model. The process of training the nonparametric model with the aforementioned additional characters. Learning methods that include this.

7. On the computer, The process involves obtaining information indicating the number of samples (the number of training data points used to train a pre-trained deep learning model) for each character type, as part of the character recognition results. Based on the recognition results, the steps include identifying characters belonging to character types for which the number of samples is less than a threshold as characters that are difficult to recognize using a pre-trained deep learning model, and determining that the identified characters will be used as additional characters to train the nonparametric model, The steps include training the nonparametric model with the aforementioned additional characters. A program that executes the command.

8. A learning method performed by computers, The process involves obtaining the classification result of a metric-trained nearest neighbor classifier as the recognition result after recognizing characters, and Based on the recognition results, the process involves identifying characters that were not correctly classified by the metrically trained nearest neighbor classifier as characters that are difficult to recognize, and determining that these identified characters will be used as additional characters to train the nonparametric model. The process of training the nonparametric model with the aforementioned additional characters. Learning methods that include this.

9. On the computer, The steps include obtaining the classification result of a metric-trained nearest neighbor classifier as the recognition result after recognizing characters, and Based on the recognition results, the steps include identifying characters that were not correctly classified by the metrically trained nearest neighbor classifier as characters that are difficult to recognize, and determining that these identified characters will be added to train the nonparametric model. The steps include training the nonparametric model with the aforementioned additional characters. A program that executes the command.

Images