Text line recognition system generation method and device, electronic device

Abstract

The present invention provides a method and device for generating a text line recognition system and an electronic device, wherein the method comprises: using a pre-trained first recognition model to score training samples; determining each classification based on each score; training a classification division model based on the training samples; and for each classification, training a second recognition model corresponding to the classification based on the training samples corresponding to the classification. The text line recognition system comprises: the classification division model and each trained second recognition model. The text line recognition system generation method provided by the present invention can not only improve the recognition rate of the predicted samples, but also improve the recognition efficiency by reducing the size of the second recognition model, thereby improving the performance of the text line recognition system as a whole.

Description

Technical Field

[0001] The present invention relates to the technical field of optical character recognition, and in particular to a method and device for generating a text line recognition system, and electronic equipment. Background Art

[0002] The two most common scenarios in OCR (Optical Character Recognition) are text recognition in scanned documents and text recognition in natural scenes taken in photos. The text in scanned documents and text in photos are very different in terms of fonts, backgrounds and other factors. Generally speaking, text recognition in natural scenes is much more difficult than text in scanned documents. In order to cater to the huge differences in fonts and backgrounds, the recognition rate of the text line recognition model is currently mainly ensured by increasing the amount of training sample data and increasing the size of the recognition model. However, when the amount of training sample data reaches a certain order of magnitude, the incremental effect of the text recognition rate brought about by simply increasing the size of the text line recognition model will become smaller and smaller, and increasing the size of the text line recognition model will cause its performance to seriously deteriorate.

[0003] It can be seen that the existing text line recognition model cannot take into account both recognition rate and performance. Summary of the invention

[0004] In view of the above problems, embodiments of the present invention are proposed to provide a method and device for generating a text line recognition system, an electronic device and a storage medium that overcome the above problems or at least partially solve the above problems.

[0005] In a first aspect, an embodiment of the present invention discloses a method for generating a text line recognition system, wherein the method comprises:

[0006] Use the pre-trained first recognition model to score the training samples;

[0007] Determine the categories according to the scores;

[0008] Training a classification model based on the training samples;

[0009] For each of the categories, the second recognition model corresponding to the category is trained based on the training samples corresponding to the category.

[0010] Wherein, the text line recognition system includes: the classification model and each of the second recognition models that have been trained.

[0011] Optionally, after the step of training the second recognition model corresponding to each of the categories according to the training samples corresponding to the categories, the method further includes:

[0012] Inputting the predicted sample into the classification model to obtain the target classification corresponding to the predicted sample;

[0013] The predicted sample is input into a second recognition model corresponding to the target classification to obtain text line information corresponding to the predicted sample.

[0014] Optionally, before the step of scoring the training samples using the pre-trained first recognition model, the method further includes:

[0015] determining a first preset number of samples of the first type;

[0016] determining a second preset number of samples of the second type;

[0017] Extracting a third preset number of first samples from the first type of samples;

[0018] Extracting a fourth preset number of second samples from the second type of samples;

[0019] Using the first sample and the second sample as the training samples to train the first recognition model;

[0020] The other samples of the first type of samples except the first sample and the other samples of the second type of samples except the second sample are determined as test samples.

[0021] Optionally, the step of determining the divided categories according to the scores includes:

[0022] According to the above scoring, determine the number of categories and the score area corresponding to each category;

[0023] For each of the training samples, the category to which the training sample belongs is determined according to the score corresponding to the training sample and the score range corresponding to each category.

[0024] Optionally, the first type of samples are text lines captured in natural scenes, and the second type of samples are scanned text lines.

[0025] In a second aspect, an embodiment of the present invention discloses a text line recognition system generation device, wherein the device comprises:

[0026] A first module is used to score training samples using a pre-trained first recognition model;

[0027] The second module is used to determine the categories according to the scores;

[0028] The third module is used to train a classification model based on the training samples;

[0029] The fourth module is used to train the second recognition model corresponding to each classification based on the training samples corresponding to the classification.

[0030] Wherein, the text line recognition system includes: the classification model and each of the second recognition models that have been trained.

[0031] Optionally, the device further comprises:

[0032] A fifth module is used for, in the fourth module, training the second recognition model corresponding to each of the categories according to the training samples corresponding to the categories, and then inputting the predicted samples into the classification division model to obtain the target category corresponding to the predicted samples;

[0033] The sixth module is used to input the predicted sample into the second recognition model corresponding to the target classification to obtain the text line information corresponding to the predicted sample.

[0034] Optionally, the device further comprises:

[0035] A seventh module is used to determine a first preset number of first type samples before the first module uses the pre-trained first recognition model to score the training samples;

[0036] An eighth module is used to determine a second preset number of second type samples;

[0037] A ninth module, configured to extract a third preset number of first samples from the first type of samples;

[0038] A tenth module, configured to extract a fourth preset number of second samples from the second type of samples;

[0039] An eleventh module is used to train the first recognition model by using the first sample and the second sample as the training samples;

[0040] The twelfth module is used to determine other samples of the first type of samples except the first sample, and other samples of the second type of samples except the second sample, as test samples.

[0041] Optionally, the second module includes:

[0042] A determination submodule is used to determine the number of categories and the score area corresponding to each category according to the scores;

[0043] The classification determination submodule is used to determine, for each of the training samples, the classification to which the training sample belongs according to the score corresponding to the training sample and the score area corresponding to each classification.

[0044] In a third aspect, an embodiment of the present invention discloses an electronic device, comprising: one or more processors; and one or more machine-readable media having instructions stored thereon; when the instructions are executed by the one or more processors, the processors execute the text line recognition system generation method as described in any one of the above items.

[0045] In a fourth aspect, an embodiment of the present invention discloses a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method for generating a text line recognition system as described in any one of the above items.

[0046] The text line recognition system generation scheme provided in the embodiment of the present invention uses a pre-trained first recognition model to score the training samples; determines the categories divided according to the scores of each training sample; trains the classification division model according to the training samples; for each category, trains the second recognition model corresponding to the category according to the training samples corresponding to the category. When recognizing the text line in the prediction sample based on the text line recognition system, after identifying the target category to which the prediction sample belongs, it can be predicted according to the second recognition model corresponding to the target category. Since the second recognition model corresponding to each category is smaller than the existing large unified text line recognition model, it can improve the recognition rate of the prediction sample and its prediction performance. In addition, the text line recognition system generated by the embodiment of the present invention does not need to improve the recognition rate by blindly adding networks, so it has stable performance. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] Figure 1 is a flowchart of steps of a method for generating a text line recognition system according to an embodiment of the present invention;

[0048] Figure 2 It is a structural block diagram of a text line recognition system generating device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0049] The exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although the exemplary embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be limited by the embodiments set forth herein. On the contrary, these embodiments are provided to enable a more thorough understanding of the present disclosure and to fully convey the scope of the present disclosure to those skilled in the art.

[0050] Reference Figure 1 , showing a step flow chart of a method for generating a text line recognition system according to an embodiment of the present invention.

[0051] The method for generating a text line recognition system according to an embodiment of the present invention may include the following steps:

[0052] Step 101: Use the pre-trained first recognition model to score the training samples.

[0053] The first recognition model is a unified recognition model. After the first recognition model is trained with training samples, each training sample is scored with the first recognition model, and the score is the self-recognition rate.

[0054] Before the step of using the pre-trained first recognition model to score the training samples, the first recognition model needs to be trained. An optional method of training the first recognition model includes the following steps:

[0055] Step 1: Determine a first preset number of first type samples;

[0056] Optionally, the first type of sample is: a natural scene text line obtained by shooting, and the second type of sample is a scanned text line. In the specific implementation process, a text simulation tool can be used to simulate the text line in the natural scene and the text line in the scanned scene.

[0057] Step 2: Determine a second preset number of second type samples;

[0058] The first preset number and the second preset number can be configured by a person skilled in the art according to actual needs, and no specific limitation is imposed on this in the embodiments of the present application. The first preset number and the second preset number can be the same or different. For example, the first preset number and the second preset number are both set to 10 million.

[0059] Step 3: extracting a third preset number of first samples from the first type of samples;

[0060] Step 4: extracting a fourth preset number of second samples from the second type samples;

[0061] The third preset number and the fourth preset number can be configured by those skilled in the art according to actual needs, and no specific restrictions are imposed on this in the embodiments of the present application. The third preset number and the fourth preset number may be the same or different. For example: the third preset number and the fourth preset number are both set to 9.5 million. For example: in the actual implementation process, a text simulation tool can be used to simulate 10 million lines of text in natural scenes and 10 million lines of text in scanning scenes; 9.5 million lines of text in natural scenes and 9.5 million lines of text in scanning scenes are randomly selected as training samples, each training sample constitutes a training sample set, and the remaining 1 million lines of samples are used as test samples to constitute a test sample set.

[0062] Step 5: Using the first sample and the second sample as training samples to train the first recognition model;

[0063] The method of training the grand unified recognition model based on samples can refer to the existing model training method, and no specific limitation is made in the embodiments of the present application.

[0064] Step 6: Determine the other samples in the first type of samples except the first sample, and the other samples in the second type of samples except the second sample as test samples.

[0065] The test samples are used to test the prediction effect of the trained grand unified recognition model.

[0066] Step 102: Determine the categories to be divided according to the scores.

[0067] After using the pre-trained first recognition model to score the training samples and obtain the scores, technical personnel in this field can set classification rules for the scored samples based on prior knowledge, and then train a classification model based on the classification rules. The trained classification model can quickly determine the category to which the sample belongs.

[0068] In an optional embodiment, the method of determining each classification according to the scoring of each training sample by the first recognition model includes:

[0069] First, according to each score, determine the number of categories and the score area corresponding to each category;

[0070] The specific number of categories and the corresponding relationship between each category and the score area can be flexibly set by technical personnel in this field according to actual needs, and no specific limitation is made in the embodiments of the present application.

[0071] For example, the scoring results can be manually divided into three categories: difficult, medium, and easy, and the self-recognition rate of the training samples can be used as the score. For example, 0-70 points are set as the difficult recognition field as the first category, 70-90 points are set as the medium difficulty recognition field as the second category, and 90 points or above are set as the easy recognition field as the third category.

[0072] Secondly, for each training sample, the category to which the training sample belongs is determined based on the score corresponding to the training sample and the score area corresponding to each category.

[0073] Step 103: Train a classification model based on the training samples.

[0074] The classification model can use any existing appropriate model, such as resnet18, resnet50, and resnet101_fpn network models. The value specifies the total number of convolutional layers and fully connected layers contained in the model, excluding pooling layers.

[0075] Step 104: For each category, the second recognition model corresponding to the category is trained based on the training samples corresponding to the category.

[0076] Common optical character recognition algorithms include CRNN+CTC, CRNN+attention, and CNN+Transformer.

[0077] CRNN+CTC: CNN is used to extract the features of text lines, RNN is used to extract contextual information, and CTC is used as the loss function;

[0078] CRNN+attention: CNN is used to extract the features of text lines, RNN is used to extract contextual information, and the traditional seq2seq attention mechanism is used for decoding, with softmax cross entropy as the loss function; Seq2seq is sequence to sequence, such as machine translation is to convert one text into another, and OCR recognition is to convert images into text.

[0079] CNN+Transformer: Use CNN to extract the features of text lines, Transformer as decoder, and softmax cross entropy as loss function. The second recognition model and the corresponding algorithm can be selected from the commonly used recognition algorithms and models listed above, or can be selected by technical personnel in this field from existing models and algorithms that have not been listed according to actual needs. Select an exclusive second recognition model and algorithm for each category. The specific rules for selecting the second recognition model for each category can be set by technical personnel in this field according to actual needs, and no specific restrictions are made in the embodiments of the present application. For example: Determine the size of the corresponding second recognition model based on the degree of recognition difficulty corresponding to the size of the recognition model and the classification. The greater the recognition difficulty, the larger the second recognition model; conversely, the smaller the recognition difficulty, the smaller the second recognition model.

[0080] After the second recognition model corresponding to each classification is trained, the text line recognition system generation process is completed. The text line recognition system includes: the classification division model and the trained second recognition models.

[0081] The method for generating a text line recognition system provided by an embodiment of the present invention uses a pre-trained first recognition model to score training samples; determines the categories divided according to the scores of each training sample; trains the classification division model according to the training samples; for each category, trains the second recognition model corresponding to the category according to the training samples corresponding to the category. When recognizing text lines in the prediction samples based on the text line recognition system, after identifying the target category to which the prediction sample belongs, it can be predicted according to the second recognition model corresponding to the target category. Since the second recognition model corresponding to each category is smaller than the existing unified text line recognition model, it can improve the recognition rate of the prediction sample and its prediction performance. In addition, the text line recognition system generated by the method provided by the embodiment of the present invention does not need to improve the recognition rate by blindly adding networks, so it has stable performance.

[0082] In an optional embodiment, after the step of training the second recognition model corresponding to each classification according to the training samples corresponding to the classification, the method further includes the following steps:

[0083] First, the prediction sample is input into the classification model to obtain the target classification corresponding to the prediction sample;

[0084] Secondly, the predicted sample is input into the second recognition model corresponding to the target classification to obtain the text line information corresponding to the predicted sample.

[0085] For example: after the prediction sample is input into the classification division model, the classification division model outputs the result of the first classification, and the prediction sample is predicted by the second recognition model corresponding to the first classification. Based on the method shown in the embodiment of the present application, a text line recognition system is generated. When predicting the prediction sample, the classification division model is first used to classify the prediction sample into a category, and then the second recognition model corresponding to the category to which the prediction sample belongs is used to recognize the prediction sample, which can not only improve the text line recognition rate but also improve the text line recognition speed.

[0086] Reference Figure 2 , showing a structural block diagram of a text line recognition system generating device according to an embodiment of the present invention.

[0087] The text line recognition system generation device of the embodiment of the present invention may include the following modules:

[0088] The first module 201 is used to score the training samples using a pre-trained first recognition model;

[0089] The second module 202 is used to determine the categories according to the scores;

[0090] The third module 203 is used to train a classification model based on the training samples;

[0091] The fourth module 204 is used to train the second recognition model corresponding to each of the categories according to the training samples corresponding to the category.

[0092] Wherein, the text line recognition system includes: the classification model and each of the second recognition models that have been trained.

[0093] Optionally, the device further comprises:

[0094] A fifth module is used for, in the fourth module, training the second recognition model corresponding to each of the categories according to the training samples corresponding to the categories, and then inputting the predicted samples into the classification division model to obtain the target category corresponding to the predicted samples;

[0095] The sixth module is used to input the predicted sample into the second recognition model corresponding to the target classification to obtain the text line information corresponding to the predicted sample.

[0096] Optionally, the device further comprises:

[0097] A seventh module is used to determine a first preset number of first type samples before the first module uses the pre-trained first recognition model to score the training samples;

[0098] An eighth module is used to determine a second preset number of second type samples;

[0099] A ninth module, configured to extract a third preset number of first samples from the first type of samples;

[0100] A tenth module, configured to extract a fourth preset number of second samples from the second type of samples;

[0101] An eleventh module is used to train the first recognition model by using the first sample and the second sample as the training samples;

[0102] The twelfth module is used to determine other samples of the first type of samples except the first sample, and other samples of the second type of samples except the second sample, as test samples.

[0103] Optionally, the second module includes:

[0104] A determination submodule is used to determine the number of categories and the score area corresponding to each category according to the scores;

[0105] The classification determination submodule is used to determine, for each of the training samples, the classification to which the training sample belongs according to the score corresponding to the training sample and the score area corresponding to each classification.

[0106] The text line recognition system generation device provided by the embodiment of the present invention uses a pre-trained first recognition model to score the training samples; determines the categories divided according to the scores of each training sample; trains the classification division model according to the training samples; for each category, trains the second recognition model corresponding to the category according to the training samples corresponding to the category. When recognizing the text line in the prediction sample based on the text line recognition system, after identifying the target category to which the prediction sample belongs, it can be predicted according to the second recognition model corresponding to the target category. Since the second recognition model corresponding to each category is smaller than the existing large unified text line recognition model, it can improve the recognition rate of the prediction sample and its prediction performance. In addition, the text line recognition system generated by the device provided by the embodiment of the present invention does not need to improve the recognition rate by blindly adding networks, so it has stable performance.

[0107] As for the device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0108] In an embodiment of the present invention, an electronic device is also provided. The electronic device may include one or more processors and one or more machine-readable media storing instructions thereon, the instructions being, for example, application programs. When the instructions are executed by the one or more processors, the processors execute the above-mentioned text line recognition system generation method.

[0109] In an embodiment of the present invention, a non-transitory computer-readable storage medium is also provided, on which a computer program is stored, and the program can be executed by a processor of an electronic device to complete the above-mentioned text line recognition system generation method. For example, the non-transitory computer-readable storage medium can be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, etc.

[0110] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.

[0111] It will be appreciated by those skilled in the art that the embodiments of the present invention may be provided as methods, devices, or computer program products. Therefore, the embodiments of the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the embodiments of the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.

[0112] The embodiments of the present invention are described with reference to the flowcharts and / or block diagrams of the methods, terminal devices (systems), and computer program products according to the embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of the processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing terminal device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing terminal device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0113] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing terminal device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce a manufactured product including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0114] These computer program instructions can also be loaded onto a computer or other programmable data processing terminal device so that a series of operating steps are executed on the computer or other programmable terminal device to produce a computer-implemented process, thereby providing instructions for executing on the computer or other programmable terminal device to implement the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0115] Although the preferred embodiments of the present invention have been described, those skilled in the art may make other changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the embodiments of the present invention.

[0116] Finally, it should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or terminal device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or terminal device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or terminal device including the elements.

[0117] The above is a detailed introduction to a text line recognition system generation method and device, electronic device and storage medium provided by the present invention. Specific examples are used in this article to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core idea; at the same time, for general technical personnel in this field, according to the idea of ​​the present invention, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as a limitation on the present invention.

Claims

1. A method for generating a text line recognition system, characterized in that: The method comprises: Use the pre-trained first recognition model to score the training samples; Determine the categories according to the scores; The step of determining the categories according to the scores includes: According to the above scoring, determine the number of categories and the score area corresponding to each category; For each of the training samples, determining the category to which the training sample belongs according to the score corresponding to the training sample and the score range corresponding to each category; Training a classification model based on the training samples; For each of the categories, training the second recognition model corresponding to the category based on the training samples corresponding to the category; Wherein, the text line recognition system includes: the classification model and each of the second recognition models that have been trained.

2. The method according to claim 1, characterized in that: After the step of training the second recognition model corresponding to each of the categories according to the training samples corresponding to the categories, the method further includes: Inputting the predicted sample into the classification model to obtain the target classification corresponding to the predicted sample; The predicted sample is input into a second recognition model corresponding to the target classification to obtain text line information corresponding to the predicted sample.

3. The method according to claim 1, characterized in that Before the step of using the pre-trained first recognition model to score the training samples, the method further includes: determining a first preset number of samples of the first type; determining a second preset number of samples of the second type; Extracting a third preset number of first samples from the first type of samples; Extracting a fourth preset number of second samples from the second type of samples; Using the first sample and the second sample as the training samples to train the first recognition model; The other samples of the first type of samples except the first sample and the other samples of the second type of samples except the second sample are determined as test samples.

4. The method according to claim 3, characterized in that The first type of samples are: text lines of natural scenes captured, and the second type of samples are scanned text lines.

5. A text line recognition system generation device, characterized in that: The device comprises: A first module is used to score training samples using a pre-trained first recognition model; The second module is used to determine the categories according to the scores; Wherein, the second module comprises: A determination submodule is used to determine the number of categories and the score area corresponding to each category according to the scores; A classification determination submodule, for determining, for each of the training samples, the classification to which the training sample belongs based on the score corresponding to the training sample and the score area corresponding to each classification; The third module is used to train a classification model based on the training samples; The fourth module is used to train the second recognition model corresponding to each of the categories according to the training samples corresponding to the categories; Wherein, the text line recognition system includes: the classification model and each of the second recognition models that have been trained.

6. The device according to claim 5, characterized in that The device also includes: A fifth module is used for, in the fourth module, training the second recognition model corresponding to each of the categories according to the training samples corresponding to the categories, and then inputting the predicted samples into the classification division model to obtain the target category corresponding to the predicted samples; The sixth module is used to input the predicted sample into the second recognition model corresponding to the target classification to obtain the text line information corresponding to the predicted sample.

7. The device according to claim 5, characterized in that The device also includes: A seventh module, configured to determine a first preset number of first type samples before the first module uses the pre-trained first recognition model to score the training samples; An eighth module is used to determine a second preset number of second type samples; A ninth module, configured to extract a third preset number of first samples from the first type of samples; A tenth module, configured to extract a fourth preset number of second samples from the second type of samples; An eleventh module is used to train the first recognition model by using the first sample and the second sample as the training samples; The twelfth module is used to determine other samples of the first type of samples except the first sample, and other samples of the second type of samples except the second sample, as test samples.

8. An electronic device, characterized in that: include: one or more processors; and one or more machine-readable media having instructions stored thereon; When the instructions are executed by the one or more processors, the processors are caused to execute the method for generating a text line recognition system as described in any one of claims 1 to 4.

Images