Character recognition method and device, OCR system and readable storage medium

By generating a long-tail content vector library and replacing recognition characters with confidence below the threshold using recall characters, the problem of unsatisfactory recognition in the long-tail problem is solved, the accuracy of character recognition is improved, the number of model training is reduced, and the number of models is enhanced. The system is robust and reduces costs.

CN119027957APending Publication Date: 2024-11-26北京电科智芯科技有限公司
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202410994349.5
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-07-23
Publication Date
2024-11-26

AI Technical Summary

Technical Problem

In actual business scenarios, image-based character recognition methods have poor recognition results due to the small number of category samples in the training dataset, and supplementing the training dataset to enhance model training will increase time, labor and cost, and in the new business type data The difficulty increases when generated, and how to improve the recognition accuracy and reduce the number of model training is an urgent problem.

Method used

By obtaining the long-tail content library in advance, a long-tail content vector library is generated, and the pattern recognition and character cutting of the recognized pictures is performed. The OCR recognition engine is used to recognize characters and obtain confidence. Characters with confidence below the threshold are converted into vectors. Get the corresponding recall characters in the content vector library and replace the original characters to improve the recognition results.

Benefits of technology

It improves the accuracy of character recognition, reduces the number of model training times, enhances the robustness of the recognition system, reduces time, labor and cost, and does not need to change the original OCR recognition engine.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119027957A_ABST
    Figure CN119027957A_ABST
Patent Text Reader

Abstract

The invention relates to the technical field of character recognition, and discloses a character recognition method and device, an OCR system and a readable storage medium, and the method comprises the steps: generating a long-tail content vector library in advance according to long-tail characters; performing format recognition and character cutting on the to-be-recognized picture; identifying the character cutting result by using an OCR (Optical Character Recognition) engine to obtain a first character identification result and the confidence coefficient of each identified character in the first character identification result; converting the recognition character with the confidence lower than a threshold value into a first vector, acquiring a second vector corresponding to the first vector from a long-tail content vector library, and converting the second vector into a recall character; and combining and restoring the recognition characters with the confidence not lower than the threshold value and the recall characters to obtain a second character recognition result. According to the technical scheme, the recognition characters with the confidence lower than the threshold value in the first character recognition result are processed and replaced through the long-tail content vector library to obtain the final character recognition result, so that the character recognition accuracy is improved, and an original recognition engine does not need to be changed.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the field of character recognition technology, and in particular to a character recognition method, device, OCR system and readable storage medium. Background Art

[0002] The current image-based character recognition method is to build a machine learning model, annotate the corresponding training data set, use the training data set to train the machine learning model, and then use the trained model for character recognition. This method can achieve the expected good recognition effect when the model is reasonably constructed and the training data set is complete enough. However, in actual business scenarios, due to factors such as the richness and diversity of business scenarios and the limitation of training costs, it is inevitable to encounter the long-tail problem, that is, some categories in the training data set contain fewer character samples, which makes the recognition effect less than ideal.

[0003] In order to solve the problem of unsatisfactory recognition effect caused by the long-tail problem, the current method is to enhance the model training by adding more training data sets. However, this method will make the time, manpower and cost of character recognition relatively high, and with the continuous generation of new business type data, the implementation of the above method is becoming more and more difficult.

[0004] In summary, how to improve the accuracy of character recognition and reduce the number of model training times is a technical problem that needs to be urgently solved by technical personnel in this field. Summary of the invention

[0005] In view of this, the purpose of the present application is to provide a character recognition method, device, OCR system and readable storage medium, which are used to improve the accuracy of character recognition and reduce the number of model training times.

[0006] In order to achieve the above objectives, this application provides the following technical solutions:

[0007] A character recognition method, comprising:

[0008] Pre-acquire a long-tail content library, and generate a long-tail content vector library according to the long-tail characters in the long-tail content library;

[0009] Perform layout recognition on the image to be recognized, and perform character segmentation on the layout recognition result;

[0010] Recognize the character segmentation result by using an OCR recognition engine to obtain a first character recognition result and a confidence level of each recognized character in the first character recognition result;

[0011] Converting the recognized characters whose confidence is lower than a threshold into a first vector, acquiring a second vector corresponding to the first vector from the long-tail content vector library, and converting the second vector into a recalled character;

[0012] The recalled characters are used to replace the recognized characters whose confidence is lower than the threshold, and the recognized characters whose confidence is not lower than the threshold are combined with the recalled characters to restore and obtain a second character recognition result.

[0013] Optionally, generating a long-tail content vector library according to the long-tail characters in the long-tail content library includes:

[0014] Convert each of the long-tail characters in the long-tail content library into a corresponding picture;

[0015] Encode the pictures corresponding to the long-tail characters to generate corresponding vectors;

[0016] The generated vectors are stored in a database to obtain the long-tail content vector library.

[0017] Optionally, before converting each of the long-tail characters in the long-tail content library into a corresponding picture, the method further includes:

[0018] Each of the long-tail characters is converted into long-tail characters in multiple font styles.

[0019] Optionally, it also includes:

[0020] When the long-tail characters added in the long-tail content library are determined, the added long-tail characters are converted into corresponding pictures, the pictures corresponding to the added long-tail characters are encoded, and the corresponding vectors generated by the encoding are stored in the long-tail content vector library.

[0021] Optionally, before generating corresponding pictures for the added long-tail characters, the method further includes:

[0022] The added long-tail characters are converted into long-tail characters in multiple font styles.

[0023] Optionally, acquiring a second vector corresponding to the first vector from the long-tail content vector library includes:

[0024] Calculating the similarity between each vector in the long-tail content vector library and the first vector;

[0025] A vector corresponding to the maximum similarity is obtained from each vector in the long-tail content vector library, and the vector corresponding to the maximum similarity is determined as a second vector corresponding to the first vector.

[0026] Optionally, calculating the similarity between each vector in the long-tail content vector library and the first vector includes:

[0027] The cosine similarity between each vector in the long-tail content vector library and the first vector is calculated.

[0028] A character recognition device, comprising:

[0029] A pre-generation module, used to pre-acquire a long-tail content library and generate a long-tail content vector library according to the long-tail characters in the long-tail content library;

[0030] The recognition and cutting module is used to perform layout recognition on the image to be recognized and perform character cutting on the layout recognition result;

[0031] A character recognition module, used to recognize the character segmentation result by using an OCR recognition engine, and obtain a first character recognition result and a confidence level of each recognized character in the first character recognition result;

[0032] a conversion acquisition module, configured to convert the recognized characters whose confidence level is lower than a threshold into a first vector, obtain a second vector corresponding to the first vector from the long-tail content vector library, and convert the second vector into a recalled character;

[0033] The combination restoration module is used to use the recalled characters to replace the recognized characters whose confidence is lower than the threshold, and to restore the recognized characters whose confidence is not lower than the threshold by combining them with the recalled characters to obtain a second character recognition result.

[0034] An OCR system, comprising:

[0035] Memory for storing computer programs;

[0036] A processor is used to implement the steps of the character recognition method as described in any one of the above when executing the computer program.

[0037] A readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the character recognition method as described in any one of the above items are implemented.

[0038] The present application provides a character recognition method, device, OCR system and readable storage medium, wherein the method includes: pre-acquiring a long-tail content library, generating a long-tail content vector library according to the long-tail characters in the long-tail content library; performing layout recognition on the image to be recognized, and performing character segmentation on the layout recognition result; using an OCR recognition engine to recognize the character segmentation result, and obtaining a first character recognition result and the confidence of each recognized character in the first character recognition result; converting the recognized characters whose confidence is lower than a threshold into a first vector, obtaining a second vector corresponding to the first vector from the long-tail content vector library, and converting the second vector into a recalled character; replacing the recognized characters whose confidence is lower than a threshold with the recalled character, combining and restoring the recognized characters whose confidence is not lower than the threshold with the recalled character, and obtaining a second character recognition result.

[0039] The above technical solution disclosed in the present application performs layout recognition and character segmentation on the image to be recognized, recognizes the character segmentation result using the OCR recognition engine, obtains the first character recognition result and the confidence of each recognized character therein, then converts the recognized characters with a confidence lower than a threshold into a first vector, and obtains a second vector corresponding to the first vector from a long-tail content vector library pre-generated according to the long-tail characters in the long-tail content library, converts the second vector into a recall character, and then uses the recall character to replace the recognized characters with a confidence lower than the threshold, and then combines the recognized characters with a confidence not lower than the threshold with the recall character to restore, and obtains the second character recognition result. The above method can realize the processing of the recognized characters with a confidence lower than the threshold in the first character recognition result using the long-tail content vector library, without the need to retrain the recognition model in the OCR recognition engine by supplementing more training data sets, so as to improve the character recognition accuracy without changing the original OCR recognition engine, thereby enhancing the robustness of the recognition system, reducing the time, manpower and cost consumed by character recognition, and improving the efficiency of character recognition.

[0040] Additional aspects and advantages of the present application will be given in part in the description below, and in part will become apparent from the description below, or will be learned through the practice of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] Figure 1 A flowchart of a character recognition method provided in an embodiment of the present application;

[0042] Figure 2 A flowchart of another character recognition method provided in an embodiment of the present application;

[0043] Figure 3 A structural diagram of a character recognition device provided in an embodiment of the present application;

[0044] Figure 4 A schematic diagram of the structure of an OCR system provided in an embodiment of the present application. DETAILED DESCRIPTION

[0045] The current image-based character recognition method is to build a machine learning model (including deep neural networks), annotate the corresponding training data set to train the machine learning model, and then use the trained machine learning model for character recognition. This implementation method can achieve the expected good recognition effect when the model is reasonably constructed and the training data set is complete enough. However, in actual business scenarios, due to factors such as the richness and diversity of business scenarios and the limitation of training costs, it is inevitable to encounter the long-tail problem, that is, the number of samples of some categories in the training data set is small, and the long-tail problem will lead to unsatisfactory recognition results.

[0046] In order to solve the problem of unsatisfactory recognition results caused by the long-tail problem, at present, the model is enhanced by adding more training data sets, that is, the model is repeatedly trained by adding more long-tail data. However, repeated model training will bring higher costs. For example, it will lead to an increase in the time spent on character recognition, an increase in manpower, an increase in costs, etc., and with the continuous generation of new business type data, the above method is becoming more and more difficult to implement.

[0047] To this end, the present application provides a character recognition method, device, OCR system and readable storage medium, which are used to improve the accuracy of character recognition and reduce the number of model training times.

[0048] Embodiments of the present application are described in detail below, and examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the present application, and should not be construed as limiting the present application.

[0049] See also Figure 1 , which shows a flow chart of a character recognition method provided by an embodiment of the present application. The character recognition method provided by an embodiment of the present application may include:

[0050] S11: pre-acquire a long-tail content library, and generate a long-tail content vector library according to the long-tail characters in the long-tail content library.

[0051] It should be noted that the execution entity of the character recognition method provided in the embodiment of the present application may be the character recognition device or OCR system mentioned later, etc. The embodiment of the present application takes the OCR system as an example for explanation.

[0052] First, business personnel can define and maintain special characters (i.e., long-tail characters), such as rare characters, uncommon characters, etc. These long-tail characters are often not fully covered and trained in ordinary OCR (Optical Character Recognition) systems due to their low frequency of occurrence. In order to accurately identify these long-tail characters, business personnel can establish a long-tail content library, which stores long-tail knowledge (specifically, multiple long-tail characters). That is, the long-tail content library is a content library defined and maintained by business personnel, which contains multiple long-tail characters (not only letters, numbers, etc., but also symbols such as operation symbols and punctuation marks).

[0053] The OCR system can obtain the long-tail content library in advance, and generate a long-tail content vector library according to the long-tail characters in the long-tail content library, wherein the long-tail content vector library contains vectors corresponding to each long-tail character in the long-tail content library. By converting the long-tail characters in the long-tail content library into vectors to generate the long-tail content vector library, it is easy to realize similarity calculation, retrieval, etc., so as to perform character recognition with the help of the external long-tail content vector library, improve the accuracy of character recognition, and make the OCR recognition model in the OCR recognition engine not have to be reconstructed, thereby enhancing the robustness of the OCR system.

[0054] It should be noted that step S11 is a pre-executed process, that is, after obtaining the long-tail content library and generating the long-tail content vector library in advance, when performing character recognition, the long-tail content vector library generated by step S11 can be directly used, without having to perform step S11 every time character recognition is performed. Moreover, step S11 can be performed offline to prepare for character recognition in an offline state, so that the long-tail content vector library in step S11 can be directly used when performing character recognition, thereby improving the efficiency and accuracy of online character recognition.

[0055] In addition, in order to further improve the accuracy of character recognition, the long-tail content library and long-tail content vector library mentioned above can be for a business scenario, such as an electric power business scenario or an Internet scenario or an automobile business scenario, that is, each business scenario can correspond to a long-tail content library and a long-tail content vector library. That is, for a certain target business scenario, the long-tail characters in the target business scenario can be obtained, and the business personnel in the target business scenario can build a long-tail content library corresponding to the target business scenario based on the long-tail characters in the target business scenario. The OCR system can obtain the long-tail content library corresponding to the target business scenario, and generate a long-tail content vector library corresponding to the target business scenario based on the long-tail content library corresponding to the target business scenario. Of course, it is also possible not to distinguish between business scenarios.

[0056] S12: Performing layout recognition on the image to be recognized, and performing character segmentation on the layout recognition result.

[0057] When character recognition is required, the OCR system can obtain the image to be recognized (including the characters to be recognized), and can perform layout recognition on the image to be recognized to obtain the layout recognition result. Then, the layout recognition result can be character segmented, and the layout recognition result can be segmented into multiple fine-grained object targets to obtain the character segmentation result. That is, after obtaining the image to be recognized, the image to be recognized can be firstly subjected to layout recognition and detection, and segmented into multiple fine-grained image targets.

[0058] Among them, in order to improve the accuracy of character recognition, after obtaining the picture to be recognized and before performing layout recognition on the picture to be recognized, the picture to be recognized can also be preprocessed. Specifically, the picture to be recognized can be binarized, noise removed, tilt corrected, etc., so as to improve the quality of the picture to be recognized, thereby improving the accuracy of character recognition.

[0059] S13: using an OCR recognition engine to recognize the character segmentation result, and obtaining a first character recognition result and a confidence level of each recognized character in the first character recognition result.

[0060] After the character segmentation result is obtained, the character segmentation result may be recognized by using an OCR recognition engine to obtain a first character recognition result and the confidence level of each recognized character in the first character recognition result.

[0061] Among them, the OCR recognition engine can be a general OCR recognition engine, and can be one of a variety of OCR recognition engines. The OCR recognition engine used in the embodiment of the present application can be selected according to needs. The OCR recognition engine includes a recognition model, that is, the recognition model in the OCR recognition engine is used to recognize the character segmentation result to obtain the first character recognition result and the confidence of each recognized character in the first character recognition result. The recognition model in the OCR recognition engine can be a baseline model, or it can be a model obtained after training with a labeled training data set, etc.

[0062] S14: converting the recognized characters whose confidence is lower than the threshold into a first vector, acquiring a second vector corresponding to the first vector from the long-tail content vector library, and converting the second vector into a recalled character.

[0063] After obtaining the confidence of each recognized character in the first character recognition result, the first character recognition result and the confidence may be transitioned to filter out recognized characters with confidences lower than a threshold.

[0064] Among them, if the confidence of the recognized characters is not lower than the threshold (the threshold can be set through preliminary experiments or based on experience, etc.), it means that these recognized characters have greater credibility. At this time, it can be determined that these recognized characters (i.e., recognized characters with confidence not lower than the threshold) are correctly recognized, so no additional processing is required.

[0065] If the confidence of the recognized characters is lower than the threshold, it indicates that these recognized characters have a lower credibility. At this time, it can be determined that these recognized characters (i.e., recognized characters with confidence lower than the threshold) are recognized incorrectly and need additional processing to improve the accuracy of character recognition. Specifically, for recognized characters with confidence lower than the threshold, the closest vector is obtained from the long-tail content vector library by retrieval and recall, and the closest vector is converted into a character to replace the recognized characters with confidence lower than the threshold for output. Specifically, the recognized characters with confidence lower than the threshold can be converted into a first vector first, and then the second vector corresponding to the first vector can be obtained from the long-tail content vector library, wherein the second vector corresponding to the first vector can be the vector closest to the first vector in the long-tail content vector library, and after the second vector corresponding to the first vector is obtained from the long-tail content vector library, the second vector can be converted into a character to convert the second vector into a recalled character (the recalled character is the character obtained by converting the second vector). The process of converting the recognized characters with confidence lower than the threshold into the first vector may be: converting the recognized characters with confidence lower than the threshold into corresponding pictures (specifically, this may be implemented using a picture generation tool or a picture generator), encoding the pictures corresponding to the recognized characters with confidence lower than the threshold (specifically, this may be implemented using a picture encoder), so as to be mapped into the first vector. As for converting the second vector into the recall character, when executing step S11, the mapping relationship between each long-tail character in the long-tail content library and each vector in the long-tail content vector library is obtained. On this basis, the specific process of converting the second vector into the recall character may be: determining the long-tail character corresponding to the second vector according to the mapping relationship, and determining the long-tail character corresponding to the second vector as the recall character obtained by converting the second vector; or the second vector may be decoded (specifically, it may be decoded using a decoder) to convert the second vector into the recall character.

[0066] S15: Using the recalled characters to replace the recognized characters whose confidence is lower than the threshold, combining the recognized characters whose confidence is not lower than the threshold with the recalled characters to restore them, and obtaining a second character recognition result.

[0067] After converting the second vector corresponding to the first vector into a recalled character, the recalled character can be used to replace the recognized characters whose confidence is lower than the threshold, and then the recognized characters whose confidence is not lower than the threshold and the recalled characters can be combined and restored (i.e., combined and unified in layout restoration) to obtain the second character recognition result (i.e., the final recognition result). That is, the long-tail content vector library can be used to correct the recognized characters whose confidence is lower than the threshold in the first character recognition result obtained by the OCR recognition engine, and then, based on the correction result and the recognized characters whose confidence is not lower than the threshold in the first character recognition result obtained by the OCR recognition engine, the final character recognition result can be obtained by combining and restoring the layout. After obtaining the second character recognition result, the second character recognition result can be output so that relevant personnel can obtain the corresponding character recognition result in a timely manner.

[0068] From the above, it can be seen that the embodiment of the present application focuses on the baseline recognition, plugs in the long-tail content vector library, and compares and learns the recognized characters with confidence lower than the threshold (which can be called long-tail recognition targets) through retrieval enhancement, without retraining the recognition model in the OCR recognition engine. When dynamically responding to the needs of business changes, it can not only conveniently and easily improve the recognition accuracy of long-tail problems, but also does not need to retrain the recognition model in the OCR recognition engine, and does not need to change the original OCR recognition engine, so that the system does not need to be reconstructed, the robustness of the system is enhanced, and the time spent on character recognition is shortened, and the manpower consumption and cost of character recognition are reduced. Moreover, from the above, it can be seen that the embodiment of the present application constructs an OCR recognition framework based on the "divide and conquer" idea, solves the long-tail problem based on the plug-in long-tail content vector library, divides the OCR recognition task into general recognition objects and long-tail recognition objects, and uses the long-tail content vector library to process the long-tail recognition objects through retrieval, so as to improve the character recognition accuracy without changing the original OCR recognition engine, so that the system has stronger robustness.

[0069] The above technical solution disclosed in the embodiment of the present application performs layout recognition and character segmentation on the image to be recognized, recognizes the character segmentation result using the OCR recognition engine, obtains the first character recognition result and the confidence of each recognized character therein, then converts the recognized characters with a confidence lower than a threshold into a first vector, and obtains a second vector corresponding to the first vector from a long-tail content vector library pre-generated according to the long-tail characters in the long-tail content library, converts the second vector into a recall character, and then uses the recall character to replace the recognized characters with a confidence lower than the threshold, and then combines the recognized characters with a confidence not lower than the threshold with the recall character to restore, and obtains the second character recognition result. The above method can realize the processing of the recognized characters with a confidence lower than the threshold in the first character recognition result using the long-tail content vector library, without the need to retrain the recognition model in the OCR recognition engine by supplementing more training data sets, so as to improve the character recognition accuracy without changing the original OCR recognition engine, thereby enhancing the robustness of the recognition system, reducing the time, manpower and cost consumed by character recognition, and improving the efficiency of character recognition.

[0070] See also Figure 2 , which shows a flow chart of another character recognition method provided by an embodiment of the present application. A character recognition method provided by an embodiment of the present application generates a long-tail content vector library according to long-tail characters in a long-tail content library, which may include:

[0071] Convert each long-tail character in the long-tail content library into a corresponding picture;

[0072] Encode the images corresponding to each long-tail character and generate the corresponding vector;

[0073] The generated vectors are stored in a database to obtain a long-tail content vector library.

[0074] In the embodiment of the present application, the method of generating a long-tail content vector library according to the long-tail characters in the long-tail content library may specifically be:

[0075] (1) Each long-tail character in the long-tail content library is converted into a corresponding image through an image generation tool, an image generator, or an image recognizer (that is, each long-tail character is converted into an image format), so as to facilitate the generation of a corresponding vector;

[0076] (2) Encode the image corresponding to each long-tail character using an image encoder to map the image code corresponding to each long-tail character into a vector form; specifically, a VGG network (a convolutional neural network proposed by the Visual Geometry Group of the University of Oxford) can be used as the image encoder.

[0077] (3) The corresponding vector generated by encoding the image corresponding to each long-tail character is stored in a database to obtain a long-tail content vector library.

[0078] The above method can reduce the complexity of generating the long-tail content vector library and improve the generation efficiency of the long-tail content vector library.

[0079] It should be noted that Figure 2 The OCR detection in the image is used to perform layout recognition and character segmentation on the image to be identified. Specifically, the image to be identified is subjected to layout recognition and detection, and is segmented into multiple fine-grained image targets.

[0080] A character recognition method provided in an embodiment of the present application may further include, before converting each long-tail character in the long-tail content library into a corresponding image:

[0081] Each long-tail character is converted into long-tail characters in multiple font styles.

[0082] In an embodiment of the present application, before converting each long-tail character in the long-tail content library into a corresponding picture, each long-tail character in the long-tail content library can be converted into long-tail characters of multiple font styles through a character editing tool, and can be converted into as many font styles as possible. Then, the long-tail characters of each font style can be converted into a corresponding picture, and each corresponding picture can be encoded to generate a corresponding vector, and each generated vector can be stored.

[0083] In the above manner, each long-tail character can correspond to multiple vectors, and each vector corresponds to a font style of the long-tail character, so that the font style can be taken into account during character recognition, thereby improving the accuracy of detection recall and enhancing the robustness of the system.

[0084] A character recognition method provided in an embodiment of the present application may further include:

[0085] When the added long-tail characters in the long-tail content library are determined, the added long-tail characters are converted into corresponding pictures, the pictures corresponding to the added long-tail characters are encoded, and the corresponding vectors generated by the encoding are stored in the long-tail content vector library.

[0086] In an embodiment of the present application, when a business person finds a new long-tail character, the new long-tail character can be stored in the long-tail content library, and a notification of the addition of a long-tail character in the long-tail content library can be sent to the OCR system, so that the OCR system can determine the addition of a long-tail character in the long-tail content library. Alternatively, the OCR can monitor the long-tail content library regularly or in real time, and compare the long-tail content library at the current monitoring moment with the long-tail content library at the previous monitoring moment to determine whether a long-tail character is added to the long-tail content library. If it is determined that a long-tail character is added to the long-tail content, it can also be determined which one or which long-tail characters are added.

[0087] When the OCR system determines that long-tail characters are added to the long-tail content library, the added long-tail characters can be obtained and converted into corresponding pictures through a picture generation tool, a picture generator, or a picture recognizer, etc. Then, the picture corresponding to the added long-tail characters can be encoded using a picture encoder to generate a vector corresponding to the added long-tail characters, and the vector corresponding to the added long-tail characters is stored in the long-tail content vector library to update the long-tail content vector library, thereby facilitating further improving the accuracy of character recognition.

[0088] A character recognition method provided by an embodiment of the present application may further include, before generating corresponding images for the added long-tail characters:

[0089] Convert the increased long-tail characters into long-tail characters in multiple font styles.

[0090] In an embodiment of the present application, before generating corresponding images for the added long-tail characters, the added long-tail characters can be converted into long-tail characters of multiple font styles, and then image conversion and image encoding are performed to convert the long-tail characters of each font style into corresponding vectors, thereby facilitating improving the accuracy of character recognition.

[0091] A character recognition method provided by an embodiment of the present application, which obtains a second vector corresponding to a first vector from a long-tail content vector library, may include:

[0092] Calculate the similarity between each vector in the long-tail content vector library and the first vector;

[0093] A vector corresponding to the maximum similarity is obtained from each vector in the long-tail content vector library, and the vector corresponding to the maximum similarity is determined as a second vector corresponding to the first vector.

[0094] In the embodiment of the present application, the specific process of obtaining the second vector corresponding to the first vector from the long-tail content vector library may be:

[0095] (1) Calculate the similarity between each vector in the long-tail content vector library and the first vector respectively;

[0096] (2) Retrieve and sort each vector in the long-tail content vector library based on similarity. Specifically, each vector in the long-tail content vector library may be sorted in descending order of similarity, or each vector in the long-tail content vector library may be sorted in descending order of similarity. Then, the vector corresponding to the maximum similarity may be obtained based on the sorting of each vector in the long-tail content vector library, and the vector corresponding to the maximum similarity may be determined as the second vector corresponding to the first vector. Of course, the vector corresponding to the maximum similarity may also be obtained from each vector in the long-tail content vector library in other ways, which is not limited in the present embodiment of the application.

[0097] Of course, when obtaining the second vector corresponding to the first vector, the second vector can also be determined and obtained in combination with the context of the recognized character corresponding to the first vector and having a confidence level lower than a threshold. In addition, the vector with the smallest Euclidean distance among the vectors in the long-tail content vector library can be obtained by calculating the Euclidean distance between each vector in the long-tail content vector library and the first vector, and the vector can be determined as the second vector corresponding to the first vector.

[0098] The above method can improve the accuracy of obtaining the second vector corresponding to the first vector, and facilitate improving the accuracy of character recognition.

[0099] A character recognition method provided by an embodiment of the present application, which calculates the similarity between each vector in a long-tail content vector library and a first vector, may include:

[0100] The cosine similarity between each vector in the long-tail content vector library and the first vector is calculated.

[0101] In an embodiment of the present application, the cosine similarity between each vector in the long-tail content vector library and the first vector may be calculated, and the second vector corresponding to the first vector may be determined based on the cosine similarity between each vector in the long-tail content vector library and the first vector.

[0102] The present application also provides a character recognition device. Figure 3 , which shows a structural schematic diagram of a character recognition device provided by an embodiment of the present application, which may include:

[0103] A pre-generation module 31 is used to pre-acquire a long-tail content library and generate a long-tail content vector library according to long-tail characters in the long-tail content library;

[0104] The recognition and cutting module 32 is used to perform layout recognition on the image to be recognized and perform character cutting on the layout recognition result;

[0105] The character recognition module 33 is used to recognize the character segmentation result by using the OCR recognition engine to obtain a first character recognition result and the confidence of each recognized character in the first character recognition result;

[0106] A conversion acquisition module 34, configured to convert the recognized characters whose confidence level is lower than a threshold into a first vector, obtain a second vector corresponding to the first vector from a long-tail content vector library, and convert the second vector into a recalled character;

[0107] The combination restoration module 35 is used to replace the recognized characters whose confidence is lower than the threshold with the recalled characters, and to restore the recognized characters whose confidence is not lower than the threshold by combining them with the recalled characters to obtain a second character recognition result.

[0108] In a character recognition device provided in an embodiment of the present application, the pre-generation module 31 may include:

[0109] A first conversion unit, used to convert each long-tail character in the long-tail content library into a corresponding picture;

[0110] A first encoding unit, used to encode the image corresponding to each long-tail character to generate a corresponding vector;

[0111] The storage unit is used to store the generated vectors in a database to obtain a long-tail content vector library.

[0112] In the character recognition device provided in the embodiment of the present application, the pre-generation module 31 may further include:

[0113] The second conversion unit is used to convert each long-tail character in the long-tail content library into a long-tail character in multiple font styles before converting each long-tail character in the long-tail content library into a corresponding picture.

[0114] In the character recognition device provided in the embodiment of the present application, the pre-generation module 31 may further include:

[0115] The third conversion unit is used to convert the added long-tail characters into corresponding pictures when determining the added long-tail characters in the long-tail content library, encode the pictures corresponding to the added long-tail characters, and store the corresponding vectors generated by the encoding in the long-tail content vector library.

[0116] In the character recognition device provided in the embodiment of the present application, the pre-generation module 31 may further include:

[0117] The fourth conversion unit is used to convert the added long-tail characters into long-tail characters of multiple font styles before generating corresponding pictures for the added long-tail characters.

[0118] In a character recognition device provided in an embodiment of the present application, the conversion acquisition module 34 may include:

[0119] A calculation unit, used to calculate the similarity between each vector in the long-tail content vector library and the first vector;

[0120] The acquisition unit is used to acquire the vector corresponding to the maximum similarity from each vector in the long-tail content vector library, and determine the vector corresponding to the maximum similarity as the second vector corresponding to the first vector.

[0121] In a character recognition device provided in an embodiment of the present application, a computing unit may include:

[0122] The calculation subunit is used to calculate the cosine similarity between each vector in the long-tail content vector library and the first vector.

[0123] The present application also provides an OCR system. Figure 4 , which shows a schematic diagram of the structure of an OCR system provided in an embodiment of the present application, which may include:

[0124] A memory 41, used for storing computer programs;

[0125] The processor 42, when used to execute the computer program stored in the memory 41, can implement the following steps:

[0126] A long-tail content library is obtained in advance, and a long-tail content vector library is generated according to the long-tail characters in the long-tail content library; layout recognition is performed on the image to be recognized, and character segmentation is performed on the layout recognition result; the character segmentation result is recognized by using an OCR recognition engine to obtain a first character recognition result and the confidence of each recognized character in the first character recognition result; the recognized characters whose confidence is lower than a threshold are converted into a first vector, a second vector corresponding to the first vector is obtained from the long-tail content vector library, and the second vector is converted into a recalled character; the recognized characters whose confidence is lower than a threshold are replaced by the recalled character, and the recognized characters whose confidence is not lower than the threshold are combined and restored with the recalled character to obtain a second character recognition result.

[0127] The present application also provides a readable storage medium, in which a computer program is stored. When the computer program is executed by a processor, the following steps can be implemented:

[0128] A long-tail content library is obtained in advance, and a long-tail content vector library is generated according to the long-tail characters in the long-tail content library; layout recognition is performed on the image to be recognized, and character segmentation is performed on the layout recognition result; the character segmentation result is recognized by using an OCR recognition engine to obtain a first character recognition result and the confidence of each recognized character in the first character recognition result; the recognized characters whose confidence is lower than a threshold are converted into a first vector, a second vector corresponding to the first vector is obtained from the long-tail content vector library, and the second vector is converted into a recalled character; the recognized characters whose confidence is lower than a threshold are replaced by the recalled character, and the recognized characters whose confidence is not lower than the threshold are combined and restored with the recalled character to obtain a second character recognition result.

[0129] The description of the relevant parts of a character recognition device, an OCR system, and a readable storage medium provided in an embodiment of the present application can be found in the detailed description of the corresponding parts of a character recognition method provided in an embodiment of the present application, and will not be repeated here.

[0130] It should be noted that the logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing logical functions, and can be specifically implemented in any readable storage medium for use by an instruction execution system, device or equipment (such as a computer-based system, a system including a processor, or other system that can fetch instructions from an instruction execution system, device or equipment and execute instructions), or in combination with these instruction execution systems, devices or equipment. For the purpose of this specification, "readable storage medium" can be any device that can contain, store, communicate, propagate or transmit a program for use by an instruction execution system, device or equipment, or in combination with these instruction execution systems, devices or equipment. More specific examples of readable storage media (a non-exhaustive list) include the following: an electrical connection portion with one or more wirings (electronic device), a portable computer disk box (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable and programmable read-only memory (EPROM or flash memory), an optical fiber device, and a portable compact disk read-only memory (CDROM). In addition, the readable storage medium may even be paper or other suitable medium on which the program is printed, since the program can be obtained electronically, for example, by optically scanning the paper or other medium and then editing, interpreting or processing in other suitable ways if necessary, and then stored in a computer memory.

[0131] It should be understood that the various parts of the present application can be implemented by hardware, software, firmware or a combination thereof. In the above-mentioned embodiments, multiple steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented by hardware, as in another embodiment, it can be implemented by any one of the following technologies known in the art or their combination: a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0132] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present application. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner.

[0133] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" and "second" may explicitly or implicitly include at least one of the features. In the description of this application, the meaning of "plurality" is at least two, such as two, three, etc., unless otherwise clearly and specifically defined.

[0134] In this application, unless otherwise clearly specified and limited, the terms "installed", "connected", "connected", "fixed" and the like should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection or an indirect connection through an intermediate medium, it can be the internal connection of two elements or the interaction relationship between two elements, unless otherwise clearly defined. For ordinary technicians in this field, the specific meanings of the above terms in this application can be understood according to specific circumstances.

[0135] Although the embodiments of the present application have been shown and described above, it can be understood that the above embodiments are exemplary and cannot be understood as limitations on the present application. Ordinary technicians in this field can change, modify, replace and modify the above embodiments within the scope of the present application.

Claims

1. A character recognition method, characterized in that: include: Pre-acquire a long-tail content library, and generate a long-tail content vector library according to the long-tail characters in the long-tail content library; Perform layout recognition on the image to be recognized, and perform character segmentation on the layout recognition result; Recognize the character segmentation result by using an OCR recognition engine to obtain a first character recognition result and a confidence level of each recognized character in the first character recognition result; Converting the recognized characters whose confidence is lower than a threshold into a first vector, acquiring a second vector corresponding to the first vector from the long-tail content vector library, and converting the second vector into a recalled character; The recalled characters are used to replace the recognized characters whose confidence is lower than the threshold, and the recognized characters whose confidence is not lower than the threshold are combined with the recalled characters to restore and obtain a second character recognition result.

2. The character recognition method according to claim 1, characterized in that: Generating a long-tail content vector library according to the long-tail characters in the long-tail content library includes: Convert each of the long-tail characters in the long-tail content library into a corresponding picture; Encode the pictures corresponding to the long-tail characters to generate corresponding vectors; The generated vectors are stored in a database to obtain the long-tail content vector library.

3. The character recognition method according to claim 2, characterized in that: Before converting each of the long-tail characters in the long-tail content library into a corresponding picture, the method further includes: Each of the long-tail characters is converted into long-tail characters in multiple font styles.

4. The character recognition method according to claim 2, characterized in that: Also includes: When the long-tail characters added in the long-tail content library are determined, the added long-tail characters are converted into corresponding pictures, the pictures corresponding to the added long-tail characters are encoded, and the corresponding vectors generated by the encoding are stored in the long-tail content vector library.

5. The character recognition method according to claim 4, characterized in that: Before generating corresponding pictures from the added long-tail characters, the method further includes: The added long-tail characters are converted into long-tail characters in multiple font styles.

6. The character recognition method according to any one of claims 1 to 5, characterized in that: Acquiring a second vector corresponding to the first vector from the long-tail content vector library includes: Calculating the similarity between each vector in the long-tail content vector library and the first vector; A vector corresponding to the maximum similarity is obtained from each vector in the long-tail content vector library, and the vector corresponding to the maximum similarity is determined as a second vector corresponding to the first vector.

7. The character recognition method according to claim 6, characterized in that: Calculating the similarity between each vector in the long-tail content vector library and the first vector includes: The cosine similarity between each vector in the long-tail content vector library and the first vector is calculated.

8. A character recognition device, characterized in that: include: A pre-generation module, used to pre-acquire a long-tail content library and generate a long-tail content vector library according to the long-tail characters in the long-tail content library; The recognition and cutting module is used to perform layout recognition on the image to be recognized and perform character cutting on the layout recognition result; A character recognition module, used to recognize the character segmentation result by using an OCR recognition engine, and obtain a first character recognition result and a confidence level of each recognized character in the first character recognition result; a conversion acquisition module, configured to convert the recognized characters whose confidence level is lower than a threshold into a first vector, obtain a second vector corresponding to the first vector from the long-tail content vector library, and convert the second vector into a recalled character; The combination restoration module is used to use the recalled characters to replace the recognized characters whose confidence is lower than the threshold, and to restore the recognized characters whose confidence is not lower than the threshold by combining them with the recalled characters to obtain a second character recognition result.

9. An OCR system, characterized in that: include: Memory for storing computer programs; A processor, configured to implement the steps of the character recognition method as claimed in any one of claims 1 to 7 when executing the computer program.

10. A readable storage medium, characterized in that: The readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the character recognition method according to any one of claims 1 to 7 are implemented.