Super-large-scale character detection and recognition method based on character-level features

Abstract

The invention discloses a super-large-scale character detection and recognition model training method and component in the technical field of computers.According to the super-large-scale character detection and recognition model training method and component, a character detection and recognition model can be obtained through training, the character detection and recognition model can detect the position and direction of each character in any image, and the super-large-scale character detection and recognition model training method and component are suitable for character detection and recognition. The character detection model can determine the distribution position of characters in an image according to the character detection and recognition task of the characters in the image, the character detection model can determine the distribution position of the characters in the image, and therefore finer-grained character-level character detection and recognition can be achieved, and therefore the character detection model can carry out character detection and extraction on template images and irregular character images; correspondingly, the invention provides a character image data generation method containing character-level labels, a super-large-scale character detection and recognition method and a super-large-scale character detection and recognition assembly based on character-level features, and also has the above technical effects.

Description

Technical Field

[0001] The present invention relates to a very large-scale text detection and recognition method, and in particular to a very large-scale text detection and recognition method based on character-level annotation data and a target detection framework. Background Art

[0002] Text detection and recognition is one of the important tasks in the field of computer vision. Most of the existing text detection and recognition methods are based on large-scale annotated data and deep network training to extract text features and then detect and recognize text. Therefore, model structure design and data annotation quality have an important impact on text detection and recognition performance. For text line recognition and end-to-end text detection and recognition tasks, the current mainstream methods need to rely on the object detection module for text positioning, while the bounding box regression metric of the existing object detection has the same value under certain different regression results, which has a great impact on the accuracy of text positioning. For single-word recognition tasks, most of the existing methods rely on the structural optimization of a single model to improve the accuracy. The accuracy improvement is limited in the Chinese calligraphy recognition task containing rich character shape changes and a large number of similar characters. In addition, the existing end-to-end text detection and recognition methods have poor text detection and recognition performance on text images containing arbitrary layouts, special fonts, and complex deformations. Some researchers have tried to use language knowledge to assist in improving text recognition accuracy, but this strategy will fail in text detection and recognition tasks containing a large number of special nouns outside the vocabulary (such as brand names, license plates, etc.). However, most existing scene text image datasets adopt a data processing strategy of manual shooting and manual annotation, and only provide word-level annotations. The data acquisition efficiency is low and the directionality is poor, and errors caused by manual operations may be introduced. Summary of the invention

[0003] To address the above problems, the present invention unifies text line recognition and end-to-end scene text recognition into character-level text detection and recognition tasks, which can effectively improve the accuracy and efficiency of text detection and recognition. The present invention specifically includes the following steps:

[0004] S1: Collect text labels, font libraries and background images of the target language scene in advance, and generate corresponding text image data according to preset text labels and related attributes. The generated data also includes text images, character-level annotations and word-level annotations.

[0005] S2: Select a mainstream object detection framework and use the bounding box regression metric MPDIoU and loss function based on the minimum point distance to train the text detection and recognition model. The specific calculation formulas are shown in equations (1) to (3).

[0006]

[0007] in express The coordinates of the upper left and lower right points, express The coordinates of the upper left and lower right points, and Respectively represent the square of the distance between the upper left point and the lower right point of the prediction box and the annotation box, w 2 and h 2 Represent the square of the input image width and height respectively.

[0008] S3: For text detection and recognition tasks in multilingual scenarios, since the number of character categories may be very large, which will lead to a decrease in model performance, multiple text detection and recognition sub-models can be trained according to language categories, and the recognition results can be screened after step-by-step recognition in the test phase.

[0009] S4: Input a test text image, use a dynamic input scaling module to adjust the test image to a resolution that meets the model input requirements, and input the text detection and recognition model to obtain the detection and recognition results. The calculation formula of the dynamic input scaling module is shown in equations (4)-(5).

[0010]

[0011] where h old and w old Indicates the original height and width of the input image, h new and w new Represents the height and width of the input image after dynamic scaling.

[0012] S5: Sort the text detection and recognition results according to row and column coordinates according to actual application needs, or set a clustering threshold based on character distance to group characters into words or text lines.

[0013] Compared with the prior art, the present invention has the following beneficial effects:

[0014] (1) Flexibility: The present invention can theoretically autonomously generate text images in any language scenario and cover all character categories of the target language. The frequency of occurrence of different characters in real scenarios often varies greatly. Therefore, the present invention is more flexible and efficient than the existing manual shooting and manual annotation data acquisition methods.

[0015] (2) Real-time performance: The present invention realizes end-to-end text recognition based on character-level features. Compared with the existing word-level text recognition method, it has clearer directionality, faster inference speed, and lower computing power consumption.

[0016] (3) High recognition accuracy: It has a high recognition rate for both natural scene text images and optical character images, with an average recognition accuracy of over 98%. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 It is the overall flow chart of the present invention.

[0018] Figure 2 It is the framework diagram of the testing phase of the present invention.

[0019] Figure 3 These are two output forms of the test results of the present invention. DETAILED DESCRIPTION

[0020] In order to improve the accuracy and interpretability of text detection and recognition models, the present invention proposes a method for automatically generating text detection and recognition datasets based on text labels and related attributes. The method requires the pre-collection of character labels, ttf fonts and background images of the target language, and can provide more specific character-level annotations for the training of text detection and recognition models. Based on this method, text images of any language in the world can be generated in theory. The mainstream object detection framework (such as the YOLO series) is used, and the bounding box regression metric method is replaced by the bounding box regression metric method MPDIoU based on the minimum point distance, so as to train a very large-scale text detection and recognition model based on character-level features.

[0021] The test phase of the model of the present invention first dynamically scales the input test image, selects the optimal resolution for the input test image, and satisfies the input requirements of the model. The test result includes the text label, position information and confidence level in the image. Based on the test result, the characters in the output result can be sorted by row and column coordinates according to the actual application needs, and the characters are clustered and combined into words or text lines according to the position.

Claims

1. A large-scale text detection and recognition model based on character-level features, which is trained on text image data containing character-level annotations and an object detection framework, and is characterized by Regression optimization is performed using a bounding box regression metric based on minimum point distance.

2. A large-scale text detection and recognition model based on character-level features as claimed in claim 1, characterized in that A dynamic scaling module is used to select the optimal resolution for the input test image to avoid possible information loss or information redundancy caused by the fixed scaling module of the target detection framework. The specific calculation formula is as follows.

3. A large-scale text detection and recognition model based on character-level features as claimed in claims 1 and 2, characterized in that It can provide character-level text detection and recognition results, including text labels and location information. It can sort the text detection and recognition results according to row and column coordinates according to actual application needs, or set clustering thresholds based on character distance to group characters into words or text lines.

Images