Text recognition method
By improving the CRNN structure and introducing CTC Loss loss function and intensive loss function of editing distance, the problem of slow convergence speed and difficulty in dealing with single and double lines of text in text recognition is solved, and a more efficient text recognition effect is achieved.
Patent Information
- Application Number
- CN202411633122.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-11-15
- Publication Date
- 2025-02-21
AI Technical Summary
In text recognition, traditional CRNN models have problems such as slow convergence speed, limited recognition effect, and difficult to deal with single and even text.
By establishing an improved CRNN structure, introducing a CTC Loss loss function for editing distance, and combining dense loss functions for supervision, a two-row output sequence branch and a single-row output sequence branch are constructed to identify single-line and double-line text at the same time.
It significantly improves the convergence speed and recognition accuracy of the model, simplifies the processing process, and improves the applicability and recognition efficiency of the model.
Smart Images
Figure CN119495089A_ABST
Abstract
Description
Technical Field
[0001] The invention belongs to the technical field of text recognition and relates to a text recognition method. Background Art
[0002] In the field of text recognition, CRNN (Convolutional Recurrent Neural Network) has become a popular algorithm in recent years, specifically for end-to-end single-line text recognition. The CRNN model consists of three core modules: first, CNN (Convolutional Neural Network) is responsible for extracting image features; then, RNN (Recurrent Neural Network) is used to process the extracted feature sequence; finally, CTC Loss (Connected Temporal Classification Loss Function) optimizes the match between the output sequence and the target sequence. CRNN can process text sequences of any length, its model is lightweight and has excellent recognition effect, so it is widely used in various image text recognition tasks.
[0003] In traditional CTC Loss, the input text image is processed by CNN and RNN to extract feature sequences, and the output sequence dimension is (T, class_nums), where T represents the length of the output sequence or the number of nodes, and class_nums represents the number of character categories. Based on this output, the model can combine multiple paths, each of which has a corresponding probability. The core of CTCLoss is to maximize the "probability sum of the correct path / probability sum of all paths". Among them, the "correct path" refers to the path whose output result after post-processing is completely consistent with the label (that is, the path with an edit distance of 0); the "wrong path" refers to the path that is not completely consistent with the label path, that is, the path with an edit distance greater than 0. Traditional CTC Loss has a significant flaw, that is, the loss function gives the same penalty and optimization to the path with only one wrong character and the completely wrong path, which makes the iteration process of the model parameters not smooth enough, see Figure 1 shown.
[0004] Usually in CRNN training, only CTC loss is used to calculate the loss and guide back propagation to optimize model parameters. The output of CRNN is a feature map with a width of w=nw=nw=n and a height of 1. CTC loss is more inclined to a global supervision method, which supervises the entire feature map as a whole. This global supervision method may lead to slower convergence of the model and limited effect.
[0005] The traditional CRNN model is only suitable for single-line text recognition. Figure 2 However, in some application scenarios, the input may contain single-line and double-line text. If the double-line text is forcibly cut into a single line for recognition, not only will an extra step be added, but the stability of the cutting algorithm will be difficult to guarantee. Summary of the invention
[0006] In order to solve the above-mentioned problems existing in the current technology, the present invention provides a text recognition method, which significantly improves the convergence speed and recognition accuracy of the model, can simultaneously recognize single-line and double-line text versions, simplifies the processing flow, and improves the applicability and recognition efficiency of the model.
[0007] The technical solution adopted by the present invention is:
[0008] A text recognition method, the specific steps of which are as follows:
[0009] S1, establishing a text recognition model based on an improved CRNN structure, wherein the output of the improved CRNN structure includes a double-line output sequence branch and a single-line output sequence branch;
[0010] S2, input pictures to train the text recognition model. After a single-line picture is input into the text recognition model, a single-line output sequence branch is trained. After a double-line picture is input into the text recognition model, a double-line output sequence branch is trained. The loss function is the CTC Loss with edit distance introduced as the loss function.
[0011] S3, with the recognition picture input into the text recognition model, recognizes both single-line text and double-line text.
[0012] Furthermore, the output of the improved CRNN structure in step S1 also includes a single-row / double-row classification branch for detecting, analyzing and classifying the feature graph, and performing corresponding labeling.
[0013] Furthermore, the CTC Loss function in step S2 is:
[0014]
[0015] The calculation method of the editing coefficient is:
[0016]
[0017] Here, d represents the edit distance of the path, len represents the length of the label characters, and min(len,d) ensures that the loss calculation is more reasonable by constraining the maximum length of the edit distance so that it does not exceed the length of the label characters.
[0018] Furthermore, in step S2, the CTC Loss function is combined with a dense loss function for joint supervision.
[0019] Furthermore, the improved CRNN structure extracts image features through a convolutional neural network.
[0020] Furthermore, the improved CRNN structure converts the text in the image into a character sequence through a bidirectional LSTM.
[0021] Furthermore, the feature map in the double-row output sequence branch is first split up and down, and then spliced left and right into a single-row feature map.
[0022] Beneficial effects of the present invention:
[0023] 1. By introducing the edit distance, the CTC Loss loss function is improved, making the optimization process of the loss function smoother. This not only significantly improves the training convergence speed, but also improves the training accuracy.
[0024] 2. By improving the CRNN structure, both single-line text recognition and double-line text recognition can be achieved.
[0025] 3. By introducing a dense loss function, the shortcomings of CTC loss are supplemented, so the labeling rules remain consistent, thereby improving the overall accuracy and significantly accelerating the convergence speed. BRIEF DESCRIPTION OF THE DRAWINGS
[0026] Figure 1 Schematic diagram of the path of the traditional CTC Loss of the present invention.
[0027] Figure 2 Schematic diagram of the traditional CRNN structure of the present invention.
[0028] Figure 3 It is a schematic diagram of the improved CRNN structure of the present invention.
[0029] Figure 4 It is a schematic diagram of the present invention for single-line text recognition.
[0030] Figure 5 It is a schematic diagram of the present invention for double-line text recognition.
[0031] Figure 6 Schematic diagram of the edit distance of the present invention.
[0032] Figure 7 Schematic diagram of dense supervision of the present invention. DETAILED DESCRIPTION
[0033] The present invention is further described below in conjunction with specific embodiments, but the present invention is not limited to these specific embodiments. Those skilled in the art should recognize that the present invention covers all possible alternatives, improvements and equivalents within the scope of the claims.
[0034] In the description of the present invention, it should be understood that the terms "center", "longitudinal", "lateral", "length", "width", "thickness", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "clockwise", "counterclockwise" and the like indicate positions or positional relationships based on the positions or positional relationships shown in the accompanying drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as limiting the present invention. In addition, the terms "first" and "second" are only used for descriptive purposes, and cannot be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Therefore, the features defined as "first" and "second" may explicitly or implicitly include one or more of the features. In the description of the present invention, unless otherwise specified, the meaning of "multiple" is two or more, unless otherwise clearly defined.
[0035] In the present invention, unless otherwise clearly specified and limited, the terms "installed", "connected", "connected", "fixed" and the like should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection, or it can be an indirect connection through an intermediate medium, or it can be the internal communication of two components. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.
[0036] In the present invention, unless otherwise clearly specified and limited, a first feature being "above" or "below" a second feature may include that the first and second features are in direct contact, or may include that the first and second features are not in direct contact but are in contact through another feature between them. Moreover, a first feature being "above", "above" and "above" a second feature includes that the first feature is directly above and obliquely above the second feature, or simply indicates that the first feature is higher in level than the second feature. A first feature being "below", "below" and "below" a second feature includes that the first feature is directly below and obliquely below the second feature, or simply indicates that the first feature is lower in level than the second feature.
[0037] This embodiment provides a text recognition method, and the specific steps are as follows:
[0038] S1, establish a text recognition model based on the improved CRNN structure, see Figure 3 As shown, the output of the improved CRNN structure includes a double-row output sequence branch and a single-row output sequence branch; the output of the improved CRNN structure also includes a single-row / double-row classification branch for detecting, analyzing and classifying feature maps, and performing corresponding labeling.
[0039] The improved CRNN structure extracts image features through convolutional neural networks.
[0040] The improved CRNN structure converts the text in the image into a character sequence through a bidirectional LSTM.
[0041] The feature map in the double-row output sequence branch is first split up and down, and then spliced left and right into a single-row feature map.
[0042] The improved CRNN structure is used to recognize both single-line and double-line characters at the same time. This improvement can handle the above problems more concisely, without the need for additional cutting and splicing steps, and without the need to determine in advance whether the input image is single-line or double-line, thereby improving the accuracy and efficiency of recognition.
[0043] S2, input pictures to train the text recognition model. After a single line of pictures is input into the text recognition model, a single line output sequence branch is trained. Figure 4 As shown, the double-line image is input into the text recognition model and then the double-line output sequence branch is trained, see Figure 5 As shown in the figure, the loss function is the CTC Loss that introduces the edit distance as the loss function; the CTC Loss loss function is:
[0044]
[0045] The calculation method of the editing coefficient is:
[0046]
[0047] Here, d represents the edit distance of the path, len represents the length of the label characters, and min(len,d) constrains the maximum length of the edit distance so that it does not exceed the length of the label characters, thereby ensuring that the loss calculation is more reasonable. By introducing the edit distance, see Figure 6 As shown in the figure, the model can weight different paths according to the degree of error of the path, so that the optimization process of the loss function is smoother and the training speed can be significantly faster.
[0048] The CTC Loss loss function is combined with a dense loss function for joint supervision. Dense supervision is to independently supervise each position in the feature map and optimize the performance of each feature point one by one. In contrast, global supervision methods such as CTC loss usually provide a unified supervision signal for the entire feature map or overall features. Although this helps capture global information, it may ignore the details of local features and converge slowly. Dense supervision, on the other hand, ensures that the model adjusts the weight of each local feature more finely during the learning process, thereby improving the overall accuracy and significantly speeding up the convergence speed by independently supervising each pixel or feature vector in the feature map. Figure 7As shown in Figure 1, the schematic diagram of dense supervision provides a label for each result in the feature map with height 1. Since the dense loss function is not used independently and is intended to supplement the shortcomings of CTC loss, the labeling rules remain consistent.
[0049] exist Figure 7 In the example, the center position of each character needs to be predicted correctly, such as "bed", "front", "bright", "moon", and "light". There needs to be at least one clear "_" separator category between every two characters, or it is called the background category. For positions with clear labels, such as "moon", the supervision goal of this position is to maximize the probability of "moon", and other clear labels (such as "front" / "_", etc.) are also supervised according to this rule. The "*" part in the figure indicates that this position can be fuzzy matched, and the specific rules are based on the labels on both sides of the "*". For example, if there is "bed" on the left side of the "*" and "_" on the right side, then the supervision requirement for the "*" is to maximize the sum of the probability values of "bed" and "_", without paying attention to the individual probabilities of "bed" or "_".
[0050] When using CTC loss for training, it is only necessary to label the category for the entire text, without the need to accurately label the position of each character. However, if dense supervision is introduced, the position of each character needs to be indicated at the same time, which will obviously increase the complexity of the labeling work. However, since the training data in OCR recognition is usually generated through synthesis, the position of the characters is known during the data synthesis process. Therefore, "dense supervision + CTC supervision" can be used for the synthetic data part at the same time, and only "CTC supervision" can be used for the data of the real scene. Doing so will not increase the workload of labeling, and at the same time can make full use of the known information of the synthetic data.
[0051] S3, with the recognition picture input into the text recognition model, recognizes both single-line text and double-line text.
[0052] The present invention improves the CTC Loss loss function by introducing the edit distance, thereby making the optimization process of the loss function smoother. Not only does it significantly improve the training convergence speed, but it also improves the training accuracy to a certain extent. By improving the CRNN structure, both single-line text recognition and double-line text recognition are possible. By introducing a dense loss function to supplement the shortcomings of CTC loss, the labeling rules remain consistent, thereby improving the overall accuracy and significantly accelerating the convergence speed.
Claims
1. A text recognition method, the specific steps of which are as follows: S1, establishing a text recognition model based on an improved CRNN structure, wherein the output of the improved CRNN structure includes a double-line output sequence branch and a single-line output sequence branch; S2, input pictures to train the text recognition model. After a single-line picture is input into the text recognition model, a single-line output sequence branch is trained. After a double-line picture is input into the text recognition model, a double-line output sequence branch is trained. The loss function is the CTC Loss with edit distance introduced as the loss function. S3, with the recognition picture input into the text recognition model, recognizes both single-line text and double-line text.
2. A text recognition method according to claim 1, characterized in that: The output of the improved CRNN structure described in step S1 also includes a single-row / double-row classification branch for detecting, analyzing and classifying the feature map, and performing corresponding labels.
3. A text recognition method according to claim 1, characterized in that: The CTC Loss function in step S2 is: The calculation method of the editing coefficient is: Here, d represents the edit distance of the path, len represents the length of the label characters, and min(len,d) constrains the maximum length of the edit distance.
4. A text recognition method according to claim 1, characterized in that: In step S2, the CTC Loss function is combined with a dense loss function for joint supervision.
5. A text recognition method according to claim 1, characterized in that: The improved CRNN structure extracts image features through convolutional neural networks.
6. A text recognition method according to claim 5, characterized in that: The improved CRNN structure converts the text in the image into a character sequence through a bidirectional LSTM.
7. A text recognition method according to claim 1, characterized in that: The feature map in the double-row output sequence branch is first split up and down, and then spliced left and right into a single-row feature map.