Long text line recognition method based on word segmentation
By using a word segmentation-based method to detect and recognize long lines of text, this method solves the problem that existing character recognition algorithms struggle to recognize long lines of text, achieving efficient and accurate identification of document information, and is applicable to various scenarios.
Patent Information
- Application Number
- CN202410897895.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-05
- Publication Date
- 2024-11-01
- Estimated Expiration
- 2044-07-05
AI Technical Summary
In existing OCR document recognition technologies, character recognition algorithms struggle to recognize long lines of text, leading to frequent missed recognitions. Meanwhile, large model methods have limited application scenarios on small embedded platforms.
A word-segmentation-based approach is adopted to perform text line detection, cropping, word detection, and character recognition on long text lines. The YOLOv8 algorithm is used for text line detection, and the CRNN algorithm is used for character recognition. The CRNN+RNN+CTC algorithm framework and MobileNetV3 feature enhancement are combined to achieve accurate recognition through word segmentation and concatenation.
It improves the recognition accuracy of document text lines of varying lengths, reduces the requirements for hardware facilities, is applicable to various scenarios, and achieves efficient document information verification.
Smart Images

Figure CN118429989B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of certificate recognition, and in particular to a long text line recognition method based on word segmentation. Background Art
[0002] The existing OCR document recognition technology mainly uses character recognition algorithms for recognition. However, since character recognition algorithms are not easy to recognize long text lines, in actual scenarios, when recognizing long text lines, it is easy to miss recognition. In addition, the method based on large models is usually used to solve the problem of long text recognition, but the use scenarios are limited and cannot be applied to small embedded platforms. Summary of the invention
[0003] One object of the present invention is to provide a method for recognizing long text lines based on word segmentation, which can segment long text lines according to words based on a word detection algorithm, recognize each part separately, then splice and verify them to obtain a final recognition result with a high recognition accuracy.
[0004] Other advantages and features of the present invention will become more apparent from the following detailed description and may be realized by means of the instrumentalities and combinations particularly pointed out in the appended claims.
[0005] According to one aspect of the present invention, a long text line recognition method based on word segmentation of the present invention, which can achieve the above-mentioned purpose and other purposes and advantages, comprises the following steps:
[0006] (A) Perform text line detection on the document image to be identified to obtain the text line detection result;
[0007] (B) cropping each text line detection result generated to obtain a text line image, determining whether word detection is required, performing word detection if it is required, and segmenting the image, and proceeding to the next step if it is not required;
[0008] (C) If the text line image is segmented, the recognition results are spliced and output. If the text line image is not segmented, character recognition is performed and the algorithm results are directly output.
[0009] According to one embodiment of the present invention, the step (B) includes the sub-steps of: determining the length of each of the detected text line detection results, scaling the height of the text line image and scaling the width in proportion to the text line image size used in the character recognition algorithm crnn training, and determining whether the text line image needs to be segmented and the number of segments as follows: (a) when the width of the original text line image after scaling is less than or equal to a preset threshold value L, the text line image does not need to be segmented; (b) when the width of the original text line image after scaling is greater than the preset threshold value L, the text line image needs to be segmented, and the formula for the number of segments N is: , where math.ceil represents rounding N up when the result has a remainder, and W1 is the width of the original text line image after being scaled.
[0010] According to one embodiment of the present invention, step (B) includes sub-steps: performing word detection, finding all coordinate points that can be segmented, ensuring that the characters are not cut, and performing word detection on the text line image. There are two types of targets, namely word class and character class.
[0011] According to one embodiment of the present invention, the step (B) includes the sub-steps of: segmenting the text line image to be segmented, looping the coordinates of each word, judging whether the width of the horizontal coordinate of the starting position and the horizontal coordinate of the lower right point of the current word exceeds the preset width of the training image size of crnn, if the width of the lower right horizontal coordinate of the previous word and the horizontal coordinate of the starting position is less than the preset width, and the width of the lower right horizontal coordinate of the current word and the horizontal coordinate of the starting position is greater than or equal to the preset width, obtaining the first segmented horizontal coordinate, segmenting the text line image, and setting the starting position horizontal coordinate to the lower right horizontal coordinate value of the current word, continuing to loop the word coordinates, obtaining subsequent segmentation coordinate positions, and continuing to segment the remaining text line images.
[0012] According to one embodiment of the present invention, the step (C) includes the sub-steps of: performing character recognition on the text line image obtained in step (B) without word detection, the character recognition algorithm uses the crnn algorithm, the algorithm framework is CRNN+RNN+CTC, the algorithm backbone is MobileNetV3, feature enhancement uses rnn, result prediction uses CTC, and the recognition result of the text line image is processed.
[0013] According to an embodiment of the present invention, the step (C) includes the sub-steps of: for the segmented text line images, sequentially connecting the recognition results with spaces to form a final recognition result.
[0014] According to one embodiment of the present invention, the step (A) includes sub-steps: the text line detection algorithm adopts the yolov8 algorithm, the document image I is feature encoded to obtain enhanced image features, the image features and the annotated text line coordinate information are input into the loss function for loss calculation during training, the loss calculation result is reversely optimized, the loss function is removed when predicting the text line coordinate position, and the text line result prediction is performed.
[0015] According to an embodiment of the present invention, the step (A) includes sub-steps: the feature extraction formula is I1 =encode(I), where encode is encoding.
[0016] According to one embodiment of the present invention, step (A) includes sub-steps: the formula of the loss function is: ,in, is the coordinate box regression loss, is the target horizontal coordinate loss, is the category loss function, is the target classification loss.
[0017] According to one embodiment of the present invention, step (A) includes sub-steps: using the SIOU loss function, the formula is , , where B is the predicted coordinate frame, The coordinate box for the label. is the intersection of two coordinate frames, is the union of two coordinate frames, |.| is the absolute value, is the angle loss between the predicted label and the annotated label, is the shape loss, is the target horizontal coordinate loss, and the distance loss is calculated for the width of the target result and the predicted result coordinate box, where (x1, y1) and (x2, y2) are the upper left and lower right vertices of the text line coordinate box predicted by the algorithm, and (x1', y1') and (x2', y2') are the upper left and lower right vertices of the real coordinate box of the labeled text line. ,in, represents multiplication, is the category loss function, which is used to determine the target category. The cross entropy loss function is used, and the formula is , n is the number of detection targets, is the corresponding label, Represents the category confidence, i is the i-th image, 0≤i≤n, where, For the target classification loss, the loss of the target is calculated. The loss result here is calculated using the cross entropy loss function.
[0018] The beneficial effects of the present invention are: there is no need to use a large model to identify long text lines, which saves time costs and has low requirements for hardware facilities. It is widely applicable to document text lines of varying lengths, and has a high accuracy in content recognition of long and short text lines, creating good algorithm conditions for efficient verification of passenger identity information. BRIEF DESCRIPTION OF THE DRAWINGS
[0019] Figure 1 The present invention is a flowchart of a method for recognizing long text lines based on word segmentation according to an embodiment of the present invention.
[0020] Figure 2 It is an algorithm flow chart of the long text line recognition method based on word segmentation according to the above embodiment of the present invention.
[0021] Figure 3 Schematic diagram of the loss function of the long text line recognition method based on word segmentation according to the above embodiment of the present invention.
[0022] Figure 4 It is a schematic diagram of text line detection results of the long text line recognition method based on word segmentation according to the above embodiment of the present invention.
[0023] Figure 5 It is a schematic diagram of a text line image after word detection in the long text line recognition method based on word segmentation according to the above embodiment of the present invention.
[0024] Figure 6 The method for recognizing long text lines based on word segmentation of the above embodiment of the present invention is Figure 5 Schematic diagram of the text line image after segmentation. DETAILED DESCRIPTION
[0025] The following description is used to disclose the present invention so that those skilled in the art can implement the present invention. The preferred embodiments described below are only examples, and those skilled in the art can think of other obvious variations. The basic principles of the present invention defined in the following description can be applied to other embodiments, variations, improvements, equivalents, and other technical solutions that do not deviate from the spirit and scope of the present invention.
[0026] Those skilled in the art should understand that, in the disclosure of the present invention, the terms "longitudinal", "lateral", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside" and the like indicating the orientation or position relationship are based on the orientation or position relationship shown in the drawings, which are only for the convenience of describing the present invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operate in a specific orientation. Therefore, the above terms should not be understood as limiting the present invention.
[0027] It is understood that the term "one" should be understood as "at least one" or "one or more", that is, in one embodiment, the number of an element can be one, while in another embodiment, the number of the element can be multiple, and the term "one" cannot be understood as a limitation on the number. In addition, in the description of the present invention, unless otherwise specified, the meaning of "multiple" is two or more than two.
[0028] In the description of the present invention, it should be understood that "first", "second", etc. are only used for descriptive purposes and cannot be understood as indicating or implying relative importance. In the description of the present invention, it should be noted that, unless otherwise clearly specified and limited, "connected" and "connected" should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection or an indirect connection through a medium. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.
[0029] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples, without contradiction.
[0030] like Figures 1 to 2 As shown, a long text line recognition method based on word segmentation according to an embodiment of the present invention is disclosed. The long text line recognition method based on word segmentation is applied to the OCR document recognition scenario, which can overcome the defects of character recognition algorithms and large model methods, improve the recognition accuracy of text lines of different lengths, and contribute to the accuracy of document recognition. The long text line recognition method based on word segmentation includes the following:
[0031] Step 1: Perform text line detection on the document image to be identified. In detail, the text line detection algorithm can be but not limited to the yolov8 algorithm. The steps of the yolov8 algorithm process are as follows: feature encoding of the document image I to obtain enhanced image features, and during training, the image features and the annotated text line coordinate information are sent to the loss function for loss calculation. The loss calculation result is used to reversely optimize the algorithm effect, and the loss function is removed when predicting the text line coordinate position to predict the text line result. The feature extraction formula is I1 = encode (I), where encode is the code, and the loss function formula is ,in, is the coordinate box regression loss,
[0032] Preferably, the present invention adopts the SIOU loss function, and the formula of the SIOU loss function is: , Among them, B is the predicted coordinate frame, The coordinate box for the label. is the intersection of two coordinate frames, is the union of two coordinate frames, |.| is the absolute value, is the angle loss between the predicted label and the annotated label, is the shape loss, The target horizontal coordinate loss is calculated by calculating the distance loss between the target result and the predicted result coordinate frame width. like Figure 3 As shown, (x1, y1) and (x2, y2) are the upper left and lower right vertices of the text line coordinate box (i.e., the prediction box) predicted by the algorithm, and (x1', y1') and (x2', y2') are the upper left and lower right vertices of the real coordinate box (i.e., the target box) of the labeled text line. ,in, represents multiplication, is the category loss function, which is used to determine the target category. The cross entropy loss function is used, and the formula is , n is the number of detection targets, is the corresponding label, Represents the category confidence, i is the i-th image, 0≤i≤n, where, The target classification loss is used to measure the accuracy of whether the predicted box contains the target object. The target classification loss prediction result and the actual label result are used to calculate the target loss. The loss result calculation here uses the cross entropy loss function to obtain the text line detection result. Figure 4 As shown;
[0033] Step 2: Crop each text line detection result generated in step 1 to obtain a text line image, determine whether word detection is needed, if it is necessary, perform word detection and segment the image, if it is not necessary, proceed to step 3. In detail, the length of the detected text line detection results is judged one by one. Considering that the character recognition algorithm has a good recognition effect on short and medium text lines, it is easy to cause character omission when performing character recognition on text lines with a long length and too many characters. For example, the training sample size of the character recognition algorithm crnn is 32×320, 32 is the height of the image input to the algorithm, and 320 is the height of the image input to the algorithm. is the width of the image input to the algorithm. When the image height is 32, if the image width exceeds 320, it is regarded as a longer text line. Therefore, a segmentation step is added to this step of the present invention. This step is divided into three sub-steps. The details are as follows: (1) Determine whether the text line image needs to be segmented. Specifically, the height of the text line image is scaled and the width is scaled proportionally according to the text line image size used in the character recognition algorithm crnn training. Preferably, the width and height of the text line image trained by the crnn algorithm are preset to 320 and 32 respectively. The width of the original image size is W, and the height is H. The scaling ratio S formula is: , where the original text line image is scaled to have a width and height of W1 and 32 respectively. The method for determining whether the text line image needs to be segmented and the number of segments is as follows: (a) When W1 is less than or equal to 320, the text line image does not need to be segmented, and the text line image is sent to the crnn algorithm in step 3 for character recognition; (b) When W1 is greater than 320, the text line image needs to be segmented, and the formula for the number of segments N is: , where math.ceil means that when the result has a remainder, N is rounded up;
[0034] (2) Perform word detection, find all the coordinate points that can be segmented, ensure that the characters are not cut, and perform word detection on the text line image. The detection algorithm can be but is not limited to yolov8. There are two types of targets, namely word classes and character classes. Preferably, the character classes are uppercase English and punctuation marks. Take any text line as an example, and the word detection results obtained are as follows: Figure 5 As shown, it is a text line image after word detection, wherein the detected word classes include 5 and the detected character classes include 29;
[0035] (3) Segment the text line image that needs to be segmented. Specifically, loop through the coordinates of each word to determine whether the width of the horizontal coordinate of the starting position and the horizontal coordinate of the lower right point of the current word exceeds the training image size width of 320 of crnn. If the width of the lower right horizontal coordinate of the previous word and the horizontal coordinate of the starting position is less than 320, and the width of the lower right horizontal coordinate of the current word and the horizontal coordinate of the starting position is greater than or equal to 320, obtain the horizontal coordinate of the first segmentation, segment the text line image, and set the horizontal coordinate of the starting position to the lower right horizontal coordinate value of the current word. Continue to loop through the word coordinates to obtain the subsequent segmentation coordinate positions, and continue to segment the remaining text line images to obtain all the segmented text line images, such as Figure 6 As shown, for Figure 5 The text line image after segmentation;
[0036] Step 3: Perform character recognition on the text line image obtained in step 2 without word detection. Specifically, preferably, the character recognition algorithm uses the crnn algorithm, the algorithm framework is CRNN+RNN+CTC, the algorithm backbone is MobileNetV3, the feature enhancement uses rnn, the result prediction uses CTC, and the recognition result of the text line image is processed. If the text line image is segmented, the recognition result is spliced and output. If the text line image is not segmented, the algorithm result is directly output. The steps are as follows:
[0037] (a) Directly output the recognition result for the unsegmented text line image;
[0038] (b) For the segmented text line image, the recognition results are connected with spaces in sequence to form the final recognition result. For example, for the segmentation result of the text line image in step 2, the output recognition result after reorganization is: K***M MO**MEDM***D IBR**IM RA**A.
[0039] The long text line recognition method based on word segmentation of the present invention can instantly recognize or extract the character content of the captured image after the imaging device captures the image, with good immediacy, or directly receive the image and extract or recognize the character content, thereby realizing instant recognition of text lines of various lengths and outputting the recognized text content, which is beneficial to the document recognition needs of various verification scenarios.
Claims
1. A long text line recognition method based on word segmentation, characterized in that: The following steps are involved: (A) Perform text line detection on the document image to be recognized to obtain the text line detection result. The text line detection algorithm adopts the yolov8 algorithm to perform feature encoding on the document image I to obtain the enhanced image features. During training, the image features and the annotated text line coordinate information are sent to the loss function for loss calculation. The loss calculation result is used to reversely optimize the algorithm effect. When predicting the text line coordinate position, the loss function is removed to predict the text line result. The feature extraction formula is I1=encode(I), where encode is the code; the loss function formula is loss=loss box +loss wiou +loss cls +loss obj Among them, loss box is the coordinate frame regression loss, loss wiou is the target horizontal coordinate loss, loss cls is the category loss function, loss obj is the target classification loss; The step (A) includes sub-steps: the coordinate frame regression loss adopts the SIOU loss function, and the formula is Among them, B is the predicted coordinate frame, B GT is the coordinate box marked by the label, ∩ is the intersection of two coordinate boxes, ∪ is the intersection of two coordinate boxes, |.| is the absolute value, Δ is the angle loss between the predicted label and the marked label, Ω is the shape loss, loss wiou is the target horizontal coordinate loss, and the distance loss is calculated for the width of the target result and the predicted result coordinate box, where (x1, y1) and (x2, y2) are the upper left and lower right vertices of the text line coordinate box predicted by the algorithm, and (x1', y1') and (x2', y2') are the upper left and lower right vertices of the real coordinate box of the labeled text line. where · represents multiplication, loss cls is the category loss function, which is used to determine the target category. The cross entropy loss function is used, and the formula is n is the number of detected targets, y i is the corresponding label, x i Represents the category confidence, i is the i-th image, 0≤i≤n, where, loss obj The target classification loss is calculated to obtain the target loss. The loss result here is calculated using the cross entropy loss function. (B) cropping each text line detection result generated to obtain a text line image, determining whether word detection is required, performing word detection if determined to be required, and segmenting the image, and proceeding to the next step if determined not to be required; The step (B) includes the sub-steps of segmenting the text line image to be segmented, looping the coordinates of each word, judging whether the width of the horizontal coordinate of the starting position and the horizontal coordinate of the lower right point of the current word exceeds the preset width of the training image size of crnn, if the width of the lower right horizontal coordinate of the previous word and the horizontal coordinate of the starting position is less than the preset width, and the width of the lower right horizontal coordinate of the current word and the horizontal coordinate of the starting position is greater than or equal to the preset width, obtaining the horizontal coordinate of the first segmentation, segmenting the text line image, and setting the horizontal coordinate of the starting position to the lower right horizontal coordinate value of the current word, continuing to loop the word coordinates, obtaining subsequent segmentation coordinate positions, and continuing to segment the remaining text line images; (C) If the text line image is segmented, the recognition results are spliced and then output; if the text line image is not segmented, character recognition is performed and the algorithm results are directly output.
2. The long text line recognition method according to claim 1, characterized in that: The step (B) includes the sub-steps of determining the length of each of the detected text line detection results, scaling the height of the text line image and scaling the width in proportion to the text line image size used in the character recognition algorithm crnn training, and determining whether the text line image needs to be segmented and the number of segments as follows: (a) when the width of the original text line image after scaling is less than or equal to a preset threshold value L, the text line image does not need to be segmented; (b) when the width of the original text line image after scaling is greater than the preset threshold value L, the text line image needs to be segmented, and the formula for the number of segments N is, Wherein, math.ceil represents rounding N up when the result has a remainder, and W1 is the width of the original text line image after being scaled.
3. The long text line recognition method according to claim 2, characterized in that: The step (B) includes sub-steps: performing word detection, finding all coordinate points that can be segmented, ensuring that the characters are not cut, and performing word detection on the text line image. There are two types of targets, namely word class and character class.
4. The long text line recognition method according to claim 1, characterized in that: The step (C) includes the sub-steps of: performing character recognition on the text line image obtained in step (B) without word detection, using a CRNN algorithm for character recognition, the algorithm framework is CRNN+RNN+CTC, the algorithm backbone is MobileNetV3, RNN is used for feature enhancement, CTC is used for result prediction, and the recognition result of the text line image is processed.
5. The long text line recognition method according to claim 4, characterized in that: The step (C) includes the sub-steps of: for the segmented text line images, sequentially connecting the recognition results with spaces to form a final recognition result.
