A line classification post-processing method for multi-line character OCR recognition results

Abstract

The present invention discloses a line classification post-processing method for multi-line character OCR recognition results, which takes the text bounding box information determined by the OCR engine recognition as input, classifies the unstructured characters into different lines by adopting the idea of ​​dimensionality reduction clustering, and has the high-precision line classification capability for tilted text with a high probability of character line classification failure, thereby structuring the OCR recognition results. The present invention can perform effective post-processing judgment and correction on the OCR engine recognition results, and improve the recognition result accuracy of tilted text.

Description

Technical Field

[0001] The invention belongs to the technical field of character recognition, and in particular relates to a line classification post-processing method for character OCR recognition results. Background Art

[0002] In the field of character recognition, general texts are structured and strictly follow the line arrangement criteria. OCR recognition results are generally output together by line. However, when the text is tilted, the OCR recognition results will have line separation errors. The arrangement of text lines in unstructured text is relatively random. Especially when the OCR recognition image is tilted, there will be problems such as recognizing the same line of characters as multiple lines of characters or recognizing different lines of characters in series. Problems such as the lack of regular line structure of text, complex background, random position in the image, and difficult font recognition all affect the line to which the OCR recognition result belongs, affecting the recognition accuracy and regularity. In the case of successful recognition of a single character, OCR recognition must also meet the requirement that the line arrangement of the output characters is regular, so as not to affect all the final recognition results. However, the existing OCR recognition engine has carried out a lot of pre-processing and recognition algorithm accuracy improvement work, but still lacks an efficient and accurate post-processing judgment and correction mechanism for the recognition results, resulting in the problem of different lines of characters being often arranged in the same line when recognizing tilted text. Therefore, in order to solve the above problems, it is necessary to perform post-processing of the character results of OCR recognition by row classification. Without relying on human intervention, the character arrangement judgment and correction of the OCR engine recognition output can be performed, and the text with incorrect character row output can be automatically classified and corrected to output the correct character results arranged in row and order. This is of great significance for the rapid and accurate recognition of parts with multi-line character encoding in the industrial field. It is an indispensable means to ensure the recognition accuracy and is the quality assurance of industrial intelligent and automated production.

[0003] Chinese patent publication CN111401371A discloses a text detection and recognition method, system and computer equipment. For tilted and rotated images to be recognized, a network training is performed by constructing a rotated image data set to predict the tilt angle, and then text recognition is performed on the tilt-corrected images to output the character results with correct row classification. However, it is still aimed at the pre-processing process in character recognition and does not have the ability to judge and correct the arrangement structure of the recognized character results. When the prediction angle is wrong and the recognized character result is incorrectly classified, new recognition errors will still be introduced.

[0004] The development of existing OCR technology mainly focuses on the algorithms of pre-processing and recognition process, and there is still a lack of research on post-processing of recognition result classification. Accurately recognizing characters does not mean that the arrangement structure of characters can be accurately understood, which is one of the keys for machines to understand text. Summary of the invention

[0005] In order to overcome the shortcomings of the prior art, the present invention provides a line classification post-processing method for multi-line character OCR recognition results, which takes the text bounding box information determined by the OCR engine recognition as input, classifies unstructured characters into different lines using the idea of ​​dimensionality reduction clustering, and has the ability to classify tilted text with high probability of character line classification failure, thereby structuring the OCR recognition results. The present invention can perform effective post-processing judgment and correction on the OCR engine recognition results, and improve the accuracy of the recognition results of tilted text.

[0006] The technical solution adopted by the present invention to solve the technical problem includes the following steps:

[0007] Step 1: Given the OCR recognition result {O = S 1 ,…,S i ,…,S m}, a total of m characters, get the original data of the bounding box of all characters; the bounding box data of the i-th, 1≤i≤m characters is expressed as: the coordinates of the upper left corner point P i (x i ,y i ), frame width l i and frame height h i ; The upper left corner of the OCR recognized image is the coordinate origin O(0,0);

[0008] Enter the total number of lines of text to be recognized Sum, record Line = 1;

[0009] Step 2: Traverse the y coordinates of all the characters in O, and take the point with the minimum y value as the initial point, which is recorded as the initial point P. 1 (x 1 ,y 1 );

[0010] Step 3: Take the unit vector in the X-axis direction as the initial vector A = (1,1); 1 is the starting point of the vector, divided by P 1 The remaining points P j ,2≤j≤m is the end point of the vector, construct other vectors B j =(x j -x 1 ,y j -y 1 ), we get the vector set E = {A,B 2 ,…,B j ,…,B m};

[0011] Step 4: Calculate the initial vector A and other vectors B according to formula (1) j The angle θ betweenj , reduce the original two-dimensional bounding box data to a one-dimensional vector angle data set G = {θ 2 ,…,θ j ,…,θ m};

[0012]

[0013] Step 5: Use formula (2) to perform nonlinear log normalization on the data set G and obtain the normalized result G′={θ′ 2 ,…,θ′ j ,…,θ′ m}.

[0014]

[0015] where θ max is the θ in the set G j Maximum value, Represents θ in the set G j The minimum value is rounded down;

[0016] Step 6: Use K-Means clustering algorithm to divide the similarity of normalized sample data G′ into k categories of clusters C = {C 1 ,…,C e ,…,C k}, randomly select k cluster centers: μ 1 ,…,μ e ,…,μ k , with the goal of minimizing the following loss function E:

[0017]

[0018] Step 6-1: Take k = 2, that is, use K-Means clustering to get two cluster results; let C = {C 1 ,C 2}, the cluster center is: μ 1 ,μ 2 ;

[0019] Step 6-2: Initialize the cluster e=1,2;Calculate each element θ′ in the sample data G′ j With each cluster center μ e Distance According to d GK The size of θ′ determines j The cluster label λ e , and then classify each element into the cluster label λ e The corresponding cluster, that is

[0020] Step 6-3: If the condition of loss function E is not met or not all rows are identified, recalculate the center point corresponding to each cluster If μ e ≠μ′ e , then the assignment updates μ e =μ′ e ; Otherwise, keep the current cluster center unchanged; repeat steps 6-2 and 6-3, and when the cluster centers no longer change, output the final two clusters

[0021] Step 7: The characters in the same row are classified into a set C e The internal elements have the highest similarity, and the calculated loss function E e Minimum, then take the minimum loss function E of each cluster min The corresponding cluster C e For similar direction vectors, obtain the corresponding vector end point coordinates, that is, output the same row result U e ;

[0022] Step 8: Assign the remaining characters as new input characters and subtract U from O e , let Line add 1;

[0023] Step 9: If Sum>Line+1 is satisfied, continue to perform the operations from step 2 to step 8 to classify the new line characters; if Sum=Line+1 is satisfied after iteration, stop iteration, identify and classify all Sum line characters: {U 1 ,U 2 ,…,U Sum}.

[0024] The beneficial effects of the present invention are as follows:

[0025] 1. The present invention formulates a line classification post-processing method for multi-line character OCR recognition results, and performs line classification correction on the characters output serially by the OCR engine recognition, thereby solving the problem of disordered lines in text OCR recognition results in tilted text with multiple lines of characters, and improving the accuracy of OCR recognition results by post-processing.

[0026] 2. The present invention has strong generalization ability and can be applied to any OCR engine to improve the accuracy of its recognition output results. The proposed method can be executed from any line of the text, solving the problem of positioning and selecting the first character when the text is tilted.

[0027] 3. The present invention has a high line classification accuracy for text image recognition and line division when the text image is rotated within 90 degrees, and has strong robustness. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] Figure 1Flow chart of the method of the present invention.

[0029] Figure 2 This is an example of reducing the dimensionality of text pictures and two-dimensional scattered point data input into one-dimensional angle data in an embodiment of the present invention.

[0030] Figure 3 This is an example diagram of the angle data clustering results according to an embodiment of the present invention.

[0031] Figure 4 This is the character row classification result after the first iteration of the embodiment of the present invention.

[0032] Figure 5 This is the final character row classification result of the embodiment of the present invention. DETAILED DESCRIPTION

[0033] The present invention is further described below in conjunction with the accompanying drawings and embodiments.

[0034] like Figure 1 As shown, a line classification post-processing method for multi-line character OCR recognition results includes the following steps:

[0035] Step 1: Given the OCR recognition result {O = S 1 ,…,S i ,…,S m}, a total of m characters, get the original data of the bounding box of all characters. The bounding box data of the i-th, 1≤i≤m characters is expressed as: the coordinates of the upper left corner point P i (x i ,y i ), frame width l i and frame height h i ; The upper left corner of the OCR recognized image is the coordinate origin O(0,0);

[0036] Enter the total number of lines of text to be recognized Sum, record Line = 1;

[0037] Step 2: Traverse the y coordinates of all the characters in O, and take the point corresponding to the minimum y value as the initial point, recorded as the initial point P 1 (x 1 ,y 1 ).

[0038] Step 3: In order to improve the generalization ability of the algorithm and reduce the interference of the chaotic arrangement structure of the two-dimensional point data in the tilted state of the text, the dimensionality reduction method of converting the two-dimensional discrete point data into one-dimensional vector data is adopted to reconstruct the structural relationship between all characters in the text. The unit vector in the X-axis direction is used as the initial vector A = (1,1); 1 is the starting point of the vector, divided by P 1 The remaining points P j,2≤j≤m is the end point of the vector, construct other vectors B j =(x j -x 1 ,y j -y 1 ), we get the vector set E = {A,B 2 ,…,B j ,…,B m};

[0039] Step 4: Calculate the initial vector A and other vectors B according to formula (1) j The angle θ between j , reduce the original two-dimensional bounding box data to a one-dimensional vector angle data set G = {θ 2 ,…,θ j ,…,θ m};

[0040]

[0041] Step 5: Normalize the data based on the characteristics of the text data. For characters in the same row, the angle between the above vectors is j Generally, the directions are highly similar and are all within a small range, while the vector angle θ of characters in different rows j Generally, there will be small or large values. Therefore, for a given data set G, in order to reduce the data dispersion, the data set G is nonlinearly log-normalized using formula (2), and the normalized result G′={θ′ 2 ,…,θ′ j ,…,θ′ m}.

[0042]

[0043] where θ max is the θ in the set G j Maximum value, Represents θ in the set G j The minimum value is rounded down;

[0044] Step 6: Use K-Means clustering algorithm to divide the similarity of normalized sample data G′ into k categories of clusters C = {C 1 ,…,C e ,…,C k}, randomly select k cluster centers: μ 1 ,…,μ e ,…,μ k , with the goal of minimizing the following loss function E:

[0045]

[0046] Step 6-1: Take k = 2, that is, use K-Means clustering to get two cluster results; let C = {C 1 ,C 2}, the cluster center is: μ 1 ,μ 2 ;

[0047] Step 6-2: Initialize the cluster e=1,2;Calculate each element θ′ in the sample data G′ j With each cluster center μ e Distance According to d GK The size of θ′ determines j The cluster label λ e , and then classify each element into the cluster label λ e The corresponding cluster, that is

[0048] Step 6-3: If the condition of loss function E is not met or not all rows are identified, recalculate the center point corresponding to each cluster If μ e ≠μ′ e , then the assignment updates μ e =μ′ e ; Otherwise, keep the current cluster center unchanged; repeat steps 6-2 and 6-3, and when the cluster centers no longer change, output the final two clusters

[0049] Step 7: The characters in the same row are classified into a set C e The internal elements have the highest similarity, and the calculated loss function E e Minimum, then take the minimum loss function E of each cluster min The corresponding cluster C e is a set of similar direction vectors, and the corresponding vector end point coordinates are obtained, that is, the output is the same row result U e ;

[0050] Step 8: Assign the remaining characters as new input characters and subtract U from O e , let Line add 1;

[0051] Step 9: If Sum>Line+1 is satisfied, continue to perform the operations from step 2 to step 8 to classify the new line characters; if Sum=Line+1 is satisfied after iteration, stop iteration, identify and classify all Sum line characters: {U 1 ,U 2 ,…,U Sum}. Specific embodiment:

[0053] 1. Given the OCR recognition result O = {S1 ,…,S i ,…,S m}, a total of m characters. Read the bounding box data of the i-th (1≤i≤m) character: the coordinates of the upper left corner point P i (x i ,y i ), frame width l i and frame height h i . OCR recognizes the upper left corner of the default image as the coordinate origin O(0,0). Enter the total number of lines of text to be recognized Sum = 4, and record Line = 1. Figure 2 As shown, OCR character recognition has 38 characters in total, and the characters are outputted in sequence as O = {S, 1, 0, 1, D, N, 4, 1, 2, P, 4, 5, 1, 7, 0, 9, 3, 0, 1, 1, 4, 7, 1, 2, 1, B, 3, 1, R, T, V, U, 3, 2, 8, 4, F, 2}. The OCR recognition result is outputted as {S10, 1DN412, P4517093011, 47121B31, RTVU32, 84F2} by line, and the line division result is 6 lines and the character line division is incorrect. In addition, it is impossible to determine the line to which the character belongs based on the OCR result.

[0054] 2. Traverse the y coordinates of all the characters, and take the point corresponding to the minimum y value as the initial point, recorded as the initial point P 1 (x 1 ,y 1 ). The initial point P in this example is 1 It is the first character 'S' of the first line.

[0055] 3. The unit vector in the X-axis direction is used as the initial vector A = (1,1). In order to accurately determine the characters in each line and define the cutoff condition for the last character in each line, P 1 is the starting point of the vector, divided by P 1 The remaining points P j (2≤j≤m) is the end point of the vector, and other vectors B are constructed. j =(x j -x 1 ,y j -y 1 ), construct a vector set E = {A, B 2 ,…,B j ,…,B m}.

[0056] 4. Calculate the initial vector A and other vectors B according to the following formula j The angle θ between j , reduce the original data to a one-dimensional data set G = {θ 2 ,…,θ j ,…,θ m}. The two-dimensional scattered point data is reduced to one-dimensional vector angle data, such as Figure 2 The angle between the vectors marked is shown.

[0057]

[0058] 5. Use the following formula to perform nonlinear log normalization on the data and obtain the normalized result G′={θ′ 2 ,…,θ′ j ,…,θ′ m}.

[0059]

[0060] where θ max is the θ in the set G j Maximum value, Represents θ in the set G j The minimum value is rounded down.

[0061] 6. Use K-Means clustering algorithm to divide the similarity of normalized sample data G′ into k categories of clusters C = {C 1 ,…,C e ,…,C k}. Randomly select k cluster centers: μ 1 ,…,μ e ,…,μ k The goal is to minimize the following loss function E.

[0062]

[0063] 6.1. In the present invention, k=2, that is, two types of cluster results are obtained through K-Means clustering. Let C={C 1 ,C 2}, the cluster center is: μ 1 ,μ 2 .

[0064] 6.2. Initialize the cluster (where e = 1, 2). Calculate each element θ′ in the sample data G′ j With each cluster center μ e Distance According to d GK The size of θ′ determines j The cluster label λ e , and then classify each element into the cluster label λ e The corresponding cluster, that is

[0065] 6.3. If the condition of loss function E is not met or all rows are not identified, recalculate the center point corresponding to each cluster If μ e ≠μ′ e , then the assignment updates μ e =μ′ e Otherwise, keep the current cluster center unchanged. Repeat steps 6.2 and 6.3. When the cluster centers no longer change, output the final two clusters. The clustering results of angle data are as follows Figure 3 shown.

[0066] 7. The character vectors in the same row have basically the same direction, and the classified set is C e The internal elements have the highest similarity, and the calculated loss function E e Minimum, then take the minimum loss function of each cluster (E min ) corresponds to the cluster C e For similar direction vectors, obtain the corresponding vector end point coordinates, that is, output the same row result U e The character row classification results after the first iteration of this example are as follows Figure 4 As shown, the first line of characters output is U 1 ={S10123011}.

[0067] 8. Assign the remaining characters as new input characters, that is, O = OU 1 , let Line = Line + 1. In this example, after the first update, O = {1,D,N,4,1,7,0,9,3,1,P,4,5,1,2,1,B,U,3,2,4,7,R,T,V,8,4,F,2}, Line = 2

[0068] 9. If Sum>Line+1 is satisfied, continue to perform steps 2 to 8 to classify new line characters. If Sum=Line+1 is satisfied after iteration, stop iteration and identify and classify all Sum line characters: {U 1 ,U 2 ,…,U Sum}. The final character row classification output of this example is as follows Figure 5 As shown: 1 ,U 2 ,…,U 4}={S10123011,1DN4170931,P45121BU32,47RTV84F2}, and the correct result of character row classification is obtained.

Claims

1. A line classification post-processing method for multi-line character OCR recognition results, characterized in that: The steps include: Step 1: Given the OCR recognition result {O = S1,…,S i ,…,S m }, a total of m characters, get the original data of the bounding boxes of all characters; the bounding box data of the i-th, 1≤i≤m characters is expressed as: the coordinates of the upper left corner point P i (x i ,y i ), frame width l i and frame height h i ; The upper left corner of the OCR recognized image is the coordinate origin O(0,0); Enter the total number of lines of text to be recognized Sum, record Line = 1; Step 2: Traverse the y coordinates of all the characters in O, and take the point with the minimum y value as the initial point, recorded as the initial point P1(x1,y1); Step 3: Take the unit vector in the X-axis direction as the initial vector A = (1,1); take P1 as the starting point of the vector, and the remaining points P except P1 j ,2≤j≤m is the end point of the vector, construct other vectors B j =(x j -x1,y j -y1), we get the vector set E = {A, B2,…, B j ,…,B m }; Step 4: Calculate the initial vector A and other vectors B according to formula (1) j The angle θ between j , reduce the original two-dimensional bounding box data to a one-dimensional vector angle data set G = {θ2,…,θ j ,…,θ m }; Step 5: Use formula (2) to perform nonlinear log normalization on the data set G and obtain the normalized result G′={θ′2,…,θ′ j ,…,θ′ m }; where θ max is the θ in the set G j Maximum value, Represents θ in the set G j The minimum value is rounded down; Step 6: Use K-Means clustering algorithm to divide the similarity of normalized sample data G′ into k categories of clusters C = {C1,…,C e ,…,C k }, randomly select k cluster centers: μ1,…,μ e ,…,μ k , with the goal of minimizing the following loss function E: Step 6-1: Take k = 2, that is, use K-Means clustering to get two cluster results; let C = {C1, C2}, and the cluster centers are: μ1, μ2; Step 6-2: Initialize the cluster e=1,2;Calculate each element θ′ in the sample data G′ j With each cluster center μ e Distance According to d GK The size of θ′ determines j The cluster label λ e , and then classify each element into the cluster label λ e The corresponding cluster, that is Step 6-3: If the condition of loss function E is not met or not all rows are identified, recalculate the center point corresponding to each cluster If μ e ≠μ′ e , then the assignment updates μ e =μ′ e ; Otherwise, keep the current cluster center unchanged; repeat steps 6-2 and 6-3, and when the cluster centers no longer change, output the final two clusters Step 7: The characters in the same row are classified into a set C e The internal elements have the highest similarity, and the calculated loss function E e Minimum, then take the minimum loss function E of each cluster min The corresponding cluster C e For similar direction vectors, obtain the corresponding vector end point coordinates, that is, output the same row result U e ; Step 8: Assign the remaining characters as new input characters and subtract U from O e , let Line add 1; Step 9: If Sum>Line+1, continue to perform the operations from step 2 to step 8 to classify the new line characters; if Sum=Line+1 after iteration, stop iteration, identify and classify all Sum line characters: {U1,U2,…,U Sum }.

Images