An OCR character recognition method, an electronic device, and a storage medium
By combining neural networks and rotation correction techniques with vertical projection segmentation, the problem of low accuracy in Chinese character recognition in natural scenes is solved, achieving fast and accurate text recognition results.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- BEIJING ZHONGDIAN HUIZHI TECH CO LTD
- Filing Date
- 2022-09-13
- Publication Date
- 2026-04-14
AI Technical Summary
The accuracy of text recognition in natural scenes is low, especially for Chinese characters. Due to factors such as individual shape differences, diverse arrangement and layout, complex backgrounds, and blurring and distortion, existing technologies are unable to effectively solve this problem.
We employ a neural network-based OCR text recognition method, which improves text recognition accuracy by combining feature extraction, model training, rotation correction, vertical projection segmentation, and an OCR recognition engine with voting and template segmentation methods.
It enables rapid segmentation and accurate recognition of text in natural scenes, improving the efficiency of OCR text recognition.
Smart Images
Figure CN115457565B_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of optical character recognition technology, and particularly relates to an OCR text recognition method, electronic device and storage medium. Background Technology
[0002] With the development of the information age, text information has shifted from paper books to computer text. This raises the question of how to effectively store text from books onto computers. OCR technology emerged to address this need. OCR refers to the process by which electronic devices use optical character recognition to translate character shapes into computer text. However, text in natural scenes exhibits characteristics such as significant individual shape variations, diverse arrangement layouts, complex and irregular backgrounds, and blurriness or distortion. Furthermore, the sheer number of characters and the similarity in some character shapes further complicate the accuracy of text recognition in natural scenes, particularly for Chinese characters. Therefore, a universal and effective algorithm is needed to solve the problem of Chinese character detection and recognition in natural scenes. Summary of the Invention
[0003] In view of this, the present invention provides an OCR text recognition method, electronic device and storage medium that can improve the accuracy of text recognition and efficiently extract text, so as to solve the above-mentioned technical problems. Specifically, the following technical solutions are adopted to achieve this.
[0004] In a first aspect, the present invention provides an OCR text recognition method, comprising the following steps:
[0005] The text data to be recognized is acquired, and the text data is preprocessed to obtain a text image;
[0006] The text image is trained by extracting feature information from the neural network and annotating the data to obtain the corresponding training model. The text region of the text image is predicted, located and cropped according to the training model. The text image is then rotated and corrected by the image rotation correction model to obtain the target text image.
[0007] The text in the target text image is segmented to obtain character regions. The points on the character regions are vertically mapped to the bottom position. The number of black pixels at each position after vertical projection is counted. The statistical results at each position are used as the standard for segmenting characters, so that multiple characters in the text are segmented into single characters.
[0008] The segmented individual characters are fed into the OCR recognition engine for text recognition, and the recognized text is output.
[0009] As a further improvement to the above technical solution, the dots on the character area are vertically mapped to the bottommost position, including:
[0010] S20: After vertical projection, count the number of black pixels at each position in the target text image;
[0011] S21: Perform preliminary character segmentation based on the number of black pixels, and record the segmentation position and the width of the segmentation result;
[0012] S22: Find the modulo of the width of the segmentation result, and use it as the width of the segmentation target character in the target text image, which is based on the fact that printed text has the same character width;
[0013] S23: Based on the digit value in step 3, verify the width in S21. If the difference is large and the calculated width exceeds two-thirds of the digit value of the width, it is determined that there is an overlapping part of the character and further segmentation operation is required. Execute S24 below. Otherwise, it is determined that the character width is normal.
[0014] S24: Divide the calculated width by the modulo of the width and round up to get the result. Use this result as the loop count to obtain the number of characters contained in the overlapping area. Near the center positions of two adjacent characters, find the position with the smallest projection point and use it as the cutting position of the overlapping characters.
[0015] As a further improvement to the above technical solution, the text image is obtained by rotating and correcting it using an image rotation correction model, including:
[0016] The text in the text image is merged into lines, and the text with the longest length is selected for text recognition to obtain its confidence score. After comparison with the confidence score threshold, it is determined whether it is an upright text.
[0017] The preset confidence threshold is 0.5. If the confidence score of the text recognized by the text recognition model is less than 0.5, the text is determined to be upside down; otherwise, if the confidence score is greater than 0.5, the text is determined to be upright.
[0018] In the process of merging text boxes, all text boxes are sorted by their top-left corner coordinates from smallest to largest (if the coordinates are the same, they are sorted by their horizontal coordinates from smallest to largest). The text boxes are then checked for inclusion in a specific row by continuously increasing the y-axis value. After several rows are identified, the text boxes in each row are sorted by the x-axis, and then the text boxes in each row are merged from left to right.
[0019] As a further improvement to the above technical solution, a voting method is used to determine the orientation category of the current text image. The specific process includes:
[0020] Let A be a sample set and b be a test sample. A distance calculation method is selected as the distance metric between the two samples. The distances between samples in set A and the test sample b are calculated and sorted in ascending order. The k nearest samples are selected, and b is then predicted to have a category or value based on the information from these k nearest samples, where k ≤ 20, k ∈ N. * ;
[0021] When the distance metric is Euclidean distance, the two-dimensional expression for Euclidean distance is: Where x1, y1, x2, and y2 represent the x and y coordinates of the two points;
[0022] Given a set A containing elements {0,1}, where 0 is classified as inverted and 1 as upright, a one-dimensional absolute value function is used as the distance variable. When k=1, the element with the closest distance to the original set A is obtained by calculating the confidence level and sorting the values in ascending order. The category of this element is then the category value of the text box.
[0023] As a further improvement to the above technical solution, a template segmentation method is used to obtain single characters, including:
[0024] To obtain the left boundary of the first character, the vector V stores the character pixels of each column. The first V(f1)≠0 in V is the left boundary position f1 of the first character template. The left boundary of the second character template is determined from the arrangement position of the characters in the text line. The arrangement position is greater than the right boundary position of the first character template, and at the same time, V(f2)≠0 is satisfied.
[0025] The process of determining the left boundary of the third and subsequent character templates is the same as that of the second character. The expression for determining the left boundary of the character template is j = 1, V(f j )≠0 and V(i)=0; i=1,2...f j -1; the left boundary expression for the subsequent character is j≥2, V(f j )≠0,f j >r j And V(i)=0; i=r j-1 ,r j-1 +1...f j -1, where f j R represents the left boundary of the j-th character. j Indicates the right boundary of the j-th character;
[0026] Set the right boundary response function to e V(r) The expression for constructing the character template response function based on the left boundary expression of the subsequent characters is as follows: Where λ represents the robustness factor used to improve the value of the response function inside the character template, L is the left boundary of the character, and r is the right boundary of the character;
[0027] By changing the right boundary position of the character template, the width of the character template is gradually increased, and the corresponding function response value Mr(L,r) is calculated. When Mr(L,r) reaches its maximum value, the left and right boundaries of the current template are the left and right boundaries of a single character.
[0028] As a further improvement to the above technical solution, the text image is obtained by rotating and correcting it using an image rotation correction model, including:
[0029] The predicted text region image after initial rotation is input into the text recognizer for text recognition and outputs its text confidence score. The confidence score determines whether the current image is upright.
[0030] If the object is upright, the initial rotation angle is the final rotation angle; otherwise, the initial rotation angle plus 180° equals the final rotation angle.
[0031] As a further improvement to the above technical solution, a corresponding training model is obtained by extracting feature information from text images and training labeled data based on neural networks, including:
[0032] The full convolutional neural network is used to extract features and learn the corresponding text line parameters, including (x, y, w, h, θ) and confidence scores, as well as the four coordinates of the quadrilateral. Non-maximum suppression is used to filter the geometry of the generated text detection boxes. The overall feature extraction network includes a feature extraction layer, a feature fusion layer, and an output layer.
[0033] Feature extraction layer: First, feature extraction is performed, followed by four stages of convolution. The size of the convolution kernel decreases sequentially, being 1 / 32, 1 / 16, 1 / 8, and 1 / 4 of the input image, respectively, while the number of kernels increases sequentially. Feature maps are extracted in each stage of the convolutional layer. Larger feature maps are used to predict smaller text lines, while smaller convolutional layers are used to predict larger text lines.
[0034] Feature fusion layer: Features from the top of the feature extraction network are merged sequentially downwards according to corresponding rules. First, the feature map obtained in the feature extraction stage is input into the upper pooling layer for expansion. Then, the feature maps from the upper and lower layers are fused and input into the convolutional layer for convolution calculation to reduce the number of channels and computational cost. After four processing steps, the final feature map is generated and input into the output layer to obtain the final result. The expression for this stage is as follows: g i The fusion rules for different stages are represented by the following expression:
[0035] Output layer: Determines the shape of the text box, including rotated rectangles and arbitrary quadrilaterals. When a text box with the shape of a rotated rectangle is detected, the output includes the confidence score of the prediction box, the position parameters (x,y,w,h) of the text box, and the rotation angle θ of the text box. When a text box with the shape of an arbitrary quadrilateral is detected, the output includes the confidence score of the prediction box and the position information of the text.
[0036] As a further improvement to the above technical solution, the text data to be recognized is acquired, and the text data is preprocessed to obtain a text image, including:
[0037] The Niblack algorithm is used to preprocess the text data. Within a window of a preset size, the mean gray value and gray variance of other pixels in the target point window are calculated to determine the nearest threshold. The expression for the optimal threshold is t(i,j)=m(i,j)+p*s(i,j), where t(i,j) represents the optimal threshold obtained, p represents the preset correction value, m(i,j) represents the mean gray value of adjacent pixels in the target point window, and s(i,j) represents the gray variance of adjacent pixels in the target point window.
[0038] For each pixel in the image, a calculation is performed based on the neighboring pixels. If the result is close to the grayscale mean m(i,j), it is determined to be the image background; otherwise, it is determined to be the text image.
[0039] Secondly, the present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the steps of the above-described OCR text recognition method.
[0040] Thirdly, the present invention also provides a storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the steps of the OCR text recognition method as described in any one of claims 1-8.
[0041] This invention provides an OCR text recognition method, electronic device, and storage medium. It acquires text data to be recognized, preprocesses the text data to obtain a text image, extracts feature information from the text image based on a neural network, and trains the model by labeling the data. Based on the training model, it predicts, locates, and crops the text region of the text image. It then uses an image rotation correction model to rotate and correct the text image to obtain the target text image. The text in the target text image is segmented into character regions. Points on the character regions are vertically mapped to the bottommost position. The number of black pixels at each position after vertical projection is counted. The statistical results at each position are used as the standard for character segmentation, thus segmenting multiple characters in the text into single characters. The segmented single characters are then fed into an OCR recognition engine for text recognition, outputting the recognized text. This method enables rapid segmentation and accurate recognition of text in text images, thereby improving the efficiency of OCR text recognition. Attached Figure Description
[0042] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of the present invention and should not be regarded as a limitation on the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.
[0043] Figure 1 A flowchart of the OCR text recognition method provided by the present invention;
[0044] Figure 2 A process diagram of character segmentation provided by the present invention;
[0045] Figure 3 This is a structural block diagram of the electronic device provided by the present invention. Detailed Implementation
[0046] Embodiments of the present invention are described in detail below. Examples of these embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and should not be construed as limiting the present invention.
[0047] It should be noted that when an element is said to be "fixed" to another element, it can be directly on the other element or there may be an intervening element. When an element is said to be "connected" to another element, it can be directly connected to the other element or there may be an intervening element. Conversely, when an element is said to be "directly" on another element, there is no intervening element. The terms "vertical," "horizontal," "left," "right," and similar expressions used in this document are for illustrative purposes only.
[0048] See Figure 1 This invention provides an OCR text recognition method, comprising the following steps:
[0049] S10: Obtain the text data to be recognized and preprocess the text data to obtain a text image;
[0050] S11: Based on the neural network, feature information is extracted from the text image and labeled data is used to train the corresponding training model. The text region of the text image is predicted, located and cropped according to the training model. The text image is rotated and corrected through the image rotation correction model to obtain the target text image.
[0051] S12: Segment the text in the target text image to obtain character regions, vertically map the points on the character regions to the bottom position, count the number of black pixels at each position after vertical projection, and use the statistical results at each position as the standard for segmenting characters, so that multiple characters in the text are segmented into single characters.
[0052] S13: The segmented single characters are fed into the OCR recognition engine for text recognition, and the recognized text is output.
[0053] In this embodiment, the text image is rotated and corrected to obtain the target text image through the image rotation correction model. This includes: inputting the predicted text region image after preliminary rotation into a text recognizer for text recognition and outputting its text confidence score; determining whether the current image is upright based on the confidence score; if upright, the preliminary rotation angle is the final rotation angle; otherwise, the preliminary rotation angle plus 180° equals the final rotation angle. The text data to be recognized is acquired and preprocessed to obtain a text image. This includes: preprocessing the text data using the NiBlack algorithm; calculating the mean gray value and gray variance of other pixels within a pre-defined window to determine the nearest threshold; the expression for the optimal threshold is t(i,j) = m(i,j) + p*s(i,j), where t(i,j) represents the optimal threshold, p represents a pre-set correction value, m(i,j) represents the mean gray value of adjacent pixels within the target point window, and s(i,j) represents the gray variance of adjacent pixels within the target point window; for each pixel in the image, the gray value is calculated individually based on its neighboring pixels. If the gray value is close to the mean gray value m(i,j), it is determined to be the image background; otherwise, it is determined to be the text image.
[0054] It should be noted that image text recognition includes steps such as text localization, text extraction, and text recognition. Text extraction includes image preprocessing of digital text images to achieve segmentation of the target text from the image background, as well as text segmentation. In the rotation correction process of the express delivery slip's main body, the image of the express delivery slip's main body is rotated and corrected by comprehensively analyzing the prediction information of the cascaded barcode detection model, the text detection model, and the text recognition model. The process includes: First, using a barcode detector to predict the position of the barcode to obtain the barcode's position information. If the barcode is detected, the initial rotation angle is directly calculated, and the process jumps to the third step; otherwise, the process proceeds to the second step. Second, using a text detector to predict the text box information of the image, the overall long and short sides of the text box are calculated and determined, and then the initial rotation angle is calculated. Third, if the image comes directly from the first step, it is rotated according to the initial rotation angle, and then the text detector is used to predict the text box of the initially rotated image. If the image comes from the second step, both the image and the text box are rotated according to the initial rotation angle. The image is classified as upright or inverted after rotation and the inverted image is corrected. The entire image can be classified as upright or inverted by voting on the upright or inverted classification results of each text box. The recognition of a single text image gives a confidence value reflecting whether the recognition result is text, and this value is sensitive to Chinese characters.
[0055] It should be understood that text data constitutes a relatively small proportion of image data. If the original image is directly used as input to the text recognition model, the text information will be distorted or even lost due to image scaling, resulting in a deterioration in the performance of both text detection and text recognition models. Therefore, it is necessary to locate the text in the original image before text recognition. Although the arbitrary orientation of text does not affect the performance of the text detection model, the input to the text recognition model is horizontal text. Before text recognition, the text data needs to be rotated and corrected, and the resulting upright image is used as input to the text recognizer, thereby improving the accuracy and speed of text recognition.
[0056] See Figure 2 This maps the points on the character region vertically to the bottommost position, including:
[0057] S20: After vertical projection, count the number of black pixels at each position in the target text image;
[0058] S21: Perform preliminary character segmentation based on the number of black pixels, and record the segmentation position and the width of the segmentation result;
[0059] S22: Find the modulo of the width of the segmentation result, and use it as the width of the segmentation target character in the target text image, which is based on the fact that printed text has the same character width;
[0060] S23: Based on the digit value in step 3, verify the width in S21. If the difference is large and the calculated width exceeds two-thirds of the digit value of the width, it is determined that there is an overlapping part of the character and further segmentation operation is required. Execute S24 below. Otherwise, it is determined that the character width is normal.
[0061] S24: Divide the calculated width by the modulo of the width and round up to get the result. Use this result as the loop count to obtain the number of characters contained in the overlapping area. Near the center positions of two adjacent characters, find the position with the smallest projection point and use it as the cutting position of the overlapping characters.
[0062] In this embodiment, vertical projection is a common geometric projection method used in two-dimensional plane maps. It refers to projecting a vertical line from a point on the ground onto a horizontal plane. In character segmentation, vertical projection means that all points on the segmented character area are vertically mapped to the bottommost position. Then, the number of black pixels at each bottommost position after vertical projection is counted, and the statistical result at each position is used as the standard for character segmentation. Each character has a certain pixel width. The purpose of counting the number of black pixels at each position after vertical projection is to segment characters and, if characters overlap, to find the optimal segmentation position based on the counted pixels. If characters have overlapping areas, the segmentation result may be that a single character area contains multiple characters other than the target font. It is necessary to ensure that the segmentation result contains only one character.
[0063] Optionally, the target text image is obtained by rotating the text image using an image rotation correction model, including:
[0064] The text in the text image is merged into lines, and the text with the longest length is selected for text recognition to obtain its confidence score. After comparison with the confidence score threshold, it is determined whether it is an upright text.
[0065] The preset confidence threshold is 0.5. If the confidence score of the text recognized by the text recognition model is less than 0.5, the text is determined to be upside down; otherwise, if the confidence score is greater than 0.5, the text is determined to be upright.
[0066] In the process of merging text boxes, all text boxes are sorted by their top-left corner coordinates from smallest to largest (if the coordinates are the same, they are sorted by their horizontal coordinates from smallest to largest). The text boxes are then checked for inclusion in a specific row by continuously increasing the y-axis value. After several rows are identified, the text boxes in each row are sorted by the x-axis, and then the text boxes in each row are merged from left to right.
[0067] In this embodiment, a voting method is used to determine the orientation of the current text image. The specific process includes: A is a sample set, and b is a test sample. A distance calculation method is selected as the distance metric between the two samples. The distances between the samples in set A and the test sample b are calculated and sorted in ascending order. The k closest samples are selected, and b is then predicted to have a category or value based on the information from the k closest samples, where k ≤ 20, k ∈ N. * When the distance metric is Euclidean distance, the two-dimensional expression for Euclidean distance is: Where x1, y1, x2, and y2 represent the x and y coordinates of two points; the preset set A contains elements {0, 1}, where 0 is inverted and 1 is upright. A one-dimensional absolute value function is used as the distance variable. When k = 1, the element with the closest distance is obtained by calculating the confidence level and the original distance in set A and sorting them from smallest to largest. The category of this element is the category value of the text box.
[0068] Optionally, a template segmentation method is used to obtain single characters, including:
[0069] To obtain the left boundary of the first character, the vector V stores the character pixels of each column. The first V(f1)≠0 in V is the left boundary position f1 of the first character template. The left boundary of the second character template is determined from the arrangement position of the characters in the text line. The arrangement position is greater than the right boundary position of the first character template, and at the same time, V(f2)≠0 is satisfied.
[0070] The process of determining the left boundary of the third and subsequent character templates is the same as that of the second character. The expression for determining the left boundary of the character template is j = 1, V(f k )≠0 and V(i)=0; i=1,2...f j -1; the left boundary expression for the subsequent character is j≥2, V(f j )≠0,f j >r j And V(i)=0; i=r j-1 ,r j-1 +1...f j -1, where f j R represents the left boundary of the j-th character. j Indicates the right boundary of the j-th character;
[0071] Set the right boundary response function to e V(r) The expression for constructing the character template response function based on the left boundary expression of the subsequent characters is as follows: Where λ represents the robustness factor used to improve the value of the response function inside the character template, L is the left boundary of the character, and r is the right boundary of the character;
[0072] By changing the right boundary position of the character template, the width of the character template is gradually increased, and the corresponding function response value Mr(L,r) is calculated. When Mr(L,r) reaches its maximum value, the left and right boundaries of the current template are the left and right boundaries of a single character.
[0073] In this embodiment, Chinese characters with left and right structures have gaps of varying sizes inside. Directly segmenting characters using the vertical projection method would lead to incomplete or incorrect character segmentation. Therefore, it's necessary to first determine the left boundary of the template, then change the right boundary, and finally determine the optimal left and right boundary positions of the characters based on the template's corresponding function value. This ensures the optimal path for character segmentation, preventing incorrect segmentation due to large gaps in the left and right structures of the Chinese characters. During the segmentation process, the template's interior and edges should contain as many character and background pixels as possible, and the smaller the template area, the better, ensuring complete segmentation of characters within the template and their adjacent characters. Since the image to be processed is a text line image with a fixed height, a constructor is needed for the template's interior and right boundary. To achieve a good segmentation effect, the template's interior should contain as many character pixels as possible. Therefore, the template's interior response function is essentially the total number of character pixels contained within the template, i.e., ∑V(i).
[0074] It should be noted that when text information is distributed across most of the image area, a text detector can be used directly to predict the text location. However, if the text information occupies a small portion of the image, it is difficult to detect the text by directly inputting the image into the text detector. Taking a waybill as an example, after detecting the main body of the waybill, it needs to be rotated and corrected. The rotation angle needs to be calculated based on the image information. Image rotation can be performed using either barcode location information or text boxes, but the former is much faster than the latter. For images without barcodes, to improve the robustness of the entire model, it is necessary to combine segmentation information and image feature information to locate and extract the target text image. Several text boxes are selected as targets. The text recognizer's recognition confidence is sensitive to text, and the target text boxes are classified into nearest neighbors. The selected text boxes are then used to vote on whether the current image is upright. If it is upright, the final rotation angle is equal to the initial rotation angle; otherwise, the final rotation angle is equal to the initial rotation angle plus 180°. The barcode detection model predicts a rectangular box for the barcode, while the text detection model predicts several text boxes for all text regions. The rotation angle is calculated for each individual text box. For a single barcode rectangle, its side length needs to be calculated to determine the long side, and the rotation angle can be calculated. The set of text boxes for a text region is calculated by determining the average side length of the entire text box in two directions. After distinguishing the long and short sides, the rotation angle of each text box is calculated and its average value is taken to improve the speed and accuracy of text recognition.
[0075] Optionally, a corresponding training model is obtained by extracting feature information from text images and training labeled data based on neural networks, including:
[0076] The full convolutional neural network is used to extract features and learn the corresponding text line parameters, including (x, y, w, h, θ) and confidence scores, as well as the four coordinates of the quadrilateral. Non-maximum suppression is used to filter the geometry of the generated text detection boxes. The overall feature extraction network includes a feature extraction layer, a feature fusion layer, and an output layer.
[0077] Feature extraction layer: First, feature extraction is performed, followed by four stages of convolution. The size of the convolution kernel decreases sequentially, being 1 / 32, 1 / 16, 1 / 8, and 1 / 4 of the input image, respectively, while the number of kernels increases sequentially. Feature maps are extracted in each stage of the convolutional layer. Larger feature maps are used to predict smaller text lines, while smaller convolutional layers are used to predict larger text lines.
[0078] Feature fusion layer: Features from the top of the feature extraction network are merged sequentially downwards according to corresponding rules. First, the feature map obtained in the feature extraction stage is input into the upper pooling layer for expansion. Then, the feature maps from the upper and lower layers are fused and input into the convolutional layer for convolution calculation to reduce the number of channels and computational cost. After four processing steps, the final feature map is generated and input into the output layer to obtain the final result. The expression for this stage is as follows: g i The fusion rules for different stages are represented by the following expression:
[0079] Output layer: Determines the shape of the text box, including rotated rectangles and arbitrary quadrilaterals. When a text box with the shape of a rotated rectangle is detected, the output includes the confidence score of the prediction box, the position parameters (x,y,w,h) of the text box, and the rotation angle θ of the text box. When a text box with the shape of an arbitrary quadrilateral is detected, the output includes the confidence score of the prediction box and the position information of the text.
[0080] In this embodiment, the distances to the four sides of the text box are predicted for each point within the text box. A weighted average of the predicted distances is then used to predict the positions of the four vertices of the border or text box. However, for long texts, using vertices from one end to predict vertices from another end is prone to breakage or unpredictability due to limitations in the receptive field. All pixels in the text constitute the shape of the text box; only boundary pixels are needed to predict the coordinates of the regressed vertices. Boundary pixels are further divided into head and tail pixels. A weighted average of the predicted values of all boundary pixels is used to predict the two vertices at the head or tail ends. Then, a weighted average is taken based on the predicted offsets X, Y, X, Y, X, and Y of the top-left, bottom-left, and bottom-right corners from the head and tail pixels to finally obtain the predicted coordinates of the four vertices. The pixels in the image are sorted by the boundary confidence threshold, and adjacent pixels are grouped into a set. The set is then expanded to obtain a new set. The intersection of all sets is then checked. This process is repeated until the final set is obtained, which is the range of the text box. The final position of the vertex of the text box is predicted by the weighted average distance from all the first and last pixels to the vertex.
[0081] See Figure 3 The present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, it implements the steps of the above-described OCR text recognition method. The OCR text recognition method includes: acquiring text data to be recognized and preprocessing the text data to obtain a text image; extracting feature information from the text image based on a neural network and training it with labeled data to obtain a corresponding training model; predicting, locating, and cropping the text region of the text image according to the training model; rotating and correcting the text image using an image rotation correction model to obtain a target text image; segmenting the text in the target text image to obtain character regions; vertically mapping the points on the character regions to the bottommost position; counting the number of black pixels at each position after vertical projection; using the statistical results at each position as the standard for character segmentation, thus segmenting multiple characters in the text into single characters; and feeding the segmented single characters into an OCR recognition engine for text recognition, outputting the recognized text.
[0082] In addition, the present invention also provides a storage medium storing a computer program thereon, which, when executed by a processor, implements the steps of the OCR text recognition method as described in any one of claims 1-8. The OCR text recognition method includes: acquiring text data to be recognized and preprocessing the text data to obtain a text image; extracting feature information from the text image based on a neural network and training it with labeled data to obtain a corresponding training model; predicting, locating, and cropping the text region of the text image according to the training model; rotating and correcting the text image using an image rotation correction model to obtain a target text image; segmenting the text in the target text image to obtain character regions; vertically mapping the points on the character regions to the bottommost position; counting the number of black pixels at each position after vertical projection; using the statistical results at each position as the standard for character segmentation, thus segmenting multiple characters in the text into single characters; and sending the segmented single characters into an OCR recognition engine for text recognition, outputting the recognized text.
[0083] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.
[0084] In all examples shown and described herein, any specific values should be interpreted as merely exemplary and not as limitations; therefore, other examples of exemplary embodiments may have different values.
[0085] It should be noted that similar labels and letters in the following figures indicate similar items. Therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures.
[0086] The above-described embodiments are merely illustrative of several implementations of the present invention, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of the invention. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention, and these modifications and improvements all fall within the scope of protection of the present invention.
Claims
1. An OCR text recognition method, characterized in that, Includes the following steps: The text data to be recognized is acquired, and the text data is preprocessed to obtain a text image; The text image is trained by extracting feature information from the neural network and annotating the data to obtain the corresponding training model. The text region of the text image is predicted, located and cropped according to the training model. The text image is then rotated and corrected by the image rotation correction model to obtain the target text image. The text in the target text image is segmented to obtain character regions. The points on the character regions are vertically mapped to the bottom position. The number of black pixels at each position after vertical projection is counted. The statistical results at each position are used as the standard for segmenting characters, so that multiple characters in the text are segmented into single characters. The segmented single characters are fed into the OCR recognition engine for text recognition, and the recognized text is output. Vertically map the points on the character region to the bottommost position, including: S20: After vertical projection, count the number of black pixels at each position in the target text image; S21: Perform preliminary character segmentation based on the number of black pixels, and record the segmentation position and the width of the segmentation result; S22: Find the modulo of the width of the segmentation result, and use it as the width of the segmentation target character in the target text image, which is based on the fact that printed text has the same character width; S23: Based on the digit value in step 3, verify the width in S21. If the difference is large and the calculated width exceeds two-thirds of the digit value of the width, it is determined that there is an overlapping part of the character and further segmentation operation is required. Execute S24 below. Otherwise, it is determined that the character width is normal. S24: Divide the calculated width by the modulo of the width and round up to get the result. Use this result as the loop count to obtain the number of characters contained in the overlapping area. Near the center positions of two adjacent characters, find the position with the smallest projection point and use it as the cutting position of the overlapping characters. The text image is obtained by rotating and correcting the text image using an image rotation correction model, including: The text in the text image is merged into lines, and the text with the longest length is selected for text recognition to obtain its confidence score. After comparison with the confidence score threshold, it is determined whether it is an upright text. The preset confidence threshold is 0.
5. If the confidence score of the text recognized by the text recognition model is less than 0.5, the text is determined to be upside down; otherwise, if the confidence score is greater than 0.5, the text is determined to be upright. In the process of merging text boxes, all text boxes are sorted by their top-left corner coordinates from smallest to largest (if the coordinates are the same, they are sorted by their horizontal coordinates from smallest to largest). The text boxes are then checked for being in a particular row by continuously increasing the y-axis value. After finding several rows, the text boxes in each row are sorted by the x-axis and then merged from left to right. Single characters are obtained using template segmentation, including: To obtain the left boundary of the first character, the vector V stores the pixel values of each column of characters. The first character in V... The left boundary position of the first character template The left boundary of the second character template is determined by the arrangement of characters in the text line. This arrangement is greater than the right boundary of the first character template, and simultaneously satisfies... ; The process of determining the left boundary of the third and subsequent character templates is the same as that of the second character. The expression for determining the left boundary of the character template is as follows: The left boundary expression for subsequent characters is: ,in This represents the left boundary of the j-th character. Indicates the right boundary of the j-th character; Set the right boundary response function as The expression for constructing the character template response function based on the left boundary expression of the subsequent characters is as follows: ,in The robustness factor is used to increase the influence of the response function value inside the character template, where L is the left boundary of the character and r is the right boundary of the character. By gradually increasing the width of the character template by changing its right boundary position, the corresponding function response value is calculated. ,when When the maximum value is reached, the left and right boundaries of the current template are the left and right boundaries of a single character.
2. The OCR text recognition method according to claim 1, characterized in that, The orientation of the current text image is determined by voting. The specific process includes: Let A be a sample set and b be a test sample. By selecting a distance calculation method as the distance metric between the two samples, the distances between samples in set A and the test sample b are calculated and sorted in ascending order. The k nearest samples are selected, and b is then predicted to have a category or value based on the information from these k nearest samples. ; When the distance metric is Euclidean distance, the two-dimensional expression for Euclidean distance is: ,in , , and Represents the x and y coordinates of two points; Elements in set A 0 represents inverted, and 1 represents upright. A one-dimensional absolute value function is chosen as the distance variable. When the confidence level is calculated and the original distance in set A is sorted in ascending order, the element with the closest distance is obtained, and the category of that element is the category value of the text box.
3. The OCR text recognition method according to claim 1, characterized in that, The text image is obtained by rotating and correcting the text image using an image rotation correction model, including: The predicted text region image after initial rotation is input into the text recognizer for text recognition and outputs its text confidence score. The confidence score determines whether the current image is upright. If the object is upright, the initial rotation angle is the final rotation angle; otherwise, the initial rotation angle plus 180° equals the final rotation angle.
4. The OCR text recognition method according to claim 1, characterized in that, Based on neural networks, feature information is extracted from text images and labeled data is used to train corresponding training models, including: Feature extraction and learning are performed using a fully convolutional neural network to generate corresponding text line parameters, including... The feature extraction network consists of a feature extraction layer, a feature fusion layer, and an output layer. The network uses non-maximum suppression to filter the geometry of the generated text detection box based on the confidence level and the four coordinates of the quadrilateral. Feature extraction layer: First, feature extraction is performed, followed by four stages of convolution, with the kernel size decreasing sequentially, corresponding to the input image size... , , and The number of features increases sequentially. At each stage of the convolutional layer, feature maps are extracted. The larger feature maps are used to predict the smaller text lines, and the smaller convolutional layers are used to predict the larger text lines. Feature fusion layer: Features from the top of the feature extraction network are merged sequentially downwards according to corresponding rules. First, the feature map obtained in the feature extraction stage is input into the upper pooling layer for expansion. Then, the feature maps from the upper and lower layers are fused and input into the convolutional layer for convolution calculation to reduce the number of channels and computational cost. After four processing steps, the final feature map is generated and input into the output layer to obtain the final result. The expression for this stage is as follows: , The fusion rules for different stages are represented by the following expression: ; Output layer: Determines the shape of the text box, including rotated rectangles and arbitrary quadrilaterals. When a text box with the shape of a rotated rectangle is detected, the output includes the confidence score of the predicted bounding box and the text box's... Position parameters and text box rotation angle When detecting text boxes of arbitrary quadrilateral shape, the output includes the confidence score of the predicted box and the position information of the text.
5. The OCR text recognition method according to claim 1, characterized in that, The text data to be recognized is acquired, and the text data is preprocessed to obtain a text image, including: The NiBlack algorithm is used to preprocess the text data. Within a window of a preset size, the mean and variance of the grayscale values of other pixels within the target point window are calculated to determine the nearest threshold. The expression for the optimal threshold is: ,in This represents the optimal threshold value obtained, where p represents the preset correction value. This represents the average grayscale value of adjacent pixels within the target point window. This represents the grayscale variance of adjacent pixels within the target point window; For each pixel in the image, the grayscale mean is calculated individually based on its neighboring pixels. If they are similar, they are judged as image background; otherwise, they are judged as text image.
6. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the steps of the OCR text recognition method as described in any one of claims 1-5.
7. A storage medium having a computer program stored thereon, characterized in that, When a computer program is executed by a processor, it implements the steps of the OCR text recognition method as described in any one of claims 1-5.