Text recognition and analysis system and method based on deep learning
Through deep learning technology, combined with image preprocessing and feature extraction, a text recognition model is constructed using convolutional neural networks and recurrent neural networks, which solves the problem of low accuracy in complex backgrounds in traditional optical character recognition and achieves more efficient text recognition Effect.
Patent Information
- Application Number
- CN202411316439.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-09-20
- Publication Date
- 2025-01-10
AI Technical Summary
Traditional optical character recognition technology has low recognition accuracy under complex backgrounds and special characters and cannot adapt to most environments.
Using a text recognition and analysis system based on deep learning, through image preprocessing, text area positioning and feature extraction, a text recognition model is constructed using convolutional neural networks and recurrent neural networks, and combining Canny edge detection and Sobel operator for text area positioning. Decode output using the Connectivist time classification layer.
It improves the accuracy and efficiency of text recognition, can accurately identify text information in complex contexts, and enhances the performance and generalization capabilities of the model.
Smart Images
Figure CN119296116A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of text recognition systems, and in particular to a text recognition and analysis system and method based on deep learning. Background Art
[0002] Deep learning reduces the burden of feature engineering and improves the performance and generalization ability of the model by automatically learning high-level abstract features from data. Therefore, preprocessing the acquired text image and locating the text area in the preprocessed image is a very important part of building a text detection model. Preprocessing helps to improve the performance and accuracy of the model. By eliminating noise, errors and inconsistencies in the original data, preprocessing can make the data more interpretable, reduce the difficulty of model training, improve the performance of the model, and provide a reliable foundation for subsequent analysis. In image processing, text area positioning is also an important prerequisite for text recognition, and its accuracy and efficiency directly affect the subsequent text recognition and processing process. However, general text area positioning often has generalization and low computational efficiency. For traditional optical character recognition technology, the recognition accuracy of OCR technology is low in complex backgrounds and special characters, so traditional OCR cannot adapt to most environments. Summary of the invention
[0003] The purpose of the present invention is to provide a text recognition and analysis system and method based on deep learning to solve the problems raised in the above background technology.
[0004] In order to solve the above technical problems, the present invention provides the following technical solution: a text recognition and analysis method based on deep learning, comprising the following analysis steps:
[0005] S1, obtaining a text image to be predicted, preprocessing the text image to be predicted, and obtaining a primary image;
[0006] S2, locate the region of the primary image, extract the strip-shaped target, and mark it as a text region;
[0007] S3, construct a text recognition model, extract features from the marked text area through the text recognition model, and obtain a text sequence in the text area;
[0008] S4. Decode the text sequence, output the text recognition result, obtain the text information, and output the text information.
[0009] According to the above technical solution, in step S1, it also includes:
[0010] The preprocessing of the text image to be tested is used to load the image stored in the file system, convert the image from the RGB color space to the grayscale color space, and finally binarize the text image according to different grayscale thresholds.
[0011] According to the above technical solution, in step S2, it also includes:
[0012] S2-1, obtain a pixel map of the primary image, use Canny edge detection to obtain edge pixel points of the primary image, use the Sobel operator to perform a convolution operation on the pixel points at the edge of the image, and obtain the vertical gradient value of the edge pixel points at the edge of the image, where the gradient direction is perpendicular to the edge direction of the line;
[0013] S2-2, select an edge pixel point p on the image boundary, calculate the gradient direction θ of the edge pixel point p by using the gradient value, form an extension line along the gradient direction θ after passing through the edge pixel point p, extract the intersection point between the extension line and the image boundary and mark it as the edge pixel point q, calculate the gradient direction of the edge pixel point q by using the gradient value again and compare it with the gradient direction θ of the edge pixel point p, when the gradient directions are opposite, the edge where the pixel point q is located is parallel to the edge where the edge pixel point p is located, take the line connecting the edge pixel point p and the edge pixel point q as a cross section of the line, and mark the number of pixels passed on the line as the line width;
[0014] S2-3. The adjacent cross sections with set line width are divided into a group. The cross sections in the same group constitute a band target. The characters are composed of the band target with set line width. The image composed of lines is defined as a character. The area containing characters in the primary image is positioned as a text area.
[0015] According to the above technical solution, in step S3, it also includes:
[0016] Construct a text recognition model based on convolutional neural network and recurrent neural network. The convolutional neural network automatically extracts text features through the convolution layer and pooling layer structure. The text image is input, scanned through the convolution layer, and the feature map is extracted. The pooling layer is used to reduce the dimension of the feature map and reduce the amount of calculation while retaining important information.
[0017] The recurrent neural network processes the context information of the text through the feature graph extracted by the convolutional neural network, and combines the feature graph extracted by the convolutional neural network into a text sequence.
[0018] According to the above technical solution, in step S4, it also includes:
[0019] The text region after the positioning of the text image to be tested is obtained, the text sequence in the text region is extracted using the text recognition model, and the output text sequence is decoded and output as text information using the connectionist temporal classification layer.
[0020] A text recognition and analysis system based on deep learning, the system includes: a data preprocessing module, a text area positioning module, a feature extraction module and a text recognition and analysis module;
[0021] The data preprocessing module is used to perform denoising and binarization on the text image to be tested; the text area positioning module is used to identify and locate the text area of the preprocessed text image by whether the image is composed of strokes; the feature extraction module is used to extract character features from the image in the text area to be tested and extract the text sequence through these features to construct a text recognition model; the text recognition and analysis module is used to decode the text sequence extracted from the text area to be tested by the constructed text recognition model using the connectionist temporal classification layer and determine the final text recognition result;
[0022] The output end of the data preprocessing module is connected to the input end of the text area positioning module; the output end of the text area positioning module is connected to the input end of the feature extraction module; the output end of the feature extraction module is connected to the input end of the text recognition and analysis module.
[0023] According to the above technical solution, the data preprocessing module includes a color change unit and a binary segmentation unit;
[0024] The color change unit is used to convert the color image in the text image to be tested into a grayscale image; the binary segmentation unit is used to convert the grayscale image into a binary image;
[0025] The output end of the color changing unit is connected to the input end of the binary segmentation unit.
[0026] According to the above technical solution, the text area positioning module includes a gradient comparison unit and a judgment positioning unit;
[0027] The gradient calculation unit extracts edge pixels from the pixel map of the primary image, and determines whether the image is composed of lines by comparing the gradient direction and the line distance; the determination and positioning unit is used to represent the band-shaped target group with the same width as a character area composed of lines, locate the character area composed of lines and determine it as a text area;
[0028] The output end of the gradient comparison unit is connected to the input end of the determination and positioning unit.
[0029] According to the above technical solution, the feature extraction module includes a convolutional neural unit and a recurrent neural unit;
[0030] The convolutional neural unit uses a convolutional layer to slide on the image using multiple convolution kernels to perform convolution operations, thereby extracting local features of the image. At the same time, the pooling layer downsamples the feature map output by the convolutional layer. After alternating stacking of multiple convolutional layers and pooling layers, the convolutional neural network can extract hierarchical features from low-level to high-level in the image; the recurrent neural unit processes the contextual information of the text through the feature map extracted by the convolutional neural network, and combines the feature map extracted by the convolutional neural network into a text sequence;
[0031] The output end of the convolutional neural unit is connected to the input end of the recurrent neural unit.
[0032] According to the above technical solution, the text recognition and analysis module includes a decoding calculation unit and a decoding output unit;
[0033] The decoding calculation unit calculates the sum of the probabilities of all paths that can be mapped to the sequence based on the text sequence extracted by the text recognition model using the connectionist time classification layer on the output of the sequence modeling network; the decoding output unit obtains the final text recognition result through the final decoding result of the connectionist time classification layer;
[0034] The output end of the decoding calculation unit is connected to the input end of the decoding output unit.
[0035] Compared with the prior art, the beneficial effects of the present invention are: in order to solve the problem that traditional optical character technology cannot accurately recognize special characters under complex backgrounds in text recognition, the present invention utilizes the characteristic that characters are composed of lines, performs calculations and measurements on the composition of the image, and more accurately identifies and locates the text area, so that the subsequent text recognition model can more accurately and clearly recognize the text information. BRIEF DESCRIPTION OF THE DRAWINGS
[0036] Figure 1 It is a flowchart of the text recognition and analysis method based on deep learning of the present invention. DETAILED DESCRIPTION
[0037] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0038] See also Figure 1, In the first embodiment of the present invention, a text recognition and analysis method based on deep learning is provided, the method comprising: obtaining a text image to be predicted, preprocessing the text image to be predicted to obtain a primary image, the preprocessing of the text image to be predicted is used to load an image stored in a file system, and at the same time, the image is converted from an RGB color space to a grayscale color space, and finally the text image is binarized according to different grayscale thresholds;
[0039] Obtain the pixel map of the primary image, use Canny edge detection to obtain the edge pixel points of the primary image, use the Sobel operator to perform convolution operation on the pixel points at the edge of the image, and obtain the vertical gradient value of the edge pixel points at the edge of the image, and its gradient direction is perpendicular to the edge direction of the line;
[0040] Select an edge pixel point p on the image boundary, calculate the gradient direction θ of the edge pixel point p through the gradient value, form an extension line along the gradient direction θ after passing through the edge pixel point p, extract the intersection point between the extension line and the image boundary and mark it as the edge pixel point q, calculate the gradient direction of the edge pixel point q through the gradient value again and compare it with the gradient direction θ of the edge pixel point p. When the gradient directions are opposite, the edge where the pixel point q is located is parallel to the edge where the edge pixel point p is located, take the line connecting the edge pixel point p and the edge pixel point q as a cross section of the line, and mark the number of pixels passed on the line as the line width;
[0041] Adjacent cross sections with set line width are divided into a group. The cross sections in the same group constitute a band target. Characters are composed of band targets with set line width. The image composed of lines is defined as a character, and the area containing characters in the primary image is located as a text area.
[0042] The text recognition model combines convolutional neural networks and recurrent neural networks. The convolutional neural network can automatically extract rich feature representations from text images through the convolutional layer and pooling layer structure. The convolutional layer scans the input image through a set of learnable filters; the filter slides on the input image and performs a dot product operation at each position to output a feature map. Each feature map captures the local features of the characters in the input image. By stacking multiple convolutional layers, complex character features can be stacked and expressed; the pooling layer usually follows the convolutional layer to reduce the dimension of the feature map and reduce the amount of calculation while retaining important information. The pooling layer helps the convolutional neural network achieve feature invariance. Even if the position of the feature in the image changes slightly, the pooling layer can ensure the same output; these features include not only the local details of the characters, but also capture the spatial relationship between characters;
[0043] The recurrent neural network processes the context information of the text through the feature graph extracted by the convolutional neural network, and combines the feature graph extracted by the convolutional neural network into a text sequence;
[0044] By combining the advantages of convolutional neural networks and recurrent neural networks, the model can more accurately recognize characters and words in text images. The convolutional neural network extracts local features in the image, while the recurrent neural network combines these features into meaningful text sequences.
[0045] The text region after the positioning of the text image to be tested is obtained, the text sequence in the text region is extracted using the text recognition model, and the output text sequence is decoded and output as text information using the connectionist temporal classification layer.
[0046] For example: in a complex environment with many irregular patterns and large areas of background color, the background area is extracted and removed through image preprocessing, and a predetermined line width is set according to the required text information. The text is located on the preprocessed primary image, and the irregular patterns and characters that do not conform to the line width are removed to complete the positioning of the text area. A text recognition model is constructed and text features are extracted from the located text area. A text sequence is output, and the output text sequence is used as input. The text information obtained by outputting the connectionist temporal classification layer is used as the result.
[0047] In the second embodiment, a text recognition and analysis system based on deep learning is provided, which includes: a data preprocessing module, a text area positioning module, a feature extraction module and a text recognition and analysis module;
[0048] The data preprocessing module is used to perform denoising and binarization on the text image to be tested; the text area positioning module is used to identify and locate the text area of the preprocessed text image by whether the image is composed of strokes; the feature extraction module is used to extract character features from the image in the text area to be tested and extract the text sequence through these features to construct a text recognition model; the text recognition and analysis module is used to decode the text sequence extracted from the text area to be tested by the constructed text recognition model using the connectionist temporal classification layer and determine the final text recognition result;
[0049] The output end of the data preprocessing module is connected to the input end of the text area positioning module; the output end of the text area positioning module is connected to the input end of the feature extraction module; the output end of the feature extraction module is connected to the input end of the text recognition and analysis module.
[0050] The data preprocessing module includes a color change unit and a binary segmentation unit;
[0051] The color change unit is used to convert the color image in the text image to be tested into a grayscale image; the binary segmentation unit is used to convert the grayscale image into a binary image;
[0052] The output end of the color changing unit is connected to the input end of the binary segmentation unit.
[0053] The text area positioning module includes a gradient comparison unit and a judgment positioning unit;
[0054] The gradient calculation unit extracts edge pixels from the pixel map of the primary image, and determines whether the image is composed of lines by comparing the gradient direction and the line distance; the determination and positioning unit is used to represent the band-shaped target group with the same width as a character area composed of lines, locate the character area composed of lines and determine it as a text area;
[0055] The output end of the gradient comparison unit is connected to the input end of the determination and positioning unit.
[0056] The feature extraction module includes a convolutional neural unit and a recurrent neural unit;
[0057] The convolutional neural unit uses a convolutional layer to slide on the image using multiple convolution kernels to perform convolution operations, thereby extracting local features of the image. At the same time, the pooling layer downsamples the feature map output by the convolutional layer. After alternating stacking of multiple convolutional layers and pooling layers, the convolutional neural network can extract hierarchical features from low-level to high-level in the image; the recurrent neural unit processes the contextual information of the text through the feature map extracted by the convolutional neural network, and combines the feature map extracted by the convolutional neural network into a text sequence;
[0058] The output end of the convolutional neural unit is connected to the input end of the recurrent neural unit.
[0059] The text recognition and analysis module includes a decoding calculation unit and a decoding output unit;
[0060] The decoding calculation unit calculates the sum of the probabilities of all paths that can be mapped to the sequence based on the text sequence extracted by the text recognition model using the connectionist time classification layer on the output of the sequence modeling network; the decoding output unit obtains the final text recognition result through the final decoding result of the connectionist time classification layer;
[0061] The output end of the decoding calculation unit is connected to the input end of the decoding output unit.
[0062] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device.
[0063] Finally, it should be noted that the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art can still modify the technical solutions described in the aforementioned embodiments or replace some of the technical features therein by equivalents. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.
Claims
1. A text recognition and analysis method based on deep learning, characterized by: The method comprises the following steps: S1, obtaining a text image to be predicted, preprocessing the text image to be predicted, and obtaining a primary image; S2, locate the region of the primary image, extract the strip-shaped target, and mark it as a text region; S3, construct a text recognition model, extract features from the marked text area through the text recognition model, and obtain a text sequence in the text area; S4. Decode the text sequence, output the text recognition result, obtain the text information, and output the text information.
2. The text recognition and analysis method based on deep learning according to claim 1, characterized in that: In step S1, it also includes: The preprocessing of the text image to be tested is used to load the image stored in the file system, convert the image from the RGB color space to the grayscale color space, and finally binarize the text image according to different grayscale thresholds.
3. The text recognition and analysis method based on deep learning according to claim 1, characterized in that: The step S2 comprises: S2-1, obtain a pixel map of the primary image, use Canny edge detection to obtain edge pixel points of the primary image, use the Sobel operator to perform a convolution operation on the pixel points at the edge of the image, and obtain the vertical gradient value of the edge pixel points at the edge of the image, where the gradient direction is perpendicular to the edge direction of the line; S2-2, select an edge pixel point p on the image boundary, calculate the gradient direction θ of the edge pixel point p by using the gradient value, form an extension line along the gradient direction θ after passing through the edge pixel point p, extract the intersection point between the extension line and the image boundary and mark it as the edge pixel point q, calculate the gradient direction of the edge pixel point q by using the gradient value again and compare it with the gradient direction θ of the edge pixel point p, when the gradient directions are opposite, the edge where the pixel point q is located is parallel to the edge where the edge pixel point p is located, take the line connecting the edge pixel point p and the edge pixel point q as a cross section of the line, and mark the number of pixels passed on the line as the line width; S2-3. The adjacent cross sections with set line width are divided into a group. The cross sections in the same group constitute a band target. The characters are composed of the band target with set line width. The image composed of lines is defined as a character. The area containing characters in the primary image is positioned as a text area.
4. The text recognition and analysis method based on deep learning according to claim 1, characterized in that: In step S3, it also includes: Construct a text recognition model based on convolutional neural network and recurrent neural network. The convolutional neural network automatically extracts text features through the convolution layer and pooling layer structure. The text image is input, scanned through the convolution layer, and the feature map is extracted. The pooling layer is used to reduce the dimension of the feature map and reduce the amount of calculation while retaining important information. The recurrent neural network processes the context information of the text through the feature graph extracted by the convolutional neural network, and combines the feature graph extracted by the convolutional neural network into a text sequence.
5. The text recognition and analysis method based on deep learning according to claim 1, characterized in that: In step S4, it also includes: The text region after the positioning of the text image to be tested is obtained, the text sequence in the text region is extracted using the text recognition model, and the output text sequence is decoded and output as text information using the connectionist temporal classification layer.
6. A text recognition and analysis system based on deep learning, which is applied to the text recognition and analysis method based on deep learning according to any one of claims 1 to 5, characterized in that: The system includes: a data preprocessing module, a text area positioning module, a feature extraction module and a text recognition and analysis module; The data preprocessing module is used to perform denoising and binarization on the text image to be tested; the text area positioning module is used to identify and locate the text area of the preprocessed text image by whether the image is composed of strokes; the feature extraction module is used to extract character features from the image in the text area to be tested and extract the text sequence through these features to construct a text recognition model; the text recognition and analysis module is used to decode the text sequence extracted from the text area to be tested by the constructed text recognition model using the connectionist temporal classification layer and determine the final text recognition result; The output end of the data preprocessing module is connected to the input end of the text area positioning module; the output end of the text area positioning module is connected to the input end of the feature extraction module; the output end of the feature extraction module is connected to the input end of the text recognition and analysis module.
7. The text recognition and analysis system based on deep learning according to claim 6, characterized in that: The data preprocessing module includes a color change unit and a binary segmentation unit; The color change unit is used to convert the color image in the text image to be tested into a grayscale image; the binary segmentation unit is used to convert the grayscale image into a binary image; The output end of the color changing unit is connected to the input end of the binary segmentation unit.
8. The text recognition and analysis system based on deep learning according to claim 6, characterized in that: The text area positioning module includes a gradient comparison unit and a judgment positioning unit; The gradient calculation unit extracts edge pixels from the pixel map of the primary image, and determines whether the image is composed of lines by comparing the gradient direction and the line distance; the determination and positioning unit is used to represent the band-shaped target group with the same width as a character area composed of lines, locate the character area composed of lines and determine it as a text area; The output end of the gradient comparison unit is connected to the input end of the determination and positioning unit.
9. The text recognition and analysis system based on deep learning according to claim 6, characterized in that: The feature extraction module includes a convolutional neural unit and a recurrent neural unit; The convolutional neural unit uses a convolutional layer to slide on the image using multiple convolution kernels to perform convolution operations, thereby extracting local features of the image. At the same time, the pooling layer downsamples the feature map output by the convolutional layer. After alternating stacking of multiple convolutional layers and pooling layers, the convolutional neural network can extract hierarchical features from low-level to high-level in the image; the recurrent neural unit processes the contextual information of the text through the feature map extracted by the convolutional neural network, and combines the feature map extracted by the convolutional neural network into a text sequence; The output end of the convolutional neural unit is connected to the input end of the recurrent neural unit.
10. The text recognition and analysis system based on deep learning according to claim 6, characterized in that: The text recognition and analysis module includes a decoding calculation unit and a decoding output unit; The decoding calculation unit calculates the sum of the probabilities of all paths that can be mapped to the sequence based on the text sequence extracted by the text recognition model using the connectionist time classification layer on the output of the sequence modeling network; the decoding output unit obtains the final text recognition result through the final decoding result of the connectionist time classification layer; The output end of the decoding calculation unit is connected to the input end of the decoding output unit.