Text recognition method, device and equipment based on multi-attribute supervision

Abstract

The invention relates to the field of text recognition, and provides a text recognition method, device and equipment based on multi-attribute supervision, and the method comprises the steps: obtaining a to-be-recognized text image, and carrying out the preprocessing of the to-be-recognized text image; performing feature extraction on the preprocessed to-be-recognized text image to obtain shared image features; position features of characters in the preprocessed to-be-recognized text image are determined through a trained position attention decoder; the trained position attention decoder is obtained by supervising position information of learning characters; inputting the shared image features into a first time sequence feature extraction model to obtain a classification result of a dictionary font library; and carrying out feature fusion on the shared image features and the position features, and inputting the fused features into a second time sequence feature extraction model to obtain a classification result of the similar attribute word stock. According to the method, the problem of how to improve the recognition precision of the OCR model on similar characters and complex text structures in the prior art is solved, and the performance of text similar character recognition is effectively improved.

Description

Technical Field

[0001] The present invention relates to the field of image recognition technology, and in particular to a text recognition method, device and equipment based on multi-attribute supervision. Background Art

[0002] With the rapid development of computer vision, machine learning and deep learning technologies, optical character recognition (OCR) technology is also constantly improving. OCR technology is widely used in many fields such as document digitization, automated data entry, and intelligent transportation systems. Its core function is to convert text information in images into editable and searchable electronic text formats. From early template matching methods to modern deep neural network models, the accuracy and efficiency of OCR algorithms have been significantly improved.

[0003] At present, deep learning technology has become the mainstream method in the field of OCR, and many excellent OCR algorithms have emerged, such as PaddleOCR, Tesseract, Transformer OCR, etc. These algorithms use structures such as deep convolutional neural networks (CNN) and recurrent neural networks (RNN) to efficiently and accurately recognize standard texts. For example, PaddleOCR combines technologies such as CRNN (convolutional recurrent neural network) and Attention OCR to achieve text recognition in multiple languages; Tesseract uses LSTM (long short-term memory network) to capture the long-term dependencies of text sequences and improve recognition accuracy.

[0004] Although existing deep learning techniques have made significant progress in standard text recognition, they still have some limitations when dealing with text lines of indefinite length, especially similar characters and complex text structures. Summary of the invention

[0005] The present invention provides a text recognition method, device and equipment based on multi-attribute supervision, which solves the problem in the prior art of how to improve the recognition accuracy of the OCR model on similar characters and complex text structures, and effectively improves the performance of text similar character recognition.

[0006] The present invention provides a text recognition method based on multi-attribute supervision, comprising the following steps: Acquire a text image to be recognized, and preprocess the text image to be recognized; Extract features from the preprocessed text image to be recognized to obtain shared image features; Determine the position features of the characters in the preprocessed text image to be recognized by using a trained position attention decoder; the trained position attention decoder is obtained by supervised learning of the position information of the characters; Inputting the shared image features into a first temporal feature extraction model to obtain a classification result of a dictionary character library; The shared image features and the position features are subjected to feature fusion, and the fused features are input into a second temporal feature extraction model to obtain a classification result of a similar attribute character library.

[0007] According to a text recognition method based on multi-attribute supervision provided by the present invention, a position attention decoder is trained, specifically including: marking the center position of the characters in each line of text image through an image binarization algorithm to obtain the position information of the characters; determining that the structure of the position attention decoder includes multiple convolutional layers and a fully connected layer, and the convolutional layer adopts a spatial attention mechanism; and supervised learning of the position information of the characters through cross loss and segmentation loss functions to obtain a trained attention decoder.

[0008] According to a text recognition method based on multi-attribute supervision provided by the present invention, before recognizing a text image to be recognized, the method further includes: determining the similarity between characters through the fully connected weights of a trained dictionary classification model; dividing the dictionary into several categories of similar attributes according to the similarity; and training the trained dictionary classification model through a Chinese text training set.

[0009] According to a text recognition method based on multi-attribute supervision provided by the present invention, the dictionary classification model is trained, specifically including: training the Chinese text training set through a convolutional neural network and a long short-term memory network structure to obtain a trained dictionary classification model.

[0010] According to a text recognition method based on multi-attribute supervision provided by the present invention, the first time series feature extraction model and the second time series feature extraction model both adopt a recurrent neural network structure to extract time series features from input image features; during the classification process, the probability distribution of the extracted time series features is calculated to obtain the final classification result.

[0011] According to a text recognition method based on multi-attribute supervision provided by the present invention, before recognizing the text image to be recognized, the method further includes: performing cluster analysis on the dictionary through k-means clustering to generate categories of similar attributes.

[0012] The present invention also provides a text recognition device based on multi-attribute supervision, comprising the following modules: A preprocessing module, used for acquiring a text image to be recognized and preprocessing the text image to be recognized; A shared feature extraction module is used to extract features from the preprocessed text image to be recognized to obtain shared image features; A position feature extraction module is used to determine the position features of the characters in the preprocessed text image to be recognized by using a trained position attention decoder; the trained position attention decoder is obtained by supervised learning of the position information of the characters; A dictionary classification module, used for inputting the shared image features into a first temporal feature extraction model to obtain a classification result of a dictionary character library; The similar attribute classification module is used to fuse the shared image features and the position features, input the fused features into the second temporal feature extraction model, and obtain the classification result of the similar attribute character library.

[0013] The present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the text recognition method based on multi-attribute supervision as described above is implemented.

[0014] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements any of the above-described text recognition methods based on multi-attribute supervision.

[0015] The present invention also provides a computer program product, comprising a computer program, wherein when the computer program is executed by a processor, the computer program implements any of the above-mentioned text recognition methods based on multi-attribute supervision.

[0016] The present invention provides a text recognition method, device and equipment based on multi-attribute supervision, including the following beneficial effects: by obtaining the text image to be recognized and performing preprocessing, extracting shared image features, and using the trained position attention decoder to determine the character position features, the shared image features are further used for dictionary character library classification and similar attribute character library classification after fusion with the position features, thereby effectively improving the performance of text similar character recognition, especially when processing similar characters and complex text structures, it can provide more accurate and reliable recognition results. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0018] Figure 1 It is a flowchart of the text recognition method based on multi-attribute supervision provided by the present invention.

[0019] Figure 2It is the structure diagram of the position attention decoder provided by the present invention.

[0020] Figure 3 It is the structure diagram of the similar attribute branch provided by the present invention.

[0021] Figure 4 It is the framework diagram of the training model based on multi-attribute supervision provided by the present invention.

[0022] Figure 5 It is the framework diagram of the inference model provided by the present invention.

[0023] Figure 6 It is the schematic structural diagram of the text recognition device based on multi-attribute supervision provided by the present invention.

[0024] Figure 7 It is the schematic structural diagram of the electronic device provided by the present invention. Detailed implementation manners

[0025] To make the objectives, technical solutions and advantages of the present invention clearer, the technical solutions in the present invention will be clearly and completely described below with reference to the accompanying drawings in the present invention. Obviously, the described embodiments are some but not all of the embodiments of the present invention. All other embodiments obtained by those of ordinary skill in the art based on the embodiments in the present invention without making creative efforts shall fall within the protection scope of the present invention.

[0026] The OCR algorithm has been continuously developed with the progress of computer vision, machine learning and deep learning technologies. From the early template matching method to the modern deep neural network, the algorithm has become more complex and accurate. In the field of automated text recognition, optical character recognition (OCR) technology has been widely used to convert text images into editable text formats. Although existing deep learning technologies (such as deep learning algorithms like PaddleOCR, Tesseract, Transformer OCR, etc.) have made significant progress in standard text recognition, there are still certain limitations in processing variable-length text lines, especially in the recognition of similar characters (such as Chinese characters '日' and '曰', '0' and 'O' for digital and alphabetic confusion) and similar characters with more strokes (such as "鐡" and "鐵"). Affected by the picture quality, this may cause confusion in the OCR model during the recognition process and affect the recognition accuracy.

[0027] Currently, for many recognition errors, the conventional error correction method is to add a natural language model, which corrects the recognized errors or texts with low confidence by learning the semantic information between sentences. Since natural language models generally belong to large models, their inference speed is slow on the CPU, and they cannot correct some texts without much semantics.

[0028] Since existing text recognition algorithms are divided into corresponding categories according to the number of dictionary libraries, the larger the font library is, the more similar characters there are, and the lower the accuracy is. However, although adding a language model can improve the accuracy of text recognition, it is impossible to achieve this in many usage scenarios where only CPU is available. In order to solve the above-mentioned OCR recognition problems, improve the interpretability of the model and reduce the uncertainty of generated content, the present invention constructs a supervised deep learning method of hierarchical multi-attribute categories, increases the category learning of similar attributes, and increases the inter-class distance of numbers and letters; uses a position attention decoder to enhance the attention of characters, which can effectively improve the performance of text similar character recognition. The mask module also supervises the number of characters, and it can provide consistent high-precision recognition results for text lines of different sizes, especially when dealing with similar characters and complex text structures.

[0029] Combine the following Figure 1-Figure 7 Embodiments of the present invention are described in detail.

[0030] Figure 1 is a flowchart of a text recognition method based on multi-attribute supervision provided by the present invention, such as Figure 1 As shown, the method comprises the following steps: S110, obtaining a text image to be recognized, and preprocessing the text image to be recognized.

[0031] Specifically, you can directly obtain photos or images containing text from electronic devices (such as smartphones, digital cameras, or tablets). Make sure that there is sufficient lighting when taking pictures and the text is clearly visible. You can also retrieve stored image files containing text from the network or database.

[0032] The purpose of preprocessing is to improve the accuracy and efficiency of text recognition. If necessary, the acquired color image is converted into a grayscale image to reduce the processing complexity and improve the recognition speed. For some specific recognition algorithms, it may be necessary to further convert the grayscale image into a binary image. Apply filtering algorithms (such as Gaussian filtering, median filtering, etc.) to remove noise and impurities in the image to smooth the image and reduce interference in the recognition process. Enhance the contrast and clarity of the image through contrast stretching, histogram equalization, etc., so that the boundary between the text and the background is clearer. Detect and correct the tilt angle of the image to ensure that the text line is parallel to the edge of the image, thereby improving recognition accuracy. Segment the text area in the image from the background or other non-text areas so that only the area containing the text is processed later. Adjust the size and resolution of the image through interpolation algorithms (such as bilinear interpolation, nearest neighbor interpolation, etc.) to meet the requirements of the recognition algorithm.

[0033] S120 , extracting features from the preprocessed text image to be recognized to obtain shared image features.

[0034] Specifically, after the preprocessing is completed, the optimized text image to be recognized will be subjected to feature extraction to obtain shared image features. The feature extraction model is constructed using deep learning technology, especially convolutional neural network (CNN). The model extracts features of the input image step by step through multiple convolutional layers, and can capture edges, textures, and higher-level abstract features in the image. After each convolution operation, the nonlinear expression capability is increased through activation functions (such as ReLU), and the dimension of the feature map is reduced through pooling operations (such as maximum pooling), while retaining important information.

[0035] During the training process, the weight parameters of the convolutional neural network are continuously optimized through the back-propagation algorithm, so that the model can learn the ability to extract effective features from text images.

[0036] After sufficient training, the model will be able to extract shared image features from preprocessed text images. These shared image features are representative features that can be shared in different recognition tasks. They will be passed to subsequent recognition modules to support a more accurate and efficient text recognition process. In this way, it can ensure that the text recognition system can maintain good performance and accuracy when facing various complex scenarios and font styles.

[0037] S130. Determine the position features of the characters in the preprocessed text image to be recognized through the trained position attention decoder.

[0038] The trained position attention decoder is obtained through supervised learning of the position information of characters.

[0039] According to a text recognition method based on multi-attribute supervision provided by the present invention, a position attention decoder is trained, specifically including: marking the center position of the characters in each line of text image through an image binarization algorithm to obtain the position information of the characters; determining that the structure of the position attention decoder includes multiple convolutional layers and a fully connected layer, and the convolutional layer adopts a spatial attention mechanism; and supervising the learning of the position information of the characters through cross loss and segmentation loss functions to obtain a trained attention decoder.

[0040] Specifically, when training the position attention decoder, we first use an image binarization algorithm (such as common algorithms such as OSTU) to accurately mark the center position of the characters in each line of text image, so as to obtain the position information of the characters. This information will be used as a supervisory signal during decoder training. Next, we design the network structure of the position attention decoder, which mainly consists of multiple convolutional layers and a fully connected layer. In the convolutional layer, the spatial attention mechanism is particularly used to more effectively capture the subtle differences in character positions.

[0041] In order to supervise the learning of the position information of these characters, the cross entropy loss and segmentation loss functions are introduced in the training process. Through the combined effect of these two loss functions, the position attention decoder can gradually learn how to extract accurate character position features from the input text image. After this training process is completed, the decoder can be used in subsequent text recognition tasks to provide accurate character position information, thereby enhancing the performance and accuracy of overall text recognition. In practical applications, the decoder can efficiently and accurately predict the character positions in the text image to be recognized based on its trained weights.

[0042] Specifically, we add a character position attention decoder, use position information supervision to enhance character attention, and learn more features to align text and position in a fine-grained manner, which is more conducive to the regression of the number of words in a text line. The structure of the position attention decoder is shown in the figure below: Figure 2 As shown in the figure, considering that the text gaps in the text lines are small, the position attention decoder mainly refers to the multi-scale prediction method of the key point algorithm PFLD, which consists of two 3*3 convolutional layers (the step size is only for the x direction), one 5*5 convolutional layer (the step size is only for the x direction), and a fully connected FC layer, in which spatial attention is used in the convolutional layer.

[0043] S140: Input the shared image features into a first temporal feature extraction model to obtain a classification result of a dictionary character library.

[0044] Specifically, in the process of text recognition, after the initial image preprocessing and feature extraction, shared image features containing key information of the text image are obtained. Subsequently, these shared image features are input into the first temporal feature extraction model. This model is built on a recurrent neural network (RNN) or its variants (such as LSTM) and has powerful sequence data processing capabilities. Inside the model, the shared image features are processed step by step in the order of time steps or sequences to extract temporal features related to the text character sequence. These temporal features are further used in the classification task of the dictionary character library.

[0045] Specifically, the model will perform dictionary character classification prediction on the input shared image features based on the learned feature representation, and output the dictionary character category to which each character belongs. This process utilizes the classification ability learned by the model during the training phase, enabling the model to accurately map the characters in the text image to the corresponding dictionary character category, thus providing a reliable foundation for subsequent text recognition.

[0046] S150, performing feature fusion on the shared image features and the position features, inputting the fused features into a second temporal feature extraction model, and obtaining a classification result of a similar attribute character library.

[0047] According to a text recognition method based on multi-attribute supervision provided by the present invention, before recognizing a text image to be recognized, the similarity between characters is determined by the fully connected weights of a trained dictionary classification model; the dictionary is divided into several categories of similar attributes according to the similarity; and the trained dictionary classification model is trained using a Chinese text training set.

[0048] Specifically, before recognizing the text image to be recognized, the present invention first uses a trained dictionary classification model to determine the similarity between characters. This step is to calculate the similarity value between characters by using a weight matrix through the calculation of the fully connected layer of the characters in the dictionary, usually using a measurement method such as cosine similarity. Based on the calculated similarity, the dictionary is divided into several categories with similar attributes, such as numbers, letters, Chinese characters, etc. Chinese characters can also be subdivided into smaller categories based on further similarity. The trained dictionary classification model is trained with a large number of Chinese text training sets, ensuring the accuracy and generalization ability of the model in character classification.

[0049] Specifically, similar attribute classification is added, and the dictionary library is divided into multiple categories, such as numbers, letters, and similar Chinese characters. The similarity of similar Chinese characters is calculated according to the trained dictionary classification model, or the category weights of Chinese characters are divided into several categories by k-mean, which are called similar attribute categories in this article. This method can increase the inter-class distance of numbers and letters and reduce the interference of structural dissimilarities; the similarity calculation adopts the cosine similarity method. The cosine similarity uses the theory that the smaller the vector angle, the higher the similarity. By calculating the cosine value of the angle between two vectors a and b, the specific formula is as follows: In the process of text recognition, after obtaining the shared image features and the position features of the characters, the system will first perform a feature fusion step. The purpose of this step is to combine the global information of the image with the specific position information of the characters to enhance the local feature expression ability of the model. Feature fusion is usually achieved through convolution operations or other feature fusion techniques, merging the shared image features and position features in the spatial dimension to form a fused feature map. Subsequently, these fused features are input into the second temporal feature extraction model, which is also based on a recurrent neural network (RNN) or its improved version (such as LSTM or multi-attention mechanism module) to further extract temporal features related to similar attribute word libraries. Based on these features, the model classifies and predicts the similar attribute word library for the fused features, and finally outputs the similar attribute category to which each character belongs. This process makes full use of the complementarity of position features and global image features, and improves the model's ability to distinguish similar characters.

[0050] like Figure 3The diagram shows the branch structure of similar attributes, similar attribute supervision training, using position attention features and global shared features for adaptive fusion, enhancing local character features, reducing background information interference, and extracting and classifying the fused features for temporal features. The present invention upsamples the position multi-scale feature p and fuses it with the global shared feature f through convolution. The fused features are input into the temporal feature module (LSTM module or multi-attention mechanism module) to extract the temporal features, and then two full connections are performed for similar attribute classification. Experiments have verified that two full connections can improve accuracy.

[0051] By building a classification system based on similar attributes, the present invention can more effectively handle the recognition problem of similar characters, especially in the case of dealing with Chinese characters with mixed numbers and letters and complex strokes, which significantly improves the recognition accuracy. Classifying dictionaries according to similar attributes not only improves the recognition effect, but also makes the decision-making process of the model more transparent, which is helpful for error analysis and model tuning in practical applications.

[0052] According to a text recognition method based on multi-attribute supervision provided by the present invention, a dictionary classification model is trained, specifically including: training a Chinese text training set through a convolutional neural network and a long short-term memory network structure to obtain a trained dictionary classification model.

[0053] Specifically, first prepare a sufficient Chinese text training set, which should contain diverse text samples to cover all possible characters and character combinations. Then, use a convolutional neural network (CNN) to perform preliminary feature extraction on the text image. CNN can effectively extract hierarchical feature representations from the original image. Subsequently, these features are passed to the long short-term memory network (LSTM) structure. As a special recurrent neural network (RNN), LSTM is good at processing sequence data and can capture contextual dependencies between characters. By combining the advantages of CNN and LSTM, the model can learn the accurate representation of characters in text images and the temporal relationship between characters. During the training process, the back propagation algorithm and an appropriate loss function (such as cross entropy loss) are used to optimize the model weights until the model achieves satisfactory performance on the validation set, thereby obtaining a trained dictionary classification model.

[0054] By combining the structure of convolutional neural networks and long short-term memory networks, the model can learn the deep features of text images and the temporal dependencies between characters, thereby significantly improving the accuracy and robustness of dictionary classification. This structure enables the model to show stronger differentiation ability when processing complex texts, especially texts containing similar characters. Training with a rich Chinese text training set helps the model learn more diverse character features and text patterns, thereby enhancing the model's generalization ability to new text samples. This means that the model can still maintain good recognition performance when faced with unknown or unseen text.

[0055] The trained dictionary classification model can not only be directly used for character classification tasks, but also provide a reliable initial feature representation for subsequent multi-attribute supervised learning. In the hierarchical multi-attribute supervised training framework, the output of the dictionary classification model can be used as the input for steps such as similar attribute classification and position attention decoding, further improving the accuracy and efficiency of the entire text recognition system.

[0056] According to a text recognition method based on multi-attribute supervision provided by the present invention, the first temporal feature extraction model and the second temporal feature extraction model both adopt a recurrent neural network structure to extract temporal features of input image features; during the classification process, the probability distribution of the extracted temporal features is calculated to obtain the final classification result.

[0057] Specifically, the first temporal feature extraction model and the second temporal feature extraction model both adopt the structure of a recurrent neural network (RNN) or its improved version (such as a long short-term memory network LSTM or a multi-attention mechanism network) to effectively extract temporal features from input image features. In specific implementation, the shared image features and the fused features (including position information) are respectively fed into the first temporal feature extraction model and the second temporal feature extraction model. These models can capture the temporal dependencies and contextual information in image features through their internal recurrent connection mechanisms, thereby generating more expressive temporal feature representations. During the classification process, the model will further process the extracted temporal features, and calculate the probability distribution of each character or character combination belonging to each category through structures such as a fully connected layer or a softmax layer. Finally, the classification result of each character or character combination is determined based on the maximum value of the probability distribution or the set threshold.

[0058] According to a text recognition method based on multi-attribute supervision provided by the present invention, before recognizing a text image to be recognized, a dictionary is clustered and analyzed by k-means clustering to generate categories of similar attributes.

[0059] Specifically, in the text recognition method based on multi-attribute supervision, in order to further improve the recognition capability of similar characters, the present invention further includes a step of clustering the dictionary by k-means clustering before recognizing the text image to be recognized.

[0060] First, the fully connected weights of the trained dictionary classification model are used to calculate the similarity between each character in the dictionary to obtain a similarity matrix. Then, the k-means clustering algorithm is used to perform cluster analysis on this similarity matrix, dividing them into several categories according to the similarity between the characters. These categories represent different similarity attributes.

[0061] For example, numbers, letters, and Chinese characters can be clustered into different categories. For Chinese characters, they can be further subdivided according to their strokes, structure, and other features to form more detailed similar attribute categories. During the clustering process, the k-means algorithm will continuously iterate and update the cluster center until the convergence condition is reached, and finally a stable similar attribute category division is obtained.

[0062] By clustering the dictionary using k-means clustering, similar characters can be automatically grouped into the same category, thereby enhancing the model's ability to distinguish similar characters. This helps reduce errors caused by confusion between similar characters during text recognition and improves overall recognition accuracy.

[0063] The present invention mainly improves the accuracy of similar characters through hierarchical supervised learning. The specific steps are as follows: (1) Use the conventional convolutional neural network (CRNN) and long short-term memory (LSTM) structure to train a dictionary classification model in advance; (2) Using the trained dictionary classification model, calculate the similarity between each character through the fully connected weights, and divide the dictionary into several similar character libraries according to the similarity (other clustering methods, such as k-mean clustering, can also be used). Manually make minor adjustments to the misclassified characters, with numbers and letters in one category each, to form similar attribute categories; (3) Using common image binarization algorithms such as Osthu, mark the center position of the characters in each line of the text image to obtain position information; (4) A position attention decoder module designed according to the present invention uses cross entropy loss and segmentation loss functions to supervise the learning of character position information; (5) According to the hierarchical multi-attribute supervised training model framework designed by the present invention, Figure 4 As shown, the features of the position attention decoder are fused with the shared image features to enhance the local feature expression capability, and the fused features are input into the temporal feature extraction module 2 (RNN2) to classify the similar attribute word library; (6) According to the hierarchical multi-attribute supervised training model framework designed by the present invention, shared features are input into the time series feature extraction model 1 (RNN1) to perform classification regression on the character library; (7) In the inference stage, the position attention decoder and similar attributes do not need to participate in the calculation, and only the classification results of the dictionary word library and the similar attribute word library are output. The inference model is as follows: Figure 5 shown.

[0064] The present invention proposes an efficient classification algorithm based on multi-attribute supervision, which can improve the text recognition accuracy without increasing the reasoning speed. The specific implementation method of network training reasoning in this method is as follows: (1) Organize the Chinese text training set, train the dictionary classification branch separately, and save the weights; (2) Prepare similar attribute categories and divide the dictionary into four categories {symbols, numbers, letters, and Chinese characters}. Chinese characters can be further subdivided according to similarity. Use the fully connected weights trained in (1) to cross-check the dictionary, merge similar characters based on confidence, and manually adjust the misclassified characters. (3) Prepare character position labels. The present invention uses an open source image binarization method to obtain the center position of each character, but is not limited to other methods (instance segmentation or semantic segmentation); (4) Training is performed on the framework based on multi-attribute supervision of the present invention, learning shared features, using the shared features for character position feature and time series feature extraction, classifying similar attribute categories according to the time series features, and calculating their CTC loss; using multi-scale character position features to locate the character center point, calculating the L1 loss and cross entropy loss, adding all losses according to the loss coefficient to obtain the model loss, and using the model loss for supervised training; (5) During inference, only the dictionary classification branch is used to ensure that the text recognition accuracy is improved without increasing parameters.

[0065] In order to prove the feasibility and effectiveness of the present invention, a specific example is described in combination with the address text in the actual ID card image: (1) Obtain an ID card scene image and use the existing card detection model to detect the ID card image; (2) Use the direction model to classify the card image, output the direction category of the card image, align and rotate it; (3) Using the existing text detection algorithm DBnet, perform text detection on the rotated card image and obtain the text line coordinates; (4) Use perspective transformation to crop text line images and obtain text images of all characters; (5) Normalize the text image and scale it to 48*480; (6) The normalized text image is input into the efficient text recognition model based on multi-attribute supervision of the present invention, and reasoning is performed to output the predicted text and average confidence. Through result analysis, this method effectively avoids the similarity between numbers and letters, and also learns the structural features of similar characters.

[0066] In actual tests, the present invention has significantly improved the recognition accuracy of numbers and letters, especially the recognition of some numbers, such as characters [O, 0, Z, 2]. At the same time, it also strengthens the structural characteristics of similar characters, uses position attention supervised learning, reduces background interference, and reduces the phenomenon of missed recognition and multiple recognition of characters. The overall accuracy on the test set is improved by 4.5%.

[0067] The text recognition device based on multi-attribute supervision provided by the present invention is described below. The text recognition device based on multi-attribute supervision described below and the text recognition method based on multi-attribute supervision described above can be referred to each other.

[0068] like Figure 6 The present invention provides a text recognition device based on multi-attribute supervision, comprising: A preprocessing module 610 is used to obtain a text image to be recognized and preprocess the text image to be recognized; A shared feature extraction module 620 is used to extract features from the preprocessed text image to be recognized to obtain shared image features; The position feature extraction module 630 is used to determine the position features of the characters in the preprocessed text image to be recognized by using a trained position attention decoder; the trained position attention decoder is obtained by supervised learning of the position information of the characters; A dictionary classification module 640 is used to input the shared image features into the first temporal feature extraction model to obtain a classification result of the dictionary character library; The similar attribute classification module 650 is used to fuse the shared image features and the position features, input the fused features into the second temporal feature extraction model, and obtain the classification result of the similar attribute character library.

[0069] Figure 7 An example of a physical structure diagram of an electronic device is shown in FIG. Figure 7As shown, the electronic device may include: a processor 710, a communication interface 720, a memory 730 and a communication bus 740, wherein the processor 710, the communication interface 720 and the memory 730 communicate with each other through the communication bus 740. The processor 710 may call the logic instructions in the memory 730 to execute a text recognition method based on multi-attribute supervision, the method comprising: obtaining a text image to be recognized, preprocessing the text image to be recognized; extracting features of the preprocessed text image to be recognized to obtain shared image features; determining the position features of the characters in the preprocessed text image to be recognized by a trained position attention decoder; the trained position attention decoder is obtained by supervised learning of the position information of the characters; inputting the shared image features into a first temporal feature extraction model to obtain the classification results of the dictionary character library; performing feature fusion on the shared image features and the position features, inputting the fused features into a second temporal feature extraction model to obtain the classification results of the similar attribute character library.

[0070] In addition, the logic instructions in the above-mentioned memory 730 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on this understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art or the part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods of each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program codes.

[0071] On the other hand, the present invention also provides a computer program product, which includes a computer program. The computer program can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the text recognition method based on multi-attribute supervision provided by the above methods, and the method includes: obtaining a text image to be recognized and preprocessing the text image to be recognized; extracting features from the preprocessed text image to be recognized to obtain shared image features; determining the position features of characters in the preprocessed text image to be recognized through a trained position attention decoder; the trained position attention decoder is obtained through supervised learning of the position information of the characters; inputting the shared image features into a first temporal feature extraction model to obtain the classification results of a dictionary character library; performing feature fusion on the shared image features and the position features, and inputting the fused features into a second temporal feature extraction model to obtain the classification results of a similar attribute character library.

[0072] On the other hand, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, is implemented to execute the text recognition method based on multi-attribute supervision provided by the above-mentioned methods, the method comprising: obtaining a text image to be recognized, and preprocessing the text image to be recognized; performing feature extraction on the preprocessed text image to be recognized to obtain shared image features; determining the position features of the characters in the preprocessed text image to be recognized by a trained position attention decoder; the trained position attention decoder is obtained by supervised learning of the position information of the characters; the shared image features are input into a first temporal feature extraction model to obtain the classification results of a dictionary character library; the shared image features and the position features are feature fused, and the fused features are input into a second temporal feature extraction model to obtain the classification results of a similar attribute character library.

[0073] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, i.e., they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Those of ordinary skill in the art may understand and implement it without creative effort.

[0074] Through the description of the above implementation modes, those skilled in the art can clearly understand that each implementation mode can be implemented by means of software plus a necessary general hardware platform, or of course by hardware. Based on such an understanding, the above technical solution can essentially or in other words be embodied in the form of a software product that contributes to the prior art. The computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a magnetic disk, an optical disk, etc., and includes a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods of each embodiment or some parts of the embodiment.

[0075] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A text recognition method based on multi-attribute supervision, characterized in that: include: Acquire a text image to be recognized, and preprocess the text image to be recognized; Extract features from the preprocessed text image to be recognized to obtain shared image features; Determine the position features of the characters in the preprocessed text image to be recognized by using a trained position attention decoder; the trained position attention decoder is obtained by supervised learning of the position information of the characters; Inputting the shared image features into a first temporal feature extraction model to obtain a classification result of a dictionary character library; The shared image features and the position features are subjected to feature fusion, and the fused features are input into a second temporal feature extraction model to obtain a classification result of a similar attribute character library.

2. The text recognition method based on multi-attribute supervision according to claim 1, characterized in that: Train the position attention decoder, including: The center position of the characters in each line of text image is marked by an image binarization algorithm to obtain the position information of the characters; The structure of the position attention decoder is determined to include multiple convolutional layers and a fully connected layer, wherein the convolutional layer adopts a spatial attention mechanism; The position information of the characters is supervised and learned through cross loss and segmentation loss functions to obtain a trained attention decoder.

3. The text recognition method based on multi-attribute supervision according to claim 1, characterized in that: Before recognizing the text image to be recognized, the method further includes: Determine the similarity between characters through the fully connected weights of the trained dictionary classification model; The dictionary is divided into several categories with similar attributes according to the similarity; the trained dictionary classification model is trained by using a Chinese text training set.

4. The text recognition method based on multi-attribute supervision according to claim 3 is characterized in that: Training the dictionary classification model specifically includes: The Chinese text training set is trained through convolutional neural network and long short-term memory network structure to obtain a trained dictionary classification model.

5. The text recognition method based on multi-attribute supervision according to claim 1, characterized in that: The first temporal feature extraction model and the second temporal feature extraction model both adopt a recurrent neural network structure to extract temporal features from input image features; During the classification process, the probability distribution of the extracted time series features is calculated to obtain the final classification result.

6. The text recognition method based on multi-attribute supervision according to claim 1, characterized in that: Before recognizing the text image to be recognized, the method further includes: The dictionary is clustered by k-means clustering to generate categories with similar attributes.

7. A text recognition device based on multi-attribute supervision, characterized in that: include: A preprocessing module, used for acquiring a text image to be recognized and preprocessing the text image to be recognized; A shared feature extraction module is used to extract features from the preprocessed text image to be recognized to obtain shared image features; A position feature extraction module is used to determine the position features of the characters in the preprocessed text image to be recognized by using a trained position attention decoder; the trained position attention decoder is obtained by supervised learning of the position information of the characters; A dictionary classification module, used for inputting the shared image features into a first temporal feature extraction model to obtain a classification result of a dictionary character library; The similar attribute classification module is used to fuse the shared image features and the position features, input the fused features into the second temporal feature extraction model, and obtain the classification result of the similar attribute character library.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the text recognition method based on multi-attribute supervision as described in any one of claims 1 to 6 is implemented.

9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the text recognition method based on multi-attribute supervision as claimed in any one of claims 1 to 6 is implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the text recognition method based on multi-attribute supervision as claimed in any one of claims 1 to 6 is implemented.

Images