Text recognition method and device, storage medium and electronic equipment
Through a multi-level text recognition model, including feature extraction, border regression and semantic segmentation, the problem of low accuracy of irregular text recognition in the prior art is solved, and efficient recognition of curved text is achieved.
Patent Information
- Application Number
- CN202510436128.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-04-08
- Publication Date
- 2025-08-12
AI Technical Summary
Existing text recognition methods have low accuracy when identifying irregular text, especially when bending text, and existing end-to-end methods are complex in training and fail to fully utilize the complementary advantages of text detection models and text recognition models.
A multi-level text recognition model is adopted, including feature extraction sub-model, text box determination sub-model, text detection sub-model and text recognition sub-model. Through feature extraction, border regression and semantic segmentation, the target text box is gradually determined and recognized.
The recognition accuracy of irregular texts is improved, especially the recognition effect of curved texts. Through the collaborative work of multi-level models, efficient recognition of irregular texts is achieved.
Smart Images
Figure CN120472466A_ABST
Abstract
Description
Technical Field
[0001] The present disclosure relates to the field of image processing technology, and in particular to a text recognition method, device, storage medium, and electronic device. Background Art
[0002] Common text recognition methods can be categorized as printed text and handwritten text. Handwritten text recognition, in addition to facing various challenges with printed text, is also affected by handwriting style. In some application scenarios, the text images to be recognized are not always straight and horizontal. For example, in intelligent test recording, if the user-uploaded image contains irregular text, intelligent recording and formatting requires recognition of the irregular text, thereby improving the accuracy of test paper restoration.
[0003] Related technologies typically require training two models, optimizing the text detection model and the text recognition model separately, failing to fully leverage the highly correlated and complementary strengths of the two. Existing end-to-end approaches focus only on horizontal and directional text recognition, resulting in lower accuracy for irregular text such as curved shapes. Summary of the Invention
[0004] The present invention aims to provide a text recognition method, device, storage medium and electronic device for improving the accuracy of text recognition.
[0005] According to a first aspect of an embodiment of the present disclosure, a text recognition method is provided, the method comprising: Obtaining a text image containing target text to be recognized; Inputting the text image into a pre-trained text recognition model to obtain a text recognition result of the target text output by the text recognition model; The text recognition model is used to obtain the feature image of the text image, determine a pending text box based on the feature image, perform border regression on the pending text box to obtain a target text box, and perform semantic segmentation on the image corresponding to the target text box to obtain a text recognition result, wherein the text box is used to determine the text area of the target text in the feature image.
[0006] Optionally, the text recognition model includes a feature extraction sub-model, a text box determination sub-model, a text detection sub-model, and a text recognition sub-model, wherein the output of the feature extraction sub-model is coupled to the input of the text box determination sub-model, the output of the text box determination sub-model is coupled to the input of the text detection sub-model, and the output of the text detection sub-model is coupled to the input of the text recognition sub-model; the text recognition model is used to: Extracting image features of the text image through the feature extraction sub-model to obtain the feature image, and inputting the feature image into the text box determination sub-model; Determine the pending text box according to the feature image using the text box determination sub-model; Adjusting the size of the pending text box to a target size; Performing bounding box regression on the adjusted pending text box through the text detection sub-model to obtain a target text box; and The text recognition sub-model is used to perform semantic segmentation on the image corresponding to the target text box to obtain a text recognition result.
[0007] Optionally, the text recognition model is trained in the following manner: Acquire training sample data, wherein the training sample data includes text image samples and sample recognition results corresponding to the text image samples; The preset training model is trained using the training sample data to obtain the text recognition model.
[0008] Optionally, the training a preset training model using the training sample data to obtain the text recognition model includes: Using the text image sample as input of the feature extraction sub-model and the sample feature image as output of the text detection sub-model to train the feature extraction sub-model; The output of the feature extraction sub-model is used as the input of the text box determination sub-model, and the sample undetermined text box is used as the output of the text box determination sub-model to train the text box determination sub-model; The output of the text box determination sub-model is used as the input of the text detection sub-model, and the sample detection result is used as the output of the text detection sub-model to train the text detection sub-model; The output of the text box determination sub-model is used as the input of the text recognition sub-model, and the sample recognition result is used as the output of the text recognition sub-model to train the text recognition sub-model.
[0009] Optionally, the text box determination sub-model is trained in the following manner: Acquire training sample sub-data, wherein the training sample sub-data includes a sample feature image and a sample pending text box corresponding to the sample image feature; The preset training sub-model is trained using the training sample sub-data to obtain the text box determination sub-model.
[0010] Optionally, the training of a preset training sub-model using the training sample sub-data to obtain the text box determination sub-model includes: For each pixel point in each of the sample feature images, setting multiple preset anchor frames; The anchor box containing the target text is regarded as the positive anchor box, and the anchor box not containing the target text is regarded as the negative anchor box; Randomly select a preset number of target positive anchor frames from the positive anchor frame, and randomly select a preset number of target negative anchor frames from the negative anchor frame; The preset training sub-model is trained using the target positive anchor frame, the target negative anchor frame, and the sample pending text frame to obtain the text frame determination sub-model.
[0011] According to a second aspect of an embodiment of the present disclosure, a text recognition device is provided, the device comprising: An acquisition module, used for acquiring a text image containing a target text to be recognized; A recognition module, configured to input the text image into a pre-trained text recognition model and obtain a text recognition result of the target text output by the text recognition model; The text recognition model is used to obtain the feature image of the text image, determine a pending text box based on the feature image, perform border regression on the pending text box to obtain a target text box, and perform semantic segmentation on the image corresponding to the target text box to obtain a text recognition result, wherein the text box is used to determine the text area of the target text in the feature image.
[0012] Optionally, the text recognition model includes a feature extraction sub-model, a text box determination sub-model, a text detection sub-model, and a text recognition sub-model, wherein the output of the feature extraction sub-model is coupled to the input of the text box determination sub-model, the output of the text box determination sub-model is coupled to the input of the text detection sub-model, and the output of the text detection sub-model is coupled to the input of the text recognition sub-model; the text recognition model is used to: Extracting image features of the text image through the feature extraction sub-model to obtain the feature image, and inputting the feature image into the text box determination sub-model; Determine the pending text box according to the feature image using the text box determination sub-model; Adjusting the size of the pending text box to a target size; Performing bounding box regression on the adjusted pending text box through the text detection sub-model to obtain a target text box; and The text recognition sub-model is used to perform semantic segmentation on the image corresponding to the target text box to obtain a text recognition result.
[0013] Optionally, the text recognition model is trained in the following manner: Acquire training sample data, wherein the training sample data includes text image samples and sample recognition results corresponding to the text image samples; The preset training model is trained using the training sample data to obtain the text recognition model.
[0014] Optionally, the training a preset training model using the training sample data to obtain the text recognition model includes: Using the text image sample as input of the feature extraction sub-model and the sample feature image as output of the text detection sub-model to train the feature extraction sub-model; The output of the feature extraction sub-model is used as the input of the text box determination sub-model, and the sample undetermined text box is used as the output of the text box determination sub-model to train the text box determination sub-model; The output of the text box determination sub-model is used as the input of the text detection sub-model, and the sample detection result is used as the output of the text detection sub-model to train the text detection sub-model; The output of the text box determination sub-model is used as the input of the text recognition sub-model, and the sample recognition result is used as the output of the text recognition sub-model to train the text recognition sub-model.
[0015] Optionally, the text box determination sub-model is trained in the following manner: Acquire training sample sub-data, wherein the training sample sub-data includes a sample feature image and a sample pending text box corresponding to the sample image feature; The preset training sub-model is trained using the training sample sub-data to obtain the text box determination sub-model.
[0016] Optionally, the training of the preset training sub-model using the training sample sub-data to obtain the text box determination sub-model includes: For each pixel point in each of the sample feature images, setting multiple preset anchor frames; The anchor box containing the target text is regarded as the positive anchor box, and the anchor box not containing the target text is regarded as the negative anchor box; Randomly select a preset number of target positive anchor frames from the positive anchor frame, and randomly select a preset number of target negative anchor frames from the negative anchor frame; The preset training sub-model is trained using the target positive anchor frame, the target negative anchor frame, and the sample pending text frame to obtain the text frame determination sub-model.
[0017] According to a third aspect of an embodiment of the present disclosure, a non-temporary computer-readable storage medium is provided, on which a computer program is stored, which, when executed by a processor, implements the steps of any one of the methods described in the first aspect of the present disclosure.
[0018] According to a fourth aspect of the embodiments of the present disclosure, there is provided an electronic device, including: a memory having a computer program stored thereon; A processor is used to execute the computer program in the memory to implement the steps of any one of the methods described in the first aspect of the present disclosure.
[0019] Through the above technical solution, the present disclosure first obtains a text image containing the target text to be recognized. The text image is then input into a pre-trained text recognition model to obtain a text recognition result of the target text output by the text recognition model. Among them, the text recognition model is used to obtain a feature image of the text image, determine the pending text box according to the feature image, perform border regression on the pending text box to obtain a target text box, and perform semantic segmentation on the image corresponding to the target text box to obtain a text recognition result, wherein the text box is used to determine the text area of the target text in the feature image. The present disclosure first determines the target text box containing the target text to be recognized according to the feature image corresponding to the text image, and performs semantic segmentation on the target text box to obtain a text recognition result, which can accurately identify irregular text and improve the accuracy of text recognition.
[0020] Other features and advantages of the present disclosure will be described in detail in the following detailed description. BRIEF DESCRIPTION OF THE DRAWINGS
[0021] The accompanying drawings are used to provide a further understanding of the present disclosure and constitute a part of the specification. Together with the following detailed description, they are used to explain the present disclosure but do not constitute a limitation of the present disclosure. In the accompanying drawings: Figure 1 The present invention is a flowchart showing a text recognition method according to an exemplary embodiment.
[0022] Figure 2 The present invention is a flowchart illustrating a method for training a text recognition model according to an exemplary embodiment.
[0023] Figure 3 The present invention is a flowchart illustrating another method for training a text recognition model according to an exemplary embodiment.
[0024] Figure 4 The figure is a block diagram of a text recognition device according to an exemplary embodiment.
[0025] Figure 5 It is a block diagram of an electronic device according to an exemplary embodiment. DETAILED DESCRIPTION
[0026] The following describes the specific embodiments of the present disclosure in detail with reference to the accompanying drawings. It should be understood that the specific embodiments described herein are only used to illustrate and explain the present disclosure and are not intended to limit the present disclosure.
[0027] It should be noted that all actions of acquiring signals, information or data in the present disclosure are carried out in compliance with the corresponding data protection laws and policies of the country where they are located and with the authorization given by the owner of the corresponding device.
[0028] Before introducing the text recognition method, device, storage medium, and electronic device provided by the present disclosure, the application scenarios involved in each embodiment of the present disclosure are first introduced. With the advent of the intelligent era, the application scenarios of irregular text (such as handwritten text recognition and irregular printed text recognition) are becoming more and more numerous. For example, in educational scenarios, essay grading, automatic test grading, test paper entry, etc., and in business applications, handwritten receipt recognition, card recognition, signature recognition, etc., all require the recognition of handwritten text images. Currently, common text recognition models can often accurately recognize standard printed text, but for irregular printed text or handwritten text, common text recognition models are often unable to accurately recognize.
[0029] Currently, regular text recognition is typically performed using one-stage and end-to-end approaches. The one-stage approach requires separate training and optimization of the text detection and recognition models, failing to fully leverage the highly correlated and complementary nature of these two models. Existing end-to-end approaches, on the other hand, have complex and inefficient training processes and focus only on horizontal and directional text recognition, making it difficult to accurately recognize irregular text (such as curved text).
[0030] Figure 1 is a flowchart illustrating a text recognition method according to an exemplary embodiment. Figure 1 As shown, the method may include: Step 101: Acquire a text image containing target text to be recognized.
[0031] The text image to be recognized may be an image containing handwritten text, such as an image obtained by photographing or scanning a test paper, homework, or receipt. It may also be an image containing printed text, such as a store name or artistic text captured in a natural scene.
[0032] Step 102: Input the text image into a pre-trained text recognition model to obtain a text recognition result of the target text output by the text recognition model.
[0033] Among them, the text recognition model is used to obtain the feature image of the text image, determine the pending text box based on the feature image, perform border regression on the pending text box to obtain the target text box, and perform semantic segmentation on the image corresponding to the target text box to obtain the text recognition result, wherein the text box is used to determine the text area of the target text in the feature image.
[0034] For example, after obtaining a text image containing the target text to be recognized, the text image can be input into a pre-trained text recognition model. The text recognition model can then process the text image and output a text recognition result for the target text. Specifically, the text recognition model can first extract features from the text image to obtain multiple feature images of different sizes. For example, feature images can be obtained using network models such as GoogLeNet and ResNet. The target text box can then be obtained from the feature image. For example, the target text box can be obtained using Faster RCNN or RPN. In some embodiments, the size of the target text box can be fixed through ROI Alignment or ROI Pooling to facilitate subsequent text detection and recognition. Bounding box regression can then be performed on the target text box to obtain a more accurate target text box. The text box is used to determine the text area of the target text in the feature image. Finally, semantic segmentation can be performed on the image corresponding to the target text box to obtain the text recognition result. This allows accurate recognition of irregular text (such as curved text).
[0035] In summary, the present disclosure first obtains a text image containing the target text to be recognized. The text image is then input into a pre-trained text recognition model to obtain a text recognition result of the target text output by the text recognition model. The text recognition model is used to obtain a feature image of the text image, determine a pending text box based on the feature image, perform border regression on the pending text box to obtain a target text box, and perform semantic segmentation on the image corresponding to the target text box to obtain a text recognition result, wherein the text box is used to determine the text area of the target text in the feature image. The present disclosure first determines the target text box containing the target text to be recognized based on the feature image corresponding to the text image, and performs semantic segmentation on the target text box to obtain a text recognition result, which can accurately identify irregular text and improve the accuracy of text recognition.
[0036] In one application scenario, the text recognition model includes a feature extraction sub-model, a text box determination sub-model, a text detection sub-model, and a text recognition sub-model. The output of the feature extraction sub-model is coupled to the input of the text box determination sub-model, the output of the text box determination sub-model is coupled to the input of the text detection sub-model, and the output of the text detection sub-model is coupled to the input of the text recognition sub-model. The text recognition model is used to: The image features of the text image are extracted through the feature extraction sub-model to obtain a feature image, and the feature image is input into the text box to determine the sub-model.
[0037] For example, the text recognition model may include a feature extraction sub-model, a text box determination sub-model, a text detection sub-model, and a text recognition sub-model. After a text image containing the target text to be recognized is input into the text recognition model, the image features of the text image may be extracted by the feature extraction sub-model to obtain a feature image. The feature extraction sub-model may include, for example, ResNet50 and FPN (Feature Pyramid Network). Figure 3 As shown in the figure, ResNet50 extracts features from text images, and the output feature maps C1, C2, C3, C4, C5, and C6 are sized 1 / 2, 1 / 4, 1 / 8, 1 / 16, 1 / 32, and 1 / 64 of the text image, respectively. FPN processes the feature maps output by ResNet50 to obtain fused feature maps P2, P3, P4, P5, and P6, the sizes of which are 1 / 4, 1 / 8, 1 / 16, 1 / 32, and 1 / 64 of the text image, respectively. Among them, the fused feature map P2 contains the richest image detail features, and the fused feature map P6 contains the richest image semantic features. Since feature maps of different sizes have different sensitivities to text of different sizes, for example, P2 is more likely to detect smaller text, while P6 is more likely to detect larger text, this is conducive to detecting text regions of different sizes.
[0038] The text box determination sub-model determines the pending text box according to the feature image.
[0039] Adjust the size of the pending text box to the target size.
[0040] For example, the feature extraction sub-model can input the obtained feature image into the text box determination sub-model. The text box determination sub-model can then determine the candidate text box based on the feature image. The candidate text box can be understood as the region within the feature image where the text is located. For example, the text box determination sub-model can be an RPN (Region Proposal Network). Taking the RPN as an example, the RPN essentially sets a dense array of candidate anchor boxes at the original image scale and then determines which anchor boxes are positive anchor boxes (i.e., containing objects) and which are negative anchor boxes (i.e., not containing objects). This is a binary classification problem. After the text box determination sub-model outputs the candidate text box, a preset bounding box regression algorithm is used to regress the adjusted candidate text box, thereby correcting the detection box position and obtaining a more accurate candidate text box. For example, the bounding box regression algorithm can be Bounding Box Regression.
[0041] The adjusted pending text box is regressed through the text detection sub-model to obtain the target text box. And, The text recognition sub-model is used to perform semantic segmentation on the image corresponding to the target text box to obtain the text recognition result.
[0042] For example, the positioning of the pending text box obtained by the feature extraction sub-model is still not accurate enough, so the text detection sub-model can be further used to use the preset border regression algorithm to perform border regression on the adjusted pending text box again, so as to obtain a more accurate target text box.
[0043] In some embodiments, the target text box output by the text detection sub-model can be input into the text recognition sub-model, and the text recognition sub-model performs semantic segmentation on the image corresponding to the target text box to obtain the text recognition result. Among them, the text recognition result may include a global text instance segmentation map (Global word map), a single character segmentation map (Character map) and a background segmentation map (Background map). The global text instance segmentation map can be understood as the area where all the words extracted from the text image are located, the single character segmentation map can be understood as each word extracted from the text image, and the background segmentation map can be understood as the area in the text image excluding the area where the words are located. In this way, irregular text (such as curved text) can also be accurately recognized.
[0044] Figure 2 is a flowchart illustrating a method for training a text recognition model according to an exemplary embodiment. Figure 2 As shown, the method may include: Step 201: Acquire training sample data, where the training sample data includes text image samples and sample recognition results corresponding to the text image samples.
[0045] Step 202: Train the preset training model using training sample data to obtain a text recognition model.
[0046] For example, when training a text recognition model, multiple text image samples can be used as input to the text recognition model, and multiple sample recognition results corresponding to the multiple text image samples can be used as the output of the recognition model. This allows the model to be trained so that, when multiple text image samples are input, the output of the recognition model matches the multiple sample recognition results. For example, a loss can be determined based on the output of the text recognition model and the multiple text image samples. With the goal of reducing this loss, a backpropagation algorithm can be used to modify the neuron parameters in the text recognition model. Neuron parameters can include, for example, neuron weights and biases. The above steps are repeated until the loss meets a preset condition, such as being less than a preset loss threshold, thereby achieving the purpose of training the recognition model.
[0047] Figure 3 is a flowchart illustrating another method for training a text recognition model according to an exemplary embodiment. Figure 3 As shown, step 202 can be implemented by the following steps: Step 2021: Use the text image sample as the input of the feature extraction sub-model and use the sample feature image as the output of the text detection sub-model to train the feature extraction sub-model.
[0048] In step 2022, the output of the feature extraction sub-model is used as the input of the text box determination sub-model, and the sample pending text box is used as the output of the text box determination sub-model to train the text box determination sub-model.
[0049] In step 2023, the output of the text box determination sub-model is used as the input of the text detection sub-model, and the sample detection result is used as the output of the text detection sub-model to train the text detection sub-model.
[0050] In step 2024, the output of the text box determination sub-model is used as the input of the text recognition sub-model, and the sample recognition result is used as the output of the text recognition sub-model to train the text recognition sub-model.
[0051] For example, the text image samples in the training sample data can be used as the input of the feature extraction sub-model, and the sample feature images can be used as the output of the text detection sub-model to train the feature extraction sub-model. The feature images output by the feature extraction sub-model can be used as the input of the text box determination sub-module, and the sample undetermined text boxes can be used as the output of the text box determination sub-model to train the text detection sub-model. Thereafter, the outputs of the text detection sub-model can be used as the inputs of the text detection sub-model and the text recognition sub-model respectively. Accordingly, the sample detection results can be used as the output of the text detection sub-model to train the text detection sub-model, and the sample recognition results can be used as the output of the text recognition sub-model to train the text recognition sub-model. In other words, during the training process of the text recognition model, the text detection sub-model and the text recognition sub-model are parallel. In this way, the text detection sub-model and the text recognition sub-model can be tuned separately to minimize the loss of both the text detection sub-model and the text recognition sub-model, thereby improving the recognition accuracy of the text recognition model.
[0052] In another application scenario, the text box determination sub-model can be trained in the following way: The training sample sub-data includes a sample feature image and a sample pending text box corresponding to the sample feature image.
[0053] The preset training sub-model is trained through the training sample sub-data to obtain the text box determination sub-model.
[0054] For example, training sample sub-data can be first obtained, where the training sample sub-data includes a sample feature image and a sample undetermined text box corresponding to the sample feature image. The sample feature image can be the feature image output by the feature extraction sub-model. A second preset sub-training model can then be trained using the sample feature image as input and the sample undetermined text box as output.
[0055] In another application scenario, the text box determination sub-model can be trained in the following way: For each pixel in each sample feature image, multiple preset anchor boxes are set.
[0056] The anchor boxes containing the target text are regarded as positive anchor boxes, and the anchor boxes not containing the target text are regarded as negative anchor boxes.
[0057] A preset number of target positive anchor frames are randomly selected from the positive anchor frames, and a preset number of target negative anchor frames are randomly selected from the negative anchor frames.
[0058] The preset training sub-model is trained through the target positive anchor box, the target negative anchor box and the sample pending text box to obtain the text box determination sub-model.
[0059] Exemplarily, a preset number of anchor boxes can be preset first. The preset number k can be 6 for example. Then, taking each feature image output by the feature extraction sub-model as a sample feature image, each sample feature image is traversed, and a preset number of anchor boxes are respectively equipped for each pixel point as initial detection boxes. Among them, each anchor box is divided into positive and negative, so the category of each pixel point consists of 2k scores, and each anchor box also corresponds to 4 offsets (x, y, w, h), so the offset of each pixel point consists of 4k coordinates, where (x, y) is the center point coordinate of the anchor box, w is the width of the anchor box, and h is the height of the anchor box. Since using all anchor boxes for training will cause the problem of unbalanced positive and negative samples, a preset number of target positive anchor boxes can be randomly selected from the positive anchor boxes, and a preset number of target negative anchor boxes can be randomly selected from the negative anchor boxes. For example, 256 positive anchor boxes (positive anchor) and 256 negative anchor boxes (negative anchor) can be selected for training. Among them, when the IoU (English: Intersection over Union, Chinese: intersection over union) between the anchor box and the ground truth > 0.7, this anchor box is considered a positive anchor box. When the IoU between the anchor box and the ground truth < 0.3, this anchor box is considered a negative anchor box. Anchor boxes with 0.3 < IoU < 0.7 do not participate in the RPN training. By using the target positive anchor boxes, target negative anchor boxes and sample pending text boxes to train the preset training sub-model, the text box determination sub-model can be obtained.
[0060] In summary, the present disclosure first obtains a text image including a target text to be recognized. Then, the text image is input into a pre-trained text recognition model to obtain the text recognition result of the target text output by the text recognition model. Among them, the text recognition model is used to obtain the feature image of the text image, determine the pending text box according to the feature image, perform border regression on the pending text box to obtain the target text box, and perform semantic segmentation on the image corresponding to the target text box to obtain the text recognition result. The text box is used to determine the text area of the target text in the feature image. The present disclosure first determines the target text box including the target text to be recognized according to the feature image corresponding to the text image, and performs semantic segmentation on the target text box to obtain the text recognition result, which can accurately recognize irregular texts and improve the accuracy of text recognition.
[0061] Figure 4 is a block diagram of a text recognition device shown according to an exemplary embodiment, as Figure 4 shown, the device 300 includes: The acquisition module 301 is configured to acquire a text image containing target text to be recognized.
[0062] The recognition module 302 is used to input the text image into a pre-trained text recognition model to obtain a text recognition result of the target text output by the text recognition model.
[0063] The text recognition model is used to obtain the feature image of the text image, determine the pending text box based on the feature image, perform border regression on the pending text box to obtain the target text box, and perform semantic segmentation on the image corresponding to the target text box to obtain the text recognition result, where the text box is used to determine the text area of the target text in the feature image.
[0064] In one application scenario, the text recognition model includes a feature extraction sub-model, a text box determination sub-model, a text detection sub-model, and a text recognition sub-model. The output of the feature extraction sub-model is coupled to the input of the text box determination sub-model, the output of the text box determination sub-model is coupled to the input of the text detection sub-model, and the output of the text detection sub-model is coupled to the input of the text recognition sub-model. The text recognition model is used to: The image features of the text image are extracted through the feature extraction sub-model to obtain a feature image, and the feature image is input into the text box to determine the sub-model.
[0065] The text box determination sub-model determines the pending text box according to the feature image.
[0066] Adjust the size of the pending text box to the target size.
[0067] The adjusted pending text box is regressed through the text detection sub-model to obtain the target text box. And, The text recognition sub-model is used to perform semantic segmentation on the image corresponding to the target text box to obtain the text recognition result.
[0068] In another application scenario, the text recognition model is trained in the following way: The training sample data includes text image samples and sample recognition results corresponding to the text image samples.
[0069] The preset training model is trained using training sample data to obtain a text recognition model.
[0070] In another application scenario, the preset training model is trained using training sample data to obtain a text recognition model including: The text image samples are used as the input of the feature extraction sub-model, and the sample feature images are used as the output of the text detection sub-model to train the feature extraction sub-model.
[0071] The output of the feature extraction sub-model is used as the input of the text box determination sub-model, and the sample pending text box is used as the output of the text box determination sub-model to train the text box determination sub-model.
[0072] The output of the text box determination sub-model is used as the input of the text detection sub-model, and the sample detection result is used as the output of the text detection sub-model to train the text detection sub-model.
[0073] The output of the text box determination sub-model is used as the input of the text recognition sub-model, and the sample recognition result is used as the output of the text recognition sub-model to train the text recognition sub-model.
[0074] In another application scenario, the text box determination sub-model is trained in the following way: The training sample sub-data includes a sample feature image and a sample pending text box corresponding to the sample image feature.
[0075] The preset training sub-model is trained through the training sample sub-data to obtain the text box determination sub-model.
[0076] In another application scenario, the preset training sub-model is trained by training sample sub-data, and the obtained text box determination sub-model includes: For each pixel in each sample feature image, multiple preset anchor boxes are set.
[0077] The anchor boxes containing the target text are regarded as positive anchor boxes, and the anchor boxes not containing the target text are regarded as negative anchor boxes.
[0078] A preset number of target positive anchor frames are randomly selected from the positive anchor frames, and a preset number of target negative anchor frames are randomly selected from the negative anchor frames.
[0079] The preset training sub-model is trained through the target positive anchor box, the target negative anchor box and the sample pending text box to obtain the text box determination sub-model.
[0080] Regarding the apparatus in the above embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment of the method, and will not be elaborated here.
[0081] In summary, the present disclosure first obtains a text image containing the target text to be recognized. The text image is then input into a pre-trained text recognition model to obtain a text recognition result of the target text output by the text recognition model. The text recognition model is used to obtain a feature image of the text image, determine a pending text box based on the feature image, perform border regression on the pending text box to obtain a target text box, and perform semantic segmentation on the image corresponding to the target text box to obtain a text recognition result, wherein the text box is used to determine the text area of the target text in the feature image. The present disclosure first determines the target text box containing the target text to be recognized based on the feature image corresponding to the text image, and performs semantic segmentation on the target text box to obtain a text recognition result, which can accurately identify irregular text and improve the accuracy of text recognition.
[0082] Figure 5 FIG. 4 is a block diagram of an electronic device 400 according to an exemplary embodiment. Figure 5 As shown, the electronic device 400 may include: a processor 401 , a memory 402 , and may further include one or more of a multimedia component 403 , an input / output (I / O) interface 404 , and a communication component 405 .
[0083] The processor 401 is used to control the overall operation of the electronic device 400 to complete all or part of the steps in the above-mentioned text recognition method. The memory 402 is used to store various types of data to support the operation of the electronic device 400. This data may include, for example, instructions for any application or method operating on the electronic device 400, as well as application-related data such as contact information, sent and received messages, images, audio, video, etc. The memory 402 can be implemented by any type of volatile or non-volatile storage device, or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk. The multimedia component 403 may include a screen and an audio component. The screen may be, for example, a touch screen, and the audio component is used to output and / or input audio signals. For example, the audio component may include a microphone for receiving external audio signals. The received audio signals may be further stored in the memory 402 or transmitted via the communication component 405. The audio component also includes at least one speaker for outputting audio signals. The I / O interface 404 provides an interface between the processor 401 and other interface modules, which may be a keyboard, a mouse, buttons, etc. These buttons may be virtual buttons or physical buttons. The communication component 405 is used for wired or wireless communication between the electronic device 400 and other devices. Wireless communication, such as Wi-Fi, Bluetooth, Near Field Communication (NFC), 2G, 3G, 4G, NB-IOT, eMTC, or other 5G, etc., or a combination of one or more thereof, is not limited here. Therefore, the corresponding communication component 405 may include: a Wi-Fi module, a Bluetooth module, an NFC module, etc.
[0084] In an exemplary embodiment, the electronic device 400 may be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components to perform the above-mentioned text recognition method.
[0085] In another exemplary embodiment, a computer-readable storage medium including program instructions is further provided. When the program instructions are executed by a processor, the steps of the above-described text recognition method are implemented. For example, the computer-readable storage medium may be the aforementioned memory 402 including the program instructions. The program instructions may be executed by the processor 401 of the electronic device 400 to perform the above-described text recognition method.
[0086] In another exemplary embodiment, a computer program product is further provided. The computer program product includes a computer program that can be executed by a programmable device, and the computer program has a code portion for performing the above-mentioned text recognition method when executed by the programmable device.
[0087] The preferred embodiments of the present disclosure are described in detail above in conjunction with the accompanying drawings. However, the present disclosure is not limited to the specific details of the above embodiments. Within the technical concept of the present disclosure, various simple modifications can be made to the technical solutions of the present disclosure, and these simple modifications all fall within the scope of protection of the present disclosure.
[0088] It should also be noted that the various specific technical features described in the above specific embodiments can be combined in any appropriate manner without contradiction. In order to avoid unnecessary repetition, the present disclosure will not further describe various possible combinations.
[0089] In addition, the various embodiments of the present disclosure may be arbitrarily combined, and as long as they do not violate the concept of the present disclosure, they should also be regarded as the contents disclosed by the present disclosure.
Claims
1. A text recognition method, characterized in that: The method comprises: Obtaining a text image containing target text to be recognized; Inputting the text image into a pre-trained text recognition model to obtain a text recognition result of the target text output by the text recognition model; The text recognition model is used to obtain the feature image of the text image, determine a pending text box based on the feature image, perform border regression on the pending text box to obtain a target text box, and perform semantic segmentation on the image corresponding to the target text box to obtain a text recognition result, wherein the text box is used to determine the text area of the target text in the feature image.
2. The method according to claim 1, characterized in that The text recognition model includes a feature extraction sub-model, a text box determination sub-model, a text detection sub-model and a text recognition sub-model. The output end of the feature extraction sub-model is coupled to the input end of the text box determination sub-model, the output end of the text box determination sub-model is coupled to the input end of the text detection sub-model, and the output end of the text detection sub-model is coupled to the input end of the text recognition sub-model. The text recognition model is used to: Extracting image features of the text image through the feature extraction sub-model to obtain the feature image, and inputting the feature image into the text box determination sub-model; Determine the pending text box according to the feature image using the text box determination sub-model; Adjusting the size of the pending text box to a target size; Performing bounding box regression on the adjusted pending text box through the text detection sub-model to obtain a target text box; and The text recognition sub-model is used to perform semantic segmentation on the image corresponding to the target text box to obtain a text recognition result.
3. The method according to claim 2, characterized in that The text recognition model is trained in the following way: Acquire training sample data, wherein the training sample data includes text image samples and sample recognition results corresponding to the text image samples; The preset training model is trained using the training sample data to obtain the text recognition model.
4. The method according to claim 3, characterized in that The training of a preset training model using the training sample data to obtain the text recognition model includes: Using the text image sample as input of the feature extraction sub-model and the sample feature image as output of the text detection sub-model to train the feature extraction sub-model; The output of the feature extraction sub-model is used as the input of the text box determination sub-model, and the sample undetermined text box is used as the output of the text box determination sub-model to train the text box determination sub-model; The output of the text box determination sub-model is used as the input of the text detection sub-model, and the sample detection result is used as the output of the text detection sub-model to train the text detection sub-model; The output of the text box determination sub-model is used as the input of the text recognition sub-model, and the sample recognition result is used as the output of the text recognition sub-model to train the text recognition sub-model.
5. The method according to claim 4, characterized in that The text box determination sub-model is trained in the following way: Acquire training sample sub-data, wherein the training sample sub-data includes a sample feature image and a sample pending text box corresponding to the sample image feature; The preset training sub-model is trained using the training sample sub-data to obtain the text box determination sub-model.
6. The method according to claim 5, characterized in that The step of training a preset training sub-model using the training sample sub-data to obtain the text box determination sub-model includes: For each pixel point in each of the sample feature images, setting multiple preset anchor frames; The anchor box containing the target text is regarded as the positive anchor box, and the anchor box not containing the target text is regarded as the negative anchor box; Randomly select a preset number of target positive anchor frames from the positive anchor frame, and randomly select a preset number of target negative anchor frames from the negative anchor frame; The preset training sub-model is trained using the target positive anchor frame, the target negative anchor frame, and the sample pending text frame to obtain the text frame determination sub-model.
7. A text recognition device, characterized in that: The device comprises: An acquisition module, used for acquiring a text image containing a target text to be recognized; A recognition module, configured to input the text image into a pre-trained text recognition model and obtain a text recognition result of the target text output by the text recognition model; The text recognition model is used to obtain the feature image of the text image, determine a pending text box based on the feature image, perform border regression on the pending text box to obtain a target text box, and perform semantic segmentation on the image corresponding to the target text box to obtain a text recognition result, wherein the text box is used to determine the text area of the target text in the feature image.
8. The device according to claim 7, characterized in that The text recognition model includes a feature extraction sub-model, a text box determination sub-model, a text detection sub-model and a text recognition sub-model. The output end of the feature extraction sub-model is coupled to the input end of the text box determination sub-model, the output end of the text box determination sub-model is coupled to the input end of the text detection sub-model, and the output end of the text detection sub-model is coupled to the input end of the text recognition sub-model. The text recognition model is used to: Extracting image features of the text image through the feature extraction sub-model to obtain the feature image, and inputting the feature image into the text box determination sub-model; Determine the pending text box according to the feature image using the text box determination sub-model; Adjusting the size of the pending text box to a target size; Performing bounding box regression on the adjusted pending text box through the text detection sub-model to obtain a target text box; and The text recognition sub-model is used to perform semantic segmentation on the image corresponding to the target text box to obtain a text recognition result.
9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.
10. An electronic device, characterized in that: include: a memory having a computer program stored thereon; A processor, configured to execute the computer program in the memory to implement the steps of the method according to any one of claims 1 to 6.