Text recognition method and device, feature extraction neural network training method and device

By learning character position high heat maps on single-line text and training feature extraction neural networks, directly identifying space-level dense texts, the problems of multi-line text recognition accuracy and inefficiency in the prior art are solved, and efficient and accurate text recognition is achieved.

CN114255467BActive Publication Date: 2024-11-08ALIBABA GROUP HOLDING LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202011003216.5
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2020-09-22
Publication Date
2024-11-08
Estimated Expiration
2040-09-22

AI Technical Summary

Technical Problem

The prior art is difficult to accurately identify space-level dense two-dimensional text, especially because it is difficult to accurately segment lines between lines of multiple lines, resulting in low recognition accuracy and efficiency.

Method used

Using an extended CTC method, a high-heat map of character position is learned on a single-line text by constructing a pseudo-label, a feature extraction neural network is trained, and a intensive prediction of space-level dense text is directly performed, and the final recognition results are obtained by using CTC decoding.

Benefits of technology

It improves the accuracy and efficiency of space-level intensive text recognition, avoids the accuracy loss caused by detector performance, and realizes accurate recognition of multi-line text.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN114255467B_ABST
    Figure CN114255467B_ABST
Patent Text Reader

Abstract

The embodiments of this specification provide a text recognition method and device, and a feature extraction neural network training method and device, wherein the text recognition method includes obtaining a text image to be recognized; inputting the text image into a feature extraction neural network for processing to obtain a two-dimensional feature map of the text image, wherein the two-dimensional feature map includes characters in the text image; predicting the characters in the two-dimensional feature map to obtain a two-dimensional prediction result map including the predicted characters; processing the predicted characters in the two-dimensional prediction result map to obtain recognition results of the characters in the text image.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The embodiments of this specification relate to the field of computer technology, and in particular to a text recognition method. One or more embodiments of this specification also relate to a text recognition device, a feature extraction neural network training method, a feature extraction neural network training device, a computing device, and a computer-readable storage medium. Background Art

[0002] With the development of deep learning and the expansion of application scenarios, the main research object in the field of document OCR (Optical Character Recognition, OCR for short, optical character recognition in Chinese) has shifted from document text recognition mainly for simple printed text to document text recognition mainly for complex handwritten text. At present, the mainstream text recognition decoders in the industry (such as CTC and Attention) can generally only solve the problem of one-dimensional text or short-sequence two-dimensional text recognition.

[0003] Therefore, there is an urgent need to provide a text recognition method that can accurately recognize long sequences of dense two-dimensional text. Summary of the invention

[0004] In view of this, the present specification provides a text recognition method. One or more embodiments of the present specification also relate to a text recognition device, a feature extraction neural network training method, a feature extraction neural network training device, a computing device, and a computer-readable storage medium to solve the technical defects existing in the prior art.

[0005] According to a first aspect of an embodiment of this specification, a text recognition method is provided, comprising:

[0006] Get the text image to be recognized;

[0007] Inputting the text image into a feature extraction neural network for processing to obtain a two-dimensional feature map of the text image, wherein the two-dimensional feature map includes characters in the text image;

[0008] Predicting the characters in the two-dimensional feature map to obtain a two-dimensional prediction result map containing the predicted characters;

[0009] The predicted characters in the two-dimensional prediction result image are processed to obtain recognition results of the characters in the text image.

[0010] According to a second aspect of an embodiment of this specification, a feature extraction neural network training method is provided, comprising:

[0011] Constructing an initial feature extraction neural network and obtaining a sample image training set containing text, wherein the sample image training set includes sample images and text labels corresponding to the sample images;

[0012] Processing the sample image based on the initial feature extraction neural network to obtain a two-dimensional feature map of the sample image, and determining a character position high heat map of the sample image based on the two-dimensional feature map;

[0013] Obtaining a one-dimensional feature map of the sample image according to the two-dimensional feature map of the sample image and the character position high heat map;

[0014] Classifying the one-dimensional feature map to obtain a text prediction result of the sample image;

[0015] A loss function is calculated based on the text label corresponding to the sample image and the text prediction result, and the initial feature extraction neural network is trained according to the loss function to obtain the feature extraction neural network.

[0016] According to a third aspect of the embodiments of this specification, a text recognition method is provided, including:

[0017] Displaying a picture input interface to the user based on a user's call request;

[0018] Acquire a text image to be recognized input by the user based on the image input interface;

[0019] Inputting the text image into a feature extraction neural network for processing to obtain a two-dimensional feature map of the text image, wherein the two-dimensional feature map includes characters in the text image;

[0020] Predicting the characters in the two-dimensional feature map to obtain a two-dimensional prediction result map containing the predicted characters;

[0021] The predicted characters in the two-dimensional prediction result image are processed to obtain recognition results of the characters in the text image and return them to the user.

[0022] According to a fourth aspect of the embodiments of this specification, a text recognition method is provided, including:

[0023] Receiving a call request sent by a user, wherein the call request carries a text image to be recognized;

[0024] Inputting the text image into a feature extraction neural network for processing to obtain a two-dimensional feature map of the text image, wherein the two-dimensional feature map includes characters in the text image;

[0025] Predicting the characters in the two-dimensional feature map to obtain a two-dimensional prediction result map containing the predicted characters;

[0026] The predicted characters in the two-dimensional prediction result image are processed to obtain recognition results of the characters in the text image and return them to the user.

[0027] According to a fifth aspect of the embodiments of this specification, a feature extraction neural network training method is provided, comprising:

[0028] Displaying a picture input interface to the user based on a user's call request;

[0029] Receive a sample picture training set containing text input by the user based on the picture input interface, wherein the sample picture training set includes sample pictures and text labels corresponding to the sample pictures

[0030] Constructing an initial feature extraction neural network, processing the sample image based on the initial feature extraction neural network to obtain a two-dimensional feature map of the sample image, and determining a character position high heat map of the sample image based on the two-dimensional feature map;

[0031] Obtaining a one-dimensional feature map of the sample image according to the two-dimensional feature map of the sample image and the character position high heat map;

[0032] Classifying the one-dimensional feature map to obtain a text prediction result of the sample image;

[0033] A loss function is calculated based on the text label corresponding to the sample image and the text prediction result, and the initial feature extraction neural network is trained according to the loss function to obtain the feature extraction neural network.

[0034] According to a sixth aspect of the embodiments of this specification, a feature extraction neural network training method is provided, comprising:

[0035] Receive a call request sent by a user, wherein the call request carries a sample picture training set containing text, and the sample picture training set includes sample pictures and text labels corresponding to the sample pictures;

[0036] Construct an initial feature extraction neural network;

[0037] Processing the sample image based on the initial feature extraction neural network to obtain a two-dimensional feature map of the sample image, and determining a character position high heat map of the sample image based on the two-dimensional feature map;

[0038] Obtaining a one-dimensional feature map of the sample image according to the two-dimensional feature map of the sample image and the character position high heat map;

[0039] Classifying the one-dimensional feature map to obtain a text prediction result of the sample image;

[0040] A loss function is calculated based on the text label corresponding to the sample image and the text prediction result, and the initial feature extraction neural network is trained according to the loss function to obtain the feature extraction neural network and return it to the user.

[0041] According to a seventh aspect of the embodiments of this specification, a text recognition device is provided, including:

[0042] A first acquisition module is configured to acquire a text image to be recognized;

[0043] A first obtaining module is configured to input the text image into a feature extraction neural network for processing to obtain a two-dimensional feature map of the text image, wherein the two-dimensional feature map includes characters in the text image;

[0044] A second obtaining module is configured to predict the characters in the two-dimensional feature map to obtain a two-dimensional prediction result map containing the predicted characters;

[0045] The third acquisition module is configured to process the predicted characters in the two-dimensional prediction result image to obtain the recognition results of the characters in the text image.

[0046] According to an eighth aspect of the embodiments of this specification, there is provided a feature extraction neural network training device, comprising:

[0047] A second acquisition module is configured to construct an initial feature extraction neural network and acquire a sample image training set containing text, wherein the sample image training set includes sample images and text labels corresponding to the sample images;

[0048] A third acquisition module is configured to process the sample image based on the initial feature extraction neural network to obtain a two-dimensional feature map of the sample image, and determine a character position high heat map of the sample image based on the two-dimensional feature map;

[0049] A fourth obtaining module is configured to obtain a one-dimensional feature map of the sample image according to the two-dimensional feature map of the sample image and the character position high heat map;

[0050] A fifth obtaining module is configured to classify the one-dimensional feature map to obtain a text prediction result of the sample image;

[0051] The sixth acquisition module is configured to calculate a loss function based on the text label corresponding to the sample image and the text prediction result, and train the initial feature extraction neural network according to the loss function to obtain the feature extraction neural network.

[0052] According to a ninth aspect of the embodiments of this specification, a text recognition device is provided, including:

[0053] A first interface display module is configured to display a picture input interface for the user based on a user's call request;

[0054] A third acquisition module is configured to acquire the text image to be recognized input by the user based on the image input interface;

[0055] a seventh obtaining module, configured to input the text image into a feature extraction neural network for processing, and obtain a two-dimensional feature map of the text image, wherein the two-dimensional feature map includes characters in the text image;

[0056] an eighth obtaining module, configured to predict the characters in the two-dimensional feature map to obtain a two-dimensional prediction result map containing the predicted characters;

[0057] The ninth acquisition module is configured to process the predicted characters in the two-dimensional prediction result image, obtain the recognition results of the characters in the text image and return them to the user.

[0058] According to a tenth aspect of the embodiments of this specification, a text recognition device is provided, including:

[0059] A first request receiving module is configured to receive a call request sent by a user, wherein the call request carries a text image to be recognized;

[0060] A first processing module is configured to input the text image into a feature extraction neural network for processing to obtain a two-dimensional feature map of the text image, wherein the two-dimensional feature map includes characters in the text image;

[0061] A first prediction module is configured to predict the characters in the two-dimensional feature map to obtain a two-dimensional prediction result map containing the predicted characters;

[0062] The second processing module is configured to process the predicted characters in the two-dimensional prediction result image, obtain the recognition results of the characters in the text image and return them to the user.

[0063] According to an eleventh aspect of the embodiments of this specification, there is provided a feature extraction neural network training device, comprising:

[0064] A second interface display module is configured to display a picture input interface for the user based on a call request from the user;

[0065] The sample receiving module is configured to receive a sample picture training set containing text input by the user based on the picture input interface, wherein the sample picture training set includes sample pictures and text labels corresponding to the sample pictures

[0066] A third processing module is configured to construct an initial feature extraction neural network, process the sample image based on the initial feature extraction neural network to obtain a two-dimensional feature map of the sample image, and determine a character position high heat map of the sample image based on the two-dimensional feature map;

[0067] A tenth obtaining module, configured to obtain a one-dimensional feature map of the sample image according to the two-dimensional feature map of the sample image and the character position high heat map;

[0068] A first classification module is configured to classify the one-dimensional feature map to obtain a text prediction result of the sample image;

[0069] The first training module is configured to calculate a loss function based on the text label corresponding to the sample image and the text prediction result, and train the initial feature extraction neural network according to the loss function to obtain the feature extraction neural network.

[0070] According to a twelfth aspect of the embodiments of this specification, there is provided a feature extraction neural network training device, comprising:

[0071] A second request receiving module is configured to receive a call request sent by a user, wherein the call request carries a sample picture training set containing text, and the sample picture training set includes sample pictures and text labels corresponding to the sample pictures;

[0072] a fourth processing module, configured to construct an initial feature extraction neural network, process the sample image based on the initial feature extraction neural network to obtain a two-dimensional feature map of the sample image, and determine a character position high heat map of the sample image based on the two-dimensional feature map;

[0073] An eleventh obtaining module is configured to obtain a one-dimensional feature map of the sample image according to the two-dimensional feature map of the sample image and the character position high heat map;

[0074] A second classification module is configured to classify the one-dimensional feature map to obtain a text prediction result of the sample image;

[0075] The second training module is configured to calculate a loss function based on the text label corresponding to the sample image and the text prediction result, and train the initial feature extraction neural network according to the loss function to obtain the feature extraction neural network and return it to the user.

[0076] According to a thirteenth aspect of the embodiments of this specification, a computing device is provided, including:

[0077] Memory and processor;

[0078] The memory is used to store computer executable instructions, and the processor is used to execute the computer executable instructions. When the instructions are executed by the processor, the steps of the text recognition method or the steps of the feature extraction neural network training method are implemented.

[0079] According to the fourteenth aspect of the embodiments of this specification, a computer-readable storage medium is provided, which stores computer-executable instructions, which, when executed by a processor, implement the steps of the text recognition method or the steps of the feature extraction neural network training method.

[0080] One embodiment of the present specification implements a text recognition method and device, wherein the text recognition method includes obtaining a text image to be recognized; inputting the text image into a feature extraction neural network for processing to obtain a two-dimensional feature map of the text image, wherein the two-dimensional feature map includes characters in the text image; predicting the characters in the two-dimensional feature map to obtain a two-dimensional prediction result map including the predicted characters; processing the predicted characters in the two-dimensional prediction result map to obtain the recognition result of the characters in the text image. The text recognition method uses a feature extraction neural network obtained by training a single-line text image to accurately predict the two-dimensional feature map of the text image to be recognized containing multiple lines of text, and can obtain accurate text characters in the text image by processing the predicted characters in the accurate two-dimensional feature map. BRIEF DESCRIPTION OF THE DRAWINGS

[0081] Figure 1 This is an example diagram of a specific application scenario of a text recognition method provided by an embodiment of this specification;

[0082] Figure 2 is a flow chart of a first text recognition method provided by an embodiment of this specification;

[0083] Figure 3 It is a flowchart of obtaining a recognition result of a text image according to a two-dimensional prediction result graph in a text recognition method provided by an embodiment of this specification;

[0084] Figure 4is a flow chart of a first feature extraction neural network training method provided by an embodiment of this specification;

[0085] Figure 5 It is a specific training process diagram of a feature extraction neural network training method provided by an embodiment of this specification;

[0086] Figure 6 is a flow chart of a second text recognition method provided by an embodiment of this specification;

[0087] Figure 7 is a flowchart of a third text recognition method provided by an embodiment of this specification;

[0088] Figure 8 is a flow chart of a second feature extraction neural network training method provided by an embodiment of this specification;

[0089] Fig. 9 is a flow chart of a third feature extraction neural network training method provided by an embodiment of this specification;

[0090] Fig.10 is a structural schematic diagram of a first text recognition device provided by an embodiment of this specification;

[0091] Fig.11 is a structural diagram of a second text recognition device provided by an embodiment of this specification;

[0092] Fig.12 is a structural diagram of a third text recognition device provided by an embodiment of this specification;

[0093] Fig.13 It is a structural schematic diagram of a first feature extraction neural network training device provided by an embodiment of this specification;

[0094] Fig.14 It is a structural schematic diagram of a second feature extraction neural network training device provided by an embodiment of this specification;

[0095] Fig.15 It is a structural schematic diagram of a third feature extraction neural network training device provided by an embodiment of this specification;

[0096] Fig.16 It is a structural block diagram of a computing device provided by an embodiment of this specification. DETAILED DESCRIPTION

[0097] Many specific details are described in the following description to facilitate a full understanding of this specification. However, this specification can be implemented in many other ways than those described herein, and those skilled in the art can make similar generalizations without violating the connotation of this specification, so this specification is not limited to the specific implementation disclosed below.

[0098] The terms used in one or more embodiments of this specification are only for the purpose of describing specific embodiments, and are not intended to limit one or more embodiments of this specification. The singular forms of "a", "said" and "the" used in one or more embodiments of this specification and the appended claims are also intended to include plural forms, unless the context clearly indicates other meanings. It should also be understood that the term "and / or" used in one or more embodiments of this specification refers to and includes any or all possible combinations of one or more associated listed items.

[0099] It should be understood that although the terms first, second, etc. may be used to describe various information in one or more embodiments of this specification, this information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of one or more embodiments of this specification, the first may also be referred to as the second, and similarly, the second may also be referred to as the first. Depending on the context, the word "if" as used herein may be interpreted as "at the time of" or "when" or "in response to determining".

[0100] First, the terms involved in one or more embodiments of this specification are explained.

[0101] OCR: English: Optical Character Recognition, abbreviated as: OCR, Chinese: Optical Character Recognition.

[0102] CTC: Connectionist Temporal Classifier.

[0103] Attention: Attention mechanism.

[0104] Dense prediction: predict the classification of each feature point on the feature map.

[0105] In this specification, a text recognition method is provided. One or more embodiments of this specification also involve a text recognition device, a feature extraction neural network training method, a feature extraction neural network training device, a computing device, and a computer-readable storage medium, which are described in detail one by one in the following embodiments.

[0106] In the field of document OCR, for paragraph-level text (i.e., multi-line text), the existing recognition technology can generally only solve the problem of one-dimensional text or short-sequence two-dimensional text recognition. When performing paragraph-level dense text recognition, it is necessary to first perform text detection, find the detection box or feature of the single-line text, and then perform one-dimensional text recognition. Finally, the recognition results are stitched together in order to form the recognition result. However, the performance of the detection-before-recognition mechanism is limited by the accuracy and speed of the detector, especially for dense text with multiple lines. It is difficult to accurately separate lines with quadrilateral boxes. For text with inaccurate detection, it will have a huge impact on the accuracy of subsequent recognition. Therefore, the embodiment of the present specification proposes a method for recognizing dense text at the paragraph level based on extended CTC. The method does not require the use of a detector, but only learns a high heat map of character positions on a single line of text by constructing pseudo labels, and uses CTC decoding to train a feature extraction neural network (such as a convolutional neural network). In actual application, dense predictions can be directly performed on dense text at the paragraph level, and the final text recognition result can be obtained through post-processing. The text recognition method of the embodiment of the present specification can effectively improve the recognition accuracy and efficiency of dense text at the paragraph level.

[0107] See also Figure 1 , Figure 1 An example diagram showing a specific application scenario of a text recognition method provided according to an embodiment of the present specification is shown.

[0108] Figure 1 The application scenario includes a terminal and a server. Specifically, a user sends a text image containing multiple lines of text to a server through a terminal. After receiving the text image containing multiple lines of text, the server inputs the text image into a pre-trained convolutional neural network, convolves to obtain a two-dimensional feature map of the text image, and then classifies the characters corresponding to each feature point in the two-dimensional feature map (i.e., character prediction) to obtain a two-dimensional dense prediction result map, wherein the two-dimensional dense prediction result map stores the text characters in the text image; finally, the text characters in the two-dimensional dense prediction result map are post-processed such as eight-field merging, traversal, and sorting to form the final characters in the text image, i.e., the multiple lines of text content in the text image.

[0109] See also Figure 2 , Figure 2 A flowchart of a first text recognition method provided according to an embodiment of the present specification is shown, which specifically includes the following steps.

[0110] Step 202: Obtain a text image to be recognized.

[0111] The text image includes but is not limited to an image containing any text characters, such as a text image containing texts such as words, numbers, symbols and / or special characters. In practical applications, the text image is a text image containing dense text, that is, a text image containing multiple lines of text, such as a text image of 50 lines, 100 lines or 200 lines of text.

[0112] Specifically, obtaining a text image to be recognized can be understood as obtaining a text image to be recognized containing multiple lines of text. In subsequent processing, the processing is performed on the text image containing multiple lines of text.

[0113] Step 204: Input the text image into a feature extraction neural network for processing to obtain a two-dimensional feature map of the text image, wherein the two-dimensional feature map includes the characters in the text image.

[0114] Among them, the feature extraction neural network can be a convolutional neural network, or it can be other machine learning networks that can process text images to obtain two-dimensional feature maps. The present application does not impose any limitations on this. For ease of understanding, the embodiments of this specification are described in detail using the convolutional neural network as an example of the feature extraction neural network.

[0115] In a specific implementation, before the text image is input into the feature extraction neural network, the feature extraction neural network will be trained so that the feature extraction neural network can accurately output the two-dimensional feature map of the text image when it is used specifically.

[0116] Specifically, the training steps of the feature extraction neural network are as follows:

[0117] Constructing an initial feature extraction neural network and obtaining a sample image training set containing text, wherein the sample image training set includes sample images and text labels corresponding to the sample images;

[0118] Processing the sample image based on the initial feature extraction neural network to obtain a two-dimensional feature map of the sample image, and determining a character position high heat map of the sample image based on the two-dimensional feature map;

[0119] Obtaining a one-dimensional feature map of the sample image according to the two-dimensional feature map of the sample image and the character position high heat map;

[0120] Classifying the one-dimensional feature map to obtain a text prediction result of the sample image;

[0121] A loss function is calculated based on the text label corresponding to the sample image and the text prediction result, and the initial feature extraction neural network is trained according to the loss function to obtain the feature extraction neural network.

[0122] Among them, the sample image is different from the above-mentioned text image to be recognized. The sample image is a sample image containing a single line of text, that is, each sample image contains only one line of text consisting of words, digital symbols and / or special characters, while the text image to be recognized is a dense text image containing multiple lines of text.

[0123] When the sample images are single-row sample images, the sample label corresponding to each sample image is the actual text content in each sample image. For example, if the sample image includes "I love China", then the text label corresponding to the sample image is "I love China".

[0124] In the specific implementation, firstly, an initial feature extraction neural network is constructed, which includes an input layer, a hidden layer, a convolution layer, a pooling layer and a fully connected layer, etc., then the obtained sample image is input into the initial feature extraction neural network through the input layer of the initial feature extraction neural network, and the sample image is convolved through the convolution layer of the initial feature extraction neural network to obtain a two-dimensional feature map corresponding to the sample image, and then some convolution operations are performed on the two-dimensional feature map to obtain a character position high heat map of the sample image; a one-dimensional feature map of the sample image is calculated based on the two-dimensional feature map of the sample image and the character position high heat map; the text in the sample image is classified and predicted through the one-dimensional feature map to obtain a text prediction result of the sample image; finally, the loss function of the initial feature extraction neural network is calculated based on the real text label corresponding to the sample image and the text prediction result of the sample image, and the loss function is used to adjust the parameters of each layer of the initial feature extraction neural network through the principle of back propagation to obtain the feature extraction neural network after final training.

[0125] In the embodiments of the present specification, when training the feature extraction neural network, only text images of a single line of text are used to train the feature extraction neural network, so that the feature extraction neural network obtained after training can be applied to the recognition method of multi-line, paragraph-level dense text, avoiding the use of a text detector, thereby avoiding the loss of accuracy caused by the detector performance, and the feature extraction neural network can greatly improve the recognition efficiency of text images of paragraph-level dense text in practical applications.

[0126] In actual use, in order to supervise the character position high heat map when generating it and ensure the inference accuracy of the feature extraction neural network, the character position high heat map can be supervised in the form of pseudo labels. The specific implementation method is as follows:

[0127] After obtaining the sample image training set containing text, the method further includes:

[0128] The sample image is preprocessed to generate a pseudo label for the sample image.

[0129] Specifically, the sample images are preprocessed, including but not limited to 5*5 corrosion, Otsu binarization, inversion, and proportional scaling of the sample images. The Otsu binarization method is a global threshold segmentation method and an efficient algorithm for binarizing images. By performing the above series of preprocessing on the sample images, the preprocessed sample images are obtained, and the preprocessed sample images are pseudo labels.

[0130] In actual application, when the two-dimensional feature map of the sample image is convolved to obtain the character position high heat map, the character position high heat map will be supervised by pseudo labels to ensure the accuracy of the character position high heat map. The specific implementation method is as follows:

[0131] The determining of the character position high heat map of the sample image based on the two-dimensional feature map comprises:

[0132] A character position heat map of the sample image is determined based on the pseudo label of the sample image and the two-dimensional feature map.

[0133] In the specific implementation, the pseudo-label of the sample image obtained by preprocessing is of the same size as the character position high heat map generated based on the convolution of the two-dimensional feature map. A mean square error loss is calculated by the activation value of the pixel point in the character position high heat map and the activation value of the pixel point in the pseudo-label. The character position high heat map is adjusted based on the mean square error loss to obtain the final and accurate character position high heat map.

[0134] In practical applications, since only the high heat of each character can be achieved through a two-dimensional feature map, but the position of each high heat character is unclear, a certain character position sequence is required to accurately arrange the position of each character. Therefore, generating pseudo labels can determine which characters are high heat when generating a character position high heat map, thereby generating an accurate character position high heat map.

[0135] In the embodiments of the present specification, pseudo labels are generated and supervised learning is performed to obtain a heat map of the character positions in a single-line text image. When the feature extraction neural network is subsequently used for text recognition, each feature point on the dense text at the page level can be directly predicted and classified, so that a corresponding result will be generated at each position where a character is located, thereby ensuring the integrity of the recognized text.

[0136] In another embodiment of the present specification, obtaining a one-dimensional feature map of the sample image according to the two-dimensional feature map of the sample image and the character position high heat map includes:

[0137] The two-dimensional feature map of the sample image and the character position high heat map are summed in the vertical dimension to obtain a one-dimensional feature map of the sample image.

[0138] Specifically, since the feature extraction neural network is trained through sample images of single-line text, it is necessary to obtain the recognition results of the text on the one-dimensional image. Therefore, after obtaining the two-dimensional feature map and the character position high heat map of the sample image, the two-dimensional feature map and the character position high heat map of the sample image are vertically summed to obtain the one-dimensional feature map of the sample image, so that the training parameters of the feature extraction neural network can be adjusted based on the prediction results in the one-dimensional feature map to ensure the prediction accuracy of the feature extraction neural network.

[0139] In a specific implementation, the classifying of the one-dimensional feature map to obtain the text prediction result of the sample image includes:

[0140] Classifying the one-dimensional feature map to obtain an initial text prediction result of the sample image;

[0141] The initial text prediction result is subjected to CTC decoding to obtain a text prediction result of the sample image.

[0142] Specifically, after obtaining the one-dimensional feature map of the sample image, the one-dimensional feature map will be classified, that is, the characters in the one-dimensional feature map will be predicted to obtain the initial text prediction result of the sample image, and then the initial text prediction result of the sample image will be CTC decoded, that is, the repeatedly predicted characters will be removed to obtain the final text prediction result of the sample image.

[0143] In the embodiments of the present specification, CTC decoding is performed on the initial text prediction result of the one-dimensional feature map classification prediction to remove the repeatedly predicted characters, so as to ensure the accuracy of the text prediction result of the sample image obtained by prediction.

[0144] After obtaining the final text prediction result of the sample image, the text prediction result of the sample image is compared with the actual text label of the sample image, and the CTC loss is calculated for the initial feature extraction neural network. The initial feature extraction neural network is trained through the calculated loss function to obtain the final adjusted and accurate feature extraction neural network.

[0145] In the embodiments of the present specification, when training the feature extraction neural network, a text image of a single line of text is used to train the feature extraction neural network based on the CTC loss function. By generating pseudo labels, a high heat map of the character positions in the single line of text is supervised and learned, and the initial text prediction results in the predicted one-dimensional feature map are processed by CTC decoding to ensure the recognition accuracy and efficiency of the feature extraction neural network obtained by training.

[0146] Step 206: predicting the characters in the two-dimensional feature map to obtain a two-dimensional prediction result map containing the predicted characters.

[0147] The predicted characters may be understood as characters in the predicted text image.

[0148] Specifically, characters in a two-dimensional feature map of a text image are predicted to obtain a two-dimensional prediction result map of the text image, wherein the two-dimensional prediction result map stores prediction results obtained by predicting the characters in the two-dimensional feature map.

[0149] Step 208: Process the predicted characters in the two-dimensional prediction result image to obtain recognition results of the characters in the text image.

[0150] Specifically, the processing of the predicted characters in the two-dimensional prediction result image to obtain the recognition results of the characters in the text image includes:

[0151] Merging eight neighborhoods of the predicted characters in the two-dimensional prediction result graph to obtain a target character generated after the predicted characters are merged, wherein the target character carries a horizontal coordinate and a vertical coordinate in the two-dimensional prediction result graph;

[0152] The target characters are arranged based on the horizontal coordinates and the vertical coordinates of the target characters, and the recognition results of the characters in the text image are determined according to the arrangement results of the target characters.

[0153] Among them, the purpose of eight-neighborhood merging is to merge the characters obtained by repeated prediction into one character without affecting other identical characters in different fields.

[0154] For example, if the predicted characters in the two-dimensional prediction result graph are "I love China, I am Chinese", then after the eight-neighborhood merging of the predicted characters in the two-dimensional prediction result graph, the target characters after the merging of the predicted characters are "I love China, I am Chinese". Each target character carries the ordinate and abscissa in the two-dimensional prediction result graph.

[0155] In the specific implementation, the predicted characters in the two-dimensional prediction result image are first merged into eight neighborhoods to obtain the target characters, and then the target characters are arranged according to the horizontal and vertical coordinates of the target characters, and the arranged target characters are used as the target recognition results of the characters in the text image, that is, the specific text content in the text image.

[0156] In the embodiment of the present specification, the predicted characters in the two-dimensional prediction result image are deduplicated by merging eight neighborhoods. While obtaining accurate target characters, the processing time of the target characters can also be saved, and the arrangement efficiency of the target characters can be improved.

[0157] In another embodiment of the present specification, the arranging the target characters based on the horizontal coordinates and the vertical coordinates of the target characters, and determining the recognition results of the characters in the text image according to the arrangement results of the target characters includes:

[0158] sorting the target characters in ascending order based on the horizontal coordinates of the target characters to form a sorted first target character set;

[0159] storing the first target character in the first target character set into the second target character set, determining the ordinate of the first target character as the current sliding average, and deleting the first target character in the first target character set;

[0160] traversing the other target characters in the first target character set except the first target character according to the current sliding average value to obtain a third target character set, wherein the third target character set includes at least one second target character set;

[0161] Based on the ordinate of the first target character in each second target character set, the second target character sets in the third target character set are sorted and then merged, and the recognition result of the characters in the text image is determined according to the merged result.

[0162] Among them, the first target character set and the second target character set are both sets for storing target characters. The difference is that the first target character set initially contains all target characters, while the second target character set contains target characters located in the same row determined based on the horizontal and vertical coordinates. There is one first target character set and at least one second target character set. Generally, for text images with dense text at the page level, there are multiple second target character sets.

[0163] Specifically, firstly, all target characters are sorted in ascending order based on the horizontal coordinate of each target character, and the sorted target characters are stored in the first target character set. The first target character in the first target character set, that is, the target character with the largest horizontal coordinate, is stored in the second target character set, and the vertical coordinate of the first target character is used as the current sliding average, and the first target character in the first target character set is deleted.

[0164] Then, based on the current sliding average, the other target characters in the first target character set except the first target character are traversed to obtain a third target character set including at least one second target character set; finally, the vertical coordinate of the first target character in each second target character set is obtained, and all the second target character sets are sorted and merged based on the vertical coordinate, and the target characters in all the sorted and merged second target character sets are used as the recognition results of the characters in the text image.

[0165] In the embodiments of the present specification, based on the horizontal and vertical coordinates of the target characters, the target characters of each row are first obtained, and then the target characters of each row are sorted and merged based on the vertical coordinates, so as to achieve accurate recognition of the characters in the text image, avoid the omission of row characters and the mismatch of column text characters, and improve the user experience.

[0166] In another embodiment of the present specification, the first target character is the i-th target character, where i∈[1,n], and i is a positive integer;

[0167] Correspondingly, traversing other target characters in the first target character set except the first target character according to the current sliding average value to obtain a third target character set includes:

[0168] S1, obtaining the i+1th target character in the first target character set, and calculating the sliding average value of the i+1th target character based on the current sliding average value;

[0169] S2: Determine whether the sliding average value of the i+1 target character is less than a preset sliding threshold value.

[0170] If yes, then storing the i+1th target character into the second target character set, deleting the i+1th target character in the first target character set, and updating the current sliding average value based on the sliding average value of the i+1th target character;

[0171] Determine whether i+1 is greater than n,

[0172] If yes, then the second target character set is stored in the third target character set, and it is determined whether the first target character set is empty.

[0173] If yes, then obtain a third target character set;

[0174] If not, a new second target character set is created, the i-th target character in the first target character set is stored in the newly created second target character set, the ordinate of the i-th target character is determined as the current sliding average, the i-th target character in the first target character set is deleted, and step S1 is continued.

[0175] If not, then i is incremented by 1 and step S1 is continued.

[0176] If not, i is incremented by 1 and step S1 is continued.

[0177] Specifically, i is 1 for illustration, the target characters are sorted in ascending order based on the horizontal coordinates of all target characters to form a sorted first target character set, and the first target character in the first target character set is stored in the second target character set, the vertical coordinate of the first target character is determined as the current sliding average value, and after deleting the first target character in the first target character set, the second target character in the first target character set is continued to be obtained, that is, the next target character next to the first target character, and the sliding average value of the second target character is obtained based on the current sliding average value and a preset calculation formula, wherein the sliding average value of the first target character is:

[0178] y 滑动(1) =y of the first target character, where y 滑动(1) represents the sliding average of the first target character, and y of the first target character represents the vertical coordinate of the first target character.

[0179] The default calculation formula is:

[0180] y 滑动(n) =0.9*y 滑动(n-1) +0.1*y of the nth character

[0181] Among them, y 滑动(n) represents the sliding average of the nth target character, y 滑动(n-1) represents the sliding average of the previous target character n, that is, the current sliding average, and the y of the nth character represents the ordinate of the nth character.

[0182] That is, taking n as 2 as an example, when calculating the sliding average of the second target character, the vertical coordinate of the second target character and the sliding average of the first target character are substituted into the preset calculation formula to calculate the sliding average of the second target character.

[0183] Then, it is determined whether the sliding average value of the second target character is less than a preset sliding threshold value, wherein the preset sliding threshold value can be set according to the actual application, for example, set to 2, 3, etc., and this specification does not impose any limitation on this; if so, the second target character is also stored in the second target character set where the first target character is located, and the second target character in the first target character set is deleted, and the sliding average value of the second target character is used as the current sliding average value; if not, it means that the second target character and the first target character are not in the same line of text. In this case, the second target character is not processed first, and the third target character is processed in the above manner.

[0184] When the sliding average value of the second target character is less than the preset sliding threshold value, after the second target character is stored in the second target character set, it is necessary to determine whether the second character is the last target character in the first target character set. If so, the second target character set including the first and second target characters is stored in the third target character set, and it is determined whether the first target character set is empty. If the first target character set is empty, it means that all target characters have been traversed and the third target character set can be directly obtained. If it is not empty, it means that although there are still target characters in the first target character set, other target characters are not in the same row as the first and second target characters. At this time, a new second target character set is created, and the largest first target character remaining in the first target character set is stored in the newly created second target character set. The above steps are repeated until there are no target characters in the first target character set, that is, all target characters have been stored in the corresponding second target character set, and then all second target character sets are stored in the third target character set.

[0185] In the embodiments of the present specification, all target characters are first arranged in a single row based on the horizontal and vertical coordinates, and then the target characters arranged in a single row are stored in a large set. Subsequently, the final character recognition result of the text image can be obtained by arranging the target characters in each row in columns. In this way, the recognition result of the text image can be obtained completely and quickly.

[0186] In another embodiment of the present specification, the step of sorting and merging the second target character sets in the third target character set based on the ordinate of the first target character in each second target character set, and determining the recognition result of the characters in the text image according to the merging result includes:

[0187] Obtain all second target character sets in the third target character set;

[0188] Determine the ordinate of the first target character in each second target character set;

[0189] The target characters in all the second target character sets are arranged based on the ordinate of the first target character in each second target character set, and the arrangement result of the target characters in all the second target character sets is determined as the recognition result of the characters in the text picture.

[0190] In specific implementation, when arranging each row of target characters in the third target character set, first obtain the vertical coordinate of the first target character in each row, and then arrange all rows of target characters based on the vertical coordinate of the first target character in each row to obtain an accurate and neat final recognition result of the text image.

[0191] In the embodiments of the present specification, the text recognition method uses a feature extraction neural network obtained through training based on a single-line text image to accurately predict a two-dimensional feature map of a text image to be recognized containing multiple lines of text, and can obtain accurate text characters in the text image by processing the predicted characters in the accurate two-dimensional feature map.

[0192] See also Figure 3 , Figure 3 A flowchart of obtaining a recognition result of a text image according to a two-dimensional prediction result graph in a text recognition method provided according to an embodiment of the present specification is shown, which specifically includes the following steps:

[0193] Step 302: Obtain a two-dimensional prediction result graph.

[0194] Step 304: performing eight-neighborhood merging on the predicted characters in the two-dimensional prediction result image to obtain a merged target character.

[0195] Step 306: Sort the target characters in ascending order according to the horizontal coordinates of the target characters to obtain a sequence S.

[0196] Step 308: Take the first target character in sequence S, put it into sequence D, and use the vertical coordinate of the first target character as the current sliding average.

[0197] Step 310: traverse the next character in the sequence S in order, and calculate the sliding average value of the next character based on the ordinate of the next character and the current sliding average value.

[0198] Step 312: Determine whether the absolute value of the sliding average value of the next character is less than 3. If so, execute step 314; if not, continue to execute step 310.

[0199] Step 314: put the next character into sequence D, update the calculated sliding average value of the next character as the current sliding average value, and remove the next character from sequence S.

[0200] Step 316: Determine whether the last target character of the sequence S has been traversed. If so, execute step 318; if not, continue to execute step 310.

[0201] That is, determine whether the traversal has reached the end of the sequence S.

[0202] Step 318: Put sequence D into set D.

[0203] Step 320: Determine whether the sequence S is empty. If so, execute step 322; if not, continue to execute step 308.

[0204] Step 322: Obtain the ordinate of the first character in each sequence D in the set D, and sort and merge all the sequences D based on the ordinate of the first character in each sequence D.

[0205] Step 324: taking the target characters in all the sorted and merged sequences D as the final recognition result of the text image.

[0206] In the embodiments of the present specification, the method of performing the above-mentioned processing on the predicted characters in the two-dimensional prediction result graph can solve the decoding of dense text at the length level with any number of characters, avoid the problem of attention map drift in the attention mechanism, solve the problem of being unable to be data-driven on dense text at the length level, and achieve accurate and rapid recognition of dense text at the length level, wherein data-driven can be understood as the current convolutional neural network model, and many of them can improve the performance of the convolutional neural network model by adding a large amount of labeled data, that is, the performance can be driven by data; and being unable to be data-driven means that no matter how much data is added, the above-mentioned problem still cannot be solved, because this is an unsolvable problem in modeling.

[0207] See also Figure 4 , Figure 4 A flow chart of a first feature extraction neural network training method provided according to an embodiment of the present specification is shown, which specifically includes the following steps.

[0208] Step 402: construct an initial feature extraction neural network, and obtain a sample image training set containing text, wherein the sample image training set includes sample images and text labels corresponding to the sample images.

[0209] Among them, the sample image is different from the above-mentioned text image to be recognized. The sample image is a sample image containing a single line of text, that is, each sample image contains only one line of text consisting of words, digital symbols and / or special characters, while the text image to be recognized is a dense text image containing multiple lines of text.

[0210] When the sample images are single-row sample images, the sample label corresponding to each sample image is the actual text content in each sample image. For example, if the sample image includes "I love China", then the text label corresponding to the sample image is "I love China".

[0211] Step 404: Process the sample image based on the initial feature extraction neural network to obtain a two-dimensional feature map of the sample image, and determine a character position high heat map of the sample image based on the two-dimensional feature map.

[0212] Step 406: Obtain a one-dimensional feature map of the sample image according to the two-dimensional feature map of the sample image and the character position high heat map.

[0213] Step 408: Classify the one-dimensional feature map to obtain a text prediction result of the sample image.

[0214] Step 410: Calculate a loss function based on the text label corresponding to the sample image and the text prediction result, and train the initial feature extraction neural network according to the loss function to obtain the feature extraction neural network.

[0215] In specific implementation, the feature extraction neural network can be a convolutional neural network, or other machine learning networks that can process sample images to obtain two-dimensional feature maps. This application does not impose any limitations on this.

[0216] In the specific implementation, firstly, an initial feature extraction neural network is constructed, which includes an input layer, a hidden layer, a convolution layer, a pooling layer and a fully connected layer, etc., then the obtained sample image is input into the initial feature extraction neural network through the input layer of the initial feature extraction neural network, and the sample image is convolved through the convolution layer of the initial feature extraction neural network to obtain a two-dimensional feature map corresponding to the sample image, and then some convolution operations are performed on the two-dimensional feature map to obtain a character position high heat map of the sample image; a one-dimensional feature map of the sample image is calculated based on the two-dimensional feature map of the sample image and the character position high heat map; the text in the sample image is classified and predicted through the one-dimensional feature map to obtain a text prediction result of the sample image; finally, the loss function of the initial feature extraction neural network is calculated based on the real text label corresponding to the sample image and the text prediction result of the sample image, and the loss function is used to adjust the parameters of each layer of the initial feature extraction neural network through the principle of back propagation to obtain the feature extraction neural network after final training.

[0217] In the embodiments of the present specification, when training the feature extraction neural network, only text images of a single line of text are used to train the feature extraction neural network, so that the feature extraction neural network obtained after training can be applied to the recognition method of multi-line, paragraph-level dense text, avoiding the use of a text detector, thereby avoiding the loss of accuracy caused by the detector performance, and the feature extraction neural network can greatly improve the recognition efficiency of text images of paragraph-level dense text in practical applications.

[0218] In actual use, in order to supervise the character position high heat map when generating it and ensure the inference accuracy of the feature extraction neural network, the character position high heat map can be supervised in the form of pseudo labels. The specific implementation method is as follows:

[0219] After obtaining the sample image training set containing text, the method further includes:

[0220] The sample image is preprocessed to generate a pseudo label for the sample image.

[0221] Specifically, the sample images are preprocessed, including but not limited to 5*5 corrosion, Otsu binarization, inversion, and proportional scaling of the sample images. The Otsu binarization method is a global threshold segmentation method and an efficient algorithm for binarizing images. By performing the above series of preprocessing on the sample images, the preprocessed sample images are obtained, and the preprocessed sample images are pseudo labels.

[0222] In actual application, when the two-dimensional feature map of the sample image is convolved to obtain the character position high heat map, the character position high heat map will be supervised by pseudo labels to ensure the accuracy of the character position high heat map. The specific implementation method is as follows:

[0223] The determining of the character position high heat map of the sample image based on the two-dimensional feature map comprises:

[0224] A character position heat map of the sample image is determined based on the pseudo label of the sample image and the two-dimensional feature map.

[0225] In the specific implementation, the pseudo-label of the sample image obtained by preprocessing is of the same size as the character position high heat map generated based on the convolution of the two-dimensional feature map. A mean square error loss is calculated by the activation value of the pixel point in the character position high heat map and the activation value of the pixel point in the pseudo-label. The character position high heat map is adjusted based on the mean square error loss to obtain the final and accurate character position high heat map.

[0226] In practical applications, since only the high heat of each character can be achieved through a two-dimensional feature map, but the position of each high heat character is unclear, a certain character position sequence is required to accurately arrange the position of each character. Therefore, generating pseudo labels can determine which characters are high heat when generating a character position high heat map, thereby generating an accurate character position high heat map.

[0227] In the embodiments of the present specification, pseudo labels are generated and supervised learning is performed to obtain a heat map of the character positions in a single-line text image. When the feature extraction neural network is subsequently used for text recognition, each feature point on the dense text at the page level can be directly predicted and classified, so that a corresponding result will be generated at each position where a character is located, thereby ensuring the integrity of the recognized text.

[0228] In another embodiment of the present specification, obtaining a one-dimensional feature map of the sample image according to the two-dimensional feature map of the sample image and the character position high heat map includes:

[0229] The two-dimensional feature map of the sample image and the character position high heat map are summed in the vertical dimension to obtain a one-dimensional feature map of the sample image.

[0230] Specifically, since the feature extraction neural network is trained through sample images of single-line text, it is necessary to obtain the recognition results of the text on the one-dimensional image. Therefore, after obtaining the two-dimensional feature map and the character position high heat map of the sample image, the two-dimensional feature map and the character position high heat map of the sample image are vertically summed to obtain the one-dimensional feature map of the sample image, so that the training parameters of the feature extraction neural network can be adjusted based on the prediction results in the one-dimensional feature map to ensure the prediction accuracy of the feature extraction neural network.

[0231] In a specific implementation, the classifying of the one-dimensional feature map to obtain the text prediction result of the sample image includes:

[0232] Classifying the one-dimensional feature map to obtain an initial text prediction result of the sample image;

[0233] The initial text prediction result is subjected to CTC decoding to obtain a text prediction result of the sample image.

[0234] Specifically, after obtaining the one-dimensional feature map of the sample image, the one-dimensional feature map will be classified, that is, the characters in the one-dimensional feature map will be predicted to obtain the initial text prediction result of the sample image, and then the initial text prediction result of the sample image will be CTC decoded, that is, the repeatedly predicted characters will be removed to obtain the final text prediction result of the sample image.

[0235] In the embodiments of the present specification, CTC decoding is performed on the initial text prediction result of the one-dimensional feature map classification prediction to remove the repeatedly predicted characters, so as to ensure the accuracy of the text prediction result of the sample image obtained by prediction.

[0236] After obtaining the final text prediction result of the sample image, the text prediction result of the sample image is compared with the actual text label of the sample image, and the CTC loss is calculated for the initial feature extraction neural network. The initial feature extraction neural network is trained by calculating the loss function to obtain the final adjusted and accurate feature extraction neural network.

[0237] In the embodiments of the present specification, when training the feature extraction neural network, a text image of a single line of text is used to train the feature extraction neural network based on the CTC loss function. By generating pseudo labels, a high heat map of the character positions in the single line of text is supervised and learned, and the initial text prediction results in the predicted one-dimensional feature map are processed by CTC decoding to ensure the recognition accuracy and efficiency of the feature extraction neural network obtained by training.

[0238] See also Figure 5 , Figure 5A specific training process diagram of a feature extraction neural network training method provided according to an embodiment of the present specification is shown, which specifically includes the following steps.

[0239] Among them, the feature extraction neural network is a convolutional neural network.

[0240] Step 502: construct an initial convolutional neural network, obtain a single-line input image and a text label corresponding to the single-line input image, and input the single-line input image into the convolutional layer of the initial convolutional neural network for convolution.

[0241] Step 504: Obtain a two-dimensional feature map after convolution of a single row of input images.

[0242] Step 506: Perform preprocessing such as corrosion, binarization, inversion, and proportional scaling on the single-line input image to generate a pseudo label.

[0243] Step 508: Generate a character position heat map based on the two-dimensional feature map.

[0244] Step 510: Calculate the mean square error loss of the character position high heat map based on the pseudo-labels, and adjust the character position high heat map based on the mean square error loss.

[0245] Step 512: vertically sum the two-dimensional feature map and the character position heat map to obtain a one-dimensional feature map.

[0246] Step 514: classify the one-dimensional feature map to obtain a classified initial prediction sequence.

[0247] Step 516: Perform CTC decoding on the classified initial prediction sequence to obtain a target prediction sequence.

[0248] Step 518: Calculate the CTC loss function according to the target prediction sequence of the single-line input image and the corresponding text label, and train the initial convolutional neural network based on the CTC loss function to obtain a convolutional neural network.

[0249] In the embodiment of this specification, the convolutional neural network uses ResNet as the backbone, and obtains a two-dimensional feature map by inputting a single row of input images into the convolutional neural network, and generates a corresponding text high heat map, multiplies the text high heat map with the two-dimensional feature map, and sums it in the column dimension to obtain a one-dimensional feature map, classifies the one-dimensional feature map and CTC decodes it, generates pseudo labels for the input images, and supervises the mean square error loss with the text high heat map; and supervises the predicted sequence and text labels using CTC loss to achieve fast and accurate training of the convolutional neural network. In addition, the backbone of the convolutional neural network in the embodiment of this specification can have more choices, such as DenseNet, ResNest, etc.; and the process of generating pseudo labels can be done by other image processing methods, as long as it can basically represent the center position of all characters in the text image, and no limitation is made here.

[0250] See also Figure 6 , Figure 6 A flow chart of a second text recognition method provided according to an embodiment of the present specification is shown, which specifically includes the following steps.

[0251] Step 602: Displaying a picture input interface to the user based on the user's call request.

[0252] Step 604: Obtain the text image to be recognized input by the user based on the image input interface.

[0253] Step 606: Input the text image into a feature extraction neural network for processing to obtain a two-dimensional feature map of the text image, wherein the two-dimensional feature map includes the characters in the text image.

[0254] Step 608: predicting the characters in the two-dimensional feature map to obtain a two-dimensional prediction result map containing the predicted characters.

[0255] Step 610: Process the predicted characters in the two-dimensional prediction result image, obtain recognition results of the characters in the text image and return them to the user.

[0256] It should be noted that, for the part of the second text recognition method provided in the embodiments of this specification that corresponds to the embodiment of the first text recognition method mentioned above, reference can be made to the detailed description in the embodiment of the first text recognition method mentioned above, and will not be repeated here.

[0257] In the embodiments of the present specification, the text recognition method uses a feature extraction neural network obtained through training based on a single-line text image to accurately predict a two-dimensional feature map of a text image to be recognized containing multiple lines of text, and can obtain accurate text characters in the text image by processing the predicted characters in the accurate two-dimensional feature map.

[0258] See also Figure 7 , Figure 7 A flowchart of a third text recognition method provided according to an embodiment of the present specification is shown, which specifically includes the following steps.

[0259] Step 702: Receive a call request sent by a user, wherein the call request carries a text image to be recognized.

[0260] Step 704: Input the text image into a feature extraction neural network for processing to obtain a two-dimensional feature map of the text image, wherein the two-dimensional feature map includes the characters in the text image.

[0261] Step 706: predicting the characters in the two-dimensional feature map to obtain a two-dimensional prediction result map containing the predicted characters.

[0262] Step 708: Process the predicted characters in the two-dimensional prediction result image, obtain recognition results of the characters in the text image and return them to the user.

[0263] It should be noted that, for the part of the third text recognition method provided in the embodiments of this specification that corresponds to the embodiment of the first text recognition method mentioned above, reference can be made to the detailed description in the embodiment of the first text recognition method mentioned above, and will not be repeated here.

[0264] In the embodiments of the present specification, the text recognition method uses a feature extraction neural network obtained through training based on a single-line text image to accurately predict a two-dimensional feature map of a text image to be recognized containing multiple lines of text, and can obtain accurate text characters in the text image by processing the predicted characters in the accurate two-dimensional feature map.

[0265] See also Figure 8 , Figure 8 A flow chart of a second feature extraction neural network training method provided according to an embodiment of the present specification is shown, which specifically includes the following steps.

[0266] Step 802: Displaying a picture input interface to the user based on the user's call request.

[0267] Step 804: receiving a sample picture training set containing text input by the user based on the picture input interface, wherein the sample picture training set includes sample pictures and text labels corresponding to the sample pictures.

[0268] Step 806: construct an initial feature extraction neural network, process the sample image based on the initial feature extraction neural network, obtain a two-dimensional feature map of the sample image, and determine a character position high heat map of the sample image based on the two-dimensional feature map.

[0269] Step 808: Obtain a one-dimensional feature map of the sample image according to the two-dimensional feature map of the sample image and the character position high heat map.

[0270] Step 810: Classify the one-dimensional feature map to obtain a text prediction result of the sample image.

[0271] Step 812: Calculate a loss function based on the text label corresponding to the sample image and the text prediction result, and train the initial feature extraction neural network according to the loss function to obtain the feature extraction neural network.

[0272] It should be noted that, for the part of the second feature extraction neural network training method provided in the embodiments of this specification that corresponds to the embodiment of the first feature extraction neural network training method mentioned above, reference can be made to the detailed description in the embodiment of the first feature extraction neural network training method mentioned above, and will not be repeated here.

[0273] In the embodiments of the present specification, when training the feature extraction neural network, a text image of a single line of text is used to train the feature extraction neural network based on the CTC loss function. By generating pseudo labels, a high heat map of the character positions in the single line of text is supervised and learned, and the initial text prediction results in the predicted one-dimensional feature map are processed by CTC decoding to ensure the recognition accuracy and efficiency of the feature extraction neural network obtained by training.

[0274] See also Fig. 9 , Fig. 9 A flowchart of a third feature extraction neural network training method provided according to an embodiment of the present specification is shown, which specifically includes the following steps.

[0275] Step 902: receiving a call request sent by a user, wherein the call request carries a sample picture training set containing text, and the sample picture training set includes sample pictures and text labels corresponding to the sample pictures.

[0276] Step 904: construct an initial feature extraction neural network.

[0277] Step 906: Process the sample image based on the initial feature extraction neural network to obtain a two-dimensional feature map of the sample image, and determine a character position high heat map of the sample image based on the two-dimensional feature map.

[0278] Step 908: Obtain a one-dimensional feature map of the sample image according to the two-dimensional feature map of the sample image and the character position heat map.

[0279] Step 910: classify the one-dimensional feature map to obtain a text prediction result of the sample image.

[0280] Step 912: Calculate the loss function based on the text label corresponding to the sample image and the text prediction result, and train the initial feature extraction neural network according to the loss function to obtain the feature extraction neural network and return it to the user.

[0281] It should be noted that, for the part of the third feature extraction neural network training method provided in the embodiments of this specification that corresponds to the embodiment of the first feature extraction neural network training method mentioned above, reference can be made to the detailed description in the embodiment of the first feature extraction neural network training method mentioned above, and will not be repeated here.

[0282] In the embodiments of the present specification, when training the feature extraction neural network, a text image of a single line of text is used to train the feature extraction neural network based on the CTC loss function. By generating pseudo labels, a high heat map of the character positions in the single line of text is supervised and learned, and the initial text prediction results in the predicted one-dimensional feature map are processed by CTC decoding to ensure the recognition accuracy and efficiency of the feature extraction neural network obtained by training.

[0283] Corresponding to the above method embodiment, this specification also provides a text recognition device embodiment, Fig.10 FIG. 1 shows a schematic diagram of the structure of a first text recognition device provided by an embodiment of the present specification. Fig.10 As shown, the device comprises:

[0284] The first acquisition module 1002 is configured to acquire a text image to be recognized;

[0285] The first obtaining module 1004 is configured to input the text image into a feature extraction neural network for processing to obtain a two-dimensional feature map of the text image, wherein the two-dimensional feature map includes characters in the text image;

[0286] The second obtaining module 1006 is configured to predict the characters in the two-dimensional feature map and obtain a two-dimensional prediction result map containing the predicted characters;

[0287] The third obtaining module 1008 is configured to process the predicted characters in the two-dimensional prediction result image to obtain recognition results of the characters in the text image.

[0288] Optionally, the third obtaining module 1008 is further configured to:

[0289] Merging eight neighborhoods of the predicted characters in the two-dimensional prediction result graph to obtain a target character generated after the predicted characters are merged, wherein the target character carries a horizontal coordinate and a vertical coordinate in the two-dimensional prediction result graph;

[0290] The target characters are arranged based on the horizontal coordinates and the vertical coordinates of the target characters, and the recognition results of the characters in the text image are determined according to the arrangement results of the target characters.

[0291] Optionally, the third obtaining module 1008 is further configured to:

[0292] The step of arranging the target characters based on the horizontal coordinates and the vertical coordinates of the target characters and determining the recognition results of the characters in the text image according to the arrangement results of the target characters includes:

[0293] sorting the target characters in ascending order based on the horizontal coordinates of the target characters to form a sorted first target character set;

[0294] storing the first target character in the first target character set into the second target character set, determining the ordinate of the first target character as the current sliding average, and deleting the first target character in the first target character set;

[0295] traversing the other target characters in the first target character set except the first target character according to the current sliding average value to obtain a third target character set, wherein the third target character set includes at least one second target character set;

[0296] Based on the ordinate of the first target character in each second target character set, the second target character sets in the third target character set are sorted and then merged, and the recognition result of the characters in the text image is determined according to the merged result.

[0297] Optionally, the first target character is the i-th target character, where i∈[1,n], and i is a positive integer;

[0298] Accordingly, the third obtaining module 1008 is further configured to:

[0299] S1, obtaining the i+1th target character in the first target character set, and calculating the sliding average value of the i+1th target character based on the current sliding average value;

[0300] S2: Determine whether the sliding average value of the i+1 target character is less than a preset sliding threshold value.

[0301] If yes, then storing the i+1th target character into the second target character set, deleting the i+1th target character in the first target character set, and updating the current sliding average value based on the sliding average value of the i+1th target character;

[0302] Determine whether i+1 is greater than n,

[0303] If yes, then the second target character set is stored in the third target character set, and it is determined whether the first target character set is empty.

[0304] If yes, then obtain a third target character set;

[0305] If not, a new second target character set is created, the i-th target character in the first target character set is stored in the newly created second target character set, the ordinate of the i-th target character is determined as the current sliding average, the i-th target character in the first target character set is deleted, and step S1 is continued.

[0306] If not, then i is incremented by 1 and step S1 is continued.

[0307] If not, i is incremented by 1 and step S1 is continued.

[0308] Optionally, the third obtaining module 1008 is further configured to:

[0309] Obtain all second target character sets in the third target character set;

[0310] Determine the ordinate of the first target character in each second target character set;

[0311] The target characters in all the second target character sets are arranged based on the ordinate of the first target character in each second target character set, and the arrangement result of the target characters in all the second target character sets is determined as the recognition result of the characters in the text picture.

[0312] Optionally, the feature extraction neural network is obtained by training a network training module, and the network training module is configured as follows:

[0313] Constructing an initial feature extraction neural network and obtaining a sample image training set containing text, wherein the sample image training set includes sample images and text labels corresponding to the sample images;

[0314] Processing the sample image based on the initial feature extraction neural network to obtain a two-dimensional feature map of the sample image, and determining a character position high heat map of the sample image based on the two-dimensional feature map;

[0315] Obtaining a one-dimensional feature map of the sample image according to the two-dimensional feature map of the sample image and the character position high heat map;

[0316] Classifying the one-dimensional feature map to obtain a text prediction result of the sample image;

[0317] A loss function is calculated based on the text label corresponding to the sample image and the text prediction result, and the initial feature extraction neural network is trained according to the loss function to obtain the feature extraction neural network.

[0318] Optionally, the device further includes:

[0319] The label generation module is configured to pre-process the sample image to generate a pseudo label for the sample image.

[0320] Optionally, the network training module is further configured to:

[0321] A character position heat map of the sample image is determined based on the pseudo label of the sample image and the two-dimensional feature map.

[0322] Optionally, the network training module is further configured to:

[0323] The two-dimensional feature map of the sample image and the character position high heat map are summed in the vertical dimension to obtain a one-dimensional feature map of the sample image.

[0324] Optionally, the network training module is further configured to:

[0325] Classifying the one-dimensional feature map to obtain an initial text prediction result of the sample image;

[0326] The initial text prediction result is subjected to CTC decoding to obtain a text prediction result of the sample image.

[0327] The above is a schematic scheme of a text recognition device of this embodiment. It should be noted that the technical scheme of the text recognition device and the technical scheme of the first text recognition method mentioned above are of the same concept, and the details of the technical scheme of the text recognition device that are not described in detail can be found in the description of the technical scheme of the above text recognition method.

[0328] Corresponding to the above method embodiment, this specification also provides a text recognition device embodiment, Fig.11FIG. 2 shows a schematic diagram of the structure of a second text recognition device provided by an embodiment of the present specification. Fig.11 As shown, the device comprises:

[0329] The first interface display module 1102 is configured to display a picture input interface for the user based on a user's call request;

[0330] The third acquisition module 1104 is configured to acquire the text image to be recognized input by the user based on the image input interface;

[0331] A seventh obtaining module 1106 is configured to input the text image into a feature extraction neural network for processing to obtain a two-dimensional feature map of the text image, wherein the two-dimensional feature map includes characters in the text image;

[0332] An eighth obtaining module 1108 is configured to predict the characters in the two-dimensional feature map to obtain a two-dimensional prediction result map containing the predicted characters;

[0333] The ninth obtaining module 1110 is configured to process the predicted characters in the two-dimensional prediction result image, obtain the recognition results of the characters in the text image and return them to the user.

[0334] The above is a schematic scheme of a text recognition device of this embodiment. It should be noted that the technical scheme of the text recognition device and the technical scheme of the second text recognition method described above are of the same concept, and the details of the technical scheme of the text recognition device that are not described in detail can be found in the description of the technical scheme of the text recognition method described above.

[0335] Corresponding to the above method embodiment, this specification also provides a text recognition device embodiment, Fig.12 FIG. 2 shows a schematic diagram of the structure of a third text recognition device provided by an embodiment of this specification. Fig.12 As shown, the device comprises:

[0336] The first request receiving module 1202 is configured to receive a call request sent by a user, wherein the call request carries a text image to be recognized;

[0337] The first processing module 1204 is configured to input the text image into a feature extraction neural network for processing to obtain a two-dimensional feature map of the text image, wherein the two-dimensional feature map includes characters in the text image;

[0338] A first prediction module 1206 is configured to predict the characters in the two-dimensional feature map to obtain a two-dimensional prediction result map containing the predicted characters;

[0339] The second processing module 1208 is configured to process the predicted characters in the two-dimensional prediction result image, obtain the recognition results of the characters in the text image and return them to the user.

[0340] The above is a schematic scheme of a text recognition device of this embodiment. It should be noted that the technical scheme of the text recognition device and the technical scheme of the third text recognition method mentioned above are of the same concept, and the details of the technical scheme of the text recognition device that are not described in detail can be found in the description of the technical scheme of the text recognition method mentioned above.

[0341] Corresponding to the above method embodiment, this specification also provides a feature extraction neural network training device embodiment, Fig.13 FIG. 1 is a schematic diagram showing the structure of a first feature extraction neural network training device provided by an embodiment of the present specification. Fig.13 As shown, the device comprises:

[0342] The second acquisition module 1302 is configured to construct an initial feature extraction neural network and acquire a sample image training set containing text, wherein the sample image training set includes sample images and text labels corresponding to the sample images;

[0343] A third obtaining module 1304 is configured to process the sample image based on the initial feature extraction neural network to obtain a two-dimensional feature map of the sample image, and determine a character position high heat map of the sample image based on the two-dimensional feature map;

[0344] A fourth obtaining module 1306 is configured to obtain a one-dimensional feature map of the sample image according to the two-dimensional feature map of the sample image and the character position high heat map;

[0345] A fifth obtaining module 1308 is configured to classify the one-dimensional feature map to obtain a text prediction result of the sample image;

[0346] The sixth acquisition module 1310 is configured to calculate a loss function based on the text label corresponding to the sample image and the text prediction result, and train the initial feature extraction neural network according to the loss function to obtain the feature extraction neural network.

[0347] Optionally, the device further includes:

[0348] The preprocessing module is configured to preprocess the sample image to generate a pseudo label for the sample image.

[0349] Optionally, the third obtaining module 1304 is further configured to:

[0350] A character position heat map of the sample image is determined based on the pseudo label of the sample image and the two-dimensional feature map.

[0351] Optionally, the fourth obtaining module 1306 is further configured to:

[0352] The two-dimensional feature map of the sample image and the character position high heat map are summed in the vertical dimension to obtain a one-dimensional feature map of the sample image.

[0353] Optionally, the fifth obtaining module 1308 is further configured to:

[0354] Classifying the one-dimensional feature map to obtain an initial text prediction result of the sample image;

[0355] The initial text prediction result is subjected to CTC decoding to obtain a text prediction result of the sample image.

[0356] The above is a schematic scheme of a feature extraction neural network training device of this embodiment. It should be noted that the technical scheme of the feature extraction neural network training device and the technical scheme of the first feature extraction neural network training method mentioned above belong to the same concept, and the details not described in detail in the technical scheme of the feature extraction neural network training device can be found in the description of the technical scheme of the feature extraction neural network training method mentioned above.

[0357] Corresponding to the above method embodiment, this specification also provides a feature extraction neural network training device embodiment, Fig.14 FIG. 2 shows a schematic diagram of the structure of a second feature extraction neural network training device provided by an embodiment of the present specification. Fig.14 As shown, the device comprises:

[0358] The second interface display module 1402 is configured to display a picture input interface for the user based on a user's call request;

[0359] The sample receiving module 1404 is configured to receive a sample picture training set containing text input by the user based on the picture input interface, wherein the sample picture training set includes sample pictures and text labels corresponding to the sample pictures.

[0360] The third processing module 1406 is configured to construct an initial feature extraction neural network, process the sample image based on the initial feature extraction neural network to obtain a two-dimensional feature map of the sample image, and determine a character position high heat map of the sample image based on the two-dimensional feature map;

[0361] A tenth obtaining module 1408 is configured to obtain a one-dimensional feature map of the sample image according to the two-dimensional feature map of the sample image and the character position high heat map;

[0362] A first classification module 1410 is configured to classify the one-dimensional feature map to obtain a text prediction result of the sample image;

[0363] The first training module 1412 is configured to calculate a loss function based on the text label corresponding to the sample image and the text prediction result, and train the initial feature extraction neural network according to the loss function to obtain the feature extraction neural network.

[0364] The above is a schematic scheme of a feature extraction neural network training device of this embodiment. It should be noted that the technical scheme of the feature extraction neural network training device and the technical scheme of the second feature extraction neural network training method mentioned above belong to the same concept, and the details not described in detail in the technical scheme of the feature extraction neural network training device can be found in the description of the technical scheme of the feature extraction neural network training method mentioned above.

[0365] Corresponding to the above method embodiment, this specification also provides a feature extraction neural network training device embodiment, Fig.15 FIG. 2 shows a schematic diagram of the structure of a third feature extraction neural network training device provided by an embodiment of this specification. Fig.15 As shown, the device comprises:

[0366] The second request receiving module 1502 is configured to receive a call request sent by a user, wherein the call request carries a sample picture training set containing text, and the sample picture training set includes sample pictures and text labels corresponding to the sample pictures;

[0367] The fourth processing module 1504 is configured to construct an initial feature extraction neural network, process the sample image based on the initial feature extraction neural network to obtain a two-dimensional feature map of the sample image, and determine a character position high heat map of the sample image based on the two-dimensional feature map;

[0368] An eleventh obtaining module 1506 is configured to obtain a one-dimensional feature map of the sample image according to the two-dimensional feature map of the sample image and the character position high heat map;

[0369] A second classification module 1508 is configured to classify the one-dimensional feature map to obtain a text prediction result of the sample image;

[0370] The second training module 1510 is configured to calculate a loss function based on the text label corresponding to the sample image and the text prediction result, and train the initial feature extraction neural network according to the loss function to obtain the feature extraction neural network and return it to the user.

[0371] The above is a schematic scheme of a feature extraction neural network training device of this embodiment. It should be noted that the technical scheme of the feature extraction neural network training device and the technical scheme of the third feature extraction neural network training method mentioned above belong to the same concept, and the details not described in detail in the technical scheme of the feature extraction neural network training device can be found in the description of the technical scheme of the feature extraction neural network training method mentioned above.

[0372] Fig.16 The block diagram of a computing device 1600 according to one embodiment of the present specification is shown. The components of the computing device 1600 include but are not limited to a memory 1610 and a processor 1620. The processor 1620 is connected to the memory 1610 via a bus 1630, and the database 1650 is used to store data.

[0373] The computing device 1600 also includes an access device 1640 that enables the computing device 1600 to communicate via one or more networks 1660. Examples of these networks include a public switched telephone network (PSTN), a local area network (LAN), a wide area network (WAN), a personal area network (PAN), or a combination of communication networks such as the Internet. The access device 1640 may include one or more of any type of network interface (e.g., a network interface card (NIC)) whether wired or wireless, such as an IEEE 802.11 wireless local area network (WLAN) wireless interface, a World Wide Interoperability for Microwave Access (Wi-MAX) interface, an Ethernet interface, a universal serial bus (USB) interface, a cellular network interface, a Bluetooth interface, a near field communication (NFC) interface, and the like.

[0374] In one embodiment of the present specification, the above components of the computing device 1600 and Fig.16 Other components not shown in the figure may also be connected to each other, for example, via a bus. It should be understood that Fig.16 The computing device structure block diagram shown is only for the purpose of illustration, and is not intended to limit the scope of this specification. Those skilled in the art can add or replace other components as needed.

[0375] Computing device 1600 may be any type of stationary or mobile computing device, including a mobile computer or mobile computing device (e.g., a tablet computer, a personal digital assistant, a laptop computer, a notebook computer, a netbook, etc.), a mobile phone (e.g., a smart phone), a wearable computing device (e.g., a smart watch, smart glasses, etc.), or other types of mobile devices, or a stationary computing device such as a desktop computer or PC. Computing device 1600 may also be a mobile or stationary server.

[0376] The processor 1620 is used to execute the following computer executable instructions, which, when executed by the processor, implement the steps of the text recognition method or the steps of the feature extraction neural network training method.

[0377] The above is a schematic scheme of a computing device of this embodiment. It should be noted that the technical scheme of the computing device and the technical scheme of the above-mentioned text recognition method or feature extraction neural network training method belong to the same concept, and the details not described in detail in the technical scheme of the computing device can be referred to the description of the technical scheme of the above-mentioned text recognition method or feature extraction neural network training method.

[0378] An embodiment of the present specification also provides a computer-readable storage medium storing computer instructions, which, when executed by a processor, implement the steps of the text recognition method or the steps of the feature extraction neural network training method.

[0379] The above is a schematic scheme of a computer-readable storage medium of this embodiment. It should be noted that the technical scheme of the storage medium and the technical scheme of the above-mentioned text recognition method or feature extraction neural network training method belong to the same concept, and the details not described in detail in the technical scheme of the storage medium can be referred to the description of the technical scheme of the above-mentioned text recognition method or feature extraction neural network training method.

[0380] The above is a description of a specific embodiment of the specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the embodiments and still achieve the desired results. In addition, the processes depicted in the drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0381] The computer instructions include computer program codes, which may be in source code form, object code form, executable files or some intermediate forms, etc. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording medium, USB flash drive, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electric carrier signal, telecommunication signal and software distribution medium, etc. It should be noted that the content contained in the computer-readable medium may be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, computer-readable media do not include electric carrier signals and telecommunication signals.

[0382] It should be noted that, for the above-mentioned method embodiments, for the sake of simplicity of description, they are all expressed as a series of action combinations, but those skilled in the art should be aware that the embodiments of this specification are not limited by the order of the actions described, because according to the embodiments of this specification, some steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the embodiments of this specification.

[0383] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0384] The preferred embodiments of this specification disclosed above are only used to help explain this specification. The optional embodiments do not describe all the details in detail, nor do they limit the invention to the specific implementation methods described. Obviously, many modifications and changes can be made according to the content of the embodiments of this specification. This specification selects and specifically describes these embodiments in order to better explain the principles and practical applications of the embodiments of this specification, so that technicians in the relevant technical field can understand and use this specification well. This specification is limited only by the claims and their full scope and equivalents.

Claims

1. A text recognition method, comprising: Get the text image to be recognized; Inputting the text image into a feature extraction neural network for processing to obtain a two-dimensional feature map of the text image, wherein the two-dimensional feature map includes characters in the text image; Predicting the characters in the two-dimensional feature map to obtain a two-dimensional prediction result map containing the predicted characters; Performing eight-neighborhood merging on the predicted characters in the two-dimensional prediction result image to obtain a target character generated after the predicted characters are merged, wherein the target character carries a horizontal coordinate and a vertical coordinate in the two-dimensional prediction result image, and the eight-neighborhood merging is used to merge the repeated predicted characters in the two-dimensional prediction result image into one character; The target characters are arranged based on the horizontal coordinates and the vertical coordinates of the target characters, and the recognition results of the characters in the text image are determined according to the arrangement results of the target characters.

2. According to the text recognition method of claim 1, the step of arranging the target characters based on the horizontal coordinates and the vertical coordinates of the target characters and determining the recognition results of the characters in the text image according to the arrangement results of the target characters comprises: sorting the target characters in ascending order based on the horizontal coordinates of the target characters to form a sorted first target character set; storing the first target character in the first target character set into the second target character set, determining the ordinate of the first target character as the current sliding average, and deleting the first target character in the first target character set; traversing the other target characters in the first target character set except the first target character according to the current sliding average value to obtain a third target character set, wherein the third target character set includes at least one second target character set; Based on the ordinate of the first target character in each second target character set, the second target character sets in the third target character set are sorted and then merged, and the recognition result of the characters in the text image is determined according to the merged result.

3. The text recognition method according to claim 2, wherein the first target character is the i-th target character; Correspondingly, traversing other target characters in the first target character set except the first target character according to the current sliding average value to obtain a third target character set includes: S1, obtaining the i+1th target character in the first target character set, and calculating the sliding average value of the i+1th target character based on the current sliding average value; S2: Determine whether the sliding average value of the i+1 target character is less than a preset sliding threshold value. If yes, then storing the i+1th target character into the second target character set, deleting the i+1th target character in the first target character set, and updating the current sliding average value based on the sliding average value of the i+1th target character; Determine whether i+1 is greater than n, If yes, then the second target character set is stored in the third target character set, and it is determined whether the first target character set is empty. If yes, then obtain a third target character set; If not, a new second target character set is created, the i-th target character in the first target character set is stored in the newly created second target character set, the ordinate of the i-th target character is determined as the current sliding average, the i-th target character in the first target character set is deleted, and step S1 is continued. If not, then i is incremented by 1 and step S1 is continued. If not, i is incremented by 1 and step S1 is continued.

4. The text recognition method according to claim 3, wherein the second target character sets in the third target character set are sorted and then merged based on the ordinate of the first target character in each second target character set, and the recognition result of the characters in the text image is determined according to the merged result, comprising: Obtain all second target character sets in the third target character set; Determine the ordinate of the first target character in each second target character set; The target characters in all the second target character sets are arranged based on the ordinate of the first target character in each second target character set, and the arrangement result of the target characters in all the second target character sets is determined as the recognition result of the characters in the text picture.

5. According to the text recognition method of claim 1, the training steps of the feature extraction neural network are as follows: Construct an initial feature extraction neural network and obtain a sample image training set containing text, where: The sample image training set includes sample images and text labels corresponding to the sample images, and the sample images are sample images containing a single line of text; Processing the sample image based on the initial feature extraction neural network to obtain a two-dimensional feature map of the sample image, and determining a character position high heat map of the sample image based on the two-dimensional feature map; Sum the two-dimensional feature map of the sample image and the character position high heat map in the vertical dimension to obtain a one-dimensional feature map of the sample image; Classifying the one-dimensional feature map to obtain a text prediction result of the sample image; A loss function is calculated based on the text label corresponding to the sample image and the text prediction result, and the initial feature extraction neural network is trained according to the loss function to obtain the feature extraction neural network.

6. The text recognition method according to claim 5, after obtaining the sample image training set containing text, further comprising: The sample image is preprocessed to generate a pseudo label for the sample image.

7. The text recognition method according to claim 6, wherein determining the character position heat map of the sample image based on the two-dimensional feature map comprises: A character position heat map of the sample image is determined based on the pseudo label of the sample image and the two-dimensional feature map.

8. According to the text recognition method of claim 5, the step of classifying the one-dimensional feature map to obtain the text prediction result of the sample image comprises: Classifying the one-dimensional feature map to obtain an initial text prediction result of the sample image; The initial text prediction result is subjected to CTC decoding to obtain a text prediction result of the sample image.

9. According to the text recognition method according to any one of claims 1-8, the feature extraction neural network comprises a convolutional neural network.

10. A feature extraction neural network training method comprising: Constructing an initial feature extraction neural network and obtaining a sample image training set containing text, wherein the sample image training set includes sample images and text labels corresponding to the sample images, and the sample images are sample images containing a single line of text; Processing the sample image based on the initial feature extraction neural network to obtain a two-dimensional feature map of the sample image, and determining a character position high heat map of the sample image based on the two-dimensional feature map; Sum the two-dimensional feature map of the sample image and the character position high heat map in the vertical dimension to obtain a one-dimensional feature map of the sample image; Classifying the one-dimensional feature map to obtain a text prediction result of the sample image; A loss function is calculated based on the text label corresponding to the sample image and the text prediction result, and the initial feature extraction neural network is trained according to the loss function to obtain the feature extraction neural network.

11. The feature extraction neural network training method according to claim 10, after obtaining the sample image training set containing text, further comprising: The sample image is preprocessed to generate a pseudo label for the sample image.

12. The feature extraction neural network training method according to claim 11, wherein determining the character position high heat map of the sample image based on the two-dimensional feature map comprises: A character position heat map of the sample image is determined based on the pseudo label of the sample image and the two-dimensional feature map.

13. The feature extraction neural network training method according to claim 12, wherein the step of classifying the one-dimensional feature graph to obtain the text prediction result of the sample image comprises: Classifying the one-dimensional feature map to obtain an initial text prediction result of the sample image; The initial text prediction result is subjected to CTC decoding to obtain a text prediction result of the sample image.

14. A text recognition method, comprising: Displaying a picture input interface to the user based on a user's call request; Acquire a text image to be recognized input by the user based on the image input interface; Inputting the text image into a feature extraction neural network for processing to obtain a two-dimensional feature map of the text image, wherein the two-dimensional feature map includes characters in the text image; Predicting the characters in the two-dimensional feature map to obtain a two-dimensional prediction result map containing the predicted characters; Performing eight-neighborhood merging on the predicted characters in the two-dimensional prediction result image to obtain a target character generated after the predicted characters are merged, wherein the target character carries a horizontal coordinate and a vertical coordinate in the two-dimensional prediction result image, and the eight-neighborhood merging is used to merge the repeated predicted characters in the two-dimensional prediction result image into one character; The target characters are arranged based on the horizontal coordinates and the vertical coordinates of the target characters, and the recognition results of the characters in the text image are determined according to the arrangement results of the target characters and returned to the user.

15. A text recognition method, comprising: Receiving a call request sent by a user, wherein the call request carries a text image to be recognized; Inputting the text image into a feature extraction neural network for processing to obtain a two-dimensional feature map of the text image, wherein the two-dimensional feature map includes characters in the text image; Predicting the characters in the two-dimensional feature map to obtain a two-dimensional prediction result map containing the predicted characters; Performing eight-neighborhood merging on the predicted characters in the two-dimensional prediction result image to obtain a target character generated after the predicted characters are merged, wherein the target character carries a horizontal coordinate and a vertical coordinate in the two-dimensional prediction result image, and the eight-neighborhood merging is used to merge the repeated predicted characters in the two-dimensional prediction result image into one character; The target characters are arranged based on the horizontal coordinates and the vertical coordinates of the target characters, and the recognition results of the characters in the text image are determined according to the arrangement results of the target characters and returned to the user.

16. A feature extraction neural network training method comprising: Displaying a picture input interface to the user based on a user's call request; Receiving a sample picture training set containing text input by the user based on the picture input interface, wherein the sample picture training set includes sample pictures and text labels corresponding to the sample pictures, and the sample pictures are sample pictures containing a single line of text; Constructing an initial feature extraction neural network, processing the sample image based on the initial feature extraction neural network to obtain a two-dimensional feature map of the sample image, and determining a character position high heat map of the sample image based on the two-dimensional feature map; Sum the two-dimensional feature map of the sample image and the character position high heat map in the vertical dimension to obtain a one-dimensional feature map of the sample image; Classifying the one-dimensional feature map to obtain a text prediction result of the sample image; A loss function is calculated based on the text label corresponding to the sample image and the text prediction result, and the initial feature extraction neural network is trained according to the loss function to obtain the feature extraction neural network.

17. A feature extraction neural network training method comprising: Receive a call request sent by a user, wherein the call request carries a sample picture training set containing text, the sample picture training set includes sample pictures and text labels corresponding to the sample pictures, and the sample pictures are sample pictures containing a single line of text; Construct an initial feature extraction neural network; Processing the sample image based on the initial feature extraction neural network to obtain a two-dimensional feature map of the sample image, and determining a character position high heat map of the sample image based on the two-dimensional feature map; Sum the two-dimensional feature map of the sample image and the character position high heat map in the vertical dimension to obtain a one-dimensional feature map of the sample image; Classifying the one-dimensional feature map to obtain a text prediction result of the sample image; A loss function is calculated based on the text label corresponding to the sample image and the text prediction result, and the initial feature extraction neural network is trained according to the loss function to obtain the feature extraction neural network and return it to the user.

18. A text recognition device, comprising: A first acquisition module is configured to acquire a text image to be recognized; A first obtaining module is configured to input the text image into a feature extraction neural network for processing to obtain a two-dimensional feature map of the text image, wherein the two-dimensional feature map includes characters in the text image; A second obtaining module is configured to predict the characters in the two-dimensional feature map to obtain a two-dimensional prediction result map containing the predicted characters; The third obtaining module is configured to perform eight-neighborhood merging on the predicted characters in the two-dimensional prediction result image to obtain a target character generated after the predicted characters are merged, wherein the target character carries the horizontal coordinate and the vertical coordinate in the two-dimensional prediction result image, and the eight-neighborhood merging is used to merge the repeated predicted characters in the two-dimensional prediction result image into one character. The target characters are arranged based on the horizontal coordinates and the vertical coordinates of the target characters, and the recognition results of the characters in the text image are determined according to the arrangement results of the target characters.

19. A feature extraction neural network training device, comprising: A second acquisition module is configured to construct an initial feature extraction neural network and acquire a sample image training set containing text, wherein the sample image training set includes sample images and text labels corresponding to the sample images, and the sample images are sample images containing a single line of text; A third acquisition module is configured to process the sample image based on the initial feature extraction neural network to obtain a two-dimensional feature map of the sample image, and determine a character position high heat map of the sample image based on the two-dimensional feature map; A fourth obtaining module is configured to sum the two-dimensional feature map of the sample image and the character position high heat map in a vertical dimension to obtain a one-dimensional feature map of the sample image; A fifth obtaining module is configured to classify the one-dimensional feature map to obtain a text prediction result of the sample image; The sixth acquisition module is configured to calculate a loss function based on the text label corresponding to the sample image and the text prediction result, and train the initial feature extraction neural network according to the loss function to obtain the feature extraction neural network.

20. A text recognition device, comprising: A first interface display module is configured to display a picture input interface for the user based on a user's call request; A third acquisition module is configured to acquire the text image to be recognized input by the user based on the image input interface; a seventh obtaining module, configured to input the text image into a feature extraction neural network for processing, and obtain a two-dimensional feature map of the text image, wherein the two-dimensional feature map includes characters in the text image; an eighth obtaining module, configured to predict the characters in the two-dimensional feature map to obtain a two-dimensional prediction result map containing the predicted characters; A ninth obtaining module is configured to perform eight-neighborhood merging on the predicted characters in the two-dimensional prediction result image to obtain a target character generated after the predicted characters are merged, wherein the target character carries a horizontal coordinate and a vertical coordinate in the two-dimensional prediction result image, and the eight-neighborhood merging is used to merge the repeated predicted characters in the two-dimensional prediction result image into one character. The target characters are arranged based on the horizontal coordinates and the vertical coordinates of the target characters, and the recognition results of the characters in the text image are determined according to the arrangement results of the target characters and returned to the user.

21. A text recognition device, comprising: A first request receiving module is configured to receive a call request sent by a user, wherein the call request carries a text image to be recognized; A first processing module is configured to input the text image into a feature extraction neural network for processing to obtain a two-dimensional feature map of the text image, wherein the two-dimensional feature map includes characters in the text image; A first prediction module is configured to predict the characters in the two-dimensional feature map to obtain a two-dimensional prediction result map containing the predicted characters; The second processing module is configured to perform eight-neighborhood merging on the predicted characters in the two-dimensional prediction result image to obtain a target character generated after the predicted characters are merged, wherein the target character carries the horizontal coordinate and the vertical coordinate in the two-dimensional prediction result image, and the eight-neighborhood merging is used to merge the repeated predicted characters in the two-dimensional prediction result image into one character. The target characters are arranged based on the horizontal coordinates and the vertical coordinates of the target characters, and the recognition results of the characters in the text image are determined according to the arrangement results of the target characters and returned to the user.

22. A feature extraction neural network training device, comprising: A second interface display module is configured to display a picture input interface for the user based on a call request from the user; A sample receiving module is configured to receive a sample picture training set containing text input by the user based on the picture input interface, wherein the sample picture training set includes sample pictures and text labels corresponding to the sample pictures, and the sample pictures are sample pictures containing a single line of text; A third processing module is configured to construct an initial feature extraction neural network, process the sample image based on the initial feature extraction neural network to obtain a two-dimensional feature map of the sample image, and determine a character position high heat map of the sample image based on the two-dimensional feature map; A tenth obtaining module is configured to sum the two-dimensional feature map of the sample image and the character position high heat map in a vertical dimension to obtain a one-dimensional feature map of the sample image; A first classification module is configured to classify the one-dimensional feature map to obtain a text prediction result of the sample image; The first training module is configured to calculate a loss function based on the text label corresponding to the sample image and the text prediction result, and train the initial feature extraction neural network according to the loss function to obtain the feature extraction neural network.

23. A feature extraction neural network training device, comprising: A second request receiving module is configured to receive a call request sent by a user, wherein the call request carries a sample picture training set containing text, the sample picture training set includes sample pictures and text labels corresponding to the sample pictures, and the sample pictures are sample pictures containing a single line of text; a fourth processing module, configured to construct an initial feature extraction neural network, process the sample image based on the initial feature extraction neural network to obtain a two-dimensional feature map of the sample image, and determine a character position high heat map of the sample image based on the two-dimensional feature map; An eleventh obtaining module is configured to sum the two-dimensional feature map of the sample image and the character position high heat map in a vertical dimension to obtain a one-dimensional feature map of the sample image; A second classification module is configured to classify the one-dimensional feature map to obtain a text prediction result of the sample image; The second training module is configured to calculate a loss function based on the text label corresponding to the sample image and the text prediction result, and train the initial feature extraction neural network according to the loss function to obtain the feature extraction neural network and return it to the user.

24. A computing device comprising: Memory and processor; The memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions. When the instructions are executed by the processor, the steps of the text recognition method described in any one of claims 1-9, 14, and 15 are implemented, or the steps of the feature extraction neural network training method described in any one of claims 10-13, 16, and 17 are implemented.

25. A computer-readable storage medium storing computer instructions, which, when executed by a processor, implement the steps of the text recognition method described in any one of claims 1-9, 14, and 15 or the steps of the feature extraction neural network training method described in any one of claims 10-13, 16, and 17.