Text correction model training method, text recognition method, and related device

Abstract

The application provides a text correction model training method, a text recognition method and related equipment, and relates to the technical field of text recognition. The text correction model training method comprises: performing text detection on a first sample image to obtain a first target image and a second target image, the first target image being a partial image of the first sample image including a text region, and the second target image being an image obtained by removing background information from the first target image; performing text recognition on the first target image to obtain a first text recognition result and text features corresponding to the first text recognition result; inputting the second target image, the first text recognition result and the text features corresponding to the first text recognition result into a text correction model, training the text correction model based on an output of the text correction model, and the output of the text correction model comprising a correction result and a confidence degree corresponding to the correction result. The embodiment of the application can improve the accuracy of the trained text correction model.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of text recognition, and in particular to a text correction model training method, a text recognition method and related equipment. BACKGROUND

[0002] With the development of information processing technology, the optical character recognition technology based on machine deep learning for character recognition has been greatly improved. Optical character recognition needs to correct text based on the text recognition result to ensure the accuracy of the text recognition result. Currently, when training a text correction model, the text correction model and the text recognition model are in a decoupled state, resulting in a low accuracy of the trained text correction model. SUMMARY

[0003] The embodiments of the present application provide a text correction model training method, a text recognition method and related equipment to solve the problem that the text correction model and the text recognition model are in a decoupled state when training a text correction model, resulting in a low accuracy of the trained text correction model.

[0004] To solve the above technical problems, the present application is implemented as follows:

[0005] In a first aspect, the embodiments of the present application provide a text correction model training method, which comprises:

[0006] performing text detection on a first sample image to obtain a first target image and a second target image, the first target image being a partial image of the first sample image including a text region, and the second target image being an image obtained by removing background information from the first target image;

[0007] performing text recognition on the first target image to obtain a first text recognition result and a text feature corresponding to the first text recognition result;

[0008] inputting the second target image, the first text recognition result and the text feature corresponding to the first text recognition result into a text correction model, training the text correction model based on an output of the text correction model, the output of the text correction model including a correction result and a confidence degree corresponding to the correction result.

[0009] In a second aspect, the embodiments of the present application provide a text recognition method, which comprises:

[0010] obtaining a second text recognition result of a to-be-processed image;

[0011] performing text correction on the second text recognition result by using a text correction model, the text correction model being the text correction model of the first aspect.

[0012] In a third aspect, an embodiment of the present application provides a device for training a text correction model, the device comprising:

[0013] a detection module configured to perform text detection on a first sample image to obtain a first target image and a second target image, the first target image being a partial image of the first sample image including a text region, and the second target image being an image obtained by removing background information from the first target image;

[0014] a recognition module configured to perform text recognition on the first target image to obtain a first text recognition result and a text feature corresponding to the first text recognition result;

[0015] a training module configured to input the second target image, the first text recognition result, and the text feature corresponding to the first text recognition result into a text correction model, and train the text correction model based on an output of the text correction model, the output of the text correction model including a correction result and a confidence degree corresponding to the correction result.

[0016] In a fourth aspect, an embodiment of the present application provides a device for text recognition, the device comprising:

[0017] an acquisition module configured to acquire a second text recognition result of a to-be-processed image;

[0018] a correction module configured to perform text correction on the second text recognition result by using a text correction model, the text correction model being the text correction model of the first aspect.

[0019] In a fifth aspect, an embodiment of the present application provides an electronic device, which comprises a memory, a processor, and a program stored in the memory and capable of running on the processor, and when the program is executed by the processor, the steps in the method for training a text correction model according to the first aspect are implemented; or when the program is executed by the processor, the steps in the method for text recognition according to the second aspect are implemented.

[0020] In a sixth aspect, an embodiment of the present application provides a computer readable storage medium, which stores a computer program, and when the computer program is executed by a processor, the steps in the method for training a text correction model according to the first aspect are implemented; or when the computer program is executed by the processor, the steps in the method for text recognition according to the second aspect are implemented.

[0021] In the embodiment of the present application, text detection is performed on a first sample image to obtain a first target image and a second target image, the first target image is a partial image including a text region in the first sample image, and the second target image is an image after the first target image removes background information; text recognition is performed on the first target image to obtain a first text recognition result and a text feature corresponding to the first text recognition result; the second target image, the first text recognition result and the text feature corresponding to the first text recognition result are input into a text correction model, the text correction model is trained based on an output of the text correction model, the output of the text correction model includes a correction result and a confidence corresponding to the correction result. In this way, when training the text correction model, the text correction model and the text recognition model are tightly coupled through the second target image and the text feature corresponding to the first text recognition result, which can improve the accuracy of the trained text correction model and reduce the cost of data labeling. BRIEF DESCRIPTION OF DRAWINGS

[0022] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings needed in the description of the embodiments of the present application will be briefly introduced as follows. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor on the basis of these drawings.

[0023] Figure 1 is a flowchart of a text correction model training method provided by the embodiment of the present application;

[0024] Figure 2 is a structural schematic diagram of a detection sub-model provided by the embodiment of the present application;

[0025] Figure 3 is a structural schematic diagram of a recognition sub-model provided by the embodiment of the present application;

[0026] Figure 4 is a structural schematic diagram of a text correction model provided by the embodiment of the present application;

[0027] Figure 5 is a schematic diagram of calculating a BERT network input provided by the embodiment of the present application;

[0028] Figure 6 is a flowchart of a text recognition method provided by the embodiment of the present application;

[0029] Figure 7 is a structural schematic diagram of a text correction model training device provided by the embodiment of the present application;

[0030] Figure 8is a structure schematic diagram of a text correction model training device provided by an embodiment of the present application;

[0031] Figure 9 is a structure schematic diagram of a text recognition device provided by an embodiment of the present application;

[0032] Figure 10 is a structure schematic diagram of an electronic device provided by an embodiment of the present application. DETAILED DESCRIPTION

[0033] The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are some but not all of the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative efforts should fall within the protection scope of the present application.

[0034] Reference is made to Figure 1 , Figure 1 is a flowchart of a text correction model training method provided by an embodiment of the present application, as shown in Figure 1 The method comprises the following steps:

[0035] Step 101, performing text detection on a first sample image to obtain a first target image and a second target image, the first target image being a partial image including a text region in the first sample image, and the second target image being an image after removing background information from the first target image.

[0036] The text recognition model can comprise a detection sub-model and a recognition sub-model. Network structures that can achieve text detection effects can all be used as network structures of the detection sub-model. For example, the detection sub-model can be an EAST (Efficient and Accurate Scene Text) model. The first sample image can be subjected to text detection by the detection sub-model. For example, if the detection sub-model is an EAST model, the output of the EAST model can be four diagonal points representing a text box in the first sample image, an inclination angle of the text box, and scores of whether each position point in the first sample image is a text region. The first target image can be obtained by the four diagonal points representing the text box and the inclination angle of the text box in the first sample image. The second target image can be obtained by the first target image and the scores of whether each position point is a text region.

[0037] As a specific implementation, the EAST model can be a fully convolutional network, which can comprise a plurality of convolution (conv) layers and a plurality of unpooling layers. For example, Figure 2As shown, the EAST model can include eight convolutional layers, i.e., a first convolutional layer to an eighth convolutional layer, and three de-pooling layers, i.e., a first de-pooling layer to a third de-pooling layer. The input of the first convolutional layer is the first sample image, the input of the second convolutional layer to the fourth convolutional layer is the output of the previous convolutional layer, the input of the first de-pooling layer is the output of the fourth convolutional layer, the input of the fifth convolutional layer is the concatenation result of the output of the first de-pooling layer and the output of the third convolutional layer, the input of the sixth convolutional layer is the concatenation result of the output of the second de-pooling layer and the output of the second convolutional layer, the input of the seventh convolutional layer is the concatenation result of the output of the third de-pooling layer and the output of the first convolutional layer, and the input of the eighth convolutional layer is the output of the seventh convolutional layer.

[0038] In addition, through the convolution operation and the de-pooling operation of the EAST model, a feature map with the same size as the first sample image can be obtained. Figure 2 The score map represents the score of whether a position point in the first sample image is a text region, the text boxes represent the distance from the position point to the four edges of the text box, and the text angle represents the inclination angle of the text box of the position point. Through the text boxes and the text angle, an inclined text detection box can be obtained, and the first target image can be the result obtained by cropping the first sample image according to the text detection box. The value of the position point corresponding to the score map score less than the preset score in the first sample image is set to 0, i.e., the position of the non-text region is set to 0, and the second target image can be obtained by cropping according to the text detection box, so that the background information in the first target image can be removed, and the influence of the background factor can be removed. The preset score can be 0.4, or 0.6, or 0.8, and the like, and exemplarily can be 0.5. The generation of the first target image and the second target image can correspond to rotating the picture according to the angle of each text detection box so that the text box is horizontal and then cropping.

[0039] It should be noted that the detection sub-model can be trained on a certain real data set, so that it does not need to be retrained or jointly trained. The data for training the detection sub-model can come from the sample set. In order to improve the accuracy of the sample annotation in the sample set, the sample image with the intersection over union of the text box detected by the detection sub-model and the annotated text box less than 0.7 can be deleted from the sample set.

[0040] In step 102, text recognition is performed on the first target image to obtain a first text recognition result and a text feature corresponding to the first text recognition result.

[0041] The network structure that can achieve text recognition effect can be used as the network structure of the recognition sub-model. For example, the recognition sub-model can include a CRNN (Convolutional Recurrent Neural Network) and a CTC (Connectionist Temporal Classification) model. The first target image can be subjected to text recognition by using the recognition sub-model. For example, the recognition sub-model includes a CRNN and a CTC model. The CNN extracts a feature sequence from the first target image, the RNN predicts a label (true value) distribution of the feature sequence obtained from the convolutional layer, and the CTC converts the label distribution obtained from the recurrent layer into a first text recognition result through de-duplication and integration.

[0042] As shown in FIG. 1, Figure 3 For example, the text content in the first sample image includes "sales members produce the same products". Through the output of the CTC, it is determined that t2, t3 and t6, t7 and t12, t13 output by the RNN are the same character, the input of the RNN corresponding to these time sequences is spliced as a feature sequence of the corresponding character, the input of the RNN at other positions is taken as a feature sequence of the corresponding character, denoted as fi, i represents the serial number of the character, each character corresponds to a feature sequence, and f1 to fn are text features corresponding to the first text recognition result.

[0043] It should be noted that the recognition sub-model can be trained on a certain real data set, so that it does not need to be retrained or jointly trained. The data for training the recognition sub-model can come from the real data set and the data generated by the detection sub-model. In order to make the length of the recognized text equal to the length of the real text, sample data in which the length of the recognized text is not equal to the length of the real text in the data generated by the recognition sub-model can be deleted.

[0044] In step 103, the second target image, the first text recognition result, and the text features corresponding to the first text recognition result are input into a text correction model. The text correction model is trained based on the output of the text correction model. The output of the text correction model includes a correction result and a confidence degree corresponding to the correction result.

[0045] The loss value of the correction result can be calculated to update the model parameters of the text correction model in reverse; or the loss value of the correction result and the loss value of the confidence can be calculated respectively, and the model parameters of the text correction model are updated in reverse based on the loss value of the correction result and the loss value of the confidence. For example, the loss value of the correction result and the loss value of the confidence can be weighted and averaged as the loss value of the output of the text correction model to update the model parameters of the text correction model in reverse. The loss value can be calculated in the form of cross entropy.

[0046] It should be noted that the trained text correction model can be used for text recognition, which can improve the accuracy of optical character recognition. A second text recognition result of the to-be-processed image can be obtained, and the trained text correction model is used to perform text correction on the second text recognition result. The second text recognition result of the to-be-processed image can include text detection on the to-be-processed image to obtain a third target image and a fourth target image, the third target image being a partial image including a text region in the to-be-processed image, and the fourth target image being an image obtained by removing background information from the third target image; text recognition is performed on the third target image to obtain a second text recognition result and text features corresponding to the second text recognition result. The trained text correction model is used to perform text correction on the second text recognition result, which can include inputting the fourth target image, the second text recognition result, and the text features corresponding to the second text recognition result into the trained text correction model to perform text correction. The output of the text correction model can include a correction result and a confidence corresponding to the correction result. When the confidence is lower than a preset threshold, the second text recognition result is taken as a final text recognition result; and when the confidence is higher than the preset threshold, the correction result is taken as a final text recognition result.

[0047] In the embodiment of the present application, text detection is performed on a first sample image to obtain a first target image and a second target image, the first target image is a partial image including a text region in the first sample image, and the second target image is an image after the first target image removes background information; text recognition is performed on the first target image to obtain a first text recognition result and a text feature corresponding to the first text recognition result; the second target image, the first text recognition result and the text feature corresponding to the first text recognition result are input into a text correction model, the text correction model is trained based on an output of the text correction model, the output of the text correction model includes a correction result and a confidence corresponding to the correction result. In this way, when training the text correction model, the text correction model and the text recognition model are tightly coupled through the second target image and the text feature corresponding to the first text recognition result, which can improve the accuracy of the trained text correction model and reduce the cost of data labeling.

[0048] Optionally, the text correction model includes a first text correction network and a second text correction network.

[0049] The second target image, the first text recognition result and the text feature corresponding to the first text recognition result are input into the text correction model, including:

[0050] The text feature corresponding to the first text recognition result is input into the first text correction network to obtain a first sub-correction result.

[0051] The second target image and the first text recognition result are input into the second text correction network to obtain a second sub-correction result and the confidence.

[0052] The correction result is determined based on the first sub-correction result and the second sub-correction result.

[0053] The first sub-correction result can be a probability distribution of characters at each position point of the text, or can be a feature value of characters at each position point of the text. The second sub-correction result can be a probability distribution of characters at each position point of the text, or can be a feature value of characters at each position point of the text. The characters at each position point of the text can be determined through the probability distribution or the feature value. The correction result can be a weighted average of the first sub-correction result and the second sub-correction result. For example, the correction result Q can be:

[0054] Qi = α * Pi + (1 - α) * Ti

[0055] Wherein, Pi can be a first sub-correction result, Ti can be a second sub-correction result, i is greater than 1 and less than n, n is the number of characters in the correction result, and a is a preset value. For example, a can be 0.1.

[0056] In addition, the first text correction network can include a convolutional layer, a fully connected layer connected to the convolutional layer, and a normalization layer connected to the fully connected layer; or, as shown in Figure 4 The first text correction network can include a convolutional layer, a normalization layer connected to the convolutional layer; or, a network structure with the same effect can also be used as the network structure of the first text correction network, and the present embodiment does not limit this. As shown in Figure 4 The second text correction network can include a BERT (Bidirectional Encoder Representation from Transformers) network and a convolutional network (CNN), or a network structure with the same effect can also be used as the network structure of the second text correction network, and the present embodiment does not limit this.

[0057] In this embodiment, the first text correction network and the second text correction network are used to build a text correction model, a correction model structure of a text correction model tightly coupled with a text recognition model is proposed, and a better text correction effect can be obtained.

[0058] Optionally, the second text correction network includes a BERT network and a convolutional network, and the second target image and the first text recognition result are input into the second text correction network to obtain a second sub-correction result and the confidence, including:

[0059] The second target image is input into the convolutional network to obtain a convolutional vector;

[0060] The first text recognition result corresponding token embedding vector and position embedding vector, and the convolutional vector are input into the BERT network to obtain a second sub-correction result and the confidence.

[0061] The first text recognition result corresponding token embedding vector can be a word vector, and the first word of the word vector can be an E[CLS] mark used to distinguish whether the text needs to be corrected. The position embedding vector can be used to represent the learned position features. As shown in Figure 5As shown, taking the first text recognition result of "selling the same products produced by members" as an example, the input of the BERT network can be the superposition of Token Embeddings vectors, Position Embeddings vectors and convolution vectors. The convolution vectors can be superimposed on E[CLS] in the Token Embeddings vectors, and the image information can be used to constrain whether the text needs to be corrected. The second sub-correction result can include T1 to Tn, which can be the probability distribution of the characters at each position point of the text.

[0062] As shown in the second text correction network including the BERT network and the convolution network, Figure 4 As shown, the text content in the second target image can be "selling the same products produced by members", and the second target image is processed by the CNN and superimposed on E[CLS] in the Token Embeddings vectors as [CLS] of the input of the BERT network. Tok1 to Tokn obtained by superimposing the Token Embeddings vectors and the Position Embeddings vectors corresponding to the first text recognition result "selling the same products produced by members" are E1 to En of the input of the BERT network. The BERT network deletes the unnecessary SegmentEmbeddings (part of the embedding) vectors. The BERT network outputs the confidence and the second sub-correction result T1 to Tn. The second sub-correction result T1 to Tn is weighted and averaged with the first sub-correction result after the text features f1 to fn corresponding to the first text recognition result are normalized, and the loss value is calculated. When training, 50% of the samples that are different from the real text in the first text recognition result can be retained, and at least one character in the other samples is replaced by a character with a higher similarity than a preset similarity.

[0063] In this embodiment, by inputting the convolution vectors obtained by processing the second target image by the convolution network into the BERT network, the image information in the text detection process can be applied to the text correction, and the accuracy of the trained text correction model can be improved.

[0064] Optionally, the first text correction network includes a convolution layer, a fully connected layer connected to the convolution layer, and a normalization layer connected to the fully connected layer.

[0065] In which, the text features f1 to fn corresponding to the first text recognition result are processed by the convolution layer, the fully connected layer connected to the convolution layer, and the normalization layer connected to the fully connected layer, respectively, to obtain the first sub-correction result with the same dimension as the second sub-correction result.

[0066] In this embodiment, the text features corresponding to the first text recognition result are processed by a convolutional layer, a fully connected layer connected to the convolutional layer, and a normalization layer connected to the fully connected layer to obtain a first sub-correction result, so that the correction result can be determined according to the first sub-correction result.

[0067] Optionally, before the text detection on the first sample image, the method further includes:

[0068] extracting a feature vector corresponding to each character in the preset text content;

[0069] replacing at least one character in the preset text content based on the feature vector;

[0070] performing fusion processing on the replaced preset text content based on a preset background image to obtain a second sample image;

[0071] adding the second sample image to the sample set to obtain an expanded sample set;

[0072] The first sample image is any one sample image in the expanded sample set.

[0073] The feature vector corresponding to each character in the preset text content can be extracted by a character shape model. Replacing at least one character in the preset text content based on the feature vector can be replacing the at least one character in the preset text content with a character having a similarity higher than a preset similarity with the feature vector. The preset text content can be a sentence randomly selected from a pre-stored set of semantically complete sentences. For example, a part of the semantically complete sentence set can be randomly selected to replace one or more characters with other characters similar in shape. For the replaced characters in the preset text content, a label indicating that the character is incorrect can be set. The fusion processing on the replaced preset text content based on the preset background image can be Poisson fusion processing on the replaced preset text content based on the preset background image. For example, the replaced preset text content can be superimposed on a background image similar to the sample image in the sample set, and basic transformations such as rotation, tilt, and color jitter can be performed. The superimposed image can be binarized, then dilated and eroded to obtain a mask for the replaced preset text content. The binarized image and the preset background image can be Poisson fused based on the mask to obtain a second sample image, and the replaced preset text content and its text position in the second sample image can be recorded as a label.

[0074] In the embodiment, the preset text content after replacement is fused based on a preset background image to obtain a second sample image, and the second sample image is added to a sample set, so that the sample set can be expanded, and the influence of background information on the text correction model can be weakened.

[0075] Referring to Figure 6 , Figure 6 is a flowchart of a text recognition method provided by an embodiment of the present application, as shown in Figure 6 , the method comprises the following steps:

[0076] Step 201: obtaining a second text recognition result of a to-be-processed image;

[0077] Step 202: performing text correction on the second text recognition result by using a text correction model, wherein the text correction model is the text correction model provided by the embodiment of the present application.

[0078] In the embodiment of the present application, a second text recognition result of a to-be-processed image is obtained, and a text correction model is used to perform text correction on the second text recognition result, wherein the text correction model is the text correction model provided by the embodiment of the present application. In this way, when the text correction model is trained, the text correction model and the text recognition model are tightly coupled through the second target image and the text features corresponding to the first text recognition result, which can improve the accuracy of the trained text correction model, and the trained text correction model can be applied to optical character recognition, which can improve the accuracy of optical character recognition.

[0079] Referring to Figure 7 , Figure 7 is a structural schematic diagram of a training device of a text correction model provided by an embodiment of the present application, as shown in Figure 7 , the device 300 comprises:

[0080] a detection module 301, configured to perform text detection on a first sample image to obtain a first target image and a second target image, wherein the first target image is a partial image of the first sample image including a text region, and the second target image is an image obtained by removing background information from the first target image;

[0081] an identification module 302, configured to perform text recognition on the first target image to obtain a first text recognition result and text features corresponding to the first text recognition result;

[0082] The training module 303 is configured to input the second target image, the first text recognition result, and a text feature corresponding to the first text recognition result into a text correction model, train the text correction model based on an output of the text correction model, and the output of the text correction model includes a correction result and a confidence degree corresponding to the correction result.

[0083] Optionally, the text correction model includes a first text correction network and a second text correction network.

[0084] The training module 303 is specifically configured to:

[0085] input the text feature corresponding to the first text recognition result into the first text correction network to obtain a first sub-correction result;

[0086] input the second target image and the first text recognition result into the second text correction network to obtain a second sub-correction result and the confidence degree;

[0087] train the text correction model based on the output of the text correction model.

[0088] The correction result is determined based on the first sub-correction result and the second sub-correction result.

[0089] Optionally, the second text correction network includes a BERT network and a convolutional network, and the training module 303 is specifically further configured to:

[0090] input the second target image into the convolutional network to obtain a convolutional vector;

[0091] input a token embedding vector and a position embedding vector corresponding to the first text recognition result, and the convolutional vector into the BERT network to obtain the second sub-correction result and the confidence degree.

[0092] Optionally, the first text correction network includes a convolutional layer, a fully connected layer connected to the convolutional layer, and a normalization layer connected to the fully connected layer.

[0093] Optionally, as shown in Figure 8 The apparatus 300 further includes:

[0094] The extraction module 304 is configured to extract a feature vector corresponding to each character in preset text content.

[0095] The replacement module 305 is configured to replace at least one character in the preset text content based on the feature vector.

[0096] The processing module 306 is configured to perform fusion processing on the replaced preset text content based on a preset background image to obtain a second sample image.

[0097] an adding module 307, configured to add the second sample image into the sample set to obtain an expanded sample set;

[0098] The first sample image is any one sample image in the expanded sample set.

[0099] The training device of the text correction model can implement Figure 1 The various processes implemented in the method embodiments are not repeated here to avoid repetition.

[0100] Referring to Figure 9 , Figure 9 is a structural schematic diagram of a text recognition device provided in an embodiment of the present application, as shown in the figure, the device 400 includes: Figure 9

[0101] The obtaining module 401 is configured to obtain a second text recognition result of a to-be-processed image.

[0102] The correction module 402 is configured to perform text correction on the second text recognition result by using a text correction model, and the text correction model is the text correction model described in the embodiments of the present application.

[0103] The text recognition device can implement Figure 6 The various processes implemented in the method embodiments are not repeated here to avoid repetition.

[0104] As shown in Figure 10 , the embodiments of the present application further provide an electronic device 500, which includes a processor 501, a memory 502, and a program stored in the memory 502 and executable on the processor 501, the program being executed by the processor 501 to implement the various processes of the above-described text correction model training method embodiments, or the program being executed by the processor 501 to implement the various processes of the above-described text recognition method embodiments and achieve the same technical effects, and the same technical effects are not repeated here to avoid repetition.

[0105] The embodiments of the present application further provide a computer readable storage medium, which stores a computer program, the computer program being executed by a processor to implement the various processes of the above-described text correction model training method embodiments, or the computer program being executed by a processor to implement the various processes of the above-described text recognition method embodiments and achieve the same technical effects, and the same technical effects are not repeated here to avoid repetition. The computer readable storage medium is, for example, a ROM, a RAM, a magnetic disc, or an optical disc.

[0106] ​It should be noted that, in this document, the terms "comprises", "comprising", or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but can also include other elements not expressly listed or inherent to such process, method, article, or apparatus. An element proceeded by "comprises... a" does not, without more constraints, exclude the presence of additional identical elements in the process, method, article, or apparatus that comprises the element.

[0107] Those skilled in the art can clearly understand the above-mentioned embodiment method can be realized by means of software and necessary general hardware platform, of course, also can be realized by hardware, but in many cases, the former is a better embodiment. Based on such understanding, the technical solutions of the present application can be embodied in the form of software product, and the computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), including a plurality of instructions to make a terminal (may be a mobile phone, computer, server, air conditioner, or network equipment, etc.) execute the method described in various embodiments of the present application.

[0108] The embodiments of the present application are described above in conjunction with the drawings, but the present application is not limited to the above-mentioned specific embodiments, and the above-mentioned specific embodiments are only illustrative, not restrictive. Those skilled in the art can make many forms under the inspiration of the present application without departing from the purpose of the present application and the scope protected by the claims, which all belong to the protection of the present application.

Claims

1. A training method for a text error correction model, characterized in that, The method comprises: text detection is performed on a first sample image to obtain a first target image and a second target image, the first target image being a partial image including a text region in the first sample image, and the second target image being an image obtained after the first target image removes background information; text recognition is performed on the first target image to obtain a first text recognition result and a text feature corresponding to the first text recognition result; the second target image, the first text recognition result and the text feature corresponding to the first text recognition result are input into a text correction model, the text correction model is trained based on an output of the text correction model, the output of the text correction model includes a correction result and a confidence degree corresponding to the correction result; the text correction model includes a first text correction network and a second text correction network; the inputting of the second target image, the first text recognition result and the text feature corresponding to the first text recognition result into the text correction model comprises: the text feature corresponding to the first text recognition result is input into the first text correction network to obtain a first sub-correction result; the second target image and the first text recognition result are input into the second text correction network to obtain a second sub-correction result and the confidence degree; wherein the correction result is determined based on the first sub-correction result and the second sub-correction result.

2. The method of claim 1, wherein, The second text correction network includes a BERT network and a convolution network, and the inputting of the second target image and the first text recognition result into the second text correction network to obtain the second sub-correction result and the confidence degree comprises: the second target image is input into the convolution network to obtain a convolution vector; a token embedding vector and a position embedding vector corresponding to the first text recognition result and the convolution vector are input into the BERT network to obtain the second sub-correction result and the confidence degree.

3. The method of claim 1, wherein, The first text correction network includes a convolution layer, a fully connected layer connected to the convolution layer, and a normalization layer connected to the fully connected layer.

4. The method of claim 1, wherein, Before the text detection on the first sample image, the method further comprises: extracting a feature vector corresponding to each character in preset text content; replacing at least one character in the preset text content based on the feature vector; performing fusion processing on the replaced preset text content based on a preset background image to obtain a second sample image; adding the second sample image to a sample set to obtain an expanded sample set; wherein the first sample image is any one sample image in the expanded sample set.

5. A text recognition method characterized by, The method comprises: obtaining a second text recognition result of a to-be-processed image; performing text correction on the second text recognition result by using a text correction model, the text correction model being the text correction model of any one of claims 1-4. 6.A device for training a text correction model, comprising: The device comprises: The detection module is configured to perform text detection on the first sample image to obtain a first target image and a second target image, the first target image being a partial image including a text region in the first sample image, and the second target image being an image obtained by removing background information from the first target image. The recognition module is configured to perform text recognition on the first target image to obtain a first text recognition result and a text feature corresponding to the first text recognition result. The training module is configured to input the second target image, the first text recognition result, and the text feature corresponding to the first text recognition result into a text correction model, train the text correction model based on an output of the text correction model, and obtain a correction result and a confidence degree corresponding to the correction result. The text correction model includes a first text correction network and a second text correction network. The training module is specifically configured to: input the text feature corresponding to the first text recognition result into the first text correction network to obtain a first sub-correction result; input the second target image and the first text recognition result into the second text correction network to obtain a second sub-correction result and the confidence degree; train the text correction model based on the output of the text correction model; and determine the correction result based on the first sub-correction result and the second sub-correction result.

7. A text recognition apparatus characterized by comprising: The apparatus includes: an acquisition module configured to acquire a second text recognition result of a to-be-processed image; a correction module configured to perform text correction on the second text recognition result by using a text correction model, the text correction model being the text correction model of any one of claims 1-4.

8. An electronic device, comprising: include: a memory, a processor, and a program stored on the memory and executable on the processor, the program being executed by the processor to implement the steps in the training method of the text correction model according to any one of claims 1-4; or the program being executed by the processor to implement the steps in the text recognition method according to claim 5.

Images