Text recognition model training method, text recognition method and electronic device

Abstract

The present disclosure provides a training method for a text recognition model, a text recognition method and an electronic device, and relates to the field of image processing technology, in order to solve the problem in the prior art that when encoding and decoding a picture with text to obtain a recognition result, there are differences in the text contained in the picture, resulting in a low recognition accuracy. The training method includes: obtaining a training text image and a marking result of the training text image; wherein the training text image includes text content of at least one text category, and the text content of different text categories is different; inputting the training text image into a preset recognition model for processing, and determining the prediction result of the training text image; based on the prediction result and the marking result, training the preset recognition model until the preset recognition model converges to obtain a text recognition model.

Description

Technical Field

[0001] The present disclosure relates to the field of image processing technology, and in particular to a training method for a text recognition model, a text recognition method, and an electronic device. Background Art

[0002] At present, in natural scene text recognition, it is necessary to identify character sequences from pictures with text (for Chinese, one character is a Chinese character, for English, one character is a letter). For example, first encode the picture with text, and then decode the sequence to get the character sequence. However, in the specific application process, due to the differences in the text contained in the picture, the recognition accuracy is low. Summary of the invention

[0003] The present disclosure provides a training method for a text recognition model, a text recognition method and an electronic device to solve the problem in the prior art that when encoding and then decoding a picture with text to obtain a recognition result, there are differences in the text contained in the picture, resulting in a low recognition accuracy.

[0004] In order to achieve the above objectives, the present disclosure provides the following technical solutions:

[0005] In a first aspect, the present disclosure provides a method for training a text recognition model, comprising: obtaining a training text image and a labeling result of the training text image; wherein the training text image includes text content of at least one text category, and the text content of different text categories is different; inputting the training text image into a preset recognition model for processing to determine a prediction result of the training text image; based on the prediction result and the labeling result, training the preset recognition model until the preset recognition model converges to obtain a text recognition model.

[0006] In some feasible examples, a training text image is input into a preset recognition model for processing to determine a prediction result of the training text image, including: determining a first tensor and a second tensor corresponding to the training text image based on the training text image; wherein the dimension of the first tensor is greater than the dimension of the second tensor; inputting the first tensor into a preconfigured classifier to determine a prediction category of the training text image; wherein the prediction category is any one of the text categories; inputting the second tensor into a preconfigured recognition head to determine a first probability matrix corresponding to each text category of the training text image; wherein one text category corresponds to one recognition head, and one text category corresponds to one first decoding algorithm; according to the second decoding algorithm corresponding to the prediction category, the first probability matrix corresponding to the text category to which the training text image belongs is decoded to determine a prediction result.

[0007] In some feasible examples, based on the training text image, a first tensor and a second tensor corresponding to the training text image are determined, including: performing feature extraction on the training text image to determine the first image after the feature extraction; hard coding the first image to determine the first tensor corresponding to the first image; and performing dimension compression on the first tensor to determine the second tensor after the compressed dimension.

[0008] In some practicable examples, inputting the first tensor into a preconfigured classifier to determine the predicted category of the training text image includes: inputting the first tensor into a preconfigured classifier to determine the classification probability of the training text image corresponding to each text classification. According to the classification probability, determining that the predicted category of the training text image is the text classification corresponding to the maximum classification probability.

[0009] In some implementable examples, a first probability matrix corresponding to the text category to which a training text image belongs is decoded according to a second decoding algorithm corresponding to the prediction category to determine a prediction result, including: decoding a first probability matrix corresponding to the text category to which a training text image belongs according to a second decoding algorithm corresponding to the prediction category to determine a first sequence; querying a character sequence matching the first sequence in a preconfigured dictionary library to determine that the prediction result is a character corresponding to the character sequence.

[0010] In some feasible examples, a preset recognition model is trained based on the prediction results and the marking results until the preset recognition model converges to obtain a text recognition model, including: determining a first loss value according to a first loss function, a prediction category and a marking result corresponding to the classifier; determining a second loss value according to a second loss function, a prediction result and a marking result corresponding to the prediction category; adjusting the network parameters of the classifier based on the first loss value, adjusting the network parameters of the preset recognition model based on the second loss value, returning to execute inputting the training text image into the preset recognition model for processing, determining the prediction result of the training text image, until both the classifier and the recognition head converge to obtain the text recognition model.

[0011] In some feasible examples, the text category includes any one of a language category and a mathematics category; when the text category is a language category or a mathematics category, the first decoding algorithm includes an attention-based method, the second decoding algorithm includes a greedy algorithm, and the second loss function includes a multi-classification cross entropy loss function.

[0012] In some feasible examples, the text category includes an English category; when the text category is an English category, the first decoding algorithm includes a method of temporal class classification based on a neural network, the second decoding algorithm includes a greedy algorithm or a beam search algorithm, and the second loss function includes a loss function of temporal class classification based on a neural network.

[0013] In some feasible examples, the first loss function includes a multi-classification cross entropy loss function.

[0014] In a second aspect, the text recognition method provided by the present disclosure includes: obtaining a text image to be recognized; wherein the text image includes text content of any one of at least one text category, and the text content of different text categories is different; inputting the text image to be recognized into a text recognition model to determine the text content included in the text image to be recognized; wherein the text recognition model is trained by any one of the methods provided in the first aspect.

[0015] In a third aspect, the present disclosure provides a training device for a text recognition model, comprising: an acquisition unit, for acquiring a training text image and a labeling result of the training text image; wherein the training text image comprises text content of at least one text category, and the text content of different text categories is different; a processing unit, for inputting the training text image acquired by the acquisition unit into a preset recognition model for processing, and determining a prediction result of the training text image; the processing unit is also used to train the preset recognition model based on the prediction result and the labeling result acquired by the acquisition unit, until the preset recognition model converges to obtain a text recognition model.

[0016] In some implementable examples, the processing unit is specifically used to determine the first tensor and the second tensor corresponding to the training text image based on the training text image acquired by the acquisition unit; wherein the dimension of the first tensor is greater than the dimension of the second tensor; the processing unit is specifically used to input the first tensor into a preconfigured classifier to determine the predicted category of the training text image; wherein the predicted category is any one of the text categories; the processing unit is specifically used to input the second tensor into a preconfigured recognition head to determine the first probability matrix corresponding to the training text image in each text category; wherein one text category corresponds to one recognition head, and one text category corresponds to one first decoding algorithm; the processing unit is specifically used to decode the first probability matrix corresponding to the text category to which the training text image acquired by the acquisition unit belongs according to the second decoding algorithm corresponding to the predicted category, and determine the prediction result.

[0017] In some feasible examples, the processing unit is specifically used to perform feature extraction on the training text image acquired by the acquisition unit, and determine the first image after the feature extraction; the processing unit is specifically used to hard-code the first image, and determine the first tensor corresponding to the first image; the processing unit is specifically used to perform dimension compression on the first tensor, and determine the second tensor after the compressed dimension.

[0018] In some feasible examples, the processing unit is specifically used to input the first tensor into a preconfigured classifier to determine the classification probability of the training text image corresponding to each text classification; the processing unit is specifically used to determine, based on the classification probability, that the predicted category of the training text image is the text classification corresponding to the maximum classification probability.

[0019] In some feasible examples, the processing unit is specifically used to decode the first probability matrix corresponding to the text category to which the training text image acquired by the acquisition unit belongs according to the second decoding algorithm corresponding to the prediction category, and determine the first sequence; the processing unit is specifically used to query a character sequence matching the first sequence in a preconfigured dictionary library, and determine that the prediction result is the character corresponding to the character sequence.

[0020] In some implementable examples, the processing unit is specifically used to determine a first loss value based on a first loss function, a predicted category, and a labeled result corresponding to the classifier; the processing unit is specifically used to determine a second loss value based on a second loss function, a predicted result, and a labeled result corresponding to the predicted category; the processing unit is specifically used to adjust the network parameters of the classifier based on the first loss value, adjust the network parameters of the preset recognition model based on the second loss value, return to execute inputting the training text image into the preset recognition model for processing, determine the prediction result of the training text image, until the classifier and the recognition head converge, and obtain the text recognition model.

[0021] In some feasible examples, the text category includes any one of a language category and a mathematics category; when the text category is a language category or a mathematics category, the first decoding algorithm includes an attention-based method, the second decoding algorithm includes a greedy algorithm, and the second loss function includes a multi-classification cross entropy loss function.

[0022] In some feasible examples, the text category includes an English category; when the text category is an English category, the first decoding algorithm includes a method of temporal class classification based on a neural network, the second decoding algorithm includes a greedy algorithm or a beam search algorithm, and the second loss function includes a loss function of temporal class classification based on a neural network.

[0023] In some feasible examples, the first loss function includes a multi-classification cross entropy loss function.

[0024] In a fourth aspect, the text recognition device provided by the present disclosure includes: an acquisition unit, used to acquire a text image to be recognized; wherein the text image includes text content of any one of at least one text category, and the text content of different text categories is different; a processing unit, used to input the text image to be recognized acquired by the acquisition unit into a text recognition model to determine the text content included in the text image to be recognized; wherein the text recognition model is trained by any of the methods described in the first aspect.

[0025] In a fifth aspect, the present disclosure provides an electronic device, comprising: a communication interface, a processor, a memory, and a bus; the memory is used to store computer-executable instructions, and the processor and the memory are connected via the bus. When the electronic device is running, the processor executes the computer-executable instructions stored in the memory, so that the electronic device executes the training method of the text recognition model provided in the first aspect above.

[0026] In a sixth aspect, the present disclosure provides a computer-readable storage medium, comprising instructions. When the instructions are executed on a computer, the computer executes the text recognition model training method provided in the first aspect.

[0027] In a seventh aspect, the present disclosure provides a computer program product, which, when executed on a computer, enables the computer to execute the training method of a text recognition model designed in the first aspect.

[0028] In an eighth aspect, the present disclosure provides an electronic device, comprising: a communication interface, a processor, a memory, and a bus; the memory is used to store computer-executable instructions, and the processor and the memory are connected via the bus. When the electronic device is running, the processor executes the computer-executable instructions stored in the memory, so that the electronic device performs the text recognition method provided in the second aspect above.

[0029] In a ninth aspect, the present disclosure provides a computer-readable storage medium, comprising instructions. When the instructions are executed on a computer, the computer is caused to execute the text recognition method provided in the second aspect.

[0030] In a tenth aspect, the present disclosure provides a computer program product, which, when executed on a computer, enables the computer to execute the text recognition method designed in the second aspect.

[0031] It should be noted that the above computer instructions may be stored in whole or in part on a first computer-readable storage medium, wherein the first computer-readable storage medium may be packaged together with the processor of the electronic device, or may be packaged separately from the processor of the electronic device, which is not limited in the present disclosure.

[0032] The description of the third, fifth, sixth and seventh aspects in the present disclosure can refer to the detailed description of the first aspect; and the beneficial effects of the description of the third, fifth, sixth and seventh aspects can refer to the beneficial effects analysis of the first aspect, which will not be repeated here.

[0033] The descriptions of the fourth, eighth, ninth and tenth aspects of the present disclosure may refer to the detailed description of the first aspect; and the beneficial effects of the descriptions of the fourth, eighth, ninth and tenth aspects may refer to the beneficial effects analysis of the second aspect, which will not be repeated here.

[0034] In the present disclosure, the names of the training device for the text recognition model and the text recognition device do not limit the devices or functional modules themselves. In actual implementation, these devices or functional modules may appear with other names. As long as the functions of each device or functional module are similar to those of the present disclosure, they fall within the scope of the claims of the present disclosure and their equivalent technologies.

[0035] These and other aspects of the present disclosure will become more apparent from the following description.

[0036] Compared with the prior art, the technical solution provided by the present invention has the following advantages:

[0037] The training method of the text recognition model provided by the embodiment of the present disclosure trains the preset recognition model through training text images of different text categories, so as to determine the prediction result of the training text image. Afterwards, the preset recognition model is trained according to the prediction result and the marking result of the training text image until the preset recognition model converges to obtain the text recognition model. In this way, when the text image to be recognized is recognized through the text recognition model, since the text recognition model is trained through training text images of different text categories, the text content contained in the text image to be recognized can be more accurately recognized. This solves the problem in the prior art that when encoding and decoding a picture with text to obtain a recognition result, there are differences in the text contained in the picture, resulting in a low recognition accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the present disclosure.

[0039] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0040] Figure 1 One of the flowcharts of the training method of the text recognition model provided in the embodiment of the present disclosure;

[0041] Figure 2 A second flowchart of a method for training a text recognition model provided in an embodiment of the present disclosure;

[0042] Figure 3 A third flowchart of a method for training a text recognition model provided in an embodiment of the present disclosure;

[0043] Figure 4 A flowchart of a text recognition method provided by an embodiment of the present disclosure;

[0044] Figure 5 One of the structural diagrams of the server provided in the embodiment of the present disclosure;

[0045] Figure 6 A second structural diagram of a server provided in an embodiment of the present disclosure;

[0046] Figure 7 A schematic diagram of the structure of a computer program product of a text recognition model training method provided in an embodiment of the present disclosure;

[0047] Figure 8 A third structural diagram of a server provided in an embodiment of the present disclosure;

[0048] Fig. 9 A fourth structural diagram of a server provided in an embodiment of the present disclosure;

[0049] Fig.10 A schematic diagram of the structure of a computer program product of a text recognition method provided by an embodiment of the present disclosure. DETAILED DESCRIPTION

[0050] In order to more clearly understand the above-mentioned objectives, features and advantages of the present disclosure, the scheme of the present disclosure will be further described below. It should be noted that the embodiments of the present disclosure and the features in the embodiments can be combined with each other without conflict.

[0051] In the following description, many specific details are set forth to facilitate a full understanding of the present disclosure, but the present disclosure may also be implemented in other ways different from those described herein; it is obvious that the embodiments in the specification are only part of the embodiments of the present disclosure, rather than all of the embodiments.

[0052] It should be noted that, in this article, relational terms such as "first" and "second" are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

[0053] Resnet18 in the disclosed embodiment refers to Resnet with 18 convolutional layers. Resnet is a well-known natural scene image classification network, which effectively solves the problem of model performance degradation when the number of network model layers increases, so that the network can increase the number of layers (deepen) to extract more complex feature patterns.

[0054] The Transformer in the disclosed embodiment refers to a famous classic network structure proposed in "Attention is all out of touch", which consists of two parts: an encoder and a decoder. Among them, the decoder includes multiple basic modules one, and the basic module one is mainly composed of a multi-head self-attention layer, a jump connection, a layer normalization, and a feedforward neural network. The decoder includes multiple basic modules two. Among them, the basic module two is different from the basic module one and includes two layers of multi-head self-attention layers.

[0055] The following describes the training process of the text recognition model:

[0056] like Figure 1 As shown, a flowchart of a text recognition model training method shown in an embodiment of the present disclosure includes the following S10-S12.

[0057] S10: The server obtains a training text image and a labeling result of the training text image, wherein the training text image includes text content of at least one text category, and the text content of different text categories is different.

[0058] In some examples, the training text images include a large number of training text images of different text categories (each text category includes text images of straight text, inclined text and curved text, as well as conventional blurred and photocopied text images, and some other text images, but these text images are all single-line text images). For example, the training text images include three text categories, namely, Chinese, mathematics and English. Of course, this text category can be expanded at any time (the selection of the three text categories here is very representative. The size of the Chinese and English dictionaries is very different, and the grammatical rules are different. Chinese is recognized by words, while English is recognized by letters. Chinese characters can form any word sentence, but English words are indeed fixed. Therefore, English can use dictionaries to further improve accuracy after recognition, but Chinese can hardly do so. In mathematics, except for simple pure formulas, Chinese characters are generally combined with numbers or English words are combined with numbers. In general, Chinese has stronger context dependence and English is weaker, while mathematics has more complex rules, such as the subscripts and subscripts of summation symbols, various latex formulas, etc., involving a wider range of content).

[0059] It should be noted that the above example is described by taking the training text image including three text categories as an example. The user can set the number of text categories included in the training text image according to the needs, and the embodiments of the present disclosure do not limit this.

[0060] Specifically, the labeling result of the training text image includes the text category corresponding to the training text image and the text content contained in the training text image.

[0061] S11. The server inputs the training text image into a preset recognition model for processing to determine a prediction result of the training text image.

[0062] In some examples, when the server inputs a training text image into a preset recognition model for processing, each input training text image only contains text content of the same text category, ensuring that the preset recognition model can recognize text content of different text categories.

[0063] S12. The server trains a preset recognition model based on the prediction results and the marking results until the preset recognition model converges to obtain a text recognition model.

[0064] As can be seen from the above, in order to ensure that the text recognition model can accurately identify the text content contained in the text image to be recognized, the training method of the text recognition model provided by the embodiment of the present disclosure includes text images of straight text, inclined text and curved text in each text category, and also includes conventional blurred and photocopied text images, as well as some other text images, but these text images are all single-line text images, so that when the preset recognition model is trained by training text images of different text categories, the recognition accuracy of the text recognition model can be better increased. In this way, when the text image to be recognized is recognized by the text recognition model, since the text recognition model is trained by training text images of different text categories, the text content contained in the text image to be recognized can be more accurately recognized. This solves the problem in the prior art that when encoding and decoding a picture with text to obtain a recognition result, there are differences in the text contained in the picture, resulting in a low recognition accuracy.

[0065] In some possible implementation examples, combined with Figure 1 ,like Figure 2 As shown, the above S11 can be specifically implemented through the following S110-S113.

[0066] S110: The server determines a first tensor and a second tensor corresponding to the training text image according to the training text image, wherein the dimension of the first tensor is greater than the dimension of the second tensor.

[0067] S111: The server inputs the first tensor into a pre-configured classifier to determine a predicted category of the training text image, wherein the predicted category is any one of the text categories.

[0068] S112, the server inputs the second tensor into a pre-configured recognition head to determine a first probability matrix corresponding to each text category of the training text image, wherein one text category corresponds to one recognition head, and one text category corresponds to one first decoding algorithm.

[0069] In some examples, in the training method of the text recognition model provided by the embodiments of the present disclosure, the total number of pre-configured recognition heads is the same as the total number of text categories included in the training text image, and the first decoding algorithm includes a method based on the attention mechanism Attention and a method based on the neural network temporal classification (Connectionist Temporal Classification, CTC). Among them, the CTC-based method has a faster processing speed and is suitable for text categories with weak semantic relationships; the Attention-based method has a slightly slower processing speed and is suitable for text categories with strong semantic relationships. For example: In combination with the example given in S10 above, the training text image includes 3 text categories, which are distributed as Chinese categories, mathematics categories and English categories. Because the text content corresponding to different text categories is different. Therefore, for the recognition head used to identify language categories, based on the Attention method, an attention layer, a gated recurrent unit (GRU) layer, a fully connected layer and a word embedding layer are set (the input of GRU consists of two parts, the current input x1 and the hidden state h0 of the previous moment. The specific operation is to use h0 as the query of the attention layer, and the output of the second sub-part as the key and value to obtain the current input x1, and then merge x1 and the output of the previous moment together as x1, pass through GRU, and get h1, h1 passes through the fully connected layer to get the current predicted character, and then the current predicted character passes through the word embedding layer to get the representation vector, which will be merged with the input x2 of the next moment as x2).

[0070] For the recognition head used to identify English categories, a two-layer bidirectional long short-term memory (LSTM) is set based on the CTC method to contextualize the input, and then a fully connected layer is included for mapping.

[0071] The recognition head for identifying math categories uses the same structure as the recognition head for identifying language categories. Since complex math formulas need to take into account more information in the same column, the recognition head for identifying math categories uses the second tensor as q and each column of the first tensor as k and v. Based on the operation of the attention mechanism, a new feature is calculated as the input of the recognition head for identifying math categories. The rest is the same as the recognition head for identifying language categories.

[0072] S113: The server decodes the first probability matrix corresponding to the text category to which the training text image belongs according to the second decoding algorithm corresponding to the predicted category, and determines a prediction result.

[0073] In some examples, combined with the example given in S112 above, since the text content corresponding to different text categories is different, different decoding algorithms need to be used when decoding the first probability matrix corresponding to each text category of the training text image. For example, when the text content contained in the training text image is a text category of a language category or a mathematics category, the second decoding algorithm includes a greedy algorithm, and the second loss function includes a multi-classification cross entropy loss function. When the text content contained in the training text image is an English category, the second decoding algorithm includes a greedy algorithm or a beam search algorithm. In this way, the first probability matrix corresponding to different text categories can be decoded to ensure the accuracy of the prediction results.

[0074] In some possible implementation examples, combined with Figure 2 ,like Figure 3 As shown, the above S110 can be specifically implemented through the following S1100-S1102.

[0075] S1100: The server extracts features from a training text image and determines a first image after feature extraction.

[0076] In some examples, the training method of the text recognition model provided by the embodiments of the present disclosure adopts a modified Resnet18 network. The modified Resnet18 network includes 4 blocks. Each block is composed of several convolutional layers, and the output of each block is the input of the next block.

[0077] Specifically, in the training method of the text recognition model provided by the embodiment of the present disclosure, after the training text image passes through the first two blocks in the modified Resnet18 network, the height and width of the training text image are halved, and the number of input channels and the number of output channels are doubled. Afterwards, when the training text image processed by the first two blocks passes through the last two blocks, the width and height of the training text image processed by the first two blocks remain unchanged, and the number of input channels and the number of output channels are doubled. For example: the number of output channels of the first block is 64, and finally a set of feature maps with H / 4 (indicating 1 / 4 of the height of the training text image), W / 4 (indicating 1 / 4 of the width of the training text image), and the number of output channels is 512.

[0078] In this way, the server can input the training text image into the modified Resnet18 network, and extract features of the training text image through the modified Resnet18 network to obtain the first image. Since the modified Resnet18 network does not perform excessive compression on the training text image, it can ensure that sufficient feature information is extracted to ensure subsequent operations and avoid information loss.

[0079] S1101. The server hard codes a first image to determine a first tensor corresponding to the first image.

[0080] In some examples, the training method of the text recognition model provided by the embodiments of the present disclosure improves the encoder of the Transformer architecture. The specific improvements include:

[0081] 1. In the position encoding part of the encoder, hard coding is used directly, that is, for the input (H / 4, W / 4, 512), according to the position of H and W, from top to bottom and from left to right, use 0, 1... as the encoding of each position respectively.

[0082] Second, the final output of the encoder is no longer two tensors, but one tensor, that is, the encoder still obtains a tensor like (H / 4, W / 4, 512).

[0083] In this way, after the server inputs the first image into the encoder of the Transformer architecture, it can hard-code the first image to obtain the first tensor corresponding to the first image.

[0084] S1102. The server performs dimension compression on the first tensor to determine a second tensor after the dimension compression.

[0085] In some examples, the training method of the text recognition model provided by the embodiments of the present disclosure uses multiple convolutional layers to compress the dimension of the first tensor. For example, the first tensor is (H / 4, W / 4, 512), and after multiple convolutional layers, (H / 4, W / 4, 512) is dimensionally compressed, and the second tensor after the compressed dimension is determined to be (1, L, 512). Among them, 1 represents the number of input images, L represents the set maximum decoding length, and 512 refers to the number of output channels.

[0086] Thus: the server compresses the dimension of the first tensor through multiple convolutional layers, and can determine the second tensor after the compressed dimension.

[0087] In some possible implementation examples, combined with Figure 2 ,like Figure 3 As shown, the above S111 can be specifically implemented through the following S1110 and S1111.

[0088] S1110. The server inputs the first tensor into a preconfigured classifier to determine the classification probability of the training text image corresponding to each text classification.

[0089] In some examples, in the training method of the text recognition model provided by the embodiments of the present disclosure, the preconfigured classifier includes N convolutional layers and M fully connected layers, where N and M are integers greater than or equal to 1, and the number of nodes in the last fully connected layer is equal to the total number of text categories included in the training text image.

[0090] Specifically, N is equal to 3, and M is equal to 2.

[0091] In this way, after the server inputs the first tensor into the pre-configured classifier, it can obtain the classification probability of the text image corresponding to each text classification after being processed by N convolutional layers and M fully connected layers in the classifier.

[0092] S1111. The server determines, based on the classification probability, that the predicted category of the training text image is the text category corresponding to the maximum classification probability.

[0093] In some possible implementation examples, combined with Figure 2 ,like Figure 3 As shown, the above S113 can be specifically implemented through the following S1130 and S1131.

[0094] S1130: The server decodes the first probability matrix corresponding to the text category to which the training text image belongs according to the second decoding algorithm corresponding to the predicted category, and determines a first sequence.

[0095] S1131. The server searches a pre-configured dictionary library for a character sequence that matches the first sequence, and determines that a prediction result is a character corresponding to the character sequence.

[0096] In some examples, in order to be able to identify the text content contained in the training text image, the training text image needs to be encoded so that the server can identify the text content contained in the training text image. In order for the user to determine the text content contained in the training text image, the encoding corresponding to the text content needs to be converted to form content that the user can recognize. Therefore, by collecting historical manual annotation results (including training text images and annotation results of the training text images), that is, annotating the text character information on the training text image, a character sequence can be formed, and at the same time, a dictionary library is established for each text category of the training text image according to the manual annotation results (the dictionary library only includes independent characters). In this way, the server decodes the first probability matrix corresponding to the text category to which the training text image belongs according to the second decoding algorithm corresponding to the predicted category, and determines the first sequence. Since each character corresponds to a character sequence, a character sequence matching the first sequence can be queried in the pre-configured dictionary library, and the predicted result is determined to be the character corresponding to the character sequence. In this way, it can be converted into text content that the user can recognize, ensuring user experience.

[0097] In some possible implementation examples, combined with Figure 1 ,like Figure 2 As shown, the above S12 can be specifically implemented through the following S120-S122.

[0098] S120. The server determines a first loss value according to the first loss function, the predicted category, and the labeling result corresponding to the classifier.

[0099] S121. The server determines a second loss value according to a second loss function, a prediction result and a marking result corresponding to the prediction category.

[0100] S122. The server adjusts the network parameters of the classifier based on the first loss value, adjusts the network parameters of the preset recognition model based on the second loss value, returns to execute inputting the training text image into the preset recognition model for processing, determines the prediction result of the training text image, and obtains the text recognition model until the classifier and the recognition head converge.

[0101] In some examples, in order to ensure that the obtained text recognition model can accurately identify the text content of each text category, when training the classifier and the recognition head, the training text image input each time is from the same text category. For example, when the recognition head for identifying the language category is currently trained, the text content of the training text image is all in the language category. At this time, the recognition head for identifying the English category and the recognition head for identifying the mathematics category assume that their output results are completely correct, that is, the loss values ​​of the recognition head for identifying the English category and the recognition head for identifying the mathematics category are both set to 0, so that the update of the gradient of the recognition head for identifying the language category can be effectively controlled. In this way, after multiple trainings, until the classifier and the recognition head converge, a text recognition model is obtained.

[0102] In some feasible examples, the text category includes any one of a language category and a mathematics category; when the text category is a language category or a mathematics category, the first decoding algorithm includes an Attention-based method, the second decoding algorithm includes a greedy algorithm, and the second loss function includes a multi-classification cross entropy loss function.

[0103] In some feasible examples, the text category includes an English category; when the text category is an English category, the first decoding algorithm includes a neural network-based temporal class classification method, the second decoding algorithm includes a greedy algorithm or a beam search algorithm, and the second loss function includes a CTC loss function.

[0104] In some feasible examples, the first loss function includes a multi-classification cross entropy loss function.

[0105] The following describes the processing process of the text recognition method:

[0106] like Figure 4 As shown, a flowchart of a text recognition method shown in an embodiment of the present disclosure includes the following S20 and S21.

[0107] S20: The server obtains a text image to be recognized, wherein the text image includes text content of any one of at least one text category, and the text content of different text categories is different.

[0108] S21, the server inputs the text image to be recognized into the text recognition model to determine the text content included in the text image to be recognized. The text recognition model is trained by the training method of the text recognition model in the above example.

[0109] The above mainly introduces the solution provided by the embodiment of the present invention from the perspective of the method. In order to achieve the above functions, it includes hardware structures and / or software modules corresponding to the execution of each function. Those skilled in the art should easily realize that, in combination with the units and algorithm steps of each example described in the embodiments disclosed herein, the present invention can be implemented in the form of hardware or a combination of hardware and computer software. Whether a function is executed in the form of hardware or computer software driving hardware depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of the present invention.

[0110] The embodiment of the present invention can divide the functional modules of the server according to the above method example. For example, each functional module can be divided according to each function, or two or more functions can be integrated into one processing module. The above integrated module can be implemented in the form of hardware or in the form of software functional modules. It should be noted that the division of modules in the embodiment of the present invention is schematic and is only a logical functional division. There may be other division methods in actual implementation.

[0111] like Figure 5 As shown, an embodiment of the present invention provides a schematic diagram of the structure of a server 10. The server 10 includes an acquisition unit 101 and a processing unit 102.

[0112] An acquisition unit is used to acquire a training text image and a labeling result of the training text image; wherein the training text image includes text content of at least one text category, and the text content of different text categories is different; a processing unit is used to input the training text image acquired by the acquisition unit into a preset recognition model for processing to determine a prediction result of the training text image; the processing unit is also used to train the preset recognition model based on the prediction result and the labeling result acquired by the acquisition unit until the preset recognition model converges to obtain a text recognition model.

[0113] In some implementable examples, the processing unit is specifically used to determine the first tensor and the second tensor corresponding to the training text image based on the training text image acquired by the acquisition unit; wherein the dimension of the first tensor is greater than the dimension of the second tensor; the processing unit is specifically used to input the first tensor into a preconfigured classifier to determine the predicted category of the training text image; wherein the predicted category is any one of the text categories; the processing unit is specifically used to input the second tensor into a preconfigured recognition head to determine the first probability matrix corresponding to the training text image in each text category; wherein one text category corresponds to one recognition head, and one text category corresponds to one first decoding algorithm; the processing unit is specifically used to decode the first probability matrix corresponding to the text category to which the training text image acquired by the acquisition unit belongs according to the second decoding algorithm corresponding to the predicted category, and determine the prediction result.

[0114] In some feasible examples, the processing unit is specifically used to perform feature extraction on the training text image acquired by the acquisition unit, and determine the first image after the feature extraction; the processing unit is specifically used to hard-code the first image, and determine the first tensor corresponding to the first image; the processing unit is specifically used to perform dimension compression on the first tensor, and determine the second tensor after the compressed dimension.

[0115] In some feasible examples, the processing unit is specifically used to input the first tensor into a preconfigured classifier to determine the classification probability of the training text image corresponding to each text classification; the processing unit is specifically used to determine, based on the classification probability, that the predicted category of the training text image is the text classification corresponding to the maximum classification probability.

[0116] In some feasible examples, the processing unit is specifically used to decode the first probability matrix corresponding to the text category to which the training text image acquired by the acquisition unit belongs according to the second decoding algorithm corresponding to the prediction category, and determine the first sequence; the processing unit is specifically used to query a character sequence matching the first sequence in a preconfigured dictionary library, and determine that the prediction result is the character corresponding to the character sequence.

[0117] In some implementable examples, the processing unit is specifically used to determine a first loss value based on a first loss function, a predicted category, and a labeled result corresponding to the classifier; the processing unit is specifically used to determine a second loss value based on a second loss function, a predicted result, and a labeled result corresponding to the predicted category; the processing unit is specifically used to adjust the network parameters of the classifier based on the first loss value, adjust the network parameters of the preset recognition model based on the second loss value, return to execute inputting the training text image into the preset recognition model for processing, determine the prediction result of the training text image, until the classifier and the recognition head converge, and obtain the text recognition model.

[0118] In some feasible examples, the text category includes any one of a language category and a mathematics category; when the text category is a language category or a mathematics category, the first decoding algorithm includes an attention-based method, the second decoding algorithm includes a greedy algorithm, and the second loss function includes a multi-classification cross entropy loss function.

[0119] In some feasible examples, the text category includes an English category; when the text category is an English category, the first decoding algorithm includes a method of temporal class classification based on a neural network, the second decoding algorithm includes a greedy algorithm or a beam search algorithm, and the second loss function includes a loss function of temporal class classification based on a neural network.

[0120] In some feasible examples, the first loss function includes a multi-classification cross entropy loss function.

[0121] Among them, all relevant contents of each step involved in the above method embodiment can be referred to the functional description of the corresponding functional module, and its role will not be repeated here.

[0122] Of course, the server 10 provided in the embodiment of the present invention includes but is not limited to the above modules, for example, the server 10 may also include a storage unit 103. The storage unit 103 may be used to store the program code of the write server 10, and may also be used to store data generated by the write server 10 during operation, such as data in a write request.

[0123] Figure 6 A schematic diagram of the structure of a server 10 provided in an embodiment of the present invention is shown in FIG. Figure 6 As shown, the server 10 may include: at least one processor 51 , a memory 52 , a communication interface 53 and a communication bus 54 .

[0124] Combine the following Figure 6 The components of the server 10 are described in detail:

[0125] The processor 51 is the control center of the server 10, and may be a processor or a general term for multiple processing elements. For example, the processor 51 is a central processing unit (CPU), or an application specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of the present invention, such as one or more DSPs, or one or more field programmable gate arrays (FPGAs).

[0126] In a specific implementation, as an embodiment, the processor 51 may include one or more CPUs, such as Figure 6 Also, as an embodiment, the server 10 may include multiple processors, such as Figure 6 51 and processor 56 are shown in FIG. Each of these processors may be a single-core processor (Single-CPU) or a multi-core processor (Multi-CPU). The processor here may refer to one or more devices, circuits, and / or processing cores for processing data (e.g., computer program instructions).

[0127] The memory 52 may be a read-only memory (ROM) or other types of static storage devices that can store static information and instructions, a random access memory (RAM) or other types of dynamic storage devices that can store information and instructions, or an electrically erasable programmable read-only memory (EEPROM), a compact disc read-only memory (CD-ROM) or other optical disc storage, optical disc storage (including compressed optical disc, laser disc, optical disc, digital versatile disc, Blu-ray disc, etc.), a magnetic disk storage medium or other magnetic storage device, or any other medium that can be used to carry or store the desired program code in the form of instructions or data structures and can be accessed by a computer, but is not limited thereto. The memory 52 may exist independently and be connected to the processor 51 via a communication bus 54. The memory 52 may also be integrated with the processor 51.

[0128] In a specific implementation, the memory 52 is used to store the data of the present invention and execute the software program of the present invention. The processor 51 can execute various functions of the air conditioner by running or executing the software program stored in the memory 52 and calling the data stored in the memory 52.

[0129] The communication interface 53 uses any transceiver-like device for communicating with other devices or communication networks, such as a Radio Access Network (RAN), a Wireless Local Area Network (WLAN), a terminal, a cloud, etc. The communication interface 53 may include an acquisition unit 102 to implement an acquisition function.

[0130] The communication bus 54 may be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus. The bus may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 6 Only one thick line is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0131] As an example, combining Figure 5 The function implemented by the acquisition unit 102 in the server 10 is the same as Figure 6 The function of the communication interface 53 in the embodiment is the same as that of the processing unit 101. Figure 6 The functions of the processor 51 in the embodiment are the same as those of the storage unit 103. Figure 6 The function of the memory 52 in is the same.

[0132] Another embodiment of the present invention further provides a computer-readable storage medium, in which instructions are stored. When the instructions are executed on a computer, the computer executes the method shown in the above method embodiment.

[0133] In some embodiments, the disclosed methods may be implemented as computer program instructions encoded in a machine-readable format on a computer-readable storage medium or on other non-transitory media or articles of manufacture.

[0134] Figure 7 The diagram schematically shows a conceptual partial view of a computer program product provided by an embodiment of the present invention, wherein the computer program product includes a computer program for executing a computer process on a computing device.

[0135] In one embodiment, the computer program product is provided using a signal bearing medium 410. The signal bearing medium 410 may include one or more program instructions, which when executed by one or more processors may provide the above-mentioned Figure 1 Thus, for example, reference to Figure 1 In the embodiment shown in , one or more features of S10-S12 may be undertaken by one or more instructions associated with the signal bearing medium 410. In addition, Figure 7 The program instructions in also describe example instructions.

[0136] In some embodiments, the signal-bearing medium 410 may include a computer-readable medium 411, such as, but not limited to, a hard drive, a compact disk (CD), a digital video disk (DVD), a digital tape, a memory, a read-only memory (ROM), or a random access memory (RAM), and the like.

[0137] In some implementations, signal bearing medium 410 may include computer recordable medium 412 such as, but not limited to, memory, read / write (R / W) CD, R / W DVD, and the like.

[0138] In some implementations, signal bearing medium 410 may include communication medium 413 such as, but not limited to, digital and / or analog communication media (eg, fiber optic cables, waveguides, wired communication links, wireless communication links, etc.).

[0139] The signal bearing medium 410 may be communicated by a wireless form of communication medium 413 (eg, a wireless communication medium conforming to the IEEE 802.41 standard or other transmission protocol). The one or more program instructions may be, for example, computer executable instructions or logic implemented instructions.

[0140] In some possible implementation examples, such as for Figure 1 The described writing data device may be configured to provide various operations, functions, or actions in response to one or more program instructions via computer-readable medium 411 , computer-recordable medium 412 , and / or communication medium 413 .

[0141] like Figure 8 As shown, an embodiment of the present invention provides a schematic diagram of the structure of a server 20. The server 20 includes an acquisition unit 201 and a processing unit 202.

[0142] An acquisition unit is used to acquire a text image to be recognized; wherein the text image includes text content of any one of at least one text category, and the text content of different text categories is different; a processing unit is used to input the text image to be recognized acquired by the acquisition unit into a text recognition model to determine the text content included in the text image to be recognized; wherein the text recognition model is trained by any of the methods provided in the first aspect.

[0143] Among them, all relevant contents of each step involved in the above method embodiment can be referred to the functional description of the corresponding functional module, and its role will not be repeated here.

[0144] Of course, the server 20 provided in the embodiment of the present invention includes but is not limited to the above modules, for example, the server 20 may further include a storage unit 203. The storage unit 203 may be used to store the program code of the write server 20, and may also be used to store data generated by the write server 20 during operation, such as data in a write request.

[0145] Fig. 9 A schematic diagram of the structure of a server 20 provided in an embodiment of the present invention is shown in FIG. Fig. 9 As shown, the server 20 may include: at least one processor 61 , a memory 62 , a communication interface 63 and a communication bus 64 .

[0146] Combine the following Fig. 9 The components of the server 20 are described in detail:

[0147] The processor 61 is the control center of the server 20, and may be a processor or a general term for multiple processing elements. For example, the processor 61 is a central processing unit (CPU), or an application specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of the present invention, such as one or more DSPs, or one or more field programmable gate arrays (FPGAs).

[0148] In a specific implementation, as an embodiment, the processor 61 may include one or more CPUs, such as Fig. 9 Also, as an embodiment, the server 20 may include multiple processors, such as Fig. 9 61 and processor 66 are shown in FIG. Each of these processors may be a single-core processor (Single-CPU) or a multi-core processor (Multi-CPU). The processor here may refer to one or more devices, circuits, and / or processing cores for processing data (e.g., computer program instructions).

[0149] The memory 62 may be a read-only memory (ROM) or other types of static storage devices that can store static information and instructions, a random access memory (RAM) or other types of dynamic storage devices that can store information and instructions, or an electrically erasable programmable read-only memory (EEPROM), a compact disc read-only memory (CD-ROM) or other optical disc storage, optical disc storage (including compressed optical disc, laser disc, optical disc, digital versatile disc, Blu-ray disc, etc.), a magnetic disk storage medium or other magnetic storage device, or any other medium that can be used to carry or store the desired program code in the form of instructions or data structures and can be accessed by a computer, but is not limited thereto. The memory 62 may exist independently and be connected to the processor 61 via a communication bus 64. The memory 62 may also be integrated with the processor 61.

[0150] In a specific implementation, the memory 62 is used to store the data of the present invention and execute the software program of the present invention. The processor 61 can execute various functions of the air conditioner by running or executing the software program stored in the memory 62 and calling the data stored in the memory 62.

[0151] The communication interface 63 uses any transceiver-like device to communicate with other devices or communication networks, such as Radio Access Network (RAN), Wireless Local Area Networks (WLAN), terminals, cloud, etc. The communication interface 63 may include an acquisition unit 201 to implement the acquisition function.

[0152] The communication bus 64 may be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus. The bus may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Fig. 9 Only one thick line is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0153] As an example, combining Figure 8The function implemented by the acquisition unit 201 in the server 20 is the same as Fig. 9 The function of the communication interface 63 in FIG. 1 is the same as that of the processing unit 202. Fig. 9 The function of the processor 61 in the embodiment is the same as that of the storage unit 203. Fig. 9 The function of the memory 62 in is the same.

[0154] Another embodiment of the present invention further provides a computer-readable storage medium, in which instructions are stored. When the instructions are executed on a computer, the computer executes the method shown in the above method embodiment.

[0155] In some embodiments, the disclosed methods may be implemented as computer program instructions encoded in a machine-readable format on a computer-readable storage medium or on other non-transitory media or articles of manufacture.

[0156] Fig.10 The diagram schematically shows a conceptual partial view of a computer program product provided by an embodiment of the present invention, wherein the computer program product includes a computer program for executing a computer process on a computing device.

[0157] In one embodiment, the computer program product is provided using a signal bearing medium 510. The signal bearing medium 510 may include one or more program instructions, which when executed by one or more processors may provide the above-mentioned Figure 2 Thus, for example, reference to Figure 2 In the embodiment shown in , one or more features of S20 and S21 may be undertaken by one or more instructions associated with the signal bearing medium 510. In addition, Fig.10 The program instructions in also describe example instructions.

[0158] In some embodiments, the signal-bearing medium 510 may include a computer-readable medium 511, such as, but not limited to, a hard drive, a compact disk (CD), a digital video disk (DVD), a digital tape, a memory, a read-only memory (ROM), or a random access memory (RAM), and the like.

[0159] In some implementations, signal bearing medium 510 may include computer recordable medium 512 such as, but not limited to, memory, read / write (R / W) CD, R / W DVD, and the like.

[0160] In some implementations, signal bearing medium 510 may include communication medium 513 such as, but not limited to, digital and / or analog communication media (eg, fiber optic cables, waveguides, wired communication links, wireless communication links, etc.).

[0161] The signal bearing medium 510 may be communicated by a wireless form of communication medium 513 (eg, a wireless communication medium conforming to the IEEE 802.41 standard or other transmission protocol). The one or more program instructions may be, for example, computer executable instructions or logic implemented instructions.

[0162] In some possible implementation examples, such as for Figure 7 The described writing data device may be configured to provide various operations, functions, or actions in response to one or more program instructions via computer-readable medium 511 , computer-recordable medium 512 , and / or communication medium 513 .

[0163] Through the description of the above implementation methods, technical personnel in the relevant field can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional modules is used as an example. In actual applications, the above-mentioned functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above.

[0164] In the several embodiments provided by the present invention, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the modules or units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0165] The units described as separate components may or may not be physically separated, and the components shown as units may be one physical unit or multiple physical units, that is, they may be located in one place or distributed in multiple different places. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0166] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0167] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a readable storage medium. Based on this understanding, the technical solution of the embodiment of the present invention is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for a device (which can be a single-chip microcomputer, chip, etc.) or a processor (processor) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: various media that can store program codes, such as USB flash drives, mobile hard drives, ROM, RAM, magnetic disks or optical disks.

[0168] The above description is only a specific embodiment of the present disclosure, so that those skilled in the art can understand or implement the present disclosure. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present disclosure. Therefore, the present disclosure will not be limited to the embodiments described herein, but will conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A training method for a text recognition model, characterized in that: include: Acquire a training text image and a labeling result of the training text image; wherein the training text image includes text content of at least one text category, and the text content of different text categories is different; each text category includes text images of straight text, inclined text and curved text, and blurred and photocopied text images, and each text image is a single-line text image; Input the training text image into a preset recognition model for processing to determine the prediction result of the training text image; wherein the preset recognition model includes a modified Resnet18 network, a Transformer architecture and multiple convolutional layers, the modified Resnet18 network includes 4 Block blocks, each Block block is composed of several convolutional layers, the output of each Block is the input of the next Block block, the Transformer architecture is used to hard-code the image output by the Resnet18 network to obtain the first tensor corresponding to the image, the position encoding part of the encoder of the Transformer architecture uses hard coding, and the multiple convolutional layers are used to perform dimension compression on the first tensor output by the Transformer architecture; Based on the prediction result and the marking result, the preset recognition model is trained until the preset recognition model converges to obtain the text recognition model.

2. The text recognition model training method according to claim 1, characterized in that: The step of inputting the training text image into a preset recognition model for processing and determining a prediction result of the training text image includes: Determine, according to the training text image, a first tensor and a second tensor corresponding to the training text image; wherein the dimension of the first tensor is greater than the dimension of the second tensor; Inputting the first tensor into a preconfigured classifier to determine a predicted category of the training text image; wherein the predicted category is any one of the text categories; Input the second tensor into a pre-configured recognition head to determine a first probability matrix corresponding to each of the text categories of the training text image; wherein one text category corresponds to one recognition head, and one text category corresponds to one first decoding algorithm; According to the second decoding algorithm corresponding to the predicted category, the first probability matrix corresponding to the text category to which the training text image belongs is decoded to determine a prediction result.

3. The text recognition model training method according to claim 2, characterized in that: The step of determining the first tensor and the second tensor corresponding to the training text image according to the training text image includes: Performing feature extraction on the training text image to determine a first image after feature extraction; Hard-coding the first image to determine a first tensor corresponding to the first image; Perform dimension compression on the first tensor to determine a second tensor with compressed dimensions.

4. The text recognition model training method according to claim 2, characterized in that: The step of inputting the first tensor into a preconfigured classifier to determine a predicted category of the training text image includes: Inputting the first tensor into a preconfigured classifier to determine the classification probability of the training text image corresponding to each text classification; According to the classification probability, the predicted category of the training text image is determined to be the text category corresponding to the maximum classification probability.

5. The text recognition model training method according to claim 2, characterized in that: The step of decoding the first probability matrix corresponding to the text category to which the training text image belongs according to the second decoding algorithm corresponding to the predicted category to determine the prediction result includes: Decoding a first probability matrix corresponding to the text category to which the training text image belongs according to a second decoding algorithm corresponding to the predicted category to determine a first sequence; A character sequence matching the first sequence is searched in a preconfigured dictionary library, and the prediction result is determined to be a character corresponding to the character sequence.

6. The text recognition model training method according to claim 2, characterized in that: The step of training the preset recognition model based on the prediction result and the marking result until the preset recognition model converges to obtain the text recognition model comprises: Determine a first loss value according to a first loss function corresponding to the classifier, the predicted category, and the labeling result; Determining a second loss value according to a second loss function corresponding to the prediction category, the prediction result, and the marking result; The network parameters of the classifier are adjusted based on the first loss value, and the network parameters of the preset recognition model are adjusted based on the second loss value. The training text image is input into the preset recognition model for processing, and the prediction result of the training text image is determined. The text recognition model is obtained until the classifier and the recognition head converge.

7. The text recognition model training method according to claim 6, characterized in that: The text category includes any one of a language category and a mathematics category; When the text category is a language category or a mathematics category, the first decoding algorithm includes an attention-based method, the second decoding algorithm includes a greedy algorithm, and the second loss function includes a multi-classification cross entropy loss function.

8. The text recognition model training method according to claim 6, characterized in that: The text categories include English categories; When the text category is an English category, the first decoding algorithm includes a method of temporal class classification based on a neural network, the second decoding algorithm includes a greedy algorithm or a beam search algorithm, and the second loss function includes a loss function of temporal class classification based on a neural network.

9. The text recognition model training method according to claim 6, characterized in that: The first loss function includes a multi-classification cross entropy loss function.

10. A text recognition method, characterized in that: include: Acquire a text image to be recognized; wherein the text image includes text content of any one of at least one text category, and the text content of different text categories is different; The text image to be recognized is input into a text recognition model to determine the text content included in the text image to be recognized; wherein the text recognition model is trained by the method described in any one of claims 1-9.

11. A training device for a text recognition model, characterized in that: include: An acquisition unit, used to acquire a training text image and a labeling result of the training text image; wherein the training text image includes text content of at least one text category, and the text content of different text categories is different; each text category includes text images of straight text, inclined text and curved text, and blurred and photocopied text images, and each text image is a single-line text image; A processing unit, used for inputting the training text image acquired by the acquisition unit into a preset recognition model for processing, and determining a prediction result of the training text image; wherein the preset recognition model includes a modified Resnet18 network, a Transformer architecture and a plurality of convolutional layers, the modified Resnet18 network includes 4 Block blocks, each Block block is composed of a plurality of convolutional layers, the output of each Block is the input of the next Block block, the Transformer architecture is used to hard-code the output image of the Resnet18 network to obtain a first tensor corresponding to the image, the position encoding part of the encoder of the Transformer architecture uses hard coding, and the plurality of convolutional layers are used to perform dimension compression on the first tensor output by the Transformer architecture; The processing unit is further used to train the preset recognition model based on the prediction result and the marking result obtained by the acquisition unit until the preset recognition model converges to obtain the text recognition model.

12. A text recognition device, characterized in that: include: An acquisition unit, used for acquiring a text image to be recognized; wherein the text image includes text content of any one of at least one text category, and the text content of different text categories is different; a processing unit, used for inputting the text image to be recognized acquired by the acquisition unit into a text recognition model to determine the text content included in the text image to be recognized; wherein the text recognition model is trained by the method described in any one of claims 1 to 9.

13. An electronic device, characterized in that: include: A processor, wherein the processor is used to execute a computer program stored in a memory, wherein the computer program, when executed by the processor, implements the steps of the method described in any one of claims 1 to 9, or the computer program, when executed by the processor, implements the steps of the method described in claim 10.

14. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method described in any one of claims 1 to 9 are implemented, or when the computer program is executed by a processor, the steps of the method described in claim 10 are implemented.

Images