A text recognition method, device and equipment and storage medium

Abstract

The application relates to a text recognition method, a device, equipment and a storage medium. The text recognition method comprises the following steps: acquiring an image to be recognized, wherein the image to be recognized comprises target text; inputting the image to be recognized into a pre-constructed text recognition model, wherein the text recognition model comprises a feature extraction module and a dimension transformation module; performing feature extraction on the image to be recognized through the feature extraction module to generate a two-dimensional feature map, wherein a first feature map in the two-dimensional feature map marks the features of a first part of characters, a second feature map marks the features of a second part of characters, and the first part of characters and the second part of characters constitute the target text; performing dimension transformation on the first feature map and the second feature map through the dimension transformation module to generate a one-dimensional first feature vector; and recognizing the target text according to the first feature vector to generate a text recognition result of the image to be recognized. The application can accurately recognize the text in the image, and the text recognition effect is good.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and in particular to a text recognition method, apparatus, device, and storage medium. Background Technology

[0002] Currently, text recognition technology is widely used in applications such as photo-based question searching, intelligent grading, and automatic text input. The main purpose of text recognition is to identify the specific content of text lines, including printed and handwritten text recognition. Mathematical formula recognition, as a branch of text recognition, is a crucial step in intelligent grading; the accuracy of the recognition results directly affects the accuracy of the grading results. Mathematics problems in compulsory education contain a large number of arithmetic operations and comparison problems. Compared to integer problems, fraction problems often present a challenge due to the distinct hierarchical structure of fractions, making fraction recognition a difficult aspect of mathematical formula recognition. This results in existing recognition algorithms performing poorly on fraction problems. Summary of the Invention

[0003] To address the aforementioned technical problems, this application provides a text recognition method, apparatus, device, and storage medium that can accurately recognize text in images with good recognition results.

[0004] According to one aspect of this application, a text recognition method is provided, comprising:

[0005] Obtain an image to be recognized, wherein the image to be recognized includes target text;

[0006] The image to be recognized is input into a pre-trained text recognition model, wherein the text recognition model includes a feature extraction module and a dimension transformation module;

[0007] The feature extraction module extracts features from the image to be identified to generate a two-dimensional feature map. The first feature map in the two-dimensional feature map marks the features of the first part of the characters, and the second feature map marks the features of the second part of the characters. The first part of the characters and the second part of the characters constitute the target text.

[0008] The dimension transformation module performs dimension transformation on the first feature map and the second feature map to generate a one-dimensional first feature vector.

[0009] The target text is identified based on the first feature vector, and the text recognition result of the image to be recognized is generated.

[0010] According to another aspect of this application, a text recognition device is provided, comprising:

[0011] An acquisition unit is used to acquire an image to be recognized, wherein the image to be recognized includes target text;

[0012] An input unit is used to input the image to be recognized into a pre-trained text recognition model, wherein the text recognition model includes a feature extraction module and a dimension transformation module;

[0013] An extraction unit is used to extract features from the image to be identified through the feature extraction module to generate a two-dimensional feature map. The first feature map in the two-dimensional feature map marks the features of the first part of the characters, and the second feature map marks the features of the second part of the characters. The first part of the characters and the second part of the characters constitute the target text.

[0014] The transformation unit is used to perform dimensional transformation on the first feature map and the second feature map through the dimension transformation module to generate a one-dimensional first feature vector.

[0015] The recognition unit is used to recognize the target text based on the first feature vector and generate the text recognition result of the image to be recognized.

[0016] According to another aspect of this application, an electronic device is provided, the electronic device comprising: a processor; and a memory storing a program, wherein the program includes instructions that, when executed by the processor, cause the processor to perform the text recognition method described above.

[0017] According to another aspect of this application, a non-transitory computer-readable storage medium is provided that stores computer instructions for causing the computer to perform a text recognition method.

[0018] According to another aspect of this application, a computer program product is provided, including a computer program that, when executed by a processor, implements the above-described text recognition method.

[0019] The technical solution provided in this application has the following advantages compared with the prior art:

[0020] The text recognition method includes: acquiring an image to be recognized, wherein the image to be recognized includes target text; inputting the image to be recognized into a pre-constructed text recognition model, wherein the text recognition model includes a feature extraction module and a dimension transformation module; extracting features from the image to be recognized through the feature extraction module to generate a two-dimensional feature map, wherein a first feature map in the two-dimensional feature map marks the features of a first part of the characters, and a second feature map marks the features of a second part of the characters, the first part of the characters and the second part of the characters constitute the target text; transforming the first feature map and the second feature map through the dimension transformation module to generate a one-dimensional first feature vector; recognizing the target text based on the first feature vector, and generating a text recognition result for the image to be recognized. This application can accurately recognize text in images, and the recognition effect is good. Attached Figure Description

[0021] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0022] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0023] Figure 1 A flowchart of the text recognition model training method provided in the embodiments of this application;

[0024] Figure 2 A flowchart illustrating the annotation of sample images provided in this application embodiment;

[0025] Figure 3 A flowchart illustrating dimensional transformation provided in this application embodiment;

[0026] Figure 4 This is a schematic diagram of the structure of the text recognition model provided in the embodiments of this application;

[0027] Figure 5 A flowchart of the text recognition method provided in the embodiments of this application;

[0028] Figure 6 This is a schematic diagram of the structure of the text recognition device provided in the embodiments of this application;

[0029] Figure 7 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0030] To better understand the above-described objectives, features, and advantages of this application, embodiments of the application will be described in more detail below with reference to the accompanying drawings. While some embodiments of the application are shown in the drawings, it should be understood that the application can be implemented in various forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided to provide a more thorough and complete understanding of the application. It should be understood that the accompanying drawings and embodiments are for illustrative purposes only and are not intended to limit the scope of protection of this application.

[0031] It should be understood that the steps described in the method embodiments of this application may be performed in different orders and / or in parallel. Furthermore, the method embodiments may include additional steps and / or omit the steps shown. The scope of this application is not limited in this respect.

[0032] The term "comprising" and its variations as used herein are open-ended, meaning "including but not limited to". The term "based on" means "at least partially based on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one additional embodiment"; the term "some embodiments" means "at least some embodiments". Definitions of other terms will be given in the following description. It should be noted that the concepts of "first", "second", etc., mentioned in this application are used only to distinguish different devices, modules, or units, and are not intended to limit the order of functions performed by these devices, modules, or units or their interdependencies.

[0033] It should be noted that the terms "a" and "a plurality of" used in this application are illustrative rather than restrictive, and those skilled in the art should understand that, unless otherwise expressly indicated in the context, they should be understood as "one or more".

[0034] The names of the messages or information exchanged between multiple devices in the embodiments of this application are for illustrative purposes only and are not intended to limit the scope of these messages or information.

[0035] In recent years, with the rise of deep learning, a large number of methods specifically for text recognition have emerged. Among these, methods suitable for recognizing fractional arithmetic problems mainly fall into the following categories: The first category is text recognition methods based on Convolutional Neural Networks (CNNs) and Connectionist Temporal Classification (CTC) algorithms. A common structure is a combination of CNN and CTC. The advantages of this method are fast computation, low computational cost, and simple model; the disadvantage is its inability to capture the sequential information between features. The second category is deep learning methods represented by Convolutional Recurrent Neural Networks (CRNNs). Compared to the first category, the main feature of this type of method is the introduction of Recurrent Neural Networks (RNNs) to learn the temporal relationships between features. Its typical structure is a combination of CNN, RNN, and CTC. The advantages of this type of method are high algorithmic efficiency and the ability to capture the sequential information between features well; however, its weakness lies in its poor performance in recognizing formulas with a hierarchical structure, such as fractions. The third category is text recognition methods based on the attention mechanism. By introducing the attention mechanism, it can capture key regions in the image well when processing sequence features, such as fraction lines. The disadvantage is that the model structure is complex and the computation is large, and the recognition speed is slow for expressions with a large number of characters.

[0036] Currently, in the field of fraction calculation recognition, algorithms based on the Attention mechanism outperform those based on the CRNN structure. However, in intelligent grading scenarios, besides fraction calculations, there are many other types of text that need to be considered, such as long Chinese texts (more than 15 characters long), English, and Pinyin in word problems. Attention-based algorithms are less accurate and efficient than CRNN-based algorithms for these types of texts. Experiments show that when the character length exceeds 9 characters, the Attention-based algorithm is inferior to the CRNN-based algorithm in both time consumption and accuracy. In summary, Attention-based algorithms are suitable for short texts with complex structures, while CRNN is suitable for long texts with simple structures. However, in intelligent grading scenarios, long texts account for a high proportion, such as in word problems, multiple-choice questions, and fill-in-the-blank questions. Furthermore, intelligent grading has high real-time requirements for algorithms, and neither Attention-based nor CRNN-based algorithms can meet the needs of intelligent grading operations, nor can their recognition performance and accuracy be guaranteed.

[0037] To address the aforementioned technical problems, this application provides a text recognition method. By improving the structure of the text recognition model and the training samples, this application is applicable to the text recognition of different arithmetic problems in intelligent grading scenarios, and it also performs well in recognizing fraction arithmetic problems, significantly improving the accuracy of fraction arithmetic problem recognition. Detailed descriptions are provided below through one or more embodiments.

[0038] Specifically, text recognition methods can be executed by a terminal or a server. Specifically, the terminal or server can use a text recognition model to recognize the target text in the image to be recognized. The entity executing the text recognition model training method and the entity executing the text recognition method can be the same or different.

[0039] For example, in one application scenario, the server trains a text recognition model. The terminal obtains the trained text recognition model from the server and uses it to recognize the target text in the image to be recognized. This image to be recognized can be a photograph taken by the terminal. Alternatively, the image to be recognized can be obtained by the terminal from another device. Or, the image to be recognized can be an image obtained by the terminal after image processing of a preset image; this preset image can be a photograph taken by the terminal or obtained by the terminal from another device. Here, no specific limitation is made to the other devices.

[0040] In another application scenario, the server trains a text recognition model. Further, the server uses the trained text recognition model to identify the target text in the image to be recognized. The server acquires the image to be recognized in a similar way to the terminal acquiring the image described above, and will not be elaborated further here.

[0041] In another application scenario, the terminal trains a text recognition model. Further, the terminal uses the trained text recognition model to identify the target text in the image to be recognized.

[0042] It is understood that the text recognition model training method and text recognition method provided in the embodiments of this application are not limited to the possible scenarios described above. Since the trained text recognition model can be applied in the text recognition method, the text recognition model training method can be introduced below before introducing the text recognition method.

[0043] The following example, using server-side text recognition model training, illustrates a text recognition model training method, specifically the training process. It's understood that this method is also applicable to terminal-side text recognition model training scenarios.

[0044] Figure 1The flowchart of the text recognition model training method provided in this application embodiment is applied to a server and specifically includes, as follows: Figure 1 The following steps S110 to S130 are shown:

[0045] S110. Obtain a sample image and determine the true encoding result of the second character in the sample image.

[0046] Understandably, multiple sample images are acquired as training samples for the text recognition model. These sample images can be question images from intelligent grading scenarios, specifically word problems, multiple-choice questions, and fill-in-the-blank questions, such as various test questions and handwritten answers in primary and secondary school textbooks. Each sample image includes a text line, which can be long or short, containing multiple texts, each including at least one second character. The text can be fractions, letters, operators, etc. For example, a fraction includes multiple second characters such as numerator and denominator. After acquiring the sample images, each image is labeled to obtain the true encoding results of all second characters in the sample image. This is the data preparation before model training, using the true encoding results as labels.

[0047] Optionally, the determination of the true encoding result of the second character in the sample image can be achieved through the following steps:

[0048] The second character in the sample image is labeled according to a preset format to generate a labeling result; a converted sample result corresponding to the labeling result is generated according to a preset conversion rule; a string recognition sample result corresponding to the converted sample result is determined based on a second preset character table; the string recognition sample result is encoded according to a first preset character table to generate a sample real encoding result of the second character.

[0049] The annotation results include the fraction bar identifier, the first part of the sample characters, and the second part of the sample characters.

[0050] Understandably, fractions are two-dimensional sequence texts with an up-down structure, while other texts are one-dimensional sequence texts. Therefore, fractional formulas are transcribed line by line, and other texts can be processed according to conventional data preparation methods. The specific methods of conventional data preparation are not limited here.

[0051] Currently, in terms of data preparation, the commonly used transcription rule is based on the LaTeX format. For example, the fraction 1 / 2 will be transcribed as \frac{1}{2} and used as a label for model training. However, this type of transcription has two problems: 1) The number of transcribed characters is large, with the addition of characters such as \frac in addition to the numerator and denominator, which increases the labeling cost. At the same time, the large number of characters also burdens model training and prediction, resulting in increased model training time; 2) The mechanism of model structure and recognition algorithm is not considered.

[0052] Understandably, to address the aforementioned technical problems, this application provides a transcription rule applicable to fractional arithmetic. Specifically, the second character is labeled according to an optimized preset format to obtain the labeling result. For example, the fraction 1 / 2 in the above example is transcribed as {1} / {2}, and {1} / {2} is the labeling result. This transcription format effectively reduces the number of transcribed characters. Here, {1} is the first part of the sample characters, / is the fraction bar identifier, and {2} is the second part of the sample characters. The three combine to form the complete text 1 / 2, where 1, / , and 2 are all second characters in the sample image. Subsequently, conversion rules are set and a script is written to convert the labeling result, generating the corresponding converted sample result. The converted sample result is based on the labeling result, with a direction identifier added to each second character to distinguish the denominator and numerator in the fraction. After the conversion is complete, a second preset character set needs to be constructed. For text other than fractions, the existing construction method can be used, which will not be elaborated here. In addition, during the construction of the second preset character set, twenty special characters are selected to identify the numerator (0-9) and denominator (0-9). These special characters can be Tibetan, Uyghur, etc., and the selection principle is to avoid repetition with other characters (Chinese characters, letters, etc.) in the second preset character set. This application uses Tibetan as an example; for instance, the number 1 as the numerator (denoted as numerator 1) is represented by Tibetan. To identify, molecule 2 is written in Tibetan. To indicate, the number 1 in the denominator is represented by Tibetan script. To indicate, the denominator 2 is written in Tibetan. To identify the fraction, a second preset character table is constructed based on the association between Tibetan characters and the numbers 0-9 in both the numerator and denominator. Then, the string recognition sample results are determined based on this second preset character table. In the example above, the string recognition sample result for fraction 1 / 2 is... Next, a first preset character table is constructed. This first preset character table is used to identify the relationship between the characters in the string encoding and the second preset character table, for example, to identify the Tibetan script for molecule 1. The string encoding is 10, which identifies the Tibetan character of molecule 2. The string encoding is 11, which identifies the Tibetan character in the denominator 1. The string encoding is 21, which identifies the Tibetan character in the denominator 2. The string encoding is 22. After completing the construction of the first preset character table, the characters in the string recognition sample results are encoded based on the first preset character table to generate the sample's true encoding result. In the above example, the string recognition sample result... The actual encoding result of the sample is 10-22. Understandably, the method for constructing the first and second preset character tables based on other characters is not limited here and can be determined by the user according to their needs.

[0053] Understandably, during the annotation process, the annotation results can be obtained manually, and subsequent conversion and encoding operations can be automatically executed by writing scripts, which can effectively reduce the complexity of manual annotation, reduce the annotation error rate and annotation cost.

[0054] Optionally, the above-mentioned generation of the transformed sample result corresponding to the annotation result according to the preset transformation rules can be achieved through the following steps:

[0055] Based on the score line identifier, determine the first direction identifier corresponding to the first part of the sample characters and the second direction identifier corresponding to the second part of the sample characters; combine the first part of the sample characters and the first direction identifier into a first whole, and combine the second part of the sample characters and the second direction identifier into a second whole; concatenate the first whole and the second whole to generate the transformed sample result corresponding to the annotation result.

[0056] Understandably, the fraction bar identifier in the annotation results is used to determine the first direction identifier corresponding to the first part of the sample characters and the second direction identifier corresponding to the second part of the sample characters. In other words, the fraction bar identifier determines whether the first part of the sample characters and the second part of the sample characters are the numerator or denominator in the same fraction. For example, if the first part of the sample characters is before the fraction bar identifier, then the first part of the sample characters is defined as the characters on the numerator, and the first direction identifier is added to each second character in the first part of the sample characters, which represents the numerator. If the second part of the sample characters is after the fraction bar identifier, then the second part of the sample characters is defined as the characters on the denominator, and the second direction identifier is added to each second character in the second part of the sample characters, which represents the denominator. Based on the above example, in the annotation result {1} / {2}, the first part of the sample characters {1} is before the fraction bar symbol / , indicating that the second character 1 in the first part of the sample characters {1} is the numerator, and the first direction symbol ↑ is determined for it; the second part of the sample characters {2} is after the fraction bar symbol / , indicating that the second character 2 in the second part of the sample characters {2} is the denominator, and the second direction symbol ↓ is determined for it. Conversely, the first part of the sample characters before the fraction bar symbol can also be used as the denominator, and the second part of the sample characters after the fraction bar symbol can be used as the numerator. The specific distinction method is not limited. Then, each second character in the first part of the sample characters and the first direction symbol are combined into a first whole, such as ↑1, and each second character in the second part of the sample characters is combined into a second whole, such as ↓2. Finally, the first whole and the second whole are concatenated to generate the corresponding transformed sample result of the annotation result. For example, the transformed sample result of the annotation result {1} / {2} is ↑1↓2.

[0057] Understandably, the numerator and denominator each include at least one second character, meaning the number of characters in the numerator and / or denominator may be greater than 1. For example, 12 / 34, where the numerator and denominator each include 2 characters. In this case, when converting the annotation result, the splitting of all characters in the numerator and denominator is done by alternating up and down, and the characters in the numerator and denominator are sequentially and alternately spliced ​​together, that is, by alternating splicing one numerator and one denominator.

[0058] Understandably, fractions can be further classified into proper fractions and improper fractions. Improper fractions also exist as mixed numbers, such as one and twelve thirty-fourths. In this case, when labeling the integer 1, the curly braces representing the first part of the sample characters and the second part of the sample characters can be omitted. When converting the labeling results, the integer 1 can also be omitted from the arrow representing the direction.

[0059] For example, see Figure 2 , Figure 2 This is a flowchart of sample image annotation provided in the embodiments of this application. Taking the annotation of scores in a sample image as an example, the process is as follows: 1) Annotation: Annotate the scores according to a preset format. The annotation process is as follows: 1) Annotation: The annotation results include the third part of the sample characters, the first part of the sample characters, the fraction bar identifier, and the second part of the sample characters in sequence. The annotation result is 1{12} / {34}, where the integer 1 is the third part of the sample characters, / is the fraction bar identifier, {12} is the first part of the sample characters, and {34} is the second part of the sample characters. 2) First conversion: Generate the conversion sample result corresponding to the annotation result 1{12} / {34} according to the preset conversion rules. Specifically, the integer 1 is not added with an arrow, ↑ is added before the numerators 1 and 2, and ↓ is added before the denominators 3 and 4. The numerator and denominator are concatenated alternately, and so on, resulting in the conversion sample result 1↑1↓3↑2↓4. 3) Second conversion: Based on the second preset character table, determine the string recognition sample result corresponding to the conversion sample result 1↑1↓3↑2↓4. Treat an arrow and a number as a whole, determine the character corresponding to each whole in the second preset character table, and concatenate them in sequence. Based on the above example, the resulting string recognition result is 1. in, This represents molecule 1 (↑1). This represents the denominator 3 (↓3). This represents molecule 2 (↑2). This represents the denominator 4 (↓4). 4) Encoding: Based on the first preset character table, the string recognition sample result 1 Encoding is performed to generate the sample true encoding results for the score. The encoding corresponding to each string is determined in the first preset character table. The obtained sample true encoding results are 2-10-23-11-24, where encoding 2 is the string encoding for the integer 1, and 10 is... The string encoding, 23 is The string encoding, 11 is String encoding, 24 bits String encoding.

[0060] S120. Input the sample image into the text recognition model, extract the features of the sample image through the feature extraction module to generate a two-dimensional sample feature map; perform dimensional transformation on the two-dimensional sample feature map through the dimensional transformation module to generate a one-dimensional first sample feature vector; construct the temporal relationship of the first sample feature vector through the attention module in the text recognition model to generate a one-dimensional second sample feature vector.

[0061] The text recognition model includes a feature extraction module, a dimension transformation module, an attention module, and a training module.

[0062] Understandably, based on the above S110, after the sample images are labeled, they can be preprocessed. Specifically, bilinear interpolation is used to transform the sample images containing handwritten characters into images of a preset dimension, resulting in processed sample images. For example, the preset dimension could be 32 in height, 512 in width, and 3 channels. The processed sample images are then input into a pre-built text recognition model, which includes a feature extraction module, a dimension transformation module, an attention module, and a training module. After the 32*512*3 processed sample image passes through the feature extraction module, it outputs a two-dimensional sample feature map with a dimension of 2*32*512. The two-dimensional sample feature map is then input into the dimension transformation module for dimension transformation, outputting a one-dimensional first sample feature vector with a dimension of 1*64*512. The first sample feature vector is then input into the attention module to construct the temporal relationship between characters, resulting in a one-dimensional second sample feature vector with a dimension of 1*64*256. Subsequently, the second sample feature vector is input into the training module, and the training model is used to train the text recognition model based on the second sample feature vector and the sample's true encoding result.

[0063] The two-dimensional sample feature map includes a first sample feature map and a second sample feature map. The first sample feature map includes multiple first sample feature values, and the second sample feature map includes multiple second sample feature values.

[0064] Optionally, the two-dimensional sample feature map can be transformed to generate a one-dimensional first sample feature vector, which can be achieved through the following steps:

[0065] Based on the sorting of each feature value in the sample feature map, the multiple first sample feature values ​​and the second sample feature values ​​are sequentially and alternately concatenated to generate a one-dimensional first sample feature vector.

[0066] Understandably, each sample feature map contains multiple ordered sample feature values, and each sample feature value has a corresponding index. Based on the sorting of each feature value in the sample feature map, multiple first sample feature values ​​and multiple second sample feature values ​​are sequentially and alternately concatenated to convert the two-dimensional feature map into a one-dimensional first sample feature vector. For example, a first sample feature value is concatenated with a second sample feature value, and so on, until all sample feature values ​​are concatenated.

[0067] Understandably, the first sample feature map is about the features of the first part of the sample characters, and the second sample feature map is about the features of the second part of the sample characters. Therefore, the process of transforming the two-dimensional sample feature map to obtain the one-dimensional first sample feature vector corresponds to the process of transforming the annotation result into a script to obtain the transformed sample result. That is, the two lines of text above and below the score are converted into one line of text. The first sample feature vector is the feature vector about the transformed sample result.

[0068] For example, see Figure 3 , Figure 3 This is a flowchart of dimensional transformation provided in an embodiment of this application. Figure 3 The image shows a 2*4 two-dimensional sample feature map (a 2*4 feature map), meaning only the height and width of the feature map are shown. The following will use... Figure 3 To illustrate the dimensionality transformation process, a 2*4 two-dimensional sample feature map can be considered as being composed of a 1*4 first sample feature map (first row, dark gray background) and a 1*4 second sample feature map (second row, light gray background). Figure 3 The numbers in the feature maps shown represent the indices of the feature values. In the first sample feature map, the four first sample feature values ​​are numbered 0-4, and in the second sample feature map, the four second sample feature values ​​are numbered 5-8. A dimensionality transformation is performed on the first and second sample feature maps to obtain a one-dimensional first sample feature vector (1*8 feature map). Here, the first sample feature value with indices 1 is followed by the second sample feature value with indices 5, then the first sample feature value with indices 2, and so on. Based on the indices of the feature values, the first and second sample feature values ​​are alternately concatenated to obtain the one-dimensional first sample feature vector.

[0069] S130. Calculate the loss based on the second sample feature vector and the sample real encoding result, and update the network parameters of the text recognition model based on the loss.

[0070] Understandably, based on the above S120, the second sample feature vector and the sample real encoding result are input into the training module to calculate the loss, and the network parameters of the text recognition model are updated by the training module based on the loss.

[0071] Optionally, the loss calculation in S130 above based on the second sample feature vector and the true encoding result of the sample can be implemented through the following steps:

[0072] The sample prediction encoding result is obtained based on the second sample feature vector; the loss is calculated based on the sample prediction encoding result and the sample true encoding result.

[0073] Understandably, based on the feature vector of the second sample, the string encoding is predicted to obtain the sample predicted encoding result. Then, the sample predicted encoding result is compared with the sample real encoding result to calculate the loss. The method for calculating the loss can be the connection time classification algorithm (CTC). The specific method is not limited and can be determined according to the user's needs.

[0074] For example, see Figure 4 , Figure 4 This is a schematic diagram of the text recognition model provided in this application embodiment. The text recognition model adopts a CNN+RNN+CTC structure, and adds an operation to reorganize the output tensor. Specifically, the text recognition model includes a feature extraction module, a dimension transformation module, an attention module, and a training module. The feature extraction module can use an improved residual network (ResNet18). By modifying the pooling layer parameters of ResNet18, the dimension of the feature map output by ResNet18 is 2*32*512, where 2 is the height of the feature map (corresponding to the upper and lower structures of the score), 32 is the width of the feature map, and 512 is the number of channels of the feature map. The output feature map is a two-dimensional sample feature map. It can be understood that characters other than the score are also divided into two parts, that is, the text is also separated in the middle like the score, and the features of the upper part and the features of the lower part are extracted separately. The dimension transformation module performs a dimension transformation on the two-dimensional sample feature map based on the above dimension transformation process. The 2*32*512 two-dimensional sample feature map is transformed to a dimension of 1*64*512, denoted as the one-dimensional first sample feature vector. Thus, the alternating transformation method for the two-dimensional sample feature map corresponds to the alternating transformation method for the annotation results, making the patterns learned by the text recognition model during training more consistent. The attention module can use a two-layer bidirectional LSTM module. The transformed feature vector (first sample feature vector) is fed into the attention module to construct the temporal relationship between characters. The resulting feature vector is the one-dimensional second sample feature vector, with a dimension of 1*64*256. The training module selects the CTC loss function as the objective function. During the text recognition model training process, the second sample feature vector and the sample ground truth are fed into the CTC loss function to calculate the loss.

[0075] The text recognition model training method provided in this application divides the text into two parts during the sample image annotation process, and uses an alternating transformation between the two parts. During the internal recognition process of the text recognition model, a two-dimensional feature map is extracted, and the two feature maps are used to alternate between vertical transformation to correspond the annotation process and the recognition process. This makes the patterns learned by the model during training more consistent, and the trained text recognition model can be better applied to score recognition scenarios. At the same time, it can also take into account other text recognition scenarios besides scores, and the recognition accuracy is also relatively high.

[0076] Based on the above embodiments, Figure 5 This is a flowchart of a text recognition method provided in an embodiment of this application. The following embodiments are described using a terminal as an example, and specifically include the following: Figure 5 The following steps S510 to S550 are shown:

[0077] S510. Obtain the image to be recognized.

[0078] The image to be identified includes target text.

[0079] Understandably, the image to be recognized can be a user-written question image. The image to be recognized includes at least one target text, which may be characters, letters, numbers, etc. The numbers may be in the form of fractions with an upper and lower structure. Specifically, the following embodiment uses a fraction as an example to illustrate the target text.

[0080] S520. Input the image to be recognized into the pre-trained text recognition model.

[0081] The text recognition model includes a feature extraction module, a dimension transformation module, and an attention module.

[0082] Understandably, based on S510, the image to be recognized is input into the text recognition model trained in the above embodiment, and the recognition process is carried out in the text recognition model.

[0083] S530. The feature extraction module extracts features from the image to be identified to generate a two-dimensional feature map.

[0084] In the two-dimensional feature map, the first feature map marks the features of the first part of the characters, and the second feature map marks the features of the second part of the characters. The first part of the characters and the second part of the characters constitute the target text.

[0085] Understandably, based on the above S520, bilinear interpolation is used to transform the dimension of the image to be recognized into a preset dimension, which is 32*512*3. Then, the feature extraction module extracts features of the target text in the transformed image to generate a two-dimensional feature map. The dimension of the two-dimensional feature map is 2*32*512. The two-dimensional feature map includes a first feature map and a second feature map. The first feature map marks the features of the upper half of the target text, which is denoted as the first part of the characters. The second feature map marks the features of the lower half of the target text, which is denoted as the second part of the characters. The first part of the characters and the second part of the characters constitute the complete target text. In other words, the target text is viewed as an upper and lower structure, divided into two parts of characters, and the features of each part are extracted to obtain the two-dimensional feature map.

[0086] S570. The first feature map and the second feature map are transformed by the dimension transformation module to generate a one-dimensional first feature vector.

[0087] The first feature map includes multiple first feature values, and the second feature map includes multiple second feature values.

[0088] Optionally, in step S570 above, the first feature map and the second feature map are subjected to dimensionality transformation to generate a one-dimensional first feature vector, which is specifically achieved through the following steps:

[0089] The first feature value and the second feature value are concatenated to generate a one-dimensional first feature vector.

[0090] Understandably, based on the above S530, after obtaining the two-dimensional feature map, the dimension transformation module transforms the multiple first feature values ​​included in the first feature map and the multiple second feature values ​​included in the second feature map to generate a one-dimensional first feature vector. The dimension of the first feature vector is 1*64*512. For the specific transformation process, please refer to the transformation process during the training of the above text recognition model, which will not be elaborated here.

[0091] S550. Identify the target text based on the first feature vector and generate the text recognition result of the image to be recognized.

[0092] Optionally, the above S550 can be implemented through the following steps:

[0093] The attention module constructs the temporal relationship of the first feature vector to generate a one-dimensional second feature vector, the dimension of which includes height, width and number of channels; the number of channels of the second feature vector is extended to a preset length to obtain a one-dimensional third feature vector, wherein the first feature vector, the second feature vector and the third feature vector have the same height and width; the target text is identified based on the third feature vector to generate the text recognition result of the image to be recognized.

[0094] Understandably, the text recognition model also includes an attention module. Based on the above S570, after obtaining the first feature vector, the first feature vector is input into the attention module. The attention module constructs the temporal relationship between characters and generates a one-dimensional second feature vector with a dimension of 1*64*256. Understandably, the dimensions of the two-dimensional feature map, the first feature vector, and the second feature vector are the same as the dimensions of the two-dimensional sample feature map, the first sample feature vector, and the second sample feature vector generated during model training.

[0095] Understandably, text recognition models also include an inference module. During model application, the training module is unnecessary; only the feature extraction, dimensionality transformation, attention, and inference modules are required. In other words, the inference module replaces the training module. Alternatively, the inference module and the text recognition model can be independent. During model application, the training module is unnecessary; only the feature extraction, dimensionality transformation, and attention modules are required. That is, after the image to be recognized is input into the text recognition model, the model performs the recognition process among its internal modules and directly outputs a one-dimensional second feature vector. Then, the inference module generates the text recognition result of the image based on this second feature vector.

[0096] Understandably, each feature vector has dimensions including height, width, and number of channels. After obtaining the second feature vector, the number of channels of the second feature vector is expanded to a preset length through a linear layer, while the width and height remain unchanged, to obtain a one-dimensional third feature vector. The dimension of the third feature vector is 1*64*map_len, where the preset length is the length of the first preset word table, and the preset length is denoted as map_len.

[0097] Optionally, the above-mentioned identification of the target text based on the third feature vector and generation of the text recognition result of the image to be recognized can be achieved through the following steps:

[0098] The third feature vector is decoded to generate a string encoding result; the string recognition result corresponding to the string encoding result is determined according to the first preset character table; the string recognition result is converted to generate the text recognition result of the image to be recognized.

[0099] Understandably, after obtaining the third feature vector, a softmax operation is performed on the third feature vector along the dimension containing `map_len`. Then, a greedy search algorithm is used to decode the softmax-encoded third feature vector to generate a string encoding result, such as 2-10-23-11-24. After decoding, the string encoding result is further decoded according to a first preset character table to generate a string recognition result, such as 1.

[0100] Optionally, the string recognition result described above can be converted to generate the text recognition result of the image to be recognized. This can be achieved through the following steps:

[0101] The conversion result corresponding to the string recognition result is determined based on the second preset character table; the direction indicator in the conversion result is identified, and multiple first characters included in the conversion result are combined based on the direction indicator to generate the text recognition result of the image to be recognized.

[0102] Understandably, after generating the string recognition result, the string recognition result is transformed based on the second preset character table to generate a transformed result, such as 1↑1↓3↑2↓4. The direction indicators in the transformed result are recognized, including the first direction indicator ↑ representing the numerator and the second direction indicator ↓ representing the denominator. Based on the direction indicators, multiple first characters included in the transformed result are combined to generate the text recognition result of the image to be recognized, for example... In other words, the position of the first character in the fraction is determined based on the direction indicator before each first character and its order in the conversion result. Here, multiple first characters refer to 1, 1, 3, 2, and 4. In the conversion result, 1↑1↓3↑2↓4, a direction indicator and a first character are treated as a whole and sorted. The whole ↓3 has the sequence number 3, and ↓ represents the denominator, meaning that the first character 3 is the first character in the denominator. The whole ↓4 has the sequence number 5, and ↓ represents the denominator, meaning that the first character 4 is the last character in the denominator. Therefore, the denominator of the target text is 34.

[0103] Understandably, the reasoning process and the annotation process are two opposite processes. When reasoning, it is sufficient to obtain the recognition result (score), and there is no need to continue reasoning forward based on the text recognition result to obtain the annotation result.

[0104] The text recognition method provided in this application inputs the image to be recognized into a pre-trained text recognition model for recognition. The complete target text is treated as a top-bottom structure, and the feature values ​​of the characters in the top and bottom parts are extracted to form a two-dimensional feature map. Without affecting the rest of the text except for fractions, this method better adapts to the top-bottom structure of fractions, accurately extracting the features of the numerator and denominator. Subsequently, the two-dimensional feature map undergoes dimensional transformation, alternately concatenating the first and second feature values ​​to obtain a one-dimensional first feature vector, completing the transformation from two-dimensional to one-dimensional features. This alternating concatenation method, while distinguishing the numerator and denominator, also establishes associations with the remaining text divided into two parts, ensuring the accuracy of the remaining text recognition; that is, the features of the top and bottom parts belonging to the same text are adjacent. Finally, based on the first feature vector, a forward inference process, the reverse of the annotation process, is performed to obtain the text recognition result of the image to be recognized. This method is suitable for intelligent grading scenarios of fraction equations, and is not limited by text length or text type. It can accurately recognize the text in the image, and the text recognition effect is good.

[0105] Figure 6 This is a schematic diagram of the structure of a text recognition device provided in an embodiment of this application. The text recognition device provided in this embodiment can execute the processing flow provided in the text recognition method embodiment, such as... Figure 6 As shown, the text recognition device 600 includes an acquisition unit 610, an input unit 620, an extraction unit 630, a transformation unit 670, and a recognition unit 650, wherein:

[0106] The acquisition unit 610 is used to acquire an image to be recognized, wherein the image to be recognized includes target text;

[0107] The input unit 620 is used to input the image to be recognized into a pre-trained text recognition model, wherein the text recognition model includes a feature extraction module and a dimension transformation module;

[0108] The extraction unit 630 is used to extract features from the image to be identified through the feature extraction module to generate a two-dimensional feature map, wherein the first feature map in the two-dimensional feature map marks the features of the first part of the characters, and the second feature map marks the features of the second part of the characters, and the first part of the characters and the second part of the characters constitute the target text;

[0109] Transformation unit 670 is used to perform dimensional transformation on the first feature map and the second feature map through the dimensional transformation module to generate a one-dimensional first feature vector.

[0110] The recognition unit 650 is used to recognize the target text based on the first feature vector and generate the text recognition result of the image to be recognized.

[0111] Optionally, in device 600, the first feature map includes a first feature value, and the second feature map includes a second feature value.

[0112] Optionally, the transformation unit 670 is used for:

[0113] The first feature value and the second feature value are concatenated to generate a one-dimensional first feature vector.

[0114] Optionally, the text recognition model in device 600 may further include an attention module.

[0115] Optionally, the identification unit 650 is used for:

[0116] The attention module constructs the temporal relationship of the first feature vector to generate a one-dimensional second feature vector, wherein the dimension of the feature vector includes height, width and number of channels;

[0117] The number of channels of the second feature vector is expanded to a preset length to obtain a one-dimensional third feature vector, wherein the first feature vector, the second feature vector, and the third feature vector have the same height and width;

[0118] The target text is identified based on the third feature vector, and the text recognition result of the image to be recognized is generated.

[0119] Optionally, the identification unit 650 is used for:

[0120] The third feature vector is decoded to generate a string encoding result;

[0121] The string recognition result corresponding to the string encoding result is determined according to the first preset character table;

[0122] The string recognition result is converted to generate the text recognition result of the image to be recognized.

[0123] Optionally, the identification unit 650 is used for:

[0124] The conversion result corresponding to the string recognition result is determined based on the second preset character table;

[0125] The direction identifier in the conversion result is identified, and multiple first characters included in the conversion result are combined based on the direction identifier to generate the text recognition result of the image to be recognized.

[0126] Optionally, device 600 is also used for:

[0127] Acquire a sample image and determine the true encoding result of the second character in the sample image;

[0128] The sample image is input into the text recognition model, and the features of the sample image are extracted by the feature extraction module to generate a two-dimensional sample feature map. The two-dimensional sample feature map is transformed by the dimension transformation module to generate a one-dimensional first sample feature vector. The temporal relationship of the first sample feature vector is constructed by the attention module in the text recognition model to generate a one-dimensional second sample feature vector.

[0129] The loss is calculated based on the second sample feature vector and the true encoding result of the sample, and the network parameters of the text recognition model are updated based on the loss.

[0130] Optionally, device 600 is also used for:

[0131] The second character in the sample image is labeled according to a preset format, and the labeling result is generated.

[0132] Generate the corresponding transformed sample results according to the preset transformation rules;

[0133] The string recognition sample result corresponding to the conversion sample result is determined based on the second preset character table;

[0134] The string recognition sample result is encoded according to the first preset character table to generate the sample real encoding result of the second character.

[0135] Optionally, the annotation results in device 600 include a fraction bar identifier, a first part of sample characters, and a second part of sample characters.

[0136] Optionally, device 600 is also used for:

[0137] Based on the fraction line identifier, determine the first direction identifier corresponding to the first part of sample characters and the second direction identifier corresponding to the second part of sample characters;

[0138] The first part of the sample characters and the first direction identifier are combined into a first whole, and the second part of the sample characters and the second direction identifier are combined into a second whole;

[0139] The first whole and the second whole are spliced ​​together to generate the transformed sample result corresponding to the annotation result.

[0140] The device provided in this embodiment has the same implementation principle and technical effect as the aforementioned method embodiment. For the sake of brevity, any parts not mentioned in the device embodiment can be referred to the corresponding content in the aforementioned method embodiment.

[0141] An exemplary embodiment of this application also provides an electronic device, including: at least one processor; and a memory communicatively connected to the at least one processor. The memory stores a computer program executable by the at least one processor, the computer program being executed by the at least one processor to cause the electronic device to perform a method according to an embodiment of this application.

[0142] An exemplary embodiment of this application also provides a computer program product, including a computer program, wherein, when executed by a computer's processor, the computer program is used to cause the computer to perform a method according to an embodiment of this application.

[0143] refer to Figure 7 The present invention describes a structural block diagram of an electronic device 700 that can serve as a server or client of this application, which is an example of a hardware device that can be applied to various aspects of this application. The electronic device is intended to represent various forms of digital electronic computer devices, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the application described and / or claimed herein.

[0144] like Figure 7 As shown, the electronic device 700 includes a computing unit 701, which can perform various appropriate actions and processes based on a computer program stored in a read-only memory (ROM) 702 or a computer program loaded from a storage unit 708 into a random access memory (RAM) 703. The RAM 703 may also store various programs and data required for the operation of the device 700. The computing unit 701, ROM 702, and RAM 703 are interconnected via a bus 704. An input / output (I / O) interface 705 is also connected to the bus 704.

[0145] Multiple components in electronic device 700 are connected to I / O interface 705, including: input unit 706, output unit 707, storage unit 708, and communication unit 709. Input unit 706 can be any type of device capable of inputting information to electronic device 700. Input unit 706 can receive input digital or character information and generate key signal inputs related to user settings and / or function control of electronic device. Output unit 707 can be any type of device capable of presenting information and may include, but is not limited to, a display, speaker, video / audio output terminal, vibrator, and / or printer. Storage unit 704 may include, but is not limited to, disk and optical disk. Communication unit 709 allows electronic device 700 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks, and may include, but is not limited to, modems, network cards, infrared communication devices, wireless communication transceivers, and / or chipsets, such as Bluetooth™ devices, WiFi devices, WiMax devices, cellular communication devices, and / or the like.

[0146] The computing unit 701 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 701 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 701 performs the various methods and processes described above. For example, in some embodiments, the text recognition method or text recognition mechanism can be implemented as a computer software program tangibly contained in a machine-readable medium, such as storage unit 708. In some embodiments, part or all of the computer program can be loaded and / or installed on the electronic device 700 via ROM 702 and / or communication unit 709. In some embodiments, the computing unit 701 can be configured to perform the text recognition method by any other suitable means (e.g., by means of firmware).

[0147] The program code used to implement the methods of this application may be written in any combination of one or more programming languages. This program code may be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable data processing device, such that when executed by the processor or controller, the functions / operations specified in the flowcharts and / or block diagrams are implemented. The program code may be executed entirely on a machine, partially on a machine, as a standalone software package partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0148] In the context of this application, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. Machine-readable media can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0149] As used in this application, the terms "machine-readable medium" and "computer-readable medium" refer to any computer program product, device, and / or apparatus (e.g., disk, optical disk, memory, programmable logic device (PLD)) for providing machine instructions and / or data to a programmable processor, including machine-readable media that receive machine instructions as machine-readable signals. The term "machine-readable signal" refers to any signal for providing machine instructions and / or data to a programmable processor.

[0150] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device for displaying information to the user (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor); and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the computer. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0151] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as a data server), or computing systems that include middleware components (e.g., an application server), or computing systems that include frontend components (e.g., a user computer with a graphical user interface or web browser through which a user can interact with embodiments of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., a communication network). Examples of communication networks include local area networks (LANs), wide area networks (WANs), and the Internet.

[0152] Computer systems can include clients and servers. Clients and servers are generally located far apart and typically interact through communication networks. Client-server relationships are created by computer programs running on the respective computers and having a client-server relationship with each other.

[0153] The above description is merely a specific embodiment of this application, enabling those skilled in the art to understand or implement this application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of this application. Therefore, this application is not to be limited to the embodiments described herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A text recognition method, characterized in that, include: Obtain an image to be recognized, wherein the image to be recognized includes the target text of a mathematical equation; The image to be recognized is input into a pre-trained text recognition model, wherein the text recognition model includes a feature extraction module and a dimension transformation module; The feature extraction module extracts features from the image to be identified to generate a two-dimensional feature map. The first feature map in the two-dimensional feature map marks the features of the first part of the characters, and the second feature map marks the features of the second part of the characters. The first part of the characters and the second part of the characters constitute the complete target text. The first part of the characters is the upper half of the target text, and the second part of the characters is the lower half of the target text. The dimension transformation module performs alternating dimension transformations on the first feature map and the second feature map to generate a one-dimensional first feature vector. The target text is identified based on the first feature vector, and the text recognition result of the image to be recognized is generated.

2. The method according to claim 1, characterized in that, The first feature map includes a first feature value, and the second feature map includes a second feature value. The step of transforming the first feature map and the second feature map to generate a one-dimensional first feature vector includes: The first feature value and the second feature value are concatenated to generate a one-dimensional first feature vector.

3. The method according to claim 1, characterized in that, The text recognition model further includes an attention module, wherein the step of recognizing the target text based on the first feature vector and generating the text recognition result of the image to be recognized includes: The attention module constructs the temporal relationship of the first feature vector to generate a one-dimensional second feature vector, wherein the dimension of the feature vector includes height, width and number of channels; The number of channels of the second feature vector is expanded to a preset length to obtain a one-dimensional third feature vector, wherein the first feature vector, the second feature vector, and the third feature vector have the same height and width; The target text is identified based on the third feature vector, and the text recognition result of the image to be recognized is generated.

4. The method according to claim 3, characterized in that, The step of identifying the target text based on the third feature vector and generating the text recognition result of the image to be recognized includes: The third feature vector is decoded to generate a string encoding result; The string recognition result corresponding to the string encoding result is determined according to the first preset character table; The string recognition result is converted to generate the text recognition result of the image to be recognized.

5. The method according to claim 4, characterized in that, The step of converting the string recognition result to generate the text recognition result of the image to be recognized includes: The conversion result corresponding to the string recognition result is determined based on the second preset character table; The direction identifier in the conversion result is identified, and multiple first characters included in the conversion result are combined based on the direction identifier to generate the text recognition result of the image to be recognized.

6. The method according to claim 1, characterized in that, The training steps of the text recognition model include: Acquire a sample image and determine the true encoding result of the second character in the sample image; The sample image is input into the text recognition model, and the features of the sample image are extracted by the feature extraction module to generate a two-dimensional sample feature map. The two-dimensional sample feature map is transformed by the dimension transformation module to generate a one-dimensional first sample feature vector. The temporal relationship of the first sample feature vector is constructed by the attention module in the text recognition model to generate a one-dimensional second sample feature vector. The loss is calculated based on the second sample feature vector and the true encoding result of the sample, and the network parameters of the text recognition model are updated based on the loss.

7. The method according to claim 6, characterized in that, Determining the true encoding result of the second character in the sample image includes: The second character in the sample image is labeled according to a preset format, and the labeling result is generated. Generate the corresponding transformed sample results according to the preset transformation rules; The string recognition sample result corresponding to the conversion sample result is determined based on the second preset character table; The string recognition sample result is encoded according to the first preset character table to generate the sample real encoding result of the second character.

8. The method according to claim 7, characterized in that, The annotation result includes a score line identifier, a first part of sample characters, and a second part of sample characters. Generating the corresponding conversion sample result according to a preset conversion rule includes: Based on the fraction line identifier, determine the first direction identifier corresponding to the first part of sample characters and the second direction identifier corresponding to the second part of sample characters; The first part of the sample characters and the first direction identifier are combined into a first whole, and the second part of the sample characters and the second direction identifier are combined into a second whole; The first whole and the second whole are spliced ​​together to generate the transformed sample result corresponding to the annotation result.

9. A text recognition device, characterized in that, include: An acquisition unit is used to acquire an image to be recognized, wherein the image to be recognized includes the target text of a mathematical equation; An input unit is used to input the image to be recognized into a pre-trained text recognition model, wherein the text recognition model includes a feature extraction module and a dimension transformation module; The extraction unit is used to extract features from the image to be identified through the feature extraction module to generate a two-dimensional feature map. The first feature map in the two-dimensional feature map marks the features of the first part of the characters, and the second feature map marks the features of the second part of the characters. The first part of the characters and the second part of the characters constitute the complete target text. The first part of the characters is the upper half of the target text, and the second part of the characters is the lower half of the target text. The transformation unit is used to perform alternating dimensional transformations on the first feature map and the second feature map through the dimensional transformation module to generate a one-dimensional first feature vector. The recognition unit is used to recognize the target text based on the first feature vector and generate the text recognition result of the image to be recognized.

10. An electronic device, characterized in that, The electronic device includes: Processor; and Stored program memory, The program includes instructions that, when executed by the processor, cause the processor to perform the text recognition method according to any one of claims 1 to 8.

11. A non-transitory computer-readable storage medium storing computer instructions, characterized in that, The computer instructions are used to cause the computer to perform the text recognition method according to any one of claims 1 to 8.

Images