Text recognition method, device, storage medium and electronic device

Abstract

The present disclosure relates to a text recognition method, device, storage medium and electronic device, the method comprising: obtaining a text frame in the image to be recognized by positioning and detecting the text in the image to be recognized, wherein the text in each of the text frames has the same running direction; determining the running direction of the text in the text frame for each of the text frames; and obtaining the text recognition result of the text frame according to the running direction of the text frame. By adopting this method, no matter what font type and font size the text in the image to be recognized is, and no matter what running direction the text in the image to be recognized is, and what typesetting method is used, the text frame in the image to be recognized can be detected, and the text recognition result corresponding to the running direction of each detected text frame can be obtained. Therefore, this method can perform text recognition on images of any format.

Description

Technical Field

[0001] The present disclosure relates to the technical field of text recognition, and in particular, to a text recognition method, device, storage medium and electronic device. Background Art

[0002] In daily life, people often have to deal with a large number of pictures, texts and reports. In order to reduce people's workload and improve processing efficiency, text recognition technology came into being. Text recognition technology can be applied to many fields, such as document and image retrieval, letter and package sorting, manuscript editing and proofreading, statistical report and card summary and analysis, bank check processing, commodity invoice statistics summary, commodity code recognition, various types of certificate recognition, and sensitive word inspection, etc.

[0003] In the related art, optical character recognition (OCR) technology is used to recognize text in images. Specifically, a certain type of image samples with a fixed format are used for model training, and then the trained model is used to perform text recognition on the images of the fixed format, thereby obtaining the text content in the image. However, this trained model cannot be used to recognize text in images of other formats. Summary of the invention

[0004] The purpose of the present disclosure is to provide a text recognition method, device, storage medium and electronic device to solve the problems existing in the related art.

[0005] In order to achieve the above object, according to the first part of the embodiment of the present disclosure, a text recognition method is provided, the method comprising:

[0006] By performing positioning detection on the text in the image to be recognized, a text frame in the image to be recognized is obtained, wherein the text in each of the text frames has the same running direction;

[0007] For each of the text boxes, determining the text direction of the text in the text box; and

[0008] According to the running direction of the text box, a text recognition result of the text box is obtained.

[0009] Optionally, for each of the text boxes, determining the running direction of the text in the text box includes:

[0010] For each of the text boxes, determine the center point coordinates of each character in the text box; and

[0011] The text direction of the text box is determined according to the center point coordinates of each character in the text box.

[0012] Optionally, the writing direction includes a horizontal writing direction and a vertical writing direction;

[0013] Determining the text direction of the text box according to the center point coordinates of each character in the text box includes:

[0014] Calculating the X-axis offset and the Y-axis offset of the center point coordinates of each character pair in the text box, wherein the character pair represents two adjacent characters in the text box;

[0015] Calculate the average value of the X-axis offsets and the average value of the Y-axis offsets of all the text pairs in the text frame;

[0016] When the average value of the X-axis offset is greater than or equal to the average value of the Y-axis offset, determining the text direction of the text box to be the horizontal text direction;

[0017] When the average value of the X-axis offset is smaller than the average value of the Y-axis offset, the text direction of the text box is determined to be the vertical text direction.

[0018] Optionally, obtaining a text recognition result of the text box according to the text direction of the text box includes:

[0019] If the text direction of the text box is the horizontal text direction, the text box is input into the horizontal recognition model to obtain the text recognition result of the horizontal text;

[0020] If the text running direction of the text box is the vertical text running direction, the text box is input into the vertical recognition model to obtain the text recognition result of the vertical text running.

[0021] Optionally, before determining the running direction of the text in each text box, the method further includes:

[0022] For each of the text boxes, the text box is input into a horizontal recognition model and a vertical recognition model respectively, to obtain a first recognition result output by the horizontal recognition model and a second recognition result output by the vertical recognition model;

[0023] The obtaining of the text recognition result of the text box according to the text direction of the text box includes:

[0024] The text recognition result of the text box is selected from the first recognition result and the second recognition result according to the text direction.

[0025] Optionally, determining the running direction of the text in the text box includes:

[0026] When the first recognition result is not empty and the length of the text box is greater than the height, determining that the text direction of the text box is a horizontal text direction;

[0027] When the first recognition result is not empty, the length of the text box is less than or equal to the height, and the length of the first recognition result is less than the length of the second recognition result, determining that the text direction of the text box is a vertical text direction;

[0028] When the first recognition result is not empty, the length of the text box is less than or equal to the height, the length of the first recognition result is greater than or equal to the length of the second recognition result, and the number of characters in the second recognition result is greater than 1, determining that the text direction of the text box is the vertical text direction;

[0029] When the first recognition result is not empty, the length of the text box is less than or equal to the height, the length of the first recognition result is greater than or equal to the length of the second recognition result, and the number of characters in the second recognition result is less than or equal to 1, the writing direction of the text box is determined to be the horizontal writing direction.

[0030] Optionally, before performing location detection on the text in the image to be recognized, the method further includes:

[0031] Correcting the image to be recognized into a frontal view;

[0032] After obtaining the text frame in the image to be recognized, the method further includes:

[0033] The obtained text frame is corrected so that the shape of the text frame is a rectangle.

[0034] Optionally, the correcting the obtained text frame so that the shape of the text frame is a rectangle includes:

[0035] Determine a circumscribed quadrilateral having the largest area intersection and union ratio with the text box;

[0036] Determine the coordinates of four target vertices of the circumscribed quadrilateral according to the coordinates of the four vertices of the circumscribed quadrilateral;

[0037] Determine a transformation matrix according to the coordinates of the four vertices of the circumscribed quadrilateral and the coordinates of the four target vertices;

[0038] The text frame is transformed according to the transformation matrix to obtain the text frame having a rectangular shape.

[0039] Optionally, the horizontal recognition model includes at least one of a seq2seq decoding module and a CTC decoding module, and the vertical recognition model includes at least one of a seq2seq decoding module and a CTC decoding module.

[0040] Optionally, the horizontal recognition model includes a seq2seq decoding module and a CTC decoding module, and the vertical recognition model includes a seq2seq decoding module and a CTC decoding module;

[0041] The method further comprises:

[0042] Before inputting the text box in the vertical writing direction into the vertical recognition model, the text box is rotated by a preset angle in a preset direction.

[0043] According to the second part of the embodiment of the present disclosure, there is provided a text recognition device, including:

[0044] A generating module is configured to obtain a text frame in the image to be recognized by performing positioning detection on the text in the image to be recognized, wherein the text in each of the text frames has the same running direction;

[0045] A first determination module is configured to determine, for each of the text boxes, the text direction of the text in the text box;

[0046] The execution module is configured to obtain a text recognition result of the text box according to the text direction of the text box.

[0047] Optionally, the first determining module includes:

[0048] The first determination submodule is configured to determine the center point coordinates of each character in the text box for each of the text boxes; and

[0049] The second determination submodule is configured to determine the text direction of the text box according to the center point coordinates of each character in the text box.

[0050] Optionally, the writing direction includes a horizontal writing direction and a vertical writing direction;

[0051] The second determining submodule is specifically configured to:

[0052] Calculating the X-axis offset and the Y-axis offset of the center point coordinates of each character pair in the text box, wherein the character pair represents two adjacent characters in the text box;

[0053] Calculate the average value of the X-axis offsets and the average value of the Y-axis offsets of all the text pairs in the text frame;

[0054] When the average value of the X-axis offset is greater than or equal to the average value of the Y-axis offset, determining the text direction of the text box to be the horizontal text direction;

[0055] When the average value of the X-axis offset is smaller than the average value of the Y-axis offset, the text direction of the text box is determined to be the vertical text direction.

[0056] Optionally, the execution module includes:

[0057] A first execution submodule is configured to input the text box into a horizontal recognition model to obtain the text recognition result of the horizontal text when the text direction of the text box is the horizontal text direction;

[0058] The second execution submodule is configured to input the text box into a vertical recognition model to obtain the text recognition result of the vertical text when the text running direction of the text box is the vertical text running direction.

[0059] Optionally, the device further comprises:

[0060] An input module is configured to input each of the text boxes into a horizontal recognition model and a vertical recognition model respectively before determining the running direction of the text in the text box for each of the text boxes, so as to obtain a first recognition result output by the horizontal recognition model and a second recognition result output by the vertical recognition model;

[0061] The execution module includes:

[0062] The selection submodule is configured to select the text recognition result of the text box from the first recognition result and the second recognition result according to the text direction.

[0063] Optionally, the first determining module includes:

[0064] A third determination submodule is configured to determine that the text direction of the text box is a horizontal text direction when the first recognition result is not empty and the length of the text box is greater than the height;

[0065] a fourth determination submodule, configured to determine that the text direction of the text box is a vertical text direction if the first recognition result is not empty, the length of the text box is less than or equal to the height, and the length of the first recognition result is less than the length of the second recognition result;

[0066] a fifth determination submodule, configured to determine that the text direction of the text box is the vertical text direction when the first recognition result is not empty, the length of the text box is less than or equal to the height, the length of the first recognition result is greater than or equal to the length of the second recognition result, and the number of characters in the second recognition result is greater than 1;

[0067] The sixth determination submodule is configured to determine that the writing direction of the text box is the horizontal writing direction when the first recognition result is not empty, the length of the text box is less than or equal to the height, the length of the first recognition result is greater than or equal to the length of the second recognition result, and the number of characters in the second recognition result is less than or equal to 1.

[0068] Optionally, the device further comprises:

[0069] A first correction module is configured to correct the image to be recognized into a front view before positioning and detecting the text in the image to be recognized;

[0070] The second correction module is configured to correct the obtained text frame after obtaining the text frame in the image to be recognized, so that the shape of the text frame is a rectangle.

[0071] Optionally, the second correction module is specifically configured to:

[0072] Determine a circumscribed quadrilateral having the largest area intersection and union ratio with the text box;

[0073] Determine the coordinates of four target vertices of the circumscribed quadrilateral according to the coordinates of the four vertices of the circumscribed quadrilateral;

[0074] Determine a transformation matrix according to the coordinates of the four vertices of the circumscribed quadrilateral and the coordinates of the four target vertices;

[0075] The text frame is transformed according to the transformation matrix to obtain the text frame having a rectangular shape.

[0076] Optionally, the horizontal recognition model includes at least one of a seq2seq decoding module and a CTC decoding module, and the vertical recognition model includes at least one of a seq2seq decoding module and a CTC decoding module.

[0077] Optionally, the horizontal recognition model includes a seq2seq decoding module and a CTC decoding module, and the vertical recognition model includes a seq2seq decoding module and a CTC decoding module;

[0078] The device also includes:

[0079] The rotation module is configured to rotate the text box in the vertical writing direction by a preset angle in a preset direction before inputting the text box in the vertical writing direction into the vertical recognition model.

[0080] According to the third part of the embodiments of the present disclosure, a computer-readable storage medium is provided, on which a computer program is stored, and when the program is executed by a processor, the steps of any one of the methods described in the first part above are implemented.

[0081] According to a fourth aspect of the embodiments of the present disclosure, there is provided an electronic device, including:

[0082] a memory having a computer program stored thereon;

[0083] A processor is used to execute the computer program in the memory to implement the steps of any method described in the first part above.

[0084] By adopting the above technical solution, at least the following technical effects can be achieved:

[0085] By positioning and detecting the text in the image to be recognized, a text frame in the image to be recognized is obtained, wherein each text frame is a partial image including text in the image to be recognized, and the text in each text frame has the same running direction. For each text frame, the running direction of the text in the text frame is determined, and the text recognition result of the text frame is obtained according to the running direction of the text frame. In this way, the text recognition result of the image to be recognized can be obtained. By adopting this method, no matter what font type and font size the text in the image to be recognized is, no matter what running direction the text in the image to be recognized is and what typesetting method is used, the text frame in the image to be recognized can be detected, and the text recognition result corresponding to the running direction can be obtained according to the specific running direction of each text frame detected. Compared with the related art, this method can perform text recognition on images of different formats, that is, this method is applicable to text recognition on images of any format, solving the problems existing in the related art.

[0086] Other features and advantages of the present disclosure will be described in detail in the following detailed description. BRIEF DESCRIPTION OF THE DRAWINGS

[0087] The accompanying drawings are used to provide a further understanding of the present disclosure and constitute a part of the specification. Together with the following specific embodiments, they are used to explain the present disclosure but do not constitute a limitation of the present disclosure. In the accompanying drawings:

[0088] Figure 1 The figure is a flowchart of a text recognition method according to an exemplary embodiment of the present disclosure.

[0089] Figure 2A neural network for positioning and detecting text in an image to be recognized is shown according to an exemplary embodiment of the present disclosure.

[0090] Figure 3 The present invention is a flowchart of a method for determining a text direction of a text box according to an exemplary embodiment of the present invention.

[0091] Figure 4 It is an architectural block diagram of a horizontal recognition model according to an exemplary embodiment of the present disclosure.

[0092] Figure 5 The flowchart is another method for determining the writing direction of a text box according to an exemplary embodiment of the present disclosure.

[0093] Figure 6 The flowchart is another method for determining the writing direction of a text box according to an exemplary embodiment of the present disclosure.

[0094] Figure 7 is a flowchart showing another text recognition method according to an exemplary embodiment of the present disclosure.

[0095] Figure 8 is a flowchart of another text recognition method according to an exemplary embodiment of the present disclosure.

[0096] Fig. 9 The block diagram of a text recognition device according to an exemplary embodiment of the present disclosure is shown.

[0097] Fig.10 It is a block diagram of an electronic device according to an exemplary embodiment of the present disclosure. DETAILED DESCRIPTION

[0098] The specific implementation of the present disclosure is described in detail below in conjunction with the accompanying drawings. It should be understood that the specific implementation described herein is only used to illustrate and explain the present disclosure, and is not used to limit the present disclosure.

[0099] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present disclosure. Instead, they are merely examples of devices and methods consistent with some aspects of the present disclosure as detailed in the appended claims.

[0100] In daily life, people often have to deal with a large number of pictures, texts and reports. For example, the text in the picture is reviewed for sensitive words. Another example is to identify the copy or picture of the certificate. Another example is to collect the text information in the bill picture, etc.

[0101] In the related art, a trained model is generally used to recognize text in an image. For example, for ID card images, a large number of ID card images are first used as training samples to train an ID card recognition model, and then the ID card image to be recognized is input into the trained ID card recognition model to obtain the text recognition result output by the ID card recognition model. For another example, for ordinary invoice images, a large number of ordinary invoice images are first used as training samples to train an invoice recognition model, and then the ordinary invoice image to be recognized is input into the trained invoice recognition model to obtain the text recognition result output by the invoice recognition model.

[0102] However, each model trained by this method in the related art can only recognize images of a fixed format corresponding to the model, that is, the ID card recognition model cannot recognize the text in ordinary invoice images, and the invoice recognition model cannot recognize the text in ID card images. It is not difficult to understand that the reason why each model cannot be applied to images other than the fixed format corresponding to the model is that the non-fixed arrangement of text will increase the difficulty of text recognition, and when the document number and font of the text to be recognized in the image change dramatically, the difficulty of text recognition will also increase.

[0103] In view of this, the embodiments of the present disclosure provide a text recognition method, device, storage medium and electronic device to solve the problems existing in the related art.

[0104] Figure 1 is a flowchart of a text recognition method according to an exemplary embodiment of the present disclosure. Figure 1 As shown, the method may include the following steps:

[0105] S11, obtaining a text frame in the image to be recognized by performing positioning detection on the text in the image to be recognized, wherein the text in each text frame has the same running direction;

[0106] S12, for each of the text boxes, determining the text direction of the text in the text box;

[0107] S13. Obtaining a text recognition result of the text box according to the text direction of the text box.

[0108] The text box is a partial image including text in the image to be recognized. The text in each text box has the same text direction. For example, the text direction can be horizontal, vertical, and oblique.

[0109] A feasible implementation method may be to use Figure 2The joint detection neural network based on pixel segmentation and attention mechanism shown in the figure performs positioning detection on the text in the image to be recognized to obtain the text frame in the image to be recognized, for example, the following can be obtained: Figure 2 The text box shown as a white box in the lower right corner of the picture.

[0110] After obtaining the text box in the image to be recognized, the direction of the text box can be determined. Figure 2 , it can be determined without a doubt Figure 2 The text box shown in the white box in the middle has a horizontal direction.

[0111] After determining the text direction of the text box, the recognition result of the text box can be obtained based on the text direction of the text box. For example, according to the text direction of the text box, each character in the text box is recognized in turn to obtain the corresponding character recognition result.

[0112] By adopting this method, the text in the image to be recognized is positioned and detected to obtain the text frame in the image to be recognized, wherein each text frame is a partial image including text in the image to be recognized, and the text in each text frame has the same running direction. For each text frame, the running direction of the text in the text frame is determined, and the text recognition result of the text frame is obtained according to the running direction of the text frame. In this way, the text recognition result of the image to be recognized can be obtained. By adopting this method, no matter what font type and font size the text in the image to be recognized is, no matter what running direction the text in the image to be recognized is and what typesetting method is used, the text frame in the image to be recognized can be detected, and the specific running direction of the text in each text frame can be obtained, such as horizontal, vertical or oblique. According to the specific running direction of each text frame detected, the text recognition result corresponding to the running direction can be obtained. Compared with the related art, this method can perform text recognition for images of different formats, that is, this method is applicable to performing text recognition for images of any format, solving the problems existing in the related art.

[0113] Optionally, for each of the text boxes, determining the writing direction of the text in the text box may include the following steps:

[0114] For each of the text boxes, the center point coordinates of each character in the text box are determined; and according to the center point coordinates of each character in the text box, the text direction of the text box is determined.

[0115] Specifically, the FasterRCNN network can be used to locate each single character in the image to be recognized, and the center point coordinates of each character can be obtained. For each text box, the direction of the text box can be determined according to the center point coordinates of each character in the text box.

[0116] It should be noted here that, in the present disclosure, the text direction specifically refers to whether the text is arranged left and right or up and down. It is not difficult to understand that the text direction can be oblique text, concave and convex text, etc. in addition to horizontal horizontal text and vertical vertical text. For oblique text and concave and convex text, both can be classified horizontally and vertically according to whether the text is arranged left and right or up and down. Therefore, in the present disclosure, all types of text directions can be divided into horizontal text directions and vertical text directions.

[0117] For details, see Figure 3 , you can determine the direction of each text box by following these steps:

[0118] S31. For each text box, calculate the X-axis offset and the Y-axis offset of the center point coordinates of each character pair in the text box, where the character pair represents two adjacent characters in the text box.

[0119] A text pair refers to any two adjacent characters in the text box. For example, assuming that the characters in the text box are ABCDE in horizontal / vertical / diagonal directions, then all the text pairs in the text box can be determined to be AB, BC, CD, and DE. Next, the X-axis offset and Y-axis offset of the two characters in each text pair are calculated. For example, the X-axis offset of the text pair AB is calculated to be |A x -B x |, the Y-axis offset of the text to AB is |A y -B y |, where || represents the absolute value.

[0120] S32, calculating an average value of the X-axis offsets and an average value of the Y-axis offsets of all the text pairs in the text box.

[0121] For example, the formula The average value of the X-axis offsets of all the character pairs AB, BC, CD, and DE is calculated. Similarly, the average value of the Y-axis offsets of all the character pairs can be calculated.

[0122] S33: When the average value of the X-axis offset is greater than or equal to the average value of the Y-axis offset, determine that the text direction of the text box is the horizontal text direction.

[0123] After determining the average X-axis offset and the average Y-axis offset of all text pairs in the text box, if the average X-axis offset is greater than or equal to the average Y-axis offset, the text direction of the text box is determined to be the horizontal text direction.

[0124] It should be noted that when the average value of the X-axis offset Greater than or equal to the average value of the Y-axis offset hour, and The ratio is greater than or equal to 1. According to trigonometric functions, the angle between the projection of the mean offset of the center point of the text in the text box and the X-axis must be between -45° and 45°.

[0125] S34: When the average value of the X-axis offset is less than the average value of the Y-axis offset, determine that the text direction of the text box is the vertical text direction.

[0126] After determining the average X-axis offset and the average Y-axis offset of all text pairs in the text box, if the average X-axis offset is smaller than the average Y-axis offset, the text direction of the text box is determined to be the vertical text direction.

[0127] It is not difficult to understand that when the average value of the X-axis offset Less than the average Y-axis offset hour, and The ratio is less than 1. According to trigonometric functions, the angle between the projection of the mean offset of the center point of the text in the text box and the X-axis must be between 45° and 90° or -90° and -45°.

[0128] Therefore, by adopting the above method of determining the text direction of each text box, in addition to determining the text direction of a text box with horizontal text as the horizontal text direction and determining the text direction of a text box with vertical text as the vertical text direction, the text direction of an inclined text box can also be determined as horizontal or vertical. In this way, it can be determined whether the characters in all types of text boxes with text directions in the image to be recognized (i.e., the text boxes in which the angle between the projection of the mean value of the text center point offset and the X-axis is an arbitrary value) are horizontal or vertical.

[0129] Further, after determining the text direction of each text box in the image to be recognized, obtaining the text recognition result of the text box according to the text direction of the text box may include the following steps:

[0130] If the text running direction of the text box is the horizontal text running direction, the text box is input into the horizontal recognition model to obtain the text recognition result of the horizontal text running; if the text running direction of the text box is the vertical text running direction, the text box is input into the vertical recognition model to obtain the text recognition result of the vertical text running.

[0131] It should be noted that the horizontal recognition model is trained using text box samples with horizontal text. The vertical recognition model is trained using text box samples with vertical text. Therefore, inputting a text box with horizontal text direction into the horizontal recognition model can obtain the text recognition result of the text box with horizontal text direction. Inputting a text box with vertical text direction into the vertical recognition model can obtain the text recognition result of the text box with vertical text direction.

[0132] Among them, the specific training process of the horizontal recognition model and the vertical recognition model is similar to the model training method in the related technology, which will not be repeated here.

[0133] Optionally, the horizontal recognition model includes a seq2seq decoding module and a CTC decoding module, and the vertical recognition model includes a seq2seq decoding module and a CTC decoding module.

[0134] Among them, the seq2seq (Attentional Sequence to Sequence) decoding module uses a sequence-to-sequence recognition method based on the attention mechanism to decode the encoded feature vector. Due to the introduction of the recursive neural network and the use of the sequence attention module by the seq2seq decoder itself, the seq2seq decoding module not only relies on the characteristics of the characters themselves, but also relies on the contextual information characteristics between the characters during the decoding process. The text sequence recognition result (i.e., text recognition result) finally output by the seq2seq decoding module includes the category of each character and the corresponding probability of the category.

[0135] The CTC (Connectionist Temporal Classification) decoding module uses sequence connection classification to decode the encoded feature vector. Since recursive neural networks are not introduced and CTC only considers image texture features, the CTC decoding module mainly relies on the characteristics of the characters themselves for decoding. The CTC decoding module finally outputs the text sequence recognition result, including each character category and the corresponding probability of the category.

[0136] It should be noted that the horizontal recognition model and the vertical recognition model can share the encoding module. The feature extraction layer parameters in the encoding module can be obtained by training using networks such as ResNet, InceptionV4, and DenseNet.

[0137] For example, taking the horizontal recognition model as an example, the architecture of the horizontal recognition model can be as follows Figure 4It should be understood that when the decoding module in the horizontal recognition model includes a seq2seq decoding module and a CTC decoding module, the feature vector of the text box can be decoded by the seq2seq decoding module and the CTC decoding module respectively, and then the output results of the seq2seq decoding module and the CTC decoding module are probabilistically fused to obtain the final text recognition result of the text box.

[0138] Among them, the calculation formula of probability fusion is: or

[0139] S=argmax(min(P(S11),P(S12),…,P(S1 m )),min(P(S21),P(S22),…,P(S2 n ))),

[0140] Among them, S represents the final text recognition result, S1 and S2 represent the output results of the CTC decoding module and the seq2seq decoding module respectively, and m and n represent the number of characters in the output results of the CTC decoding module and the seq2seq decoding module respectively.

[0141] Optionally, the method may further include:

[0142] Before inputting the text box in the vertical writing direction into the vertical recognition model, the text box is rotated by a preset angle in a preset direction.

[0143] For example, before a text box with a vertical text direction is input into the vertical recognition model, the text box may be rotated 90 degrees in a clockwise direction.

[0144] It should be noted that the use of Figure 4 In the process of recognizing a text box in the vertical text direction by a vertical recognition model with a similar architecture, a feature map with a length of W and a height of 1 can be obtained after passing through the feature extraction layer. Then, during decoding, the feature map is decoded in sequence along the X-axis direction. In this process, if the height of the text box in the vertical text direction is directly compressed to 1, the text in the text box will be completely unrecognizable. Therefore, in an embodiment of the present disclosure, before the text box in the vertical text direction is input into the vertical recognition model, the text box is rotated by a preset angle in a preset direction. When each character in the text box is sequentially recognized along the X-axis direction in the vertical recognition model, the text recognition is actually performed sequentially according to the original text direction from top to bottom or from bottom to top in the text box.

[0145] In an achievable implementation manner, before determining the direction of the text in each text box, the method may further include the following steps:

[0146] For each of the text boxes, the text box is input into the horizontal recognition model and the vertical recognition model respectively to obtain a first recognition result output by the horizontal recognition model and a second recognition result output by the vertical recognition model. And according to the text direction, the text recognition result of the text box is selected from the first recognition result and the second recognition result.

[0147] A possible implementation method is that when the running direction of the text box is uncertain, that is, when the above-mentioned FasterRCNN network is not used to locate each single character in the image to be recognized to determine the running direction of the text box, the text box can be input into the horizontal recognition model and the vertical recognition model respectively, so as to obtain a first recognition result output by the horizontal recognition model and a second recognition result output by the vertical recognition model.

[0148] The text direction of the text box can also be determined based on the first recognition result and the second recognition result of the text box. After the text direction of the text box is determined based on the first recognition result and the second recognition result of the text box, the final text recognition result of the text box can be selected from the first recognition result and the second recognition result according to the determined text direction.

[0149] In this achievable implementation, the horizontal recognition model includes at least one of a seq2seq decoding module and a CTC decoding module, and the vertical recognition model includes at least one of a seq2seq decoding module and a CTC decoding module.

[0150] Specifically, when the text direction of the text box is uncertain, if the text box is input into the horizontal recognition model and the vertical recognition model respectively, then in order to obtain the recognition result faster, the horizontal recognition model and the vertical recognition model can be decoded by only one of the seq2seq decoding module and the CTC decoding module.

[0151] For details, see Figure 5 Determining the text direction of the text box based on the first recognition result and the second recognition result of the text box may include:

[0152] S51. When both the first recognition result and the second recognition result are empty, determine that the writing direction of the text box is a default writing direction, and the default writing direction is the horizontal writing direction or the vertical writing direction.

[0153] It should be understood that when both the first recognition result and the second recognition result of the text box are empty, the content of the text box cannot be determined, that is, the first recognition result and the second recognition result are meaningless. Therefore, in this case, it can be determined that the text direction of the text box is the horizontal text direction or the vertical text direction.

[0154] S52: When the first recognition result is empty and the second recognition result is not empty, determine that the writing direction of the text box is the vertical writing direction.

[0155] When the first recognition result is empty but the second recognition result is not empty, it indicates that the horizontal recognition model cannot recognize the content of the text box, while the vertical recognition model can recognize the content of the text box. Therefore, in this case, it can be determined that the text direction of the text box is the vertical direction.

[0156] S53: When the first recognition result is not empty and the length of the text box is greater than the height, determine that the text direction of the text box is a horizontal text direction.

[0157] When the first recognition result is not empty, it means that the horizontal recognition model can recognize the text box, and the text direction of the text box may be horizontal. In a possible case, when the horizontal recognition model can recognize the text box, the result of recognizing the text box may be 0 or non-0. Therefore, under the premise of considering the accuracy of the recognition result, when it is determined that the length of the text box is greater than the height, it can be further determined that the text direction of the text box is the horizontal direction.

[0158] The length of the text box is the horizontal length, and the height is the vertical length.

[0159] S54: When the first recognition result is not empty, the length of the text box is less than or equal to the height, and the length of the first recognition result is less than the length of the second recognition result, determine that the text direction of the text box is a vertical text direction.

[0160] In one possible case, in the above step S53, if it is further determined that the length of the text box is less than or equal to the height, it means that even if the horizontal recognition model can recognize the text box, the text box may still be in a vertical direction. In this case, if it is further determined that the length of the first recognition result is less than the second recognition result, it is determined that the text box is in a vertical direction.

[0161] S55. When the first recognition result is not empty, the length of the text box is less than or equal to the height, the length of the first recognition result is greater than or equal to the length of the second recognition result, and the number of characters in the second recognition result is greater than 1, determine that the writing direction of the text box is the vertical writing direction.

[0162] In one possible situation, if it is further determined in the above step S54 that the length of the first recognition result is greater than or equal to the second recognition result, then further, if it is determined that the number of characters in the second recognition result is greater than 1, the text direction of the text box is determined to be the vertical text direction.

[0163] S56. When the first recognition result is not empty, the length of the text box is less than or equal to the height, the length of the first recognition result is greater than or equal to the length of the second recognition result, and the number of characters in the second recognition result is less than or equal to 1, determine that the writing direction of the text box is the horizontal writing direction.

[0164] If it is further determined in the above step S55 that the number of characters in the second recognition result is less than or equal to 1, then the text direction of the text box is determined to be the horizontal text direction.

[0165] After the running direction of the text box is determined according to the first recognition result and the second recognition result, a corresponding result is selected from the first recognition result and the second recognition result as the text recognition result of the text box according to the determined running direction of the text box.

[0166] See also Figure 6 The specific implementation steps of the method for determining the text direction of the text box based on the first recognition result and the second recognition result of the text box may be:

[0167] S601, determining whether the first recognition result of the text box is empty;

[0168] If yes, execute S602, if no, execute S605.

[0169] S602, determining whether the second recognition result of the text box is empty;

[0170] If yes, execute S603, if no, execute S604.

[0171] S603, determining that the text direction of the text box is a horizontal text direction;

[0172] S604, determining that the text direction of the text box is a vertical text direction;

[0173] S605, determining whether the length of the text box is greater than the height;

[0174] If yes, execute S606, if no, execute S607.

[0175] S606, determining that the text direction of the text box is a horizontal text direction;

[0176] S607, determining whether the length of the second recognition result is greater than the length of the first recognition result;

[0177] If yes, execute S608, if no, execute S609.

[0178] S608, determining that the text direction of the text box is a vertical text direction;

[0179] S609, determining whether the number of characters in the second recognition result is greater than 1;

[0180] If yes, execute S610, if no, execute S611.

[0181] S610, determining that the text direction of the text box is a vertical text direction;

[0182] S611: Determine that the text direction of the text box is a horizontal text direction.

[0183] Among them, S603, S606 and S611 may be the same step. S604, S608 and S610 may be the same step. It should be noted that the order of the judgment steps in the above steps may be adjusted without violating the idea of ​​the above method.

[0184] Optionally, for any of the above-mentioned text recognition methods, before performing positioning detection on the text in the image to be recognized, the method may further include: correcting the image to be recognized to a frontal view.

[0185] Inevitably, the image to be recognized uploaded by the user may be a positive image, an inverted image, an image rotated clockwise, or an image rotated counterclockwise. The inverted or rotated image will affect the text positioning detection, and will affect the recognition process of the horizontal or vertical recognition model. Therefore, in order to improve the accuracy of text positioning detection and the correctness of the text recognition process, the embodiment of the present disclosure can correct the image to be recognized to a positive view before positioning the text in the image to be recognized. In a specific implementation method, a classification model trained through refinement can be used to judge the direction of the image to be recognized, and the direction of the image to be recognized can be corrected according to the classification result.

[0186] Optionally, after obtaining the text frame in the image to be recognized, the method may further include:

[0187] The obtained text frame is corrected so that the shape of the text frame is a rectangle.

[0188] When shooting an image, the shooting angle will affect the shape of the object being shot. Therefore, it is not difficult to understand that the text box obtained from the image to be recognized may have an angle tilt (i.e., deformation), and thus the text in the text box may have an angle tilt. In view of this, in order to overcome the problem of text tilt caused by the shooting angle, the shape of the text box can be corrected so that the shape of the text box is a rectangle.

[0189] In an achievable implementation manner, the correction of the obtained text frame to make the shape of the text frame a rectangle may specifically include the following steps:

[0190] Determine a circumscribed quadrilateral with the largest area intersection and union ratio with the text box; determine the coordinates of four target vertices of the circumscribed quadrilateral according to the coordinates of the four vertices of the circumscribed quadrilateral; determine a transformation matrix according to the coordinates of the four vertices of the circumscribed quadrilateral and the coordinates of the four target vertices; transform the text box according to the transformation matrix to obtain the text box with a rectangular shape.

[0191] Specifically, the target length and width of the circumscribed quadrilateral can be calculated based on the coordinates of the four vertices of the circumscribed quadrilateral (x0, y0), (x1, y1), (x2, y2), and (x3, y3). For example, the target length is The target width is Furthermore, according to the target length and width of the circumscribed quadrilateral, the coordinates of the four target vertices of the circumscribed quadrilateral can be determined as (0,0), (w,0), (w,h), (0,h). According to the coordinates of the four vertices of the circumscribed quadrilateral and the coordinates of the four target vertices, a transformation matrix can be obtained through RANSAC random sampling. For example, the transformation matrix is The text frame can be transformed according to the transformation matrix to obtain a text frame with a rectangular shape.

[0192] In this way, a text box whose projection and X-axis have an angle in the range of (-45°, 45°) can be corrected to a horizontal text box, that is, a text box whose deformation angle is in the range of (-45°, 45°) can be corrected to a horizontal text box. A text box whose projection and X-axis have an angle in the range of (45°, 90°) or (-90°, -45°) can be corrected to a vertical text box, that is, a text box whose deformation angle is in the range of (45°, 90°) or (-90°, -45°) can be corrected to a vertical text box. It should be noted that this method of correcting a text box does not change the direction of the text in the text box.

[0193] Figure 7 is a flowchart of another text recognition method according to an exemplary embodiment of the present disclosure, such as Figure 7 As shown, the method may include the following steps:

[0194] S71, correcting the image to be recognized into a front view;

[0195] S72, obtaining a text frame in the image to be recognized by performing positioning detection on the text in the image to be recognized;

[0196] S73, correcting the obtained text frame so that the shape of the text frame is a rectangle;

[0197] S74, for each text frame, input the text frame into the horizontal recognition model and the vertical recognition model respectively, to obtain a first recognition result output by the horizontal recognition model and a second recognition result output by the vertical recognition model;

[0198] Among them, the decoding modules in the horizontal recognition model and the vertical recognition model are seq2seq decoding modules or CTC decoding modules.

[0199] S75, determining the text direction of the text box according to the first recognition result and the second recognition result of the text box;

[0200] S76. Select a text recognition result of the text box from the first recognition result and the second recognition result according to the text direction of the text box.

[0201] Using the above method, when the decoding module in the horizontal recognition model and the vertical recognition model is a CTC decoding module, the text recognition speed of the above method is faster than when the decoding module in the horizontal recognition model and the vertical recognition model is a seq2seq decoding module, but the accuracy is relatively low. The reason is that the seq2seq decoding module adopts the attention mechanism and combines the context information of the text in the decoding process.

[0202] Figure 8 is a flowchart of another text recognition method according to an exemplary embodiment of the present disclosure, such as Figure 8 As shown, the method may include the following steps:

[0203] S81, correcting the image to be recognized into a front view;

[0204] S82, obtaining a text frame in the image to be recognized by performing positioning detection on the text in the image to be recognized;

[0205] S83. For each text box, determine the center point coordinates of each character in the text box; and determine the text direction of the text box according to the center point coordinates of each character in the text box.

[0206] S84, correcting the obtained text frame so that the shape of the text frame is a rectangle;

[0207] S85, if the text box has a horizontal text direction, input the text box into a horizontal recognition model, wherein the horizontal recognition model includes a seq2seq decoding module and a CTC decoding module;

[0208] S86. If the text box has a vertical text direction, input the text box into a vertical recognition model, wherein the vertical recognition model includes a seq2seq decoding module and a CTC decoding module;

[0209] S87. Perform probability fusion processing on the results output by the seq2seq decoding module and the CTC decoding module to obtain a text recognition result of the text box.

[0210] Using this method, Figure 7 Compared with the two methods shown in FIG. 1 , step S83 is added, and the seq2seq decoding module and the CTC decoding module are used to decode the text frame respectively, and the probability fusion processing process is also added. Therefore, this method is better than Figure 7 The two methods shown have slower recognition speeds but higher text recognition accuracy.

[0211] If the above Figure 7 The decoding module in the horizontal recognition model and the vertical recognition model shown in the figure is a method corresponding to the CTC decoding module as the first method, Figure 7 The decoding modules in the horizontal recognition model and the vertical recognition model shown in the figure are seq2seq decoding modules corresponding to the method as the second method. Figure 8 The corresponding method is regarded as the third method. After the experiment, the specific differences obtained by using these three methods to perform text recognition on the same image to be recognized are shown in Table 1.

[0212] Table 1

[0213] Way Accuracy time consuming First method 85％ 1.5508s Second method 89％ 1.9984s Third Way 91％ 2.4624s

[0214] In one achievable implementation, corresponding operation options may be set for the above three methods in the user operation interface, so that the user can flexibly select any one of the above three methods to identify the text information in the image to be identified according to needs.

[0215] For example, when the user has a high requirement for the real-time performance of text recognition but a low requirement for the accuracy of the recognition result, the user can select the first method to recognize the text in the image to be recognized so as to quickly obtain the text information in the image to be recognized.

[0216] As another example, when the user has a relatively balanced demand for real-time performance and accuracy of text recognition, the second method can be selected to recognize text in the image to be recognized.

[0217] As another example, when the user has very high requirements for the accuracy of text recognition, the third method mentioned above can be selected to recognize the text in the image to be recognized, so as to obtain a text recognition result with high accuracy.

[0218] Based on the same inventive concept, the embodiment of the present disclosure also provides a text recognition device, such as Fig. 9 As shown, the device 900 includes:

[0219] The generating module 901 is configured to obtain a text frame in the image to be recognized by performing positioning detection on the text in the image to be recognized, wherein the text in each of the text frames has the same running direction;

[0220] A first determination module 902 is configured to determine, for each of the text boxes, the text direction of the text in the text box;

[0221] The execution module 903 is configured to obtain a text recognition result of the text box according to the text direction of the text box.

[0222] By adopting this device, the text in the image to be recognized is positioned and detected to obtain the text frame in the image to be recognized, wherein each text frame is a partial image including text in the image to be recognized, and the text in each text frame has the same running direction. For each text frame, the running direction of the text in the text frame is determined, and the text recognition result of the text frame is obtained according to the running direction of the text frame. In this way, the text recognition result of the image to be recognized can be obtained. By adopting this device, no matter what font type and font size the text in the image to be recognized is, no matter what running direction the text in the image to be recognized is, and what typesetting method is used, the text frame in the image to be recognized can be detected, and the specific running direction of the text in each text frame can be determined, such as horizontal, vertical or oblique. According to the specific running direction of each text frame detected, the text recognition result corresponding to the running direction can be obtained. Compared with the related art, this method can perform text recognition for images of different formats, that is, this device is suitable for performing text recognition for images of any format, solving the problems existing in the related art.

[0223] Optionally, the first determining module 902 includes:

[0224] The first determination submodule is configured to determine the center point coordinates of each character in the text box for each of the text boxes; and

[0225] The second determination submodule is configured to determine the text direction of the text box according to the center point coordinates of each character in the text box.

[0226] Optionally, the writing direction includes a horizontal writing direction and a vertical writing direction;

[0227] The second determining submodule is specifically configured to:

[0228] Calculating the X-axis offset and the Y-axis offset of the center point coordinates of each character pair in the text box, wherein the character pair represents two adjacent characters in the text box;

[0229] Calculate the average value of the X-axis offsets and the average value of the Y-axis offsets of all the text pairs in the text frame;

[0230] When the average value of the X-axis offset is greater than or equal to the average value of the Y-axis offset, determining the text direction of the text box to be the horizontal text direction;

[0231] When the average value of the X-axis offset is smaller than the average value of the Y-axis offset, the text direction of the text box is determined to be the vertical text direction.

[0232] Optionally, the execution module 903 includes:

[0233] A first execution submodule is configured to input the text box into a horizontal recognition model to obtain the text recognition result of the horizontal text when the text direction of the text box is the horizontal text direction;

[0234] The second execution submodule is configured to input the text box into a vertical recognition model to obtain the text recognition result of the vertical text when the text running direction of the text box is the vertical text running direction.

[0235] Optionally, the device 900 further includes:

[0236] An input module is configured to input each of the text boxes into a horizontal recognition model and a vertical recognition model respectively before determining the running direction of the text in the text box for each of the text boxes, so as to obtain a first recognition result output by the horizontal recognition model and a second recognition result output by the vertical recognition model;

[0237] The execution module 903 includes:

[0238] The selection submodule is configured to select the text recognition result of the text box from the first recognition result and the second recognition result according to the text direction.

[0239] Optionally, the first determining module 902 includes:

[0240] A third determination submodule is configured to determine that the text direction of the text box is a horizontal text direction when the first recognition result is not empty and the length of the text box is greater than the height;

[0241] a fourth determination submodule, configured to determine that the text direction of the text box is a vertical text direction if the first recognition result is not empty, the length of the text box is less than or equal to the height, and the length of the first recognition result is less than the length of the second recognition result;

[0242] a fifth determination submodule, configured to determine that the text direction of the text box is the vertical text direction when the first recognition result is not empty, the length of the text box is less than or equal to the height, the length of the first recognition result is greater than or equal to the length of the second recognition result, and the number of characters in the second recognition result is greater than 1;

[0243] The sixth determination submodule is configured to determine that the writing direction of the text box is the horizontal writing direction when the first recognition result is not empty, the length of the text box is less than or equal to the height, the length of the first recognition result is greater than or equal to the length of the second recognition result, and the number of characters in the second recognition result is less than or equal to 1.

[0244] Optionally, the device 900 further includes:

[0245] A first correction module is configured to correct the image to be recognized into a front view before positioning and detecting the text in the image to be recognized;

[0246] The second correction module is configured to correct the obtained text frame after obtaining the text frame in the image to be recognized, so that the shape of the text frame is a rectangle.

[0247] Optionally, the second correction module is specifically configured to:

[0248] Determine a circumscribed quadrilateral having the largest area intersection and union ratio with the text box;

[0249] Determine the coordinates of four target vertices of the circumscribed quadrilateral according to the coordinates of the four vertices of the circumscribed quadrilateral;

[0250] Determine a transformation matrix according to the coordinates of the four vertices of the circumscribed quadrilateral and the coordinates of the four target vertices;

[0251] The text frame is transformed according to the transformation matrix to obtain the text frame having a rectangular shape.

[0252] Optionally, the horizontal recognition model includes at least one of a seq2seq decoding module and a CTC decoding module, and the vertical recognition model includes at least one of a seq2seq decoding module and a CTC decoding module.

[0253] Optionally, the horizontal recognition model includes a seq2seq decoding module and a CTC decoding module, and the vertical recognition model includes a seq2seq decoding module and a CTC decoding module;

[0254] The device 900 further includes:

[0255] The rotation module is configured to rotate the text box in the vertical writing direction by a preset angle in a preset direction before inputting the text box in the vertical writing direction into the vertical recognition model.

[0256] Regarding the device in the above embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment of the method, and will not be elaborated here.

[0257] Fig.10 FIG. 7 is a block diagram of an electronic device 700 according to an exemplary embodiment. Fig.10 As shown, the electronic device 700 may include: a processor 701 and a memory 702. The electronic device 700 may also include one or more of a multimedia component 703, an input / output (I / O) interface 704, and a communication component 705.

[0258] The processor 701 is used to control the overall operation of the electronic device 700 to complete all or part of the steps in the above-mentioned text recognition method. The memory 702 is used to store various types of data to support the operation of the electronic device 700, and these data may include, for example, instructions for any application or method used to operate on the electronic device 700, and application-related data, such as contact data, sent and received messages, pictures, audio, video, etc. The memory 702 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (Static Random Access Memory, referred to as SRAM), electrically erasable programmable read-only memory (Electrically Erasable Programmable Read-Only Memory, referred to as EEPROM), erasable programmable read-only memory (Erasable Programmable Read-Only Memory, referred to as EPROM), programmable read-only memory (Programmable Read-Only Memory, referred to as PROM), read-only memory (Read-Only Memory, referred to as ROM), magnetic memory, flash memory, magnetic disk or optical disk. The multimedia component 703 may include a screen and an audio component. The screen may be, for example, a touch screen, and the audio component is used to output and / or input audio signals. For example, the audio component may include a microphone for receiving external audio signals. The received audio signal may be further stored in the memory 702 or sent through the communication component 705. The audio component also includes at least one speaker for outputting audio signals. The I / O interface 704 provides an interface between the processor 701 and other interface modules, and the other interface modules may be keyboards, mice, buttons, etc. These buttons may be virtual buttons or physical buttons. The communication component 705 is used for wired or wireless communication between the electronic device 700 and other devices. Wireless communication, such as Wi-Fi, Bluetooth, Near Field Communication (NFC), 2G, 3G, 4G, NB-IOT, eMTC, or other 5G, etc., or a combination of one or more of them, is not limited here. Therefore, the corresponding communication component 705 may include: Wi-Fi module, Bluetooth module, NFC module, etc.

[0259] In an exemplary embodiment, the electronic device 700 can be implemented by one or more application specific integrated circuits (ASIC), digital signal processors (DSP), digital signal processing devices (DSPD), programmable logic devices (PLD), field programmable gate arrays (FPGA), controllers, microcontrollers, microprocessors or other electronic components to execute the above-mentioned text recognition method.

[0260] In another exemplary embodiment, a computer-readable storage medium including program instructions is also provided, and when the program instructions are executed by a processor, the steps of the above-mentioned text recognition method are implemented. For example, the computer-readable storage medium can be the above-mentioned memory 702 including program instructions, and the above-mentioned program instructions can be executed by the processor 701 of the electronic device 700 to complete the above-mentioned text recognition method.

[0261] The preferred embodiments of the present disclosure are described in detail above in conjunction with the accompanying drawings; however, the present disclosure is not limited to the specific details in the above embodiments. Within the technical concept of the present disclosure, a variety of simple modifications can be made to the technical solution of the present disclosure, and these simple modifications all fall within the protection scope of the present disclosure.

[0262] It should also be noted that the various specific technical features described in the above specific embodiments can be combined in any suitable manner without contradiction. In order to avoid unnecessary repetition, the present disclosure will not further describe various possible combinations.

[0263] In addition, various embodiments of the present disclosure may be arbitrarily combined, and as long as they do not violate the concept of the present disclosure, they should also be regarded as the contents disclosed by the present disclosure.

Claims

1. A method for character recognition, characterized in that: The method comprises: By performing positioning detection on the text in the image to be recognized, a text frame in the image to be recognized is obtained, wherein the text in each of the text frames has the same running direction; For each of the text boxes, determining the text direction of the text in the text box; and obtaining the text recognition result of the text box according to the text direction of the text box; The step of determining the direction of the text in each text box includes: For each of the text boxes, determine the center point coordinates of each character in the text box; and determine the text direction of the text box according to the center point coordinates of each character in the text box; The writing direction includes a horizontal writing direction and a vertical writing direction; Determining the text direction of the text box according to the center point coordinates of each character in the text box includes: Calculating the X-axis offset and the Y-axis offset of the center point coordinates of each character pair in the text box, wherein the character pair represents two adjacent characters in the text box; Calculate the average value of the X-axis offsets and the average value of the Y-axis offsets of all the text pairs in the text frame; When the average value of the X-axis offset is greater than or equal to the average value of the Y-axis offset, determining the text direction of the text box to be the horizontal text direction; When the average value of the X-axis offset is less than the average value of the Y-axis offset, determining the text direction of the text box to be the vertical text direction; Before performing location detection on the text in the image to be recognized, the method further includes: Correcting the image to be recognized into a frontal view; After obtaining the text frame in the image to be recognized, the method further includes: Correcting the obtained text frame so that the shape of the text frame is a rectangle; The step of correcting the obtained text frame so that the text frame is in a rectangular shape includes: Determine a circumscribed quadrilateral having the largest area intersection and union ratio with the text box; Determine the coordinates of four target vertices of the circumscribed quadrilateral according to the coordinates of the four vertices of the circumscribed quadrilateral; Determine a transformation matrix according to the coordinates of the four vertices of the circumscribed quadrilateral and the coordinates of the four target vertices; The text frame is transformed according to the transformation matrix to obtain the text frame having a rectangular shape.

2. The method according to claim 1, characterized in that: The obtaining of the text recognition result of the text box according to the text direction of the text box includes: If the text direction of the text box is the horizontal text direction, the text box is input into the horizontal recognition model to obtain the text recognition result of the horizontal text; If the text running direction of the text box is the vertical text running direction, the text box is input into the vertical recognition model to obtain the text recognition result of the vertical text running.

3. The method according to claim 1, characterized in that: Before determining the running direction of the text in each text box, the method further includes: For each of the text boxes, the text box is input into a horizontal recognition model and a vertical recognition model respectively, to obtain a first recognition result output by the horizontal recognition model and a second recognition result output by the vertical recognition model; The obtaining of the text recognition result of the text box according to the text direction of the text box includes: The text recognition result of the text box is selected from the first recognition result and the second recognition result according to the text direction.

4. The method according to claim 3, characterized in that: Determining the running direction of the text in the text box includes: When the first recognition result is not empty and the length of the text box is greater than the height, determining that the text direction of the text box is a horizontal text direction; When the first recognition result is not empty, the length of the text box is less than or equal to the height, and the length of the first recognition result is less than the length of the second recognition result, determining that the text direction of the text box is a vertical text direction; When the first recognition result is not empty, the length of the text box is less than or equal to the height, the length of the first recognition result is greater than or equal to the length of the second recognition result, and the number of characters in the second recognition result is greater than 1, determining that the text direction of the text box is the vertical text direction; When the first recognition result is not empty, the length of the text box is less than or equal to the height, the length of the first recognition result is greater than or equal to the length of the second recognition result, and the number of characters in the second recognition result is less than or equal to 1, the writing direction of the text box is determined to be the horizontal writing direction.

5. The method according to claim 3, characterized in that: The horizontal recognition model includes at least one of a seq2seq decoding module and a CTC decoding module, and the vertical recognition model includes at least one of a seq2seq decoding module and a CTC decoding module.

6. The method according to claim 2, characterized in that: The horizontal recognition model includes a seq2seq decoding module and a CTC decoding module, and the vertical recognition model includes a seq2seq decoding module and a CTC decoding module; The method further comprises: Before inputting the text box in the vertical writing direction into the vertical recognition model, the text box is rotated by a preset angle in a preset direction.

7. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the steps of the method described in any one of claims 1 to 6 are implemented.

8. An electronic device, characterized in that: include: a memory having a computer program stored thereon; A processor, configured to execute the computer program in the memory to implement the steps of the method according to any one of claims 1 to 6.

Images