A text image recognition method and device

Abstract

An embodiment of the present invention discloses a text image recognition method and device; after obtaining a text image to be recognized, the text image to be recognized includes at least one text content, and features are extracted from the text content in the text image to be recognized to obtain a text feature set of the text content. Then, based on the text feature set, semantic feature information of the text content is determined, and based on the semantic feature information, recognition parameters of the recognition text feature set are adjusted to obtain target recognition parameters. Based on the target recognition parameters, the text feature set is recognized to obtain a text recognition result of the text image to be recognized; this solution can improve the recognition accuracy of text images, especially the recognition accuracy of low-quality text images.

Description

Technical Field

[0001] The present invention relates to the field of communication technologies, and in particular to a text image recognition method and device. Background Art

[0002] In recent years, with the rapid development of neural network technology in the field of artificial intelligence, image recognition technology, which applies neural networks to image recognition, has also made significant progress. In particular, neural networks are being applied to recognize text content in text images. Existing text image recognition methods often use attention algorithms to encode and decode text images for recognition.

[0003] During the research and practice of the existing technology, the inventors of the present invention found that for the existing text image recognition method, since it only relies on limited local text features, it is difficult to process some low-quality text images to be recognized, which greatly reduces the accuracy of text image recognition. Summary of the Invention

[0004] Embodiments of the present invention provide a text image recognition method and device, which can improve the accuracy of text image recognition.

[0005] A text image recognition method, comprising:

[0006] Acquire a text image to be recognized, where the text image to be recognized includes at least one text content;

[0007] Extracting features of the text content in the to-be-recognized text image to obtain a text feature set of the text content;

[0008] Determining semantic feature information of the text content based on the text feature set;

[0009] Based on the semantic feature information, adjusting the recognition parameters for identifying the text feature set to obtain target recognition parameters;

[0010] The text feature set is identified according to the target recognition parameters to obtain a text recognition result of the text image to be identified.

[0011] Optionally, the embodiment of the present invention may further provide another text image recognition method, including:

[0012] Displaying an education live broadcast page, wherein the education live broadcast page includes text content and an identification control;

[0013] In response to the recognition operation of the recognition control, displaying a text content selection page, the text content selection page including a text content selection control;

[0014] In response to a selection operation of the text content selection control, generating a text image to be recognized according to the text content, and sending the text image to be recognized to a server;

[0015] Obtaining a text recognition result from the server for the text image to be recognized;

[0016] A recognition result page is displayed, wherein the recognition result page includes a text image to be recognized and a text recognition result of the text image to be recognized.

[0017] Accordingly, an embodiment of the present invention provides a text image recognition device, comprising:

[0018] A first acquiring unit is configured to acquire a text image to be recognized, wherein the text image to be recognized includes at least one text content;

[0019] An extraction unit, configured to extract features of the text content in the to-be-recognized text image to obtain a text feature set of the text content;

[0020] a determining unit, configured to determine semantic feature information of the text content based on the text feature set;

[0021] an adjusting unit, configured to adjust recognition parameters for identifying the text feature set based on the semantic feature information to obtain target recognition parameters;

[0022] The recognition unit is used to recognize the text feature set according to the target recognition parameter to obtain a text recognition result of the text image to be recognized.

[0023] Optionally, an embodiment of the present invention may further provide another text image recognition device, including:

[0024] A live broadcast page display unit, configured to display an education live broadcast page, the education live broadcast page including text content and an identification control;

[0025] a selection page display unit, configured to display a text content selection page in response to a recognition operation of the recognition control, wherein the text content selection page includes a text content selection control;

[0026] a generating unit, configured to generate a text image to be recognized according to the text content in response to a selection operation of the text content selection control, and send the text image to be recognized to a server;

[0027] A second acquiring unit, configured to acquire a text recognition result of the server for the text image to be recognized;

[0028] The recognition result display unit is used to display a recognition result page, wherein the recognition result page includes a text image to be recognized and a text recognition result of the text image to be recognized.

[0029] Optionally, in some embodiments, the adjustment unit can be specifically used to obtain the initialization parameters of the decoding network and the dimensional information of the hidden layer in the text recognition model, and use the initialization parameters as the recognition parameters for identifying the text feature set. The text recognition model is used to identify the text content of the text image to be identified; according to the dimensional information, the dimension of the semantic feature information is converted to obtain the converted semantic feature information; according to the converted semantic feature information, the recognition parameters are adjusted to obtain the target recognition parameters.

[0030] Optionally, in some embodiments, the adjustment unit can be specifically used to replace the recognition parameters with the converted semantic feature information to initialize the decoding network, and use the converted semantic feature information as the target recognition parameter; the text feature set is recognized according to the target recognition parameter to obtain the text recognition result of the text image to be recognized, including: according to the target recognition parameter, the text feature set is decoded using the initialized decoding network to obtain the text recognition result of the text image to be recognized.

[0031] Optionally, in some embodiments, the recognition unit can be specifically used to determine the character recognition order of the text content in the text image to be recognized based on the text feature set; based on the character recognition order, determine the current text characters that currently need to be recognized in the text content; and recognize the current text characters according to the target recognition parameters to obtain the text recognition result of the text image to be recognized.

[0032] Optionally, in some embodiments, the recognition unit can be specifically used to update the target recognition parameter according to the text feature set to obtain a first updated recognition parameter, and recognize the current text character based on the first updated recognition parameter to obtain a first character recognition result of the current text character; filter out the next text character of the current text character in the text content according to the character recognition order to obtain a target text character; update the first updated recognition parameter according to the first character recognition result to obtain a second updated recognition parameter, and recognize the target text character based on the second updated recognition parameter to obtain a second character recognition result of the target text character; return to execute the step of filtering out the next text character of the current text character in the text content according to the character recognition order, until all text characters in the text content are recognized, and obtain the text recognition result of the text image to be recognized.

[0033] Optionally, in some embodiments, the recognition unit can be specifically used to perform feature extraction on the target recognition parameter to obtain a first recognition feature corresponding to the target recognition parameter; fuse the first recognition feature with the text feature in the text feature set to obtain a first global text feature; and update the target recognition parameter according to the first global text feature to obtain a first updated recognition parameter.

[0034] Optionally, in some embodiments, the recognition unit can be specifically used to perform feature extraction on the first updated recognition parameter to obtain a second recognition feature of the first updated recognition parameter; determine the character feature of the current text character recognized based on the first character recognition result; fuse the second recognition feature with the character feature to obtain a second global text feature; and update the first updated recognition parameter based on the second global text feature to obtain a second updated recognition parameter.

[0035] Optionally, in some embodiments, the extraction unit can be specifically used to identify the text position information of the text content in the text image to be identified; correct the text image to be identified based on the text position information to obtain a corrected text image; and use the encoding network of the recognition model to perform feature extraction on the text content of the corrected text image to obtain a text feature set of the text content.

[0036] Optionally, in some embodiments, the extraction unit can be specifically used to identify at least one image control point of the text content and the target position of the image control point in the text image to be recognized; determine the current position of the image control point in the text image to be recognized based on the text position information; calculate the positional relationship between the text image to be recognized and the corrected text image based on the target position and current position of the image control point; and convert the text image to be recognized into the corrected text image based on the positional relationship.

[0037] Optionally, in some embodiments, the extraction unit can be specifically used to extract image features of the corrected text image to obtain image feature information of the corrected text image; perform bidirectional recognition on the text content based on the image feature information to obtain multiple initial text features in different directions; and splice the initial text features to obtain a text feature set of the text content.

[0038] Optionally, in some embodiments, the determination unit can be specifically used to convert the dimension of the text feature set to obtain a target text feature set of a target dimension; use the semantic recognition network of the recognition model to recognize the target text feature set to obtain the contextual features of the text content; and determine the semantic feature information of the text content based on the contextual features.

[0039] Optionally, in some embodiments, the text image recognition device may further include a processing unit, which may be specifically used to store the text recognition result in response to the save operation of the save control, and return to display the education live broadcast page, the education live broadcast page also including a viewing control, the viewing control is used to view the text recognition result; in response to the viewing operation of the viewing control, a text viewing page is displayed, the text viewing page includes the text recognition result and an editing control; in response to the editing operation of the editing control, the text recognition result is edited.

[0040] In addition, an embodiment of the present invention further provides an electronic device, including a processor and a memory, wherein the memory stores an application program, and the processor is configured to run the application program in the memory to implement the text image recognition method provided by an embodiment of the present invention.

[0041] In addition, an embodiment of the present invention further provides a computer-readable storage medium, which stores a plurality of instructions suitable for loading by a processor to execute the steps in any text image recognition method provided by an embodiment of the present invention.

[0042] In an embodiment of the present invention, after obtaining a text image to be recognized, the text image to be recognized includes at least one text content, and features are extracted from the text content in the text image to be recognized to obtain a text feature set of the text content. Then, based on the text feature set, semantic feature information of the text content is determined. Then, based on the semantic feature information, recognition parameters of the recognition text feature set are adjusted to obtain target recognition parameters. Based on the target recognition parameters, the text feature set is recognized to obtain a text recognition result of the text image to be recognized. Since the scheme determines the semantic feature information of the text content after extracting the text feature set of the text content of the text image to be recognized, the semantic feature information is used as global information to guide the recognition of text characters in the text content, thereby improving the recognition accuracy of text images, especially the recognition accuracy of low-quality text images. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative work.

[0044] Figure 1 Schematic diagram of a scenario of a text image recognition method provided by an embodiment of the present invention;

[0045] Figure 2 Schematic diagram of a scenario on the server side in the text image recognition method provided by an embodiment of the present invention;

[0046] Figure 3 1 is a flow chart of a text image recognition method provided by an embodiment of the present invention;

[0047] Figure 4 is a framework diagram of a text recognition model provided by an embodiment of the present invention;

[0048] Figure 5 is another flowchart of the text image recognition method provided by an embodiment of the present invention;

[0049] Figure 6 Schematic diagram of an education live broadcast page provided by an embodiment of the present invention;

[0050] Figure 7 is a schematic diagram of a content selection page provided by an embodiment of the present invention;

[0051] Figure 8 1 is a schematic diagram of a process for generating a text image to be recognized provided by an embodiment of the present invention;

[0052] Figure 9 is a schematic diagram of a recognition result page provided by an embodiment of the present invention;

[0053] Figure 10 is a schematic diagram of a text viewing page provided by an embodiment of the present invention;

[0054] Figure 11 Schematic diagram of the interaction process in the text image recognition method provided by an embodiment of the present invention;

[0055] Figure 12 is a network structure diagram of a text recognition model provided by an embodiment of the present invention;

[0056] Figure 13 is a structural diagram of a first text image recognition device provided by an embodiment of the present invention;

[0057] Figure 14is a structural diagram of a second text image recognition device provided by an embodiment of the present invention;

[0058] Figure 15 is another structural diagram of a second text image recognition device provided by an embodiment of the present invention;

[0059] Figure 16 It is a structural diagram of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0060] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without making any creative efforts shall fall within the scope of protection of the present invention.

[0061] Embodiments of the present invention provide a text image recognition method, apparatus, and computer-readable storage medium. The text image recognition apparatus can be integrated into an electronic device, which can be a server, a terminal, or other device. Specifically, embodiments of the present invention provide a text image recognition apparatus for a first electronic device (referred to as a first text image recognition apparatus for distinction), and a text image recognition apparatus for a second electronic device (referred to as a second text image recognition apparatus for distinction).

[0062] Among them, the first electronic device can be a server or other device, and the server can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, network acceleration services (Content Delivery Network, CDN), and basic cloud computing services such as big data and artificial intelligence platforms. The second electronic device can be a terminal or other device, wherein the terminal can be a smart phone, tablet computer, laptop computer, desktop computer, smart speaker, smart watch, etc., but is not limited to this. The terminal and the server can be directly or indirectly connected through wired or wireless communication, and this application does not limit this.

[0063] In the embodiment of the present invention, a text image recognition method is introduced by taking the first electronic device as a server and the second electronic device as a terminal as an example.

[0064] For example, see Figure 1The text image recognition system provided by the embodiment of the present invention may include a server 10, or may include a server 10 and a terminal 20, where the server 10 and the terminal are connected via a network, for example, via a wired or wireless network.

[0065] Among them, the server 10 can be used to obtain a text image to be recognized, which includes at least one text content. Then, feature extraction is performed on the text content in the text image to be recognized to obtain a text feature set of the text content. According to the text feature set, semantic feature information of the text content is determined. Then, based on the semantic feature information, the recognition parameters of the recognition text feature set are adjusted to obtain target recognition parameters. According to the target recognition parameters, the text feature set is recognized to obtain a text recognition result of the text image to be recognized. Specifically, Figure 2 shown.

[0066] The semantic feature information may be feature information of semantic similarity of each text character in the text content.

[0067] Artificial intelligence can be used to identify text images. For example, a text recognition model based on a semantically enhanced codec framework can be used to identify character parameter information and scene information in the test frame, as well as generate simulated operation information. Artificial intelligence (AI) refers to the theories, methods, techniques, and application systems that use digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use knowledge to achieve optimal results. In other words, AI is a comprehensive technology in computer science that attempts to understand the essence of intelligence and produce new intelligent machines that can respond in a manner similar to human intelligence. AI is the study of the design principles and implementation methods of various intelligent machines, enabling them to have the capabilities of perception, reasoning, and decision-making.

[0068] Artificial intelligence (AI) technology is a comprehensive discipline encompassing a wide range of fields, encompassing both hardware and software technologies. Foundational AI technologies generally include sensors, specialized AI chips, cloud computing, distributed storage, big data processing, operating / interaction systems, and mechatronics. AI software technologies primarily encompass computer vision, speech processing, natural language processing, and machine learning / deep learning.

[0069] Among them, terminal 20 can be used to display an education live broadcast page, which includes text content and recognition controls. In response to the recognition operation of the recognition control, a text content selection page is displayed, which includes a text content selection control. In response to the selection operation of the text content selection control, a text image to be recognized is generated according to the text content, and the text image to be recognized is sent to the server. Then, the text recognition result of the server for the text image to be recognized is obtained, and a recognition result page is displayed. The recognition result page includes the text image to be recognized and the text recognition result of the text image to be recognized.

[0070] Among them, the response is used to indicate the conditions or states on which the executed operations depend. When the dependent conditions or states are met, one or more operations executed can be real-time or have a set delay; unless otherwise specified, there is no restriction on the order of execution of the multiple operations executed.

[0071] It should be noted that the text image recognition method can be executed by the first text image recognition device alone, or can be executed jointly by the first text image recognition device and the second text image recognition device through interaction.

[0072] It should be noted that the order of description of the following embodiments is not intended to limit the preferred order of the embodiments.

[0073] This embodiment will be described from the perspective of a first text image recognition device. The first text image recognition device may be integrated into an electronic device, which may be a server or other device.

[0074] A text image recognition method, comprising:

[0075] A text image to be recognized is obtained, where the text image to be recognized includes at least one text content, features are extracted from the text content in the text image to obtain a text feature set of the text content, semantic feature information of the text content is determined based on the text feature set, recognition parameters of the recognition text feature set are adjusted based on the semantic feature information to obtain target recognition parameters, and the text feature set is recognized based on the target recognition parameters to obtain a text recognition result of the text image to be recognized.

[0076] like Figure 3 As shown, the specific process of the text image recognition method is as follows:

[0077] 101. Obtain a text image to be recognized.

[0078] The text image to be recognized includes at least one text content. The so-called text content may be content including text characters, for example, a piece of text in a different language, a piece of code, or a piece of symbol, etc. having text characters.

[0079] For example, one or more text images to be identified sent by the terminal can be directly obtained. For example, the user selects the text content to be identified on the education live broadcast page, and the terminal generates a text image to be identified based on the text content, and sends the text image directly to the first text image recognition device, so that the first text image recognition device can obtain the text image to be identified. The text image to be identified can also be obtained from the Internet or other databases. For example, images containing text content can be crawled from the Internet or other databases, and the crawled images containing text content can be used as text images to be identified. It is also possible to directly receive images containing text content uploaded by users, and use the received images containing text content as text images to be identified. It is also possible to pre-process the received images containing text content to obtain text images to be identified. For example, a text area containing text content can be identified in the image, and the image can be cropped according to the text area, and the area not containing text content can be cropped to obtain an image of the text area containing only text content, and the image of the text area can be used as the text image to be identified.

[0080] 102. Perform feature extraction on the text content in the text image to be recognized to obtain a text feature set of the text content.

[0081] For example, the text position information of the text content can be identified in the text image to be recognized. Based on the text position information, the text image to be recognized is corrected to obtain a corrected text image. The encoding network of the recognition model is used to extract the text content of the corrected text image features to obtain a text feature set of the text content. Specifically, the text feature set can be as follows:

[0082] S1. Identify text position information of text content in a text image to be identified.

[0083] The text position information may be position information of a text box including text content in the text image to be recognized.

[0084] For example, a text area containing text content is identified in the text image to be recognized, and the position information of the text area is obtained as the text position information of the text content. For example, a text area containing content is identified in the text image to be recognized, and the position information of the virtual text box corresponding to this text area in the entire text image to be recognized is calculated, and the position information is used as the text position information of the text content.

[0085] S2. Correct the text image to be recognized according to the text position information to obtain a corrected text image.

[0086] The correction of the text image to be recognized mainly corrects the distorted text characters in the text content, thereby obtaining a corrected text image.

[0087] For example, the correction network of the recognition model can be used to identify the image control points (control points) corresponding to the text content and the target positions of the image control points in the text image to be recognized. The image control points can be reference points selected on the image for establishing the geometric transformation function in geometric transformations such as image geometric correction, projection transformation and image registration. They can be understood as control points that surround the text characters to be recognized in the text content. There can generally be multiple image control points, and multiple image control points can form the minimum virtual bounding box that surrounds the text characters to be recognized in the text content. For example, a shallow convolutional neural network (Shallow CNN) can be used to identify at least one image control point of the text content and the target position of the image control point in the text image to be recognized.

[0088] According to the text position information, the current position of the image control point in the text image to be recognized is determined, for example, the distance relationship between the image control point and the text area of ​​the text content is calculated, and the current position of the image control point in the text image to be recognized is determined based on the distance relationship and the text position information. Based on the image control point, the target position and the current position, the positional relationship between the text content in the text image to be recognized and the corrected text image is calculated, for example, by using a thin plate spline interpolation algorithm, the positional relationship between the text content in the text image to be recognized and the corrected text image is calculated by comparing the target position and the current position of the image control point. According to the positional relationship, the text image to be recognized is converted into the corrected text image, for example, according to the positional relationship, the image control point is moved or otherwise operated in the text image to make the current position of the image control point coincide with the target position, so that the text image to be recognized can be converted into the corrected text image, thereby completing the correction of the text image to be recognized.

[0089] S3. Use the encoding network of the recognition model to extract features of the text content of the corrected text image to obtain a text feature set of the text content.

[0090] For example, image feature extraction can be performed on the corrected text image to obtain image feature information of the corrected text image. Based on the image feature information, bidirectional recognition can be performed on the text content to obtain initial text feature sets in different directions. The text features in the initial text feature sets can be spliced ​​to obtain a text feature set of the text content. The specific method can be as follows:

[0091] (1) Perform image feature extraction on the corrected text image to obtain image feature information of the corrected text image.

[0092] For example, by performing multi-scale image feature extraction on the corrected text image, multi-scale image feature information of the corrected text image can be obtained. For example, a residual network with 45 layers or other layers can be used to perform multi-scale image feature extraction on the corrected text image to obtain multi-scale image feature information of the corrected text image. This image feature information can also be the visual feature of the text image to be identified.

[0093] (2) Based on the image feature information, the text content is bidirectionally recognized to obtain the initial text features in different directions.

[0094] For example, based on the image feature information, the text characters in the text content are identified from left to right and from right to left respectively, and the initial text features of each text character in different directions are obtained. For example, 256 hidden units and a two-layer bidirectional long short-term memory network (BiLSTM) are used to filter out the target image features of each text character in the text content from the image feature information, and the BiLSTM network is used to identify the target image features from two different directions respectively, and the initial text features of each character in different directions are obtained.

[0095] It should also be noted that the recognition of the initial text features of each text character needs to rely on the recognition result of the previous text character.

[0096] (3) The initial text features are concatenated to obtain a text feature set of the text content.

[0097] For example, according to the order of the text characters in the text content, different initial text features obtained by recognizing the same character twice are concatenated to obtain a text feature set of the text content. For example, if the text content includes three text characters A, B, and C, the target image features corresponding to these three text characters are recognized from left to right, and the initial text features corresponding to these three text characters are L1, L2, and L3 respectively. The target image features corresponding to these three text characters are recognized from right to left, and the initial text features corresponding to these three text characters are R1, R2, and R3 respectively. At this time, two initial text features of each text character can be obtained. For example, the initial text features of text character A can be L1 and R1. These two initial text features are concatenated to obtain the text features of text character A. The initial text features of all text characters in the text content are concatenated to obtain a text feature set of the text content. The size of the text feature set can be L×C, where L is the width of the feature map corresponding to the last image feature and C is the depth of the feature map.

[0098] 103. Determine semantic feature information of the text content based on the text feature set.

[0099] For example, the dimension of the text feature is transformed to obtain a target text feature set of the target dimension. For example, taking the size of the text feature set as L×C as an example, the text feature set is expanded into a one-dimensional text feature with a dimension of K, K=L×C, and the one-dimensional text feature with a dimension of K is used as the target text feature set. The semantic recognition network of the recognition model is used to recognize the target text feature set to obtain the contextual features of the text content. For example, the feedforward neural network of the FastText network (a semantic recognition network) can be used to recognize the one-dimensional text feature with a dimension of K to obtain the contextual features of the text content. According to the contextual features, the semantic feature information of the text content is determined. For example, a linear function can be used to recognize the contextual features to obtain the semantic features corresponding to each text character in the text content, thereby obtaining the semantic feature information of the text content. The linear function can be shown as formula (1):

[0100] S=W2σ(W1I+b1)+b2 (1)

[0101] Among them, S is the semantic feature of the text characters in the text content, W1 and W2 are conversion parameters, b1 and b2 are network parameters, and σ is the linear rectification function (Rectified Linear Unit, ReLU).

[0102] Among them, for the semantic recognition network, choosing the FastText network can use the sub-word algorithm to deal with the problem of "out of vocabulary", thereby greatly improving the accuracy of recognizing the semantic feature information of the text content.

[0103] 104. Based on the semantic feature information, adjust the recognition parameters of the recognition text feature set to obtain target recognition parameters.

[0104] The recognition parameters may be initialization parameters in a decoding network, and are mainly used to initialize the decoding network to guide the recognition of text characters.

[0105] For example, the initialization parameters of the decoding network and the dimension information of the hidden layer in the text recognition model are obtained. For example, the decoding network can be a unidirectional GRU (a variant of an LSTM network) network. The initialization parameters in the GRU network are generally random parameters. It is necessary to identify the current initialization parameters in the decoding network, use the initialization parameters as the recognition parameters for recognizing the text feature set, and obtain the dimension information of the hidden layer of the GRU network. The dimension of the hidden layer can be 512. According to the dimension information, the dimension of the semantic feature information is converted to obtain the converted semantic feature information. For example, taking the dimension of the hidden layer as 512 as an example, the dimension of the semantic feature vector in the semantic feature information can be converted to 512 to obtain the converted semantic feature information. According to the converted semantic feature information, the recognition parameters are adjusted to obtain the target recognition parameters. For example, the recognition parameters in the decoding network are directly replaced with the converted semantic information. At this time, it is equivalent to replacing the initialization parameters of the decoding network with the converted semantic information. At this time, loading the converted semantic information can complete the initialization of the decoding network, thereby obtaining the initialized decoding network and using the converted semantic feature information as the target recognition parameters. The initialization here is equivalent to allowing the decoding network to learn the semantic feature information of the text content in the text image to be recognized. In this way, when the decoding network is used to recognize the text feature set, the decoding network is guided by the semantic feature information, thereby improving the recognition accuracy of the text image to be recognized.

[0106] 105. According to the target recognition parameters, the text feature set is recognized to obtain a text recognition result of the text image to be recognized.

[0107] For example, according to the target recognition parameters, the initialized decoding network is used to decode the text feature set to obtain the text recognition result of the text image to be recognized, which can be specifically as follows:

[0108] C1. Determine the character recognition order of the text content in the text image to be recognized based on the text feature set.

[0109] The character recognition order may be the order in which text characters in the text content are recognized.

[0110] For example, the feature order of text features is determined in the text feature set, and the text characters in the text content are sorted according to the feature order to obtain the character recognition order of the text content in the text image to be recognized. For example, when the text features in the text feature set correspond one-to-one to the text characters in the text content, the text features corresponding to each text character are determined in the text content, and then the text characters in the text content are sorted according to the feature order, and the order of the sorted text characters is used as the character recognition order. When the text features in the text feature set do not correspond one-to-one to the text characters in the text content, the character recognition order can be the original arrangement order of the text characters in the text content.

[0111] C2. Based on the character recognition order, determine the current text characters that need to be recognized in the text content.

[0112] For example, the decoding network recognizes a text character in the text content one at a time, such as a word, a letter, or a word. Based on the character recognition order, a current text character to be recognized is determined each time in the text content. For example, when recognizing text characters in the text content for the first time, the current text character to be recognized is the text character ranked first in the character recognition order, and so on.

[0113] C3. Recognize the current text characters according to the target recognition parameters to obtain a text recognition result of the text image to be recognized.

[0114] For example, according to the text feature set, the target recognition parameter is updated to obtain the first updated recognition parameter, and based on the first updated recognition parameter, the current text character is updated to obtain the first character recognition result of the current text character, and according to the character recognition order, the next text character of the current text character is screened out in the text content to obtain the target text character, and according to the first character recognition result, the first updated recognition parameter is updated to obtain the second updated recognition parameter, and based on the second updated recognition parameter, the target text character is recognized to obtain the second character recognition result of the target text character, and the step of screening out the next text character of the current text character in the text content according to the character recognition order is returned to execute until all the text characters in the text content are recognized to obtain the text recognition result of the text image to be recognized. Specifically, it can be as follows:

[0115] (1) According to the text feature set, the target recognition parameter is updated to obtain a first updated recognition parameter, and based on the first updated recognition parameter, the current text character is updated to obtain a first character recognition result of the current text character.

[0116] For example, feature extraction is performed on the target recognition parameter to obtain a first recognition feature corresponding to the target recognition parameter. For example, if the target recognition parameter is the converted semantic recognition feature information, this is equivalent to extracting a semantic feature vector from the converted semantic feature recognition feature information and using the semantic feature vector as the recognition feature corresponding to the target recognition parameter. The first recognition feature is fused with the text features in the text feature set to obtain a first global text feature. For example, the semantic feature vector is concatenated with the feature vector of the text feature set. Based on the concatenated feature vector, updated gating state parameters and reset gating state parameters are generated. The gating state parameters are fused with the semantic feature vector to obtain a fused semantic feature vector. The fused semantic feature vector is further concatenated with the text features in the text feature set to obtain a first global text feature. Based on the first global text feature, the target recognition parameter is updated to obtain a first updated recognition parameter. For example, the first global text feature and the target recognition parameter are fused by updating the gating state parameter to obtain a first updated recognition parameter.

[0117] Based on the first updated recognition parameter, the current text character is recognized to obtain the first character recognition result of the current text character. For example, the recognition formula shown in formula (2) can be used to recognize the current text character to obtain the first character result of the current text character, which is as follows:

[0118] y t =σ(W o ·h t ) (2)

[0119] Among them, y t is the character feature corresponding to the character recognition result, W o is the identification parameter, h t For identification features.

[0120] (2) According to the character recognition order, the next text character of the current text character is filtered out in the text content to obtain the target text character.

[0121] For example, the character identifier of the next text character of the current text character is queried in the character recognition sequence, and based on the character identifier, the target text character is filtered out in the text content. For example, taking the character recognition sequence of text character A-text character B-text character C, and the current text character is text character A as an example, the character identifier of the next text character of the current text character is text character B. The text character corresponding to text character B is filtered out in the text content to obtain the target text character.

[0122] (3) According to the first character recognition result, the first updated recognition parameter is updated to obtain a second updated recognition parameter, and based on the second updated recognition parameter, the target text word is recognized to obtain a second character recognition result of the target text character.

[0123] For example, feature extraction is performed on the first updated recognition parameter to obtain a second recognition feature of the first updated recognition parameter, for example, semantic features are extracted from the first updated recognition parameter, and the extracted semantic features are used as the second recognition features. According to the first character recognition result, the character features of the current text character recognition are determined. For example, when the first character recognition result is a character vector after the current text character recognition, the character vector can be used as the character feature. When the first character recognition result is a specific word or word recognized, feature extraction is performed on the recognized word or word to obtain the character feature. The second recognition feature is fused with the character feature to obtain a second global text feature. For example, the semantic feature vector of the first updated recognition parameter is spliced ​​with the character feature vector. According to the spliced ​​feature vector, an updated gating state parameter and a reset gating state parameter are generated. The gating state parameter is fused with the semantic feature vector to obtain a fused semantic feature vector. The fused semantic feature vector is spliced ​​with the character feature again to obtain a second global text feature. Based on the second global text feature, the first updated recognition parameter is updated. For example, the second global text feature and the first updated recognition parameter are fused by updating the gated state parameter to obtain the second updated recognition parameter.

[0124] Based on the second updated recognition parameter, the target text character is recognized to obtain a second character recognition result of the target text character. The specific recognition method is the same as the first character recognition result, and formula (2) is also used for recognition, which will not be repeated here.

[0125] (4) Return to execute the step of filtering out the next text character of the current text character in the text content according to the character recognition order, until all the text characters in the text content are recognized, and obtain the text recognition result of the text image to be recognized.

[0126] For example, after the target text character is recognized as the current text character at this time, the step of filtering out the next text character of the current text character in the text content according to the character recognition order can be returned to, that is, filtering out the third text character that needs to be recognized in the text content. First, according to the character recognition result of the previous text character, the current recognition parameters are updated, and the third text character is recognized based on the updated recognition parameters to obtain the character recognition result of the third text character. When the next text character of the third text character still exists in the text content, the next text character is recognized based on the recognition result of the third text character and the updated recognition parameters corresponding to the third text character, and so on, until all the text characters in the text content are recognized, so that the text recognition result of the text image to be recognized can be obtained.

[0127] Among them, it should be noted that the decoding network of the text recognition model is based on the recognition result and recognition parameters of the previous text character when recognizing each text character. Except for the recognition of the first text character, the recognition of the first text character mainly relies on the converted semantic feature information and the current text feature set. It can be found that there are two outputs and two outputs in the decoding network each time. Starting from the second text character, the two inputs are the character recognition result of the previous text character and the recognition parameters updated by the previous text character.

[0128] Among them, for the text recognition model, the basic model mainly includes the encoding network (encoder), the semantic recognition network (Semantic) and the decoding network (decoder). The decoding network can also include the attention mechanism module. The overall framework can be as follows Figure 4 As shown, the framework of the text recognition model can also be a semantically enhanced encoding and decoding framework (SE-ASTER). The encoding network is used to encode the text features of the text image to be recognized, thereby obtaining a text feature set. The semantic recognition network is mainly used to perform semantic recognition on the text features in the text feature set, and identify the semantic features of each text character in the text content that are semantically similar, thereby obtaining semantic feature information. The decoding network is mainly used to decode the text feature set. Before decoding the text feature set, the decoding network is initialized using the semantic feature information. The initialized decoding network is used to perform unidirectional decoding on the text feature set to generate a recognition result for each text character in the text content of the text image to be recognized, thereby obtaining a text recognition result for the text image to be recognized.

[0129] The text recognition model can be set according to the needs of actual applications. In addition, it should be noted that the text recognition model can be pre-set by maintenance personnel or trained by the text recognition model itself. Moreover, since the text recognition model has multiple networks, the training method for the text recognition model adopts end-to-end training. The specific training process can be as follows:

[0130] (1) Collect text image samples, which have been annotated with text content.

[0131] For example, multiple original text images can be collected, such as from a database or the Internet, and images containing text content can be filtered out from the original images to obtain original text images. Text content can be annotated in the original text images to obtain text image samples.

[0132] (2) Use the preset text recognition model to recognize the text content in the image text sample and obtain the recognition result.

[0133] For example, a preset text recognition model is used to extract features of the text content in the text image to be recognized, and a text feature set of the text content is obtained. According to the text feature set, the semantic feature information of the text content is determined. Based on the semantic feature information, the recognition parameters of the text feature set are adjusted to obtain target recognition parameters. According to the target recognition parameters, the text feature set is recognized to obtain a text recognition result of the text image to be recognized.

[0134] (3) The preset text recognition model is converged according to the prediction results and annotation results of the text image samples to obtain the text recognition model.

[0135] For example, a loss function can be used to converge a preset text recognition model based on the prediction results and annotation results of text image samples to obtain a text recognition model. For example, the loss function can be specifically shown as formula (3):

[0136] L=L rec +λL sem (3)

[0137] Among them, L rec is the standard cross entropy loss, L sem is the cosine loss function, and λ is a hyperparameter used to balance the loss, which can be set to 1 or other values. sem The main purpose is to improve the training speed of the text recognition model. Therefore, there is no contrastive loss function.

[0138] Among them, for L sem For example, the specific calculation process can be shown in formula (4), which can be as follows:

[0139] L sem =1-cos(S,em)

[0140] Among them, S is the predicted semantic feature information, and em is the word vector in the FastText model.

[0141] From the above, it can be seen that after the embodiment of the present invention obtains the text image to be recognized, the text image to be recognized includes at least one text content, and features are extracted from the text content in the text image to be recognized to obtain a text feature set of the text content, and then, based on the text feature set, the semantic feature information of the text content is determined, and then, based on the semantic feature information, the recognition parameters of the recognition text feature set are adjusted to obtain target recognition parameters, and based on the target recognition parameters, the text feature set is recognized to obtain the text recognition result of the text image to be recognized; because this scheme also determines the semantic feature information of the text content after extracting the text feature set of the text content of the text image to be recognized, the semantic feature information is used as global information to guide the recognition of text characters in the text content, thereby improving the recognition accuracy of text images, especially the recognition accuracy of low-quality text images.

[0142] This embodiment will be described from the perspective of a second text image recognition device, which can be specifically integrated into an electronic device, which can be a terminal or other device; wherein, the terminal can include a mobile phone, a tablet computer, a laptop computer, a personal computer (PC), a wearable device, a virtual reality device or other smart device that can access business resources.

[0143] A text image recognition method, comprising:

[0144] An education live broadcast page is displayed, which includes text content and a recognition control. In response to the recognition operation of the recognition control, a text content selection page is displayed, which includes a text content selection control. In response to the selection operation of the text content selection control, a text image to be recognized is generated according to the text content, and the text image to be recognized is sent to a server, and the text recognition result of the server for the text image to be recognized is obtained, and a recognition result page is displayed, which includes the text image to be recognized and the text recognition result of the text image to be recognized.

[0145] like Figure 5 As shown, the specific process of the text image recognition method is as follows:

[0146] 201. Display the education live broadcast page.

[0147] The educational live broadcast page can be a page for educational live broadcasts. Educational live broadcasts primarily combine online education with live broadcasts, characterized by the editor providing explanations based on text and graphics presented to the audience. The educational live broadcast page includes text content and recognition controls. The text content can be the graphics and text displayed by the editor during the live broadcast for explanation.

[0148] For example, the user can operate the user operation page of the live broadcast client to trigger the display of the education live broadcast page. For example, the user operates the content search page of the live broadcast client to trigger the display of the education live broadcast page. The education live broadcast page can be displayed as follows: Figure 6 As shown, the content search page can be a page for users to search for various types of educational live broadcast content. Users can use the search control on the content search page to search for the desired educational live broadcast content. For example, users can trigger the display of the educational live broadcast page by operating the content recommendation page on the live broadcast client. The content recommendation page can be a page that recommends various educational live broadcast content to users based on their viewing habits or personal information. Users can select the desired educational live broadcast content on the content recommendation page using the selection control.

[0149] Among them, the control can be expressed in various forms, such as input boxes, icons, and buttons.

[0150] 202. In response to the recognition operation of the recognition control, display a text content selection page.

[0151] The text content selection page includes a text content selection control.

[0152] For example, when a user triggers the recognition operation of the recognition control on the education live broadcast page, in response to the recognition operation of the recognition control, a screenshot is taken of the area displaying the text content in the currently displayed education live broadcast page to obtain an initial text image, and a text content selection page is displayed based on the initial text content image. For example, a text content selection page is generated based on the initial text content image, and then the text content selection page is displayed, and the text content selection page includes the initial text content image and the text content selection control, such as Figure 7 As shown, the text content selection control may be an adjustable text content selection box.

[0153] 203. In response to the selection operation of the text content selection control, generate a text image to be recognized according to the text content, and send the text image to be recognized to the server.

[0154] For example, when a user triggers a text content selection control on a text content selection page and selects a target text content to be recognized in the text content, for example, the user adjusts the position and size of the text content selection box on the text content selection page, and uses the text content of the adjusted content selection box as the target text content. In response to the selection operation of the text content selection control, the image corresponding to the target text content is used as the text image to be recognized, for example, the target area of ​​the target text content is recognized in the initial text content image, and the target area is screenshotted in the initial text content, thereby obtaining the text image to be recognized. Specifically, Figure 8 Then, the text image to be recognized is directly sent to the server. When the number of text images to be recognized is large or the memory is large, the text image to be recognized can also be sent to the server indirectly. For example, the text image to be recognized is stored in a third-party database, and then the storage address is sent to the server. The server obtains the text image to be recognized from the third-party database based on the storage address.

[0155] 204. Obtain text recognition results from the server for the text image to be recognized.

[0156] For example, the text recognition result returned by the server can be directly received. For example, after receiving the text image to be recognized and performing recognition, the server directly sends the recognition result to the second text image recognition device. A request for obtaining the recognition result can also be sent to the server. After receiving the acquisition request, the server determines whether the recognition of the text image to be recognized is completed. If the recognition is completed, the recognition result is directly returned. If the recognition is not yet completed or the recognition is not yet complete, the recognition result can be returned after the recognition of the text image to be recognized is completed. The recognition result can also be obtained from the memory or cache of the server storing the recognition result. For example, the memory or cache storing the recognition result in the server can be read regularly or periodically to filter out the recognition result of the text image to be recognized in the memory or cache.

[0157] 205. Display the recognition result page.

[0158] The recognition result page includes the text image to be recognized and the recognition result of the text image to be recognized.

[0159] For example, after receiving the recognition result of the text image to be recognized, the recognition result page is displayed according to the recognition result of the text image to be recognized. For example, according to the recognition result of the text image to be recognized, the recognition result page is generated and the recognition result page is displayed. Figure 9 shown.

[0160] Optionally, after the recognition result page is displayed, the recognition result page may further include a save control, so that the user can save and edit the recognition result. Therefore, the text image recognition method may further include:

[0161] In response to the save operation of the save control, the text recognition result is stored and the education live broadcast page is returned for display. The education live broadcast page also includes a view control, which is used to check the duplicate text recognition results; in response to the view operation of the view control, the text view page is displayed, and the text view page includes the text recognition result and the edit control; in response to the edit operation of the edit control, the text recognition result is edited.

[0162] For example, when the user triggers the save control on the recognition result page, the text result is stored in response to the save operation of the save control. For example, the text recognition result can be stored in the memory or cache of the live broadcast client in the form of text. At this time, you can return to displaying the education live broadcast page, which can also include a viewing control. When the user triggers the view control on the education live broadcast page, in response to the view operation of the view control, the text view page is displayed, specifically as follows Figure 10 As shown, the text viewing page includes an editing control. The text viewing page can be a subpage of the education live broadcast page or an independent page displayed on the live broadcast client. When the user triggers the editing control on the text viewing page, the text recognition result is edited in response to the editing operation of the editing control. For example, the text of the text recognition result can be modified, deleted, or replaced to obtain the edited text recognition result. The edited text recognition result can also be stored in the memory or cache of the live broadcast client, or it can directly replace the text recognition result stored for the first time.

[0163] From the above, it can be seen that after the education live broadcast page is displayed in this embodiment, the education live broadcast page includes text content and recognition controls. In response to the recognition operation of the recognition control, a text content selection page is displayed. The text content selection page includes a text content selection control. In response to the selection operation of the text content selection control, a text image to be recognized is generated according to the text content, and the text image to be recognized is sent to the server. The text recognition result of the server for the text image to be recognized is obtained, and a recognition result page is displayed. The recognition result page includes the text image to be recognized and the text recognition result of the text image to be recognized. Since the scheme recognizes the text content of the text image to be recognized based on the semantic feature information of the text content in the text image to be recognized by the server, the recognition accuracy of the text image is improved, especially the recognition accuracy of low-quality text images.

[0164] The method described in the above embodiment will be further described in detail below with examples.

[0165] In this embodiment, the first text image recognition device is integrated in the server, the second text image recognition device is integrated in the terminal, the encoding network of the text recognition model is a residual module and a BiLSTM module, the semantic recognition network is a FastText network, the decoding network is a GRU, and the application scenario of the first text image recognition device and the second text image recognition device is the recognition of text images to be recognized generated in educational live broadcasts.

[0166] (1) Training of text recognition model

[0167] For example, the text recognition model is trained in an end-to-end manner, where multiple networks are trained as a whole. Specifically, the training can be as follows:

[0168] (1) Collect text image samples, which have been annotated with text content.

[0169] For example, original images are crawled from a database or the Internet, and images containing text content are filtered out from the original images to obtain original text images. Text content is annotated in the original text images to obtain text image samples.

[0170] (2) Use the preset text recognition model to recognize the text content in the image text sample and obtain the recognition result.

[0171] For example, a preset text recognition model is used to extract features of the text content in the text image to be recognized, and a text feature set of the text content is obtained. According to the text feature set, the semantic feature information of the text content is determined. Based on the semantic feature information, the recognition parameters of the text feature set are adjusted to obtain target recognition parameters. According to the target recognition parameters, the text feature set is recognized to obtain a text recognition result of the text image to be recognized.

[0172] (3) The preset text recognition model is converged according to the prediction results and annotation results of the text image samples to obtain the text recognition model.

[0173] For example, a loss function can be used to converge a preset text recognition model based on the prediction results and annotation results of text image samples to obtain a text recognition model. For example, the loss function can be specifically shown as formula (3):

[0174] L=L rec +λL sem (3)

[0175] Among them, L rec is the standard cross entropy loss, L sem is the cosine loss function, and λ is a hyperparameter used to balance the loss, which can be set to 1 or other values. semThe main purpose is to improve the training speed of the text recognition model. Therefore, there is no contrastive loss function.

[0176] Among them, for L sem For example, the specific calculation process can be shown in formula (4), which can be as follows:

[0177] L sem =1-cos(S,em)

[0178] Among them, S is the predicted semantic feature information, and em is the word vector in the FastText model.

[0179] (2) Application of text recognition model

[0180] like Figure 11 As shown in FIG, a text image recognition method, the specific process is as follows:

[0181] 301. The terminal displays the education live broadcast page.

[0182] For example, the user can trigger the terminal to display the education live broadcast page by operating on the content search page of the live broadcast client, and the user can trigger the terminal to display the education live broadcast page by operating on the content recommendation page of the live broadcast client.

[0183] 302. The terminal displays a text content selection page in response to the recognition operation of the recognition control.

[0184] For example, when a user triggers the recognition operation of the recognition control on the education live broadcast page, the terminal responds to the recognition operation of the recognition control, takes a screenshot of the area displaying the text content in the currently displayed education live broadcast page, obtains the initial text image, generates a text content selection page based on the initial text content image, and then displays the text content selection page.

[0185] 303. In response to the selection operation of the text content selection control, the terminal generates a text image to be recognized according to the text content, and sends the text image to be recognized to the server.

[0186] For example, the user adjusts the position and size of the text content selection box on the text content selection page. The terminal uses the text content of the adjusted content selection box as the target text content, identifies the target area of ​​the target text content in the initial text content image, and takes a screenshot of the target area in the initial text content, thereby obtaining the text image to be recognized and sending the text image to be recognized to the server.

[0187] 304. The server obtains the text image to be recognized.

[0188] For example, when the terminal sends a text image to be recognized to the server, the server can directly receive the text image to be recognized sent by the terminal. When the terminal sends the storage address of the text image to be recognized to the server, the server obtains the text image to be recognized based on the storage address. The server can also directly obtain the text image to be recognized that currently needs to be recognized from the terminal's memory or cache. The server can also pre-process the received image containing text content, for example, identifying the text area with text content in the image, and cropping the image based on the text area, cropping the area that does not contain text content, thereby obtaining an image of the text area containing only text content, and using the image of the text area as the text image to be recognized.

[0189] 305. The server identifies text position information of text content in the text image to be identified.

[0190] For example, the server identifies a text area containing content in the text image to be recognized, calculates the position information of the virtual text box corresponding to the text area in the entire text image to be recognized, and uses the position information as the text position information of the text content.

[0191] 306. The server corrects the text image to be recognized according to the text position information to obtain a corrected text image.

[0192] For example, the server can use a shallow convolutional neural network to identify at least one image control point of the text content and the target position of the image control point in the text image to be identified. The distance relationship between the image control point and the text area of ​​the text content is calculated, and the current position of the image control point in the text image to be identified is determined based on the distance relationship and the text position information. Using a thin plate spline interpolation algorithm, by comparing the target position and current position of the image control point, the positional relationship between the text content in the text image to be identified and the corrected text image is calculated. Based on the positional relationship, the image control point is moved or otherwise operated in the text image to make the current position of the image control point coincide with the target position, and the text image to be identified can be converted into a corrected text image, thereby completing the correction of the text image to be identified.

[0193] 307. The server uses the encoding network of the recognition model to extract features of the text content of the corrected text image to obtain a text feature set of the text content.

[0194] For example, the server can perform image feature extraction on the corrected text image to obtain image feature information of the corrected text image. Based on the image feature information, the server can perform bidirectional recognition on the text content to obtain initial text feature sets in different directions. The server can then concatenate the text features in the initial text feature sets to obtain a text feature set of the text content. The specific steps can be as follows:

[0195] (1) The server extracts image features from the corrected text image to obtain image feature information of the corrected text image.

[0196] For example, the server can use a residual network with 45 layers or other numbers of layers to perform multi-scale image feature extraction on the corrected text image to obtain multi-scale image feature information of the corrected text image, which can also be the visual features of the text image to be identified.

[0197] (2) The server performs bidirectional recognition on the text content based on the image feature information to obtain the initial text features in different directions.

[0198] For example, the server uses 256 hidden units and a two-layer bidirectional long short-term memory network (BiLSTM) to filter out the target image features of each text character in the text content from the image feature information, and uses the BiLSTM network to recognize the target image features from two different directions to obtain the initial text features of each character in different directions.

[0199] (3) The server concatenates the initial text features to obtain a text feature set of the text content.

[0200] For example, based on the order of the text characters in the text content, the different initial text features obtained by recognizing the same character twice can be concatenated to obtain a text feature set for the text content. The size of this text feature set can be L×C, where L is the width of the feature map corresponding to the last image feature, and C is the depth of the feature map.

[0201] 308. The server determines semantic feature information of the text content based on the text feature set.

[0202] For example, taking the size of a text feature set as L×C, the server expands the text feature set into a one-dimensional text feature with a dimension of K, where K = L×C, and thus uses the one-dimensional text feature with a dimension of K as the target text feature set. The server can use the feedforward neural network of the FastText network to identify the one-dimensional text feature with a dimension of K to obtain the contextual features of the text content. The linear function shown in formula (1) can be used to identify the contextual features to obtain the semantic features corresponding to each text character in the text content, thereby obtaining the semantic feature information of the text content.

[0203] 309. The server adjusts the recognition parameters of the recognition text feature set based on the semantic feature information to obtain target recognition parameters.

[0204] For example, the server obtains the initialization parameters in the GRU network, uses the initialization parameters as the recognition parameters for the text feature set, and obtains the dimension information of the hidden layer of the GRU network, which can be 512. The dimension of the semantic feature vector in the semantic feature information is converted to 512 to obtain the converted semantic feature information. The initialization parameters in the GRU network are directly replaced with the converted semantic information, and the converted semantic feature information is used as the target recognition parameter. Loading this converted semantic information can complete the initialization of the GRU network, thereby obtaining the initialized GRU network.

[0205] 310. The server determines a character recognition order for the text content in the text image to be recognized based on the text feature set.

[0206] For example, when the text features in the text feature set correspond one-to-one to the text characters in the text content, the server determines the text features corresponding to each text character in the text content, and sorts the text characters in the text content according to the feature order, and uses the order of the sorted text characters as the character recognition order. When the text features in the text feature set do not correspond one-to-one to the text characters in the text content, the character recognition order can be the original arrangement order of the text characters in the text content.

[0207] 311. The server determines the current text characters that need to be recognized in the text content based on the character recognition order.

[0208] For example, the GRU network recognizes text content one text character at a time. When recognizing a text character in the text content for the first time, the current text character that needs to be recognized is the text character that ranks first in the character recognition order, and so on.

[0209] 312. The server recognizes the current text characters according to the target recognition parameters to obtain a text recognition result of the text image to be recognized.

[0210] For example, the server updates the target recognition parameter according to the text feature set to obtain the first updated recognition parameter, and based on the first updated recognition parameter, updates the current text character to obtain the first character recognition result of the current text character, and filters out the next text character of the current text character in the text content according to the character recognition order to obtain the target text character, updates the first updated recognition parameter according to the first character recognition result to obtain the second updated recognition parameter, and recognizes the target text character based on the second updated recognition parameter to obtain the second character recognition result of the target text character, and returns to execute the step of filtering out the next text character of the current text character in the text content according to the character recognition order, until all the text characters in the text content are recognized, and the text recognition result of the text image to be recognized is obtained. The specific steps can be as follows:

[0211] (1) The server updates the target recognition parameters according to the text feature set to obtain first updated recognition parameters, and updates the current text characters based on the first updated recognition parameters to obtain a first character recognition result of the current text characters.

[0212] For example, the server extracts a semantic feature vector from the converted semantic feature recognition feature information, and uses the semantic feature vector as the recognition feature corresponding to the target recognition parameter. The semantic feature vector is spliced ​​with the feature vector of the text feature set, and based on the spliced ​​feature vector, an updated gating state parameter and a reset gating state parameter are generated. The gating state parameter is fused with the semantic feature vector to obtain a fused semantic feature vector, and the fused semantic feature vector is spliced ​​again with the text features in the text feature set to obtain a first global text feature. The first global text feature and the target recognition parameter are fused by updating the gating state parameter to obtain a first updated recognition parameter. Based on the first updated recognition parameter, the recognition formula shown in formula (2) is used to recognize the current text character to obtain the first character recognition result of the current text character.

[0213] (2) The server selects the next text character of the current text character from the text content according to the character recognition order to obtain the target text character.

[0214] For example, the character identifier of the next text character of the current text character is searched in the character recognition sequence, and the target text character is filtered out in the text content according to the character identifier.

[0215] (3) The server updates the first updated recognition parameter according to the first character recognition result to obtain a second updated recognition parameter, and recognizes the target text character based on the second updated recognition parameter to obtain a second character recognition result of the target text character.

[0216] For example, the server extracts semantic features from the first updated recognition parameters and uses the extracted semantic features as the second recognition features. When the first character recognition result is the character vector after the current text character recognition, the character vector can be used as the character feature. When the first character recognition result is the recognized specific word or word, the recognized word or word is subjected to feature extraction to obtain the character feature. The semantic feature vector of the first updated recognition parameter is spliced ​​with the character feature vector. According to the spliced ​​feature vector, the updated gated state parameters and the reset gated state parameters are generated. The gated state parameters are fused with the semantic feature vector to obtain the fused semantic feature vector. The fused semantic feature vector is spliced ​​with the character feature again to obtain the second global text feature. By updating the gated state parameters, the second global text feature and the first updated recognition parameter are fused to obtain the second updated recognition parameter. Based on the second updated recognition parameter, the target text character is recognized to obtain the second character recognition result of the target text character.

[0217] (4) The server returns to execute the step of filtering out the next text character of the current text character in the text content according to the character recognition order, until all the text characters in the text content are recognized, and obtains the text recognition result of the text image to be recognized.

[0218] For example, after the server completes the recognition of the target text character as the current text character at this time, it can return to execute the step of filtering out the next text character of the current text character in the text content according to the character recognition order, that is, filtering out the third text character that needs to be recognized in the text content. First, according to the character recognition result of the previous text character, the current recognition parameters are updated, and the third text character is recognized based on the updated recognition parameters to obtain the character recognition result of the third text character. When the next text character of the third text character still exists in the text content, the next text character is recognized based on the recognition result of the third text character and the updated recognition parameters corresponding to the third text character, and so on, until all the text characters in the text content are recognized, so that the text recognition result of the text image to be recognized can be obtained.

[0219] Among them, the overall network architecture diagram can be seen as follows: Figure 12As shown in the figure, it is mainly divided into four parts. The first is the correction module. The correction module uses a shallow recognition model to identify the image control points of the text image to be identified. Through the image control points, the thin plate spline interpolation algorithm is used to correct the text image to obtain the corrected text image. The second is the encoding network. The encoding network uses the residual module and the BiLSTM network to encode the corrected text image to obtain the text feature set of the text content. The third is the semantic recognition network. The semantic recognition network uses the FastText network through a linear function based on the text feature set to determine the semantic feature information of the text content. The fourth is the encoding network. The encoding network initializes the GRU network through the semantic feature information, and uses the initialized GRU network to recognize the text feature set, thereby obtaining the text recognition result of the text image to be identified.

[0220] 313. The terminal obtains a text recognition result from the server for the text image to be recognized.

[0221] For example, after receiving and recognizing an image of text to be recognized, the server directly sends the recognition results to the terminal. The terminal can also send a request to the server to obtain the recognition results. After receiving the request, the server determines whether the recognition of the image to be recognized is complete. If so, the server directly returns the recognition results. If not, the server waits until the recognition of the image to be recognized is complete before returning the recognition results. The terminal can also directly read the recognition results from the server's memory or cache, which stores the recognition results.

[0222] 314. The terminal displays the recognition result page.

[0223] For example, after receiving the recognition result of the text image to be recognized, the terminal generates a recognition result page according to the recognition result of the text image to be recognized, and displays the recognition result page.

[0224] Optionally, after the recognition result page is displayed, the recognition result page may further include a save control, and the user may also save and edit the recognition result. For example, when the user triggers the save control on the recognition result page, in response to the save operation of the save control, the text recognition result may be stored in the memory or cache of the live broadcast client in the form of text. At this point, you can return to displaying the education live broadcast page, which may also include a view control. When the user triggers the view control on the education live broadcast page, in response to the view operation of the view control, the text view page is displayed. When the user triggers the edit control on the text view page, in response to the edit operation of the edit control, the text recognition result is edited to obtain the edited text recognition result. The edited text recognition result may also be stored in the memory or cache of the live broadcast client, or it may directly replace the text recognition result stored for the first time.

[0225] From the above, it can be seen that after the server of this embodiment obtains the text image to be recognized, the text image to be recognized includes at least one text content, and performs feature extraction on the text content in the text image to be recognized to obtain a text feature set of the text content, and then, based on the text feature set, determines the semantic feature information of the text content, and then, based on the semantic feature information, adjusts the recognition parameters of the recognition text feature set to obtain target recognition parameters, and recognizes the text feature set based on the target recognition parameters to obtain the text recognition result of the text image to be recognized; because this scheme also determines the semantic feature information of the text content after extracting the text feature set of the text content of the text image to be recognized, the semantic feature information is used as global information to guide the recognition of text characters in the text content, thereby improving the recognition accuracy of text images, especially the recognition accuracy of low-quality text images.

[0226] In order to better implement the above method, an embodiment of the present invention further provides a text image recognition device (i.e., a first text image recognition device). The first text image recognition device can be integrated in a server, which can be a single server or a server cluster composed of multiple servers.

[0227] For example, Figure 13 As shown, the first text image recognition device may include a first acquisition unit 401, an extraction unit 402, a determination unit 403, an adjustment unit 404, and a recognition unit 405, as follows:

[0228] (1) a first acquisition unit 401;

[0229] The first acquiring unit 401 is configured to acquire a text image to be recognized, where the text image to be recognized includes at least one text content.

[0230] For example, the first acquisition unit 401 can be specifically used by users to filter out text content that needs to be identified on the education live broadcast page. The terminal generates a text image to be identified based on the text content, and sends the text image directly to the first text image recognition device. The first text image recognition device can obtain the text image to be identified, and can also obtain the text image to be identified from the Internet or other databases.

[0231] (2) Extraction unit 402;

[0232] The extraction unit 402 is configured to extract features from the text content in the text image to be recognized, and obtain a text feature set of the text content.

[0233] For example, the extraction unit 402 can be specifically used to identify the text position information of the text content in the text image to be identified, correct the text image to be identified based on the text position information to obtain a corrected text image, and use the encoding network of the recognition model to extract features of the text content of the corrected text image features to obtain a text feature set of the text content.

[0234] (3) determining unit 403;

[0235] The determining unit 403 is configured to determine semantic feature information of the text content according to the text feature set.

[0236] For example, determination unit 403 can be specifically used to convert the dimensions of text features to obtain a target text feature set of a target dimension, use the semantic recognition network of the recognition model to recognize the target text feature set, obtain the contextual features of the text content, and determine the semantic feature information of the text content based on the contextual features.

[0237] (4) adjustment unit 404;

[0238] The adjustment unit 404 is configured to adjust the recognition parameters of the recognition text feature set based on the semantic feature information to obtain target recognition parameters.

[0239] For example, the adjustment unit 404 can be specifically used to obtain the initialization parameters of the decoding network and the dimensional information of the hidden layer in the text recognition model, convert the dimension of the semantic feature information according to the dimensional information to obtain the converted semantic feature information, and adjust the recognition parameters according to the converted semantic feature information to obtain the target recognition parameters.

[0240] (5) Identification unit 405;

[0241] The recognition unit 405 is used to recognize the text feature set according to the target recognition parameter to obtain a text recognition result of the text image to be recognized.

[0242] For example, the recognition unit 405 can be specifically used to determine the character recognition order of the text content in the text image to be recognized based on the text feature set, determine the current text characters that need to be recognized in the text content based on the character recognition order, and recognize the current text characters according to the target recognition parameters to obtain the text recognition result of the text image to be recognized.

[0243] In specific implementation, the above units can be implemented as independent entities, or can be arbitrarily combined to be implemented as the same or several entities. The specific implementation of the above units can be found in the previous method embodiments and will not be repeated here.

[0244] From the above, it can be seen that in this embodiment, after the first acquisition unit 401 acquires the text image to be recognized, the text image to be recognized includes at least one text content, the extraction unit 402 performs feature extraction on the text content in the text image to be recognized to obtain a text feature set of the text content, and then the determination unit 403 determines the semantic feature information of the text content based on the text feature set, and then the adjustment unit 404 adjusts the recognition parameters of the recognition text feature set based on the semantic feature information to obtain the target recognition parameters, and the recognition unit 405 recognizes the text feature set based on the target recognition parameters to obtain the text recognition result of the text image to be recognized; because this scheme also determines the semantic feature information of the text content after extracting the text feature set of the text content of the text image to be recognized, and uses the semantic feature information as global information to guide the recognition of text characters in the text content, thereby improving the recognition accuracy of the text image, especially the recognition accuracy of low-quality text images.

[0245] In order to better implement the above method, an embodiment of the present invention also provides a text image recognition device (i.e., a second text image recognition device), which can be integrated into a terminal, which can include a smart phone, tablet computer, laptop computer and / or personal computer, etc.

[0246] For example, Figure 14 As shown, the second text image recognition device may include a live page display unit 501, a selection page display unit 502, a generation unit 503, a second acquisition unit 504 and a recognition result display unit 505, as follows:

[0247] (1) Live page display unit 501;

[0248] The live broadcast page display unit 501 is used to display an education live broadcast page, which includes text content and identification controls.

[0249] For example, the live broadcast page display unit 501 can be specifically used to enable the user to operate the user operation page of the live broadcast client to trigger the display of the education live broadcast page.

[0250] (2) Selecting the page display unit 502;

[0251] The selection page display unit 502 is configured to display a text content selection page in response to a recognition operation of the recognition control, where the text content selection page includes a text content selection control.

[0252] For example, the selection page display unit 502 can be specifically used to take a screenshot of the area displaying the text content in the currently displayed education live broadcast page in response to the recognition operation of the recognition control when the user triggers the recognition operation of the recognition control, obtain the initial text image, and display the text content selection page based on the initial text content image.

[0253] (3) generating unit 503;

[0254] The generating unit 503 is configured to generate a text image to be recognized according to the text content in response to a selection operation of the text content selection control, and send the text image to be recognized to the server.

[0255] For example, the generating unit 503 may be configured to, when a user triggers a text content selection control on a text content selection page to select a target text content to be recognized in the text content, send an image corresponding to the target text content as a text image to be recognized to a server in response to the selection operation of the text content selection control.

[0256] (4) a second acquisition unit 504;

[0257] The second acquiring unit 504 is configured to acquire a text recognition result from the server for the text image to be recognized.

[0258] For example, the second acquiring unit 504 may be configured to directly receive the text recognition result returned by the server, or acquire the text recognition result from a memory or cache of the server that stores the recognition result.

[0259] (5) Recognition result display unit 505;

[0260] The recognition result display unit 505 is used to display a recognition result page, which includes the text image to be recognized and the text recognition result of the text image to be recognized.

[0261] For example, the recognition result display unit 505 may be specifically configured to generate a recognition result page according to the recognition result of the text image to be recognized, and display the recognition result page.

[0262] Optionally, the second text image recognition device may further include a processing unit 506, such as Figure 15 As shown, the details are as follows:

[0263] The processing unit 506 is configured to process the text recognition result of the text image to be recognized.

[0264] For example, the processing unit 506 can be specifically used to store the text recognition results in response to the save operation of the save control, and return to display the education live broadcast page, which also includes a view control, and the view control is used to check the duplicate text recognition results; in response to the view operation of the view control, display the text view page, the text view page includes the text recognition results and the edit control; in response to the edit operation of the edit control, edit the text recognition results.

[0265] In specific implementation, the above units can be implemented as independent entities, or can be arbitrarily combined to be implemented as the same or several entities. The specific implementation of the above units can be found in the previous method embodiments and will not be repeated here.

[0266] From the above, it can be seen that in this embodiment, after the live page display unit 501 displays the education live page, the education live page includes text content and recognition controls. The selection page display unit 502 displays a text content selection page in response to the recognition operation of the recognition control. The text content selection page includes a text content selection control. The generation unit 503 generates a text image to be recognized according to the text content in response to the selection operation of the text content selection control, and sends the text image to be recognized to the server. The second acquisition unit 504 obtains the text recognition result of the server for the text image to be recognized. The recognition result display unit 505 displays the recognition result page. The recognition result page includes the text image to be recognized and the text recognition result of the text image to be recognized. Since the scheme recognizes the text content of the text image to be recognized based on the semantic feature information of the text content in the text image to be recognized by the server, the recognition accuracy of the text image is improved, especially the recognition accuracy of low-quality text images.

[0267] An embodiment of the present invention further provides an electronic device, such as Figure 16 , which shows a schematic structural diagram of an electronic device involved in an embodiment of the present invention, specifically:

[0268] The electronic device may include one or more processing core processors 601, one or more computer-readable storage media memories 602, a power supply 603, an input unit 604 and other components. Those skilled in the art will understand that Figure 16 The electronic device structure shown in the figure does not constitute a limitation of the electronic device, and may include more or fewer components than shown in the figure, or combine certain components, or arrange components differently.

[0269] The processor 601 is the control center of the electronic device, connecting the various parts of the entire electronic device using various interfaces and lines. By running or executing software programs and / or modules stored in the memory 602 and calling data stored in the memory 602, it performs various functions of the electronic device and processes data, thereby performing overall detection of the electronic device. Optionally, the processor 601 may include one or more processing cores; preferably, the processor 601 may integrate an application processor and a modem processor, wherein the application processor mainly processes the operating system, user interface, and application programs, and the modem processor mainly handles wireless communications. It is understood that the above-mentioned modem processor may not be integrated into the processor 601.

[0270] The memory 602 can be used to store software programs and modules. The processor 601 executes various functional applications and data processing by running the software programs and modules stored in the memory 602. The memory 602 may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc.; the data storage area may store data created according to the use of the electronic device, etc. In addition, the memory 602 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other volatile solid-state storage device. Accordingly, the memory 602 may also include a memory controller to provide the processor 601 with access to the memory 602.

[0271] The electronic device also includes a power supply 603 for supplying power to various components. Preferably, the power supply 603 can be logically connected to the processor 601 via a power management system, thereby enabling the power management system to manage charging, discharging, and power consumption. The power supply 603 can also include one or more DC or AC power supplies, a recharging system, a power failure detection circuit, a power converter or inverter, a power status indicator, and other arbitrary components.

[0272] The electronic device may further include an input unit 604, which may be used to receive input digital or character information and generate keyboard, mouse, joystick, optical or trackball signal input related to user settings and function control.

[0273] Although not shown, the electronic device may further include a display unit, etc., which will not be described in detail here. Specifically, in this embodiment, the processor 601 in the electronic device will load the executable files corresponding to the processes of one or more application programs into the memory 602 according to the following instructions, and the processor 601 will run the application programs stored in the memory 602 to implement various functions as follows:

[0274] A text image to be recognized is obtained, where the text image to be recognized includes at least one text content, features are extracted from the text content in the text image to obtain a text feature set of the text content, semantic feature information of the text content is determined based on the text feature set, recognition parameters of the recognition text feature set are adjusted based on the semantic feature information to obtain target recognition parameters, and the text feature set is recognized based on the target recognition parameters to obtain a text recognition result of the text image to be recognized.

[0275] or

[0276] An education live broadcast page is displayed, which includes text content and a recognition control. In response to the recognition operation of the recognition control, a text content selection page is displayed, which includes a text content selection control. In response to the selection operation of the text content selection control, a text image to be recognized is generated according to the text content, and the text image to be recognized is sent to a server, and the text recognition result of the server for the text image to be recognized is obtained, and a recognition result page is displayed, which includes the text image to be recognized and the text recognition result of the text image to be recognized.

[0277] The specific implementation of the above operations can be found in the previous embodiments and will not be described in detail here.

[0278] From the above, it can be seen that after the embodiment of the present invention obtains the text image to be recognized, the text image to be recognized includes at least one text content, and features are extracted from the text content in the text image to be recognized to obtain a text feature set of the text content, and then, based on the text feature set, the semantic feature information of the text content is determined, and then, based on the semantic feature information, the recognition parameters of the recognition text feature set are adjusted to obtain target recognition parameters, and based on the target recognition parameters, the text feature set is recognized to obtain the text recognition result of the text image to be recognized; because this scheme also determines the semantic feature information of the text content after extracting the text feature set of the text content of the text image to be recognized, the semantic feature information is used as global information to guide the recognition of text characters in the text content, thereby improving the recognition accuracy of text images, especially the recognition accuracy of low-quality text images.

[0279] Those skilled in the art will appreciate that all or part of the steps in the various methods of the above embodiments may be accomplished by instructions, or by controlling related hardware through instructions. The instructions may be stored in a computer-readable storage medium and loaded and executed by a processor.

[0280] To this end, an embodiment of the present invention provides a computer-readable storage medium storing a plurality of instructions that can be loaded by a processor to execute the steps of any of the text image recognition methods provided in the embodiments of the present invention. For example, the instructions can execute the following steps:

[0281] A text image to be recognized is obtained, where the text image to be recognized includes at least one text content, features are extracted from the text content in the text image to obtain a text feature set of the text content, semantic feature information of the text content is determined based on the text feature set, recognition parameters of the recognition text feature set are adjusted based on the semantic feature information to obtain target recognition parameters, and the text feature set is recognized based on the target recognition parameters to obtain a text recognition result of the text image to be recognized.

[0282] or

[0283] An education live broadcast page is displayed, which includes text content and a recognition control. In response to the recognition operation of the recognition control, a text content selection page is displayed, which includes a text content selection control. In response to the selection operation of the text content selection control, a text image to be recognized is generated according to the text content, and the text image to be recognized is sent to a server, and the text recognition result of the server for the text image to be recognized is obtained, and a recognition result page is displayed, which includes the text image to be recognized and the text recognition result of the text image to be recognized.

[0284] The specific implementation of the above operations can be found in the previous embodiments and will not be repeated here.

[0285] The computer-readable storage medium may include a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc.

[0286] Since the instructions stored in the computer-readable storage medium can execute the steps in any text image recognition method provided in the embodiments of the present invention, the beneficial effects that can be achieved by any text image recognition method provided in the embodiments of the present invention can be achieved. Please refer to the previous embodiments for details and will not be repeated here.

[0287] According to one aspect of the present application, a computer program product or computer program is provided, comprising computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the methods provided in various optional implementations of the aforementioned text image recognition aspects.

[0288] The above is a detailed introduction to a text image recognition method, device and computer-readable storage medium provided in an embodiment of the present invention. Specific examples are used herein to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core ideas. At the same time, for those skilled in the art, according to the ideas of the present invention, there may be changes in the specific implementation methods and application scopes. In summary, the content of this specification should not be understood as limiting the present invention.

Claims

1. A text image recognition method, characterized in that: include: Acquire a text image to be recognized, where the text image to be recognized includes at least one text content; Extracting features of the text content in the to-be-recognized text image to obtain a text feature set of the text content; Determining semantic feature information of the text content based on the text feature set; Based on the semantic feature information, adjusting the recognition parameters for identifying the text feature set to obtain target recognition parameters, wherein the recognition parameters of the text feature set are initialization parameters of a decoding network in a text recognition model, and the target recognition parameters are obtained by adjusting the recognition parameters according to the converted semantic feature information; Performing feature extraction on the target recognition parameter to obtain a first recognition feature corresponding to the target recognition parameter; fusing the first recognition feature with a text feature in the text feature set to obtain a first global text feature; updating the target recognition parameter based on the first global text feature to obtain a first updated recognition parameter, and updating the current text character based on the first updated recognition parameter to obtain a first character recognition result of the current text character; and screening the next text character of the current text character in the text content according to a character recognition order to obtain a target text character; Extracting semantic features from the first updated recognition parameter as second recognition features of the first updated recognition parameter; determining character features of the recognized characters of the current text based on the first character recognition result; Fusing the second recognition feature with the character feature to obtain a second global text feature; Based on the second global text feature, the first updated recognition parameter is updated to obtain a second updated recognition parameter, and based on the second updated recognition parameter, the target text character is recognized to obtain a second character recognition result of the target text character; Return to the step of filtering out the next text character of the current text character in the text content according to the character recognition order, until all the text characters in the text content are recognized, and obtain the text recognition result of the text image to be recognized. Each time the decoding network is used to recognize the text characters in the text feature set, it is guided by the converted semantic feature information.

2. The text image recognition method according to claim 1, characterized in that: The step of adjusting the recognition parameters for identifying the text feature set based on the semantic feature information to obtain target recognition parameters includes: Obtaining initialization parameters of a decoding network and dimension information of a hidden layer in a text recognition model, and using the initialization parameters as recognition parameters for identifying the text feature set, wherein the text recognition model is used to identify the text content of the text image to be recognized; Converting the dimension of the semantic feature information according to the dimension information to obtain converted semantic feature information; The recognition parameters are adjusted according to the converted semantic feature information to obtain the target recognition parameters.

3. The text image recognition method according to claim 2, characterized in that: The step of adjusting the recognition parameters according to the converted semantic feature information to obtain the target recognition parameters includes: Replacing the recognition parameters with the converted semantic feature information to initialize the decoding network, and using the converted semantic feature information as the target recognition parameters; The method of identifying the text feature set according to the target recognition parameters to obtain a text recognition result of the text image to be recognized includes: decoding the text feature set using an initialized decoding network according to the target recognition parameters to obtain a text recognition result of the text image to be recognized.

4. The text image recognition method according to claim 3, characterized in that: The method of decoding the text feature set using an initialized decoding network according to the target recognition parameters to obtain a text recognition result of the text image to be recognized includes: Determining a character recognition order of text content in the to-be-recognized text image based on the text feature set; Based on the character recognition order, determining a current text character that currently needs to be recognized in the text content; The current text characters are recognized according to the target recognition parameters to obtain a text recognition result of the text image to be recognized.

5. The text image recognition method according to claim 2, characterized in that: The feature extraction of the text content in the to-be-recognized text image to obtain a text feature set of the text content includes: Recognizing text position information of the text content in the to-be-recognized text image; Correcting the to-be-recognized text image according to the text position information to obtain a corrected text image; The encoding network of the recognition model is used to perform feature extraction on the text content of the corrected text image to obtain a text feature set of the text content.

6. The text image recognition method according to claim 5, characterized in that: Correcting the to-be-recognized text image according to the text position information to obtain a corrected text image includes: Identifying at least one image control point of the text content and a target position of the image control point in the text image to be identified; Determining the current position of the image control point in the to-be-recognized text image according to the text position information; Calculating the positional relationship between the text content in the to-be-recognized text image and the corrected text image based on the target position and the current position of the image control point; According to the positional relationship, the to-be-recognized text image is converted into the corrected text image.

7. The text image recognition method according to claim 5, characterized in that: The encoding network of the recognition model is used to extract features from the text content of the corrected text image to obtain a text feature set of the text content, including: Performing image feature extraction on the corrected text image to obtain image feature information of the corrected text image; Performing bidirectional recognition on the text content based on the image feature information to obtain multiple initial text features in different directions; The initial text features are spliced ​​together to obtain a text feature set of the text content.

8. The text image recognition method according to claim 2, characterized in that: Determining the semantic feature information of the text content based on the text feature set includes: Converting the dimension of the text feature set to obtain a target text feature set of a target dimension; Using the semantic recognition network of the recognition model to recognize the target text feature set to obtain contextual features of the text content; Determine semantic feature information of the text content based on the context feature.

9. A text image recognition method, characterized in that: include: Displaying an education live broadcast page, wherein the education live broadcast page includes text content and an identification control; In response to the recognition operation of the recognition control, displaying a text content selection page, the text content selection page including a text content selection control; In response to a selection operation of the text content selection control, generating a text image to be recognized according to the text content, and sending the text image to be recognized to a server; Obtaining a text recognition result of the server for the text image to be recognized, wherein the text recognition result is obtained by the server after performing text image recognition on the text image to be recognized using the text image recognition method according to any one of claims 1 to 8; A recognition result page is displayed, wherein the recognition result page includes a text image to be recognized and a text recognition result of the text image to be recognized.

10. The text image recognition method according to claim 9, characterized in that: The recognition result page further includes a save control. After the recognition result page is displayed, the following further includes: In response to a save operation of the save control, the text recognition result is stored, and the education live broadcast page is returned for display, wherein the education live broadcast page further includes a view control for viewing the text recognition result; In response to a viewing operation of the viewing control, displaying a text viewing page, the text viewing page including the text recognition result and an editing control; In response to an editing operation of the editing control, the text recognition result is edited.

11. A text image recognition device, characterized in that: include: A first acquiring unit is configured to acquire a text image to be recognized, wherein the text image to be recognized includes at least one text content; An extraction unit, configured to extract features of the text content in the to-be-recognized text image to obtain a text feature set of the text content; a determining unit, configured to determine semantic feature information of the text content based on the text feature set; an adjusting unit, configured to adjust recognition parameters for identifying the text feature set based on the semantic feature information to obtain target recognition parameters, wherein the recognition parameters of the text feature set are initialization parameters of a decoding network in a text recognition model, and the target recognition parameters are obtained by adjusting the recognition parameters according to the converted semantic feature information; a recognition unit configured to perform feature extraction on the target recognition parameter to obtain a first recognition feature corresponding to the target recognition parameter; fuse the first recognition feature with a text feature in the text feature set to obtain a first global text feature; update the target recognition parameter based on the first global text feature to obtain a first updated recognition parameter; and update the current text character based on the first updated recognition parameter to obtain a first character recognition result of the current text character; and screen the next text character of the current text character in the text content according to a character recognition order to obtain a target text character; Extracting semantic features from the first updated recognition parameter as second recognition features of the first updated recognition parameter; determining character features of the recognized characters of the current text based on the first character recognition result; Fusing the second recognition feature with the character feature to obtain a second global text feature; Based on the second global text feature, the first updated recognition parameter is updated to obtain a second updated recognition parameter, and based on the second updated recognition parameter, the target text character is recognized to obtain a second character recognition result of the target text character; Return to the step of filtering out the next text character of the current text character in the text content according to the character recognition order, until all the text characters in the text content are recognized, and obtain the text recognition result of the text image to be recognized. Each time the decoding network is used to recognize the text characters in the text feature set, it is guided by the converted semantic feature information.

12. A text image recognition device, characterized in that: include: A live broadcast page display unit, configured to display an education live broadcast page, the education live broadcast page including text content and an identification control; a selection page display unit, configured to display a text content selection page in response to a recognition operation of the recognition control, wherein the text content selection page includes a text content selection control; a generating unit, configured to generate a text image to be recognized according to the text content in response to a selection operation of the text content selection control, and send the text image to be recognized to a server; a second acquiring unit, configured to acquire a text recognition result of the server for the text image to be recognized, the text recognition result being obtained by the server after performing text image recognition on the text image to be recognized using the text image recognition method according to any one of claims 1 to 8; The recognition result display unit is used to display a recognition result page, wherein the recognition result page includes a text image to be recognized and a text recognition result of the text image to be recognized.

13. An electronic device, characterized in that: The invention comprises a processor and a memory, wherein the memory stores a computer program, and the processor is used to run the computer program in the memory to execute the text image recognition method according to any one of claims 1 to 10.

14. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, and the computer program is suitable for being loaded by a processor to execute the text image recognition method according to any one of claims 1 to 10.

15. A computer program product, characterized in that The computer program product stores a computer program, and the computer program is suitable for being loaded by a processor to execute the text image recognition method according to any one of claims 1 to 10.

Images