Text recognition method, apparatus, device, and storage medium

Abstract

The application provides a text recognition method, device and equipment and a storage medium. The method comprises the following steps: obtaining an image to be recognized including at least one line of text, first obtaining a first feature of the image to be recognized through a feature extraction network, the first feature being a visual feature; then further extracting a first sequence feature containing global information and local information of the text from the first feature through a convolutional Transformer network; and finally obtaining text information in the image to be recognized, i.e. a text recognition result, from the first sequence feature through a decoding module. The convolutional Transformer network in this embodiment is different from an ordinary Transformer network, and comprises a plurality of convolutional Transformer sub-networks connected in sequence, each convolutional Transformer sub-network comprising a linear projection layer and a convolutional projection layer, the linear projection layer being used to extract global information of image text, and the convolutional projection layer being used to extract local information of image text. Through fusion of the global and local information of the image text, the text recognition effect in a complex scene can be improved.

Description

Technical Field

[0001] The present application relates to the field of computer vision technology, and in particular to a text recognition method, apparatus, device, and storage medium. Background Art

[0002] Optical character recognition (OCR), also known as optical text recognition, is a branch of computer vision (CV) research that uses image processing and pattern recognition techniques to identify optical characters. With the development of the OCR cloud platform, it supports an increasing number of scenarios, including recognition of cards, invoices, product images, and educational examination questions.

[0003] In recent years, research interest in the Transformer architecture has gradually extended from the field of natural language processing (NLP) to the field of computer vision. However, the Transformer structure, which originated in NLP, is not fully adapted to OCR, especially for text recognition in complex scenarios, such as multi-line text or text containing mathematical formulas. Its text recognition effect is poor. Summary of the Invention

[0004] The embodiments of the present application provide a text recognition method, apparatus, device, and storage medium to improve text recognition effects in complex scenarios.

[0005] A first aspect of an embodiment of the present application provides a text recognition method, comprising:

[0006] Acquire an image to be recognized, wherein the image to be recognized includes at least one line of text;

[0007] Acquire a first feature of the image to be identified through a feature extraction network, where the first feature is a visual feature;

[0008] Extracting a first sequence feature containing global information and local information from the first feature through a convolutional Transformer network; the convolutional Transformer network includes N convolutional Transformer subnetworks connected in sequence, each convolutional Transformer subnetwork includes a linear projection layer and a convolutional projection layer, the linear projection layer is used to extract the global information, and the convolutional projection layer is used to extract the local information, where N is a positive integer greater than or equal to 2;

[0009] The text information in the image to be identified is obtained from the first sequence features through a decoding module.

[0010] In an optional embodiment of the first aspect of the present application, extracting a first sequence feature containing global information and local information from the first feature through the convolutional Transformer network includes:

[0011] Inputting the first feature into the first convolutional Transformer subnetwork in the convolutional Transformer network to obtain a second sequence feature;

[0012] The second sequence feature is input into a second convolutional Transformer subnetwork connected to the first convolutional Transformer subnetwork to obtain a third sequence feature, until the first sequence feature is obtained after passing through the last convolutional Transformer subnetwork in the convolutional Transformer network.

[0013] In an optional embodiment of the first aspect of the present application, the linear projection layer includes a first convolutional layer, and the convolutional projection layer includes the first convolutional layer and a second convolutional layer; the first convolutional layer is a 1×1 convolutional layer, and the second convolutional layer is a 1×n depth convolutional layer, where n is a positive integer greater than or equal to 3.

[0014] In an optional embodiment of the first aspect of the present application, each convolutional Transformer sub-network further includes a multi-head attention mechanism layer, a normalization layer, and a multi-layer perceptron (MLP) layer; inputting the first feature into the first convolutional Transformer sub-network in the convolutional Transformer network to obtain a second sequence feature includes:

[0015] Input the first features into the linear projection layer and the convolutional projection layer in the first convolutional Transformer subnetwork respectively, and obtain a first set of QKV vectors and a second set of QKV vectors after linear transformation;

[0016] After concatenating the first set of QKV vectors and the second set of QKV vectors, the concatenation is input into the multi-head attention mechanism layer to obtain the second feature;

[0017] A third feature is generated according to the first feature and the second feature, and the second sequence feature is obtained after the third feature is processed by the normalization layer and the MLP layer.

[0018] In an optional embodiment of the first aspect of the present application, each convolutional Transformer sub-network further includes a third convolutional layer, wherein the third convolutional layer is a 1×3 depthwise separable convolutional layer; and the method further includes:

[0019] Inputting the first feature into the third convolutional layer in the first convolutional Transformer subnetwork to obtain a fourth feature;

[0020] Generating a third feature according to the first feature and the second feature includes:

[0021] The first feature, the second feature, and the fourth feature are weighted and summed to generate the third feature.

[0022] In an optional embodiment of the first aspect of the present application, the text recognition method further includes: inputting the first feature into a feature rearrangement module, sampling and rearranging the first feature to obtain a rearranged first feature;

[0023] Correspondingly, extracting a first sequence feature containing global information and local information from the first feature through the convolutional Transformer network includes: extracting a first sequence feature containing global information and local information from the rearranged first feature through the convolutional Transformer network.

[0024] In an optional embodiment of the first aspect of the present application, the decoding module includes a CTC decoding module; and obtaining text information in the image to be recognized from the first sequence features by the decoding module includes:

[0025] The CTC decoding module is used to obtain text information in the image to be recognized from the first sequence features.

[0026] A second aspect of the present invention provides a text recognition method, the method comprising:

[0027] Receiving an image to be recognized from an electronic device, wherein the image to be recognized includes at least one line of text;

[0028] Acquire a first feature of the image to be identified through a feature extraction network, where the first feature is a visual feature;

[0029] Extracting a first sequence feature containing global information and local information from the first feature through a convolutional Transformer network; the convolutional Transformer network includes N convolutional Transformer subnetworks connected in sequence, each convolutional Transformer subnetwork includes a linear projection layer and a convolutional projection layer, the linear projection layer is used to extract the global information, and the convolutional projection layer is used to extract the local information, where N is a positive integer greater than or equal to 2;

[0030] Acquire text information in the image to be recognized from the first sequence features through a decoding module;

[0031] The text message is sent to the electronic device.

[0032] A third aspect of the embodiments of the present application provides a text recognition device, including:

[0033] An acquisition module, configured to acquire an image to be recognized, wherein the image to be recognized includes at least one line of text;

[0034] A processing module is used to obtain a first feature of the image to be identified through a feature extraction network, where the first feature is a visual feature; extract a first sequence feature containing global information and local information from the first feature through a convolutional Transformer network; and obtain text information in the image to be identified from the first sequence feature through a decoding module; wherein the convolutional Transformer network includes N convolutional Transformer subnetworks connected in sequence, each convolutional Transformer subnetwork includes a linear projection layer and a convolutional projection layer, the linear projection layer is used to extract the global information, and the convolutional projection layer is used to extract the local information, and N is a positive integer greater than or equal to 2.

[0035] In an optional embodiment of the third aspect of the present application, the processing module is configured to input the first feature into a first convolutional Transformer subnetwork in the convolutional Transformer network to obtain a second sequence feature;

[0036] The second sequence feature is input into a second convolutional Transformer subnetwork connected to the first convolutional Transformer subnetwork to obtain a third sequence feature, until the first sequence feature is obtained after passing through the last convolutional Transformer subnetwork in the convolutional Transformer network.

[0037] In an optional embodiment of the third aspect of the present application, the linear projection layer includes a first convolutional layer, and the convolutional projection layer includes the first convolutional layer and a second convolutional layer; the first convolutional layer is a 1×1 convolutional layer, and the second convolutional layer is a 1×n depth convolutional layer, where n is a positive integer greater than or equal to 3.

[0038] In an optional embodiment of the third aspect of the present application, each convolutional Transformer sub-network further includes a multi-head attention mechanism layer, a normalization layer, and a multi-layer perceptron MLP layer; the processing module is used to:

[0039] Input the first features into the linear projection layer and the convolutional projection layer in the first convolutional Transformer subnetwork respectively, and obtain a first set of QKV vectors and a second set of QKV vectors after linear transformation;

[0040] The first group of QKV vectors and the second group of QKV vectors are spliced, and a second feature is obtained by inputting the spliced vectors into a multi-head attention mechanism layer;

[0041] A third feature is generated according to the first feature and the second feature, and the third feature is processed by the normalization layer and the MLP layer to obtain the second sequence feature.

[0042] In an optional embodiment of the third aspect of the application, each convolutional Transformer subnetwork further includes a third convolutional layer, which is a 1*3 depth separable convolutional layer; and the processing module is configured to input the first feature into the third convolutional layer in the first convolutional Transformer subnetwork to obtain a fourth feature.

[0043] The first feature, the second feature, and the fourth feature are weighted and summed to generate the third feature.

[0044] In an optional embodiment of the third aspect of the application, the processing module is configured to input the first feature into a feature rearrangement module, sample and rearrange the first feature to obtain a rearranged first feature.

[0045] The convolutional Transformer network extracts a first sequence feature containing global information and local information from the rearranged first feature.

[0046] In an optional embodiment of the third aspect of the application, the decoding module includes a CTC decoding module; and the processing module is configured to acquire text information in the image to be recognized from the first sequence feature by the CTC decoding module.

[0047] A fourth aspect of the embodiments of the application provides a text recognition device, which includes:

[0048] A receiving module is configured to receive an image to be recognized from an electronic device, and the image to be recognized includes at least one line of text;

[0049] A processing module is configured to acquire a first feature of the image to be recognized by a feature extraction network, the first feature being a visual feature; extract a first sequence feature containing global information and local information from the first feature by a convolutional Transformer network; the convolutional Transformer network includes N convolutional Transformer subnetworks connected in sequence, each convolutional Transformer subnetwork includes a linear projection layer and a convolutional projection layer, the linear projection layer is configured to extract the global information, and the convolutional projection layer is configured to extract the local information, N being a positive integer greater than or equal to 2; and acquire text information in the image to be recognized from the first sequence feature by a decoding module.

[0050] The sending module is used to send the text information to the electronic device.

[0051] The fifth aspect of an embodiment of the present application provides an electronic device, comprising: a memory, a processor, and a computer program; the computer program is stored in the memory and is configured to be executed by the processor to implement the method as described in any one of the first aspects of the present application, or the method as described in the second aspect.

[0052] A sixth aspect of an embodiment of the present application provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program is executed by a processor to implement the method as described in any one of the first aspect of the present application or the method described in the second aspect.

[0053] A seventh aspect of the embodiments of the present application provides a computer program product, including a computer program, which, when executed by a processor, implements the method described in any one of the first aspect or the second aspect of the present application.

[0054] The embodiments of the present application provide a text recognition method, apparatus, device and storage medium, the method comprising: obtaining an image to be recognized including at least one line of text, first obtaining a first feature of the image to be recognized through a feature extraction network, the first feature being a visual feature. Then, through a convolutional Transformer network, a first sequence feature containing global information and local information of the text is further extracted from the first feature, and finally, through a decoding module, the text information in the image to be recognized is obtained from the first sequence feature, that is, the text recognition result. The convolutional Transformer network of this embodiment is different from an ordinary Transformer network, and comprises a plurality of convolutional Transformer subnetworks connected in sequence, each convolutional Transformer subnetwork comprising a linear projection layer and a convolutional projection layer, the linear projection layer being used to extract the global information of the image text, and the convolutional projection layer being used to extract the local information of the image text. By fusing the global and local information of the image text, the text recognition effect in complex scenes can be improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] Figure 1A A system diagram of a text recognition method provided in an embodiment of the present application;

[0056] Figure 1B Based on Figure 1A A flowchart of a text recognition method of the system shown;

[0057] Figure 2 A schematic diagram of the structure of an improved text recognition model provided in an embodiment of the present application;

[0058] Figure 3 A flowchart of a text recognition method provided in an embodiment of the present application;

[0059] Figure 4 A schematic diagram of the structure of a convolutional Transformer network provided in an embodiment of the present application;

[0060] Figure 5A A schematic diagram of the internal processing of a convolutional Transformer subnetwork provided in an embodiment of the present application;

[0061] Figure 5B A schematic diagram of the internal processing of a convolutional Transformer subnetwork provided in an embodiment of the present application;

[0062] Figure 6 A schematic diagram of the structure of an improved text recognition model provided in an embodiment of the present application;

[0063] Figure 7 A flowchart of a text recognition method provided in an embodiment of the present application;

[0064] Figure 8 A schematic diagram of the structure of a text recognition device provided in an embodiment of the present application;

[0065] Figure 9 A schematic diagram of the structure of a text recognition device provided in an embodiment of the present application;

[0066] Figure 10 A hardware structure diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0067] To make the purpose, technical solutions, and advantages of the embodiments of this application more clear, the technical solutions in the embodiments of this application will be clearly and completely described below in conjunction with the drawings in the embodiments of this application. Obviously, the described embodiments are part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0068] The following first briefly describes the professional terms involved in the embodiments of this application.

[0069] First, text recognition uses various pattern recognition algorithms to locate the text in the text image, extract the features of the text, and then match it with the standard characters in the recognition dictionary to achieve the purpose of identifying the content of the document.

[0070] Second, Transformer is a novel network architecture designed to solve sequence-to-sequence tasks. It is a conversion model that relies on a self-attention mechanism to compute input and output representations without using sequence-aligned recurrent neural networks or convolutional neural networks.

[0071] Third, CTC (Connectionist Temporal Classification), a method for calculating loss functions that does not require alignment, is used to train deep neural networks and can be applied to text line recognition and speech recognition.

[0072] Fourth, local information, namely local image features, refers to the local expression of image features, which reflects the local characteristics of the image.

[0073] Fifth, global information, or global image features, refers to features that can represent the entire image. Global image features are relative to local image features and are used to describe the overall characteristics of objects in an image, such as the contextual information of text in an image.

[0074] Sixth, Depthwise Separable Convolution is an algorithm obtained by improving the standard convolution calculation in convolutional neural networks. It reduces the number of parameters required for convolution calculation by splitting the correlation between spatial dimensions and channel (depth) dimensions, and can improve the efficiency of convolution kernel parameters.

[0075] Seventh, convolutional neural networks (CNN) are a type of feedforward neural network that includes convolution calculations and has a deep structure, including convolutional layers, pooling layers, and fully connected layers.

[0076] Eighth, long short-term memory (LSTM) is a special type of recurrent neural network (RNN). It is primarily implemented through three gating logics (forget, input, and output) to address the problems of vanishing and exploding gradients during long sequence training. Compared to ordinary RNNs, LSTMs perform better on longer sequences.

[0077] Ninth, the Residual Network (ResNet), the core of which is to establish "short-circuit connections" (shortcuts, skip connections) between the previous layer and the subsequent layer, which helps the back propagation of gradients during training, thereby training deeper CNN networks.

[0078] Tenth, the DenseNet (Densely connected convolutional networks) model, its basic idea is consistent with ResNet, but it establishes a dense connection between all the previous layers and the subsequent layers, that is, each layer receives all the previous layers as its additional input.

[0079] The following first briefly introduces the system architecture of the technical solution provided in the embodiment of the present application.

[0080] For example, Figure 1A This is a system diagram of the text recognition method provided in the embodiment of the present application. Figure 1A As shown, the system 100 of the text recognition method may include a server 101, a communication network 102, and one or more user devices 103. Figure 1A A plurality of user equipments are shown in FIG.

[0081] Server 101 can be any suitable server for storing information, data, programs, and / or any other suitable type of content. In some embodiments, server 101 can perform any suitable function. For example, in some embodiments, server 101 can be used for text recognition, including recognizing Chinese text, English text, mathematical formulas, etc. In some embodiments, server 101 can be used to send text recognition results to a user device.

[0082] The communication network 102 can be any suitable combination of one or more wired and / or wireless networks. For example, the communication network 102 can include any one or more of the following: the Internet, an intranet, a wide area network (WAN), a local area network (LAN), a wireless network, a digital subscriber line (DSL) network, a frame relay network, an asynchronous transfer mode (ATM) network, a virtual private network (VPN) and / or any other suitable communication network. The user device 103 can be connected to the communication network 102 by one or more communication links (e.g., communication link 104), and the communication network 102 can be linked to the server 101 via one or more communication links (e.g., communication link 105). The communication link can be any communication link suitable for transmitting data between the user device 103 and the server 101, such as a network link, a dial-up link, a wireless link, a hard-wired link, any other suitable communication link or any suitable combination of such links.

[0083] User device 103 may include any one or more user devices suitable for receiving and capturing image data. In some embodiments, user device 103 may include any suitable type of device. For example, in some embodiments, user device 103 may include a mobile device, a tablet computer, a laptop computer, a desktop computer, a wearable computer, a game console, a media player, and / or any other suitable type of user device, without limitation in this embodiment of the present application.

[0084] Although server 101 is illustrated as one device, in some embodiments, any suitable number of devices may be used to perform the functions performed by server 101. For example, in some embodiments, multiple devices may be used to implement the functions performed by server 101. Alternatively, the functions of server 101 may be implemented using cloud services.

[0085] For example, Figure 1B Based on Figure 1A Schematic diagram of the flow chart of the text recognition method of the system shown in FIG. Figure 1B As shown, the user device sends the image to be recognized to the server. The server extracts the text information in the image to be recognized based on the pre-trained text recognition model and obtains the text recognition result. The server sends the text recognition result to the user device.

[0086] In related technologies, there are mainly two types of text recognition solutions:

[0087] The first is a CTC-based text recognition model, which includes two subtypes: a CNN+CTC-based text recognition model and a CNN+LSTM+CTC-based text recognition model. The first model consumes fewer computing resources, but its text recognition performance is poor for complex scenarios, such as multi-line text recognition or text containing mathematical formulas. The second model, while introducing LSTM for semantic modeling based on the first model, can improve the model's text recognition performance in complex scenarios to a certain extent, the introduction of an autoregressive process makes it prone to gradient explosion and inability to parallelize computations during model training. The same issue also exists during model prediction.

[0088] The second is a text recognition model based on the attention mechanism. This model has the following major issues: First, the recognition of each character depends on the output at the previous time point, that is, serial operation, which makes the model operation time-consuming. Second, the recognition of Chinese text lines is prone to attention drift, resulting in a sharp decline in text recognition performance.

[0089] Currently, research on the aforementioned text recognition models primarily focuses on improving their recognition accuracy, often without considering their computational complexity. However, in practical applications, computational efficiency is a crucial metric for model application. Furthermore, these models are primarily designed for English text recognition and are not suitable for Chinese text recognition.

[0090] In this regard, an embodiment of the present application proposes a text recognition method and designs a text recognition solution based on dual modeling of local and global information of image text. It utilizes the CNN's ability to extract local information of image text and the Transformer's ability to extract global information of image text to construct an improved text recognition model.

[0091] For example, Figure 2 A structural diagram of an improved text recognition model provided in an embodiment of the present application is shown as follows: Figure 2 As shown in the figure, the improved text recognition model mainly consists of three parts: a feature extraction network, a convolutional Transformer network, and a decoding module (such as CTC decoding). The feature extraction network is used to extract the visual features of the input image, the convolutional Transformer network is used to perform context modeling on the visual features, and obtain sequence features that integrate the local and global information of the image and text. The CTC decoding module is used to decode the sequence features and output the text information of the image.

[0092] This improved text recognition model strengthens the integration of local and global information within image text, enhancing text recognition performance. Furthermore, decoding modules (such as CTC decoding) rely on dense predictions, and the original pixel arrangement cannot correspond to labels. By introducing a feature rearrangement module after the feature extraction network, the decoding module can recognize two-dimensional text, such as formulas, enhancing the text recognition model's ability to recognize text in complex scenarios.

[0093] The text recognition solution provided by this application is described in detail below through specific embodiments. It should be noted that the technical solutions provided by the embodiments of this application may include some or all of the following contents, and the following specific embodiments may be combined with each other. For the same or similar concepts or processes, they may not be repeated in some embodiments.

[0094] For example, Figure 3 This is a flowchart of a text recognition method provided in an embodiment of the present application. Figure 2 As shown, the text recognition method of this embodiment can be applied to the server shown in FIG1 , and the method includes:

[0095] Step 301: Acquire an image to be recognized, where the image to be recognized includes at least one line of text.

[0096] Step 302: Obtain a first feature of the image to be identified through a feature extraction network, where the first feature is a visual feature.

[0097] In some embodiments, the feature extraction network can adopt a convolutional backbone network (Convolutional Backbone) such as DenseNet, ResNet, Inception, MobileNet, etc. Compared with the patch embedding in ViT (i.e., VisionTransformer) (the process of obtaining patch embedding includes: image → block → mapping → feature), the convolution operation of the feature extraction network can better retain the detailed information of the image text, retain translation invariance, and is more in line with the fine-grained characteristics of text.

[0098] Optionally, in some embodiments, in order to reduce the computational complexity of the network while meeting a certain accuracy, a pruned ConvNeXt-Tiny may be used as a feature extraction network.

[0099] Step 303: Extract a first sequence feature containing global information and local information from the first feature through a convolutional Transformer network.

[0100] For example, Figure 4 A structural diagram of a convolutional Transformer network provided in an embodiment of the present application. As shown in Figure 4, the convolutional Transformer network includes N convolutional Transformer sub-networks connected in sequence. Where N is a positive integer greater than or equal to 2, for example, N is 6. It should be understood that the more convolutional Transformer sub-networks there are, the stronger their learning ability is. In some embodiments, N can be a positive integer greater than 6. Specifically, Figure 4 The input of the first convolutional Transformer subnetwork shown is used as the input of the convolutional Transformer network, and is processed in sequence by the second convolutional Transformer subnetwork until the Nth convolutional Transformer subnetwork, and finally the output of the Nth convolutional Transformer subnetwork is used as the output of the convolutional Transformer network.

[0101] It should be noted that, compared to a typical Transformer network, each convolutional Transformer subnetwork in the convolutional Transformer network provided in this embodiment includes both a linear projection layer and a convolutional projection layer. The linear projection layer is used to extract global information from the first feature, while the convolutional projection layer is used to extract local information from the first feature. The information extracted by the linear and convolutional projection layers is fused in subsequent layers to improve the model's recognition of text in images.

[0102] In an optional embodiment, a first sequence feature containing global information and local information is extracted from the first feature through a convolutional Transformer network, specifically including: inputting the first feature into the first convolutional Transformer subnetwork in the convolutional Transformer network to obtain a second sequence feature; inputting the second sequence feature into the second convolutional Transformer subnetwork connected to the first convolutional Transformer subnetwork to obtain a third sequence feature, until the first sequence feature is obtained after passing through the last convolutional Transformer subnetwork in the convolutional Transformer network.

[0103] In the above embodiment, the first feature is used as the input of the convolutional Transformer network. After data processing by N end-to-end connected convolutional Transformer sub-networks in the convolutional Transformer network, a first sequence feature containing local and global information of the image text is finally obtained.

[0104] It should be noted that the lengths of the first sequence feature, the second sequence feature, and the third sequence feature are the same as the first feature.

[0105] In an optional embodiment, in each convolutional Transformer subnetwork, the linear projection layer includes a first convolutional layer, which is a 1×1 convolutional layer. The convolutional projection layer includes a first convolutional layer and a second convolutional layer, which is a 1×n depthwise convolutional layer, where n is a positive integer greater than or equal to 3, for example, 5 or 7. It should be noted that the linear projection layer and the convolutional projection layer can share the first convolutional layer.

[0106] In an optional embodiment, in addition to the linear projection layer and the convolutional projection layer, each convolutional Transformer subnetwork also includes a multi-head attention mechanism layer, a normalization layer, and a multi-layer perceptron MLP layer.

[0107] As an example, the first feature is input into the first convolutional Transformer subnetwork in the convolutional Transformer network to obtain the second sequence feature, specifically including:

[0108] The first feature is respectively input into the linear projection layer and the convolutional projection layer in the first convolutional Transformer subnetwork, and the first set of QKV vectors and the second set of QKV vectors are obtained after linear transformation; the first set of QKV vectors and the second set of QKV vectors are concatenated and input into the multi-head attention mechanism layer to obtain the second feature; the third feature is generated based on the first and second features, and the third feature is processed by the normalization layer and the MLP layer to obtain the second sequence feature.

[0109] In order to improve the recognition effect of text recognition models in complex scenes, it is usually necessary to set up multiple convolutional Transformer sub-networks to extract more semantic information of image text. The following takes the first convolutional Transformer sub-network in the convolutional Transformer network as an example to explain its internal processing in detail.

[0110] For example, Figure 5A This is a schematic diagram of the internal processing of a convolutional Transformer sub-network provided in an embodiment of the present application. After the feature extraction network extracts the first feature of the input image, such as Figure 5A As shown, the first feature can be input into the first convolutional Transformer subnetwork of the convolutional Transformer network. The first feature is processed by the first convolutional layer (i.e., linear projection layer) to obtain the first set of QKV vectors, for example Figure 5A In addition, the first feature is processed by the first convolution layer and the second convolution layer (the first convolution layer and the second convolution layer can be regarded as convolution projection layers) to obtain the second set of QKV vectors, for example Figure 5A Before entering the multi-head attention mechanism layer, the two sets of QKV vectors are concatenated: Q1 and Q2, K1 and K2, and V1 and V2. The concatenated QKV vectors can be recorded as Q3, K3, and V3. After processing the concatenated QKV vectors by the multi-head attention mechanism layer, they are summed with the first feature and then passed through the normalization layer and the MLP layer to output the second sequence feature.

[0111] It should be noted that the internal structure and processing flow of other convolutional Transformer subnetworks in the convolutional Transformer network can refer to the first convolutional Transformer subnetwork.

[0112] In an optional embodiment, in each convolutional Transformer subnetwork, in addition to the linear projection layer, the convolutional projection layer, the multi-head attention mechanism layer, the normalization layer, and the multi-layer perceptron MLP layer, it also includes a third convolution layer, which is a 1×3 depth-separable convolution layer.

[0113] As an example, the first feature is input into the first convolutional Transformer subnetwork in the convolutional Transformer network to obtain the second sequence feature, specifically including:

[0114] The first feature is input into the linear projection layer and the convolutional projection layer in the first convolutional Transformer subnetwork respectively, and the first set of QKV vectors and the second set of QKV vectors are obtained after linear transformation; the first set of QKV vectors and the second set of QKV vectors are concatenated and input into the multi-head attention mechanism layer to obtain the second feature.

[0115] At the same time, the first feature is fed into the third convolutional layer of the first convolutional Transformer subnetwork to generate the fourth feature. The third feature is then generated by weighted summation of the first, second, and fourth features. The third feature is processed by the normalization layer and the MLP layer to generate the second sequence feature.

[0116] For example, Figure 5B A schematic diagram of the internal processing of a convolutional Transformer sub-network provided in an embodiment of the present application. Figure 5A Based on the processing flow shown, Figure 5B As shown in the figure, adding a 1×3 depth-wise separable convolutional layer, i.e., the third convolutional layer, can perform weighted summation of features by learning the weight α, further strengthening the fusion of local semantic information of image text and improving the text recognition effect.

[0117] Step 304: Obtain text information in the image to be recognized from the first sequence features through a decoding module.

[0118] In one optional embodiment, the decoding module includes a CTC decoding module; obtaining text information in the image to be recognized from the first sequence of features via the decoding module specifically includes: obtaining text information in the image to be recognized from the first sequence of features via the CTC decoding module. Using the CTC decoding module reduces the learning difficulty of the Transformer, eliminating the need to learn how to perform label alignment. It also ensures the continuity of visual feature alignment, avoiding information crosstalk within the Transformer.

[0119] The embodiment of the present application shows a text recognition method, which obtains an image to be recognized including at least one line of text, and first obtains the first feature of the image to be recognized through a feature extraction network, where the first feature is a visual feature. Then, a first sequence feature containing global information and local information of the text is further extracted from the first feature through a convolutional Transformer network, and finally, the text information in the image to be recognized is obtained from the first sequence feature through a decoding module, that is, the text recognition result. The convolutional Transformer network of this embodiment is different from an ordinary Transformer network, and includes a plurality of convolutional Transformer subnetworks connected in sequence, each convolutional Transformer subnetwork including a linear projection layer and a convolutional projection layer, the linear projection layer is used to extract the global information of the image text, and the convolutional projection layer is used to extract the local information of the image text. By fusing the global and local information of the image text, the text recognition effect in complex scenes can be improved.

[0120] For example, Figure 6 This is a schematic diagram of the structure of an improved text recognition model provided in the embodiment of the present application. Figure 6 As shown in FIG, the improved text recognition model includes a feature extraction network, a feature rearrangement module, a convolutional Transformer network and a decoding module. Figure 2 The difference between the text recognition model shown is that a feature rearrangement module is added between the feature extraction network and the convolutional Transformer network. The feature rearrangement module is used to sample and rearrange the first feature. For example, the feature rearrangement module can use the grid sample function to sample and rearrange the first feature. In the embodiment of the present application, the feature rearrangement module is mainly aimed at scenes where the image includes formula text or multiple lines of text. For example, for the recognition of mathematical formulas, the feature rearrangement module will adjust the image features instead of extracting feature information in a preset order from top to bottom and from left to right.

[0121] As an example, the feature rearrangement module can be constructed using a multi-layer CNN network.

[0122] It should be noted that during the training of the improved text recognition model, the feature reordering module does not require label supervision and relies on the final recognition loss for gradient inversion. Specifically, when the features generated by the feature reordering module do not match the correct text order, the CTC loss increases; conversely, when the features match the correct text order, the CTC loss decreases. The feature reordering module is trained by changing the CTC loss.

[0123] based on Figure 6 The text recognition model shown in the embodiment of the present application also provides a text recognition method. For example, Figure 7This is a flowchart of a text recognition method provided in an embodiment of the present application. Figure 7 As shown, the text recognition method of this embodiment can be applied to the server shown in FIG1 , and the method includes:

[0124] Step 701: Acquire an image to be recognized, where the image to be recognized includes at least one line of text.

[0125] Step 702: Obtain a first feature of the image to be identified through a feature extraction network, where the first feature is a visual feature.

[0126] Step 703: Input the first feature into the feature rearrangement module, perform sampling rearrangement on the first feature, and obtain the rearranged first feature.

[0127] Step 704: Extract a first sequence feature containing global information and local information from the rearranged first feature through a convolutional Transformer network.

[0128] Step 705: Obtain text information in the image to be recognized from the first sequence features through a decoding module.

[0129] In an embodiment of the present application, at least one line of text in the image to be recognized includes two-dimensional text such as mathematical formulas. Conventional top-to-bottom or left-to-right sequential recognition may result in errors in the recognition of two-dimensional text. Therefore, a feature reordering module is provided in the text recognition model. Before the visual features are input into the convolutional Transformer network, the feature reordering module randomly adjusts the order of the visual features, thereby improving the recognition capability of the decoding module (such as the CTC decoding module) for two-dimensional text.

[0130] The present application also provides a text recognition method, including:

[0131] Step 1: Receive an image to be recognized from an electronic device, where the image to be recognized includes at least one line of text.

[0132] Step 2: Obtain the first feature of the image to be identified through the feature extraction network, where the first feature is a visual feature.

[0133] Step 3: Extract the first sequence feature containing global information and local information from the first feature through the convolutional Transformer network.

[0134] Among them, the convolutional Transformer network includes N convolutional Transformer sub-networks connected in sequence. Each convolutional Transformer sub-network includes a linear projection layer and a convolutional projection layer. The linear projection layer is used to extract global information, and the convolutional projection layer is used to extract local information. N is a positive integer greater than or equal to 2.

[0135] Step 4, obtaining text information in the to-be-recognized image from the first sequence feature through a decoding module.

[0136] Step 5, sending the text information to an electronic device.

[0137] The steps 3 to 4 can refer to the above embodiment.

[0138] The text recognition method provided by the embodiment of the present application is described above, and the text recognition device provided by the embodiment of the present application will be described below.

[0139] The embodiment of the present application can divide the text recognition device into functional modules according to the above method embodiment. For example, each functional module can be divided according to each function, or two or more functions can be integrated into one processing module. The integrated module can be implemented in the form of hardware or in the form of a software functional module. It should be noted that the division of the modules in the embodiment of the present application is illustrative, and is only a logical functional division. In actual implementation, another division mode can be used. The following will be described taking the division of each functional module corresponding to each function as an example.

[0140] Figure 8 A structural schematic diagram of a text recognition device provided by the embodiment of the present application is shown in FIG. 8. As shown in FIG. 8, the text recognition device 800 of the embodiment includes an acquisition module 801 and a processing module 802. Figure 8

[0141] The acquisition module 801 is configured to acquire a to-be-recognized image, wherein the to-be-recognized image includes at least one line of text.

[0142] The processing module 802 is configured to acquire a first feature of the to-be-recognized image through a feature extraction network, wherein the first feature is a visual feature; extract a first sequence feature including global information and local information from the first feature through a convolutional Transformer network; obtain text information in the to-be-recognized image from the first sequence feature through a decoding module; and wherein the convolutional Transformer network includes N convolutional Transformer sub-networks connected in sequence, each convolutional Transformer sub-network includes a linear projection layer and a convolutional projection layer, the linear projection layer is configured to extract the global information, the convolutional projection layer is configured to extract the local information, and N is a positive integer greater than or equal to 2.

[0143] In an optional embodiment of the present embodiment, the processing module 802 is configured to input the first feature into a first convolutional Transformer sub-network in the convolutional Transformer network to obtain a second sequence feature. ​

[0144] The second sequence feature is input into a second convolutional Transformer subnetwork connected to the first convolutional Transformer subnetwork to obtain a third sequence feature, until the first sequence feature is obtained after passing through the last convolutional Transformer subnetwork in the convolutional Transformer network.

[0145] In an optional embodiment of this embodiment, the linear projection layer includes a first convolutional layer, and the convolutional projection layer includes the first convolutional layer and a second convolutional layer; the first convolutional layer is a 1×1 convolutional layer, and the second convolutional layer is a 1×n depth convolutional layer, where n is a positive integer greater than or equal to 3.

[0146] In an optional embodiment of this embodiment, each convolutional Transformer sub-network further includes a multi-head attention mechanism layer, a normalization layer, and a multi-layer perceptron MLP layer; the processing module 802 is used to:

[0147] Input the first features into the linear projection layer and the convolutional projection layer in the first convolutional Transformer subnetwork respectively, and obtain a first set of QKV vectors and a second set of QKV vectors after linear transformation;

[0148] After concatenating the first set of QKV vectors and the second set of QKV vectors, the concatenation is input into the multi-head attention mechanism layer to obtain the second feature;

[0149] A third feature is generated according to the first feature and the second feature, and the second sequence feature is obtained after the third feature is processed by the normalization layer and the MLP layer.

[0150] In an optional embodiment of this embodiment, each convolutional Transformer sub-network further includes a third convolutional layer, where the third convolutional layer is a 1×3 depthwise separable convolutional layer; the processing module 802 is configured to input the first feature into the third convolutional layer in the first convolutional Transformer sub-network to obtain a fourth feature;

[0151] The first feature, the second feature, and the fourth feature are weighted and summed to generate the third feature.

[0152] In an optional embodiment of this embodiment, the processing module 802 is configured to input the first feature into a feature rearrangement module, perform sampling rearrangement on the first feature, and obtain a rearranged first feature;

[0153] A first sequence feature containing global information and local information is extracted from the rearranged first feature through the convolutional Transformer network.

[0154] In an optional embodiment of this embodiment, the decoding module includes a CTC decoding module; the processing module 802 is used to obtain text information in the image to be recognized from the first sequence features through the CTC decoding module.

[0155] The text recognition device provided in this embodiment obtains an image to be recognized including at least one line of text through an acquisition module. The processing module first obtains the first feature of the image to be recognized through a feature extraction network, and the first feature is a visual feature; then, through a convolutional Transformer network, a first sequence feature containing global information and local information is extracted from the first feature; finally, the text information in the image to be recognized is obtained from the first sequence feature through a decoding module. The convolutional Transformer network includes a plurality of convolutional Transformer sub-networks connected in sequence, and each convolutional Transformer sub-network includes a linear projection layer and a convolutional projection layer. The linear projection layer is used to extract global information, and the convolutional projection layer is used to extract local information. By fusing the global and local information of the image text, the text recognition capability of the text recognition device in complex scenes can be improved.

[0156] Figure 9 This is a structural diagram of a text recognition device provided in an embodiment of the present application. Figure 9 As shown, the text recognition device 900 of this embodiment includes: a receiving module 901 , a processing module 902 and a sending module 903 .

[0157] The receiving module 901 is configured to receive an image to be recognized from an electronic device, wherein the image to be recognized includes at least one line of text;

[0158] Processing module 902 is configured to obtain a first feature of the image to be identified through a feature extraction network, where the first feature is a visual feature; extract a first sequence feature containing global information and local information from the first feature through a convolutional Transformer network; the convolutional Transformer network includes N convolutional Transformer subnetworks connected in sequence, each convolutional Transformer subnetwork includes a linear projection layer and a convolutional projection layer, the linear projection layer is used to extract the global information, and the convolutional projection layer is used to extract the local information, where N is a positive integer greater than or equal to 2; and obtain text information in the image to be identified from the first sequence feature through a decoding module;

[0159] The sending module 903 is configured to send the text message to the electronic device.

[0160] The text recognition device provided in this embodiment can execute the technical solution of the aforementioned method embodiment. Its implementation principle and technical effects are similar and will not be described in detail here.

[0161] It should be noted that in some of the processes described in the above embodiments and the accompanying drawings, multiple operations that appear in a specific order are included, but it should be clearly understood that these operations may not be executed in the order in which they appear in this article or in parallel. They are only used to distinguish between different operations, and the sequence number itself does not represent any execution order. In addition, these processes may include more or fewer operations, and these operations may be executed in sequence or in parallel. It should be noted that the descriptions of "first", "second", etc. in this article are used to distinguish different messages, devices, modules, etc., and do not represent the order of precedence, nor do they limit "first" and "second" to be different types.

[0162] Figure 10 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. Figure 10 As shown, the electronic device includes a memory 1001 and a processor 1002. Memory 1001 is used to store computer programs and can be configured to store various other data to support operations on a text recognition device or server. Processor 1002, coupled to memory 1001, is used to execute the computer program in memory 1001 to implement the technical solutions of the aforementioned method embodiments. The implementation principles and technical effects are similar and will not be further described here.

[0163] Optional, such as Figure 10 As shown, the electronic device also includes: a firewall 1003, a load balancer 1004, a communication component 1005, a power supply component 1006 and other components. Figure 10 Only some components are shown schematically, which does not mean that the electronic device only includes Figure 10 Components shown.

[0164] An embodiment of the present application provides a computer-readable storage medium storing a computer program. When the computer program / instructions are executed by a processor, the processor implements the technical solution of the aforementioned method embodiment.

[0165] An embodiment of the present application provides a computer program product, including a computer program / instruction. When the computer program / instruction is executed by a processor, the processor implements the technical solution of the cloud device in the aforementioned method embodiment.

[0166] The present application provides a chip comprising: a processing module and a communication interface. The processing module is capable of executing the technical solution of the cloud device in the aforementioned method embodiment. Optionally, the chip further comprises a storage module (e.g., a memory) configured to store instructions, and the processing module configured to execute the instructions stored in the storage module. Execution of the instructions stored in the storage module causes the processing module to execute the technical solution of the cloud device in the aforementioned method embodiment.

[0167] The above storage may be an object storage service (OSS).

[0168] The above-mentioned memory can be implemented by any type of volatile or non-volatile memory device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk.

[0169] The above-mentioned communication component is configured to facilitate wired or wireless communication between the device where the communication component is located and other devices. The device where the communication component is located can access a wireless network based on a communication standard, such as WiFi, 2G, 3G, 4G / LTE, 5G and other mobile communication networks, or a combination thereof. In an exemplary embodiment, the communication component receives a broadcast signal or broadcast-related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component also includes a near field communication (NFC) module to facilitate short-range communication. For example, the NFC module can be implemented based on radio frequency identification (RFID) technology, infrared data association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology and other technologies.

[0170] The power supply assembly provides power to various components of the device in which the power supply assembly is located. The power supply assembly may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to the device in which the power supply assembly is located.

[0171] It will be understood by those skilled in the art that embodiments of the present invention may be provided as methods, systems, or computer program products. Thus, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware. Furthermore, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0172] The present invention is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device produce a device for implementing the functions specified in one or more processes in the flowchart and / or one or more boxes in the block diagram.

[0173] These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce a product including an instruction device that implements the functions specified in one or more processes in the flowchart and / or one or more boxes in the block diagram.

[0174] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, so that the instructions executed on the computer or other programmable device provide steps for implementing the functions specified in one or more processes in the flowchart and / or one or more boxes in the block diagram.

[0175] In a typical configuration, a computing device includes one or more processors (CPUs), input / output interfaces, network interfaces, and memory.

[0176] Memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. Memory is an example of a computer-readable medium.

[0177] Computer-readable media includes permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. The information can be computer-readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer-readable media does not include transitory computer-readable media (transitory media), such as modulated data signals and carrier waves.

[0178] It should also be noted that the terms "comprises," "includes," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, commodity, or apparatus that includes a series of elements includes not only those elements but also other elements not explicitly listed, or includes elements inherent to such process, method, commodity, or apparatus. In the absence of further limitations, an element defined by the phrase "comprises a ..." does not exclude the presence of other identical elements in the process, method, commodity, or apparatus that includes the element.

[0179] The above are merely embodiments of the present application and are not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present application should all be included within the scope of the claims of the present application.

Claims

1. A text recognition method, characterized in that: include: Acquire an image to be recognized, wherein the image to be recognized includes at least one line of text; Acquire a first feature of the image to be identified through a feature extraction network, where the first feature is a visual feature; Extracting a first sequence feature containing global information and local information from the first feature through a convolutional Transformer network; the convolutional Transformer network includes N convolutional Transformer subnetworks connected in sequence, each convolutional Transformer subnetwork includes a linear projection layer and a convolutional projection layer, the linear projection layer is used to extract the global information, and the convolutional projection layer is used to extract the local information, where N is a positive integer greater than or equal to 2; The text information in the image to be identified is obtained from the first sequence features through a decoding module.

2. The method according to claim 1, characterized in that Extracting a first sequence feature containing global information and local information from the first feature through the convolutional Transformer network includes: Inputting the first feature into the first convolutional Transformer subnetwork in the convolutional Transformer network to obtain a second sequence feature; The second sequence feature is input into a second convolutional Transformer subnetwork connected to the first convolutional Transformer subnetwork to obtain a third sequence feature, until the first sequence feature is obtained after passing through the last convolutional Transformer subnetwork in the convolutional Transformer network.

3. The method according to claim 1, characterized in that The linear projection layer includes a first convolutional layer, and the convolutional projection layer includes the first convolutional layer and a second convolutional layer; the first convolutional layer is a 1×1 convolutional layer, and the second convolutional layer is a 1×n depth convolutional layer, where n is a positive integer greater than or equal to 3.

4. The method according to claim 2, characterized in that Each convolutional Transformer subnetwork further includes a multi-head attention mechanism layer, a normalization layer, and a multi-layer perceptron (MLP) layer; the first feature is input into the first convolutional Transformer subnetwork in the convolutional Transformer network to obtain a second sequence feature, including: Input the first features into the linear projection layer and the convolutional projection layer in the first convolutional Transformer subnetwork respectively, and obtain a first set of QKV vectors and a second set of QKV vectors after linear transformation; After concatenating the first set of QKV vectors and the second set of QKV vectors, the concatenation is input into the multi-head attention mechanism layer to obtain the second feature; A third feature is generated according to the first feature and the second feature, and the second sequence feature is obtained after the third feature is processed by the normalization layer and the MLP layer.

5. The method according to claim 4, characterized in that Each convolutional Transformer subnetwork further includes a third convolutional layer, where the third convolutional layer is a 1×3 depthwise separable convolutional layer; and the method further includes: Inputting the first feature into the third convolutional layer in the first convolutional Transformer subnetwork to obtain a fourth feature; Generating a third feature according to the first feature and the second feature includes: The first feature, the second feature, and the fourth feature are weighted and summed to generate the third feature.

6. The method according to any one of claims 1 to 5, characterized in that The method further includes: inputting the first feature into a feature rearrangement module, sampling and rearranging the first feature to obtain a rearranged first feature; Correspondingly, extracting a first sequence feature containing global information and local information from the first feature through the convolutional Transformer network includes: extracting a first sequence feature containing global information and local information from the rearranged first feature through the convolutional Transformer network.

7. The method according to claim 6, characterized in that The decoding module includes a CTC decoding module; obtaining text information in the image to be recognized from the first sequence features through the decoding module includes: The CTC decoding module is used to obtain text information in the image to be recognized from the first sequence features.

8. A text recognition method, characterized in that: include: Receiving an image to be recognized from an electronic device, wherein the image to be recognized includes at least one line of text; Acquire a first feature of the image to be identified through a feature extraction network, where the first feature is a visual feature; Extracting a first sequence feature containing global information and local information from the first feature through a convolutional Transformer network; the convolutional Transformer network includes N convolutional Transformer subnetworks connected in sequence, each convolutional Transformer subnetwork includes a linear projection layer and a convolutional projection layer, the linear projection layer is used to extract the global information, and the convolutional projection layer is used to extract the local information, where N is a positive integer greater than or equal to 2; Acquire text information in the image to be recognized from the first sequence features through a decoding module; The text message is sent to the electronic device.

9. A text recognition device, characterized in that: include: An acquisition module, configured to acquire an image to be recognized, wherein the image to be recognized includes at least one line of text; A processing module is used to obtain a first feature of the image to be identified through a feature extraction network, where the first feature is a visual feature; extract a first sequence feature containing global information and local information from the first feature through a convolutional Transformer network; and obtain text information in the image to be identified from the first sequence feature through a decoding module; wherein the convolutional Transformer network includes N convolutional Transformer subnetworks connected in sequence, each convolutional Transformer subnetwork includes a linear projection layer and a convolutional projection layer, the linear projection layer is used to extract the global information, and the convolutional projection layer is used to extract the local information, and N is a positive integer greater than or equal to 2.

10. A text recognition device, characterized in that: include: A receiving module, configured to receive an image to be recognized from an electronic device, wherein the image to be recognized includes at least one line of text; A processing module is configured to obtain, through a feature extraction network, a first feature of the image to be identified, where the first feature is a visual feature; extract, through a convolutional Transformer network, a first sequence feature containing global information and local information from the first feature; the convolutional Transformer network includes N convolutional Transformer subnetworks connected in sequence, each convolutional Transformer subnetwork includes a linear projection layer and a convolutional projection layer, the linear projection layer is configured to extract the global information, and the convolutional projection layer is configured to extract the local information, where N is a positive integer greater than or equal to 2; and obtain, through a decoding module, text information in the image to be identified from the first sequence feature; The sending module is used to send the text information to the electronic device.

11. An electronic device, characterized in that: include: A memory, a processor, and a computer program; the computer program is stored in the memory and configured to be executed by the processor to implement the method according to any one of claims 1 to 8.

12. A computer-readable storage medium, characterized in that A computer program is stored thereon, and the computer program is executed by a processor to implement the method according to any one of claims 1 to 8.

13. A computer program product, characterized in that The invention comprises a computer program, which implements the method according to any one of claims 1 to 8 when the computer program is executed by a processor.

Images