Text recognition methods and devices

Abstract

This application provides a text recognition method and apparatus. The method includes: obtaining a text image; extracting features from the text image to obtain shallow text features; performing shift convolution processing on the shallow text features to obtain first intermediate text features; performing multiple sets of self-attention calculations with different window sizes on the first intermediate text features using a group-by-group multi-window self-attention module, fusing the calculated multiple sets of self-attention features to obtain second intermediate text features; performing residual connection between the shallow text features and the second intermediate text features to obtain deep text features; and recognizing text in the text image based on the shallow and deep text features. The solution of this application can improve the accuracy of text image recognition.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and in particular to a text recognition method and apparatus. Background Technology

[0002] With the continuous development of image processing technology, the application scenarios for text recognition in images are increasing. For example, text recognition technology can be applied to many fields such as license plate recognition, street sign recognition, document retrieval, parts identification in industrial automation, and navigation matching in autonomous driving.

[0003] However, images in natural scenes generally have low resolution, and the text contained in the images usually varies in terms of layout, font, and size, which can easily lead to text recognition errors and result in low accuracy. Summary of the Invention

[0004] In view of this, this application provides a text recognition method and apparatus to improve the accuracy of text recognition.

[0005] To achieve the above objectives, this application provides a text recognition method, comprising:

[0006] Obtain a text image;

[0007] Feature extraction is performed on the text image to obtain shallow text features of the text image;

[0008] The shallow text features are subjected to shift convolution processing to obtain the first intermediate text features of the text image;

[0009] The first intermediate text feature is subjected to multiple self-attention calculations with different window sizes using a multi-window self-attention module. The calculated multi-window self-attention features are then fused to obtain the second intermediate text feature of the text image.

[0010] The shallow text features are residually connected with the second intermediate text features to obtain the deep text features of the text image;

[0011] Based on the shallow and deep text features, the text in the text image is identified.

[0012] In one possible implementation, identifying the text in the text image based on the shallow text features and deep text features includes:

[0013] Based on the shallow and deep text features, a super-resolution text image is constructed from the text image.

[0014] Identify the text in the super-resolution text image.

[0015] In another possible implementation, constructing the super-resolution text image based on the shallow and deep text features includes:

[0016] The shallow text features and deep text features are fused to obtain the fused text features of the text image;

[0017] The fused text features are reconstructed pixel by pixel to obtain the reconstructed text image;

[0018] The reconstructed text image is upsampled using a pixel-level attention module to obtain a super-resolution text image.

[0019] In another possible implementation, the step of performing shift convolution processing on the shallow text features to obtain the first intermediate text features of the text image includes:

[0020] The shallow text features are subjected to a first shift convolution process to obtain the first text features of the text image.

[0021] The first text feature is activated using the first activation function;

[0022] The activated first text features are subjected to a second shift convolution process to obtain the second text features of the text image.

[0023] Based on the second text feature, the first intermediate text feature of the text image is determined.

[0024] In another possible implementation, determining the first intermediate text feature of the text image based on the second text feature includes:

[0025] The second text feature is residually connected with the shallow text feature to obtain the first intermediate text feature of the text image.

[0026] In another possible implementation, before extracting the shallow features of the text image, the method further includes:

[0027] Perform text correction on the text image;

[0028] The second activation function is used to activate the text image after text correction.

[0029] In another possible implementation, the pixel-level attention module includes: a relationship matrix R of transformed pixels as follows:

[0030]

[0031] Where softmax represents the normalization function;

[0032] θ() and φ() represent different fully connected layer transformations, respectively;

[0033] f i l The image f that needs to be upsampled l The features of any pixel i in the image f, i∈n, n belongs to the image f. l The set of pixels;

[0034] Represents image f l Features of pixel j, j∈N i N i Represents image f l The set of k nearest neighbor pixels of pixel i;

[0035] C is the image f l The number of channels.

[0036] In another aspect, this application also provides a text recognition device, comprising:

[0037] Image acquisition unit, used to acquire text images;

[0038] The feature extraction unit is used to extract features from the text image to obtain shallow text features of the text image;

[0039] A shift convolution unit is used to perform shift convolution processing on the shallow text features to obtain the first intermediate text features of the text image;

[0040] The attention calculation unit is used to perform multiple sets of self-attention calculations on the first intermediate text features using a multi-window self-attention module, and to fuse the calculated sets of self-attention features to obtain the second intermediate text features of the text image.

[0041] The feature connection unit is used to perform a residual connection between the shallow text features and the second intermediate text features to obtain the deep text features of the text image;

[0042] A text recognition unit is used to recognize text in the text image based on the shallow text features and deep text features.

[0043] In one possible implementation, the text recognition unit includes:

[0044] An image construction subunit is used to construct a super-resolution text image of the text image based on the shallow text features and deep text features;

[0045] A text recognition subunit is used to recognize text in the super-resolution text image.

[0046] In yet another possible implementation, the image construction subunit includes:

[0047] The feature fusion subunit is used to fuse the shallow text features and deep text features to obtain the fused text features of the text image.

[0048] A pixel recombination unit is used to reconstruct the fused text features into a pixel-based image to obtain a reconstructed text image.

[0049] A pixel-level processing unit is used to upsample the reconstructed text image using a pixel-level attention module to obtain a super-resolution text image of the text image.

[0050] As described above, after extracting the shallow text features of the text image, this application performs shift convolution processing on the shallow text features to achieve feature fusion between different channels. Furthermore, this application utilizes a multi-window self-attention module to perform multiple sets of self-attention calculations with different window sizes on the shift convolution processed text features, and fuses the calculated sets of self-attention features. This allows the fused second intermediate text features to extract text features from a deeper and more global perspective. Therefore, residual connection between the shallow text features and the second intermediate text features can reduce the impact of gradient vanishing on the accuracy and comprehensiveness of the text features, ultimately resulting in deeper text features that more accurately reflect global features. Thus, by using both shallow and deep text features, the local details and global text features of the text image can be more accurately and comprehensively represented, naturally improving the accuracy of text recognition. Attached Figure Description

[0051] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only embodiments of this application. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0052] Figure 1 This paper illustrates a flowchart of a text recognition method provided in an embodiment of this application.

[0053] Figure 2 This illustration shows a schematic diagram of the implementation principle framework of the text recognition method provided in an embodiment of this application;

[0054] Figure 3This illustration shows another flowchart of the text recognition method provided in an embodiment of this application;

[0055] Figure 4 A schematic diagram of the composition structure of a text recognition device provided in an embodiment of this application is shown. Detailed Implementation

[0056] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this application.

[0057] like Figure 1 As shown, it illustrates a flowchart of an embodiment of the text recognition method provided in this application. The solution of this application can be applied to any computer device with image processing capabilities, such as a personal computer or a server, without limitation.

[0058] The method in this embodiment may include:

[0059] S101, Obtain the text image.

[0060] Among them, the text image is an image containing text information, which is also the image from which the text needs to be identified in this application.

[0061] Generally, the resolution of the text image is low, resulting in a low accuracy rate for text recognition.

[0062] S102, perform feature extraction on the text image to obtain the shallow text features of the text image.

[0063] Among them, shallow text features can be in the form of images; therefore, shallow text features can also be called shallow text feature maps.

[0064] Shallow text features can reflect the detailed information of text in a text image, but the receptive field of shallow text features is relatively small.

[0065] For example, conventional shallow feature extraction methods can be used to extract shallow features from text images to obtain shallow text features of the text image.

[0066] In one possible implementation, a shallow feature extraction model can be used to extract shallow text features from the text image. For example, the shallow feature extraction model can be a convolutional neural network consisting of a convolutional layer with a kernel size of 9. Correspondingly, this shallow extraction model can reduce the image channels of the text image to 64 and fill in 4 pixels around each pixel of the text image to complete feature extraction and obtain shallow text features.

[0067] S103, perform shift convolution processing on the shallow text features to obtain the first intermediate text features of the text image.

[0068] The shift-conv method divides the shallow text features into five groups based on the number of image channels. The first four groups are then shifted along different spatial dimensions, including translation (up, down, left, and right). The last group remains unchanged. The shifted features are then fused with the neighboring pixel information of the first four groups and the last group (which remains unchanged) through a convolutional layer (e.g., a 1x1 convolutional layer).

[0069] For ease of distinction, this application refers to the features obtained by shifting and convolving the shallow text features as the first intermediate text features.

[0070] In this application, depending on the actual needs, the number of shift convolution operations on shallow text features can be one or multiple. In one alternative approach, in order to better integrate the relationships between multiple pixels, expand the receptive field, and enable local pixels to learn information from global pixels, multiple convolution shift operations can be performed on the shallow text features in this application.

[0071] S104, the first intermediate text feature is calculated using a multi-window self-attention module with different window sizes. The calculated multi-window self-attention features are then fused to obtain the second intermediate text feature of the text image.

[0072] Among them, the Group-wise Multi-scale Self-Attention (GMSA) module can be pre-trained without any restrictions.

[0073] The GMSA module performs self-attention calculations on the first intermediate text feature using multiple groups of different window sizes. First, the input first intermediate text feature is divided into multiple groups according to its image channels, resulting in multiple groups of text features. For different groups of text features, different window sizes are used for self-attention calculations. The number of groups into which the input first intermediate text feature is divided can vary depending on the application scenario.

[0074] For example, the GSMA module can divide the first intermediate text features into three groups, resulting in a first group of text features, a second group of text features, and a third group of text features. For the first group of text features, a 4-pixel window is used to calculate the self-attention of the first group of text features; for the second group of text features, an 8-pixel window is used to calculate the self-attention of the second group of text features; and for the third group of text features, a 12-pixel window is used to calculate the self-attention of the third group of text features. Based on this, the self-attention features calculated by these three windows are fused using a 1*1 convolutional neural network. The feature fusion based on the convolutional neural network can be accomplished through a rolling operation, without restriction.

[0075] It is understandable that by performing self-attention calculation on the first intermediate text features of a text image in a multi-window manner, it is possible not only to mine the feature information of the text image at multiple scales, but also, due to the efficiency of the GMSA module, using GMSA to mine text features in depth can save the computational cost of mining deep text features.

[0076] For ease of distinction, this application refers to the text features obtained after processing by the GMSA module as the second intermediate text features.

[0077] S105, perform a residual connection between the shallow text features and the second intermediate text features to obtain the deep text features of the text image.

[0078] For example, the trained residual network can be used to perform residual connections between shallow text features and second intermediate text features.

[0079] It is understandable that residual connections can reduce gradient vanishing, thereby enabling the obtained deep text features to more realistically and accurately reflect the deep features of the text image.

[0080] S106, Based on the shallow text features and deep text features, identify the text in the text image.

[0081] Understandably, this application, while performing conventional feature extraction on text images to obtain shallow text features, further extracts deep text features based on these shallow text features, in order to more comprehensively and accurately reflect the text features in the text images, thereby improving the accuracy of text recognition.

[0082] The process of recognizing text based on shallow and deep text features can have many possibilities, and this application does not impose any restrictions on it.

[0083] In one possible implementation, this application can first construct a super-resolution text image based on shallow and deep text features. Then, it identifies the text within the super-resolution text image, thereby determining the text contained within the text image.

[0084] As can be seen, after extracting the shallow text features of the text image, this application performs shift convolution processing on the shallow text features to achieve feature fusion between different channels. Based on this, this application also utilizes a multi-window self-attention module to perform multiple sets of self-attention calculations with different window sizes on the text features after shift convolution processing, and fuses the calculated multiple sets of self-attention features, so that the fused second intermediate text features can extract text features from the text image more deeply and globally. Therefore, residual connection between the shallow text features and the second intermediate text features can reduce the impact of gradient vanishing on the accuracy and comprehensiveness of text features, thus ultimately obtaining deeper text features that more accurately reflect global features. Therefore, by using shallow and deep text features, the local details and global text features of the text image can be more accurately and comprehensively represented, naturally improving the accuracy of text recognition.

[0085] Furthermore, in the process of extracting deep text features, this application employs a shift-convolution operation to extract shallow text features instead of using a traditional recurrent neural network (RNN). This reduces the time and computational overhead associated with using RNNs for feature extraction. Moreover, this application uses the GMSA module to further mine the features after the shift-convolution operation, reusing the high performance and low overhead of the GMSA module. This facilitates more efficient extraction of deep text features from text images while reducing computational costs.

[0086] Understandably, in order to reduce the impact of text distortion and other deformations on text recognition and further improve the accuracy of text recognition, this application performs text correction on the text image before extracting shallow text features. For example, a trained Spatial Transformer Networks (STN) model is used to perform image correction on the text image to correct the text in the text image.

[0087] Furthermore, to improve the text correction effect, this application can also utilize an activation function to activate the text image after text correction. For example, the activation function can be a Parametric Rectified Linear Unit (PReLU).

[0088] In this application, in order to further improve the accuracy of text recognition, a super-resolution text image can be constructed based on the shallow and deep text features of the text image, and then text recognition can be performed on the super-resolution text image.

[0089] There are various specific implementations for constructing super-resolution text images based on shallow and deep text features.

[0090] In one possible implementation, this application can first fuse the shallow text features and the application text features to obtain the fused text features of the text image. Then, the fused text features are pixel-recombined, and the recombined text features are actually a high-resolution text image.

[0091] However, some defects may exist in the pixel recombination process. Therefore, in order to reduce the defects in the pixel recombination process and further enrich the local information of pixels in the image, this application can also use a pixel-wise attention module (PAM) to upsample the recombined text image to obtain a super-resolution text image. The PAM module is also called a pixel-wise self-attention module.

[0092] like Figure 2 As shown, it illustrates a schematic diagram of the implementation principle framework of the text recognition method according to an embodiment of this application.

[0093] Combination Figure 2 As can be seen, the scheme of this application first extracts shallow text features from a low-resolution text image, and then extracts deep text features based on these shallow text features. On this basis, this application performs feature fusion on the shallow and deep text features, and then uses a pixel reconstruction module to reconstruct the fused text features pixel by pixel to obtain a reconstructed text image. Finally, the PAM module is used to perform pixel-by-pixel attention calculation on the reconstructed text image to achieve upsampling and obtain a super-resolution text image. Accordingly, by performing text recognition on the super-resolution text image, the text contained in the text image can be identified.

[0094] The essence of PAM upsampling the reconstructed text image is as follows: for each pixel in the reconstructed text image, weights are assigned to the neighboring nodes of the pixel based on the attention mechanism, and the weights of the corresponding features of each neighboring node of the pixel are added to the feature of the pixel to construct the feature of a new pixel. The feature of the new pixel is then inserted into the surrounding area of ​​the pixel in the text image.

[0095] Upsampling images using the PAM module is highly efficient and consumes little video memory, thus improving the performance of text recognition. Furthermore, the PAM module takes into account the local information of pixels in the image during the upsampling process, thereby enhancing the richness of text information in super-resolution text images.

[0096] In this application, in order to enable the PAM module to better integrate the features of each pixel in the reconstructed text image, the PAM module includes a module that performs fully connected layer transformation on the features of the pixels.

[0097] For example, in this application, the PAM module upsamples the image f that needs to be upsampled. l A new image f obtained through conversion processing l+1 The conversion formula can be expressed as Formula 1 below:

[0098]

[0099] Where, N i Represents image f l Let ω() be the set of k nearest neighbor pixels of any pixel i, i∈n, where n belongs to the pixel set of the image, ω() be the feature transformation function, and R be the relationship matrix of the transformed pixels, such as the relationship matrix between pixel i and pixel j.

[0100] In this application, the relation matrix can be represented by the following formula:

[0101]

[0102] Here, softmax represents the normalization function. θ() and φ() represent different fully connected layer transformations. i l Represents image f l Features of pixel i, f j l Represents image f l The features of pixel j. C is the image f. l The number of channels.

[0103] Wherein, image f l It can represent the text image input to the PAM module, such as the reconstructed text image; or it can be any layer of text image converted by the PAM module, without any restrictions.

[0104] Understandably, in order to more accurately represent the features in text images, this application may include multiple shift-convolution operations when performing shift-convolution processing on shallow text images. The text recognition method of this application will be described below with reference to one possible scenario of shift-convolution processing.

[0105] like Figure 3 The diagram illustrates a flowchart of another embodiment of the text recognition method provided in this application. The method of this embodiment may include:

[0106] S301, Obtain the text image to be recognized and perform text correction on the text image.

[0107] For example, the STN model can be used for text correction of text images. The STN model allows the neural network to learn how to perform spatial transformations on the input image to enhance the model's geometric invariance, thereby improving the image correction effect.

[0108] S302, use an activation function to activate the text image after text correction.

[0109] For example, the PReLU module can be used to activate text images. The purpose of this activation function is to increase the non-linear relationship between the layers of the neural network. PReLU can adaptively learn the parameters of the linear unit ReLU and improve the model accuracy with negligible additional computational cost, thereby further improving the text image correction effect.

[0110] S303, extract features from the corrected and activated text image to obtain the shallow text features of the text image.

[0111] For example, the trained shallow feature extraction module can be used to extract features from text images to obtain shallow text features.

[0112] S304, perform the first shift convolution on the shallow text features to obtain the first text features of the text image.

[0113] For ease of distinction, this application refers to the first shift convolution process as the first shift convolution process, the image features obtained after the first shift convolution process as the first text features, the subsequent second shift convolution process as the second shift convolution process, and the text features obtained after the second shift convolution process as the second text features.

[0114] The first and second shift convolutions can both be processed using different shift convolution modules. For details, please refer to the previous introduction on shift convolution (shift-conv), which will not be repeated here.

[0115] It is understandable that both the first and second text features can be in the form of feature maps, without any restrictions.

[0116] S305, the first text feature is activated using an activation function.

[0117] In this application, the activation functions used in step S305 and step S302 can be the same type of interception function or different types of activation functions. For ease of distinction, these two activation functions can be referred to as the first function and the second activation function, respectively, based on their order of appearance.

[0118] For example, in step S305, the activation function can be a rectified linear unit (ReLU).

[0119] Adding an activation function after convolution shifting can enhance the nonlinear relationship between the network layers that implement convolution shifting, which is beneficial for better expressing the features in text images.

[0120] S306, Perform a second shift convolution on the activated first text features to obtain the second text features of the text image.

[0121] S307, Based on the second text feature, determine the first intermediate text feature of the text image.

[0122] For example, the second text feature can be identified as the first intermediate text feature.

[0123] In one alternative approach, this application can perform a residual connection between the second text features and the previously obtained shallow text features to obtain the first intermediate text features of the text image. For example, the trained first residual network can be used to perform a residual connection between the second text features and the shallow text features.

[0124] S308, the GMSA module is used to perform self-attention calculations on the first intermediate text feature with multiple sets of different window sizes, and the calculated sets of self-attention features are fused to obtain the second intermediate text feature of the text image.

[0125] This step can be referred to in the relevant description of the previous embodiments, and will not be repeated here.

[0126] S309, perform a residual connection between the shallow text features and the second intermediate text features to obtain the deep text features of the text image.

[0127] For example, the trained second residual network is used to perform residual connections between shallow text features and second intermediate text features, and the features obtained by the residual connections are determined as deep text features.

[0128] Understandably, deep text features can also be in the form of feature maps, without any restrictions.

[0129] S310, the shallow text features and deep text features are fused to obtain the fused text features of the text image.

[0130] The purpose of fusing shallow and deep text features is to add the principal pixels of the shallow and deep text features. There are various ways to fuse them, and there are no restrictions on them.

[0131] S311, Perform pixel recombination on the fused text features to obtain the recombined text image.

[0132] It is understandable that fusing text features is essentially a feature map.

[0133] Pixel shuffle is essentially a process of rearranging the pixels of an image. By rearranging pixels, a low-resolution image can be transformed into a high-resolution image. Pixel shuffle itself is also a form of upsampling. Therefore, compared to the text image obtained in step S301, the reshuffled text image is also a high-resolution text image.

[0134] S312 uses the PAM module to upsample the reconstructed text image to obtain a super-resolution text image.

[0135] Step S312 can be found in the previous description and will not be repeated here.

[0136] S313 performs text recognition on the super-resolution text image to obtain the recognized text.

[0137] It is understood that steps S310 to S313 are illustrated using one implementation method of text recognition based on shallow text features and deep text features as an example. Other implementation methods mentioned above are also applicable to this embodiment and are not limited thereto.

[0138] It is understood that all models (such as network models or function modules) involved in the text recognition method of this application need to be trained in advance. In order to achieve better text recognition results, the gradient profile loss function can be used as the objective function for training the model in this application.

[0139] To facilitate understanding, the process of training the various models involved in the above embodiments in this application will be described below using one possible scenario as an example.

[0140] First, multiple text image samples labeled with high-resolution images can be obtained.

[0141] For example, multiple high-resolution images that meet the resolution requirements can be obtained first. The high-resolution images can be downsampled to obtain images as text image samples. The high-resolution image of the text image sample is the high-resolution image labeled by the text image sample.

[0142] Of course, there are many other ways to obtain text image samples, and there are no restrictions on this.

[0143] Secondly, for each text image sample, a hypertext resolution image of the text image sample is constructed based on the scheme of this application.

[0144] The process of constructing text image samples using the scheme of this application can be found in the previous section. Figure 2 as well as Figure 3 Related information.

[0145] For example, an STN network is used to correct text image samples. The corrected text image samples are then input into a PReLu unit for activation. The activated text image samples are then input into a shallow feature extraction module. The shallow text features output from the shallow feature extraction module are then processed sequentially through a first convolutional shift module, a ReLu module, and a second convolutional shift module. The feature map output from the second convolutional shift module is then input into a residual network for residual connection. The residually connected feature map is then input into a GMSA module. The feature map output from the GMSA module is then input into another residual network for residual connection, resulting in deep text features. Finally, the deep and shallow text features are fused and processed sequentially through a pixel reconstruction module and a PAM module to obtain a super-resolution image of the text image samples.

[0146] Next, based on the first gradient of each text image sample and the second gradient of the super-resolution image corresponding to each text image sample, and the set loss function, the loss function value is calculated.

[0147] For ease of distinction, the gradient of the text image sample is called the first gradient, while the gradient of the super-resolution image of the text image sample is called the second gradient.

[0148] For example, for each text image sample, the gradient difference between the first gradient of the text image sample and the second gradient of the super-resolution image of that text image sample can be used. Based on this, combined with the gradient difference corresponding to each text image sample, the mean squared error (MSE) function is used to calculate the mean squared error value of the gradient difference of each text image sample, and this mean squared error value is determined as the loss function value.

[0149] Of course, this is just one example of how to calculate the loss function value. Other methods of calculating the loss function value by combining the gradient difference between the text image sample and its corresponding super-resolution image are also applicable to this embodiment, and there are no restrictions on them.

[0150] Finally, if the loss function value does not converge, adjust the parameters of each model and reconstruct the super-resolution image for each text image sample until the loss function converges, and end the model training.

[0151] Corresponding to the text recognition method of this application, this application also provides a text recognition device.

[0152] like Figure 4 The diagram illustrates a structural composition of a text recognition device provided in an embodiment of this application. The device in this embodiment may include:

[0153] Image acquisition unit 401 is used to acquire text images;

[0154] The feature extraction unit 402 is used to extract features from the text image to obtain shallow text features of the text image;

[0155] The shift convolution unit 403 is used to perform shift convolution processing on the shallow text features to obtain the first intermediate text features of the text image;

[0156] Attention calculation unit 404 is used to perform multiple sets of self-attention calculations with different window sizes on the first intermediate text features using a multi-window self-attention module, and to fuse the calculated multiple sets of self-attention features to obtain the second intermediate text features of the text image.

[0157] The feature connection unit 405 is used to perform a residual connection between the shallow text features and the second intermediate text features to obtain the deep text features of the text image.

[0158] The text recognition unit 406 is used to recognize the text in the text image based on the shallow text features and the deep text features.

[0159] In one possible implementation, the text recognition unit may include:

[0160] An image construction subunit is used to construct a super-resolution text image of the text image based on the shallow text features and deep text features;

[0161] A text recognition subunit is used to recognize text in the super-resolution text image.

[0162] Furthermore, the image construction subunit includes:

[0163] The feature fusion subunit is used to fuse the shallow text features and deep text features to obtain the fused text features of the text image.

[0164] A pixel recombination unit is used to reconstruct the fused text features into a pixel-based image to obtain a reconstructed text image.

[0165] A pixel-level processing unit is used to upsample the reconstructed text image using a pixel-level attention module to obtain a super-resolution text image of the text image.

[0166] In one possible implementation, the shifted convolutional unit includes:

[0167] The first shift convolution unit is used to perform a first shift convolution process on the shallow text features to obtain the first text features of the text image;

[0168] The first activation unit is used to activate the first text feature using a first activation function;

[0169] The second shift convolution unit is used to perform a second shift convolution process on the activated first text features to obtain the second text features of the text image;

[0170] The feature determination unit is used to determine the first intermediate text features of the text image based on the second text features.

[0171] In yet another possible implementation, the feature determining unit includes:

[0172] The feature determination subunit is used to perform a residual connection between the second text feature and the shallow text feature to obtain the first intermediate text feature of the text image.

[0173] In yet another possible implementation, the device further includes:

[0174] An image correction unit is used to perform text correction on the text image before the feature extraction unit extracts the shallow features of the text image;

[0175] The second activation unit is used to activate the text image after text correction using the second activation function.

[0176] It should be noted that the various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. Furthermore, the features described in the various embodiments of this specification can be substituted or combined with each other, enabling those skilled in the art to implement or use this application. For apparatus embodiments, since they are basically similar to method embodiments, the description is relatively simple; relevant parts can be referred to the descriptions of the method embodiments.

[0177] Finally, it should be noted that in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes the element.

[0178] The above description of the disclosed embodiments enables those skilled in the art to make or use this application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of this application. Therefore, this application is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

[0179] The above are merely preferred embodiments of this application. It should be noted that those skilled in the art can make various improvements and modifications without departing from the principles of this application, and these improvements and modifications should also be considered within the scope of protection of this application.

Claims

1. A text recognition method, characterized by, The method comprises the following steps: obtaining a text image; performing feature extraction on the text image to obtain a shallow text feature of the text image; performing shift convolution processing on the shallow text feature to obtain a first intermediate text feature of the text image; performing self-attention calculation on the first intermediate text feature by using a group-by-group multi-window self-attention module with different window sizes to obtain a plurality of groups of self-attention features, and fusing the plurality of groups of self-attention features to obtain a second intermediate text feature of the text image; performing residual connection on the shallow text feature and the second intermediate text feature to obtain a deep text feature of the text image; based on the shallow text feature and the deep text feature, identifying the text in the text image, comprising: performing fusion processing on the shallow text feature and the deep text feature to obtain a fusion text feature of the text image; performing pixel reorganization on the fusion text feature to obtain a reorganized text image; performing up-sampling on the reorganized text image by using a pixel-level attention module to obtain a super-resolution text image of the text image; based on the shallow text feature and the deep text feature, constructing a super-resolution text image of the text image; identifying the text in the super-resolution text image; the step of performing up-sampling on the reorganized text image by using the pixel-level attention module to obtain the super-resolution text image of the text image comprises: for each pixel point in the reorganized text image, assigning weights to neighbor nodes of the pixel point based on an attention mechanism, adding the weights of the features of the neighbor nodes of the pixel point to the feature of the pixel point, and constructing a feature of a new pixel point, and inserting the feature of the new pixel point into the surrounding of the pixel point in the text image.

2. The method of claim 1, wherein, the step of performing shift convolution processing on the shallow text feature to obtain the first intermediate text feature of the text image comprises: performing first shift convolution processing on the shallow text feature to obtain a first text feature of the text image; activating the first text feature by using a first activation function; performing second shift convolution processing on the activated first text feature to obtain a second text feature of the text image; based on the second text feature, determining the first intermediate text feature of the text image.

3. The method of claim 2, wherein, the step of determining the first intermediate text feature of the text image based on the second text feature comprises: performing residual connection on the second text feature and the shallow text feature to obtain the first intermediate text feature of the text image.

4. The method of claim 1, wherein, before extracting the shallow feature of the text image, the method further comprises: performing text correction on the text image; activating the text-corrected text image by using a second activation function.

5. The method of claim 1, wherein, The pixel-level attention module comprises a conversion relationship matrix of pixels : wherein ; wherein denotes a normalization function; and denote different fully connected layer transformations, respectively; representing an image that needs up-sampling any one of the pixels characteristics of, , belonging to the image set of pixels; representative image central pixel characteristic, , representative image central pixel set of​ is the number of channels of the image .

6. A text recognition apparatus characterized by comprising: The method comprises the following steps: an image obtaining unit is configured to obtain a text image; a feature extraction unit is configured to perform feature extraction on the text image to obtain a shallow text feature of the text image; a shift convolution unit is configured to perform shift convolution processing on the shallow text feature to obtain a first intermediate text feature of the text image; An attention calculation unit is configured to perform self-attention calculation on the first intermediate text feature by using a group-wise multi-window self-attention module with different window sizes, fuse the calculated self-attention features, and obtain a second intermediate text feature of the text image. A feature connection unit is configured to perform residual connection on the shallow text feature and the second intermediate text feature, and obtain a deep text feature of the text image. A text recognition unit is configured to recognize text in the text image based on the shallow text feature and the deep text feature. The text recognition unit includes an image construction subunit configured to construct a super-resolution text image of the text image based on the shallow text feature and the deep text feature, and a text recognition subunit configured to recognize text in the super-resolution text image. The image construction subunit includes a feature fusion subunit configured to fuse the shallow text feature and the deep text feature, and obtain a fused text feature of the text image, and a pixel recombination unit configured to recombine the fused text feature, and obtain a recombined text image. A pixel-level processing unit is configured to perform up-sampling on the recombined text image by using a pixel-level attention module, and obtain a super-resolution text image of the text image. The pixel-level processing unit is specifically configured to assign weights to neighbor nodes of each pixel in the recombined text image based on an attention mechanism, add the weights of the features of the neighbor nodes of each pixel to the feature of the pixel, construct a feature of a new pixel, and insert the feature of the new pixel into the text image around the pixel.

Images