Text recognition method and device, electronic equipment and medium

Abstract

The application discloses a character recognition method and device, electronic equipment and medium, and belongs to the field of character recognition algorithms. The character recognition method comprises the following steps: obtaining a character picture, wherein the character picture comprises at least one character; inputting the character picture into a grouping convolutional neural network model for prediction to obtain character sequence prediction information corresponding to the character picture; and obtaining a character recognition result corresponding to the character picture based on the character sequence prediction information.

Description

Technical Field

[0001] This application belongs to the field of artificial intelligence technology, specifically relating to a text recognition method, device, electronic device, and medium. Background Technology

[0002] With the development of smart terminal technology, text recognition technology is being used more and more widely. This text recognition technology can be used to extract text from images.

[0003] In related technologies, when electronic devices perform text recognition, they usually reduce the number of network parameters in each layer of the convolutional neural network model to reduce the amount of computation and parameters, thereby improving the recognition speed. However, this method will reduce the recognition accuracy of the convolutional neural network model, resulting in poor overall recognition performance. Summary of the Invention

[0004] The purpose of this application is to provide a text recognition method, device, electronic device, and medium that can solve the problem of low recognition accuracy of convolutional neural network models, resulting in poor overall recognition performance.

[0005] To solve the above-mentioned technical problems, this application is implemented as follows:

[0006] In a first aspect, embodiments of this application provide a text recognition method, the method comprising: acquiring a text image, the text image including at least one character; inputting the text image into a grouped convolutional neural network model for prediction to obtain text sequence prediction information corresponding to the text image; and obtaining a text recognition result corresponding to the text image based on the text sequence prediction information.

[0007] Secondly, embodiments of this application provide a text recognition device, which includes: an acquisition module, a prediction module, and a processing module, wherein: the acquisition module is used to acquire a text image, the text image including at least one character; the prediction module is used to input the text image acquired by the acquisition module into a grouped convolutional neural network model for prediction to obtain text sequence prediction information corresponding to the text image; and the processing module is used to obtain a text recognition result corresponding to the text image based on the text sequence prediction information obtained by the prediction module.

[0008] Thirdly, embodiments of this application provide an electronic device including a processor and a memory, wherein the memory stores programs or instructions executable on the processor, and the programs or instructions, when executed by the processor, implement the steps of the method described in the first aspect.

[0009] Fourthly, embodiments of this application provide a readable storage medium on which a program or instructions are stored, which, when executed by a processor, implement the steps of the method described in the first aspect.

[0010] Fifthly, embodiments of this application provide a chip, the chip including a processor and a communication interface, the communication interface being coupled to the processor, the processor being used to run programs or instructions to implement the method as described in the first aspect.

[0011] In a sixth aspect, embodiments of this application provide a computer program product stored in a storage medium, which is executed by at least one processor to implement the method described in the first aspect.

[0012] In this embodiment, the electronic device can acquire a text image, which includes at least one character; input the text image into a grouped convolutional neural network model for prediction to obtain text sequence prediction information corresponding to the text image; and based on the text sequence prediction information, obtain the target character recognition result corresponding to the text image. Thus, because the grouped convolutional neural network model has fewer parameters and can divide the input data into multiple groups for simultaneous processing, the computational load of the model can be reduced while maintaining recognition accuracy, thereby improving the recognition performance of the electronic device. Attached Figure Description

[0013] Figure 1 This is a schematic flowchart of a text recognition method provided in an embodiment of this application;

[0014] Figure 2 This is a schematic diagram of the structure of the convolutional recurrent neural network model provided in the embodiments of this application;

[0015] Figure 3 This is a schematic diagram of the structure of the grouped convolutional neural network model provided in the embodiments of this application;

[0016] Figure 4 This is a schematic diagram of the structure of a character recognition device provided in an embodiment of this application;

[0017] Figure 5 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application;

[0018] Figure 6 This is a hardware schematic diagram of an electronic device provided in an embodiment of this application. Detailed Implementation

[0019] The technical solutions of the embodiments of this application will be clearly described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this application. All other embodiments obtained by those skilled in the art based on the embodiments of this application are within the scope of protection of this application.

[0020] The terms "first," "second," etc., used in the specification and claims of this application are used to distinguish similar objects and not to describe a specific order or sequence. It should be understood that such use of data can be interchanged where appropriate so that embodiments of this application can be implemented in orders other than those illustrated or described herein, and the objects distinguished by "first," "second," etc., are generally of the same class and the number of objects is not limited; for example, a first object can be one or more. Furthermore, in the specification and claims, "and / or" indicates at least one of the connected objects, and the character " / " generally indicates that the preceding and following objects are in an "or" relationship.

[0021] The text recognition method, apparatus, electronic device, and medium provided in this application will be described in detail below with reference to the accompanying drawings and through specific embodiments and application scenarios.

[0022] Currently, text recognition technology is widely used. Compared to cloud computing, mobile optical character recognition (OCR) algorithms can extract text from images offline. This algorithm offers significant advantages such as low latency, protection of data privacy and security, reduced cloud energy consumption, and independence from network stability, making it suitable for scenarios involving timeliness, cost, and privacy considerations. However, due to the limited computing resources of mobile electronic devices, they cannot run complex OCR algorithm models to meet users' needs for fast and accurate text recognition in images.

[0023] The OCR algorithm model described above employs a network structure based on a Convolutional Recurrent Neural Network (CRNN) temporal classification algorithm (CTC). This network structure mainly consists of three parts: a convolutional neural network, a recurrent neural network, and a transcription neural network. The CRNN is constructed from a series of convolutional layers, pooling layers, and batch normalization (BN) layers. After the image is input into the CRNN, it is transformed into a feature map with characteristic information and output as a sequence, serving as the input to the recurrent layers. The recurrent neural network is composed of a bidirectional long short-term memory (LSTM), which has a strong ability to capture information from sequences and can obtain more contextual information to better recognize textual information in the image and obtain a predicted sequence. The transcription neural network uses the CTC algorithm to convert the predicted sequence obtained by the recurrent neural network into a labeled sequence, which is used to obtain the final recognition result.

[0024] In related technologies, electronic devices need to employ computationally inefficient models for text recognition while achieving good recognition results. To enable the CRNN network model to be applied to electronic devices, the number of parameters in the convolutional layers of the CRNN model needs to be reduced to decrease computational complexity, thereby achieving real-time performance and reducing the size of the CRNN network model. However, this method of reducing the number of parameters significantly reduces the accuracy of text recognition, resulting in poor final text recognition performance.

[0025] In the text recognition method, apparatus, electronic device, and medium provided in this application embodiment, the electronic device can acquire a text image, which includes at least one character; input the text image into a grouped convolutional neural network model for prediction to obtain text sequence prediction information corresponding to the text image; and based on the text sequence prediction information, obtain the text recognition result corresponding to the text image. Thus, since the grouped convolutional neural network model has fewer parameters and can divide the input data into multiple groups for simultaneous processing, the computational load of the grouped convolutional neural network model can be reduced while maintaining recognition accuracy, thereby improving the recognition performance of the electronic device.

[0026] The text recognition method provided in this embodiment can be executed by a text recognition device, which can be an electronic device, or a control module or processing module within the electronic device. The following description uses an electronic device as an example to illustrate the technical solution provided in this application embodiment.

[0027] This application provides a text recognition method, such as... Figure 1 As shown, the character recognition method may include the following steps 201 to 203:

[0028] Step 201: The electronic device acquires the text image.

[0029] In this embodiment of the application, the above-mentioned text image includes at least one character.

[0030] For example, the above text can be Chinese characters, English, or other languages, and this application embodiment does not limit this.

[0031] In this embodiment of the application, the aforementioned text image may be a text image that has been processed in grayscale by an electronic device.

[0032] In this embodiment of the application, the grayscale processing described above involves uniformly processing the red (R), green (G), and blue (B) values ​​in the text image to make R = G = B.

[0033] For example, the text images mentioned above have the same size and height.

[0034] For example, the electronic device can scale the size of the aforementioned text images to make all the text images the same size.

[0035] Step 202: The electronic device inputs the text image into the grouped convolutional neural network model for prediction, and obtains the text sequence prediction information corresponding to the text image.

[0036] In this embodiment of the application, the above-mentioned grouped convolutional neural network model includes grouped convolutional layers, which are used to extract at least two sets of image feature information corresponding to the above-mentioned text image.

[0037] In this embodiment of the application, the above-mentioned text sequence prediction information is obtained based on the above-mentioned at least two sets of image feature information.

[0038] In the embodiments of this application, the above-mentioned grouped convolutional neural network model is generated by improving the network structure model of CRNN+CTC.

[0039] For example, the recurrent neural network in the aforementioned CRNN is removed and replaced with a convolutional neural network (CNN) + CTC network structure model. Then, the number of parameters in each layer of the CNN is reduced, and some standard convolutions are replaced with group convolutions with the same kernel size and 1*1 kernels, which have fewer parameters. Finally, to compensate for the decrease in recognition accuracy caused by removing the recurrent neural network and reducing the number of parameters, the network depth of the CNN is increased to improve the representational capability of the aforementioned grouped convolutional neural network model.

[0040] It should be noted that the above-mentioned increase in CNN network depth can be achieved by customizing a convolutional module consisting of three alternating convolutional groups with 3*3 kernels and convolutions with 1*1 kernels.

[0041] In the embodiments of this application, the improved CNN+CTC refers to a predictive model for text recognition of text images that can be deployed on electronic devices.

[0042] For example, the above sequence position can be a grouped convolutional neural network model that predicts the position based on multiple probability values ​​set according to the text position order in the above text image.

[0043] Step 203: The electronic device obtains the text recognition result corresponding to the text image based on the text sequence prediction information.

[0044] In this embodiment of the application, the text sequence prediction information may include a text sequence prediction matrix.

[0045] For example, the above text sequence is used to indicate the positional order of the text in the above text image.

[0046] Optionally, in this embodiment of the application, step 203 above, "the electronic device obtains the text recognition result corresponding to the text image based on the text sequence prediction information," may include the following steps 203a to 203c:

[0047] Step 203a: The electronic device calculates the target prediction probability information based on the text sequence prediction information.

[0048] In this embodiment of the application, the target prediction probability information is used to characterize the probability of each character index corresponding to each sequence position in the text sequence corresponding to the text sequence prediction information.

[0049] For example, each of the above text indices corresponds to a text in the character library.

[0050] In this embodiment of the application, the target prediction probability information may include a text sequence prediction probability matrix.

[0051] In this embodiment of the application, the electronic device can use a normalized exponential function to perform probability calculation on the text sequence prediction matrix to obtain the text sequence prediction probability matrix.

[0052] In the embodiments of this application, the normalized exponential function can be the softmax function.

[0053] It should be noted that the normalized exponential function mentioned above is used to uniformly convert the values ​​of the above text sequence prediction matrix into probability values ​​ranging from 0 to 1.

[0054] Step 203b: The electronic device determines the text prediction result at each sequence position based on the target prediction probability information.

[0055] In this embodiment of the application, each sequence position may correspond to multiple text prediction results. The electronic device can determine the text prediction result with the highest prediction probability among the multiple text prediction results as the text prediction result of that sequence position.

[0056] In this embodiment of the application, the electronic device can use the prediction information corresponding to the maximum probability value at each sequence position in the above-mentioned text sequence prediction probability as the recognition result index of the sequence position, and then index the text prediction result corresponding to the prediction information from the character set dictionary pre-stored by the electronic device to obtain the text recognition result at each sequence position.

[0057] Step 203c: The electronic device determines the text recognition result corresponding to the text image based on the text prediction result at each sequence position.

[0058] In this embodiment, the electronic device can repeat the above indexing steps to obtain the text recognition result sequence corresponding to the above text sequence. Then, the electronic device can merge the duplicate recognition results of adjacent sequence positions through CTC and remove the empty recognition results to obtain the final text recognition result.

[0059] The generation of the character set dictionary used in the embodiments of this application will be explained below:

[0060] For example, an electronic device can count the frequency of all Chinese characters that appear when training the above-mentioned grouped convolutional neural network model, and take the Chinese characters with a frequency greater than a preset threshold as the character set dictionary.

[0061] In this way, by calculating the probability of the character recognition result corresponding to each sequence position, and selecting the recognition result with the highest probability from the multiple recognition results as the final character recognition result, the accuracy of character recognition is improved.

[0062] In the text recognition method provided in this application embodiment, an electronic device can acquire a text image, which includes at least one character; input the text image into a grouped convolutional neural network model for prediction, and obtain text sequence prediction information corresponding to the image features in the text image; based on the text sequence prediction information, obtain the text recognition result corresponding to the text image. Thus, because the grouped convolutional neural network model has fewer parameters, and because it can divide the input data into multiple groups for simultaneous processing, the computational load of the grouped convolutional neural network model can be reduced while maintaining recognition accuracy, thereby improving the recognition performance of the electronic device.

[0063] Optionally, in the embodiments of this application, the above-mentioned grouped convolutional neural network model includes: a first standard convolutional layer, a group convolutional layer, a second standard convolutional layer, and a fully connected layer.

[0064] In this embodiment, the first standard convolutional layer, the group convolutional layer, the second standard convolutional layer, and the fully connected layer are connected sequentially.

[0065] In this embodiment of the application, the first standard convolutional layer includes a target standard convolutional unit, and the first standard convolutional layer includes a convolutional kernel.

[0066] It should be noted that the aforementioned target standard convolutional unit is used to reduce the number of parameters in the above grouped convolutional neural network model.

[0067] In the embodiments of this application, each convolution in the first standard convolutional layer includes a convolutional kernel.

[0068] For example, the first standard convolutional layer mentioned above can be a convolutional layer composed of 3*3 convolution, pooling layer, 3*3 convolution, pooling layer, 1*1 convolution, and pooling layer.

[0069] For example, the target standard convolutional unit mentioned above can be a 1*1 convolution.

[0070] It should be noted that the 1*1 convolution mentioned above is used to indicate the size of the feature, so as to avoid the previous 3*3 convolution having too many parameters.

[0071] In this embodiment, the aforementioned group convolutional layer includes a target group convolutional unit, and the aforementioned group convolutional layer includes M convolutional kernels, where M is an integer greater than 1.

[0072] It should be noted that the aforementioned target group convolutional units are used to reduce the computational cost of the aforementioned grouped convolutional neural network model.

[0073] For example, the above-mentioned group convolutional layer can be a group convolutional layer composed of 1*1 convolution, 3*3 group convolution, 1*1 convolution, 3*3 group convolution, 1*1 convolution, 3*3 group convolution, 1*1 convolution, and pooling layer.

[0074] For example, the target group convolutional unit mentioned above can be a 3*3 group convolution.

[0075] In this embodiment, the second standard convolutional layer includes a convolutional kernel.

[0076] Thus, by setting target standard convolutional units and target group convolutional units in the grouped convolutional neural network model, the number of parameters and computational cost of the grouped convolutional model can be reduced, thereby improving the recognition efficiency of electronic devices.

[0077] Optionally, in this embodiment of the application, step 202 above, "the electronic device inputs the text image into the grouped convolutional neural network model for prediction, and obtains the text sequence prediction information corresponding to the text image," may include the following steps 202a to 202d:

[0078] Step 202a: After the electronic device inputs the text image into the grouped convolutional neural network model, it uses the first standard convolutional layer to extract the first image feature information of the text image.

[0079] In this embodiment of the application, the first image feature information is used to characterize the text region features in the text image.

[0080] For example, the electronic device can sequentially use a 3*3 convolution, a pooling layer, a 3*3 convolution, a pooling layer, a 1*1 convolution, and a pooling layer (i.e., the first standard convolutional layer mentioned above) to extract primary features (i.e., the first image feature information mentioned above) from the text image.

[0081] Step 202b: The electronic device uses a group convolutional layer to group the first image feature information to obtain M groups of image feature information, and uses M convolutional kernels in the group convolutional layer to extract key image feature information from each group of image feature information, and fuses the obtained M groups of key image feature information to obtain the first key image feature information.

[0082] In the embodiments of this application, each convolutional kernel in the above-mentioned group of convolutional layers is used to process a set of image feature information.

[0083] In this embodiment of the application, the first key image feature information is used to characterize the text feature information in the text region features.

[0084] For example, the electronic device can sequentially employ 1*1 convolution, group convolution, 1*1 convolution, group convolution, 1*1 convolution, group convolution, 1*1 convolution, group convolution, 1*1 convolution, and pooling layer (i.e., the aforementioned group convolution layer) to extract intermediate features from the aforementioned primary features. The aforementioned 1*1 convolution is used to process the irregular results of the output of the previous pooling layer to improve the network's expressive power. Then, again sequentially employing 1*1 convolution, group convolution, 1*1 convolution, group convolution, 1*1 convolution, group convolution, 1*1 convolution, and pooling layer, it extracts high-level features (i.e., the aforementioned first key image feature information) from the aforementioned intermediate features. The aforementioned group convolution is a group convolution with a kernel size of 3*3 and a group number of 4. This group convolution divides the aforementioned first image feature information into 4 groups, each group is convolved using a 3*3 convolution kernel to obtain its own key image feature information. Then, the 4 groups of key image feature information are merged to obtain a single convolution output (i.e., the aforementioned first key image feature information).

[0085] It should be noted that the number of parameters in the above group convolution with a 3*3 kernel is only one-quarter of the number of parameters in the convolution with a 3*3 kernel.

[0086] Step 202c: The electronic device uses a second standard convolutional layer to extract the text sequence features of the first key image feature information.

[0087] In this embodiment of the application, the above-mentioned text sequence features are used to characterize the text content of the text in the above-mentioned text image.

[0088] For example, after obtaining the first key image feature information, the electronic device can first use a 1*1 convolution to process the irregular information in the first key image feature information, and then use a 2*2 convolution (i.e. the second standard convolutional layer) to convert the height dimension of the processed first key image feature information to 1 (i.e., remove the height dimension), thereby extracting the above-mentioned text sequence features from the first key image feature information after removing the height dimension.

[0089] Step 202d: The electronic device uses a fully connected layer to obtain the text sequence prediction information corresponding to the text sequence features.

[0090] In related technologies, after obtaining the aforementioned text sequence features, two LSTMs are used to extract the sequence features, and these text sequence features are then converted into a text sequence prediction matrix. However, this LSTM cannot be processed in parallel, and its processing efficiency in electronic devices is low, resulting in poor text recognition performance.

[0091] In this embodiment, after obtaining the aforementioned text sequence features, the electronic device can use a fully connected layer to reduce the feature dimension size of the text sequence features, thereby reducing the number of parameters in the next fully connected layer. Then, another fully connected layer is used to convert the text sequence features into a text sequence prediction matrix (i.e., the aforementioned text sequence prediction information).

[0092] It should be noted that the feature dimension size mentioned above is equal to the number of characters in the character set dictionary plus one.

[0093] It is understandable that an electronic device can add a null character to the total number of characters included in the character set dictionary, and then set the feature dimension size according to the number of characters after adding the null character, so that the feature dimension size is equal to the number of characters after adding the null character.

[0094] Thus, by using an improved grouped convolutional neural network model to process the input text image, the electronic device can obtain the corresponding text sequence prediction information more quickly. Furthermore, by using a fully connected layer to process the first key image feature information, the number of parameters of the grouped convolutional neural network model is further reduced, thereby improving the text recognition performance of the electronic device.

[0095] Optionally, in this embodiment of the application, after step 201 above, the text recognition method provided in this embodiment of the application further includes step 201a as shown below:

[0096] Step 201a: The electronic device crops the text image into N sub-text images.

[0097] In this embodiment of the application, each of the above N sub-text images contains at least one character, where N is an integer greater than 1.

[0098] In this embodiment of the application, the image size and height of the above N sub-text images are all equal.

[0099] In this embodiment of the application, the electronic device can detect the position of all text lines in the above-mentioned text image, and then crop all text line images (i.e., the above-mentioned N sub-text images) according to the detected position coordinates, and then scale the above-mentioned text line images to convert them into images with equal height.

[0100] It should be noted that the height of the above text line image matches the data size that the above grouped convolutional neural network model can process.

[0101] Further optionally, in this embodiment of the application, in conjunction with the above step 201a, the step 202, "the electronic device inputs the text image into the grouped convolutional neural network model for prediction, and obtains the text sequence prediction information corresponding to the text image," may include the following step 202e:

[0102] Step 202e: The electronic device inputs N sub-text images into a grouped convolutional neural network model for prediction, and obtains the text sequence prediction information corresponding to each of the N sub-text images.

[0103] In this embodiment of the application, the electronic device can input the first sub-text image among the above N sub-text images into a grouped convolutional neural network model for prediction, and after obtaining the prediction result, input the second sub-text image, and so on for prediction.

[0104] In this embodiment, after obtaining the text sequence prediction information corresponding to each of the N sub-text images, the electronic device can obtain the text recognition result based on the prediction information. Then, according to the detected text position coordinates, the text recognition result is arranged to obtain the target text recognition result of the aforementioned text image.

[0105] In this way, by cropping the text image one by one, the computational load of the above-mentioned grouped convolutional neural network model can be reduced, further improving the recognition speed and ensuring the recognition accuracy.

[0106] The training process of the grouped convolutional neural network model used in the embodiments of this application will be described by way of example below:

[0107] For example, the training process of the above-mentioned grouped convolutional neural network model may include the following steps S1 to S4:

[0108] Step S1: Data collection and expansion.

[0109] In this embodiment of the application, during the data collection process, in order to make the aforementioned grouped convolutional neural network model applicable to various scenarios, the collected text images need to include as many different scenarios as possible (such as cards, books and newspapers, screenshots, screens, posters, street scenes, handwritten text, etc.). Then, the collected text images need to be manually annotated to obtain corresponding text tag files.

[0110] Because manual data collection and annotation are inefficient, data augmentation is necessary to expand the data. This data augmentation can be achieved through two methods: data augmentation and font synthesis.

[0111] Data augmentation is the process of transforming labeled real data into new data through random geometric deformation, blurring, brightness and contrast adjustment, image compression, and other methods.

[0112] Font synthesis involves creating text images using font files and corpora, and then enhancing the realism and diversity of the synthesized images through random backgrounds, text colors, fonts, geometric deformations, perspective changes, blurring, brightness and contrast adjustments, and image compression.

[0113] In this embodiment of the application, sufficient training data can be obtained through the three methods described above: real data acquisition, data augmentation, and font synthesis.

[0114] Step S2: Data preprocessing.

[0115] In this embodiment of the application, before the collected data is fed into the model for training, the data needs to be processed uniformly, specifically: size scaling, width sorting, and dictionary creation.

[0116] Size scaling: The model design requires the input text images to have a fixed height of 32 pixels, while the width is not fixed. Therefore, the data needs to be scaled proportionally to a height of 32 pixels.

[0117] Width Sorting: Text images vary in length, but during training, multiple text images are often input in batches. This requires that the text images in a batch have the same width and height. However, when the widths of text images within the same batch differ significantly, forcibly adjusting the widths to be uniform can distort the text in some images, leading to substantial information loss and hindering effective training. Therefore, the text images in the training set can be sorted according to their aspect ratio. Several text images with adjacent aspect ratios are taken as a batch, and all text images within the batch are uniformly scaled using the size of the text image with the smallest width.

[0118] Step S3: Model building.

[0119] In the embodiments of this application, such as Figure 2As shown, the classic CRNN network structure consists of a CNN based on 3x3 convolutions and a Recurrent Neural Network (RNN) based on LSTM. When an electronic device inputs a text image with a height of 32 pixels into the model, it first extracts image feature information through a CNN. For example, it sequentially uses one 3x3 convolution (3x3 Conv), a pooling layer, one 3x3 convolution, a pooling layer, two 3x3 convolutions, a pooling layer, and two 3x3 convolutions, a pooling layer to extract image feature information, while gradually increasing the feature dimension from 64 to 512. Next, it generates sequence features through an image mapping sequence structure. Then, it uses two LSTMs to extract the sequence features from the image feature information and outputs the sequence features as a sequence prediction matrix.

[0120] It should be noted that the aforementioned CNN mainly consists of convolutional and pooling layers with progressively increasing feature dimensions and 3x3 kernels, used to extract image feature information; the aforementioned RNN consists of two LSTM layers, used to extract sequence features and convert the sequence features into a sequence prediction matrix. However, the computational cost of this CRNN network structure is too high, and its performance and model size cannot meet the requirements of electronic devices. In addition, LSTM is not conducive to deployment on electronic devices.

[0121] In the embodiments of this application, in order to enable the model to have better performance and effects on electronic devices with limited computing power, such as Figure 3 As shown, we significantly reduced the feature dimension size; and removed LSTM, which is not easy to deploy on electronic devices, and instead used fully connected layers (FC) to convert sequence features into sequence prediction matrices; in addition, we used only CNN networks instead of CNN+RNN networks to extract image feature information, and the CNN network also abandoned the original scheme of using 3*3 convolution kernels, and instead replaced some 3*3 convolution kernels with group convolutions and 1*1 convolutions with smaller parameters, and improved the model's feature learning ability by using deeper network layers.

[0122] For example, to reduce the number of parameters while ensuring good feature learning ability, we reduce the feature dimension size from 32 to 192. Then, we first use 3*3 convolution, pooling layer, 3*3 convolution, 1*1 convolution (1*1 Conv), pooling layer to extract primary image feature information from the input text image. The added 1*1 convolution is used to increase the feature dimension size to avoid the previous 3*3 convolution having too many parameters. Then, we use 1*1 convolution, group convolution (3*3 group Conv), 1*1 convolution, group convolution, 1*1 convolution, group convolution, 1*1 convolution, group convolution, 1*1 convolution, pooling layer to extract intermediate image feature information from the above primary image feature information. The first 1*1 convolution is used to add non-linear excitation to the output of the previous pooling layer to improve the network's expressive ability. Next, a series of convolutional layers (1x1, group convolutions, 1x1, group convolutions, 1x1, group convolutions, 1x1, group convolutions, 1x1 convolutions, and pooling layers) are used to extract high-level image features from the intermediate-level image features. Finally, a 1x1 convolution is applied to add non-linear activation to the high-level image features, and a 2x2 convolution is used to convert the height dimension to 1. Then, the height dimension is removed, and the feature dimensions and width dimensions are swapped to meet the requirements of the next layer, thus transforming the four-dimensional high-level image features into a three-dimensional feature sequence. This feature sequence is then passed through a fully connected layer with fewer parameters to reduce the feature dimension, thus reducing the number of parameters in the next layer. Finally, another fully connected layer converts the reduced feature dimension sequence into a sequence prediction matrix. The resulting sequence prediction matrix is ​​the output of the entire model.

[0123] It should be noted that the combination of alternating group convolutions and 1*1 convolutions repeated 3 times, compared with the structure of two 3*3 convolutions in the traditional CRNN, increases the network depth while reducing the number of parameters, thus improving the model's representation ability.

[0124] Step S4: Model training and quantization.

[0125] In this embodiment, model training involves dividing the text images into multiple batches, each batch consisting of a fixed number of text images, and then randomly feeding them into the model. When a batch of text images is fed into the model, the model built in step S3 is used to calculate layer by layer to obtain a text sequence prediction matrix. Then, a normalized exponential function (softmax) is used to convert the values ​​in the text sequence prediction matrix into a text sequence prediction probability matrix with values ​​ranging from 0 to 1. Next, based on the text sequence prediction probability matrix, a greedy algorithm is used to determine the prediction result corresponding to the highest probability value as the prediction result for that sequence position, and the predicted text sequence is obtained based on the character set dictionary index mapping. The classic loss function (CTC loss) is used to calculate the loss value between the predicted text sequence and the corresponding label text sequence in the text image. Based on the loss value, a stochastic optimizer (Adaptive Momentum, Adam) is used to backpropagate the model and update the model parameters. The initial learning rate of the stochastic optimizer is set to 0.0005, and then gradually decreased using a cosine learning rate decreasing method. Then, the above operation is repeated for the next batch of text images to update the model parameters again. After multiple rounds of parameter updates, the loss value drops to a suitable range and tends to stabilize, thus completing the training of the model.

[0126] Model quantization: In order to accelerate the model inference speed and maintain good accuracy, the parameters are stored and the model is inferred using a half-precision (Full Precise Float, FP) method to obtain the above-mentioned grouped convolutional neural network model.

[0127] The text recognition method provided in this application can be executed by a text recognition device. This application uses a text recognition device executing the text recognition method as an example to illustrate the text recognition device provided in this application.

[0128] This application provides a text recognition device, such as... Figure 4 As shown, the text recognition device 400 includes: an acquisition module 401, a prediction module 402, and a processing module 403, wherein: the acquisition module 401 is used to acquire a text image, the text image including at least one character; the prediction module 402 is used to input the text image acquired by the acquisition module 401 into a grouped convolutional neural network model for prediction to obtain text sequence prediction information corresponding to the text image; the processing module 403 is used to obtain the text recognition result corresponding to the text image based on the text sequence prediction information obtained by the prediction module 402.

[0129] Optionally, in this embodiment, the grouped convolutional neural network model includes: a first standard convolutional layer, a group convolutional layer, a second standard convolutional layer, and a fully connected layer; the prediction module 402 is specifically used for: inputting the text image obtained by the acquisition module 401 into the grouped convolutional neural network model, extracting the first image feature information of the text image using the first standard convolutional layer; grouping the first image feature information using the group convolutional layer to obtain M groups of image feature information, extracting key image feature information from each group of image feature information using M convolutional kernels in the group convolutional layer, and fusing the obtained M groups of key image feature information to obtain the first key image feature information, where each convolutional kernel in the group convolutional layer is used to process a group of image feature information, and M is an integer greater than 1; extracting the text sequence features of the first key image feature information using the second standard convolutional layer; and obtaining the text sequence prediction information corresponding to the text sequence features using the fully connected layer.

[0130] Optionally, in this embodiment, the first standard convolutional layer, the group convolutional layer, the second standard convolutional layer, and the fully connected layer are connected sequentially; the first standard convolutional layer includes a target standard convolutional unit, which is used to reduce the number of parameters in the group convolutional neural network model, and the first standard convolutional layer includes one convolutional kernel; the group convolutional layer includes a target group convolutional unit, which is used to reduce the computational cost of the group convolutional neural network model, the group convolutional layer includes M convolutional kernels, and the second standard convolutional layer includes one convolutional kernel.

[0131] Optionally, in this embodiment of the application, the text recognition device 400 further includes a cropping module, wherein: the cropping module is used to crop the text image into N sub-text images after the acquisition module 401 acquires the text image, each sub-text image containing at least one character, and N is an integer greater than 1; the prediction module 402 is specifically used to input the N sub-text images obtained by the cropping module into a grouped convolutional neural network model for prediction, to obtain the text sequence prediction information corresponding to each of the N sub-text images.

[0132] Optionally, in this embodiment of the application, the processing module 403 is specifically used to: calculate target prediction probability information based on the text sequence prediction information obtained by the prediction module 402, wherein the target prediction probability information is used to characterize the probability of each character index corresponding to each sequence position in the text sequence corresponding to the text sequence prediction information, wherein each character index corresponds to a character in the character library; determine the text prediction result at each sequence position based on the target prediction probability information; and determine the text recognition result corresponding to the text image based on the text prediction result at each sequence position.

[0133] The text recognition device provided in this application embodiment can acquire text images, which include at least one character; input the text image into a grouped convolutional neural network model for prediction to obtain text sequence prediction information corresponding to the text image; and based on the text sequence prediction information, obtain the text recognition result corresponding to the text image. Thus, because the grouped convolutional neural network model has fewer parameters and can divide the input data into multiple groups for simultaneous processing, the computational load of the grouped convolutional neural network model can be reduced while maintaining recognition accuracy, thereby improving the recognition effect of the text recognition device.

[0134] The text recognition device in this application embodiment can be an electronic device or a component within an electronic device, such as an integrated circuit or a chip. The electronic device can be a terminal or other devices besides a terminal. For example, the electronic device can be a mobile phone, tablet computer, laptop computer, PDA, in-vehicle electronic device, mobile internet device (MID), augmented reality (AR) / virtual reality (VR) device, robot, wearable device, ultra-mobile personal computer (UMPC), netbook, or personal digital assistant (PDA), etc. It can also be a server, network attached storage (NAS), personal computer (PC), television (TV), ATM, or self-service machine, etc. This application embodiment does not specifically limit the device.

[0135] The text recognition device in this application embodiment can be a device with an operating system. This operating system can be Android, iOS, or other possible operating systems; this application embodiment does not specifically limit it.

[0136] The character recognition device provided in this application embodiment can achieve... Figure 1 The various processes implemented in the method implementation examples will not be described again here to avoid repetition.

[0137] Optionally, such as Figure 5As shown, this application embodiment also provides an electronic device 600, including a processor 601 and a memory 602. The memory 602 stores a program or instructions that can run on the processor 601. When the program or instructions are executed by the processor 601, they implement the various steps of the above-described text recognition method embodiment and can achieve the same technical effect. To avoid repetition, they will not be described again here.

[0138] It should be noted that the electronic devices in the embodiments of this application include the mobile electronic devices and non-mobile electronic devices described above.

[0139] Figure 6 A schematic diagram of the hardware structure of an electronic device to implement an embodiment of this application.

[0140] The electronic device 100 includes, but is not limited to, components such as: radio frequency unit 101, network module 102, audio output unit 103, input unit 104, sensor 105, display unit 106, user input unit 107, interface unit 108, memory 109, and processor 110.

[0141] Those skilled in the art will understand that the electronic device 100 may also include a power supply (such as a battery) for supplying power to various components. The power supply may be logically connected to the processor 110 through a power management system, thereby enabling functions such as managing charging, discharging, and power consumption through the power management system. Figure 6 The electronic device structure shown does not constitute a limitation on the electronic device. The electronic device may include more or fewer components than shown, or combine certain components, or have different component arrangements, which will not be elaborated here.

[0142] The processor 110 is configured to: acquire a text image, the text image including at least one character; input the text image into a grouped convolutional neural network model for prediction to obtain text sequence prediction information corresponding to the text image; and obtain the text recognition result corresponding to the text image based on the text sequence prediction information.

[0143] Optionally, in this embodiment, the grouped convolutional neural network model includes: a first standard convolutional layer, a group convolutional layer, a second standard convolutional layer, and a fully connected layer; the processor 110 is specifically used for: inputting the text image into the grouped convolutional neural network model, using the first standard convolutional layer to extract the first image feature information of the text image; using the group convolutional layer to group the first image feature information to obtain M groups of image feature information, and using M convolutional kernels in the group convolutional layer to extract key image feature information from each group of image feature information, and fusing the obtained M groups of key image feature information to obtain the first key image feature information, where each convolutional kernel in the group convolutional layer is used to process a group of image feature information, and M is an integer greater than 1; using the second standard convolutional layer to extract the text sequence features of the first key image feature information; and using the fully connected layer to obtain the text sequence prediction information corresponding to the text sequence features.

[0144] Optionally, in this embodiment, the first standard convolutional layer, the group convolutional layer, the second standard convolutional layer, and the fully connected layer are connected sequentially; the first standard convolutional layer includes a target standard convolutional unit, which is used to reduce the number of parameters in the group convolutional neural network model, and the first standard convolutional layer includes one convolutional kernel; the group convolutional layer includes a target group convolutional unit, which is used to reduce the computational cost of the group convolutional neural network model, the group convolutional layer includes M convolutional kernels, and the second standard convolutional layer includes one convolutional kernel.

[0145] Optionally, in this embodiment of the application, the processor 110 is further configured to crop the text image into N sub-text images, each sub-text image containing at least one character, where N is an integer greater than 1; specifically, the processor 110 is configured to input the N sub-text images into a grouped convolutional neural network model for prediction, thereby obtaining the text sequence prediction information corresponding to each of the N sub-text images.

[0146] Optionally, in this embodiment of the application, the processor 110 is specifically used to: calculate target prediction probability information based on the text sequence prediction information obtained by the prediction module 402, wherein the target prediction probability information is used to characterize the probability of each character index corresponding to each sequence position in the text sequence corresponding to the text sequence prediction information, wherein each character index corresponds to a character in the character library; determine the text prediction result at each sequence position based on the target prediction probability information; and determine the text recognition result corresponding to the text image based on the text prediction result at each sequence position.

[0147] In the electronic device provided in this application embodiment, the electronic device can acquire a text image, which includes at least one character; input the text image into a grouped convolutional neural network model for prediction to obtain text sequence prediction information corresponding to the text image; and based on the text sequence prediction information, obtain the text recognition result corresponding to the text image. Thus, because the grouped convolutional neural network model has fewer parameters, and because it can divide the input data into multiple groups for simultaneous processing, the computational load of the grouped convolutional neural network model can be reduced while maintaining recognition accuracy, thereby improving the recognition performance of the electronic device.

[0148] It should be understood that, in this embodiment, the input unit 104 may include a graphics processing unit (GPU) 1041 and a microphone 1042. The GPU 1041 processes image data of still images or videos obtained by an image capture device (such as a camera) in video capture mode or image capture mode. The display unit 106 may include a display panel 1061, which may be configured in the form of a liquid crystal display, an organic light-emitting diode, or the like. The user input unit 107 includes at least one of a touch panel 1071 and other input devices 1072. The touch panel 1071 is also called a touch screen. The touch panel 1071 may include a touch detection device and a touch controller. Other input devices 1072 may include, but are not limited to, a physical keyboard, function keys (such as volume control buttons, power buttons, etc.), a trackball, a mouse, and a joystick, which will not be described in detail here.

[0149] The memory 109 can be used to store software programs and various data. The memory 109 may primarily include a first storage area for storing programs or instructions and a second storage area for storing data. The first storage area may store the operating system, application programs or instructions required for at least one function (such as sound playback, image playback, etc.). Furthermore, the memory 109 may include volatile memory or non-volatile memory, or both. The non-volatile memory may be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. Volatile memory can be random access memory (RAM), static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDRSDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM), and direct memory bus RAM (DRRAM). The memory 109 in the embodiments of this application includes, but is not limited to, these and any other suitable types of memory.

[0150] Processor 110 may include one or more processing units; optionally, processor 110 integrates an application processor and a modem processor, wherein the application processor mainly handles operations involving the operating system, user interface, and applications, and the modem processor mainly handles wireless communication signals, such as a baseband processor. It is understood that the aforementioned modem processor may also not be integrated into processor 110.

[0151] This application also provides a readable storage medium storing a program or instructions. When the program or instructions are executed by a processor, they implement the various processes of the above-described text recognition method embodiments and achieve the same technical effect. To avoid repetition, they will not be described again here.

[0152] The processor is the processor in the electronic device described in the above embodiments. The readable storage medium includes computer-readable storage media, such as computer read-only memory (ROM), random access memory (RAM), magnetic disk, or optical disk.

[0153] This application embodiment also provides a chip, which includes a processor and a communication interface. The communication interface is coupled to the processor. The processor is used to run programs or instructions to implement the various processes of the above-described text recognition method embodiments and can achieve the same technical effect. To avoid repetition, it will not be described again here.

[0154] It should be understood that the chip mentioned in the embodiments of this application may also be referred to as a system-on-a-chip, system chip, chip system, or system-on-a-chip, etc.

[0155] This application provides a computer program product, which is stored in a storage medium and executed by at least one processor to implement the various processes of the above-described text recognition method embodiments, and can achieve the same technical effect. To avoid repetition, it will not be described again here.

[0156] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element. Furthermore, it should be noted that the scope of the methods and apparatuses in the embodiments of this application is not limited to performing functions in the order shown or discussed, but may also include performing functions substantially simultaneously or in the reverse order, depending on the functions involved. For example, the described methods may be performed in a different order than described, and various steps may be added, omitted, or combined. Additionally, features described with reference to certain examples may be combined in other examples.

[0157] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a computer software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal (which may be a mobile phone, computer, server, or network device, etc.) to execute the methods described in the various embodiments of this application.

[0158] The embodiments of this application have been described above with reference to the accompanying drawings. However, this application is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of this application without departing from the spirit and scope of the claims, and all of these forms are within the protection scope of this application.

Claims

1. A character recognition method, characterized in that, The method includes: Acquire a text image, wherein the text image includes at least one character; The text image is input into a grouped convolutional neural network model for prediction to obtain the text sequence prediction information corresponding to the text image. Based on the text sequence prediction information, the text recognition result corresponding to the text image is obtained; After obtaining the text image, the method further includes: The text image is cropped into N sub-text images, each sub-text image contains at least one character, and each sub-text image includes the text line from the text image; N is an integer greater than 1; The step of inputting the text image into a grouped convolutional neural network model for prediction to obtain the text sequence prediction information corresponding to the text image includes: The N sub-text images are input into a grouped convolutional neural network model for prediction, and the text sequence prediction information corresponding to each of the N sub-text images is obtained. The grouped convolutional neural network model includes: a first standard convolutional layer, a group convolutional layer, a second standard convolutional layer, and a fully connected layer; The step of inputting the text image into a grouped convolutional neural network model for prediction to obtain the text sequence prediction information corresponding to the text image includes: After the text image is input into the grouped convolutional neural network model, the first image feature information of the text image is extracted using the first standard convolutional layer. The first image feature information is grouped using the group convolutional layer to obtain M groups of image feature information. The key image feature information in each group of image feature information is extracted using M convolutional kernels in the group convolutional layer. The M groups of key image feature information are then fused to obtain the first key image feature information. Each convolutional kernel in the group convolutional layer is used to process a group of image feature information, where M is an integer greater than 1. The second standard convolutional layer is used to extract the text sequence features of the first key image feature information; The fully connected layer is used to obtain the text sequence prediction information corresponding to the text sequence features; wherein, the first standard convolutional layer includes a target standard convolutional unit, and the target standard convolutional unit is a 1*1 convolution.

2. The method according to claim 1, characterized in that, The first standard convolutional layer, the group convolutional layer, the second standard convolutional layer, and the fully connected layer are connected in sequence; The target standard convolutional unit is used to reduce the number of parameters in the grouped convolutional neural network model, and the first standard convolutional layer includes a convolutional kernel; The group convolutional layer includes a target group convolutional unit, which is used to reduce the computational cost of the group convolutional neural network model. The group convolutional layer includes M convolutional kernels. The second standard convolutional layer includes a convolutional kernel.

3. The method according to claim 1, characterized in that, The step of obtaining the text recognition result corresponding to the text image based on the text sequence prediction information includes: Based on the text sequence prediction information, target prediction probability information is calculated. The target prediction probability information is used to characterize the probability of each character index corresponding to each sequence position in the text sequence corresponding to the text sequence prediction information. Each character index corresponds to a character in the character library. Based on the target prediction probability information, the text prediction result at each sequence position is determined; Based on the text prediction results at each sequence position, the text recognition result corresponding to the text image is determined.

4. A character recognition device, characterized in that, The device includes: an acquisition module, a prediction module, and a processing module, wherein: The acquisition module is used to acquire text images, wherein the text images include at least one character; The prediction module is used to input the text image obtained by the acquisition module into the grouped convolutional neural network model for prediction, and obtain the text sequence prediction information corresponding to the text image. The processing module is used to obtain the text recognition result corresponding to the text image based on the text sequence prediction information obtained by the prediction module; The device further includes: a trimming module, wherein: The cropping module is used to crop the text image into N sub-text images after the acquisition module acquires the text image. Each sub-text image contains at least one character, and each sub-text image includes the text line in the text image; N is an integer greater than 1. The prediction module is specifically used to input the N sub-text images obtained by the cropping module into the grouped convolutional neural network model for prediction, and obtain the text sequence prediction information corresponding to each of the N sub-text images; The grouped convolutional neural network model includes: a first standard convolutional layer, a group convolutional layer, a second standard convolutional layer, and a fully connected layer; The prediction module is specifically used for: After the text image obtained by the acquisition module is input into the grouped convolutional neural network model, the first image feature information of the text image is extracted using the first standard convolutional layer. The first image feature information is grouped using the group convolutional layer to obtain M groups of image feature information. The key image feature information in each group of image feature information is extracted using M convolutional kernels in the group convolutional layer. The M groups of key image feature information are then fused to obtain the first key image feature information. Each convolutional kernel in the group convolutional layer is used to process a group of image feature information, where M is an integer greater than 1. The second standard convolutional layer is used to extract the text sequence features of the first key image feature information; The fully connected layer is used to obtain text sequence prediction information corresponding to the text sequence features; The first standard convolutional layer includes a target standard convolutional unit, which is a 1*1 convolution.

5. The apparatus according to claim 4, characterized in that, The first standard convolutional layer, the group convolutional layer, the second standard convolutional layer, and the fully connected layer are connected in sequence; The target standard convolutional unit is used to reduce the number of parameters in the grouped convolutional neural network model, and the first standard convolutional layer includes a convolutional kernel; The group convolutional layer includes a target group convolutional unit, which is used to reduce the computational cost of the group convolutional neural network model. The group convolutional layer includes M convolutional kernels. The second standard convolutional layer includes a convolutional kernel.

6. The apparatus according to claim 4, characterized in that, The processing module is specifically used for: Based on the text sequence prediction information obtained by the prediction module, target prediction probability information is calculated. The target prediction probability information is used to characterize the probability of each character index corresponding to each sequence position in the text sequence corresponding to the text sequence prediction information. Each character index corresponds to a character in the character library. Based on the target prediction probability information, the text prediction result at each sequence position is determined; Based on the text prediction results at each sequence position, the text recognition result corresponding to the text image is determined.

7. An electronic device, characterized in that, It includes a processor and a memory, the memory storing a program or instructions that can run on the processor, the program or instructions being executed by the processor to implement the steps of the character recognition method as described in any one of claims 1 to 3.

8. A readable storage medium, characterized in that, The readable storage medium stores a program or instructions that, when executed by a processor, implement the steps of the character recognition method as described in any one of claims 1 to 3.

Images