Text recognition method, apparatus, and electronic device

Abstract

The application discloses a text recognition method, belongs to the technical field of optical character recognition, and helps to improve text recognition accuracy. The method comprises the following steps: inputting a target image into a convolutional neural network in a pre-trained character recognition model, obtaining a feature map with a height of D and a width of n output by the convolutional neural network for the target image, wherein D and n are integers greater than 1; reorganizing the feature map to obtain a feature sequence of the target image; encoding and mapping the feature sequence through a sequence encoding network in the character recognition model to obtain an encoded sequence; and decoding the encoded sequence through a CTC decoder in the character recognition model to obtain a character recognition result of the target image. According to the method, the feature map with a height greater than 1 is extracted, so that character recognition can be performed based on more fine-grained features, and the text recognition accuracy of complex texts such as arc-shaped text images and seal images is improved.

Description

Technical Field

[0001] This application relates to the field of optical character recognition technology, and in particular to text recognition methods, apparatus, electronic devices, and computer-readable storage media. Background Technology

[0002] Traditional methods for text recognition in images involve segmenting the image into text line images, then further segmenting each text line into individual characters, and finally recognizing each character separately. With the widespread application of neural network models, end-to-end text recognition based on deep learning has emerged. In deep learning-based end-to-end text recognition schemes, the explicit segmentation of text lines is eliminated. Instead, optical character recognition is transformed into a sequence learning problem. Although the input image scale and text length vary, after passing through CNNs (Convolutional Neural Networks) and RNNs (Recurrent Neural Networks), and undergoing CTC (Connectionist Temporal Classification) translation and transcription at the output stage, the entire text image can be recognized. However, existing deep learning-based end-to-end text recognition methods still require improvement in accuracy when processing text recognition in images such as seals or curved text. Summary of the Invention

[0003] This application provides a text recognition method that helps improve the accuracy of text recognition for complex text images such as curved text images and seal images.

[0004] In a first aspect, embodiments of this application provide a text recognition method, including:

[0005] The target image is input into a convolutional neural network in a pre-trained character recognition model to obtain a feature map with height D and width n output by the convolutional neural network for the target image, where D and n are integers greater than 1;

[0006] Based on the D groups of features in the feature map that correspond to the same width position but different height positions, a feature subsequence corresponding to the width position is obtained;

[0007] The feature subsequences corresponding to each of the width positions are sequentially concatenated to form the feature sequence of the target image;

[0008] The feature sequence is encoded and mapped using a sequence coding network in the character recognition model to obtain the encoded sequence;

[0009] The character recognition result of the target image is obtained by decoding the encoded sequence using the CTC decoder in the character recognition model.

[0010] Secondly, embodiments of this application provide a text recognition device, including:

[0011] The feature map acquisition module is used to input the target image into the convolutional neural network in the pre-trained character recognition model, and acquire the feature map with height D and width n output by the convolutional neural network for the target image, where D and n are integers greater than 1;

[0012] The first feature conversion module is used to obtain a feature subsequence corresponding to the width position based on D groups of features in the feature map that correspond to the same width position but different height positions.

[0013] The second feature conversion module is used to sequentially concatenate the feature subsequences corresponding to each of the width positions into the feature sequence of the target image;

[0014] The feature encoding module is used to encode and map the feature sequence through the sequence encoding network in the character recognition model to obtain the encoded sequence;

[0015] The decoding output module is used to decode the encoded sequence through the CTC decoder in the character recognition model to obtain the character recognition result of the target image.

[0016] Thirdly, embodiments of this application also disclose an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the text recognition method described in embodiments of this application.

[0017] Fourthly, embodiments of this application provide a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, performs the steps of the text recognition method disclosed in embodiments of this application.

[0018] The text recognition method disclosed in this application involves inputting a target image into a convolutional neural network in a pre-trained character recognition model to obtain a feature map with height D and width n output by the convolutional neural network for the target image, where D and n are integers greater than 1; obtaining feature subsequences corresponding to the width positions based on D groups of features corresponding to the same width position but different height positions in the feature map; sequentially concatenating the feature subsequences corresponding to each width position to form a feature sequence of the target image; encoding and mapping the feature sequence through a sequence coding network in the character recognition model to obtain an encoded sequence; and decoding the encoded sequence through a CTC decoder in the character recognition model to obtain the character recognition result of the target image. This method configures the height of the output feature map of the convolutional neural network, enabling the text recognition model to extract more fine-grained image features from the target image. As a result, when the subsequent sequence coding network performs feature encoding based on these finer-grained image features, it can not only effectively avoid noise in the image but also perform text recognition based on more detailed features. This improves the accuracy of text recognition for complex text images such as curved text images and seal images.

[0019] The above description is only an overview of the technical solution of this application. In order to better understand the technical means of this application and to implement it in accordance with the contents of the specification, and to make the above and other objects, features and advantages of this application more obvious and understandable, the following are specific embodiments of this application. Attached Figure Description

[0020] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0021] Figure 1 This is a flowchart of a text recognition method according to an embodiment of this application;

[0022] Figure 2 This is a schematic diagram of the character recognition model structure in one embodiment of this application;

[0023] Figure 3 This is a schematic diagram showing the correspondence between the input image and the output feature map of a convolutional neural network in one embodiment of this application;

[0024] Figure 4This is a schematic diagram illustrating the process of reconstructing a feature map output by a convolutional neural network into a feature sequence in one embodiment of this application;

[0025] Figure 5 This is a schematic diagram illustrating the correspondence between feature sequences input to a sequence coding network and their positions in an image, as used in existing technologies.

[0026] Figure 6 This is a schematic diagram illustrating the correspondence between the feature sequences input to the sequence coding network and their positions in the image in an embodiment of this application;

[0027] Figure 7 This is another flowchart of a text recognition method in one embodiment of this application;

[0028] Figure 8 This is a schematic diagram of the structure of a text recognition device in one embodiment of this application;

[0029] Figure 9 A block diagram schematically illustrates an electronic device for performing the method according to this application; and

[0030] Figure 10 A storage unit for holding or carrying program code implementing the method according to this application is illustrated schematically. Detailed Implementation

[0031] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0032] Example 1

[0033] This application discloses a text recognition method, such as... Figure 1 As shown, the method includes steps 110 to 150.

[0034] Step 110: Input the target image into the convolutional neural network in the pre-trained character recognition model, and obtain the feature map with height D and width n output by the convolutional neural network for the target image, where D and n are integers greater than 1.

[0035] The target image described in this embodiment is a grayscale image obtained after preprocessing the image of the text to be recognized. The height and width of the target image will be normalized to the input image size of the pre-trained character recognition model, for example, the height is H and the width is W.

[0036] In the embodiments of this application, the character recognition model is a pre-trained offline neural network model. The character recognition model may be a CRNN model.

[0037] The CRNN model combines CNN and RNN networks and trains them together. The CRNN model is mainly used to achieve end-to-end recognition of text sequences of variable length to a certain extent. It doesn't require segmenting individual characters first, but rather transforms text recognition into a time-dependent sequence learning problem; it's an image-based sequence recognition method. The CRNN model mainly consists of three layers: a CNN (convolutional layer), which uses a deep CNN to extract features from the input image to obtain a feature map; an RNN (recurrent layer), which uses a bidirectional RNN (BLSTM) to predict the feature sequence, learning each feature vector in the sequence and outputting the predicted label (true value) distribution; and a CTC loss (transcriptional layer), which uses CTC loss to transform the series of label distributions obtained from the recurrent layer into the final label sequence.

[0038] In existing technologies, for an image with height H and width W input to a CRNN model, after convolution processing by a CNN, the output is a feature map with height 1 and width W / m, where m is the width of the time window. Then, the feature vector sequence needed by the RNN is extracted from the feature map output by the CNN, and feature classification is performed for each width.

[0039] Taking a target image of size HxWx1 as an example, where H represents height, W represents width, and 1 represents channel value (here, grayscale), this target image is input into a CRNN model in existing technology. After being processed by the feature layers of a CNN (a series of convolutional layers, pooling layers, batch normalization layers, and activation layers), a feature map of the form 1xnxL is obtained. Here, 1 represents the height of the feature map, n represents the width of the feature map, and L is the number of feature maps.

[0040] like Figure 2 As shown, the character recognition model used in this embodiment includes: a convolutional neural network 210, a data structure conversion layer 220, a sequence encoding network 230, and a CTC decoder 240. In the text recognition stage, the convolutional neural network 210 is used to extract features from the input image (such as a target image) to obtain a feature map; the data structure conversion layer 220 is used to convert the feature map output by the convolutional neural network 210 into a feature sequence that meets the input requirements of the sequence encoding network 230; the sequence encoding network 230 is used to encode the input feature sequence to obtain an encoded sequence; and the CTC decoder 240 is used to transcribe the encoded sequence output by the sequence encoding network 230 into a character encoded sequence, i.e., the text recognition result.

[0041] In the embodiments of this application, the convolutional neural network 210 is configured to output L feature maps with a height of D and a width of n. The values ​​of L, the height D, and the width n are integers greater than 1. The number of feature maps L and the width n are determined based on the computing power of the processing device executing this method and industry experience. Further, the value of the height D is determined based on the curvature of the text lines matched in the target image. The larger the curvature of the text lines, the larger the height value D. For example, for a character recognition model used to recognize text in a seal image, the value of D can be set to 3 or 4, or a larger value; while for a character recognition model recognizing multiple or single text lines with curvature, the value of D can be set to 2.

[0042] In the specific implementation process, a character recognition model can be constructed first according to the specific application scenario of the text recognition method. The height value D of the feature map output by the convolutional neural network 210 in the character recognition model can be determined, and the structure of the convolutional neural network 210 can be designed according to the output requirement of the feature map height value D. In this way, after training, the obtained character recognition model will obtain a feature map of height D for each input image after feature extraction by the convolutional neural network 210.

[0043] In some embodiments of this application, adjustments can be made. Figure 2 The sampling window size or stride of one or more convolutional layers and / or max pooling layers located at the back end of the convolutional neural network 210 configured with the height of the feature map output by the convolutional neural network 210.

[0044] In this embodiment, taking the configuration of the height value of the output feature map of the convolutional neural network 210 as 3 as an example, after inputting the aforementioned target image of size HxWx1 into the character recognition model designed in this embodiment, the convolutional neural network 210 in the character recognition model will extract a feature map of height D from the input target image. That is, the feature map output by the convolutional neural network 210 in the character recognition model is a feature map of the form DxnxL, where D is 3, representing the height of the feature map. It can be understood that in the character recognition model designed in this embodiment, the feature map output by the convolutional neural network 210 is L feature maps with a width of n and a height of 3 (i.e., D). According to the mapping relationship between the feature map output by the convolutional neural network 210 and the pixel position of the target image, it can be concluded that the image features of the same width position of the target image (corresponding to the same time window of the convolutional neural network) are represented by D feature vectors, which have stronger feature expression capabilities and are more suitable for expressing text lines with curvature.

[0045] Because convolutional layers, max-pooling layers, and activation functions operate on local regions, they are translation-invariant. Therefore, each feature vector in the output feature map of the convolutional neural network 210 corresponds to a rectangular region (i.e., the receptive field) of the input image (such as the target image), and these rectangular regions have the same order as the corresponding columns on the feature map from left to right. Figure 3 Taking the input image and feature map shown as an example, if the feature maps output by the convolutional neural network 210 are divided into n time windows from left to right according to their width, and the input image is divided into n rectangular regions from left to right, then the feature vector corresponding to each time window in each feature map corresponds to a rectangular region in the input image. For example, Figure 3 Each feature map located in the cuboid region 320 is a feature vector of the image region 310 in the input image. In the embodiments of this application, the height of each feature vector is D, that is, each feature vector has D dimensions, and different dimensions of the feature vector correspond to different height positions in the input image.

[0046] Therefore, the L feature maps of the input image extracted by the convolutional neural network 210 designed in this embodiment of the application have D sets of features at a corresponding width position, and each set of features corresponds to a different image height.

[0047] Step 120: Based on the D groups of features in the feature map that correspond to the same width position but different height positions, obtain the feature subsequence corresponding to the width position.

[0048] In this embodiment, the sequence coding network 230 is implemented based on sequence network structures such as RNN or LSTM. The input data of the sequence coding network 230 is a feature sequence; therefore, it is necessary to convert the feature map output by the convolutional neural network 210 into a feature sequence that the sequence coding network 230 can process.

[0049] In the embodiments of this application, a data structure conversion layer 220 is set between the convolutional neural network 210 and the sequence coding network 230 to realize the conversion from feature map to feature sequence.

[0050] In some embodiments of this application, obtaining a feature subsequence corresponding to the width position based on D groups of features in the feature map that correspond to the same width position but different height positions includes: for D groups of features in the feature map that correspond to the same width position, concatenating the D groups of features from front to back in order of their positions in the target image from top to bottom to obtain a feature subsequence corresponding to the width position.

[0051] Depend on Figure 3As can be seen from the feature map mapping diagram, the feature vector corresponding to the same width position in each feature map (such as the feature vector in the cuboid shown in 320) expresses the image features of a rectangular image region (such as rectangular region 310) in the input image. In the embodiments of this application, by improving the structure of the convolutional neural network 210, the height of the feature map output by the convolutional neural network 210 is D. That is, the feature vector corresponding to the same width position in each feature map is represented by D-dimensional features, where each dimension of the features is used to express the image features of image regions at different height positions in the corresponding image at that width position. In other words, the features of the image region at each width position in the input image are expressed by D sets of feature vectors, and each set of feature vectors is used to express the image features of an image region at a specified image height position at that width position.

[0052] When performing feature data structure transformation, the D groups of feature vectors (i.e. feature vectors corresponding to each height) of the corresponding image region for a time window are used as the feature subsequences corresponding to that image region. Then, the feature subsequences corresponding to each image region are concatenated in sequence to obtain a feature vector sequence of the input image.

[0053] For example, for a convolutional neural network 210 with n time windows, the feature vectors in the feature map are represented as follows: For example, where j represents the height position, taking values ​​from 1 to D, and i represents the time window number, i.e., the width position, taking values ​​from 1 to n, the feature map output by this convolutional neural network 210, corresponding to the height position 1 of the input image, can be represented by the feature vector. The vector sequence formed, corresponding to the feature map of the height position D of the input image, can be represented as a sequence of feature vectors. The vector sequence formed.

[0054] The image height position is determined based on the feature height output by the designed convolutional neural network 210. For example, when the feature height output by the convolutional neural network 210 is configured as D, the input image (such as the target image) can be divided into D rows from top to bottom along the height direction, with each row corresponding to an image height position. For example, the top row can be defined as corresponding to image height position 1, and the bottom row as corresponding to image height position D, thereby determining the height position in the input image.

[0055] Take the D groups of feature vectors corresponding to the same width position i (i.e., the feature vectors at height positions 1 to D corresponding to the same width position i), and concatenate these D groups of features from front to back according to the top-to-bottom order of the image height positions corresponding to each group of feature vectors (as mentioned above, the order of height positions from 1 to D). This will give you the feature subsequence corresponding to the width position i. For example, the feature vector corresponding to height position 1 of the target image... Feature vector corresponding to height position 2 of the target image …until the feature vector corresponding to the height position D of the target image… By concatenating the data from front to back, we obtain the feature subsequence at width position i. The effect of the data structure transformation can be seen in [reference needed]. Figure 4 , Figure 4 The top three rows are feature vectors at different heights in the feature map, and the bottom row is the feature subsequence obtained after data structure transformation.

[0056] In some embodiments of this application, the feature vector corresponding to a certain time window can be obtained by fusing the feature vectors corresponding to the same image height position in the L feature maps output by the convolutional neural network 210 for that time window. For example, the feature vector corresponding to the image height position 1 in a column of feature vectors corresponding to the first time window in the L feature maps can be fused to obtain the feature vector. The feature vector corresponding to the image height position D in the column of feature vectors corresponding to the nth time window in the L feature maps is fused to obtain the feature vector.

[0057] Step 130: The feature subsequences corresponding to each of the width positions are sequentially concatenated to form the feature sequence of the target image.

[0058] Following the aforementioned method, the feature subsequence corresponding to each width position of the target image can be obtained.

[0059] For a convolutional neural network 210 with n time windows (i.e., the feature map of the target image corresponds to n width positions), if the convolutional neural network 210 is configured to output a feature map with a height of D, the L feature maps with a width of n and a height of D output by the convolutional neural network 210 will, after data structure transformation, result in a feature sequence of length D×n obtained by concatenating n feature subsequences of length D.

[0060] In existing technologies, the features input from the convolutional layer to the recurrent layer in a CRNN model are as follows: Figure 5The sequence of feature vectors corresponding to each column of image regions (i.e., each width position) is shown. In this embodiment, by configuring the height of the output feature map of the convolutional neural network 210 to a value greater than 1, the features input to the sequence coding network 230 (equivalent to a recurrent layer in the prior art) are as follows. Figure 6 The sequence of feature vectors corresponding to each grid region is shown. Since the feature vectors input to the sequence coding network 230 are generated based on feature maps at multiple different image height positions, they carry more detailed image features, resulting in a more granular representation of image features. This makes the character recognition model more robust to interference and the recognition results more accurate. For example, based on... Figure 6 When the sequence of feature vectors corresponding to each grid region is encoded and transcribed, the interference caused by the features of the pentagram region in the grid to text recognition will be greatly reduced. If recognition is performed using existing technology, the image features of the pentagram will account for a large proportion in the feature vector corresponding to the width position of the pentagram region, which will lead to character recognition errors at that width position.

[0061] Step 140: The feature sequence is encoded and mapped using the sequence coding network in the character recognition model to obtain the encoded sequence.

[0062] Next, the feature sequence of length D×n obtained after data structure transformation is input into the sequence coding network 230, and the sequence coding network 230 encodes and maps the feature sequence to obtain the encoded sequence.

[0063] For specific implementations of sequence coding networks, please refer to the specific implementations of RNN networks (i.e., recurrent layers) in CRNN networks in the prior art. They will not be repeated in the embodiments of this application.

[0064] Step 150: The encoded sequence is decoded by the CTC decoder in the character recognition model to obtain the character recognition result of the target image.

[0065] Subsequently, the encoded sequence obtained after encoding mapping processing by the sequence coding network 230 is input to the CTC decoder 240. The CTC decoder 240 transcribes the encoded sequence output by the sequence coding network 230 to obtain the text recognition result of the target image. The text recognition result is the character encoding sequence corresponding to the text lines in the target image.

[0066] For a detailed implementation of the CTC decoder, please refer to the CTC decoder in the existing CRNN network; it will not be repeated in the embodiments of this application.

[0067] To facilitate readers' understanding of the text recognition method disclosed in the embodiments of this application, the training scheme of the character recognition model will be further elaborated below.

[0068] See Figure 7 In some embodiments of this application, before inputting the target image into a convolutional neural network in a pre-trained character recognition model and obtaining a feature map with height D and width n output by the convolutional neural network for the target image, the method further includes: step 100.

[0069] Step 100: Train the character recognition model.

[0070] The training samples used to train the character recognition model are: labeled sample images, where the labels are the true encoded sequences of the characters in the sample images, and the convolutional neural network is configured to output a feature map of height D. The configuration and network structure of the convolutional neural network in the character recognition model can be found in the preceding description.

[0071] The training of the character recognition model includes: performing the following character recognition steps on each training sample to obtain the predicted encoding sequence of the character in the corresponding sample image: inputting the sample image into the convolutional neural network in the character recognition model to obtain a sample feature map with height D and width n output by the convolutional neural network for the sample image; obtaining a sample feature subsequence corresponding to the width position based on D groups of features corresponding to the same width position and different height positions in the sample feature map; sequentially concatenating the sample feature subsequences corresponding to each width position into a feature sequence of the sample image; encoding and mapping the feature sequence through the sequence coding network to obtain a sample encoding sequence; decoding the sample encoding sequence through the CTC decoder to obtain the predicted encoding sequence of the character in the sample image; calculating the CTC loss based on the predicted encoding sequence and the real encoding sequence of the character in the sample image of each training sample; and iteratively training the character recognition model by optimizing the CTC loss.

[0072] In some embodiments of this application, the sample image includes one or more of the following images: a stamp image, an arc-shaped text line image, and a multi-line text image.

[0073] During the training of the character recognition model, for each training sample, the model performs feature extraction, encoding mapping, and transcription operations to obtain the predicted encoding sequence corresponding to each training sample. Then, based on the labels (i.e., the true encoding sequences) of all training samples and the predicted encoding sequences obtained by the character recognition model, the CTC (Connectionist Temporal Classification) loss of the character recognition model is calculated using the CTC loss function. With the goal of optimizing the CTC loss, the model parameters in the convolutional neural network 210 and the sequence encoding network 230 of the character recognition model are adjusted to achieve the purpose of training the character recognition model.

[0074] During model training, for the current training sample, after the sample image of the training sample is input into the character recognition model, the convolutional neural network 210 of the character recognition model extracts features from the input sample image. The convolutional neural network 210 outputs a sample feature map with height D and width n for the sample image. The specific implementation method of feature extraction of the input image by the convolutional neural network 210 is described above and will not be repeated here.

[0075] Subsequently, the sample feature map is input to the data structure transformation layer 220. The data structure transformation layer 220 performs data structure transformation on the sample feature map to obtain the feature sequence required by the matching sequence coding network 230. For example, based on D groups of features corresponding to the same width position but different height positions in the sample feature map, sample feature sub-sequences corresponding to the respective width positions are obtained. Then, the sample feature sub-sequences corresponding to each width position are sequentially concatenated to form the feature sequence of the sample image. For a specific implementation method of performing data structure transformation on the sample feature map to obtain the feature sequence corresponding to the sample image, please refer to the specific implementation method of performing data structure transformation on the feature map of the target image to obtain the feature sequence in the text recognition stage above; it will not be repeated here.

[0076] Next, the feature sequence of the sample image is input into the sequence coding network 230, and the feature sequence is encoded and mapped by the sequence coding network 230 to obtain the sample coding sequence. For a specific implementation of encoding and mapping the feature sequence of the sample image using the sequence coding network to obtain the sample coding sequence, please refer to the text recognition stage. The specific implementation of encoding and mapping the feature sequence of the target image using the sequence coding network in the character recognition model to obtain the coding sequence will not be repeated here.

[0077] Subsequently, the sample encoding sequence is decoded using the CTC decoder to obtain the predicted encoding sequence of the characters in the sample image. For a specific implementation of decoding the sample encoding sequence using the CTC decoder to obtain the predicted encoding sequence of the characters in the sample image, please refer to the text recognition stage. The specific implementation of decoding the encoding sequence using the CTC decoder in the character recognition model to obtain the character recognition result of the target image is not repeated here.

[0078] In the embodiments of this application, the CTC loss function can be the general CTC loss function in the existing CRNN model. For the optimization methods of the model parameters and the iterative training process of the character recognition model, please refer to the prior art, and will not be repeated here.

[0079] The text recognition method disclosed in this application involves inputting a target image into a convolutional neural network in a pre-trained character recognition model to obtain a feature map with height D and width n output by the convolutional neural network for the target image, where D and n are integers greater than 1; obtaining feature subsequences corresponding to the width positions based on D groups of features corresponding to the same width position and different height positions in the feature map; sequentially concatenating the feature subsequences corresponding to each width position to form the feature sequence of the target image; encoding and mapping the feature sequence through a sequence coding network in the character recognition model to obtain an encoded sequence; and decoding the encoded sequence through a CTC decoder in the character recognition model to obtain the character recognition result of the target image, thereby improving the text recognition accuracy of complex text images such as curved text images and seal images.

[0080] The text recognition method disclosed in this application configures the height of the output feature map of the convolutional neural network, enabling the text recognition model to extract more fine-grained image features from the target image when performing feature extraction. In this way, when the subsequent sequence coding network performs feature encoding based on the more fine-grained image features, it can not only effectively avoid noise in the image, but also perform text recognition based on more detailed features, thereby improving the accuracy of text recognition of the target image.

[0081] The text recognition method disclosed in this application significantly improves the recognition performance of complex text images (such as multi-line text images, seal images, and arc-shaped text line images).

[0082] Example 2

[0083] This application discloses a text recognition device, such as... Figure 8 As shown, the device includes:

[0084] The feature map acquisition module 810 is used to input the target image into the convolutional neural network in the pre-trained character recognition model, and acquire the feature map with height D and width n output by the convolutional neural network for the target image, where D and n are integers greater than 1;

[0085] The first feature conversion module 820 is used to obtain a feature subsequence corresponding to the width position based on D groups of features in the feature map that correspond to the same width position but different height positions.

[0086] The second feature conversion module 830 is used to sequentially concatenate the feature subsequences corresponding to each of the width positions into a feature sequence of the target image;

[0087] The feature encoding module 840 is used to encode and map the feature sequence through the sequence encoding network in the character recognition model to obtain an encoded sequence.

[0088] The decoding output module 850 is used to decode the encoded sequence through the CTC decoder in the character recognition model to obtain the character recognition result of the target image.

[0089] In some embodiments of this application, the first feature conversion module 820 is further configured to:

[0090] For the D groups of features corresponding to the same width position in the feature map, the D groups of features are concatenated from front to back according to the top-to-bottom order of their positions in the target image to obtain the feature subsequence corresponding to the width position.

[0091] In some embodiments of this application, the apparatus further includes:

[0092] A character recognition model training module (not shown in the figure) is used to train the character recognition model. The training samples for training the character recognition model are labeled sample images, where the labels are the true encoding sequences of the characters in the sample images. The convolutional neural network is configured to output a feature map of height D. The character recognition model training module is further used for:

[0093] For each of the training samples, perform the following character recognition steps to obtain the predicted encoding sequence of the character in the corresponding sample image:

[0094] The sample image is input into the convolutional neural network in the character recognition model to obtain a sample feature map with height D and width n output by the convolutional neural network for the sample image;

[0095] Based on the D groups of features in the sample feature map that correspond to the same width position but different height positions, a sample feature subsequence corresponding to the width position is obtained;

[0096] The sample feature subsequences corresponding to each of the width positions are sequentially concatenated to form the feature sequence of the sample image;

[0097] The feature sequence is encoded and mapped using the sequence coding network to obtain the sample encoded sequence;

[0098] The CTC decoder decodes the sample encoding sequence to obtain the predicted encoding sequence of the characters in the sample image;

[0099] The character recognition model is iteratively trained by calculating the CTC loss based on the predicted encoding sequence and the real encoding sequence of the characters in the sample images of each training sample, and by optimizing the CTC loss.

[0100] In some embodiments of this application, the value of the height D is determined based on the radian of the text line matched by the target image.

[0101] In some embodiments of this application, the sample image includes one or more of the following images: a stamp image, an arc-shaped text line image, and a multi-line text image.

[0102] The text recognition device disclosed in this application is used to implement the text recognition method described in Embodiment 1 of this application. The specific implementation methods of each module of the device will not be repeated here. Please refer to the specific implementation methods of the corresponding steps in the method embodiment.

[0103] The text recognition device disclosed in this application improves the accuracy of text recognition for complex text images such as curved text images and seal images by inputting a target image into a convolutional neural network in a pre-trained character recognition model. The device obtains a feature map with height D and width n output by the convolutional neural network for the target image, where D and n are integers greater than 1. Based on D groups of features corresponding to the same width position but different height positions in the feature map, feature sub-sequences corresponding to the width positions are obtained. These feature sub-sequences are sequentially concatenated to form a feature sequence for the target image. The feature sequence is encoded and mapped using a sequence coding network in the character recognition model to obtain an encoded sequence. Finally, the encoded sequence is decoded using a CTC decoder in the character recognition model to obtain the character recognition result for the target image.

[0104] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the apparatus embodiments, since they are fundamentally similar to the method embodiments, the description is relatively simple; relevant parts can be referred to the descriptions in the method embodiments.

[0105] The above provides a detailed description of a text recognition method and apparatus provided in this application. Specific examples have been used to illustrate the principles and implementation methods of this application. The description of the above embodiments is only for the purpose of helping to understand the method and its core idea. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the idea of ​​this application. Therefore, the content of this specification should not be construed as a limitation of this application.

[0106] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0107] The various component embodiments of this application can be implemented in hardware, or as software modules running on one or more processors, or a combination thereof. Those skilled in the art will understand that microprocessors or digital signal processors (DSPs) can be used in practice to implement some or all of the functions of some or all of the components in the electronic device according to the embodiments of this application. This application can also be implemented as a device or apparatus program (e.g., a computer program and computer program product) for performing part or all of the methods described herein. Such a program implementing this application can be stored on a computer-readable medium, or can be in the form of one or more signals. Such signals can be downloaded from an Internet website, provided on a carrier signal, or provided in any other form.

[0108] For example, Figure 9An electronic device is shown that can implement the methods according to this application. The electronic device can be a PC, mobile terminal, personal digital assistant, tablet computer, etc. The electronic device conventionally includes a processor 910 and a memory 920, and program code 930 stored on the memory 920 and executable on the processor 910. When the processor 910 executes the program code 930, it implements the methods described in the above embodiments. The memory 920 can be a computer program product or a computer-readable medium. The memory 920 can be an electronic memory such as flash memory, EEPROM (Electrically Erasable Programmable Read-Only Memory), EPROM, hard disk, or ROM. The memory 920 has a storage space 9201 for the program code 930 of a computer program for performing any of the method steps described above. For example, the storage space 9201 for the program code 930 can include various computer programs for implementing the various steps in the above methods. The program code 930 is computer-readable code. These computer programs can be read from or written to one or more computer program products. These computer program products include program code carriers such as hard disks, CDs, memory cards, or floppy disks. The computer program includes computer-readable code that, when executed on an electronic device, causes the electronic device to perform the method according to the above embodiments.

[0109] This application also discloses a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the steps of the text recognition method as described in this application.

[0110] Such a computer program product can be a computer-readable storage medium, which can have the same characteristics as... Figure 9 The memory 920 in the illustrated electronic device is similarly arranged as storage segments, storage spaces, etc. Program code can be stored, for example, in a compressed form on the computer-readable storage medium. The computer-readable storage medium is typically as shown in the reference... Figure 10 The portable or fixed storage unit is described above. Typically, the storage unit includes computer-readable code 930', which is code read by a processor and, when executed by the processor, implements the various steps of the method described above.

[0111] The terms "an embodiment," "embodiment," or "one or more embodiments" as used herein mean that a particular feature, structure, or characteristic described in connection with an embodiment is included in at least one embodiment of this application. Furthermore, please note that the examples of the phrase "in one embodiment" do not necessarily all refer to the same embodiment.

[0112] Numerous specific details are set forth in the specification provided herein. However, it will be understood that embodiments of this application may be practiced without these specific details. In some instances, well-known methods, structures, and techniques have not been shown in detail so as not to obscure the understanding of this specification.

[0113] In the claims, any reference signs placed between parentheses should not be construed as limiting the claims. The word "comprising" does not exclude the presence of elements or steps not listed in the claims. The word "a" or "an" preceding an element does not exclude the presence of a plurality of such elements. This application can be implemented by means of hardware comprising several different elements and by means of a suitably programmed computer. In a unit claim enumerating several means, several of these means may be embodied by the same item of hardware. The use of the words first, second, and third, etc., does not indicate any order. These words can be interpreted as names.

[0114] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application.

Claims

1. A text recognition method, characterized in that, include: The target image is input into a convolutional neural network in a pre-trained character recognition model to obtain a feature map with height D and width n output by the convolutional neural network for the target image, where D and n are integers greater than 1; the value of D is determined based on the radian of the text lines in the target image. Based on the D groups of features in the feature map that correspond to the same width position but different height positions, the features are sequentially concatenated in the target image in the order from top to bottom according to their corresponding positions in the target image, to obtain the feature subsequence corresponding to the width position; each feature vector of the feature map corresponds to a rectangular region of the target image, and the rectangular region has the same order as the corresponding column on the feature map from left to right; The feature subsequences corresponding to each of the width positions are sequentially concatenated to form the feature sequence of the target image; The feature sequence is encoded and mapped using a sequence coding network in the character recognition model to obtain the encoded sequence; The character recognition result of the target image is obtained by decoding the encoded sequence using the CTC decoder in the character recognition model.

2. The method according to claim 1, characterized in that, The step of obtaining the feature subsequence corresponding to the width position based on D groups of features in the feature map that correspond to the same width position but different height positions includes: For the D groups of features corresponding to the same width position in the feature map, the D groups of features are concatenated from front to back according to the top-to-bottom order of their positions in the target image to obtain the feature subsequence corresponding to the width position.

3. The method according to claim 1 or 2, characterized in that, Before inputting the target image into a convolutional neural network in a pre-trained character recognition model and obtaining a feature map with height D and width n output by the convolutional neural network for the target image, the method further includes: Training the character recognition model, wherein the training samples used to train the character recognition model are: labeled sample images, where the labels are the true encoding sequences of the characters in the sample images, and the convolutional neural network is configured to output a feature map of height D. Training the character recognition model includes: performing the following character recognition steps on each training sample to obtain the predicted encoding sequence of the character in the corresponding sample image: The sample image is input into the convolutional neural network in the character recognition model to obtain a sample feature map with height D and width n output by the convolutional neural network for the sample image; Based on the D groups of features in the sample feature map that correspond to the same width position but different height positions, a sample feature subsequence corresponding to the width position is obtained; The sample feature subsequences corresponding to each of the width positions are sequentially concatenated to form the feature sequence of the sample image; The feature sequence is encoded and mapped using the sequence coding network to obtain the sample encoded sequence; The CTC decoder decodes the sample encoding sequence to obtain the predicted encoding sequence of the characters in the sample image; The character recognition model is iteratively trained by calculating the CTC loss based on the predicted encoding sequence and the real encoding sequence of the characters in the sample images of each training sample, and by optimizing the CTC loss.

4. The method according to claim 3, characterized in that, The sample images include one or more of the following: stamp images, arc-shaped text line images, and multi-line text images.

5. A text recognition device, characterized in that, include: The feature map acquisition module is used to input the target image into the convolutional neural network in the pre-trained character recognition model, and acquire the feature map with height D and width n output by the convolutional neural network for the target image, where D and n are integers greater than 1; the value of D is determined according to the radian of the text lines in the target image; The first feature conversion module is used to sequentially concatenate D groups of features in the feature map that correspond to the same width position but different height positions, according to the top-to-bottom order of the positions of each group of features in the target image, to obtain a feature subsequence corresponding to the corresponding width position; each feature vector of the feature map corresponds to a rectangular region of the target image, and the rectangular region has the same order as the corresponding column on the feature map from left to right; The second feature conversion module is used to sequentially concatenate the feature subsequences corresponding to each of the width positions into the feature sequence of the target image; The feature encoding module is used to encode and map the feature sequence through the sequence encoding network in the character recognition model to obtain the encoded sequence; The decoding output module is used to decode the encoded sequence through the CTC decoder in the character recognition model to obtain the character recognition result of the target image.

6. The apparatus according to claim 5, characterized in that, The first feature conversion module is further configured to: For the D groups of features corresponding to the same width position in the feature map, the D groups of features are concatenated from front to back according to the top-to-bottom order of their positions in the target image to obtain the feature subsequence corresponding to the width position.

7. The apparatus according to claim 5 or 6, characterized in that, The device further includes: A character recognition model training module is used to train the character recognition model, wherein the training samples used to train the character recognition model are: labeled sample images, where the labels are the true encoding sequences of characters in the sample images, and the convolutional neural network is configured to output a feature map of height D. The character recognition model training module is further used for: For each of the training samples, perform the following character recognition steps to obtain the predicted encoding sequence of the character in the corresponding sample image: The sample image is input into the convolutional neural network in the character recognition model to obtain a sample feature map with height D and width n output by the convolutional neural network for the sample image; Based on the D groups of features in the sample feature map that correspond to the same width position but different height positions, a sample feature subsequence corresponding to the width position is obtained; The sample feature subsequences corresponding to each of the width positions are sequentially concatenated to form the feature sequence of the sample image; The feature sequence is encoded and mapped using the sequence coding network to obtain the sample encoded sequence; The CTC decoder decodes the sample encoding sequence to obtain the predicted encoding sequence of the characters in the sample image; The character recognition model is iteratively trained by calculating the CTC loss based on the predicted encoding sequence and the real encoding sequence of the characters in the sample images of each training sample, and by optimizing the CTC loss.

8. An electronic device, comprising a memory, a processor, and program code stored in the memory and executable on the processor, characterized in that, When the processor executes the program code, it implements the text recognition method according to any one of claims 1 to 4.

9. A computer-readable storage medium having program code stored thereon, characterized in that, When the program code is executed by the processor, it implements the steps of the text recognition method according to any one of claims 1 to 4.

Images