Multi-class text sequence recognition method and device, electronic device

Abstract

The application discloses a multi-class text sequence recognition method and device, and belongs to the technical field of character recognition. The method is executed through a text sequence recognition model comprising at least one feature extraction layer, a feature mapping layer and a transcription layer. The feature mapping layer comprises M convolution layers arranged in parallel. The method comprises the following steps: performing feature extraction on an input image through the feature extraction layer to obtain N feature maps; distributing the N feature maps to the M convolution layers for feature mapping to obtain grouped feature mapping results; merging the grouped feature mapping results to obtain feature mapping results of a predetermined number of character categories corresponding to the N feature maps; and performing transcription on the feature mapping results to obtain a text sequence recognition result. The feature mapping layer is improved to a plurality of convolution layers in the method, the feature maps output by the feature extraction layer are grouped and subjected to feature mapping, and then the feature mapping results are merged, so that the number of parameters of the text sequence recognition model is reduced, and the model training and inference speed is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of character recognition, in particular to a multi-class text sequence recognition method and device, an electronic device, and a computer readable storage medium. BACKGROUND

[0002] In the prior art, a text sequence recognition model is usually constructed based on a CRNN network. CRNN stands for Convolutional Recurrent Neural Network, which is mainly used for end-to-end recognition of indefinite-length text sequences, without cutting individual characters first, but converting text recognition into a sequence learning problem with time-dependent.

[0003] In the prior art, a text sequence recognition model using CRNN technology usually includes a convolutional layer, a fully connected layer, and a transcription layer. The convolutional layer is used to extract features from an image input into the text sequence recognition model, and each neuron extracts features. The fully connected layer is used to classify and map the features extracted by the convolutional layer to obtain a feature mapping result. The transcription layer is used to transcribe the feature mapping result output by the fully connected layer to output a recognition result corresponding to the label space. In the prior art, the fully connected layer processes the features extracted by each neuron of the convolutional layer to obtain a classification mapping result, which requires the fully connected layer to have more network parameters, thereby causing the text sequence recognition model to have at least the following defects: large amount of calculation during operation, slow training and inference speed.

[0004] It can be seen that the multi-class text sequence recognition method in the prior art still needs to be improved. SUMMARY

[0005] The embodiments of the present application provide a multi-class text sequence recognition method and device, and an electronic device, to improve the problem of large amount of calculation during model operation, slow training and inference speed caused by too many parameters of the fully connected layer of the text sequence recognition model.

[0006] In a first aspect, the embodiments of the present application provide a multi-class text sequence recognition method, comprising:

[0007] The pre-trained text sequence recognition model includes at least one feature extraction layer and a feature mapping layer, wherein the feature mapping layer includes M convolutional layers arranged in parallel, and the method comprises:

[0008] extracting features of the input image through the at least one feature extraction layer to obtain N feature maps;

[0009] distributing the N feature maps to the M convolution layers for feature mapping to obtain a grouped feature mapping result output by each of the convolution layers, wherein M and N are positive integers, and M is less than or equal to N;

[0010] merging the obtained grouped feature mapping results to obtain a feature mapping result corresponding to a preset number of character categories for the N feature maps;

[0011] transcribing the feature mapping result to obtain a text sequence recognition result of the input image.

[0012] In a second aspect, an embodiment of the present application provides a multi-category text sequence recognition device, comprising:

[0013] The pre-trained text sequence recognition model comprises at least one feature extraction layer and a feature mapping layer, wherein the feature mapping layer comprises M convolution layers arranged in parallel, and the device comprises:

[0014] a feature extraction module configured to extract features of an input image through the at least one feature extraction layer to obtain N feature maps;

[0015] a grouped feature mapping result acquisition module configured to distribute the N feature maps to the M convolution layers for feature mapping to obtain a grouped feature mapping result output by each of the convolution layers, wherein M and N are positive integers, and M is less than or equal to N;

[0016] a merging module configured to merge the obtained grouped feature mapping results to obtain a feature mapping result corresponding to a preset number of character categories for the N feature maps;

[0017] a transcription recognition module configured to transcribe the feature mapping result to obtain a text sequence recognition result of the input image.

[0018] In a third aspect, an embodiment of the present application further discloses an electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the multi-category text sequence recognition method of the embodiment of the present application.

[0019] In a fourth aspect, an embodiment of the present application provides a computer readable storage medium having a computer program stored thereon, wherein the program is executable by a processor to implement the steps of the multi-category text sequence recognition method disclosed in the embodiment of the present application.

[0020] The multi-class text sequence recognition method disclosed by the embodiments of the present application is executed through a text sequence recognition model, the text sequence recognition model comprises at least one feature extraction layer, a feature mapping layer and a transcription layer, wherein the feature mapping layer comprises M convolution layers arranged in parallel, the method performs feature extraction on an input image through the feature extraction layer to obtain N feature maps; then, the N feature maps are distributed to the M convolution layers for feature mapping to obtain a grouped feature mapping result output by each convolution layer, wherein M and N are positive integers, and M is less than or equal to N; then, the obtained grouped feature mapping results are combined to obtain feature mapping results of a preset number of character categories corresponding to the N feature maps; finally, the feature mapping results are transcribed to obtain a text sequence recognition result of the input image. The above scheme improves the feature mapping layer into a plurality of convolution layers, performs feature mapping on the feature maps output by the feature extraction layer in groups, then combines the feature mapping results, and under the premise of obtaining the same output result as the full connection layer in the prior art, effectively reduces the number of parameters of the text sequence recognition model, and helps to improve the model training and inference speed.

[0021] The above description is only a summary of the technical solutions of the present application. In order to enable the technical means of the present application to be more clearly understood, and to be implemented according to the content of the description, and in order to enable the above and other purposes, characteristics and advantages of the present application to be more apparent and easy to understand, the following specific embodiments of the present application are described. BRIEF DESCRIPTION OF DRAWINGS

[0022] In order to make the purposes, technical solutions and advantages of the embodiments of the present application more clear, the technical solutions in the embodiments of the present application will be described clearly and completely below in combination with the drawings of the embodiments of the present application. Obviously, the described embodiments are only some of the embodiments of the present application, not all. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor fall within the scope of protection of the present application.

[0023] Figure 1 is a flowchart of the multi-class text sequence recognition method of the embodiments of the present application;

[0024] Figure 2 is a structural schematic diagram of a text recognition model of the multi-class text sequence recognition method of the embodiments of the present application;

[0025] Figure 3 is a structural schematic diagram of a text recognition model of the multi-class text sequence recognition method of the multi-class text sequence recognition method of the embodiments of the present application;

[0026] Figure 4 is a structural schematic diagram of a multi-class text sequence recognition device in the embodiments of the present application;

[0027] Figure 5A block diagram schematically illustrates an electronic device for performing the method according to this application; and

[0028] Figure 6 A storage unit for holding or carrying program code implementing the method according to this application is illustrated schematically. Detailed Implementation

[0029] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0030] This application discloses a multi-category text sequence recognition method, such as... Figure 1 As shown, the method includes steps 110 to 140.

[0031] The multi-category text sequence recognition method disclosed in this application is implemented through a text sequence recognition model with a specific structure. Before performing text recognition, the text sequence recognition model with this specific structure needs to be trained in advance.

[0032] like Figure 2 As shown, the pre-trained text sequence recognition model includes at least one feature extraction layer 210, a feature mapping layer 220, a merging layer 230, and a transcription layer 240, wherein the feature mapping layer 220 includes M parallel convolutional layers 2201. Here, M is an integer greater than 1.

[0033] The value of M is determined based on experimental results.

[0034] The following section, in conjunction with the structure of the text sequence recognition model, further elaborates on the specific technical solutions of the multi-category text sequence recognition method disclosed in the embodiments of this application.

[0035] Step 110: Extract features from the input image through the at least one feature extraction layer to obtain N feature maps.

[0036] Optionally, the input image can be a text line image, which includes a text sequence.

[0037] The at least one feature extraction layer 210 is connected in sequence, the first feature extraction layer 210 performs feature extraction on the input image to obtain a feature map of a first size, and then the second feature extraction layer 210 performs feature extraction on the feature map of the first size to obtain a feature map of a second size. After each feature extraction layer 210 processes the feature map output by the previous feature extraction layer 210, the last feature extraction layer 210 inputs N feature maps of a specified size.

[0038] In some embodiments of the present application, the feature extraction layer includes a convolution layer, a Batch Normalization layer and an activation function. The convolution layer performs convolution operation on the input image input to the feature extraction layer or the feature map output by the previous feature extraction layer based on a preset convolution kernel and step length to obtain a feature map of a specified size; the Batch Normalization layer is used to normalize the feature map of the specified size obtained by the convolution layer to stabilize the feature distribution of the feature map; and the activation function is used to perform nonlinear mapping on the feature map processed by the Batch Normalization layer to obtain the hidden layer output vector of the current feature extraction layer 210.

[0039] Taking a text sequence recognition model including two feature extraction layers 210 as an example, for an input image of a size of CxHxW, where C represents the number of channels, H represents the height of the input image, and W represents the width of the input image, the first feature extraction layer 210 can obtain 64 feature maps of a size of 64xH / 2xW / 2 after performing feature extraction on the input image, and the second feature extraction layer 210 can obtain 512 feature maps of a size of 512xH / 4xW / 4 after performing feature extraction on the 64 feature maps of the size of 64xH / 2xW / 2.

[0040] Optionally, when the network structure of the feature extraction layer 210 changes, for example, the convolution kernel size or the step length of the convolution layer changes, the size and number of the output feature map will change accordingly, and in the embodiments of the present application, the number and size of the feature map output by the feature extraction layer 210 are not limited. That is, in the embodiments of the present application, the convolution kernel size, convolution step length and other structural parameters of the convolution layer in the feature extraction layer 210 are not limited.

[0041] In some embodiments of the present application, the activation function can be implemented by using a GELU (Gaussian Error Linear Unit) function.

[0042] ​​​In the prior art text recognition model, the ReLU (The Rectified Linear Unit) activation function is usually used after the batch normalization layer in the feature extraction layer to map the feature map output by the batch normalization layer to the output space of the hidden layer neurons. The expression of relu(x) = max(x, 0) is: relu(x) = max(x, 0). The ReLU activation function maps the data less than 0 to 0 and keeps the data greater than 0 unchanged. Since the derivative of the ReLU activation function is 0 when x <= 0, the part of values will not affect the network parameter update in the training process, which affects the performance of the neural network to some extent. For example, the inventors found through experiments that when processing text images, the small changes between the features of characters with similar shapes, such as the small changes between the number "1" and the lowercase character "l" (the lowercase form of the character "L"), are easily ignored, resulting in text recognition errors.

[0043] The expression of the GELU activation function is: GELU(x) = x x (x), where (x) refers to the Gaussian normal distribution cumulative function of x. The GELU activation function has the feature of mapping in combination with the context, which can take into account the small changes between the features of characters with similar shapes to better learn the language rules.

[0044] Due to the characteristics of the convolutional layer based on pixel analysis characters, for characters with similar shapes, the language rules learned by combining the GELU function can improve the accuracy of text sequence recognition by considering the pixel features of the context where the characters are located in the GELU function. For example, the character "1" in the text sequence "0123" will be more easily recognized as the number "1" rather than the lowercase letter of the character "L".

[0045] Step 120, distributing the N feature maps to the M convolutional layers for feature mapping to obtain a grouped feature mapping result output by each convolutional layer.

[0046] Wherein, M and N are positive integers, and M is less than or equal to N.

[0047] Next, the N feature maps output by the last feature extraction layer 210 are distributed to the M convolutional layers 2201, so that each convolutional layer 2201 is used to map the features of the feature extraction layer 210, and each convolutional layer 2201 only processes part of the feature maps output by the feature extraction layer 210, so that the speed of feature mapping of the text sequence recognition model can be improved.

[0048] In some embodiments of the present application, the distributing the N feature maps to the M convolutional layers for feature mapping to obtain a grouped feature mapping result output by each convolutional layer comprises: dividing the N feature maps into M groups according to a preset correspondence between the feature maps output by the feature extraction layer and the convolutional layers to obtain a group of feature maps corresponding to each convolutional layer; and for each group of feature maps, performing feature mapping on the group of feature maps by the corresponding convolutional layer to obtain a grouped feature mapping result mapped to a specified character category, wherein the specified character category is part of the preset number of character categories preconfigured for the corresponding convolutional layer.

[0049] The preset number of character categories is a label category of the text sequence recognition model. For example, if the text sequence recognition model needs to recognize a character set containing 10,000 characters, the preset number of character categories is equal to 10,000. That is, the preset number of character categories is the character category that the text sequence recognition model needs to recognize.

[0050] Optionally, during the model design phase, a preset correspondence between the feature maps output by the feature extraction layer 210 and the M convolutional layers 2201 can be preconfigured, so that each convolutional layer 2201 processes part of the feature maps output by the feature extraction layer 210 according to the preset correspondence.

[0051] Optionally, the preset correspondence between the feature maps output by the feature extraction layer and the convolutional layers comprises a correspondence in which each feature map output by the feature extraction layer is input to a convolutional layer. For example, when the feature extraction layer 210 outputs N feature maps, the N feature maps can be divided into M groups, each group including different feature maps, and each group of feature maps is set to correspond to a specified convolutional layer 2201, so that the N feature maps are distributed to the M convolutional layers 2201 for feature mapping processing. Wherein N is an integer greater than 1, and the value of N is determined according to the network structure and parameters of the feature extraction layer 210. For example, N can take the value of 512, or other integer values.

[0052] In the embodiments of the present application, the specific number of feature maps included in each group is not limited.

[0053] Optionally, during the model design phase, a classification mapping space of the feature mapping result output by each convolutional layer 2201 can also be preconfigured, and each convolutional layer 2201 is configured to perform feature mapping only in a specified character category space. For example, the preset number of character categories can be divided into M groups, and each group of character categories is configured as a classification mapping space of a convolutional layer 2201.

[0054] Afterwards, for each of the convolutional layers 2201, the feature map corresponding to the convolutional layer 2201 is mapped by the convolutional layer 2201 to obtain a grouped feature mapping result output by the convolutional layer 2201. In this way, the M convolutional layers can obtain M grouped feature mapping results in total, each of which corresponds to part of the character categories in the recognition category space of the text sequence recognition model, and the M grouped feature mapping results can cover the entire category space of the text sequence recognition model.

[0055] The embodiment of the present application can greatly reduce the number of parameters of the model by improving the feature mapping layer equivalent to the fully connected layer into a network structure of multiple parallel convolutional layers, thereby reducing the amount of computation of feature mapping in the training process and the inference process and improving the running speed of the model. Taking the replacement of the fully connected layer with M convolutional layers as an example, if the dimension of the feature map output by the feature extraction layer 210 is (512, 1, W / 4), where 512 is the number of channels, 1 is the height of the feature map, and W / 4 is the width of the feature map, after the feature map is evenly divided into M groups according to the channels (it is assumed that 512 can be divided by M here), the input dimension of each convolutional layer can be represented as (512 / M, 1, W / 4), and the output dimension is (D / M, 1, W / 4), where D is the number of categories that can be recognized by the text sequence recognition model, M is the number of convolutional layers, and W is the output image width. Based on this structure parameter, the number of parameters of the M convolutional layers can be calculated as (512 / M) x (D / M) x M, that is, 512 x D / M.

[0056] However, in the prior art, as shown in Figure 3 The text sequence recognition model usually includes at least one feature extraction layer 310, a fully connected layer 320, and a transcription layer 330, where all neurons of the fully connected layer 320 need to be connected with all neurons of the previous layer, which increases the complexity of correlation modeling and increases the number of parameters. Taking the dimension of the feature map input to the fully connected layer 320 as (512, 1, W / 4) and the feature dimension output by the fully connected layer as (D, 1, W / 4) as an example, the number of parameters of the fully connected layer 320 is 512 x D, and the number of parameters is positively correlated with D. The larger the number of categories that can be recognized by the text sequence recognition model, the more parameters the fully connected layer has. A large number of parameters means a large amount of computation, which has a loss on the model speed. Moreover, in the case where D is large enough, the correlation between categories is gradually weakened, which is not conducive to the generalization of the model.

[0057] In the case of the same number of categories D, compared with the full connection layer in the prior art, the network structure design of the feature mapping layer disclosed in the embodiments of the present application can reduce the parameter quantity (512 x D-512 x D / M). The larger M is, the more the parameter quantity of the feature mapping layer is reduced. The network structure design of the feature mapping layer disclosed in the embodiments of the present application not only can reduce the network parameters of the feature mapping layer and improve the operation speed of the model, but also is helpful for modeling the correlation between categories of each convolutional layer and for generalization of the model.

[0058] In the embodiments of the present application, the plurality of convolutional layers 2201 included in the feature mapping layer 220 are mutually independent convolutional layers, and the convolutional layers 2201 do not share parameters.

[0059] In some embodiments of the present application, when the N feature maps are distributed to the M convolutional layers for feature mapping, M groups of feature maps can also be generated from the N feature maps, wherein the number of feature maps in each group is less than N, the intersection of the M groups of feature maps is equal to the N feature maps, and one or more feature maps can be divided into multiple feature map combinations. Then, each group of feature maps is sent to a convolutional layer for feature mapping.

[0060] Step 130, merging the obtained grouping feature mapping results to obtain the feature mapping results of the N feature maps corresponding to a predetermined number of character categories.

[0061] The foregoing step of distributing the N feature maps to the M convolutional layers for feature mapping can obtain M grouping feature mapping results, each corresponding to a pre-configured group of character categories, and the M grouping feature mapping results can cover the entire character category space that can be recognized by the text sequence recognition model.

[0062] Next, the M grouping feature mapping results obtained are merged by the merging layer 230 to obtain the feature mapping results of the N feature maps corresponding to a predetermined number of character categories.

[0063] As described above, some embodiments of the present application, in which part of the predetermined number of character categories corresponding to the grouping feature mapping results, the merging of the obtained grouping feature mapping results to obtain the feature mapping results of the N feature maps corresponding to a predetermined number of character categories includes: merging the grouping feature mapping results according to the character categories corresponding to the grouping feature mapping results to obtain the feature mapping results of the N feature maps corresponding to a predetermined number of character categories.

[0064] Taking the dimension of the grouped feature mapping result output by each convolutional layer 2201 as an example, the dimension is represented as (D / M, 1, W / 4), where D is the total number of character categories that can be recognized by the text sequence recognition model, M is the number of convolutional layers 2201, and W is the output image width. After the M grouped feature mapping results are merged in the mapping category dimension by the merging layer 230, a feature mapping result with a dimension of (D, 1, W / 4) can be obtained. The feature mapping result obtained after merging can cover D mapping categories. In the embodiments of the present application, the feature mapping result obtained after merging is taken as the feature mapping result corresponding to the preset number of character categories for the N feature maps.

[0065] At step 140, the feature mapping result is transcribed to obtain the text sequence recognition result of the input image.

[0066] Next, the feature mapping result corresponding to the preset number of character categories obtained after merging is input to the transcription layer 240, and the transcription layer 240 transcribes the feature mapping result to obtain the text sequence recognition result of the input image.

[0067] The specific implementation of transcribing the feature mapping result by the transcription layer to obtain the text sequence recognition result of the input image is prior art, and will not be described in detail in the embodiments of the present application.

[0068] During the training process of the text sequence recognition model, the above steps 110 to 140 are performed for each training sample to obtain the prediction result corresponding to the training sample. Then, the model error is calculated based on the prediction result and the sample label (i.e., the true value of the text sequence corresponding to the training sample), and the model parameters are optimized by using the gradient update method to perform iterative training of the model.

[0069] The training method of the text sequence recognition model is described in the prior art, and will not be described in detail in the embodiments of the present application.

[0070] For a large number of training samples, the use of the feature mapping layer with the foregoing structure to reduce the model parameters can effectively improve the convergence speed of the model, thereby improving the training speed of the model. On the other hand, due to the use of grouped feature mapping, the correlation between the categories and the features is stronger, which helps to improve the generalization of the model.

[0071] The multi-class text sequence recognition method disclosed by the embodiments of the present application is executed through a text sequence recognition model, and the text sequence recognition model comprises at least one feature extraction layer, a feature mapping layer and a transcription layer, wherein the feature mapping layer comprises M convolution layers arranged in parallel, the method performs feature extraction on an input image through the feature extraction layer to obtain N feature maps; then the N feature maps are distributed to the M convolution layers for feature mapping to obtain a grouped feature mapping result output by each convolution layer, wherein M and N are positive integers, and M is less than or equal to N; then the obtained grouped feature mapping results are combined to obtain a feature mapping result corresponding to a preset number of character categories for the N feature maps; finally, the feature mapping result is transcribed to obtain a text sequence recognition result of the input image. The above scheme improves the feature mapping layer into a plurality of convolution layers, performs feature mapping on the feature maps output by the feature extraction layer in groups, then combines the feature mapping results, and under the premise of obtaining the same output result as the full connection layer in the prior art, effectively reduces the number of parameters of the text sequence recognition model, and helps to improve the model training and inference speed.

[0072] In a multi-class text sequence scenario, for example, in a text sequence scenario supporting the GB18030 character set, the text sequence recognition model needs to perform classification mapping of nearly 20,000 categories. Using the feature mapping layer disclosed in the embodiments of the present application for feature mapping can greatly reduce the model parameters of the text sequence recognition model compared with using the full connection layer network structure in the prior art, thereby improving the inference and training speed. On the other hand, using the feature mapping layer disclosed in the embodiments of the present application for feature grouping mapping reduces the character category space corresponding to one feature mapping, so that the relevance between the categories and the features is stronger, which helps to improve the generalization of the model.

[0073] The embodiments of the present application also disclose a multi-class text sequence recognition device, wherein a pre-trained text sequence recognition model comprises at least one feature extraction layer and a feature mapping layer, wherein the feature mapping layer comprises M convolution layers arranged in parallel, as shown in Figure 4 The device comprises:

[0074] The feature extraction module 410 is configured to perform feature extraction on an input image through the at least one feature extraction layer to obtain N feature maps.

[0075] The grouped feature mapping result acquisition module 420 is configured to distribute the N feature maps to the M convolution layers for feature mapping to obtain a grouped feature mapping result output by each convolution layer, wherein M and N are positive integers, and M is less than or equal to N.

[0076] The merging module 430 is configured to merge the obtained grouping feature mapping results to obtain feature mapping results of the N feature maps corresponding to the preset plurality of character categories.

[0077] The transcription recognition module 440 is configured to transcribe the feature mapping results to obtain a text sequence recognition result of the input image.

[0078] Optionally, the grouping feature mapping result acquisition module 420 is further configured to:

[0079] According to the preset correspondence between the feature maps output by the feature extraction layer and the convolution layers, the N feature maps are divided into M groups to obtain a group of feature maps corresponding to each convolution layer.

[0080] For each group of feature maps, the corresponding convolution layer is used to perform feature mapping on the group of feature maps to obtain a grouping feature mapping result mapped to a specified character category, wherein the specified character category is part of the preset plurality of character categories preconfigured for the corresponding convolution layer.

[0081] Optionally, the preset correspondence between the feature maps output by the feature extraction layer and the convolution layers comprises:

[0082] The correspondence that each feature map output by the feature extraction layer is used as an input of a convolution layer.

[0083] Optionally, the part of the plurality of character categories corresponding to the grouping feature mapping result, the merging module 430 is further configured to:

[0084] According to the character categories corresponding to the grouping feature mapping results, the grouping feature mapping results are merged to obtain feature mapping results of the N feature maps corresponding to the preset plurality of character categories.

[0085] Optionally, the feature extraction layer comprises a convolution layer, a batch normalization layer and an activation function, and the activation function is implemented by using a GELU function.

[0086] The multi-category text sequence recognition device disclosed in the embodiments of the present application is used to implement the multi-category text sequence recognition method described in the embodiments of the present application, and the specific implementation manners of the modules of the device will not be described again, and can be referred to the specific implementation manners of the corresponding steps of the method embodiments.

[0087] The multi-class text sequence recognition device disclosed by the embodiments of the present application is executed through a text sequence recognition model, and the text sequence recognition model comprises at least one feature extraction layer, a feature mapping layer and a transcription layer, wherein the feature mapping layer comprises M convolution layers arranged in parallel, the device extracts features of an input image through the feature extraction layer to obtain N feature maps; then, the N feature maps are distributed to the M convolution layers for feature mapping to obtain a grouped feature mapping result output by each convolution layer, wherein M and N are positive integers, and M is less than or equal to N; then, the obtained grouped feature mapping results are combined to obtain a feature mapping result corresponding to a plurality of preset character categories for the N feature maps; finally, the feature mapping result is transcribed to obtain a text sequence recognition result of the input image. The above scheme improves the feature mapping layer into a plurality of convolution layers, groups the feature maps output by the feature extraction layer for feature mapping, and then combines the feature mapping results, thereby effectively reducing the number of parameters of the text sequence recognition model on the premise of obtaining the same output result as the full connection layer in the prior art, and helping to improve the model training and inference speed.

[0088] In a multi-class text sequence scenario, for example, in a text sequence scenario supporting the GB18030 character set, the text sequence recognition model needs to perform classification mapping of nearly 20,000 categories. Using the feature mapping layer disclosed in the embodiments of the present application for feature mapping can greatly reduce the model parameters of the text sequence recognition model compared with using the full connection layer network structure in the prior art, thereby improving the inference and training speed. On the other hand, using the feature mapping layer disclosed in the embodiments of the present application for feature grouping mapping reduces the character category space corresponding to one feature mapping, so that the relevance between the categories and the features is stronger, which helps to improve the generalization of the model.

[0089] Each of the embodiments in the specification is described in a progressive manner, and each embodiment focuses on the difference from other embodiments. The same and similar parts between the embodiments can be referred to each other. For the device embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and the relevant parts can be referred to the part of the method embodiments.

[0090] The multi-class text sequence recognition method and device provided by the present application are described in detail above, and the principles and implementation modes of the present application are described by applying specific examples. The above embodiment description is only used to help understand the method and a core idea thereof; meanwhile, for those skilled in the art, according to the idea of the present application, the specific implementation mode and application range can be changed; and the above description should not be understood as a limitation of the present application.

[0091] The apparatus embodiments described above are only illustrative, wherein the units described as separate components can or can not be physically separate, and the components displayed as units can or can not be physical units, i.e., can be located in one place, or can be distributed to multiple network units. Part or all of the modules can be selected to achieve the purpose of the embodiment according to actual needs. Those skilled in the art can understand and implement without creative labor.

[0092] The various component embodiments of the present application can be implemented in hardware, or in software modules running on one or more processors, or in a combination thereof. Those skilled in the art will understand that a microprocessor or a digital signal processor (DSP) can be used in practice to implement some or all of the functions of some or all of the components in the electronic device according to the embodiments of the present application. The present application can also be implemented as a device or apparatus program (e.g., a computer program and a computer program product) for executing part or all of the methods described herein. Such a program implementing the present application can be stored on a computer readable medium or can have the form of one or more signals. Such signals can be downloaded from an Internet website, or provided on a carrier signal, or in any other form.

[0093] For example, Figure 5 An electronic device that can implement the methods according to the present application is shown. The electronic device can be a PC, a mobile terminal, a personal digital assistant, a tablet computer, etc. The electronic device traditionally comprises a processor 510 and a memory 520 and program code 530 stored on the memory 520 and executable on the processor 510, which when executed by the processor 510 implements the methods described in the above embodiments. The memory 520 can be a computer program product or a computer readable medium. The memory 520 can be an electronic memory such as a flash memory, an EEPROM (electrically erasable programmable read-only memory), an EPROM, a hard disk, or a ROM. The memory 520 has a storage space 5201 for program code 530 of computer programs for performing any of the method steps in the above methods. For example, the storage space 5201 for program code 530 can comprise individual computer programs for implementing the various steps in the above methods, respectively. The program code 530 is computer readable code. The computer programs can be read out from or written into one or more computer program products. The computer program products include program code carriers such as hard disks, compact discs (CDs), memory cards, or floppy disks. The computer programs include computer readable code which, when executed on the electronic device, causes the electronic device to perform the methods according to the above embodiments.

[0094] This application also discloses a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the steps of the multi-category text sequence recognition method as described in this application.

[0095] Such a computer program product can be a computer-readable storage medium, which can have the same characteristics as... Figure 5 The memory 520 in the illustrated electronic device is similarly arranged with storage segments, storage spaces, etc. Program code can be stored, for example, in a compressed form on the computer-readable storage medium. The computer-readable storage medium is typically as shown in the reference. Figure 6 The portable or fixed storage unit is described above. Typically, the storage unit includes computer-readable code 530', which is code read by a processor and, when executed by the processor, implements the various steps of the method described above.

[0096] The terms "an embodiment," "embodiment," or "one or more embodiments" as used herein mean that a particular feature, structure, or characteristic described in connection with an embodiment is included in at least one embodiment of this application. Furthermore, please note that the examples of the phrase "in one embodiment" do not necessarily all refer to the same embodiment.

[0097] Numerous specific details are set forth in the specification provided herein. However, it will be understood that embodiments of this application may be practiced without these specific details. In some instances, well-known methods, structures, and techniques have not been shown in detail so as not to obscure the understanding of this specification.

[0098] In the claims, any reference signs placed between parentheses should not be construed as limiting the claims. The word "comprising" does not exclude the presence of elements or steps not listed in the claims. The word "a" or "an" preceding an element does not exclude the presence of a plurality of such elements. This application can be implemented by means of hardware comprising several different elements and by means of a suitably programmed computer. In a unit claim enumerating several means, several of these means may be embodied by the same item of hardware. The use of the words first, second, and third, etc., does not indicate any order. These words can be interpreted as names.

[0099] Finally, it should be noted that the above examples are only used to illustrate the technical solutions of the present application, and are not intended to limit the same; although the present application has been described in detail with reference to the foregoing examples, those of ordinary skill in the art should understand that they can still modify the technical solutions recorded in the foregoing examples, or make equivalent replacements for some of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A method for recognizing multi-class text sequences, characterized in that, The pre-trained text sequence recognition model includes: at least one feature extraction layer and a feature mapping layer, wherein the feature mapping layer includes M parallel convolutional layers, and the method includes: The input image is processed by at least one feature extraction layer to extract features, resulting in N feature maps. The N feature maps are distributed to the M convolutional layers for feature mapping, resulting in grouped feature mapping results output by each convolutional layer, where M and N are positive integers, and M is less than or equal to N; The obtained grouped feature mapping results are merged to obtain the feature mapping results of the N feature maps corresponding to a number of preset character categories; The feature mapping result is transcribed to obtain the text sequence recognition result of the input image; The feature extraction layer includes a convolutional layer, a batch normalization layer, and an activation function, wherein the activation function is implemented using the GELU function.

2. The method according to claim 1, characterized in that, The step of distributing the N feature maps to the M convolutional layers for feature mapping, to obtain the grouped feature mapping result output by each convolutional layer, includes: Based on the preset correspondence between the feature maps output by the feature extraction layer and the convolutional layer, the N feature maps are divided into M groups to obtain a set of feature maps corresponding to each convolutional layer; For each set of feature maps, feature mapping is performed on the set of feature maps through the corresponding convolutional layer to obtain the grouped feature mapping result mapped to a specified character category, wherein the specified character category is: a portion of the preset character categories pre-configured for the corresponding convolutional layer.

3. The method according to claim 2, characterized in that, The preset correspondence between the feature maps output by the feature extraction layer and the convolutional layer includes: Each feature map output by the feature extraction layer corresponds to an input of the convolutional layer.

4. The method according to claim 1, characterized in that, The grouped feature mapping results correspond to a subset of character categories from the preset number of character categories. The process of merging the obtained grouped feature mapping results to obtain the feature mapping results of the N feature maps corresponding to the preset number of character categories includes: Based on the character categories corresponding to the grouped feature mapping results, the grouped feature mapping results are merged to obtain the feature mapping results of the N feature maps corresponding to a preset number of character categories.

5. A multi-category text sequence recognition device, characterized in that, The pre-trained text sequence recognition model includes: at least one feature extraction layer and a feature mapping layer, wherein the feature mapping layer includes M parallel convolutional layers, and the device includes: The feature extraction module is used to extract features from the input image through the at least one feature extraction layer to obtain N feature maps; The group feature mapping result acquisition module is used to distribute the N feature maps to the M convolutional layers for feature mapping, and obtain the group feature mapping result output by each convolutional layer, where M and N are positive integers, and M is less than or equal to N; The merging module is used to merge the obtained grouped feature mapping results to obtain the feature mapping results of the N feature maps corresponding to a preset number of character categories; The transcription recognition module is used to transcribe the feature mapping result to obtain the text sequence recognition result of the input image.

6. The apparatus according to claim 5, characterized in that, The grouping feature mapping result acquisition module is further used for: Based on the preset correspondence between the feature maps output by the feature extraction layer and the convolutional layer, the N feature maps are divided into M groups to obtain a set of feature maps corresponding to each convolutional layer; For each set of feature maps, feature mapping is performed on the set of feature maps through the corresponding convolutional layer to obtain the grouped feature mapping result mapped to a specified character category, wherein the specified character category is: a portion of the preset character categories pre-configured for the corresponding convolutional layer; The feature extraction layer includes a convolutional layer, a batch normalization layer, and an activation function, wherein the activation function is implemented using the GELU function.

7. The apparatus according to claim 5, characterized in that, The merging module is further configured to: (The grouping feature mapping result corresponds to a subset of character categories from the preset character categories.) Based on the character categories corresponding to the grouped feature mapping results, the grouped feature mapping results are merged to obtain the feature mapping results of the N feature maps corresponding to a preset number of character categories.

8. An electronic device, comprising a memory, a processor, and program code stored in the memory and executable on the processor, characterized in that, When the processor executes the program code, it implements the multi-category text sequence recognition method according to any one of claims 1 to 4.

9. A computer-readable storage medium having program code stored thereon, characterized in that, When the program code is executed by the processor, it implements the steps of the multi-category text sequence recognition method according to any one of claims 1 to 4.

Images