Character recognition method, device, computer equipment and storage medium

Abstract

The present application provides a character recognition method, apparatus, computer device, and storage medium, wherein the method comprises: obtaining an image to be recognized, wherein the image to be recognized contains characters to be recognized; in the process of extracting features from the image to be recognized through a feature extraction network, performing target vector replacement on at least one feature map extracted by the feature extraction network to obtain a target feature map corresponding to the image to be recognized; wherein the target vector refers to a vector arranged along the character direction in the feature map, and target vector replacement refers to replacing another target vector in the feature map with a target vector in the feature map; and performing character recognition based on the target feature map to determine the characters in the image to be recognized. This technical solution fully establishes the association relationship between characters and improves the accuracy of character recognition.

Description

Technical Field

[0001] The present application relates to the field of image recognition, and in particular to character recognition methods, devices, computer equipment, and storage media. Background Art

[0002] Optical character recognition (OCR) is the process by which an electronic device (such as a scanner or camera) detects characters printed on a paper document, determines their shape by detecting patterns of light and dark, and then translates these shapes into text using character recognition methods. Traditional OCR solutions typically first use projection methods to cut out individual characters, which are then fed into convolutional neural networks (CNNs) for classification.

[0003] With the development of technology, end-to-end OCR recognition solutions based on deep learning have been proposed. Instead of segmenting characters, character recognition is transformed into a sequence learning problem. Character segmentation is integrated into deep learning, and text recognition is performed directly on text images containing characters to determine the characters in the text image. End-to-end OCR recognition solutions based on deep learning mainly use convolutional feature extraction and sequence feature prediction, then translate and transcribe the predicted labels and output text. Because characters have certain correlations, establishing correlations between characters during character recognition can improve recognition efficiency. Therefore, how to establish correlations between characters has become a technical problem that needs to be solved urgently. Summary of the Invention

[0004] The present application provides a character recognition method, apparatus, computer equipment, and storage medium to establish associations between characters and improve the accuracy of character recognition.

[0005] In a first aspect, a character recognition method is provided, comprising:

[0006] Acquire an image to be recognized, wherein the image to be recognized includes characters to be recognized;

[0007] During feature extraction of the image to be recognized by the feature extraction network, target vector replacement is performed on at least one feature map extracted by the feature extraction network to obtain a target feature map corresponding to the image to be recognized; wherein the target vector refers to a vector arranged along the character direction in the feature map, and the target vector replacement refers to replacing another target vector in the feature map with a target vector in the feature map;

[0008] Character recognition is performed based on the target feature map to determine the characters in the image to be recognized.

[0009] In this technical solution, after obtaining the image to be recognized, in the process of extracting features from the image to be recognized through the feature extraction network, the target vector is replaced on the feature map obtained by the feature extraction network to obtain the target feature map corresponding to the image to be recognized, and then character recognition is performed based on the target feature map, thereby determining the characters in the image to be recognized. Among them, the target vector refers to the vector arranged along the character direction in the feature map, and the target vector replacement refers to replacing another target vector in the feature map with a target vector in the feature map. Replacing the target vector on the feature map during the feature extraction process means replacing the vector arranged along the character direction in the feature map during the feature extraction process. Since the vector arranged along the character direction can indicate part of the feature information of a certain character, replacing the vector arranged along the character direction in the feature map during the feature extraction process can establish a connection between characters, so that the target feature map contains the correlation between characters, which can improve the accuracy of character recognition.

[0010] In combination with the first aspect, in a possible implementation, the feature extraction network includes a plurality of feature extraction structures connected in sequence, each feature extraction structure includes at least one convolutional layer, wherein, in two adjacent feature extraction structures, the next feature extraction structure is used to extract features from the feature atlas output by the previous feature extraction structure; the target vector is replaced on at least one feature map extracted by the feature extraction network to obtain the target feature map corresponding to the image to be identified, including: obtaining a first feature map, the first feature map is a feature map output by the first feature extraction structure in the feature extraction network, the first feature map includes multiple feature maps, and the first feature extraction structure is a preset structure set. any feature extraction structure, the preset structure set includes at least one feature extraction structure in the feature extraction network; performing target vector replacement on at least one feature map in the first feature map set to update the first feature map set, and inputting the updated first feature map set into the second feature extraction structure to obtain a second feature map set, the second feature extraction structure being the next feature extraction structure connected to the first feature extraction structure, and the second feature map set being the feature map set output by the second feature extraction structure; determining the third feature map set or the updated third feature map set as the target feature map, and the third feature map set being the feature map set output by the last feature extraction structure in the feature extraction network. In the process of feature extraction, by performing target vector replacement on some feature maps output by some feature extraction structures to update the feature map, the feature map can contain both the feature information of the characters and the association information between the characters, thereby improving the accuracy of character recognition.

[0011] In conjunction with the first aspect, in one possible implementation, the target vector replacement of at least one feature map in the first feature map set to update the first feature map set includes: in the first feature map, for each first target vector, replacing the first target vector with a target vector adjacent to the first target vector to obtain an updated first feature map, the first feature map being the feature map in the first feature map set that needs to have its target vector replaced, and the first target vector being any target vector to be replaced in the first feature map. Since adjacent characters have a higher correlation, using adjacent target vectors for target vector replacement can better establish an association relationship between characters.

[0012] In combination with the first aspect, in a possible implementation, at least one feature map in the first feature map set includes 2n feature maps in the first feature map set, where n is a positive integer greater than or equal to 1; the target vector replacement of at least one feature map in the first feature map set to update the first feature map set includes: in the second feature map, for each second target vector, the second target vector is replaced by a target vector adjacent to the second target vector in the first character direction to obtain an updated second feature map, the second feature map is any feature map of n of the 2n feature maps, and the second target vector is any target vector to be replaced in the second feature map; in the third feature map, for each third target vector, the third target vector is replaced by a target vector adjacent to the third target vector in the second character direction to obtain an updated third feature map, the third feature map is any feature map of the other n feature maps of the 2n feature maps, and the third target vector is any target vector to be replaced in the third feature map; the first character direction and the second character direction are two opposite character directions. When performing target vector replacement, adjacent target vectors in two character directions are used for replacement, which can fully establish the association relationship between adjacent characters and help improve the accuracy of character recognition.

[0013] In combination with the first aspect, in a possible implementation, the n feature maps are the first n feature maps in the first feature map set, and the other n feature maps are the last n feature maps in the first feature map set.

[0014] In conjunction with the first aspect, in one possible implementation, the feature extraction network includes M feature extraction structures, the preset structure set includes the i-th feature extraction structure in the feature extraction network, 2≤i≤(M-1), and M is a positive integer greater than 4. By extracting feature extraction structures excluding the first and last two feature extraction structures in the feature extraction network, the feature graphs are all replaced with target vectors, thereby fully establishing the association relationship between characters.

[0015] In conjunction with the first aspect, in one possible implementation, before performing target vector replacement on at least one feature map extracted by the feature extraction network, the method further includes: determining the number of characters to be recognized contained in the image to be recognized; and determining the number of the at least one feature map based on the number of characters to be recognized. Determining the number of target vector replacements required by determining the number of characters to be recognized before performing target vector replacement enables more reasonable associations between characters, thereby improving character recognition accuracy.

[0016] In a second aspect, a character recognition device is provided, comprising:

[0017] An image acquisition module is used to acquire an image to be recognized, wherein the image to be recognized includes characters to be recognized;

[0018] a replacement module, configured to, during feature extraction of the image to be identified by the feature extraction network, perform target vector replacement on at least one feature map extracted by the feature extraction network to obtain a target feature map corresponding to the image to be identified; wherein the target vector refers to a vector arranged along the character direction in the feature map, and target vector replacement refers to replacing another target vector in the feature map with a target vector in the feature map;

[0019] The character determination module is used to perform character recognition based on the target feature map to determine the characters in the image to be recognized.

[0020] In a third aspect, a computer device is provided, comprising a memory and one or more processors, wherein the one or more processors are used to execute one or more computer programs stored in the memory, and when the one or more processors execute the one or more computer programs, the computer device implements the character recognition method of the first aspect.

[0021] In a fourth aspect, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores a computer program, and the computer program includes program instructions. When the program instructions are executed by a processor, the processor executes the character recognition method of the first aspect.

[0022] The present application can achieve the following technical effects: replacing the target vector of the feature map during the feature extraction process means replacing the vectors arranged along the character direction in the feature map during the feature extraction process. Since the vectors arranged along the character direction can indicate part of the feature information of a certain character, replacing the vectors arranged along the character direction in the feature map during the feature extraction process can establish a connection between characters, so that the target feature map contains the correlation between characters, which can improve the accuracy of character recognition. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] Figure 1 This is a schematic diagram of the architecture of an OCR recognition system;

[0024] Figure 2 A flowchart of a character recognition method provided in an embodiment of the present application;

[0025] Figure 3 The process of replacing the target vector with the feature map is shown;

[0026] Figure 4 A schematic diagram of a specific configuration of a feature extraction network provided in an embodiment of the present application;

[0027] Figure 5 1 is a structural diagram of a character recognition device provided in an embodiment of the present application;

[0028] Figure 6 It is a structural diagram of a computer device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0029] The technical solutions in the embodiments of the present application will be described below in conjunction with the drawings in the embodiments of the present application.

[0030] The technical solution of the present application can be applied to various OCR recognition scenarios, and in particular to various types of computer devices, such as laptops, servers, etc.

[0031] To facilitate understanding, we first introduce an OCR recognition scheme. Figure 1 , Figure 1 This is a schematic diagram of the architecture of an OCR recognition system. Figure 1As shown, the OCR recognition system 10 is a recurrent convolutional network (CRNN), which specifically includes three parts, namely, a convolution structure 101, a recurrent structure 102 and a transcription structure 103 from top to bottom, wherein the convolution structure 101 includes multiple convolution layers for extracting features of the input text image to obtain a feature map; the recurrent structure 102 includes a bidirectional long short-term memory network (BiLSTM), which is used to convert the feature map into a feature sequence and then predict the feature sequence to output a predicted label distribution; the transcription structure 103 is used for temporal classification (CTC) loss based on a neural network to convert the predicted label distribution into a final label sequence, and obtain the text in the text image based on the label sequence. Among them, in the CRNN network, the features are temporally associated through the BiLSTM in the cyclic structure 102. This processing method can establish the association relationship between characters to a certain extent. However, since only the features of the characters themselves are extracted when extracting features, the association relationship between characters established only by BiLSTM may be insufficient, that is, the association between the established characters is not strong enough, which will cause certain errors in character recognition.

[0032] In view of this, the present application proposes a new technical idea. Before performing sequence feature prediction based on the feature map and translating the predicted labels to transcribe the characters in the output image, in the process of extracting features from the image to obtain the feature map, the target vector is replaced for the feature map extracted by the feature extraction network, that is, the vectors arranged along the character direction are replaced in the feature map, so that in addition to the features of each character, the feature map also contains the association relationship between characters. By establishing the association relationship between characters in the feature extraction stage, the association between characters can be improved, thereby facilitating better sequence feature prediction and character output, and improving the accuracy of character recognition.

[0033] The technical solution of this application is described in detail below.

[0034] See also Figure 2 , Figure 2 A flowchart of a character recognition method provided in an embodiment of the present application is provided. The method can be applied to computer devices, such as Figure 2 As shown, the method includes the following steps:

[0035] S201: Obtain an image to be recognized.

[0036] Here, the image to be recognized refers to an image whose characters need to be recognized. The image to be recognized contains characters to be recognized. The characters to be recognized refer to characters that need to be recognized, including but not limited to Chinese characters, English characters, numeric characters, etc.

[0037] Specifically, the image to be identified can be obtained by photographing or scanning a paper document containing characters; or, it can be obtained by taking a screenshot of a document scanned into an electronic file. This application does not limit the method of obtaining the image to be identified.

[0038] S202, in the process of extracting features of the image to be identified through the feature extraction network, performing target vector replacement on at least one feature map extracted by the feature extraction network to obtain a target feature map corresponding to the image to be identified.

[0039] In the embodiment of the present application, the process of extracting features of an image to be identified by a feature extraction network refers to the process of inputting the image to be identified into the feature extraction network, and identifying and extracting various image features (such as color features, texture features, size features, spatial features, etc.) of the image to be identified by a method or structure for extracting image features such as a network operator or convolution kernel in the feature extraction network, thereby obtaining a feature map that can characterize various image features of the image to be identified. For example, the feature extraction network can be the aforementioned Figure 1 Convolutional architecture in 101.

[0040] The target vector refers to the vector arranged along the character direction in the feature map. Specifically, when the character direction is horizontal (i.e., the characters are arranged horizontally in the image to be recognized), the target vector can be a vector arranged horizontally in the feature map, that is, the target vector is a column vector in the feature map; when the character direction is vertical (i.e., the characters are arranged vertically in the image to be recognized), the target vector is a vector arranged vertically in the feature map, that is, the target vector is a row vector in the feature map.

[0041] Target vector replacement refers to replacing another target vector in a feature map with a target vector in the feature map. Taking the target vector as a column vector in the feature map as an example, target vector replacement refers to replacing another column vector in the feature map with a column vector in the feature map. Performing target vector replacement on a feature map refers to performing one or more target vector replacement processes in the feature map to obtain a new feature map; wherein the target vector replaced in each replacement process is different. The number of times the target vector replacement process is performed in the feature map and the target vector to be replaced in the target feature map can be set according to specific needs. For example, see Figure 3 , Figure 3 The process of replacing the target vector with the feature map is shown. Figure 3In the feature graph T1, the column vector L2 of the feature graph T1 is replaced by the column vector L3 of the feature graph T1, the column vector L1 of the feature graph T1 is replaced by the column vector L2 of the feature graph T1, the column vector L4 of the feature graph is replaced by the column vector L3 of the feature graph, the column vector L5 of the feature graph is replaced by the column vector L4 of the feature graph, and the column vector L1 of the feature graph is replaced by the column vector L5 of the feature graph; after five target vector replacement processes, the feature graph T2 can be obtained, and the feature graph T2 is the new feature graph corresponding to the feature graph T1. Since the feature graph is obtained by extracting features from the image to be recognized, the feature graph is used to reflect the image features of the image to be recognized, and the target vector is a vector arranged along the character direction in the feature graph, which is used to indicate the feature information of a certain character. Performing one or more target vector replacement processes in the feature graph can integrate the feature information of other characters into the feature graph, thereby establishing a connection between characters.

[0042] In some possible scenarios, the feature extraction network extracts features from the image to be identified based on a convolutional neural network to obtain a feature map. The feature extraction network includes sequentially connected feature extraction structures, each of which includes at least one convolutional layer. In each of two adjacent feature extraction structures, the next feature extraction structure is used to extract features from the feature map output by the previous feature extraction. A target vector replacement can be performed on at least one feature map extracted by the feature extraction network through the following steps A1-A4 to obtain a target feature map corresponding to the image to be identified.

[0043] A1. Obtain the first feature atlas.

[0044] A2. Replace the target vector of at least one feature map in the first feature map set to update the first feature map set.

[0045] A3. Input the updated first feature atlas into the second feature extraction structure to obtain a second feature atlas.

[0046] A4. Determine the third feature atlas or the updated third feature atlas as the target feature map corresponding to the image to be recognized.

[0047] Among them, the first feature atlas in the above step A1 is the feature atlas output by the first feature extraction structure in the feature extraction network structure, the first feature atlas includes multiple feature maps, the first feature extraction structure is any feature extraction structure in the preset structure set, and the preset structure set includes at least one feature extraction structure in the feature extraction network. The first feature extraction structure can be understood as a preset feature extraction structure in the feature extraction network that requires target vector replacement; the preset structure set is a set of feature extraction structures that require target vector replacement. The preset feature extraction structure that requires target vector replacement can be set according to the structure of the feature extraction network and specific needs.

[0048] In a specific implementation, the feature extraction network includes M feature extraction structures, where M is a positive integer greater than 4. The preset structure set may include the i-th feature extraction structure in the feature extraction network, 2≤i≤(M-1), that is, the second feature extraction structure, the third feature extraction structure...the (M-1)th feature extraction structure in the feature extraction network are all feature extraction structures that require target vector replacement, that is, the first feature extraction structure.

[0049] The feature extraction network is Figure 1 Taking the convolution structure 101 in FIG. 1 as an example, a specific configuration of the convolution structure 101 can be as follows: Figure 4 As shown, it specifically includes five feature extraction structures, namely feature extraction structure Q1, feature extraction structure Q2, feature extraction structure Q3, feature extraction structure Q4 and feature extraction structure Q5. Among them, the feature extraction structure Q1 includes a convolution layer and a maximum pooling layer, which is used to extract features from a 32*160 text image grayscale image to obtain 64 16*80 feature maps; the feature extraction structure Q2 includes a convolution layer and a maximum pooling layer, which is used to extract features from 64 16*80 feature maps to obtain 128 8*40 feature maps; the feature extraction structure Q3 includes 2 convolution layers and a maximum pooling layer, which is used to extract features from 128 8*40 feature maps to obtain 256 4*40 feature maps; the feature extraction structure Q4 includes 2 convolution layers and a maximum pooling layer, which is used to extract features from 256 4*40 feature maps to obtain 512 2*40 feature maps; the feature extraction structure Q5 includes a convolution layer, which is used to extract features from 512 2*40 feature maps to obtain 512 1*40 feature maps. Figure 4The feature extraction network shown in FIG2 can determine the feature extraction structure Q2, the feature extraction structure Q3, and the feature extraction structure Q4 as the structures to be subjected to feature extraction, i.e., the first feature extraction structure. Corresponding to the first feature extraction structure, the first feature atlas can include 64 16*80 feature maps output by the feature extraction structure Q2, or 128 8*40 feature maps output by the feature extraction structure Q3, or 512 2*40 feature maps output by the feature extraction structure Q4.

[0050] By removing the first and last two feature extraction structures from the feature extraction network and extracting feature graphs, the target vectors are replaced on all feature extraction structures, so as to fully establish the association relationship between characters.

[0051] In particular, for the above step A2, in a feasible implementation, the target vector of at least one feature map in the first feature map set may be replaced based on adjacent target vector replacement.

[0052] For the feature graph that needs to replace the target vector in the first feature graph set (hereinafter referred to as the first feature graph), in the first feature graph, for each target vector to be replaced (hereinafter referred to as the first target vector) in the first feature graph, the first target vector is replaced with a target vector adjacent to the first target vector to obtain an updated first feature graph.

[0053] Among them, which feature maps in the first feature map set are determined as the first feature map and which target vectors in the first feature map are determined as the first target vector can be set according to specific needs. Specifically, a preset number of feature maps in the first feature map set can be determined as the first feature map. Alternatively, the first feature map is selected proportionally, and the number of first features is equal to the product of the number of features in the first feature map set and the preset ratio. Selecting the first feature map proportionally can ensure that the feature map output by the first feature extraction structure contains both the original features of the characters and the associated features between adjacent characters, thereby helping to improve the accuracy of character recognition. Specifically, all target vectors in the first feature map or other target vectors excluding the first and last two target vectors can be determined as the first target vector. In this way, as many target vectors as possible in the first feature map can be replaced, so that the association relationship between adjacent characters can be fully established.

[0054] Since in a feature map, for some target vectors in the middle position (such as Figure 3For the column vector L2 in , there are two target vectors adjacent to it. In order to establish an association relationship between it and the two adjacent target vectors, the number of feature maps that need to replace the target vector can be set to 2n, where n is a positive integer greater than or equal to 1; for any feature map among the n feature maps that need to replace the target vector (hereinafter referred to as the second feature map), in the second feature map, for each target vector to be replaced in the second feature map (hereinafter referred to as the second target vector), the second target vector can be replaced by a target vector adjacent to the second target vector in the first character direction to obtain an updated second feature map; for any feature in the other n feature maps that need to replace the target vector (hereinafter referred to as the third feature map), in the third feature map, for each target vector to be replaced in the third feature map (hereinafter referred to as the third target vector), the third target vector can be replaced by a target vector adjacent to the third target vector in the second character direction to obtain an updated third feature map.

[0055] The first character direction and the second character direction are two opposite character directions. Taking the horizontal character direction as an example, the first character direction can be from left to right, and the second character direction can be from right to left; or, the first character direction can be from right to left, and the second character direction can be from left to right. The target vector to be replaced can be the other target vectors excluding the first and last two target vectors in the feature map to be replaced, that is, the second target vector is the other target vector excluding the first and last two target vectors in the second feature map, and the third target vector is the other target vector excluding the first and last two target vectors in the third feature map.

[0056] In a specific implementation, among the 2n feature maps that need to undergo vector replacement, the n feature maps that need to undergo target vector replacement can be the first n feature maps in the first feature map set, that is, the feature maps output by the first n channels in the first feature extraction structure; the other n feature maps that need to undergo target vector replacement can be the last n feature maps in the first feature map set, that is, the feature maps output by the last n channels in the first feature extraction structure.

[0057] The first feature extraction structure is Figure 4Taking the feature extraction structure Q2 in as an example, the first feature map set includes 128 8*40 feature maps, then the first 16 8*40 feature maps and the last 16 8*40 feature maps can be selected as the 2n feature maps that need to be replaced by vectors. In the 2n replaced feature maps, the target vector is a column vector, and the column vectors are arranged in sequence according to the character direction, and the arrangement order is 1 to 40. Then, for the first 16 8*40 feature maps, the first column vector in the feature map can be used to replace the second column vector in the feature map, and the second column vector in the feature map can be used to replace the third column vector in the feature map, and so on, the 38th column vector in the feature map can be used to replace the 39th column vector in the feature map to update each feature map and obtain the updated first 16 8*40 feature maps. For the last 16 8*40 feature maps, the 39th column vector in the feature map can be replaced by the 40th column vector in the feature map, the 38th column vector in the feature map can be replaced by the 39th column vector in the feature map, and so on, the 2nd column vector in the feature map can be replaced by the 3rd column vector in the feature map to update each feature map and obtain the last 16 8*40 feature maps after update. In this way, the first 16 8*40 feature maps after update, the last 16 8*40 feature maps after update, and the 96 8*40 feature maps that have not been updated constitute the updated first feature map set. It should be understood that when the first feature extraction structure is Figure 4 When the feature extraction structure Q3 and the feature extraction structure Q4 are Figure 4 The feature extraction structure Q2 in is the same and will not be described in detail.

[0058] When performing target vector replacement, adjacent target vectors in two character directions are used for replacement, which can fully establish the association relationship between adjacent characters; since the association relationship between adjacent characters in a character is closest, fully establishing the association relationship between adjacent characters can help improve the accuracy of character recognition.

[0059] Optionally, other target vector replacement methods may be used to establish association relationships between characters, which is not limited in this application.

[0060] The second feature extraction structure in step A3 is the next feature extraction structure connected to the first feature extraction structure, and the second feature atlas is the feature atlas output by the second feature extraction structure. Figure 4 The feature extraction structure Q2 in the first feature extraction structure is the feature extraction structure Q3, and the second feature atlas includes 256 4*40 feature maps output by the feature extraction structure Q3; the first feature extraction structure is Figure 4The feature extraction structure Q3 in the first feature extraction structure is the feature extraction structure Q4, and the second feature atlas includes 512 2*40 feature maps output by the feature extraction structure Q4; the first feature extraction structure is Figure 4 If the feature extraction structure Q4 is used, the second feature extraction structure is the feature extraction structure Q5, and the second feature atlas includes 512 1*40 feature maps output by the feature extraction structure Q5.

[0061] Wherein, for the above step A4, the third feature atlas is the feature atlas output by the last feature extraction structure in the feature extraction network. In the case that the last feature extraction structure in the feature extraction network is not the preset feature extraction structure that needs to be replaced with the target vector, the third feature atlas is determined as the target feature map. For example, the configuration of the feature extraction network is as follows Figure 4 As shown, if feature extraction structures Q2, Q3, and Q4 are determined to be structures requiring feature extraction, the 512 1*40 feature maps output by feature extraction structure Q5 can be determined as target feature maps corresponding to the image to be recognized. If the last feature extraction structure in the feature extraction network is a preset feature extraction structure requiring target vector replacement, the updated third feature map set is determined as the target feature map.

[0062] S203: Perform character recognition based on the target feature map corresponding to the image to be recognized to determine the characters in the image to be recognized.

[0063] In a feasible implementation, the target feature map corresponding to the image to be identified can be input into Figure 1 In the loop structure 102 shown, the output of the transcription structure 103 is obtained to determine the characters in the image to be recognized.

[0064] In the above technical solution, after obtaining the image to be recognized, in the process of extracting features of the image to be recognized through the feature extraction network, the target vector is replaced on the feature map obtained by the feature extraction network to obtain the target feature map corresponding to the image to be recognized, and then character recognition is performed based on the target feature map, thereby determining the characters in the image to be recognized. Among them, the target vector refers to the vector arranged along the character direction in the feature map, and the target vector replacement refers to replacing another target vector in the feature map with a target vector in the feature map. Replacing the target vector on the feature map during the feature extraction process means replacing the vector arranged along the character direction in the feature map during the feature extraction process. Since the vector arranged along the character direction can indicate part of the feature information of a certain character, therefore, replacing the vector arranged along the character direction in the feature map during the feature extraction process can establish a connection between characters, so that the target feature map contains the correlation between characters, which can improve the accuracy of character recognition.

[0065] Optionally, in some possible situations, the number of feature maps requiring target vector replacement may be determined based on the number of characters contained in the image to be recognized. Before performing target vector replacement on at least one feature map extracted by the feature extraction network, the number of characters to be recognized contained in the image to be recognized may be determined; based on the number of characters to be recognized contained in the image to be recognized, the number of feature maps requiring target vector replacement in the feature extraction network may be determined.

[0066] Specifically, the number of characters to be recognized contained in the image to be recognized can be determined using a character number prediction model; the character number prediction model can be obtained through pre-training. Alternatively, the number of characters to be recognized contained in the image to be recognized can be determined by segmenting the characters using a projection method.

[0067] Specifically, when the number of characters to be recognized is large, it means that a target vector in the feature graph contains more correlations between characters, and the number of feature graphs requiring target vector replacement can be reduced; when the number of characters to be recognized is small, it means that a target vector in the feature graph contains fewer correlations between characters, and the number of feature graphs requiring target vector replacement can be increased. The ratio of feature graphs requiring target vector replacement to the first feature graph set can be determined based on the number of characters to be recognized, and the product of the total number of feature graphs contained in the first feature graph set and the ratio is determined as the number of feature graphs requiring target vector replacement.

[0068] Before performing target vector replacement, determining the number of target vector replacements required by determining the number of characters to be recognized can more reasonably establish association relationships between characters, thereby improving the accuracy of character recognition.

[0069] The above describes the method of the present application. In order to better implement the method of the present application, the device of the present application will be described below.

[0070] See also Figure 5 , Figure 5 Schematic diagram of a character recognition device provided by an embodiment of the present application. The character recognition device may be a computer device or a part of a computer device. Figure 5 As shown, the character recognition device 30 includes:

[0071] An image acquisition module 301 is configured to acquire an image to be recognized, wherein the image to be recognized includes characters to be recognized;

[0072] a replacement module 302 configured to perform target vector replacement on at least one feature map extracted by the feature extraction network during feature extraction of the image to be recognized, so as to obtain a target feature map corresponding to the image to be recognized; wherein a target vector refers to a vector arranged along a character direction in the feature map, and target vector replacement refers to replacing a target vector in the feature map with another target vector in the feature map;

[0073] The character determination module 303 is configured to perform character recognition based on the target feature map to determine the characters in the image to be recognized.

[0074] In some possible designs, the feature extraction network includes multiple feature extraction structures connected in sequence, each feature extraction structure includes at least one convolutional layer, wherein, in two adjacent feature extraction structures, the next feature extraction structure is used to extract features from the feature atlas output by the previous feature extraction structure; the above-mentioned replacement module 302 is specifically used to: obtain a first feature atlas, the first feature atlas is the feature atlas output by the first feature extraction structure in the feature extraction network, the first feature atlas includes multiple feature maps, the first feature extraction structure is any feature extraction structure in a preset structure set, and the preset structure set includes at least one feature extraction structure in the feature extraction network; perform target vector replacement on at least one feature map in the first feature atlas to update the first feature atlas, and input the updated first feature atlas into the second feature extraction structure to obtain a second feature atlas, the second feature extraction structure is the next feature extraction structure connected to the first feature extraction structure, and the second feature atlas is the feature atlas output by the second feature extraction structure; determine the third feature atlas or the updated third feature atlas as the target feature map, the third feature atlas being the feature atlas output by the last feature extraction structure in the feature extraction network.

[0075] In some possible designs, the above-mentioned replacement module 302 is specifically used to: in the first feature map, for each first target vector, replace the first target vector with a target vector adjacent to the first target vector to obtain an updated first feature map, where the first feature map is a feature map in the first feature map set that requires target vector replacement, and the first target vector is any target vector to be replaced in the first feature map.

[0076] In some possible designs, at least one feature map in the first feature map set includes 2n feature maps in the first feature map set, where n is a positive integer greater than or equal to 1; the above-mentioned replacement module 302 is specifically used to: in the second feature map, for each second target vector, use a target vector adjacent to the second target vector in the first character direction to replace the second target vector to obtain an updated second feature map, the second feature map is any feature map of n of the 2n feature maps, and the second target vector is any target vector to be replaced in the second feature map; in the third feature map, for each third target vector, use a target vector adjacent to the third target vector in the second character direction to replace the third target vector to obtain an updated third feature map, the third feature map is any feature map of the other n feature maps of the 2n feature maps, and the third target vector is any target vector to be replaced in the third feature map; the first character direction and the second character direction are two opposite character directions.

[0077] In some possible designs, the n feature maps are the first n feature maps in the first feature map set, and the other n feature maps are the last n feature maps in the first feature map set.

[0078] In one possible design, the feature extraction network includes M feature extraction structures, and the preset structure set includes the i-th feature extraction structure in the feature extraction network, 2≤i≤(M-1), and M is a positive integer greater than 4.

[0079] In some possible designs, the character recognition device 30 further includes a quantity determination module 304 for determining the number of characters to be recognized contained in the image to be recognized; and determining the number of the at least one feature map according to the number of characters to be recognized.

[0080] It should be noted that Figure 5 For the contents not mentioned in the corresponding embodiments, please refer to the description of the aforementioned method embodiments and will not be repeated here.

[0081] After acquiring the image to be recognized, the above-mentioned device performs target vector replacement on the feature map obtained by the feature extraction network during the process of extracting features from the image to be recognized through the feature extraction network, obtains the target feature map corresponding to the image to be recognized, and then performs character recognition based on the target feature map, thereby determining the characters in the image to be recognized. Among them, the target vector refers to the vector arranged along the character direction in the feature map, and the target vector replacement refers to replacing another target vector in the feature map with a target vector in the feature map. Performing target vector replacement on the feature map during the feature extraction process means replacing the vector arranged along the character direction in the feature map during the feature extraction process. Since the vector arranged along the character direction can indicate part of the feature information of a certain character, replacing the vector arranged along the character direction in the feature map during the feature extraction process can establish a connection between characters, so that the target feature map contains the correlation between characters, which can improve the accuracy of character recognition.

[0082] See also Figure 6 , Figure 6 4 is a schematic diagram of a computer device provided in an embodiment of the present application, wherein the computer device 40 includes a processor 401 and a memory 402. The processor 401 is connected to the memory 402, for example, the processor 401 may be connected to the memory 402 via a bus.

[0083] The processor 401 is configured to support the computer device 40 in executing the corresponding functions of the method in the above method embodiment. The processor 401 can be a central processing unit (CPU), a network processor (NP), a hardware chip, or any combination thereof. The above hardware chip can be an application specific integrated circuit (ASIC), a programmable logic device (PLD), or a combination thereof. The above PLD can be a complex programmable logic device (CPLD), a field-programmable gate array (FPGA), a generic array logic (GAL), or any combination thereof.

[0084] Memory 402 is used to store program code, etc. Memory 402 may include volatile memory (VM), such as random access memory (RAM); non-volatile memory (NVM), such as read-only memory (ROM), flash memory, hard disk drive (HDD), or solid-state drive (SSD); or a combination of the aforementioned types of memory.

[0085] The processor 401 may call the program code to perform the following operations:

[0086] Acquire an image to be recognized, wherein the image to be recognized includes characters to be recognized;

[0087] During feature extraction of the image to be recognized by the feature extraction network, target vector replacement is performed on at least one feature map extracted by the feature extraction network to obtain a target feature map corresponding to the image to be recognized; wherein the target vector refers to a vector arranged along the character direction in the feature map, and the target vector replacement refers to replacing another target vector in the feature map with a target vector in the feature map;

[0088] Character recognition is performed based on the target feature map to determine the characters in the image to be recognized.

[0089] An embodiment of the present application further provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, wherein the computer program includes program instructions, and when the program instructions are executed by a computer, the computer executes the method as described in the above embodiment.

[0090] Those skilled in the art will appreciate that all or part of the processes in the above-described method embodiments can be implemented by instructing related hardware through a computer program. The program can be stored in a computer-readable storage medium, and when executed, the program can include the processes in the above-described method embodiments. The storage medium can be a magnetic disk, an optical disk, a read-only memory (ROM), or a random access memory (RAM).

[0091] The above disclosure is only a preferred embodiment of the present application, and certainly cannot be used to limit the scope of rights of the present application. Therefore, equivalent changes made according to the claims of the present application are still within the scope covered by the present application.

Claims

1. A character recognition method, characterized in that: include: Acquire an image to be recognized, wherein the image to be recognized includes characters to be recognized; During feature extraction of the image to be recognized by the feature extraction network, target vector replacement is performed on at least one feature map extracted by the feature extraction network to obtain a target feature map corresponding to the image to be recognized; wherein the target vector refers to a vector arranged along the character direction in the feature map, and target vector replacement refers to replacing another target vector in the feature map with a target vector in the feature map; Character recognition is performed based on the target feature map to determine the characters in the image to be recognized.

2. The method according to claim 1, characterized in that The feature extraction network includes a plurality of sequentially connected feature extraction structures, each of which includes at least one convolutional layer, wherein, between two adjacent feature extraction structures, the next feature extraction structure is used to extract features from the feature atlas output by the previous feature extraction structure; The step of performing target vector replacement on at least one feature map extracted by the feature extraction network to obtain a target feature map corresponding to the image to be identified includes: Obtaining a first feature atlas, where the first feature atlas is a feature atlas output by a first feature extraction structure in the feature extraction network, the first feature atlas includes a plurality of feature maps, and the first feature extraction structure is any feature extraction structure in a preset structure set, and the preset structure set includes at least one feature extraction structure in the feature extraction network; Performing target vector replacement on at least one feature map in the first feature map set to update the first feature map set, and inputting the updated first feature map set into a second feature extraction structure to obtain a second feature map set, where the second feature extraction structure is a next feature extraction structure connected to the first feature extraction structure, and the second feature map set is a feature map set output by the second feature extraction structure; The third feature atlas or the updated third feature atlas is determined as the target feature map, where the third feature atlas is the feature atlas output by the last feature extraction structure in the feature extraction network.

3. The method according to claim 2, characterized in that The replacing the target vector of at least one feature map in the first feature map set to update the first feature map set includes: In the first feature map, for each first target vector, the first target vector is replaced by a target vector adjacent to the first target vector to obtain an updated first feature map, where the first feature map is a feature map in the first feature map set in which the target vector needs to be replaced, and the first target vector is any target vector to be replaced in the first feature map.

4. The method according to claim 2, characterized in that The at least one feature map in the first feature map set includes 2n feature maps in the first feature map set, where n is a positive integer greater than or equal to 1; The replacing the target vector of at least one feature map in the first feature map set to update the first feature map set includes: In the second feature map, for each second target vector, the second target vector is replaced with a target vector adjacent to the second target vector in the first character direction to obtain an updated second feature map, where the second feature map is any one of n feature maps among the 2n feature maps, and the second target vector is any target vector to be replaced in the second feature map; In the third feature map, for each third target vector, the third target vector is replaced with a target vector adjacent to the third target vector in the second character direction to obtain an updated third feature map, where the third feature map is any feature map of the other n feature maps of the 2n feature maps, and the third target vector is any target vector to be replaced in the third feature map; The first character direction and the second character direction are two opposite character directions.

5. The method according to claim 4, characterized in that The n feature maps are the first n feature maps in the first feature map set, and the other n feature maps are the last n feature maps in the first feature map set.

6. The method according to any one of claims 2 to 5, characterized in that: The feature extraction network includes M feature extraction structures, and the preset structure set includes the i-th feature extraction structure in the feature extraction network, 2≤i≤(M-1), and M is a positive integer greater than 4.

7. The method according to any one of claims 1 to 5, characterized in that Before performing target vector replacement on at least one feature map extracted by the feature extraction network, the method further includes: Determining the number of characters to be recognized contained in the image to be recognized; The number of the at least one feature map is determined according to the number of the characters to be recognized.

8. A character recognition device, characterized in that: include: An image acquisition module is used to acquire an image to be recognized, wherein the image to be recognized includes characters to be recognized; a replacement module, configured to, during feature extraction of the image to be identified by the feature extraction network, perform target vector replacement on at least one feature map extracted by the feature extraction network to obtain a target feature map corresponding to the image to be identified; wherein the target vector refers to a vector arranged along the character direction in the feature map, and target vector replacement refers to replacing another target vector in the feature map with a target vector in the feature map; The character determination module is used to perform character recognition based on the target feature map to determine the characters in the image to be recognized.

9. A computer device, characterized in that: The computer device comprises a memory and a processor, wherein the processor is configured to execute one or more computer programs stored in the memory, and when the processor executes the one or more computer programs, the computer device implements the method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, wherein the computer program includes program instructions, and when the program instructions are executed by a processor, the processor is caused to perform the method according to any one of claims 1 to 7.

Images