A character recognition method, device, electronic device, and storage medium

Abstract

An embodiment of the present invention discloses a character recognition method, device, electronic device, and storage medium. The method includes: extracting image features from a character image to be processed input into a character recognition network to obtain a feature image corresponding to the character image to be processed; encoding the feature image through each encoder node in an encoder-decoder module to determine context information corresponding to the feature image, and determining first information to be decoded input into each decoder node according to a gate function set for each decoder node; and determining a character recognition result for the character image to be processed based on the decoding processing results of each decoder node for each first information to be decoded. The context information input into each decoder node can be weighted through the gate function set for each decoder node, so as to control the ability of the character recognition network to learn context information, improve the learning quality of the context information, and improve the accuracy of the character recognition result.

Description

Technical Field

[0001] The present invention relates to the technical field of machine learning, and in particular, to a character recognition method, apparatus, electronic device, and storage medium. Background Art

[0002] With the rapid development of machine learning, character recognition technology has received increasing attention from researchers in the correction business. Taking the correction business of math homework as an example, the character image to be processed is an image including various math formulas, the character recognition result is various math formulas in the image, and the basic process of the math homework correction business is as follows: The user takes a picture of the math homework to be corrected and uploads the image to the electronic device. The electronic device automatically recognizes the math formulas in the image to obtain the character recognition result, and then based on the character recognition result, the score is determined. It can be seen that the recognition of characters is of great significance for the correction business.

[0003] Currently, for character recognition in the correction business, all context information of the character image to be processed is usually used for recognition. However, due to the randomness of characters (such as various math formulas) in the correction business, the context information often does not contain useful information. Therefore, inappropriate learning of context information during character recognition will reduce the accuracy of the character recognition result. Summary of the Invention

[0004] Embodiments of the present invention provide a character recognition method, apparatus, electronic device, and storage medium, which can weight the context information input to each decoder node through a gate function set for each decoder node, thereby controlling the ability of the character recognition network to learn context information, improving the learning quality of context information, and further improving the accuracy of the character recognition result.

[0005] On the one hand, embodiments of the present invention provide a character recognition method, including:

[0006] Performing image feature extraction on a character image to be processed input to the character recognition network through a feature extraction module in the character recognition network to obtain a feature image corresponding to the character image to be processed. The character recognition network further includes an encoder-decoder module, and the encoder-decoder module includes at least one encoder node and at least one decoder node;

[0007] Encoding the feature image through each of the encoder nodes in the encoder-decoder module to determine context information corresponding to the feature image;

[0008] Determining first information to be decoded input to each of the decoder nodes according to a gate function set for each of the decoder nodes, where the first information to be decoded includes context information weighted by the gate function;

[0009] Based on the decoding processing results of each decoder node for each of the first information to be decoded, determine the character recognition result for the character image to be processed.

[0010] On the other hand, an embodiment of the present invention provides a character recognition device, including:

[0011] A feature extraction module, configured to perform image feature extraction on a character image to be processed input to the character recognition network through a feature extraction module in the character recognition network, to obtain a feature image corresponding to the character image to be processed, where the character recognition network further includes an encoder-decoder module, and the encoder-decoder module includes at least one encoder node and at least one decoder node;

[0012] A processing module, configured to encode the feature image through each of the encoder nodes in the encoder-decoder module to determine context information corresponding to the feature image;

[0013] The processing module is further configured to determine first information to be decoded input to each of the decoder nodes according to a gate function set for each of the decoder nodes, where the first information to be decoded includes context information weighted by the gate function;

[0014] The processing module is further configured to determine a character recognition result for the character image to be processed based on the decoding processing results of each of the decoder nodes for each of the first information to be decoded.

[0015] Correspondingly, an embodiment of the present invention further provides an electronic device, including: a processor and a storage device; the storage device is configured to store program instructions; the processor, when calling the program instructions, is configured to perform: performing image feature extraction on a character image to be processed input to the character recognition network through a feature extraction module in the character recognition network, to obtain a feature image corresponding to the character image to be processed, where the character recognition network further includes an encoder-decoder module, and the encoder-decoder module includes at least one encoder node and at least one decoder node; encoding the feature image through each of the encoder nodes in the encoder-decoder module to determine context information corresponding to the feature image;

[0016] Determining first information to be decoded input to each of the decoder nodes according to a gate function set for each of the decoder nodes, where the first information to be decoded includes context information weighted by the gate function; determining a character recognition result for the character image to be processed based on the decoding processing results of each of the decoder nodes for each of the first information to be decoded.

[0017] Accordingly, an embodiment of the present invention further provides a computer storage medium, in which program instructions are stored, and when the program instructions are executed, they are used to implement the above-mentioned various methods.

[0018] In an embodiment of the present invention, an electronic device may perform image feature extraction on a to-be-processed character image input into a character recognition network through a feature extraction module in the character recognition network to obtain a feature image corresponding to the to-be-processed character image. The character recognition network further includes an encoder-decoder module, and the encoder-decoder module includes at least one encoder node and at least one decoder node. Further, each encoder node in the encoder-decoder module may encode the feature image to determine context information corresponding to the feature image, and determine first to-be-decoded information input into each decoder node according to a gate function set for each decoder node (the first to-be-decoded information includes the context information weighted by the gate function). Furthermore, based on the decoding processing results of each decoder node for each first to-be-decoded information, a character recognition result for the to-be-processed character image is determined. By adopting such a method, during the character recognition process, the context information input into each decoder node can be weighted by the gate function set for each decoder node, so as to control the ability of the character recognition network to learn context information, improve the learning quality of context information, and further improve the accuracy of the character recognition result. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required for use in the description of the embodiments or the prior art. Obviously, the following drawings are only some embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0020] Figure 1 is a flowchart of a character recognition method according to an embodiment of the present invention;

[0021] Figure 2 is a flowchart of another character recognition method according to an embodiment of the present invention;

[0022] Figure 3 is a schematic diagram of a to-be-processed character image according to an embodiment of the present invention;

[0023] Figure 4 is a flowchart of yet another character recognition method according to an embodiment of the present invention;

[0024] Figure 5 is a schematic diagram of the structure of a character recognition device according to an embodiment of the present invention;

[0025] Figure 6 It is a schematic structural diagram of an electronic device according to an embodiment of the present invention. Detailed implementation manners

[0026] With the research and progress of artificial intelligence technology, artificial intelligence technology has been studied and applied in multiple fields. For example, common smart homes, smart wearable devices, virtual assistants, smart speakers, smart marketing, driverless, autonomous driving, drones, robots, smart healthcare, smart customer service, etc. It is believed that with the development of technology, artificial intelligence technology will be applied in more fields and play an increasingly important role.

[0027] Artificial Intelligence (AI) is to use a digital computer or a machine controlled by a digital computer to simulate, extend, and expand human intelligence, a theory, method, technology, and application system that can perceive the environment, acquire knowledge, and use knowledge to obtain the best results. In other words, artificial intelligence is a comprehensive technology in computer science. It attempts to understand the essence of intelligence and produce a new intelligent machine that can react in a way similar to human intelligence. Artificial intelligence is also to study the design principles and implementation methods of various intelligent machines, so that the machines have the functions of perception, reasoning, and decision-making.

[0028] Among them, Machine Learning (ML) is an important direction in the field of computer science and the field of artificial intelligence. Machine learning is an interdisciplinary subject that involves multiple disciplines such as probability theory, statistics, approximation theory, convex analysis, and algorithm complexity theory. It specifically studies how a computer simulates or realizes human learning behavior to acquire new knowledge or skills and reorganize the existing knowledge structure to continuously improve its own performance. Machine learning is the core of artificial intelligence and the fundamental way to make a computer intelligent. Its applications cover all fields of artificial intelligence. Machine learning and deep learning usually include technologies such as artificial neural networks, belief networks, reinforcement learning, transfer learning, inductive learning, and rote learning.

[0029] Taking the quick calculation correction service as an example, the arithmetic expressions in the character images to be processed have random characteristics. All the numbers and operators in the arithmetic expressions can be considered as randomly written by users and do not have a displayed context relationship. This results in very weak semantic information in the context information in the character recognition network, which not only cannot help the character recognition network effectively learn, but may also cause the character recognition network to "memorize" specific arithmetic expression combinations incorrectly, which is not conducive to arithmetic expression recognition. For example, the specific arithmetic expression combination incorrectly "memorized" by the character recognition network is "1+1=2". When the character recognition network next recognizes the specific arithmetic expression combination "1+1=", it can directly determine the character recognition result as "1+1=2" based on the previously "memorized" specific arithmetic expression combination. However, the actual arithmetic expression included in the character image to be processed may be "1+1=3".

[0030] In order to improve the learning quality of the context information corresponding to the character images to be processed during character recognition, and thus improve the accuracy of the character recognition results. An embodiment of the present invention proposes a character recognition method, which can be applied to an electronic device. The method can perform image feature extraction on the character images to be processed input into the character recognition network through a feature extraction module in the character recognition network to obtain the feature images corresponding to the character images to be processed. The character recognition network further includes an encoder-decoder module, and the encoder-decoder module includes at least one encoder node and at least one decoder node. Further, each encoder node in the encoder-decoder module can encode the feature images to determine the context information corresponding to the feature images, and determine the first information to be decoded input into each decoder node according to the gate function set for each decoder node (the first information to be decoded includes the context information weighted by the gate function). Then, based on the decoding processing results of each decoder node for each first information to be decoded, the character recognition result for the character image to be processed is determined. By adopting such a method, during the character recognition process, the context information input into each decoder node can be weighted by the gate function set for each decoder node, so as to control the ability of the character recognition network to learn context information, improve the learning quality of the context information, and thus improve the accuracy of the character recognition results.

[0031] Among them, the above-mentioned electronic device can be a terminal device, including but not limited to a computer (such as a personal computer), a mobile phone with a touch-sensitive surface (such as a touch screen display and / or a touchpad), a laptop computer or other portable devices such as a tablet computer, or it can also be a server. In one embodiment, the construction and training of the character recognition network can be completed by the server, and the trained character recognition network is deployed on the terminal device, and the terminal device performs character recognition on the character images to be processed through the character recognition network.

[0032] In one embodiment, the above character recognition network may include a feature extraction module and an encoder-decoder module based on an attention mechanism. The encoder-decoder module includes three cascaded modules, namely: an encoder module, a decoder module, and a fully-connected output module. Among them, in one embodiment, the encoder module may be set as a bidirectional long short-term memory module, and this encoder module includes at least one encoder node; the decoder module may be set as a unidirectional long short-term memory module, and this decoder module includes at least one decoder node.

[0033] Exemplarily, taking the quick calculation and marking service as an example, the character image to be processed is an image including a mathematical formula, and the character recognition result is a mathematical formula. Refer to Figure 1 The flowchart of a character recognition method shown in Figure 1 As shown in 10, the feature extraction module and the encoder-decoder module based on the attention mechanism included in the character recognition network are respectively as shown in Figure 1 11 and 12 in. The encoder-decoder module 12 includes at least one encoder node 120 and at least one decoder node 121. Among them, the feature extraction module 11 includes five cascaded convolutional layer modules, and the number of channels is 64, 128, 256, 256, and 512 respectively. The size of the convolutional kernel is 3x3. The output between adjacent modules will first pass through a combination of a batch normalization module, a pooling module, and a rectified linear unit module, and then enter the adjacent convolutional layer module. The strides of the pooling modules are: 2x2, 2x2, 1x2, 2x1, 2x1 respectively. The encoder-decoder module based on the attention mechanism includes three cascaded modules, namely an encoder module, a decoder module, and a fully-connected output module. The encoder module is set as a bidirectional long short-term memory module with 256 hidden layer nodes and a hidden layer depth of 2; the decoder module is set as a unidirectional long short-term memory module with 256 hidden layer nodes and a hidden layer depth of 2, and the attention mechanism is applied to the decoder module; the fully-connected output module is used to connect the decoder hidden layer and the output character list. In this case, the electronic device can input the character image 10 to be processed into the character recognition network. The feature extraction module 11 in the character recognition network can perform image feature extraction on the character image 10 to be processed, and determine the feature image corresponding to the character image 10 to be processed.

[0034] Further, each encoder node 120 in the encoder-decoder module 12 can encode the feature image to determine the context information corresponding to the feature image, and determine the first information to be decoded input to each decoder node 121 according to the gate function set for each decoder node 121 (the first information to be decoded includes the context information weighted by the gate function), and then based on the decoding processing results of each decoder node 121 for each first information to be decoded, determine the character recognition result "78 - 67 = 11" for the character image to be processed.

[0035] Please refer to Figure 2 FIG. is a schematic flowchart of a character recognition method according to an embodiment of the present invention. The method according to the embodiment of the present invention can be executed by an electronic device. The method according to the embodiment of the present invention includes the following steps.

[0036] S201: Extract image features of the character image to be processed input to the character recognition network through the feature extraction module in the character recognition network to obtain a feature image corresponding to the character image to be processed. The character recognition network further includes an encoder-decoder module, and the encoder-decoder module includes at least one encoder node and at least one decoder node.

[0037] Among them, the above-mentioned character recognition network may include a feature extraction module and an encoder-decoder module based on an attention mechanism. The encoder-decoder module includes three cascaded modules, namely: an encoder module, a decoder module, and a fully connected output module. The encoder module can be set as a bidirectional long short-term memory module and includes at least one encoder node; the decoder module can be set as a unidirectional long short-term memory module and includes at least one decoder node. The feature extraction module may include N (N is an integer greater than 1) cascaded convolutional layer modules, and the output between adjacent modules can enter the adjacent convolutional layer module through a combination of a batch normalization module, a pooling module, and a rectified linear unit module.

[0038] Exemplarily, N can be 5, and the number of channels corresponding to the five cascaded convolutional layer modules included in the feature extraction module can be 64, 128, 256, 256, and 512 respectively, and the kernel size of the convolutional kernels is 3x3; the step sizes of the above-mentioned pooling modules can be: 2x2, 2x2, 1x2, 2x1, 2x1.

[0039] S202: Encode the feature image through each encoder node in the encoder-decoder module to determine the context information corresponding to the feature image.

[0040] In one embodiment, the specific implementation manner of encoding the feature image through each encoder node in the encoder-decoder module to determine the context information corresponding to the feature image may be: splitting the feature image to obtain at least one feature image region, where one feature image region corresponds to one encoder node, and encoding each corresponding feature image region through each encoder node in the encoder-decoder module to determine the context information corresponding to the feature image.

[0041] In one embodiment, the electronic device may pre-number each encoder node in the cascade. After splitting the feature image to obtain at least one feature image region, the one-to-one correspondence between the feature image region and the encoder node may be determined based on the front-to-back order of each character's position in the processed character image and the sorting of each encoder node number. Exemplarily, assuming that the mathematical formula included in the to-be-processed character image is a horizontal formula: 78 - 67 = 11, then the front-to-back order of each character's position in the processed character image refers to the order from left to right of each character's position in the horizontal formula in the processed character image. Refer to Figure 3 As shown, assuming that the mathematical formula included in the to-be-processed character image is a vertical formula: 78 - 67 = 11, then the front-to-back order of each character's position in the processed character image refers to the order from top to bottom of each character's position in the vertical formula in the processed character image.

[0042] Exemplarily, assuming that the to-be-processed character image is Figure 3 As shown, the included mathematical formula is a vertical formula "78 - 67 = 11", the characters included in the mathematical formula are respectively the numbers "78", "67", "11", the included operator is "-", and the included line break character is "—". The electronic device may split the feature image corresponding to the to-be-processed character image to obtain 5 feature image regions, and each feature image region includes the image features corresponding to each of the above characters. Specifically, sorting according to the position order from top to bottom of each character's position in the processed character image, they are respectively: "78", "-", "67", "—", "11". The first feature image region includes the image features corresponding to "78", the second feature image region includes the image features corresponding to "-", the third feature image region includes the image features corresponding to "67", the fourth feature image region includes the image features corresponding to "—", and the fifth feature image region includes the image features corresponding to "11". Among them, the line break character in the vertical formula may be equivalent to the "=" in the horizontal formula "78 - 67 = 11".

[0043] Further, based on the position order from top to bottom of each of the above characters in the processed character image and the numbers of each encoder node, in the feature image, the encoder node numbered 1 can encode the first feature image region, the encoder node numbered 2 can encode the second feature image region, the encoder node numbered 3 can encode the third feature image region, the encoder node numbered 4 can encode the fourth feature image region, and the encoder node numbered 5 can encode the fifth feature image region, thereby determining the context information corresponding to the above feature image.

[0044] S203: Determine the first information to be decoded input to each decoder node according to the gate function set for each decoder node. The first information to be decoded includes the context information weighted by the gate function.

[0045] In one embodiment, the first information to be decoded input to each decoder node further includes the output vector of the previous decoder node. The gate function is used to control the learning strength of the decoder node for the context information, and can be a fixed value (such as a weighting coefficient), or a dynamically changing function for balancing the weights of the image features and the context information.

[0046] S204: Based on the decoding processing results of each decoder node for each first information to be decoded, determine the character recognition result for the character image to be processed.

[0047] In one embodiment, the decoding processing results of each decoder node for each first information to be decoded include the distribution probabilities for the respective corresponding character recognitions. A decoder node determines the distribution probability for a character recognition. The distribution probability can be used to indicate, for example: the probability of being the character '+' is 70%, and the probability of being the character is '30%'. Further, the electronic device can obtain the character with the highest probability from the decoding processing results of each decoder node for each first information to be decoded, and based on the number order of the decoder nodes, combine each character with the highest probability, thereby determining the character recognition result for the character image to be processed. The number order for the decoder nodes corresponds to the number order for the above encoder nodes. For example, the encoder node numbered 1 corresponds to the decoder node numbered 1.

[0048] Exemplarily, assume that the arithmetic expression included in the character image to be processed is "1+2=4". The decoder numbered 1 determines the distribution probability for the character "1" in this arithmetic expression, the decoder numbered 2 determines the distribution probability for the character "+", the decoder numbered 3 determines the distribution probability for the character "2", the decoder numbered 4 determines the distribution probability for the character "=", and the decoder numbered 5 determines the distribution probability for the character "4", as shown in Table 1. In this case, the characters with the highest probabilities can be selected from the distribution probabilities determined by each decoder node for the respective corresponding character recognitions, which are: "1", "+", "2", "=", "4". Based on the order of the decoder node numbers, the characters with the highest probabilities for each are combined, and thus the character recognition result for the character image to be processed is determined as: "1+2=4":

[0049] Table 1

[0050]

[0051] In one embodiment, at least one adversarial character image can also be generated for the character recognition network through the sample generation module of the generative adversarial network, and the network parameters of the character recognition network can be adjusted based on the at least one adversarial character image. In this way, the character recognition network can be further optimized based on adversarial hard example samples, which is beneficial to further improving the accuracy of the character recognition result of the character recognition network for the character image to be processed.

[0052] In the embodiment of the present invention, the image features of the character image to be processed input into the character recognition network can be extracted through the feature extraction module in the character recognition network to obtain the feature image corresponding to the character image to be processed. The character recognition network further includes an encoder-decoder module, and the encoder-decoder module includes at least one encoder node and at least one decoder node. Further, the feature image can be encoded through each encoder node in the encoder-decoder module to determine the context information corresponding to the feature image, and the first information to be decoded input into each decoder node can be determined according to the gate function set for each decoder node (the first information to be decoded includes the context information weighted by the gate function). Then, based on the decoding processing results of each decoder node for each first information to be decoded, the character recognition result for the character image to be processed is determined. In this way, during the character recognition process, the context information input into each decoder node can be weighted through the gate function set for each decoder node, so as to control the ability of the character recognition network to learn context information, improve the learning quality of context information, and further improve the accuracy of the character recognition result.

[0053] Please refer to again Figure 4, which is a schematic flowchart of another character recognition method according to an embodiment of the present invention. The method according to the embodiment of the present invention can be executed by an electronic device. The method according to the embodiment of the present invention includes the following steps.

[0054] S401: Obtain at least one sample character image, and each sample character image in the at least one sample character image is labeled with its corresponding character information. Exemplarily, assuming that the sample character image includes a mathematical formula, then the character information labeled on the sample character image is the mathematical formula.

[0055] S402: Train a character recognition network based on each sample character image in the at least one sample character image.

[0056] In one embodiment, the specific implementation of training a character recognition network based on each sample character image in the at least one sample character image includes: determining target context information input to each decoder node in the character recognition network based on a gate function set for each decoder node in the character recognition network, where the target context information is obtained by weighting the context information by the above-mentioned gate function, and the context information is obtained by encoding a feature image of the sample character image. Further, the target context information, the output vector of the previous decoder node, and the character information input to the previous decoder node can be used as the second information to be decoded input to each decoder node, and based on the decoding processing results of each decoder node for each second information to be decoded, a character recognition result for the sample character image is determined, and the network parameters of the character recognition network are adjusted according to the difference information between the character recognition result and the character information labeled on the sample character image.

[0057] Among them, the character information input to the previous decoder node can be understood as the training target input to the previous decoder node. For example, if the previous decoder node is used to decode the character "=" in the formula "1 + 1 = 2", then the training target corresponding to the previous decoder node is the character "=".

[0058] Exemplarily, assuming that the output of the t-th (t is an integer greater than 0) decoder node is defined as s_t, the context information obtained by encoding the feature image of the sample character image is defined as c_t, the output vector of the previous decoder node is defined as s_{t - 1}, and the character information input to the previous decoder node is defined as y_{t - 1}. In this case, during the training process, the s_t can be represented by Equation 1.1:

[0059] s_t = g(s_{t - 1}, y_{t - 1}, c_t) Equation 1.1

[0060] Further, after setting the gating function for each decoder node in the character recognition network, the s_t can be expressed as Equation 1.2:

[0061] Equation 1.2

[0062] Among them, w_t is the input weight of the context information. To more intuitively control the learning of the character recognition network, the gating function can be directly set to a fixed value (that is, w_t in Equation 1.2 can be set to a fixed value), but the generalization performance is limited and it is difficult to cover complex scenarios. Therefore, in order to enable the character recognition network to cover complex scenarios, the gating function can also be set to a dynamically changing function for dynamically controlling the value of w_t.

[0063] In one embodiment, the gating function set for each decoder node in the character recognition network can be the logistic function sigma, and the function value of the logistic function corresponding to each decoder node is determined by the second information to be decoded input to each decoder node. Exemplarily, the logistic function can be expressed as Equation 1.3, which can dynamically control the control of the weight w_t of all decoder nodes by the character recognition network.

[0064] w_t = sigma(s_{t-1}, y_{t-1}, c_t) Equation 1.3

[0065] S403: Use the feature extraction module in the character recognition network to perform image feature extraction on the character image to be processed input to the character recognition network, and obtain the feature image corresponding to the character image to be processed. The character recognition network further includes an encoder-decoder module, and the encoder-decoder module includes at least one encoder node and at least one decoder node.

[0066] S404: Encode the feature image through each encoder node in the encoder-decoder module to determine the context information corresponding to the feature image.

[0067] S405: Determine the first information to be decoded input to each decoder node according to the gating function set for each decoder node. The first information to be decoded includes the context information weighted by the gating function.

[0068] S406: Based on the decoding processing results of each decoder node for each first information to be decoded, determine the character recognition result for the character image to be processed. Among them, the specific implementation manners of steps S403 to S406 can refer to the relevant descriptions of steps S201 to S204 in the above embodiments, and will not be elaborated here.

[0069] An embodiment of the present invention further provides a computer storage medium, in which program instructions are stored. When the program instructions are executed, they are used to implement the corresponding methods described in the above embodiments.

[0070] Please refer to Figure 5 , which is a schematic structural diagram of a character recognition device according to an embodiment of the present invention. The character recognition device according to the embodiment of the present invention can be arranged in an electronic device.

[0071] In one implementation of the device according to the embodiment of the present invention, the device includes the following structure.

[0072] A feature extraction module 50, configured to perform image feature extraction on a to-be-processed character image input to the character recognition network through a feature extraction module in the character recognition network, to obtain a feature image corresponding to the to-be-processed character image. The character recognition network further includes an encoder-decoder module, and the encoder-decoder module includes at least one encoder node and at least one decoder node;

[0073] A processing module 51, configured to encode the feature image through each of the encoder nodes in the encoder-decoder module to determine context information corresponding to the feature image;

[0074] The processing module 51 is further configured to determine first information to be decoded input to each of the decoder nodes according to a gate function set for each of the decoder nodes. The first information to be decoded includes context information weighted by the gate function;

[0075] The processing module 51 is further configured to determine a character recognition result for the to-be-processed character image based on the decoding processing results of each of the decoder nodes for each of the first information to be decoded.

[0076] In one embodiment, the first information to be decoded input to each of the decoder nodes further includes an output vector of a previous decoder node.

[0077] In one embodiment, the processing module 51 is specifically configured to split the feature image to obtain at least one feature image region, and one feature image region corresponds to one encoder node; encode each corresponding feature image region through each of the encoder nodes in the encoder-decoder module to determine context information corresponding to the feature image.

[0078] In one embodiment, the processing module 51 is further configured to obtain at least one sample character image, where each sample character image in the at least one sample character image is labeled with its corresponding character information; and train the character recognition network based on each sample character image in the at least one sample character image.

[0079] In one embodiment, the processing module 51 is further specifically configured to determine the target context information input to each decoder node of the character recognition network based on the gate function set for each decoder node of the character recognition network, where the target context information is obtained by weighting the context information by the gate function, and the context information is encoded from the feature image of the sample character image; use the target context information, the output vector of the previous decoder node, and the character information input to the previous decoder node as the second information to be decoded input to each decoder node; determine the character recognition result for the sample character image based on the decoding processing results of each decoder node for each of the second information to be decoded; and adjust the network parameters of the character recognition network according to the difference information between the character recognition result and the character information labeled for the sample character image.

[0080] In one embodiment, the gate function set for each decoder node of the character recognition network is a logistic function, and the function value of the logistic function corresponding to each decoder node is determined by the second information to be decoded input to each decoder node.

[0081] In one embodiment, the processing module 51 is further specifically configured to generate at least one adversarial character image for the character recognition network through the sample generation module of the generative adversarial network; and adjust the network parameters of the character recognition network based on the at least one adversarial character image.

[0082] In the embodiments of the present invention, the specific implementation of each of the above modules may refer to the description of the relevant content in the embodiments corresponding to the foregoing respective drawings.

[0083] Please refer to Figure 6 , which is a schematic structural diagram of an electronic device according to an embodiment of the present invention. The electronic device according to the embodiment of the present invention includes structures such as a power supply module, and includes a processor 601 and a storage device 602. Data can be exchanged between the processor 601 and the storage device 602, and the processor 601 implements the corresponding character recognition function.

[0084] The storage device 602 may include a volatile memory, such as a random-access memory (RAM); the storage device 602 may also include a non-volatile memory, such as a flash memory, a solid-state drive (SSD), etc.; the storage device 602 may further include a combination of the above types of memories.

[0085] The processor 601 may be a central processing unit 601 (CPU). In one embodiment, the processor 601 may also be a Graphics Processing Unit (GPU). The processor 601 may also be a combination of a CPU and a GPU. In the electronic device, multiple CPUs and GPUs may be included as needed for corresponding character recognition. In one embodiment, the storage device 602 is used to store program instructions. The processor 601 may call the program instructions to implement various methods involved in the above embodiments of the present invention.

[0086] In the first possible implementation manner, the processor 601 of the electronic device calls the program instructions stored in the storage device 602 to perform image feature extraction on the to-be-processed character image input to the character recognition network through a feature extraction module in the character recognition network, to obtain a feature image corresponding to the to-be-processed character image. The character recognition network further includes an encoder-decoder module, and the encoder-decoder module includes at least one encoder node and at least one decoder node; encodes the feature image through each of the encoder nodes in the encoder-decoder module to determine context information corresponding to the feature image; determines first to-be-decoded information input to each of the decoder nodes according to a gate function set for each of the decoder nodes, where the first to-be-decoded information includes the context information weighted by the gate function; determines a character recognition result for the to-be-processed character image based on the decoding processing results of each of the decoder nodes for each of the first to-be-decoded information.

[0087] In one embodiment, the first to-be-decoded information input to each of the decoder nodes includes an output vector of the previous decoder node.

[0088] In one embodiment, the processor 601 is specifically configured to split the feature image to obtain at least one feature image region, where one feature image region corresponds to one encoder node; and encode each corresponding feature image region through each encoder node in the encoder-decoder module to determine the context information corresponding to the feature image.

[0089] In one embodiment, the processor 601 is further configured to obtain at least one sample character image, where each sample character image in the at least one sample character image is labeled with its corresponding character information; and train the character recognition network based on each sample character image in the at least one sample character image.

[0090] In one embodiment, the processor 601 is further specifically configured to determine the target context information input to each decoder node based on the gate function set for each decoder node in the character recognition network, where the target context information is obtained by weighting the context information by the gate function, and the context information is obtained by encoding the feature image of the sample character image; use the target context information, the output vector of the previous decoder node, and the character information input to the previous decoder node as the second information to be decoded input to each decoder node; determine the character recognition result for the sample character image based on the decoding processing results of each decoder node for each of the second information to be decoded; and adjust the network parameters of the character recognition network according to the difference information between the character recognition result and the character information labeled for the sample character image.

[0091] In one embodiment, the gate function set for each decoder node in the character recognition network is a logistic function, and the function value of the logistic function corresponding to each decoder node is determined by the second information to be decoded input to each decoder node.

[0092] In one embodiment, the processor 601 is further specifically configured to generate at least one adversarial character image for the character recognition network through the sample generation module of the generative adversarial network; and adjust the network parameters of the character recognition network based on the at least one adversarial character image.

[0093] In the embodiments of the present invention, the specific implementation of the processor 601 may refer to the description of the relevant content in the embodiments corresponding to the foregoing respective drawings.

[0094] Those of ordinary skill in the art can understand that all or part of the processes in the methods of the above embodiments can be completed by instructing relevant hardware through a computer program. The program can be stored in a computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the above methods. Among them, the storage medium can be a magnetic disk, an optical disk, a read-only memory (ROM), or a random access memory (RAM), etc.

[0095] The above-disclosed are only some embodiments of the present invention. Of course, the scope of rights of the present invention cannot be limited thereby. Those of ordinary skill in the art can understand all or part of the processes of the above embodiments, and the equivalent changes made according to the claims of the present invention still fall within the scope covered by the invention.

Claims

1. A character recognition method, characterized in that, Including: Performing image feature extraction on a character image to be processed input into the character recognition network through a feature extraction module in the character recognition network, obtaining a feature image corresponding to the character image to be processed, where the character recognition network further includes an encoder-decoder module, and the encoder-decoder module includes at least one encoder node and at least one decoder node; Encoding the feature image through each of the encoder nodes in the encoder-decoder module to determine context information corresponding to the feature image; Inputting the output vector of the previous decoder node of each decoder node, the character information input into the previous decoder node, and the context information corresponding to the feature image into a gate function set for each decoder node to obtain input weights corresponding to each decoder node; Processing the context information corresponding to the feature image based on the input weights corresponding to each decoder node to obtain context information corresponding to each decoder node; Determining the output vector of the previous decoder node of each decoder node, the character information input into the previous decoder node, and the context information corresponding to each decoder node as the first information to be decoded input into each decoder node; Determining a character recognition result for the character image to be processed based on the decoding processing results of each decoder node for each of the first information to be decoded.

2. The method according to claim 1, wherein The encoding the feature image through each of the encoder nodes in the encoder-decoder module to determine context information corresponding to the feature image includes: Splitting the feature image to obtain at least one feature image region, where one feature image region corresponds to one encoder node; Encoding each corresponding feature image region through each of the encoder nodes in the encoder-decoder module to determine context information corresponding to the feature image.

3. The method according to claim 1, wherein Before performing image feature extraction on a character image to be processed input into the character recognition network through a feature extraction module in the character recognition network and obtaining a feature image corresponding to the character image to be processed, the method further includes: Obtaining at least one sample character image, where each sample character image in the at least one sample character image is labeled with its corresponding character information; Training the character recognition network based on each sample character image in the at least one sample character image.

4. The method according to claim 3, wherein The training the character recognition network based on each sample character image in the at least one sample character image includes: Determining target context information input into each decoder node based on a gate function set for each decoder node in the character recognition network, where the target context information is obtained by weighted processing of context information by the gate function, and the context information is obtained by encoding the feature image of the sample character image; Using the target context information, the output vector of the previous decoder node, and the character information input into the previous decoder node as the second information to be decoded input into each decoder node; Determine the character recognition result for the sample character image based on the decoding processing results of each decoder node for each of the second information to be decoded; Adjust the network parameters of the character recognition network according to the difference information between the character recognition result and the character information annotated for the sample character image.

5. The method according to claim 4, wherein The gate function set for each decoder node in the character recognition network is a logistic function, and the function values of the logistic functions corresponding to the respective decoder nodes are determined by the second information to be decoded input to the respective decoder nodes.

6. The method according to claim 1, wherein The method further includes: Generate at least one adversarial character image for the character recognition network through the sample generation module of the generative adversarial network; Adjust the network parameters of the character recognition network based on the at least one adversarial character image.

7. A character recognition device, characterized in that, The device includes: A feature extraction module, configured to perform image feature extraction on the character image to be processed input to the character recognition network through the feature extraction module in the character recognition network, to obtain a feature image corresponding to the character image to be processed, where the character recognition network further includes an encoder-decoder module, and the encoder-decoder module includes at least one encoder node and at least one decoder node; A processing module, configured to encode the feature image through each of the encoder nodes in the encoder-decoder module to determine the context information corresponding to the feature image; The processing module is further configured to input the output vector of the previous decoder node of each decoder node, the character information input to the previous decoder node, and the context information corresponding to the feature image into the gate function set for each decoder node, to obtain the input weights corresponding to the respective decoder nodes; process the context information corresponding to the feature image based on the input weights corresponding to the respective decoder nodes to obtain the context information corresponding to the respective decoder nodes; determine the output vector of the previous decoder node of each decoder node, the character information input to the previous decoder node, and the context information corresponding to the respective decoder nodes as the first information to be decoded input to the respective decoder nodes; The processing module is further configured to determine the character recognition result for the character image to be processed based on the decoding processing results of each decoder node for each of the first information to be decoded.

8. An electronic device, characterized in that, Comprising a processor and a storage device, the processor and the storage device are connected to each other, wherein the storage device is configured to store a computer program, the computer program includes program instructions, and the processor is configured to call the program instructions to execute the method according to any one of claims 1-6.

9. A computer storage medium, characterized in that, Program instructions are stored in the computer storage medium, and when the program instructions are executed, they are configured to implement the method according to any one of claims 1-6.

Images