Scene character recognition method and system based on semantic enhancement encoder decoder framework

Abstract

The invention provides a scene character recognition method and system based on a semantic enhancement encoder decoder framework, and the method comprises the steps of correcting a text in any shape on a target image into a horizontal text, and obtaining a corrected image; inputting the corrected image into a convolutional neural network to extract visual features, and extracting sequence information from the visual features by using a recurrent neural network; predicting global semantic information according to the sequence information; and initializing the state of a GRU (Gateway Recirculation Unit) based on an attention mechanism by utilizing the global semantic information, calculating an attention weight according to the visual feature and the hidden state of each decoding time of theGRU, weighting the visual feature according to the attention weight, and predicting each character on the image. According to the invention, global information can be effectively utilized to overcomethe defect that local information is used in an existing method, and meanwhile, a gap between visual information and semantic information is reduced, so that the model can better process a low-quality image.

Description

technical field [0001] The invention relates to the field of computer image character recognition, in particular to a scene character recognition method and system based on a semantically enhanced encoder-decoder framework. Background technique [0002] Text detection and recognition of scene images is a research hotspot in recent years. Text recognition is the core part of the whole process. Its task is to transcribe the text in the picture into a text format that can be directly edited by the computer. With the development of deep learning, this field has made rapid progress. Inspired by the field of machine translation, the current mainstream methods are based on the encoder-decoder structure. The encoder extracts rich visual features through convolutional neural networks and cyclic neural networks, and the decoder obtains the required features through the attention mechanism. According to the text The order of the sequence predicts each character in the sequence. [00...

AI Technical Summary

Problems solved by technology

[0005] 3. The existing methods lack certain supervision on the use of global information, resulting in the inability to effectively learn the global information, thus not being very effective in the recognition process

Images