Scene character recognition method based on visual language modeling network

Abstract

The invention discloses a scene character recognition method based on a visual language modeling network, and the method comprises the steps: enabling a visual model to directly recognize a complete word-level result in a shielded character image feature in a training process, and guiding the visual model to deduce the shielded character content according to visual context information, so that thevisual model is endowed with language ability; and therefore, under the condition that an extra language model structure does not need to be introduced, the visual model captures language informationin the visual context in a self-adaptive mode to enhance visual features, and therefore the recognition capacity is improved. Moreover, the whole generation process of the character-level mask only needs the original word-level annotation without introducing additional annotation information; in a test process, only a backbone network and a visual semantic reasoning module are used for identification, so that a position-aware mask generation module is only used in a training process, and extra calculation overhead is not introduced.

Description

technical field [0001] The invention relates to the technical field of natural scene text recognition, in particular to a scene text recognition method based on a visual language modeling network. Background technique [0002] Natural scene text recognition is a general text recognition technology, which has become a hot research direction in the field of computer vision and document analysis in recent years, and has been widely used in autonomous driving, license plate recognition, and helping the visually impaired. The goal of this task is to convert the text content in the image into editable text. [0003] Due to the characteristics of low resolution, complex background, and susceptibility to noise interference in natural scenes, traditional text recognition technology cannot be applied to natural scenes. Therefore, character recognition technology in natural scenes has great research significance. [0004] With the development of deep learning technology in the field ...

AI Technical Summary

Problems solved by technology

But there are two problems in these methods: 1) Because of the additional introduction of the language model structure, resulting in a large amount of additional computational overhead

2) Due to the modeling of visual information and language information in two separate modules, it is difficult for the network to fully consider and effectively fuse two independent information to achieve accurate text recognition

Therefore, the speed and accuracy of scene text recognition still need to be improved.

Images