Semantic enhanced scene text recognition method and device

Abstract

The invention provides a semantic enhanced scene text recognition method and device, and the method comprises the steps of extracting a visual feature map and a context feature sequence of a scene text image through an encoder of a scene text recognition model; and determining enhanced feature expression based on the visual feature map, the context feature sequence and the position code of the feature map; obtaining global visual information and semantic information of a scene text image; adopting a specially designed recurrent neural network unit for decoding by a decoder, wherein the unit can balance independence and correlation of context information; and performing multi-head attention operation on the implicit state vector and the expanded enhanced feature expression to obtain a local apparent feature vector. The local apparent feature vector and the hidden layer output of the recurrent neural network unit jointly participate in character prediction at the current moment, so that the correlation between semantic information and visual information is enhanced. The multi-head attention mechanism design can capture saliency information and auxiliary information of the features, so that the accuracy of a scene text recognition result is relatively high.

Description

Technical field [0001] The present invention relates to the field of computer technology, and more particularly to a semantic enhanced scene text identification method and apparatus. Background technique [0002] Compared with the traditional optical character identification (OCR) of high quality document images, natural scene text recognition can be applied in a broader field, such as photo analysis, license plate identification, picture advertising filtering, scene understanding, commodity identification, street view, bill Identification, etc. Due to the complexity of the text text, it is more difficult to identify, the main difficulties include: picture background is extremely rich, often facing low-brightness, low contrast, illumination, perspective deformation and incident shielding, etc.; text layout There may be problems such as distortion, pleats, commutation; the text may also be diverse, the word is different color. [0003] Conventional scene text recognition mainly us...

AI Technical Summary

Problems solved by technology

[0004] The present invention provides a semantically enhanced scene text recognition method and device to solve the defects of low accuracy and high complexity of scene text recognition in the prior art

Images