Visual rich document information extraction method for actual OCR scene

Abstract

The invention discloses a visual rich document information extraction method for an actual OCR scene. The method comprises the following steps: collecting a visual rich text image in the actual scene; extracting text word embedding features and position embedding features of character levels and word levels by utilizing a pre-training word embedding model; training a named entity classification module; constructing a global document graph structure based on graph convolution GAT, and introducing a self-attention mechanism; training a named entity boundary positioning module; constructing a multi-feature aggregation structure; and training an error semantic correction module, adopting a decoding structure of a GRU, extracting a coding hidden state of a corresponding dimension feature according to an optimal path of a CRF, and guiding output of a decoder every time by taking category information of a named entity as prior guidance information to obtain entity naming information in a standard format. According to the visual rich document information extraction method, the precision of the visual rich document information extraction method in actual OCR detection and recognition application is effectively improved, and the visual rich document information extraction method is of great significance to structured storage of visual rich document information.

Description

technical field [0001] The invention belongs to the technical field of visual information extraction, and in particular relates to a method for extracting visually rich document information in an actual OCR scene. Background technique [0002] Visual Information Extraction (VIE), as an important part of Natural Language Processing (NLP), aims to directly extract structured information from unstructured document images, which is a key step in understanding document images. The extracted structured information is widely used in many occasions, such as fast indexing, efficient archiving, and document analysis. The typical method is to formulate the information extraction problem as a sequential labeling problem. In recent years, information extraction from document images (such as invoices, ID cards, and purchase receipts, etc.) has become a research hotspot. [0003] Since visual features such as glyphs, text position, layout, and font size are important cues for extracting i...

AI Technical Summary

Problems solved by technology

[0003] Since visual features such as glyph, text position, layout, and font size are important cues for extracting information from document images, many methods incorporate document images into sequential labeling models In , better results were obtained compared to using only plain text, however, most of the existing research assumes that the OCR (Optical Character Recognition) results are accurate and cannot deal with the case of flawed OCR results

On the other hand, it is very complicated to achieve error-free OCR prediction of document images, and manually annotated positioning boxes cannot be directly used for information extraction in defective OCR results, because defective OCR results usually contain a large number of repeated or missed content, which directly affects the performance of the VIE model

In addition, VIE systems that fuse the positions of text segments will face the problem of unclear boundaries of named entities, which will cause a lot of post-processing to get the final correct result

Although VIE models should consider the problem that human annotations cannot fully match OCR results, as a downstream task of OCR, it has often been ignored in previous studies

Images