Spatial-Aware CNN Attention for Document Field Extraction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Deep-learning (DL) systems for document information extraction require lengthy annotation processes, are time-consuming, rely on pre-existing OCR systems, and often fail to accurately extract fields due to dependence on these systems.
Innovation Solution
A convolutional neural network (CNN) based system that combines feature maps with positional information to create a spatial-aware feature map, using an encoder-decoder pattern with an attention layer in the decoder for end-to-end training without additional annotations, eliminating the need for well-performing OCR systems.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If a pre-existing OCR system is used for document information extraction, then the system can recognize fields in particular languages or formats, but the extraction accuracy is limited by the OCR system's performance and additional models are required
Solution Approach 1:
The patent combines the OCR functionality and document information extraction into a single integrated deep learning model. The encoder processes the entire document image and generates feature maps that are directly used by the decoder to extract both text and structured information, eliminating the need for separate OCR and extraction models. This unified approach improves extraction accuracy while reducing system complexity.
Solution Approach 2:
The deep learning model is designed to perform multiple functions simultaneously: optical character recognition, text detection, and structured information extraction. The single model can handle various document types, languages, and formats without requiring separate specialized models, making the system more versatile and accurate.
2Reliability
If a deep-learning system is trained with annotated documents, then field extraction accuracy can be improved, but the annotation process is lengthy and time-consuming
Solution Approach 1:
The patent uses pre-trained language models and pre-trained OCR components as starting points for the document information extraction task. These pre-trained models provide initial knowledge about language patterns, text recognition, and document structures, which significantly reduces the amount of task-specific annotated data needed and accelerates the training process.
Solution Approach 2:
The patent introduces an encoder-decoder architecture with feature maps as an intermediary representation between the input document image and the output extracted information. The encoder transforms the image into intermediate feature representations that capture both visual and semantic information, making the learning process more efficient and reducing the need for extensive annotated training data.
3Adaptability or versatility
If additional models are used to perform extraction for certain document types, then coverage of different document formats is improved, but the system becomes more complex and dependent on multiple pre-existing systems
Solution Approach 1:
The patent designs a universal deep learning model that can handle multiple document types, languages, and formats through its encoder-decoder architecture. The model uses attention mechanisms and spatial feature maps to adapt to different document layouts and structures without requiring separate specialized models, thereby improving versatility while maintaining system simplicity.
Data Source
AI summary
Disclosed herein are system, method, and computer program product embodiments for document information extraction without additional annotations. An embodiment operates by receiving an input representing a document and a key. The embodiment processes the input using a convolutional neural network to obtain a feature map. The embodiment combines the feature map with positional information to obtain a spatial-aware feature map. The embodiment then repeatedly performs the following decoding process: generate attention weights, generate a context vector based on the spatial-aware feature map and the generated attention weights using an attention layer, process the context vector, the key, and an input vector using a recurrent neural network (RNN) to obtain a RNN state, and generate an output vector based on the RNN state and the context vector using a projection layer. The embodiment then extracts a field based on the result of the decoding process.