Spatial Encoding for Structured Data Extraction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current document understanding systems face challenges in accurately extracting structured data from documents, particularly images of forms, due to noisy data from optical character recognition and the inability to effectively utilize spatial information for training machine learning models, leading to incorrect context predictions and poor performance in information extraction.
Innovation Solution
The method involves preprocessing images, using spatial features and empty patch detection to enhance data extraction, by encoding hierarchical spatial information into tokens and incorporating this information into machine learning models, such as bi-directional long short-term memory and conditional random fields, to improve the accuracy of structured data extraction.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If spatial information is not encoded into tokens, then the machine learning model cannot utilize spatial cues for context prediction, but encoding hierarchical spatial information increases data processing complexity
Solution Approach 1:
The patent segments spatial information into hierarchical levels (document level, page level, block level, line level, word level) and encodes each level separately as distinct features. This segmentation allows the system to manage complex spatial information in manageable chunks that can be independently processed and combined, resolving the contradiction between utilizing spatial cues and maintaining processing simplicity.
Solution Approach 2:
The patent transforms 2D spatial coordinates from document images into 1D sequential features by encoding hierarchical spatial relationships as ordered token sequences. This dimensionality transformation allows spatial information to be integrated into the existing 1D sequence processing framework of transformers, enabling spatial cue utilization without requiring complex 2D processing architecture.
2Loss of information
If optical character recognition is used on document images, then text can be extracted, but noisy data is generated that reduces extraction accuracy
Solution Approach 1:
The patent introduces spatial encoding as an intermediary layer between optical character recognition and the transformer model. Instead of directly feeding noisy OCR output into the model, the system first encodes spatial relationships of recognized elements and uses these spatial features as mediating signals to guide context prediction, thereby reducing the impact of OCR noise on final extraction accuracy.
Solution Approach 2:
The patent implements feedback mechanisms where spatial features and contextual information from the transformer model are used to refine and correct OCR output. The model learns to identify and correct OCR errors by comparing predicted contextual patterns against actual recognized text, with spatial positioning information providing additional constraints for accurate correction.
3Reliability
If traditional machine learning models are used without spatial features, then the system is simpler to implement, but context predictions are incorrect and performance is poor
Solution Approach 1:
The patent merges spatial feature encoding with the transformer architecture by integrating spatial tokens into the standard token sequence. This merging allows the model to simultaneously process both textual content and spatial relationships using the same attention mechanism, improving information extraction performance without requiring separate processing pipelines or complex multi-model architectures.
Data Source
AI summary
A system and method for extracting data from a piece of content using spatial information about the piece of content. The system and method may use a conditional random fields process or a bidirectional long short term memory and conditional random fields process to extract structured data using the spatial information.