Document Image Tokenization for Non-Text Entity Extraction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods for entity extraction from unstructured documents struggle with documents containing non-text elements like signatures and checkboxes, as they lack the capability to handle such features and often rely solely on optical character recognition, which fails to capture semantic information.
Innovation Solution
A method involving multiple fine-tuned object detection models to identify text and non-text objects in documents, followed by natural language processing to extract entities, using techniques like key-value detection and deep neural networks.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If optical character recognition is used for entity extraction, then text processing is simple, but non-text elements like signatures and checkboxes cannot be handled
Solution Approach 1:
The system segments the document processing task into separate detection modules: text object detection, non-text object detection (for signatures, checkboxes, stamps), and entity extraction. Each module handles specific element types independently, enabling comprehensive processing while maintaining modular architecture that manages complexity
Solution Approach 2:
The patent applies a unified object detection framework that can detect multiple types of elements (text, signatures, checkboxes, stamps) using the same architectural paradigm. The system uses a pre-trained model that can be adapted to detect different object categories, creating a universal solution that handles diverse document elements without requiring completely separate processing systems
2Measurement precision
If multiple fine-tuned object detection models are used to detect different regions of interest, then detection accuracy improves, but computational complexity increases
Solution Approach 1:
The detection system is segmented into multiple specialized models, each fine-tuned to detect specific types of regions of interest (text objects, signatures, checkboxes, stamps). Each model focuses on one or a few element types, which improves detection accuracy for each category while allowing the system to manage overall complexity through modular design
Solution Approach 2:
The system uses a pre-trained object detection model as a baseline and then applies fine-tuning with modified parameters (training data, loss functions, detection thresholds) for each specific element type. This parameter adaptation allows the models to achieve high accuracy for their specific tasks while maintaining the underlying architectural simplicity of the original pre-trained model
3Loss of information
If region of interest detection is applied to identify text and non-text objects, then entity extraction completeness improves, but processing time increases
Solution Approach 1:
The processing pipeline segments document analysis into parallel detection stages: text object detection, non-text object detection, and entity extraction. By detecting and categorizing regions of interest in parallel rather than sequentially, the system maintains complete information capture while reducing overall processing time through concurrent operations
Solution Approach 2:
The system performs preliminary detection and classification of regions of interest before entity extraction. By pre-identifying what elements are present and their types, the system can optimize subsequent processing steps and avoid unnecessary analysis, thereby maintaining information completeness while improving processing efficiency
Data Source
AI summary
A method of document image processing comprises, based on at least a document page image, generating a plurality of semantic tokens that includes a plurality of word tokens and a plurality of special tokens. Each special token among the plurality of special tokens represents a non-textual semantic element of the document image, and generating the plurality of semantic tokens includes predicting, for each special token among the plurality of special tokens, a token type of the special token. The method also comprises generating, for each semantic token among the plurality of semantic tokens, a corresponding semantic token embedding among a plurality of semantic token embeddings; and applying a trained model to process an input that is based on the plurality of semantic token embeddings and a plurality of visual token embeddings based on at least the document page image to generate a semantic processing result.