Document Image Information Extraction Using Knowledge Graphs and FCA
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional machine learning techniques for extracting information from scanned document images face challenges in seamlessly integrating vision-based and symbolic representations, struggling with detail detection, relational identification, and abstraction levels, especially in data-deficient domains, and fail to capture human-like performance by not incorporating visual and textual cues effectively.
Innovation Solution
A system utilizing a cycle generative adversarial network (GAN) for preprocessing, optical character recognition (OCR) for text extraction, and Formal Concept Analysis (FCA) to generate knowledge graphs, which are then compared with template datasets to determine similarity metrics and apply rule sets for information extraction, with manual annotation for images below a confidence threshold.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If conventional vision-based models are used for entity detection in scanned document images, then the processing speed is maintained, but the detection precision and ability to capture fine details deteriorate
Solution Approach 1:
The system segments the information extraction process into multiple specialized modules: a vision model for detecting entities and their spatial relationships, an OCR technique for text recognition, and a knowledge graph generation component. Each module focuses on specific tasks, allowing high detection precision while maintaining processing efficiency through parallel and sequential optimization.
2Adaptability or versatility
If deep learning-symbolic techniques are integrated for information extraction, then the ability to reason over visual and textual components improves, but the system complexity increases
Solution Approach 1:
The system introduces a knowledge graph as an intermediary structure that bridges vision-based entity detection and symbolic reasoning. The knowledge graph represents entities, their attributes, and spatial relationships in a structured format, enabling seamless integration of visual and textual cues while managing system complexity through a well-defined intermediate representation.
Solution Approach 2:
The system merges deep learning-based vision models with symbolic knowledge representation by combining the strengths of both approaches. The vision model detects entities and spatial relationships, while the knowledge graph stores and reasons over this information, creating a unified system that leverages both perceptual and logical capabilities.
3Measurement precision
If conventional OCR and template matching are used, then the implementation simplicity is maintained, but the information extraction accuracy deteriorates
Solution Approach 1:
The system implements feedback mechanisms where the knowledge graph is continuously refined by comparing extracted information with domain knowledge and background information. This feedback loop enables the system to learn from errors and improve extraction accuracy over time, while the structured knowledge representation makes the complexity manageable through systematic organization.
Data Source
AI summary
This disclosure relates to a method and system for extracting information from images of one or more templatized documents. A knowledge graph with a fixed schema based on background knowledge is used to capture spatial and semantic relationships of entities present in scanned document and an adaptive lattice-based approach based on formal concepts analysis (FCA) is used to determine a similarity metric that utilizes both spatial and semantic information to determine if the structure of the scanned document image adheres to any of the known document templates. If a known document template whose structure is closely matching the structure of the scanned document is detected, then an inductive rule learning based approach is used to learn symbolic rules to extract information present in scanned document image and if a new document template is detected, then future scanned document images belonging to new document template are automatically processed using the learnt rules.