Vision-Assisted Graph Extraction for Spatially Structured Documents
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional methods for extracting graph structures from documents fail to utilize spatial features and understand empty space, leading to inefficiencies in processing visual diagrams like flow charts and organizational charts.
Innovation Solution
A vision-assisted network discovery algorithm that combines vision and network algorithms to derive graph structures by analyzing spatial features and text information, using techniques such as adaptive thresholding, dilation, and watershed algorithms to organize diagrams into structured representations.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of information
If conventional OCR and natural language algorithms are used to extract graph structures, then text recognition is achieved, but spatial features and empty space understanding are lost
Solution Approach 1:
The patent segments the graph extraction process into distinct modules: vision-based spatial feature extraction, text recognition, and structure assembly. The visual processing module separately analyzes spatial relationships, colors, and layouts, while the text recognition module handles OCR. This segmentation preserves spatial features that would be lost in conventional unified approaches while managing algorithmic complexity through modular design.
Solution Approach 2:
The patent introduces a vision-based spatial feature extraction module as an intermediary between the input diagram and the graph structure output. This intermediary module processes visual information (colors, spatial relationships, layouts) and transforms it into structured data that can be combined with text recognition results, thereby preserving spatial features that conventional OCR would discard.
2Measurement precision
If vision-based algorithms are implemented to understand spatial features, then graph structure extraction accuracy is improved, but processing speed decreases
Solution Approach 1:
The patent applies preliminary action by pre-processing the input image through vision-based algorithms to extract spatial features, identify nodes, and determine relationships before the main graph construction phase. Techniques like adaptive thresholding, morphological operations, and contour detection are performed in advance to create a structured representation that speeds up subsequent processing while maintaining high accuracy.
Solution Approach 2:
The patent replaces conventional mechanical text-line-by-line processing with vision-based algorithms that can simultaneously analyze spatial relationships, colors, and layouts. This substitution enables parallel processing of multiple visual features, improving both accuracy and efficiency by avoiding sequential analysis of each element.
3Loss of information
If standard OCR programs are used, then text extraction is achieved, but spatial relationships and diagram structure are not captured
Solution Approach 1:
The patent implements a universal vision-based processing module that can handle multiple diagram types (flowcharts, organizational charts, network diagrams, etc.) through a single unified approach. The module extracts spatial relationships, identifies nodes and connections, and reconstructs graph structures applicable to various diagram formats, thereby increasing adaptability while preserving spatial information that standard OCR would lose.
4Loss of information
If conventional extraction methods are used, then simple text recognition is achieved, but understanding of flow charts, organizational charts, and tables is lost
Solution Approach 1:
The patent introduces a vision-based intermediate processing layer that analyzes spatial relationships, identifies diagram-specific structures (flowchart symbols, organizational hierarchy patterns, table layouts), and transforms them into structured graph representations. This intermediary module bridges the gap between simple text recognition and complex diagram understanding without requiring entirely new algorithms for each diagram type.
Data Source
AI summary
Various methods and processes, apparatuses or systems, and media for deterministically deriving underlying graph structure and associated text information in a document are disclosed. A processor implements a vision-based algorithm and a network-based algorithm that may extract and structure a diagram from an image obtained from the document. The processor deterministically derives underlying graph structure and associated text information in the document by applying the vision-based algorithm and the network-based algorithm, thereby allowing encoding of graph content and reasoning into downstream applications including LLM inputs, graphical question-answering, and information extraction tasks. The processor also implements OCR algorithm for text fields, and then isolates which piece of text belongs to which node by examining the spatial coordinates of the text against bounding box of the node and executes cross-page resolution.