Flowchart Information Extraction via Node Detection and Clustering
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing technologies face challenges in accurately and efficiently extracting information from flowchart images due to variations in shape, size, flow types, formats, colors, content density, quality, and fonts, which hinders their conversion into searchable and interactive formats.
Innovation Solution
A system and method that detects closed-shaped data nodes, localizes text within them, reconstructs connecting lines, and assembles free text using orientation-based two-dimensional clustering, ultimately compiling the information into a searchable and interactive flow diagram file.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If traditional information extraction methods are used on flowchart images, then the process is simple, but the extraction accuracy and completeness deteriorate due to variations in shape, size, flow types, formats, colors, content density, quality, and fonts
Solution Approach 1:
The extraction system is divided into multiple independent modules: closed-shaped data node detection module, text localization module, connecting line reconstruction module, and free text assembly module. Each module handles a specific component of the flowchart, allowing the system to manage complexity through modular processing while maintaining high extraction accuracy across diverse flowchart variations
Solution Approach 2:
The system transitions from two-dimensional image processing to three-dimensional spatial relationship analysis by detecting orientation angles and assembling text blocks based on angular relationships. This dimensional approach enables accurate reconstruction of flowchart structure despite variations in font, size, and layout
2Productivity
If manual information extraction is performed, then accuracy can be maintained, but time consumption and productivity deteriorate
Solution Approach 1:
The system performs automated detection and extraction of flowchart information without requiring manual intervention. The automated algorithms independently detect closed-shaped nodes, localize text, reconstruct connecting lines, and assemble free text blocks, enabling rapid processing while maintaining accuracy comparable to manual extraction
Solution Approach 2:
Manual visual analysis and transcription are replaced with automated computer vision algorithms including contour detection, text localization, line reconstruction, and orientation-based clustering. This substitution enables high-speed processing of flowchart images while maintaining extraction accuracy
3Quantity of substance
If the flowchart image is converted to a compact electronic file format, then storage size is reduced, but the complexity of conversion and data loss risk increases
Solution Approach 1:
The system creates a structured data representation (JSON format) that copies and preserves all essential information from the flowchart image, including node content, connecting lines, and free text. This digital copy maintains data completeness while enabling compact storage and easy manipulation compared to the original image file
Data Source
AI summary
A method of extracting information from a flowchart image comprising a plurality of closed-shaped data nodes having text enclosed within, connecting lines connecting the plurality of closed-shaped data nodes and free text adjacent to the connecting lines includes receiving the flowchart image, detecting the closed-shaped data nodes, localizing the text enclosed within the closed-shaped data nodes, and masking the localized text.to generate an annotated image. Lines in the annotated image are the detected to reconstruct them as closed-shaped data nodes and connecting lines. A tree frame with the plurality of closed-shaped data nodes and the connecting lines is extracted. The free text is then localized. Chunks of the free text oriented and positioned proximally together are assembled into text blocks using an orientation-based two-dimensional clustering.