Image-Only Schematic Link Extraction Using ML and Signal Processing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing techniques struggle to accurately extract links and connectivity from schematic diagrams in image-only formats due to challenges such as non-continuous lines, scanning artifacts, and lack of standardized visual cues, making it difficult to create digital twins and update diagrams efficiently.
Innovation Solution
A combination of multiple ML models, including a link segmenter and keypoint detector, along with signal processing, is used to produce predictions about link segments and starting/stopping points, which are then combined to generate a graph connectivity matrix, improving accuracy by minimizing shared mistakes.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If traditional line detection techniques are used, then the extraction process is simple, but the accuracy of link detection is insufficient due to non-continuous lines, scanning artifacts, and inability to distinguish crossings from junctions
Solution Approach 1:
The patent segments the link detection task into multiple independent ML models: a link segmenter model that identifies individual line segments, and a keypoint detector model that identifies connection points. Each model focuses on a specific aspect of link detection, improving overall accuracy while maintaining manageable complexity through modular architecture.
Solution Approach 2:
The patent combines multiple ML model predictions into a composite extraction system. By integrating outputs from the link segmenter model, keypoint detector model, and crossing detector model, the system achieves superior link detection accuracy that exceeds what any single model could achieve alone.
2Productivity
If manual review and data entry are used, then extraction accuracy can be high, but the process is time-consuming and labor-intensive
Solution Approach 1:
The patent implements self-service through automated ML models that perform link detection and connectivity extraction without human intervention. The system automatically processes schematic diagrams, identifies links, detects keypoints, and generates connectivity information, eliminating the need for manual review and data entry while maintaining high accuracy through sophisticated algorithms.
Solution Approach 2:
The patent replaces the mechanical process of manual review and data entry with automated computational systems. ML models substitute human analysts, using pattern recognition and image processing to extract link information automatically, thereby dramatically improving productivity while maintaining or exceeding the accuracy that manual methods could achieve.
3Ease of operation
If image-only format is used, then the schematic diagram is easy to store and transmit, but the information is hard to validate and difficult to consume by design and modeling software
Solution Approach 1:
The patent introduces an intermediary extraction system that bridges image-only schematic diagrams and machine-readable data formats. The ML-based extraction pipeline acts as a mediator, converting visual information from images into structured connectivity data that design and modeling software can consume, thereby preserving information accessibility while preventing loss of machine-readable content.
Solution Approach 2:
The patent creates a digital copy of the link connectivity information from the image-only schematic diagram. By extracting and storing connectivity data in machine-readable formats, the system preserves the essential information from the original image while making it accessible and usable by software applications without requiring the original image format.
4Measurement precision
If multiple ML models are combined, then extraction accuracy is significantly enhanced, but the system complexity and computational requirements increase
Solution Approach 1:
The patent applies preliminary action by using the link segmenter model to identify and segment potential link regions before applying the more computationally intensive keypoint detector and crossing detector models. This preprocessing step reduces the search space and computational load for subsequent models, maintaining high accuracy while optimizing resource consumption.
Data Source
AI summary
In example embodiments, techniques are provided for using a combination of multiple ML models and signal processing to extract links and connectivity from a schematic diagram in an image-only format. A first ML model (i.e. link segmenter) may produce a first set of predictions about the positions of link segments in the schematic diagram (e.g., in the form of a segmentation map). A second ML model (i.e. keypoint detector) may produce a second set of predictions about starting and stopping points of link segments in the schematic diagram (e.g., in the form of one or more heatmaps). A signal processing module may combine the first set of predictions and the second set of predictions to produce a description of links and connectivity they provide (e.g., combining the segmentation map with data from the one or more heatmaps). The results of the combining may be saved as a graph connectivity matrix.