Path-Based Distance Calculation via Minimum Spanning Tree
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing data processing systems face high computational complexity in computing similarity matrices for large datasets, particularly when using distance measures other than Euclidean distance, which can lead to inefficient clustering and retrieval processes.
Innovation Solution
The implementation of a path-based distance measure that computes distances between objects in an undirected graph by constructing a minimum spanning tree, allowing for efficient computation of pairwise distances and clustering by leveraging the properties of a tree structure, which reduces computational time.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If path-based distance measures are used to capture data structure, then measurement precision is improved, but computational complexity increases
Solution Approach 1:
The patent segments the complete graph into a minimum spanning tree structure, retaining only the essential edges needed to connect all nodes. This segmentation reduces the number of paths to evaluate while preserving the topological structure necessary for accurate distance measurement, thereby maintaining measurement precision while reducing computational complexity
Solution Approach 2:
The patent extracts the minimum spanning tree from the complete graph by removing redundant edges that do not contribute to the essential connectivity or shortest path information. This extraction process eliminates unnecessary computational overhead while preserving the critical structural information needed for accurate path-based distance calculation
2Productivity
If all-pair distance measures are pre-computed for clustering, then productivity is improved, but loss of time increases
Solution Approach 1:
The patent performs preliminary computation of the minimum spanning tree structure before conducting all-pair distance calculations. By establishing this efficient tree structure in advance, subsequent distance queries can be answered rapidly using the pre-established paths, reducing the time penalty of pre-computation while enabling efficient clustering operations
Data Source
AI summary
A method is disclosed for computing a path-based distance between a pair of nodes in an undirected graph comprising a set of nodes and a set of edges connecting pairs of nodes in the undirected graph with each edge having an edge weight. The method includes: computing a minimum spanning tree (MST) over the undirected graph; computing a path-based distance between the pair of nodes in the MST; and setting the path-based distance between the pair of nodes in the undirected graph equal to the path-based distance between the pair of nodes in the MST. The path-based distance between the pair of nodes in the MST is computed as the largest edge weight along the only path between the pair of nodes in the minimum spanning tree. A similarity matrix may be constructed from the path-based distances, and used for spectral clustering.