Sparse Graph Clustering for Database Query Optimization
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Large dense graphs used in computer systems for modeling shipping or data networks require significant resources and time to process queries, often exceeding the capabilities of available computing systems, especially when precision is not critical.
Innovation Solution
Converting dense graphs into sparse graphs by clustering nodes based on relevant parameters, such as geographic location, and assigning weights to edges in the sparse graph for approximating edge weights, allowing for faster query processing and reduced memory usage.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If a dense graph is used to model shipping or data networks, then measurement precision and reliability are improved, but processing time and memory usage increase significantly
Solution Approach 1:
The patent segments the dense graph into multiple clusters of nodes, where each cluster represents a group of geographically or functionally related nodes. This segmentation reduces the overall graph size while preserving the essential connectivity patterns, enabling faster query processing without completely sacrificing accuracy.
Solution Approach 2:
The patent creates a simplified copy of the dense graph structure by generating a sparse graph that approximates the original. This sparse graph copy contains only essential edges and nodes, allowing rapid querying while the full dense graph remains stored for when maximum precision is required.
2Measurement precision
If a dense graph is used to model shipping or data networks, then measurement precision and reliability are improved, but device complexity and memory requirements increase
Solution Approach 1:
The dense graph is segmented into clusters, and only essential inter-cluster edges are retained in the sparse graph. This segmentation dramatically reduces memory requirements by eliminating redundant intra-cluster edge representations while preserving the overall network topology.
Solution Approach 2:
The patent extracts only the most important edges from the dense graph to create the sparse graph. By taking out and retaining only essential connections (those that span different clusters or represent critical pathways), the system reduces memory usage while maintaining sufficient accuracy for approximate queries.
3Productivity
If node clustering is performed to reduce graph size, then processing speed and memory efficiency are improved, but measurement precision deteriorates
Solution Approach 1:
The patent implements a dynamic approach where the system can switch between using the sparse graph for rapid approximate queries and the dense graph for precise queries when needed. This dynamic selection allows the system to optimize between speed and precision based on the specific query requirements.
Solution Approach 2:
The patent changes the granularity parameter of graph representation by allowing flexible adjustment of cluster sizes and densities. By modifying this parameter, the system can tune the balance between processing speed and query accuracy, creating sparse graphs at different levels of abstraction.
Data Source
AI summary
In a processing time, a dense graph is received, nodes of the dense graph are clustered using the first parameter to generate a sparse graph, a weight for each edge of the sparse graph is assigned; and the weights are stored in computer memory for use in a run time. In the run time after the processing time, a request for a weight of an edge between two first nodes of the dense graph is received, using a weight stored in computer memory, an approximated weight for the request is found and returned.