Hybrid In-Memory BFS-DFS Graph Query Processing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Relational database systems face challenges in efficiently processing graph queries due to suboptimal use of graph structure, high memory usage, and irregular access patterns, particularly when handling homogeneous and heterogeneous graphs, as existing methods do not leverage the graph structure effectively and result in excessive memory consumption and poor locality during neighbor traversals.
Innovation Solution
A hybrid in-memory BFS-DFS approach is integrated into relational database systems, combining breadth-first and depth-first neighbor traversal algorithms to optimize memory locality and pipeline friendliness, using compressed sparse rows (CSR) representation and specialized match operators to efficiently process graph pattern matching queries without exporting data to external engines.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If SQL join operators are used for graph queries, then existing SQL engine can process queries, but memory usage explodes and performance degrades due to scanning all vertices instead of leveraging graph structure
Solution Approach 1:
The patent extracts the graph traversal logic from general SQL join operations by introducing specialized graph table operators that directly access graph structures (CSR format) and perform neighbor traversals without scanning all vertices. This extraction enables efficient graph query processing while avoiding the memory explosion of traditional SQL joins.
Solution Approach 2:
The patent applies local quality by making the graph processing runtime pipeline-friendly with controlled memory footprint, allowing different parts of the system (graph operators vs. relational operators) to have optimized characteristics suited to their specific functions.
2Manufacturing precision
If BFS traversal is used for graph queries, then all vertices at a given level are matched before next level, but memory footprint becomes large and pipeline execution is blocked
Solution Approach 1:
The patent transforms the static BFS approach into a dynamic hybrid BFS-DFS execution model where the system can adaptively switch between breadth-first and depth-first traversal strategies based on query characteristics and data distribution, optimizing both memory usage and processing efficiency.
Solution Approach 2:
The patent segments the graph traversal into independent pipeline stages with controlled memory footprint, allowing results to be produced incrementally and shipped to downstream operators without materializing all intermediate results in memory.
3Quantity of substance
If DFS traversal is used for graph queries, then memory footprint is reduced and pipeline execution is enabled, but memory locality deteriorates and neighbor iteration efficiency decreases
Solution Approach 1:
The patent implements dynamic traversal strategy selection that adapts between BFS and DFS based on query patterns, allowing the system to optimize for memory locality when beneficial while maintaining low memory footprint through controlled exploration depth.
Solution Approach 2:
The patent employs nested data structures (CSR format with compressed row pointers and column pointers) that enable efficient neighbor iteration by nesting neighbor information within vertex records, improving memory locality without increasing overall memory footprint.
4Adaptability or versatility
If graph queries are processed without leveraging graph structure, then SQL engine can be used, but neighbor traversal scans all vertices causing poor performance
Solution Approach 1:
The patent applies local quality by making the graph processing runtime pipeline-friendly with controlled memory footprint, allowing different parts of the system (graph operators vs. relational operators) to have optimized characteristics suited to their specific functions.
Solution Approach 2:
The patent extracts the graph traversal logic from general SQL join operations by introducing specialized graph table operators that directly access graph structures (CSR format) and perform neighbor traversals without scanning all vertices.
5Productivity
If hash-join is used for graph queries, then join operations are optimized, but hash table construction is repeated for every join and query increasing overhead
Solution Approach 1:
The patent performs preliminary action by pre-building and caching graph structures (CSR format) and neighbor information before query execution, eliminating the need to reconstruct join structures for every query and reducing repeated computational overhead.
Data Source
AI summary
An in-memory graph query runtime is integrated inside a database management system and is capable of performing simple patter-matching queries against homogeneous graphs. The runtime efficiently combines breadth-first (BFS) and depth-first (DFS) neighbor traversal algorithms to achieve a hybrid runtime that takes the best from both sides. As a result, the hybrid runtime is able to process arbitrarily large queries with a fixed amount of memory, optimizing for memory locality.