Graph Query Projection via In-Memory Index Caching
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Graph queries in relational database systems face inefficiencies due to high memory usage and irregular access patterns, particularly when projecting properties, as existing solutions either rely on costly random data access or require excessive memory resources for in-memory graph indexes.
Innovation Solution
The implementation of a caching mechanism, materialized paths data structure, and lazy materialization buffer, combined with data prefetching and specialized control flows, minimizes storage accesses and optimizes memory usage by caching properties of visited graph components and prefetching likely future data, thereby reducing the need for random access and memory overhead.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If in-memory graph index is used to process graph queries, then query processing speed is improved, but memory resources required increase significantly
Solution Approach 1:
The patent pre-computes and stores path patterns in the graph index before queries are executed. By organizing the graph data structure in advance with pre-calculated neighbor relationships and path information, the system enables faster query processing without needing to materialize entire subgraphs in memory during query execution, thus reducing peak memory requirements while maintaining high query speed.
Solution Approach 2:
The patent extracts only the necessary path pattern information from the full graph data and stores it in a compact index structure. Instead of keeping the entire graph in memory, only the essential connectivity patterns and path relationships are pre-computed and stored, allowing fast query processing with reduced memory footprint by extracting and storing only what is needed for efficient query execution.
2Adaptability or versatility
If random data access is used to retrieve projection properties from graph indexes, then property retrieval flexibility is improved, but access efficiency deteriorates
Solution Approach 1:
The patent pre-organizes projection property data alongside the graph path patterns in the index structure. By pre-computing and storing property values in a coordinated manner with the path patterns, the system enables efficient retrieval of projection properties during query execution without requiring random access to separate storage locations, thus improving access efficiency while maintaining the flexibility to retrieve any projected property.
3Adaptability or versatility
If traditional join-oriented approach is used to process graph queries, then system compatibility is improved, but query execution cost increases
Solution Approach 1:
The patent segments the graph query processing into distinct phases: path pattern matching using the specialized graph index, and property projection using pre-organized property data. By dividing the query processing into these separate stages with specialized optimization for each, the system achieves better performance than a monolithic join-oriented approach while maintaining compatibility with existing relational database systems through the use of standard SQL interfaces.
Data Source
AI summary
Techniques are provided for processing a graph query by exploiting an in-memory graph index and minimizing the number of storage accesses needed to project properties of generated paths. A predefined number of paths from a graph query runtime is accumulated, using different data structures, before executing storage accesses necessary to retrieve all properties needed. A first data structure stores all paths from the graph query runtime that hit cache(s) entirely. A second data structure stores paths that do not hit caches at any level or only a subset of the levels does. Once any of these data structures are full, result rows are produced based on the two data structures prior to extracting more paths from the graph query runtime.