Main-Memory CSR Graph Indexing for Parallel RDBMS Analytics
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing relational database management systems (RDBMS) face performance issues in graph analytics due to the inefficiency of constructing graphs outside the RDBMS, leading to increased complexity and data transfer costs, and are unable to effectively execute graph algorithms like PageRank or express queries such as path finding queries efficiently.
Innovation Solution
The solution involves constructing a main-memory representation of a property graph using compressed sparse row (CSR) encodings, which are efficiently built and maintained within the RDBMS without altering the database schema, leveraging parallelism and integration with in-memory columnar representations to support heterogeneous graph elements and seamless graph index sharing.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If graphs are constructed outside the RDBMS in dedicated graph engines, then graph analytics performance is improved, but data management complexity and data transfer costs increase
Solution Approach 1:
The patent merges graph construction and storage capabilities directly into the RDBMS by implementing graph index structures (CSR encodings) within the database system itself. This integration eliminates the need for separate graph engines while maintaining graph analytics performance, as the RDBMS can now natively construct and query graph structures from relational tables without external dependencies
2Reliability
If graphs are constructed outside the RDBMS, then dedicated graph analytics capabilities are achieved, but loading and data transfer costs increase
Solution Approach 1:
The patent extracts graph construction logic from external graph engines and brings it directly into the RDBMS environment. By implementing CSR (Compressed Sparse Row) encoding mechanisms within the database, the system can construct graph indexes directly from relational tables without requiring data extraction or transfer to external systems, thereby eliminating data transfer costs while maintaining graph analytics capabilities
3Adaptability or versatility
If RDBMS uses traditional table joins for graph algorithms, then compatibility with relational queries is maintained, but execution speed deteriorates
Solution Approach 1:
The patent implements dynamic query execution that automatically selects between traditional table join methods and optimized graph index-based methods based on the query type. The system maintains compatibility with standard SQL queries while providing accelerated graph algorithm execution through CSR encodings when applicable, allowing the RDBMS to adaptively choose the most efficient execution path for each query
4Adaptability or versatility
If CSR encodings are built sequentially in the RDBMS, then integration with legacy features is maintained, but construction time increases
Solution Approach 1:
The patent segments the CSR construction process into independent parallel tasks that can be executed concurrently. The construction of different CSR structures (source CSR, destination CSR, reverse destination CSR) is divided into discrete operations that can run in parallel, significantly reducing overall construction time while maintaining full integration with legacy RDBMS features and query processors
Data Source
Figure 1
Figure 2
Figure 3
AI summary
Herein are techniques that concurrently populate entries in a compressed sparse row (CSR) encoding, of a type of edge of a heterogenous graph. In an embodiment, a computer obtains a mapping of a relational schema to a graph data model. The relational schema defines vertex tables that correspond to vertex types in the graph data model, and edge tables that correspond to edge types in the graph data model. Each edge type is associated with a source vertex type and a target vertex type. For each vertex type, a sequence of persistent identifiers of vertices is obtained. Based on the mapping and for a CSR representation of each edge type, a source array is populated that, for a same vertex ordering as the sequence of persistent identifiers for the source vertex type, is based on counts of edges of the edge type that originate from vertices of the source vertex type. For the CSR, the computer populates, in parallel and based on said mapping, a destination array that contains canonical offsets as sequence positions within the sequence of persistent identifiers of the vertices.