Fragmented Graph Traversal in Column Store
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current data graph traversal methods in database management systems are inefficient due to the lack of optimized algorithms for processing large-scale property graphs, especially in memory-centric systems leveraging multi-core CPUs and GPUs, which leads to suboptimal performance in querying and data retrieval.
Innovation Solution
The implementation of a fragmented incremental traversal algorithm within a graph engine that divides edge tables into fragments, allowing for parallel processing and efficient traversal by selecting the appropriate algorithm based on graph statistics and query characteristics, thereby optimizing the traversal of property graphs.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If traditional graph traversal algorithms are used in memory-centric systems, then the system can process data graphs, but the traversal efficiency is low and query execution is slow
Solution Approach 1:
The patent divides the graph traversal process into multiple independent fragments that can be processed in parallel. The vertex set is partitioned into multiple fragments, and each fragment can be traversed independently by different processing units (CPU cores or GPU threads), enabling concurrent execution and significantly improving traversal efficiency while reducing query execution time
Solution Approach 2:
The patent transitions from traditional single-threaded or sequential graph traversal to a multi-dimensional parallel processing model by utilizing both multi-core CPUs and GPUs. This dimensional expansion allows simultaneous traversal of multiple graph fragments across different processing units, dramatically increasing productivity and reducing execution time
2Quantity of substance
If graph data is stored in traditional row-based format, then data can be accessed, but memory usage is inefficient and read operations are excessive
Solution Approach 1:
The patent segments graph data into columnar format where vertices and edges are stored separately in optimized structures. This segmentation allows the system to load only the specific fragments needed for each query into memory, reducing overall memory usage while improving data retrieval efficiency by eliminating unnecessary read operations
Solution Approach 2:
The patent extracts only the necessary graph fragments required for query execution and loads them into memory, leaving the rest of the graph data on storage. This selective extraction minimizes memory usage and reduces the number of read operations by focusing only on relevant data portions
Data Source
AI summary
A system, computer-implemented method, and a computer-readable storage medium for a traversal of a property graph, are provided. The edge table of the property graph is divided into a plurality of fragments. A first fragment is selected for traversal using a set of selected vertices, where the traversal identifies a set of edges. Based on the set of edges, a set of adjacent vertices is determined during the traversal. A set of discovered vertices in the property graph is determined based on the set of selected vertices and the set of adjacent vertices.