C-Graph Edge-Set Traversal for Concurrent Property Graph Queries
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing graph processing systems face challenges in handling concurrent queries efficiently, particularly in large-scale distributed environments, due to poor data locality and high overhead in managing multiple queries simultaneously, leading to increased response times and resource competition.
Innovation Solution
The C-Graph framework implements an edge-set based graph traversal system that supports concurrent property graph queries by maintaining global vertex states, utilizing synchronous and asynchronous communication interfaces, and optimizing edge-sets for sparsity and cache locality, allowing for efficient traversal across distributed nodes.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If graph processing systems handle multiple concurrent queries simultaneously, then query throughput increases, but response time increases and resource competition intensifies
Solution Approach 1:
The patent segments the graph data into multiple partitions distributed across different nodes. Each partition contains a subset of vertices and edges, allowing concurrent queries to be processed in parallel across different partitions. This segmentation enables the system to handle multiple queries simultaneously without excessive resource competition, as each query can be routed to relevant partitions independently.
Solution Approach 2:
The patent introduces a new dimension of concurrency control through edge-set based traversal. Instead of traditional vertex-centric approaches that process queries sequentially or with heavy locking, the system uses edge-sets that can be traversed and processed in parallel across multiple threads and queries. This dimensional shift in the traversal model allows high concurrency while maintaining efficient resource utilization.
2Productivity
If graph data is distributed across multiple nodes, then system scalability increases, but data locality deteriorates
Solution Approach 1:
The patent performs preliminary organization of graph data into edge-sets during the preprocessing phase. Edge-sets are constructed to group edges that are likely to be traversed together, optimizing for both distribution and locality. This preliminary action ensures that when queries are executed, the data is already organized in an optimal structure that balances distribution across nodes with locality within each node, eliminating the need for costly data movement during query execution.
3Productivity
If edge-sets are optimized for sparsity and cache locality, then traversal efficiency increases, but memory usage increases
Solution Approach 1:
The patent applies local quality optimization by organizing edge-sets with specific properties tailored to traversal needs. Each edge-set is structured to maximize cache locality for the specific traversal patterns it supports, while sparsity optimization reduces the memory footprint by only storing relevant edges. This localized optimization approach allows the system to achieve high traversal efficiency without uniformly increasing memory usage across the entire graph, as each edge-set is optimized independently based on its specific characteristics.
Data Source
Figure 1
Figure 2
Figure 3~3(B)
AI summary
A graph processing system for concurrent property graph queries of a property graph implemented in a distributed network computes on respective nodes a subgraph shard represented as edge-sets containing vertices within a certain range. Each node stores data for a subgraph shard that contains a range of local vertices that are a subset of all vertices of the property graph. Each subgraph shard also has boundary vertices having edges that connect the subgraph shard to boundary vertices of another subgraph shard. Upon receipt of concurrent queries of the property graph, a query of the subgraph shards is scheduled in accordance with an initial vertex for each concurrent user query. The property graph is traversed by traversing edge-sets within a subgraph shard on each node and during traversal messaging is used to send values of boundary vertices to at least one other node having another subgraph shard sharing the boundary vertices.