Type-Segregated CSR Storage for Efficient Heterogeneous Graph Analytics
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current graph analytics systems inefficiently utilize memory and access vertices and edges in heterogeneous graphs due to treating them as untyped, leading to extra memory consumption and inefficient access when only a subset of types are accessed, and they do not support mixed graphs with both directed edges and undirected arcs.
Innovation Solution
Segregate vertices and edges by type, store properties in separate vectors, and use compressed sparse row (CSR) format for directed edges, with edge maps for reverse traversal, and encode undirected arcs efficiently to minimize data access and maximize locality.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If vertices and edges are treated as untyped in a universal property graph model, then the system can handle completely unstructured data, but memory utilization becomes inefficient when dealing with heterogeneous graphs that have typed vertices and edges
Solution Approach 1:
The patent segments vertices and edges by type, creating separate collections for different vertex types and edge types. Each collection stores only the properties relevant to that specific type, eliminating the need to store default or NULL values for properties that don't apply to particular elements. This segmentation resolves the contradiction by maintaining the ability to handle heterogeneous data while improving memory utilization through targeted storage.
Solution Approach 2:
The patent implements local quality by allowing different vertex and edge types to have different sets of properties. Instead of a universal property set for all elements, each type maintains its own relevant properties. This enables efficient memory usage for heterogeneous graphs while preserving the flexibility to handle various data structures.
2Device complexity
If all vertices are consolidated into a single collection regardless of type, then the system structure is simplified, but memory access becomes inefficient when only a subset of types are accessed
Solution Approach 1:
The patent divides the single vertex collection into multiple type-specific collections. This segmentation allows the system to access only the relevant vertex types needed for a particular query or operation, significantly reducing memory access time and improving performance while maintaining manageable system structure through organized collection hierarchy.
3Adaptability or versatility
If a universal property graph model is used, then the system can represent heterogeneous graphs, but it cannot efficiently support mixed graphs with both directed edges and undirected arcs
Solution Approach 1:
The patent segments edge relationships into separate collections for directed edges and undirected arcs. This allows the system to optimize data structures and access patterns specifically for each edge type, enabling efficient graph analysis operations on mixed graphs while maintaining the versatility to represent different relationship types.
Data Source
AI summary
Techniques are provided herein for efficient representation of heterogeneous graphs in memory. In an embodiment, vertices and edges of the graph are segregated by type. Each property of a type of vertex or edge has values stored in a respective vector. Directed or undirected edges of a same type are stored in compressed sparse row (CSR) format. The CSR format is more or less repeated for edge traversal in either forward or reverse direction. An edge map translates edge offsets obtained from traversal in the reverse direction for use with data structures that expect edge offsets in the forward direction. Subsequent filtration and/or traversal by type or property of vertex or edge entails minimal data access and maximal data locality, thereby increasing efficient use of the graph.