Delta-Logged Graph Iterators for Fast Traversal
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing graph processing systems face challenges in updating graphs stored in-memory while maintaining snapshot isolation and analytical performance, particularly due to performance degradation caused by repeated data-access indirections during concurrent modifications.
Innovation Solution
The implementation of delta-logged data structures and optimized graph iterators that leverage spatial and temporal locality, reducing indirections through techniques like spatial amortization and recursion, and utilizing specialized iterators for direct property access and full-range iterations, along with software prefetching and caching to enhance memory efficiency and computation speed.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If delta-logged data structures are used to enable fast graph updates, then update speed is improved, but memory access performance degrades due to repeated data-access indirections
Solution Approach 1:
The patent pre-computes and stores version offsets and element mappings in the delta log structure before traversal operations begin. This preliminary action allows iterators to directly compute element locations using stored offsets and simple arithmetic, avoiding repeated indirections through the delta log and achieving memory access performance comparable to read-only graphs
Solution Approach 2:
The patent introduces versioned element iterators as an intermediary layer between the delta-logged graph structure and traversal operations. These iterators maintain state about the current graph version and pre-computed mappings, mediating access to graph elements by translating logical element IDs into physical memory locations without requiring multiple indirections through the delta log
2Reliability
If multiple graph versions are maintained concurrently with snapshot isolation, then data consistency is improved, but traversal performance degrades due to version checking overhead
Solution Approach 1:
The patent extracts version verification logic from the graph traversal path by pre-computing and storing version-specific element mappings and offsets in the delta log. This allows iterators to verify version consistency through simple comparisons of pre-stored version identifiers rather than repeatedly checking the delta log during traversal, maintaining snapshot isolation while minimizing performance overhead
Solution Approach 2:
The patent performs preliminary version validation and element mapping setup when iterators are initialized, rather than during each traversal step. This preliminary action ensures snapshot isolation requirements are met while allowing the actual traversal to proceed with minimal version-checking overhead, improving analytical workload performance
3Speed
If traditional graph indices are used to accelerate traversals, then analytical performance is improved, but update complexity increases significantly
Solution Approach 1:
The patent implements dynamic versioning where the delta log structure automatically adapts to graph modifications without requiring reconstruction of index structures. The system dynamically maintains versioned element mappings that are updated incrementally through the delta log, allowing fast traversals on any graph version while keeping update complexity low through the immutable nature of the delta log approach
Data Source
AI summary
Herein is fast and memory efficient iteration of elements of a graph that has (e.g. topological) changes. In an embodiment, a computer generates an element iterator that is based on a graph that contains many elements (e.g. vertices, edges, and their properties) and a delta log that represents modification(s) of the graph. The delta log only records changes, such that only some graph elements are modified and occur in the delta log. Thus, iteration of only some graph elements may need to retrieve data from the delta log. Based on the element iterator and the delta log, a first graph element is accessed during iteration. Based on the element iterator but not the delta log, a second graph element is accessed. For example, a result may be generated that is based on the first element and the second element that were iterated based on different respective data structures.