Mutable Multilevel Graph Representation for Dynamic Analytics
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current graph analytics systems face challenges in efficiently handling dynamic graph structures due to limitations in existing data structures like Compressed Sparse Row (CSR) representations, which are not mutable, leading to performance issues and inefficient memory usage.
Innovation Solution
The implementation of a mutable multilevel data structure with multiple read-only levels and a single writable level, where each read-only level includes vertex and edge tables, and the writable level stores changes since the previous read-only level, allowing for efficient updates and reduced memory usage.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If Compressed Sparse Row (CSR) representation is used for graph analytics, then memory efficiency is improved, but mutability is lost and performance deteriorates when graph structure changes dynamically
Solution Approach 1:
The graph representation is segmented into multiple levels: read-only levels (CSR structures) and a writable level (delta map). This segmentation allows the system to maintain efficient read-only CSR structures while adding mutability through the writable delta map layer, resolving the contradiction between memory efficiency and adaptability.
Solution Approach 2:
The patent merges multiple CSR representations (read-only levels) with a writable delta map into a unified multilevel graph representation. This combination preserves the memory efficiency of CSR while adding the mutability of the delta map, allowing dynamic graph updates without sacrificing space utilization.
2Productivity
If delta maps are used to store modifications in write-optimized representation, then write performance is improved, but computation speed slows down when accessing the representation
Solution Approach 1:
The delta map acts as an intermediary layer between the read-only CSR structure and the dynamic graph updates. It mediates between write operations (stored efficiently in the delta map) and read operations (performed on the optimized CSR structure), allowing fast writes while maintaining fast computation by keeping the base CSR structure unchanged and highly optimized.
3Adaptability or versatility
If log-structured approaches are used to store graph modifications, then mutability is achieved, but memory usage increases significantly
Solution Approach 1:
The patent extracts only the necessary modification data into a compact delta map structure, rather than storing complete log-structured representations of all graph changes. This extraction approach achieves mutability by storing only the differences (additions, deletions, modifications) since the last read-only level, significantly reducing memory usage compared to full log-structured approaches.
4Quantity of substance
If log-structured approaches are used to cap memory usage, then memory control is improved, but periodic merges are required which increase time consumption
Solution Approach 1:
The system performs preliminary actions by pre-allocating and maintaining a bounded delta map structure that automatically caps memory usage. Instead of requiring periodic merges to control memory, the delta map is designed with inherent memory bounds and can be efficiently merged with read-only levels on-demand, eliminating the need for time-consuming periodic merge operations while maintaining memory control.
Data Source
AI summary
A mutable multilevel data structure representing a graph structure may include multiple read-only levels and a single writable level. Each read-only level may include a vertex table (with references to edge tables on the same level or a different level containing elements of adjacency lists for some vertices) and an edge table (with elements of adjacency lists that changed since the previous read-only level). A hybrid variant may switch between a performance-optimized variant (whose edge tables include complete adjacency lists for vertices whose edge sets were modified) and a space-optimized variant (whose edge tables include only newly added adjacency list elements). The vertex tables and/or the writable level may be implemented using copy-on-write arrays, each including an indirection table and multiple fixed-sized data pages. Computations may be run on the read-only levels or on the writable level and read-only levels.