Time-Versioned Graph Database Segmentation for Access Latency
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Large-scale property graph databases face performance issues due to increased access times as they grow in size, storing both current and historical data, with existing methods like snapshot and linked-list techniques being inefficient for fine-grained access and leading to storage and time management complications.
Innovation Solution
Implementing a system with a current table and a history table using time information as keys, allowing efficient data access by creating a valid range for entries based on creation and deletion times, and using tree-based structures for searching, which enables efficient retrieval of data without needing exact timestamps.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If databases store more historical data to satisfy user access requirements, then data availability and user satisfaction improve, but database size grows causing increased access times and performance degradation
Solution Approach 1:
The database is segmented into multiple time-versioned tables (e.g., current table, historical tables) that store data for different time periods. This segmentation allows the system to efficiently retrieve only the relevant time-period data needed by users, rather than scanning through all historical data, thus maintaining fast access times while preserving data availability across different time ranges.
2Adaptability or versatility
If databases store all historical data without organization, then complete historical access is maintained, but search and retrieval efficiency deteriorates due to lack of structure
Solution Approach 1:
The system performs preliminary organization of historical data by creating time-versioned tables with pre-defined time ranges and metadata during data ingestion. This preliminary structuring enables efficient querying and retrieval operations later, as the system can directly access organized time-period segments without performing complex searches through unstructured historical data.
3Adaptability or versatility
If snapshot or linked-list techniques are used for versioning, then historical data retrieval is enabled, but system complexity increases and fine-grained access control becomes inefficient
Solution Approach 1:
Instead of using complex snapshot or linked-list versioning mechanisms, the patent segments historical data into simple time-versioned tables with clear time-range boundaries. Each table stores data for a specific time period with associated metadata, creating a straightforward hierarchical structure that enables efficient fine-grained access control and reduces system complexity compared to traditional versioning approaches.
Data Source
AI summary
A fine-grained scalable time-versioning support for large-scale property graph databases includes receiving a request, wherein the request includes an entity identifier (ID) and reference time information, and searching entries of a first table using the entity ID as a first key. Matching an entry is performed corresponding to the first key in the first table, determining the entry is valid based at least in part on comparing the reference time information of the request with stored time information for the entry, and responsive to the determination, creating a second key, wherein the second key concatenates an ID associated with the entity ID with the reference time information of the request. Searching a second table is performed having entries based at least in part on the second key, and responsive to searching the second table, retrieving data associated with the second key.