Temporal Graph Out-of-Order Data Handling
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Handling out-of-order data in temporal graph databases is inefficient, leading to excessive storage requirements and processing complexity due to the need to modify and duplicate snapshots across multiple timestamps, making it impractical for large streaming data networks.
Innovation Solution
The system uses two timestamps, ModificationTime and NextModificationTime, to manage data elements in a temporal graph, allowing out-of-order data to be linked directly to the graph without duplicating snapshots, enabling efficient retrieval of snapshots with O(1) time complexity by storing data elements with NextModificationTime set to MAX_TIME for current data and ModificationTime for specific timestamps.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If out-of-order data is handled by modifying and duplicating snapshots across multiple timestamps, then data completeness is improved, but storage requirements and processing complexity increase excessively
Solution Approach 1:
The patent segments the temporal graph data into two distinct timestamp fields: ModificationTime (when data was created) and NextModificationTime (when data will be updated). This segmentation allows out-of-order data to be inserted without duplicating entire snapshots, as each data element independently tracks its temporal validity range. The segmentation resolves the contradiction by maintaining data completeness through proper timestamp tracking while avoiding excessive storage duplication.
Solution Approach 2:
The patent implements a nested structure where out-of-order data elements are embedded within the existing temporal graph framework by linking them to appropriate time ranges through the dual-timestamp mechanism. Instead of creating separate snapshot duplications, out-of-order elements are nested into the continuous temporal stream by setting their ModificationTime and NextModificationTime to span the appropriate time intervals, thus maintaining completeness without proportional storage increase.
2Reliability
If out-of-order data is handled by modifying and duplicating snapshots across multiple timestamps, then data completeness is improved, but processing complexity increases excessively
Solution Approach 1:
By segmenting the temporal representation into ModificationTime and NextModificationTime fields, the patent simplifies the processing of out-of-order data. Instead of complex snapshot duplication and merging operations, the system only needs to insert or update data elements with appropriate timestamp values. This segmentation reduces processing complexity from O(n) snapshot operations to O(1) timestamp assignments while maintaining data completeness.
Solution Approach 2:
The patent avoids physical copying of snapshot data by using logical timestamp references. When out-of-order data arrives, the system creates a logical representation by setting ModificationTime and NextModificationTime fields rather than duplicating actual data snapshots. This virtual copying mechanism maintains data completeness through proper temporal indexing without the computational overhead of physical data duplication.
3Measurement precision
If traditional snapshot duplication is used for out-of-order data, then temporal accuracy is improved, but storage space usage becomes excessive
Solution Approach 1:
The patent changes the parameter representation from storing complete snapshots at each timestamp to storing only two timestamp parameters (ModificationTime and NextModificationTime) per data element. This parameter transformation maintains temporal accuracy by precisely tracking when data was created and when it becomes invalid, while reducing storage space from O(n) snapshot sizes to O(1) timestamp values per element.
Data Source
AI summary
The present technology addresses deficiencies associated with common practices for handling out of order data in a streaming data database. An aspect of the present technology is avoid storing out of order data in a snapshot but just store the out of order data as additional data linked to the temporal graph. The present technology receives out of order data and records a modification time for the data and a next modification time for the data that equals a timestamp of data previously stored in the database. If there is also data in the database for a time earlier than the timestamp of the out of order data, the earlier data is adjusted so that its next modification time matches the timestamp of the out of order data.