Bi-Temporal Index for Append-Only Data Storage
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing data storage systems face inefficiencies in updating and retrieving data over time, particularly at high data volumes, as they require reads and overwrites to maintain data validity, which can be resource-intensive and inefficient.
Innovation Solution
A bi-temporal index system that uses an append-only data store to maintain records of values with validity and recordation times, allowing for the storage and retrieval of recent and older data without updating previous entries, enabling fast data entry and lookup by using validity and recordation time pairs.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If prior indexing systems update validity data for new end times, then data accuracy is maintained, but read and write operations become inefficient at high data volumes
Solution Approach 1:
The patent segments data storage into two independent components: validity time information and recordation time information. Each component is stored separately in the index structure, allowing independent processing and查询 operations. This segmentation eliminates the need for coordinated read-write operations when updating validity data, as each time dimension can be processed independently.
Solution Approach 2:
The patent introduces a bi-temporal index that adds a second time dimension (recordation time) to the traditional single time dimension (validity time). This dimensional expansion transforms the index structure from a linear time sequence to a two-dimensional time plane, enabling efficient querying and updates by providing multiple access paths without requiring data overwrites.
2Reliability
If data is updated to maintain validity over time, then data currency is improved, but resource consumption increases due to reads and overwrites
Solution Approach 1:
The patent performs preliminary organization of data into validity-time segments and recordation-time segments during the initial storage phase. This preliminary structuring allows subsequent updates to simply append or modify specific time-dimension entries without triggering cascading read-write operations, significantly reducing resource consumption while maintaining data validity.
Solution Approach 2:
The patent creates and maintains separate index copies organized by validity time and by recordation time. When data needs to be updated for validity purposes, only the relevant validity-time copy needs modification, while the recordation-time copy remains unchanged. This copying approach eliminates the need for expensive overwrites of the entire data structure.
3Reliability
If traditional indexing updates are performed, then data currentness is maintained, but operation complexity increases
Solution Approach 1:
The patent introduces bi-temporal index structures as intermediary layers between the raw data and the query interface. These index intermediaries absorb the complexity of time-based data management, providing simple, standardized access methods for both validity-time and recordation-time queries without exposing the underlying operational complexity to users or applications.
Data Source
AI summary
A database system stores data in a bi-temporal index that maintains recordation time and a validity time for recorded values. The index for a given key may be append only, such that the stored data for the key doesn't require modification of previously-recorded data for given validity times. The bi-temporal index comprises a key-value data store that allows values to be recorded for a specific key, validity time, and recordation time. To retrieve a value, a request presents a query with a key, subject time, and observation time. Queries look for values with a validity time most recent to and before the queried subject time and a recordation time, associated with the validity time, most recent to and before the queried observation time. The queries prioritize the validity time over the recordation time and returns the value that satisfies the query request.