Temporal Database Set Emptiness Index via Asynchronous Compaction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Temporal database systems face challenges in efficiently handling large volumes of historical data and maintaining serializable isolation during transactions, leading to conflicts and contention between read and write operations, especially when dealing with ever-increasing records and parallel transactions.
Innovation Solution
A counter-like index mechanism that allows contention-free non-optimistic updates and queries, using asynchronous compaction to aggregate increment data into summary points, reducing contention and enabling parallel transactions without locking large amounts of data for extended periods.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If read-modify-write process is used to maintain ACID principles, then transaction consistency is improved, but transaction conflict and serialization overhead increase
Solution Approach 1:
The patent pre-calculates and stores empty set timestamps during write transactions, so that read transactions can directly query this pre-computed information without performing read-modify-write operations. This preliminary action eliminates the need for conflicting read operations on set membership data.
Solution Approach 2:
The patent separates the tracking of set emptiness information from the main set membership data. By maintaining a dedicated index of empty set timestamps independent of the primary set storage, read transactions can query emptiness status without accessing or blocking write operations on the actual set data.
2Reliability
If serializing read and write transactions is implemented, then transaction isolation is improved, but system performance and parallelism deteriorate
Solution Approach 1:
The patent extracts the emptiness determination logic from the main transaction processing path. By maintaining a separate index that tracks when sets become empty, the system allows read transactions to independently determine set status without serializing with write transactions, thus maintaining isolation while improving performance.
Solution Approach 2:
The empty set timestamp index acts as an intermediary data structure that mediates between write transactions (which populate it) and read transactions (which query it). This intermediary allows both transaction types to proceed in parallel without direct conflict, while still providing accurate isolation guarantees.
3Measurement precision
If comprehensive transaction history is maintained, then query accuracy is improved, but data volume and storage requirements increase
Solution Approach 1:
The patent extracts only the critical information needed for emptiness queries (the timestamps when sets become empty) from the complete transaction history. This selective extraction maintains query accuracy for emptiness determination while dramatically reducing the volume of data that must be stored and processed.
Solution Approach 2:
Instead of storing all transaction details and filtering for emptiness information during queries, the patent inverts the approach by pre-computing and storing only the emptiness timestamps. This inversion reduces data volume while maintaining the precision needed for accuracy-critical queries.
Data Source
AI summary
A temporal database system, method, and computer-readable storage medium in which a database is provided with sets of entities defined by initial tuples having a set ID, a unique timestamp, and a member increment. A write transaction is performed for sets of entities, wherein the write transaction designates the set by said set ID and produces an increment, wherein the increment is a number of entities to be added to or removed from the designated respective set of entities. New tuples including the set ID, the increment, and a new unique timestamp are created for the write transaction. Following the write transaction, an asynchronous compaction operation is performed on the new tuples. The compaction operation aggregates the increment of each new tuple into summary point counts. The compaction operation facilitates efficient queries without contention with write transactions.