Commit Time Logging in Time-Based MVCC with Segmented Storage
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Distributed database systems face challenges in managing commit times for time-based multi-version concurrency control (MVCC) due to the burden on system resources, leading to performance issues and memory consumption.
Innovation Solution
Implement commit time logging for time-based MVCC by storing commit times in both non-volatile and volatile memory using segments with type indicators, allowing for efficient conversion between time and status segments to free up storage space and maintain quick access to commit times.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If commit times are stored in volatile memory for time-based MVCC, then quick access to commit times is achieved, but memory burden and consumption increase
Solution Approach 1:
The commit log is divided into multiple segments that are stored in non-volatile memory. Only the currently active segment is kept in volatile memory for quick access, while historical segments are archived in non-volatile storage. This segmentation allows the system to maintain fast access to recent commit times while reducing overall memory consumption by distributing the storage burden across volatile and non-volatile memory hierarchies.
2Quantity of substance
If commit times are stored in non-volatile memory to reduce memory burden, then memory consumption is reduced, but access speed decreases
Solution Approach 1:
A segment management system acts as an intermediary between non-volatile storage and volatile memory. This intermediary manages the loading of segment data into volatile memory when needed and the flushing of segments to non-volatile storage when memory pressure occurs. The segment manager optimizes data placement and retrieval, ensuring that frequently accessed commit times remain in fast volatile memory while less frequently accessed data resides in non-volatile storage, thus balancing access speed and memory consumption.
3Reliability
If time-based MVCC is implemented with commit time logging, then consistency and serializability are maintained, but system resource burden increases
Solution Approach 1:
The system changes the state parameter of commit time data by converting it between different representations and storage formats. Commit times are logged with metadata indicating their state (e.g., committed, aborted, in-progress). The segment manager transforms and optimizes this data as it moves between volatile and non-volatile storage, adjusting parameters such as compression level, encoding format, and retention policy. This parameter transformation allows the system to maintain the necessary consistency information for MVCC while optimizing storage efficiency and access performance, thereby reducing overall system resource burden.
Data Source
AI summary
Commit time logging is performed as part of implementing time-based Multi-Version Concurrency Control (MVCC). A query engine may determine a commit time for a database transaction is committed to a database by applying MVCC to select different versions of data in the database according to transaction commit times. The query engine may store the commit time for the database transaction in a log structure stored in a non-volatile data store. A segment describing the log structure maintained in a volatile memory maintained in a volatile memory may be updated to include the commit time for the database transaction.