Lockless Index for MVCC Data Access
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In-memory database systems face challenges in providing efficient and lockless access to multi-version concurrency control (MVCC) information while maintaining performance and reducing memory footprint, especially in large-scale databases with high transactional volumes.
Innovation Solution
The implementation of a lockless index structure that uses bitwise operations and versioned vectors to manage row states and timestamps, allowing concurrent access and efficient storage of MVCC data, enabling efficient read and write operations without compromising response time or increasing memory usage.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If conventional MVCC access mechanisms are used to ensure consistency in in-memory databases, then reliability is improved, but device complexity and memory footprint increase
Solution Approach 1:
The patent segments the MVCC index structure into multiple blocks, where each block contains a subset of row position to timestamp mappings. This segmentation reduces the complexity of individual index blocks, enabling lockless access while maintaining consistency. The index is divided into first blocks and second blocks, each manageable independently without requiring system-wide locks.
Solution Approach 2:
The patent introduces a block handle mechanism that adds a new dimension to the index structure. Instead of directly mapping row positions to timestamps in a single large structure, the system uses block handles as intermediaries, creating a two-level indexing scheme. This dimensional change enables efficient lockless access by allowing threads to operate on specific blocks independently.
2Productivity
If lockless access to MVCC information is implemented to improve productivity, then response time is improved, but reliability may worsen due to concurrent access challenges
Solution Approach 1:
The patent uses block handles as intermediary structures between the index and the actual MVCC data. These block handles act as mediators that enable concurrent access without requiring locks on the entire MVCC structure. Threads can read or write specific blocks independently through their block handles, maintaining reliability while achieving lockless high-performance access.
Solution Approach 2:
The patent implements a copying mechanism where block handles are copied and distributed to different threads for independent operation. Each thread receives a copy of the block handle, allowing it to access and modify its designated block without interfering with other threads. This copying approach enables true lockless concurrent access while preserving data consistency.
3Reliability
If comprehensive MVCC information is stored to ensure data visibility and consistency, then reliability is improved, but memory footprint increases
Solution Approach 1:
The patent extracts only the essential MVCC information (timestamps and row states) into compact index blocks, separating this critical data from the full MVCC metadata. By taking out only the necessary visibility-determining information and storing it in an optimized index structure with block handles, the system reduces memory footprint while maintaining the reliability needed for data consistency and visibility.
Data Source
AI summary
An operation on a row of a table of a database is initiated. Thereafter, a multi-version concurrency control (MVCC) object is accessed to identify blocks associated with the row position of the row using bitwise operations. Subsequently, a row state block computed based on the row position of the row is accessed to determine a row state for the row. At least one other block is accessed, based in part on the row state, to obtain at least one timestamp from the computed offset based on the row. Next, the at least one timestamp is stored or retrieved. Related apparatus, systems, techniques and articles are also described.