Multi-layered Row Mapping Data Structure for Database Cache Optimization
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional database access systems face inefficiencies in data access latency and memory optimization due to buffer cache contention, hash table limitations, and scalability issues, leading to poor and unpredictable performance as data volume increases.
Innovation Solution
A multi-layered row mapping data structure is introduced, which allows for efficient data access at the row level by converting unique row identifiers into row addresses in a row cache memory, bypassing the inefficiencies of buffer caches and hash tables, and enabling scalable and concurrent operations.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If conventional buffer cache mechanisms are used for data storage, then data access operations can be performed, but data access latency increases and memory optimization is reduced due to buffer latch contention
Solution Approach 1:
The invention segments the buffer cache into multiple independent buffer pools, each capable of operating autonomously without requiring global buffer latches. This segmentation eliminates latch contention by allowing concurrent access to different buffer pools without mutual exclusion, directly resolving the contradiction between data access speed and device complexity.
2Productivity
If hash tables are used for data access and search operations, then data can be located, but sequential scan of overflow chains becomes expensive and memory space is wasted on key value storage
Solution Approach 1:
The invention extracts the key value storage requirement from the buffer cache structure entirely. By using direct addressing with buffer addresses as indices rather than hash tables, the system eliminates the need to store key values in the buffer cache, removing both the memory waste and the sequential scan overhead while maintaining direct data access capability.
3Reliability
If buffer caches require locks at each bucket for searching and updating, then database consistency is maintained, but scalability is reduced and concurrent operations become difficult
Solution Approach 1:
The invention segments the buffer cache into multiple independent buffer pools that can be accessed concurrently without requiring locks. Each buffer pool operates independently with its own management, allowing multiple threads to perform search and update operations simultaneously on different pools without contention, thus maintaining database consistency while enabling high concurrency.
4Ease of operation
If large data structures are implemented to facilitate database accesses, then data access operations can be performed, but memory space is wasted
Solution Approach 1:
The invention extracts and eliminates the overflow chain structure entirely by using direct addressing with buffer addresses as indices. This removes the need for large data structures to handle overflow cases, reducing memory space consumption while maintaining efficient data access operations through direct buffer address lookup.
Data Source
AI summary
A multi-layer data structure is configured to access rows, pages, and locks stored in memory of a database. The multi-layer data structure may include a row mapping table configured to convert a row identifier of a row in the database into a row address of the row to be accessed stored in a cache memory. The row mapping table may be used to compare bits of the row identifier with corresponding bits of the row address in the cache memory. Each layer of multi-layer row mapping table can be configured to store certain of the bits oldie row address and the row address can be determined by comparing the bits of the row address with the corresponding bits of the row identifier at each layer of the data structure.