Hash-Balanced Key-Value Storage Layout for Shorter Page Jumps
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing storage systems face performance degradation due to lengthy page jumps and contentions in read/write operations, particularly in LSM tree-like databases, which require locking parallel groups for consistency, leading to wait times and inefficient data access.
Innovation Solution
Implementing a key-value storage layout based on hash and balanced trees to optimize data access paths, reducing page jumps and alleviating contentions by using hash operations to determine target bucket pages and entry pages, with lock mechanisms to ensure data integrity.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If LSM tree-like database structure is used for storage, then data consistency can be maintained through locking parallel groups, but page jump length increases and read/write operation contentions worsen
Solution Approach 1:
The patent divides the storage structure into separate index pages and data pages, with the index page containing only key-value pairs for quick lookup and the data page containing actual data. This segmentation allows the index to be smaller and faster to traverse, reducing page jump length while maintaining data consistency through the linked structure between index and data pages.
Solution Approach 2:
The patent introduces a new dimension by adding an index page layer between the key-value store and data pages. This additional dimensional layer organizes data through hash-based indexing, enabling O(1) average case access time and significantly reducing the number of page jumps required compared to traditional LSM tree approaches.
2Reliability
If locking mechanism is applied to parallel groups for consistency, then data integrity is ensured, but read/write operation contentions increase
Solution Approach 1:
The patent segments the locking mechanism by applying locks at the page level rather than at the parallel group level. Each page can be locked independently, allowing concurrent read operations on different pages without blocking each other. This fine-grained segmentation significantly reduces contentions while maintaining data integrity through proper lock management.
Solution Approach 2:
The patent implements dynamic locking where locks are acquired and released based on operational needs. Read operations acquire shared locks that allow concurrent access, while write operations acquire exclusive locks. This dynamic approach enables high throughput by allowing multiple readers to access data simultaneously without blocking writers, improving overall productivity while ensuring integrity.
3Speed
If hash operation is used to determine target bucket pages, then data access speed improves, but collision handling complexity increases
Solution Approach 1:
The patent segments the hash table into multiple buckets, where each bucket can independently handle collisions. When hash collisions occur, the system divides the problem into smaller sub-problems by processing each bucket separately, reducing the complexity of collision handling while maintaining O(1) average case access speed through efficient hash distribution.
Data Source
AI summary
Storage techniques involve determining a target bucket page corresponding to a target key in at least one bucket page by performing a hash operation on the target key of a read operation. Such techniques further involve determining whether a target address corresponding to the target key exists in a plurality of records included in the target bucket page. Such techniques further involve, in response to a determination that the target address exists in the plurality of records, returning a target value from a corresponding target entry page based on the target address. In this way, there is provided a key-value storage layout based on hash and balanced tree which optimizes an access path of data, so that a length of a page jump can be reduced significantly, and then contentions for pages from read/write operations are effectively alleviated, thereby improving the storage performance.