Key-Value Store Architecture for KVSSD
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Key-value (KV) stores in the art lack support for advanced operations like range query, iterator/snapshot, prefix extractor, and time to live (TTL) operations, and rely on device input/output operations for storage management, which limits their performance and efficiency.
Innovation Solution
A high-performance KV store architecture that omits the use of a file system and write-ahead log (WAL), leveraging a metadata buffer for efficient key iterator operations and offloading user key-to-block mapping to KV solid state drives (SSDs) with computation units, enabling advanced database operations and reducing write amplification factor (WAF).
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If KV stores use device input/output operations for storage management, then storage management is simplified, but performance and efficiency are limited
Solution Approach 1:
The patent introduces a file system as an intermediary layer between the KV store and the storage device. This file system intermediary enables advanced operations (range queries, iterators, snapshots, TTL) while maintaining simplified KV store operations, thus resolving the contradiction between ease of operation and productivity.
Solution Approach 2:
The patent segments the storage management functionality into two parts: the KV store handles basic key-value operations, while the file system handles advanced operations. This segmentation allows each layer to optimize for its specific function, improving overall performance while maintaining operational simplicity.
2Reliability
If KV stores rely on conventional architecture with file system and WAL, then data consistency is ensured, but write amplification factor increases
Solution Approach 1:
The patent extracts and removes the write-ahead log (WAL) component from the conventional KV store architecture. By eliminating WAL, the system reduces write amplification factor while maintaining data consistency through alternative mechanisms in the file system layer.
Solution Approach 2:
The patent changes the operational parameters of the KV store by omitting WAL and directly using the file system for consistency management. This parameter change reduces the number of write operations (lowering WAF) while maintaining reliability through the file system's inherent consistency mechanisms.
3Loss of substance
If KV stores omit file system and WAL, then write amplification factor is reduced, but advanced operations support is limited
Solution Approach 1:
The patent makes the file system multi-functional by having it serve both as a storage management layer and as the provider of advanced operations (range queries, iterators, snapshots, TTL). This universality allows the system to support advanced operations without WAL, reducing WAF while maintaining versatility.
4Adaptability or versatility
If KV stores use host-side key management, then flexibility is maintained, but device complexity and disk writes increase
Solution Approach 1:
The patent implements self-service by having the file system automatically handle key management tasks (key-to-block mapping, metadata management) without requiring host-side intervention. This self-service approach reduces device complexity and disk writes while maintaining flexibility through the file system's automated mechanisms.
Data Source
AI summary
Provided is a method of data storage, the method including receiving a write request including a user key, determining the user key exists in a cache, generating or updating metadata corresponding to the user key, writing data corresponding to the write request to a storage device, converting the metadata to a device format corresponding to the storage device, and storing the metadata on the storage device.