Resizable Hash-Based Indexing for Key-Value SSD Metadata
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Key-value solid state drives (KVSSDs) face performance bottlenecks due to data conversion mismatches between host applications and storage devices, leading to inefficient metadata management and unpredictable workloads, which hinder scalable and efficient data access.
Innovation Solution
A resizable hash-based indexing (RHIK) scheme that uses a two-level hash table with a directory layer in DRAM and a record layer in flash, transforming variable-sized keys into fixed-size key hashes for efficient metadata management, ensuring predictable access costs and fast membership checking through probabilistic operations.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If multiple data conversion layers are used to bridge host application and storage device interfaces, then compatibility and flexibility are improved, but data management overhead and performance bottlenecks increase
Solution Approach 1:
The patent extracts the key-to-physical-address mapping functionality from the complex multi-layer conversion chain and implements it directly in the storage device controller. By taking out the essential mapping function and implementing it natively in the storage device rather than through multiple host-mediated conversion layers, the system maintains compatibility while significantly reducing data management overhead and eliminating performance bottlenecks associated with repeated data conversions and metadata management on the host side
Solution Approach 2:
The patent introduces a dedicated key-to-physical-address mapping structure as an intermediary component within the storage device controller. This intermediary layer directly maps variable-length keys to physical storage addresses without requiring multiple conversion steps through host software, thereby maintaining system compatibility while reducing the complexity of data management operations
2Adaptability or versatility
If variable-sized keys are supported to increase data flexibility, then adaptability is improved, but metadata management complexity and access time increase
Solution Approach 1:
The patent changes the parameter representation by hashing variable-sized keys into fixed-size hash values for storage in the mapping structure. This parameter transformation allows the system to maintain full support for variable-sized keys (improving data flexibility) while the fixed-size hash representation enables constant-time metadata access (reducing metadata access time), thereby resolving the contradiction between flexibility and access speed
3Speed
If fixed-size key hashes are used to speed up metadata access, then access speed is improved, but information representation efficiency decreases
Solution Approach 1:
The patent creates a hash copy of the variable-sized key and stores this fixed-size copy in the key-to-physical-address mapping structure. The hash copy enables fast constant-time metadata access for lookups, while the original variable-sized key is preserved in the data structure for actual storage and retrieval operations. This copying approach maintains full key information representation capability while achieving high-speed metadata access through the fixed-size hash copy
Data Source
AI summary
Systems and methods for optimizing the internal metadata management of key-value solid state drives (KVSSDs) are provided. A key-to-physical-address (K2P) mapping scheme (resizable hash-based indexing) that focuses on index resizing and space optimization can be used. The resizable metadata management scheme can efficiently handle a highly variable workload in an incredibly large keyspace. It can provide predictable metadata access cost, as well as fast membership checking. The metadata management structure can be designed to ensure near-constant performance in terms of metadata access cost and fast membership checking.