Hybrid Key-Value Storage Using Dual Hash Verification
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Key-value storage devices face performance bottlenecks due to the mapping of keys to logical addresses and then to physical addresses, requiring multiple main memory accesses to locate data, which hampers efficiency in distributed object storage and cloud computing environments.
Innovation Solution
A key-value storage scheme utilizing first and second non-volatile memory modules, where the first module stores keys and the second module stores user data, employing a hash function to generate different hash values for indexing and verification, thereby reducing the need for direct key comparisons and enhancing data transfer efficiency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If keys are mapped to logical addresses and then to physical addresses in traditional storage systems, then data can be stored and retrieved, but multiple main memory accesses are required which reduces processing speed and increases latency
Solution Approach 1:
The storage system is segmented into two independent non-volatile memory modules: a key store module for storing keys and a value store module for storing user data. This segmentation eliminates the need for sequential mapping through logical addresses, allowing direct access to data values through hash-based key lookup, thereby reducing memory access time and improving retrieval speed.
Solution Approach 2:
A hash function is introduced as an intermediary mechanism that directly maps keys to physical addresses in the value store. This hash function acts as a mediator that eliminates the traditional two-step mapping process (key to logical address, then logical address to physical address), reducing the number of memory accesses required and improving overall system performance.
2Reliability
If traditional key mapping methods are used, then data storage is achievable, but hash collisions increase and verification complexity increases
Solution Approach 1:
The system uses two different hash functions to generate two distinct hash values for each key, creating a two-dimensional verification space. This dimensional expansion allows the system to significantly reduce hash collisions while maintaining verification reliability, as the probability of collision decreases when checking against two independent hash values rather than one.
3Productivity
If multiple memory accesses are required for key lookup, then comprehensive data retrieval is possible, but processing throughput decreases
Solution Approach 1:
The system pre-computes and stores hash values in hash tables within the non-volatile memory modules during data insertion. When a retrieval operation is performed, the system only needs to compute the hash of the input key and perform a direct table lookup, rather than performing multiple sequential memory accesses. This preliminary action significantly improves processing throughput while reducing operational complexity.
Data Source
AI summary
A key-value storage device and method of using the same. In some embodiments, keys are stored in a key store in a first non-volatile memory and corresponding values associated with the keys are stored in a value store of a second non-volatile memory. An input command is received from a host device, the input command having a key associated with a value. Different first and second hash values are generated by applying a hash function to the key. The input command is executed responsive to the first and second hash values.