KVSSD Metadata Storage via SCM for Data Consistency
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Log-structured merge-tree (LSM-Tree) based Key-Value Solid State Disk (KVSSD) systems face data consistency issues and increased latency due to loss of index information during abnormal power failures, leading to difficulties in distinguishing valid data versions and ensuring data consistency.
Innovation Solution
Incorporating Storage Class Memory (SCM) into the typical DRAM+NAND storage architecture to store metadata, ensuring that index information is preserved across power failures and reducing the need for frequent NAND accesses by caching frequently accessed data.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If metadata is stored in DRAM, then read performance is improved, but data consistency is worsened due to loss of index information during abnormal power failures
Solution Approach 1:
The storage system is segmented into two parts: DRAM for storing frequently accessed metadata and index information, and NAND flash for storing KV data. This segmentation allows the system to leverage the high speed of DRAM for read operations while using the non-volatile NAND flash to preserve critical index information during power failures, thus resolving the contradiction between read performance and data consistency.
Solution Approach 2:
A non-volatile memory buffer is introduced as an intermediary between DRAM and NAND flash. This buffer receives metadata from DRAM and preserves it during abnormal power failures, then supplies the preserved metadata back to DRAM after power restoration. This intermediary mechanism ensures data consistency is maintained while allowing DRAM to continue providing high-speed read operations.
2Reliability
If full disk scan is performed after abnormal power failure, then data consistency is improved, but loss of time is worsened due to extended recovery duration
Solution Approach 1:
The system performs preliminary actions by continuously maintaining index information in DRAM and using non-volatile memory buffer to preserve metadata during normal operation and abnormal power failures. This preliminary preservation of index information eliminates the need for complete disk scanning after power restoration, significantly reducing recovery time while ensuring data consistency.
Solution Approach 2:
The non-volatile memory buffer acts as a cushion that protects against data loss during abnormal power failures. By beforehand preserving metadata in this buffer, the system prevents the need for time-consuming full disk scans after power restoration, thus reducing recovery time while maintaining data consistency.
3Reliability
If index information is stored in both DRAM and SCM, then reliability is improved, but device complexity is worsened
Solution Approach 1:
The non-volatile memory buffer is designed with multi-functionality: it serves as a cache buffer during normal operation, preserves metadata during abnormal power failures, and provides a recovery mechanism after power restoration. This universal component achieves data consistency without requiring complex redundant storage architectures, thus improving reliability while controlling device complexity.
Data Source
AI summary
A log-structured merge-tree (LSM-Tree) based key-value (KV) data storage method includes writing KV data into a NAND flash memory. The KV data includes a key-value pair including a key and a corresponding value. The KV data is stored in a key-value solid state drive (KVSSD), which includes a storage class memory (SCM) and the NAND flash memory. The method further includes storing metadata of the KV data in the SCM. The metadata of the KV data includes the key and index information of the corresponding value of the KV data, and the index information of the corresponding value of the KV data indicates address information of the KV data in the NAND flash memory.