Hybrid Storage System Metadata Management
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional file systems face performance and endurance issues due to high read-modify-write (RMW) overhead and write-amplification in NAND-based SSDs, particularly with small, single-sector writes, which increase complexity and cost, and require significant DRAM resources for indirection tables and defragmentation.
Innovation Solution
A hybrid storage system using a byte-addressable non-volatile memory for metadata and a block-based non-volatile memory for data, where metadata is stored in a more write-efficient device and data is stored in a device suitable for larger writes, eliminating the need for indirection tables and reducing DRAM requirements by handling address translation in system software.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If NAND-based SSD is used for storing both metadata and file data, then storage capacity is achieved, but read-modify-write overhead and write-amplification increase significantly
Solution Approach 1:
The patent segments the storage system into two distinct parts: a byte-addressable non-volatile memory device for storing metadata and a block-addressable NAND-based SSD for storing file data. This segmentation allows each component to operate in its optimal mode, with the byte-addressable memory handling small metadata writes efficiently without the read-modify-write overhead that plagues NAND-based SSDs when used for metadata storage.
2Adaptability or versatility
If conventional file system stores metadata in NAND-based SSD, then storage uniformity is maintained, but significant DRAM resources are required for indirection tables and defragmentation
Solution Approach 1:
The patent extracts the metadata storage function from the NAND-based SSD and places it in a separate byte-addressable non-volatile memory device. This extraction eliminates the need for complex indirection tables and defragmentation operations in DRAM, as the byte-addressable memory naturally handles metadata updates without requiring the additional DRAM resources that conventional NAND-based file systems need.
3Productivity
If byte-addressable non-volatile memory is used for metadata, then write efficiency is improved, but device complexity increases due to hybrid architecture
Solution Approach 1:
The patent introduces a file system layer that acts as an intermediary between the host and the hybrid storage architecture. This intermediary handles address translation and routing, presenting a unified interface to applications while managing the underlying complexity of the hybrid byte-addressable and block-addressable memory devices. This abstraction layer simplifies the interface and manages the complexity internally.
Data Source
AI summary
Provided are a computer program product, system and method for managing read/write operations in a hybrid memory device system. Determinations are made of an available physical address in a first memory device for a data block to allocate for metadata for a file or directory in a file system and a first logical address corresponding to the available physical address in a first range of logical addresses. Determinations are made of an available physical address in a second memory device for a data block to allocate for the file or directory in the file system and a second logical address corresponding to the available physical address in the second memory device in a second range of logical addresses. The second logical address is used to access the data block allocated to the file or directory in the file system.