Multi-Stream SSD Storage Separating User Data and Metadata
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In distributed storage systems using solid-state drives (SSDs), the overhead introduced by internal garbage collectors can significantly impact availability and response time due to the need for data movement and garbage collection processes, especially in systems handling large volumes of reads and writes.
Innovation Solution
The technique involves organizing physical large blocks (PLBs) into super-stripes and utilizing the UNMAP command to separate user data and metadata storage, ensuring they are written to different data streams on SSDs, thereby eliminating the need for internal garbage collector operations by marking entire erase units as unused.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If data and metadata are stored together in the same erase units on SSDs, then storage space utilization is improved, but internal garbage collector overhead increases significantly
Solution Approach 1:
The patent segments the storage system into two distinct data streams: a first data stream for user data and a second data stream for metadata. This segmentation ensures that user data and metadata are written to different erase units on the SSDs, allowing independent management and deletion operations. When user data is deleted, only the corresponding first erase units need to be reclaimed, without triggering garbage collection on erase units containing metadata, thus reducing garbage collector overhead while maintaining storage space utilization.
2Device complexity
If traditional log-structured storage is used where data and metadata share the same storage medium, then device complexity is reduced, but system availability deteriorates due to garbage collection interruptions
Solution Approach 1:
The patent divides the storage medium into separate data streams for user data and metadata, each with its own erase units. This segmentation allows the storage system to maintain a relatively simple log-structured organization while improving availability by preventing garbage collection interruptions during metadata operations. The separate streams enable independent deletion and reclamation operations that do not trigger extensive garbage collection processes.
Solution Approach 2:
The patent introduces a stream identifier as an intermediary mechanism that directs writes to appropriate data streams based on the type of data being stored. This intermediary layer manages the separation between user data and metadata without adding significant complexity to the overall storage structure, enabling the system to achieve improved availability through selective deletion operations.
3Productivity
If entire erase units are marked as unused through UNMAP command, then deletion efficiency is improved, but the requirement for separate data streams increases device complexity
Solution Approach 1:
The patent segments storage operations into two distinct data streams with separate erase units, enabling efficient bulk deletion of user data by marking entire first erase units as unused via UNMAP commands without affecting metadata storage. This segmentation makes the increased device complexity worthwhile by achieving significant improvements in deletion efficiency and overall system performance.
Solution Approach 2:
The patent changes the organizational parameter of storage by introducing separate data streams with different erase unit structures. This parameter change enables the system to mark entire erase units as unused efficiently, improving deletion productivity despite the increased complexity of managing multiple data streams and their associated erase units.
Data Source
AI summary
A method is provided for use in a storage processor, the method comprising: receiving a write request, the write request including a request to store user data in an array that includes a plurality of solid-state drives (SSD); executing the write request by: identifying metadata that is associated with the write request, and writing the user data and the metadata to different data streams that are opened on the plurality of SSDs; wherein writing the user data and the metadata to different data streams causes: (i) the user data to be stored in one or more first erase units of any of the plurality of SSDs, and (ii) the metadata to be stored in one or more second erase units of any of the plurality of SSDs, such that no part of the metadata is stored on any of the one or more first erase units, and no part of the user data is stored on any of the one or more second erase units.