KV SSD Load Balancing via Host-Side RAID Grouping
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional Key-Value Solid State Drives (KV SSDs) face issues with uneven loading and inefficient bandwidth utilization due to random storage of Key-Value pairs, leading to variations in read and write speeds across different Flash Translation Layers (FTLs).
Innovation Solution
The implementation of a system that creates Redundant Array of Independent Disks (RAID) groups for KV SSDs, where a host processor selects KV SSDs and FTL cores based on load, available space, and slab information to efficiently manage Key-Value pairs by splitting values and broadcasting keys for optimal storage and retrieval.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If Key-Value pairs are stored randomly in KV SSDs, then storage flexibility is improved, but load balancing deteriorates and bandwidth utilization becomes inefficient
Solution Approach 1:
The system segments Key-Value pairs into different categories based on their access patterns (hot, warm, cold data) and routes them to appropriate storage locations. This segmentation enables differentiated handling of different data types, improving both storage flexibility and bandwidth utilization by preventing hot data from being stored in locations with limited access speed.
Solution Approach 2:
The patent implements dynamic load balancing that continuously monitors the load status of different FTLs and adjusts the routing of Key-Value pairs in real-time. This dynamic approach allows the system to adapt to changing workload conditions, maintaining optimal bandwidth utilization while preserving storage flexibility.
2Adaptability or versatility
If Key-Value pairs are stored randomly in KV SSDs, then storage flexibility is improved, but load balancing deteriorates
Solution Approach 1:
The system employs feedback mechanisms that continuously monitor the load distribution across multiple FTLs and adjust the routing decisions accordingly. When an FTL becomes overloaded, the feedback loop redirects subsequent Key-Value pairs to underutilized FTLs, maintaining stable and uniform load distribution while preserving storage flexibility through adaptive routing.
Solution Approach 2:
The dynamic load balancing algorithm continuously adapts to changing system conditions, adjusting the distribution of Key-Value pairs across FTLs in real-time. This dynamic approach ensures stable load distribution without imposing rigid storage constraints, allowing the system to maintain both flexibility and balance.
3Device complexity
If data is stored in a single FTL, then device complexity is reduced, but read and write speed becomes uneven
Solution Approach 1:
The patent merges multiple FTLs into a unified logical storage system that presents a single interface to the host. This merging approach maintains simplicity from the host's perspective while enabling internal load distribution across multiple physical FTLs, ensuring consistent read and write speeds without increasing perceived device complexity.
Solution Approach 2:
The system introduces an intermediary layer (the controller) that manages data distribution across multiple FTLs. This intermediary maintains the simplicity of the overall device structure by abstracting the complexity of multi-FTL management, while simultaneously ensuring consistent performance by balancing loads across all FTLs.
Data Source
AI summary
Methods and systems for managing Key-Value Solid State Drives (KV SSDs). A method includes writing, by a host processor, at least one Key-value pair of at least one write command to at least one KV SSD of a plurality of KV SSDs of at least one RAID group based on at least one of slab information, available space and load. Further, the method includes reading, by the host processor, at least one value from the at least one KV SSD of the plurality of KV SSDs for at least one key of at least one read command using at least one of a consistent hashing function and slab information.