Automatic Load Rebalancing for Storage Arrays
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In storage systems with direct-mapped storage devices, the frequent garbage collection processes performed by both storage controllers and internal storage device controllers lead to increased erase block wear, reducing their lifespan and causing inefficiencies in data management, particularly when drive imbalances occur during write operations.
Innovation Solution
Implementing an automatic load rebalancing mechanism that divides segments into shards and distributes them evenly across drives in a write group, allowing the storage controller to manage garbage collection processes externally, reducing redundant operations and optimizing data distribution.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If frequent garbage collection processes are performed by storage controllers and internal storage device controllers, then data management efficiency is improved, but erase block wear increases and lifespan is reduced
Solution Approach 1:
The patent extracts the garbage collection function from internal storage device controllers and consolidates it to external storage controllers. This eliminates redundant garbage collection operations that occur when both internal and external controllers independently manage the same data blocks, thereby reducing erase block wear while maintaining data management efficiency.
Solution Approach 2:
The patent merges multiple garbage collection operations into a single coordinated process managed by external storage controllers. By combining the garbage collection functions of internal and external controllers into one unified management system, the patent prevents duplicate erase operations on the same blocks, extending erase block lifespan.
2Quantity of substance
If drive imbalances occur during write operations, then data distribution becomes inefficient, but storage system performance degrades
Solution Approach 1:
The patent implements a feedback mechanism where external storage controllers monitor the data distribution status across multiple drives in real-time. When imbalances are detected, the system automatically triggers rebalancing operations to redistribute data evenly, preventing performance degradation while maintaining efficient data distribution.
Solution Approach 2:
The patent introduces dynamic load balancing that automatically adjusts data distribution based on current system conditions. The external storage controllers continuously monitor drive utilization and dynamically migrate data to maintain balanced loads across all drives, ensuring optimal storage system performance.
Data Source
AI summary
A method of automatic load rebalancing includes determining that a new storage device was added to a storage array comprising a plurality of storage devices, wherein the new storage device is distinct from the plurality of storage devices. The method further includes, in response to the determining, identifying a first shard on a first storage device of the plurality of storage devices, wherein the first storage device has a fullness metric that is equal to or exceeds a fullness threshold. The method further includes moving, by a processing device of a storage array controller of the storage array, the first shard from the first storage device to the new storage device.