Storage System I/O Performance Stabilization During Rebalancing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing storage systems face challenges in maintaining I/O performance during data transfer between different storage devices, particularly when a scale-out storage device is used as the transfer destination, as they do not adequately address the rebalancing process and proper configuration definition.
Innovation Solution
A storage system with a storage management apparatus that forms pool volumes across nodes, allocates data input/output processes, and executes rebalancing to equalize throughput and capacities, determining the transfer destination node and order to minimize performance drops.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If data transfer is executed to a scale-out storage device, then cost is reduced, but I/O performance drops during rebalancing process
Solution Approach 1:
The storage management apparatus determines the transfer destination node and transfer order before executing data transfer. By preliminarily analyzing the cluster configuration and predicting the impact on rebalancing, the system plans the transfer sequence to minimize performance degradation, rather than reacting to performance drops after they occur.
Solution Approach 2:
The storage management apparatus monitors the actual I/O performance during the data transfer process and compares it with expected performance. When performance deviations are detected, the system adjusts the transfer plan in real-time, modifying the transfer order or pausing transfers to maintain acceptable performance levels.
2Stability of the object's composition
If rebalancing process is executed to equalize capacities and performances, then cluster balance is improved, but I/O performance drops during the process
Solution Approach 1:
The system dynamically adjusts the rebalancing process based on current I/O workload conditions. When I/O performance drops are detected during rebalancing, the storage management apparatus can pause or slow down the rebalancing operations. This dynamic control allows the system to maintain cluster balance over time while preventing severe performance degradation during critical periods.
3Productivity
If multiple volumes are transferred simultaneously, then transfer efficiency is improved, but I/O performance drops due to multiple rebalancing operations
Solution Approach 1:
The storage management apparatus segments the transfer of multiple volumes into coordinated batches, assigning each volume to a specific transfer timing based on cluster conditions. Instead of transferring all volumes simultaneously or sequentially one at a time, the system divides the transfer operation into manageable segments that can be executed in an optimized sequence, reducing the cumulative impact on rebalancing operations.
Solution Approach 2:
The system merges multiple volume transfer operations into a unified transfer plan that coordinates with rebalancing needs. By combining transfer decisions and evaluating them together rather than independently, the storage management apparatus can identify opportunities to batch transfers in a way that minimizes the total number of rebalancing operations required, thereby maintaining I/O performance.
Data Source
AI summary
A processor of each node of a storage system executes rebalancing to transfer a volume between pool volumes for the node in such a way as to equalize throughput of a data input/output process and/or data capacities between the pool volumes making up the pool. When a plurality of external volumes are transferred to a node in the storage system, the storage management apparatus determines a transfer destination node for the volumes and order of transferring the volumes and executes volume transfer in such a way as to equalize throughput of the data input/output process and/or data capacities between the pool volumes making up the pool after completion of or during the volume transfer.