Distributed Storage Load Balancer with Automated Migration
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In distributed storage systems, load balancing across storage arrays is typically manual and based on observations, lacking automation in identifying which disk images or logical groups of files to migrate and where, leading to inefficiencies in optimizing performance.
Innovation Solution
Implement online monitoring of workloads and performance across storage units to identify and migrate objects, such as disk volumes or virtual machine disk images, between storage units based on calculated normalized load metrics, enabling automated load balancing across different vendor configurations.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Extent of automation
If manual load balancing based on observations is used, then storage array performance can be optimized, but the system lacks automation in identifying what to move and where
Solution Approach 1:
The system performs self-service by automatically monitoring storage array loads, identifying imbalances, selecting candidate objects for migration, and executing migrations without human intervention. The load balancer continuously gathers performance metrics, computes normalized load metrics, and autonomously makes decisions about which disk images or logical groups of files to migrate between storage arrays, eliminating the need for manual observation and decision-making processes.
2Adaptability or versatility
If multiple storage arrays with different vendor configurations are used, then storage capacity and flexibility are increased, but load balancing becomes more complex
Solution Approach 1:
The system handles diverse vendor configurations by normalizing performance metrics through parameter transformation. It collects raw performance data from storage arrays of different vendors, applies vendor-specific normalization factors and algorithms to convert these metrics into a common standardized format, and then performs load balancing comparisons. This allows the system to work with heterogeneous storage infrastructure while maintaining consistent load balancing logic across all arrays regardless of vendor or configuration differences.
Data Source
AI summary
Distributed storage resources are managed based on data collected from online monitoring of workloads on the storage resources and performance characteristics of the storage resources. Load metrics are calculated from the collected data and used to identify workloads that are migration candidates and storage units that are migration destinations, so that load balancing across the distributed storage resources can be achieved.


