Storage Node Resource Balancing via Dynamic Multipath Switching
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing internode load balancing in dual node storage systems at a file server granularity leads to large workload fluctuations and instability due to unequal file system distribution and difficulty in modeling the impact of workload movements, making it challenging to achieve balanced workload states between nodes.
Innovation Solution
Redirecting block I/O workload of selected file systems between nodes using dynamic path state changes in a multipath configuration without relocating file servers or file systems, allowing for finer granularity balancing and gradual workload redistribution.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If internode load balancing is performed at file server granularity, then workload distribution between nodes can be adjusted, but large workload fluctuations and system instability occur due to unequal file system distribution
Solution Approach 1:
The patent segments the file system into individual files as the granular unit for load balancing. Instead of moving entire file servers or file systems between nodes, the system selectively migrates individual files from one node to another based on workload metrics. This fine-grained segmentation allows for precise control over workload distribution while maintaining system stability, as only specific files contribute to the balancing operation rather than large aggregates of data.
Solution Approach 2:
The patent implements dynamic load balancing by continuously monitoring workload metrics and adjusting file placements in real-time. The system dynamically determines which files to migrate based on current node workload states, and can reverse migrations if workload conditions change. This dynamic approach allows the system to adapt to changing conditions without causing instability, as the load balancing operation is reversible and responsive to actual system state rather than following a fixed schedule or policy.
2Productivity
If file servers or file systems are relocated between nodes to balance workload, then internode workload imbalance can be reduced, but the complexity of modeling impact and achieving balanced states increases
Solution Approach 1:
The patent applies local quality by making load balancing decisions at the individual file level rather than uniformly across entire file servers or file systems. Each file is evaluated independently based on its contribution to node workload, and migration decisions are made locally for each file based on simple criteria such as node capacity and file access patterns. This local decision-making approach simplifies the overall control logic compared to complex global optimization algorithms, while still achieving effective workload balance.
Solution Approach 2:
The patent implements self-service load balancing where the system automatically monitors its own workload state and performs migrations without external intervention. The load balancing mechanism uses simple, locally-available metrics such as node CPU utilization, memory usage, and I/O workload to make migration decisions. This self-service approach eliminates the need for complex external control systems or manual configuration, reducing overall system complexity while maintaining effective load balance.
Data Source
AI summary
In at least one embodiment, processing can include: configuring a file server on a first node of a storage system, the file server including a file system configured on a volume; configuring a first path exposing the volume to the file server as active-optimized (AO) and configuring a second path exposing the volume to the file server as active-non-optimized (ANO); for the volume, while the first path's state is AO and the second path's state is ANO, determining an internode I/O workload imbalance between the first node and the second node; and responsive to determining the internode I/O workload imbalance, reducing or removing the internode I/O workload imbalance including: redirecting I/O workload of the volume from the first node to the second node over the second path by modifying the first path's state from AO to ANO and by modifying the second path's state from ANO to AO.