Storage Controller Load Distribution for HSM Migration
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing hierarchical storage management systems face inefficiencies in data migration due to uneven load distribution among storage clusters, leading to suboptimal performance and increased time for completion of copy operations.
Innovation Solution
A method and system for scheduling copy operations in a hierarchical storage management system with a grid configuration, distributing loads among multiple storage nodes, and coordinating individual storage node operations to optimize system-level performance and throughput, including the use of a storage controller with a processor and memory to manage workload and execute copy policies.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If copy operations are performed without load distribution scheduling, then copy operations can be executed simply, but system performance and throughput are suboptimal due to uneven load distribution
Solution Approach 1:
The patent replaces manual or simple copy operations with an automated scheduling system that uses algorithms to distribute loads across storage nodes. The storage controller automatically monitors load conditions and coordinates copy operations, substituting mechanical/manual intervention with intelligent automation to optimize system performance without proportionally increasing operational complexity.
Solution Approach 2:
The scheduling system dynamically adjusts copy operation distribution based on real-time load conditions of storage nodes. Instead of static load distribution, the system continuously monitors and adapts the scheduling decisions, allowing storage nodes with lighter loads to handle more copy operations while balancing the overall system performance and preventing any single node from becoming a bottleneck.
2Productivity
If copy operations are distributed among multiple storage nodes, then system throughput is optimized, but coordination complexity increases
Solution Approach 1:
The storage controller is designed as a universal coordinating entity that handles multiple functions: monitoring load conditions, making scheduling decisions, and managing copy operations across different storage nodes. This multi-functional approach consolidates coordination complexity into a single controller rather than requiring complex peer-to-peer coordination between storage nodes, thereby enabling distributed operations without proportionally increasing system complexity.
Solution Approach 2:
The storage controller acts as an intermediary between the data migration system and individual storage nodes. It receives copy operation requests, determines optimal distribution based on load conditions, and directs operations to appropriate nodes. This intermediary role simplifies the coordination problem by centralizing decision-making logic while still enabling parallel distributed execution across multiple nodes.
3Productivity
If load distribution scheduling is implemented, then copy operation efficiency improves, but system complexity increases
Solution Approach 1:
The storage nodes are equipped with self-monitoring capabilities that automatically report their load conditions to the storage controller. Rather than requiring complex external monitoring infrastructure, the system uses self-service mechanisms where each node independently tracks its own status and makes this information available for scheduling decisions, reducing the overall system complexity while maintaining efficient load distribution.
Solution Approach 2:
The scheduling system implements feedback loops where storage nodes continuously report their load status to the controller, which adjusts scheduling decisions based on this feedback. This closed-loop control mechanism enables the system to adapt to changing conditions automatically, improving copy operation efficiency without requiring complex manual intervention or overly sophisticated scheduling algorithms.
Data Source
AI summary
Embodiments provide a method, a system, and a computer program product for performing copy operations of one or more data units in a hierarchical storage management (HSM) system. The HSM system includes an upper layer and a lower layer. The upper layer includes multiple storage nodes having a grid configuration. The method comprises scheduling a copy operations of multiple data units each of which is stored in at least one of the multiple storage nodes such that loads on the copy operations are distributed among the multiple storage nodes in which the multiple data units are stored and copying the multiple data units to the lower layer in accordance with the scheduling.