Dynamic Load Sharing for Storage Copy Workloads in Device Clusters
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional data migration techniques in clustered environments are inefficient as they only utilize a single host device for data migration, leading to slow migration times and failure to account for the performance and workload of other nodes, resulting in suboptimal resource utilization.
Innovation Solution
A migration service dynamically distributes copy workloads among multiple host devices based on their performance in previous copy cycles, using a migration node to analyze and adjust workload assignments for each node, ensuring that higher performing nodes handle more data chunks in subsequent cycles.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Device complexity
If a single host device is used for data migration, then the system complexity is reduced, but the migration time increases and resource utilization decreases
Solution Approach 1:
The patent segments the data migration workload into multiple chunks and distributes them across multiple host devices in the cluster. Each host device handles a portion of the migration task simultaneously, dividing the overall migration process into parallel sub-tasks that can be executed concurrently, thereby reducing total migration time without significantly increasing system complexity
Solution Approach 2:
The patent combines multiple host devices into a collaborative migration system where they work together to perform data migration. By merging the capabilities of multiple hosts and coordinating their efforts through a management entity, the system achieves faster migration speeds while maintaining manageable complexity through centralized coordination
2Ease of operation
If a single host device performs data migration, then resource utilization across the cluster is reduced, but the workload management becomes simpler
Solution Approach 1:
The patent implements a feedback mechanism where the system monitors the performance and workload of each host device during migration tasks. Based on this feedback, the management entity dynamically adjusts workload distribution to optimize resource utilization. Higher-performing hosts receive more migration chunks while lower-performing hosts receive fewer, ensuring efficient use of available resources while maintaining simple workload management through automated adjustments
3Productivity
If workloads are distributed among multiple host devices, then resource utilization improves, but the complexity of coordinating multiple nodes increases
Solution Approach 1:
The patent implements a universal management entity that handles multiple functions including workload distribution, performance monitoring, and dynamic adjustment across all host devices. This single multi-functional component coordinates the distributed workload without requiring complex peer-to-peer communication between hosts, thereby improving resource utilization while keeping coordination complexity manageable through centralized multi-functionality
Data Source
AI summary
An apparatus comprises a processing device configured to identify a plurality of data portions from a source storage volume to be copied to a target storage volume, and to analyze performance of respective ones of a plurality of host devices in connection with at least one copying operation. Respective ones of the plurality of host devices are assigned to perform copying of respective subsets of the plurality of data portions to the target storage volume based, at least in part, on the analysis.