Multi-tier Storage Data Mover Modules for Distributed Parallel Transfer
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional storage tiering arrangements in parallel file systems are inflexible and inefficient when transferring large files or data objects between storage tiers, leading to wasteful use of resources.
Innovation Solution
Implementing a multi-tier storage system with distributed multi-part data movement using configurable data mover modules that transfer non-overlapping parts of data objects between storage tiers, utilizing a distributed multi-part data movement protocol with phases for efficient data transfer between a fast tier and a capacity tier.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If conventional storage tiering arrangements are used in parallel file systems, then data can be moved between storage tiers, but the transfer of large files or data objects is inefficient and wasteful of storage system resources
Solution Approach 1:
The patent divides large data objects into multiple smaller parts and distributes them across multiple data mover modules for parallel transfer between storage tiers. Each data mover module handles a specific portion of the data object, enabling concurrent processing and improving overall transfer efficiency while reducing resource waste.
2Adaptability or versatility
If conventional storage tiering arrangements are used, then data movement between tiers is possible, but the system lacks flexibility when transferring large files or data objects
Solution Approach 1:
The patent implements a dynamic data transfer mechanism where the system can adaptively select the number and configuration of data mover modules based on the size and characteristics of the data object being transferred. This dynamic approach provides flexibility for handling different data types and sizes while maintaining manageable system complexity through modular design.
3Loss of time
If single data mover modules are used for data transfer, then the system structure is simpler, but transfer time for large data objects is longer
Solution Approach 1:
The patent merges multiple data mover modules to work in parallel on different portions of the same data object. By combining the capabilities of multiple modules and coordinating their operations through a distributed protocol, the system achieves faster transfer times for large data objects while managing complexity through structured coordination mechanisms.
Data Source
AI summary
An apparatus in one embodiment comprises a storage system having at least first and second storage tiers each comprising a plurality of storage devices. The storage system implements a plurality of data mover modules for controlling movement of data objects between the first and second storage tiers. A given one of the data objects is transferred between the first and second storage tiers utilizing a distributed multi-part data movement protocol in which different portions of the given data object are transferred between the first and second storage tiers by different ones of the data mover modules. For example, each of the different portions may be transferred between the first and second storage tiers by its corresponding data mover module as a plurality of distinct non-overlapping parts. In such an arrangement, each of the plurality of distinct non-overlapping parts of each of the different portions may have a corresponding unique tag identifier.