Object Copy Driver Module for Low-Overhead Volume Migration
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional volume migration techniques incur high costs and inefficiencies due to multiple read and write operations when transferring data between storage tiers, particularly when moving data from a slower, lower-cost capacity tier to a faster, higher-cost storage tier.
Innovation Solution
A cost-optimized volume migration technique using backend object copy operations that transfers metadata instead of actual data, employing an object copy driver module and work queue to manage the migration process, thereby reducing network usage and operational costs.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If conventional volume migration techniques are used to transfer data between storage tiers, then data migration is achieved, but the number of read and write operations increases significantly and network utilization increases
Solution Approach 1:
The patent uses backend object copy operations to create copies of objects directly from the source object store to the destination object store without reading the actual data through the system. This copying mechanism allows metadata to be transferred while the actual data remains in place temporarily, reducing network utilization and operational costs during migration
Solution Approach 2:
The patent extracts and transfers only the metadata of objects instead of the actual data during migration. By separating metadata from data and transferring only the essential information needed for migration tracking and management, the system significantly reduces network usage and the number of read/write operations required
2Productivity
If multiple read and write operations are performed during volume migration, then data is transferred between storage tiers, but operational costs increase
Solution Approach 1:
Backend object copy operations are used to migrate data between storage tiers by creating direct copies from source to destination object stores. This approach reduces the number of read and write operations compared to conventional techniques that require explicit read-modify-write cycles, thereby lowering operational costs
Solution Approach 2:
Only metadata is extracted and transferred during the migration process rather than the entire data set. This extraction principle minimizes the amount of data processing required while still achieving the migration objective, reducing operational costs associated with data transfer
3Productivity
If objects are tiered between storage tier and capacity tier, then storage efficiency is improved, but access speed to capacity tier objects is slower
Solution Approach 1:
The system segments data into frequently accessed objects and infrequently accessed objects, placing them in appropriate storage tiers. The backend object copy driver enables efficient migration by processing objects in batches and using work queues to manage the segmentation and transfer process between storage tiers
Solution Approach 2:
The object copy driver module acts as an intermediary that manages the tiering process between storage tier and capacity tier. It uses work queues to coordinate object migration and maintains the tiering structure, allowing efficient management of the trade-off between storage efficiency and access speed
Data Source
AI summary
Techniques are provided for migrating a volume utilizing an object copy work queue and an object copy driver module. Data of the volume is stored within objects stored across a storage tier and capacity tier of a source object store. As part of migrating the volume to a destination object store, the objects are migrated to the destination cluster. Directly copying the objects involves multiple read operations to the source object store and a write operation at the destination object store. The techniques provided herein improve the efficiency of the migration by initially sending metadata from the source object store to the destination object store for performing backend object copy operations to migrate the volume. This results in fewer operations and less network usage, thus improving the efficiency and cost of migrating the volume.