Bulk Data Migration Using Candidate Index for Storage Segments
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Data migration between storage tiers in deduplication systems is inefficient due to repeated segment identification processes, which slow down as locality degrades over time, especially during space reclamation processes.
Innovation Solution
Implementing a two-phase data migration process involving segment identification and bulk data movement, where a candidate index is used to identify and retrieve deduplicated segments once, reducing unnecessary disk accesses and optimizing storage space estimation between tiers.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If files are moved between storage tiers using traditional file-by-file migration methods, then data can be migrated between tiers, but the process is slow and inefficient due to repeated segment identification for each file
Solution Approach 1:
The patent segments the migration process into two distinct phases: (1) segment identification phase where all unique segments across multiple files are identified and stored in a candidate index, and (2) data movement phase where segments are moved in bulk. This segmentation eliminates repeated segment identification for each file, significantly improving migration efficiency.
Solution Approach 2:
The patent performs preliminary segment identification and creates a candidate index before actual data movement. This preliminary action identifies all unique segments across multiple files in advance, so that during the subsequent bulk data movement phase, segments can be transferred without repeated identification overhead, thus accelerating the overall migration process.
2Quantity of substance
If deduplication is performed on multiple tiers of storage, then storage efficiency is improved and space is saved, but segment identification must be repeatedly performed for each file migration
Solution Approach 1:
The patent merges the segment identification process across multiple files by creating a unified candidate index that contains all unique segments from all files to be migrated. This combining approach eliminates redundant segment identification operations and reduces the overall complexity of managing deduplication across multiple files during migration.
Solution Approach 2:
The patent creates a candidate index that serves as a copy or representation of all unique segments across multiple files. This index structure allows the system to work with a consolidated view of segments rather than repeatedly analyzing each file individually, simplifying the deduplication management process during migration.
3Quantity of substance
If space reclamation processes are performed in deduplication storage systems, then storage space is optimized, but locality degradation occurs which slows down subsequent file access and migration
Solution Approach 1:
The patent performs preliminary segment identification and creates a candidate index before space reclamation affects data locality. By identifying all unique segments and preparing the migration index in advance, the system captures the segment structure before locality degradation occurs, enabling efficient bulk data movement despite subsequent space reclamation processes.
Data Source
AI summary
Techniques for data migration of a storage system are described herein. According to one embodiment, a candidate index is generated to store information identifying a plurality of segments associated with one or more candidate files of a source storage tier to be migrated to a target storage tier of a storage system. Segments of the one or more candidate files are then migrated from the source storage tier to the target storage tier according to a sequential order, and based on the candidate index.