Cache-Based Deduplication Migration via Hash Sorting
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In systems where deduplication occurs only against a cache of the most recent writes, there is often a lack of deduplication of all same copies of a page, resulting in lower deduplication ratios on the target array compared to the donor array, and existing data migration approaches may not optimize deduplication effectively.
Innovation Solution
The method involves sorting hash values of data at the source and migrating it to the target according to the sorted order, using a series of READ NEXT PAGE commands with inputs including volume, page size, and session ID, and outputs including volumes and offsets, allowing for optimal deduplication by maximizing cache hits and ensuring data is reordered for higher deduplication ratios.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of manufacture
If data is migrated without sorting hash values, then migration process is simpler, but deduplication ratio on target array is lower
Solution Approach 1:
The patent applies preliminary action by sorting hash values of data blocks at the source storage system before migration. This pre-sorting organizes data in advance so that identical or similar data blocks are grouped together, enabling the target storage system's cache-based deduplication to achieve higher deduplication ratios without complex real-time processing during migration.
2Productivity
If data is migrated in random order, then cache hits are minimized, but deduplication efficiency is reduced
Solution Approach 1:
The patent changes the ordering parameter of data migration from random or sequential to hash-value-sorted order. By transforming the migration sequence based on hash values, the system maximizes cache hits in the target storage system, improving deduplication efficiency without significantly impacting migration speed.
3Speed
If cache-based deduplication is used without sorted hash values, then deduplication process is faster, but deduplication ratio is lower
Solution Approach 1:
The patent applies preliminary action by pre-sorting data blocks according to their hash values before migration. This preparation ensures that when the target storage system processes data through its cache-based deduplication mechanism, identical data blocks are encountered in sequence, maximizing both the deduplication ratio and maintaining fast cache-based processing speed.
Data Source
AI summary
A method, computer program product, and computer system for preparing, by a computing device, for migration of data from a source to a target. Hash values of the data may be sorted at the source. The data may be migrated from the source to the target according to how the data was sorted at the source.