Data Migration Preserving Storage Efficiency via Deduplication
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Traditional data migration techniques fail to preserve storage efficiency information, leading to redundant data transfer and increased bandwidth usage during data migration processes in clustered network storage environments.
Innovation Solution
The technology utilizes deduplication information to identify shared data blocks between files, allowing the destination server to create files by referencing shared data blocks instead of duplicating their content, thereby reducing bandwidth requirements and preserving storage efficiency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of energy
If traditional data migration techniques are used to transfer data from source to destination, then data migration is completed, but storage efficiency information is lost and redundant data is transferred
Solution Approach 1:
The system performs preliminary deduplication on the source data before migration, identifying and eliminating redundant data blocks in advance. This preliminary action ensures that only unique data blocks are migrated, preserving storage efficiency information and reducing bandwidth consumption during the actual migration process.
Solution Approach 2:
Instead of copying all source data blocks to the destination, the system creates a copy only for unique data blocks that were not deduplicated on the source. This selective copying approach maintains storage efficiency by avoiding redundant data transfer while ensuring complete data reconstruction at the destination.
2Reliability
If all source data is migrated to destination without deduplication, then complete data is transferred, but bandwidth consumption increases significantly
Solution Approach 1:
The system performs preliminary deduplication analysis on the source data to identify unique blocks before migration. This preliminary action ensures that data completeness is maintained by tracking which blocks need to be migrated, while simultaneously reducing bandwidth consumption by excluding redundant blocks from the transfer.
Solution Approach 2:
The system selectively copies only the unique data blocks identified through deduplication to the destination, rather than copying all source data. This ensures data completeness for the necessary blocks while minimizing bandwidth consumption by leaving out redundant duplicates.
3Loss of energy
If deduplication is performed on source data before migration, then bandwidth consumption is reduced, but additional processing time is required
Solution Approach 1:
The system performs deduplication as a preliminary action before migration, accepting the additional processing time as a necessary trade-off to achieve significant bandwidth reduction during the actual data transfer phase. The preprocessing identifies unique blocks that need to be migrated, eliminating redundant transfers.
4Reliability
If traditional migration copies all data blocks, then data integrity is maintained, but storage efficiency is not preserved at destination
Solution Approach 1:
The system performs preliminary deduplication on source data and preserves the deduplication information through migration. This allows the destination to reconstruct data with the same storage efficiency as the source, maintaining data integrity while preserving storage efficiency by avoiding redundant block creation.
Solution Approach 2:
The system selectively copies only unique data blocks to the destination and uses deduplication references to represent redundant blocks. This approach maintains data integrity by ensuring all necessary blocks are present while preserving storage efficiency by not creating duplicate copies of identical data blocks.
Data Source
AI summary
Technology is disclosed for a data migration process for a storage server that preserves storage efficiency information. The storage server receives an indication of a group of two or more files and selects among the two or more files a reference file and one or more selected files. The storage server initiates a first migration operation to copy or move the reference file from a source storage server to a destination storage server. The storage server initiates one or more additional migration operations to copy or move the selected files from the source storage server to the destination storage server. At least one of the additional migration operations include a step of transmitting to the destination storage server data blocks of the selected files that are not shared between the reference file and the selected files, but avoid transmitting to the destination storage server the blocks shared with the reference file.