Storage System Data Migration via Deduplication Ratio Analysis
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional data migration methods in deduplication storage systems are inefficient, leading to high storage costs due to overlapping data, as they fail to effectively identify and migrate data with low deduplication ratios, resulting in minimal storage space savings and increased costs.
Innovation Solution
A storage management method that determines the deduplication ratio of files by counting data segment occurrences and migrating files with a deduplication ratio lower than a threshold from local to remote storage, utilizing techniques like virtual synthesis and fast copy/overwrite for incremental backups to maintain data protection efficiency and reduce storage costs.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of energy
If data is migrated from local storage to remote storage in a deduplication storage system, then storage costs are reduced, but the deduplication ratio deteriorates due to overlapping data content
Solution Approach 1:
The system calculates the deduplication ratio of each file before migration decisions are made. By performing this preliminary analysis, the system identifies files with low deduplication ratios that are suitable for migration, thereby avoiding migration of files that would result in poor deduplication performance and ensuring that only files contributing to actual storage savings are migrated.
Solution Approach 2:
The system uses the deduplication ratio as a key parameter to determine migration eligibility. Files are migrated based on their deduplication ratio being below a certain threshold, transforming the static migration decision into a dynamic parameter-based selection process that optimizes both storage cost reduction and deduplication efficiency.
2Ease of operation
If files are migrated without calculating deduplication ratios, then migration process is simplified, but storage space savings are minimized due to overlapping data
Solution Approach 1:
The system automatically calculates deduplication ratios and makes migration decisions without requiring manual intervention. The storage management module autonomously identifies suitable files for migration based on deduplication analysis, eliminating the need for complex manual selection processes while maximizing storage space savings through data-driven decisions.
3Area of stationary object
If all files are migrated to remote storage, then local storage capacity is increased, but remote storage costs increase due to duplicate data storage
Solution Approach 1:
Instead of migrating all files, the system performs partial migration by selectively choosing only those files with low deduplication ratios for migration. This partial action approach ensures that local storage capacity is freed up effectively while avoiding the excessive cost of migrating and storing duplicate data in remote storage systems.
Data Source
AI summary
Embodiments of the present disclosure provide a storage management method, an electronic device, and a computer program product. The method includes determining at least one count corresponding to at least one data segment of a file in a file set, the file set being stored in a local storage device, and the at least one count indicating the number of occurrences of the at least one data segment in the file set. The method further includes determining a deduplication ratio of the file based on the at least one count, the deduplication ratio indicating an overlapping level of the file with other files in the file set. The method further includes migrating the file from the local storage device to a remote storage device according to a determination that the deduplication ratio of the file is lower than a threshold.