Hybrid Storage Deduplication via File Characteristic Mapping
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing deduplication processes in hybrid storage networks do not consider file characteristics or disk technologies, leading to suboptimal storage performance and resource utilization during RAID rebuilds, as they fail to account for the specific attributes of different storage systems and RAID configurations.
Innovation Solution
The method involves determining file characteristics and mapping them to storage preferences, which specify suitable storage systems based on attributes such as disk technology and RAID configurations, optimizing storage locations for redundant data sets and optimizing system resources during RAID rebuilds by restricting inline deduplication when necessary.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If traditional deduplication processes are used without considering file characteristics or disk technology, then storage space is reduced by eliminating duplicates, but storage performance and resource utilization become suboptimal
Solution Approach 1:
The patent applies local quality by selecting different storage locations for different files based on their specific characteristics. Instead of a uniform deduplication approach, the system analyzes file attributes (size, access patterns, type) and maps them to appropriate storage systems with matching characteristics, ensuring each file is stored in the most suitable location for its specific needs.
Solution Approach 2:
The system changes parameters by considering multiple file characteristics (size, access frequency, file type) and storage system attributes (disk technology, RAID configuration, performance metrics) to dynamically determine optimal storage locations. This parameter-based mapping allows the system to adapt storage decisions to specific file and storage conditions rather than using a fixed approach.
2Quantity of substance
If deduplication processes do not consider RAID configurations, then storage capacity is maximized, but system resources are consumed during RAID rebuilds
Solution Approach 1:
The patent applies preliminary action by proactively selecting storage locations with appropriate RAID configurations before rebuild operations are needed. The system analyzes RAID array states and avoids placing deduplicated files on arrays that would require costly rebuild operations, thereby preventing future resource consumption during rebuilds rather than reacting when rebuilds are necessary.
3Device complexity
If storage locations are selected without considering disk technology attributes, then deduplication is simpler to implement, but read and write performance are reduced
Solution Approach 1:
The system changes parameters by establishing a mapping between file characteristics and storage system attributes including disk technology type. This parameter-based approach systematically considers SSD versus HDD characteristics, IOPS capabilities, and other performance metrics to select optimal storage locations, transforming a potentially complex decision into a structured parameter-matching process.
Data Source
AI summary
Deduplication in a hybrid storage environment includes determining characteristics of a first data set. The first data set is identified as redundant to a second data set and the second data set is stored in a first storage system. The deduplication also includes mapping the characteristics of the first data set to storage preferences, the storage preferences specifying storage system selections for storing data sets based upon attributes of the respective storage systems. The deduplication further includes storing, as a persistent data set, one of the first data set and the second data set in one of the storage systems identified from the mapping.