Deduplication Storage Tier Migration via Per-Object Utilization
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The challenge in data backup systems is the inefficient use of storage resources due to the storage of deduplicated data, which can lead to exceeding storage tier thresholds, necessitating intelligent data placement decisions to manage storage utilization effectively.
Innovation Solution
A deduplication system that divides data into blocks, generates block signatures, and uses manifests and container indexes to identify duplicates, allowing for efficient storage and migration of data across storage tiers based on determined storage utilization, with a storage manager determining the per-copy size of objects and adjusting their storage tiers to prevent resource exhaustion.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If data is stored in deduplicated form to save storage space, then storage efficiency is improved, but storage utilization becomes difficult to track accurately, leading to threshold overages
Solution Approach 1:
The patent segments deduplicated data into discrete blocks and tracks each block's storage utilization independently. By dividing the deduplicated data structure into manageable segments (data blocks with associated metadata), the system can accurately measure and control storage utilization for each segment, preventing threshold overages while maintaining deduplication efficiency.
2Reliability
If storage tier thresholds are set to prevent resource exhaustion, then storage resource management is improved, but data placement becomes more complex, requiring intelligent allocation decisions
Solution Approach 1:
The patent implements self-service mechanisms where the storage system automatically monitors its own utilization metrics and performs intelligent data placement decisions without external intervention. The system uses automated policies to evaluate storage tier thresholds and migrate data blocks between tiers based on real-time utilization data, reducing the complexity of manual data placement while improving reliability.
Solution Approach 2:
The patent establishes feedback loops that continuously monitor storage utilization metrics and use this information to dynamically adjust data placement decisions. The system collects feedback on actual storage consumption, compares it against threshold policies, and automatically triggers data migration or retention actions, creating a closed-loop control system that simplifies management while ensuring reliable resource allocation.
Data Source
AI summary
Example implementations relate to determining a storage utilization attributable to object data stored in deduplicated form. The storage utilization attributable to the object data may be determined from an amount of object data not shared with other objects of the deduplication store and a portion of an amount of object data shared with other objects of the deduplication store. It is determined whether the storage utilization results in exceeding a storage threshold of a storage tier to which the object data is assigned. Where the storage utilization is determined to exceed the storage threshold, the object data may be reassigned to a different storage tier, or data of another object may be removed from the storage tier.