Immutable Deduplication Space Reclamation via Container References
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Deduplication systems face challenges in managing storage space efficiently in immutable deduplication systems due to the inability to compact containers until their retention period has elapsed, leading to unnecessary storage of expired data and increased costs.
Innovation Solution
Implementing threshold-based mechanisms to manage container references in immutable storage, using static or dynamic thresholds to determine when containers are unavailable for referencing, thereby allowing for predictable space reclamation and reducing unnecessary storage of expired data.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If containers are stored in immutable storage with fixed retention periods, then data integrity and compliance are ensured, but storage space cannot be reclaimed until the retention period elapses, leading to unnecessary storage of expired data
Solution Approach 1:
The patent segments the retention period into multiple phases: an immutable phase where containers cannot be modified or deleted, and a reclamation phase where containers become eligible for deletion. This segmentation allows the system to maintain data integrity during the immutable phase while enabling space reclamation during the reclamation phase, resolving the contradiction between reliability and storage space efficiency
Solution Approach 2:
The patent implements preliminary actions by establishing predetermined reclamation policies and thresholds before the retention period expires. The system proactively identifies containers that will become eligible for deletion and prepares reclamation processes in advance, allowing smooth transition from immutable storage to space reclamation without compromising data integrity
2Reliability
If the retention period is extended to ensure compliance, then data protection is improved, but storage costs increase due to prolonged retention of expired data
Solution Approach 1:
The patent introduces dynamic reclamation policies that adjust retention behavior based on multiple factors including compliance requirements, storage capacity, and data access patterns. The system can dynamically extend or reduce retention periods for different container types, allowing optimization of storage costs while maintaining necessary data protection levels
Solution Approach 2:
The patent changes key parameters such as retention period duration, reclamation thresholds, and priority levels based on container characteristics and organizational policies. By adjusting these parameters dynamically, the system can extend retention for compliance-critical data while reducing retention for less critical data, thereby optimizing the balance between data protection and storage costs
3Reliability
If all containers are retained until their retention period expires, then compliance requirements are met, but storage efficiency decreases due to accumulation of expired data
Solution Approach 1:
The patent implements a copying mechanism where containers eligible for reclamation are copied to archival storage or backup systems before deletion from primary immutable storage. This ensures compliance requirements are met by preserving data copies while improving storage efficiency by removing expired data from the primary storage system
Solution Approach 2:
The patent introduces an intermediary reclamation management system that acts as a mediator between the immutable storage system and the deletion process. This intermediary layer monitors container eligibility, manages the reclamation queue, and coordinates deletion operations, ensuring compliance is maintained while enabling efficient storage space reclamation
Data Source
AI summary
Embodiments are disclosed that provide space reclamation in immutable deduplication systems, and can include selecting a unit of data of a backup image, determining whether a duplicate unit of data is stored in an existing data storage construct (the duplicate unit of data is a duplicate of the unit of data and the existing data storage construct is stored in immutable storage), and in response to a determination that the duplicate unit of data exists in the existing data storage construct, determining whether the existing data storage construct is designated as being available to be referenced, in response to the existing data storage construct being designated as being available to be referenced, updating a reference to the duplicate unit of data, and in response to the existing data storage construct being designated as being unavailable to be referenced, storing the unit of data in a new data storage construct.