Deduplication Storage Space Reclamation via Selective Compaction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Deduplication systems face inefficiencies in storage space reclamation, particularly in remote storage environments where compaction processes are resource-intensive and time-consuming, and fail to effectively manage fragmented storage constructs.
Innovation Solution
The method involves identifying storage constructs with low usage, removing their fingerprints from the fingerprint cache, and deleting or marking them for garbage collection, thereby treating remaining data segments as unused and duplicating them into new containers during backup operations, optimizing storage space usage and reducing the life cycle of storage objects.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of energy
If compaction processes are performed on storage constructs in remote storage environments, then storage space can be reclaimed, but the processes become resource-intensive and time-consuming
Solution Approach 1:
The patent applies partial action by performing compaction only on storage constructs that meet specific criteria (low usage threshold, not referenced by recent backup images). Instead of compacting all storage constructs, the system selectively identifies and processes only those containers with container IDs present in the differential container ID list, thereby reducing resource consumption while still achieving space reclamation goals.
Solution Approach 2:
The system applies different treatment to different storage constructs based on their local characteristics. Storage constructs are evaluated individually against reclamation criteria, and only those meeting the criteria (low usage, old backup images) are selected for compaction. This localized approach ensures resources are spent only where necessary rather than uniformly across all storage constructs.
2Loss of energy
If compaction processes are performed on storage constructs, then storage space can be reclaimed, but the processes are time-consuming
Solution Approach 1:
The patent implements partial action by limiting compiction operations to only those storage constructs that satisfy specific reclamation criteria. The system processes a subset of containers (those with IDs in the differential list) rather than all containers, significantly reducing the time required for space reclamation while maintaining effectiveness.
Solution Approach 2:
The system performs preliminary identification and filtering of storage constructs before executing compaction. By pre-computing the differential container ID list and identifying candidate containers in advance, the system prepares the work queue before actual compaction begins, reducing the overall time spent on the reclamation process.
3Quantity of substance
If storage constructs are fragmented, then data can be stored efficiently, but restoration operations become slower due to poor data locality
Solution Approach 1:
The patent applies merging by consolidating data segments from fragmented storage constructs into newly created contiguous containers. During the compaction process, data segments are rewritten into new containers with sequential structures, improving data locality. This merging of scattered data into organized containers enhances restoration speed while preserving storage capacity.
Solution Approach 2:
The system changes the physical organization parameter of stored data by rewriting data segments into new containers with improved contiguity. This parameter change transforms fragmented storage into more structured storage, enhancing data locality and restoration performance without losing storage capacity.
Data Source
AI summary
A method, computer program product, computer system, and the like that provide for the efficient reclamation of storage space in a deduplication system are disclosed. The method, for example, includes identifying one or more storage constructs of a number of storage constructs and generating an indication that a reclamation operation is to be performed with respect to the one or more storage constructs. In an embodiment, each of the plurality of storage constructs includes metadata and a number of units of data. The one or more storage constructs are identified, at least in part, by determining that a portion of the number of units of data of each of the one or more storage constructs is in a state, wherein the determining is based, at least in part, on at least a portion of the metadata.