Predicting Storage Reclamation in Deduplicated Datasets
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In large-scale storage systems, it is difficult to predict the amount of storage space that will be reclaimed by deleting logical containers of data due to data deduplication, where a data block is shared among multiple containers, leading to unpredictable storage space recovery.
Innovation Solution
A method to identify data blocks referenced by targeted files for deletion, compare their reference counts against the overall reference counts in the dataset, and calculate the reclaimable storage space by deleting files that are exclusively referenced by the targeted files, ensuring accurate prediction of storage space recovery before deletion.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If data deduplication is implemented in the storage system, then storage space efficiency is improved, but the ability to predict storage space reclamation becomes difficult
Solution Approach 1:
The system performs preliminary analysis of data block references before deletion operations. By pre-calculating which data blocks are exclusively referenced by targeted logical containers, the system can predict storage space reclamation accurately without having to perform actual deletions first. This preliminary assessment resolves the contradiction by enabling measurement precision while maintaining deduplication efficiency.
Solution Approach 2:
The patent introduces an intermediary mechanism that tracks reference counts of data blocks across multiple logical containers. This intermediary tracking system allows the storage controller to determine which data blocks can be safely deleted by verifying their reference patterns, thereby enabling accurate prediction of storage space reclamation while preserving the benefits of data deduplication.
2Quantity of substance
If logical containers sharing data blocks are deleted, then storage space should be reclaimed, but shared data blocks remain due to references from other containers
Solution Approach 1:
The system implements a feedback mechanism that continuously monitors reference counts of data blocks. Before confirming storage space reclamation, the system verifies that no other logical containers reference the data blocks intended for deletion. This feedback loop ensures reliable space recovery by preventing premature deletion of shared data blocks, thereby resolving the contradiction between space reclamation and recovery guarantee.
Solution Approach 2:
The patent performs preliminary verification of data block references before executing deletion operations. By checking reference counts in advance and identifying exclusively referenced data blocks, the system guarantees that storage space will be reliably reclaimed. This preliminary action eliminates uncertainty about space recovery, resolving the contradiction between reclamation and reliability.
3Quantity of substance
If users delete multiple logical containers to reclaim space, then storage space may be recovered, but the process is time-consuming and unpredictable
Solution Approach 1:
The system performs preliminary analysis of data block references and calculates potential storage space reclamation before users initiate deletion operations. This preliminary assessment provides users with immediate feedback on expected space recovery, eliminating the need for time-consuming trial deletions and making the process predictable. The time loss is reduced by performing calculations in advance rather than during actual deletion operations.
Solution Approach 2:
The patent enables the storage system to automatically assess and report potential storage space reclamation without requiring users to perform actual deletion operations. The system serves itself by maintaining reference count information and calculating reclamation potential independently, providing users with immediate predictions. This self-service capability eliminates time-consuming manual deletion processes while maintaining accurate space recovery predictions.
Data Source
AI summary
A technique for determining an amount of storage space that would be recovered in a storage system by deleting a list of logical containers of data is provided. In one embodiment, data blocks referenced by the list of files are first identified. The data blocks (e.g., deduplicated data blocks) may be referenced by two or more files. A number of references made by the identified list of files to each data block is compared against a number of references made by all files stored in a dataset of the storage system. A total amount of storage space is then calculated based on a sum of storage spaces utilized by each data block for which the two reference numbers match. This total amount of storage space is the storage space that would be reclaimed through deletion of the identified list of files.