Deduplication Map Thinning via Tombstone Flattening
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Data deduplication systems face performance issues due to the presence of no longer relevant deduplication entries in the database, which can lead to increased processing time and resource inefficiency as the deduplication map grows in size.
Innovation Solution
The implementation of tombstones, a data structure that marks no longer relevant deduplication entries for removal, allowing for the thinning of the database through a flattening process, which merges vectors and enables early removal of these entries before a garbage collection process, thereby improving system performance.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If the deduplication map grows to store more data blocks, then the storage capacity increases, but the processing time and resource consumption increase
Solution Approach 1:
The patent applies preliminary action by performing a flattening process that proactively merges vectors and removes dead data blocks before they accumulate and cause performance degradation. The garbage collection process is triggered proactively when the deduplication map reaches a threshold size, preventing the map from growing excessively large and maintaining optimal processing performance while preserving storage capacity.
2Productivity
If the deduplication map is frequently cleaned to remove dead entries, then the processing performance improves, but the system complexity increases
Solution Approach 1:
The system implements self-service through an automated garbage collection mechanism that monitors the size of the deduplication map and autonomously triggers the flattening process when thresholds are exceeded. The controller automatically identifies dead data blocks, merges vectors, and removes obsolete entries without requiring manual intervention, thereby maintaining high processing performance while managing system complexity through automation rather than manual procedures.
Data Source
AI summary
An implementation of the disclosure provides a system comprising a storage array comprising a storage controller coupled to the storage array. The storage controller comprising a processing device to remap a plurality of deduplication references in a deduplication map to point to an earlier occurrence of duplicate data of a data block for the deduplication map. The processing device further to update an entry of the deduplication map associated with the plurality of deduplication references with a record indicating that the entry is no longer referenced and trim the entry from the deduplication map that is associated with the record.