Garbage Collection for Merged Deduplication Systems
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Deduplication in replica systems requires efficient garbage collection to manage duplicate data segments across multiple primary systems, which increases processing complexity due to the need to determine identical segments stored across multiple primary systems.
Innovation Solution
Implementing a system with cooperating deduplicators, including a primary deduplicator and a secondary deduplicator, that use segment IDs and reference counters to determine duplicate segments and manage storage efficiently, allowing for selective operation of deduplicators to store or delete segments based on their presence across systems.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If deduplication is implemented in replica systems, then storage space requirements are reduced, but garbage collection processing complexity increases due to the need to determine identical segments across multiple primary systems
Solution Approach 1:
The patent divides data into segments and uses segment IDs to track duplicates across multiple primary systems. Each segment is independently managed with its own reference count, allowing the garbage collector to process segments individually rather than analyzing entire datasets, thus reducing processing complexity while maintaining storage efficiency.
Solution Approach 2:
The patent implements a reference count mechanism that provides feedback about segment usage across primary systems. When a segment's reference count drops to zero (indicating no active references), the garbage collector is triggered to remove the segment. This feedback loop enables efficient garbage collection without requiring complex analysis of all segments.
2Productivity
If multiple primary systems are serviced by a single replica system, then system efficiency is improved, but the number of segments to manage increases requiring more processing for garbage collection
Solution Approach 1:
The patent merges the garbage collection processes of multiple primary systems into a single unified process at the replica system. By consolidating segment management and using a shared reference count mechanism, the replica system can efficiently manage segments from multiple primary systems without requiring separate garbage collection operations for each primary system.
Solution Approach 2:
The replica system's garbage collector is designed to handle segments from any primary system universally. The reference count mechanism and segment identification work the same way regardless of which primary system the segment originated from, allowing a single garbage collection process to serve multiple primary systems efficiently.
Data Source
AI summary
A method of identifying nonreferenced memory elements in a storage system is disclosed. A plurality of lists of referenced elements from a plurality of storage subsystems is input. A union of the lists of referenced elements is compiled. The union of the lists of referenced memory elements is compared to a list of previously referenced memory elements to determine previously referenced elements that are no longer referenced. The previously referenced elements that are no longer referenced is output.