Persistent Live Segment Records for Garbage Collection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional garbage collection processes in deduplicated file systems are inefficient due to their inability to effectively identify and manage live segments, leading to unnecessary scanning and resource wastage.
Innovation Solution
The implementation of live segment records (LSRs) that indicate whether segments are live, allowing for level-by-level traversal and storage space reclamation without re-scanning containers, enabling efficient garbage collection by maintaining persistent LSRs that mark dead segments for storage space reclamation.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If conventional garbage collection processes scan all LP containers multiple times to perform independent tasks, then garbage collection can be performed, but memory and processing resource efficiency deteriorates
Solution Approach 1:
The patent applies preliminary action by creating and maintaining live segment records (LSRs) that pre-track which segments are live and which are dead before the actual garbage collection work. This allows the system to avoid re-scanning containers multiple times, as the LSRs provide advance information about segment status. The LSRs are updated incrementally as files are processed, enabling efficient garbage collection without repeated full scans.
Solution Approach 2:
The patent uses live segment records as an intermediary data structure between the container manager and the garbage collection process. These LSRs act as a mediator that stores metadata about segment liveness, allowing the garbage collector to operate without directly scanning all containers repeatedly. The LSRs bridge the gap between container storage and garbage collection logic, enabling efficient resource utilization.
2Productivity
If a physical garbage collector traverses all files simultaneously using breadth-first approach, then all files can be processed, but the ability to roll per-file-tree checksums and identify missed metadata segments deteriorates
Solution Approach 1:
The patent applies segmentation by organizing the garbage collection process around individual file trees rather than processing all files simultaneously. Each file tree can be traversed independently, allowing the system to roll checksums per-file-tree and accurately identify missed metadata segments. The LSR structure supports this by maintaining segment liveness information at the file tree level, enabling precise checksum verification without the complexity of simultaneous multi-file processing.
Data Source
AI summary
A garbage collector scans containers of a storage system, each container containing a plurality of segments referenced by files of the storage system. For each of the containers being scanned constructing a working live segment record (LSR) file in memory, including generating a container LSR for each of the containers being scanned in a sequential order according to container identifiers of the containers, generating segment LSRs associated with the container LSR, each of the segment LSRs corresponding to one of the segments contained in the container, and for each of the segments contained in the container, indicating in a corresponding segment LSR whether the segment is a live segment. The segment LSRs of each container LSR of the working LSR file are translated from the memory into a persistent LSR for each of the containers in a persistent LSR file stored a persistent storage.