Live Segment Records for Deduplicated Storage Garbage Collection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional garbage collection processes in deduplicated file systems are inefficient due to their inability to effectively identify and manage live segments, leading to unnecessary scanning and resource wastage.
Innovation Solution
The implementation of live segment records (LSRs) that indicate whether segments are live, allowing for level-by-level traversal and storage space reclamation without re-scanning containers, enabling efficient identification and reclamation of dead segments.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If conventional garbage collection processes scan all LP containers multiple times to perform independent tasks, then garbage collection can be performed, but memory and processing resource usage increases significantly
Solution Approach 1:
The patent applies preliminary action by pre-computing and storing live segment information in persistent LSR files before garbage collection execution. The system performs the expensive scanning and identification work in advance, storing results in a persistent format that can be reused across multiple garbage collection cycles, thereby avoiding repeated full scans of LP containers
Solution Approach 2:
The patent creates a copy of the live segment information in persistent LSR files that mirrors the actual segment data in LP containers. This copy can be used for garbage collection decisions without needing to access or re-scan the original containers, significantly reducing I/O and processing resources during GC execution
2Productivity
If conventional garbage collection traverses all files simultaneously using breadth-first approach, then garbage collection can be performed, but the system cannot roll per-file-tree checksum and may miss metadata segments
Solution Approach 1:
The patent segments the garbage collection process by file tree, maintaining separate LSR files for each file tree that track its specific live segments. This allows the system to process and verify each file tree independently, rolling per-file-tree checksums to ensure completeness while maintaining overall system productivity through parallel processing of multiple file trees
3Measurement precision
If the system scans all containers to identify live segments, then accurate identification can be achieved, but the process is very memory and processing resource inefficient
Solution Approach 1:
The system performs the complex scanning and identification work in advance, storing results in persistent LSR files with pre-computed live segment information. This eliminates the need for repeated complex scanning operations, reducing both memory and processing complexity during actual garbage collection execution while maintaining identification accuracy
Data Source
AI summary
Containers of a storage system are scanned, each container containing segments of files, where each file is represented by a file tree having segments in a hierarchical structure. The container live segment records (LSRs) corresponding to one of the containers are created, each of the container LSRs including segment LSRs corresponding to segments contained therein. After the segment LSRs of the container LSRs have been created for all segments of the containers, the segment LSRs of the container LSRs are sequentially traversed based on levels of segments specified in the corresponding segment LSRs to determine and indicate in the corresponding segment LSRs whether the segments are live segments. After all of the segment LSRs of the container LSRs have been traversed, a garbage collection operation is performed to reclaim storage space of segments that are not live segments indicated in the segment LSRs of the container LSRs.