Backup Storage Garbage Collection via Pre-computed Expiration Metadata
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional garbage collection processes in backup storage systems are inefficient, especially in large data systems, as they require walking through all data to determine which can be deleted, leading to excessive resource consumption and prolonged processing times due to a time complexity of O(n2).
Innovation Solution
The method involves obtaining time information for a removal time point, determining a target removal period from a removal period list mapped to chunk elements, and removing corresponding chunks, thereby bypassing the need to traverse directory information and check expiration status for each chunk, reducing the time complexity to O(n).
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If conventional garbage collection processes walk through all backup data to determine deletion eligibility, then complete data verification is achieved, but processing time and resource consumption increase significantly with O(n2) time complexity
Solution Approach 1:
The patent applies preliminary action by pre-calculating and storing expiration times for all chunks during data ingestion and backup operations. This allows the garbage collection process to directly query pre-computed expiration information rather than walking through all data to determine deletion eligibility, reducing time complexity from O(n2) to O(n) while maintaining complete verification
Solution Approach 2:
The patent introduces an intermediary metadata structure that stores expiration time information separately from the actual backup data. This intermediary layer acts as a lookup table that enables rapid determination of chunk expiration status without traversing the entire backup dataset, thus maintaining reliability while reducing processing time
2Measurement precision
If conventional garbage collection processes traverse all directory information to check each chunk expiration status, then accurate identification of expired chunks is achieved, but device complexity and resource consumption increase
Solution Approach 1:
The patent segments the garbage collection process into distinct operations: querying pre-stored expiration information, comparing current time with expiration times, and removing expired chunks. This segmentation eliminates the need for complex directory traversal and chunk-by-chunk verification, maintaining accurate identification while reducing process complexity
Solution Approach 2:
The patent creates a simplified copy of expiration information in a metadata structure that mirrors the essential deletion criteria without containing the full complexity of the backup data hierarchy. This copy enables accurate expired chunk identification through simple time comparison rather than complex directory traversal
3Productivity
If backup storage systems maintain detailed tracking of all chunk expiration times, then precise garbage collection is achieved, but storage overhead and processing resources increase
Solution Approach 1:
The patent implements self-service by having the metadata structure automatically maintain expiration time information as chunks are created, modified, or deleted during normal backup operations. This self-updating mechanism ensures precise garbage collection information is always available without requiring additional processing resources or extensive storage overhead
Data Source
AI summary
Embodiments of the present disclosure provide a method, a device and a computer program product for storage management. The method comprises: obtaining time information related to a removal time point for a backup storage system, the time information indicating that a chunk in the backup storage system whose expiration time does not exceed the removal time point is to be removed; determining, from a removal period list, a target removal period whose end time does not exceed the removal time point, each removal period in the removal period list being mapped to at least one chunk element, the at least one chunk element representing at least one chunk in the backup storage system whose expiration time is within the removal period to which the at least one chunk element is mapped; determining at least one target chunk element to which the target removal period is mapped; and removing, from the backup storage system, at least one target chunk corresponding to the at least one target chunk element. Therefore, with this solution, expired chunks can be efficiently removed.