Garbage Collection via Similarity Groups for Concurrent Storage Operations
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In deduplicated storage systems, particularly in cloud-based environments, efficiently deleting dead objects without impacting live objects is challenging due to the complexity of managing reference counts and the need for scalable, cost-effective garbage collection that supports concurrent operations.
Innovation Solution
Implementing a microservice-based garbage collection system that uses similarity groups and subgroups to identify and remove unreferenced segments, allowing for concurrent reads and writes, and dynamically allocating computing resources only when needed, thereby reducing storage and computational costs.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If reference counts are used to track object references in deduplicated storage systems, then garbage collection can identify dead objects, but the system requires maintaining billions of counts which consumes significant storage and is difficult to manage
Solution Approach 1:
The patent extracts the reference counting mechanism from the system and replaces it with a marker-based approach. Instead of maintaining billions of reference counts, the system uses markers placed on objects during garbage collection to track which objects are being processed, eliminating the need for complex reference count management while preserving garbage collection functionality
Solution Approach 2:
The patent changes the fundamental parameter from storing reference counts (numeric values) to using markers (presence/absence indicators). This parameter transformation reduces the storage requirements from billions of count values to a minimal marker structure, while maintaining the ability to identify dead objects for garbage collection
2Loss of substance
If garbage collection is performed in deduplicated storage systems, then storage space can be reclaimed by removing dead objects, but the system must carefully manage segments that may be shared between live and dead objects
Solution Approach 1:
The patent segments the garbage collection process into distinct phases: marking phase where markers are placed on objects being collected, and copying phase where live objects are copied to new locations. This segmentation allows the system to safely manage shared segments by controlling access during each phase, enabling storage reclamation without compromising data integrity
Solution Approach 2:
The patent introduces markers as intermediary elements that mediate between the garbage collection process and the actual data segments. These markers serve as intermediaries to track which objects are being collected without requiring direct manipulation of the shared segments, simplifying the management of deduplicated data
3Productivity
If computing resources are allocated continuously in the storage system, then system operations can proceed smoothly, but storage and computational costs increase
Solution Approach 1:
The patent implements periodic garbage collection operations rather than continuous resource allocation. The system performs garbage collection at scheduled intervals when needed, allowing computing resources to be released during idle periods and allocated only when garbage collection or other operations require them, reducing overall computational costs while maintaining system productivity
Solution Approach 2:
The patent creates a dynamic resource allocation system where computing resources are adjusted based on actual system needs. Resources are allocated during garbage collection operations and released when not needed, creating a dynamic pattern of resource usage that maintains productivity while minimizing energy consumption and computational costs
4Loss of substance
If garbage collection operations are performed, then dead objects can be removed, but normal system operations such as reads and writes may be impacted
Solution Approach 1:
The patent segments the storage system into multiple operational zones and divides garbage collection work across multiple workers that operate on different segments simultaneously. This segmentation allows normal reads and writes to continue on unaffected segments while garbage collection proceeds on other segments, enabling concurrent operations without impacting overall system ease of use
Solution Approach 2:
The patent ensures continuity of useful actions by implementing a garbage collection system that does not interrupt normal operations. The marking and copying phases are designed to work alongside ongoing reads and writes, maintaining continuous useful action in both garbage collection and normal system operations simultaneously
Data Source
AI summary
Systems and methods enabling garbage collection operations and normal system operations concurrently. Concurrent operations are performed by configuring a similarity group to permit garbage collection and normal operations. This may include creating a new subgroup in a similarity group for write and deduplication purposes such that an impacted subgroup can be cleaned.