Chunk Segmentation and Copying Garbage Collection for Storage Efficiency
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Cloud-based data storage systems face inefficiencies in reclaiming storage space due to immutable chunks and high resource consumption when waiting for chunks to become fully unused or copying data from sparsely filled chunks, leading to poor capacity usage efficiency and inefficient garbage collection.
Innovation Solution
Implementing separate chunk types for leaf and internal nodes, with smaller leaf chunks and standard-sized internal chunks, and using a copying garbage collector to proactively reclaim space by copying valid pages from sparsely filled chunks, thereby increasing the probability of fully reclaimable chunks and improving capacity usage efficiency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If chunks are made immutable for data protection, then data reliability is improved, but capacity usage efficiency deteriorates due to inability to perform fine-grained reclamation
Solution Approach 1:
The patent segments chunks into two distinct types: immutable chunks for data protection and mutable chunks for efficient garbage collection. This segmentation allows the system to maintain reliability through immutable chunks while improving capacity efficiency through mutable chunks that support fine-grained reclamation.
Solution Approach 2:
Different quality properties are applied to different parts of the storage system: immutable chunks provide high reliability for data protection, while mutable chunks provide high efficiency for garbage collection. This local differentiation resolves the contradiction by allowing each chunk type to optimize for its specific purpose.
2Reliability
If the system waits for chunks to become fully unused before garbage collection, then data consistency is improved, but productivity deteriorates due to high resource consumption and inefficiency
Solution Approach 1:
The system performs preliminary actions by proactively copying valid pages from sparsely filled chunks before they become fully unused. This allows garbage collection to occur earlier than traditional methods would permit, improving productivity while maintaining data consistency through the copying process.
Solution Approach 2:
The patent uses copying as the core mechanism for garbage collection, creating new chunks with valid pages while the original chunks remain intact. This copying approach maintains data consistency while enabling proactive reclamation of space, resolving the contradiction between consistency and efficiency.
3Loss of substance
If sparsely filled chunks are copied to reclaim capacity, then capacity usage efficiency is improved, but device complexity increases due to additional management overhead
Solution Approach 1:
The patent changes the parameter of chunk mutability, introducing mutable chunks as a new state. This parameter change enables the system to track and manage chunk states more effectively, improving capacity efficiency while the added complexity is managed through systematic state transitions.
Data Source
AI summary
The disclosed technology generally describes separating types of data chunks in a copy-on-write/MVCC B+ tree, chunk-based data storage system, and also allocating the sizes of leaf chunks to be smaller than that of other (e.g., internal and root node) chunks. By having leaf chunks separate from node chunks, the probability of having a fully reclaimable (without copying) chunk is increased. Similarly, by having smaller sized leaf chunks relative to node chunks, the probability of having a fully reclaimable (without copying) leaf chunks is increased. The technology thus facilitates more efficient garbage collection.