On-Demand Snapshot Pruning in Hybrid SLC MLC Storage
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Data storage systems face inefficiencies due to unnecessary space consumption in higher cost storage tiers when snapshots expire, as expired point-in-time copies (PITCs) cannot be coalesced, leading to prolonged storage of historical data in SLC drives, which are more expensive and have faster write performance.
Innovation Solution
Implementing an on-demand snapshot generation method that designates data as read-only when SLC storage space reaches a predetermined threshold, transferring it to MLC storage, and using a batch pruning method to free overwritten data, thereby optimizing storage utilization and reducing wear on MLC drives.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If snapshots are automatically generated and stored in SLC drives at predetermined time intervals, then data protection and recovery capability are improved, but storage space in higher cost tiers is unnecessarily consumed by expired snapshots that cannot be coalesced
Solution Approach 1:
The system dynamically adjusts snapshot generation behavior based on storage space conditions. When SLC storage space becomes insufficient, the system transitions from automatic periodic snapshot creation to on-demand snapshot creation only when necessary, allowing flexible adaptation to changing storage requirements and preventing unnecessary space consumption by expired snapshots
Solution Approach 2:
The system changes the parameter of snapshot generation timing from fixed periodic intervals to conditional on-demand timing. By monitoring storage space thresholds and coalescing opportunities, the system modifies when snapshots are created, moved to MLC storage, or pruned, optimizing the balance between data protection and storage space utilization
2Stability of the object's composition
If expired snapshots are held in frozen state until coalescing is permitted, then data integrity is maintained, but storage space is prolongedly occupied in higher cost tiers
Solution Approach 1:
The system performs preliminary coalescing operations when opportunities arise, even before snapshots expire. By proactively merging snapshot data with active volumes when coalescing conditions are met, the system prevents expired snapshots from occupying storage space indefinitely, while still maintaining data integrity through proper coalescing procedures
Solution Approach 2:
The system discards expired snapshots that are no longer needed for data protection, transferring their storage resources to MLC tiers or freeing them for reuse. By identifying and removing expired snapshots that cannot be coalesced, the system recovers storage space that would otherwise be permanently occupied, balancing data integrity requirements with space optimization
3Quantity of substance
If data is moved from SLC to MLC storage tiers, then storage cost efficiency is improved, but system complexity increases due to additional monitoring and management mechanisms
Solution Approach 1:
The storage system automatically monitors its own state, detecting when SLC storage space is insufficient or when coalescing opportunities arise. The system self-manages the snapshot lifecycle by automatically creating, moving to MLC storage, or pruning snapshots without external intervention, reducing the need for complex manual management while maintaining cost efficiency
Solution Approach 2:
The system implements feedback mechanisms that continuously monitor storage space utilization, snapshot status, and coalescing conditions. Based on this feedback, the system automatically adjusts snapshot management behavior, creating snapshots when space is available, moving them to MLC tiers when appropriate, and pruning expired ones, thereby optimizing cost efficiency through adaptive control
Data Source
AI summary
A method of data progression in a data storage system having at least two tiers of storage space. A first tier may include storage space in a SLC SSD and a second tier may include storage space in a MLC SSD. The method may include setting a predetermined free space threshold for the first tier of storage space, monitoring free space in the first tier of storage space, and when the amount of available free space in the first tier of storage space decreases to the predetermined free space threshold, generating an on-demand snapshot of at least a portion of the data of the first tier of storage space by designating that data as read-only. The on-demand snapshot may then be transferred to the second tier of storage space, thereby freeing the corresponding portion of data of the first tier of storage space for new writes.