In-Memory Cache Extent Pre-Allocation for Snapshot Latency
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The continuous creation and deletion of snapshots in storage systems with scheduled snapshot policies or replication lead to significant snapshot load, causing IO latency spikes and hindering the reduction of Recover Point Objective (RPO), especially due to the expensive allocation and freeing of extents using on-disk data structures that require serialization.
Innovation Solution
Implementing an in-memory cache to pre-allocate and manage extents for snapshot operations, where the number of extents needed for create snapshot operations is identified and allocated from this cache, and freed extents are added back to the cache during delete operations, reducing reliance on on-disk data structures and minimizing serialization.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If continuous snapshots are created and deleted to reduce RPO, then recovery capability is improved, but system load and IO latency increase
Solution Approach 1:
The patent pre-allocates extents in an in-memory cache before snapshot operations are needed. By identifying and reserving the required extents in advance within the cache structure, the system eliminates the need for expensive on-disk allocation operations during actual snapshot creation, thereby reducing system load while maintaining continuous snapshot capability for RPO reduction.
Solution Approach 2:
The patent introduces an in-memory cache as an intermediary between the snapshot operation and the physical storage allocation. This cache layer absorbs the allocation/freeing operations, providing a buffer that reduces the impact on the main storage system. The cache acts as a mediator that handles extent management internally, preventing direct I/O operations during snapshot creation and deletion.
2Reliability
If on-disk data structures are used for extent allocation, then persistence is ensured, but serialization and IO latency increase
Solution Approach 1:
The patent segments the extent management into two parts: an in-memory cache for active allocation operations and an on-disk persistence layer for durability. The in-memory cache handles frequent allocation/freeing operations locally without serialization, while only committing changes to disk when necessary. This segmentation separates the high-performance operational layer from the persistence layer, eliminating the need for serialization during normal operations.
Solution Approach 2:
The patent performs extent identification and allocation decisions in-memory before any disk I/O occurs. By pre-computing which extents need to be allocated and managing them in the cache structure, the system avoids repeated serialization and disk access operations. The preliminary in-memory processing reduces the frequency and impact of disk serialization operations.
3Reliability
If extents are allocated and freed frequently for scheduled snapshots, then snapshot freshness is improved, but CPU load increases
Solution Approach 1:
The patent implements a self-managing in-memory cache that automatically tracks and manages extent allocations and freeings without requiring extensive CPU intervention. The cache structure maintains its own state information about allocated and available extents, enabling it to service snapshot operations autonomously. This self-service capability reduces CPU load while maintaining the ability to create and delete snapshots frequently for freshness.
Solution Approach 2:
The patent changes the state management parameter from disk-based to memory-based. By moving extent tracking from the disk to an in-memory cache, the system fundamentally alters how allocation and freeing operations are handled. This parameter change enables frequent snapshot operations to be processed in-memory with minimal CPU overhead, as the cache can quickly identify and allocate extents without expensive disk access or complex serialization.
Data Source
AI summary
A method, computer program product, and computer system for identifying, by a computing device, a number of extents needed for a create snapshot operation to create a snapshot. The number of extents may be added to an in-memory cache. The number of extents needed for the create snapshot operation may be allocated from the in-memory cache to execute the create snapshot operation. Freed extents may be added to the in-memory cache based upon, at least in part, executing a delete snapshot operation to delete the snapshot.


