Snapshot Lineage Copying via Checkpointing Cache
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current information processing systems face challenges in efficiently managing and copying snapshots between local and cloud storage systems, particularly in pausing and resuming data transfer processes, which can lead to inefficiencies and resource wastage.
Innovation Solution
The implementation of a processing device that identifies snapshot lineages, selects snapshots for copying, and uses a checkpointing cache to manage the copying process, allowing for pausing and resuming operations by tracking metadata for data progress within the cache and cloud storage.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If snapshot copying is performed continuously from local to cloud storage, then data transfer completeness is improved, but system resource consumption increases and flexibility decreases
Solution Approach 1:
The patent implements dynamic snapshot copying by introducing a checkpointing mechanism that allows the copying process to be paused and resumed. The system maintains checkpointing information in a cache that records the state of data transfer, enabling flexible control over when copying occurs while ensuring eventual completeness. This resolves the contradiction by making the copying process adaptable to system conditions rather than rigidly continuous.
Solution Approach 2:
The patent performs preliminary actions by creating checkpoints during the snapshot copying process. Before completing the full copy to cloud storage, the system first creates intermediate checkpoints in local cache, allowing the process to be interrupted and later resumed from the last checkpoint state. This ensures data transfer completeness can be achieved without requiring continuous uninterrupted transfer.
2Adaptability or versatility
If checkpointing information is maintained in cache, then pausing and resuming copying is enabled, but memory usage increases
Solution Approach 1:
The patent segments the snapshot copying process into manageable chunks by maintaining checkpointing information for individual snapshots or data blocks in the cache. Rather than loading entire snapshots into memory, the system divides them into smaller units with associated checkpoint metadata, reducing overall memory requirements while still enabling pause and resume functionality.
3Speed
If data is copied directly from local storage to cloud storage, then transfer speed is improved, but ability to pause and resume decreases
Solution Approach 1:
The patent introduces a checkpointing cache as an intermediary between local storage and cloud storage. Data flows from local storage to the checkpointing cache first, where checkpointing information is maintained, and then to cloud storage. This intermediary structure enables the system to pause copying at any checkpoint and resume later without requiring direct end-to-end transfer, thus maintaining speed while adding adaptability.
Data Source
AI summary
An apparatus comprises a processing device configured to identify a snapshot lineage comprising (i) a local snapshot lineage stored on a storage system and (ii) a cloud snapshot lineage stored on cloud storage of at least one cloud external to the storage system. The processing device is also configured to select a snapshot to be copied from the local snapshot lineage to the cloud snapshot lineage, and to copy the selected snapshot by copying data stored in the local snapshot lineage to a checkpointing cache and, responsive to determining that the copied data in the checkpointing cache has reached a specified checkpoint size, moving the copied data from the checkpointing cache to the cloud storage. The processing device is further configured to maintain, in the checkpointing cache, checkpointing information utilizable for pausing and resuming copying of the selected snapshot from the local snapshot lineage to the cloud snapshot lineage.