Snapshot Recovery Using Two-Phase Patch File Consolidation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing data management systems face challenges in achieving fast and cost-effective data recovery from cloud-based backups due to the inefficiencies of traditional snapshot methods, which often result in poor recovery point objectives (RPOs) and high costs.
Innovation Solution
A two-phase approach is implemented, comprising a dry-run phase to create a patch file image and a data-transfer phase to download specific data blocks in parallel, minimizing redundant data transfer and optimizing snapshot recovery.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If traditional snapshot methods are used for data backup, then data portability and backup capability are achieved, but recovery point objectives deteriorate and costs increase due to inefficient data retrieval
Solution Approach 1:
The system performs a dry-run phase before actual snapshot recovery to pre-identify which data blocks need to be downloaded. This preliminary action creates a download profile that guides the subsequent recovery process, ensuring that only necessary data is retrieved and minimizing recovery time
Solution Approach 2:
The snapshot recovery process is divided into two distinct phases: a dry-run phase for profiling and a data-transfer phase for actual download. This segmentation allows the system to optimize each phase independently, improving overall recovery efficiency and reducing time loss
2Reliability
If snapshots are taken frequently to improve recovery point objectives, then data freshness improves, but operational costs and system impact increase
Solution Approach 1:
The system extracts and downloads only the specific data blocks that are needed for recovery, rather than downloading entire snapshots. This selective extraction reduces the volume of data transferred, lowering operational costs while maintaining good recovery point objectives
Solution Approach 2:
Instead of downloading complete snapshots, the system performs partial downloads of only the necessary data blocks identified in the dry-run phase. This partial action approach reduces bandwidth consumption and operational costs while achieving the required recovery fidelity
3Productivity
If parallel data download is implemented to speed up recovery, then productivity improves, but data transfer complexity and resource requirements increase
Solution Approach 1:
The dry-run phase performs all necessary planning and identification of data blocks before the parallel download begins. This preliminary action creates a ready-to-execute download profile that simplifies the coordination of parallel transfers, reducing the complexity burden during the actual data transfer
Data Source
AI summary
In some examples, a data management and storage (DMS) platform comprises peer DMS nodes in a node cluster, a distributed data store comprising local and cloud storage, and at least one processor configured to perform operations in a method of creating a local consolidated patch file from a patch file chain stored in the cloud storage. Example operations comprise, in a first dry-run phase, creating a patch file image of data blocks in one or more cloud patch files stored in the cloud storage; in a second data-transfer phase, downloading at least some of the data blocks from the cloud patch files identified by the patch file image; and creating and storing, in the local storage, the local consolidated patch file using the downloaded data blocks.