Incremental Forever Offload for Cloud Object Store Data Protection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional data protection systems for cloud object stores face inefficiencies in managing incremental backups, requiring extensive time and complexity for restore operations and consuming excessive storage space, while also facing performance challenges due to caching and kernel integration issues.
Innovation Solution
The implementation of an incremental offload mechanism using extents to identify blocks of data for virtual machines, converting them into an incremental forever data format, and storing these in cloud object stores, allowing for efficient data management and recovery without the need for staging through operational recovery storage.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If conventional snapshot-based backup systems are used to store all backups in a snapshot repository, then data protection is provided, but the amount of storage space required increases significantly and restore operations become complex and time-consuming
Solution Approach 1:
The backup data is segmented into incremental chunks that are stored separately in cloud object storage rather than maintaining complete snapshots locally. Each backup increment is stored as individual objects with metadata, allowing selective retrieval without requiring the entire backup set to be present locally.
Solution Approach 2:
The patent extracts the incremental backup data from the traditional snapshot repository and stores it in cloud object storage. The local system retains only the metadata and minimal necessary data, while the actual backup increments are offloaded to the cloud, reducing local storage requirements.
2Ease of operation
If all backup increments are retained locally in a snapshot repository, then complete restore capability is maintained, but restore operations require significant time and complexity to reconstruct data from multiple snapshots
Solution Approach 1:
The system performs preliminary organization of backup data into incremental objects with associated metadata in cloud storage before restore operations are needed. The metadata includes information about data location, version, and dependencies, so that when a restore is required, the system can quickly identify and retrieve only the necessary increments without having to search through or reconstruct from multiple snapshots.
Solution Approach 2:
The patent introduces a metadata layer that acts as an intermediary between the restore request and the actual backup data in cloud storage. This metadata layer enables efficient querying and identification of required backup increments, eliminating the need for complex reconstruction algorithms and reducing restore time.
3Adaptability or versatility
If ZFS send command is used to transfer incremental changes, then data can be replicated, but the time and complexity required to merge increments increases with each transfer
Solution Approach 1:
The patent changes the parameter of how incremental data is structured and identified. Instead of using ZFS send's change tracking mechanism which requires sequential merging, the system uses cloud object storage's native object identification and metadata to track increments. This allows any increment to be independently identified and applied without requiring previous increments to be present, reducing merge complexity.
4Adaptability or versatility
If device driver solutions with gateway translation are used, then block requests can be translated to cloud storage objects, but performance challenges arise due to caching and kernel integration requirements
Solution Approach 1:
The patent extracts the performance-critical translation and caching functions from the kernel space device driver layer and implements them in user space. This eliminates the need for complex kernel integration and caching mechanisms, allowing the system to leverage cloud object storage's native performance characteristics while maintaining block-level access semantics through software abstraction.
Data Source
AI summary
Provided are techniques for providing and managing data protection by using incremental forever for storage to cloud object stores. An incremental offload is performed by using one or more extents to identify blocks of data for a version of a virtual machine in operational recovery storage to be offloaded to a cloud object store, wherein each of the one or more extents identifies locations of the blocks of data. The blocks of data are converted to objects in an incremental forever data format. The objects are offloaded to the cloud object store. The details of the offloading of the objects are stored in a local database in the operational recovery storage. The local database is copied from the operational recovery storage to the cloud object store as a database copy.