Object Store Incremental Backup Using Changed Data Blocks
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing backup protocols like NDMP struggle with scalability and efficiency when dealing with a large number of incremental backups, leading to complex and time-consuming restore processes, and prior backup APIs waste resources by backing up entire changed files instead of just data blocks, failing to preserve storage efficiencies like deduplication and compression.
Innovation Solution
A backup framework utilizing storage APIs to identify and transfer only changed data blocks, leveraging snapshot difference and copy to object store APIs to perform incremental backups and restores, preserving deduplication and compression efficiencies.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If incremental backups are performed using prior backup APIs that back up entire changed files, then backup functionality is provided, but processing and network resources are wasted and storage efficiencies like deduplication and compression are not preserved
Solution Approach 1:
The patent segments files into smaller data blocks for backup operations. Instead of backing up entire changed files, the system identifies and backs up only the specific data blocks that have changed within those files. This segmentation enables more granular control over backup data and reduces the amount of data transferred and stored.
Solution Approach 2:
The patent extracts only the changed data blocks from files for backup, rather than backing up entire files. By using storage APIs to identify changed blocks at the block level, the system separates and extracts only the necessary portions of data that have changed, leaving the rest of the file data untouched.
2Quantity of substance
If a large number of incremental backups are performed using protocols like NDMP, then backup coverage is increased, but restore processes become complex and time-consuming
Solution Approach 1:
The patent segments backup data into discrete data blocks with unique identifiers, enabling individual blocks to be tracked and retrieved independently. This segmentation allows the restore process to retrieve only the specific blocks needed rather than processing entire files or backup sets, significantly reducing restore time even when many incremental backups exist.
Solution Approach 2:
The patent uses storage APIs to create efficient copies of changed data blocks and stores them in an object store with metadata that tracks block provenance across incremental backups. This copying mechanism with proper metadata management enables rapid identification and retrieval of specific blocks from any point in the backup history without reconstructing entire backup chains.
3Adaptability or versatility
If backup protocols like NDMP are used to handle incremental backups, then backup functionality is provided, but the system becomes less scalable and more complex
Solution Approach 1:
The patent introduces storage APIs as an intermediary layer between the backup application and the underlying storage system. These APIs provide standardized interfaces for identifying changed data blocks, enabling the backup system to work with various storage types and protocols without increasing complexity. The intermediary abstracts the complexity of block-level operations from the backup logic.
Solution Approach 2:
The patent creates a universal backup framework that can handle incremental backups across different storage types and protocols through standardized storage APIs. The system is designed to be protocol-agnostic, allowing it to adapt to different storage backends (object stores, file systems, block storage) without requiring protocol-specific implementation complexity.
Data Source
AI summary
Techniques are provided for incremental backup to an object store. A request may be received from an application to perform a backup from a volume hosted by a node to a backup target within the object store. A set of changed files within the volume since a prior backup of the volume was performed to the backup target is identified, along with metadata associated with the set of changed files. The metadata is utilized to identify changed data blocks comprising data of the set of changed files that was modified since the prior backup. The changed data blocks are backed up to the object store.