Object Store Incremental Backup Using Changed Data Blocks
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing backup protocols like NDMP face scalability issues with a large number of incremental backups, leading to complex and time-consuming restores, and prior backup APIs inefficiently transfer entire changed files, wasting processing and network resources while failing to preserve storage efficiencies like deduplication and compression.
Innovation Solution
A backup framework utilizing storage APIs to identify and transfer only changed data blocks, preserving deduplication and compression efficiencies, and enabling scalable incremental backups and restores to and from cloud storage.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If prior backup APIs transfer entire changed files, then backup completeness is ensured, but processing and network resources are wasted
Solution Approach 1:
The patent segments files into smaller data blocks and identifies only the changed blocks within those files. Instead of transferring entire changed files, the system divides files into blocks, compares blocks to identify changes, and transfers only the modified blocks, thereby reducing network and processing resources while maintaining backup completeness.
Solution Approach 2:
The patent extracts and transfers only the essential changed data blocks from files, removing unnecessary data. By using block-level change identification, the system extracts only the specific blocks that have changed rather than transferring entire files, reducing resource consumption while ensuring all necessary backup data is captured.
2Productivity
If incremental backups are increased to improve backup frequency, then backup freshness is improved, but restore complexity increases
Solution Approach 1:
The patent segments backup data into block-level increments, where each incremental backup contains only changed blocks rather than entire files. This segmentation approach allows for frequent backups without proportionally increasing restore complexity, as the block-level organization enables efficient selection and application of only necessary blocks during restoration.
Solution Approach 2:
The patent uses block-level copying where changed blocks are identified and copied to incremental backup storage. This copying mechanism maintains a chain of incremental backups that can be efficiently reconstructed during restore operations, reducing the complexity burden that would otherwise result from file-level incremental backups.
3Volume of stationary object
If cloud storage is used to reduce on-premises storage requirements, then storage scalability is improved, but data transfer time increases
Solution Approach 1:
The patent extracts and transfers only the changed data blocks to cloud storage rather than transferring entire files or full backups. This extraction approach significantly reduces the volume of data that needs to be transferred over the network, thereby reducing transfer time while still achieving the goal of reducing on-premises storage requirements through cloud archival.
Solution Approach 2:
The patent segments backup data into blocks and transfers only the necessary changed blocks to cloud storage. This segmentation enables selective data transfer, reducing the overall transfer time and bandwidth consumption while maintaining the ability to restore data from cloud storage when needed.
Data Source
AI summary
Techniques are provided for incremental backup to an object store. A request may be received from an application to perform a backup from a volume hosted by a node to a backup target within the object store. A set of changed files within the volume since a prior backup of the volume was performed to the backup target is identified, along with metadata associated with the set of changed files. The metadata is utilized to identify changed data blocks comprising data of the set of changed files that was modified since the prior backup. The changed data blocks are backed up to the object store.