Backup Copy Validation via Embedded Bitmaps
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current backup and recovery processes are inefficient, requiring lengthy times and significant computing resources, and often fail to detect data corruption errors until they cause issues during restoration, especially due to limitations in incremental backups and the inability of backup proxies to track changes outside the production IO path.
Innovation Solution
The implementation of a system that uses bitmaps embedded into snapshots to track changes on the production host, allowing backup proxies to conduct differential or incremental backups by providing the changed block map, which reduces backup windows and ensures data integrity by embedding the bitmap into the snapshot, enabling efficient data rollover and validation.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of time
If incremental backups are used, then backup window time and storage space are reduced, but backup proxies cannot take advantage of incremental backups because they are outside the production I/O data path
Solution Approach 1:
A bitmap is introduced as an intermediary data structure that bridges the gap between the production host's change tracking and the backup proxy's backup operations. The bitmap is generated at the production host, transmitted to the backup proxy, and used to guide incremental backup operations, enabling the proxy to perform change-based backups without being in the I/O path
Solution Approach 2:
The backup system is segmented into distinct functional components: change tracking at the production host, bitmap generation and transmission, and incremental backup execution at the proxy. This segmentation allows each component to operate independently while maintaining coordination through the bitmap mechanism
2Productivity
If data corruption errors are not detected during backup, then backup processes run smoothly, but data corruption errors are discovered too late during restoration
Solution Approach 1:
Validation operations are performed preliminarily during the backup process itself, comparing source and destination data blocks and computing checksums to detect corruption early. This preliminary validation prevents corrupted data from being propagated through the backup chain, maintaining both efficiency and reliability
3Reliability
If full backups are performed frequently, then data recovery reliability is improved, but backup time and computing resources increase significantly
Solution Approach 1:
The system implements periodic full backups interspersed with incremental backups. Full backups are performed at scheduled intervals to establish new baseline copies, while incremental backups capture changes since the last full backup. This periodic pattern maintains recovery reliability while minimizing overall backup time and resource consumption
Data Source
AI summary
A synthetic full backup of a source volume representing a state of the volume at a current time is retrieved, the synthetic full having been generated by merging a full backup of the volume performed at an initial time with an incremental backup of the volume performed at the current time, after the initial time. A bitmap tracking changes to the volume made between the initial and current times is accessed. The bitmap is used to identify a location on the volume having changes made between the initial and current times. First data written to the location on the volume is read. Second data written to the same location on the synthetic full backup is read. First and second checksums are generated based on the first and second data, respectively. The checksums are compared and if any do not match, an indication is generated that the synthetic full is corrupt.