One-Pass Delta Streaming Filter for Continuous Data Protection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing backup technologies suffer from a high recovery point objective (RPO) due to the need to stun or quiesce computing systems during snapshot operations, limiting the frequency and efficiency of data protection.
Innovation Solution
A one-pass mechanism using two maps (primary and secondary bitmaps) to track changes and generate deltas, allowing for near continuous data protection with an RPO reduced to seconds, without the need for a splitter.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If snapshot operations are performed using traditional methods, then data protection is achieved, but the computing system must be stunned or quiesced which interrupts IO and limits the frequency of snapshots
Solution Approach 1:
The patent segments the backup process into multiple threads: a first thread captures writes to a primary bitmap, a second thread processes the primary bitmap to generate deltas, and a third thread captures writes to a secondary bitmap. This segmentation allows concurrent execution of backup operations without stuning the computing system, thereby maintaining IO throughput while achieving data protection.
Solution Approach 2:
The patent uses bitmaps to pre-track and mark dirty extents before actual backup operations. The primary bitmap captures writes in advance, and the secondary bitmap pre-tracks subsequent writes. This preliminary tracking enables efficient delta generation without interrupting ongoing IO operations, resolving the contradiction between data protection and IO throughput.
2Reliability
If multiple bitmaps and data logs are used to manage IOs during delta operations, then data protection accuracy is improved, but system complexity increases
Solution Approach 1:
The patent extracts the essential function of change tracking into two dedicated bitmaps (primary and secondary) that are maintained separately from the data storage structure. This extraction simplifies the overall system by removing the need for complex data logs and multiple intermediate structures, while still achieving accurate tracking of dirty extents for delta operations.
Solution Approach 2:
The patent uses bitmap copying where the secondary bitmap is initialized as a copy of the primary bitmap after delta generation. This copying mechanism simplifies the management of change tracking state without requiring complex data logs, reducing system complexity while maintaining data protection accuracy.
3Reliability
If conventional delta operations are performed with multiple passes of bitmaps and volumes, then complete data protection is achieved, but processing time and computational resources increase
Solution Approach 1:
The patent enables continuous useful action by having the first thread continuously capture writes to the primary bitmap while the second thread processes deltas, and the third thread simultaneously captures writes to the secondary bitmap. This continuous operation eliminates idle processing time between backup passes, achieving complete data protection without the time loss associated with traditional multi-pass operations.
Solution Approach 2:
The patent performs preliminary capture of writes in the primary bitmap before delta generation begins. This preliminary action allows the delta processing thread to work with pre-identified dirty extents without repeatedly scanning volumes, significantly reducing processing time while maintaining complete data protection.
Data Source
AI summary
One example method includes performing delta operations to protect data. During a delta operation, a primary map and a secondary map are processed using bit logic. The bit logic determines how to handle data stored at a location on the volume associated with an entry in the primary map and included in the current delta operation when a new write for the same location is received as the corresponding entry in the primary map is processed.