Lightweight Backup Filter With Concurrent Bitmap Tracking
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional light weight filter designs can only support a single concurrent backup of an asset, failing to meet the increasing demand for multiple concurrent backups.
Innovation Solution
Implement a light weight filter design (LWD) that supports multiple concurrent backups by using metadata tracking and data transfer, creating a new bitmap fragment for each backup stream and performing differential map calculations to consolidate changes, allowing multiple backups to be performed concurrently while handling ongoing IO operations.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If conventional light weight filter designs are used, then the filter structure remains simple and manageable, but only a single concurrent backup can be supported
Solution Approach 1:
The patent segments the backup tracking mechanism by creating separate bitmap fragments for each backup stream. Each bitmap fragment independently tracks changes for a specific backup, allowing multiple concurrent backups to be managed separately. This segmentation enables the system to support multiple concurrent backups while maintaining the simplicity of individual bitmap tracking structures.
Solution Approach 2:
The patent implements a universal bitmap structure that can serve multiple backup streams simultaneously. By using a consistent bitmap fragment format and consolidation approach across all backup streams, the system achieves multi-functionality where the same underlying mechanism handles multiple concurrent backups, increasing adaptability without proportionally increasing complexity.
2Adaptability or versatility
If multiple concurrent backups are implemented, then backup versatility increases, but metadata management complexity increases
Solution Approach 1:
The patent divides metadata tracking into separate bitmap fragments for each backup stream. Each fragment contains only the metadata necessary for that specific backup, preventing the need to manage all backup metadata simultaneously in a single complex structure. This segmentation reduces metadata management complexity while supporting multiple concurrent backups.
Solution Approach 2:
The patent performs preliminary consolidation of bitmap fragments into a consolidated bitmap before actual backup operations. This preliminary action organizes the metadata structure in advance, making the actual backup process simpler and reducing the complexity of real-time metadata management during concurrent operations.
3Adaptability or versatility
If multiple bitmap fragments are created for each backup stream, then concurrent backup capability improves, but memory consumption increases
Solution Approach 1:
The patent merges multiple bitmap fragments into a single consolidated bitmap for each backup stream. This consolidation reduces the total number of separate metadata structures that need to be maintained in memory, thereby reducing overall memory consumption while still supporting multiple concurrent backup streams through the fragment-consolidate pattern.
Data Source
AI summary
One example method includes receiving backup requests, creating a respective bitmap fragment for each backup request, each bitmap fragment corresponding to a respective tracking start time, for one of the requested backups, identifying a group of bitmap fragments that correspond to at timeframe extending from a time at which a previous backup was requested to a time at which the particular backup was requested, using the bitmap fragments of the group to create a consolidated bitmap that includes all changes to data in a volume resulting from IOs during the timeframe, with the consolidated bitmap, reading the data from the volume and transmit the data to a recipient, identifying new write IOs, received in parallel with transmission of the data, that are directed to data included in the particular backup, and reading, from the volume, the data to which the new write IOs are directed, and temporarily storing that data.