Bitmap Data Structure for Secondary Replication
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing data replication systems face inefficiencies when synchronization is lost between a source and a target device, often requiring inefficient full data replication due to communication failures or log wraps, leading to bandwidth and processing inefficiencies.
Innovation Solution
Implementing a secondary bitmap-based replication method that uses a hierarchical bitmap structure, including a recovery bitmap, checkpoint bitmap, and accumulating bitmap, to identify and transmit only changed data blocks, allowing for efficient data replication when log-based methods fail.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of energy
If log-based replication is used as the primary method, then data synchronization is maintained with low bandwidth usage, but when synchronization is lost, full data retransmission is required causing high bandwidth consumption
Solution Approach 1:
The system pre-generates bitmap data structures that track all data block changes on the source device before replication is needed. When synchronization is lost, this pre-computed bitmap information enables immediate identification of changed blocks without requiring full data retransmission, thus resolving the contradiction between bandwidth efficiency and synchronization reliability
Solution Approach 2:
A bitmap data structure is introduced as an intermediary mechanism between the source and target devices. This bitmap serves as a compact representation of data changes that can be transmitted efficiently even when log-based replication fails, allowing the system to maintain low bandwidth consumption while ensuring synchronization reliability through an alternative replication path
2Reliability
If full data replication is performed when synchronization is lost, then synchronization reliability is restored, but processing requirements and time consumption increase significantly
Solution Approach 1:
The system extracts only the essential synchronization information into a compact bitmap data structure, separating the critical change tracking data from the full data set. When synchronization is lost, only this extracted bitmap information needs to be processed and transmitted, dramatically reducing processing requirements and restoring replication speed while maintaining synchronization reliability
Solution Approach 2:
The system changes the parameter representation of data changes from full data block transmission to compact bitmap notation. This parameter transformation allows the system to maintain complete tracking of data changes while using minimal processing resources and time, resolving the contradiction between reliability restoration and replication efficiency
3Loss of energy
If log-based replication is used, then bandwidth efficiency is maintained, but the system is vulnerable to log wraps causing synchronization loss
Solution Approach 1:
The system implements feedback mechanisms where the target device monitors log sequence numbers and detects potential log wrap conditions. When a log wrap is detected or anticipated, the system switches from log-based replication to bitmap-based replication, maintaining bandwidth efficiency while providing resilience against synchronization loss. The bitmap data structure continuously tracks all changes, ensuring no data is lost during the transition
Solution Approach 2:
The system prepares bitmap data structures in advance that cover the entire data set, creating a safety cushion against log wraps. This pre-computed bitmap information acts as a backup mechanism that can be activated immediately when log-based replication becomes unreliable, protecting the system from synchronization loss while maintaining efficient bandwidth usage during normal operation
Data Source
AI summary
Examples of the present disclosure describe implementing bitmap-based data replication when a primary form of data replication between a source device and a target device cannot be used. According to one example, a temporal identifier may be received from the target device. If the source device determines that the primary replication method is unable to be used to replicate data associated with the temporal identifier, a secondary replication method may be initiated. The secondary replication method may utilize a recovery bitmap identifying data blocks that have changed on the source device since a previous event.