Resumable Replica Resynchronization Using Bitmap Tracking
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In distributed storage systems, resynchronization of replicas after disconnection is a time-consuming process, and replicas may become unavailable again during synchronization, leading to inefficient use of resources and prolonged synchronization times.
Innovation Solution
The use of bitmaps, such as resyncing and tracking bitmaps, to track unsynchronized data blocks, allowing for targeted data copying during resynchronization, thereby avoiding redundant data transfers and optimizing resource usage.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If full resynchronization is performed when a replica reconnects, then data consistency is ensured, but resynchronization time and resource consumption increase significantly
Solution Approach 1:
The patent divides the resynchronization process into segments by using bitmaps to identify and track specific data blocks that require synchronization. Instead of synchronizing all data blocks, the system segments the work into only those blocks marked as unsynchronized in the bitmap, significantly reducing the time and resources needed for resynchronization while maintaining data consistency.
Solution Approach 2:
The patent implements preliminary action by maintaining bitmaps that track the synchronization status of data blocks before resynchronization begins. When a replica disconnects and reconnects, the bitmap already contains information about which blocks need updating, allowing the system to immediately resume synchronization from the correct point without retransmitting already-synchronized data.
2Reliability
If full resynchronization is performed when a replica reconnects, then data consistency is ensured, but network bandwidth and CPU cycles are wasted on redundant data transfers
Solution Approach 1:
The patent extracts only the necessary information for resynchronization by using bitmaps to identify specific unsynchronized data blocks. The system extracts and transmits only those blocks that require synchronization, eliminating redundant data transfers and reducing network bandwidth consumption and CPU cycle usage while ensuring data consistency.
3Productivity
If resynchronization resumes after replica disconnection, then synchronization progress is maintained, but tracking and managing partial synchronization states increases system complexity
Solution Approach 1:
The patent uses bitmaps as simplified copies or representations of the actual data blocks' synchronization status. Instead of managing complex state information for each data block, the system maintains a compact bitmap copy that tracks which blocks need synchronization, making it easier to manage partial synchronization states while maintaining high productivity.
Data Source
AI summary
Systems and techniques are described for transferring data. A described technique includes determining that a first replica of an object stored at a first host has become available to a distributed storage system after previously being unavailable to the distributed storage system. The object includes a range of memory addresses at which data of the object is stored. In response to determining that the first replica has become available, resyncing data for the first replica is obtained. The resyncing data indicates whether each range of memory addresses is synchronized at the first replica with other replicas of the object. Tracking data for the first replica is obtained. The tracking data indicates whether data stored at the range of memory addresses of the object has been modified at a second replica while the first replica was unavailable. The resyncing data is updated based on the tracking data.