Delta Component Synchronization for Distributed Storage Maintenance
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Distributed data objects face reduced data availability and durability when components become unavailable for maintenance, and upon returning online, they may be stale due to missed I/O traffic, leading to synchronization challenges.
Innovation Solution
Generating a delta component with an address space matching the base component and a tracking bitmap to route write operations, synchronize data blocks, and remove the delta component once the base component is up-to-date.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of repair
If a component is taken offline for maintenance, then the component can be updated and maintained, but data availability and durability are reduced during the maintenance period
Solution Approach 1:
A delta component is created in advance before the base component is taken offline. This delta component serves as a temporary placeholder that can immediately accept write operations, ensuring data availability is maintained during the maintenance window without requiring the base component to be online.
Solution Approach 2:
The delta component acts as an intermediary between the unavailable base component and incoming write operations. It receives and stores data changes during maintenance, then synchronizes these changes back to the base component when it comes online, effectively mediating the data flow disruption caused by maintenance.
2Ease of operation
If a component returns online after maintenance, then the component becomes available again, but it becomes stale due to missed I/O traffic
Solution Approach 1:
A tracking bitmap is implemented that continuously monitors and records which data blocks have been modified in the delta component. When the base component comes online, this bitmap provides feedback information about exactly which blocks need to be synchronized, enabling efficient selective updates rather than blind full-synchronization.
Solution Approach 2:
The tracking bitmap is maintained continuously during the maintenance period, preliminarily identifying all modified data blocks before synchronization is needed. This preliminary tracking eliminates the need for expensive full-data comparisons when bringing the component back online, as the sync process can immediately target only the identified changed blocks.
3Reliability
If traditional synchronization methods are used to update the base component, then data consistency is restored, but bandwidth and processing requirements increase significantly
Solution Approach 1:
The tracking bitmap extracts and isolates only the specific data blocks that have changed during maintenance, separating them from the unchanged majority of the data. When synchronizing, only these extracted changed blocks are transferred back to the base component, dramatically reducing the bandwidth and processing requirements compared to traditional methods that would synchronize all data.
Solution Approach 2:
Instead of performing a complete full-data synchronization, the system applies partial action by synchronizing only the subset of data blocks that actually changed. The tracking bitmap enables this selective partial synchronization, avoiding the excessive bandwidth consumption and processing overhead of transferring and comparing entire data sets that contain mostly unchanged information.
Data Source
AI summary
The disclosure herein describes enhancing data durability of a base component using a delta component. A delta component is generated based on the base component becoming unavailable. The delta component is configured to include unwritten storage space with an address space matching the base component and a tracking bitmap associated with data blocks of the address space of the delta component. Write operations targeted for the base component are routed to the delta component. Based on the routed write operations, bits associated with data blocks affected by the write operations are changed in the tracking bitmap. Based on the base component becoming available, data blocks affected by routed write operations are identified based on the tracking bitmap and the identified data blocks are synchronized from the delta component to the base component. The delta component is then removed.