Replica Cleanup During Storage Resynchronization
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional data synchronization methods between storage nodes are CPU-intensive, memory-intensive, and bandwidth-heavy, especially during resynchronization, as they require comparing and transferring large lists of data segments, leading to inefficiencies in deletion cleanup and synchronization.
Innovation Solution
The method involves assigning unique identifiers to data segments, sorting them, and identifying gaps to determine deleted ranges, which are then transmitted to the secondary node for synchronization, allowing for O(1) time and space complexity, reducing the need for full list comparison and bandwidth usage.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If conventional data synchronization methods are used to compare and transfer large lists of data segments, then data consistency between storage nodes is achieved, but CPU time, memory usage, and bandwidth consumption increase significantly
Solution Approach 1:
The patent extracts only the essential information needed for synchronization by identifying and transmitting only the deleted data segment ranges rather than transferring complete data segment lists. This extraction approach maintains data consistency while dramatically reducing the amount of data that needs to be processed and transmitted.
Solution Approach 2:
The patent segments the data synchronization process into identifying deleted ranges and transmitting only those specific segments. By dividing the synchronization task into focused segments rather than processing entire data lists, the system achieves better efficiency while maintaining reliability.
2Reliability
If complete data segment lists are transferred during resynchronization, then deleted data can be identified and removed, but network bandwidth and transmission time are excessively consumed
Solution Approach 1:
The patent extracts only the critical deletion information (ranges of deleted data segments) and transmits this extracted data instead of complete data segment lists. This approach ensures that deletion synchronization is achieved while minimizing network bandwidth consumption and transmission time.
3Measurement precision
If conventional methods crawl the entire file system name space to identify deleted data, then accurate deletion detection is achieved, but CPU time and memory usage increase significantly
Solution Approach 1:
The patent extracts only the necessary deletion range information without crawling the entire file system name space. By extracting this specific information directly, the system achieves accurate deletion detection while avoiding the excessive CPU time and memory usage associated with comprehensive name space crawling.
Data Source
AI summary
Mechanisms are provided for efficient replica cleanup during resynchronization. According to various embodiments, a plurality of deleted data segment ranges on a first storage node may be identified. The first storage node may be configured to store a plurality of data segments. Each of the plurality of data segments may have associated therewith a respective identifier. Each of the data segment ranges may designate one or more data segments that have been deleted from the first storage node. The plurality of deleted data segment ranges may be transmitted to a second storage node configured to mirror the plurality of data segments stored on the first storage node. The plurality of deleted data segment ranges may be capable of being used to identify one or more data segments to delete from the second storage node.