Uniform Logic Replication for DDFS Segment Tree Format Translation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Replicating data between storage systems with different segment tree formats is problematic due to differences in segment sizes and structures, leading to inefficiencies in data transfer and management.
Innovation Solution
The solution involves generating fingerprints for each segment using a collision-resistant hash function and translating or resegmenting segments from one format to another as needed, allowing for efficient replication and management across systems with varying segment tree formats.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If segment trees are implemented in different formats (UST and NONUST) in different storage systems, then each system can optimize its own storage structure, but replication between systems becomes problematic due to format incompatibility
Solution Approach 1:
The patent introduces a format translation mechanism that acts as an intermediary between UST and NONUST segment tree formats. The source storage system translates its segment tree format to match the target storage system's format before replication, enabling compatibility between different formats without requiring both systems to use the same format.
Solution Approach 2:
The patent changes the segment tree format parameters (segment sizes, tree structure) dynamically based on the target storage system's requirements. The source system can translate between different segment sizes and tree structures, allowing the same data to be represented in multiple format variations suitable for different storage systems.
2Productivity
If segment sizes differ between UST and NONUST formats, then each format can be optimized for its specific use case, but direct comparison of segments for change detection becomes impossible
Solution Approach 1:
The patent dynamically adjusts segment size parameters during translation. When translating from UST to NONUST or vice versa, the system recalculates segment boundaries and sizes to match the target format's requirements, enabling accurate segment-by-segment comparison even when original segment sizes differed.
Solution Approach 2:
The patent re-segments data during format translation, dividing or combining segments to match the target format's segment structure. This ensures that segments in the translated format align properly with the target system's expected segment boundaries, enabling precise change detection.
3Reliability
If replication transfers entire files, then data integrity is ensured, but transfer efficiency is reduced when only small portions of files have changed
Solution Approach 1:
The patent divides files into segments and transfers only the segments that have changed, rather than transferring entire files. By comparing segment fingerprints or metadata, the system identifies modified segments and transfers only those portions, significantly reducing transfer bandwidth requirements while maintaining data integrity.
Solution Approach 2:
The patent extracts and transfers only the necessary modified segments from source files, leaving the unchanged segments at the source. This selective extraction approach ensures that only the minimal required data is transmitted, optimizing transfer efficiency while ensuring the target system receives all necessary changes.
Data Source
AI summary
In one embodiment, the storage system determines if a first format of a first segment tree of the first file system is different from a second format of a second segment tree of the second file system representing a file stored in the first and second file systems, respectively. The storage system identifies, in response to determining that the first and second formats are different, a second level within the first and second segment trees that have different formats. In one embodiment, the storage system further identifies one or more segments of the second level of the first segment tree that have been modified based on a comparison of fingerprints of a third level of the segment trees. For each modified second level segment, the storage system resegments the segment from the first to the second format, and replicates the resegmented segments to the target storage system.