File System Replication via Segment-Level Tracking
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing data replication methods are inefficient and bandwidth-intensive, as they often duplicate all data or require significant network bandwidth to maintain synchronization between original and replicated file systems, especially when files share identical data segments.
Innovation Solution
The method involves segmenting file system data into identifiable segments, using metadata and content handles to replicate only changed segments and references, eliminating duplicate storage and reducing bandwidth usage by transferring only new or modified segments during updates.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If the entire contents of the original storage system are copied to the replicated file system periodically, then the replicated file system is synchronized with the original, but a large amount of bandwidth is consumed and all data is duplicated regardless of modifications
Solution Approach 1:
The patent segments data into discrete data segments with unique identifiers, allowing the system to track and replicate only changed segments rather than copying entire files or datasets. This segmentation enables selective replication based on actual changes, reducing bandwidth consumption while maintaining synchronization.
Solution Approach 2:
The patent introduces a content handle parameter that changes when data is modified. By monitoring changes in content handles, the system can identify which data segments need replication, avoiding unnecessary copying of unchanged data and reducing overall bandwidth consumption during synchronization.
2Reliability
If operations are reproduced on both original and replica, then synchronization is maintained, but significant bandwidth is required and a reliable connection is needed
Solution Approach 1:
The patent extracts only the necessary change information (data segment identifiers and content handle changes) from the full operation log, sending only this condensed information to the replica. This extraction approach maintains synchronization while significantly reducing bandwidth requirements compared to transmitting complete operation records.
Solution Approach 2:
Instead of copying entire files or operations, the patent creates a lightweight representation of changes using data segment references and content handles. This copy approach transmits only the essential change information needed for synchronization, reducing bandwidth consumption while maintaining the ability to reproduce operations on the replica.
3Adaptability or versatility
If log records are used to improve operation reproduction, then reliable connection is not needed, but significant bandwidth is still required as logs can grow large
Solution Approach 1:
The patent extracts only the essential change information from complete operation logs, isolating and transmitting only the data segment identifiers and content handle changes. This extraction reduces log size and bandwidth consumption while maintaining the flexibility to operate without constant reliable connections.
Solution Approach 2:
By segmenting operations into discrete, identifiable data segment changes with unique identifiers, the patent creates a compact representation that can be efficiently transmitted. This segmentation allows the system to send only the necessary change information rather than complete operation records, reducing bandwidth requirements.
4Device complexity
If file system snapshots are used, then logs are eliminated, but bandwidth requirement remains large when files are frequently modified
Solution Approach 1:
The patent segments the file system into discrete data segments with unique identifiers, allowing snapshots to track only segment-level changes rather than file-level changes. This segmentation enables more granular comparison between snapshots, identifying only the actual data changes that need replication and reducing bandwidth consumption during frequent modifications.
Solution Approach 2:
The patent uses content handles as parameters that change only when data segments are modified. By monitoring changes in these content handle parameters between snapshots, the system can efficiently identify which segments need replication, reducing bandwidth consumption compared to comparing entire file systems or using traditional log records.
Data Source
AI summary
File system replication includes determining whether one of a plurality of files included in an original file system has been updated since a previous replication, the file having a plurality of data segments, and in the event that the file has been updated, locating among the plurality of data segments a previously stored data segment that is newly referenced by the file, and that does not require replication.