Snapshot Tracking Files for File System Replication
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
High performance distributed file systems face challenges in maintaining consistent backups and restorations due to interruptions during long-running backup and mirroring processes, leading to inconsistent data and difficulties in generating accurate point-in-time restorations.
Innovation Solution
The implementation of snapshot tracking files and snapshot change sets to establish point-in-time versions of file system clusters, allowing for the tracking of modified file system objects and incremental synchronization between primary and secondary clusters, ensuring data consistency and enabling failover and failback procedures.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If backup systems mirror data on primary file system, then data redundancy is improved, but data consistency during interruption deteriorates
Solution Approach 1:
The system creates snapshots of the file system at specific points in time before backup operations begin. These snapshots serve as predetermined reference states that allow the system to track changes and ensure consistency even when backup processes are interrupted. The snapshot tracking file records the state of file system objects at the time of snapshot creation, enabling accurate restoration to that specific point-in-time.
Solution Approach 2:
The system implements feedback mechanisms through snapshot tracking files that continuously monitor and record changes to file system objects between snapshots. This feedback loop allows the backup system to detect when changes occur and adjust the mirroring process accordingly, ensuring that the secondary file system reflects accurate point-in-time states even during interruptions.
2Productivity
If long running backup processes operate continuously, then backup coverage is improved, but process interruption increases
Solution Approach 1:
The backup process is segmented into discrete snapshot operations rather than continuous mirroring. Each snapshot captures a point-in-time state of the file system, and the tracking file records changes between snapshots. This segmentation allows the system to take multiple snapshots at different times, providing comprehensive backup coverage while isolating interruptions to individual snapshot operations rather than affecting the entire backup process.
Solution Approach 2:
Snapshots are created at predetermined points in time before backup operations begin, establishing baseline states that remain valid even if subsequent backup processes are interrupted. This preliminary action ensures that restoration can always proceed from a known consistent state, maintaining reliability while allowing continuous backup coverage through multiple snapshot creation opportunities.
3Speed
If incremental synchronization is implemented, then replication speed is improved, but data consistency tracking complexity increases
Solution Approach 1:
The system extracts only the essential change information between snapshots into the snapshot tracking file, rather than copying entire file system states. This extraction approach stores only the deltas (changes, creations, deletions) between snapshot points, significantly reducing the data volume that needs to be synchronized while maintaining the ability to reconstruct complete file system states for consistent replication.
Solution Approach 2:
The snapshot tracking file is created and populated in advance during snapshot operations, recording all necessary change information before replication begins. This preliminary preparation of change data eliminates the need for complex real-time consistency tracking during replication, as the tracking file already contains the complete change history needed for incremental synchronization.
Data Source
AI summary
Embodiments are directed towards replication and restoration of file system objects stored on file system clusters. Modified file system objects may be tracked using a snapshot tracking file corresponding to a snapshot. The snapshot tracking file may be employed to generate a snapshot change set. The snapshot change set may be used to restore a file system cluster to a version corresponding to a snapshot by reversing the changes in the snapshot change set. Also, a snapshot change set may be used to replicate file system objects by committing the included changes on another file system cluster. If a primary file system cluster becomes unavailable the file system may failover to the secondary file system cluster. If the primary file system cluster later becomes available it may be restored by using snapshot change sets generated on the secondary file system cluster.