File System Replication Using Changelog and Inode Scanning
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods for replicating file systems in use are inefficient and often result in inaccurate replicas due to the inability to synchronize changes in real-time, leading to excessive downtime and resource wastage, especially when the system is actively modified during the replication process.
Innovation Solution
A method utilizing a changelog to record modifications of the file system, initiating a scan of directory inodes, and updating a database with sequential records from the changelog to build a coherent replica target file system, allowing for replication while the source file system is in use.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If a full file system scan is performed to create a replica, then the replica can be synchronized, but the process takes several hours and the file system must be quiescent (no activity), causing excessive downtime
Solution Approach 1:
The patent applies preliminary action by performing a complete file system scan and creating the replica structure before the file system becomes active. The method scans the file system hierarchy, creates the replica directory structure, and prepares all necessary data structures in advance, so that when the file system goes online, the replica is already in place and only incremental updates are needed.
Solution Approach 2:
The patent segments the replication process into distinct phases: an initial offline scan phase where the complete file system hierarchy is traversed and the replica structure is created, and a subsequent online phase where only incremental changes are applied. This segmentation allows the time-consuming synchronization work to be done when the file system is offline, while minimizing downtime when it is active.
2Reliability
If the file system is kept quiescent during replication to ensure accuracy, then a coherent replica can be produced, but productivity and availability are severely reduced
Solution Approach 1:
The patent performs the complete file system scan and replica creation as a preliminary action during a maintenance window when the file system is offline. This ensures that the replica structure is fully established with all files and directories in their correct positions before the file system becomes active, guaranteeing replica coherence without requiring the file system to remain quiescent during normal operations.
Solution Approach 2:
The patent implements periodic action by scheduling complete replication scans at regular intervals or during planned maintenance windows, rather than requiring continuous quiescence. Between these periodic full scans, the system can use incremental updates or changelogs to keep the replica synchronized, allowing the file system to remain fully operational while still maintaining replica coherence through periodic re-synchronization.
3Productivity
If incremental updates are applied during file system operation, then availability is maintained, but the replica may become inconsistent if changes occur during the scan
Solution Approach 1:
The patent introduces an intermediary mechanism in the form of a changelog or transaction log that records all file system changes. This intermediary allows the replication process to capture changes atomically - either all changes up to a certain point are replicated, or none are - preventing partial or inconsistent states. The changelog acts as a buffer between the file system operations and the replica updates, ensuring consistency even when the file system is active.
Solution Approach 2:
The patent implements feedback by continuously monitoring the file system state through changelogs and using this information to update the replica. The system reads the changelog to determine what changes have occurred since the last replication, applies only those specific changes to the replica, and verifies consistency. This feedback loop ensures that the replica remains synchronized with the source file system without requiring the source to be quiescent.
Data Source
AI summary
A method of replicating a source file system on a storage system, the method including: utilising a changelog operable to store sequentially a data record corresponding to a modification of a source file system by a storage request; specifying a first data record of the changelog at a first time; initiating a scan of the directory inodes of the source file system at the first time; scanning the directory inodes; completing the scan at a second time later than the first time; specifying a second data record of the changelog at the second time; storing, in a database, information corresponding to the scanned directory inodes; updating the database with sequential data records from the first data record to the second data record of the changelog; and building, on a computer system, a replica target file system based on the database.