Filesystem Replication via Minimal Metadata Changelog
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current methods of file system replication are inefficient and costly, especially as file system sizes increase, due to the need to constantly scan for changes and maintain replication across physically distant locations.
Innovation Solution
A method using a minimal file system metadata changelog to efficiently replicate a file system by querying the source file system with minimal queries, determining changes, and applying actions to a target file system based on changelog entries, thereby reducing unnecessary operations and maintaining a complete copy.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If traditional file system replication methods are used to maintain data redundancy and retrieval rate, then data protection and data retrieval capability are improved, but system overhead and operational costs increase significantly
Solution Approach 1:
The patent extracts only the essential metadata information (file creation, modification, deletion events) from the source file system and stores it in a changelog. This extracted metadata is then used to replicate changes to the target file system, avoiding the need to continuously scan and copy entire file systems. This selective extraction of critical information reduces the replication overhead while maintaining data protection and consistency.
Solution Approach 2:
The patent performs preliminary actions by maintaining a changelog that records all file system operations as they occur. This preliminary recording of changes allows the replication process to only apply necessary updates to the target system rather than performing full scans and comparisons, significantly reducing the computational overhead and energy consumption of the replication process.
2Reliability
If traditional file system replication methods are used to ensure data redundancy, then data protection is improved, but operational complexity and costs increase
Solution Approach 1:
The patent introduces a changelog as an intermediary component between the source and target file systems. This changelog acts as a mediator that captures file system operations and provides a simplified interface for replication. Instead of implementing complex scanning and comparison logic, the system simply reads from the changelog and applies recorded operations to the target, significantly reducing operational complexity while maintaining data protection.
3Quantity of substance
If file system sizes increase to store more data, then data storage capacity is improved, but replication overhead and costs are exacerbated
Solution Approach 1:
The patent extracts only the essential metadata information (file creation, modification, deletion events) from the source file system and stores it in a changelog. This extracted metadata is then used to replicate changes to the target file system, avoiding the need to continuously scan and copy entire file systems. This selective extraction of critical information reduces the replication overhead while maintaining data protection and consistency.
Solution Approach 2:
Instead of performing complete file system scans and copying all data, the patent applies partial action by only replicating the specific changes recorded in the changelog. This partial replication approach focuses computational resources only on the necessary updates rather than the entire file system, significantly reducing the energy and time overhead even as file system sizes grow to accommodate larger data volumes.
4Reliability
If file system replications are stored in physically distant locations to improve data security and retrieval rate, then data protection and retrieval capability are improved, but replication overhead and operational costs increase
Solution Approach 1:
The patent performs preliminary actions by maintaining a changelog that records all file system operations as they occur. This preliminary recording of changes allows the replication process to only apply necessary updates to the target system rather than performing full scans and comparisons, significantly reducing the computational overhead and energy consumption of the replication process.
Solution Approach 2:
The patent implements continuous monitoring and recording of file system operations in the changelog, ensuring that all changes are captured as they occur. This continuous action at the source enables efficient incremental replication to distant locations, reducing the time loss associated with periodic full synchronizations while maintaining data protection across geographically distributed systems.
Data Source
AI summary
In general, the invention relates to replicating a source file system stored on a first memory by obtaining a first unread entry from a changelog associated with the source file system, querying the source file system using the first unread entry to obtain a current first object file status, a current first object file path, a current first parent directory status, and a current first parent directory path, determining, based on the querying, whether a first object file on the source file system has changed at some time after the execution of the first unread entry, if the first object file has not changed, performing a first action on a target file system, and if the first object file has changed, performing a second action on the target file system.