File Service Replication Scheduling With Selective Share Sync
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing data replication technologies inefficiently transfer data across geographically separated datacenters, leading to unnecessary data transfer and lack of control over specific synchronization, especially for high-level logical constructs like shares, resulting in inefficiencies and potential data loss.
Innovation Solution
A replication policy management and scheduling technique that manages data distribution and consolidation across multiple sites, using file-level synchronization and asynchronous replication via file-oriented snapshots, allowing precise control over replication of high-level constructs like shares or files, with customer-defined filters and dynamic mapping.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If base snapshot and incremental changes are transferred for the entire share, then data replication is achieved, but needless data transfer occurs when only a fraction of the share is of interest
Solution Approach 1:
The patent segments the share into individual files and further into blocks, allowing selective replication of only the necessary portions. Instead of transferring entire shares or volumes, the system identifies and replicates only the specific files or blocks that have changed or are required at the target site, thereby reducing unnecessary data transfer while maintaining replication reliability.
Solution Approach 2:
The patent implements local quality by allowing different replication policies and synchronization controls for different files or directories within the same share. Each file or block can be independently tracked and replicated based on its own change status, enabling precise control over what data is transferred to each target site based on local requirements.
2Device complexity
If incremental changes apply only to blocks for the entire share or volume, then replication is simplified, but control of specific synchronization such as application of deletions and overwrites on a per-file-basis is not possible
Solution Approach 1:
The patent segments the replication control mechanism into file-level and block-level operations. By maintaining file system metadata and tracking changes at the file level while performing actual data transfer at the block level, the system achieves both fine-grained control over what files to replicate and efficient block-level transfer operations, resolving the contradiction between control precision and operational simplicity.
3Reliability
If separate replication is performed for each share in a volume with a large number of underlying shares, then each share is replicated accurately, but management becomes exacerbated and inefficient
Solution Approach 1:
The patent merges multiple share replications into a unified volume-level replication operation. By treating the entire volume as a single replicable entity and using a common namespace across shares, the system can replicate data from one volume to another while automatically handling the distribution across multiple shares, thereby reducing management complexity while maintaining accurate replication of each share.
Solution Approach 2:
The patent implements a universal replication mechanism that can handle multiple shares simultaneously through a single replication process. The system uses a common namespace and unified change tracking that works across all shares in a volume, allowing one replication operation to serve multiple shares rather than requiring separate replication processes for each share.
Data Source
AI summary
A replication policy management and scheduling technique of a Files service is configured for deployment in multi-site data replication environments. The technique involves policy management for data distribution and/or data consolidation (concentration) where multiple sources, e.g., file system (FS) clusters or sites, replicate the data to a central target FS site, e.g., in a spoke and hub arrangement typical of remote office/branch office environments. The technique also involves creation and configuration of a main replication policy by a customer at a central resource manager configured to interact and manage the FS sites, each of which includes one or more file server virtual machines.