Multi-Client Change Tracking for Snapshot-Based File Replication
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing file system replication methods are inefficient and lack effective delta generation processes, particularly in enterprise environments where managing large datasets across multiple clients is challenging.
Innovation Solution
A change tracking and delta generation (CTDG) system that enables delta generation proportional to the number of changes between snapshots rather than the size of the snapshots, supporting multiple clients and allowing for dynamic enablement or disablement of change tracking.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If traditional file system replication methods are used, then data can be replicated between regions, but the delta generation process is inefficient and scales with the size of snapshots rather than the number of changes
Solution Approach 1:
The patent segments the change tracking process by creating separate change tracking instances for different clients, each maintaining its own change tracking state. This allows delta generation to focus only on changes specific to each client rather than processing entire snapshots, thereby improving efficiency and reducing time loss.
Solution Approach 2:
The patent implements dynamic change tracking by enabling and disabling change tracking on running replication operations. The system can dynamically adjust the change tracking state based on replication progress and client needs, allowing flexible optimization of delta generation efficiency without restarting replication processes.
2Adaptability or versatility
If change tracking is enabled for multiple clients, then each client can have customized replication, but the system complexity increases
Solution Approach 1:
The patent creates a universal change tracking data structure that serves multiple clients simultaneously. Each client has its own change tracking instance using the same underlying data structure format, allowing the system to handle multiple clients with consistent logic while maintaining individual customization capabilities.
Solution Approach 2:
The patent uses snapshot copying to create point-in-time references of the file system state. By copying snapshots rather than tracking every individual change event, the system reduces complexity while still enabling comprehensive multi-client replication tracking.
3Reliability
If delta generation is based on snapshot size, then all data is replicated, but the process is inefficient when only small portions change
Solution Approach 1:
The patent extracts only the changed portions of data from snapshots by using change tracking information to identify and extract delta sets. Instead of replicating entire snapshots, the system extracts and replicates only the necessary changed blocks, maintaining data completeness while dramatically improving replication efficiency.
Solution Approach 2:
The patent performs preliminary change tracking during the replication process to identify changes before actual delta generation occurs. By pre-identifying changed blocks through change tracking data structures, the system prepares the exact set of changes that need to be replicated, avoiding unnecessary processing of unchanged data.
Data Source
AI summary
A change tracking and delta generation (CTDG) system servicing multiple clients is disclosed for enabling the delta generation during replication to be proportional to the number of changes between two snapshots instead of the size of the snapshots. In some embodiments, CTDG can track changes to a file system, where change-tracking can be dynamically switched between two sub-CT modes, file level or directory level. Change-tracking (CT) keys may be generated accordingly and stored in a data structure to be used for delta generation during replications for clients. In some embodiments, change-tracking may be enabled or disabled for different clients. CTDG can coordinate the CT enablement/disablement to ensure replications can use the information in CT keys properly.