Clone File Resync Rebuild Without Destination Data Warehouse
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional replication engines require rebuilding a data warehouse at the destination volume during resync operations, leading to significant time consumption and failure to meet disaster recovery service level objectives due to the need to rebuild the entire volume, even for data modifications, which is inefficient and time-consuming.
Innovation Solution
Replicate clone files without relying on a data warehouse at the destination by tracking clone relationships in a metafile and transferring inode information and parent file information to reconstruct clone files, preserving storage efficiency by referencing shared blocks, thus avoiding the overhead of rebuilding the entire data warehouse.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If conventional replication engines rebuild the entire data warehouse at the destination volume during resync operations, then data consistency is ensured, but resync time becomes excessively long and service level objectives are not met
Solution Approach 1:
The patent extracts only the essential clone relationship information (inode information and parent file information) from the source file system and transfers it to the destination, rather than copying the entire data warehouse. This selective extraction of critical metadata enables clone file reconstruction without requiring a complete data warehouse rebuild, thus reducing resync time while maintaining data consistency.
Solution Approach 2:
The patent performs preliminary tracking of clone relationships in a metafile at the source node before the resync operation. By pre-identifying and recording clone file relationships (which files are clones of which parent files), the system prepares the necessary information in advance, allowing the destination node to efficiently reconstruct clone files during resync without having to analyze the entire file system structure from scratch.
2Reliability
If the entire data warehouse is rebuilt at the destination during resync, then complete data replication is achieved, but storage efficiency is reduced due to redundant data copying
Solution Approach 1:
The patent uses a metadata copying approach where only clone relationship information (inode numbers, parent file identifiers) is transferred from source to destination, rather than physically copying actual data blocks. The destination node reconstructs clone files by referencing the transferred parent file information, creating a lightweight copy that preserves storage efficiency while achieving complete data replication.
3Reliability
If clone files are replicated using conventional methods requiring data warehouse reconstruction, then data integrity is maintained, but the complexity of the replication process increases significantly
Solution Approach 1:
The patent segments the clone file replication process into distinct components: (1) tracking clone relationships in a metafile at the source, (2) transferring only inode and parent file information to the destination, and (3) reconstructing clone files at the destination using the transferred metadata. This segmentation simplifies the overall replication process by breaking it down into manageable, independent steps that can be executed efficiently while maintaining data integrity.
Data Source
AI summary
Techniques are provided for rebuilding a destination file system as part of a resync operation. A file system of a first node may be replicated to a second node as a destination file system. The file system may include clone files that were created as clones of parent files. The clone files may have been created in a storage efficient manner where a clone file refers to data of a parent file, instead physically storing duplicate instances of the parent data of the parent file. The techniques provided herein are capable of replicating the clone files to the second node while preserve the storage efficiency of the clone files during a resync operation that rebuilds the destination file system after recovering from a failure of the second node.