Snapshot Backup File System Traversal via Splitter Replication
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods for backing up virtual machines using snapshots are inefficient due to high computing resource requirements and storage I/O operations, particularly in virtual machine file systems, which can negatively impact storage appliance performance and limit the frequency of snapshot-based backups.
Innovation Solution
A system that includes a splitter module to replicate write operations, a data protection appliance to generate snapshots of changed data blocks, and a method to map these blocks to a file system structure using depth-first traversal and high-speed storage for efficient transfer to backup storage, reducing unnecessary disk I/O operations.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If snapshot-based backup is implemented on storage arrays, then backup capability is provided, but substantial disk I/O operations are required which negatively impact storage appliance performance
Solution Approach 1:
The system separates the backup function from the storage array by introducing a storage appliance that receives data copies from the array. The storage array performs only minimal copy operations while the storage appliance handles the substantial I/O operations for snapshot creation and file system traversal, thus segmenting the workload and protecting storage array performance.
Solution Approach 2:
The storage appliance acts as an intermediary between the storage array and the backup destination. It receives replicated data blocks, creates snapshots, and performs file system traversal to determine file associations, thereby mediating the backup process and preventing direct impact on storage array performance.
2Loss of information
If file system traversal is performed to determine file associations for changed blocks, then complete backup catalog is generated, but substantial computing resources and processor time are consumed
Solution Approach 1:
The system performs file system traversal and determines file associations in advance during the snapshot creation process, before the actual backup transfer. This preliminary action organizes the data structure so that when blocks are transferred to backup storage, the file associations are already established, avoiding repeated traversal operations.
Solution Approach 2:
The storage appliance performs the file system traversal and file association determination using its own computing resources rather than relying on the virtual machine or host system. This self-service approach consolidates the computational workload in one location and enables parallel processing capabilities.
3Loss of time
If snapshot backup time is reduced, then backup frequency can be increased, but computational resources must be optimized
Solution Approach 1:
The backup process is segmented into parallel operations: data block replication, file system traversal, snapshot creation, and backup transfer all occur concurrently rather than sequentially. This segmentation of the backup workflow into parallel tasks significantly reduces the total time required for snapshot backups.
Solution Approach 2:
The system maintains continuous backup operations by implementing incremental snapshots that build upon previous snapshots. Changed blocks are identified and transferred continuously without stopping the production workload, ensuring uninterrupted backup progress and enabling frequent snapshots.
Data Source
AI summary
Systems and methods are disclosed for efficiently performing backups using snapshots. A splitter receives write operations and replicates the write operations to a data protection appliance and to a primary storage. The data embodied in the replicated writes is stored in a high-speed storage in a snapshot over a time increment. The snapshot represents data blocks that have changed since a previous time increment. A list of changed blocks is kept in high-speed storage that corresponds to the snapshot blocks. The snapshot blocks are associated with files and file system structure by (a) traversing a file system structure (FSS) for a file system or (b) by traversing the list of changed blocks and looking up changed blocks that are in inode format in the FSS, an inode tree structure, or directly. The snapshot, list of changed blocks, and list of associated changed files are transferred from high-speed storage to backup storage.