Incremental Block-Level Backup System for Distributed Storage
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Traditional backup methods, including replication, are inefficient for long-term archiving and prone to errors, such as replicating mistakenly deleted files, and require high bandwidth, making them unsuitable for scaling effectively.
Innovation Solution
An incremental block-level backup system using a hash tree structure to efficiently identify and transfer only changed data blocks, distributing data evenly across storage drives for balanced performance and error prevention, and replicating data with real-time synchronization to maintain data integrity during I/O operations.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If traditional backup methods are used, then data can be backed up, but backup efficiency is low and bandwidth requirements are high
Solution Approach 1:
The patent extracts only the changed data blocks from the source volume and transfers them to the backup system, rather than transferring entire volumes. This is achieved by comparing metadata hashes to identify modifications, thereby reducing bandwidth consumption while maintaining backup efficiency.
Solution Approach 2:
The patent segments data into individual blocks and processes them independently. By dividing the volume into blocks and tracking changes at the block level, the system can efficiently identify and transfer only modified portions, improving backup productivity while reducing network bandwidth requirements.
2Reliability
If replication is used for backup, then data can be synchronized, but errors such as replicating mistakenly deleted files occur
Solution Approach 1:
The patent implements a feedback mechanism using metadata hashes to track data block changes. By continuously comparing source and backup metadata, the system can identify actual data modifications and distinguish them from erroneous changes like mistaken deletions, thereby improving data integrity while reducing replication errors.
Solution Approach 2:
The patent performs preliminary actions by maintaining metadata hashes of data blocks before backup operations. This preliminary metadata comparison enables the system to identify changed blocks accurately, preventing the replication of erroneous changes and ensuring reliable data synchronization.
3Reliability
If full backup methods are used, then all data is protected, but the process is time-consuming and inefficient
Solution Approach 1:
The patent applies partial action by performing incremental backups that transfer only changed data blocks rather than complete volumes. This approach maintains comprehensive data protection coverage while significantly reducing backup time through selective data transfer based on metadata comparison.
4Quantity of substance
If data is distributed across multiple storage devices, then storage capacity increases, but system complexity increases
Solution Approach 1:
The patent introduces metadata hashes as an intermediary mechanism to manage distributed storage. By using hash-based metadata to track and identify data blocks across multiple storage devices, the system increases storage capacity while minimizing the complexity of managing distributed data through efficient block-level tracking.
Data Source
AI summary
Disclosed are systems, computer-readable mediums, and methods for receiving a start replication message to replicate a source volume to a replicated volume. A source system forwards I/O requests to the replica server. A data structure associated with the replicated volume is initialized. A write request is received from the source system. The write data is written to the replicated volume and the data structure is updated. Source metadata associated with the source volume is received. The source metadata is compared with prior metadata associated with a prior point-in-time image of the source volume to determine blocks of data that have changed since the prior point-in-time image of the source volume. A first block is determined to not be retrieved based upon the data structure. A second block is determined to be retrieved based upon the data structure. The second block is received and written to the replicated volume.