Backup Replication Using Selective Copy-on-Write
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current backup systems face challenges in efficiently copying large data volumes while the operating system is actively writing to the source volume, leading to performance degradation and increased storage requirements due to the need for copy-on-write mode, which can result in resource bottlenecks and potential copy process failures.
Innovation Solution
The system implements a method to copy a source volume to a target volume by designating blocks, identifying written blocks, and entering copy-on-write mode only when necessary, reducing the time spent in copy-on-write mode and optimizing data transfer through predictive and lookahead optimizations, allowing for crash-consistent snapshots even as the operating system continues to write.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If copy-on-write mode is used to copy the source volume while the operating system is actively writing, then data consistency is maintained, but system performance degrades and storage requirements increase
Solution Approach 1:
The system enters copy-on-write mode selectively only when necessary (when blocks are being modified during the copy process), rather than maintaining it throughout the entire copy operation. This preliminary and conditional activation minimizes performance impact while ensuring data consistency when needed.
Solution Approach 2:
The system dynamically transitions between normal copy mode and copy-on-write mode based on real-time conditions. It monitors block modifications and adjusts the copying strategy accordingly, switching to copy-on-write only when block changes are detected, thereby optimizing both performance and reliability.
2Reliability
If copy-on-write mode is used to ensure crash-consistent snapshots, then reliability is improved, but the time spent in copy-on-write mode increases leading to resource bottlenecks
Solution Approach 1:
The system prepares by identifying blocks that need copying before initiating the copy process, and only enters copy-on-write mode when actual block modifications are detected during copying. This preliminary preparation reduces the duration of copy-on-write mode activation while ensuring crash consistency is maintained.
Solution Approach 2:
Instead of maintaining copy-on-write mode for the entire copy process, the system applies it partially and selectively only to the specific blocks that are being modified during the copy operation, minimizing the time loss while ensuring crash consistency for those critical blocks.
3Quantity of substance
If the system copies large data volumes while the operating system is actively writing, then backup completeness is improved, but resource bottlenecks and copy process failures increase
Solution Approach 1:
The system dynamically adjusts the copying strategy based on real-time monitoring of block modifications. It switches between normal copy and copy-on-write modes, and can pause or resume operations as needed, thereby handling large data volumes reliably without causing resource bottlenecks or failures.
Solution Approach 2:
The system continuously monitors the copying process and block modification status, using this feedback to adjust its behavior. When block changes are detected during copying, it activates copy-on-write mode; when copying progresses smoothly, it maintains normal operation, thereby ensuring reliability for large data volumes.
Data Source
AI summary
The Multiclient Backup Replication Apparatuses, Methods and Systems (“MBR”) transforms pairing request, replication data stream inputs via MBR components into pairing response, replication confirmation outputs. A replication notification for a snapshot of a backup volume at a source node is obtained. A source node named pipe for the snapshot is created. A priority for the snapshot is determined. When appropriate to send the snapshot to a replication target node, snapshot data is read from the source node named pipe and serialized into chunks Chunks associated with the snapshot and other snapshots are multiplexed into a replication data stream and sent to the replication target node via a persistent network connection. The replication data stream is received by the replication target node and chunks associated with the snapshot are deserialized. A replication target node named pipe for the snapshot is created and used to write snapshot data to a replication volume.