Storage Backup Using XOR Delta Compression for Lower Bandwidth
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing data backup methods consume significant bandwidth due to the need to copy modified data blocks in their entirety, even though only changes need to be transmitted for incremental backups.
Innovation Solution
The method involves determining the shift amount between data blocks using hash calculations, performing a cyclic shift to align data, and then using an exclusive-OR operation to identify identical data segments, which are then compressed and transmitted, reducing the amount of data sent during backups.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If incremental replication technology is used to compress and back up changed data, then data backup efficiency is improved, but bandwidth consumption remains high due to transmitting entire data blocks
Solution Approach 1:
The patent segments data blocks into smaller units by identifying identical segments through hash calculation and cyclic shift. Instead of transmitting entire data blocks, only the differences between segments are transmitted, effectively dividing the transmission task into manageable segments that reduce overall bandwidth consumption
Solution Approach 2:
The patent extracts only the essential difference information between source and backup data blocks. By using hash calculation to identify identical segments and cyclic shift to find alignment, the system extracts only the necessary change information for transmission, leaving out redundant identical data that would otherwise consume bandwidth
2Reliability
If data blocks are copied in their entirety for incremental backups, then data integrity is ensured, but the amount of data transmitted consumes significant bandwidth
Solution Approach 1:
The patent uses hash calculation to create fingerprints of data segments and compares these copies to identify identical portions. By copying and comparing hash values rather than entire data blocks, the system determines which segments are identical and can be skipped, reducing transmission volume while maintaining data integrity through verified segment matching
Solution Approach 2:
The patent changes the parameter of data representation by using cyclic shift to transform data blocks and find optimal alignment. This parameter change (cyclic shift amount) allows the system to identify identical segments even when there are positional differences, enabling more accurate identification of what needs to be transmitted
Data Source
Figure 1~2a
Figure 2b~4
Figure 5~8
AI summary
Embodiments of the present invention disclose a solution for data backup and recovery in a storage system. When a source device in the storage system backs up, to a backup-end device, a data block that is written after a snapshot Sn, the source device performs a logical operation such as an exclusive-NOR or exclusive-OR operation on the written data block and an original data block, which is recorded in the snapshot Sn, of the written data block, and then compresses a data block obtained after the logical operation, which improves a compression ratio of a data block, thereby reducing an amount of data that is sent to the backup-end device, and saving transmission bandwidth. The solution may be further applied to a scenario of data recovery in a storage system.