Block-Based Backup Replication Using File-Level Compression
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Computing devices face challenges in efficiently managing and transferring large amounts of data to remote storage systems, particularly in efficiently compressing and replicating block-based backups across distributed systems.
Innovation Solution
A method involving the generation of a block-based backup by creating a template virtual volume, identifying file extents, and compressing the backup at a file level for efficient replication across multiple backup storage devices.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If block-based backup is replicated directly without compression, then replication speed is maintained, but data volume consumed is excessive
Solution Approach 1:
The backup data is segmented into individual file levels rather than treating the entire block-based backup as a single unit. This allows compression to be applied selectively at the file level, enabling efficient identification and compression of compressible files while maintaining the integrity of the overall backup structure.
Solution Approach 2:
The data format is transformed from uncompressed block-based backup to compressed file-level format. This parameter change in data representation enables significant reduction in data volume while preserving the deterministic nature of backups through the use of hard links for duplicate files.
2Quantity of substance
If compression is applied to block-based backup, then data volume is reduced, but processing time increases
Solution Approach 1:
The system performs preliminary actions by mounting the backup and identifying compressible files before actual compression occurs. This advance preparation allows the system to target only the necessary files for compression, reducing overall processing time while achieving effective data volume reduction.
Solution Approach 2:
Instead of compressing the entire backup, the system creates hard links (a form of copying) for duplicate files identified during the file level traversal. This copying mechanism preserves data integrity while significantly reducing the amount of actual compression processing required, thereby minimizing processing time overhead.
3Productivity
If file-level compression is used, then compression efficiency is improved, but system complexity increases
Solution Approach 1:
The patent introduces a file system as an intermediary layer between the block-based backup and the compression process. This file system mounts the backup, enables file-level traversal and identification, and manages the compression operations. The intermediary simplifies the overall process by providing a familiar file system interface while handling the complexity of block-based data internally.
Solution Approach 2:
The system performs self-service by automatically traversing the mounted backup, identifying compressible files, and applying compression without requiring external intervention. The deterministic backup mechanism automatically creates hard links for duplicate files, eliminating the need for manual deduplication processes and reducing operational complexity.
Data Source
AI summary
A method for managing backups includes mounting a block-based backup on a backup storage system to obtain a mounted backup, compressing the mounted backup to generate a compressed backup, and replicating the compressed backup to generate a plurality of compressed backups.