Replication Regulation via Data Chunk Deduplication
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current backup and recovery systems face inefficiencies in data replication and management, particularly over low-bandwidth connections, where existing technologies struggle to effectively reduce data transfer volumes and optimize replication operations.
Innovation Solution
The system employs an object-based approach with deduplication operations, where data is partitioned into chunks, and cryptographic hashing is used to identify duplicates, allowing only references to stored chunks to be transmitted, thereby reducing data redundancy and optimizing replication across different storage appliances.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If traditional backup systems transmit complete backup data over the network, then data replication is achieved, but data transfer volume is excessively large especially over low-bandwidth connections
Solution Approach 1:
The patent segments backup data into individual chunks or blocks, allowing the system to identify and transmit only unique chunks rather than complete duplicate datasets. This segmentation enables granular comparison and selective replication, dramatically reducing network bandwidth consumption while ensuring data integrity.
Solution Approach 2:
The patent uses cryptographic hashing to create digital fingerprints (copies) of data chunks without transmitting the actual data. By comparing these hash copies between source and destination, the system can identify duplicates and transmit only necessary data, significantly reducing transfer volume while maintaining replication reliability.
2Speed
If backup data is replicated without deduplication, then replication speed is maintained, but network bandwidth is wasted transmitting redundant data
Solution Approach 1:
The patent performs preliminary cryptographic hashing of data chunks at the source before transmission. This preliminary action allows the destination to quickly identify existing data through hash comparison, eliminating the need to transmit redundant chunks and optimizing network bandwidth utilization while maintaining replication speed.
Solution Approach 2:
The patent transforms the replication approach by changing the parameter being transmitted from complete data blocks to selective data differences identified through cryptographic hash comparison. This parameter change enables efficient bandwidth utilization while preserving replication integrity and speed.
3Quantity of substance
If deduplication operations are implemented, then data transfer volume is reduced, but processing complexity increases
Solution Approach 1:
The patent replaces complex mechanical data comparison operations with cryptographic hashing algorithms. Instead of comparing actual data bytes between source and destination, the system computes and compares hash values, which are computationally efficient and dramatically simplify the deduplication process while maintaining accuracy.
Solution Approach 2:
The patent introduces cryptographic hash values as an intermediary between the source and destination data. These hash intermediaries serve as proxies for actual data comparison, enabling efficient duplicate detection without requiring complex direct data analysis, thus reducing processing complexity.
Data Source
AI summary
A technique includes determining at least one metric associated with a replication operation to replicate at least a portion of a first object stored on one of a client, a first storage appliance and a second storage appliance onto another one of the client, the first storage appliance and the second storage appliance. The technique further includes regulating the replication operation including selectively regulating at least one parameter of the replication operation based on the metric(s).