Deduplication-Aware Storage Replication via Standard Protocols
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional data replication methods in information processing systems often require proprietary protocols for deduplication, limiting their applicability and efficiency, especially in scenarios where bandwidth conservation and standard protocol compatibility are crucial.
Innovation Solution
Implementing deduplication-aware replication functionality using standard storage command protocols, such as iSCSI or NVMeOF, that allow for content-based signature scanning and management without the need for proprietary messaging between source and target storage systems, enabling broader compatibility and efficient bandwidth use.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If proprietary protocols are used for deduplication-aware replication, then deduplication functionality can be achieved, but protocol compatibility and system versatility are limited
Solution Approach 1:
The patent implements deduplication-aware replication using standard storage command protocols (SCSI, iSCSI, Fibre Channel, NVMe) that are universally supported by storage systems. The replication functionality is achieved through existing standard commands rather than requiring proprietary protocol extensions, making the solution compatible with diverse storage systems from different vendors.
Solution Approach 2:
The target storage system autonomously performs deduplication by scanning incoming write commands for content-based signatures and comparing them against existing data. The system self-determines which data pages are duplicates and skips transferring redundant data without requiring complex coordination protocols with the source system.
2Loss of energy
If all data pages are transferred during replication, then data completeness is ensured, but network bandwidth is wasted on duplicate data
Solution Approach 1:
The target storage system performs preliminary scanning of incoming write commands to compute content-based signatures (hash values) before actual data transfer. By pre-identifying duplicate data pages through signature matching, the system avoids wasting bandwidth transferring redundant data while ensuring all unique data is captured for complete recovery.
Solution Approach 2:
The system implements a feedback mechanism where the target storage system monitors incoming data, identifies duplicates through signature comparison, and selectively requests or accepts only non-duplicate data pages. This feedback loop ensures data completeness is maintained while optimizing bandwidth utilization by eliminating redundant transfers.
3Productivity
If content-based signature scanning is implemented, then deduplication efficiency is improved, but processing overhead increases
Solution Approach 1:
The patent replaces complex mechanical coordination mechanisms with simple content-based signature scanning using cryptographic hash functions. Instead of requiring complex protocol handshakes or data comparison operations, the system uses lightweight hash computations to rapidly identify duplicate data pages, significantly reducing processing overhead while maintaining high replication speed.
Data Source
AI summary
An apparatus obtains a set of content-based signatures for respective data pages of at least a portion of at least one storage volume replicated from a source storage system to a target storage system, issues a write for the set of content-based signatures to the target storage system, and reads from a designated location in the target storage system information indicating for each of the content-based signatures whether or not the corresponding data page is present in the target storage system. For one or more data pages not present in the target storage system, the apparatus writes the one or more data pages to respective logical addresses of the replicated storage volume in the target storage system. The issuing of a write, the reading from the designated location, and the writing of the one or more data pages are illustratively performed utilizing respective commands of a standard storage command protocol.