Data Replication via Signature Verification
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing data replication methods are complex and resource-intensive, often requiring significant processing power and carrying a risk of data loss due to deduplication, which can be catastrophic if data is shared across multiple sets.
Innovation Solution
A method involving calculating and transmitting signatures of data fragments between source and destination systems, where only unknown signatures trigger fragment transmission, optimizing differential replication and using VTL-type libraries for performance while minimizing data exchange volume.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If data deduplication is used to reduce storage volume, then storage efficiency is improved, but data loss risk increases
Solution Approach 1:
The patent uses cryptographic signatures (hashes) as copies of data identifiers to verify data availability without copying actual data. This allows the system to track which data blocks exist across replicas without storing duplicate data content, thus reducing storage volume while maintaining reliability through verification.
Solution Approach 2:
The system implements feedback mechanisms where signature verification results inform subsequent replication decisions. The destination system sends back information about which signatures are unknown, allowing the source system to selectively transmit only the necessary data blocks, optimizing both storage efficiency and reliability.
2Reliability
If synchronous replication is used to ensure data availability, then reliability is improved, but transmission time increases
Solution Approach 1:
The patent applies partial action by transmitting only the necessary portion of data - specifically, only data blocks whose signatures are unknown at the destination. This selective transmission maintains data availability reliability while minimizing transmission time by avoiding redundant data transfer.
Solution Approach 2:
The system performs preliminary signature calculation and transmission before actual data transfer. By pre-computing and sending signatures, the system enables the destination to quickly determine what data is missing, thereby reducing the overall replication time while ensuring data availability.
3Productivity
If asynchronous replication is used to reduce transmission time, then productivity is improved, but data availability decreases
Solution Approach 1:
The asynchronous replication system incorporates feedback through signature verification and unknown signature reporting. This feedback mechanism ensures that despite asynchronous operation, the destination system eventually receives all necessary data blocks, maintaining data availability while benefiting from reduced transmission time.
Solution Approach 2:
The system uses partial action by sending only unknown data blocks based on signature comparison. This approach maintains high replication speed by avoiding transmission of already-present data while ensuring complete data availability through selective retransmission of missing blocks.
4Reliability
If all data is transmitted to ensure complete replication, then reliability is improved, but data exchange volume increases
Solution Approach 1:
The patent extracts only the essential identification element - the cryptographic signature - from the complete data set. By transmitting signatures separately and using them to identify which data blocks need replication, the system ensures complete replication reliability while dramatically reducing data exchange volume by transmitting only unknown blocks.
Solution Approach 2:
The system applies partial action by transmitting only the necessary subset of data - specifically, data blocks with unknown signatures at the destination. This selective transmission maintains replication completeness while minimizing data exchange volume by excluding already-present and redundant data blocks.
Data Source
Figure 1~3
Figure 2a~6
Figure 2b~5
AI summary
The invention relates, in particular, to the replication of data between a source system and a destination system. After having calculated a signature for a fragment of data to be replicated, the source system transmits same to the destination system. The latter determines whether or not said signature is known, i.e. whether or not a fragment is associated with said signature. If so, the data to be replicated is reconstructed (with respect to fragment in question). If not, a message indicating that the signature is unknown is transmitted to the source system, which then transmits the corresponding fragment to the destination system. The latter stores and reconstructs the data to be replicated (with respect to the fragment in question).