CAS Replication via Hash Key Inspection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Content addressable storage systems face challenges in reducing latency and bandwidth usage during replication due to the need for unnecessary data transfers and the limitations of hash-based content addressing, particularly when dealing with bandwidth constraints and latency across physically distant locations.
Innovation Solution
A CAS data storage system and method that utilizes a hash key storage table to determine whether data items are already present at the destination, allowing only unique data items to be transferred by comparing generated hash keys against stored keys, and maintaining an index to track replication counts, thereby eliminating the need for round-trip acknowledgments and reducing latency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If data items are transferred during replication without checking for existing copies, then replication consistency is maintained, but bandwidth is wasted and latency increases due to unnecessary data transfer
Solution Approach 1:
The system performs preliminary action by sending hash keys before data items and maintaining a hash key storage table at the destination to track which data items already exist. This allows the destination to determine in advance whether incoming data items are duplicates, eliminating unnecessary bandwidth consumption while preserving replication consistency through the index field tracking mechanism.
2Loss of energy
If hash keys are sent to destination for verification before data transfer, then bandwidth usage is reduced, but latency increases due to round-trip communication
Solution Approach 1:
The system eliminates round-trip latency by performing preliminary action at the source: hash keys are generated and stored in a hash key storage table locally before data transfer. The destination simply checks its local hash key storage table against received hash keys and sends acknowledgment, eliminating the need for round-trip verification communication while maintaining bandwidth efficiency.
Solution Approach 2:
The hash key storage table acts as an intermediary mechanism that stores hash keys locally at both source and destination. This intermediary structure enables the system to determine data uniqueness without direct round-trip communication between source and destination, reducing latency while maintaining the bandwidth savings of hash-based verification.
3Reliability
If all data items are transferred during replication, then destination has complete copies, but latency increases due to transferring duplicate data items
Solution Approach 1:
The system extracts only the essential identifying information (hash keys) from data items and stores them in hash key storage tables at both source and destination. When replicating, the system compares hash keys to identify and exclude duplicate data items, transferring only unique items. This extraction approach ensures replication completeness for unique items while eliminating latency-wasting transfers of duplicates.
Solution Approach 2:
The system performs preliminary action by pre-storing hash keys in hash key storage tables and pre-comparing them before data transfer. The index field in the hash key storage table tracks replication status, allowing the system to determine in advance which data items need transfer and which are duplicates, thereby eliminating unnecessary transfer latency while ensuring complete replication of unique items.
4Measurement precision
If hash key storage table is maintained at source, then unique data identification is improved, but device complexity increases
Solution Approach 1:
The system creates a simplified copy of the data identification mechanism by storing only hash keys (not full data items) in the hash key storage table. This copying approach maintains precise data uniqueness identification through hash-based content addressing while significantly reducing device complexity compared to storing or processing full data items. The hash key storage table structure with index fields provides efficient uniqueness detection without complex data management overhead.
Data Source
AI summary
A CAS data storage method and apparatus comprising: receiving input data including a succession of data items with corresponding logical addresses at a source CAS data storage space for storage therein and for replication at a destination CAS data storage space, generating a hash key for each data item at the source storage space, comparing respective hash keys with hash keys stored at a hash key storage table, to determine whether respective further data items are already present at the destination storage device; transferring respective data items to the destination storage space if no match is made to a hash key stored at the hash key storage table, but not transferring respective further data items if a match is made to a hash key stored at the hash key storage table, thereby transferring to the destination storage space only unique data items.