Block Span Digest Comparison for Storage Replication
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The replication of large block storage resources over slow or bandwidth-constrained communications links is inefficient, as traditional full copy methods are not feasible when the source storage resource is under high demand.
Innovation Solution
A method that uses digest calculations to identify and copy only the differing blocks between two block storage resources, dividing spans into smaller pairs and performing comparisons to minimize the number of blocks copied, leveraging the SCSI Architecture Model-2 Specification and iSCSI protocol for efficient data transfer.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If full copy method is used to replicate block storage resources, then complete data replication is achieved, but replication efficiency deteriorates and bandwidth is excessively consumed
Solution Approach 1:
The patent divides the block storage resources into multiple block spans and processes them in parallel. Each block span is independently replicated, allowing the system to efficiently manage large volumes of data by breaking down the replication task into smaller, concurrent units of work, thereby improving overall replication efficiency without sacrificing completeness
Solution Approach 2:
The patent performs preliminary actions by calculating digests of source and destination block spans before actual data transfer. This preliminary digest calculation identifies which blocks need replication, allowing the system to avoid transferring unchanged data and significantly reduce bandwidth consumption while ensuring complete replication of only necessary data
2Reliability
If full copy method is used to replicate block storage resources, then complete data replication is achieved, but bandwidth consumption increases excessively
Solution Approach 1:
The patent performs preliminary digest calculations on both source and destination block spans to identify differences before initiating data transfer. This preliminary analysis enables the system to skip transferring blocks that are already identical, dramatically reducing bandwidth consumption while ensuring that all necessary data is replicated
Solution Approach 2:
The patent extracts and replicates only the differing blocks between source and destination storage resources. By using digest calculations to identify and extract only the necessary data portions, the system avoids transferring redundant data, thereby reducing bandwidth consumption while maintaining complete replication of all changed data
3Measurement precision
If block spans are divided into smaller pairs for digest calculation, then the precision of difference identification is improved, but the complexity of the replication process increases
Solution Approach 1:
The patent segments block storage resources into fixed-size block spans with defined boundaries. This segmentation provides a structured approach to digest calculation and comparison, improving difference identification accuracy while managing complexity through consistent, predetermined span sizes and parallel processing of multiple spans
Solution Approach 2:
The patent changes the parameter of block span size to an optimized fixed value that balances precision and complexity. By selecting an appropriate fixed span size, the system achieves sufficient difference identification accuracy without creating excessive complexity, and this parameter can be adjusted based on specific replication requirements
Data Source
AI summary
The portion of a source block storage resource to be replicated, and the corresponding portion of the block storage resource being written to, are each divided into a predefined number of equal-sized spans. A digest calculation is then generated for each span in a corresponding pair and compared. If the digests do not match, those spans are divided into still smaller spans, and digests are calculated and compare iteratively, to identify smaller areas of discrepancies and reduce the number of blocks that are actually written.