Stripe Consistency Checking Using CRC Blocks for Erasure Coding
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current methods for determining stripe consistency in data storage systems consume significant network bandwidth, affecting overall system performance due to the need to converge all data blocks on a single operation node.
Innovation Solution
The method involves obtaining and determining whether n groups of cyclic redundancy check (CRC) data blocks meet a linear relationship, where the CRC data blocks are derived from EC transformations of the original data blocks, allowing for reduced bandwidth consumption and resource usage by assessing consistency based on smaller CRC data blocks rather than full data blocks.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If all data blocks in the stripe are converged on one operation node for consistency determination, then stripe consistency can be accurately determined, but network bandwidth consumption increases significantly
Solution Approach 1:
The patent extracts only the essential verification information (CRC data blocks) from the complete data blocks and transmits this extracted information to the operation node for consistency determination. This extraction principle resolves the contradiction by maintaining determination accuracy while minimizing network bandwidth consumption.
Solution Approach 2:
The patent introduces CRC (cyclic redundancy check) data blocks as an intermediary element between the original data blocks and the consistency determination process. These intermediary CRC blocks serve as sufficient statistics for verification, eliminating the need to transmit full data blocks while preserving determination accuracy.
2Reliability
If all data blocks are transmitted to the operation node for consistency checking, then complete verification is possible, but system performance deteriorates due to high resource consumption
Solution Approach 1:
The patent extracts only the verification-critical CRC portions from complete data blocks, transmitting only these extracted elements to the operation node. This extraction maintains verification completeness while reducing computational and network resources required, thereby improving system performance.
Solution Approach 2:
The patent uses lightweight CRC data blocks instead of heavy original data blocks for verification purposes. These CRC blocks are computationally inexpensive to generate and transmit, serving as disposable verification carriers that maintain reliability while improving productivity.
3Loss of energy
If CRC data blocks are used instead of full data blocks for consistency determination, then network resource consumption is reduced, but calculation complexity increases
Solution Approach 1:
The patent changes the parameter representation from full data blocks to CRC checksum values, transforming the verification problem into polynomial arithmetic operations. This parameter transformation reduces data size and enables efficient modular arithmetic calculations, balancing resource consumption with computational complexity.
Data Source
AI summary
A method and apparatus for determining stripe consistency, including an operation node that obtains n groups of CRC data blocks corresponding to a target stripe, where the target stripe includes n first data blocks, the n first data blocks include k original data blocks and m EC data blocks of the k original data blocks, the n groups of CRC data blocks are respective CRC data blocks of n groups of second data blocks, one group of second data blocks includes m transformation results obtained by performing m times of EC transformation on one first data block, both k and m are positive integers, and n=m+k.