Stripe Consistency Verification Using CRC and EC Linear Checks
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods for determining stripe consistency in data storage systems consume significant network bandwidth due to the convergence of all data blocks on one operation node, affecting overall system performance.
Innovation Solution
Determine stripe consistency by using cyclic redundancy check (CRC) data blocks obtained through EC transformation, which are smaller than original data blocks, to reduce bandwidth consumption and resource usage.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If all data blocks in the stripe are converged on one operation node for consistency determination, then stripe consistency can be verified, but network bandwidth consumption increases significantly
Solution Approach 1:
The patent extracts only the essential verification information (CRC values and their linear relationship) from the complete data blocks, allowing consistency verification without transferring all data blocks across the network. This reduces network bandwidth consumption while maintaining verification reliability.
Solution Approach 2:
Instead of verifying consistency by converging all data blocks to one node, the patent inverts the approach by having each node verify its local data blocks' CRC values against a linear relationship, eliminating the need for data block convergence and reducing network bandwidth consumption.
2Reliability
If all data blocks are converged on one operation node, then complete stripe verification is achieved, but system performance deteriorates due to bandwidth constraints
Solution Approach 1:
The patent segments the verification process into independent local operations at each system node, where each node verifies its own data blocks' CRC values. This eliminates the bottleneck of converging all data blocks to one node, improving system performance while maintaining verification reliability.
Solution Approach 2:
The patent inverts the traditional verification approach by distributing the verification process across multiple nodes rather than concentrating it at one node. Each node independently verifies its local data, eliminating network bandwidth constraints and improving overall system performance.
Data Source
Figure 1~3
Figure 4~5
Figure 6~7
AI summary
Embodiments of this application provide a method for determining stripe consistency and an apparatus, and relate to the field of computer technologies, so that network resources consumed for determining stripe consistency can be reduced. The method includes: An operation node obtains n groups of CRC data blocks corresponding to a target stripe, where the target stripe includes n first data blocks, the n first data blocks include k original data blocks and m EC data blocks of the k original data blocks, the n groups of CRC data blocks are respective CRC data blocks of n groups of second data blocks, one group of second data blocks includes m transformation results obtained by performing m times of EC transformation on one first data block, both k and m are positive integers, and n=m+k. In addition, the operation node determines whether the n groups of CRC data blocks meet a linear relationship. If the n groups of CRC data blocks meet a linear relationship, the operation node determines that the target stripe has stripe consistency.