Erasure Code CRC Compaction for CPU Error Detection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing data encoding systems using erasure codes face scalability issues and high computing resource costs due to undetected errors during data storage, particularly with increasing data volumes, as they rely on methods that are not scalable and require recoding to detect errors, which is computationally expensive.
Innovation Solution
A system generates discrete cyclic redundancy checks (CRCs) by transposing data vectors into sub-vectors, producing a CRC matrix, mixing CRCs, and compacting them into a reduced CRC for verification, allowing for efficient detection of errors without recoding the data, using higher degree polynomials and existing CPU instructions for compatibility.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If existing systems use CRC operations over low order Galois fields to detect errors in erasure encoded data, then error detection capability is provided, but processing bottlenecks and high computing resource costs occur due to lack of scalability
Solution Approach 1:
The patent segments the data vector into multiple sub-vectors (columns) and computes discrete CRCs for each sub-vector independently. This segmentation allows parallel processing of CRC computations, improving scalability while maintaining error detection capability. The CRC matrix is then transposed and compacted efficiently using vectorized operations.
Solution Approach 2:
The patent replaces traditional sequential CRC computation mechanics with vectorized and matrix-based operations that leverage CPU instruction sets. By substituting iterative scalar operations with parallel vector operations, the system achieves higher processing throughput and scalability without sacrificing error detection reliability.
2Quantity of substance
If higher order Galois fields such as GF(256) are used for encoding and CRC generation, then data processing capacity increases, but the linearity property is lost making bitwise operations inapplicable
Solution Approach 1:
The patent changes the parameter of Galois field order from low (GF(2)) to high (GF(256)) to increase data processing capacity. Despite losing the linearity property, the patent maintains computational efficiency by using matrix transposition and compacting operations that work effectively over higher order fields, avoiding the need for complex bitwise operations.
3Reliability
If recoding is performed to detect errors during data storage, then error detection is achieved, but computational overhead and processing time increase significantly
Solution Approach 1:
The patent performs preliminary CRC computation on the data vector before storage operations. By computing discrete CRCs for each sub-vector and then compacting them into a reduced CRC in advance, the system enables rapid error detection without requiring time-consuming recoding operations later, thus reducing processing time loss.
Solution Approach 2:
The patent creates a compacted copy of the CRC information (reduced CRC) that can be quickly compared against future data without requiring the full recoding process. This copied CRC representation enables fast error detection by avoiding repetitive complex computations.
Data Source
AI summary
Technology that detects computation errors is disclosed, in which a system may include one or more processors and storage logic. The storage logic may be executed by the one or more processors to perform operations comprising: receiving a data vector, the data vector including a plurality of ordered blocks; transposing the data vector into a set of sub vectors, each of the sub vectors including a corresponding data element from each of the ordered blocks; generating a set of discrete cyclic redundancy checks (CRCs) based on the set of sub vectors; transposing the set of discrete CRCs into a set of mixed CRCs, each of the mixed CRCs including a CRC data element from each of the discrete CRCs; and compacting the set of mixed CRCs into a reduced CRC.