Block CRC Hashing for RAID Data Integrity Verification
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current data integrity verification methods in storage devices, such as RAID systems, are inefficient due to high CPU cycle consumption and processing requirements, particularly during CRC computations and data scrubbing, which can slow down data transfer rates and impact system performance.
Innovation Solution
A method that packs error detecting codes into a smaller error detecting code pack, using a 32-bit polynomial hash operation to generate a hash word for data integrity verification, allowing verification without reading the data segment, and comparing this hash word to stored parity information to ensure data integrity across multiple disk drives.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If traditional CRC computation and verification methods are used for data integrity checking, then data integrity can be verified, but CPU cycle consumption increases significantly (consuming 60% or higher of total CPU cycles)
Solution Approach 1:
The patent segments the data verification process into two independent parts: (1) CRC computation performed during data write operations, and (2) CRC verification performed during data read operations. By pre-computing and storing CRC values with the data, the verification process no longer needs to re-compute CRC for every read operation, thus reducing CPU cycle consumption during verification while maintaining data integrity checking capability
Solution Approach 2:
The patent performs preliminary CRC computation during the data write operation and stores the CRC value together with the data. This preliminary action ensures that when data is read later, the verification process only needs to retrieve and compare the pre-computed CRC value rather than performing complex CRC computation again, thereby improving CPU efficiency during verification
2Reliability
If data scrubbing is performed on a sector by sector basis with full data transfer to global memory, then errors can be detected and corrected, but bandwidth requirements and processing time increase
Solution Approach 1:
The patent extracts only the essential verification information (CRC values) from the data blocks and performs verification operations using only these extracted CRC values rather than transferring entire data sectors to global memory. This extraction approach maintains error detection capability while significantly reducing the amount of data that needs to be processed and transferred during scrubbing operations
Solution Approach 2:
The patent uses pre-computed CRC copies that are stored with the data as surrogate verification objects. Instead of copying and verifying the entire data during scrubbing operations, the system copies and verifies only the compact CRC representations, which maintains error detection accuracy while reducing processing time and bandwidth requirements
3Reliability
If full data segments are read and written into global memory for verification, then complete data integrity can be checked, but bandwidth requirements and processing overhead increase
Solution Approach 1:
The patent extracts and utilizes only the pre-computed CRC values from the data blocks for verification purposes, rather than reading and processing the entire data segments. This extraction method maintains complete data integrity verification capability while significantly reducing bandwidth consumption by operating on compact verification representations instead of full data
Data Source
AI summary
A block CRC based fast data hash provides efficient data integrity verification functions. A hash word is generated from block CRCs that are stored along with data blocks in a hard drive for each data and/or parity track of a storage system, such as a RAID array. Each storage system member writes the hash word into a global memory. Thereafter, a director verifies data integrity using all member's hash words with one or more XOR operations. Use of the hash words for data integrity verification saves system bandwidth and CPU processing resources.