Storage Controller CRC Reordering for Silent Data Corruption Detection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional storage systems fail to detect silent data corruption, which occurs when a system interruption during a write operation leaves partially written data sectors appearing correct but incorrect, leading to undetected errors.
Innovation Solution
The storage controller employs CRC reordering and ECC encoding/decoding logic to segment data into smaller portions, append CRC and ECC codes, and shift CRC codes among segments, ensuring that incomplete writes are flagged as errors during read operations.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If conventional storage systems use standard CRC checking, then data integrity is maintained for complete writes, but silent data corruption from incomplete writes cannot be detected
Solution Approach 1:
The patent divides the data payload into multiple segments and computes separate CRC codes for each segment. During read operations, the system reorders these CRC codes to detect whether all segments were successfully written. This segmentation approach enables detection of silent data corruption without requiring a completely new error detection paradigm.
Solution Approach 2:
The patent pre-computes and stores CRC codes for each data segment during the write operation, before the actual data is committed to storage. This preliminary computation of segment-specific CRC codes allows the system to later verify whether all segments were successfully written by checking if the retrieved CRC codes match the expected reordered sequence.
2Reliability
If the system implements atomic write operations to prevent partial writes, then data integrity is improved, but write performance and throughput deteriorate
Solution Approach 1:
The patent introduces CRC codes as an intermediary verification mechanism that operates independently of the write operation itself. The CRC computation and verification process acts as a mediator that provides atomicity guarantees without requiring blocking mechanisms or complex coordination protocols that would slow down write operations.
Solution Approach 2:
The patent replaces mechanical atomicity enforcement mechanisms (such as write barriers, fencing protocols, or lock-based systems) with a software-based CRC verification approach. This substitution allows the system to achieve atomic write semantics through error detection rather than through complex control mechanisms, thereby maintaining write performance.
3Productivity
If the system uses larger sector sizes for storage optimization, then storage efficiency and performance improve, but the risk of silent data corruption increases
Solution Approach 1:
The patent segments the larger data payload into multiple smaller logical segments, each with its own CRC code. This segmentation maintains the benefits of large sector sizes for storage efficiency while introducing fine-grained verification points that enable detection of partial write failures, thereby improving resilience without sacrificing performance.
Data Source
AI summary
One embodiment provides a storage controller. The storage controller includes host data segmentation logic to divide, in response to a write command from a host domain to write a data payload to a storage device, the data payload into a plurality of data segments; cyclic redundancy check (CRC) encode logic to generate a CRC code for each data segment; and CRC reordering encode logic to assign each CRC code to another data segment among the plurality of data segments.