End-to-end datapath protection via pre-computed checksums
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional data validation schemes in storage systems are inefficient and computationally burdensome, leading to CPU-based delays that become bottlenecks as storage media speeds improve, especially since checksum calculations consume hundreds of CPU clock cycles and are often performed every time data is accessed.
Innovation Solution
Calculating a checksum or error-detection code (EDC) upon data ingress during a write request and validating it upon data egress during a read request, applying protection at the point of entry and validating at the point of exit, with optional additional validation during data transfers to backend storage, and applying similar validation to metadata.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If checksum validation is performed every time data is accessed, then data integrity is ensured, but CPU performance deteriorates due to hundreds of CPU clock cycles consumed per validation
Solution Approach 1:
The patent applies preliminary action by calculating and storing the checksum with the data during the write operation, before the data is later accessed. This pre-computation eliminates the need to recalculate checksums during read operations, thereby ensuring data integrity while avoiding CPU performance degradation during data access.
2Measurement precision
If CPU-based checksum calculation is used, then data validation accuracy is maintained, but system speed deteriorates due to CPU delays becoming bottlenecks
Solution Approach 1:
The system performs the computationally intensive checksum calculation in advance during data ingestion, storing the result with the data. When data is later accessed, the pre-computed checksum is simply retrieved and compared, maintaining validation accuracy while dramatically improving system speed by avoiding repeated CPU-intensive calculations.
3Reliability
If checksum is recalculated on every read, then error detection capability is maximized, but latency increases due to repeated computational overhead
Solution Approach 1:
The patent stores the checksum with the data during the write operation, making it immediately available for comparison during read operations. This eliminates the need to recalculate the checksum on every read, thereby maintaining full error detection capability while reducing validation latency by avoiding repeated computational overhead.
Data Source
AI summary
A technique of protecting data in a data storage system includes, in response to receiving a write request that specifies data to be written in the data storage system, calculating an error detection code (EDC) of the specified data. The technique further includes persisting the specified data and the EDC in persistent cache and, in response to receiving a read request for reading the specified data, validating the specified data using the EDC before returning the specified data in satisfaction of the read request.