Storage Processor PI Comparison for End-to-End Data Integrity
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing data integrity protection mechanisms are complex and require additional resources, often not supported by host hardware and software, making it difficult to ensure data integrity during transmission between host devices and storage systems, especially in scenarios like high-performance computing where errors due to cosmic radiation or software bugs can occur.
Innovation Solution
Implementing a method where data is transmitted twice between the host device and storage device, with protection information (such as checksums or signatures) calculated for each transmission, allowing for error detection and handling, and utilizing a storage processor to compare these values to ensure data integrity without requiring significant changes to the host system.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If data integrity protection mechanisms are implemented using additional metadata and protocols, then data integrity is improved, but device complexity and overhead increase
Solution Approach 1:
The patent introduces a storage processor as an intermediary component that handles data integrity protection functions. The storage processor receives data from the host, calculates protection information (such as checksums or cryptographic hashes), and manages error detection and correction. This intermediary approach offloads the complexity from the host driver to the storage processor, allowing the host to interact with a simplified interface while the storage processor implements the complex integrity protection mechanisms.
Solution Approach 2:
The storage system implements self-service by automatically calculating protection information, detecting errors, and handling data integrity issues without requiring host intervention. The storage processor autonomously monitors data transmission, validates integrity using stored protection information, and triggers error-handling procedures when discrepancies are detected, eliminating the need for complex host drivers to manage these functions.
2Reliability
If data integrity protection mechanisms are implemented, then data integrity is improved, but additional memory and processing resources are required
Solution Approach 1:
The patent implements protection information calculation selectively based on data integrity requirements. Protection information (such as checksums or cryptographic hashes) is calculated and stored for data that requires integrity protection, while potentially skipping or using simplified protection for less critical data. This partial action approach allocates memory and processing resources only where needed, rather than uniformly applying complex protection mechanisms to all data, thereby reducing overall resource consumption while maintaining integrity for critical operations.
3Reliability
If data is transmitted twice with protection information calculation, then error detection capability is improved, but transmission time increases
Solution Approach 1:
The patent calculates protection information (such as checksums or cryptographic hashes) in advance during the data writing phase, before actual data transmission occurs. The storage processor computes the protection information and stores it alongside the data. When data is subsequently read or transmitted, the stored protection information is used for rapid error detection without requiring recalculation, thereby reducing transmission time while maintaining strong error detection capability.
Data Source
AI summary
The integrity of transmitted data can be protected by causing that data to be transmitted twice, and calculating protection information (PI) for the data from each transmission. The PI can include information such as a checksum or signature that should have the same value if the data from each transmission is the same. If the PI values are not the same, an error handling procedure can be activated, such as may retry the transmission. For write operations, the data can be transmitted twice from a source to a storage destination, while for read operations, the data can be transmitted to a recipient then sent back from the recipient to the storage device, with PI calculated for each transmission. A component such as a storage processor can perform at least this comparison step. Such approaches can also be used for network transmission or high performance computing.