Variable Length Record Protection via Block Storage Metadata
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current methods for data protection in block storage systems, such as Oracle's HARD architecture and T10 End-to-End Data Protection, are inadequate for variable length records like CKD and ECKD formats, as they fail to provide comprehensive metadata protection and are limited in scope, particularly lacking protection against 'stale data' and offering insufficient metadata space.
Innovation Solution
The implementation of high performance block storage metadata that generates an emulated record with a sequence of extended fixed-length blocks, including a checksum that covers all data blocks and footers, ensuring end-to-end data integrity by checking the checksum during transit, writing, and reading, and utilizing a Host Bus Adapter or storage controller to verify record integrity.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If conventional data protection methods (HARD architecture, T10 End-to-End Data Protection) are used, then data protection is provided for fixed-length blocks, but they fail to provide comprehensive protection for variable length records and offer insufficient metadata space
Solution Approach 1:
The patent segments the data protection mechanism by introducing record-level metadata structures that can handle variable length records differently from fixed-length blocks. Each record gets its own metadata entry with specific protection characteristics, allowing tailored protection for CKD/ECKD formats while maintaining compatibility with fixed-block operations.
Solution Approach 2:
The patent adds a new dimension to data protection by implementing end-to-end protection that spans from host to storage subsystem, introducing checksum fields and metadata structures at multiple levels (record level, block level, and track level) to create a multi-dimensional protection architecture.
2Reliability
If metadata space is increased to provide comprehensive protection, then data integrity is improved, but system complexity increases
Solution Approach 1:
The metadata is segmented into distinct functional components: checksum fields for error detection, length fields for record boundary identification, and status fields for protection state tracking. This segmentation allows the system to manage complex protection requirements through modular, manageable metadata elements.
Solution Approach 2:
The patent introduces metadata as an intermediary layer between the variable length records and the fixed-block storage system. This metadata acts as a mediator that translates between the two formats, providing protection information without requiring fundamental changes to either the record structure or storage mechanism.
3Reliability
If checksum verification is performed during transit, writing, and reading, then data corruption is prevented, but processing time increases
Solution Approach 1:
The checksum verification is performed preliminarily at multiple stages: during data transit before writing to storage, during the writing process itself, and during read operations. By distributing verification across these preliminary and intermediate stages, the system prevents corruption propagation while managing processing time through parallel verification opportunities.
Solution Approach 2:
The patent implements feedback mechanisms where checksum verification results trigger appropriate actions: successful verification allows data progression, while failed verification initiates error correction or retransmission protocols. This feedback loop ensures data integrity while optimizing processing time by avoiding unnecessary reprocessing of valid data.
Data Source
AI summary
An enhanced mechanism for providing data protection for variable length records utilizes high performance block storage metadata. In an embodiment, an emulated record that emulates a variable length record, such as a Count-Key-Data (CKD) record or an Extended-Count-Key-Data (ECKD) record, is generated by a Host Bus Adapter (HBA) of a mainframe system. The emulated record comprises a sequence of extended fixed-length blocks, each of which includes a data block and a footer. A confluence of the footers defines a high performance block storage metadata unit associated with the emulated record and includes a checksum that covers all data blocks and all footers of the entire emulated record. In one embodiment, the checksum is checked during transit of the emulated record between a HBA and a storage subsystem (e.g., by the HBA when the emulated record is received from the storage subsystem, and/or by a switch in the data transfer path), during a hardening step when writing the emulated record to a disk, and/or during a verification step when reading the emulated record from the disk.