Checksum Management for Standard Storage Devices
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional block devices lack support for checksum calculation and storage, leading to data integrity issues, particularly in enterprise-grade disks that are expensive, and hybrid solutions face challenges with data coherence and performance degradation due to separate checksum storage.
Innovation Solution
A method that determines the trustworthiness of checksums, initializes unknown checksums, calculates and updates checksums during write requests, and compares checksums during read requests to ensure data integrity, using a checksum archive and bitmap to manage checksums efficiently, even in the absence of native checksum support.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If enterprise-grade disks with native checksum support are used, then data integrity is improved, but cost increases significantly
Solution Approach 1:
The patent creates a software-based copy of the checksum functionality that exists in enterprise-grade disks. By implementing checksum calculation and verification in software rather than relying on hardware-level support, the system replicates the data integrity protection of expensive enterprise disks using standard consumer-grade hardware, thereby eliminating the cost barrier while maintaining reliability.
Solution Approach 2:
The patent replaces the mechanical/hardware checksum support mechanism with a software-based system. Instead of relying on physical disk features (mechanical system) to calculate and store checksums, the invention uses software processes to perform checksum operations, allowing standard disks to function with enterprise-level data integrity protection.
2Ease of manufacture
If checksums are stored on a separate device in hybrid solutions, then cost is reduced, but data coherence and reliability deteriorate due to false positives
Solution Approach 1:
The patent merges the data storage and checksum storage operations into a unified system. By storing checksums in the same address space as the data blocks and using consistent locking mechanisms for both operations, the system ensures that data and checksums are always coherent. This eliminates the false positive problem where separate storage locations could become desynchronized.
Solution Approach 2:
The patent implements a feedback mechanism through checksum verification on every read operation. When data is read, the system recalculates the checksum and compares it with the stored value, providing immediate feedback on data integrity. This continuous verification loop ensures that any coherence issues are detected and handled, maintaining reliability even in the hybrid architecture.
3Ease of manufacture
If checksums are stored on a separate device in hybrid solutions, then cost is reduced, but performance deteriorates due to constant disk head movement
Solution Approach 1:
The patent merges data and checksum storage into the same disk location, eliminating the need for separate seek operations. By storing checksums in the same address space as data blocks, the system can retrieve both data and its corresponding checksum in a single disk operation, thereby maintaining disk performance while enabling hybrid checksum storage.
Solution Approach 2:
The patent performs preliminary actions by pre-calculating and storing checksums alongside data during write operations. This allows the checksum to be immediately available when data is read, eliminating the need for separate checksum retrieval operations and thus preventing performance degradation from additional disk head movement.
4Reliability
If checksums are calculated and stored during write operations, then data integrity is improved, but system complexity increases
Solution Approach 1:
The patent makes the file system and storage driver multi-functional by having them perform both data management and checksum management. The same file system code that handles data allocation and retrieval also handles checksum calculation, storage, and verification. This universal approach avoids adding separate checksum management subsystems, thereby limiting the increase in system complexity while achieving comprehensive data integrity protection.
Data Source
AI summary
Systems and methods are disclosed herein for maintaining data integrity of data in a storage device. An exemplary method comprises determining whether checksums associated with data on the storage device are trusted or untrusted, responsive to determining that the checksums are trusted, retrieving the checksums from a checksum archive, otherwise initializing the checksums as unknown, when a received storage request is a write request, calculating a checksum of the data and updating the checksum in memory, when a received storage request is a read request and an in-memory checksum associated with the identified block is not unknown, calculating the checksum of the data in the identified block and comparing the checksum with an in-memory checksum associated with the identified block, determining that corruption has occurred when the checksums don't match and saving checksums associated with each of the uncorrupted data blocks of the storage device to an archive.