Compressing Metadata Blocks to Eliminate Checksums in Flash Storage
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional data storage arrays consume excessive computer resources and cause wear on flash memory by updating and verifying checksums in block metadata (BMDs) in response to file system changes, which is inefficient and wears out flash memory.
Innovation Solution
Compressing metadata blocks before writing them to secondary storage, allowing verification during decompression to eliminate the need for checksums in BMDs, thereby reducing resource consumption and flash memory wear.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If checksums are stored in block metadata (BMDs) to verify indirect block (IB) contents, then data integrity is ensured, but computer resources (bus resources, cache space, locks) are excessively consumed
Solution Approach 1:
The patent extracts the checksum verification function from the traditional BMD structure and relocates it to the compression algorithm itself. The compression algorithm inherently detects data corruption without requiring separate checksum storage, thereby eliminating the resource overhead of maintaining and verifying checksums in BMDs while preserving data integrity
Solution Approach 2:
The compression algorithm is given a dual function: it both compresses the IB data to reduce storage space and simultaneously verifies data integrity through its built-in error detection capabilities. This multi-functionality eliminates the need for separate checksum mechanisms, reducing computer resource consumption while maintaining reliability
2Reliability
If checksums are stored and updated in BMDs, then data corruption can be detected, but flash memory wear increases due to additional write operations
Solution Approach 1:
The patent removes the separate checksum storage and update mechanism from the flash memory operations. By embedding error detection within the compression algorithm, the system no longer needs to perform additional write operations to update checksums in BMDs, thereby reducing flash memory wear while maintaining the ability to detect data corruption
Solution Approach 2:
The compression algorithm serves itself by incorporating built-in error detection capabilities that eliminate the need for separate checksum management. This self-service approach allows the system to detect data corruption without requiring additional flash memory write operations, thereby extending flash memory lifespan
3Reliability
If BMDs are read and updated with checksums for every file system change, then data integrity is verified, but operation time increases
Solution Approach 1:
The patent extracts the integrity verification step from the separate BMD read-update-verify cycle and integrates it into the compression process. This eliminates the need for separate BMD operations, reducing the time required for file system changes while maintaining data integrity verification through the compression algorithm's built-in error detection
Data Source
AI summary
A technique manages file system metadata and is performed in a data storage system that maintains a file system on behalf of a set of host computers. The technique involves modifying a first metadata block of the file system in response to a file system change, the first metadata block containing file system metadata. The technique further involves performing a compression operation to generate a compressed metadata block in volatile storage from the first metadata block, the compressed metadata block having a size that is smaller than a size of the first metadata block. The technique further involves writing the compressed metadata block from the volatile storage (volatile cache memory) to non-volatile storage (e.g., flash memory).