Metadata Compression Offload for Emergency Vaulting
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Data centers face challenges in avoiding data loss during emergency shutdown due to the large, costly, and regulation-compliant lithium-ion batteries required for SAN and NAS nodes to copy volatile memory data to non-volatile storage.
Innovation Solution
Implementing power conservation logic with compression offload hardware that compresses and destages metadata and data from volatile memory to non-volatile storage using a standby power source, allowing for efficient data preservation during loss of line power, potentially using smaller batteries that do not trigger strict shipping regulations.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of energy
If compression is applied to metadata during emergency shutdown, then the amount of data requiring power for writing is reduced, but the device complexity increases due to compression hardware requirements
Solution Approach 1:
A dedicated compression offload hardware component acts as an intermediary between the processor and storage media, handling compression operations during emergency shutdown. This specialized component reduces the processing burden on the main processor while enabling efficient compression of metadata and data before power is completely lost.
Solution Approach 2:
The compression function is extracted from the main processor and implemented as a separate compression offload hardware module. This extraction allows compression operations to be performed independently during emergency shutdown without consuming processor resources, thereby reducing overall power consumption while maintaining data integrity.
2Reliability
If all metadata is copied to non-volatile storage during emergency shutdown, then data loss is prevented, but the time required for the operation increases
Solution Approach 1:
The system applies different quality treatments to different types of data: compressible metadata is compressed and prioritized for writing to non-volatile storage, while non-compressible data is written uncompressed. This local differentiation optimizes the use of limited power and time resources during emergency shutdown, ensuring critical metadata is preserved with higher reliability while managing overall operation time.
Solution Approach 2:
The system dynamically changes the compression parameter based on data characteristics, applying compression to metadata that can be compressed while leaving non-compressible data uncompressed. This parameter adjustment enables faster writing of critical information during emergency shutdown, reducing the overall time required while maintaining data integrity.
3Productivity
If compression offload hardware is used, then compression speed increases, but the device complexity and cost increase
Solution Approach 1:
Compression offload hardware is pre-configured and ready to operate during emergency shutdown events. The hardware is designed to activate automatically when power loss is detected, performing compression operations in advance of complete power failure. This preliminary action ensures that critical metadata is compressed and written to non-volatile storage before power is completely lost, improving compression speed without requiring complex real-time processing.
Data Source
AI summary
Metadata in volatile memory is selectively compressed and destaged to non-volatile storage in the event of an emergency shutdown due to loss of like power. Compression offload hardware that is normally used for data compression is used to compress the metadata, e.g. at line speed. The compressed metadata and any uncompressed metadata that was not selected for compression may be destaged to vault drives along with compressed and uncompressed data that is in the volatile memory. Compression during vaulting may decrease power consumption when operating under standby battery power.