Storage Node Metadata Compression for Failover Latency
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing storage systems face challenges in reducing memory consumption and I/O latency after a failover event, as they require frequent reading of metadata from storage devices, leading to increased memory usage and latency.
Innovation Solution
Compressing metadata portions that are not frequently accessed or necessary for immediate I/O operations and storing them in memory, allowing for decompression on demand during failover, thereby reducing memory consumption and I/O latency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If metadata is stored in the storage device, then memory consumption is reduced, but I/O latency increases after failover due to frequent reading from storage device
Solution Approach 1:
The standby storage control program pre-loads metadata from the storage device into its memory before failover occurs. This preliminary action ensures that when failover happens, the metadata is already available in memory, avoiding the latency penalty of reading from storage device during the critical failover period.
Solution Approach 2:
Each storage node (active and standby) maintains local copies of metadata in its own memory. The standby node specifically prepares the metadata it will need for failover operations. This local storage approach eliminates the need for remote I/O operations during failover, reducing latency while managing memory usage efficiently.
2Loss of time
If metadata is stored in the memory, then I/O latency after failover is reduced, but memory consumption increases
Solution Approach 1:
The standby storage control program does not load all metadata into memory, but only the specific metadata portions that are necessary for failover operations. This partial action approach reduces memory consumption compared to loading complete metadata sets, while still achieving the goal of reducing I/O latency for critical failover functions.
Solution Approach 2:
The standby node performs preliminary loading of only the essential metadata portions into memory before failover. This selective pre-loading optimizes the balance between memory usage and I/O latency by preparing only what is strictly necessary for maintaining I/O operations during failover.
3Loss of time
If all metadata is loaded into memory in advance, then I/O latency is minimized, but memory consumption becomes excessive
Solution Approach 1:
The metadata is segmented into different portions based on their importance and usage patterns during failover. The standby storage control program loads only the critical segments into memory, while less critical segments remain in the storage device. This segmentation strategy minimizes memory consumption while ensuring that essential metadata is readily accessible during failover.
Solution Approach 2:
Instead of loading all metadata into memory, the system performs partial loading of only the essential metadata portions required for failover operations. This partial action approach achieves acceptable I/O latency performance while avoiding the excessive memory consumption that would result from loading the complete metadata set.
Data Source
AI summary
Two or more nodes respectively provided with two or more storage control programs constituting each redundantization group maintain redundantization of metadata at the two or more nodes. When a node failure occurs, a failover from the corresponding active storage control program to a standby storage control program is performed. As regarding at least one standby storage control program, a node with the standby storage control program arranged therein compresses a target metadata portion including a metadata portion capable of being accessed after the failover, of metadata existing in the node as regarding the corresponding redundantization group, and stores the same in a memory of the node.