Volatile Memory Metadata Logging for Dual-Node Storage
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In dual-node data storage systems, existing implementations inefficiently use volatile memory by storing complete mirrored in-memory copies of metadata pages, leading to excessive memory consumption and potential performance degradation, especially when only one node primarily services I/O operations for a logical device.
Innovation Solution
Implement a technique where the primary node stores actual metadata changes in its volatile memory, while the non-primary node records references or pointers to persisted metadata changes, reducing memory usage without increasing latency significantly.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If complete mirrored in-memory copies of metadata pages are stored on both nodes, then data consistency and reliability are improved, but volatile memory consumption increases significantly
Solution Approach 1:
The patent applies selective copying by storing complete metadata pages only on the primary node while using persistent references (pointers) on the non-primary node. This reduces memory consumption while maintaining consistency through the reference mechanism that points to the actual metadata storage location.
Solution Approach 2:
The patent implements local quality by differentiating the storage approach between primary and non-primary nodes. The primary node stores actual metadata pages, while the non-primary node stores only persistent references. This asymmetric storage strategy optimizes memory usage based on the specific role and needs of each node.
2Speed
If complete metadata pages are stored on both nodes, then access speed is improved, but memory overhead and system complexity increase
Solution Approach 1:
The patent uses persistent references instead of full copies on the non-primary node. These references efficiently point to metadata storage locations without duplicating the actual data, reducing memory overhead while maintaining access efficiency through direct reference resolution.
Solution Approach 2:
The persistent reference acts as an intermediary between the non-primary node and the actual metadata storage. Instead of storing complete metadata, the system uses references as mediators that resolve to the actual data location, simplifying the storage architecture while maintaining access capabilities.
3Reliability
If mirrored metadata copies are maintained on both nodes, then fault tolerance is improved, but memory management overhead increases
Solution Approach 1:
The patent implements efficient copying by storing only persistent references on the non-primary node rather than full metadata copies. This reduces memory management overhead while maintaining fault tolerance through the reference mechanism that ensures metadata accessibility even if the primary node fails.
Solution Approach 2:
The patent discards the need for complete mirrored copies on the non-primary node, recovering memory resources. The persistent references maintain the necessary information to recover or access metadata if needed, achieving fault tolerance with reduced memory overhead.
Data Source
AI summary
In a dual node system, each node can store metadata updates in a volatile memory metadata log. Metadata pages can be used to access data stored on a volume. A node designated as preferred or affined for a volume can be expected to receive I/Os directed to the volume. A preferred node for a volume can record, in its volatile memory metadata log, the metadata changes for the metadata pages used to access data stored on the volume. A non-preferred or non-affined node for the volume can infrequently receive I/Os directed to the volume. A non-preferred node for a volume can record, in its volatile memory metadata log, pointers or references to the metadata changes for the metadata pages used to access data stored on the volume, where the pointers or references map to persisted copies of the meta changes as stored in a persisted metadata log.