Host-Based Locality Log Pages for NVMeF Path Selection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In software-defined storage systems, it is challenging to perform optimal path selection using advanced storage access protocols like NVMeF or NVMe/TCP, often resulting in non-optimal paths that lead to performance issues such as excessive latency and reduced throughput due to the lack of locality information about data distribution across storage nodes.
Innovation Solution
The implementation of host-based locality determination techniques, where host devices obtain and utilize locality information from storage nodes to select paths for IO operations, ensuring that data is accessed from the nearest storage node, thereby optimizing path selection and reducing latency and increasing throughput.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If proprietary protocols are used to hide data distribution details, then system flexibility and ease of operation are improved, but path selection performance deteriorates due to lack of locality information
Solution Approach 1:
The patent introduces locality log pages as an intermediary information structure that bridges the gap between the proprietary protocol abstraction layer and the path selection mechanism. These log pages contain locality information about data distribution across storage nodes, enabling the host to make informed path selection decisions while still using the proprietary protocol to hide implementation details. The intermediary allows performance optimization without sacrificing ease of operation.
2Adaptability or versatility
If data is distributed across multiple storage nodes, then system capacity and adaptability are improved, but access latency increases due to non-optimal path selection
Solution Approach 1:
The patent implements preliminary action by pre-collecting and storing locality information in locality log pages before actual data access operations. The host obtains locality information indicating which storage nodes store which portions of logical storage volumes in advance. This preliminary knowledge enables the host to select optimal paths before initiating data transfer operations, thereby reducing access latency while maintaining the benefits of distributed storage for system capacity and adaptability.
3Ease of operation
If proprietary protocols hide physical storage locations, then security and ease of operation are improved, but throughput is reduced due to inability to perform optimal path selection
Solution Approach 1:
The patent applies segmentation by dividing the information architecture into distinct layers: the proprietary protocol layer that maintains abstraction and security, and the locality information layer that enables performance optimization. The locality log pages segment the metadata about data distribution from the actual data access operations, allowing the host to perform optimal path selection based on segmented locality information while the proprietary protocol continues to hide physical storage locations, thus maintaining both ease of operation and high throughput.
Data Source
AI summary
An apparatus includes at least one processing device configured to identify a logical storage volume stored across multiple storage nodes of a distributed storage system, to obtain from each of one or more of the storage nodes locality information for one or more different portions of the logical storage volume, the locality information for a given one of the portions indicating whether or not the given portion of the logical storage volume is local to a particular storage node, and to utilize the obtained locality information in performing path selection for one or more input-output operations targeting the logical storage volume. Obtaining from each of one or more of the storage nodes locality information for one or more different portions of the logical storage volume illustratively comprises, for a given such storage node, obtaining the locality information from one or more log pages maintained by the storage node.