Host Path Selection via Address Range Distribution Mapping
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In software-defined storage systems, selecting optimal paths for data access using advanced protocols like NVMe/oF or NVMe/TCP is challenging, often leading to non-optimal performance due to excessive latency and reduced throughput caused by suboptimal path selection.
Innovation Solution
The implementation of a technique that utilizes address range distribution information to select paths for IO operations, allowing host devices to directly access storage nodes that locally store the targeted data, thereby reducing the need for additional network hops and optimizing path selection.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If proprietary protocols are used to enable seamless access to data in different storage nodes, then data accessibility is improved, but path selection performance deteriorates due to non-optimal paths being selected
Solution Approach 1:
The host device performs preliminary actions by obtaining address range distribution information from storage nodes before actual data access operations. This information is cached and used to pre-determine optimal paths, avoiding the need for real-time path selection decisions during data access and thus resolving the contradiction between ease of access and path selection performance.
Solution Approach 2:
The system implements feedback mechanisms where the host device receives address range distribution information from storage nodes, processes this feedback to generate path selection mappings, and uses these mappings to direct IO operations. This closed-loop feedback ensures optimal path selection while maintaining seamless data access through proprietary protocols.
2Adaptability or versatility
If data is distributed across multiple storage nodes, then storage capacity and flexibility are improved, but access latency increases due to additional network hops
Solution Approach 1:
The host device performs preliminary path determination by obtaining address range distribution information before data access operations. This pre-computed path mapping allows direct routing to the correct storage node without unnecessary network hops, thereby reducing access latency while maintaining the benefits of distributed storage capacity.
Solution Approach 2:
The system implements local quality by directing each IO operation to the specific storage node that locally stores the requested data, rather than using a centralized access pattern. This creates locally optimized access paths for each data segment, reducing network hops and latency while preserving the distributed capacity advantages.
3Device complexity
If path selection does not utilize address range distribution information, then system complexity is reduced, but throughput is reduced due to suboptimal routing
Solution Approach 1:
The host device performs preliminary computation to obtain and process address range distribution information, creating a path selection mapping that simplifies subsequent IO operations. By pre-computing the optimal paths, the system avoids complex real-time path selection algorithms during actual data access, thereby maintaining simplicity while improving throughput.
Data Source
AI summary
An apparatus includes at least one processing device configured to identify a logical storage volume stored across multiple storage nodes of a distributed storage system, to obtain address range distribution information for the logical storage volume from at least a subset of the storage nodes, the address range distribution information indicating, for each of a plurality of distinct address ranges of the logical storage volume, which of the storage nodes locally stores data for that address range, and to generate, from the obtained address range distribution information, a mapping of the distinct address ranges to particular ones of the storage nodes, wherein the mapping is utilized to select paths for delivery of input-output operations to the storage nodes. Obtaining address range distribution information for the logical storage volume may comprise, for example, sending log page commands to each of the storage nodes to obtain asymmetric range access and/or template information.