Intelligent Path Selection for Distributed Storage Nodes
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In software-defined storage systems, selecting optimal paths for data access using advanced protocols like NVMeF or NVMe/TCP can be challenging, leading to performance issues such as excessive latency and reduced throughput due to non-optimal path selection.
Innovation Solution
Implementing intelligent path selection techniques that involve a processing device obtaining information on the distribution of logical storage volumes across multiple storage nodes, using bitmaps to identify the correct storage node for each IO operation and selecting the most efficient path for data access, thereby ensuring direct delivery to the node storing the targeted data.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If proprietary protocols are used to enable seamless access to data in different storage nodes, then data access simplicity is improved, but path selection optimality deteriorates leading to excessive latency and reduced throughput
Solution Approach 1:
The patent segments the storage system into multiple storage nodes with distinct identities, where each node can be directly addressed. The data is divided into logical blocks that can be mapped to specific storage nodes, allowing the host to send I/O operations directly to the correct node without going through proprietary protocol abstractions, thereby reducing latency while maintaining simplicity.
Solution Approach 2:
The patent introduces a mapping mechanism that acts as an intermediary between the host and storage nodes. This mapping stores the relationship between logical blocks and physical storage nodes, enabling intelligent path selection without requiring proprietary protocols. The mapping allows direct addressing of storage nodes while maintaining the logical view for applications.
2Adaptability or versatility
If data is distributed across multiple storage nodes, then storage capacity and flexibility are improved, but path selection complexity increases leading to non-optimal data access paths
Solution Approach 1:
The patent performs preliminary actions by pre-establishing a mapping between logical blocks and physical storage nodes before data access operations. This mapping is maintained and updated in advance, allowing the host to quickly determine the correct storage node for any logical block without complex real-time path selection algorithms, thereby reducing path selection complexity while maintaining distribution flexibility.
3Loss of information
If proprietary protocols hide physical storage locations from applications, then data distribution transparency is improved, but access performance deteriorates due to non-optimal path selection
Solution Approach 1:
The patent extracts the path selection intelligence from the proprietary protocol layer and places it in the mapping mechanism. The mapping contains the information about which storage node stores which logical block, allowing the host to make informed path selection decisions. This extraction maintains transparency for applications while enabling optimal path selection to improve throughput.
Data Source
AI summary
An apparatus includes at least one processing device configured to obtain information characterizing which of a plurality of storage nodes of a distributed storage system stores respective ones of a plurality of different logical blocks of a logical storage volume of the distributed storage system. The at least one processing device is further configured, for each of a plurality of input-output operations directed to a particular one of the logical blocks of the logical storage volume, to identify, based at least in part on the obtained information, which of the plurality of storage nodes of the distributed storage system stores the particular logical block, to select a path to the identified storage node, and to send the input-output operation to the identified storage node over the selected path.