NVMe Client Kernel Extension for Multi-Node Storage Access
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing NVMe clients are unable to concurrently access multiple storage nodes in distributed storage systems, leading to performance bottlenecks and limitations in integrating distributed storage systems with operating system kernels.
Innovation Solution
The method extends a first access client using kernel extension technology to obtain a second access client with a kernel mode program that determines multiple first virtual sub-clients, allowing concurrent access to distributed storage nodes through these sub-clients, including metadata acquisition and address mapping algorithms.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If an NVMe client establishes a connection with one NVMe-oF Target service, then the connection is simple and stable, but the storage system cannot achieve high-concurrency access performance
Solution Approach 1:
The NVMe client is segmented into multiple I/O threads, with each thread capable of independently accessing different storage nodes. This segmentation enables concurrent access to multiple storage nodes simultaneously, transforming a single-threaded sequential access model into a multi-threaded parallel access model, thereby achieving high-concurrency storage access performance without requiring fundamental changes to the NVMe protocol stack
Solution Approach 2:
The NVMe client is enhanced with multi-functionality to support both traditional single-node access and new multi-node concurrent access modes. The client can dynamically select between different access patterns based on configuration, making it universally applicable to various storage scenarios while maintaining backward compatibility with existing single-node access architectures
2Productivity
If the NVMe client is extended to access multiple storage nodes, then concurrent access performance improves, but the client complexity increases
Solution Approach 1:
An address mapping algorithm acts as an intermediary between the I/O threads and storage nodes. This mapping layer abstracts the complexity of multi-node management, automatically translating logical I/O requests into physical storage node addresses. The mapping mechanism handles node selection, load balancing, and failover transparently, reducing the complexity burden on the client program while enabling concurrent access to multiple storage nodes
3Productivity
If customized client interfaces are used for distributed storage, then storage performance is optimized, but integration with existing OS kernel storage processes becomes difficult
Solution Approach 1:
The NVMe client is designed with universal compatibility to work seamlessly with existing OS kernel storage processes. By leveraging the standard NVMe interface and adding multi-node access capabilities through configurable parameters, the client maintains full adaptability with Linux kernel storage subsystems while delivering optimized distributed storage performance
Data Source
Figure 1
Figure 2~3
Figure 4~5
AI summary
Embodiments of the present disclosure provide a node access method and apparatus, and an electronic apparatus. The method includes: extending a first access client by using kernel extension technology to obtain a second access client, wherein the second access client includes a kernel mode program which is used for determining a plurality of first virtual sub-clients corresponding to the second access client; and accessing distributed storage nodes by means of the plurality of first virtual sub-clients. By means of the technical solution above, the problem in the related art that existing NVME clients cannot concurrently access a plurality of storage nodes in distributed storage is solved.