NVMe Storage RDMA Access Bypassing CPU Bottleneck
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In distributed storage systems, the performance of NVMe storage devices is not fully utilized due to the need for CPU involvement in processing access requests, which hinders write performance.
Innovation Solution
A method that allows clients to send remote direct memory access (RDMA) write requests directly to NVMe storage nodes, bypassing the CPU by querying and obtaining memory addresses from management servers or storage nodes, enabling direct access to NVMe storage devices.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If the CPU processes access requests in the distributed storage system, then the storage system can handle write requests, but the write performance is limited due to CPU involvement
Solution Approach 1:
The patent extracts the CPU from the write request processing path by enabling direct RDMA access between clients and NVMe storage devices. The client sends write requests directly to the storage device's memory space bypassing the CPU, while the CPU retains control for other operations. This separation resolves the contradiction by removing CPU involvement from the critical write path, improving write performance without eliminating CPU functionality entirely.
Solution Approach 2:
The patent introduces an RDMA gateway as an intermediary component that enables direct memory access between clients and NVMe storage devices. The gateway handles the translation and routing of write requests to the appropriate memory addresses in the storage device, allowing data to flow directly without CPU intervention. This intermediary mechanism resolves the technical contradiction by providing a dedicated path for high-performance writes while maintaining system control.
2Productivity
If the client sends write requests through the storage node CPU, then the requests can be processed, but the NVMe storage device performance cannot be fully utilized
Solution Approach 1:
The patent implements a skipping mechanism where write requests bypass the CPU processing stage entirely and are directed straight to the NVMe storage device's memory space via RDMA. By skipping the CPU intervention step, the system eliminates the time loss associated with CPU processing while maintaining proper request handling. This allows NVMe storage devices to operate at their full performance potential without being bottlenecked by CPU speed.
3Productivity
If the client directly accesses the NVMe storage device without CPU involvement, then write performance improves, but the system needs a mechanism to manage memory address mapping
Solution Approach 1:
The patent implements a self-service mechanism where the NVMe storage device automatically manages its own memory address mapping and queue management. The storage device maintains its own namespace and memory space organization, eliminating the need for external CPU intervention in address translation. This self-managing approach enables direct client access while keeping the system manageable, as the storage device handles its own addressing complexity internally.
Data Source
AI summary
In a solution for accessing a storage system, a client sends, based on an obtained start address that is of a queue of an NVMe storage device and to which an access request points and an obtained logical address that is of the NVMe storage device and to which the access request points, a remote direct memory access command to a storage node in which the NVMe storage device is located.