RDMA Connection Management in Distributed Storage
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The existing RDMA technology in distributed storage systems is limited by the number of quantization parameters (QPs), which restricts scalability and performance due to excessive connections between server nodes.
Innovation Solution
The proposed communication method establishes two network stacks and divides connection relationships between clients and server nodes, and between processes of server nodes, into two categories based on these stacks. This approach reduces the number of connections related to QPs, thereby improving system performance and scalability.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If the RDMA technology is applied to distributed storage systems with multi-process concurrency and complete cross-connections between server nodes, then communication capability between nodes is improved, but the number of QPs reaches the upper limit easily, limiting system scalability
Solution Approach 1:
The patent segments the connection management by introducing a connection manager component that divides the system into client processes, server processes, and a dedicated connection manager. This segmentation allows the connection manager to handle QP allocation and connection establishment centrally, reducing the burden on individual server nodes and enabling scalable system growth without hitting QP limits on each node.
Solution Approach 2:
The connection manager acts as an intermediary between client processes and server processes. It establishes connections on behalf of clients to server nodes, manages QP resources centrally, and coordinates communication between nodes. This intermediary approach reduces the number of direct QP connections needed between all node pairs, as the connection manager handles connection multiplexing and resource allocation efficiently.
2Device complexity
If a network proxy layer is added between the client and each server node to reduce the number of QPs, then the number of connections is reduced, but the IO stack length increases, consuming the benefits of RDMA technology
Solution Approach 1:
The patent implements local quality by allowing different parts of the system to use different communication mechanisms. Direct RDMA connections are maintained between client and server processes where needed, while the connection manager uses appropriate protocols for connection establishment and management. This selective approach preserves RDMA performance benefits for data transmission while using standardized protocols only for connection control plane operations.
Solution Approach 2:
The connection manager serves as an intermediary that handles connection establishment, QP allocation, and coordination without interfering with the high-performance RDMA data path. It manages the control plane operations separately from the data plane, allowing RDMA technology to maintain its performance benefits for actual data transmission while the intermediary handles connection management overhead.
Data Source
AI summary
Disclosed are a communication method and related components. In this solution, two network stacks are firstly established, and all connection relationships between a client and each server node and between processes of any two server nodes are then divided, so as to obtain two kinds of connection relationships, wherein the two kinds of connection relationships are respectively constructed on the basis of the two network stacks, and the number of one of the two kinds of connection relationships is related to the number of QPs.


