RDMA Connection Management in Distributed Storage

Resolve Bottlenecks,
Find Innovative Solutions
Generate Solutions

Solution Overview

Problem

The existing RDMA technology in distributed storage systems is limited by the number of quantization parameters (QPs), which restricts scalability and performance due to excessive connections between server nodes.

Innovation Solution

The proposed communication method establishes two network stacks and divides connection relationships between clients and server nodes, and between processes of server nodes, into two categories based on these stacks. This approach reduces the number of connections related to QPs, thereby improving system performance and scalability.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Adaptability or versatility

If the RDMA technology is applied to distributed storage systems with multi-process concurrency and complete cross-connections between server nodes, then communication capability between nodes is improved, but the number of QPs reaches the upper limit easily, limiting system scalability

Engineering Contradiction:
Improvecommunication capabilityVSAvoidnumber of QP connections
Core Design Contradiction:
Adaptability or versatilityVSDevice complexity

Solution Approach 1:

The patent segments the connection management by introducing a connection manager component that divides the system into client processes, server processes, and a dedicated connection manager. This segmentation allows the connection manager to handle QP allocation and connection establishment centrally, reducing the burden on individual server nodes and enabling scalable system growth without hitting QP limits on each node.

Inventive Principle:
Principle #1Segmentation

Solution Approach 2:

The connection manager acts as an intermediary between client processes and server processes. It establishes connections on behalf of clients to server nodes, manages QP resources centrally, and coordinates communication between nodes. This intermediary approach reduces the number of direct QP connections needed between all node pairs, as the connection manager handles connection multiplexing and resource allocation efficiently.

Inventive Principle:
Principle #24Intermediary (Mediator)

2Device complexity

If a network proxy layer is added between the client and each server node to reduce the number of QPs, then the number of connections is reduced, but the IO stack length increases, consuming the benefits of RDMA technology

Engineering Contradiction:
Improvenumber of connectionsVSAvoidcommunication efficiency
Core Design Contradiction:
Device complexityVSProductivity

Solution Approach 1:

The patent implements local quality by allowing different parts of the system to use different communication mechanisms. Direct RDMA connections are maintained between client and server processes where needed, while the connection manager uses appropriate protocols for connection establishment and management. This selective approach preserves RDMA performance benefits for data transmission while using standardized protocols only for connection control plane operations.

Inventive Principle:
Principle #3Local quality

Solution Approach 2:

The connection manager serves as an intermediary that handles connection establishment, QP allocation, and coordination without interfering with the high-performance RDMA data path. It manages the control plane operations separately from the data plane, allowing RDMA technology to maintain its performance benefits for actual data transmission while the intermediary handles connection management overhead.

Inventive Principle:
Principle #24Intermediary (Mediator)

Data Source

PatentUS12225084B2Communication method and related components
Publication Date: 2025.02.11 LANGCHAO ELECTRONIC INFORMATION IND CO LTD
  • US12225084B2 patent drawing
  • US12225084B2 patent drawing
  • US12225084B2 patent drawing

AI summary

Disclosed are a communication method and related components. In this solution, two network stacks are firstly established, and all connection relationships between a client and each server node and between processes of any two server nodes are then divided, so as to obtain two kinds of connection relationships, wherein the two kinds of connection relationships are respectively constructed on the basis of the two network stacks, and the number of one of the two kinds of connection relationships is related to the number of QPs.