RDMA Completion Queue Event Segmentation

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Existing RDMA systems are inefficient as applications are awakened by both send and receive completion notifications, even if they only need notifications for one type of event, leading to unnecessary resource utilization and latency.

Innovation Solution

Implementing completion queues that allow applications to selectively wait for RDMA send or receive events by using send and receive counters associated with the RDMA queue pair, enabling the interrupt handler to awaken only relevant applications based on counter values.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Reliability

If applications are awakened by both send and receive completion notifications, then all possible RDMA events are detected, but resource utilization increases and latency increases due to unnecessary wake-ups

Engineering Contradiction:

ImproveRDMA event detection completenessVSAvoidresource utilization

Core Design Contradiction:

ReliabilityVSProductivity

Solution Approach 1:

The completion queue notification mechanism is segmented into separate send completion notifications and receive completion notifications. The interrupt handler evaluates counter values to determine which type of completion occurred, and selectively awakens applications based on the specific completion type, rather than awakening all applications for all completion types.

Inventive Principle:

Principle #1Segmentation

2Reliability

If applications are awakened by both send and receive completion notifications, then all possible RDMA events are detected, but latency increases due to unnecessary processing

Engineering Contradiction:

ImproveRDMA event detection completenessVSAvoidapplication latency

Core Design Contradiction:

ReliabilityVSLoss of time

Solution Approach 1:

The notification system is segmented to distinguish between send completions and receive completions. The interrupt handler uses counter evaluation to segment the awakening process, only waking applications that are relevant to the specific completion type, thereby reducing unnecessary processing time and latency.

Inventive Principle:

Principle #1Segmentation

3Device complexity

If a single completion queue handles both send and receive completions, then device complexity is reduced, but the ability to selectively notify applications is lost

Engineering Contradiction:

Improvecompletion queue structureVSAvoidselective notification capability

Core Design Contradiction:

Device complexityVSAdaptability or versatility

Solution Approach 1:

Counters are introduced as intermediary elements between the completion queue and application notification. The send counter and receive counter act as mediators that the interrupt handler evaluates to determine which applications should be awakened, enabling selective notification while maintaining a unified completion queue structure.

Inventive Principle:

Principle #24Intermediary (Mediator)

Data Source

PatentUS9842083B2Using completion queues for RDMA event detection

Publication Date: 2017.12.12 RED HAT ISRAEL

AI summary

Systems and methods for using completion queues for Remote Direct Memory Access (RDMA) event detection. An example method may comprise: receiving a request to create a queue pair for processing Remote Direct Memory Access (RDMA) requests using an RDMA-enabled network interface controller (RNIC), the queue pair comprising a send queue and a receive queue; associating the queue pair with a completion queue associated with the RNIC, the completion queue employed to store a plurality of completion queue elements associated with completed work requests; receiving a notification of an interrupt associated with the RNIC; and responsive to determining that at least one of a number of send queues associated with the completion queue or a number of receive queues associated with the completion queue exceeds zero, identifying at least one of: a first application registered to be notified of RDMA send events or a second application registered to be notified of RDMA receive events.