Shared Write Address Protocol RDMA Flow Control
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
RDMA network adapters typically limit the number of parallel RDMA READ operations due to resource constraints, making them slower and more resource-intensive than RDMA WRITE operations, which can throttle data access protocols.
Innovation Solution
A shared write address protocol (SWAP) is implemented over an RDMA connection, allowing each module to have only one outstanding RDMA READ operation for flow control, using a HELLO message for initial flow control information and optimizing data transmission through INLINE or WRITE messages based on buffer size and message pool availability.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If multiple parallel RDMA READ operations are used for data access, then data throughput may increase, but resource constraints on RDMA network adapters cause throttling and reduced efficiency
Solution Approach 1:
The patent inverts the traditional approach by making the writer (not the reader) initiate RDMA operations. The writer actively pushes data to the reader using RDMA WRITE operations, while the reader passively receives data and manages flow control. This inversion eliminates the need for multiple outstanding READ operations, resolving the resource constraint issue while maintaining high throughput.
Solution Approach 2:
The patent introduces a shared write address (SWA) mechanism as an intermediary between writer and reader. The SWA is a memory location that both parties can access, allowing the writer to deposit data and the reader to retrieve it without direct coordination overhead. This intermediary enables efficient data transfer while minimizing RDMA READ operations to only those necessary for flow control.
2Ease of operation
If RDMA READ operations are used for flow control, then data transmission can be regulated, but each outstanding READ operation consumes valuable adapter resources
Solution Approach 1:
The patent makes the shared write address serve multiple functions: it acts as both the data transfer destination and the flow control mechanism. The reader monitors the SWA to determine when to send acknowledgments, eliminating the need for separate flow control READ operations. This multi-functionality reduces adapter resource consumption while maintaining comprehensive flow control.
Solution Approach 2:
The system enables self-service flow control where the reader autonomously monitors the shared write address and manages its own buffer space without requiring the writer to initiate READ operations for status checking. The reader service itself by polling the SWA to determine when it can accept more data, reducing overall adapter resource usage.
Data Source
AI summary
The present invention provides a system and method for a shared write address protocol (SWAP) that is implemented over a remote direct memory address (RDMA) connection. Each party to a connection establishes a flow control block that is accessible to its partner via a RDMA READ operation. The novel protocol operates so that each module needs to have one outstanding RDMA READ operation at a time, i.e., to obtain the current flow control information from its partner. In operation, if data to be transmitted is less than or equal to a buffer size, an INLINE message data structure of the SWAP protocol is utilized to send the data to be target. However, if the data is greater than the buffer size, a second determination is made as to whether sufficient space exists in the message pool for the data. If insufficient space exists, the sender will wait until sufficient space exists before utilizing a novel WRITE operation of the SWAP protocol to transmit the data.