Layer separation of transport and RDMA

Abstract

This disclosure relates to layer-separated transport and RDMA. A communication system includes a first network device and a second network device. The first network device includes (i) an RDMA initiator for generating an RDMA message comprising one or more packets and specifying an RDMA operation; and (ii) a transmission requester for transmitting the one or more packets over a network via one or more transport connections. The second network device includes (i) a transmission responder for receiving one or more packets from a network via one or more transport connections; and (ii) an RDMA target for receiving the RDMA message and performing an RDMA operation. The transmission responder sends an acknowledgment indicating successful reception of the packets. The sending of these acknowledgments is independent of successful reception of the RDMA message. In addition to the acknowledgments, the RDMA target also sends an acknowledgment indicating successful reception of the RDMA message.

Description

Technical Field

[0001] This disclosure generally relates to network communications, and more specifically to communications using separate transport and RDMA layers. Background Technology

[0002] Remote Direct Memory Access (RDMA) is a protocol that enables reading and writing data to the memory of a remote host without involving the host's software. RDMA can be implemented using various transport protocols (e.g., via InfiniBand). TM (IB) Reliable Connection (RC) Operation. Summary of the Invention

[0003] The embodiments described herein provide a communication system including a first network device and a second network device. The first network device includes: (i) a Remote Direct Memory Access (RDMA) initiator for generating an RDMA message comprising one or more packets and specifying an RDMA operation; and (ii) a transmission requester for transmitting the one or more packets via one or more transmission connections over a network. The second network device includes: (i) a transmission responder for receiving one or more packets from a network via one or more transmission connections; and (ii) an RDMA target for receiving the RDMA message carried by the one or more packets and performing an RDMA operation. The transmission responder sends one or more acknowledgments to the first network device indicating successful reception of the one or more packets, the sending of which is independent of successful reception of the RDMA message by the RDMA target. In addition to the acknowledgments sent by the transmission responder, the RDMA target also sends a confirmation to the first network device indicating successful reception of the RDMA message.

[0004] In some embodiments, the transmission requester is configured to retransmit at least one packet in response to an acknowledgment, regardless of the acknowledgment. In some embodiments, the RDMA initiator is configured to retransmit at least a portion of the RDMA message in response to an acknowledgment, regardless of the acknowledgment.

[0005] In the disclosed embodiments, the RDMA target party determines whether all one or more packets associated with the RDMA message have been successfully received by maintaining a count of received packets associated with the RDMA message and comparing that count with the expected size of the RDMA message. In one embodiment, the RDMA target party determines whether all one or more packets associated with the RDMA message have been successfully received by tracking the successful arrival of portions of the RDMA message. In an example embodiment, the RDMA target party determines whether all one or more packets associated with the RDMA message have been successfully received, regardless of the Packet Sequence Number (PSN) specified in the packet.

[0006] In one embodiment, the transport requester is used to manage multiple transport connections; and the RDMA initiator is used to execute multiple RDMA objects that generate multiple RDMA messages and interleave packets of multiple RDMA messages through multiple transport connections. In another embodiment, the transport responder is used to manage multiple transport connections; and the RDMA target is used to execute multiple RDMA objects that receive packets of multiple RDMA messages interleaved through multiple transport connections and receive multiple RDMA messages.

[0007] In some embodiments, the RDMA initiator and RDMA target are not used to prevent duplicate packet delivery. In one example embodiment, the RDMA target is used to retire a given RDMA message in response to the discovery that all packets associated with a given RDMA message have not been received within a defined timeout period. In one embodiment, the RDMA initiator is used to report the completion of the RDMA message in response to receiving both (i) an acknowledgment from the transport responder and (ii) an acknowledgment from the RDMA target.

[0008] In one embodiment, the transmission requester releases packet resources upon receiving an acknowledgment from the transmission responder. In another embodiment, the RDMA target releases message resources upon receiving a transport layer acknowledgment.

[0009] A communication method is additionally provided according to embodiments described herein, the communication method comprising: in a first network device, (i) generating a Remote Direct Memory Access (RDMA) message including one or more packets and specifying an RDMA operation, and (ii) transmitting one or more packets over a network via one or more transport connections. In a second network device, (i) receiving the one or more packets from the network via one or more transport connections, and (ii) receiving the RDMA message carried by the one or more packets and performing the RDMA operation. One or more acknowledgments indicating successful reception of the one or more packets are sent from the second network device to the first network device, the sending of these acknowledgments being independent of successful reception of the RDMA message. In addition to these acknowledgments, an acknowledgment indicating successful reception of the RDMA message is also sent from the second network device to the first network device.

[0010] An additional communication method is provided according to embodiments described herein, the communication method comprising: in an initiating network device, (i) generating an RDMA message comprising one or more packets and specifying RDMA operations in a Remote Direct Memory Access (RDMA) layer, and (ii) transmitting one or more packets over a network via one or more transport connections in a transport layer. In a destination network device: (i) receiving one or more packets from a network via one or more transport connections in a transport layer; and (ii) receiving the RDMA message carried by one or more packets in the RDMA layer, and performing RDMA operations. In both the initiating and destination network devices, the RDMA layer and the transport layer are independent of each other.

[0011] This disclosure will be more fully understood from the following detailed description of embodiments of this disclosure in conjunction with the accompanying drawings. Attached Figure Description

[0012] Figure 1 This is a schematic diagram illustrating a network communication system according to an embodiment described herein.

[0013] Figure 2 The above-described embodiments according to the present document are schematically illustrated. Figure 1 A diagram illustrating the exchange of messages and packets, and corresponding responses and acknowledgments, within the system; Figure 3 A flowchart illustrating, schematically, a method for communication using separate transport and RDMA layers according to embodiments described herein; and Figure 4 This is a schematic diagram illustrating a computing system that uses separate transport and RDMA layers for communication according to embodiments described herein. Detailed Implementation

[0014] Overview In traditional RDMA implementations, there is considerable coupling and dependency between the RDMA layer and the underlying transport layer. In one example implementation, an initiating network device (“Initiator”) generates an RDMA message to be transmitted to a target network device (“Target”). This RDMA message specifies an RDMA operation to be performed in the Target’s memory. The Initiator generates one or more packets to deliver the RDMA message and sends these packets over the network to the Target. The Target receives these packets and sends back an acknowledgment to the Initiator indicating that the packets have arrived and been successfully processed, while simultaneously receiving the RDMA message and performing the RDMA operation in its memory. The acknowledgment of the packets signifies that the arriving packet has arrived and that all preceding packets have been successfully received and passed to the message construction process.

[0015] The above scheme is suitable for lossless networks that guarantee in-order packet delivery. However, modern networks employ mechanisms such as adaptive routing and multipathing, which can lead to out-of-order (OOO) packet delivery. In such networks, as well as in lossy networks, the coupling between the RDMA layer and the transport layer becomes problematic.

[0016] The embodiments described herein provide improved techniques for conveying RDMA messages. In this context, the term "RDMA" also includes protocols such as RDMA over Converged Ethernet (RoCE). The disclosed techniques provide complete separation of the RDMA layer from the transport layer. Specifically, the initiating network device and the target network device apply two separate, independent delivery feedback mechanisms, one at the packet level (as part of the transport layer) and one at the RDMA message level (as part of the RDMA layer).

[0017] In one example embodiment, the target network devices independently send back two types of delivery feedback instructions: This indicates a transport layer acknowledgment of successful packet reception. An acknowledgment is sent regardless of whether the packet was successfully associated with an RDMA message, and generally regardless of the status of the RDMA message processing by the RDMA layer.

[0018] An RDMA layer acknowledgment indicating whether the reception of an RDMA message was successful or failed. In this context, an RDMA message is considered "successfully received" when (i) one or more packets of RDMA messages have been successfully received and (ii) the corresponding memory access operation in the target has been completed. An acknowledgment of the RDMA message is sent in addition to the acknowledgment of one or more packets of RDMA messages.

[0019] The separation between the RDMA layer and the transport layer offers significant advantages. First, RDMA messages can be efficiently transmitted over lossless networks and networks that do not guarantee in-order packet delivery. For example, the transport layer in both the initiating and destination network devices can identify lost or delayed packets and initiate retransmissions with extremely short response times. Immediate acknowledgment generation also improves the efficiency of packet-related resource allocation and the accuracy of round-trip time (RTT) measurements.

[0020] Secondly, RDMA messages can be split among multiple separate, independent transmission entities, thereby making more efficient use of available network resources, improving packet generation and reception rates, and reducing RDMA message delivery latency.

[0021] Furthermore, multiple RDMA entities can use a given transport connection without the risk of a malfunctioning RDMA entity affecting other RDMA entities sharing the same transport. Multiple RDMA entities sharing a transport connection is crucial for scalability.

[0022] Furthermore, to determine whether all packets associated with a given RDMA message have arrived, the RDMA layer of the target network device does not need to know the expected packet sequence number (PSN) of the packets. Instead, the RDMA layer can count the number of received packets associated with the RDMA message and compare that number with the expected count. The transport layer is typically responsible for preventing duplicate packets, thus eliminating this requirement for the RDMA layer.

[0023] System Description Figure 1 This is a schematic block diagram of a network communication system 20 according to an embodiment described herein. System 20 includes a pair of network devices, designated 24A and 24B respectively, which communicate via IB network 28. Figure 1 In this embodiment, network device 24 is referred to as a network interface controller (NIC). However, the disclosed techniques can generally be applied to other suitable types of network devices, such as host channel adapters (HCAs) and data processing units (DPUs).

[0024] NIC 24A serves host 32A running one or more applications (“applications”) 36A and includes host memory 40A. NIC 24B serves host 32B running one or more applications 36B and includes host memory 40B.

[0025] In this example, NIC 24A sends RDMA messages specifying the RDMA operation to be performed by NIC 24B. For example, NIC 24A can initiate an RDMA WRITE message, which transfers data from an application in application 36A to host memory 40B via NICs 24A and 24B. As another example, NIC 24A can initiate an RDMA READ message, which uses NIC 24B to read data directly from host memory 40B and then transfers the data to an application in application 36A via NIC 24A. In both cases, the data transfer does not involve the software on host 32B. In this context, NIC 24A is referred to as the "initiating NIC," and NIC 24B is referred to as the "destination NIC."

[0026] The initiating NIC 24A includes one or more ports 44 for communicating with network 28, a host interface (I / F) 48 for communicating with host 32A, and a NIC circuit system 52 that performs various processing tasks of the NIC. The NIC circuit system 52 includes an RDMA initiator 56 and a transmission requester 60. The RDMA initiator 56 performs RDMA layer initiator-side tasks. The transmission requester 60 performs transport layer requester-side tasks. In some embodiments, the RDMA initiator 56 and the transmission requester 60 are implemented in hardware and controlled by software or firmware.

[0027] The target NIC 24B also includes one or more ports 44 for communicating with network 28, a host interface (I / F) 48 for communicating with host 32A, and a NIC circuit system 52 for performing various processing tasks of the NIC. The NIC circuit system 52 includes a transport responder 64 and an RDMA target 68. The transport responder 64 performs transport layer responder-side tasks. The RDMA target 68 performs RDMA layer target-side tasks. In some embodiments, the transport responder 64 and the RDMA target 68 are implemented in hardware and controlled by software or firmware.

[0028] Typically, NICs 24A and 24B are similar or identical in design, and it is logical for them to act as either initiators or recipients. A given NIC can typically be used concurrently as the initiator of one or more RDMA messages and the recipient of one or more RDMA messages. For clarity, Figure 1 The focus is on the initiator-related elements of NIC 24A and the target-related elements of NIC 24B.

[0029] Among other tasks, the NIC circuitry 52 of NICs 24A and 24B employs two separate, independent delivery feedback mechanisms. One mechanism is a transport layer mechanism, which operates between the requester 60 and the responder 64, using packet-level acknowledgments. The other mechanism is an RDMA layer mechanism, which operates between the RDMA initiator 56 and the RDMA target 68, using message-level acknowledgments. These two delivery feedback mechanisms and the resulting independence between the RDMA layer and the transport layer are described in detail below.

[0030] exist Figure 1 In this embodiment, the RDMA target party 68 includes an internal transmission requester 60R (also referred to as a reverse transmission requester), and the RDMA initiator 56 includes a transmission responder 64R (also referred to as a reverse transmission responder). The transmission requester 60R and the transmission responder 64R are responsible for exchanging message level acknowledgments.

[0031] In some embodiments, the transport layer (e.g., transport requester 60 and / or transport responder 64) is configured to prevent the delivery of duplicate copies of the same packet, i.e., to ensure that each packet is delivered no more than once. Thus, there is no need for an RDMA layer (e.g., RDMA initiator 56 and / or RDMA target 68) to check for and avoid duplicate copies of packets.

[0032] like Figure 1 The configuration of system 20 and its components (e.g., network devices 24A and 24B) depicted is an example configuration chosen purely for clarity of concept. In alternative embodiments, any other suitable configuration may be used.

[0033] In various embodiments, network devices 24A and 24B may be implemented using appropriate software, appropriate hardware (such as one or more application-specific integrated circuits (ASICs) or field-programmable gate arrays (FPGAs)), or a combination of hardware and software.

[0034] Some components of network devices 24A and 24B may be implemented using one or more general-purpose processors, which are programmed in software to perform the techniques described herein. The software may be downloaded to the processor electronically, for example, via a network, or may be provided and / or stored alternatively or additionally on a non-transitory tangible medium, such as magnetic memory, optical memory, or electronic memory.

[0035] Separation between RDMA layer and transport layer A typical RDMA message flow begins with a request from application 36A in host 32A to write data to or read data from memory 40B of host 32B. Upon receiving the request, RDMA initiator 56 in initiating NIC 24A includes an RDMA message, such as RDMA READ, RDMA WRITE, SEND, or ATOMIC REQUEST. RDMA initiator 56 generates one or more packets to deliver the RDMA message and provides these packets to transmission requester 60. Transmission requester 60 sends the packets to destination NIC 24B via network 28. In destination NIC 24B, transmission responder 64 receives the packets and forwards them to RDMA destination 68. RDMA 68 receives the RDMA message and performs the RDMA operation specified in the message, such as writing data to or reading data from host memory 40B. RDMA 68 typically does not cache the entire RDMA message (which can be very large), but instead accesses host memory 40B on a packet-by-packet basis.

[0036] In some embodiments, as part of the process described above, the target NIC 24B sends two types of delivery feedback instructions to the initiating NIC 24A, one type being referred to as “response” and the other as “acknowledgment”.

[0037] Transport layer acknowledgment (“ACK”). An ACK is sent from the responder 64 to the requester 60 and indicates successful reception of the packet. The responder 64 sends an ACK for the packet regardless of whether the packet was successfully received by the RDMA layer (e.g., whether it was successfully placed into memory) and regardless of the processing status of the RDMA target 68 for the associated RDMA message (i.e., the RDMA message to which the packet is associated). In many cases, the responder 64 sends the ACK for the packet before all packets of the RDMA message have had a chance to arrive, and therefore before the RDMA target 68 has received the entire RDMA message.

[0038] RDMA Layer Acknowledgment. An RDMA layer acknowledgment (also simply referred to as an "acknowledgment") indicating successful or failed reception of an RDMA message is sent from the RDMA target 68 to the RDMA initiator 56. As noted above, in this context, the term "successful reception" for an RDMA message includes successful access to host memory. In addition to one or more ACKs sent by the transmission responder 64 for one or more packets delivering the same RDMA message, the RDMA target 68 also sends an acknowledgment for the RDMA message using the reverse transmission requester 60R. The RDMA initiator 56 receives the acknowledgment using the reverse transmission responder 64R and acknowledges the acknowledgment by transmitting an ACK.

[0039] In this context, the term "acknowledgment" includes both positive acknowledgment (indicating that a packet was successfully received) and negative acknowledgment (indicating that a packet was not successfully received). A positive acknowledgment can relate to a single packet or multiple packets. Similarly, the term "acknowledgment" includes both positive acknowledgment (indicating that a message was successfully received) and negative acknowledgment (indicating that a message was not successfully received). A given acknowledgment can relate to a single message or multiple messages.

[0040] In some embodiments, each of the two delivery feedback mechanisms is coupled to a corresponding retransmission mechanism. These two retransmission mechanisms belong to different layers and are separate and independent of each other. In the transport layer, the transmission requester 60 may retransmit a packet in response to an acknowledgment (e.g., in response to not receiving a positive ACK for the packet, or in response to receiving a negative ACK for the packet). The retransmitted packet is unrelated to any RDMA layer acknowledgment that may or may not be received. In the RDMA layer, the RDMA initiator 56 may retransmit at least a portion of the RDMA message in response to an acknowledgment (e.g., in response to not receiving a positive ACK for the message, or in response to receiving a negative ACK for the message).

[0041] Example message exchange Figure 2 This is a schematic illustration of the exchange of messages and packets, and corresponding responses and acknowledgments, according to embodiments described herein. Figure 2 In the example, an application in application 36A (on host 32A) initiates an RDMA message specifying an RDMA operation to be performed in host memory 40B of host 32B. The application 36A initiator NIC 24A sends the RDMA message by publishing a work queue element (WQE) on the work queue (WQ) of NIC 24A. This WQE has an identifier designated as the SWQE ID.

[0042] In response to WQE, RDMA initiator 56 in NIC 24A generates an RDMA message spanning three packets. RDMA initiator 56 provides these three packets to transmission requester 60. RDMA initiator 56 uses transport layer packet identifiers, designated PACKET_ID=X, PACKET_ID=Y, and PACKET_ID=Z, to identify the packets destined for transmission requester 60. Transmission requester 60 then transmits these three packets through network 28 to destination NIC 24B.

[0043] When sending packets, the transmission requester 60 appends the corresponding Packet Sequence Number (PSN) to each packet. In this example, packets with PACKET_ID=X are sent using PSN=1, packets with PACKET_ID=Y are sent using PSN=2, and packets with PACKET_ID=Z are sent using PSN=3. The transmission requester 60 can choose any order of assignment for PACKET_ID and PSN, but it should record the association between PSN and PACKET_ID for future correlation with received ACKs.

[0044] In the target NIC 24B, these three packets are provided to the transmission responder 64. For each successfully received packet, the transmission responder 64 sends an ACK back to the initiating NIC 24A via network 28 using the corresponding PSN. The transmission responder 64 can combine multiple ACKs and only send the last PSN of the ACK, thus also acknowledging the preceding packets. Note that the transmission responder 64 sends an ACK for the packet regardless of the packet's processing status in the RDMA layer. These three ACKs use their PSNs to indicate the corresponding IDs of the packets they are acknowledging (PACKET_ID=X, PACKET_ID=Y, and PACKET_ID=Z). The transport layer requester 60 in the initiating NIC 24A forwards the ACK to the RDMA initiator 56 using the PACKET_ID provided by the RDMA initiator 56.

[0045] In addition to sending a transport layer ACK, the transport responder 64 also provides these three packets to the RDMA target 68 in NIC 24B. The transport responder 64 uses the RDMA layer message identifier MESSAGE_ID=55 to identify the packets destined for the RDMA target 68. The message ID indicates the RDMA message to which the packet belongs. The RDMA target 68 receives the three packets belonging to message ID=55 and performs the corresponding RDMA operation packet by packet in the host memory 40B.

[0046] In some embodiments, the RDMA target 68 determines whether all packets associated with the RDMA message have arrived by counting packets with MESSAGE_ID=55. When the count reaches the expected number of packets in the RDMA message (3 in this example), the RDMA target 68 determines the completion of the RDMA message. The RDMA initiator 56 typically reports the expected number of packets in the RDMA message in at least one packet (e.g., in all packets) of the message. In this way, the RDMA target does not need to know or track the PSN of the packets belonging to each RDMA message. In some cases, because the RDMA message may be transmitted over multiple different transport connections with separate PSN sequences, the RDMA target cannot track the PSN.

[0047] In addition to counting packets, the RDMA target 68 can also track different portions of the RDMA message being received and record indications of successfully received portions of the message in the message layer. For example, different portions of the RDMA message can be tracked in terms of their offset relative to the beginning of the message (which in turn indicates their position in the host memory 40B).

[0048] Upon successful reception of an RDMA message, the RDMA target 68 performs the corresponding RDMA operation in the host memory 40B. As explained above, the RDMA target 68 typically performs the RDMA operation on a per-received-packet basis, without buffering the entire RDMA message. For example, in the case of RDMA WRITE, the RDMA target 68 writes the data transmitted in the RDMA message packet by packet into a designated address buffer in the host memory 40B. In the case of RDMA READ, the RDMA target 68 reads the data from the designated address buffer in the host memory 40B and sends the data packet by packet back to the initiating NIC 24A using the reverse transmission requester 60R.

[0049] After completing the RDMA operation, the RDMA target 68 can report completion by publishing a Completion Queue Element (CQE) on the Completion Queue (CQ). Typically, a CQE is generated only for certain types of RDMA messages (e.g., RDMA_WRITE_WITH_IMMEDIATE and various types of SEND messages). The CQE has an identifier designated as the RWQE ID. The CQE is read by an application on application 36B on host 32B.

[0050] Upon successfully receiving the RDMA message, the RDMA target party 68 sends an RDMA layer acknowledgment to the RDMA initiator 56. This acknowledgment... Figure 2The label is "Status". The RDMA target 68 provides the RDMA layer acknowledgment in a single status packet to the transmission requester 60R. The RDMA target 68 identifies the status packet destined for the transmission requester 60R via PACKET_ID=W. The transmission requester 60R (in this example, using PSN=8) sends the status packet through network 28 to the transmission responder 64R in the initiating NIC 24A.

[0051] Upon receiving a status packet, the transmission responder 64R sends a transport layer ACK for the status packet to the transmission requester 60R in the target NIC 24B. The ACK has the same PSN=8 as the status packet it acknowledges, thus indicating the delivery of PACKET_ID=W. The transmission requester 60R then reports the ACK to the RDMA target 68 using PACKET_ID=W.

[0052] In addition to responding to the status packet sent to the transmission requester 60R, the transmission responder 64R also notifies the RDMA initiator 56 that the status packet has been received. This notification carries the MESSAGE_ID of the RDMA message, i.e., MESSAGE_ID=55.

[0053] In some embodiments, the RDMA initiator 56 reports the completion of the RDMA message in response to receiving (i) a transport layer acknowledgment of a packet of SPDM message from the transport responder 64 and (ii) an acknowledgment from the RDMA target 68.

[0054] In response to this notification, RDMA initiator 56 reports the completion of the RDMA message by publishing a CQE on the CQ. This CQE has an identifier designated as SWQE ID. This CQE is read by the application 36A that initiated the RDMA message.

[0055] In some embodiments, transport layer ACKs and RDMA layer acknowledgments are also used as triggers for releasing resources. For example, a transmission requester (60 or 60R) may release packet resources upon receiving an ACK for a packet from a transmission responder (64 or 64R). As another example, an RDMA target (68) may release message resources upon receiving a transport layer ACK acknowledging an RDMA acknowledgment (“status” packet).

[0056] Figure 3 This is a schematic diagram of a flowchart illustrating a method for communication using separate transport and RDMA layers according to embodiments described herein. By way of non-limiting example, Figure 3The embodiment refers to an RDMAWRITE message spanning multiple packets. Phases 90-98 of this method are performed by the initiating NIC 24A. Phases 102-118 are performed by the target NIC 24B.

[0057] The method begins at WQE reception phase 90, where RDMA initiator 56 receives a WQE from application 36A on host 32A. The WQE specifies a data buffer within a specified address range to be written to host memory 40B on host 32B. At message preparation phase 94, RDMA initiator 56 reads the data buffer from host memory 40A and constructs an RDMA message including multiple packets carrying data. At transmission phase 98, transmission requester 60 transmits the packets to target NIC 24B via network 28 through one or more transmission connections.

[0058] At receive phase 102, the transmission responder 64 receives packets via one or more transport connections. At transport layer acknowledgment phase 106, the transmission responder 64 sends a corresponding transport layer ACK for each successfully received packet. The transport layer ACK is sent regardless of the success of the RDMA message reception (including memory access) (typically prior to the successful reception of the RDMA message).

[0059] At message reception phase 110, RDMA target 68 receives the RDMA message transmitted by the received packet and performs the corresponding memory access operation in host memory 40B. At RDMA layer acknowledgment phase 114, RDMA target 68 sends an RDMA acknowledgment (“status”) indicating successful reception of the RDMA message. At memory write phase 118, RDMA target 68 writes data to a specified address range in host memory 40B. RDMA target 68 writes data packet by packet (e.g., as soon as each packet is received).

[0060] Figure 3 The flow described is a simplified flow depicted purely for conceptual clarity. Various other suitable flows may be used in alternative embodiments. For example, in some embodiments, if all packets of a message do not arrive within a specified timeout period, the RDMA target 68 may cancel a given RDMA message. For clarity, certain operations (e.g., retransmission and transport layer acknowledgments for status packets) have been omitted from the figure.

[0061] Handling multiple transport and / or RDMA objects For clarity, the embodiments described so far refer to processing a single RDMA message. However, in many real-world scenarios, using the disclosed techniques to interleave packets of multiple RDMA messages across multiple transport connections offers significant advantages.

[0062] In one example embodiment, the transport requester 60 in the initiating NIC 24A handles multiple transport connections. These multiple transport connections can be connected to any desired number of destination NICs 24B. The RDMA initiator 56 in the initiating NIC 24A handles multiple RDMA objects that simultaneously generate multiple RDMA messages. The RDMA initiator 56 sends packets of different RDMA messages through the multiple transport connections. The RDMA initiator 56 can select transport connections for a given packet based on, for example, the destination NIC to which the message is destined, the priority of the message or packet (e.g., quality of service category), etc.

[0063] In one example embodiment, the transport responder 64 in the target NIC 24B handles multiple transport connections. These multiple transport connections can be connected to any desired number of initiating NICs 24A. The RDMA target 68 in the target NIC 24B handles multiple RDMA objects to receive multiple RDMA messages simultaneously. The RDMA target 68 receives packets of different RDMA messages through the multiple transport connections and performs the corresponding RDMA operations.

[0064] In such an embodiment, (i) a given transport connection can transmit packets belonging to different RDMA messages and interleaved with each other, and (ii) packets of a given RDMA message can be distributed across multiple transport connections.

[0065] The separation of the transport layer acknowledgment mechanism from the RDMA layer confirmation mechanism ensures that a malfunctioning RDMA object will not affect the performance of other RDMA objects sharing the same transport connection.

[0066] Example System Use Cases Figure 4 This is a schematic block diagram illustrating a computing system 1000 (e.g., a data center or high-performance computing (HPC) cluster) employing communication using separate transport and RDMA layers according to embodiments described herein. According to at least one embodiment, system 1000 includes multiple subsystems, such as multiple processing devices, multiple network devices, and multiple networks coupled to each other. The computing system 1000 is designed with multiple integrated circuits (referred to as processing devices), each of which may include one or more CPUs and GPUs, thus forming a robust and flexible architecture.

[0067] Various processing devices are interconnected via NVLink or other high-speed interconnects to enable high-speed communication between subsystems; and are also connected via NICs or DPUs to ensure efficient data transmission across computing system 1000 and to one or more external networks 1030, 1036. In this example, system 1000 includes a packet switch 1048 that connects NIC / DPU 1028 to network 1030 and a packet switch 1050 that connects NIC / DPU 1032 to network 1036.

[0068] Seamless data exchange and parallel processing are enabled through NVLink coupling of the processing device, thereby improving overall computing performance. The processing device connects to multiple networks via one or more network interface cards (NICs) or DPUs, enabling the system to handle complex multi-network tasks with high bandwidth and low latency. This configuration is ideal for demanding applications requiring massive processing power, such as artificial intelligence (AI), machine learning (ML), and data-intensive computing, while ensuring robust connectivity and scalability across a variety of networked environments. The integrated circuits of the computing system 1000 may include one or more CPUs and one or more GPUs.

[0069] Figure 4 An example architecture of a multi-GPU architecture is also shown. As illustrated, the computing system 1000 includes a processing device 1002 with a multi-GPU architecture. Specifically, the processing device 1002 may be a system-on-a-chip and includes multiple subsystems such as a CPU 1006, a GPU 1008, and a GPU 1010. The CPU 1006 may be coupled to the GPU 1008 via die-to-die (D2D) or chip-to-chip (C2C) interconnects 1012 (such as a ground reference signaling interconnect (GRS interconnect)). The CPU 1006 may be coupled to the GPU 1010 via a D2D or C2C interconnect 1014. The CPU 1006 may also be coupled to the GPU 1008 and GPU 1010 via a PCIe interconnect.

[0070] The CPU 1006 can be coupled to one or more NICs or DPUs, which in turn are coupled to one or more networks. For example, as Figure 4 As illustrated, CPU 1006 is coupled to a first NIC / DPU 1026, which is coupled to network 1030. CPU 1006 is also coupled to a second NIC / DPU 1028, which is coupled to network 1030 via switch 1048. For example, NIC / DPU 1026 and NIC / DPU 1028 can be coupled to network 1030 via Ethernet (ETH), NVLINK, or InfiniBand (IB) connections.

[0071] The computing system 1000 also includes a processing device 1004 with a multi-GPU architecture. Specifically, the processing device 1004 includes multiple subsystems, including a CPU 1016, a GPU 1018, and a GPU 1020. The CPU 1016 can be coupled to the GPU 1018 via a D2D or C2C interconnect 1022. The CPU 1016 can be coupled to the GPU 1020 via a D2D or C2C interconnect 1024. The CPU 1016 can also be coupled to the GPU 1018 and GPU 1020 via a PCIe interconnect. The CPU 1016 can be coupled to one or more NICs or DPUs, which in turn are coupled to one or more networks. For example, as... Figure 4 As illustrated, CPU 1016 is coupled to a first NIC / DPU 1032, which is coupled to network 1036. CPU 1016 is also coupled to a second NIC / DPU 1034, which is coupled to network 1036 via switch 1050. NIC / DPU 1032 and NIC / DPU 1034 can be coupled to network 1036 via Ethernet (ETH), NVLINK, or InfiniBand (IB) connections.

[0072] In at least one embodiment, processing device 1002 and processing device 1004 can communicate with each other via NIC / DPU 1038 (such as via PCIe interconnect). Processing device 1002 and processing device 1004 can also communicate with each other via high-bandwidth communication interconnect 1040 (such as NVLink interconnect or other high-speed interconnect).

[0073] Figure 4 The packet switch in the figure may include, for example, an Nvidia Quantum-2 switch. The NIC / DPU in the figure may include, for example, an Nvidia Bluefield DPU. In various embodiments, any NIC / DPU can be used as an initiating network device and / or a destination network device communicating using the layer separation technology described herein.

[0074] It should be understood that the embodiments described above are presented by way of example, and the invention is not limited to what has been specifically shown and described above. Rather, the scope of the invention includes combinations and sub-combinations of the various features described above, as well as variations and modifications that may occur to those skilled in the art upon reading the foregoing description and that are not disclosed in the prior art. Documents incorporated herein by reference should be considered part of this application, and the definitions in this specification should be considered only unless the definitions of any terms in these incorporated documents conflict with the explicit or implicit definitions in this specification.

Claims

1. A communication system, comprising: A first network device, the first network device comprising: A Remote Direct Memory Access (RDMA) initiator, the RDMA initiator being used to generate an RDMA message comprising one or more packets and specifying an RDMA operation; and A transmission requester, the transmission requester being configured to send the one or more packets over a network via one or more transmission connections; and The second network device includes: A transmission responder, the transmission responder being configured to receive the one or more packets from the network via the one or more transmission connections; and The RDMA target party is configured to receive the RDMA messages transmitted by the one or more packets and perform the RDMA operation, wherein: (i) The transmission responder is configured to send one or more acknowledgments to the first network device indicating successful reception of the one or more packets, the sending of which is independent of successful reception of the RDMA message by the RDMA target; and (ii) In addition to the response sent by the transmission responder, the RDMA target is also configured to send an acknowledgment to the first network device indicating successful reception of the RDMA message.

2. The communication system of claim 1, wherein the transmission requesting party is configured to respond to the acknowledgment and retransmit at least one of the packets regardless of the acknowledgment.

3. The communication system of claim 1, wherein the RDMA initiator is configured to retransmit at least a portion of the RDMA message in response to the acknowledgment, regardless of the response.

4. The communication system of claim 1, wherein the RDMA target is configured to determine whether all one or more packets associated with the RDMA message have been successfully received by maintaining a count of received packets associated with the RDMA message and comparing the count with the expected size of the RDMA message.

5. The communication system of claim 1, wherein the RDMA target party is configured to determine whether all one or more packets associated with the RDMA message have been successfully received by tracking the successful arrival of each portion of the RDMA message.

6. The communication system of claim 1, wherein the RDMA target is configured to determine whether all one or more packets associated with the RDMA message have been successfully received, regardless of the packet sequence number (PSN) specified in the packet.

7. The communication system according to claim 1, wherein: The transmission requester is used to handle multiple transmission connections, and The RDMA initiator is used to execute multiple RDMA objects that generate multiple RDMA messages, and to interleave the packets of the multiple RDMA messages through the multiple transport connections.

8. The communication system according to claim 1, wherein: The transmission response side is used to handle multiple transmission connections, and The RDMA target is used to execute multiple RDMA objects, which receive packets of multiple RDMA messages interleaved through the multiple transport connections and receive the multiple RDMA messages.

9. The communication system of claim 1, wherein the RDMA initiator and the RDMA target are not used to prevent duplicate packet delivery.

10. The communication system of claim 1, wherein the RDMA target is configured to withdraw the given RDMA message in response to the discovery that all packets associated with the given RDMA message have not been received within a defined timeout period.

11. The communication system of claim 1, wherein the RDMA initiator is configured to report the completion of the RDMA message in response to receiving both (i) the acknowledgment from the transmission responder and (ii) the confirmation from the RDMA target.

12. The communication system of claim 1, wherein the transmission requester is configured to release packet resources upon receiving a response from the transmission responder.

13. The communication system of claim 1, wherein the RDMA target is configured to release message resources upon receiving a transport layer acknowledgment of the confirmation.

14. A communication method, comprising: In the first network device, (i) a Remote Direct Memory Access RDMA message is generated, which includes one or more packets and specifies a Remote Direct Memory Access RDMA operation, and (ii) the one or more packets are transmitted over the network via one or more transport connections; In the second network device, (i) one or more packets are received from the network via the one or more transport connections, and (ii) the RDMA message transmitted by the one or more packets is received, and the RDMA operation is performed; The second network device sends one or more acknowledgments to the first network device indicating successful reception of the one or more packets, the sending of which is unrelated to successful reception of the RDMA message; as well as In addition to the response, an acknowledgment of successful reception of the RDMA message is sent from the second network device to the first network device.

15. The communication method according to claim 14, wherein and comprising: In response to the reply and regardless of the confirmation, at least one of the packets is retransmitted from the first network device.

16. The communication method according to claim 14, further comprising: In response to the acknowledgment and regardless of the response, at least a portion of the RDMA message is retransmitted from the first network device.

17. The communication method according to claim 14, further comprising: In the second network device, it is determined whether all one or more packets associated with the RDMA message have been successfully received by maintaining a count of received packets associated with the RDMA message and comparing the count with the expected size of the RDMA message.

18. The communication method according to claim 14, further comprising: In the first network device: Handling multiple transport connections, and Multiple RDMA objects are executed to generate multiple RDMA messages, and the packets of the multiple RDMA messages are interleaved through the multiple transport connections.

19. The communication method according to claim 14, further comprising: In the first network device: Handling multiple transport connections, and Execute multiple RDMA objects that receive packets of multiple RDMA messages interleaved through the multiple transport connections and receive the multiple RDMA messages.

20. A communication method, comprising: In the initiating network device, (i) in the Remote Direct Memory Access (RDMA) layer, an RDMA message comprising one or more packets and specifying RDMA operations is generated, and (ii) in the transport layer, the one or more packets are transmitted over the network via one or more transport connections. as well as In the target network device, (i) in the transport layer, one or more packets are received from the network via one or more transport connections, and (ii) in the RDMA layer, the RDMA message transmitted by the one or more packets is received, and the RDMA operation is performed. In both the initiating network device and the target network device, the RDMA layer and the transport layer are independent of each other.

Images