Packet transmission with flow redundancy

Abstract

A network device includes a port, a packet pipeline, and a redundancy generator. The port is to connect to the network. The packet pipeline is to transmit a sequence of packets to the network via the port. The redundancy generator is to (i) initialize one or more redundancy blocks for a group of data packets, then (ii) iteratively update the one or more redundancy blocks in response to data packets of the group passing through the packet pipeline, and (iii) generate one or more redundancy packets including the one or more redundancy blocks. The redundancy generator transmits the one or more redundancy packets to the network using the packet pipeline.

Description

Technical Field

[0001] This disclosure relates generally to packet communication systems, and more specifically to methods and systems for transport layer redundancy. Background Technology

[0002] Reliable transport protocols such as Remote Direct Memory Access (RDMA) typically rely on retransmission mechanisms to mitigate packet loss and maintain functional correctness. While providing high reliability, retransmissions can incur significant time delays, equivalent to network round-trip time (RTT). However, short message delays are crucial in many applications, such as complex machine learning (ML) tasks distributed across multiple hosts.

[0003] The embodiments described herein provide a network device including a port, a packet pipeline, and a redundancy generator. The port is used to connect to a network. The packet pipeline is used to transmit a sequence of data packets to the network via the port. The redundancy generator is used for a group of data packets to: (i) initialize one or more redundancy blocks; then (ii) iteratively update one or more redundancy blocks in response to data packets in the group passing through (or traversing) the packet pipeline; and (iii) generate one or more redundant packets including one or more redundancy blocks. The redundancy generator is used to transmit one or more redundant packets to the network using the packet pipeline.

[0004] In some embodiments, the redundancy generator is used to obtain data packets from the transport layer of a network device. In some embodiments, data packets in a group are associated with a given queue pair (QP) of the network device. In some embodiments, the redundancy generator is used to update multiple sets of redundancy blocks for multiple corresponding groups of data packets, including maintaining the corresponding state of these groups.

[0005] In the disclosed embodiments, at least two packets in the group are of different sizes, and the redundancy generator is used to set the size of the redundancy block to at least the maximum size of the packets in the group. In one example embodiment, the redundancy generator is used to specify the respective sizes of the packets in the group within one or more redundancy packets. In one embodiment, the redundancy generator is used to include packets associated with write or send commands in the group and exclude packets associated with read commands from the group. In some embodiments, one or more redundancy blocks comprise a plurality of redundancy blocks, and the redundancy generator is used to update each redundancy block in the redundancy blocks in response to a correspondingly different subset of the packets in the group.

[0006] Additionally, according to embodiments described herein, a network device is provided that includes a port, a packet pipeline, and a redundancy reconstructor. The port is used to connect to a network. The packet pipeline is used to receive a sequence of packets from the network via the port, the sequence of packets including multiple data packets and one or more redundant packets. The redundancy reconstructor is used to perform the following operations on a group of expected data packets associated with one or more redundant packets: (i) initialize one or more redundancy blocks; (ii) iteratively update one or more redundancy blocks to account for any received redundant packets and any received data packets belonging to the group; (iii) after updating the redundancy blocks to account for one or more redundant packets and all data packets in the group except for a specified number of one or more remaining data packets, reconstruct one or more remaining data packets using the one or more redundancy blocks; and (iv) provide one or more reconstructed remaining data packets to the packet pipeline.

[0007] In some embodiments, the redundancy reconstructor is used to forward one or more reconstructed remaining packets to the transport layer of the network device. In some embodiments, packets in a group are associated with a given queue pair (QP) of the network device.

[0008] In some embodiments, the redundancy reconstructor is used to update multiple sets of redundant blocks for multiple corresponding groups of data packets, including maintaining the corresponding states of these groups. In one example embodiment, based on one or more identifiers extracted from the data packet, the redundancy reconstructor is used to associate a received data packet with a group within a group.

[0009] In the disclosed embodiments, the redundancy reconstructor is used to (i) extract the size of the corresponding data packet in a group from one or more redundant packets; and (ii) update one or more redundant blocks according to the size of the data packet to accommodate the received data packet.

[0010] In some embodiments, one or more redundant blocks include multiple redundant blocks, and a redundancy reconstructor is used to update each redundant block in the redundant blocks in response to corresponding and different subsets of packets in the group.

[0011] According to the embodiments described herein, a method is also provided, comprising: transmitting a sequence of data packets to a network using a packet pipeline. For a group of data packets, one or more redundant blocks are initialized, and then, in response to data packets in the group passing through (or traversing) the packet pipeline, one or more redundant blocks are iteratively updated, and one or more redundant packets are generated, the one or more redundant packets comprising one or more redundant blocks. The one or more redundant packets are transmitted to the network using the packet pipeline.

[0012] According to embodiments described herein, a method is also provided, comprising: receiving a sequence of packets from a network using a packet pipeline, the sequence of packets including a plurality of data packets and one or more redundant packets. For a group of expected data packets associated with one or more redundant packets, the following operations are performed: (i) initializing one or more redundancy blocks; (ii) iteratively updating one or more redundancy blocks to address any received redundant packets and any received data packets belonging to the group; (iii) after updating the redundancy blocks to address the one or more redundant packets and all data packets in the group except for a specified number of one or more remaining data packets, reconstructing one or more remaining data packets using the one or more redundancy blocks; and (iv) providing one or more reconstructed remaining data packets to the packet pipeline.

[0013] This disclosure will be more fully understood by referring to the following detailed description of embodiments of this disclosure in conjunction with the accompanying drawings. Attached Figure Description

[0014] Figure 1 This is a schematic diagram illustrating a communication system including network devices using the Transport Enhancement Redundancy Protocol (TARP) according to embodiments described herein;

[0015] Figure 2 This is a schematic diagram illustrating the integration of TARP in transport layer processing according to embodiments described herein;

[0016] Figure 3 This is a schematic illustration of a sequence of data packets divided into TARP areas according to an embodiment described herein;

[0017] Figure 4 This is a schematic diagram of a flowchart illustrating a method for packet transmission using TARP in a requester network device according to an embodiment described herein;

[0018] Figure 5 A flowchart illustrating, schematically, is a method for packet reception using TARP in a responder network device according to embodiments described herein; and

[0019] Figure 6 This is a schematic diagram illustrating a computing system including a network device using TARP according to an embodiment described herein. Detailed Implementation

[0020] Overview

[0021] The embodiments described herein provide a novel protocol called Transport Enhanced Redundancy Protocol (TARP), systems and network devices employing TARP, and related methods. TARP mitigates packet loss without requiring retransmissions and without causing the latency associated with retransmissions. TARP can be used in various network devices (e.g., in Ethernet network interface controllers (NICs), InfiniBand... TM (IB) Host Channel Adapter (HCA), Data Processing Unit (DPU, also known as "Smart NIC"), etc. are implemented.

[0022] TARP is typically integrated as part of the transport layer stack in network devices. Therefore, the upper layers have highly reliable and short-latency transmissions, while remaining unaware of the underlying redundant operations. By way of example, the embodiments described herein primarily relate to RDMA, but the disclosed techniques can be used with a variety of other transport protocols. In RDMA, TARP can be used for both reliable (RC) and unreliable (UC) connections. TARP is useful in both long-RTT and short-RTT connections.

[0023] In some embodiments, a network device referred to as a "requester" sends a sequence of data packets to a network device referred to as a "responder." For a group of data packets defined in the sequence (referred to as a "TARP group" or "TARP area"), the requester calculates "redundant blocks" on the data packets in the group. The requester then generates a Redundant Packet (RDP), which includes the redundant blocks and associated metadata, and also sends the RDP to the requester. Because the redundant blocks are sent within the payload of the RDP, the redundant blocks are also referred to as "redundant payloads."

[0024] Redundancy blocks can include, for example, bitwise XORs of packets within a TARP group. Since this group can include packets of varying sizes, in some embodiments, the size of the redundancy block is set to the maximum supported packet size. For shorter packets, the XOR calculation treats the remaining bits as zero. RDP enables the responder to reconstruct any single lost packet within the group without requiring retransmission.

[0025] The requester typically generates redundant blocks incrementally in a streaming manner as data packets are being transmitted. In one example embodiment, the requester maintains a "temporary redundant payload" in memory for each TARP group currently being transmitted. After a data packet is transmitted, the requester updates the temporary redundant payload for the relevant TARP group to accommodate the packet. After the temporary redundant payload has been updated to accommodate the last packet in the TARP group, the requester generates and sends the RDP.

[0026] In a similar manner, the responder typically maintains a temporary redundant payload for each TARP group that has not yet been fully received. Upon receiving a packet (data packet or RDP) belonging to a particular TARP group, the responder updates the temporary redundant payload for that group in response to the received packet. If all packets in the group are received before the RDP, the responder can discard the RDP. Otherwise, after receiving all packets in the group except for one packet (in the case of a single RDP per TARP group), and the RDP, the responder reconstructs the lost packets based on the temporary redundant payload.

[0027] This document describes various implementations, variations, and extensions of TARP. For example, TARP can be enabled for some connections (e.g., certain queue pairs (QPs) in a requester and responder) and disabled for others (e.g., other QPs). TARP group sizes can be fixed or variable. The mapping between packets and TARP groups can be defined according to the packet sequence number (PSN) of the packets.

[0028] As another example, the embodiments described herein primarily refer to packets sent from a requester (transaction requester) to a responder (transaction responder). Additionally or alternatively, TARP can be applied to packets sent from a responder to a requester, for example, to read response packets.

[0029] In some embodiments, a variant of TARP uses multiple RDPs per TARP group. For a given TARP group, the requester computes each RDP on a different subset of packets within the group. This TARP variant enables the responder to mitigate the loss of multiple packets within the group. This document describes techniques for selecting a subset of packets and reconstructing lost packets using multiple RDPs.

[0030] System Description

[0031] Figure 1 This is a schematic block diagram illustrating a communication system 20 including network devices using TARP according to embodiments described herein. System 20 includes a network device 24 acting as a requester and a network device 28 acting as a responder. Devices 24 and 28 provide services to corresponding hosts (not visible in the figure) and communicate with each other via network 32.

[0032] For clarity, Figure 1 The simplified example illustrates only two network devices. Real-world systems typically include a large number of network devices. Communication systems such as System 20 can be used to implement data centers, high-performance computing (HPC) clusters, or any other suitable system. (The following...) Figure 6 The example system use case is illustrated.

[0033] Network devices 24 and 28 may include, for example, a NIC, HCA, or DPU. Network 32 may include, for example, an Ethernet or IB network. A given network device typically functions as a requester for some connections and a responder for others. For clarity, this diagram focuses on elements related to packet transmission in the requester and elements related to packet reception in the responder.

[0034] The requester 24 includes a host interface (I / F) 36 for communicating with one or more hosts, a network interface 40 (also referred to as a port) for communicating over network 32, and a packet pipeline for transmitting packets to network 32. Among other elements, the packet pipeline of the requester 24 includes a transport layer processing circuitry 44 that performs various transport layer processing tasks for the requester. For brevity, the circuitry 44 is also referred to herein as the "transport circuitry". In this example, the transport circuitry 44 implements the RDMA protocol stack.

[0035] The requester 24 also includes a TARP generation circuitry system 48 coupled to the transmission circuitry system 44. The TARP generation circuitry system 48 is also referred to herein as a “TARP generator” or “redundancy generator.” The TARP generator 48 generates TARP Redundancy Packets (RDPs) on each group of data packets in a streaming manner, as described in detail below. The requester 24 also includes a memory called an RDP buffer 52 for temporarily storing redundant blocks, RDPs, and / or other related information, and a TARP group state memory 60 for storing the current state of each active TARP group.

[0036] Responder 28 includes a host interface (I / F) 36 for communicating with one or more hosts served by the responder, a network interface 40 (also referred to as a port) for communicating over network 32, and a packet pipeline for receiving packets from network 32. Among other elements, the packet pipeline of responder 28 includes a transport layer processing circuitry 44 (also referred to herein as a “transport circuitry”) that performs various transport layer processing tasks for the responder. In this example, transport circuitry 44 implements the RDMA protocol stack.

[0037] The responder 28 also includes a TARP aggregation circuitry 56 (also referred to herein as a “TARP aggregator” or “redundancy reconstructor”) coupled to the transmission circuitry 44. The TARP aggregator 56 reconstructs lost packets using RDP received from the requester 24. The requester 28 also includes an RDP buffer 52 for temporarily storing redundant blocks, RDPs, and / or other relevant information, and a TARP group state memory 60 for storing the current state of each active TARP group. The packet reconstruction process, including the use of the RDP buffer and the TARP group state 60, is described in detail below.

[0038] like Figure 1 The configurations of system 20 and network devices 24 and 28 shown in the illustrations are merely example configurations chosen for clarity of concept. In alternative embodiments, any other suitable configuration may be used.

[0039] Network devices 24 and 28 can be implemented using suitable hardware, such as software, hardware, or a combination of hardware and software elements, in one or more application-specific integrated circuits (ASICs) or field-programmable gate arrays (FPGAs). RDP buffer 52 and TARP group state 60 can be implemented in any suitable memory, such as random access memory (RAM). For clarity, elements not essential for understanding the disclosed techniques have been omitted from the figures.

[0040] In some embodiments, some of the network device functions described herein may be implemented in a general-purpose processor, which is programmed in software to perform the functions described herein. The software may be downloaded to the processor electronically, for example, via a network, or alternatively or additionally, or may be provided and / or stored on a non-transitory tangible medium, such as magnetic memory, optical memory, or electronic memory.

[0041] Figure 2 This diagram schematically illustrates the integration of TARP in the transport layer processing of requester network device 24 and responder network device 28 according to embodiments described herein. It can be seen that TARP is implemented as a sublayer within a sublayer stack constituting the transport layer.

[0042] In this example, the TARP sublayer is implemented above the multipath sublayer and below the reliability sublayer. Using this order, the existing RDMA reliability sublayer (including acknowledgment and retransmission mechanisms) can be used to mitigate any remaining packet loss that TARP fails to resolve. The RDMA reliability sublayer is unaware of the packet recovery performed by the TARP sublayer. The TARP recovery mechanism is generally independent of the multipathing techniques that can be used by the multipath sublayer.

[0043] TARP connection / QP, group / region, TARP status

[0044] Typically, requester 24 processes one or more active connections with one or more responders 28 at a given time. For each active connection, the requester maintains a corresponding queue pair (QP) for submitting work requests and completion notifications. Similarly, requester 28 processes one or more active connections with one or more requesters 24 at a given time. The responders maintain a corresponding QP for each active connection for submitting work requests and completion notifications. The terms "connection" and "QP" are used interchangeably herein.

[0045] In some embodiments, TARP is enabled for certain connections (QPs) and disabled for others. The decision of whether TARP is enabled or disabled for a given QP is typically made by the requester 24. The responder 28 determines whether TARP is enabled or disabled for a particular QP by checking if an RDP is received in that particular QP. If no RDP is received, the responder does not apply the disclosed techniques. Alternatively, the requester may send an explicit message to the responder indicating which QPs have TARP enabled and / or which QPs have TARP disabled.

[0046] Within a given connection, the requester typically assigns a consecutive packet sequence number (PSN) to packets being transmitted. By convention, both the requester and responder use the PSN to detect lost packets, request and perform retransmissions, and send acknowledgments. In some embodiments, the PSN is also used to define a "TARP group" (also referred to as a "TARP area"). In this context, the terms "TARP group" or "TARP area" refer to a defined group of packets protected by a corresponding RDP (identified by the packet's PSN). (Extensions to this technique, where a TARP group is protected by two or more RDPs, will be described further below.)

[0047] Figure 3 This is a schematic illustration of a sequence of data packets for a connection divided into TARP areas according to an embodiment described herein. In this example, packets with PSN {0, 1, 2, ... 7} are defined as TARP area 0, packets with PSN {8, 9, ... 15} are defined as TARP area 1, and so on.

[0048] exist Figure 3 In the example, the TARP areas are all the same size (number of packets), and each TARP area contains a set of consecutive packets in a sequence. Alternatively, the sizes of the TARP areas may vary. TARP areas may include non-consecutive packets with non-consecutive PSNs. The number of packets in each TARP group can be set to any suitable value, such as four or eight.

[0049] In some embodiments, data packets are divided into TARP regions independent of RDMA message boundaries. As above. Figure 2 As shown, the RDMA verbs sublayer is implemented above the transport layer. The TARP sublayer operates at the packet level and is generally independent of the boundaries between RDMA messages. For example, in Figure 3In this context, the packet with PSN=12 is the last packet in the RDMA message, and the packet with PSN=13 is the first packet in the subsequent RDMA message. The boundary between the RDMA messages falls within TARP area 1. Therefore, the RDP for TARP area 1 will be calculated on the packets belonging to both RDMA messages.

[0050] Typically, both the requester 24 and the responder 28 maintain state information 60 for each QP in which TARP is enabled. Typically, both the requester and the responder can concurrently process multiple TARP-enabled QPs using the corresponding states 60 of these QPs.

[0051] In some embodiments, the state 60 of TARP-enabled QP in requester 24 includes the following:

[0052] ■ The number of PSNs in each TARP group.

[0053] ■ The PSN offset for the next redundancy calculation (i.e., the number of remaining data packets to be transmitted before generating the next RDP).

[0054] ■The starting PSN of the current TARP group.

[0055] ■ The number of data packets already included in the current redundant payload.

[0056] ■ (Contents of the current redundant payload in buffer 52)

[0057] The first two items in the list above are global, meaning they are not specific to any given TARP group. The last three items are specific to the TARP group currently being processed. These items may be discarded if necessary, in which case TARP computation for the current TARP group will be halted. For example, if the state 60 of a QP is lost for any reason, no RDP will be generated for the current TARP group, and the TARP will resume at the start of the next TARP group. In one embodiment, some RDPs may be purely informational, i.e., as noted above, sent without redundant payloads.

[0058] In some embodiments, the state 60 of TARP-enabled QP in responder 28 includes the following:

[0059] ■ The number of PSNs in each TARP group.

[0060] ■ PSN offset for the next redundancy calculation.

[0061] ■The starting PSN of the current TARP group.

[0062] ■ The number of packets already included in the current redundant payload.

[0063] ■ (Contents of the current redundant payload in buffer 52)

[0064] In the responder, the first two items in the list above are also global, while the last three items are specific to the currently being processed TARP group. If necessary, group-specific items may be discarded, in which case TARP calculation for the current TARP group will be stopped.

[0065] In some embodiments, TARP is used for RDMA write commands and RDMA send commands, but not for RDMA read commands. In one embodiment, redundancy generator 48 and redundancy aggregator 56 include packets associated with RDMA write or send commands in the current TARP group and exclude packets associated with RDMA read commands from the group. Retransmitted packets are also typically excluded from TARP.

[0066] Requester stream and responder stream

[0067] Requester Stream

[0068] Figure 4 This is a schematic illustration of a flowchart of a method for packet transmission using TARP according to an embodiment described herein. The method is performed by a requester network device (e.g., Figure 1 The requester (24) performs this action. The requester typically repeats this action for each data packet being transmitted. Figure 4 The process. For ease of explanation, Figure 4 The example refers to an implementation where each TARP group uses a single RDP. Alternative implementations using multiple RDPs per TARP group are further described below.

[0069] The method begins at packet pipeline of requester 24 transmitting a data packet to responder 28 via network 32 through port 40 at packet transmission operation 70. As part of the transmission operation, transmission circuitry 44 of requester 24 provides TARP generator 48 with (i) the data packet, (ii) the QP number (QPN) associated with the data packet identifying the connection to which the data packet belongs, and (iii) the PSN of the data packet.

[0070] At TARP enable check operation 74, TARP generator 48 checks whether TARP is enabled or disabled for the QP in question based on the QPN. If TARP is disabled, the method (for that specific packet) ends at termination operation 78.

[0071] If TARP's QP for packets is enabled, the method continues. Since TARP's QP for packets is enabled, the packets are associated with a specific TARP group. Temporary redundant payloads are maintained for that TARP in RDP buffer 52. It is assumed that the redundant payloads are initialized to all zeros before transmitting packets belonging to the TARP group begins.

[0072] At the final packet inspection operation 82, the TARP generator 48 checks whether the packet is the last packet to be transmitted in the TARP group. If the packet is not the last packet in its TARP group, the TARP generator 48 updates the redundant block (redundant payload) of the TARP group in the RDP buffer 52 at update operation 86 to accommodate the packet.

[0073] The update operation typically includes (i) reading the redundant payload from the RDP buffer 52, (ii) calculating a bitwise XOR between the redundant payload and the data packet, and (iii) writing the XOR result as the updated redundant payload into the RDP buffer 52. In the update operation, the TARP generator 48 typically includes the entire data packet, including all headers and payloads, so that the responder can fully reconstruct the data packet.

[0074] Typically, the size of the redundant payload in the RDP buffer 52 is set to the maximum expected packet size. If a packet is smaller than the maximum size, the TARP generator 48 typically sets the remaining bits up to zero to reach the maximum size and then performs a bitwise XOR. If no packet in a given TARP group reaches the maximum size, the TARP generator 48 may set the bits to zero only up to the maximum packet size in the TARP group.

[0075] Now, return to the last packet inspection operation 82. If the packet is found to be the last packet in its TARP group, the TARP generator 48 reads the redundant payload from the RDP buffer 52 at the buffer read operation 90. The TARP generator 48 generates an RDP containing the redundant payload at the RDP generation operation 94 and provides the RDP to the packet pipeline for transmission to the responder.

[0076] In addition to redundant payloads, the TARP generator 48 typically also generates RDP headers for RDP. The RDP header may include any suitable metadata or other information, such as the following:

[0077] ■ Identify the packet as an RDP operation code.

[0078] ■ An indication of whether the RDP contains redundant data. (An RDP without redundant data can be used to signal from the requester to the responder that TARP signaling is enabled for the QP in question. The responder can use this indication to initialize the QP's state 60.)

[0079] ■QP number (QPN).

[0080] ■ The size of the redundant payload used on this QP.

[0081] ■ The size of the redundant payload (i.e., the size of the largest data packet in a TARP group).

[0082] ■ The encoded length of each data packet in a TARP group.

[0083] The parameters above assume that the TARP group spans a continuous PSN sequence. To support TARP groups with non-contiguous PSNs, additional information may need to be added to the RDP header or TARP connection parameters.

[0084] In some embodiments, RDP is formatted as a transport packet (in which case the transport layer should ignore it). In these embodiments, RDP may include a basic transport header (BTH) indicating the QPN and PSN of the RDP. In other embodiments, RDP is formatted as a management datagram (MAD) packet. Alternatively, any other suitable packet format may be used.

[0085] At reset phase 98, TARP generator 48 then resets the redundant payload of the QP in buffer 52 to all zeros, preparing for the next TARP region to process the QP.

[0086] Responder Stream

[0087] Figure 5 This is a schematic illustration of a flowchart of a method for packet reception using TARP according to an embodiment described herein. The method is performed by a responder network device (e.g., Figure 1 The responder (28) performs this action. The responder typically repeats this action for each packet (data packet or RDP) being received. Figure 5 The process is as follows. Figure 4 As shown, for ease of explanation, Figure 5 An unrestricted example refers to an implementation where each TARP group uses a single RDP.

[0088] The method begins with the packet pipeline of responder 28 for receiving packets (data packets or RDPs) from network 32 via port 40 at packet receive operation 100. As part of the receive operation, the transmission circuitry 44 of responder 28 provides TARP aggregator 56 with (i) the packet, (ii) the QPN associated with the packet, and (iii) the PSN of the packet. Based on this information, TARP aggregator 56 may associate the received packet with at most one TARP group, and may not associate it with a TARP group (e.g., if the TARP QP for the packet is not enabled). More generally, TARP aggregator 56 may associate the received packet with a TARP group based on any one or more other suitable identifiers extracted from the received packet.

[0089] At TARP enable check operation 102, TARP aggregator 56 checks whether the QP to which TARP belongs for the package is enabled or disabled based on the QPN. If TARP is disabled, the method (for that specific package) ends at termination operation 104.

[0090] If QP for TARP is enabled for the packet, the method continues. Since QP for TARP is enabled for the packet, the packet is associated with a certain TARP group. Temporary redundant payload is maintained for that TARP in the RDP buffer 52 of responder 28. It is assumed that the redundant payload is initialized to all zeros before receiving packets belonging to the TARP group begins.

[0091] At packet type check operation 108, TARP aggregator 56 checks whether the received packet is a data packet or an RDP packet. If the received packet is a data packet, then at update operation 112, TARP aggregator 56 updates the redundant payload of the TARP group in buffer 52 to accommodate the data packet.

[0092] The update operation typically includes: (i) reading the redundant payload from the RDP buffer 52, (ii) calculating a bitwise XOR between the redundant payload and the data packet, and (iii) writing the XOR result as the updated redundant payload into the RDP buffer 52. In the update operation, the TARP aggregator 56 typically includes the entire data packet, which includes all headers and payload.

[0093] The size of the redundant payload in RDP buffer 52 is typically set to the maximum expected packet size. If the packet size is smaller than the maximum, TARP aggregator 56 typically sets the remaining bits up to the maximum size to zero and then performs a bitwise XOR.

[0094] At packet loss check operation 116, TARP aggregator 56 checks whether exactly a single packet is currently lost (in the TARP group and RDP). In other words, TARP aggregator 56 checks (i) whether all packets in the TARP group in question, except for one packet, have arrived. If none have arrived, the method (for that specific packet) ends at termination operation 120.

[0095] If a single packet is lost, at reconstruction operation 124, TARP aggregator 56 reconstructs the lost packet using redundant payload in buffer 52. When only a single packet is lost, the redundant buffer holds a bitwise XOR of (i) the packets received so far in the RDP and (ii) TARP areas. This bitwise XOR result is equal to the lost packet (possibly with zero padding).

[0096] The TARP aggregator 56 then delivers the recovered data packet to the transmission circuitry 44 in the packet pipeline of the responder. The packet pipeline continues to process the recovered data packet, similar to processing other data packets. As noted above, higher layers are typically unaware that the data packet has been recovered using TARP.

[0097] In some embodiments, even though the lost packets have been recovered, the TARP aggregator 56 in responder 28 notifies requester 24 that packet loss has occurred. This notification, known as a “packet loss acknowledgment,” indicates to the requester that the packet was lost but no retransmission is required. For example, this notification is used to tune congestion control (CC) mechanisms or other mechanisms in the requester. One parameter that can be tuned in the requester’s CC mechanism is the size of the TARP group for the QP in question. When packet drops are infrequent, the TARP group can be set to a larger value (thus reducing network overhead but providing more modest recovery). When packet drops are more frequent, the TARP group can be set to a smaller value (thus increasing network overhead but providing improved recovery).

[0098] In some embodiments, the TARP aggregator 56 waits for a defined period of time before issuing a packet loss acknowledgment notification, in case the packet in question is not lost but merely delayed.

[0099] In one embodiment, TARP aggregator 56 starts a timer at timer start (arm, activate) operation 128 to count defined time intervals. If a lost packet arrives before the timer expires, no notification is sent (or a normal acknowledgment is sent). Otherwise, TARP aggregator 56 sends a packet loss acknowledgment notification to requester 24.

[0100] Now, return to packet type check operation 108. If the received packet is RDP, TARP generator 56 checks at packet loss count operation 132 how many packets have been lost in the TARP group at this time.

[0101] If no packets are lost in the TARP group, no recovery is required, and the TARP aggregator discards the RDP at drop operation 136. If exactly one packet is lost, at recovery operation 140, the TARP aggregator uses the RDP and a temporary redundant payload from the TARP group to recover the lost packet. Typically, TARP aggregator 56 performs a bitwise XOR between the RDP and the redundant payload to produce the lost packet (with possible zero-padding). TARP aggregator 56 delivers the recovered packet to the transmission circuitry 44 in the packet pipeline of responder 28. At timer start operation 144 (similar to operation 128 described above), TARP aggregator 56 starts a timer in preparation for possible future packet loss acknowledgment notifications.

[0102] If two or more packets are lost, the TARP aggregator 56 updates the redundant payload in buffer 52 at redundant update operation 148 to cope with RDP.

[0103] Figure 4 and 5 The requester and responder streams shown are example streams depicted purely for conceptual clarity. In alternative embodiments, requester 24 and responder 28 may use any other suitable streams.

[0104] Each TARP group has multiple RDPs.

[0105] In the embodiments described so far, a single RDP is generated for each TARP area, making it possible to recover individual packets lost in each TARP group. In the alternative embodiments described in this section, the TARP generator 48 in requester 24 generates multiple RDPs for each TARP group, and the TARP aggregator 56 in responder 28 uses the multiple RDPs to recover multiple packets lost in the TARP group.

[0106] In the following description, the number of packets in a TARP group is denoted as k, the number of RDPs in a TARP group is denoted as r, and the total number of packets (data packets and RDPs) is denoted as n, where n = k + r. In these embodiments, once more than k packets out of n = k + r packets have been successfully received, the responder can recover the remaining data packets.

[0107] In these embodiments, the TARP generator 48 maintains r temporary redundant payloads (redundant blocks) for each TARP group in the RDP buffer 52 and updates the redundant payloads as packets are streamed via the packet pipeline of the requester 24. Once the redundant payloads are ready, the TARP generator 48 generates r RDPs, each including the corresponding redundant payload, and delivers the RDPs to the pipeline of the requester 24 for transmission to the responder 28.

[0108] In responder 28, TARP aggregator 56 also maintains r temporary redundant payloads (redundant blocks) for each TARP group in RDP buffer 52. As packets and RDPs are received and streamed via the responder's packet pipeline, TARP aggregator 56 updates the redundant payloads. When at least kr packets out of r RDPs and packets are received, TARP aggregator 56 uses these RDPs and packets to recover the remaining packets in the TARP group. TARP aggregator 56 delivers the recovered packets to the packet pipeline of responder 28.

[0109] Other features of TARP (e.g., enabling / disabling TARP for each individual QP, zero-padding of redundant payloads to accommodate different packet sizes, etc.) are similar to the single RDP implementation described above.

[0110] In some embodiments, r redundant payloads are generated from the generator matrix G of the (n, r) cyclic redundancy check (CRC) code. An example technique for deriving matrix G from the generator polynomial of the CRC can be found in "A Novel Programmable Parallel CRC Circuit" co-authored by Grymel and Furber in IEEE Transactions on Very Large Scale Integration (VLSI) Systems, Vol. 19, No. 10, October 2011.

[0111] For example, consider the generator polynomial x 4 The (12, 8) CRC scheme is derived from +x+1. In this example, each TARP group consists of eight packets (k=8) and four RDPs (r=4), that is, a total of twelve packets (n=12). TARP generator 48 generates the CRC scheme from eight packets (represented as...). Four redundant payloads (denoted as P) are derived from this. 0：3 ):

[0112] P 0:3 =G0:3，0:7 *D 0:7

[0113] in:

[0114]

[0115] The structure of the generator matrix G can be interpreted as follows:

[0116] ■ The i-th row of the matrix (i = 0...3) corresponds to the i-th redundant payload.

[0117] ■ The j-th column of the matrix (j = 0...7) corresponds to the j-th packet.

[0118] ■ Each of the four redundant payloads is constructed by performing an XOR operation on a corresponding distinct subset of the eight data packets.

[0119] ■Matrix element G i,j This indicates whether the i-th redundant payload should be updated to respond to the j-th packet (i.e., whether the i-th redundant payload should be XORed with the j-th packet). If a matrix element equals "1", the redundant payload should be updated. If a matrix element equals "0", the redundant payload should not be updated.

[0120] ■ Equivalently, along the matrix element G in the i-th row of matrix G i,j Indicates the index of the redundant payload that participates in the i-th redundant payload.

[0121] Therefore, in one embodiment, when transmitting the j-th data packet, the TARP generator 48 performs an XOR operation between the j-th data packet and the redundant payload indicated by the j-th column of matrix G. For example, when transmitting the first data packet (corresponding to the leftmost column of matrix G, j=0), the TARP generator 48 updates the redundant payloads 0, 1, and 2, but does not update 3.

[0122] In responder 28, TARP aggregator 56 accumulates four redundant payloads (represented as follows) according to a parity check matrix of the form:

[0123]

[0124] In the context of recovery calculation, the data packets are indexed as 0, 1, ..., 7, while the RDPs are indexed as 8, 9, ..., 11. It can be seen that in the responder, the H matrix has 12 columns for recovery (unlike the 8 columns in the H matrix used to generate the RDPs in the requester).

[0125] For the value v jFor each received data packet index j (0≤j≤11), C 0∶3 ^=H 0：3,j *v j In other words, for each redundant payload i (0 ≤ i ≤ 3), if H i,j If ==1, then c i =c i ^v j For example, when a packet with index 1 is received, the redundant payloads with indices 1, 2, and 3 are updated (because the values ​​of column 1 and rows 1 through 3 in the H matrix are 1).

[0126] When k or more packets (data packets and / or RDPs) of a TARP group are successfully received, the TARP aggregator 56 solves the equation To recover the remaining packets. For example, if the packets with indices 0, 1, and 2 are lost, the above equation simplifies to:

[0127]

[0128] Right now,

[0129]

[0130] The equation can be solved as follows:

[0131]

[0132] In another example scenario, since the equation has no unique solution, the loss of data packets 0, 1, 2, and 7 is irrecoverable.

[0133]

[0134] In some embodiments, the TARP aggregator 56 recovers lost packets by solving a set of linear equations using the following process (e.g., for the example case where three packets with indices 0, 1, and 2 are lost, i.e., L = 3):

[0135] Step 1 Expand matrix H using a 4x4 identity matrix to obtain:

[0136]

[0137] From this point onward, only row swapping and row appending are permitted (implemented as bitwise XOR).

[0138] Step 2 Clear the bottom left corner of H in the expanded matrix: For each row i in [0, 1, ..., L-1] (increasing), perform the following:

[0139] 1. Locate the first "1" in the row: If H i,i == 1, no operation is required. Otherwise, find the row j≥i in which H j,i == 1, and add row j to row i. If this step fails, the solution is not unique and the packet loss is irrecoverable.

[0140] 2. Clear any value "1" in the i-th column of the following rows: For rows i < j ≤ r - 1 == 3, if H j,i == 1, add row i to row j, resulting in:

[0141]

[0142] Step 3 : Clear the upper right corner of matrix H in the expanded matrix: For each row i in [L - 1, L - 2, ……, 1] (in decreasing order), perform the following: For rows 0 ≤ j < i, if H j,i == 1, add row i to row j, resulting in:

[0143]

[0144] Step 4 : The first L rows on the right side of the expanded matrix are the solution:

[0145]

[0146] The encoding schemes, values, RDP generation processes, and packet recovery processes described above are non - restrictive examples chosen purely for clarity. In alternative embodiments, any other suitable encoding schemes, values, RDP generation processes, and / or packet recovery processes may be used.

[0147] Example system use cases

[0148] Figure 6 is a block diagram schematically illustrating a computing system 1000 (e.g., a data center or a high - performance computing (HPC) cluster) according to an embodiment described herein. According to at least one embodiment, system 1000 includes multiple subsystems, such as multiple processing devices, multiple network devices, and multiple networks coupled to each other. The computing system 1000 is designed with multiple integrated circuits (referred to as processing devices), where each integrated circuit may include one or more CPUs and GPUs, thus forming a powerful and flexible architecture.

[0149] Various processing devices are interconnected via NVLink or other high-speed interconnects to enable high-speed communication between subsystems; and are also connected via NICs or DPUs to ensure efficient data transmission across computing system 1000 and to one or more external networks 1030, 1036. In this example, system 1000 includes packet switch 1048 connecting NIC / DPU 1028 to network 1030 and packet switch 1050 connecting NIC / DPU 1032 to network 1036.

[0150] Seamless data exchange and parallel processing are enabled through NVLink coupling of the processing device, thereby improving overall computing performance. The processing device connects to multiple networks via one or more network interface cards (NICs) or DPUs, enabling the system to handle complex multi-network tasks with high bandwidth and low latency. This configuration is ideal for demanding applications requiring significant processing power, such as artificial intelligence (AI), machine learning (ML), and data-intensive computing, while ensuring robust connectivity and scalability across diverse networking environments. The integrated circuits of the computing system 1000 may include one or more CPUs and one or more GPUs.

[0151] Figure 6 An example architecture of a multi-GPU architecture is also demonstrated. As illustrated, the computing system 1000 includes a processing device 1002 with a multi-GPU architecture. Specifically, the processing device 1002 may be a system-on-a-chip and includes multiple subsystems such as a CPU 1006, a GPU 1008, and a GPU 1010. The CPU 1006 may be coupled to the GPU 1008 via a chip-to-chip (D2D) or chip-to-chip (C2C) interconnect 1012 (such as a ground reference signaling interconnect (GRS interconnect)). The CPU 1006 may be coupled to the GPU 1010 via a D2D or C2C interconnect 1014. The CPU 1006 may also be coupled to the GPU 1008 and GPU 1010 via a PCIe interconnect.

[0152] The CPU 1006 can be coupled to one or more NICs or DPUs, which in turn are coupled to one or more networks. For example, as Figure 6 As illustrated, CPU 1006 is coupled to a first NIC / DPU 1026, which is coupled to network 1030. CPU 1006 is also coupled to a second NIC / DPU 1028, which is coupled to network 1030 via switch 1048. For example, NIC / DPU 1026 and NIC / DPU 1028 can be coupled to network 1030 via Ethernet (ETH), NVLINK, or InfiniBand (IB) connections.

[0153] The computing system 1000 also includes a processing device 1004 with a multi-GPU architecture. Specifically, the processing device 1004 includes multiple subsystems, including a CPU 1016, a GPU 1018, and a GPU 1020. The CPU 1016 can be coupled to the GPU 1018 via a D2D or C2C interconnect 1022. The CPU 1016 can be coupled to the GPU 1020 via a D2D or C2C interconnect 1024. The CPU 1016 can also be coupled to the GPU 1018 and GPU 1020 via a PCIe interconnect. The CPU 1016 can be coupled to one or more NICs or DPUs, which in turn are coupled to one or more networks. For example, as... Figure 6 As illustrated, CPU 1016 is coupled to a first NIC / DPU 1032, which in turn is coupled to network 1036. CPU 1016 is also coupled to a second NIC / DPU 1032, which is coupled to network 1036 via switch 1050. NIC / DPU 1032 and NIC / DPU 1034 can be coupled to network 1036 via Ethernet (ETH), NVLINK, or InfiniBand (IB) connections.

[0154] In at least one embodiment, processing device 1002 and processing device 1004 can communicate with each other via NIC / DPU 1038 (such as via PCIe interconnect). Processing device 1002 and processing device 1004 can also communicate with each other via high-bandwidth communication interconnect 1040 (such as NVLink interconnect or other high-speed interconnect).

[0155] In various embodiments, any network device in the network device of system 1000 (e.g., NIC / DPU 1026, 1028, 1032, 1034 and 1038) may use TARP in accordance with the techniques described herein. Figure 6 The packet switch in the diagram can include, for example, an Nvidia Quantum-2 switch. The NIC / DPU in the diagram can include, for example, an Nvidia Bluefield DPU.

[0156] Although the embodiments described herein are primarily for east-west traffic, the methods and systems described herein can also be used for north-south traffic, such as traffic to and from storage devices.

[0157] Therefore, it should be understood that the embodiments described above are illustrated by way of example, and the invention is not limited to the content specifically shown and described above. Rather, the scope of the invention includes combinations and sub-combinations of the various features described above, as well as variations and modifications that may occur to those skilled in the art upon reading the foregoing description and which are not disclosed in the prior art. Documents incorporated herein by reference should be considered part of this application, but if the definition of any term in such incorporated documents conflicts with the express or implied definitions in this specification, the definitions in this specification shall prevail.

Claims

1. A network device, the network device comprising: The port is used to connect to the network; A packet pipeline, the packet pipeline being used to transmit a sequence of data packets to the network via the port; as well as Redundancy generator, the redundancy generator being used for: For a group of packets, (i) one or more redundant blocks are initialized, (ii) the one or more redundant blocks are iteratively updated in response to packets in the group passing through the packet pipeline, and (iii) one or more redundant packets are generated including the one or more redundant blocks; and The packet pipeline is used to transmit one or more redundant packets to the network.

2. The network device as described in claim 1, wherein, The redundancy generator is used to obtain the data packets from the transport layer of the network device.

3. The network device as described in claim 1, wherein, The data packets in the group are associated with a given queue pair (QP) of the network device.

4. The network device as described in claim 1, wherein, The redundancy generator is used to update multiple sets of redundant blocks in multiple corresponding groups of the data packet, including maintaining the corresponding state of the groups.

5. The network device as described in claim 1, wherein, At least two data packets in the group are of different sizes, and the redundancy generator is configured to set the size of the redundancy block to be at least the maximum size of the data packets in the group.

6. The network device as described in claim 1, wherein, The redundancy generator is used to specify the appropriate size of the data packets in the group within the one or more redundancy packets.

7. The network device as described in claim 1, wherein, The redundancy generator is used to include data packets associated with write commands or send commands in the group, and to exclude data packets associated with read commands from the group.

8. The network device as claimed in claim 1, wherein, The one or more redundant blocks include a plurality of redundant blocks, and wherein the redundancy generator is configured to update each of the redundant blocks in response to a corresponding different subset of the data packets in the group.

9. A network device, the network device comprising: The port is used to connect to the network; A packet pipeline for receiving a sequence of packets from the network via the port, the sequence of packets comprising multiple data packets and one or more redundant packets; as well as A redundancy reconstructor, the redundancy reconstructor being configured to perform the following on a group of expected data packets associated with the one or more redundant packets: Initialize one or more redundant blocks; The one or more redundant blocks are iteratively updated to respond to any received redundant packets and any received packets belonging to the group; After updating the redundant blocks to accommodate the one or more redundant packets and all packets in the group except for a specified number of remaining packets, the one or more redundant blocks are used to reconstruct the one or more remaining packets; and One or more reconstructed remaining data packets are provided to the packet pipeline.

10. The network device as claimed in claim 9, wherein, The redundancy reconstructor is used to forward one or more of the reconstructed remaining data packets to the transport layer of the network device.

11. The network device as claimed in claim 9, wherein, The data packets in the group are associated with a given queue pair (QP) of the network device.

12. The network device as claimed in claim 9, wherein, The redundancy rebuilder is used to update multiple sets of redundant blocks in multiple corresponding groups of the data packet, including maintaining the corresponding state of the groups.

13. The network device as claimed in claim 12, wherein, The redundancy reconstructor is used to associate a received data packet with a group of the groups based on one or more identifiers extracted from the data packet.

14. The network device as claimed in claim 9, wherein, The redundancy reconstructor is used for: Extract the size of the corresponding data packet from the group from the one or more redundant packets; as well as The one or more redundant blocks are updated according to the size of the data packet to accommodate the received data packet.

15. The network device as claimed in claim 9, wherein, The one or more redundant blocks include a plurality of redundant blocks, and wherein the redundancy reconstructor is configured to update each of the redundant blocks in response to a corresponding different subset of the data packets in the group.

16. A method, the method comprising: Packet pipelines are used to transmit a sequence of data packets to the network; For a group of packets, (i) one or more redundant blocks are initialized, (ii) the one or more redundant blocks are iteratively updated in response to packets in the group passing through the packet pipeline, and (iii) one or more redundant packets are generated including the one or more redundant blocks; and The packet pipeline is used to transmit one or more redundant packets to the network.

17. The method of claim 16, wherein, The initialization and updating of the redundant blocks and the generation of the redundant packets are performed as part of the transport layer processing in the network device.

18. A method, the method comprising: Using a packet pipeline, a sequence of packets is received from the network, the sequence of packets comprising multiple data packets and one or more redundant packets; as well as Perform the following actions on the group of expected data packets associated with the one or more redundant packets: Initialize one or more redundant blocks; Iteratively update one or more redundant blocks to respond to any received redundant packets and any received packets belonging to the group; After updating the redundant blocks to accommodate the one or more redundant packets and all packets in the group except for a specified number of remaining packets, the one or more redundant blocks are used to reconstruct the one or more remaining packets; and One or more reconstructed remaining data packets are provided to the packet pipeline.

19. The method of claim 18, wherein, The initialization and updating of the redundant blocks and the reconstruction of the remaining data packets are performed as part of the transport layer processing in the network device.

Images