In-network compute operations utilizing encrypted communications

Abstract

To provide a device, method, and program capable of achieving a reproducible result of calculation on floating-point (FP) format data.SOLUTION: A network interface device executes the steps of: forming a communication connection with one or more devices so as to provide an endpoint for a transport layer and security protocols; rearranging the data before decryption of the data on the basis of the reception of packets out of order from one device or multiple devices; decrypting the encrypted data on the basis of the applicable cryptographic protocols; performing calculation as a portion of a series of operations on the basis of the decrypted data to generate a reproducible result; encrypting the result of the calculation on the basis of the applicable cryptographic protocols; and transferring the encrypted data to the device with a packet transferred on the basis of a remote direct memory access or reliable transport protocols.SELECTED DRAWING: Figure 10

Description

[Background technology]

[0001] Machine learning (ML) or high performance computing (HPC) clusters utilize multiple servers and graphics processing units (GPUs), tensor processing units (TPUs) or accelerators. Different switches can perform collective operations on data transmitted over the network. These systems can train ML models using iterative algorithms such as stochastic gradient descent, whereby input data is partitioned among the workers and multiple iterations are run over the training data. In each iteration, the workers compute updates to the ML model parameters based on a subset of the local data and an intermediate current model. The workers communicate these results to be aggregated into a model update, and the aggregate updates are summed over the model parameters at the node for the next iteration. These iterations are run multiple times (epochs) over the entire dataset.

[0002] FIG. 1 illustrates an end-to-end solution for machine learning (ML) training using a PS architecture. A parameter server (PS) can be used for collective operations, whereby worker nodes compute updates and send updates to the PS. The PS pushes aggregated data or data is pulled from the PS server. The PS architecture includes a worker 100 and a parameter server (PS) 120 communicatively coupled using a switch 110. The end-to-end solution of the PS architecture includes reduce-scatter and allgather operators. FIG. 1 illustrates that worker 1 has three queue pairs (QPs), each of which connects to a PS. Worker 2 and worker 3 also utilize three QPs, each of which connects to a PS.

[0003] In the reduce-scatter operator, a worker sends partitions of data to corresponding parameter servers. For example, partition a1 from worker 1, partition a2 from worker 2, and partition a3 from worker 3 are sent to PS1, while partition b1 from worker 1, partition b2 from worker 2, and partition b3 from worker 3 are sent to PS2. A similar pattern applies to PS3. As a result, data is distributed among multiple parameter servers to take advantage of the parallel computation of graphics processing units (GPUs) located in the parameter servers. After receiving data, PS first performs aggregation on the data from the workers.

[0004] In an Allgather operator, the data to be processed by the GPU is multicast to the workers. The parameter server sends the same copy of the data to the workers. In this process, the bandwidth from one PS is distributed to all the workers, and the network can become a bottleneck. [Brief description of the drawings]

[0005] [Figure 1] We present an end-to-end solution for machine learning (ML) training using a parameter server (PS) architecture. [Diagram 2] 1 illustrates an exemplary system. [Figure 3A] 1 illustrates an exemplary switch circuit. [Figure 3B] 1 illustrates an exemplary switch circuit. [Figure 4A] 1 illustrates an exemplary cluster circuit. [Figure 4B] 1 illustrates an exemplary cluster circuit. [Diagram 5] 2 shows an example of a packet format. [Figure 6] 1 shows an example of TLS record alignment with an RDMA packet. [Figure 7] 1 illustrates an exemplary scheme for reordering packets. [Figure 8] 4 illustrates an exemplary accumulation tree for an addition operation. [Figure 9] 1 illustrates an exemplary system. [Figure 10] 1 illustrates an exemplary process. [Figure 11] 1 illustrates an exemplary network interface device or packet processing device. [Figure 12A] 1 illustrates an exemplary switch. [Figure 12B] 1 illustrates an exemplary switch. [Figure 12C] 1 illustrates an exemplary switch. [Figure 13] 1 illustrates an exemplary system. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0006] PS architectures and other neural network architectures (e.g., convolutional neural networks (CNNs), recurrent neural networks (RNNs), transformer-based models, or others) can utilize arithmetic operations involving floating point (FP) format numbers. Because different sequences of FP operations can yield different results, IEEE 754-2019 (Institute of Electrical and Electronics Engineers (IEEE) Standard for Floating-Point Arithmetic standard 754) defines repeatability requirements for sequences of FP operations. In other words, the cumulative result of a list of FP numeric data depends on the order in which they are added and is not associative. However, the order of packets arriving at a network interface device from different sources is not predictable and may arrive out of order. Thus, FP numeric data may arrive at a network interface device out of order.

[0007] Communications between devices in the PS architecture or other neural network architectures can utilize cryptographic protocols to provide security for packets transmitted over computer networks. Secure Sockets Layer (SSL) and Transport Layer Security (TLS) are examples of security protocols. TLS provides end-to-end encryption at the application layer, and TLS can secure inter-application communications. TLS is a widely deployed protocol used to secure transmission control protocol (TCP) connections over the Internet. TLS is defined in at least The Transport Layer Security (TLS) Protocol Version 1.3, RFC 8446 (August 2018).

[0008] Another exemplary cryptographic protocol for secure datagram transport is Datagram Transport Layer Security (DTLS), which is defined at least by Network Working Group Request for Comments (RFC) 4347 (2006) and the Internet Engineering Task Force (IETF) Datagram Transport Layer Security (DTLS) protocol Version 1.3 (2020).

[0009] Another exemplary encryption protocol for secure datagram transport is the PSP Security Protocol (PSP). PSP is a security protocol created by Google for packet encryption. PSP uses the concept of a "Security Association" (SA) to represent a set of traffic that should be processed with a particular set of cryptographic state (keys, initialization vectors (IVs), sequence numbers, etc.).

[0010] To achieve at least reproducible results of operations on the FP data, the network interface device can order the FP data, thereby allowing a sequence of operations to be performed on the FP data to produce results in a reproducible manner, even if the network interface device receives packets carrying the FP data out of order. In some examples, the network interface device can function as both a transport protocol endpoint and a security protocol endpoint for communications with other devices. For example, the network interface device can provide a security protocol endpoint for at least protocols such as DTLS, PSP, TLS, or others. In some examples, the network interface device can track the order of receipt of encrypted data and perform decryption on the encrypted data in order. For example, the FP data is transmitted in encrypted packets, and the network interface device can decrypt the FP data before the accumulation operation. Additionally, the network interface device can encrypt the FP data generated by the accumulation or other arithmetic operations before forwarding the encrypted FP data to other network interface devices or to the host system.

[0011] 2 illustrates an exemplary system. Network interface device 200 can function as both a transport protocol endpoint and a security protocol endpoint for communication with device 210, and can function as both a transport protocol endpoint and a security protocol endpoint for communication with device 220. The system can be part of a PS architecture or a distributed ML or deep neural network (DNN) system trained to perform inference consistent with SwitchML (e.g., Sapio, A., “Scaling distributed machine learning with In-Network aggregation,” 18th USENIX Symposium on Networked Systems Design and Implementation (NSDI 21) (2021)), or can utilize one or more of a Message Passing Interface (MPI), Symmetric Hierarchical Memory Access (SHMEM), Unified Parallel C (UPC), or others. In some examples, devices 210 and 220 can represent workers in a PS environment, and network interface device 200 can perform accumulations on data from devices 210 and 220 in a reproducible or related manner and transmit the results of the accumulations to devices 210 and 220.

[0012] In some examples, the network interface device 200 may include one or more of a network interface controller (NIC), a remote direct memory access (RDMA)-enabled NIC, a SmartNIC, a router, a switch, a forwarding element, an infrastructure processing unit (IPU) with a programmable packet processing pipeline, a data processing unit (DPU), or an edge processing unit (EPU). An edge processing unit (EPU) may include a network interface device that utilizes processors and accelerators (e.g., digital signal processors (DSPs), signal processors, or wireless-specific accelerators for virtualized radio access network (vRAN), cryptographic operations, compression / decompression, etc.).

[0013] For example, device 210 can be implemented as one or more of a network interface device, a host server system, a memory pool, a storage pool, an accelerator pool, or other devices. For example, device 220 can be implemented as one or more of a network interface device, a host server system, a memory pool, a storage pool, an accelerator pool, or other devices. Various examples of host server systems are described with respect to at least FIG.

[0014] In some examples, device 210 can transmit data to network interface device 200, which processes the data for the collective operation. In some examples, device 220 can transmit data to network interface device 200, which processes the data for the collective operation. In some examples, network interface device 200 and device 210 can transmit data by packets encrypted using an encryption protocol. Exemplary encryption protocols can include one or more of DTLS, PSP, TLS, Internet Protocol Security (IPSec), IEEE 802.1AE-2008 (MACsec), or others. In some examples, the encrypted packets can be transmitted using a remote direct memory access (RDMA) protocol. Exemplary RDMA protocols include at least one or more of remote direct memory access (RDMA) over Converged Ethernet (RoCE), RoCEv2, InfiniBand, or others. Various RDMA standards are available, including Network Working Group, "A Remote Direct Memory Access Protocol Specification," Request for Comments 5040 (2007). For example, for ML flows for collective operations, the contents of the RDMA packets (e.g., the header (e.g., one or more header fields or a strict subset of the header) and / or the payload) may be encrypted by DTLS, PSP, or TLS. However, the contents of the RDMA packets of flows carrying data for non-collective operations do not need to be encrypted.

[0015] RDMA can include direct writes or reads to copy buffer contents between connections without the operating system managing the copies. A network interface device can implement direct memory access circuitry to create a channel from its RDMA circuitry through a bus to application memory. A send queue and a receive queue can be used to transfer work requests and are called a queue pair (QP). A requester can place a work request instruction in its work queue that communicates to the interface which buffer's contents to send or which buffer to receive contents from. A work request can include an identifier (e.g., a pointer or memory address of a buffer). For example, a work request placed in a send queue (SQ) can include an identifier of a message or content in a buffer (e.g., an application buffer) to be sent. In contrast, an identifier in a work request in a receive queue (RQ) can include a pointer to a buffer (e.g., an application buffer) where the contents of the incoming message can be stored. An RQ can be used to receive an RDMA-based command or an RDMA-based response. A Completion Queue (CQ) can be used to notify when instructions placed in the work queue are completed. In some examples, a QP can be assigned to the RDMA connection between network interface device 200 and device 210. In some examples, a second QP can be assigned for the connection between network interface device 200 and device 220.

[0016] In some examples, instead of using RDMA, packets can be transmitted using a reliable transport protocol, and the packet contents (e.g., headers and / or payloads) can be encrypted by DTLS, PSP, or TLS. In a reliable transport protocol, the receiver acknowledges packet reception to the data sender, and after a timeout interval, the sender attempts to resend undelivered packets, and / or the sender delays packet transmission based on detected network congestion. Non-limiting examples of reliable transport protocols include Remote Direct Memory Access (RDMA Reliable Connection) in reliable mode, InfiniBand, Transmission Control Protocol (TCP) (e.g., Internet Engineering Task Force (IETF) RFC 793 (1981)), quick UDP Internet Connections (QUIC), RDMA over Converged Ethernet v2 (RoCEv2), Amazon's scalable reliable datagram (SRD), Amazon AWS Elastic Fabric Adapter (EFA), Microsoft Azure Distributed Universal Access (DUA) and Lightweight Transport Layer (LTL), Google GCP Snap Pony Express, High Precision Congestion Control (HPCC) (e.g., Li et al., “HPCC: High Precision Congestion Control” SIGCOMM (2019)), and improved RoCE NIC (IRN) (e.g., Mittal et al., “Revisiting network support for RDMA,” SIGCOMM (2019)). 2018), Homa (e.g., Montazeri et al., "Homa: A Receiver-Driven Low-Latency Transport Protocol Using Network Priorities," SIGCOMM 2018), NDP (e.g., Handley et al., "Re-architecting Datacenter Networks and Stacks for Low Latency and High Performance," SIGCOMM 2017), EQDS (e.g., Olteanu et al., "An edge-queued datagram service for all datacenter traffic," USENIX 2022), or others.

[0017] A flow is a sequence of packets transferred between two endpoints and can generally represent a single session using a known protocol. Thus, a flow can be identified by a defined set of tuples, and for routing purposes, a flow is identified by two tuples that identify the endpoints, e.g., source and destination addresses. For content-based services (e.g., load balancers, firewalls, intrusion detection systems, etc.), flows can be distinguished at a finer granularity by using N-tuples (e.g., source address, destination address, IP protocol, transport layer source port and destination port). Packets in a flow are expected to have the same set of tuples in the packet header. A packet flow can be identified by a combination of a tuple (e.g., Ethernet type field, source and / or destination IP address, source and / or destination User Datagram Protocol (UDP) port, source / destination TCP port, or any other header field) and a unique source and destination queue pair (QP) number or identifier.

[0018] Packet may be used herein to denote various formatted collections of bits that may be transmitted across a network, such as an Ethernet frame, an IP packet, a TCP segment, a UDP datagram, etc. Also, as used in this document, references to the L2, L3, L4 and L7 layers (Layer 2, Layer 3, Layer 4 and Layer 7) are references to the second data link layer, the third network layer, the fourth transport layer and the seventh application layer, respectively, of the Open System Interconnection (OSI) layer model.

[0019] References to flows may alternatively or additionally refer to tunnels (e.g., Multiprotocol Label Switching (MPLS) Label Distribution Protocol (LDP), Segment Routing over IPv6 dataplane (SRv6) source routing, VXLAN tunnel traffic, GENEVE tunnel traffic, virtual local area network (VLAN) based network slices, Mudigonda, Jayaram, et al., “Spain: Cots data-center ethernet for multipathing over arbitrary topologies,” NSDI. Vol. 10. 2010 (hereinafter, “SPAIN”), etc.

[0020] Network interface device 200 may utilize packet processing pipeline 202 to process received packet headers, decrypt packet headers and / or payloads, perform accumulation, calculation, or arithmetic operations on data from received packets, and re-encrypt the results of the accumulation, calculation, or arithmetic operations before transmitting in one or more packets to another device (e.g., a network interface device, a host server system, or other device described herein) or before storing in memory of network interface device 200. Packet processing pipeline 202 may perform accumulation, calculation, or arithmetic operations on received data, including the following: The operations include a multiply-accumulate (MAC) operation (e.g., calculating the product of two numbers and adding the product to an accumulator), a fused multiply-add (FMA), a fused multiply-accumulate (FMAC), a matrix multiplication, a dot product, a general matrix-matrix multiplication (GEMM) operation, sum of packet data with other packet data from other workers, multiplication, division, minimum, maximum, downconversion of 16-bit numbers to 32-bit numbers, downconversion of 32-bit numbers to 16-bit numbers, FP addition, integer (INT) addition, local minimum, local maximum, AND, OR, XOR, bitwise XOR, or other data computation operations related to Allreduce, ReduceScatter, or Allgather.

[0021] The packet processing pipeline 202 can perform accumulation, calculation, or arithmetic operations on data in a variety of formats, including single precision floating point (e.g., 32-bit), half precision floating point (e.g., 16-bit), custom floating point formats (e.g., BF16), integer word (e.g., 16-bit, 8-bit, 4-bit), INT8, INT4, 8-bit binary integer, 4-bit binary integer, 2-bit binary integer, bfloat (brain floating point), 16-bit floating point format, tensor float 32-bit floating point format (TF32) that has a different number of mantissa and exponent bits relative to the Institute of Electrical and Electronics Engineers (IEEE) 754 format, or other formats.

[0022] In some examples, the network interface device 200 can offload computations to a connected server, processor, or accelerator.

[0023] The operation of the packet processing pipeline 202 can be programmed using one or more of a configuration file, OneAPI, Programming protocol independent packet processors (P4), Software for Open Networking in the Cloud (SONiC), Broadcom® Network Programming Language (NPL), NVIDIA® CUDA®, NVIDIA® DOCA™, Data Plane Development Kit (DPDK), Open Data Plane (ODP), Infrastructure Programmer Development Kit (IPDK), eBPF, x86 compatible executable binaries, or other executable binaries.

[0024] Network interface device 200 can utilize communications circuitry 204 to communicate with devices 210 and 220 over a network or fabric via one or more ports. Communications circuitry 204 can be configured to perform such communications using any one or more communications technologies (e.g., wired or wireless communications) and associated protocols (e.g., Ethernet, Bluetooth, Wi-Fi, 4G LTE, 5G, etc.). Communications circuitry 204 can include one or more network hardware resources, such as ingress queues, egress queues, direct memory access (DMA) circuitry, crossbars, shared memory switches, media access control (MAC), physical layer interfaces (PHYs), Ethernet port logic, and other network hardware resources.

[0025] 3A illustrates an exemplary system. The network interface device may include a switch circuit 300 including ingress packet processing pipelines 302-0 to 302-A (where A is an integer), a traffic manager and packet buffer 304, and egress pipelines 306-0 to 306-B (where B is an integer). One or more of the ingress pipelines 302-0 to 302-A or the egress pipelines 306-0 to 306-B may include or be configured to perform both transport protocol endpoints and security protocol endpoints, as well as collective unit circuitry 308 capable of providing accumulation or arithmetic operations for at least FP values ​​in a reproducible manner.

[0026] The collective unit circuitry 308 can perform transport layer management such as managing packet retransmissions, establishing a security session with the device based on DTLS, PSP, TLS or other security protocols, reordering packet data for decryption, performing decryption based on associated security associations, ordering FP data for reproducible operations, performing accumulation, calculation or arithmetic operations on FP data, and performing data encryption of data generated by the accumulation, calculation or arithmetic operations.

[0027] FIG. 3B illustrates an exemplary switch circuit. Ingress pipelines 350-0 through 350-A (where A is an integer) can perform packet routing, switching, and packet integrity checking. After processing by the ingress pipeline, the packets are sent to traffic manager 352, where the packets are enqueued and placed in an output buffer before being provided to the collective processing pipeline. Collective processing pipes 360-0 through 360-B (where B is an integer) can perform transport protocol initiation with devices and provide transport protocol endpoints. After processing by the collective processing pipeline, the packets are sent to traffic manager 362, where the packets are enqueued and placed in an output buffer before being provided to the egress pipeline. Egress pipelines 364-0 through 364-C (where C is an integer) can perform reproducible FP computations.

[0028] 3A and 3B may be implemented as a system on chip (SoC) that includes at least one interface to other circuits in the switch system. The switch SoC may be coupled to other devices in the switch system, such as ingress or egress ports, memory devices, or host interface circuits.

[0029] 4A illustrates an exemplary collective circuit. The collective unit circuit 400 may be implemented as one or more of: one or more processors, one or more programmable packet processing pipelines, one or more accelerators, one or more application specific integrated circuits (ASICs), one or more field programmable gate arrays (FPGAs), one or more graphics processing units (GPUs), one or more memory devices, one or more storage devices, or others.

[0030] The collective unit circuit 400 can utilize a transport layer process 402 to provide a transport layer endpoint for reliable packet delivery and retransmission by the transmitter device. For example, the transport layer process 402 can track receipt of packet sequence numbers and request retransmission of non-received packets by the transmitter network interface device based on non-reception of a sequence number or a range of sequence numbers. The transport layer process 402 can store a connection state for a flow. The connection state can include one or more of an initial packet sequence number, a number of received bytes, a received byte sequence number for keeping received packets in order, an n-tuple (e.g., source address, destination address, IP protocol, transport layer source port and / or destination port), connection flags, or others. The transport layer process 402 can trigger the transmission of a data receipt acknowledgment (ACK) to the transmitter network interface device to indicate receipt of the packet.

[0031] Data reordering for decryption 404 can selectively reorder encrypted data (e.g., records or record segments) received in out-of-order packets prior to decryption of the data by decryption 420. For DTLS encrypted communications, DTLS specifies that a single DTLS record is transmitted in a datagram and includes a record sequence number in the datagram. DTLS encrypted records can be decrypted independently of other DTLS records and out of order of packet reception. Similarly, for PSP encrypted communications, PSP specifies that a single PSP record is transmitted in a datagram and includes a record sequence number in the datagram. PSP encrypted records can be decrypted independently of other PSP records and out of order of packet reception.

[0032] For TLS encrypted communications, a TLS record can be split into multiple packets, and if the packets are received out of order, the record segments from the packets are reordered before decrypting the record. For out-of-order (OOO) packet reception, prior to decrypting the record segments, data reordering for decryption 404 can buffer the record segments in buffer 412 and reorder the record segments as described herein.

[0033] The cryptographic connection manager 406 can establish a DTLS, PSP, TLS, or other cryptographic session when a collective session with a host or network interface device (e.g., devices 210 and 220) is established. A collective flow can have a unique security context that is established when the collective flow is established. Keys can be refreshed periodically (e.g., daily) and security associations (SAs) can be updated accordingly. For a flow, the flow decoder 408 can look up the collective context and security context using a single index value such that a single lookup operation can be performed to access the collective context and security context instead of multiple lookup operations. The security context can include information such as one or more of the specific security operation to be performed, the length of the FP value (e.g., the number of bytes of floating point data in the packet), the calculation or arithmetic operation to be performed, the length of the header, the security key, the nonce, the last byte encrypted or decrypted, the packet sequence number, etc. Different collective flows can have unique security contexts that are established when the collective flow is established. The collective context can include information fields for processing packet segments of the flow by the collective operation. The collective context can include location addresses of intermediate collected values ​​in the buffer 412 or memory, numeric format (e.g., floating point, integer, or other), and calculations or arithmetic operations to be performed on the segments. The collective context and security context can be stored in a data structure in the on-chip memory of the network interface device.

[0034] Data storage for reproducible results 410 can associate data for use in FP operations with nodes of a logical tree structure. Various examples of logical tree identifiers and node assignments described with respect to Figures 8 and 9 can be stored in a collective context. Buffers 412 can store encrypted and / or decrypted packet headers and / or payloads (e.g., data) associated with reproducible and non-reproducible operations. Remote hosts can be assigned to nodes (e.g., leaves) in the tree. Data received in a packet can be assigned to a node and stored in a buffer of buffers 412 associated with the node. For example, a node can be associated with a source IP address, and receipt of a packet with such a source IP address can cause storage of the packet's data in a buffer of buffers 412 associated with the node. As described with respect to Figures 8 and 9, after a packet arrives at the network interface device and is authenticated, data from the packet can be copied to a buffer of buffers 412 associated with a tree leaf associated with the source host, and when the addition operands are stored, operations can be performed and intermediate results can be stored in buffers of buffers 412 at higher levels of leaves in the tree (e.g., parents). The final result may be at the root of the tree.

[0035] To decrypt DTLS encrypted packet content, decryption 420 can utilize Advanced Encryption Standard with Galois / Counter Mode (AES-GCM) based on the accessed security context. Data reordering for decryption 404 can reorder out-of-order received packet data to be decrypted according to DTLS.

[0036] To decrypt PSP encrypted packet contents, decryption 420 can perform key generation or extraction based on AES-GCM to extract a 128-bit or 256-bit long key and apply at least AES-GCM128 or 256 to decrypt the packet contents. For PSP, the decryption key can be derived from the packet, or the complete security association (SA) can be derived from the packet. A main key can be used to derive the SA or decryption key from a Security Parameter Index (SPI) on a per connection or per network interface device basis.

[0037] To decrypt a TLS record, techniques described in at least TLS Protocol Version 1.3, RFC8446 (August 2018), and variants thereof, can be applied by the network interface device after reordering segments of the byte stream.

[0038] The floating-point computation 430 can perform collective operations such as computation or arithmetic operations. Various examples of computation or arithmetic operations are described herein, such as MAC, FMA, FMAC, matrix multiplication, dot product, GEMM operation, sum, multiplication, division, minimum, maximum, or other data computation operations related to Allreduce, ReduceScatter, or Allgather. FP formats can include at least FP8, FP16, FP32, FP64, FPx (x is an integer), INT16, INT32, INT64, UINT16, UINT32, UINT64, BF16, and other formats. In some examples, FP8 can be based on E4M3, E5M2, or others. E4M3 can represent a 4-bit exponent and a 3-bit mantissa, and E5M2 can represent a 5-bit exponent and a 2-bit mantissa.

[0039] The encryption 440 can encrypt the packet header portion and / or the payload based on DTLS, PSP, or TLS.

[0040] FIG. 4B shows an example of an aggregate unit circuit. A packet header vector (PHV) can include one or more packet header fields of a received packet. An instruction lookup (INSTR) can look up an instruction (Inst) (e.g., read, add, store, or other) to execute to process packet payload data (PPV) based on a context from a context random access memory (RAM). Multiple aggregate units can perform arithmetic operations on one or more packet payload data in series or in parallel.

[0041] FIG. 5 shows an example of a packet format. Some portions of the packet header fields can be in the clear (e.g., unencrypted) and can be processed by the network interface device without decryption, while other portions of the packet header fields and the payload can be encrypted and are decrypted before processing by the network interface device. A PSP transport mode packet and a RoCE packet with a PSP header are shown. Additionally, a DTLS packet transmitted over UDP or RoCE is shown. As described in Internet Engineering Task Force (IETF) “The Datagram Transport Layer Security (DTLS) Protocol Version 1.3” Version 1.3 (2020) (draft-ietf-tls-dtls13-38), the DTLS header can include a connection identifier (ID), a sequence number, and a length. A RoCE packet with a TLS header is shown. The packet payload can store one or more FP values.

[0042] Figure 6 shows an example of TLS record alignment with an RDMA packet. TLS records may not include sequence numbers, and both senders and receivers keep track of how many records have been sent and received. Records can be encrypted or decrypted using the implicit record sequence number. The implicit record sequence number can be used as a nonce for AES-GCM operations, and a nonce is a value that must be unique for AES-GCM encryption / decryption operations. Since the nonce is not transmitted with the packet but is calculated independently by the endpoints, TLS records are processed to use the same nonce (implicit record number) for encryption and decryption operations. When TLS records are transmitted over a reliable transport such as TCP, packet reordering can be done based on the packet sequence number, and TLS decryption works on an in-order stream of records. However, RDMA may be transmitted over unreliable User Datagram Protocol (UDP) datagrams. Thus, packets may arrive out of order at the receiver.

[0043] A pair of network interface devices can be configured to operate in mode 600 or 650. In mode 600, a TLS record is transmitted in a single RDMA packet, and the RDMA packet sequence number (PSN) indicates the value of the nonce. For example, RDMA packets 1-3 can contain PSNs 1-3. If the TLS nonce is a 64b (64-bit) value, but the RDMA packet sequence number is 24-bit, then the Supplement to InfiniBand TMThe entire 64b TLS record sequence number can be derived from the 24b RDMA PSN, using the method described in Architecture Specification Volume 1 Release 1.2.1, Annex A17: RoCEv2 (September 2, 2014). In mode 600, TLS records from an RDMA packet can be decrypted or encrypted independently, since the PSN from the packet can be used to reliably derive a nonce value.

[0044] In mode 650, a TLS record spans more than one packet, in this case an RDMA packet, so that an encrypted TLS record can be sent in more than one RDMA packet, so that multiple RDMA packets contain parts of a TLS record (e.g., an RDMA message), but an RDMA packet does not carry more than one TLS record (e.g., the end of one TLS record and the beginning of another TLS record). An RDMA message can contain multiple RDMA packets containing a single TLS record, and the RDMA packets within one RDMA message are encrypted in order. As an endpoint, the network interface device can maintain a bitmap or data to track the state of packet arrivals in one or more flows, and can prevent packets from being processed twice in the case of retransmissions by detecting duplicate packets, or can detect missing RDMA packets and trigger retransmission of missing RDMA packets. As an endpoint, the network interface device (e.g., a packet processing pipeline or aggregation unit) can perform reordering of RDMA packets to reorder TLS record segments. Reliable transport can occur before decrypting the packets, and the RDMA SN can be authenticated without encryption (e.g., in the clear) before being used to reorder the RDMA packets. Various schemes for storing TLS record segments are described herein.

[0045] As described above, the network interface device can perform reproducible floating point operations such that accumulation is performed in a predefined order, regardless of the order of packet arrival, such that streams of floating point numbers from different hosts are processed in a predefined host order, and therefore the results are reproducible. Packet headers and / or payloads can be stored in a buffer while waiting for packet headers and / or payloads from another host to arrive in order to begin the calculation. The reproducible order is the predefined order of operations on data from different hosts, while the flow order is the relative order of packets transmitted by a single host. In some examples, memory used to store data for reproducible operation, and bitmap management that can track the arrival of packets, can be utilized for reproducibility and TLS flow reconstruction.

[0046] FIG. 7 illustrates an exemplary scheme for reordering packets. A bitmap or data structure can track arrival of packets from multiple hosts (e.g., 8 or other number) within a window of multiple packet spans (e.g., 4 or other number). TLS decryption can be performed on TLS records in RDMA PSN order row by row, while accumulation can be performed in host rank order column by column. For example, TLS decryption can be performed on TLS records received via RDMA packets with PSNs numbered 1 through 4. For example, accumulation can be performed in host rank order for the same PSN values ​​(e.g., 0 through 7) of hosts in an accumulation group.

[0047] Packet authentication can be performed based at least on Network Working Group RFC 4302, “IP Authentication Header” (December 2005). For example, parts of the packet header (e.g., the IP header) can be used to authenticate the sender or source of the packet. If packet authentication fails, intermediate buffers with the packet data can be cleared or marked as invalid, an interrupt can be generated, and system level actions can be taken (e.g., stopping the collective operation, restarting the collective operation, identifying a potentially compromised network interface device).

[0048] Next, a scheme for correlating data from pairs of hosts to provide reproducible results from the data is described. FIG. 8 illustrates an exemplary accumulation tree for an addition operation. In some examples, a network interface device can perform a predefined order of operations (((P0+P1)+(P2+P3))+((P4+P5)+(P6+P7))) based on floating point data from received packets P0-P7 received from respective hosts 0-7. However, packets from different hosts may arrive at the network interface device out of order, and for reproducible floating point operations, the addition operation cannot proceed until data for the addition of an operand pair is received.

[0049] For example, for case I, packet P0 arrives before packet P1, and data from packet P0 can be stored while waiting for data from packet P1. After receipt of data from packet P1, an add operation can be performed. For example, for case I, packet P1 arrives before packet P0, and data from packet P1 can be stored while waiting for data from packet P0. After receipt of data from packet P0, an add operation can be performed. For case I, the network interface device packet processing pipeline can execute the instruction store(packet, address) to store data of packet P0 while waiting for data from packet P1, or to store data of packet P1 while waiting for data from packet P0. Similar operations can be performed for data from commutative pairs P2 and P3, P4 and P5, and P6 and P7. Note that although examples are shown for data from pairs of packets, the examples can apply to operations on data from more than two sources.

[0050] For example, for Case II, a second datum of a group of two or more data to be processed together in an accumulation operation is received and stored at the network interface device. For example, the arrival of the last datum in one or more packets in a group of two or more data to be processed together in a commutative accumulation operation can cause retrieval of the stored data of the group, performance of an accumulation operation using the last datum, and overwriting the stored data with the result of the accumulation operation. For example, an instruction ADD(received packet data, sibling data address, parent address) can be executed to retrieve previously stored data, add the received packet data to the stored data, and store the sum in a buffer of previously stored data. An instruction PUSH(ADD(sibling 1 address, sibling 2 address, parent address)) can enqueue an instruction to perform an addition of the pair of data when there is an available cycle.

[0051] For example, for Case III, where the packet processing pipeline or other circuitry has cycles available to process the enqueued instruction, the enqueued instruction can be executed by the packet processing pipeline or other circuitry. For example, the instruction queue can store ADD(sibling 1 address, sibling 2 address, parent address). For example, P0 and P1 are siblings, P2 and P3 are siblings, P4 and P5 are siblings, and P6 and P7 are siblings. Parent A can represent the sum of P0 and P1, and parent B can represent the sum of P2 and P3, making parents A and B siblings. Based on the calculation of B but not A, siblings P0 and P1 can be read from storage and become operands of the instruction ADD(sibling 1 address, sibling 2 address, parent address), such that the sum of P0 and P1 is stored in A's parent address. In some examples, different addresses can be associated with P0 through P7.

[0052] Below is an exemplary sequence of instructions for performing a predetermined order of operations: (((P0+P1)+(P2+P3))+((P4+P5)+(P6+P7))). Note that the instructions can be stored in an instruction queue of a collective unit (e.g., INSTR). Based on the arrival of data from packet P0, a store(P0) can be executed (case I) to store the data from P0. Based on the arrival of data from packet P1, an ADD(P0,P1,P0) can be executed to store the addition of P0 and P1 (parent A) instead of the data from P0 (case II). Based on the arrival of data from packet P4, a store(P4) (case I) is executed to store the data of P4. Based on the arrival of data from packet P5, an add(P4,P5,P4) can be executed to store the addition of P4 and P5 (parent C) instead of the data of P4 (case II).

[0053] Based on the arrival of data from packet P6, execution of store(P6) (case I) can cause the storage of the data of P6. Based on the arrival of data from packet P7, execution of ADD(P6,P7,P6) can cause the storage of the addition of P6 and P7 (parent D) instead of the data from P6 (case II). The instruction Push(ADD(P4,P6)) (case III) can enqueue an addition command to cause the addition of parents C and D to be done in a commutative manner.

[0054] Based on the arrival of data from packet P2, execution of store(P2) can cause the storage of data from P2 (case I). Based on the arrival of data from packet P3, execution of ADD(P2,P3,P2) can cause the addition of P2 and P3 (parent B) and the storage of the result instead of the data of P2 (case II). The instruction Push(ADD(P0,P2,P0)) (case III) can be enqueued to cause the addition of parents A and B and store the result (parent E) instead of parent A in an exchangeable manner.

[0055] The instruction Push(ADD(P0,P4,P0)) (Case III) can be enqueued to cause the addition of parents E and F and store the result (parent G) in place of parent E. The enqueued instructions (add(P4,P6), add(P0,P2,P0), add(P0,P4,P0)) can be dequeued and executed when there is one or more available cycles in the packet processing pipeline.

[0056] FIG. 9 shows an exemplary system. In this example, 512 ranks or buffers are assigned to 512 hosts that provide data to the ranks or buffers for collective operations. However, other numbers of ranks or buffers can be assigned depending on the number of hosts that provide data to the collective operations. For example, referring to the example of FIG. 8, there are eight hosts and eight ranks or buffers can be reserved. For example, referring to the example of FIG. 8, level 1 can store data for leaves P0-P7, level 2 can store values ​​for parents A-D, level 3 can store values ​​for parents E and F, and level 4 can store values ​​for parent G (the root node). At each level, the ranks or buffers can be overwritten with sums to reduce memory usage.

[0057] In some examples, one or more segments can correspond to a single FP value. For example, segment 1 (Seg1) can represent a first segment of the FP number, segment 2 (Seg2) can represent a second segment of the FP number, segment 3 (Seg3) can represent a third segment of the FP number, etc. In some examples, the segments of a packet can be added in order.

[0058] FIG. 10 illustrates an exemplary process that may be performed by a network interface device. At 1002, the network interface device may form a communication connection with one or more devices to provide an endpoint for transport layer and security protocols. For example, the connection may utilize RDMA or a reliable transport protocol and may encrypt the header and / or payload portion based on DTLS, PSP, TLS, or other encryption schemes. The network interface device may terminate the transport layer connection by tracking at least receipt of packet sequence numbers (PSNs) and requesting retransmission of packets associated with unreceived PSNs by the sending network interface device.

[0059] At 1004, based on receipt of out-of-order packets from a device or devices, the network interface device may reorder the data before decrypting the data. For example, if an encrypted record is transmitted and received over multiple packets and the multiple packets are received out of order, the network interface device may reorder the encrypted data from the packets before decrypting the data. However, if the encrypted data is received in order or if the encrypted data can be decrypted without regard to the order in which the packets were received, the network interface device does not need to reorder the encrypted data relative to other encrypted data.

[0060] At 1006, the network interface device can decrypt the encrypted data based on an applicable cryptographic protocol. Exemplary cryptographic protocols include DTLS, PSP, TLS, IPSec, MACsec, or others. In some examples, the network interface device can obtain a context having a security context and a collective context for a packet flow using a single obtainment operation. The network interface device can decrypt the data based on metadata in the security context.

[0061] At 1008, the network interface device can perform a calculation based on the decrypted data. Various examples of calculations are described herein such as MAC, FMA, FMAC, matrix multiplication, dot product, GEMM operations, sum, multiplication, division, minimum, maximum, or other data calculation operations of the packet data with other packet data from other workers. The network interface device can perform the calculation as part of a sequence of operations to produce reproducible results. The network interface device can perform the calculation based on the metadata in the collective context.

[0062] At 1010, the network interface device can encrypt the result of the calculation based on an applicable cryptographic protocol. Exemplary cryptographic protocols include DTLS, PSP, TLS, IPSec, MACsec, or others.

[0063] At 1012, the network interface device can transmit the encrypted data to the device. The network interface device can transmit the encrypted data to the device in packets transmitted based on RDMA or a reliable transport protocol. In some examples, prior to packet transmission, the network interface device can encrypt packet headers and / or payload portions based on DTLS, PSP, TLS, or other encryption schemes.

[0064] FIG. 11 illustrates an exemplary network interface device or packet processing device. In some examples, the circuitry of the network interface device can be utilized to provide transport layer and security protocol endpoints and perform calculations on received data, as described herein. In some examples, the packet processing device 1100 can be implemented as a network interface controller, a network interface card, a host fabric interface (HFI) or a host bus adapter (HBA), and such examples can be interchangeable. The packet processing device 1100 can be coupled to one or more servers using a bus, PCIe, CXL or double data rate (DDR). The packet processing device 1100 can be embodied as part of a system-on-a-chip (SoC) that includes one or more processors, or can be included on a multi-chip package that also includes one or more processors.

[0065] Some examples of packet processing device 1100 are part of or utilized by an infrastructure processing unit (IPU) or data processing unit (DPU). An xPU can refer to at least an IPU, DPU, GPU, GPGPU, or other processing unit (e.g., accelerator device). An IPU or DPU can include a network interface having one or more programmable or fixed function processors to perform offloading of operations that may be performed by a CPU. An IPU or DPU can include one or more memory devices. In some examples, an IPU or DPU can perform virtual switch operations, manage storage transactions (e.g., compression, encryption, virtualization), and manage operations performed on other IPUs, DPUs, servers, or devices.

[0066] The network interface 1100 may include a transceiver 1102, a processor 1104, a transmit queue 1106, a receive queue 1108, a memory 1110, a host interface 1112, and a DMA engine 1152. The transceiver 1102 may transmit and receive packets in accordance with an applicable protocol, such as Ethernet as described in IEEE 802.3, although other protocols may be used. The transceiver 1102 may receive packets from and transmit packets to a network via a network medium (not shown). The transceiver 1102 may include a PHY circuit 1114 and a media access control (MAC) circuit 1116. The PHY circuit 1114 may include encoding and decoding circuitry (not shown) for encoding and decoding data packets in accordance with an applicable physical layer specification or standard. The MAC circuit 1116 may be configured to assemble data to be transmitted into packets that include a destination address and a source address along with network control information and an error detection hash value.

[0067] The processor 1104 may be any combination of processors, cores, graphics processing units (GPUs), field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), or other programmable hardware devices that enable programming of the network interface 1100. For example, a "smart network interface" may use the processor 1104 to provide packet processing capabilities at the network interface.

[0068] The processor 1104, in some embodiments, can include one or more packet processing pipelines that can be configured to perform match-actions on received packets to identify packet processing rules and next hops using information stored in a ternary content-addressable memory (TCAM) table or exact match table. For example, a match action table or circuit can be used whereby a hash of a portion of the packet is used as an index to find an entry. The packet processing pipelines can perform one or more of a packet parser, an exact match action (e.g., small exact match (SEM) engine or large exact match (LEM)), a wildcard match action (WCM), a longest prefix match block (LPM), a hash block (e.g., receive side scaling (RSS)), a packet modifier, or a traffic manager (e.g., transmission rate metering or shaping). For example, the packet processing pipelines can implement packet dropping due to access control lists (ACLs) or queue overflow.

[0069] The configuration of the operation of the processor 1104, including the data plane, can be programmed based on one or more of protocol-independent Packet Processors (P4), Software for Open Networking in the Cloud (SONiC), Broadcom® Network Programming Language (NPL), NVIDIA® CUDA®, NVIDIA® DOCA™, and Infrastructure Programmer Development Kit (IPDK), among others.

[0070] The packet allocator 1124 can provide distribution of received packets for processing by multiple CPUs or cores using receive side scaling (RSS). When the packet allocator 1124 uses RSS, the packet allocator 1124 can calculate a hash or make another decision based on the content of the received packet to determine which CPU or core should process the packet.

[0071] Interrupt coalescing 1122 can perform interrupt arbitration, whereby network interface interrupt coalescing 1122 waits for multiple packets to arrive or for a timeout to expire before generating an interrupt to the host system to process the received packet. Receive Segment Coalescing (RSC) is performed by network interface 1100, whereby portions of incoming packets are combined into segments of packets. Network interface 1100 provides the coalesced packets to the application.

[0072] A direct memory access (DMA) engine 1152 can copy packet headers, packet payloads and / or descriptors directly from the host memory to the network interface, or vice versa, instead of copying the packet to an intermediate buffer in the host and then using another copy operation from the intermediate buffer to the destination buffer.

[0073] The memory 1110 may be any type of volatile or non-volatile memory device and may store any queues or instructions used to program the network interface 1100. The transmit queues 1106 may contain data or references to data for transmission by the network interface. The receive queues 1108 may contain data or references to data received by the network interface from the network. The descriptor queues 1120 may contain descriptors that reference data or packets in the transmit queues 1106 or receive queues 1108. The host interface 1112 may provide an interface with a host device (not shown). For example, the host interface 1112 may be compatible with a PCI, PCI Express, PCI-x, Serial ATA, and / or USB compatible interface (although other interconnect standards may be used).

[0074] FIG. 12A illustrates an exemplary switch. Various implementations may be used within or in conjunction with the switch to provide transport layer and security protocol endpoints and perform computations on received data, as described herein. Switch 1204 may route packets or frames of any format or conforming to any specification from any of ports 1202-0 to 1202-X to any of ports 1206-0 to 1206-Y (or vice versa). Any of ports 1202-0 to 1202-X may be connected to one or more networks of interconnected devices. Similarly, any of ports 1206-0 to 1206-Y may be connected to one or more networks of interconnected devices.

[0075] In some examples, switch fabric 1210 can provide routing of packets from one or more ingress ports for processing before exiting switch 1204. Switch fabric 1210 can be implemented as one or more multi-hop topologies, with example topologies including torus, butterfly, buffered multi-stage, etc., or a shared memory switch fabric (SMSF), among other implementations. An SMSF can be any switch fabric connected to ingress and egress ports within a switch, where an ingress subsystem writes (stores) packet segments to the fabric's memory, while an egress subsystem reads (fetches) packet segments from the fabric's memory.

[0076] The memory 1208 can be configured to store packets received at one or more ports before they exit the port. The packet processing pipeline 1212 can include ingress and egress packet processing circuits for processing ingress packets and packets to be egressed, respectively. The packet processing pipeline 1212 can determine which port to forward a packet or frame to using a table that maps packet characteristics to an associated output port. The packet processing pipeline 1212 can be configured to perform match actions on received packets to identify packet processing rules and next hops, in some examples, using information stored in a ternary content-addressable memory (TCAM) table or an exact match table. For example, a match action table or circuit can be used whereby a hash of a portion of the packet is used as an index to find an entry (e.g., a forwarding decision based on packet header content). The packet processing pipeline 1212 can implement packet dropping due to access control lists (ACLs) or queue overflow. The packet processing pipeline 1212 can be configured to provide transport layer and security protocol endpoints and perform calculations on received data, as described herein. The configuration of the operation of the packet processing pipeline 1212, including its data plane, can be programmed using P4, C, Python, Broadcom Network Programming Language (NPL), or x86 compatible executable binaries, or other executable binaries. The processor 1216 and FPGA 1218 can be used for packet processing or modification.

[0077] Traffic manager 1213 can perform hierarchical scheduling, and transmission rate shaping and metering of packet transmissions from one or more packet queues. Traffic manager 1213 can perform congestion management such as flow control, congestion notification message (CNM) generation and reception, priority flow control (PFC), and others.

[0078] FIG. 12B illustrates an exemplary network forwarding system that can be used as a network interface device or router. The forwarding system can provide transport layer and security protocol endpoints and perform computations on received data, as described herein. For example, FIG. 12B illustrates several ingress pipelines 1220, a traffic management unit (referred to as a traffic manager) 1250, and several egress pipelines 1230. Although shown as separate structures, in some examples, the ingress pipelines 1220 and the egress pipelines 1230 can use the same circuit resources. In some examples, the egress pipelines 1230 can perform the operations of a collective unit circuit, as described herein.

[0079] The operation of the pipelines can be programmed using Programming Protocol-independent Packet Processor (P4), C, Python, Broadcom NPL, or x86 compatible executable binaries, or other executable binaries. In some examples, the pipeline circuitry is configured to synchronously process ingress and / or egress pipeline packets as well as non-packet data. That is, a particular stage of the pipeline can process any combination of ingress packets, egress packets, and non-packet data in the same clock cycle. However, in other examples, the ingress and egress pipelines are separate circuits. In some of these other examples, the ingress pipeline also processes non-packet data.

[0080] In some examples, in response to receiving a packet, the packet is directed to one of the ingress pipelines 1220, which may correspond to one or more ports of a hardware forwarding element. After passing through the selected ingress pipeline 1220, the packet is sent to the traffic manager 1250, where the packet is enqueued and placed in the output buffer 1254. In some examples, the ingress pipeline 1220 that processes the packet specifies into which queue the packet should be placed by the traffic manager 1250 (e.g., based on the packet's destination or the packet's flow identifier). The traffic manager 1250 then dispatches the packet to the appropriate egress pipeline 1230, which may correspond to one or more ports of a forwarding element. In some examples, there is no necessary correlation between which of the ingress pipelines 1220 processes the packet and which of the egress pipelines 1230 the traffic manager 1250 dispatches the packet to. That is, after a packet is received through a first port, it may be first processed by ingress pipeline 1220b, and then processed by egress pipeline 1230a for transmission out a second port, and so on.

[0081] At least one ingress pipeline 1220 includes a parser 1222, a chain of multiple match-action units or circuitry (MAUs) 1224, and a deparser 1226. Similarly, an egress pipeline 1230 can include a parser 1232, a chain of MAUs 1234, and a deparser 1236. The parser 1222 or 1232, in some examples, receives a packet as a formatted collection of bits in a particular order and parses the packet into its constituent header fields. In some examples, the parser starts at the beginning of the packet and assigns the header fields to fields (e.g., data containers) for processing. In some examples, the parser 1222 or 1232 separates the packet header (up to a specified point) from the packet's payload and sends the payload (or the entire packet, including header and payload) directly to the deparser without going through MAU processing. The egress parser 1232 can use additional metadata provided by the ingress pipeline to simplify its processing.

[0082] MAU 1224 or 1234 can perform operations on the packet data. In some examples, the MAU includes a series of stages, each stage including one or more match tables and action engines. The match table can include a set of match entries against which packet header fields are matched (e.g., using a hash table), and the match entries reference action entries. When a packet matches a particular match entry, the particular match entry references a particular action entry that specifies a set of actions to be performed on the packet (e.g., sending the packet to a particular port, modifying one or more packet header field values, dropping the packet, mirroring the packet in a mirror buffer, etc.). The action engine of the stage can perform the action on the packet, and the packet is then sent to the next stage of the MAU. For example, the MAU can provide transport layer and security protocol endpoints and perform computations on the received data, as described herein.

[0083] The deparser 1226 or 1236 can reconstruct a packet using the PHV modified by the MAU 1224 or 1234 and the payload received directly from the parser 1222 or 1232. The deparser can construct a packet that can be sent out on the physical network or to a traffic manager 1250. In some examples, the deparser can construct this packet based on the data received along with the PHV that specifies the protocol to include in the packet header and its own stored list of data container locations for each header field of the possible protocols.

[0084] Traffic manager (TM) 1250 can include a packet replicator 1252 and an output buffer 1254. In some examples, traffic manager 1250 may include other components, such as a feedback generator for sending a signal regarding an output port failure, a set of queues and a scheduler for these queues, a queue status analysis component, and further components. Packet replicator 1252 in some examples performs duplication of broadcast / multicast packets and generates multiple packets to be added to the output buffer (e.g., distributed to different egress pipelines).

[0085] The output buffer 1254 may be part of a traffic manager queuing and buffering system in some examples. The traffic manager 1250 may provide a shared buffer that corresponds to any queuing delays in the egress pipelines. In some examples, this shared output buffer 1254 may store packet data, and references (e.g., pointers) to the packet data are kept in different queues for each egress pipeline 1230. The egress pipelines may request their respective data from the common data buffer using control plane configurable queuing policies. When a packet data reference reaches the head of its queue and is scheduled for dequeuing, the corresponding packet data may be read from the output buffer 1254 and placed into the corresponding egress pipeline 1230.

[0086] FIG. 12C illustrates an exemplary switch. Various implementations may be used within or with the switch to provide transport layer and security protocol endpoints and perform calculations on received data, as described herein. The switch 1280 may include a network interface 1280 that may provide an Ethernet-compatible interface. The network interface 1280 may support 25 GbE, 50 GbE, 100 GbE, 200 GbE, and 400 GbE Ethernet port interfaces. The encryption circuitry 1284 may perform at least Media Access Control security (MACsec) or Internet Protocol Security (IPSec) decryption for received packets or encryption for packets to be transmitted.

[0087] The various circuits can perform one or more of service metering, packet counting, operations, administration, and management (OAM), protection engine, instrumentation and telemetry, and clock synchronization (e.g., based on IEEE 1588).

[0088] The database 1286 can store device profiles to configure the operation of the switch 1280. The memory 1288 can include a High Bandwidth Memory (HBM) for packet buffering. The packet processor 1290 can perform one or more of the following related to packet forwarding: next hop determination, packet counting, access list operation, bridging, routing, Multiprotocol Label Switching (MPLS), virtual private LAN service (VPLS), L2VPN, L3VPN, OAM, data center tunneling encapsulation (e.g., VXLAN and NV-GRE), or others. The packet processor 1290 can include one or more FPGAs. The buffer 1294 can store one or more packets. The traffic manager (TM) 1292 can provide per-subscriber bandwidth guarantees according to service level agreements (SLAs) and can perform hierarchical quality of service (QoS). The fabric interface 1296 may include a serializer / deserializer (SerDes) and provide an interface to a switch fabric.

[0089] The operations of the components of the example switches of Figures 12A, 12B, and / or 12C can be combined, and components of the example switches of Figures 12A, 12B, and / or 12C can be included in other examples of the switches of Figures 12A, 12B, and / or 12C. For example, the components of the example switches of Figures 12A, 12B, and / or 12C can be implemented in a switch system on chip (SoC) that includes at least one interface to other circuits in the switch system. The switch SoC can be coupled to other devices in the switch system, such as ingress or egress ports, memory devices, or host interface circuits.

[0090] FIG. 13 illustrates a system. In some examples, the circuitry can provide transport layer and security protocol endpoints and perform calculations on received data, as described herein. System 1300 includes a processor 1310 that provides processing, operational management, and execution of instructions for system 1300. Processor 1310 can include any type of microprocessor, central processing unit (CPU), graphics processing unit (GPU), XPU, processing core, or other processing hardware that provides processing for system 1300, or a combination of processors. An XPU can include one or more of a CPU, graphics processing unit (GPU), general purpose GPU (GPGPU), and / or other processing units (e.g., accelerators or programmable or fixed function FPGAs). Processor 1310 controls the overall operation of system 1300 and can be or include one or more programmable general-purpose or special-purpose microprocessors, digital signal processors (DSPs), programmable controllers, application specific integrated circuits (ASICs), programmable logic devices (PLDs), etc., or a combination of such devices.

[0091] In one example, the system 1300 includes an interface 1312 coupled to the processor 1310, which may represent a higher speed or high throughput interface for system components that require a higher bandwidth connection, such as the memory subsystem 1320 or the graphics interface component 1340 or the accelerator 1342. The interface 1312 may be a standalone component or may represent an interface circuit that may be integrated on a processor die. If present, the graphics interface 1340 interfaces to a graphics component for providing a visual display to a user of the system 1300. In one example, the graphics interface 1340 may drive a display that provides output to the user. In one example, the display may include a touch screen display. In one example, the graphics interface 1340 generates a display based on data stored in the memory 1330, or based on operations performed by the processor 1310, or both. In one example, the graphics interface 1340 generates a display based on data stored in the memory 1330, or based on operations performed by the processor 1310, or both.

[0092] The accelerators 1342 may be programmable or fixed function offload engines that can be accessed or used by the processor 1310. For example, one of the accelerators 1342 may provide a data compression (DC) function, a cryptographic service such as public key encryption (PKE), a cipher, a hash / authentication function, decryption, or other function or service. In some cases, the accelerator 1342 may be integrated into a CPU socket (e.g., a connector to a motherboard or circuit board that contains a CPU and provides an electrical interface with the CPU). For example, the accelerator 1342 may include a single-core processor or a multi-core processor, a graphics processing unit, a logical execution unit single-level cache or a logical execution unit multi-level cache, a functional unit that can be used to independently execute a program or thread, an application specific integrated circuit (ASIC), a neural network processor (NNP), programmable control logic, and a programmable processing element such as a field programmable gate array (FPGA). The accelerator 1342 can provide multiple neural networks, CPUs, processor cores, general-purpose graphics processing units, or the graphics processing units can be made available for use by artificial intelligence (AI) or machine learning (ML) models. For example, the AI ​​models can use or include any or combination of reinforcement learning schemes, Q-learning schemes, deep Q-learning, or Asynchronous Advantage Actor-Critic (A3C), combinatorial neural networks, recurrent combinatorial neural networks, or other AI or ML models.Multiple neural networks, processor cores, or graphics processing units can be made available for use by the AI ​​or ML models to perform training and / or inference operations. Exemplary accelerators 1342 include GPUs, TPUs, and Amazon Web Services Trainium.

[0093] The memory subsystem 1320 represents the main memory of the system 1300 and provides storage for code to be executed by the processor 1310 or data values ​​used in executing routines. The memory subsystem 1320 may include one or more memory devices 1330, such as read-only memory (ROM), flash memory, one or more of a variety of random access memories (RAM), such as DRAM, or other memory devices, or a combination of such devices. The memory 1330 stores and hosts, among other things, an operating system (OS) 1332 to provide a software platform for the execution of instructions within the system 1300. Additionally, applications 1334 may execute on the software platform of the OS 1332 from the memory 1330. The applications 1334 represent programs having their own operating logic for carrying out the execution of one or more functions. The processes 1336 represent agents or routines that provide auxiliary functions to the OS 1332 or one or more applications 1334 or combination. OS 1332, applications 1334, and processes 1336 provide the software logic that provides the functionality of system 1300. In one example, memory subsystem 1320 includes memory controller 1322, which is a memory controller that generates and issues commands to memory 1330. It is understood that memory controller 1322 may be a physical part of processor 1310 or a physical part of interface 1312. For example, memory controller 1322 may be an integrated memory controller integrated on circuitry with processor 1310.

[0094] Application 1334 and / or process 1336 may alternatively or additionally represent a virtual machine (VM), a container, a microservice, a processor, or other software. Various examples described herein may execute applications composed of microservices, where the microservices execute in their own processes and communicate using protocols (e.g., application program interfaces (APIs), Hypertext Transfer Protocol (HTTP) resource APIs, message services, remote procedure calls (RPCs), or Google RPC (gRPC)). The microservices may communicate with each other using a service mesh and execute in one or more data centers or edge networks. The microservices may be deployed independently with centralized management of these services. The management system may be written in different programming languages ​​and use different data storage technologies. Microservices may be characterized by one or more of polyglot programming (e.g., code written in multiple languages ​​to capture additional functionality and efficiencies not available in a single language), or lightweight container or virtual machine deployment, and distributed continuous microservice delivery.

[0095] In some examples, OS 1332 may be a Linux®, Windows® server or personal computer, FreeBSD®, Android®, MacOS®, iOS®, VMware vSphere, openSUSE, RHEL, CentOS, Debian, Ubuntu, or any other operating system. The OS and drivers may run on processors sold or designed by Intel®, ARM®, AMD®, Qualcomm®, IBM®, Nvidia®, Broadcom®, Texas Instruments®, among others.

[0096] Although not specifically shown, it is understood that the system 1300 can include one or more buses or bus systems between devices, such as a memory bus, a graphics bus, an interface bus, or others. A bus or other signal lines can communicatively or electrically couple components, or can communicatively and electrically couple components. A bus can include physical communication lines, point-to-point connections, bridges, adapters, controllers, or other circuits, or combinations thereof. A bus can include, for example, one or more of a system bus, a Peripheral Component Interconnect (PCI) bus, a HyperTransport or industry standard architecture (ISA) bus, a small computer system interface (SCSI) bus, a universal serial bus (USB), or an Institute of Electrical and Electronics Engineers (IEEE) standard 1394 bus (Firewire).

[0097] In one example, the system 1300 includes an interface 1314 that can be coupled to the interface 1312. In one example, the interface 1314 represents an interface circuit that can include standalone components and integrated circuits. In one example, multiple user interface components or peripheral components, or both, are coupled to the interface 1314. The network interface 1350 provides the system 1300 with the ability to communicate with remote devices (e.g., servers or other computing devices) over one or more networks. The network interface 1350 can include an Ethernet adapter, a wireless interconnection component, a cellular network interconnection component, a Universal Serial Bus (USB), or other wired or wireless standards-based or proprietary interface. The network interface 1350 can transmit data to a device in the same data center or rack or to a remote device, which can include transmitting data stored in the memory. The network interface 1350 can receive data from a remote device, which can include storing the received data in the memory. In some examples, the packet processing device or network interface device 1350 may represent one or more of a network interface controller (NIC), a remote direct memory access (RDMA) capable NIC, a SmartNIC, a router, a switch, a forwarding element, an infrastructure processing unit (IPU), or a data processing unit (DPU). An example IPU or DPU is described with respect to FIG.

[0098] In one example, system 1300 includes one or more input / output (I / O) interfaces 1360. I / O interface 1360 can include one or more interface components through which a user interacts with system 1300. Peripheral interface 1370 can include any hardware interface not specifically mentioned above. Peripherals generally refer to devices that connect depending on system 1300.

[0099] In one example, the system 1300 includes a storage subsystem 1380 for storing data in a non-volatile manner. In one example, in certain system implementations, at least certain components of the storage 1380 can be duplicated with components of the memory subsystem 1320. The storage subsystem 1380 includes a storage device 1384 that is or can include any conventional medium for storing large amounts of data in a non-volatile manner, such as one or more magnetic, solid state, or optical based disks, or a combination. The storage 1384 holds code or instructions and data 1386 in a persistent state (e.g., values ​​are retained despite interruption of power to the system 1300). The storage 1384 can be generally considered to be "memory," while the memory 1330 is typically an execution or operating memory for providing instructions to the processor 1310. The storage 1384 is non-volatile, while the memory 1330 can include volatile memory (e.g., the value or state of the data is indeterminate if power is interrupted to the system 1300). In one example, storage subsystem 1380 includes a controller 1382 that interfaces with storage 1384. In one example, controller 1382 may be a physical part of interface 1314 or processor 1310, or may include circuitry or logic in both processor 1310 and interface 1314.

[0100] Volatile memory is memory whose state (and therefore the data stored in it) is indeterminate if power to the device is removed. A non-volatile memory (NVM) device is memory whose state is deterministic even if power to the device is removed.

[0101] In one example, the system 1300 can be implemented using interconnected computer threads of processors, memory, storage, network interfaces, and other components. High speed interconnects include Ethernet (IEEE802.3), remote direct memory access (RDMA), InfiniBand, Internet Wide Area RDMA Protocol (iWARP), Transmission Control Protocol (TCP), User Datagram Protocol (UDP), quick UDP Internet Connections (QUIC), RDMA over Converged Ethernet (RoCE), Peripheral Component Interconnect express (PCIe), Intel QuickPath Interconnect (QPI), Intel Ultra Path Interconnect (UPI), Intel On-Chip System Fabric (IOSF), Omni-Path, Compute Express Link (CXL), HyperTransport, high speed fabrics, NVLink, Advanced Microcontroller Bus Architecture (AMBA) interconnects, OpenCAPI, Gen-Z, Infinity Fabric (IF), Cache Coherent Interconnect for The LTE standard may be based on 3GPP® 5G, LTE Accelerators, 3GPP® Long Term Evolution (4G), 3GPP® 5G, and variations thereof.Data may be copied or stored on virtualized storage nodes or may be accessed using protocols such as NVMe-oF (NVMe over Fabrics) or NVMe (e.g., non-volatile memory express (NVMe) devices capable of operating in a manner consistent with the Non-Volatile Memory Express (NVMe) Specification, revision 1.3c, published on May 24, 2018 (the "NVMe Specification"), or any derivative or variation thereof).

[0102] Communication between devices can occur using a network that provides die-to-die, chip-to-chip, circuit board-to-board, and / or package-to-package communication.

[0103] In one example, the system 1300 can be implemented using interconnected computer threads of processors, memory, storage, network interfaces, and other components. High speed interconnects such as PCIe, Ethernet, or optical interconnects (or combinations thereof) can be used.

[0104] The examples herein may be implemented with various types of computing and networking equipment such as switches, routers, rack and blade servers, such as those used in data center and / or server farm environments. Servers used in data centers and server farms include arrayed server configurations such as rack-based servers or blade servers. These servers are interconnected in communication via various network facilities, such as dividing sets of servers into Local Area Networks (LANs) with appropriate switching and routing capabilities between the LANs to form private intranets. For example, cloud hosting facilities may typically use large data centers with multiple servers. Thus, the blades include components common to conventional servers, including a main printed circuit board (main board) that provides internal wiring (e.g., buses) for coupling appropriate integrated circuits (ICs) and other components mounted on the board.

[0105] Various examples may be implemented using hardware elements, software elements, or a combination of both. In some examples, hardware elements may include devices, components, processors, microprocessors, circuits, circuit elements (e.g., transistors, resistors, capacitors, inductors, etc.), integrated circuits, ASICs, PLDs, DSPs, FPGAs, memory units, logic gates, registers, semiconductor devices, chips, microchips, chipsets, etc. In some examples, software elements may include software components, programs, applications, computer programs, application programs, system programs, machine programs, operating system software, middleware, firmware, software modules, routines, subroutines, functions, methods, procedures, software interfaces, APIs, instruction sets, computing code, computer code, code segments, computer code segments, words, values, symbols, or any combination thereof. The decision of whether an example is implemented using hardware elements and / or software elements may vary according to any number of factors, such as desired computation rates, power levels, thermal tolerances, processing cycle budgets, input data rates, output data rates, memory resources, data bus speeds, and other design or performance constraints, as desired for a given implementation. A processor may be a hardware state machine, digital control logic, a central processing unit, or a combination of one or more of any hardware, firmware, and / or software elements.

[0106] Some examples may be implemented using or as an article of manufacture or at least one computer readable medium. The computer readable medium may include a non-transitory storage medium that stores logic. In some examples, the non-transitory storage medium may include one or more types of computer readable storage media capable of storing electronic data, including volatile or non-volatile memory, removable or non-removable memory, erasable or non-erasable memory, writable or rewritable memory, etc. In some examples, the logic may include various software elements, such as software components, programs, applications, computer programs, application programs, system programs, machine programs, operating system software, middleware, firmware, software modules, routines, subroutines, functions, methods, procedures, software interfaces, APIs, instruction sets, computing code, computer code, code segments, computer code segments, words, values, symbols, or any combination thereof.

[0107] According to some examples, a computer-readable medium may include a non-transitory storage medium for storing or maintaining instructions that, when executed by a machine, computing device, or system, cause the machine, computing device, or system to perform methods and / or operations according to described examples. The instructions may include any suitable type of code, such as source code, compiled code, interpreted code, executable code, static code, dynamic code, etc. The instructions may be implemented according to a predefined computer language, manner, or syntax to instruct a machine, computing device, or system to perform a particular function. The instructions may be implemented using any suitable high-level, low-level, object-oriented, visual, compiled, and / or interpreted programming language.

[0108] One or more aspects of at least one example may be implemented by representative instructions stored on at least one machine-readable medium that represent various logic within a processor, which, when read by a machine, computing device, or system, cause the machine, computing device, or system to produce logic for performing the techniques described herein. Such representations, known as "IP cores," may be stored on tangible machine-readable media and delivered to various customers or facilities to be loaded into manufacturing machines that actually create the logic or processors.

[0109] The appearances of the phrases "an example" or "example" do not necessarily all refer to the same example or embodiment. Any aspect described herein may be combined with any other aspect described herein or with similar aspects, regardless of whether the aspects are described with respect to the same figure or element. The division, omission, or inclusion of block functions shown in the accompanying figures does not necessarily mean that hardware components, circuits, software, and / or elements for implementing those functions are necessarily divided, omitted, or included in the embodiments.

[0110] Some examples may be described using the terms "coupled" and "connected," along with their derivatives. These terms are not necessarily intended as synonyms for each other. For example, a description using the terms "connected" and / or "coupled" may indicate that two or more elements are in direct physical or electrical contact with each other. However, the term "coupled" may also mean that two or more elements are not in direct contact with each other, but still co-operate or interact with each other.

[0111] The terms "first", "second", etc. herein do not denote any order, quantity, or importance, but rather are used to distinguish one element from another. The singular forms "a", "an", "an" and "the" do not denote a limitation of quantity, but rather the presence of at least one of the referenced items. The term "asserted" as used herein with respect to a signal indicates a state of the signal where the signal is active and may be achieved by applying any logic level, either logic 0 or logic 1, to the signal. The terms "following" or "after" may indicate immediately after or after some other event. Other sequences of operations may also be performed according to alternative embodiments. Furthermore, additional operations may be added or removed depending on the particular application. Any combination of changes may be used, and those of ordinary skill in the art having the benefit of this disclosure will recognize numerous variations, modifications, and alternative embodiments thereof.

[0112] Alternative language, such as the phrase "at least one of X, Y, or Z," is otherwise understood in context as being generally used to indicate that an item, term, etc. may be either X, Y, or Z, or any combination thereof (e.g., X, Y, and / or Z), unless specifically stated otherwise. Thus, such alternative language is generally not intended to, and should not, imply that a particular embodiment requires that at least one of X, at least one of Y, or at least one of Z, respectively, be present. Furthermore, conjunctive language, such as the phrase "at least one of X, Y, and Z," should also be understood to mean X, Y, Z, or any combination thereof, including "X, Y, and / or Z," unless specifically stated otherwise.

[0113] Illustrative examples of the devices, systems, and methods disclosed herein are provided below. Embodiments of the devices, systems, and methods may include any one or more, and any combination, of the examples described below.

[0114] Example 1 includes one or more examples, including an apparatus including an interface and a circuit coupled to the interface, the circuit configured to provide an endpoint for a Datagram Transport Layer Security (DTLS) connection with a first network interface device by decrypting DTLS encrypted data from packets received from the first network interface device, provide an endpoint for a second DTLS connection with a second network interface device by decrypting DTLS encrypted data from packets received from the second network interface device, provide a transport layer endpoint for packets received from the first network interface device, and provide a second transport layer endpoint for packets received from the second network interface device, where the packets received from the first and second network interface devices provide data for an in-network computation operation, reorder the packets received from the first and second network interface devices based on the packets received out of order from the first and second network interface devices, perform a reproducible in-network computation operation based on a floating point (FP) format on the reordered data from the reordered packets, and perform DTLS encryption of data generated by the in-network computation operation on the reordered data prior to transmission.

[0115] Example 2 includes one or more examples in which packets received from the first network interface device are received in a remote direct memory access (RDMA) based manner.

[0116] Example 3 includes one or more examples in which the circuit reorders packets received from the first and second network interface devices based on record sequence numbers in the received packets.

[0117] Example 4 includes one or more examples where the circuit includes an ingress packet processing pipeline, a traffic manager, and an egress packet processing pipeline, where the egress packet processing pipeline performs in-network computation operations on reordered data from the reordered packets based on the FP format.

[0118] Example 5 includes one or more examples in which the egress packet processing pipeline performs decryption of the DTLS encrypted data prior to performance of in-network computation operations on the reordered data.

[0119] Example 6 includes one or more examples in which an egress packet processing pipeline accesses a context entry that includes a security context and a collective context.

[0120] Example 7 includes one or more examples in which the circuit stores data received from the first and second network interface devices in first and second memory buffers, the first and second memory buffers storing data from the first and second network interface devices, respectively.

[0121] Example 8 includes one or more examples in which the circuit reorders packets received from the first and second network interface devices based on DTLS record sequence numbers in the received packets.

[0122] Example 9 includes one or more examples including a switch system on a chip (SoC), the switch SoC including an interface and a circuit.

[0123] Example 10 includes one or more examples and includes at least one ingress port and at least one egress port communicatively coupled to a switch SoC.

[0124] Example 11 includes one or more of the examples and includes a non-transitory computer-readable medium having stored thereon instructions that, when executed by one or more processors, provide, on the non-transitory computer-readable medium, an endpoint for a Datagram Transport Layer Security (DTLS) connection with a first network interface device by decrypting DTLS encrypted data from packets received from a first network interface device, an endpoint for a DTLS connection with a second network interface device by decrypting DTLS encrypted data from packets received from a second network interface device, and a transport layer endpoint for packets received from the first network interface device. and configuring the network interface device to provide a transport layer endpoint for packets received from the second network interface device, where the packets received from the first and second network interface devices provide data for an in-network computation operation, reorder the packets received from the first and second network interface devices based on the packets received out of order from the first and second network interface devices, perform an in-network computation operation on the reordered data from the reordered packets based on a floating-point (FP) format, and perform DTLS encryption of data generated by the in-network computation operation on the reordered data prior to transmission.

[0125] Example 12 includes one or more examples in which the packets received from the first network interface device are received in a remote direct memory access (RDMA) based manner.

[0126] Example 13 includes one or more examples including stored instructions in a non-transitory computer-readable medium that, when executed by one or more processors, configure a network interface device to reorder packets received from the first and second network interface devices based on record sequence numbers in the received packets.

[0127] Example 14 includes one or more examples including instructions stored on a non-transitory computer-readable medium that, when executed by one or more processors, configure a network interface device to store data received from a first and second network interface device in first and second memory buffers, the first and second memory buffers storing data received from the first and second network interface devices, respectively.

[0128] Example 15 includes one or more examples including a non-transitory computer-readable medium including stored instructions that, when executed by one or more processors, configure a network interface device to reorder packets received from the first and second network interface devices based on DTLS record sequence numbers in the received packets.

[0129] Example 16 includes one or more examples including providing an endpoint for a Datagram Transport Layer Security (DTLS) connection with a first network interface device by decrypting DTLS encrypted data from packets received from the first network interface device, providing an endpoint for a DTLS connection with a second network interface device by decrypting DTLS encrypted data from packets received from the second network interface device, providing a transport layer endpoint for packets received from the first network interface device, and providing a transport layer endpoint for packets received from the second network interface device, where the packets received from the first and second network interface devices provide data for an in-network computation operation, reordering the packets received from the first and second network interface devices based on the packets received out of order from the first and second network interface devices, performing an in-network computation operation based on a floating point (FP) format on the reordered data from the reordered packets, and performing DTLS encryption of data generated by the in-network computation operation on the reordered data prior to transmission by the network interface device.

[0130] Example 17 includes one or more examples including the network interface device performing reordering of packets received from the first and second network interface devices based on record sequence numbers in the received packets.

[0131] Example 18 includes one or more examples in which a network interface device includes an ingress packet processing pipeline, a traffic manager, and an egress packet processing pipeline, where the egress packet processing pipeline performs in-network computation operations on reordered data from the reordered packets based on the FP format.

[0132] Example 19 includes one or more examples, including a network interface device performing storing data received from a first and second network interface device in a first and second memory buffer, the first and second memory buffers storing data from the first and second network interface devices, respectively.

[0133] Example 20 includes one or more examples and includes the network interface device performing reordering of packets received from the first and second network interface devices based on DTLS record sequence numbers in the received packets.

Claims

1. An apparatus including an interface and a circuit coupled to the interface, The circuit comprises: providing an endpoint for a Datagram Transport Layer Security (DTLS) connection with the first network interface device by decrypting DTLS-encrypted data from packets received from the first network interface device; providing an endpoint for a second DTLS connection with the second network interface device by decrypting DTLS encrypted data from packets received from the second network interface device; providing a transport layer endpoint for the packets received from the first network interface device; providing a second transport layer endpoint for the packets received from the second network interface device, where the packets received from the first and second network interface devices provide data for an intra-network computing operation; reordering the packets received from the first and second network interface devices based on the packets received out of order from the first and second network interface devices; performing a reproducible in-network computation operation on the reordered data from the reordered packets based on a floating point (FP) format; and configured to perform DTLS encryption of data generated by the in-network computation operation on the reordered data prior to transmission.

2. The apparatus of claim 1 , wherein the packets received from the first network interface device are received in a manner based on remote direct memory access (RDMA).

3. 3. The apparatus of claim 1, wherein the circuitry reorders the packets received from the first and second network interface devices based on record sequence numbers within the received packets.

4. 3. The apparatus of claim 1, wherein the circuitry includes an ingress packet processing pipeline, a traffic manager, and an egress packet processing pipeline, the egress packet processing pipeline performing the in-network computation operations on reordered data from the reordered packets based on the FP format.

5. 5. The apparatus of claim 4, wherein the egress packet processing pipeline performs decryption of DTLS encrypted data prior to performance of the in-network computation operations on the reordered data.

6. The apparatus of claim 4 , wherein the egress packet processing pipeline has access to a context entry that includes a security context and a collective context.

7. 3. The apparatus of claim 1, wherein the circuitry stores data received from the first and second network interface devices in first and second memory buffers, the first and second memory buffers storing data from the first and second network interface devices, respectively.

8. 3. The apparatus of claim 1, wherein the circuitry reorders the packets received from the first and second network interface devices based on DTLS record sequence numbers in the received packets.

9. 3. The apparatus of claim 1 or 2, comprising a switch system on chip (SoC), said switch SoC comprising said interface and said circuitry.

10. The apparatus of claim 9 , comprising at least one ingress port and at least one egress port communicatively coupled to the switch SoC.

11. providing an endpoint for a Datagram Transport Layer Security (DTLS) connection with the first network interface device by decrypting DTLS-encrypted data from packets received from the first network interface device; providing an endpoint for a DTLS connection with a second network interface device by decrypting DTLS encrypted data from packets received from the second network interface device; providing a transport layer endpoint for the packets received from the first network interface device; providing a transport layer endpoint for the packets received from the second network interface device, where the packets received from the first and second network interface devices provide data for an intra-network computing operation; reordering the packets received from the first and second network interface devices based on the packets received out of order from the first and second network interface devices; performing an in-network computation operation on the reordered data from the reordered packets based on a floating point (FP) format; performing DTLS encryption of data generated by said in-network computation operation on the reordered data prior to transmission; The method includes performing, on a network interface device.

12. reordering the packets received from the first and second network interface devices based on record sequence numbers in the received packets. The method of claim 11 , comprising execution by the network interface device.

13. 12. The method of claim 11, wherein the network interface device includes an ingress packet processing pipeline, a traffic manager, and an egress packet processing pipeline, the egress packet processing pipeline performing the in-network computation operations on reordered data from the reordered packets based on the FP format.

14. storing data received from the first and second network interface devices in first and second memory buffers; the network interface device executing, The method of claim 11 , wherein the first and second memory buffers store data from the first and second network interface devices, respectively.

15. reordering the packets received from the first and second network interface devices based on DTLS record sequence numbers in the received packets; The method of claim 11 , comprising execution by the network interface device.

16. A computer program product causing one or more processors to carry out the method of any one of claims 11 to 15.

17. An interface; means for providing an endpoint for a Datagram Transport Layer Security (DTLS) connection with the first network interface device by decrypting DTLS encrypted data from packets received from the first network interface device; means for providing an endpoint for a second DTLS connection with the second network interface device by decrypting DTLS encrypted data from packets received from the second network interface device; means for providing a transport layer endpoint for the packets received from the first network interface device; means for providing a second transport layer endpoint for the packets received from the second network interface device; means for reordering the packets received from the first and second network interface devices; means for performing a repeatable in-network computation operation on the reordered data from the reordered packets based on a floating point (FP) format; means for performing DTLS encryption of data generated by said in-network computation operation on the reordered data prior to transmission; An apparatus comprising:

18. 20. The apparatus of claim 17, wherein the packets received from the first network interface device are received in a remote direct memory access (RDMA) based manner.

19. 20. The apparatus of claim 17 or 18, wherein the means for reordering the packets received from the first and second network interface devices performs reordering of the packets received from the first and second network interface devices based on record sequence numbers within the received packets.

20. 20. The apparatus of claim 17 or 18, comprising an ingress packet processing pipeline, a traffic manager, and an egress packet processing pipeline, the egress packet processing pipeline performing the in-network computation operations on reordered data from the reordered packets based on the FP format.

21. 21. The apparatus of claim 20, wherein the egress packet processing pipeline performs decryption of DTLS encrypted data prior to performance of the in-network computation operations on the reordered data.

22. 21. The apparatus of claim 20, wherein the egress packet processing pipeline has access to a context entry that includes a security context and a collective context.

23. 19. The apparatus of claim 17 or 18, further comprising means for storing data received from the first and second network interface devices in first and second memory buffers, the first and second memory buffers storing data from the first and second network interface devices, respectively.

24. 20. The apparatus of claim 17 or 18, wherein the means for reordering the packets received from the first and second network interface devices reorders the packets received from the first and second network interface devices based on DTLS record sequence numbers in the received packets.

25. At least one non-transitory computer readable storage medium having stored thereon the computer program of claim 16.

Images