Intra-network computing operations with encrypted communications

Abstract

The invention relates to intra-network computing operations using encrypted communications. Examples described herein relate to an interface and a circuit module coupled to the interface. The circuitry module may provide an endpoint for transport layer security (TLS) over a remote direct memory access (RDMA) connection with a first network interface device, provide an endpoint for TLS over an RDMA connection with a second network interface device, provide a transport layer endpoint for packets received from the first network interface device, and provide a transport layer endpoint for packets received from the second network interface device. And providing a transport layer endpoint for a packet received from the second network interface device.

Description

Background Art

[0001] Machine learning (ML) or high performance computing (HPC) clusters utilize a large number of servers and graphics processing units (GPUs), tensor processing units (TPUs), or accelerators. Collective operations can be performed on data transmitted over the network by different switches. These systems can train ML models using iterative algorithms such as stochastic gradient descent, whereby the input data is divided across workers and multiple iterations are performed on the training data. In each iteration, the workers calculate updates to the ML model parameters based on a subset of the local data and an intermediate current model. The workers pass their results to be aggregated into model updates, and the aggregated updates are summed for the model parameters at the nodes for the next iteration. These iterations are performed multiple times (multiple rounds) on the entire data set.

[0002] Figure 1 shows an end-to-end solution for machine learning (ML) training using a PS architecture. A parameter server (PS) can be used for collective operations, whereby worker nodes calculate updates and send updates to the PS. The PS pushes aggregated data or pulls data from a PS server. The PS architecture includes a worker 100 and a parameter server (PS) 120 coupled in communication using a switch 110. The end-to-end solution for the PS architecture includes reduce-scatter and Allgather operators. Figure 1 shows that worker 1 has three queue pairs (QPs) and each QP is connected to a PS. Workers 2 and 3 also utilize three QPs and each QP is connected to a PS.

[0003] In the reduce-scatter operator, a worker sends partitions of data to the corresponding parameter server. For example, partition a1 from worker 1, partition a2 from worker 2, and partition a3 from worker 3 are sent to PS1, while partition b1 from worker 1, partition b2 from worker 2, and partition b3 from worker 3 are sent to PS2. A similar pattern applies to PS3. As a result, the data is scattered across multiple parameter servers to exploit the parallel computation of the Graphics Processing Units (GPUs) located at the parameter servers. After receiving the data, the PS first performs aggregation on the data from these workers.

[0004] In the Allgather operator, the data processed by the GPU is multicasted to the workers. The parameter server sends the same copy of the data to the workers. In this process, the bandwidth from one PS is distributed to all workers, and the network may be the bottleneck. BRIEF DESCRIPTION OF THE DRAWINGS

[0005] FIG. 1 illustrates an end-to-end solution for machine learning (ML) training using a parameter server (PS) architecture.

[0006] Figure 2 Describe the example system.

[0007] Figure 3A and Figure 3B Depicts an example switch circuit module.

[0008] Figure 4A and Figure 4B Depicts an example collective circuit module.

[0009] Figure 5 Describes an example of the packet format.

[0010] Figure 6 An example depicting the alignment of TLS records with RDMA packets.

[0011] Figure 7 Depicts an example manner of reordering packets.

[0012] Figure 8 Depicts an example accumulation tree for addition operations.

[0013] Fig. 9 Describe the example system.

[0014] Fig.10 Describe the example process.

[0015] Fig.11 An example network interface device or packet processing device is depicted.

[0016] Figures 12A-12C Depicts an example switch.

[0017] Fig.13 Describe the example system. DETAILED DESCRIPTION

[0018] The PS architecture and other neural network architectures (e.g., convolutional neural networks (CNNs), recurrent neural networks (RNNs), transformer-based models, or others) can utilize arithmetic operations involving floating-point (FP) format numbers. The Institute of Electrical and Electronics Engineers (IEEE) Standard 754 (IEEE 754-2019) for floating-point arithmetic standards defines reproducibility requirements for FP operation sequences, because different FP operation sequences may produce different results. In other words, the cumulative result of a string of FP numerical data depends on the order in which they are added and are not associated. However, the order of packets arriving at the network interface device from different sources is not predictable and may arrive out of order. Therefore, FP numerical data may arrive at the network interface device out of order.

[0019] Communications between devices in a PS architecture or other neural network architectures can utilize cryptographic protocols to provide security for packets transmitted over a computer network. Secure Sockets Layer (SSL) and Transport Layer Security (TLS) are examples of security protocols. TLS provides end-to-end encryption at the application layer, and TLS can protect application-to-application communications. TLS is a widely deployed protocol for protecting Transmission Control Protocol (TCP) connections on the Internet. TLS is defined in at least the Transport Layer Security (TLS) Protocol Version 1.3, RFC 8446 (August 2018).

[0020] Another example cryptographic protocol for secure datagram transmission is Datagram Transport Layer Security (DTLS). DTLS is defined at least by the Network Working Group Request for Comments (RFC) 4347 (2006) and the Internet Engineering Task Force (IETF) Datagram Transport Layer Security (DTLS) protocol version 1.3 (2020).

[0021] Another example encryption protocol for secure datagram transmission is the PSP Security Protocol (PSP). PSP is a protocol that is created by A security protocol created for encryption of packets. PSP uses the concept of a "security association" (SA) to represent a set of services to be handled with a specific set of confidentiality states (keys, initialization vectors (IVs), sequence numbers, etc.).

[0022] At least in order to obtain reproducible results for operations on FP data, the network interface device can sort the FP data so that even though the network interface device receives packets carrying FP data out of order, a sequence of operations on the FP data can be performed to generate results in a reproducible manner. In some examples, the network interface device can act as both a transport protocol endpoint and a security protocol endpoint for communication with other devices. For example, the network interface device can provide a security protocol endpoint for protocols such as at least DTLS, PSP, TLS or others. In some examples, the network interface device can track the order in which encrypted data is received and perform decryption on encrypted data in order. For example, FP data can be transmitted in encrypted packets, and the network interface device can decrypt the FP data before the accumulation operation. In addition, the network interface device can first encrypt the FP data generated by the accumulation or other arithmetic operations, and then forward the encrypted FP data to another network interface device or a host system.

[0023] Figure 2Depicting an example system. Network interface device 200 can act as both a transport protocol endpoint and a security protocol endpoint for communications to and from device 210, and can act as both a transport protocol endpoint and a security protocol endpoint for communications to and from device 220. The system can be part of a PS architecture or a distributed ML or deep neural network (DNN) system that is trained to perform reasoning consistent with SwitchML (e.g., Sapio, A., "Scaling distributed machine learning with In-Network aggregation", 18th USENIX Symposium on Networked Systems Design and Implementation (NSDI 21) (2021)) or utilize one or more of the following: message passing interface (MPI), symmetric hierarchical memory access (SHMEM), unified parallel C (UPC), or other. In some examples, devices 210 and 220 can represent workers in a PS environment, and network interface device 200 can perform accumulation on data from devices 210 and 220 in a reproducible manner or in an associated manner and transmit the accumulated results to devices 210 and 220.

[0024] In some examples, the network interface device 200 may include one or more of the following: a network interface controller (NIC), a remote direct memory access (RDMA) enabled NIC, a SmartNIC, a router, a switch, a forwarding element, an infrastructure processing unit (IPU) with a programmable packet processing pipeline, a data processing unit (DPU), or an edge processing unit (EPU). The edge processing unit (EPU) may include a network interface device that utilizes a processor and an accelerator (e.g., a digital signal processor (DSP), a signal processor, or a wireless-specific accelerator for a virtualized radio access network (vRAN), cryptographic operations, compression / decompression, etc.).

[0025] For example, the device 210 may be implemented as one or more of the following: a network interface device, a host server system, a memory pool, a storage pool, an accelerator pool, or other devices. For example, the device 220 may be implemented as one or more of the following: a network interface device, a host server system, a memory pool, a storage pool, an accelerator pool, or other devices. Fig.13 Various examples of host server systems are described.

[0026] In some examples, device 210 may transmit data to network interface device 200, and network interface device 200 is to process the data for collective operation. In some examples, device 220 may transmit data to network interface device 200, and network interface device 200 is to process the data for collective operation. In some examples, network interface device 200 and device 210 may transmit data by using encrypted packets using an encryption protocol. Example encryption protocols may include one or more of the following: DTLS, PSP, TLS, Internet Protocol Security (IPSec), IEEE 802.1AE-2008 (MACsec), or others. In some examples, encrypted packets may be transmitted using a remote direct memory access (RDMA) protocol. Examples of RDMA protocols include at least remote direct memory access (RDMA) based on converged Ethernet (RoCE), RoCEv2, InfiniBand, or others. Various RDMA standards may be utilized, including the Network Working Group's "A Remote Direct Memory Access Protocol Specification" Draft for Comments 5040 (2007). For example, for ML flows for collective operations, the contents of RDMA packets (e.g., headers (e.g., one or more header fields or a strict subset of headers) and / or payload) may be encrypted via DTLS, PSP, or TLS. However, the contents of RDMA packets for flows carrying data for non-collective operations need not be encrypted.

[0027] RDMA may involve writing or reading directly across a connection to copy the contents of a buffer without the operating system managing a copy. The network interface device may implement a direct memory access circuit module and create a channel from its RDMA circuit module through the bus to the application memory. Send queues and receive queues may be used to pass work requests and are referred to as queue pairs (QPs). A requester may place a work request instruction on its work queue, thereby conveying to the interface what buffer to send to or what buffer to receive the contents from. A work request may include an identifier (e.g., a memory address or pointer to a buffer). For example, a work request placed on a send queue (SQ) may include an identifier of a message or content in a buffer (e.g., an application buffer) to be sent. In contrast, an identifier in a work request in a receive queue (RQ) may include a pointer to a buffer (e.g., an application buffer) in which the contents of an incoming message may be stored. RQ may be used to receive RDMA-based commands or RDMA-based responses. A completion queue (CQ) may be used to notify when an instruction placed on a work queue has been completed. In some examples, a QP may be allocated for an RDMA connection between the network interface device 200 and the device 210. In some examples, a second QP may be assigned to the connection between network interface device 200 and device 220 .

[0028] In some examples, instead of using RDMA, a reliable transport protocol can be used to transmit packets, and the contents of the packets (e.g., headers and / or payloads) can be encrypted by DTLS, PSP, or TLS. In a reliable transport protocol, the receiver confirms packet reception to the data sender, and after a timeout interval, the sender attempts to retransmit undelivered packets and / or the sender delays packet transmission based on detected network congestion. Non-limiting examples of reliable transport protocols include Remote Direct Memory Access in Reliable Mode (RDMA Reliable Connections (RC)), InfiniBand, Transmission Control Protocol (TCP) (e.g., Internet Engineering Task Force (IETF) RFC 793 (1981)), Quick UDP Internet Connections (QUIC), RDMA over Converged Ethernet (RoCE) v2 (RoCEv2), Amazon's Scalable Reliable Datagram (SRD), Amazon AWS Elastic Fabric Adapter (EFA), Microsoft Azure Distributed Universal Access (DUA) and Lightweight Transport Layer (LTL), Google GCP Snap microkernel Pony Express, High Precision Congestion Control (HPCC) (e.g., Li et al., "HPCC: High Precision Congestion Control", SIGCOMM (2019)), Improved RoCE NIC (IRN) (e.g., Mittal et al., "Revisiting network support for RDMA", SIGCOMM 2018), Homa (e.g., Montazeri et al., "Homa: A Receiver-Driven Low-Latency Transport Protocol Using Network Priorities", SIGCOMM 2018), NDP (e.g., Handley et al., "Re-architecting Datacenter Networks and Stacks for Low Latency and High Performance", SIGCOMM2017), EQDS (e.g., Olteanu et al., "An edge-queued datagram service for all datacenter traffic", USENIX 2022), or others.

[0029] A stream can be a sequence of packets transmitted between two endpoints, generally representing a single session using a known protocol. Therefore, a stream can be identified by a set of defined tuples, and for the purpose of routing, a stream is identified by two tuples (e.g., source address and destination address) that identify the endpoints. For content-based services (e.g., load balancers, firewalls, intrusion detection systems, etc.), streams can be distinguished with a finer granularity by using N-tuples (e.g., source address, destination address, IP protocol, transport layer source port and destination port). The packets in a certain stream are expected to have the same set of tuples in the packet header. A packet stream can be identified by a combination of a tuple (e.g., Ethernet type field, source and / or destination IP address, source and / or destination user datagram protocol (UDP) port, source / destination TCP port, or any other header field) and a unique source and destination queue pair (QP) number or identifier.

[0030] Packets may be used herein to refer to various formatted collections of bits that may be sent over a network, such as Ethernet frames, IP packets, TCP segments, UDP datagrams, etc. Also, as used in this document, the L2, L3, L4, and L7 layers (Layer 2, Layer 3, Layer 4, and Layer 7) mentioned refer to the second data link layer, the third network layer, the fourth transport layer, and the seventh application layer of the OSI (Open Systems Interconnection) layer model, respectively.

[0031] The flows referred to may alternatively or additionally refer to tunnels, for example, Multiprotocol Label Switching (MPLS) Label Distribution Protocol (LDP), Segment Routing over IPv6 Data Plane (SRv6) source routing, VXLAN tunneling services, GENEVE tunneling services, network slicing based on virtual local area networks (VLANs), the techniques described in Mudigonda, Jayaram et al., "Spain: Cots data-center ethernet for multipathing over arbitrary topologies" (NSDI.Vol.10.2010) (hereinafter referred to as "SPAIN"), and the like.

[0032] The network interface device 200 can utilize the packet processing pipeline 202 to process received packet headers and decrypt packet headers and / or payloads, perform accumulation, calculations, or arithmetic operations on data from received packets, and re-encrypt the results of the accumulation, calculations, or arithmetic operations and transmit them in one or more packets to another device (e.g., a network interface device, a host server system, or other devices described herein), or store them in a memory of the network interface device 200. The packet processing pipeline 202 can perform accumulation, calculation or arithmetic operations on the received data, including: multiply-accumulate (MAC) operations (for example, calculating the product of two numbers and adding the product to an accumulator), fused multiply-add (FMA), fused multiply-accumulate (FMAC), matrix multiplication, dot product, general matrix-matrix multiplication (GEMM) operations, summing the packet data with other packet data from other workers, multiplication, division, minimum, maximum, 16-bit down to 32-bit, 32-bit down to 16-bit, FP addition, integer (INT) addition, local minimum, local maximum, AND, OR, XOR, bitwise XOR, or other data calculation operations related to Allreduce, ReduceScatter or Allgather.

[0033] The packet processing pipeline 202 can perform accumulation, calculation, or arithmetic operations on data in various formats such as: single-precision floating point (e.g., 32 bits), half-precision floating point (e.g., 16 bits), custom floating point formats (e.g., BF16), integer words (e.g., 16 bits, 8 bits, 4 bits), INT8, INT4, 8-bit binary integers, 4-bit binary integers, 2-bit binary integers, bfloat (brain floating point), 16-bit floating point format, tensor floating point 32-bit floating point format (TF32) with a different number of mantissa and exponent bits relative to the Institute of Electrical and Electronics Engineers (IEEE) 754 format, or other formats.

[0034] In some examples, network interface device 200 may offload computations to a connected server, processor, or accelerator.

[0035] The operation of the packet processing pipeline 202 may be programmed using one or more of the following: a configuration file, OneAPI, a packet processor independent of programming protocol (P4), software for open networking in the cloud (SONiC), Network Programming Language (NPL), DOCA TM, Data Plane Development Kit (DPDK), OpenDataPlane (ODP), Infrastructure Programmer Development Kit (IPDK), eBPF, x86-compatible executable binaries, or other executable binaries.

[0036] The network interface device 200 may utilize a communication circuit module 204 to communicate with the devices 210 and 220 via one or more ports over a network or structure. The communication circuit module 204 may be configured to communicate using any one or more communication technologies (e.g., wired or wireless communications) and associated protocols (e.g., Ethernet, 4G LTE, 5G, etc.) to perform such communications. The communication circuit module 204 may include one or more network hardware resources, such as an ingress queue, an egress queue, a direct memory access (DMA) circuit module, a crossbar switch, a shared memory switch, a media access control (MAC), a physical layer interface (PHY), Ethernet port logic, and other network hardware resources.

[0037] Figure 3A Depicting an example system. A network interface device may include a switch circuit module 300 including ingress packet processing pipelines 302-0 to 302-A (where A is an integer), a traffic manager and packet buffer 304, and egress pipelines 306-0 to 306-B (where B is an integer). One or more of the ingress pipelines 302-0 to 302-A or the egress pipelines 306-0 to 306-B may include or be configured to execute a collective unit circuit module 308 that may provide both a transport protocol endpoint and a security protocol endpoint and at least provide accumulation or arithmetic operations for FP values ​​in a reproducible manner.

[0038] The collective unit circuit module 308 can perform transport layer management, such as: managing packet retransmissions; establishing a security session with the device based on DTLS, PSP, TLS or other security protocols; reordering packet data for decryption; performing decryption based on associated security associations; ordering FP data for reproducible operations; performing accumulation, calculations or arithmetic operations on FP data; and performing data encryption on data generated by the accumulation, calculations or arithmetic operations.

[0039] Figure 3BDepicts an example switch circuit module. Ingress pipelines 350-0 to 350-A (where A is an integer) can perform packet routing, switching, and packet integrity checking. After being processed by the ingress pipeline, the packet is sent to a service manager 352, where the packet is first queued and placed in an output buffer and then provided to the collective processing pipeline. The collective processing pipelines 360-0 to 360-B (where B is an integer) can utilize a device to perform transport protocol initiation and provide a transport protocol endpoint. After being processed by the collective processing pipeline, the packet is sent to a service manager 362, where the packet is first queued and placed in an output buffer and then provided to the egress pipeline. The egress pipelines 364-0 to 364-C (where C is an integer) can perform reproducible FP calculations.

[0040] In some examples, Figure 3A and Figure 3B The switch circuit module of the switch system can be implemented as a system on chip (SoC), and the SoC includes at least one interface to other circuit modules in the switch system. The switch SoC can be coupled to other devices in the switch system, such as an ingress port or an egress port, a memory device, or a host interface circuit module.

[0041] Figure 4A Depicting an example collective circuit module. The collective unit circuit module 400 may be implemented as one or more of the following: one or more processors; one or more programmable packet processing pipelines; one or more accelerators; one or more application specific integrated circuits (ASICs); one or more field programmable gate arrays (FPGAs); one or more graphics processing units (GPUs); one or more memory devices; one or more storage devices; or other.

[0042] The collective unit circuit module 400 can utilize the transport layer processing 402 to provide a transport layer endpoint for reliable packet delivery and retransmission by the transmitter device. For example, the transport layer processing 402 can track the receipt of packet sequence numbers and request retransmission of unreceived packets by the transmitter network interface device based on the failure to receive a sequence number or sequence number range. The transport layer processing 402 can store connection status for the stream. The connection status can include one or more of the following: an initial packet sequence number, a number of bytes received, a received byte sequence number that keeps the received packets in order, an n-tuple (e.g., source address, destination address, IP protocol, transport layer source port and / or destination port), a connection flag, or other. The transport layer processing 402 can cause a data receipt confirmation (ACK) to be transmitted to the transmitter network interface device to indicate receipt of the packet.

[0043] Data reordering 404 for decryption can first selectively reorder the encrypted data (e.g., records or record segments) received in the out-of-order packets, and then the decryption 420 performs the decryption of the data. For DTLS encrypted communication, DTLS stipulates that a single DTLS record is transmitted in a datagram, and the record includes a record sequence number in the datagram. The DTLS encrypted record can be decrypted independently of other DTLS records and regardless of the order in which the packets are received. Similarly, for PSP encrypted communication, PSP stipulates that a single PSP record is transmitted in a datagram, and the record includes a record sequence number in the datagram. The PSP encrypted record can be decrypted independently of other PSP records and regardless of the order in which the packets are received.

[0044] For TLS encrypted communications, TLS records may be split across multiple packets, and if the packets are received out of order, the record segments from the packets are reordered prior to record decryption. As described herein, for out-of-order (OOO) packet reception, data reordering for decryption 404 may buffer the record segments in buffer 412 and reorder the record segments prior to record segment decryption.

[0045] When establishing a collective session with a host or network interface device (e.g., device 210 and 220), the encryption connection manager 406 can establish a DTLS, PSP, TLS or other cryptographic session. A collective stream can have a unique security context established when establishing the collective stream. The key can be refreshed regularly (e.g., every day), and the security association (SA) can be updated accordingly. For a stream, the stream decoder 408 can use a single index value to find the collective and security context, so that a single search operation can be performed to access the collective and security context without multiple search operations. The security context can include information such as one or more of the following: a specific security operation to be performed, a FP value length (e.g., the number of bytes of floating point data in a packet), a calculation or arithmetic operation to be performed, the length of the header, a security key, a random number, the last encryption or decryption byte, a packet sequence number, etc. Different collective streams can have a unique security context established when establishing the collective stream. The collective context can include an information field for processing the packet segment of the stream by a collective operation. The collective context may include the location address of the intermediate collected value in the buffer 412 or memory, the number format (e.g., floating point, integer, or other), and the calculation or arithmetic operation to be performed on the segment. The collective and security context may be stored in a data structure in the on-chip memory of the network interface device.

[0046] Data storage 410 for reproducible results can associate data used for FP operations with nodes of a logical tree structure. Figure 8 and Fig. 9Various examples of logical tree identifiers and node assignments described are stored in a collective context. Buffer 412 can store encrypted and / or decrypted packet headers and / or payloads (e.g., data) associated with reproducible operations and non-reproducible operations. Remote hosts can be assigned to nodes (e.g., leaves) in the tree. Data received in packets can be assigned to nodes and stored in a buffer associated with the node in buffer 412. For example, a node can be associated with a source IP address, and receiving a packet with such a source IP address can cause the packet's data to be stored in a buffer associated with the node in buffer 412. As described with respect to Figure 8 and Fig. 9 As described, after a packet arrives at the network interface device and is authenticated, data from the packet may be copied to a buffer in buffer 412 associated with a leaf associated with a source host, and when an addition operand is stored, an operation may occur, and an intermediate result may be stored in a higher-level leaf (e.g., parent) in the tree in the buffer in buffer 412. The final result may be at the root of the tree.

[0047] To decrypt DTLS encrypted packet contents, decryption 420 may utilize Advanced Encryption Standard with Galois / Counter Mode (AES-GCM) based on the accessed security context.Data reordering 404 for decryption may reorder out-of-order received packet data to be decrypted in accordance with DTLS.

[0048] To decrypt the PSP encrypted packet content, decryption 420 may perform key generation or key extraction based on AES-GCM to extract a 128-bit or 256-bit length key, and may apply at least AES-GCM 128 or 256 to decrypt the packet content. For the PSP, a decryption key may be derived from the packet, or a complete security association (SA) may be derived from the packet. A master key may be used to derive the SA, or a decryption key from a security parameter index (SPI) may be per connection or per network interface device.

[0049] To decrypt the TLS record, after reordering the segments of the byte stream, the network interface device may apply at least the techniques described with respect to TLS protocol version 1.3, RFC 8446 (August 2018) and its variants.

[0050] Floating point calculation 430 can perform collective operations, such as calculation or arithmetic operations. Various examples of calculation or arithmetic operations are described herein, such as: MAC, FMA, FMAC, matrix multiplication, dot product, GEMM operations, summing packet data with other packet data from other workers, multiplication, division, minimum value, maximum value, or other data calculation operations related to Allreduce, ReduceScatter or Allgather. FP format can include at least FP8, FP16, FP32, FP64, FPx (wherein x is an integer), INT16, INT32, INT64, UINT16, UINT32, UINT64, BF16 and other formats. In some examples, FP8 can be based on E4M3, E5M2 or other. E4M3 can represent 4-bit exponent and 3-bit mantissa, and E5M2 can represent 5-bit exponent and 2-bit mantissa.

[0051] Encryption 440 may encrypt the packet header portion and / or the payload based on DTLS, PSP, or TLS.

[0052] Figure 4B Depicts an example collective unit circuit module. A packet header vector (PHV) may include one or more packet header fields of a received packet. An instruction lookup (INSTR) may look up an instruction (Inst) (e.g., read, add, store, or other) to be executed to process packet payload data (PPV) based on a context from a context random access memory (RAM). Multiple collective units may perform arithmetic operations on one or more packet payload data in series or in parallel.

[0053] Figure 5 Depicts an example of a packet format. Some portions of the packet header fields may be plain text (e.g., unencrypted) and may be processed by a network interface device without decryption, while other portions of the packet header fields and the payload may be encrypted and decrypted before being processed by the network interface device. Depicts a PSP transport mode packet and a RoCE packet with a PSP header. Additionally, depicts a DTLS packet transmitted over UDP or RoCE. As described in "The Datagram Transport Layer Security (DTLS) Protocol Version 1.3" version 1.3 (2020) (draft-ietf-tls-dtls13-38) of the Internet Engineering Task Force (IETF), the DTLS header may include: a connection identifier (ID), a sequence number, and a length. Depicts a RoCE packet with a TLS header. The packet payload may store one or more FP values.

[0054] Figure 6Depicting an example of aligning TLS records with RDMA packets. TLS records may not contain sequence numbers, and both the sender and receiver track how many records have been sent and received. Implicit record sequence numbers can be used to encrypt or decrypt these records. Implicit record sequence numbers can be used as random numbers for AES-GCM operations, where the random number is a value that must be unique for AES-GCM encryption / decryption operations. Since the random number is not transmitted with the packet and is calculated independently by the endpoints, TLS records are processed so that the same random number (implicit record number) is used for encryption and decryption operations. When TLS records are transmitted over a reliable transport (such as TCP), packet reordering can occur based on packet sequence numbers, and TLS decryption operates on an ordered record stream. However, RDMA can be transmitted over unreliable User Datagram Protocol (UDP) datagrams. Therefore, packets may not arrive at the receiver in order.

[0055] A pair of network interface devices may be configured to operate in mode 600 or 650. In mode 600, one TLS record is transmitted in a single RDMA packet, and the RDMA packet sequence number (PSN) indicates the value of the random number. For example, RDMA packets 1-3 may include PSNs 1-3. In the case where the TLS random number is a 64b (64-bit) value and the RDMA packet sequence number is 24 bits, as long as the RDMA PSN does not wrap around, it can be as described in the Supplement to InfiniBand TM The full 64b TLS record sequence number is derived from the 24b RDMA PSN in the manner described in Architecture Specification Volume 1 Release 1.2.1, Annex A17: RoCEv2 (September 2, 2014). In mode 600, the TLS record from the RDMA packet can be independently decrypted or encrypted because the PSN from the packet can be used to reliably derive the random value.

[0056] In mode 650, a TLS record spans one or more packets (in this case, RDMA packets), whereby an encrypted TLS record may be sent over one or more RDMA packets such that multiple RDMA packets contain a portion of a TLS record (e.g., an RDMA message), but no RDMA packet carries more than one TLS record (e.g., the end of one TLS record and the beginning of another TLS record). An RDMA message may include multiple RDMA packets, multiple RDMA packets include a single TLS record, and the RDMA packets within an RDMA message are to be encrypted in sequence. As an endpoint, a network interface device may maintain a bitmap or data to track the state of packet arrival in one or more streams, and may prevent packets from being processed twice in the event of a retransmission by detecting duplicate packets, or may detect lost RDMA packets and cause the lost RDMA packets to be retransmitted. As an endpoint, a network interface device (e.g., a packet processing pipeline or collective unit) may perform reordering of RDMA packets, thereby reordering the TLS record segments. Reliable transport may occur before packet decryption, and the RDMA PSN may be unencrypted (e.g., plaintext) and authenticated before being used to reorder RDMA packets. Various ways of storing TLS record segments are described in this document.

[0057] As previously described, the network interface device can perform reproducible floating-point operations so that regardless of the order in which the packets arrive, accumulation is performed in a predefined order, thereby processing floating-point number streams from different hosts in a predefined host order, and the result is therefore reproducible. The packet header and / or payload can be stored in a buffer while waiting for the packet header and / or payload from another host to arrive to start calculation. The reproducible order is a predefined order for operations on data from different hosts, and the stream order is the relative order of packets transmitted by a single host. In some examples, for reproducibility and TLS stream reconstruction, a memory for storing data for reproducible operations and a bitmap management that can track packet arrival can be utilized.

[0058] Figure 7 Depicting an example manner of reordering packets. A bitmap or data structure may track the arrival of packets from multiple (e.g., 8 or other) hosts in a window of multiple (e.g., 4 or other number) packet spans. TLS decryption may be performed on TLS records in RDMA PSN order by row, while accumulation may be performed in host rank order by column. For example, TLS decryption may be performed on TLS records received via RDMA packets with PSNs numbered 1 to 4. For example, accumulation may be performed in host rank order for hosts in an accumulation group (e.g., 0 to 7) for the same PSN value.

[0059] Packet authentication may be performed based at least on Network Working Group RFC 4302 "IP Authentication Header" (December 2005). For example, portions of a packet header (e.g., an IP header) may be used to authenticate the sender or source of a packet. If packet authentication fails, intermediate buffers with packet data are cleared or marked as invalid, and an outage may be caused, and system-level actions may occur (e.g., stopping collective operation, restarting collective operation, identifying a potentially compromised network interface device).

[0060] The description next turns to a manner of correlating data from pairs of hosts to provide reproducible results from the data. Figure 8 Depicts an example accumulation tree for an addition operation. In some examples, the network interface device may perform a predetermined sequence of operations of (((P0+P1)+(P2+P3))+((P4+P5)+(P6+P7))) based on floating point data from received packets P0 to P7 received from respective hosts 0 to 7. However, packets from different hosts may arrive at the network interface device out of order, and for reproducible floating point operations, the addition operation cannot proceed until data for the addition of the operand pairs is received.

[0061] For example, for case I, packet P0 arrives before packet P1, and data from packet P0 can be stored while waiting for data from packet P1. Addition can occur after receiving data from packet P1. For example, for case I, packet P1 arrives before packet P0, and data from packet P1 can be stored while waiting for data from packet P0. Addition can occur after receiving data from packet P0. For case I, the network interface device packet processing pipeline can execute instruction store (packet, address) to store data from packet P0 while waiting for data from packet P1, or store data from packet P1 while waiting for data from packet P0. For data from exchange pairs P2 and P3, P4 and P5, and P6 and P7, similar operations may occur. It should be noted that although the example is shown for data from paired packets, the example can be applied to operations on data from more than two sources.

[0062] For example, for case II, the second data in the group of two or more data to be processed together in the accumulation operation is received and stored in the network interface device. For example, the arrival of the last data in one or more packets in the group of two or more data to be processed together in the exchange accumulation operation can cause the stored data of the group to be retrieved, the accumulation operation to be performed using the last data, and the stored data to be overwritten with the result of the accumulation operation. For example, the instruction ADD (received packet data, brother data address, parent address) can be executed to retrieve the previously stored data, the received packet data is added to the stored data, and the sum of the addition is stored in the buffer for the previously stored data. The instruction PUSH (ADD (brother 1 address, brother 2 address, parent address)) can make the instruction queue to perform the addition of paired data when there are available cycles.

[0063] For example, for case III, where the packet processing pipeline or other circuit module has (one or more) available cycles to process the enqueue instruction, the enqueue instruction can be executed by the packet processing pipeline or other circuit module. For example, the instruction queue can store ADD (brother 1 address, brother 2 address, parent address). For example, P0 and P1 are brothers, P2 and P3 are brothers, P4 and P5 are brothers, and P6 and P7 are brothers. Parent A can represent the sum of P0 and P1, and parent B can represent the sum of P2 and P3, so that parent A and parent B become brothers. Based on B's calculation instead of A's calculation, brothers P0 and P1 can be read from the storage device, and brothers P0 and P1 become operands of the instruction ADD (brother 1 address, brother 2 address, parent address), so that the sum of P0 and P1 is stored in A's parent address. In some examples, different addresses can be associated with P0 to P7.

[0064] Below is an example instruction sequence for performing a predetermined order of operations of (((P0+P1)+(P2+P3))+((P4+P5)+(P6+P7))). Note that the instructions may be stored in an instruction queue (e.g., INSTR) of a collective unit. Based on the arrival of data from packet P0, store(P0) (case I) is executed to store the data from P0. Based on the arrival of data from packet P1, the execution of ADD(P0, P1, P0) may cause the addition result of P0 and P1 to be stored in situ with the data from P0 (parent A) (case II). Based on the arrival of data from packet P4, store(P4) (case I) is executed to store the data of P4. Based on the arrival of data from packet P5, the execution of ADD(P4, P5, P4) may cause the addition result of P4 and P5 to be stored in situ with the data of P4 (parent C) (case II).

[0065] Based on the arrival of data from packet P6, the execution of store(P6) (case I) may cause the data of P6 to be stored. Based on the arrival of data from packet P7, the execution of ADD(P6, P7, P6) may cause the addition result of P6 and P7 (parent D) to be stored in situ with the data from P6 (case II). The instruction Push(ADD(P4, P6)) (case III) may enqueue an addition command to cause parent C and parent D to be added in an interchanged manner.

[0066] Based on the arrival of data from packet P2, the execution of store(P2) may cause the data from P2 to be stored (case I). Based on the arrival of data from packet P3, the execution of ADD(P2, P3, P2) may cause P2 and P3 to be added (parent B) and the result to be stored in situ in the data of P2 (case II). The instruction Push(ADD(P0, P2, P0)) (case III) may be enqueued to cause the addition of parent A and parent B and the storage of the result in situ in parent A in a swapped manner (parent E).

[0067] Instruction Push(ADD(P0, P4, P0)) (Case III) may be enqueued to cause the addition of parent E and parent F and to store the result (parent G) in situ at parent E. The enqueued instructions (add(P4, P6), add(P0, P2, P0), add(P0, P4, P0)) may be dequeued and executed when there are one or more available cycles of the packet processing pipeline.

[0068] Fig. 9 Depicts an example system. In this example, 512 hosts are assigned 512 ranks or buffers to which they contribute data for collective operations. However, other numbers of ranks or buffers may be assigned depending on the number of hosts contributing data to the collective operation. For example, referring to Figure 8 For example, there are 8 hosts and 8 ranks or buffers can be reserved. For example, refer to Figure 8 For example, level 1 can store data for leaves P0 to P7, level 2 can store values ​​of parent AD, level 3 can store values ​​of parent E and parent F, and level 4 can store values ​​of parent G (root node). At each level, the rank or buffer can be overwritten with the sum to reduce memory usage.

[0069] In some examples, one or more segments may correspond to a single FP value. For example, segment 1 (Seg1) may represent the first segment of the FP number, segment 2 (Seg2) may represent the second segment of the FP number, and segment 3 (Seg3) may represent the third segment of the FP number. In some examples, the segments of the grouping may be added sequentially.

[0070] Fig.10An example process is depicted. The process may be performed by a network interface device. At 1002, the network interface device may form a communication connection with one or more devices, thereby providing an endpoint for a transport layer and security protocol. For example, the connection may utilize RDMA or a reliable transport protocol, and encrypt headers and / or payload portions based on DTLS, PSP, TLS, or other encryption schemes. The network interface device may terminate the transport layer connection by at least the following steps: tracking the receipt of packet sequence numbers (PSNs) of packets, and requesting retransmission of packets associated with unreceived PSNs by a sender network interface device.

[0071] At 1004, based on receiving packets out of order from a device or from multiple devices, the network interface device may reorder the data prior to decryption of the data. For example, where encrypted records are transmitted and received via multiple packets and the multiple packets are received out of order, the network interface device may reorder the data prior to decryption of encrypted data from the packets. However, where encrypted data is received in order or can be decrypted without regard to the order in which the packets are received, the network interface device may not need to reorder the encrypted data relative to other encrypted data.

[0072] At 1006, the network interface device may decrypt the encrypted data based on an applicable cryptographic protocol. Example cryptographic protocols include DTLS, PSP, TLS, IPSec, MACsec, or others. In some examples, the network interface device may retrieve a context having a security context and a collective context for a packet flow using a single retrieval operation. The network interface device may decrypt the data based on metadata in the security context.

[0073] At 1008, the network interface device may perform a computation based on the decrypted data. Various examples of computations are described herein, such as: MAC, FMA, FMAC, matrix multiplication, dot product, GEMM operations, summing packet data with other packet data from other workers, multiplication, division, minimum, maximum, or other data computation operations. The network interface device may perform a computation as part of a series of operations that generates a reproducible result. The network interface device may perform the computation based on metadata in a collective context.

[0074] The network interface device may encrypt the result of the computation(s) based on an applicable cryptographic protocol at 1010. Example cryptographic protocols include DTLS, PSP, TLS, IPSec, MACsec, or others.

[0075] At 1012, the network interface device may transmit the encrypted data to the device. The network interface device may transmit the encrypted data to the device in a packet transmitted based on RDMA or a reliable transport protocol. In some examples, before the packet is transmitted, the network interface device may encrypt the packet header and / or payload portion based on DTLS, PSP, TLS, or other encryption schemes.

[0076] Fig.11 An example network interface device or packet processing device is depicted. In some examples, as described herein, the circuit modules of the network interface device can be used to provide transport layer and security protocol endpoints and perform calculations on received data. In some examples, the packet processing device 1100 can be implemented as a network interface controller, a network interface card, a host structure interface (HFI), or a host bus adapter (HBA), and such examples can be interchangeable. The packet processing device 1100 can be coupled to one or more servers using a bus, PCIe, CXL, or double data rate (DDR). The packet processing device 1100 can be implemented as part of a system on chip (SoC) including one or more processors, or can be included in a multi-chip package that also includes one or more processors.

[0077] Some examples of the packet processing device 1100 are part of a data processing unit (DPU) or an infrastructure processing unit (IPU), or are utilized by a DPU or IPU. An xPU may refer to at least an IPU, a DPU, a GPU, a GPGPU, or other processing unit (e.g., an accelerator device). An IPU or DPU may include a network interface with one or more programmable or fixed-function processors to perform offloads of operations that would otherwise be performed by a CPU. An IPU or DPU may include one or more memory devices. In some examples, an IPU or DPU may perform virtual switch operations, manage storage transactions (e.g., compression, cryptography, virtualization), and manage operations performed on other IPUs, DPUs, servers, or devices.

[0078] The network interface 1100 may include a transceiver 1102, a processor 1104, a transmit queue 1106, a receive queue 1108, a memory 1110, a host interface 1112, and a DMA engine 1172. The transceiver 1102 may be capable of receiving and transmitting packets in accordance with an applicable protocol such as Ethernet (as described in IEEE 802.3), although other protocols may also be used. The transceiver 1102 may receive packets from a network and transmit packets to a network via a network medium (not depicted). The transceiver 1102 may include a PHY circuit module 1114 and a media access control (MAC) circuit module 1116. The PHY circuit module 1114 may include an encoding and decoding circuit module (not shown) to encode and decode data packets in accordance with applicable physical layer specifications or standards. The MAC circuit module 1116 may be configured to assemble the data to be transmitted into packets, the packets including a destination address and a source address together with network control information and an error detection hash value.

[0079] The processor 1104 may be any one or a combination of a processor, a core, a graphics processing unit (GPU), a field programmable gate array (FPGA), an application specific integrated circuit (ASIC), or other programmable hardware device that allows programming of the network interface 1100. For example, an "intelligent network interface" may use the processor 1104 to provide packet processing capabilities in the network interface.

[0080] The processor 1104 may include one or more packet processing pipelines, which in some embodiments may be configured to perform match-actions on received packets using information stored in a ternary content addressable memory (TCAM) table or an exact match table to identify packet processing rules and next hops. For example, a match-action table or circuit module may be used, whereby a hash of a portion of a packet is used as an index to find an entry. The packet processing pipeline may perform one or more of the following: packet parsing (parser), exact match-action (e.g., a small exact match (SEM) engine or a large exact match (LEM)), wildcard match-action (WCM), longest prefix match block (LPM), hash block (e.g., receive side scaling (RSS)), packet modifier (modifier), or traffic manager (e.g., transmission rate metering or shaping). For example, the packet processing pipeline may implement access control lists (ACLs) or packet discards caused by queue overflow.

[0081] The configuration of the operation of the processor 1104, including its data plane, may be programmed based on one or more of the following: a protocol independent packet processor (P4), software for open networking in the cloud (SONiC), Network Programming Language (NPL), DOCA TM, Infrastructure Programmer Development Kit (IPDK), and more.

[0082] The packet distributor 1124 may use receive side scaling (RSS) to provide distribution of received packets for processing by multiple CPUs or cores. When the packet distributor 1124 uses RSS, the packet distributor 1124 may calculate a hash or make another determination based on the content of the received packet to determine which CPU or core is to process the packet.

[0083] Interrupt coalescing 1122 may perform interrupt moderation, whereby the network interface interrupt coalescing 1122 waits for multiple packets to arrive, or for a timeout to expire, before generating an interrupt to the host system processing the received packet(s). Receive segment coalescing (RSC) may be performed by the network interface 1100, whereby portions of an incoming packet are combined into segments of a packet. The network interface 1100 provides this coalesced packet to the application.

[0084] A direct memory access (DMA) engine 1172 may copy packet headers, packet payloads, and / or descriptors directly from host memory to a network interface, or vice versa, rather than copying the packets to an intermediate buffer at the host and then using another copy operation from the intermediate buffer to a destination buffer.

[0085] Memory 1110 can be any type of volatile or non-volatile memory device, and can store any queue or instructions used to program network interface 1100. Transmit queue 1106 can include data or references to data for transmission by the network interface. Receive queue 1108 can include data or references to data received from the network by the network interface. Descriptor queue 1120 can include descriptors that reference data or packets in transmit queue 1106 or receive queue 1108. Host interface 1112 can provide an interface with a host device (not depicted). For example, host interface 1112 can be compatible with PCI, PCI Express, PCI-x, Serial ATA and / or USB compatible interfaces (however, other interconnect standards can also be used).

[0086] Fig. 12ADepicting an example switch. As described herein, various examples can be used in or with a switch to provide transport layer and security protocol endpoints and perform computations on received data. Switch 1204 can route frames or packets of any format or in accordance with any specification from any port 1202-0 to 1202-X to any port in ports 1206-0 to 1206-Y (or vice versa). Any port in ports 1202-0 to 1202-X can be connected to a network of one or more interconnected devices. Similarly, any port in ports 1206-0 to 1206-Y can be connected to a network of one or more interconnected devices.

[0087] In some examples, the switch fabric 1210 can provide routing for packets from one or more ingress ports for processing before being transmitted from the switch 1204. The switch fabric 1210 can be implemented as one or more multi-hop topologies (where example topologies include torus, butterfly, buffered multi-stage, etc.), or a shared memory switch fabric (SMSF), among other implementations. An SMSF can be any switch fabric in a switch that is connected to an ingress port and an egress port, where an ingress subsystem writes (stores) packet segments into the fabric's memory, and an egress subsystem reads (removes) packet segments from the fabric's memory.

[0088] The memory 1208 may be configured to store packets received at the port before being transmitted from one or more ports. The packet processing pipeline 1212 may include inlet and outlet packet processing circuit modules to process incoming packets and outgoing packets respectively. The packet processing pipeline 1212 may use a table that maps packet characteristics to associated output ports to determine which port to transmit packets or frames to. In some examples, the packet processing pipeline 1212 may be configured to perform a match-action on the received packet to identify the packet processing rules and the next jump using information stored in a ternary content addressable memory (TCAM) table or an exact match table. For example, a match-action table or circuit module may be used, whereby a hash of a portion of the packet is used as an index to find an entry (e.g., a forwarding decision based on the content of the packet header). The packet processing pipeline 1212 may implement an access control list (ACL) or packet discard caused by a queue overflow. As described herein, the packet processing pipeline 1212 may be configured to provide a transport layer and security protocol endpoint, and perform calculations on the received data. The configuration of the operation of the packet processing pipeline 1212, including its data plane, may be programmed using P4, C, Python, Broadcom Network Programming Language (NPL), or an x86 compatible executable binary or other executable binary. The processor 1216 and FPGA 1218 may be used for packet processing or modification.

[0089] Traffic manager 1213 may perform hierarchical scheduling of packet transmissions from one or more packet queues as well as transmission rate shaping and metering. Traffic manager 1213 may perform congestion management such as flow control, congestion notification message (CNM) generation and reception, priority flow control (PFC), and others.

[0090] Fig. 12B Depicts an example network forwarding system that can be used as a network interface device or router. As described herein, the forwarding system can provide transport layer and security protocol endpoints and perform computations on received data. For example, Fig. 12B Several inlet pipelines 1220, a traffic management unit (referred to as a traffic manager) 1250, and several outlet pipelines 1230 are shown. Although shown as separate structures, in some examples, the inlet pipelines 1220 and the outlet pipelines 1230 can use the same circuit module resources. In some examples, the outlet pipelines 1230 can perform the operations of the collective unit circuit module as described herein.

[0091] The operation of the pipeline can be programmed using a packet processor (P4), C, Python, Broadcom NPL, or x86 compatible executable binary files or other executable binary files that are independent of the programming protocol. In some examples, the pipeline circuit module is configured to synchronously process inlet and / or outlet pipeline packets and non-packet data. In other words, the specific level of the pipeline can process any combination of inlet packets, outlet packets, and non-packet data in the same clock cycle. However, in other examples, the inlet pipeline and the outlet pipeline are separate circuit modules. In some examples of these other examples, the inlet pipeline also processes non-packet data.

[0092] In some examples, in response to receiving a packet, the packet is directed to one of the inlet pipelines 1220, where the inlet pipelines may correspond to one or more ports of a hardware forwarding element. After passing through the selected inlet pipeline 1220, the packet is sent to the service manager 1250, where the packet is enqueued and placed in the output buffer 1254. In some examples, the inlet pipeline 1220 that processes the packet specifies which queue the packet is to be placed into by the service manager 1250 (e.g., based on the destination of the packet or the flow identifier of the packet). The service manager 1250 then dispatches the packet to the appropriate egress pipeline 1230, where the egress pipeline may correspond to one or more ports of a forwarding element. In some examples, there is no necessary correlation between which one of the inlet pipelines 1220 processes the packet and which one of the egress pipelines 1230 the service manager 1250 dispatches the packet to. That is, the packet may initially be processed by the inlet pipeline 1220b after being received through the first port, and then subsequently processed by the egress pipeline 1230a to be sent out from the second port, and so on.

[0093] At least one ingress pipeline 1220 includes a parser 1222, a chain of multiple match-action units or circuit modules (MAUs) 1224, and an inverse parser 1226. Similarly, the egress pipeline 1230 may include a parser 1232, a chain of MAUs 1234, and an inverse parser 1236. In some examples, the parser 1222 or 1232 receives a packet as a formatted set of bits in a specific order and parses the packet into its constituent header fields. In some examples, the parser starts at the beginning of the packet and assigns the header fields to fields (e.g., data containers) for processing. In some examples, the parser 1222 or 1232 separates the packet header (up to a specified point) from the payload of the packet and sends the payload (or the entire packet, including the header and payload) directly to the inverse parser without being processed by the MAU. The egress parser 1232 can use additional metadata provided by the ingress pipeline to simplify its processing.

[0094] MAU 1224 or 1234 can perform processing on packet data. In some examples, the MAU includes a series of levels, each of which includes one or more matching tables and action engines. The matching table may include a set of matching entries, against which packet header fields are matched (e.g., using a hash table), where the matching entries reference action entries. When a packet matches a particular matching entry, the particular matching entry references a particular action entry, which specifies a set of actions to be performed on the packet (e.g., sending the packet to a particular port, modifying one or more packet header field values, discarding the packet, mirroring the packet to a mirror buffer, etc.). The action engine of this level can perform these actions on the packet, which is then sent to the next level of the MAU. For example, as described herein, (one or more) MAUs may provide transport layer and security protocol endpoints and perform calculations on received data.

[0095] The inverse parser 1226 or 1236 may reconstruct the packet using the PHV as modified by the MAU 1224 or 1234 and the payload received directly from the parser 1222 or 1232. The inverse parser may construct a packet that may be sent out over the physical network or to the traffic manager 1250. In some examples, the inverse parser may construct this packet based on data received with the PHV specifying the protocol to be included in the packet header, and its own stored list of data container locations for header fields for each possible protocol.

[0096] Traffic manager (TM) 1250 may include a packet duplicator 1252 and an output buffer 1254. In some examples, traffic manager 1250 may include other components, such as a feedback generator for sending signals about output port failures, a series of queues and schedulers for these queues, a queue status analysis component, and additional components. The packet duplicator 1252 of some examples performs duplication for broadcast / multicast packets, generating multiple packets to be added to the output buffer (e.g., to be distributed to different egress pipelines).

[0097] In some examples, the output buffer 1254 can be part of the queuing and buffering system of the service manager. The service manager 1250 can provide a shared buffer that adapts to any queuing delays in the egress pipeline. In some examples, this shared output buffer 1254 can store packet data, and references (e.g., pointers) to the packet data are stored in different queues for each egress pipeline 1230. The egress pipelines can request their corresponding data from the common data buffer using a queuing strategy configurable by the control plane. When the packet data reference reaches the head of its queue and is scheduled out of the queue, the corresponding packet data can be read out from the output buffer 1254 and read into the corresponding egress pipeline 1230.

[0098] Fig. 12C Depicting an example switch. As described herein, various examples can be used in or with a switch to provide transport layer and security protocol endpoints and perform computations on received data. The switch 1280 may include a network interface 1282 that may provide an Ethernet consistent interface. The network interface 1282 may support 25GbE, 50GbE, 100GbE, 200GbE, 400GbE Ethernet port interfaces. The cryptographic circuit module 1284 may perform at least media access control security (MACsec) or Internet Protocol security (IPSec) decryption of received packets or encryption of packets to be transmitted.

[0099] The various circuit modules may perform one or more of the following: service metering, packet counting, operations, administration and management (OAM), protection engine, instrumentation and telemetry, and clock synchronization (eg, based on IEEE 1588).

[0100] The database 1286 may store profiles of the device in order to configure the operation of the switch 1280. The memory 1288 may include a high bandwidth memory (HBM) for packet buffering. The packet processor 1290 may perform one or more of the following: decisions on the next hop related to packet forwarding, packet counting, access list operations, bridging, routing, multi-protocol label switching (MPLS), virtual private LAN service (VPLS), L2VPN, L3VPN, OAM, data center tunnel encapsulation (e.g., VXLAN and NV-GRE) or other. The packet processor 1290 may include one or more FPGAs. The buffer 1294 may store one or more packets. The traffic manager (TM) 1292 may provide bandwidth guarantees per subscriber according to a service level agreement (SLA) and perform layered quality of service (QoS). The fabric interface 1296 may include a serializer / deserializer (SerDes) and provide an interface to the switching fabric.

[0101] Can be combined Fig. 12A , 12B and / or the operation of the components of the switch of the 12C switch example, and Fig. 12A , 12B and / or 12C example switch components may be included in Fig. 12A , 12B and / or other examples of switches of the example of 12C. For example, the switch system on chip (SoC) may be implemented Fig. 12A , 12BThe switch SoC includes at least one interface to other circuit modules in the switch system. The switch SoC can be coupled to other devices in the switch system, such as ingress or egress ports, memory devices, or host interface circuit modules.

[0102] Fig.13 Depicting the system. In some examples, as described herein, the circuit module may provide transport layer and security protocol endpoints, and perform calculations on received data. System 1300 includes a processor 1310 that provides processing, operation management, and execution of instructions for system 1300. Processor 1310 may include any type of microprocessor, central processing unit (CPU), graphics processing unit (GPU), XPU, processing core, or other processing hardware that provides processing for system 1300, or a combination of processors. XPU may include one or more of the following: CPU, graphics processing unit (GPU), general purpose GPU (GPGPU), and / or other processing units (e.g., accelerator or programmable or fixed function FPGA). Processor 1310 controls the overall operation of system 1300, and may be or include one or more programmable general or special purpose microprocessors, digital signal processors (DSPs), programmable controllers, application specific integrated circuits (ASICs), programmable logic devices (PLDs), etc., or a combination of such devices.

[0103] In one example, system 1300 includes an interface 1312 coupled to processor 1310, which may represent a higher speed interface or a high throughput interface for system components (such as memory subsystem 1320 or graphics interface component 1340 or accelerator 1342) that require a higher bandwidth connection. Interface 1312 represents an interface circuit that may be a separate component or integrated onto a processor die. If present, graphics interface 1340 is connected to a graphics component for providing a visual display to a user of system 1300 via an interface. In one example, graphics interface 1340 may drive a display that provides an output to a user. In one example, the display may include a touch screen display. In one example, graphics interface 1340 generates a display based on data stored in memory 1330 or based on operations performed by processor 1310 or based on both.

[0104] The accelerator 1342 may be a programmable or fixed function offload engine that can be accessed or used by the processor 1310. For example, an accelerator among the accelerators 1342 may provide data compression (DC) capabilities, cryptographic services such as public key encryption (PKE), cryptography, hashing / authentication capabilities, decryption, or other capabilities or services. In some cases, the accelerator 1342 may be integrated into a CPU socket (e.g., a connector to a circuit board or motherboard including a CPU and providing an electrical interface with the CPU). For example, the accelerator 1342 may include a single-core or multi-core processor, a graphics processing unit, a logic execution unit, a single-level or multi-level cache, a functional unit that can be used to independently execute a program or thread, an application specific integrated circuit (ASIC), a neural network processor (NNP), programmable control logic, and a programmable processing element such as a field programmable gate array (FPGA). The accelerator 1342 may provide multiple neural networks, CPUs, processor cores, general-purpose graphics processing units, or may make the graphics processing unit available for use by artificial intelligence (AI) or machine learning (ML) models. For example, the AI ​​model may use or include any or a combination of the following: a reinforcement learning scheme, a Q-learning scheme, deep-Q learning, or asynchronous dominant actor-critic (A3C), a combination neural network, a recursive combination neural network, or other AI or ML models. Multiple neural networks, processor cores, or graphics processing units may be made available to the AI ​​or ML model to perform learning and / or reasoning operations. Example accelerators 1342 include GPUs, TPUs, and Amazon Web Services Trainium.

[0105] The memory subsystem 1320 represents the main memory of the system 1300 and provides storage for code to be executed by the processor 1310 or data values ​​to be used when executing routines. The memory subsystem 1320 may include one or more memory devices 1330, such as read-only memory (ROM), flash memory, one or more random access memories (RAM) (such as DRAM), or other memory devices, or a combination of such devices. The memory 1330 stores and hosts (among other things) an operating system (OS) 1332 to provide a software platform for the execution of instructions in the system 1300. In addition, applications 1334 can be executed on the software platform of the OS 1332 from the memory 1330. Applications 1334 represent programs that have their own operating logic to complete the execution of one or more functions. Processes 1336 represent agents or routines that provide auxiliary functions to the OS 1332 or one or more applications 1334 or a combination. The OS 1332, applications 1334, and processes 1336 provide software logic to provide functions for the system 1300. In one example, the memory subsystem 1320 includes a memory controller 1322, which is a memory controller used to generate and issue commands to the memory 1330. It will be understood that the memory controller 1322 can be a physical part of the processor 1310 or a physical part of the interface 1312. For example, the memory controller 1322 can be an integrated memory controller integrated onto a circuit with the processor 1310.

[0106] Application 1334 and / or process 1336 may alternatively or additionally refer to a virtual machine (VM), a container, a microservice, a processor, or other software. The various examples described herein may execute an application consisting of a microservice, wherein the microservice runs in its own process and communicates using a protocol (e.g., an application program interface (API), a hypertext transfer protocol (HTTP) resource API, a message service, a remote procedure call (RPC), or a Google RPC (gRPC)). Microservices may communicate with each other using a service grid, and may be executed in one or more data centers or edge networks. These services may be independently deployed using centralized management of microservices. The management system may be written in different programming languages ​​and use different data storage technologies. Microservices may be characterized by one or more of the following: polyglot programming (e.g., code written in multiple languages ​​to capture additional functionality and efficiency not available in a single language), or lightweight container or virtual machine deployment, and distributed continuous microservice delivery.

[0107] In some examples, OS 1332 may be Server or personal computer, VMware vSphere, openSUSE, RHEL, CentOS, Debian, Ubuntu, or any other operating system. The OS and drivers can be found at Texas etc., sold or designed to execute on processors.

[0108] Although not specifically shown, it will be understood that system 1300 may include one or more buses or bus systems between devices, such as a memory bus, a graphics bus, an interface bus, or other. A bus or other signal line may couple components together communicatively or electrically, or both communicatively and electrically. A bus may include a physical communication line, a point-to-point connection, a bridge, an adapter, a controller, or other circuit modules, or a combination. A bus may include, for example, a system bus, a peripheral component interconnect (PCI) bus, a hypertransport or industry standard architecture (ISA) bus, a small computer system interface (SCSI) bus, a universal serial bus (USB), or one or more of the Institute of Electrical and Electronics Engineers (IEEE) standard 1394 bus (Firewire).

[0109] In one example, system 1300 includes an interface 1314 that can be coupled to interface 1312. In one example, interface 1314 represents an interface circuit that can include independent components and integrated circuit modules. In one example, multiple user interface components or peripheral components or both are coupled to interface 1314. Network interface 1350 provides the ability to communicate with remote devices (e.g., servers or other computing devices) through one or more networks to system 1300. Network interface 1350 may include an Ethernet adapter, a wireless interconnect component, a cellular network interconnect component, a USB (universal serial bus) or other interfaces based on wired or wireless standards or proprietary. Network interface 1350 can transmit data to devices or remote devices in the same data center or rack, which can include sending data stored in memory. Network interface 1350 can receive data from a remote device, which can include storing received data into memory. In some examples, the packet processing device or network interface device 1350 may refer to one or more of the following: a network interface controller (NIC), a remote direct memory access (RDMA) enabled NIC, a SmartNIC, a router, a switch, a forwarding element, an infrastructure processing unit (IPU), or a data processing unit (DPU). An example IPU or DPU is described with respect to FIG.

[0110] In one example, the system 1300 includes one or more input / output (I / O) interfaces 1360. The I / O interfaces 1360 may include one or more interface components through which a user interacts with the system 1300. The peripheral device interface 1370 may include any hardware interface not specifically mentioned above. A peripheral device generally refers to a device that is connected to the system 1300 in a subordinate manner.

[0111] In one example, the system 1300 includes a storage subsystem 1380 for storing data in a non-volatile manner. In one example, in certain system implementations, at least some components of the storage subsystem 1380 may overlap with components of the memory subsystem 1320. The storage subsystem 1380 includes (one or more) storage devices 1384, which may be or include any conventional media for storing large amounts of data in a non-volatile manner, such as one or more magnetic, solid-state, or optical-based disks or a combination. The storage device 1384 saves code or instructions and data 1386 in a persistent state (e.g., the value is retained despite the interruption of power to the system 1300). Although the memory 1330 is generally an execution or operating memory that provides instructions to the processor 1310, the storage device 1384 may be broadly viewed as "memory." The storage device 1384 is non-volatile, while the memory 1330 may include volatile memory (e.g., if the power to the system 1300 is interrupted, the value or state of the data is uncertain). In one example, storage subsystem 1380 includes controller 1382 that interfaces with storage device 1384. In one example, controller 1382 is a physical part of interface 1314 or processor 1310, or may include circuits or logic in both processor 1310 and interface 1314.

[0112] Volatile memory is memory whose state (and therefore the data stored therein) is undefined if power to the device is interrupted. Non-volatile memory (NVM) devices are memory whose state is defined even if power to the device is interrupted.

[0113] In an example, the system 1300 may be implemented using an interconnected computing sled of processors, memory, storage, network interfaces, and other components. The high-speed interconnect may be based on: Ethernet (IEEE 802.3), Remote Direct Memory Access (RDMA), InfiniBand, Internet Wide Area RDMA Protocol (iWARP), Transmission Control Protocol (TCP), User Datagram Protocol (UDP), Quick UDP Internet Connection (QUIC), RDMA over Converged Ethernet (RoCE), Peripheral Component Interconnect Express (PCIe), Intel Quick Path Interconnect (QPI), Intel Ultra Path Interconnect (UPI), Intel System-on-Chip Fabric (IOSF), Omni-Path, Compute Express Link (CXL), HyperTransport, High-Speed ​​Fabric, NVLink, Advanced Microcontroller Bus Architecture (AMBA) Interconnect, OpenCAPI, Gen-Z, Infinity Fabric (IF), Cache Coherent Interconnect for Accelerators (CCIX), 3GPP Long Term Evolution (LTE) (4G), 3GPP 5G, and variants thereof. Data may be accessed or copied or stored to a virtualized storage node using a protocol such as NVMe over Fabric (NVMe-oF) or NVMe (e.g., a Non-Volatile Memory Express (NVMe) device may operate in a manner consistent with the Non-Volatile Memory Express (NVMe) Specification Revision 1.3c, dated May 24, 2018 (the “NVMe Specification”) or a derivative or variant thereof).

[0114] Communications between devices may occur using a network that provides die-to-die communications, chip-to-chip communications, board-to-board communications, and / or package-to-package communications.

[0115] In an example, the system 1300 may be implemented using an interconnected computing sled of processors, memory, storage, network interfaces, and other components. A high-speed interconnect such as PCIe, Ethernet, or optical interconnect (or a combination thereof) may be used.

[0116] The examples herein may be implemented in various types of computing and networking devices such as switches, routers, racks, and blade servers, such as those employed in data center and / or server farm environments. Servers used in data centers and server farms include arrayed server configurations, such as rack-based servers or blade servers. These servers are interconnected in communications via various network provisions, such as partitioning a collection of servers into local area networks (LANs) with appropriate switching and routing facilities between LANs to form a private intranet. For example, cloud hosting facilities may typically employ large data centers with a large number of servers. Blades include separate computing platforms configured to perform server-type functions, i.e., "servers on cards." Thus, blades include components common to conventional servers, including a main printed circuit board (motherboard) that provides internal wiring (e.g., a bus) for coupling appropriate integrated circuits (ICs) and other components mounted on the board.

[0117] Various examples can be realized using hardware elements, software elements or a combination of the two. In some examples, hardware elements may include devices, components, processors, microprocessors, circuits, circuit elements (e.g., transistors, resistors, capacitors, inductors, etc.), integrated circuits, ASICs, PLDs, DSPs, FPGAs, memory cells, logic gates, registers, semiconductor devices, chips, microchips, chipsets, etc. In some examples, software elements may include software components, programs, applications, computer programs, application programs, system programs, machine programs, operating system software, middleware, firmware, software modules, routines, subroutines, functions, methods, processes, software interfaces, APIs, instruction sets, computing codes, computer codes, code segments, computer code segments, words, values, symbols or any combination thereof. As desired by a given implementation, determining whether to use hardware elements and / or software elements to realize an example may vary according to factors such as any number of the following: desired computing rate, power level, heat resistance, processing cycle budget, input data rate, output data rate, memory resources, data bus speed and other design or performance constraints. A processor may be a hardware state machine, digital control logic, a central processing unit, or one or more combinations of any hardware, firmware, and / or software elements.

[0118] Some examples may be implemented using or as an article of manufacture or at least one computer-readable medium. Computer-readable media may include non-transitory storage media for storing logic. In some examples, non-transitory storage media may include one or more types of computer-readable storage media capable of storing electronic data, including volatile or non-volatile memory, removable or non-removable memory, erasable or non-erasable memory, writable or rewritable memory, and the like. In some examples, logic may include various software elements, such as software components, programs, applications, computer programs, application programs, system programs, machine programs, operating system software, middleware, firmware, software modules, routines, subroutines, functions, methods, processes, software interfaces, APIs, instruction sets, computing codes, computer codes, code segments, computer code segments, words, values, symbols, or any combination thereof.

[0119] According to some examples, a computer-readable medium may include a non-transitory storage medium for storing or maintaining instructions, which, when executed by a machine, a computing device, or a system, causes the machine, the computing device, or the system to perform methods and / or operations according to the described examples. Instructions may include any suitable type of code, such as source code, compiled code, interpreted code, executable code, static code, dynamic code, and the like. Instructions may be implemented according to a predefined computer language, manner, or syntax for instructing a machine, a computing device, or a system to perform a function. Instructions may be implemented using any suitable high-level, low-level, object-oriented, visual, compiled, and / or interpreted programming language.

[0120] One or more aspects of at least one example may be implemented by representative instructions stored on at least one machine-readable medium, which represent various logic within the processor, which when read by a machine, computing device or system causes the machine, computing device or system to fabricate logic to perform the techniques described herein. Such representations, known as "IP cores," may be stored on a tangible, machine-readable medium and supplied to various customers or manufacturing facilities to load into manufacturing machines that actually make the logic or processor.

[0121] The appearance of the phrase "one example" or "example" does not necessarily refer to the same example or embodiment. Any aspect described herein may be combined with any other aspect or similar aspect described herein, regardless of whether the aspects are described with respect to the same figure or element. The division, omission, or inclusion of block functions depicted in the drawings does not imply that hardware components, circuits, software, and / or elements for implementing these functions will necessarily be divided, omitted, or included in the embodiments.

[0122] The expressions "coupled" and "connected," along with their derivatives, may be used to describe some examples. These terms are not necessarily intended to be synonyms for each other. For example, descriptions using the terms "connected" and / or "coupled" may indicate that two or more elements are in direct physical or electrical contact with each other. However, the term "coupled" may also mean that two or more elements are not in direct contact with each other, but still cooperate or interact with each other.

[0123] The terms "first", "second" and the like herein do not represent any order, quantity or importance, but are used to distinguish one element from another element. The terms "one" and "certain" herein do not represent the limitation of quantity, but represent the existence of at least one of the mentioned items. The term "assertion" used in this article about a signal represents a certain state of a signal, in which the signal is active, and the state can be realized by applying any logic level (or logic 0 or logic 1) to the signal. The term "following" or "after..." can refer to following or following after a certain or some other event. According to alternative embodiments, other operation sequences can also be performed. In addition, depending on the specific application, additional operations can be added or removed. Any combination of variations can be used, and those of ordinary skill in the art who benefit from the present disclosure will understand its many changes, modifications and alternative embodiments.

[0124] Unless expressly stated otherwise, disjunctive language such as the phrase "at least one of X, Y, or Z" is understood within the context to be generally used to indicate that an item, term, etc. may be either X, Y, or Z, or any combination thereof (e.g., X, Y, and / or Z). Thus, such disjunctive language is generally not intended to imply, and should not imply, that certain embodiments require that at least one of X, at least one of Y, or at least one of Z each be present. Additionally, conjunctive language such as the phrase "at least one of X, Y, and Z" should also be understood to mean X, Y, Z, or any combination thereof, including "X, Y, and / or Z," unless expressly stated otherwise.

[0125] Illustrative examples of the apparatus, systems, and methods disclosed herein are provided below. Embodiments of the apparatus, systems, and methods may include any one or more of the examples described below and any combination.

[0126] Example 1 includes one or more examples and includes a device, the device including an interface and a circuit module coupled to the interface, the circuit module being configured to: on a remote direct memory access (RDMA) connection with a first network interface device, provide an endpoint for TLS by decrypting transport layer security (TLS) encrypted data from a packet received from the first network interface device; reorder the TLS encrypted data received from the first network interface device before decrypting the TLS encrypted data based on packets received out of order from the first network interface device; on an RDMA connection with a second network interface device, provide an endpoint for TLS by decrypting the TLS encrypted data from a packet received from the second network interface device point; reordering the TLS encrypted data received from the second network interface device before decryption of the TLS encrypted data based on packets received out of order from the second network interface device; providing a transport layer endpoint for packets received from the first network interface device; providing a transport layer endpoint for packets received from the second network interface device, wherein the packets received from the first network interface device and the second network interface device provide data for an in-network computing operation; performing an in-network computing operation on the decrypted data based on a floating point (FP) format; and performing TLS encryption of data generated by the in-network computing operation on the decrypted data before transmitting over an RDMA connection, wherein the in-network computing operation includes a reproducible FP operation.

[0127] Example 2 includes one or more examples wherein the circuit module is to reorder encrypted data in packets received from the first network interface device and the second network interface device based on RDMA sequence numbers in the received packets.

[0128] Example 3 includes one or more examples, wherein the circuit module includes an ingress packet processing pipeline, a traffic manager, and an egress packet processing pipeline, and wherein the egress packet processing pipeline is to perform an in-network computation operation on the reordered data.

[0129] Example 4 includes one or more examples wherein the egress packet processing pipeline is to perform decryption of TLS encrypted data prior to performing in-network compute operations on the reordered data.

[0130] Example 5 includes one or more examples wherein the egress packet processing pipeline is to access a context entry comprising a security context and a collective context.

[0131] Example 6 includes one or more examples, wherein the circuit module is to store data received from the first network interface device and the second network interface device into the first memory buffer and the second memory buffer, and wherein the first memory buffer and the second memory buffer are to be stored from the corresponding first network interface device and the second network interface device.

[0132] Example 7 includes one or more examples wherein the circuit module is to reorder received packets from the first network interface device and the second network interface device based on RDMA sequence numbers in the received packets.

[0133] Example 8 includes one or more examples and includes a switch system on chip (SoC), wherein the switch SoC includes an interface and a circuit module.

[0134] Example 9 includes one or more examples and includes at least one ingress port and at least one egress port communicatively coupled to a switch SoC.

[0135] Example 10 includes one or more examples and includes at least one non-transitory computer-readable medium, the non-transitory computer-readable medium including instructions stored thereon, the instructions, if executed by one or more processors, causing the non-transitory computer-readable medium to configure a network interface device to: on a remote direct memory access (RDMA) connection with a first network interface device, provide an endpoint for TLS by decrypting transport layer security (TLS) encrypted data from packets received from the first network interface device; reorder the TLS encrypted data received from the first network interface device before decryption based on packets received out of order from the first network interface device; on an RDMA connection with a second network interface device, decrypt the TLS encrypted data from packets received from the second network interface device; Decryption of TLS encrypted data of received packets to provide an endpoint for TLS; reordering the TLS encrypted data prior to decryption of the TLS encrypted data received from the second network interface device based on packets received out of order from the second network interface device; providing a transport layer endpoint for packets received from the first network interface device; providing a transport layer endpoint for packets received from the second network interface device, wherein the packets received from the first network interface device and the second network interface device provide data for an in-network computing operation; performing an in-network computing operation on the decrypted data based on a floating point (FP) format; and performing TLS encryption of data generated by the in-network computing operation on the decrypted data before transmitting via an RDMA connection, wherein the in-network computing operation includes a reproducible FP operation.

[0136] Example 11 includes one or more examples and includes instructions stored thereon, which, if executed by one or more processors, cause a non-transitory computer-readable medium to: configure a network interface device to reorder encrypted data in packets received from a first network interface device and a second network interface device based on RDMA sequence numbers in the received packets.

[0137] Example 12 includes one or more examples and includes instructions stored thereon, which, if executed by one or more processors, cause a non-transitory computer-readable medium to: configure a network interface device to store data received from a first network interface device and a second network interface device into a first memory buffer and a second memory buffer, and wherein the first memory buffer and the second memory buffer are to be stored from the respective first network interface device and the second network interface device.

[0138] Example 13 includes one or more examples and includes instructions stored thereon, which, if executed by one or more processors, cause a non-transitory computer-readable medium to: configure a network interface device to reorder received packets based on RDMA sequence numbers in packets received from a first network interface device and a second network interface device.

[0139] Example 14 includes one or more examples wherein the network interface device includes an ingress packet processing pipeline, a traffic manager, and an egress packet processing pipeline, and wherein the egress packet processing pipeline is to perform an in-network computation operation on the reordered data.

[0140] Example 15 includes one or more examples wherein the egress packet processing pipeline is to perform decryption of TLS encrypted data prior to performing an in-network compute operation on the reordered data, and the egress packet processing pipeline is to access a context entry comprising a security context and a collective context.

[0141] Example 16 includes one or more examples, and includes a method, comprising: a network interface device performing: on a remote direct memory access (RDMA) connection with a first network interface device, providing an endpoint for TLS by decrypting transport layer security (TLS) encrypted data from packets received from the first network interface device; reordering the TLS encrypted data received from the first network interface device before decrypting the TLS encrypted data based on packets received out of order from the first network interface device; on an RDMA connection with a second network interface device, providing an endpoint for TLS by decrypting the TLS encrypted data from packets received from the second network interface device; based on packets received out of order from the second network interface device, reordering the TLS encrypted data before decrypting the TLS encrypted data received from the first network interface device; The invention relates to a method for transmitting packets out of order received by a network interface device, reordering the TLS encrypted data prior to decryption of the TLS encrypted data received from a second network interface device; providing a transport layer endpoint for packets received from a first network interface device; providing a transport layer endpoint for packets received from a second network interface device, wherein the packets received from the first network interface device and the second network interface device provide data for an in-network computing operation; performing the in-network computing operation on the decrypted data based on a floating point (FP) format; and performing TLS encryption of data generated by the in-network computing operation on the decrypted data prior to transmission over an RDMA connection, wherein the in-network computing operation includes a reproducible FP operation.

[0142] Example 17 includes one or more examples, and includes the network interface device performing: reordering encrypted data in packets received from the first network interface device and the second network interface device based on RDMA sequence numbers in the received packets.

[0143] Example 18 includes one or more examples wherein the network interface device includes an ingress packet processing pipeline, a traffic manager, and an egress packet processing pipeline, and wherein the egress packet processing pipeline is to perform an in-network computation operation on the reordered data.

[0144] Example 19 includes one or more examples, and includes the network interface device performing: storing data received from the first network interface device and the second network interface device into the first memory buffer and the second memory buffer, and wherein the first memory buffer and the second memory buffer are to be stored from the corresponding first network interface device and the second network interface device.

[0145] Example 20 includes one or more examples, and includes the network interface device performing: reordering packets received from the first network interface device and the second network interface device based on RDMA sequence numbers in the packets received.

Claims

1. A device comprising: interface; as well as A circuit module coupled to the interface, the circuit module being configured to: providing an endpoint for a transport layer security (TLS) endpoint by decrypting TLS encrypted data from packets received from a first network interface device over a remote direct memory access (RDMA) connection with the first network interface device, reordering TLS encrypted data received from said first network interface device prior to said decryption of said TLS encrypted data based on packets received out of order from said first network interface device, providing an endpoint for TLS over the RDMA connection with the second network interface device by decrypting TLS encrypted data from packets received from the second network interface device, reordering the TLS encrypted data received from the second network interface device prior to said decryption of the TLS encrypted data received from the second network interface device based on packets received out of order from the second network interface device, providing a transport layer endpoint for said packets received from said first network interface device, providing a transport layer endpoint for the packets received from the second network interface device, wherein the packets received from the first network interface device and the second network interface device provide data for an in-network computing operation, Performing in-network computational operations on the decrypted data based on floating point (FP) format, and TLS encryption of data generated by the in-network computing operation on the decrypted data is performed prior to transmission over the RDMA connection, wherein the in-network computing operation includes a reproducible FP operation.

2. The device according to claim 1, wherein: The circuit module is to reorder encrypted data in packets received from the first network interface device and the second network interface device based on RDMA sequence numbers in the packets received.

3. The device according to any one of claims 1 to 2, wherein: The circuit module includes an ingress packet processing pipeline, a traffic manager, and an egress packet processing pipeline, and wherein the egress packet processing pipeline is to perform the in-network computing operation on the reordered data.

4. The device of claim 3, wherein: The egress packet processing pipeline is to perform decryption of TLS encrypted data prior to performing the in-network computing operation on the reordered data.

5. The device of claim 3, wherein: The egress packet processing pipeline has access to a context entry including a security context and a collective context.

6. The device according to any one of claims 1 to 5, wherein: The circuit module is to store data received from the first network interface device and the second network interface device into a first memory buffer and a second memory buffer, and wherein the first memory buffer and the second memory buffer are to be stored from the corresponding first network interface device and the second network interface device.

7. The device according to any one of claims 1 to 6, wherein: The circuit module is to reorder the received packets based on RDMA sequence numbers in the packets received from the first network interface device and the second network interface device.

8. The device of any one of claims 1 to 7, comprising a switch system on chip (SoC), wherein: The switch SoC includes the interface and the circuit module.

9. The apparatus of claim 8, comprising at least one ingress port and at least one egress port communicatively coupled to the switch SoC.

10. A non-transitory computer-readable medium comprising instructions stored thereon, which, if executed by one or more processors, cause the non-transitory computer-readable medium to: Configure the network interface device to: providing an endpoint for a transport layer security (TLS) endpoint by decrypting TLS encrypted data from packets received from a first network interface device over a remote direct memory access (RDMA) connection with the first network interface device, reordering TLS encrypted data received from said first network interface device prior to said decryption of said TLS encrypted data based on packets received out of order from said first network interface device, providing an endpoint for TLS over the RDMA connection with the second network interface device by decrypting TLS encrypted data from packets received from the second network interface device, reordering the TLS encrypted data received from the second network interface device prior to said decryption of the TLS encrypted data received from the second network interface device based on packets received out of order from the second network interface device, providing a transport layer endpoint for said packets received from said first network interface device, providing a transport layer endpoint for the packet received from the second network interface device, wherein: The packets received from the first network interface device and the second network interface device provide data for an in-network computing operation, Performing in-network computational operations on the decrypted data based on floating point (FP) format, and TLS encryption of data generated by the in-network computing operation on the decrypted data is performed prior to transmission over the RDMA connection, wherein the in-network computing operation includes a reproducible FP operation.

11. The non-transitory computer readable medium of claim 10, comprising instructions stored thereon that, if executed by one or more processors, cause the non-transitory computer readable medium to: The network interface device is configured to reorder encrypted data in packets received from the first network interface device and the second network interface device based on RDMA sequence numbers in the received packets.

12. The non-transitory computer-readable medium of any one of claims 10 to 11, comprising instructions stored thereon that, if executed by one or more processors, cause the non-transitory computer-readable medium to: The network interface device is configured to store data received from the first network interface device and the second network interface device into a first memory buffer and a second memory buffer, and wherein, The first memory buffer and the second memory buffer are to be stored from the respective first network interface device and the second network interface device.

13. The non-transitory computer-readable medium of any one of claims 10 to 12, comprising instructions stored thereon that, if executed by one or more processors, cause the non-transitory computer-readable medium to: The network interface device is configured to reorder received packets from the first network interface device and the second network interface device based on RDMA sequence numbers in the received packets.

14. The non-transitory computer readable medium of any one of claims 10 to 13, wherein the network interface device comprises an ingress packet processing pipeline, a traffic manager, and an egress packet processing pipeline, and wherein, The egress packet processing pipeline is to perform the in-network computational operations on the reordered data.

15. The non-transitory computer-readable medium of any one of claims 10 to 14, wherein the egress packet processing pipeline is to perform decryption of TLS encrypted data prior to performing the in-network computing operation on the reordered data, and The egress packet processing pipeline has access to a context entry including a security context and a collective context.

16. A method comprising: The network interface device performs: providing an endpoint for a transport layer security (TLS) endpoint by decrypting TLS encrypted data from packets received from a first network interface device over a remote direct memory access (RDMA) connection with the first network interface device, reordering TLS encrypted data received from said first network interface device prior to said decryption of said TLS encrypted data based on packets received out of order from said first network interface device, providing an endpoint for TLS over the RDMA connection with the second network interface device by decrypting TLS encrypted data from packets received from the second network interface device, reordering the TLS encrypted data received from the second network interface device prior to said decryption of the TLS encrypted data received from the second network interface device based on packets received out of order from the second network interface device, providing a transport layer endpoint for said packets received from said first network interface device, providing a transport layer endpoint for the packets received from the second network interface device, wherein the packets received from the first network interface device and the second network interface device provide data for an in-network computing operation, Performing in-network computational operations on the decrypted data based on floating point (FP) format, and TLS encryption of data generated by the in-network computing operation on the decrypted data is performed prior to transmission over the RDMA connection, wherein the in-network computing operation includes a reproducible FP operation.

17. The method of claim 16, comprising: The network interface device performs: Encrypted data in packets received from the first network interface device and the second network interface device are reordered based on RDMA sequence numbers in the received packets.

18. The method according to any one of claims 16 to 17, wherein: The network interface device comprises an ingress packet processing pipeline, a traffic manager and an egress packet processing pipeline, and wherein the egress packet processing pipeline is to perform the in-network computation operation on the reordered data.

19. The method of any one of claims 16 to 18, comprising: The network interface device performs: Data received from the first network interface device and the second network interface device are stored in a first memory buffer and a second memory buffer, and wherein the first memory buffer and the second memory buffer are to be stored from the respective first network interface device and the second network interface device.

20. The method of any one of claims 16 to 19, comprising: The network interface device performs: The received packets are reordered based on RDMA sequence numbers in packets received from the first network interface device and the second network interface device.

Images