Reliable memory transaction transfer

Abstract

The present disclosure relates to reliable memory transaction transmission. In some embodiments, an apparatus can include circuitry configured to send a plurality of data packets each corresponding to one or more memory transactions and including a respective packet sequence number (PSN). The circuitry can receive a first data packet from an endpoint, determine a first expected packet sequence number (EPSN) for the first data packet and a first PSN included in the first data packet. In response to determining that the first EPSN is equal to the first PSN, the circuitry can send an acknowledgement (ACK) message to the endpoint identifying a second EPSN and indicating that the first data packet is an in-order data packet. In response to determining that the first EPSN is not equal to the first PSN, the circuitry can send a negative acknowledgement (NACK) message to the endpoint identifying the first EPSN and indicating that the first data packet is an out-of-order data packet.

Description

Cross-reference of related applications

[0001] This application claims the benefit and priority of U.S. Provisional Application No. 63 / 794,854, filed April 25, 2025; U.S. Provisional Application No. 63 / 823,615, filed June 13, 2025; and U.S. Provisional Application No. 63 / 839,762, filed July 7, 2025, all of which are incorporated herein by reference in their entirety for all purposes. Technical Field

[0002] This disclosure generally relates to systems and methods for network interface architectures, and more specifically, to Ethernet-based load-store systems that provide reliable memory transaction transfer between computing endpoints. Background Technology

[0003] In decoupled and heterogeneous computing systems, endpoints of CPUs, GPUs, DPUs (Data Processing Units), and memory controllers may increasingly require remote load / store semantics extending beyond board or slot boundaries. This necessitates the transfer of memory transactions across network architectures operating at 100G / 200G / 400G and higher line rates. Summary of the Invention

[0004] In one aspect, this disclosure relates to an apparatus comprising: a network interface; and circuitry coupled to the network interface and configured to: transmit via the network interface a plurality of packets, each corresponding to one or more memory transactions and containing a corresponding packet sequence number (PSN); receive a first packet from an endpoint via the network interface; determine a first expected packet sequence number (EPSN) of the first packet and a first PSN contained in the first packet; update the first EPSN to a second EPSN in response to determining that the first EPSN is equal to the first PSN and transmit via the network interface an acknowledgment (ACK) message identifying the second EPSN and indicating that the first packet is an ordered packet to the endpoint; and transmit via the network interface a negative acknowledgment (NACK) message identifying the first EPSN and indicating that the first packet is an out-of-order packet to the endpoint in response to determining that the first EPSN is not equal to the first PSN.

[0005] In another aspect, this disclosure relates to a circuit coupled to a network interface and configured to: transmit via the network interface a plurality of packets, each corresponding to one or more memory transactions and containing a corresponding packet sequence number (PSN); receive a first packet from an endpoint via the network interface; determine a first expected packet sequence number (EPSN) of the first packet and a first PSN contained in the first packet; update the first EPSN to a second EPSN in response to determining that the first EPSN is equal to the first PSN and send an acknowledgment (ACK) message identifying the second EPSN and indicating that the first packet is an ordered packet to the endpoint via the network interface; and send a negative acknowledgment (NACK) message identifying the first EPSN and indicating that the first packet is an out-of-order packet to the endpoint via the network interface in response to determining that the first EPSN is not equal to the first PSN.

[0006] In another aspect, this disclosure relates to a method comprising: transmitting, via a network interface, a plurality of packets, each corresponding to one or more memory transactions and containing a corresponding packet sequence number (PSN); receiving, via the network interface, a first packet from an endpoint; determining, via the circuit, a first expected packet sequence number (EPSN) of the first packet and a first PSN contained in the first packet; in response to determining that the first EPSN is equal to the first PSN, updating the first EPSN to a second EPSN by the circuit and transmitting, via the network interface, an acknowledgment (ACK) message identifying the second EPSN and indicating that the first packet is an ordered packet to the endpoint; and in response to determining that the first EPSN is not equal to the first PSN, transmitting, via the network interface, a negative acknowledgment (NACK) message identifying the first EPSN and indicating that the first packet is an out-of-order packet to the endpoint. Attached Figure Description

[0008] The various objects, aspects, features, and advantages of this disclosure will become more apparent and better understood through a detailed description taken in conjunction with the accompanying drawings, in which similar reference numerals identify corresponding elements throughout. In the drawings, the same reference numerals generally indicate the same, functionally similar, and / or structurally similar elements.

[0009] Figure 1A It is a block diagram depicting a network environment comprising one or more access points communicating with one or more devices or stations, according to some embodiments.

[0010] Figure 1B and 1C This is a block diagram depicting a computing device used in conjunction with the methods and systems described herein, according to some embodiments.

[0011] Figure 2 The description describes an example system environment based on some embodiments, in which an Ethernet-based load-store system integrates a system-on-a-chip (SoC) with one or more transport subsystems.

[0012] Figure 3 This is a diagram illustrating the data path and associated credit loop in an Ethernet-based load-store system according to some embodiments.

[0013] Figure 4 This is a diagram illustrating the data path connection between the SoC logic and the Ethernet-based load-memory system according to some embodiments.

[0014] Figure 5 This is a diagram illustrating an interface for transmitting request and response transactions between a source endpoint and a destination endpoint, according to some embodiments.

[0015] Figure 6 This is a diagram illustrating the credit loop established between the SoC and the Ethernet-based load-memory system according to some embodiments.

[0016] Figure 7 This is a diagram illustrating the bit mapping and flow of write requests and write responses between SoC logic and an Ethernet-based load-memory system according to some embodiments.

[0017] Figure 8 This is a diagram illustrating the bit mapping and flow of read requests and read responses between SoC logic and an Ethernet-based load-memory system according to some embodiments.

[0018] Figure 9 This is a diagram illustrating an example network transmission path of an Ethernet-based load-store system according to some embodiments.

[0019] Figure 10 This is a diagram illustrating an example network receive path of an Ethernet-based load-store system according to some embodiments.

[0020] Figure 11 This is a diagram illustrating an example of the integration of an Ethernet-based load-store system with SoC logic according to some embodiments.

[0021] Figure 12 This is a flowchart illustrating an example process for transmitting memory transactions over an Ethernet network, according to some embodiments.

[0022] Figure 13 This is a flowchart illustrating another instance of a process for transmitting memory transactions over an Ethernet network, according to some embodiments.

[0023] Details of various embodiments of the method and system are set forth in the accompanying drawings and the description below. Detailed Implementation

[0024] The following disclosure provides numerous different embodiments or instances for implementing various features of the provided subject matter. Specific examples of components and arrangements are described below to simplify this disclosure. These are, of course, merely examples and are not intended to be limiting. For example, in the following description, a first feature communicating with or communicatively coupled to a second feature may include embodiments in which the first feature directly communicates with or is directly coupled to the second feature, and may also include embodiments in which an additional feature may intervene between the first and second features, such that the first feature indirectly communicates with or is indirectly coupled to the second feature. Additionally, in various instances, references to numbers and / or letters may be repeated. This repetition is for simplicity and clarity and does not in itself indicate a relationship between the various embodiments and / or configurations discussed.

[0025] The following IEEE standards (including any draft versions of such standards) are hereby incorporated herein by reference in their entirety and constitute the part disclosed herein for all purposes: IEEE 802.1 and IEEE 802.3 standards, including but not limited to IEEE 802.1Q. TM IEEE 802.1AX TM IEEE 802.1CB TM IEEE 802.3ae TM IEEE 802.3ba TM IEEE 802.3bj TM IEEE 802.3by TM IEEE 802.3cd TM and IEEE 802.3ck TM Standards. Although this disclosure may reference certain aspects of these standards, it is in no way limited by them.

[0026] The following description of sections and corresponding contents of this specification may be helpful in reading the descriptions of the various embodiments described below:

[0027] Section A describes the network and computing environments that can be used to practice the embodiments described herein; and

[0028] Section B describes embodiments of systems and methods for providing extended Ethernet frameworks.

[0029] A. Computing and Network Environment

[0030] Before discussing specific embodiments of the solutions of the present invention, it may be helpful to describe the operating environment and associated system components (e.g., hardware elements) in conjunction with the methods and systems described herein. References Figure 1A This describes an embodiment of a network environment. In brief, the network environment includes a wireless communication system comprising one or more access points (APs) or network devices 106, one or more stations or wireless communication devices 102, and network hardware components or network hardware 192. Wireless communication device 102 may, for example, include a laptop computer, tablet computer, personal computer, and / or cellular phone device. Reference Figure 1B and 1C More detailed descriptions are provided for embodiments of each station or wireless communication device 102 and AP or network device 106. In one embodiment, the network environment may be an ad hoc network environment, an infrastructure wireless network environment, a subnet environment, etc. Network device 106 or AP may be operatively coupled to network hardware 192 via a local area network (LAN) connection. In some embodiments, network device 106 is a 5G base station. Network hardware 192, which may include routers, gateways, switches, bridges, modems, system controllers, appliances, etc., may provide LAN connectivity for the communication system. Each of network device 106 or AP may have an associated antenna or antenna array for communicating with wireless communication devices in its area. Wireless communication device 102 may register with a specific network device 106 or AP to receive services from the communication system (e.g., via SU-MIMO or MU-MIMO configuration). For direct connections (e.g., point-to-point communication), some wireless communication devices may communicate directly via assigned channels and communication protocols. Some of wireless communication devices 102 may be mobile or relatively stationary relative to network device 106 or AP.

[0031] In some embodiments, network device 106 or AP includes means or modules (comprising a combination of hardware and software) that allow wireless communication device 102 to connect to a wired network using Wi-Fi or other standards. Network device 106 or AP may sometimes be referred to as a wireless access point (WAP). Network device 106 or AP can be implemented (e.g., configured, designed, and / or built) for operation in a wireless local area network (WLAN). In some embodiments, network device 106 or AP can be connected as a standalone device to a router (e.g., via a wired network). In other embodiments, network device 106 or AP can be a component of a router. Network device 106 or AP can provide network access to multiple devices. Network device 106 or AP can, for example, connect to a wired Ethernet connection and use a radio frequency link to provide wireless connectivity for other devices 102 to utilize that wired connection. Network device 106 or AP can be implemented to support standards for transmitting and receiving data using one or more radio frequencies. Those standards and the frequencies they use may be defined by IEEE (e.g., the IEEE 802.11 standard). Network device 106 or AP may be configured and / or used to support public Internet hotspots and / or extend the Wi-Fi signal range of the network over the network.

[0032] In some embodiments, access point or network device 106 may be used for (e.g., in a home, vehicle, or building) wireless networks (e.g., IEEE 802.11, Bluetooth, ZigBee, any other type of radio frequency-based network protocol, and / or variations thereof). Each of the wireless communication devices 102 may include a built-in radio and / or be coupled to a radio. Such wireless communication devices 102 and / or access point or network device 106 may operate according to various aspects of the present disclosure as presented herein to enhance performance, reduce cost and / or size, and / or enhance broadband applications. Each wireless communication device 102 may have the capability to act as a client node seeking access to resources (e.g., data and connections to networked nodes such as servers) via one or more access point or network devices 106.

[0033] The network connection may include any type and / or form of network, and may include any of the following: point-to-point network, broadcast network, telecommunications network, data communication network, computer network. The network topology may be a bus, star, or ring network topology. The network may have any such network topology known to those skilled in the art capable of supporting the operations described herein. In some embodiments, different types of data may be transmitted via different protocols. In other embodiments, the same type of data may be transmitted via different protocols.

[0034] The communication device 102 and the access point or network device 106 can be deployed as any type and form of computing device and / or executed on any type and form of computing device, such as a computer, network device or appliance capable of communicating and performing the operations described herein on any type and form of network. Figure 1B and 1C A block diagram depicting a computing device 100 that can be used to implement embodiments of wireless communication device 102 or network device 106. (See diagram for reference.) Figure 1B and 1C As shown, each computing device 100 includes a processor 121 (e.g., a central processing unit) and a main memory unit 122. Figure 1B As shown, the computing device 100 may include a storage device 128, a mounting device 116, a network interface 118, an I / O controller 123, display devices 124a to 124n, a keyboard 126, and a pointing device 127, such as a mouse. The storage device 128 may include an operating system and / or software. Figure 1C As shown, each computing device 100 may also include additional optional components such as memory port 103, bridge 170, one or more input / output devices 130a to 130n, and cache memory 140 that communicates with the central processing unit or processor 121.

[0035] The central processing unit or processor 121 is any logic circuit system that responds to and processes instructions fetched from main memory unit 122. In many embodiments, the central processing unit or processor 121 is provided by a microprocessor unit, such as a microprocessor unit manufactured by Intel Corporation of Santa Clara, California; a microprocessor unit manufactured by International Business Machines of White Plains, New York; or a microprocessor unit manufactured by Advanced Micro Devices of Sunnyvale, California. The computing device 100 may be based on any of these processors, or any other processor capable of operating as described herein.

[0036] Main memory unit 122 may be one or more memory chips capable of storing data and allowing direct access to any storage location by a microprocessor or processor 121, such as any type or variant of static random access memory (SRAM), dynamic random access memory (DRAM), ferroelectric RAM (FRAM), NAND flash memory, NOR flash memory, and solid-state drive (SSD). Main memory unit 122 may be based on any of the memory chips described above, or any other available memory chip capable of operating as described herein. Figure 1B In the embodiment shown, the processor 121 communicates with the main memory unit 122 via the system bus 150 (described in more detail below). Figure 1C An embodiment of computing device 100 is depicted, wherein the processor communicates directly with main memory unit 122 via memory port 103. For example, in Figure 1C In this context, the main memory unit 122 can be DRDRAM.

[0037] Figure 1C An embodiment is depicted in which the main processor 121 communicates directly with the cache memory 140 via an auxiliary bus (sometimes referred to as a back-side bus). In other embodiments, the main processor 121 uses a system bus 150 to communicate with the cache memory 140. The cache memory 140 typically has a faster response time than the main memory cell 122 and is provided by, for example, SRAM, BSRAM, or EDRAM. Figure 1C In the embodiments shown, processor 121 communicates with various I / O devices 130 via local system bus 150. Various buses can be used to connect the central processing unit or processor 121 to any of the I / O devices 130, such as VESA VL bus, ISA bus, EISA bus, Micro Channel Architecture (MCA) bus, PCI bus, PCI-X bus, PCI Fast Bus, or NuBus. In embodiments where the I / O device is a video display 124, processor 121 can communicate with the display 124 using an Advanced Graphics Port (AGP). Figure 1C An embodiment of a computer or computer system 100 is depicted, wherein the main processor 121 may communicate directly with the I / O device 130b, for example, via HYPERTRANSPORT, RAPIDIO, or INFINIBAND communication technologies. Figure 1C An embodiment in which local bus and direct communication are mixed is also depicted: processor 121 communicates with I / O device 130a using local interconnect bus, while communicating directly with I / O device 130b.

[0038] Various I / O devices 130a to 130n may be present in the computing device 100. Input devices include keyboards, mice, trackpads, trackballs, microphones, dials, touchpads, touchscreens, and drawing tablets. Output devices include video displays, speakers, inkjet printers, laser printers, projectors, and dye-to-sublimation printers. The I / O devices can be controlled by the I / O controller 123, such as... Figure 1B As shown in the figure. The I / O controller can control one or more I / O devices, such as keyboard 126 and pointing device 127, such as mouse or optical pen. In addition, the I / O devices can also provide storage and / or mounting media for computing device 100. In other embodiments, computing device 100 can provide USB connectivity (not shown) to receive handheld USB storage devices, such as the USB flash drive series devices manufactured by Twintech Industry, Inc. of Los Alamitos, California.

[0039] Refer again Figure 1B The computing device 100 may support any suitable installation device 116, such as a disk drive, CD-ROM drive, CD-R / RW drive, DVD-ROM drive, flash drive, tape drive of various formats, USB device, hard disk drive, network interface, or any other device suitable for installing software and programs. The computing device 100 may further include storage devices for storing the operating system and other related software, and for storing application software programs (e.g., any program or software 120 for implementing (e.g., configured and / or designed for) the systems and methods described herein), such as one or more hard disk drives or redundant arrays of independent disks. Optionally, any of the installation devices 116 may also be used as storage devices. Furthermore, the operating system and software can be run from bootable media.

[0040] Furthermore, the computing device 100 may include a network interface 118 for interfacing with a network via various connections, including but not limited to standard telephone lines, LAN or WAN links (e.g., 802.11, T1, T3, 56kb, X.25, SNA, DECNET), broadband connections (e.g., ISDN, Frame Relay, ATM, Gigabit Ethernet, SONET-based Ethernet), wireless connections, or some combination of any and all of the above. Connections can be established using various communication protocols (e.g., TCP / IP, IPX, SPX, NetBIOS, Ethernet, ARCNET, SONET, SDH, Fiber Distributed Data Interface (FDDI), RS232, IEEE 802.11, IEEE 802.11a, IEEE 802.11b, IEEE 802.11g, IEEE 802.11n, IEEE 802.11ac, IEEE 802.11ad, CDMA, GSM, WiMax, and direct asynchronous connections). In one embodiment, computing device 100 communicates with other computing devices 100' via any type and / or form of gateway or tunneling protocol, such as Secure Sockets Layer (SSL) or Transport Layer Security (TLS). Network interface 118 may include a built-in network adapter, network interface card, PCMCIA network card, card bus network adapter, wireless network adapter, USB network adapter, modem, or any other device suitable for interfacing computing device 100 to any type of network capable of communicating and performing the operations described herein.

[0041] In some embodiments, computing device 100 may include or be connected to one or more display devices 124a to 124n. Therefore, any of the I / O devices 130a to 130n and / or I / O controller 123 may include any type and / or form of suitable hardware, software, or a combination of hardware and software to support, enable, or provide computing device 100 with connectivity to and use of display devices 124a to 124n. For example, computing device 100 may include any type and / or form of video adapter, video card, driver, and / or library to interface with, communicate with, connect to, or otherwise use display devices 124a to 124n. In one embodiment, a video adapter may include multiple connectors to interface with display devices 124a to 124n. In other embodiments, computing device 100 may include multiple video adapters, each connected to display devices 124a to 124n. In some embodiments, any portion of the operating system of computing device 100 may be configured to use multiple display devices 124a to 124n. In other embodiments, I / O device 130 may be a bridge between system bus 150 and external communication buses (such as USB bus, Apple Desktop bus, RS-232 serial connection, SCSI bus, FireWire bus, FireWire 800 bus, Ethernet bus, AppleTalk bus, Gigabit Ethernet bus, Asynchronous Transfer Mode bus, Fibre Channel bus, Fibre Channel bus, Serial Attached Small Computer System Interface bus, USB connection, or HDMI bus).

[0042] Figure 1B and 1CThe computing device 100 of the type described herein can operate under the control of an operating system that controls task scheduling and access to system resources. The computing device 100 can run any operating system, such as any version of Microsoft Windows, different versions of Unix and Linux, any version of MAC OS for Macintosh computers, any embedded operating system, any real-time operating system, any open-source operating system, any proprietary operating system, any operating system for mobile computing devices, or any other operating system capable of running on a computing device and performing the operations described herein. Typical operating systems include, but are not limited to: Android, produced by Google Inc.; Windows 7, 8, and 10, produced by Microsoft Corporation of Redmond, Washington; macOS, produced by Apple Computer of Cupertino, California; WebOS, produced by Research In Motion (RIM); OS / 2, produced by International Business Machines of Armonk, New York; and Linux, a free operating system distributed by Caldera Corp. of Salt Lake City, Utah, or any type and / or form of Unix operating system, etc.

[0043] The computer system or computing device 100 may be any workstation, telephone, desktop computer, laptop or notebook computer, server, handheld computer, mobile phone or other portable telecommunications device, media playback device, gaming system, mobile computing device, or any other type and / or form of computing, telecommunications or media device capable of communication. In some embodiments, the computing device 100 may have a different processor, operating system and input device consistent with the device. For example, in one embodiment, the computing device 100 is a smartphone, mobile device, tablet computer or personal digital assistant. Furthermore, the computing device 100 may be any workstation, desktop computer, laptop or notebook computer, server, handheld computer, mobile phone, any other computer, or other form of computing or telecommunications device capable of communication and having sufficient processor power and memory capacity to perform the operations described herein.

[0044] The aspects of the operating environment and components described above will become apparent in the context of the systems and methods disclosed herein.

[0045] B. Systems and methods for providing reliable memory transaction transfers between computing endpoints

[0046] In one aspect, when transmitting memory transactions over Ethernet, semantic mismatches may exist between bus-level memory transactions and packet streams, where address / control and data phases travel as separate channels on the compute side but may need to be serialized into packet payloads and headers for network transport where frames may be reordered, copied, or dropped. This mismatch can complicate the pairing of write addresses and write data, the correlation of responses and requests across many incomplete operations, and the enforcement of burst size and alignment constraints without incurring excessive per-packet overhead. Additional challenges arise from the need to achieve high line efficiency for small, latency-sensitive operations by packaging multiple transactions per packet, while avoiding head-of-line penalties for the earliest aggregation operations, partial packet loss ambiguity, and / or excessive queuing delays. Furthermore, endpoints and intermediate devices exhibit asymmetrical production and consumption rates, and admission control needs to be protected at the transaction granularity of limited interfaces and packetization buffers, while supporting multi-tenant isolation and QoS partitioning to prevent one traffic class or destination from causing congestion breakdowns in other traffic classes.

[0047] On the other hand, reliable delivery over Ethernet remains challenging due to transient congestion, micro-bursts, and rare physical layer errors, which manifest as loss, duplication, or corruption. Any of these, if not detected and recovered in a timely manner, can violate the strict ordering expectations of memory semantics. Timely and bandwidth-efficient feedback is difficult to achieve because acknowledgment signaling and data contend for reverse path capacity, and overly sparse feedback increases retransmission latency and in-flight buffer occupancy, while overly frequent feedback increases control overhead and processing stress at high transaction rates. The choice of retransmission strategy is not insignificant: when loss is sparse but ordered, backtracking recovery wastes bandwidth, while selective retransmission requires deeper receiver state and reassembly capabilities.

[0048] On another front, congestion visibility and control are limited on conventional Ethernet paths; static or poorly tuned congestion windows may underutilize the architecture or lead to persistent queue and tail latency inflation, and their interaction with link-level flow control mechanisms (such as pause-based schemes) can create pathological pause propagation, deadlock risks, or starvation in multi-priority deployments. Furthermore, variable round-trip times, clock domain tolerances, and asymmetric traffic patterns that impede reverse path signaling can disrupt accurate recovery timing; misconfigured timers or static thresholds can trigger spurious retransmissions or delayed loss detections, amplify head-end congestion, and reduce the overall actual throughput of memory transaction traffic.

[0049] To address these limitations, this disclosure provides an Ethernet-based load-and-store system (referred to as the "System") that provides reliable memory transaction transfer between computing endpoints. In some embodiments, the term "load-and-store system" may refer to a network attachment subsystem that provides remote load (read) and store (write) semantics via packetized transactions (e.g., Advanced Extensible Interface (AXI) transactions) transmitted over high-speed Ethernet and de-packetized back into AXI at the host boundary. The System may be integrated with an Ethernet device (also referred to as an "Ethernet port device" or "Ethernet PHY") operating at, for example, a line rate of 400 Gb / s, and may employ reliable transmission with acknowledgment and negative acknowledgment, lossless flow control per virtual channel, and credit-based flow control per AXI channel to maintain order and prevent overflow. In some embodiments, the Ethernet device may implement or include one or more Ethernet port functions, including, but not limited to, Media Access Control (MAC), Physical Writer Device (PCD) or Physical Writer Sublayer (PCS), Physical Media Dependent Layer (PMD), and / or Serializer / Deserializer (SERDES) and their associated functions.

[0050] In some embodiments, an apparatus for transmitting memory transactions over an Ethernet network may include a first interface, a second interface, and circuitry. The term "memory transaction" can refer to an operation that communicates read, write, or associated control semantics between devices, including address / control information and, where applicable, associated data for a fixed-length burst of fixed size. In some embodiments, such transactions originate from a host-visible channel and are translated into packet payloads for transmission over the architecture while maintaining the relevance between requests and responses. The term "Ethernet network" can refer to a packet-switched communication architecture capable of carrying Ethernet frames at high line rates and supporting encapsulation suitable for transaction payloads, including environments employing MAC / PCS functionality and optional link-level reliability or flow control features. The term "first interface" can refer to a host-facing interface that communicates with a computing subsystem using multiple logical channels that decouple address / control from data and responses and accept or send transmissions (of fixed or variable size) compatible with the host's bus semantics. The term "second interface" can refer to a network-facing interface configured to exchange Ethernet packets with the architecture and host host link-layer functions, including, but not limited to, the MAC processing circuitry and PCS processing circuitry that together implement a high-speed Ethernet port. The term "circuit" can refer to the logic coupled between the first and second interfaces, which generates and interprets network packets in a transaction-aware manner, including functions such as packetization requests, depacketization responses, integrity checks, ordering management, and interaction with per-channel flow control mechanisms.

[0051] In some embodiments, the first interface may be configured to communicate with a host device. The term "host device" may refer to a processor complex, SoC, accelerator, or other computing entity that issues and consumes memory transactions through the first interface (e.g., by presenting and / or receiving write addresses and write data on different channels and receiving corresponding responses after network transmission). The second interface may be configured to exchange Ethernet packets with an Ethernet network. The term "Ethernet packet" may refer to a frame transmitted over an Ethernet network that encapsulates one or more transaction units and a transport header, potentially aggregating multiple operations destined for a common endpoint up to a configured packet size threshold while preserving the atomicity of each transaction. Circuitry may be coupled to the first and second interfaces and configured to packetize one or more memory transaction requests received from the first interface into a first Ethernet packet. The term "packetization" may refer to forming a first Ethernet packet from one or more memory transaction requests received at the first interface, including arranging opaque control fields and data into a packaged payload, assigning transport metadata (e.g., destination identifiers and sequence numbers), and respecting configurations (e.g., maximum packet size) without splitting individual transactions. The term "memory transaction request" can refer to a host-initiated operation that initiates memory semantics, such as a read address or a write request including a write address and corresponding write data. The circuit aligns and aggregates the memory semantics for efficient transmission over an Ethernet network. The circuit can be configured to transmit a first Ethernet packet to the Ethernet network via a second interface. The circuit can be configured to depacketize a second Ethernet packet received from the second interface into one or more memory transaction responses or one or more transaction requests. The term "depacketization" can refer to processing the second Ethernet packet received at the second interface to recover one or more memory transaction responses or one or more memory transaction requests, thereby verifying integrity, interpreting transmission fields, and reconstructing channel-specific response information for presentation to the host device. The circuit can be configured to transmit one or more memory transaction responses or one or more memory transaction requests to the host device via a first interface. The term "memory transaction request" can refer to one or more requests for reading data or one or more requests for writing data issued by the circuit to the host device via the first interface. For example, the host device can issue a write or read request to the first interface. These transactions can be packetized and transmitted to the second interface. These packetized requests can be received by a second interface of another Ethernet load-store subsystem.

[0052] The term "memory transaction response" can refer to a completion associated with a previous request, such as read data with a response code or a write acknowledgment associated with an earlier write, which the circuitry issues to the host device via a first interface according to the host's sorting and identification rules. For example, a request from another Ethernet load-memory subsystem can be received, de-packetized, and then transmitted to the host device at a second interface. These requests can be processed by the host device (specifically, the memory subsystem). This processing can then result in a "response" transaction, which is then returned to the Ethernet subsystem and / or host device that sent the request. Response transactions can follow the same flow as requests, but in the reverse direction. That is, a response can be transmitted to the first interface, packetized, and then sent out via the second interface.

[0053] In some embodiments, the second interface may include a Media Access Control (MAC) processing circuitry and a Physical Write Sublayer (PCS) processing circuitry. The term "Media Access Control (MAC) processing circuitry" may refer to the link-layer logic within the second interface that performs Ethernet framing, delimitation, checksum and disposal, and prioritization or pause behavior consistent with high-rate operation, and cooperates with higher-level transport functions carrying transaction payloads. The term "Physical Write Sublayer (PCS) processing circuitry" may refer to the physical-layer logic within the second interface that encodes and decodes symbols for multi-channel operation, including scrambling, alignment, forward error correction (FEC), and channel distribution, thereby enabling robust delivery of Ethernet packets encapsulating memory transactions.

[0054] In some embodiments, the first interface may be configured to communicate with the host device using multiple channels. The term "multiple channels" may refer to disparate logical paths on the first interface that separate address / control, data, and response, thereby allowing independent flow control and skew tolerance, enabling the address and data phases of a write request to arrive on different channels while maintaining circuit compatibility. The first interface may be configured to receive a memory write request from the host device, comprising a write address and write data received through different channels of the multiple channels. The term "memory write request" may refer to a host-initiated operation containing a write address and corresponding write data communicated on different channels of the first interface, the circuit aligning and encapsulating the write address and corresponding write data into a single transaction unit for transmission. The circuit may be configured to align the write address and write data before packetizing the memory write request. The term "write address" may refer to an address identifying the destination location and attributes of the write operation and associated control metadata, the circuit associating the destination location and attributes with the corresponding data before packetization. The term "write data" can refer to the payload bytes of a write operation communicated on a data channel, such as when circuitry pairs it with a write address and then includes it in a fixed-length or variable-length tick within a packetized transaction. The term "align write address and write data" can refer to associating the address / control phase and data phase of a write operation so that, despite independent channel timing, circuitry forms a coherent transaction boundary before packetization, thereby preserving the pairing semantics of hosts across the network.

[0055] In some embodiments, the circuitry may maintain one or more per-channel transmit queues. The term "per-channel transmit queue" may refer to a queue structure maintained by the circuitry for a given channel, such that transactions are graded, ordered, and rate-matched before packetization, thereby allowing progress to continue on the channel with available capacity even when another channel experiences temporary backpressure.

[0056] In some embodiments, the circuitry may be configured to perform per-channel credit-based flow control between the sender and receiver of one or more transactions. The sender (e.g., a host device) may decrement a credit counter after issuing a transaction, and the receiver (e.g., an Ethernet subsystem) may return credits to the sender when buffer resources (e.g., memory buffers) consumed by the transmitted transaction are released. The term "credit" may refer to a permission token representing the receiver's available buffer for a given channel, wherein the sender consumes credits when issuing a transaction and regains credits when the receiver signals acceptance into its buffer. The term "per-channel credit-based flow control" may refer to a closed-loop admission mechanism applied independently to each channel to ensure that transaction injection is consistent with receiver capacity and that congestion on one channel does not stall unrelated channels holding credits. The term "credit counter" may refer to a bounded counter maintained at the sender of each channel that tracks available credits and gates the issuance of additional transactions until credits are returned from the receiver.

[0057] In some embodiments, the circuitry may be configured to perform end-to-end integrity checks on each of the first and second Ethernet packets using end-to-end Cyclic Redundancy Check (ECRC) to protect at least the payload of each packet across the Ethernet network. The term "end-to-end integrity check" may refer to an acknowledgment performed by the circuitry that protects the packet payload carrying transaction content across the Ethernet network independently of the link-local FCS, wherein the verification is performed at the final receiver before depacketization into the host-visible channel. The term "end-to-end Cyclic Redundancy Check (ECRC)" may refer to a tail value calculated on the transaction payload of the Ethernet packet and appended for end-to-end verification, thereby enabling detection of corruption that may occur anywhere along the path and supporting payload-level protection for both requests and responses.

[0058] In some embodiments, the circuitry may be configured to aggregate multiple memory transactions destined for a common endpoint into a single Ethernet packet up to a configurable packet size threshold. The term "endpoint" may refer to a source or destination device that can be identified in transport metadata and is capable of initiating or consuming packets corresponding to a memory transaction, including, but not limited to, host devices or remote peers addressed by a configured identifier. The circuitry may not split a single transaction across multiple Ethernet packets.

[0059] In some embodiments, the apparatus may include a network interface and circuitry coupled to the network interface. The term "network interface" may refer to a circuitry system that couples the apparatus to a packet-switched network and is operable to send and receive multiple data packets (including transaction content carrying transmission metadata such as sequence numbers and acknowledgments). The circuitry may be configured to send multiple data packets via the network interface. The term "multiple data packets" may refer to multiple Ethernet frames sent or received by the circuitry in association with one or more memory transactions, wherein each data packet may contain transmission fields for ordering and reliability and may co-package several transactions up to a configured threshold. Each data packet may correspond to one or more memory transactions and contain a corresponding data packet sequence number (PSN). The term "data packet sequence number (PSN)" may refer to a sequence value inserted into the transmission header of each data packet to support ordered delivery and reliability, allowing the receiver to distinguish between ordered data packets and out-of-order events and to implement cumulative acknowledgment behavior.

[0060] In some embodiments, the circuitry may be configured to receive a first data packet from an endpoint via a network interface. The circuitry may be configured to determine a first expected packet sequence number (EPSN) for the first data packet and a first PSN contained within the first data packet. The term "expected packet sequence number (EPSN)" may refer to a receiver state that tracks the expected correct and ordered progression of the next PSN, which advances after the received ordered data packet and is communicated in an acknowledgment to inform the sender's reliability logic. In response to determining that the first EPSN equals the first PSN, the circuitry may be configured to update the first EPSN to a second EPSN and send an acknowledgment (ACK) message to the endpoint via the network interface, identifying the second EPSN and indicating that the first data packet is an ordered data packet. The term "acknowledgment (ACK) message" may refer to a control message sent to the sender that identifies the updated EPSN and indicates that one or more data packets of at most a specific sequence have been received in sequence, thereby enabling the sender to release associated buffers. The term "ordered data packet" may refer to a received data packet whose PSN matches the current EPSN, indicating acceptance, EPSN progression, and the generation of acknowledgment information, which may be included in a dedicated ACK message or carried on reverse path traffic. In response to determining that the first EPSN is not equal to the first PSN, the circuit can be configured to send a Negative Acknowledgment (NACK) message to the endpoint via the network interface, which identifies the first EPSN and indicates that the first data packet is out of order. The term "Negative Acknowledgment (NACK) message" can refer to a control message that identifies which EPSN is missing when the PSN of a received data packet does not match the current EPSN, thereby signaling a gap and prompting the sender to retransmit from the lost sequence.

[0061] In some embodiments, the corresponding PSN may be included in a reliable transmission header in each header. The term "reliable transmission header" may refer to a header carried within each packet that conveys fields supporting reliability and routing semantics, such as PSN and EPSN values ​​and endpoint identifiers, thereby allowing the receiver and sender to coordinate orderly delivery and acknowledgment. In some embodiments, in response to determining that a first EPSN is not equal to a first PSN, the circuit may be configured to discard the first packet, increment a sequence error counter, and send a NACK to the endpoint. The term "sequence error counter" may refer to a counter that increments when the PSN does not match the current EPSN, thereby providing software-visible telemetry of sequencing anomalies observed during reception and facilitating diagnosis of network conditions or endpoint behavior. The term "discard packet" may refer to the receiver choosing not to accept packets into depacketization or host-visible queues, for example, when an ECRC integrity check fails or when the PSN is not equal to the current EPSN, wherein the receiver optionally records the event and sends a NACK.

[0062] In some embodiments, the circuitry may be configured to receive a second ACK message that identifies an EPSN greater than the PSN of a plurality of previously transmitted data packets. The circuitry may be configured to release one or more buffers associated with one or more implicitly acknowledged data packets. The term "buffer" may refer to storage resources maintained by the circuitry for temporarily storing transaction content at various pipeline points, including deeper packet storage associated with packet formation and retransmission, which may be released after the acknowledgment overwritten sequence. The term "implicitly acknowledged data packet" may refer to a transmitted data packet that is considered acknowledged when the received ACK conveys an EPSN greater than its PSN, thereby allowing the circuitry to release buffers for multiple earlier data packets without needing to receive individual per-packet acknowledgments.

[0063] In some embodiments, the circuitry may be configured to receive multiple ordered data packets. The circuitry may be configured to determine a specific EPSN using the maximum PSN of the multiple ordered data packets. The circuitry may be configured to send a single ACK message identifying the specific EPSN and indicating the multiple ordered data packets.

[0064] In some embodiments, the circuit may be configured to use end-to-end CRC (ECRC) checksum protection with respect to the payload associated with each data packet by discarding packets that fail the ECRC checksum. In some embodiments, if an out-of-order data packet is received (e.g., the received PSN is not equal to the EPSN), the circuit may generate a NACK message.

[0065] In some embodiments, the circuitry may be configured to prioritize the transmission of ACK and NACK messages over the transmission of data packets. The term "prioritized transmission" may refer to scheduling behavior in which the circuitry selects to send control messages, such as ACK or NACK messages, before data packets under certain conditions, so that reliability feedback is performed with minimal delay and the sender-side buffer can be quickly reclaimed.

[0066] In some embodiments, a method may include a circuit sending multiple data packets via a network interface. Each data packet may correspond to one or more memory transactions and contain a corresponding PSN. The method may include a circuit receiving a first data packet from an endpoint via the network interface. The method may include a circuit determining a first EPSN of the first data packet and a first PSN contained in the first data packet. The method may include, in response to determining that the first EPSN is equal to the first PSN, the circuit updating the first EPSN to a second EPSN and sending an acknowledgment (ACK) message identifying the second EPSN and indicating that the first data packet is an ordered data packet to the endpoint via the network interface. The method may include, in response to determining that the first EPSN is not equal to the first PSN, a negative acknowledgment (NACK) message identifying the first EPSN and indicating that the first data packet is an out-of-order data packet to the endpoint via the network interface. The term "out-of-order data packet" may refer to a received data packet whose PSN is different from the current EPSN, which the receiver may discard and report via NACK while maintaining the EPSN at an expected value to maintain strict ordering semantics.

[0067] The embodiments of this disclosure offer at least the following advantages and benefits. First, the embodiments of this disclosure provide a device for transmitting memory transactions over an Ethernet network, offering the advantages of high throughput and deterministic handling of multichannel memory semantics across packet switching architectures. By transforming AXI-style request and response streams into compact Ethernet packet payloads, the device allows the host subsystem to issue write addresses, write data, read requests, and corresponding responses through a familiar multichannel interface, while relying on a packetization engine that preserves transaction boundaries, ordering, and relevance across the network. This approach enables the use of conventional bus-level semantics on the host side while leveraging the scalability and coverage of high-speed Ethernet interfaces, including link-level capabilities supporting, for example, priority-based flow control and link-level replay, thereby allowing remote endpoints to behave as extensions of the host's memory space, even when separated by significant physical distances.

[0068] Secondly, the embodiments in this disclosure provide a per-packet reliability mechanism that offers the advantage of robust end-to-end correctness, despite the possibility of loss, corruption, or reordering in a best-effort Ethernet architecture. By using the PSN carried in the reliable transport header, the expected sequence state maintained by the receiver, and the ECRC applied to the transaction payload, the device can distinguish between ordered and out-of-order packets, generate ACK or NACK as appropriate, and discard corrupted or out-of-order packets before they reach the host's visible channel. This improves operational predictability because packets are acknowledged according to ECRC protection and strict reordering rules before being accepted, ensuring that the host only receives correctly ordered memory read and write operations, while the device autonomously manages retransmission triggering and acknowledgment merging.

[0069] Third, the embodiments in this disclosure can incorporate per-channel credit-based flow control and packet aggregation strategies, which offer the advantages of efficient utilization of both interface bandwidth and internal buffer resources. By applying credit loops to each transaction channel, the circuitry regulates the issuance of requests and responses, ensuring that the sender injects only as many in-flight operations as the receiver can buffer, thereby preventing overruns and allowing non-congested channels to proceed independently of those experiencing backpressure. Furthermore, the ability to aggregate multiple transactions destined for a common endpoint into a single Ethernet packet up to a programmable threshold improves line efficiency by reducing header overhead and optimizing network data path usage, while avoiding splitting any single transaction across packets ensures atomicity and simplifies both integrity checks and depacketization logic. These combined behaviors contribute to predictable throughput, reduced head-up congestion, and better arbitration efficiency across virtual channels and traffic classes.

[0070] In light of the foregoing, the accompanying drawings and the following description illustrate various examples of techniques for providing reliable memory transaction transfers between computing endpoints using an Ethernet-based load-memory system. The drawings and the following description are non-limiting examples and can be implemented in any of a variety of other configurations while still remaining within the scope of this disclosure. Other embodiments may be used alternatively or in place of other embodiments. Details that are obvious to those skilled in the art may be omitted. Some embodiments may be practiced with additional components or steps and / or without using all the components or steps described.

[0071] Figure 2The example system environment 2000, according to some embodiments, describes an Ethernet-based load-memory system 200 that integrates a system-on-a-chip (SoC) 202 with one or more transport subsystems 220-0, 220-1. The Ethernet-based load-memory system 200 can integrate the SoC 202 with the transport subsystems 220-0, 220-1 to provide a 400 gigabits per second (400G) Ethernet network interface controller (NIC) for reliably transmitting memory read and write transactions between up to 128 endpoints within a single-layer network. Figure 2 A central processing unit (CPU) 201 is shown coupled to an 8×100G Ethernet PHY 210 via a port macro interface. The Ethernet PHY 210 implements MAC and / or PCS functions and exposes two Ethernet ports (e.g., Ethernet port 0 and Ethernet port 1), allowing two system examples to share a single Ethernet PHY. Within the transport subsystem, Figure 2 The system identifies the transmit (TX) and receive (RX) pipelines, as well as TX scheduling and flow control blocks and RX scheduling and flow control blocks, and TX packing and unpacking functions for assembling and disassembling packet payloads. Host-side connectivity is provided by the Advanced Extensible Interface (AXI) data path master and slave interfaces for move request and response transactions, the AXI target interface for configuration and state access, and the debug interface for observability and initial startup. The TX path packages one or more AXI memory transactions into a reliable transport packet that may contain zero or more transactions protected by end-to-end integrity checks, while the RX path performs complementary parsing and verification of transactions and delivers transactions back to the SoC.

[0072] In some embodiments, transport protocols employing PSN, expected sequence tracking, and acknowledgment (ACK) or negative acknowledgment (NACK) signaling can be used to maintain ordered delivery of memory transactions for each channel between the source and destination systems. Packets can be delivered to the destination using a delivery header interpreted by the intervening switch. System 200 can be configured to use: (1) an optimized non-standard packet delivery header format called the Artificial Intelligence Configuration Header (AFH), which reduces protocol overhead; or (2) a larger, less efficient, standards-based Layer 2 Internet Protocol Version 4 and User Datagram Protocol (L2-IPv4-UDP) header format that retains the option to comply with standards. Two virtual channels (VCs) are supported and mapped to separate lossless flow control classes to achieve deadlock avoidance and performance isolation, and a proprietary credit-based flow control interface is applied at the SoC boundary for each AXI channel and each VC. On the Ethernet side, the interface supports per-VC priority flow control (PFC) and Super Ethernet Alliance (UEC) credit-based flow control (CBFC). Ethernet ports further support Link-Level Replay (LLR), which retryes packets at the link level when a transmission failure is detected, thereby reducing, but not eliminating, the need for system-level retransmissions. Figure 2 In the embodiment described herein, the two subsystems are integrated with an 8×100G Ethernet PHY that provides a MAC / PCS interface, and the block diagram clearly shows the two subsystems (or examples) SUB-0 (220-0) and SUB-1 (220-1), the AXI data path master and slave interfaces, the AXI target interface, the debug interface, and two Ethernet ports labeled “Ethernet Port 0” and “Ethernet Port 1”.

[0073]

[0074]

[0075]

[0076] Table 1. Instance System Settings

[0077] Figure 3 Figure 3000 illustrates the data path and associated credit loop in an Ethernet-based load-store system 300 according to some embodiments. Figure 3This document describes the Advanced Scalable Interface Version 4 (AXI4) data path and associated credit loop for coupling the SoC to the Ethernet-based load-memory system 300 via a single 400G Ethernet port. It explicitly illustrates the write address, write data, write response, read address, and read data channel, as well as their corresponding credit return signaling in both the SoC-to-system and system-to-SoC directions. In this embodiment, the system provides an AXI4 slave interface and an AXI4 master interface 320 for the 400G Ethernet interface, each operating at a 1 GHz AXI data path clock (AXI_DP_CLK). Figure 3 Five AXI4 channels are depicted: Write Address Channel (AW) 301, Write Data Channel (W) 302, Write Response Channel (B) 303, Read Address Channel (AR) 304, and Read Data Channel (R) 305, which route between the SoC and the system across that clock domain. The read and write data paths use a 512-bit data bus, allowing the W and R channels to carry 512-bit data clock cycles. In some embodiments, the transfer may have a burst length of two clock cycles per transaction, resulting in a fixed 128-byte transfer on the AXI data path. In some embodiments, variable-size bursts may be transferred, such as bursts containing N 128-byte transfers (N is a positive integer), with a maximum total data size of 2KB.

[0078] Figure 3 This further demonstrates that each AXI4 channel on both the AXI4 master side 320 and the AXI4 slave side 324 is associated with a separate credit interface (e.g., master credit interface 322, slave credit interface 326), where credit flows from the receiver side back to the transmitter side to regulate the ingress rate and prevent downstream interface buffer overflow. Specifically, Figure 3 Discrete credit signaling instructs each direction's write address, write data, read address, write response, and read data channels, corresponding to the credit amount returned per channel when space is released at the receiver, such as AW credit 306, W credit 307, AR credit 308, B credit 309, and R credit 310. This per-channel, per-direction credit mechanism is sized to cover round-trip latency and is applied independently to each AXI4 master channel and each AXI4 slave channel, allowing the SoC to control or accept traffic on a channel-level granularity. Figure 3 In the setup shown, request traffic containing read address transactions and write address and write data transactions is mapped to the first virtual channel, and response traffic containing write response transactions and read data transactions is mapped to the second virtual channel. This allows for lossless flow control and deadlock avoidance across interfaces while maintaining the order of traffic within each channel. In summary, Figure 3The 1 GHz AXI_DP_CLK, 5 AXI4 channels, a fixed 512-bit data width with two-beat bursts, and separate credit interfaces associated with each AXI4 master and slave channel are described to enable deterministic and ordered transmission of memory transactions at line rates on 400G Ethernet ports.

[0079] Figure 4 Figure 400 illustrates the data path connection of the interface between SoC logic 410 and Ethernet-based load-memory system 420 according to some embodiments. Figure 4 It also describes the associated interface buffers, functional partitions, and virtual channel (VC) mappings for each channel application. The SoC logic 410 can have different types of endpoint components as sources or destinations (e.g., processing elements (PEs), direct memory access (DMA) engines, high-bandwidth memory (HBM) controllers, etc.). Figure 4 In this context, AXI transactions are presented as request traffic (including read address (AR), write address (AW), and write data (W) transactions) and response traffic (including write response (B) and read data (R) transactions); requests are mapped to a first virtual channel identified as VC0, and responses are mapped to a second virtual channel identified as VC1, thereby maintaining ordering and providing isolation between the request and response streams. The system instantiates and stores a per-channel interface buffer at the AXI interface boundary to "unpack" the AXI transaction, and... Figure 4 These buffers are visually associated with VC0 or VC1 to indicate their request or response class membership; these interface buffers are relatively small and differ from the larger “packed” data buffers residing deep within the transmit (TX) and receive (RX) data paths. In some embodiments, system 420 may maintain one or more per-channel transmit (TX) queues 422 and / or one or more per-channel receive queues 424. In some embodiments, additional queues per channel may be used to support differentiated service classes.

[0080] Figure 4This further demonstrates that the SoC provides corresponding per-channel buffers near the system to keep the buffer depth shallow and maintain independence from changes in the SoC architecture. In this arrangement, the buffers that receive transactions from the system, i.e., AXI master AR, AXI master AW, AXI master W, AXI slave B, and AXI slave R directions, return 401 credits to the system when space is released. The buffers that send transactions to the system, i.e., AXI master B, AXI master R, AXI slave AR, and AXI slave AW directions, track available credits by decrementing the credit counter for each sent transaction and incrementing the counter for each returned credit (which is returned 402 by the system to the SoC). Within the system, the compute-to-system channel interfaces are organized into two functional groups: a control group containing write responses and read request traffic; and a control / data group containing read responses with data and write requests with write data traffic. Although write requests (AW) and write data (W) use different channels as required by AXI4, transactions across these two channels arrive in pairs and in the same order, and the system aligns each pair at the head of its respective interface buffer before moving them forward as a single unit into the TX data path.

[0081] Figure 4 The bandwidth constraint governing the rate at which the TX data path can accept incoming operations from the AXI side is also described. Specifically, the TX data path divides the incoming channels into two groups: a control group consisting of B and AR channels, and a control / data group consisting of R channels and combined AW / W channels; the TX data path can receive at most one control transaction and one control / data transaction every two interface clock cycles, and each group is arbitrated by a work-saving cyclic arbitrator that selects at most once every two clock cycles. When the total arrival rate of any group exceeds this acceptance rate, Figure 4 The interface buffer is backed up and per-channel interface-level flow control is asserted via a credit loop mechanism described elsewhere, which in turn causes AXI transactions to accumulate in the per-channel buffer of the SoC until the credit is returned.

[0082] Figure 4 The demonstration shows per-channel unpacking buffers at the interface, per-channel buffers on the SoC side with explicit credit return and credit billing, VC0 and VC1 mappings for request and response traffic, alignment of paired write addresses and write data operations before TX injection, and how the dual-arbitrator bandwidth model is integrated to regulate entry into the TX data path while maintaining ordered delivery semantics and isolating control and control / data flows.

[0083] Figure 5 Figure 500 illustrates an interface for transmitting request and response transactions between a source endpoint 501 and a destination endpoint 502 according to some embodiments. Figure 5This describes an AXI4 interface that carries request transactions between source endpoint 501 and destination endpoint 502 via a first virtual channel identified as VC0 550. Source endpoint 501 may include a SoC coupled to an Ethernet-based load-store system 521 implementing reliable network transport. Similarly, destination endpoint 502 may include a SoC coupled to an Ethernet-based load-store system 522 implementing reliable network transport. Figure 5 It also describes the AXI4 interface that carries response transactions between the source endpoint 501 and the destination endpoint 502 via a second virtual channel identified as VC1 551.

[0084] exist Figure 5 In the write process described, the SoC logic at the source endpoint issues a write request to the system's AXI4 slave interface on its AXI4 master interface. The request is then packetized in the source system's transmit data path, forwarded by the switch, received and unpacked by the destination system's receive data path, and then presented on the destination system's AXI4 master interface to the destination SoC's AXI4 slave interface. The write response returns from the destination SoC to the source SoC in the opposite direction, traversing the destination system's transmit data path, the switch, and the source system's receive data path, before being delivered on the response channel of the source SoC's AXI4 master interface. Throughout this write transaction, request traffic is carried end-to-end on a first virtual channel identified as VC0, and response traffic is carried in the opposite direction on a second virtual channel identified as VC1. This preserves the ordered delivery semantics of each channel while implementing class-based lossless flow control across network paths.

[0085] Figure 5 It also indicates that the same interface relationship applies to read operations, where the read address channel is used as the request path and the read data channel is used as the response path. Specifically, the read request originates from the source SoC on the AXI4 read address channel, is injected into the source system's AXI4 slave interface, and follows the same source transmit data path, switch, and destination receive data path sequence before being sent to the destination SoC by the destination system's AXI4 master interface. The corresponding read data then returns from the destination SoC to the source SoC using the destination system's transmit data path, switch, and source system's receive data path, and is delivered on the AXI4 read data channel at the source. Consistent with the write case, in Figure 5 In this process, read address requests are mapped to VC0, and read data responses are mapped to VC1, thereby maintaining the end-to-end path between endpoints and maintaining the mapping between requests on VC0 and responses on VC1.

[0086] Combined, Figure 5The description describes a request originating from the source SoC's AXI4 master interface, entering the source system's AXI4 slave interface, passing through the source transmit data path, intermediate switches, and the destination receive data path, and being delivered to the destination system's AXI4 master interface for consumption by the destination SoC. The corresponding response passes through the reverse path and returns to the source SoC's AXI4 master interface. Virtual channel VC0 is assigned to all request traffic, and virtual channel VC1 is assigned to all response traffic for both writing and reading.

[0087] Figure 6 Figure 600 illustrates a credit loop established between SoC 610 and Ethernet-based load-store system 620 including Ethernet interface 622, according to some embodiments. Figure 6 It also describes the credit return logic, credit counter logic, and associated interface buffers related to the transmit (TX) and receive (RX) data paths and VC partitions. Figure 6 In this process, each AXI4 channel divides the credits, such that each discrete credit represents a transaction in its corresponding channel, and Figure 6 The receiver-side credit return logic is shown, which is co-located with each VCRX buffer, while the sender-side credit counter is co-located with each VCTX buffer. Figure 6 It also presents AXI4 master and AXI4 slave channel groups that are separate from and aligned with the RX and TX data paths, respectively. This is a visually clear graphical convention that makes the direction of transaction flow and the corresponding credit return path clear.

[0088] like Figure 6 As demonstrated, the credit loop performs flow control on incoming traffic for two complementary purposes. First, the credit mechanism provides rate adaptation, where the SoC-to-system channel might otherwise oversubscribe to the maximum accept rate of the TX data path; by limiting transmissions to available credits, the input rate is automatically limited to within the TX pipeline's acceptable range without exceeding the limit. Second, the credit mechanism prevents overflow of the downstream packetized TX buffer, which stores packet payloads in queues indexed by endpoints and VCs; because the TX buffer space is allocated a fixed amount per VC, returning credits only when the space is actually freed ensures that the SoC does not inject additional transactions exceeding the buffer capacity.

[0089] Figure 6This further reflects the round-trip time of each credit cycle, sized to span the interface between the sender and receiver. A complete cycle includes sending a transaction and decrementing the sender's credit counter, receiving and storing the transaction at the receiver, dequeuing the transaction to move forward, returning a credit from the receiver, and finally receiving that returned credit at the sender and incrementing the sender's counter. To maintain full-rate operation during this round-trip interval, the sender initializes with sufficient credits to keep the pipeline full until credits begin to return. In the illustrated embodiment, the system supports up to 128 credits per channel, which accommodates approximately 128 nanoseconds of round-trip latency at a 1 GHz AXI data path clock (AXI_DP_CLK), including signal flight time and internal credit processing latency within both the SoC and the system.

[0090] Figure 6 This illustrates per-channel credit return from the receiver to the transmitter, receiver-side credit return logic with per-VC RX buffer, and the placement of transmitter-side credit counters with per-VC TX buffer. It also demonstrates the explicit alignment (for clarity) of the AXI4 master and AXI4 slave channel groups with the RX and TX data paths and the sizing of the credit pool to a target round-trip time, ensuring that inbound traffic matches the TX data path rate and preventing packet buffer overflow under a fixed per-VC allocation model. In some embodiments, system 620 may include one or more per-channel credit counters 624 (which may decrement or increment) and / or one or more per-channel credit returns 622 (which may return credits to SoC 610).

[0091]

[0092] Table 2. AXI Credit Revolving Returns

[0093] Table 2 shows the interface bit per channel used to return credits. The receiver-side logic for each channel can drive this signal to the sender. When this signal is set, a credit is returned to the sender, which then increments its credit counter.

[0094] In some embodiments, credit initialization is implemented via a hardware protocol, where the receiver side of each AXI channel assigns initial credits to the sender side only after both ends of the interface have exited reset and indicated readiness to process traffic. The SoC asserts the S2E_READY signal to indicate that it has exited reset and is ready to send or receive transactions, while the Ethernet-based load-store system asserts E2S_READY for the same purpose. Because the SoC and system may exit reset at different times, credit distribution will not begin until mutual readiness is observed. When reset is active, the sender's credit counter is cleared, and after the reset cancel assertion, the receiver temporarily holds the entire credit pool for that channel.

[0095] In some embodiments, once the receiver completes its internal initialization, it enables its receive logic, asserts readiness, and immediately begins sending credits from its credit return pool to the sender via the per-channel credit return interface upon detecting the sender's readiness. During this interval, the sender remains enabled but keeps its credit counter at zero until returned credits begin to arrive. Each credit indicates that only one AXI transaction is allowed to be issued, so when a credit returns, the sender increments its counter and begins issuing transactions based on the receiver's available buffer capacity. This mechanism applies uniformly to write address (AW), write data (W), read address (AR), write response (B), and read data (R) channels, thereby ensuring that no transaction is issued until the receiver can safely accept it.

[0096] In some embodiments, the credit cycle is sized to cover the expected round-trip time (RTT) between the sender and receiver, including the time to send a transaction, store it at the receiver, forward it downstream, return the credit, and process the returned credit at the sender. To maintain full-bandwidth operation across this interval, the sender initializes with enough credits to keep the pipeline full until the credit return cycle begins. The Ethernet-based load-store system supports 128 credits per channel, which corresponds to approximately 128 nanoseconds of RTT at a 1 GHz AXI data path clock (AXI_DP_CLK), including on-chip signal propagation and credit processing time within both the SoC and the system. This credit depth ensures that incoming traffic is automatically rate-matched to the transmit data path and that the downstream packet buffer does not overflow.

[0097] In some embodiments, the AXI signaling used by the system is divided into AXI control bits and AXI data bits. The control group contains address, metadata, and transaction type information, while the data group carries a fixed-size payload tick. In Ethernet-based load-store systems, these fields are treated as opaque, meaning the system does not interpret or modify most AXI fields but instead packages them verbatim and transmits them to the destination. Only a small subset of fields are examined or repurposed, such as fields required for transport functions (e.g., endpoint identifiers for selecting the destination or control type fields for determining the number of control bytes that may or must be packaged). All remaining AXI control and data bits are simply encapsulated within a packaged transaction format and reproduced unchanged at the destination AXI interface, preserving the original AXI semantics while enabling efficient network-based memory transaction delivery.

[0098] In some embodiments, the system receives two types of control information at the AXI boundary. The first type includes transport control bits that accompany each transaction, are consumed by the system for packetization and reliable transport, and are also carried end-to-end. These include XPUID and xCTLTYPE. XPUID is a 7-bit endpoint identifier supplied by the SoC at the source to select the destination endpoint; before packetization, the system remaps this value to a configured identifier such that at the destination, the presented XPUID identifies the source endpoint, not the destination. Therefore, XPUID is intentionally asymmetric: at the source, it points to the destination, and at the destination, it points back to the source. xCTLTYPE is a two-bit control type indicator that the system retains from source to destination, so that the receiving SoC can decode it to determine how many control bytes were packetized in the transaction. Depending on the SoC integration, these transport fields may be conveyed in AXI-xUSER or other sideband fields.

[0099]

[0100] Table 3. System Input Control Fields

[0101] In some embodiments, the second class includes opaque AXI control fields that are not interpreted by the system—such as addresses, cache attributes, and transaction identifiers. These are named in the write address (AW), write response (B), read address (AR), and read data (R) channels.<CHANNEL_IDENTIFIER> Each channel of the CTL is loaded on the control bus. The write data (W) channel does not have a separate control bus because it consists entirely of WDATA and inherits control from the associated AW transaction.

[0102] In some embodiments, for each transaction, the system packages up to 18 bytes of control, consisting of up to 9 bits of transfer control (XPUID plus CTLTYPE; see Table 3 above), up to 132 bits of opaque AXI control, and 3 bits of source channel type. For combined write request / write data paths, a single type is used. The packaged control length is variable and is selected from 2 bytes to 18 bytes in 2-byte increments by a configurable lookup table indexed by {channel_type[1:0], ctl_type[1:0]}, thus providing 4 pre-configured sizes for each channel type across 16 entries. At the destination, the original xCTLTYPE accompanies the delivered transaction, so the SoC can reconstruct the AXI mapping and interpret the opaque control bytes on the appropriate AXI signal exactly as it was transmitted.

[0103] In some embodiments, the system transmits AXI data specifically via a standard AXI data bus, WDATA for writing the payload, and RDATA for reading the payload, imposing strict constraints on the size and structure of each data move. In some embodiments, each AXI data transfer may be fixed at 128 bytes, and may require AXI burst fields (AWBURST[1:0] for writing and ARBURST[1:0] for reading) to indicate the burst length of the two clock cycles. Because the data bus is 512 bits wide, 64 bytes are carried per clock cycle, and thus the forced two-clock bursts produce exactly a 128-byte transfer. These fixed-size two-clock transfers ensure consistent packing and unpacking behavior within the system and allow transmit and receive data paths to operate with predictable alignment and buffering rules, as described in the reliable transmission architecture of the original disclosure. In some embodiments, each data transfer may have a variable length.

[0104] Tables 4 to 8 below define the interface signaling (AXI signaling) and packetization for each channel.

[0105]

[0106] Table 4. Write Address (AW) Interface Signaling and Packetization

[0107]

[0108] Table 5. Write Data (W) Interface Signaling and Packaging

[0109]

[0110] Table 6. Write Response (B) Interface Signaling and Packaging

[0111]

[0112] Table 7. Read Address (AR) Interface Signaling and Packetization

[0113]

[0114]

[0115] Table 8. Data Reading (R) Interface Signaling and Packaging

[0116] In some embodiments, the SoC maps its AXI4 signals to interface fields exposed by the system on a per-channel basis. The xVALID and xDATA signals on the system interface are dedicated signals that directly correspond to the AXI4 VALID and DATA signals on the SoC side. All remaining AXI4 control fields are collected and encoded by the SoC logic into three categories of system control inputs: xCTL, xXPUID, and xCTLTYPE. The xXPUID field, identifying the communication endpoint, is remapped by the system such that its meaning is reversed between source and destination. At the source, the SoC provides an XPUID value identifying the destination endpoint. After packaging, the system replaces this value with the configured XPUID presented by the destination system to its SoC to identify the source endpoint. This remapping allows the same transaction header to serve both addressing and return path routing. In contrast, the xCTLTYPE field remains unchanged from source to destination to indicate the number of control bytes packaged for that transaction, ensuring that the receiving SoC can reconstruct the appropriate AXI control signals.

[0117] In some embodiments, the xCTL field contains opaque AXI control bits, such as address, attribute, and identifier fields. These bits retain their AXI semantics but are not interpreted by the system and can be mapped from any AXI control signal selected by the SoC. The system does not rely on AXI features incompatible with its fixed burst model. For example, the WLAST and RLAST fields are not used because the system transmits the payload as two fixed 64-byte ticks. Similarly, xSIZE and xLEN are ignored because the burst size and transmit length are fixed and do not vary by transaction. Together, these mappings allow the system to retain the AXI transaction meaning while transmitting all necessary control information across the network in packet form and faithfully reconstructing the control information at the destination.

[0118] Figure 7 and Figure 8 This section explains the bit mapping and streaming of write requests and write responses, as well as read requests and read responses, between the System-on-Chip (SoC) logic and the Ethernet-based load-memory system. It demonstrates how the general-purpose AXI4 bus presented by the SoC is mapped to an internal control bus, identified as xCTL, used for network transmission, and then mapped back to the destination AXI4. Figure 7 and Figure 8In each of these, the SoC-side AXI4 signals are divided into control and data packets. The control field is collected into the xCTL bus, while the data field remains on a dedicated data path. The gray-shaded fields indicate AXI4 bits that are not supported or consumed by Ethernet-based load-memory systems. As depicted, the xVALID and xDATA signals are not remapped into xCTL but instead retain unmapped control strobes and data paths at the AXI4 boundaries. This aligns with the requirement to package only control plane metadata into the transport header while handling data ticks on dedicated write (W) and read (R) data channels.

[0119] Figure 7 Figure 700 illustrates the write request and write response bit mapping and stream between SoC logic 710, 770 and Ethernet-based load-memory systems 720, 760 according to some embodiments. Figure 7 In this process, the write procedure begins when the source SoC 710 issues a write address (AW) transaction and the corresponding write data (W) is transmitted in ticks on its AXI4 master interface. The SoC logic maps the AW control field to the xCTL bus according to a packing strategy, while the W data is presented as a 512-bit tick that is not folded into the xCTL. The Ethernet-based load-store system 720 packages the xCTL control bytes and data payload (if present) into a network packet for transmission, thereby preserving per-channel ordering and associating the write request with the requested virtual channel. At the destination, the receive path (e.g., the Ethernet-based load-store system 760) unpacks the control bytes from the transmission header back into the AXI4 control signals and forwards the data ticks on the AXI4 data bus to the AXI4 slave interface of the destination SoC 770; the corresponding write response (B) control returns along the reverse path and is mapped back into the xCTL from the destination SoC's AXI4 interface for transmission as a response virtual channel packet. Figure 7 and Figure 8 It is emphasized that certain AXI4 fields are opaque to Ethernet-based load-store systems and are conveyed as packet control bytes without being interpreted, while fields such as endpoint identifiers are used by the transport for routing and remapping between source and destination, as described in this disclosure.

[0120] Figure 8 Figure 800 illustrates the read request and read response bit mapping and stream between SoC logic 810, 870 and Ethernet-based load-memory systems 820, 860 according to some embodiments. Figure 8The diagram illustrates a similar mapping used for read traffic. Read address (AR) control from the source SoC 810 is collected into xCTL and transmitted as a request-type packet, while read data (R) response from the destination SoC 870, along with its control on the AXI4 R channel and its data on the 512-bit data bus, is returned as a response-type packet. Figure 8 The SoC 810 provides the xTYPE field to characterize the control byte footprint for a given transaction type, allowing the Ethernet-based load-store system 820 to pack only the number of control bytes actually needed for that transaction. This optimizes in-line overhead when multiple transaction forms using a subset of AXI4 control fields or narrower encodings are transmitted over a channel. In conjunction with this, the number of packed control bytes per channel type can be varied using a lookup indexed by the channel and control type indication, where the packing control size is capped to ensure deterministic packing and unpacking across endpoints. This variability in xTYPE orientation enables efficient transmission while still preserving end-to-end ordering and integrity guarantees for reliable transmission, such as when certain write requests do not require all identification and attribute fields, or when read responses can omit fields unnecessary for destination reconstruction of AXI4 semantics.

[0121] In some embodiments, the system is integrated with an Ethernet PHY 8×100 (e.g., via eight 100G channels) that provides two 400 gigabits per second (2×400G) Ethernet interfaces compliant with the Ultra Ethernet Consortium (UEC) 1.0. In this configuration, two instances of the system share a single 8×100 Ethernet PHY, allowing both to access their high-speed Ethernet ports.

[0122] In some embodiments, the 8×100 Ethernet PHY incorporates the complete physical and link layer feature set required for 400G Ethernet operation, including the MAC layer, PCS, forward error correction (FEC), physical media attachment (PMA), physical media dependent (PMD) functions, and analog front end (AFE). Each of the two 400G ports supports link-level replay (LLR) for link-level retransmission, priority flow control (PFC) compliant with IEEE 802.1Qbb, and credit-based flow control (CBFC) as defined by UEC 1.0. The 8×100 Ethernet PHY also supports software-controlled channel switching and polarity reversal, which simplifies system-level board routing and layout.

[0123] In some embodiments, the system itself may operate with a CBFC compatible with PFC or UEC, and a selected flow control mechanism may be applied per port and per virtual channel (VC). This per-VC granularity matches the use of separate lossless flow control classes for request and response traffic within the system, thereby ensuring end-to-end lossless behavior across Ethernet links according to the configured flow control mode.

[0124] In some embodiments, each instance of the system exposes an AXI4 slave interface, which the system-on-chip (SoC) uses for configuration and status access. This interface does not directly map internal registers or memory into the SoC address space. Instead, the system implements an indirect access mechanism, whereby software writes to a smaller set of externally visible registers, which in turn address and access the system's internal register file and memory structures.

[0125] Table 9 below lists the target interface parameters.

[0126]

[0127] Table 9. AXI Target Interface Parameters

[0128] In some embodiments, each 400G port of the system provides a dedicated interrupt signal, and the software can determine the cause of the interrupt by maintaining a corresponding set of interrupt status bits for each port. These status bits identify the specific event or condition that triggered the interrupt and allow the system software to react appropriately. The system also provides software-configurable interrupt masking, allowing each interrupt source to be selectively enabled or suppressed, so that only relevant events generate external interrupts. In addition to interrupts generated by the system itself, the Ethernet PHY 8×100 also independently generates two interrupt outputs and maintains its own set of status bits. To simplify integration, these port macro interrupts are logically mapped to the interrupt signal of port 0, thereby allowing the software to handle both system-level and port macro interrupt conditions through a unified reporting mechanism.

[0129] In some embodiments, the system may receive three clocks from the SoC, as shown in Table 10 below.

[0130]

[0131] Table 10. Clock Input

[0132] In some embodiments, the system relies on a single active-low reset signal, System_RESET_N, supplied by the System-on-Chip (SoC), to reinitialize all logic within the Ethernet-based load-memory system. When this reset signal is asserted, it clears and reinitializes each internal block, including the complete data path of two 400 gigabits per second (400G) ports, all system-to-SoC interface logic, AXI target interfaces, and the integrated 8×100 Ethernet PHY (e.g., via eight 100G channels), which contains MAC, PCS, and other physical layer components. This unified reset mechanism ensures that the entire subsystem, including the Ethernet PHY and all transport and packetization logic, returns to a clean and deterministic state under SoC control.

[0133] In some embodiments, a Reliable Transport Header (RTH) is appended to each packet generated by the system and provides the control information required for reliable, lossless delivery of memory transactions between endpoints. In some embodiments, the RTH contains fields identifying the source and destination endpoint processors (XPUs), allowing the system to correctly route packets and enabling the destination to generate return traffic to the initiating XPU, including acknowledgments. The term "XPU" can refer to one or more computing devices, encompassing CPUs, GPUs, DPUs, or other processing subsystems capable of sending or receiving memory transactions (e.g., AXI-based memory transactions) across a network architecture. The header also carries a partition (also known as a "global job identifier"), which allows packets to be associated with higher-level operations or workloads across the architecture. Additionally, the RTH embeds all the information required for the system's Acknowledgment (ACK) and Negative Acknowledgment (NACK) protocols, including the PSN and EPSN. These fields allow the receiver to determine whether packets arrived in order and signal the sender if retransmission is required. Together, the RTH fields enable the system to implement its end-to-end reliable transmission mechanism, ensuring ordered delivery, loss detection, and retransmission across the Ethernet architecture.

[0134]

[0135] Table 11. Reliable Transmission Header Format

[0136] In some embodiments of the reliable transport protocol, acknowledgment (ACK) and negative acknowledgment (NACK) messages are generated by the receiver endpoint processor (XPU) and sent to the sender XPU to report delivery status and drive retransmission. The information required to communicate these messages is carried in the Reliable Transport Header (RTH) using opcodes, indicating ACK or NACK, a PSN for marking the transmitted data packet, and an EPSN for informing the sender of the next ordered sequence number the receiver is ready to accept. ACK and NACK information may be carried on data packets that have already carried payloads in the opposite direction, or may be sent in a separate control packet without payload; in one embodiment, a dedicated ACK is a compact 64-byte packet used when no reverse path data is available within a configured timeout window. Upon receiving an ACK, the sender releases the state of all data packets up to the acknowledged PSN and proceeds with transmission; upon receiving a NACK, the sender backtracks to the indicated EPSN and retransmits from that point using a back-on-N procedure.

[0137] Table 12 below shows the overhead of each of the formats that include RTH. IPG refers to the Ethernet packet gap.

[0138]

[0139] Table 12. Transmission header overhead

[0140] In some embodiments, even in scenarios where errors may occur within the system or network switching infrastructure, the system implements reliable transport protocols to ensure lossless, ordered delivery of memory transactions between endpoint processors (XPUs). While link-level replay (LLR) can prevent packet loss on individual Ethernet links, it cannot correct packet loss caused by soft errors within the system or switching equipment, and such loss, though rare, requires additional end-to-end mechanisms. Therefore, a reliable transport layer ensures correctness beyond what link-level protection can provide on its own.

[0141] To achieve this, in some embodiments, the system uses an ACK-based Go-Back-N protocol, where each transmitted data packet carries a PSN assigned by the sender. The receiving XPU maintains an EPSN identifying its expected next ordered data packet. When a data packet arrives with a PSN equal to its EPSN, the receiver accepts the data packet, increments its EPSN, and relays this updated value back to the sender. This return information is embedded in the Reliable Transport Header (RTH) using a field that contains an opcode identifying ACK and the PSN and EPSN values ​​required to advance the sender's state. If the receiver instead observes a PSN that does not match the EPSN, it concludes that a data packet has been lost, discards out-of-order packets, updates error statistics, and returns a NACK using the RTH, thereby notifying the sender that the correct EPSN can be recovered from for retransmission.

[0142] In some embodiments, during normal operation, the sending XPU transmits data packets forward to the receiving XPU, while the receiving XPU returns ACK or NACK information in the opposite direction, which may be carried over to other data packets or dedicated control packets if necessary. Acknowledgments allow the sender to free up buffer space and advance its transmission window, while negative acknowledgments cause the sender to revert to the indicated EPSN and retransmit all subsequent data packets, thus ensuring no gaps in the delivered sequence. In this way, a reliable transport protocol detects lost or corrupted data packets, repairs the loss through controlled retransmission, and maintains strictly ordered delivery across the network, providing robust end-to-end reliability even where data packet loss would otherwise have interrupted communication.

[0143] In some embodiments, for each destination endpoint processor (XPU), the sender maintains an EPSN that uniquely identifies each packet it transmits. Each outgoing packet includes the current PSN in its Reliable Transmission Header (RTH), and thereafter, the sender increments the PSN in preparation for the next transmission. At the receiving XPU, the transmission logic maintains an expected EPSN for each source XPU, indicating the next ordered packet the receiver is ready to accept. When a packet arrives, the receiver compares the packet's PSN with its EPSN. If they match, the packet is accepted, and the receiver advances its EPSN to indicate the next expected PSN value.

[0144] In some embodiments, once an ordered data packet is received, the receiver can notify the sender that the packet has been successfully received and that the receiver has advanced its EPSN. To achieve this, the receiver will transmit the updated EPSN back to the sender. Whenever the receiver has a data packet to send in the opposite direction on the same virtual channel, it carries the EPSN in the RTH of that data packet. This opportunistic acknowledgment avoids generating additional data packets and improves bandwidth efficiency. The receiver waits for a duration defined by the ACK timeout window to determine if the data packet in this opposite direction has become available. If no data packet is sent within this window, the receiver generates a dedicated 64-byte ACK packet containing only the RTH and no payload, thus ensuring that acknowledgment continues without excessive delay.

[0145] In some embodiments, if the receiver processes multiple ordered packets before sending an acknowledgment, it returns the EPSN corresponding to the highest received PSN, implicitly acknowledging all earlier packets in the sequence. When the sender receives an ACK containing the updated EPSN, it releases all buffer space and internal state associated with the acknowledged packet and continues moving forward in the reliable transport protocol.

[0146] In some embodiments, when the Receive Endpoint Processor (XPU) encounters a packet whose PSN does not match its EPSN, the packet is immediately discarded due to its out-of-order nature, and the receiver generates a negative acknowledgment (NACK) pointing back to the sender. Any later-arriving packets that still do not match the unchanged EPSN are also discarded, thus ensuring that no out-of-order packets enter the system until the lost packets are recovered. The NACK notifies the sender that a packet has been lost and specifies the next PSN expected by the receiver, allowing the sender to pinpoint precisely where the sequence gap occurred. Upon receiving the NACK, the sender backtracks its transmission queue to the packets associated with the indicated EPSN and retransmits all subsequent packets in order according to the system's back-to-N semantics.

[0147] In some embodiments, the reliable transport protocol also addresses situations where a packet is lost from sender to receiver without a NACK being returned. This is because the lost packet was the last packet transmitted and no further forward traffic prompted for a NACK, or because the NACK itself was lost in the opposite direction. To ensure recovery in these scenarios, the sender maintains a retransmission timer. If any incomplete packet remains unacknowledged after a configurable timeout, the sender assumes the packet or its corresponding NACK has been lost. It then backtracks its queue to the oldest unacknowledged packet and retransmits forward from that point. This timer-based protection ensures that each lost packet is eventually detected and corrected, even without explicit NACK signaling, and guarantees reliable end-to-end delivery under all permissible network and device failure modes.

[0148] Figure 9 Figure 900 illustrates an example network transmission path of an Ethernet-based load-store system 920 according to some embodiments. Figure 9 The process is described as a phased process, through which AXI4 transactions from the SoC are converted into Ethernet packets at a 400G transmit port provided by an 8x100G Ethernet PHY. In the illustrated arrangement, the transmit path accepts request-type inputs, including write request addresses and controls, write data and read request addresses and controls, and response-type inputs, including write response controls and read responses with data. These inputs are first normalized and then delivered to a packetization stage 921, which aggregates the entire AXI4 transaction into a packet payload on a per-destination and per-virtual-channel basis. The packetization strategy ensures that bytes from a single transaction are not split across packets, and that packet sizes grow towards a configured threshold of up to 4 kilobytes, while also allowing sub-threshold packets to be transmitted when traffic timing requires.

[0149] Downstream of packaging Figure 9The diagram illustrates the transmission buffering and queuing 922, which organizes packet payloads into a transmission buffer and queues for each {destination endpoint, virtual channel}. This organization supports simultaneous interleaved growth of up to 256 packets, one per active queue, and provides sufficient depth to bridge the load round-trip time for reliable transmissions, allowing packets to be available for potential retransmissions before being acknowledged. Subsequent transmission scheduling and congestion control phase 923 arbitrates between non-empty queues and enforces ordering and service discipline. In one embodiment, protocol control traffic (e.g., negative acknowledgments and dedicated acknowledgments) is given strict priority, while the initial data transmission and any required retransmissions are subsequently scheduled. Within and across virtual channels, scheduling combines arrival order skew with configurable preferences for larger packets to improve online efficiency, and a weighted deficit round-robin approach can be applied at the port to enforce byte-weighted fairness between virtual channels. A fixed congestion window per {destination, virtual channel} limits in-flight data, making destinations exceeding its window temporarily ineligible until acknowledgment advances the window.

[0150] Before being sent to the physical interface Figure 9 The packet construction phase describes the insertion of link layer and network layer headers and a reliable transport header, followed by the generation of an ECRC on the payload. The system supports standard Layer 2, Internet Protocol Version 4 (IP4), and User Datagram Protocol (UDP) encapsulation or AI-based header format reduction with reduced overhead, and configuration tables provide per-virtual-channel and per-destination constants for header formation. The flow control receive block 924 at the transmit boundary accepts pause signaling for each virtual channel in the form of priority flow control or Hyper-Ethernet Alliance credit-based flow control and applies it to the scheduler and queue gates, while the Ethernet PHY interface presents a 400G Ethernet transmit port including MAC and PCS functionality. Figure 9 The marked functions, such as packetization 921, transmit buffering and queuing 922, transmit scheduling and congestion control 923, flow control reception 924, packet construction 925, and interfacing with Ethernet PHY 910, work together to transform AXI4 request and response traffic from the SoC into reliably delivered Ethernet packets at line rate.

[0151] In some embodiments, "packaging" is a process by which the system, using the XPUID provided by the System-on-Chip (SoC) for destination selection, accumulates bytes from arriving transactions into a single Ethernet packet on a per-destination endpoint (XPU) and per-virtual-channel (VC) basis. Transactions from different AXI channels can be co-packaged as long as they target the same {destination XPU, VC}, because at this stage, packaging is intentionally channel-independent.

[0152] In some embodiments, each Ethernet packet may contain an integer number of complete transactions; the system never splits the bytes of a single transaction across multiple packets. Packet growth is regulated by a packet size threshold per VC, and packing continues until the packet size is greater than or equal to that configured threshold. When the threshold is met, the next arriving transaction begins a new packet. The threshold can be configured to a maximum value of 4 kilobytes.

[0153] In some embodiments, because transactions may arrive interleaved, a single packet operation may be incorporated into up to two control transactions and two control / data transactions before the threshold comparison is re-evaluated. Therefore, the worst-case packet size is strictly less than (threshold - 1 + (2 × maximum control transactions + 2 × maximum control / data transactions)), as follows.

[0154] Worst-case packet size < packet size threshold - 1 + (2 × maximum control transaction + 2 × maximum data / control transaction) (Inequality 1)

[0155] In some embodiments, where transactions have a fixed data size, there is a packet size threshold of 4 kilobytes, a maximum control transaction size of 18 bytes, and a maximum control / data transaction size of 146 bytes, with the maximum transmitted packet size ≤ 4,424 bytes. Packets smaller than the thresholds may still be transmitted when traffic patterns require it, and any packets smaller than the 64-byte Ethernet minimum are zero-padded before transmission to satisfy link-layer framing rules.

[0156] In some embodiments, the transmit packet buffer holds byte-packed memory transactions that have been assembled into packet payloads and are awaiting transmission on a 400G Ethernet port. These payloads are the result of a system packing process that aggregates them into a continuous stream of bytes before a complete AXI transaction is scheduled to be sent. The transmit packet buffer provides a temporary storage area where these fully packed payloads reside as they are organized into per-destination, per-virtual-channel queues and await selection by the transmit scheduler. Its capacity and organization allow the system to maintain the required load round-trip time for reliable transport protocols and support the construction of interleaved packets across many active queues without loss or reordering.

[0157] In some embodiments, the transmit (TX) buffer space plays two important roles in the reliable transport architecture of an Ethernet-based load-store system. First, it supports simultaneous interleaved packetization across all active TX queues, allowing multiple destinations and virtual channel (VC) streams to grow packet payloads in parallel without blocking each other. Second, it provides sufficient buffer depth to cover the load round-trip time (RTT) of the reliable transport protocol, ensuring that all packet data required for potential retransmissions remains available until a corresponding acknowledgment (ACK) is received. These requirements arise because the system may or must retain each transmitted packet in the buffer until an ACK or retransmission event distinguishes its state.

[0158] In some embodiments, during initialization, the software allocates a dedicated TX buffer space for each VC, which is not shared between VCs. This separation ensures that request and response traffic occupying different VCs will not compete for buffer resources and will not cause deadlocks or head-of-line blockages due to cross-VC interference. The fixed allocation size for each VC allows each VC to independently maintain the worst-case combination of packet assembly behavior and loading RTT requirements without exhausting the buffer capacity.

[0159] In some embodiments, the system performs packing at the tail of each transmit (TX) queue, where arriving memory transactions are appended to the currently formed data packet of that queue. Each new transaction is merged into the same data packet as long as the accumulated data packets remain below its configured per-virtual-channel (VC) data packet size threshold. When the threshold is reached or exceeded, the next incoming transaction automatically starts a new data packet, thus ensuring that each data packet remains within the system's maximum packing limit.

[0160] In some embodiments, the TX buffer space achieves this behavior by allowing transactions destined for different {endpoint, VC} queues to arrive interleaved. Since the system supports up to 256 TX queues, each corresponding to a different combination of destination XPUs and VCs, multiple packets can grow concurrently, one per active queue. In highly interleaved traffic patterns, many packets can accumulate bytes in parallel, and the final size of each packet is limited by both the per-VC threshold and the total available TX buffer capacity. This buffering structure ensures that the packetization process remains efficient even under heavy loads and mixed traffic conditions, while preventing any packet from exceeding the space allowed by the overall TX buffer allocation and maintaining the strict transaction integrity and packet size guarantees defined by the system's reliable transport design.

[0161] In some embodiments, reliable transport protocols allow the sender to hold data packets in a transmit buffer until an acknowledgment (ACK) is received for that packet, ensuring retransmission is possible in the event of a packet loss. The total amount of data packets that can be buffered is determined by the load round-trip time (RTT) between the sender and receiver at an operational 400G port rate. This RTT covers the entire sequence of events involved in reliable delivery: the time required for the sender to transmit data packets, the time for the receiver to receive and process data packets, the time for the receiver to generate and transmit the corresponding ACK, and the time required for that ACK to return to the sender and update the sender's transmission status.

[0162] In some embodiments, because the system operates using two virtual channels (VCs) that independently carry request and response traffic, each VC may or must satisfy buffering requirements, resulting in a total storage requirement equal to the per-VC RTT buffer multiplied by the number of VCs. The architecture is designed around a target load RTT of 4 microseconds, which includes latency introduced by the network switches and internal system pipelines. This ensures that sufficient transmit-side packet data is still available for any necessary retransmissions while maintaining full-rate operation across both virtual channels.

[0163] In some embodiments, the transmit buffer is organized into queues per {destination endpoint (XPU), virtual channel (VC)}, resulting in 256 queues for 128 endpoints across two VCs. Within a given VC, buffer space is not pre-reserved for each queue; instead, all queues in that VC opportunistically share the VC's allocation upon arrival, which maintains isolation between VCs while allowing flexible utilization within each VC.

[0164] In some embodiments, packet transmission uses a two-stage hierarchical scheduling approach. The queue scheduler first independently selects from the non-empty queues within each VC, and then the VC scheduler selects a port to serve between the two VCs. Each active transmit queue can request service with one of the following packet classes: Negative Acknowledgment (NACK) control packets without payload, dedicated acknowledgment (ACK) control packets without payload, and a retransmission or first packet carrying payload triggered by a received NACK or retransmission timeout. The queue scheduler applies strict priorities in that order, where NACKs and retransmissions are expected to be infrequent and the dedicated ACK rate is shaped by acknowledgment timeouts and, when available, acknowledgments loaded into the opposite direction of the data. When scheduling packets, the queue scheduler balances two configurable orthogonal priorities: arrival order, which minimizes latency based on the time it takes for the first byte of a packet to be packed; and packet size, which improves line efficiency by supporting larger packets with proportionally smaller header overheads. The tradeoff between these two objectives is software-tuned so that smaller, earlier packets do not cause larger, later packets to be unprocessed, or vice versa.

[0165] In some embodiments, during the second phase, the port scheduler arbitrates across VCs with the same strict priority among NACK, dedicated ACK, and data traffic, and uses a Weighted Deficit Round Robin (WDRR) algorithm for cross-VC data arbitration. The weight of each VC is software-configurable, ranging from 1 to 16, and is applied to the byte count including preamble and inter-packet gaps. The WDRR phase can optionally bias service towards smaller packets when one VC tends to generate significantly larger payloads than another, a situation that queue-level size bias cannot address because it operates only within a single VC. The system also supports optional port shaping on a per-400 gigabits-per-second port basis to prevent packet data from accumulating in the switch output buffer (OBM), and the scheduler accepts per-VC lossless link-level flow control signals in the form of Priority Flow Control (PFC) or Super Ethernet Alliance Credit-Based Flow Control (CBFC), depending on software selection.

[0166] In some embodiments, for congestion control, to limit in-flight data, and to protect destinations from overload, the system enforces a fixed congestion window per {destination XPU, VC}. The window size and the maximum allowed in-flight bytes per VC are software-configurable. If a destination exceeds its configured in-flight allowance, the scheduler temporarily marks it as ineligible until it is determined that the window should be advanced and the destination should be restored to ineligibility.

[0167] In some embodiments, for packet construction, prior to transmission, the packet construction phase uses a software-programmed template to construct online frames, the template specifying the Layer 2, Layer 3 / Layer 4 headers and the Reliable Transport Header (RTH) for each {VC, XPUID} combination. Using this configuration and the current transmit queue state, the system generates the L2 / L3 / L4 headers, sends the RTH reflection fields (e.g., destination endpoint and sequence status), appends an ECRC to the payload, and inserts zero padding as needed to meet the configured minimum Ethernet frame size.

[0168] Figure 10 Figure 1000 illustrates an example network receiving path of an Ethernet-based load-store system 1020 according to some embodiments. Figure 10 This demonstrates how to process, acknowledge, and unpack incoming Ethernet packets and how to deliver them to the SoC as AXI transactions. In this embodiment, the receive path begins by accepting Ethernet packets arriving from a 400G Ethernet PHY providing underlying MAC and / or PCS functionality via Ethernet PHY interface 1010. Once a packet is received, the system performs standard protocol checks and / or system-specific integrity check sequences 1021 to ensure that only valid packets can enter the reliable transport layer. These checks include verifying the Ethernet Frame Check Sequence (FCS), confirming that the encapsulation matches the configured transport format (e.g., Layer 2 - Internet Protocol Version 4 - User Datagram Protocol (L2 - IPv4 - UDP) or Artificial Intelligence Architecture Header (AFH)), checking IPv4 header fields where applicable, and confirming the reliable transport header, including its version, cyclic redundancy check, and partition identifier.

[0169] After the data packet passes the header and data integrity check 1021, Figure 10 This demonstrates the receiver-side logic of the system executing a reliable transport protocol. At this stage, the system evaluates the PSN contained in the reliable transport header against the expected sequence number maintained for the corresponding endpoint and virtual channel (VC). When packets arrive in order, the expected sequence number is advanced, and an acknowledgment is prepared to be sent back in the opposite direction. If a packet is out of order or lost, the system discards the packet, increments the error statistic, and generates a negative acknowledgment to request a retransmission. This reliable transport process ensures lossless, ordered delivery of packetized storage transactions even in the event of soft errors within the switching architecture or occasional packet loss.

[0170] After the agreement was confirmed, Figure 10Buffers 1022 depict the packet payload on a per-virtual-channel basis in dedicated receive buffers. Since network headers, padding, and ECRC are removed during parsing, only the payload portion of each packet is written to these buffers. The system then unpacks the byte-packed payloads 1023 into "unpacked" AXI transactions and stores them in shallow per-channel unpacking FIFOs. These FIFOs correspond to AXI channel semantics: read response and write response transactions are associated with the response virtual channel, while read request and write request metadata and payloads are associated with the request virtual channel. Flow control at receive boundaries is enforced by the unpacking FIFOs; when they reach capacity, they assert backpressure to enter the packing buffer stage, which in turn asserts lossless flow control toward the MAC / PCS layer, such as Priority Flow Control (PFC) or Super Ethernet Alliance Credit-Based Flow Control (CBFC). Once space becomes available, unpacking transactions are forwarded to the SoC via the AXI master and AXI slave interfaces, completing the conversion from Ethernet packets to AXI-compliant memory transactions.

[0171] In some embodiments, upon receiving a network frame, the system parses the data packet and applies a series of acknowledgment checks before allowing any payload to be reliably transmitted. The parser operates on Ethernet frames and first verifies the Frame Check Sequence (FCS); any frame that fails the FCS is discarded and recorded in an error counter (e.g., Sequence Error Counter 1050). The system also enforces a configured Maximum Transmit Unit (MTU) and discards and counts any frames whose length exceeds that MTU. The software selects the network header format used by the system—Layer 2 / Internet Protocol Version 4 / User Datagram Protocol (L2-IPv4-UDP) or Artificial Intelligence Architecture Header (AFH)—and Layer 2 checks confirm that the Ethernet type of the received frame corresponds to the configured format; frames with mismatched Ethernet types are similarly discarded and counted.

[0172] In some embodiments, if the packet is L2-IPv4-UDP, then the following IPv4 / UDP checks are performed on the IPv4 packet, as shown in Table 13. Most of these checks are derived from section 5.2.2 of RFC 1812. If any of these errors occur, the packet is dropped and the associated counter is incremented.

[0173]

[0174] Table 13. IPv4 packet inspection.

[0175] In some embodiments, the UDP checksum for all frames is ignored. A UDP length check is performed. If this fails, the packet is dropped and the counter is incremented.

[0176] In some embodiments, after the system parses the incoming Ethernet frame and identifies the configured encapsulation (Layer 2 / Internet Protocol Version 4 / User Datagram Protocol or AI Structure header), it expects the next 8 bytes to be a Reliable Transport Header (RTH). The system verifies the RTH version and confirms the ECRC; failure of either test results in the packet being dropped and the error counter being incremented. The RTH also carries a partition field, which may or must match a software configuration value used for multi-tenancy and / or job isolation; partition mismatch also results in a decreasing and increasing counter.

[0177] In some embodiments, packets filtered by header and integrity are subject to reliable transmission checks. The receiver compares the PSN of the packet in the RTH with the EPSN of the receiver in that stream. If the PSN is not equal to the EPSN, the packet is considered out of order and discarded, the associated counter is incremented, and a Negative Reverse Path Acknowledgment (NACK) is generated to indicate that retransmission should begin from the indicated EPSN.

[0178] In some embodiments, the payload from packets that have passed all checks is written to a received packet buffer on a per-virtual-channel (VC) basis, with one packet buffer per VC. Only the payload portion is stored; the link-layer and network-layer headers, ECRC, and any padding are discarded during parsing. The packet buffer is then read to unpack the byte-packed payload back into an AXI transaction, which is queued in a small, unpacked first-in-first-out (FIFO) structure indexed by {interface channel, VC}. In this organization, the VC0 packet buffer holds request-type transactions unpacked into a write address (AW), write data (W), and read address (AR) FIFO, while the VC1 packet buffer holds response-type transactions unpacked into a read data (R) and write response (B) FIFO. These unpacked FIFOs correspond to the channelized AXI data path and are identical to the logical structure referenced in the AXI data path / credit loop description.

[0179] In some embodiments, flow control at the receiving side is enforced hierarchically. If the unpacking FIFO for a given {interface channel, VC} saturates due to downstream backpressure from the on-chip system, then flow control is asserted toward the corresponding packet buffer for that traffic class (request or response), which temporarily halts reading from the packet buffer. At the network boundary, the RX packet buffer serves as the source of lossless link-level flow control, and its size determines the switch's priority flow control (PFC) response time, the hysteresis to limit the worst-case increase in line rate for PFC XON / XOFF switching, and sufficient burst absorption to accommodate transient surges. This arrangement prevents overflow while maintaining ordered delivery and ensuring that the system can cleanly backpressure the network under heavy load.

[0180] In some embodiments, Table 14 lists the statistical counters supported per 400G port of the system. These statistical counters are intended to be used for traffic monitoring and performance debugging.

[0181]

[0182]

[0183]

[0184] Table 14. Software Accessibility Statistics

[0185] In some embodiments, Table 15 lists the errors indicated to the software via the interrupt status register every 400G port:

[0186]

[0187]

[0188] Table 15. Error Events

[0189] In some embodiments, the 8×100 Ethernet PHY generates its own set of interrupt signals that report conditions associated with its internal subsystems, including PCS status, MAC events, Physical Media Dependent (PMD) conditions, and general port status indications. In some embodiments, interrupts generated by these port macros are not exposed as separate signals at the system level; instead, they are logically merged into the interrupt signal of 400G port 0. This merging allows software to handle both system-level and port macro-level interrupt events through a single interrupt path, while still using the corresponding interrupt status register to distinguish specific causes.

[0190] Figure 11 Figure 1100 illustrates an example integration of an Ethernet-based load-store system 1120 with a system-on-chip (SoC) logic 1110 according to some embodiments. Figure 11This describes the integration of an Ethernet-based load-memory system with SoC logic, where the "subsystem" includes two examples of Ethernet-based load-memory systems, 1125-0 and 1125-1, and an 8x100G Ethernet PHY 1122. In some embodiments, the term "System-on-Chip (SoC) logic" can refer to host-side digital logic that presents master and slave interfaces (e.g., AXI interfaces) to the load-memory system, including buffers, mapping logic, and flow control circuitry that generates transactions (e.g., AXI transactions) and receives returned responses. For example, the SoC logic may implement functions such as mapping AXI control fields to xCTL, xXPUID, and xCTLTYPE signals, managing per-channel credit counters for rate control, and providing readiness signaling, such as S2E_READY, to coordinate initialization with a remote transport subsystem. Depending on the implementation, the SoC logic may include a processor core, memory controller, interconnect architecture, or other computing elements that issue memory transactions requiring packetization and transmission across high-speed Ethernet.

[0191] In some embodiments, the 8×100 Ethernet PHY (1122) provides MAC and / or PCS functions for two 400 gigabits per second (400G) Ethernet ports, and Figure 11 Further details are provided regarding the timing 1112, interrupt 1114, reset 1116, and / or control bus 1118 used to coordinate the data paths on the two ports with the SoC 1110. The subsystem receives the port macro reference clock and three clocks provided by the SoC, identified as the core clock (CORE_CLK), the AXI data path clock (AXI_DP_CLK), and the AXI target clock (AXI_TGT_CLK). The integration also exposes one interrupt signal from each 400G port of the Ethernet-based load-memory system and two interrupts from the Ethernet PHY mapped to port 0, as well as an Advanced Extensible Interface (AXI) target interface used by software to configure internal registers and memory via a small SoC address window.

[0192] In some embodiments, a centralized reset of the integration is achieved via a single active-low top-level signal identified as System_RESET_N, which is expected to be driven at the SoC level by control and status register bits. Asserting System_RESET_N resets all logic within the subsystem, including the two 400G port pipelines, all Ethernet-based load-memory system and SoC data path interfaces, the AXI target interface block, and the Ethernet PHY. Following the deassertion, the software driver sequences the power-on initialization by ensuring clock stability, deasserting the reset, initializing and configuring the subsystem, and enabling the transmit (TX) and receive (RX) pipelines. The driver also performs port macro-initialization, including PCS and MAC configuration and Physical Media Dependent (PMD) tasks, and initializes and enables the required line-rate operation of the application standard 400G pipelines. Figure 11 Additionally, the integrated support for a per-example soft reset procedure 1124 is demonstrated, which allows software to reset a single 400G port pipeline without interfering with adjacent ports within a shared Ethernet PHY. During this soft reset, packet data residing in the MAC, TX, and RX pipelines of the affected port is refreshed, the configuration state in those pipelines and MAC is reset, and depending on the software policy, the link may remain connected or be intentionally disconnected.

[0193] In some embodiments, the AXI credit looping behavior at the SoC boundary supports up to 128 credits per channel, resulting in a round-trip latency of approximately 128 clock cycles over AXI_DP_CLK at 1 GHz. The latency budget includes the time from the source to transmit the transaction, the pipeline flight time to the receiver, receiver-side buffering and credit generation, credit return flight time, and source-side credit processing. In this calculation, an Ethernet-based load-and-store system requires approximately 20 clock cycles as the source, including transaction transmission and credit return processing, and approximately 20 clock cycles as the receiver, including reception, processing, and credit return. Credit depth and timing ensure that once software initialization is complete, the sender can maintain full bandwidth while waiting for credit returns, and the ingress inherently matches the downstream packet buffer and scheduler rate.

[0194] Figure 11 The components shown, such as two Ethernet-based load-store system examples with a shared 8×100 Ethernet PHY, unified reset control via System_RESET_N, software-orchestrated power-on and soft-reset flows, per-port interrupts, AXI target access for configuration, and explicit timing domains at CORE_CLK, AXI_DP_CLK, and AXI_TGT_CLK, provide a concrete SoC-level integration blueprint for implementing dual-port 400G operation with deterministic flow control and reliable transmission semantics.

[0195] Figure 12 This is a flowchart illustrating an example process 1200 for transmitting memory transactions over an Ethernet network according to some embodiments. In some embodiments, process 1200 can transmit memory transactions over an Ethernet network. In some embodiments, process 1200 is executed by one or more processors (e.g., processor 121) or one or more circuits of a device (e.g., device 102, network device 106, Ethernet-based load-store system 200, 300, 420, 521, 522, 620, 720, 760, 820, 860, 920, 1020, 1120). In other embodiments, process 1200 is executed by other entities. In some embodiments, process 1200 includes more than Figure 12 The steps shown in the text may be more, fewer, or different from those shown in the text.

[0196] In step 1210, the device's circuitry may receive one or more memory transaction requests from the host device via a first interface (e.g., AXI interface 230). The memory transaction requests may include one or more memory read requests or one or more memory write requests. The circuitry may be coupled to both the first and second interfaces (e.g., Ethernet PHY 210). The first interface may be configured to communicate with the host device. In some embodiments, the second interface may be configured to exchange Ethernet packets with an Ethernet network. The second interface may include a Media Access Control (MAC) processing circuitry and a Physical Writer Sublayer (PCS) processing circuitry.

[0197] In step 1220, the circuit may convert one or more memory transaction request packets received from the first interface into a first Ethernet packet. In step 1230, the circuit may transmit the first Ethernet packet to the Ethernet network via the second interface.

[0198] In step 1240, the circuit may receive a second Ethernet packet from the Ethernet network via the second interface. In step 1250, the circuit may depacketize the second Ethernet packet received from the second interface into one or more memory transaction responses. The one or more memory transaction responses include one or more memory read responses or one or more memory write responses.

[0199] In step 1260, the circuit may send one or more memory transaction responses to the host device via the first interface.

[0200] In some embodiments, a first interface (e.g., an AXI master interface) may be configured to communicate with a host device using multiple channels (e.g., AW, W, B, AR, R channels). The first interface may be configured to receive a memory write request from the host device, which includes a write address (e.g., write address 301) and write data (e.g., write data 302) received through different channels among the multiple channels. Circuitry may be configured to align the write address and write data before packetizing the memory write request. In some embodiments, circuitry may maintain one or more per-channel transmit queues (e.g., per-channel TX queue 422).

[0201] In some embodiments, the circuitry may be configured to perform credit-based flow control per channel (e.g., credit-based channels 306, 307, 308, 309, 310) between the sender and receiver of one or more transactions. The sender may decrement a credit counter (e.g., credit counter 624) after sending a transaction, and the receiver may return credits to the sender (e.g., credit return 622) when buffer resources (e.g., memory buffers) consumed by the transmitted transaction are released (e.g., based on a received ACK).

[0202] In some embodiments, the circuit may be configured to perform end-to-end integrity checks on each of the first Ethernet packet and the second Ethernet packet using end-to-end cyclic redundancy check (ECRC) to protect at least the payload of each packet across the Ethernet network.

[0203] In some embodiments, the circuitry can be configured to aggregate multiple memory transactions destined for a common endpoint into a single Ethernet packet up to a configurable packet size threshold. The circuitry may not split a single transaction across multiple Ethernet packets.

[0204] Figure 13 This is a flowchart illustrating an example process 1300 for transmitting memory transactions over an Ethernet network according to some embodiments. In some embodiments, process 1300 can transmit memory transactions over an Ethernet network. In some embodiments, process 1300 is executed by one or more processors (e.g., processor 121) or one or more circuits of a device (e.g., device 102, network device 106, Ethernet-based load-store system 200, 300, 420, 521, 522, 620, 720, 760, 820, 860, 920, 1020, 1120). In other embodiments, process 1300 is executed by other entities. In some embodiments, process 1300 includes more than Figure 13 The steps shown in the text may be more, fewer, or different from those shown in the text.

[0205] In step 1310, the device circuitry may transmit multiple data packets via a network interface (e.g., Ethernet PHY 210). Each data packet may correspond to one or more memory transactions and contain a corresponding data packet sequence number (PSN). In some embodiments, the corresponding PSN may be included in the reliable transmission header of each header.

[0206] In step 1320, the circuit may receive a first data packet from the endpoint via a network interface. In step 1330, the circuit may determine the first expected data packet sequence number (EPSN) of the first data packet and the first PSN contained in the first data packet.

[0207] In step 1340, in response to determining that the first EPSN is equal to the first PSN, the circuit may update the first EPSN to the second EPSN and send an acknowledgment (ACK) message identifying the second EPSN and indicating that the first data packet is an ordered data packet to the endpoint via the network interface.

[0208] In step 1350, in response to determining that the first EPSN is not equal to the first PSN, the circuit may send a negative acknowledgment (NACK) message identifying the first EPSN and indicating that the first data packet is an out-of-order data packet to the endpoint via the network interface.

[0209] In some embodiments, in response to determining that the first EPSN is not equal to the first PSN, the circuit may discard the first data packet, increment the sequence error counter (e.g., sequence error counter 1050), and send a NACK to the endpoint.

[0210] In some embodiments, the circuit may receive a second ACK message identifying an EPSN that is greater than a plurality of previously transmitted data packets. The circuit may release one or more buffers associated with one or more implicitly acknowledged data packets.

[0211] In some embodiments, the circuit may receive multiple ordered data packets. The circuit may be configured to determine a specific EPSN using the maximum PSN of the multiple ordered data packets. The circuit may send a single ACK message that identifies the specific EPSN and indicates the multiple ordered data packets.

[0212] In some embodiments, the circuit may use ECRC checksum protection to protect the payload associated with each data packet by discarding packets that fail the end-to-end CRC (ECRC) check. In some embodiments, if an out-of-order data packet is received (e.g., the received PSN is not equal to the EPSN), the circuit may generate a NACK message. In some embodiments, the circuit may prioritize the transmission of ACK and NACK messages over the transmission of data packets.

[0213] A reference to "or" can be interpreted as inclusive, such that any term described using "or" can refer to a single, more than one, or any of all descriptive terms. A reference to at least one of a list of combined terms can be interpreted as inclusive or to refer to a single, more than one, or any of all descriptive terms. For example, a reference to "at least one of 'A' and 'B'" can include only 'A', only 'B', or both 'A' and 'B'. Such references used in conjunction with "include" or other open terms can include additional items.

[0214] It should be noted that certain paragraphs of this disclosure may refer to terms such as “first” and “second” in relation to subsets of transmit space streams, probe frames, responses, and devices for identification or distinction from one another. These terms are not intended to associate entities (e.g., first and second devices) merely temporally or sequentially, although in some cases these entities may include such a relationship. These terms also do not limit the number of possible entities (e.g., STA, AP, beamformer, and / or beamformee) that can operate within the system or environment. It should be understood that the described system may provide any or multiple of those components, and these components may be provided on a standalone machine or, in some embodiments, on multiple machines in a distributed system. Furthermore, bit field positions may be changed, and multiple bit words may be used. Additionally, the described system and method may be provided as one or more computer-readable programs or executable instructions embodied on or in one or more articles of manufacture (e.g., floppy disk, hard disk, CD-ROM, flash memory card, PROM, RAM, ROM, or magnetic tape). The program can be implemented in any programming language (e.g., LISP, PERL, C, C++, C#) or in any bytecode language (e.g., JAVA). The software program or executable instructions can be stored as object code on or in one or more artifacts.

[0215] While the foregoing written description of the methods and systems enables those skilled in the art to make and use embodiments thereof, it should be understood and appreciated that variations, combinations, and equivalents of specific embodiments, methods, and examples exist herein. Therefore, the methods and systems of the present invention should not be limited to the foregoing embodiments, methods, and examples, but rather to all embodiments and methods within the scope and spirit of this disclosure.

Claims

1. An apparatus comprising: Network interface; and A circuit, coupled to the network interface and configured to: Multiple data packets, each corresponding to one or more memory transactions and containing a corresponding data packet sequence number (PSN), are sent via the network interface. Receive the first data packet from the endpoint via the network interface; Determine the first expected data packet sequence number (EPSN) of the first data packet and the first PSN contained in the first data packet; In response to determining that the first EPSN is equal to the first PSN, the first EPSN is updated to the second EPSN and an ACK message identifying the second EPSN and indicating that the first data packet is an ordered data packet is sent to the endpoint via the network interface; and In response to determining that the first EPSN is not equal to the first PSN, a negative acknowledgment (NACK) message identifying the first EPSN and indicating that the first data packet is an out-of-order data packet is sent to the endpoint via the network interface.

2. The apparatus of claim 1, wherein the corresponding PSN is included in the reliable transmission header of each header.

3. The apparatus of claim 1, wherein the circuitry is further configured to: In response to determining that the first EPSN is not equal to the first PSN and discarding the first data packet, incrementing the sequence error counter, and sending the NACK to the endpoint.

4. The apparatus of claim 1, wherein the circuitry is further configured to: Receive a second ACK message for the EPSN whose identifier is greater than that of multiple previously transmitted data packets, and Release one or more buffers associated with one or more implicitly acknowledged packets.

5. The apparatus of claim 1, wherein the circuitry is further configured to: Receive multiple ordered data packets; The specific EPSN is determined using the maximum PSN of the plurality of ordered data packets; and Send a single ACK message that identifies the specific EPSN and indicates the ordered data packets.

6. The apparatus of claim 1, wherein the circuitry is further configured to: The ECRC check is used to protect the payload associated with each data packet by discarding packets that fail the end-to-end CRC ECRC check.

7. The apparatus of claim 1, wherein the circuitry is further configured to: The transmission of ACK and NACK messages takes precedence over the transmission of data packets.

8. A circuit coupled to a network interface and configured to: Multiple data packets, each corresponding to one or more memory transactions and containing a corresponding data packet sequence number (PSN), are sent via the network interface. Receive the first data packet from the endpoint via the network interface; Determine the first expected data packet sequence number (EPSN) of the first data packet and the first PSN contained in the first data packet; In response to determining that the first EPSN is equal to the first PSN, the first EPSN is updated to the second EPSN, and an ACK message identifying the second EPSN and indicating that the first data packet is an ordered data packet is sent to the endpoint via the network interface; and In response to determining that the first EPSN is not equal to the first PSN, a negative acknowledgment (NACK) message identifying the first EPSN and indicating that the first data packet is an out-of-order data packet is sent to the endpoint via the network interface.

9. The circuit of claim 8, wherein the corresponding PSN is included in the reliable transmission header of each header.

10. The circuit of claim 8, wherein the circuit is further configured to: In response to determining that the first EPSN is not equal to the first PSN and discarding the first data packet, incrementing the sequence error counter, and sending the NACK to the endpoint.

11. The circuit of claim 8, wherein the circuit is further configured to: Receive a second ACK message for the EPSN whose identifier is greater than that of multiple previously transmitted data packets, and Release one or more buffers associated with one or more implicitly acknowledged packets.

12. The circuit of claim 8, wherein the circuit is further configured to: Receive multiple ordered data packets; The specific EPSN is determined using the maximum PSN of the plurality of ordered data packets; and Send a single ACK message that identifies the specific EPSN and indicates the ordered data packets.

13. The circuit of claim 8, wherein the circuit is further configured to: The ECRC check is used to protect the payload associated with each data packet by discarding packets that fail the end-to-end CRC ECRC check.

14. A method comprising: The circuit sends multiple data packets, each corresponding to one or more memory transactions and containing a corresponding data packet sequence number (PSN), via a network interface. The circuit receives the first data packet from the endpoint via the network interface; The circuit determines the first expected data packet sequence number (EPSN) of the first data packet and the first PSN contained in the first data packet; In response to determining that the first EPSN is equal to the first PSN, the circuit updates the first EPSN to the second EPSN and sends an ACK message identifying the second EPSN and indicating that the first data packet is an ordered data packet to the endpoint via the network interface; and In response to determining that the first EPSN is not equal to the first PSN, the circuit sends a negative acknowledgment (NACK) message identifying the first EPSN and indicating that the first data packet is an out-of-order data packet to the endpoint via the network interface.

15. The method of claim 14, wherein the corresponding PSN is included in the reliable transmission header of each header.

16. The method of claim 14, further comprising: In response to determining that the first EPSN is not equal to the first PSN and discarding the first data packet, incrementing the sequence error counter, and sending the NACK to the endpoint.

17. The method of claim 14, further comprising: Receive a second ACK message for the EPSN whose identifier is greater than that of multiple previously transmitted data packets, and Release one or more buffers associated with one or more implicitly acknowledged packets.

18. The method of claim 14, further comprising: Receive multiple ordered data packets; The maximum PSN of the multiple ordered data packets is used to determine the specific EPSN; and Send a single ACK message that identifies the specific EPSN and indicates the ordered data packets.

19. The method of claim 14, further comprising: The ECRC check is used to protect the payload associated with each data packet by discarding packets that fail the end-to-end CRC ECRC check.

20. The method of claim 14, further comprising: The transmission of ACK and NACK messages takes precedence over the transmission of data packets.

Images