Cross-domain CXL.mem translation for memory decomposition, prefetching, and multi-host operations enables composable abstraction and resource isolation.

Abstract

This provides a method for translating CXL messages, enabling communication between entities using different protocol parameters and allowing for flexible system topologies for multi-host access to Compute Express Link (CXL) devices. [Solution] The method includes receiving a sublossive (M2S) request from the first CXL.mem master to the first entity, converting the first CXL.mem M2S request into a second CXL.mem M2S request in which the value of at least one field selected from MemOpcode, Tag, or Address differs between the first and second requests, and sending the second CXL.mem M2S request to the second entity.

Description

Technical Field

[0001] Cross-Reference to Related Applications This application claims priority to the following: U.S. Provisional Patent Application No. 63 / 991,122, filed on February 25, 2026; U.S. Provisional Patent Application No. 63 / 931,124, filed on December 4, 2025; U.S. Provisional Patent Application No. 63 / 906,709, filed on October 28, 2025; U.S. Provisional Patent Application No. 63 / 895,053, filed on October 7, 2025; U.S. Provisional Patent Application No. 63 / 874,393, filed on September 2, 2025; U.S. Provisional Patent Application No. 63 / 856,653, filed on August 3, 2025; U.S. Provisional Patent Application No. 63 / 826,342, filed on June 18, 2025; U.S. Provisional Patent Application No. 63 / 811,859, filed on May 25, 2025; and U.S. Provisional Patent Application No. 63 / 784,089, filed on April 5, 2025.

Background Art

[0002] Compute Express Link (CXL) is an interconnect technology that enables cache-coherent memory access and high-bandwidth communication between a host and a device in modern computing systems. CXL is built on the physical and electrical interfaces defined by PCIe, while adding protocols that support memory semantics and cache coherency operations. The CXL specification defines protocols such as CXL.io for input / output operations, CXL.mem for memory access, and CXL.cache for cache coherency.

[0003] CXL.mem enables memory access between CXL hosts and CXL devices through master-to-sublossier (M2S) and sublossier-to-master (S2M) message channels. M2S messages transmit requests and data from the master (e.g., host) to the subordinate (e.g., memory device), while S2M messages transmit responses and data in the reverse direction. CXL.mem M2S request messages include fields such as MemOpcode, which indicates the type of memory operation; a tag that identifies the transaction; and Address, which specifies the physical address targeted by the memory operation.

[0004] CXL defines various device types for memory connectivity, including single logical devices (SLDs), multi-logical devices (MLDs), and global fabric-attached memory devices (GFDs). These device types support different functionalities and may utilize different addressing schemes. CXL hosts use the host physical address (HPA) space for memory access, while CXL devices may internally use the device physical address (DPA) space.

[0005] CXL switches route CXL traffic between hosts and devices on a fabric topology. A typical CXL switch uses a Host Managed Device Memory (HDM) decoder to determine routing based on address ranges and direct requests to the appropriate downstream port. In a typical CXL topology, the CXL switches forwarding requests while retaining the original address values ​​that were included in the request. [Overview of the Initiative]

[0006] Data center architectures often utilize distributed memory systems where compute hosts access shared memory resources. Different entities within such systems may employ different addressing schemes, tag management conventions, or memory operation types, potentially leading to incompatibility when entities attempt to communicate using CXL.mem. Furthermore, standard CXL topologies can impose constraints on how hosts access CXL devices. CXL switches route requests based on their HDM decoder configuration, but retain the original address values ​​instead of performing address translation. Some implementations provide methods and systems for translating between CXL.mem messages, enabling communication between entities using different protocol parameters and allowing for flexible system topologies for multi-host access to CXL devices.

[0007] In various implementations, the method of converting between CXL messages involves the first entity receiving a subloophair (M2S) request from the master of CXL.mem. The computer converts the first CXL.mem M2S request into a second CXL.mem M2S request, and if the value of at least one field selected from MemOpcode, tags, or addresses differs between the first and second CXL.mem M2S requests, it sends the second CXL.mem M2S request to the second entity.

[0008] Other implementations, in order to enable multi-host access to the CXL device, include receiving a CXL.mem Master-to-Subordinate (M2S) request from the first entity with the first physical address; receiving a second CXL.mem M2S request from the second entity with the second physical address, and translating the first and second physical addresses to the third and fourth physical addresses within the address space used by the CXL device; generating third and fourth CXL.mem M2S requests consisting of the third and fourth physical addresses; and then sending the third and fourth CXL.mem M2S requests to the CXL device. [Brief explanation of the drawing]

[0009] [Figure 1A] This example shows a processing pipeline for extracting passenger protocol messages from carrier protocol communications based on IEEE 802.3 PMA. [Figure 1B] This shows an example of a packet structure suitable for L3 switching operations. [Figure 1C] This shows an example of a packet structure suitable for L2 switching operations. [Figure 2A-2C] Figure 1B shows three examples of variations of passenger protocol PDUs that can be encapsulated within the carrier protocol PDU shown. [Figure 3] This example shows a processor with an RPU-based interface and a system where IEEE 802.3 PMA is connected to a CXL device. [Figure 4A] This shows an example of a system consisting of a processor with a CXL EP and PHY based on IEEE 802.3 PMA. [Figure 4B] This example of a TFD demonstrates CXL.mem communication over a carrier protocol using a PHY based on IEEE 802.3 PMA. [Figure 5A] This shows an example of a system consisting of a processor with multiple interfaces. [Figure 5B] This example shows a system consisting of a processor capable of handling external requests via a CCG optimized for processing CXL.mem traffic. [Figure 6A] This shows an example of a system consisting of memory switches, memory pools, or global fabric-attached memory devices (GCDs). [Figure 6B] This shows an example of a system consisting of a host, a memory expander, and a coupled memory pool. [Figure 7A] This shows an example of a system consisting of a memory pool with two or more MxPUs. [Figure 7B] This example shows a system consisting of at least one MxPU and a memory pool comprising at least one xPU or CPU. [Figure 8A] This example shows a system consisting of a processor, a memory pool comprising DRAM and an RPU, which performs physical address translation between hosts. [Figure 8B] This example shows a system where the processor consists of a memory pool made up of CXL multi-head devices (MHDs) connected to DRAM. [Figure 9] This example shows a system consisting of an AI memory switch or memory pool, including a CXL multi-head device (MHD). [Figure 10A] This example shows a system where entities are coupled to a CXL device and the RPU of an ARM architecture processor via an IEEE 802.3 PHY. [Figure 10B] This example shows how TFD translates a CXL.mem message into an ARM CHI request. [Figure 11] This example demonstrates multi-host memory pooling or sharing using a switch-based topology with a physical layer based on IEEE 802.3 PMA. [Figure 12A] This shows an example of a system that converts between the first and second CXL.cache files. [Figure 12B] This shows an example of a transaction flow diagram (TFD) illustrating the conversion between a CXL.cache H2D SnpInv request and a CXL.cache D2H CLFlush request. [Figure 12C] This shows an example of a TFD illustrating the conversion between a CXL.cache H2D SnpCur request and a CXL.cache D2H RdCurr request. [Figure 13A] This shows an example of a system that performs conversion between CXL.cache messages, such as between a CXL.cache Host-to-Device (H2D) request and a CXL.cache Device-to-Host (D2H) request. [Figure 13B] This shows an example of a TFD illustrating conversion between CXL.cache transactions. [Figure 14A] This shows an example of a system that performs conversion between CXL.cache messages, for example between a CXL.cache H2D request including Snoops and a CXL.cache D2H request including a read opcode, [Figure 14B] This shows an example of a TFD illustrating the conversion between a CXL.cache H2D request (including SnpData) and a CXL.cache D2H request (RdShared). [Figure 14C] This shows an example of a TFD illustrating the conversion between a CXL.cache H2D request including SnpData and a CXL.cache D2H request including RdOwn. [Figure 14D] This shows an example of a TFD illustrating the conversion between a CXL.cache H2D request (including SnpInv) and a CXL.cache D2H request (including CLFlush). [Figure 15A] This shows an example of a TFD illustrating the conversion between CXL.cache H2D SnpInv and CXL.cache D2H RdOwnNoData. [Figure 15B]The TFD illustrates the conversion between CXL.cache H2D SnpInv and CXL.cache D2H RdOwn, showing an example of returning a cache line in the Modified state (GO-M). [Figure 15C] The TFD illustrates the conversion between CXL.cache H2D SnpInv and CXL.cache D2H RdOwn, showing an example of returning an exclusive cache line (GO-E). [Figure 16A] This shows an example of a system that performs translation between CXL.cache messages. [Figure 16B] This shows an example TFD illustrating the conversion between CXL.cache H2D SnpData and CXL.cache D2H RdShared, which returns a cache line in the shared state (GO-S). [Figure 17A] This shows an example of a system that performs conversion between CXL.cache messages that affect the state transition of cache lines. [Figure 17B] This shows an example transaction flow diagram (TFD) illustrating the conversion between a CXL.cache D2H CLFlush request and a CXL.cache H2D SnpInv request. [Figure 17C] This shows an example TFD illustrating the translation between CXL.cache D2H RdCurr and CXL.cache H2D SnpCur. [Figure 18A] This shows an example TFD illustrating intent-based translation between CXL.mem M2S SnpCur and CXL.cache D2H RdCurr. [Figure 18B] This shows an example TFD illustrating intent-based translation between CXL.mem M2S SnpData and CXL.cache D2H RdShared. [Figure 18C] This shows an example TFD illustrating intent-based translation between CXL.mem M2S SnpInv and CXL.cache D2H RdOwn. [Figure 19A]This shows an example of a TFD illustrating intent-based conversion between CXL.cache D2H RdCurr and CXL.mem M2S SnpCur. [Figure 19B] This shows an example of a TFD illustrating intent-based conversion between CXL.cache D2H RdShared and CXL.mem M2S SnpData. [Figure 19C] This shows an example of a TFD illustrating intent-based translation between CXL.cache D2H RdOwn and CXL.mem M2S SnpInv. [Figure 20A] This shows an example of a system that translates between the first CXL.mem and the second CXL.mem. [Figure 20B] This shows an example transaction flow diagram (TFD) illustrating the conversion between a CXL.mem M2S MemRdData request and a CXL.mem M2S MemRd request, with optional inferred memory reads also available. [Figure 21A] This shows an example of a system consisting of computers with buffers / caches, which are converted between the first and second CXL.mem files. [Figure 21B] The TFD example shows the conversion between the first CXL.mem M2S MemSpecRd and the second CXL.mem M2S MemSpecRd, and optionally it is possible to start a third CXL.mem M2S MemSpecRd. [Figure 21C] This shows an example of a TFD illustrating the translation between CXL.mem M2S MemSpecRd and CXL.mem M2S MemRd*. [Figure 22A] This shows an example of a system that converts between three CXL.mem interfaces. [Figure 22B] This shows an example of a TFD illustrating the translation between four CXL.mem M2S requests. [Figure 23A] This example shows a system consisting of a processor / switch and a CXL device, configured to allow external entities to access resources linked to the processor. [Figure 23B] This shows an example TFD illustrating the conversion between the first CXL.mem transactions, including MemRd*. [Figure 24A] According to CXL.mem, this provides an example of a system consisting of a processor configured to communicate with multiple hosts. [Figure 24B] This shows an example of a TFD (Transaction Function Decision) where two CXL.mem transactions directed to different memory locations are combined into a single processor. [Figure 25A] This shows an example of a conversion system between CXL.mem and CXL.io. [Figure 25B] This shows an example of a TFD illustrating the conversion between CXL.mem M2S requests and CXL.io UIOMRd. [Figure 25C] This shows an example of a TFD illustrating the translation between CXL.mem M2S requests and CXL.io MRds. [Figure 26A] This example shows a system consisting of a first host connected to the first memory, a second host connected to the second memory, and a computer that performs conversions between CXL.mem and CXL.io. [Figure 26B] This shows an example of a TFD illustrating the conversion between CXL.mem M2S requests (RwD) and CXL.io memory write requests (MWr). [Figure 26C] This shows an example of a TFD illustrating the conversion between CXL.mem M2S requests (RwD) and CXL.io UIO memory write requests (UIOMWr). [Figure 27A] This shows an example of a system that performs conversions between CXL protocols, such as between CXL.io and CXL.mem. [Figure 27B] This shows an example of a TFD illustrating the translation between CXL.io TLP and CXL.mem messages. [Figure 27C] This shows an example of a TFD illustrating the translation between CXL.io UIO TLP and CXL.mem messages. [Figure 28A]This shows an example of a system that converts between CXL.mem and PCIe. [Figure 28B] This shows an example transaction flow diagram (TFD) illustrating the conversion between CXL.mem M2S requests and PCIe MRd. [Figure 28C] This shows an example TFD illustrating the conversion between CXL.mem M2S requests and PCIe UIOMRd. [Figure 29A] This example shows a system consisting of a first and second entity coupled with a computer that converts between CXL.mem and PCIe. [Figure 29B] CXL.mem shows an example of a TFD demonstrating the conversion between M2S RwD and PCIe MWr. [Figure 29C] CXL.mem shows an example TFD demonstrating conversion between M2S RwD and PCIe UIOMWr. [Figure 30A] This example shows a system consisting of cables that convert between CXL-based and PCIe-based traffic. [Figure 30B] This shows an example of a TFD illustrating the conversion via active cabling between CXL.mem transactions and PCIe transactions. [Figure 31A] This shows an example of a system that converts between PCIe and CXL.mem. [Figure 31B] This shows an example of a TFD illustrating the conversion between PCIe TLP and CXL.mem messages. [Figure 31C] This shows an example of a TFD illustrating the conversion between PCIe UIO TLP and CXL.mem messages. [Figure 32A] This shows an example of a system that converts CXL-based traffic to PCIe-based traffic. [Figure 32B] This shows an example TFD demonstrating conversion between UIOMRd and PCIe UIOMRd at CXL.io. [Figure 32C]CXL.io shows an example of a TFD (TFD) illustrating the conversion between UIOMRd and PCIe MRd. [Figure 33A] This shows an example of a system that converts traffic between CXL.io. [Figure 33B] This shows an example of a TFD illustrating the translation between CXL.io MRd and CXL.io UIOMRd. [Figure 33C] This shows an example of a TFD illustrating the translation between UIOMRd and CXL.io MRd. [Figure 34A] This diagram shows a system where Fabric Resource Entities (FREs) are connected to the CXL fabric, computers are linked to the FREs, and target entities are linked to the computers. [Figure 34B] This flowchart shows how to enable bidirectional communication using FRE within a CXL fabric. [Figure 35A] This shows an example of a system that performs conversions between first and second PCIe-based protocols. [Figure 35B] This shows an example of a TFD (Transaction Format) illustrating the conversion between PCIe MRd and PCIe UIOMRd. [Figure 35C] This shows an example of a TFD (Transaction Format) illustrating the conversion between PCIe UIOMRd and PCIe MRd. [Figures 36A-36B] This document presents two approaches to converting xPU designs to CXL memory devices. [Figure 37] This example demonstrates the construction of a CXL MHD memory pool based on an xPU, including a CXL RP. [Figure 38] This example shows how to convert an xPU design to a CXL memory device. [Figure 39] This example shows a processor consisting of an RPU that can translate between different combinations of CXL device types. [Figures 40A-40B] This shows an example of a system consisting of an MxPU and an EP or GFD. [Figure 41A]This example illustrates a system where an external entity can access memory resources mapped to the address space used by the processor's coherent interconnect. [Figure 41B] This example shows a transaction flow diagram (TFD) illustrating the RPU conversion of UIOMRd memory read requests and CXL.mem M2S requests. [Figure 42A] This example shows a system where a processor / switch is configured to allow external entities to access resources coupled to the processor. [Figure 42B] This shows an example of a TFD illustrating the translation that a processor using MemRd performs between the first and second CXL.mem files. [Figure 43A] This shows an example of a system consisting of a CXL device and a processor made up of CXL RPs. [Figure 43B] This shows an example TFD illustrating MRd requests, CXL.mem M2S requests, and CXL.io UIOMRd requests, as well as CXL.io conversions. [Figure 44A] This shows an example of a system consisting of a processor and a CXL endpoint. [Figure 44B] This shows an example of a TFD illustrating the conversion between CXL.mem and CXL.cache messages. [Figure 45A] This example shows a system consisting of processors whose coherent interconnects are linked via a CXL EP and ISoL interface. [Figure 45B] This example shows how TFD converts a CXL.mem M2S Read request to an ISoL request. [Figure 46A] This example shows a system composed of entities such as processors and node controllers, which configures translation between CXL-based messages and ISoL messages. [Figure 46B] This shows an example of a TFD (Transaction Format) demonstrating the conversion between CXL.mem and Intel UPI. [Figure 47A] This example shows a system consisting of a processor, node controller, or switch, where a CXL device is assigned to translate between CXL-based traffic and ISoL traffic. [Figure 47B] This shows an example of a TFD illustrating the translation between CXL.mem and UPI, translating errors and signs of data corruption (e.g., poison indications). [Figure 48A] This example shows a system consisting of a processor or RPU that assigns translations between CXL-based traffic and ISoL traffic. [Figure 48B] This shows an example of a TFD illustrating the conversion between CXL.mem messages and ISoL messages. [Figure 49A] This represents a system consisting of an xPU design based on prior art such as processor design, and includes reusable areas. [Figure 49B] This shows examples of multi-head device (MHD) implementations based on xPU or MxPU designs (e.g., processor designs), which include reusable areas. [Figure 49C] This shows an example of a processor consisting of termination circuits implemented at the interface between silicon die regions. [Figure 50A] This shows an example where the silicon device functions as the established xPU design before the modifications. [Figure 50B] This shows an example of a silicon device that can provide the functionality of CXL MHD. [Figure 50C] This shows an example of a silicon device that can provide the functionality of an UALink switch. [Figure 51A] This example shows a system consisting of processors including a CXL EP, configured to allow external entities to access memory resources mapped to the address space used by the processor's coherent connections. [Figure 51B] This example demonstrates how TFD converts a CXL.mem M2S request to an M2S request used by the processor's coherent interconnect. [Figure 52A] This example shows a system that includes a CXL device containing a processor, configured to allow external entities to access memory resources allocated to the processor's coherent interconnect. [Figure 52B] This example shows two CXL.mem requests where the TFD is mapped to the address space used by the processor's coherent interconnect. [Figure 53] This example demonstrates how to connect processors in a memory pool using a node controller. [Figure 54A] This example shows a system that can access DRAM and other resources through a CXL device port and a coherent interconnect interface. [Figure 54B] This shows an example of a TFD illustrating address translation between CXL.mem M2S Req MemRd and ARM CHI REQ ReadOnce. [Figure 55A] This shows an example of a system consisting of a CXL memory switch appliance with a CPU equipped with processing cores and a memory controller. [Figure 55B] This TFD example demonstrates a multi-host memory access scenario where two entities access memory through a shared coherent interconnect infrastructure. [Modes for carrying out the invention]

[0010] Some implementations of the following methods involve processing pipelines, such as Fabric Processing Units (FPUs), which extract and reconstruct CXL PDUs from carrier protocol communications received at the physical layer. This may also be done by software-defined methods, such as protocol processing firmware running on the FPU. Carrier protocols can refer to protocols that transfer or encapsulate other protocol data across networks and fabrics, such as Ethernet, Ultra Ethernet Transport (UET), Ethernet for Scale-Up Networking (ESUN), Scale-Up Ethernet (SUE), NVLink, and UALink. Passenger protocols can refer to transport protocols in which data is encapsulated within a carrier protocol, such as CXL.mem, CXL.cache, and CXL.io. A carrier protocol PDU refers to a protocol data unit of a carrier protocol, and a CXL PDU refers to a protocol data unit of a CXL subprotocol. A CXL PDU may represent data indicating a CXL message or a subset thereof, and may include fields that have been formatted, reduced, or otherwise adapted for transmission within a carrier protocol PDU.

[0011] Modern data center architectures benefit from transporting CXL traffic over network fabrics that utilize an IEEE 802.3-based physical layer, enabling rack-scale and cluster-scale deployments, memory partitioning, configurable infrastructure, and remote memory access. When a CXL PDU is encapsulated within a carrier protocol PDU, the carrier protocol may use a different field format, omit fields unnecessary for transport, or encode the CXL information in a carrier-specific representation. Processing pipelines within RPUs or similar processing units may receive the carrier protocol PDU, extract the encapsulated CXL information, perform field conversions between the carrier format and the CXL format, and reconstruct a complete CXL PDU compliant with the CXL specification for delivery to CXL devices or hosts. The pipeline may include stages such as physical coding sublayer (PCS) processing, analysis, validation, stream extraction, access control, field conversion, stream editing, address translation, and protocol translation, with different stages being used depending on the carrier protocol type, passenger protocol type, and system configuration.

[0012] Various implementations include: receiving transmissions of protocol data units (PDUs) of a carrier protocol (carrier protocol PDU) encapsulating data representing a Compute Express Link (CXL) PDU through a physical layer based on an IEEE 802.3 physical medium attachment (PMA); parsing the carrier protocol PDU to identify the passenger protocol type; extracting fields of the CXL PDU from the carrier protocol PDU and converting at least one extracted field from the format associated with the carrier protocol PDU to a field compliant with the CXL specification; and reconstructing the output CXL PDU, including the translated fields and at least one additional field not present in the carrier protocol PDU. The processing pipeline may be implemented in hardware, firmware, software, or a combination thereof, within an FPU, RPU, or similar processing unit coupled with the physical layer based on an IEEE 802.3 PMA. The physical layer receives transmissions from external entities such as hosts, accelerators, switches, and memory devices and communicates the CXL traffic encapsulated within the carrier protocol PDU. The analysis phase examines protocol-specific patterns and markers within the data stream to identify the carrier protocol type and the type of CXL subprotocol encapsulated within, such as CXL.mem, CXL.cache, and CXL.io. The extraction phase identifies and extracts fields of interest from the carrier protocol PDU, including CXL opcodes, physical addresses, transaction identifiers, and other protocol-specific fields. The conversion phase may convert one or more fields extracted from the representations and encodings used by the carrier protocol into representations compliant with the CXL specification. For example, converting carrier-specific command encodings to CXL opcodes.During the reconstruction phase, a complete CXL PDU can be assembled by combining translated fields with additional fields not included in the carrier protocol PDU (such as reserved fields, effectiveness indicators, and metadata fields that are omitted from the carrier PDU to save bandwidth but are used by the CXL device for proper analysis and processing).

[0013] In some implementations of this method, at least one additional field is reconstructed based on either a configuration or a default value, and access rights are further verified based on either a physical address or an opcode extracted from the CXL PDU. If access rights are not met, processing of the output CXL PDU is blocked. Reconstruction of fields not present in the carrier protocol PDU may utilize configuration parameters stored in RPU registers or memory, or default values ​​defined in the CXL specification or system policy. For example, the Valid field is reconstructed as Valid based on the presence of a properly formed carrier protocol PDU, and the Reserved (RSVD) field is reconstructed using a configured default value such as zero. Access control verification is performed by an Access Control List (ACL) module in the processing pipeline, which evaluates the extracted CXL fields against the access policy. The ACL module examines the physical address to determine if access to the requested memory region is permitted, or checks the opcode to determine if the requested operation type is permitted. If access rights are not met, the pipeline discards the CXL PDU, generates an error response, or logs the access violation. This access control operates at the CXL protocol level, not the carrier protocol level, enabling fine-grained memory access control specific to the CXL address space and operation type.

[0014] Some implementations also include translating the physical address extracted from the CXL PDU from the first physical address space to the second physical address space. Address translation is performed by the address translation stage in the processing pipeline, downstream from the field extraction and translation stages. The first physical address space corresponds to the host physical address (HPA) space used by the external entity that initiated the CXL request, and the second physical address space may correspond to the HPA space used by the local host or the device physical address (DPA) space used by the local CXL device. Translation is implemented using lookup tables, page tables, hash tables, base offset calculations, or programmable translation functions. Address translation enables entities using different address spaces to communicate via CXL on the carrier protocol fabric, eliminating the need to share a common address space or address mapping.

[0015] In some implementations, an output CXL PDU is translated from the first CXL channel to the second CXL channel, with the first and second CXL channels treated as different CXL subprotocols. Protocol translation may occur in the protocol translator stage of the processing pipeline, downstream from the stream editing and address translation stages. The first CXL channel may be CXL.mem, the second CXL channel CXL.cache, or vice versa, depending on the system configuration and the type of entities involved. For example, if the receiving entity is a CXL host issuing snoop requests to a local CXL device, the protocol translator may translate a CXL.mem M2S request message to a CXL.cache H2D request message. Conversely, if the receiving entity is a memory device processing memory access requests, the protocol translator may translate a CXL.cache H2D request message to a CXL.mem M2S request message. Cross-channel translation enables heterogeneous CXL deployments where different entities communicate over a common carrier protocol fabric while utilizing different CXL subprotocols.

[0016] In some implementations of this approach, the carrier protocol consists of at least one of Ethernet, Ultra Ethernet Transport (UET), Ethernet for Scale-Up Networking (ESUN), Scale-Up Ethernet (SUE), NVLink, or UALink, and the physical layer operates at a lane rate of at least 100 Gbps. Different carrier protocols may utilize IEEE 802.3 PMA in different ways while maintaining compatibility with the PMA interface specification. Ethernet may combine standard Ethernet framing with MAC layer processing, while UET, ESUN, and SUE may utilize optimized framing adapted for high-performance computing and artificial intelligence workloads. UALink can employ UALink-specific data link and transport layer protocols while utilizing IEEE 802.3 PMA as the physical layer. A lane rate of at least 100 Gbps aligns with the IEEE 802.3 PMA specification, such as speeds of 100 GBASE or higher, and provides sufficient bandwidth for CXL memory traffic with high-frequency, low-latency transactions. The increase in lane speeds, such as 200 Gbps and 400 Gbps, could further increase the available bandwidth for encapsulated CXL traffic.

[0017] Some implementations also receive a CXL response PDU, remove at least one field from it, encapsulate the remaining fields within a second carrier protocol PDU, and transmit the second carrier protocol PDU through the physical layer based on IEEE 802.3 PMA. The return path may also act as a reverse pipeline that processes CXL responses generated by the local CXL device or host and transmits them to the remote entity through the carrier protocol fabric. Removing fields from the CXL response PDU can reduce bandwidth consumption by omitting fields that can be reconfigured at the receiving end (valid fields, pending fields, metadata fields with default values ​​or derivable values). Encapsulation may involve generating carrier protocol headers and trailers appropriate for the carrier protocol type used. Return path processing may mirror the stages of the inbound processing pipeline in reverse order. The stream editor removes unnecessary fields, the field translator converts CXL-specific field representations to carrier-specific formats, and the PCS performs encoding operations such as 64B / 66B encoding, scrambling, and FEC encoding for transmission over IEEE 802.3 PMA.

[0018] In some implementations of this technique, a CXL PDU consists of a CXL.mem Master-to-Subordinate (M2S) PDU, with extracted fields consisting of the MemOpcode and Address fields, and additional fields not present in the carrier protocol PDU, such as the Valid, Reserved (RSVD), MetaField, SnpType, or Traffic Class (TC) field. A CXL.mem M2S PDU may refer to a master-to-sublocate message from an M2S Req channel carrying no-data requests such as reads or invalidations, or from an M2S RwD channel carrying data-containing requests such as writes. Both channel types may include the MemOpcode and Address fields as the primary extracted fields, and may also include other fields such as Valid, MetaField, SnpType, Tag, TC, and RSVD. When a carrier protocol PDU encapsulates information from a CXL.mem, it may include the MemOpcode and Address fields. These contain the operation type and target address necessary for routing and processing. Other fields such as Valid, RSVD, MetaField, SnpType, and TC can be omitted from the carrier protocol PDU if their values ​​can be reconstructed. The Valid field can be reconstructed based on the presence of a properly formed carrier protocol PDU. The RSVD field can be reconstructed with a default value of zero. The MetaField, SnpType, and TC fields can be reconstructed based on context derived from RPU configuration parameters, system policies, or MemOpcode or other extracted fields. The tag field, which provides a transaction identifier indicating the correlation of responses, can also be extracted from the carrier protocol PDU.

[0019] In some implementations of this technique, the translation converts the command field of the carrier protocol PDU into a MemOpcode field compliant with the CXL specification, and the MemOpcode field is converted to one of the following: MemRd, MemRdData, MemWr, MemWrPtl, or MemInv. The carrier protocol may use a carrier-specific command field to encode memory operation types, with a format, bit width, and encoding different from the CXL mem-opcode field. The field translator can map carrier-specific command encodings to numerical values ​​such as MemRd for memory read operations, MemRdData for memory read operations with specific metadata processing, MemWr for full cache line write operations, MemWrPtl for partial cache line writes, and MemInv for invalidation operations. The mapping is configurable, allowing the same processing pipeline to support different carrier protocols that utilize different command encoding schemes.

[0020] In some implementations of this method, a CXL PDU consists of a CXL.cache host-to-device (H2D) request PDU, with extracted fields consisting of an opcode field, an address field, and a unique queue ID (UQID) field. Additional fields not present in the carrier protocol PDU consist of at least one of either a valid field or a reserved (RSVD) field. A CXL.cache H2D request PDU may include fields such as Valid, Opcode, Address, UQID, and RSVD as defined in the CXL specification. The opcode field may contain H2D request opcodes such as SnpData, SnpInv, and SnpCur, indicating the type of snoop operation. The address field may hold the physical address of the cache line targeted by the snoop operation. The UQID field identifies the host entry that initiated the request and enables response routing. The Valid and RSVD fields are optional as they allow for the reconstruction of the PDU's values. The Valid field uses a default value based on the existence of the PDU, while the RSVD field uses a default value. Different frit modes defined in the CXL specification (such as 68B frit, 256B frit, and PBR frit) include different sets of additional fields, such as CacheID, SPID, and DPID, which may be extracted or reconstructed depending on the frit mode negotiated for the target CXL link.

[0021] In some implementations of this technique, a CXL PDU consists of a CXL.cache device-to-host (D2H) request PDU, with extracted fields consisting of an Opcode field, a Command Queue Identifier (CQID) field, and an Address field, and additional fields not present in the carrier protocol PDU consisting of at least one of the Valid field, NonTemporal (NT) field, or Reserved (RSVD) field. A D2H request PDU in CXL.cache may contain fields such as Valid, Opcode, CQID, NT, Address, and RSVD fields as defined in the CXL specification. The direction of a D2H request conveys a request from device to host and uses D2H request opcodes such as RdShared, RdOwn, RdCurr, RdOwnNoData, and CLFlush. The CQID field may identify a device tracker entry related to the request. The NT (NonTemporal) field provides a cache hint indicating that the requested data will not be reused in the near future, allowing the host not to cache the data in a preferred cache location or to treat it as non-cache. The RSVD field in a D2H request may be larger than that in an H2D request, and the NT field may hold default values ​​that can be reconstructed based on the system configuration. The field set of a D2H request differs from that of an H2D request in the transaction identifier (CQID vs. UQID) and available fields (NT vs. CacheID in certain frit modes), which can result in different extraction and reconstruction behavior in each direction.

[0022] In some implementations of this method, a CXL PDU is a CXL.io Transaction Layer Packet (TLP), the extracted fields consist of a Type field, an Address field, and a Tag field, and the translation converts the Command field of the Carrier Protocol PDU to the Fmt / Type field of the CXL.io TLP. It utilizes TLPs based on the CXL.io PCIe Transaction Layer specification. The TLP header may include fields such as Fmt / Type (encoding the packet format and transaction type), TC (Traffic Class), Attr (Attribute), EP (Error Poisoned), Length, Requester ID, Tag, and Address. The Fmt and Type fields together determine the TLP type, such as Memory Read (MRd), Memory Write (MWr), Data Completed (CplD), and Configuration Read / Write. Unlike PCIe Fmt / Type encoding, carrier protocols may encode CXL.io transaction types using carrier-specific command fields. The field translator can map carrier-specific commands to the appropriate Fmt / Type values. In frit mode, the FMT field is absorbed into the type field, and the conversion can support both frit mode and non-flit mode TLP header formats. Tag and address fields can be extracted from the carrier protocol PDU for routing and transaction tracking, while other TLP header fields such as TC, Attr, and EP can be omitted from the carrier PDU and reconstructed using default values ​​or configuration parameters.

[0023] In some implementations of this method, an active cable consists of first and second pluggable modules coupled by a physical medium, and the active cable incorporates hardware circuitry configured to perform the method. In some implementations of this method, a device acting as a switch is provided, and this device consists of switching circuitry configured to perform the method.

[0024] Various implementations include the following devices: a physical layer based on an IEEE 802.3 physical media attachment (PMA) configured to receive protocol data units (PDUs) of a carrier protocol (carrier protocol PDU) that encapsulates data representing a Compute Express Link (CXL) PDU; and a processing unit (processor unit) coupled to the physical layer configured to parse the carrier protocol PDU and identify the passenger protocol type. It extracts fields of the CXL PDU from the carrier protocol PDU; converts at least one extracted field from the format associated with the carrier protocol PDU into a field compliant with the CXL specification; and reconstructs an output CXL PDU containing the translated field and at least one additional field not present in the carrier protocol PDU. The devices may be implemented as an integrated circuit, SoC, FPGA, SmartNIC, data processing unit (DPU), fabric processing unit (FPU), or other appropriate hardware platform including a physical layer based on an IEEE 802.3 PMA and these processing units. The processing unit may contain dedicated hardware logic, a programmable processor, or a combination thereof. The processing unit can implement pipeline stages such as analysis, extraction, translation, and reconstruction as hardware pipeline stages that process carrier protocol PDUs at line rate, or as software-driven stages that provide configurability and flexibility. The device may include a physical layer based on IEEE 802.3 PMA for communication with external organizations and may also include processing units and pipeline instances for parallel processing of carrier protocol PDUs.

[0025] Some implementations of the following methods are also relevant to the transmission of bandwidth-optimized CXL protocol data over carrier protocols using a physical layer based on IEEE 802.3 PMA. When CXL PDUs are transmitted over a carrier protocol fabric, the bandwidth available for encapsulated CXL traffic may be limited by carrier protocol overhead, physical layer data rate, and number of concurrent transactions. Bandwidth optimization can be achieved by encapsulating some of the CXL PDU fields within the carrier protocol PDU and omitting fields that can be reconstructed at the receiving end. Different field inclusion levels may be used depending on available bandwidth, quality of service requirements, and system configuration.

[0026] For example, a CXL.mem M2S request may include fields such as Valid, MemOpcode, MetaField, SnpType, Address, Tag, TC, and RSVD. The full format may include all fields, the abbreviated format may omit the Valid and RSVD fields, and the Essential Format may include the MemOpcode, Address, and Tag fields. Omitted fields can be reconstructed by the receiving processing unit based on configuration parameters, default values, or context derived from the received fields. Similar bandwidth optimizations can also be applied to CXL.cache and CXL.io PDUs by identifying and reconstructing optional fields.

[0027] Various implementations include the following methods: receiving a protocol data unit (PDU) of a carrier protocol (carrier protocol PDU) that encapsulates some of the fields of a Compute Express Link (CXL) PDU through a physical layer based on an IEEE 802.3 physical media attachment (PMA). This subset excludes at least one field as defined in the CXL specification. Extracting a subset of fields from the carrier protocol PDU; reconstructing at least one excluded field; generating a CXL request consisting of the extracted subset of fields and at least one reconstructed field. This method enables bandwidth-efficient transmission of CXL traffic over the carrier protocol fabric by transmitting CXL PDU fields that cannot be derived or reconstructed by the receiver. The sender may analyze the CXL PDU and identify fields that contain information that cannot be derived from other fields or system configuration, and include those fields in the carrier protocol PDU. The receiver can extract the included fields, reconstruct the excluded fields, and assemble a complete CXL request compliant with the CXL specification. Bandwidth savings can be proportional to the number and size of excluded fields and may vary depending on the CXL subprotocol, message type, and selected field inclusion level. For CXL.mem M2S requests, depending on the selected format level, the savings can range from omitting a few bits (Valid and RSVD) to omitting several fields (MetaField, SnpType, TC, Valid, RSVD).

[0028] In some implementations of this method, at least one excluded field is reconstructed based on a context derived from at least one configuration, default value, or subset of fields. Depending on the type of excluded field and the available information, different reconstruction methods may be used. Configuration-based reconstruction can utilize values ​​stored in the processing unit's registers or memory, such as system-wide settings for traffic classes or metadata policies. Default value reconstruction may utilize values ​​defined in the CXL specification or system conventions. For example, reserved fields may be zero, and valid indications may be valid fields if the carrier protocol PDU is properly formed. Context-based reconstruction can derive field values ​​from other extracted fields, such as inferring the SnpType value from the MemOpcode when a particular operation type suggests a specific snooping behavior. The reconstruction method for each excluded field can be independently configured, allowing the system to adapt to different deployment scenarios and revisions to the CXL specification.

[0029] In some implementations, this method further receives a CXL response with at least one field of the CXL response, as defined in the CXL specification, excluded, and sends a carrier protocol PDU encapsulating a subset of the CXL response through a physical layer based on IEEE 802.3 PMA. Bandwidth optimization can be applied symmetrically to both the request and response paths. CXL responses include S2M No Data Response (NDR) messages, S2M Data Response (DRS) messages in CXL.mem, D2H responses and D2H data messages in CXL.cache, and may include fields that can be omitted during transmission. The processing unit parses the CXL response, identifies fields that the remote receiver can reconstruct, excludes those fields, and encapsulates the remaining fields within the carrier protocol PDU for transmission. The level of field reduction on the response path may be the same as or different from the level used on the request path, depending on the specific response message type and the reconstructible fields.

[0030] In some implementations, this technique further includes a configuration in which a remote processing unit receives the carrier protocol PDU, extracts a subset of the CXL response, and reconstructs at least one excluded CXL response field. The remote processing unit can receive the carrier protocol PDU transmitted from the local processing unit, extract an encapsulated subset of the CXL response, and reconstruct at least one excluded field. The remote processing unit can reconstruct the excluded response field using its own configuration parameters, default values, or context. The remote processing unit may be implemented within an RPU, SmartNIC, DPU, or other appropriate entity coupled to the physical layer based on IEEE 802.3 PMA. The configuration parameters used by the remote processing unit for reconstruction may be synchronized with the parameters used by the local processing unit for field exclusion. This is done through out-of-band configuration protocols or negotiation during connection establishment.

[0031] In some implementations of this method, a CXL PDU consists of a sublossal (M2S) request from a CXL.mem master, with a subset of fields consisting of MemOpcode, MetaField, SnpType, Address, Tag, and TC fields, and at least one excluded field consisting of the Valid and Reserved (RSVD) fields. In the simplified format of the CXL.mem M2S request, the Valid and Reserved fields can be omitted while retaining the fields that have meaning in the operation (MemOpcode for operation type, MetaField and SnpType for consistency behavior, Address for target location, Tag for transaction tracking, and TC for traffic classification). The Valid field can be reconstructed based on the existence of a properly formed carrier protocol PDU, as it means that the encapsulated CXL information is valid upon receipt of a properly framed and validated carrier protocol PDU. Reserved fields can be reconstructed with a default value of zero, as crisppread fields are defined in the CXL specification to be cleared by the sender and ignored by the receiver.

[0032] In some implementations of this approach, a CXL PDU consists of a CXL.mem M2S request, with a subset of fields consisting of MemOpcode, Address, and Tag, and at least one excluded field consisting of Valid, MetaField, SnpType, TC, and Reserved (RSVD). As an example, it may include the minimum fields for a basic CXL.mem operation: MemOpcode specifying the memory operation, Address identifying the target physical address, and Tag enabling transaction tracking and response correlation. Excluded fields (Valid, MetaField, SnpType, TC, RSVD) can be reconstructed using default values ​​or configuration parameters. MetaField can be reconstructed as No-Operation if metadata operations are not used in the system configuration, SnpType can be reconstructed based on MemOpcode (defaulting to No-Ops or a configured snoop type, for example), and TC can be reconstructed using configured traffic class priorities. The inherent format can result in maximum bandwidth savings and is suitable for deployments where carrier protocol bandwidth is constrained or where additional fields contain predictable values.

[0033] In some implementations of this technique, a CXL PDU consists of a CXL.cache host-to-device (H2D) request, with a subset of fields consisting of an opcode field, an address field, and a UQID field, and at least one excluded field that is either a valid field or a reserved (RSVD) field. A CXL.cache H2D request may include the Opcode, Address, UQID, Valid, and RSVD fields in 68B flip format. The opcode field may include snoop opcodes such as SnpData, SnpInv, and SnpCur. The address field contains the cache line address that snoop is targeting. The UQID identifies the host entry that initiated the request. The Valid and RSVD fields can be omitted and reconstructed from the carrier protocol PDU. In 256B flip and PBR flip formats, additional fields such as CacheID, SPID, and DPID are also subject to field reduction and reconstruction, and the set of excluded fields may differ depending on the negotiated flip mode.

[0034] In some implementations of this technique, a CXL PDU consists of a CXL.cache device-to-host request (D2H), with a subset of fields consisting of an opcode field, a command queue identifier (CQID) field, and an address field, and at least one excluded field consisting of either a Valid field, a NonTemporal (NT) field, or a Reserved (RSVD) field. A D2H request for CXL.cache may include the Opcode, CQID, NT, Address, Valid, and RSVD fields. The D2H request sends a device invocation request to the host with an opcode such as RdShared, RdOwn, RdCurr, RdOwnNoData, or CLFlush. The CQID identifies a device tracker entry for response routing. The NT field provides a cache hint indicating that the requested data will not be reused in the near future, allowing the host to either not cache the data in a preferred cache location or treat it as non-cacheable, and may also retain a default value in many configurations. The RSVD field in D2H requests is excluded because it has more reserved bits than H2D requests. NT fields are rebuilt using the configured default cache policy, and the RSVD field may be rebuilt as zero.

[0035] In some implementations of this method, a CXL PDU consists of a CXL.io Transaction Layer Packet (TLP), with a subset of fields consisting of the Fmt / Type field, Address field, Requester ID field, and Tag field, and at least one excluded field consisting of either the Traffic Class (TC) field, Attribute (Attr) field, or reserved field. A CXL.io TLP header may include fields such as Fmt / Type, TC, Attr, EP, Length, Requester ID, Tag, and Address. The Fmt / Type field (or the combined Type field in flit mode) determines the transaction type and packet format. The Address and Tag fields are retained for routing and transaction tracking. The Requester ID indicates the outgoing function. The TC field, which indicates the traffic class for quality control, can be reconstructed using a configured default traffic class. The Attr field, which contains attributes such as mitigation ordering and Northnoop hints, can be reconstructed from the configuration. Reserved fields and other padding can be reconstructed using zero values. The bandwidth savings achieved by excluding these fields may be proportional to the size of the TLP header, which depends on whether the TLP utilizes 32-bit or 64-bit addressing, and whether it uses frit mode or non-flit mode.

[0036] In some implementations, this approach further involves selecting between a first format containing a subset of fields and a second format consisting of at least one subset of available fields, based on either available bandwidth, quality parameters, or configuration settings. The processing unit can dynamically select different field inclusion levels based on runtime conditions and system configuration. If the carrier protocol fabric has sufficient bandwidth, a full or reduced format with more fields can be selected to minimize reconstruction overhead and maximize protocol compliance. If bandwidth is limited, an essential format with fewer fields may be selected to maximize the number of CXL transactions that can be forwarded within the available bandwidth. Quality parameters may influence the selection; for example, a full format might be used for high-priority traffic classes, while a reduced format is used for low-priority traffic. The selection is made per connection, per transaction, or per time interval, negotiated between the sender and receiver at connection establishment, or renegotiated during operation in response to changing network conditions.

[0037] In some implementations of this method, an active cable consists of first and second pluggable modules coupled by a physical medium, and the active cable incorporates hardware circuitry configured to perform the method. In some implementations of this method, a device acting as a switch is provided, and this device consists of switching circuitry configured to perform the method.

[0038] In various implementations, a device configured to receive protocol data units (PDUs) of a carrier protocol (carrier protocol PDU) at the physical layer based on IEEE 802.3 physical media attachment (PMA), and encapsulating a subset of the fields of the Compute Express Link (CXL) PDU, excluding at least one field as defined in the CXL specification; the processing unit is configured to extract the subset of fields from the carrier protocol PDU, reconstruct at least one excluded field, and generate a CXL request consisting of the extracted subset of fields and the reconstructed field. This device can be implemented as an RPU, SmartNIC, DPU, network interface, switch, active cable, or any other suitable device that receives the carrier protocol PDU through the physical layer based on IEEE 802.3 PMA and processes the encapsulated CXL traffic. The processing unit may include hardware logic, a programmable processor, or a combination thereof, configured to perform the extraction, reconstruction, and CXL request generation operations. The device supports multiple field inclusion levels and can be configured to select different levels depending on system requirements. The device may also include CXL devices and CXL ports connected to processing units to deliver generated CXL requests to the CXL fabric.

[0039] In some implementations of the device, the processing unit is configured to reconstruct at least one excluded field based on either its configuration, a default value, or a context derived from a subset of the field. The processing unit may maintain configuration registers or memory structures that store reconstruction parameters for each field type and each CXL subprotocol. Configuration parameters may be programmed during system initialization, updated during operation via a management interface, and negotiated with remote entities when a connection is established. Default values ​​can be defined by the CXL specification version targeted by the system, and fields with default behavior defined in the specification can be automatically reconstructed without explicit configuration.

[0040] In some implementations of the device, a processing unit is further configured to receive the CXL response, exclude at least one field of the CXL response as defined in the CXL specification, and transmit a carrier protocol PDU encapsulating a subset of the CXL response through a physical layer based on IEEE 802.3 PMA. This device can support bidirectional bandwidth optimization by applying field reduction in both the request and response directions. The processing unit analyzes the transmitted CXL response and identifies fields that can be excluded during transmission, while maintaining the ability of the remote entity to reconstruct the complete response. The exclusion of response fields may be coordinated with the remote entity to facilitate proper reconstruction at the receiving end.

[0041] In some implementations of the device, the processing unit is configured to further select between the first format (first subset of fields) and the second field (subset) based on either available bandwidth, quality parameters, or configuration settings. The device monitors bandwidth utilization on the carrier protocol fabric and dynamically adjusts the field inclusion level to optimize performance. When bandwidth utilization is low, the device may select a format with more fields to minimize reconfiguration overhead. When bandwidth utilization is high or approaching the congestion threshold, the device may select a format with fewer fields to reduce per-transaction bandwidth consumption and maintain throughput. Quality of service parameters, including priority levels, latency targets, and throughput guarantees, influence the selection of field inclusion formats for different traffic classes and transaction types.

[0042] Figure 1A shows an example processing pipeline for extracting passenger protocol messages from carrier protocol communications received at the physical layer based on an IEEE 802.3 physical medium attachment (PMA). This pipeline processes various carrier protocols (Ethernet, ESUN, SUE, UALink, NVLink, etc.) that utilize an IEEE 802.3-based PMA and extracts encapsulated passenger protocol messages such as CXL.mem, CXL.cache, and CXL.io messages. The figure illustrates a transmission received by an IEEE 802.3-based PMA, which forwards the carrier protocol data stream to the Physical Coding Sublayer (PCS). The PCS may include protocol-specific processing operations that vary depending on the carrier protocol used. In the Ethernet example, the PCS may perform operations such as 64B / 66B block decoding, block framing, block synchronization, descrambling, lane descrambling, and / or forward error correction (FEC) decoding. In the case of UALink and NVLink, the PCS can perform additional or alternative operations specific to these protocols while maintaining compatibility with the IEEE 802.3 PMA interface.

[0043] The PCS forwards the processed data to the parser. The data may take the form of a partially segmented stream, and although the protocol boundaries are identified, detailed field analysis has not yet been performed. The parser can analyze the received data to identify the protocol structure and extract protocol-specific information. The parser can perform analytical operations such as examining protocol-specific patterns and markers in the data stream to identify the carrier protocol type, the passenger protocol type encapsulated within the carrier protocol, and segmenting the protocol data units (PDUs) of the carrier protocol (carrier PDU) according to the framing rules of a particular carrier protocol (such as Ethernet framing). This includes delimiting frame boundaries (frame synchronization with operations similar to those performed in Ethernet MACs), and identifying areas of interest within the carrier protocol PDU (headers, addresses, identifiers, payload sections, and other fields required in subsequent stages of the processing pipeline). The parser adds metadata fields to the processed data, including metadata that identifies the protocol type and data patterns.

[0044] The parser forwards structured data, including location information such as an offset array to a location of interest within the carrier PDU, to the validation checker. The validation checker performs validation operations on the received data structure. For example, it verifies the conformity of the data stream format and performs validity checks on the carrier PDU (including cyclic redundancy checks (CRC) and other carrier protocol-specific error detection logic). The validation checker forwards the validated data structure to the stream extractor.

[0045] The stream extractor processes validated data structures and extracts relevant fields of the passenger protocol from the carrier PDU, including data indicating the opcode and physical address of the passenger protocol to be used. The stream extractor can identify and extract various passenger protocol fields required for message processing, regardless of the specific type of passenger protocol. The stream extractor forwards its output to an optional Access Control List (ACL) module, which verifies access permissions based on the extracted fields. The ACL sends proven data to a field translator, which performs field-level conversions between carrier and passenger protocol formats. For example, it converts fields from a carrier PDU (e.g., the command field) to fields compliant with the passenger protocol specification (e.g., the CXL opcode field required to construct a CXL request).

[0046] The field translator sends the translated field data to the stream editor, which reconstructs the complete passenger protocol message. The stream editor can perform several operations, including converting and formatting the PDU structure to conform to a valid passenger protocol PDU (such as CXL), and inserting required or optional passenger protocol fields that are not included in the carrier PDU. For example, reservation fields that are required for elements using the passenger protocol but can be omitted from the carrier PDU. For example, the CXL device correctly parses the values ​​of omitted fields based on the RPU configuration and configurable default values, and removes intermediate data and metadata (e.g., CXL requests) that were collected and generated in the processing pipeline but are not part of the quasi-passenger PDU.

[0047] If address translation or protocol translation is not required, the stream editor forwards the Passenger PDU to an element that uses the Passenger Protocol, such as a CXL device in the RPU. If address translation or protocol translation is required, the stream editor forwards the Passenger PDU, which contains a physical address in the second HPA space, such as a CXL.mem M2S Req message, to an optional address translator. The address translator translates physical addresses between different physical address spaces, for example, from the second HPA space to the first HPA space, and forwards the result to an optional protocol translator. The protocol translator can perform protocol-specific translations, such as translating a CXL.mem M2S Req message to a CXL.cache D2H Req message. The translated Passenger PDU is then forwarded to the appropriate protocol device, for example, forwarding a CXL.cache D2H Req message containing a physical address in the first HPA space to an element that uses the Passenger Protocol, such as a CXL host.

[0048] Figure 1B shows an example of a packet structure suitable for Layer 3 (L3) switching operations. L3 switching typically refers to packet forwarding based on network layer information, using IP addresses or similar network layer identifiers to make routing decisions between different network segments or subnets. The illustrated packet structure represents a typical carrier protocol PDU that can be processed by standard network equipment, and contains an encapsulated passenger protocol PDU. The packet structure includes a carrier protocol header, which is shown as an Ethernet header and may contain standard Ethernet fields (destination MAC address, source MAC address, EtherType, and length fields). Following the Ethernet header is an IP header containing network layer routing information (source and destination IP addresses, protocol identifiers, and possibly IP-specific fields necessary for L3 routing decisions). A UDP header follows the IP header and may contain transport layer information such as source and destination port numbers, which can identify a specific service or application. The packet also includes a carrier protocol encapsulation header, which may carry metadata specific to the encapsulation scheme. This header may include a PDU Version (Ver) field indicating the version and format of the protocol PDU structure, allowing the parser to correctly interpret the packet structure. The SourceNodeID field may identify the originating node in a format specific to the overlay network or fabric topology. The DestinationNodeID field can identify the target node to route within the overlay network. The SegmentationID field may be used for infrastructure virtualization, such as providing isolation between different tenants or logical networks in a multi-tenant environment. The PDU Sequence Number (PSN) field may provide ordering information for reliable delivery and reconfiguration of segmented messages.The PassengerProtocol (PassengerProt) field identifies the type of passenger protocol encapsulated within the PDU, such as CXL.io, CXL.cache, or CXL.mem, allowing the parser to apply the appropriate processing rules. The Passenger Protocol PDU section may contain the actual passenger protocol message, such as a CXL.mem PDU, that is transmitted over the network. The Passenger Protocol PDU section may contain fields required by the passenger protocol specification, as well as some fields that enable message reconstruction, which are described further below.

[0049] Furthermore, packets may contain a carrier protocol encapsulation trailer, which may include additional metadata and integrity information. This trailer may include an encapsulated CRC (E-CRC) field that provides error detection for the encapsulated packet portion, potentially offering stronger integrity protection than standard Ethernet FCS. A data poison (poison) field may be used to mark data that has been confirmed to be corrupted, allowing protocols such as CXL and UPI to propagate the error indication across the network. A reported load field may convey congestion and load information from the source device, and may also include congestion information from intermediate components along the path, such as the CXL DevLoad indicator, enabling network-aware load balancing and congestion management. The packet structure may also include optional pad fields that can be used to meet the carrier protocol's minimum frame size requirements, and may ultimately conclude with an Ethernet frame check sequence (FCS) that provides error detection for the entire Ethernet frame according to the Ethernet specification.

[0050] Figure 1C shows an example of a packet structure suitable for Layer 2 (L2) switching operations. L2 switching typically refers to packet forwarding based on data link layer information, using MAC addresses to make forwarding decisions within network segments or broadcast domains. The carrier protocol optimized header may contain condensed addressing and routing information suitable for L2 forwarding decisions. The carrier protocol encapsulation header, passenger protocol PDU, and carrier protocol encapsulation trailer have similar fields to those described in the L3 packet structure in Figure 1B and may perform similar functions adapted to the L2 switching context. The packet structure retains optional Pad fields and Ethernet FCS to ensure compatibility with the frame requirements defined in the Ethernet specification.

[0051] Figures 2A to 2C show three example variations of a passenger protocol PDU that can be encapsulated within the carrier protocol PDU shown in Figure 1B. These variations demonstrate different levels of field inclusion that can be used to maintain the ability to reconstruct a complete passenger protocol message while optimizing bandwidth utilization. The bottom of Figure 2A shows a complete passenger protocol PDU for a CXL.mem message compliant with revision 1.1 of the CXL specification. The complete PDU includes the fields required by CXL.mem: a Valid field (1 bit) indicating whether the message contains valid data; a MemOpcode field (4 bits) specifying the type of memory operation; a MetaField field (2 bits) containing metadata about the transaction; a SnpType field (3 bits) indicating the snoop type for the cache coherency operation; an Address field (46 bits) storing the physical address of the operation; a Tag field (16 bits) providing a unique identifier for transaction tracking; a TC field (2 bits) potentially indicating traffic class or priority information; and a Reserved (RSVD) field (10 bits) reserved for future use or protocol compliance. This complete format is used when full protocol compliance is required or when the carrier protocol has sufficient bandwidth to accommodate the field.

[0052] Figure 2B shows an example of a reduced passenger protocol PDU, where some reconstructible fields are omitted from the carrier PDU to reduce bandwidth requirements. The reduced PDU retains the MemOpcode field (4 bits), MetaField field (2 bits), SnpType field (3 bits), Address field (46 bits), Tag field (16 bits), and TC field (2 bits). The reconstructed fields are not included in the carrier PDU but are required by the CXL device needed to parse the CXL request and can be reconstructed by the RPU's stream editor. In Figure 2B, the fields to be reconstructed are the 1-bit valid field and the 10-bit reserved (RSVD) field. The valid field can be reconstructed based on the presence of a properly formed PDU, and the RSVD field can be reconstructed using a configured default value.

[0053] Figure 2C shows an example of a critical passenger protocol PDU, which may contain the minimum fields necessary for the operation of a basic passenger protocol. A basic PDU consists of a MemOpcode field (4 bits) that specifies the operation to perform, an Address field (46 bits) that identifies the physical address, and a Tag field (16 bits) that enables transaction tracking and response correlation. Fields omitted from the required PDU, such as Valid, MetaField, SnpType, TC, and RSVD fields, can be reconstructed by the stream editor using protocol-specific default values ​​and configuration parameters.

[0054] Some implementations of the following devices relate to a processor architecture that integrates an RPU with a physical layer based on IEEE 802.3 PMA, enabling CXL protocol communication with external entities across carrier protocol fabrics. Modern data center deployments can benefit from isolated and composite architectures where processing and memory resources (e.g., scale-up memory resources for storing KV cache entries) are isolated and interconnected across high-speed fabrics. By integrating the CXL device and RPU within the processor's IC package, the processor can communicate with accelerators, GPUs, memory expanders, storage devices, switches, or other processors that use the CXL protocol encapsulated within a carrier protocol PDU transmitted over ESUN, SUE, UALink, NVLink, or an IEEE 802.3-based physical layer.

[0055] The RPU acts as a translation bridge between the processor's internal CXL domain and the external carrier protocol domain, where carrier protocol PDUs carrying encapsulated CXL information are sent and received through the physical layer based on IEEE 802.3 PMA. Translations include extracting the CXL field from the incoming carrier protocol PDU, converting the field format between the carrier and the CXL representation, reconstructing the complete CXL PDU, and performing the reverse operation of the transmitted CXL traffic. The RPU also performs address translation between different physical address spaces and tag translation between different tag spaces, enabling interoperability across fabric boundaries.

[0056] In various implementations, the following devices are present: an integrated circuit package (IC package) combining a processing core and a memory controller; a memory channel connected to memory accessible via the memory controller; a physical layer based on IEEE 802.3 Physical Medium Attachment (PMA) configured to communicate with external entities; and a resource provisioning unit (RPU) consisting of Compute Express Link (CXL) devices. The RPU is coupled between the processing core and the physical layer based on IEEE 802.3 PMA, and performs translation between CXL-based protocol data units (PDUs) communicated via CXL devices and carrier protocol PDUs that encapsulate data indicating CXL opcodes and physical addresses. Carrier protocol PDUs are transmitted and received through the physical layer based on IEEE 802.3 PMA. The devices may be implemented as a SoC, a multi-chip module (MCM), a chiplet-based design, or other IC form incorporating the processing core, memory controller, and RPU. The processing core may include a general-purpose CPU core, an accelerator core, or a combination thereof, and may be coupled to the memory controller via an on-chip interconnect, a coherent fabric, or a direct interface. Memory accessible through the memory controller may include DRAM, HBM, or other memory technologies, and may be connected via one or more memory channels using DDR, LPDDR, or HBM interfaces. The RPU may be integrated within an IC package, or it may be implemented on a separate die within the same package and coupled to the processing core via an on-chip or die-to-die interface. CXL devices within the RPU include one or more CXL devices, CXL endpoints, or CXL ports that communicate with CXL-based PDUs, communicating according to one or more CXL subprotocols such as CXL.io, CXL.mem, and CXL.cache.Carrier protocol PDUs may encapsulate CXL information in various forms, including the complete CXL PDU, a subset of the CXL PDU fields, or carrier-specific CXL opcodes and physical address encodings.

[0057] In some implementations, the processing core is connected to the memory controller via a coherent interconnect, and the processing core responds to snoop requests using physical addresses in the host physical address (HPA) space. The coherent interconnect may be implemented as a ring-based interconnect, a mesh-based interconnect, a crossbar, or as an on-chip fabric that maintains cache coherence between processing cores, LLC slices, home agents, and other coherent agents within the IC package. The processing core may include a cache hierarchy (such as L1, L2, and L3 caches) and may participate in coherence protocols that process snoop requests to maintain data integrity. When a snoop request is received targeting a physical address in the HPA space, the processing core may respond by providing cache data, invalidating a cache copy, or indicating the state of the cache line, depending on the type of snoop and the current coherence state of the cache line. The HPA space defines the range of physical addresses that processing cores and other agents within the IC package use to access memory resources.

[0058] In some implementations, the device further comprises a Memory Management Unit (MMU) that works in conjunction with the processing cores, and the MMU is configured to translate virtual addresses to physical addresses in the host physical address space. The MMU may be integrated within each processing core or shared among multiple groups of processing cores. The MMU can use page tables, translation lookaside buffers (TLBs), and other address translation structures to map virtual addresses generated by software running on the processing cores to physical addresses in the HPA space. The MMU may support multiple page sizes, multi-layer page table walks, and IOMMU functionality for device-initiated address translation. The combination of the presence of the MMU and coherent interconnection allows the processing cores to run software that utilizes virtual memory, and the underlying CXL transactions may operate on physical addresses in the HPA space.

[0059] In some implementations of the device, the RPU is further configured to translate carrier protocol PDUs received via the IEEE 802.3 PMA-based physical layer into CXL requests communicated through a CXL device, allowing external entities to access memory through the IEEE 802.3 PMA-based physical layer. This configuration includes the RPU, memory controller, and memory channel. In the inbound direction, the RPU receives carrier protocol PDUs from external entities such as remote processors, GPUs, accelerators, and memory fabric switches, extracts and translates the encapsulated CXL information into CXL requests that are transmitted to the processor's internal CXL domain via a CXL device. The CXL requests reach the memory controller via the on-chip interconnect, and the controller accesses memory through the memory channel to process the requests. This inbound path may allow remote entities to read and write from the processor's memory without requiring a direct CXL link, utilizing the carrier protocol fabric and IEEE 802.3 PMA as the transmission medium. Inbound translation involves extracting the CXL.mem M2S request fields, reconstructing any omitted fields, address translation from the physical address space of external entities to the processor's HPA space, and delivering the reconstructed CXL request via the CXL device.

[0060] In some implementations of the device, the RPU is further configured to convert CXL requests from the processing core into carrier protocol PDUs and transmit them via the physical layer based on IEEE 802.3 PMA. This conversion allows the processing core to access resources linked to external entities via the RPU and access the physical layer based on IEEE 802.3 PMA. In the outbound direction, the processing core may generate CXL requests for resources accessible through external entities, such as HBM linked to a remote GPU, storage buffers linked to a remote storage device, MMIO registers of a remote accelerator, or memory pooled across fabrics. The CXL request passes through an on-chip interconnect to a CXL device within the RPU, which converts the PCIL request into a carrier protocol PDU and transmits it to the target external entity via the physical layer based on IEEE 802.3 PMA. Outbound translation involves translating CXL-based PDU fields into carrier protocol representations, physical address translation from the processor's HPA space to the physical address space of external entities, encapsulating the translated fields within carrier protocol headers and trailers, and transmitting the carrier protocol PDU via IEEE 802.3 PMA.

[0061] In some implementations of the device, an RPU is further configured to translate CXL requests from the processing core into carrier protocol PDUs and transmit them via the physical layer based on IEEE 802.3 PMA. This translation allows the processing core to access resources bound to external entities via the RPU and access the physical layer based on IEEE 802.3 PMA. In a bidirectional configuration, the RPU functions as a full-duplex translation bridge, supporting simultaneous inbound and outbound CXL traffic on the carrier protocol fabric. On the one hand, external entities can access the processor's memory through the RPU, and on the other hand, the processing core can access resources bound to external entities through the same RPU. Bidirectional translation maintains separate translation contexts for inbound and outbound traffic to accommodate different address spaces, tag spaces, and protocol requirements, while utilizing a shared pipeline stage for bidirectional common operations such as carrier protocol framing and physical layer processing. Bidirectional operation is particularly beneficial in deployments that maintain a peer-to-peer relationship where processors and accelerators need to access each other's memory and resources.

[0062] In some implementations of the device, a CXL device consists of at least one CXL endpoint or CXL port, and the CXL device operates as either a CXL Type-2 device, a CXL Type-3 device, or a Global Fabric Attached Memory Device (GFD). CXL devices within the RPU may take different forms depending on the type of CXL transaction they support and the processor architecture. A CXL device implements the device-side CXL protocol logic and presents itself as a CXL endpoint visible in the internal CXL fabric within the IC package. When operating as a CXL Type-2 device, it may support the CXL.io, CXL.cache, and CXL.mem subprotocols, enabling both memory access and cache consistency operations. When operating as a CXL Type-3 device, it may support the CXL.io and CXL.mem subprotocols, allowing the device to access memory without causing cache coherency. When operating as a GFD, it may support CXL.mem transactions optimized for fabric-attached memory pooling, potentially simplifying the design by omitting CXL.io processing. The choice of CXL device type may depend on the intended use case, the type of external entity being serviced, and the processor's internal coherence architecture.

[0063] In some implementations, the device further consists of a root port, a fully coherent request node (RN-F), and a fully coherent home node (HN-F), with the root port linked to the RPU. The RN-F allows external entities to access the device's memory, while the HN-F allows processing cores to access resources linked to the external entities. The root port may provide a CXL or PCIe root complex interface connected to both the RN-F and HN-F nodes within the processor's coherent interconnect. The RN-F node acts as a fully coherent request node, issuing requests on behalf of external entities, allowing them to access the processor's memory through a fully cache-consistent coherent interconnect. The HN-F node acts as a fully coherent home node, providing a home agent proxy for resources linked to external entities, allowing processing cores to send consistent read / write requests to those resources through the coherent interconnect. This dual-node architecture may support bidirectional coherent access. The RN-F path allows external entities such as GPUs to read from the processor's DRAM, while the HN-F path allows processing cores to read from the memory of external entities such as HBM or storage buffers. Depending on the implementation, the root port may be connected to a second RPU, or it may share the CXL device path with the RPU.

[0064] In some implementations of the device, the RPU is connected to the processing core via a coherent interconnection with at least one CXL / CCIX gateway (CCG). The CCG may act as a bridge between the CXL protocol domain and the ARM AMBA CHI protocol domain used in the coherent interconnect.

[0065] In some implementations, the RPU is further coupled to the processing core via at least one I / O Coherent Request Node (RN-I) to handle CXL.io traffic. The RN-I node may provide a route for CXL.io or PCIe-based non-coherent traffic that does not participate in the cache coherency protocol.

[0066] In some implementations, the device further comprises a second RPU (a second CXL device and a second physical layer based on IEEE 802.3 PMA), with the second RPU coupled to the root port and configured to handle more consistent CXL.mem traffic, and the second RPU configured to handle coherent CXL.mem traffic via the root port. A dual RPU architecture may provide two different paths that play different roles or are optimized for different access patterns for CXL.mem traffic. The first RPU is coupled to the CXL device and interconnect components and can handle CXL.mem traffic through a path optimized for device-style memory access patterns. The second RPU is connected to the root port with associated coherent nodes such as RN-F and HN-F and can handle CXL.mem traffic through a path that supports bidirectional coherent access between the processor and external entities. The two RPUs operate independently, each possibly having a physical layer based on IEEE 802.3 PMA, allowing the processors to communicate with external entities in parallel or provide redundant paths to the same external entity. The two paths may serve different combinations of CXL subprotocols; the first path through the CXL device may support CXL-to-CHI and CXL.mem and CXL.io via RN-D nodes, while the second path through the root port may support CXL.mem and CXL.cache via RN-F and HN-F nodes.

[0067] In some implementations of the device, the CXL device consists of a Global Fabric-Attached Memory Device (GFD) that supports CXL.mem transactions. The GFD is connected to the processing core via a CXL / CCIX gateway (CCG) and is optimized for handling CXL.mem traffic. The GFD may also operate as a specialized CXL device dedicated to memory access operations, simplifying the design. This simplified architecture is suitable for processors and accelerators (such as xPUs and custom CPU designs) primarily intended to handle external memory requests (such as memory pooling and memory partitioning) through CXL.mem fabric access.

[0068] In some implementations of the device, the RPU is further configured to translate physical addresses between a primary physical address space used by an external entity and a secondary physical address space used by the processing core. The external entity may use a primary physical address space (e.g., primary HPA space) that is different from the secondary physical address space (e.g., secondary HPA space) used by the processing core within the IC package. The RPU can perform address translation as part of the extraction and encapsulation processing pipeline, mapping physical addresses from the receiving carrier protocol PDU to the processor's HPA space and mapping physical addresses of the outgoing CXL request to the external entity's HPA space for the incoming request. Address translation may be implemented using lookup tables, base and offset calculations, page table structures, or other translation mechanisms. The translation allows external entities with different physical address spaces to access the processor's memory through the RPU, and the processing core may be able to access resources in different physical address spaces through different external entities.

[0069] In some implementations of this device, a carrier protocol PDU consists of an encapsulation header selected from at least one field: the PDU version field, source node identifier, destination node identifier, segment identifier, PDU sequence number, or passenger protocol identifier. The encapsulation header can be placed within a carrier protocol PDU, between the carrier protocol header (Ethernet, IP, UDP header, etc.) and the passenger protocol PDU payload. The PDU version field can indicate the version and format of the encapsulation structure, enabling the processing pipeline to correctly interpret the packet. The source node identifier and destination node identifier may hold node addresses within the fabric topology for routing purposes. The segmentation identifier may provide tenant isolation or logical network segmentation. The PDU sequence number may provide ordering information for reliable delivery or for reconstructing fragmented messages across the fabric. The passenger protocol identifier may indicate the type of CXL subprotocol encapsulated within the PDU (e.g., CXL.io, CXL.cache, CXL.mem), enabling the processing pipeline to apply appropriate extraction and transformation rules.

[0070] In some implementations of the device, the carrier protocol PDU is further composed of an encapsulation trailer, whose fields are selected from either an encapsulation CRC (E-CRC) field, a data poisoning (Poison) field, or a ReportedLoad field. The encapsulation trailer can be placed after the passenger protocol PDU payload or before carrier protocol trailer fields such as the Ethernet Frame Check Sequence (FCS). The E-CRC field provides error detection specific to the encapsulated portion of the packet and may provide additional integrity protection beyond the standard Ethernet FCS. The Poison field can propagate data poisoning indications through the carrier protocol fabric, ensuring end-to-end semantics of CXL poison even when CXL traffic is encapsulated within the carrier protocol PDU. The ReportedLoad field may convey load or congestion information from the source device or intermediate network components along the path.

[0071] In some implementations of the device, the ReportedLoad field conveys at least one congestion or load information, which is supplemented by congestion information from intermediate components along the path. The ReportedLoad field can function similarly to the CXL DevLoad indicator, conveying information about the load and congestion state at the source device and, if necessary, also corresponding to intermediate points in the carrier protocol fabric path. Intermediate components such as switches, routers, and fabric managers may supplement their own congestion values ​​as carrier protocol PDUs traverse the fabric. The receiving RPU can leverage the ReportedLoad information for load balancing decisions, quality of service management, congestion avoidance, or adaptive routing. The extension by intermediate components provides a more comprehensive view of fabric congestion than source-only reporting, enabling more effective end-to-end congestion management.

[0072] In some implementations of the device, segmentation identifiers provide isolation between different tenants or logical networks. Segmentation identifiers can enable infrastructure virtualization within the carrier protocol fabric, allowing multiple tenants, virtual machines, and logical networks to share the same physical fabric infrastructure while maintaining isolation of CXL traffic. Different segmentation identifier values ​​correspond to different tenants or logical domains, and RPUs and intermediate switching elements may utilize segmentation identifiers to enforce access control and traffic isolation. Segmentation identifiers function similarly to VLAN identifiers in Ethernet or segment identifiers in overlay networks, but are specifically applied to CXL traffic forwarded on the carrier protocol fabric. This isolation prevents CXL requests from one tenant from being visible to or interfered with by CXL traffic from other tenants.

[0073] In some implementations, the carrier protocol PDU further consists of an Ethernet header, an IP header, and a UDP header suitable for Layer 3 (L3) switching operations. The L3 variant of the carrier protocol PDU may include standard Ethernet, IP, and UDP headers before the encapsulation header, allowing the carrier protocol PDU to be routed without modification to standard L3 network equipment, IP routers, and data center switches. The Ethernet header may carry the MAC address for hop-by-hop forwarding. The IP header may carry the source and destination IP addresses for network layer routing decisions between subnets. The UDP header may hold the port number for service identification or enable load balancing by changing port numbers between flows. The L3 variant is suitable for deployments traversing network segments, subnets, or routing domains within a data center fabric.

[0074] In some implementations of the device, the carrier protocol PDU consists of a carrier protocol optimized header suitable for Layer 2 (L2) switching operations. The L2 variant of the carrier protocol PDU may employ a condensed or optimized header structure to reduce overhead per packet compared to the L3 variant. The optimized header can carry address and routing information suitable for L2 forwarding decisions based on MAC addresses and other data link layer identifiers without the overhead of IP or UDP headers. The capsule header, passenger protocol PDU, and capsule trailer have similar fields to the L3 variant and may have functions adapted to the L2 switching context. The L2 variant is suitable for deployments where L3 routing is not required, such as within a rack or top-of-rack switch domain, where CXL traffic remains within a network segment or broadcast domain.

[0075] In some implementations, the device further has a root port connected to the switch's upstream port (USP), the switch consists of downstream ports (DSPs) connected to the CXL Type-3 device, and the RPU is connected to the switch via a physical layer based on IEEE 802.3 PMA, with both the root port and the RPU accessing the CXL Type-3 device through the switch. A switch-based dual-path topology may allow two different paths to access the same CXL Type-3 memory device. The first path connects the root port of the IC package to the switch's USP, providing a CXL or PCIe native path for the processing core to access the memory within the CXL Type-3 device through the switch's DSP. The other path connects the carrier protocol fabric (physical layer based on IEEE 802.3 PMA), the RPU, and external entities (remote processors, GPUs, accelerators, etc.) through the switch, allowing these external entities to access the same CXL Type-3 memory device. The switch may be a port-based routing (PBR) switch or a fabric switch that supports multiple uplink connections. This dual-path topology enables memory pooling and memory sharing scenarios where hosts access shared memory resources through different connection paths (native CXL or CXL-over-carrier protocol).

[0076] In some implementations of the device, the carrier protocol consists of at least one of the following: Ethernet, Ultra Ethernet Transport (UET), Ethernet for Scaling Up Networking (ESUN), or Scale Up Ethernet (SUE). Furthermore, the RPU is configured to extract CXL PDUs from the carrier protocol PDU and encapsulate the CXL PDUs back into the carrier protocol PDU. Different carrier protocols may use different framing, encoding, and header structures, while still utilizing IEEE 802.3 PMA. The RPU can support any of these carrier protocols and may adjust its extraction and encapsulation behavior depending on the carrier protocol in use. For Ethernet, the RPU may process standard Ethernet frames with MAC layer framing. For UET, the RPU may process frames using UET-specific framing optimized for high-performance computing workloads. For ESUN / SUE, the RPU may process frames using ESUN / SUE-specific framing optimized for scale-up interconnect topologies. Extraction operations include parsing the carrier protocol header, identifying and extracting the encapsulated CXL PDU fields, and converting the carrier-specific field representation to a CXL-compliant format. The encapsulation operation involves converting the CXL PDU field into a carrier-specific representation, generating the carrier protocol header and trailer, and transmitting the resulting carrier protocol PDU through the physical layer.

[0077] In some implementations of the device, the RPU is further configured to translate tags between the first tag space used by the external entity and the second tag space used by the processing core. Tags can function as transaction identifiers that enable correlation and tracking of responses within a CXL transaction. Different organizations can utilize different tag spaces, each with its own scope and allocation policies. The RPU maintains tag translation tables and mapping functions to translate tags from the external entity's tag space to the processor's tag space (inbound transactions) and from the processor's tag space back to the external entity's tag space. Tag translation allows the RPU to manage concurrent transactions from multiple external entities without tag collisions, mapping each entity's tag value to a non-overlapping range within the processor's tag space. In some implementations, the RPU may not terminate the CXL protocol, in which case the tags may pass through unchanged.

[0078] Various implementations employ the following techniques: communicating with external entities via a physical layer based on IEEE 802.3 Physical Media Attachment (PMA); and a Resource Provisioning Unit (RPU) composed of Compute Express Link (CXL) devices converts between CXL-based protocol data units (PDUs) communicated via CXL devices and carrier protocol PDUs that encapsulate data indicating CXL opcodes and physical addresses, with the carrier protocol PDUs being transmitted and received via a physical layer based on IEEE 802.3 PMA. This conversion is continuous during operation, handling both incoming carrier protocol PDUs carrying CXL requests from external entities and outgoing CXL requests from the processing core to external entities. This method may be implemented within the RPU in hardware, firmware, software, or a combination thereof.

[0079] In some implementations of this technique, the translation converts a carrier protocol PDU received via the IEEE 802.3 PMA-based physical layer into a CXL request communicated through a CXL device, allowing an external entity to access memory through the IEEE 802.3 PMA-based physical layer (RPU, memory controller) and memory channel. The inbound translation scheme involves receiving the carrier protocol PDU from the physical layer, parsing the carrier protocol header to identify the encapsulated CXL information, extracting CXL fields such as opcodes and physical addresses, converting the fields from carrier-specific format to CXL-compliant format, reconstructing any omitted CXL fields, and delivering the resulting CXL request to the processor's CXL domain, including the memory channel. The CXL request reaches the memory controller via the on-chip fabric, and the controller accesses memory through the memory channel to process the request. This technique may also include generating a CXL response upon completion of the memory access and converting that response into a carrier protocol PDU to return to the external entity.

[0080] In some implementations of this method, translation translates CXL requests from the processing core into carrier protocol PDUs and transmits them via a physical layer based on IEEE 802.3 PMA, allowing the processing core to access resources linked to external entities via the RPU and physical layer. Outbound translation methods include receiving CXL requests from the processing core via the on-chip fabric and CXL device, translating the CXL PDU fields into carrier-specific representations, translating physical addresses from the processor's HPA space to the physical address space of the external entity, and encapsulating the translated fields within a carrier protocol PDU with appropriate headers and trailers. The carrier protocol PDU is then transmitted via a physical layer based on IEEE 802.3 PMA. Resources linked to external entities include HBM, DRAM, storage buffers, MMIO registers, or other addressable elements. This method may also include receiving a carrier protocol PDU carrying a response from the external entity and translating that response into a CXL response to deliver it to the processing core.

[0081] In some implementations, this technique further involves the RPU translating physical addresses between a primary physical address space used by the external entity and a secondary physical address space used by the processing core. Address translation may be performed as part of inbound and outbound translation operations. For inbound transactions, the RPU translates the physical address from the external entity's address space to the processor's address space, and then delivers the CXL request to the internal CXL domain. For outbound transactions, the RPU translates the physical address from the processor's address space to the external entity's address space, and then encapsulates the CXL request within a carrier protocol PDU. Address translation may utilize lookup tables, base and offset calculations, or other programmable translation mechanisms that can be configured during system initialization or at runtime.

[0082] In some implementations of this method, a carrier protocol PDU consists of an encapsulation header selected from at least one field: the PDU version field, source node identifier, destination node identifier, segment identifier, PDU sequence number, or passenger protocol identifier. The encapsulation header enables routing, ordering, multi-tenancy isolation, and protocol identification across the carrier protocol fabric. During inbound translation, the RPU parses the encapsulation header to identify the destination, identify the passenger protocol type, validate the sequence, and apply tenant isolation policies. During outbound translation, the RPU may generate an encapsulation header with field values ​​appropriate for the target external entity. This may include source and destination node identifiers for fabric routing, segmentation identifiers for tenant isolation, PDU sequence numbers for ordering, and passenger protocol identifiers indicating the CXL subprotocol in transit.

[0083] In some implementations of this technique, the carrier protocol consists of at least one Ethernet, Ultra Ethernet Transport (UET), Ethernet for Scale-Up Networking (ESUN), or Scale-Up Ethernet (SUE), and the conversion extracts a CXL PDU from the carrier protocol PDU and encapsulates the CXL PDU back into the carrier protocol PDU. This technique is applicable to a variety of carrier protocols that utilize IEEE 802.3 PMA, including Ethernet for general data center networks, UET for optimized high-performance computing fabrics, and ESUN / SUE for scale-up interconnect topologies. The extraction and encapsulation operations can be tailored to the specific framing, header, and encoding rules of the carrier protocol being used. This method may support automatic detection of the carrier protocol type based on patterns or markers in the received data stream, or it may be statically configured depending on the system deployment.

[0084] In some implementations of this method, an active cable consists of first and second pluggable modules coupled by a physical medium, and the active cable incorporates hardware circuitry configured to perform the method. In some implementations of this method, a device acting as a switch is provided, and this device consists of switching circuitry configured to perform the method.

[0085] Figure 3 shows an example of a system in which a processor (e.g., MxPU) is composed of processing cores (Core 0 to Core 5) and further composed of an MMU. The cores are connected to the LLC section and coherence engine via coherent interconnects, which are on-chip processor interconnects such as Intel ring / mesh / crossbar or ARM CHI ring / mesh / crossbar. The MxPU includes external interfaces. ISoL ports (e.g., ARM CHI C2C or Intel UPI) are connected to the coherent interconnects via a coherent interconnect interface called Ring-to-ISoL (R2ISoL). PCIe root ports (RPs) are coupled to PCIe lane configurations (x16, DMA, etc.) via R2PCIe, and the RPs of CXLs are coupled via R2CXL, and there are also RPU-based interfaces. The RPU-based interface connects the RPU to a physical layer based on IEEE 802.3 PMA, interacts with CXL devices, and connects to the R2CXL interface and coherent interconnect. CXL devices function as CXL EPs, GFDs, or other device types that communicate according to CXL-based protocols such as CXL.mem or CXL.io. Entities (entities / consumers / hosts / switches) connect to the MxPU via a physical layer based on IEEE 802.3 PMA. CXL devices enable communication with external entities using the CXL protocol encapsulated over a carrier protocol supported by IEEE 802.3 PMA. Meanwhile, the RPU performs the corresponding translation between the carrier domain and the coherent interconnect domain within the MxPU. The MxPU further includes a home agent (HA) and a memory controller (MC) connected to memory (e.g., DRAM) via DDR memory channels.

[0086] Translation and bridging logic enable interoperability between different communication standards while meeting performance and consistency requirements. The IEEE 802.3 PMA layer provides a standardized physical interface usable by various protocols, offering an established foundation for high-speed communication. This PMA layer and its variants may function as the physical transport for one or more protocols, including Ethernet, UALink, NVLink, Ethernet for Scale-Up Networking (ESUN), Scale-Up Ethernet (SUE), and / or other high-performance interconnect technologies.

[0087] Figure 4A shows an example of a system where a processor, such as an MxPU, consists of a processor including a PHY that transmits and receives data according to a carrier protocol. The processor consists of a processing core with an MMU and LLC, and is connected to various components such as a home agent (HA), memory controller (MC), and CBox (LLC coherence engine) via a coherent interconnect, such as a ring-based interconnect. The processor also includes ISoL ports such as ARM CHI C2C and Intel UPI, which are connected to the coherent interconnect via a Ring-to-ISoL (R2ISoL) interface. The memory controller is connected to DRAM via a DDR memory channel. The PHY, based on IEEE 802.3 PMA, enables communication with entities using a carrier protocol that encapsulates CXL messages. IEEE 802.3 PMA provides a physical layer interface for transmitting and receiving frames carrying encapsulated CXL protocol data. The processor further includes a CXL EP associated with the RPU and PHY, with the RPU connected to the coherent interconnect via a Ring-to-CXL (R2CXL) interface. Alternatively, the RPU can be connected to the coherent interconnect almost directly. The CXL EP exposes a Type 3 CXL device or a Type 2 CXL device. The RPU extracts CXL messages from carrier protocol frames received through the PHY, performs physical address translation between the entity's HPA space and the processor's physical address space, encapsulates the CXL response in the carrier protocol, and sends it via the PHY.

[0088] Figure 4B shows an example of a TFD demonstrating CXL.mem communication over a carrier protocol using a PHY based on IEEE 802.3 PMA. Entities such as hosts and switches send frames through the carrier protocol, and the frames encapsulate CXL.mem M2S requests. The PHY based on IEEE 802.3 PMA receives these frames and provides them to the RPU. The encapsulated requests include a CXL.mem read opcode such as MemRd, MemRdData, MemRdTEE, or MemRdDataTEE, a physical address (AS.2.1) from a second physical address space, and optionally a tag (p.2.1) from the entity's tag space. The RPU extracts the CXL.mem request from the carrier protocol frame, translates it to a physical address (AS.1.1) from the first physical address space used in the coherent interconnect, and optionally translates the tag (p.2.1) from the tag space of the coherent interconnect. The diagram illustrates an example where the tag (p.2.1) passes through the RPU unchanged because the RPU does not terminate the protocol. The RPU performs the translation, generates a read request compliant with the Coherent Interconnect protocol, and sends it to the home agent (also called the home node) and / or memory controller. The requested data is retrieved from the LLC or DRAM and returned via the Coherent Interconnect. The CXL EP generates a CXL.mem S2M DRS message and optionally a CXL.mem S2M NDR message containing the data. The RPU optionally returns the response tag from the processor's tag space to the entity's tag space, encapsulates the CXL.mem response within a carrier protocol frame, and sends it to the requested entity via a PHY based on IEEE 802.3 PMA.

[0089] Some implementations of the following devices relate to processor architectures connected via PCIL protocol interfaces and interconnect components or RPUs, enabling CXL communication with external entities via a physical layer based on IEEE 802.3 PMA. The interconnect within the processor is implemented as a mesh interconnect, routing messages through crosspoints and similar routing elements between processing cores, cache controllers, home agents, memory controllers, and interface agents. Interconnect components such as ARM CCG bridge the CXL protocol domain with the CHI protocol domain on the interconnect, allowing CXL devices connected to the RPU to exchange coherent and non-coherent traffic with agents on the interconnect. CXL devices are implemented on at least one RPU or interconnect component and may function as CXL endpoint logic enabling CXL transactions between external entities and the interconnect. The RPU and interconnect components can jointly perform translations between CXL transactions related to data communicated with external entities via a physical layer based on IEEE 802.3 PMA and interconnect transactions communicated via the interconnect. The conversion process involves extracting CXL information from carrier protocol PDUs, reconstructing CXL transactions, performing address and tag conversions, converting CXL transactions into interconnect transactions, and encapsulating CXL responses for transmission to external authorities.

[0090] Various implementations include: a processor consisting of processing cores coupled via an interconnect; interconnect components connected to that interconnect; a Resource Provisioning Unit (RPU) coupled to the interconnect component; a physical layer based on IEEE 802.3 Physical Media Attachment (PMA) configured to work with the RPU and communicate with external entities; a Compute Express Link (CXL) device implemented on at least one of the RPUs or interconnect components; and at least one of the RPUs or interconnect components configured to translate between CXL transactions and interconnect transactions; CXL transactions are associated with data communicated with external entities via a physical layer based on IEEE 802.3 PMA, while interconnect transactions are communicated through the interconnect. The processor may be implemented as an SoC, a multi-chip module, or a chiplet-based design including processor cores, interconnects, and various interface agents. The interconnect may provide a scalable on-chip fabric that routes transactions between agents based on packet identifiers, node addresses, and other routing information. The interconnect component converts interconnect transactions (such as CXL.mem M2S and S2M transactions, and CXL.cache H2D and D2H transactions) and CHI read, write, snoop, and data transactions, and is used to communicate with agents on the interconnect. The CXL device is implemented in at least one RPU or interconnect component, presents CXL endpoint functionality, and may implement CXL protocol logic for one or more CXL subprotocols.The RPU and interconnect components can jointly perform protocol translation from carrier to CXL and from CXL to interconnect, allowing external entities connected via IEEE 802.3 PMA to exchange CXL traffic with the processor's interconnect coupling agent. Multiple interconnect components may be used for port provisioning and processing different CXL subprotocols.

[0091] In the context of coherent interconnects, interconnect components can refer to various types of devices, blocks, or functional entities that participate in, terminate, bridge, gateway, aggregate, or otherwise interface with a coherent or non-coherent fabric. Examples of unrestricted interconnect components include router modules, request nodes, home nodes, dependent nodes, gateways, bridges, and domain bridges. For example, in certain revisions of ARM-based coherent mesh architectures such as ARM Neoverse and CoreLink CMN families, interconnect components include: crosspoint (XP) router blocks; request nodes such as fully coherent request nodes (RN-F), I / O coherent request nodes (RN-I), or I / O coherent request nodes with distributed virtual memory support (RN-D). Home nodes (such as fully coherent home nodes (HN-F) and I / O coherent home nodes (HN-I)); gateways (such as CXL / CCIX gateway (CCG) blocks used in coherent multichip links (CML) and external CXL connections, or CCIX gateways (CXG) which are bridging between CHI and CXS interfaces); and bridges such as AMBA 5 CHI to ACE5-Lite bridges (SBSX), AMBA domain bridges (ADB), CHI domain bridges (CDB), and CXS domain bridges (CXSDB). Different types, classifications, or naming conventions for interconnect components may be defined in the ARM architecture and other revisions of coherent interconnect architectures.

[0092] In some implementations of the device, the interconnect component consists of an ARM CXL / CCIX gateway (CCG). The CCG is translatable between the CXL and CHI protocol domains. The CCG may be connected to the interconnect via one or more ports at a crosspoint.

[0093] In some implementations of the device, the interconnect consists of a mesh interconnect, which is composed of crosspoints (XPs) configured to route interconnect transactions between processing cores, interconnect components, and memory controllers based on packet identifiers. Crosspoints act as routing elements at intersections in the mesh topology, examining fields within packets to identify the appropriate output port and routing path. Packet identifiers include target node identifiers, address-based routing information, or other fields defined by the interconnect protocol for mesh routing. Crosspoints may connect to processing cores, LLC slices, home agents, memory controllers, interconnect components, RN-D nodes, SN-F nodes, and other agents on the mesh.

[0094] In some implementations, the device is further connected to an interconnect with an I / O coherent request node that supports Distributed Virtual Memory (RN-D), and the RN-D is configured to handle CXL.io non-coherent communication between the CXL device and the processing core. The RN-D can handle reading and writing CXL.io configurations, memory-mapped I / O (MMIO) access, and other non-coherent transactions. The RN-D may support Distributed Virtual Memory (DVM) operations, which may enable synchronization of virtual memory management operations across the interconnect.

[0095] In some implementations, the device is further configured with sub-ronand (SN-F) nodes and a memory controller, with the memory controller connected to DRAM via DDR PHY and memory channels. The SN-F node may function as a dependent agent on the interface between the interconnect protocol domain and the memory controller. The SN-F node may include snoop filter functionality to track the state and location of cache lines across the interconnect. The memory controller can access DRAM through DDR PHY interfaces and memory channels that support memory technologies such as DDR4, DDR5, LPDDR5, and HBM. When a CXL.mem request from an external entity is translated through the RPU, the request is routed through the interconnect to the home node, which can then access the SN-F node and the memory controller to read from and write to DRAM.

[0096] In some implementations of the device, the interconnect component consists of a CXL Streaming (CXS) interface and a Coherent Multi-Chip Link (CML) gateway that utilizes the CXS interface, or an Accelerator Cache Coherent Interconnect (CCIX) gateway (CXG). The use of the CXS interface enables modular designs that combine different CXL device configurations with different interconnect component implementations. The CXS interface may support flow control, credit management, and virtual channels for different CXL subprotocols. CML gateways and CXGs may utilize the CXS interface as a streaming interface protocol for exchanging CXL transactions between CXS devices and interconnects. CML gateways may extend coherent multi-chip link capabilities across chip boundaries, while CXGs may provide CCIX-based gateway functionality, bridging between CXL and CHI protocol domains with CCIX compatibility. Optionally, a 32-bit cyclic redundancy check (CRC-32) may be applied to CXS interface-compliant transactions to detect bit errors. This is useful when the CXS interface crosses die boundaries within multi-chip modules or chiplet-based designs, and when signal consistency requirements differ from those of the on-die interconnect.

[0097] In some implementations of the device, the interconnect component is configured to assign a CHI ReadShared request to the home node in response to a CXL.mem request, and the home node is coupled with the memory controller. When the RPU receives a CXL.mem read request from an external entity through the physical layer based on IEEE 802.3 PMA and converts it into a CXL transaction, the interconnect component converts the CXL.mem request into a CHI-assigned ReadShared request and directs it to the home node responsible for the target address.

[0098] In some implementations of the device, the home node is configured to send a ReadNoSnp request to the memory controller, and the memory controller is configured to return data to the interconnect component using a CompData response. This composite response optimization can potentially reduce transaction latency by allowing the memory controller to send the response data directly to the interconnect component as a requester, without the need to route the data through the home node.

[0099] In some implementations, the CXL device consists of a Global Fabric-Attached Memory Device (GFD) that supports CXL.mem transactions, and the interconnect components are optimized for handling CXL.mem traffic. The GFD may also operate as a special CXL device that supports CXL.mem transactions without supporting the CXL.io or CXL.cache subprotocols. By limiting the supported subprotocol to CXL.mem, the GFD and associated interconnect components can be optimized specifically for memory access operations, simplifying the design by eliminating separate CXL.io processing paths managed by RN-D and RN-I nodes. This simplified architecture may be suitable for processors and accelerators designed to service external memory requests through fabric-attached memory pooling, where CXL.io configuration and enumeration may be handled by alternative mechanisms or not utilized at all.

[0100] In some implementations of the device, data communicated with external entities via the physical layer based on IEEE 802.3 PMA consists of protocol data units (PDUs) of the carrier protocol encapsulating CXL PDUs, and the RPU is configured to extract CXL PDUs from the carrier protocol PDUs. Carrier protocols include Ethernet, Ultra Ethernet Transport (UET), Ethernet for Scaling Up Networking (ESUN), Scale Up Ethernet (SUE), UALink, NVLink, or other protocols that utilize IEEE 802.3 PMA for data transmission. Carrier protocol PDUs can encapsulate CXL information within the carrier protocol header or trailer, covering fields such as source and destination node identifiers, segmentation identifiers, PDU sequence numbers, and passenger protocol identifiers. The RPU can extract encapsulated CXL PDUs by parsing the carrier protocol header, identifying the passenger protocol type, extracting CXL fields from the carrier protocol payload, converting field formats between the carrier and CXL representations, and reconstructing a complete CXL PDU compliant with the CXL specification. The extracted and reconstructed CXL transactions are propagated to the interconnect components and can be converted into interconnect transactions.

[0101] In some device implementations, the external entity consists of at least one of the following: a GPU, accelerator, or switch, and is configured to access memory connected to the processor's memory controller via the RPU and interconnect components. External entities such as GPUs, accelerators (including AI / ML accelerators, FPGAs, and data processing units), and switches may communicate with the processor via a carrier protocol fabric through a physical layer based on IEEE 802.3 PMA. The RPU and interconnect components translate carrier protocol PDUs from these external entities into interconnect transactions that reach the home node and memory controller, accessing DRAM to process requests. This path allows the external entity to read from and write to the processor's memory, such as shared memory access in heterogeneous computing environments, memory pooling across fabrics, and remote direct memory access (RDMA) operations.

[0102] In various implementations, the following methods are used: receiving data from external sources through a physical layer based on IEEE 802.3 Physical Media Attachment (PMA); converting CXL transactions to interconnect transactions communicated over the interconnect using a Compute Express Link (CXL) device implemented in one of the resource provisioning units (RPUs) or an interconnect component connected to the processor's interconnect; and the processor consists of processing cores coupled via the interconnect, with CXL transactions associated with data received from external sources through a physical layer based on IEEE 802.3 PMA. Receiving data involves receiving carrier protocol PDUs from external entities through a physical layer based on IEEE 802.3 PMA. Conversion involves extracting CXL information from the received data, reconstructing CXL transactions compliant with the CXL specification, and converting the CXL transactions to interconnect transactions for routing over the interconnect. A CXL device is implemented in at least one of the RPUs or interconnect components and may provide endpoint functionality for the CXL protocol used during conversion.

[0103] In some implementations of this method, the data consists of carrier protocol protocol data units (PDUs) that encapsulate the CXL PDUs, and the transformation involves extracting the CXL PDUs from the carrier protocol PDUs and reconstructing the CXL transactions from the extracted CXL PDUs. The carrier protocol PDUs include a payload containing carrier protocol-specific headers and trailers (Ethernet, UET, ESUN, SUE, etc.), a header encapsulated with routing and identification metadata, and CXL PDU fields. The extraction involves parsing the carrier protocol structure, identifying the CXL subprotocol type from the passenger protocol identifier, extracting CXL fields such as opcodes, addresses, and transaction identifiers, and transforming field formats where the carrier protocol uses a different encoding than the CXL specification. The reconstruction involves assembling the extracted and transformed fields as a complete CXL transaction, inserting fields omitted from the carrier protocol PDU for bandwidth optimization, leveraging configuration parameters or default values ​​for the omitted fields.

[0104] In some implementations of this method, an active cable consists of first and second pluggable modules coupled by a physical medium, and the active cable incorporates hardware circuitry configured to perform the method. In some implementations of this method, a device acting as a switch is provided, and this device consists of switching circuitry configured to perform the method.

[0105] Some of the following device implementations relate to a processor architecture in which the CXL root port connects to the interconnect and exchanges CXL traffic with external entities via a physical layer based on the RPU and IEEE 802.3 PMA. The root port provides root complex functionality and may connect to the interconnect via different nodes depending on the type of CXL traffic it corresponds to. In one configuration, the root port may be coupled with a fully coherent request node (RN-F) and a fully coherent home node (HN-F), enabling bidirectional coherent access where external entities access the processor's memory and the processor's core accesses resources bound to the external entity. In another configuration, the root port is coupled to a CXL / CCIX gateway (CCG) and an RN-D node, providing coherent CXL.mem processing through the CCG and non-coherent CXL.io processing through the RN-D node.

[0106] In various implementations, the following devices exist: a processor consisting of processing cores coupled via an interconnect; a Compute Express Link (CXL) root port connected to the interconnect; a Resource Provisioning Unit (RPU) coupled to the CXL root port; and a physical layer based on IEEE 802.3 Physical Media Attachment (PMA) configured to work with the RPU and communicate with external entities. Furthermore, at least one RPU or CXL root port is configured to translate between CXL transactions and interconnect transactions; CXL transactions are associated with data communicated with external entities via the IEEE 802.3 PMA-based physical layer, while interconnect transactions are communicated through the interconnect. The CXL root port may provide route complexity functionality to CXL devices and endpoints connected or accessed via the RPU and IEEE 802.3 PMA. The root port may be connected to the interconnect via one or more intermediate nodes that translate CXL protocol transactions and interconnect protocol transactions and communicate with agents on the interconnect. The specific node configuration for connecting the root port to the interconnect may vary depending on the type of CXL traffic supported and the consistency requirements of the deployment. The RPU and CXL root port can perform translations between CXL transactions related to data from external entities (received as carrier protocol PDUs via IEEE 802.3 PMA) and interconnect transactions communicated through the interconnect.

[0107] In some implementations of the device, the CXL root port is connected to a fully coherent request node (RN-F) and a fully coherent home node (HN-F) on the interconnect. The RN-F and HN-F may be incorporated into a gateway or bridge node structure connected between the root port and the interconnect. The RN-F acts as a fully coherent request node participating in the interconnect coherency protocol, enabling it to issue fully coherent requests such as ReadShared, ReadUnique, and MakeUnique to external entities that receive CXL traffic via the RPU and root port. The HN-F acts as a fully coherent home node managing a portion of the address space, enabling the processor core to issue coherent requests to resources accessible through external entities. The combination of the RN-F and HN-F enables bidirectional coherent communication between the processor and external entities via the CXL root port path.

[0108] In some implementations of the device, the RN-F allows external entities to access the processor's memory controller via a physical layer based on IEEE 802.3 PMA, through the RPU, CXL root port, and RN-F. When an external entity, such as a GPU or storage device, sends a read / write request encapsulated within a carrier protocol PDU, the RPU extracts the CXL request, converts it into a CXL transaction, and delivers it to the RN-F node on the interconnect via the root port. The RN-F issues a corresponding interconnect request (such as ReadShared or WriteBack) to the home node responsible for the target address, allowing the home node to access the memory controller or DRAM to process the request. The response data returns via the interconnect to the RN-F, root port, RPU, and physical layer, and may be delivered to the external entity. This path allows the external entity to access the processor's DRAM with full cache coherency, and if the target cache line exists in any of the processor's caches, the coherency protocol can handle the corresponding snoops and state transitions.

[0109] In some implementations of the device, the HN-F allows the processing core to access resources connected to external entities via the interconnect, CXL root port, RPU, and a physical layer based on IEEE 802.3 PMA. The HN-F may act as a home node proxy for an address range mapped to resources connected to external entities (e.g., HBM linked to a GPU, storage buffers linked to a storage device, memory mapping registers of a remote accelerator). When the processing core performs a coherent read or write to an address within this range, the request is routed to the HN-F through the interconnect, and the HN-F may translate it into a CXL transaction that reaches the RPU via the root port. The RPU may encapsulate the CXL transaction within a carrier protocol PDU and send it to the external entity through the physical layer to provide a service. The response from the external entity may return to the processing core via the reverse path. This outbound path allows the processor core to consistently access external resources without being aware that it can access resources through the carrier protocol fabric.

[0110] In some implementations, the CXL root port is connected to the interconnect using a CXL / CCIX gateway (CCG) and an I / O coherent request node (RN-D). In this configuration, the CXL root port may connect to the interconnect via the CCG to handle coherent traffic and via the RN-D node to handle non-coherent traffic. The CCG may translate CXL.mem and CXL.cache transactions arriving via the root port for communication with agents on the interconnect. The RN-D node may handle CXL.io or PCIe traffic that does not require full cache coherence, but can participate in DVM operations. This configuration offers a different functional balance than the RN-F / HN-F configuration and may be advantageous for workloads that primarily use CXL.mem for memory access and combine it with CXL.io for device configuration and management.

[0111] In some implementations of the device, the CCG is optimized for handling CXL.mem traffic. This configuration of the CCG is specifically optimized for CXL.mem transactions, potentially simplifying translation logic by focusing on memory reads, writes, and associated memory operations without the overhead of supporting consistent CXL.cache operations over the root port path. This optimization can reduce the logical space, power consumption, and latency of CXL.mem traffic. If CXL.cache operations over the root port path are not expected (e.g., cache consistency is managed or not utilized over another path), the CCG further simplifies the design by omitting or disabling the translation logic for CXL.cache.

[0112] In some implementations of the device, the RN-D is configured to handle CXL.io or PCIe traffic communicated through the CXL root port. The RN-D receives CXL.io or PCIe transactions from the root port and sends them to the interconnect, routing them to the appropriate agent. CXL.io traffic includes configuration read / write for device enumeration and management, MMIO access for device control, and other non-coherent transactions defined in the CXL.io (PCIe-based) protocol. The RN-D may support DVM operations, which enable synchronization of virtual memory management across the interconnect. By separating CXL.mem traffic (handled by the CCG) and CXL.io traffic (handled by the RN-D) on the same root port, the root port can potentially support the full range of CXL subprotocols while leveraging nodes specialized for each traffic type.

[0113] In some implementations, the device further has a sub-name (SN-F) and a memory controller, the memory controller is connected to DRAM via DDR PHY and memory channels. The memory subsystem of the root port architecture is similar to the CXL device architecture, with the SN-F node acting as a dependent agent that interfaces between the interconnect protocol domain and the memory controller. CXL traffic arriving via the root port path is ultimately processed by the memory controller, which accesses DRAM via DDR PHY and memory channels, after going through interconnect and home node processing.

[0114] In some implementations of the device, data communicated with external entities via the physical layer based on IEEE 802.3 PMA consists of protocol data units (PDUs) of a carrier protocol that encapsulate CXL PDUs, and the RPU is configured to extract the CXL PDUs from the carrier protocol PDUs. The carrier protocol PDU may include Ethernet, UET, ESUN, SUE, or other carrier protocol frames that carry the encapsulated CXL information within the header, payload, and trailer. The RPU parses the carrier protocol structure, extracts the CXL fields, converts between the carrier and CXL field formats, and sends the reconstructed CXL transaction to the root port to communicate with the interconnect.

[0115] In some device implementations, the external entity consists of at least one of the following: a GPU, an accelerator, or a switch, and is configured to access memory connected to the processor's memory controller via an RPU, CXL root port, and interconnect. External entities communicating through the carrier protocol fabric can access the processor's memory through the root port path, and the RPU and CXL root port translate carrier protocol PDUs into interconnect transactions that route to the home node and memory controller. The type of external entity can influence the traffic patterns and CXL subprotocols used. A GPU may generate high-bandwidth CXL.mem read / write requests for shared memory access, an accelerator may combine CXL.mem access with CXL.io for device control, and a switch may aggregate and route CXL traffic from downstream entities. The root port path may benefit certain entity types that benefit from route complexity enumeration and management capabilities.

[0116] In some implementations of the device, the interconnect is configured as a mesh interconnect, which is composed of crosspoints (XPs) configured to route interconnect transactions between processing cores, CXL root ports, and memory controllers based on packet identifiers. Crosspoints in a root port architecture may route interconnect transactions between processing cores, nodes to which the root ports connect with the interconnect (RN-F, HN-F, CCG, RN-D nodes, etc.), home agents, SN-F nodes, memory controllers, and other agents. Routing based on packet identifiers may allow crosspoints to send transactions directly from source to destination on the mesh topology without centralized routing control. Crosspoints may support virtual channels, quality control levels, and flow control mechanisms defined by the interconnect protocol.

[0117] In various implementations, the following methods are used: receiving data from external sources through a physical layer based on IEEE 802.3 Physical Media Attachment (PMA); translating between CXL transactions and interconnect transactions by connecting either a Resource Provisioning Unit (RPU) or a Compute Express Link (CXL) root port to the processor's interconnect; the processor consists of processing cores coupled via the interconnect, with CXL transactions associated with data received from external sources through a physical layer based on IEEE 802.3 PMA, and interconnect transactions communicated through the interconnect. Receiving data may include receiving carrier protocol PDUs from external sources such as GPUs, accelerators, and switches. Translation involves extracting CXL information from the received data, reconstructing the CXL transactions, and translating the CXL transactions into interconnect transactions and routing them to agents on the interconnect via the root port and its coupling nodes.

[0118] In some implementations of this method, the CXL root port is connected to a fully coherent request node (RN-F) and a fully coherent home node (HN-F) on the interconnect. The RN-F allows external entities to access memory connected to the processor's memory controller, while the HN-F allows processing cores to access resources bound to the external entities. The bidirectional coherent access scheme may also allow external entities to read from and write to the processor's memory via the RN-F path, or for processor cores to read from and write to resources bound to external entities via the HN-F path. The inbound path through the RN-F means that the RPU extracts and transforms the incoming data into a CXL transaction, which the root port delivers to the RN-F and issues as an interconnect request. The outbound path through the HN-F may involve processing cores issuing consistent requests, which the HN-F transforms into CXL transactions for transmission to external entities via the root port and RPU. Both paths can operate simultaneously, enabling fully duplex coherent communication between the processor and external entities.

[0119] In some implementations of this method, an active cable consists of first and second pluggable modules coupled by a physical medium, and the active cable incorporates hardware circuitry configured to perform the method. In some implementations of this method, a device acting as a switch is provided, and this device consists of switching circuitry configured to perform the method.

[0120] Figure 5A shows an example of a system consisting of processors with interfaces that can utilize the physical layer based on IEEE 802.3 PMA. The first RPU includes CXL devices connected to both CCG nodes that handle coherent CXL.mem and / or CXL.cache transactions and RN-D nodes that handle non-coherent CXL.io transactions. The CXL devices may be implemented on at least one of the RPUs or CCGs. The system may connect the first RPU to the CCG via a CXS interface, providing a path for consistent communication. The second RPU includes root ports connected to both a fully coherent request node (RN-F) and a fully coherent home node (HN-F), which may be incorporated into a gateway or bridge node structure, enabling external entities such as GPUs and storage devices to read bidirectionally via the RN-F node from the processor's DRAM, and vice versa, from the processor cores reading from the GPU's HBM or storage device buffers via the HN-F node.

[0121] Figure 5B shows an example of a system consisting of an xPU or CPU, which is a custom CPU design that may incorporate acceleration cores and interfaces and utilize a physical layer based on IEEE 802.3 PMA. The Global Fabric Attached Memory Device (GFD) used in the first RPU operates as a dedicated CXL device supporting only CXL.mem transactions, allowing external requests to be processed through a CCG optimized for handling CXL.mem traffic, simplifying the design by eliminating separate CXL.io processing paths typically managed by RN-D and RN-I nodes. Additionally, an optional second RPU, either containing or coupled with a root port, is included and connected to the interconnect via a CCG and a DVM-enabled I / O Coherent Request Node (RN-D). The RN-D can handle CXL.io or PCIe traffic.

[0122] Note that lines in a diagram may represent multiple ports, interfaces, or links. For example, a single line connecting the CCG and XP may represent two ports, such as a port for the Request Agent (RA) proxy and a port for the Home Agent (HA) proxy.

[0123] In various implementations, the system consists of the following: a processor consisting of a coherent interconnect; the processor is connected to memory with a capacity of at least 64GB; the processor is configured to access memory using physical addresses in the host physical address (HPA) space and to run an operating system (OS) that utilizes the virtual address space; a Memory Management Unit (MMU) configured to map addresses in the virtual address space to physical addresses in the HPA space to access memory; and a Resource Provisioning Unit (RPU) consisting of a Compute Express Link (CXL) device configured to communicate with entities according to a CXL-based protocol. Furthermore, the RPU is connected to the coherent interconnect and configured to perform inter-host physical address translation. This inter-host physical address translation allows entities to access memory via the CXL device. The OS uses the MMU to perform virtual-to-physical address mapping, and the MMU translates OS-level virtual addresses to physical addresses in the HPA space. Processes, applications, and user programs running under the control of the OS can access memory using the MMU with virtual addresses, and the MMU enforces memory protection and isolation between different processes and applications. Device drivers operating within the OS kernel space may utilize the MMU to access memory-mapped device registers and manage DMA buffers. If the processor supports virtualization, the hypervisor can utilize the MMU to manage the memory mapping of virtual machines (VMs), and the hypervisor and guest OS further utilize the MMU to manage the memory mapping of processes within the VM, optionally supporting nested virtualization with multiple address translation levels. In some cases, the MMU may translate addresses from one physical address space, such as the guest physical address space (GPA), to another physical address space, such as the HPA space.Infrastructure code and firmware running on hidden cores may utilize the MMU to access memory regions allocated for infrastructure tasks such as memory telemetry collection and memory pool management operations. Additionally, hardware components within the system, such as the DMA engine, may use the functions of the MMU or IOMMU to perform address translation when moving data between different memory regions.

[0124] Processors, MMUs, and RPUs can be implemented as semiconductor devices that combine processing power and memory pooling capabilities. Processors are multi-core processors based on x86, ARM, RISC-V, or other instruction set architectures and may include various cache hierarchy levels. The HPA space used by a processor is the physical address space that the processor uses to access memory. RPUs can be implemented as dedicated hardware logic, firmware running on a dedicated core, or a combination of both, and may maintain translation tables or use programmable mappings to translate between different HPA spaces used by external entities and the processor's local HPA space.

[0125] Optionally, messages received by the RPU may include additional messages that do not contain HPAs, such as messages compliant with the CXL protocol, and these messages may be processed by the RPU without performing physical address translation between hosts. Furthermore, the RPU may process additional messages that have virtual addresses instead of host physical addresses, and messages containing host physical addresses may coexist with other types of messages that are processed differently by the RPU. This ensures that the description of messages with host physical addresses does not restrict the existence or processing of other types of messages communicated through entities or processors. In addition, the RPU can apply different processing methods depending on the type and requirements of the message, including forwarding without modification, modifying content without address translation, or performing other translations or modifications that differ from the inter-host physical address translation described above.

[0126] In some system implementations, entities utilize a second HPA space, and inter-host physical address translation translates physical addresses within the second HPA space to physical addresses within the HPA space. The second HPA space used by the entity may have a different size, layout, or addressing scheme than the HPA space used by the processor. Inter-host physical address translation involves offset calculation, range remapping, or lookup table operations for address translation between the two HPA spaces. The RPU can support a configurable translation window that defines which parts of the entity's HPA space are mapped to the processor's HPA area, and can also implement protection logic to prevent unauthorized access to memory areas outside the allocated range.

[0127] In some implementations, the system is further configured with a CXL root port, configured to communicate with a CXL memory expander that utilizes the Device Physical Address (DPA) space. At least one of the operating systems, system firmware, or memory expander is configured to map physical addresses in the HPA space to physical addresses in the DPA space, allowing entities to leverage memory and the CXL memory expander. The CXL memory expander may be a CXL Type 3 device that provides additional memory capacity to the system. The memory expander's DPA space represents the device-local physical addresses used internally by the expander. The OS or system firmware maintains a mapping table that associates HPA ranges with the memory expander's DPA ranges, enabling transparent access to the expanded memory. Furthermore, the HPA-to-DPA mapping may be further maintained by the memory expander's internal firmware, software, or hardware.

[0128] In some system implementations, the RPU further comprises a second CXL device configured to communicate with a second entity using a second protocol based on CXL, where the second entity utilizes a third HPA space. Furthermore, the RPU translates physical addresses within the third HPA space to physical addresses, enabling the second entity to leverage the CXL memory expander. The system allows multiple entities to access the CXL memory expander using coordinated address translation. Different entities may utilize separate HDM regions or virtual CXL devices exposed by the RPU, each holding a portion of the memory expander's capacity. Alternatively, the memory expander may expose multiple HDM regions or multiple logical devices (LDs), which are mapped to multiple entities through RPU translation. The RPU can maintain separate translation contexts for each different entity, ensuring proper isolation of memory access from different entities while allowing shared access to the memory region specified during multi-entity sharing configuration. The system may implement a QoS (Quality of Service) mechanism to fairly allocate memory expander bandwidth among multiple entities.

[0129] In some system implementations, the RPU is further configured with a second CXL device set up to communicate with a second entity using a second protocol based on CXL, the second entity utilizing a third HPA space, and further configured to translate physical addresses within the third HPA space to physical addresses within the HPA space. This allows the second entity to utilize memory. When multiple entities access memory (e.g., DRAM), the system may implement a memory partitioning scheme that allocates specific memory regions to different entities. The RPU may enforce access control so that entities can only access the memory regions to which they are allocated. The system can support dynamic memory reallocation between entities based on workload requirements and management policies, and can also implement memory tiering and migration functions to move data between allocated regions of different entities when workload access patterns change or are reconfigured.

[0130] In some implementations of the system, entities consist of hosts connected to the processor via at least one CXL root port or CXL switch, and a second CXL-based protocol is distinct from the CXL-based protocol. Supporting different CXL protocols for different entities enables heterogeneous system configurations where entities with different capabilities can utilize or share memory pools. For example, one entity might use CXL.mem for simple memory expansion, while another might use CXL.cache for cache-consistent shared memory. The RPU can maintain protocol-specific state machines and translation logic for different combinations of supported protocols, enabling interoperability between entities using different CXL protocol subsets.

[0131] In some implementations of the system, the processor consists of a Modified Processing Unit (MxPU), and the memory consists of Dynamic Random Access Memory (DRAM), with the RPU enabling the use of DRAM with a capacity of at least 256GB. The MxPU may be derived from established CPU or GPU designs, with modifications to support CXL device functionality and inter-host address translation. The large DRAM capacity (≥256GB) is achieved through multiple memory channels supporting high-capacity DRAM modules. The MxPU may implement memory compression, deduplication, and other techniques to effectively increase the available memory capacity exposed to entities beyond the physical DRAM capacity.

[0132] In some system implementations, memory consists of dynamic random access memory (DRAM) coupled to the processor via a memory channel, and CXL devices consist of global fabric-attached memory (G-FAM) devices (GFDs). Memory channels may also include channels for parallel transmission to increase memory bandwidth and reduce latency. A memory channel may support one or more DRAM modules such as DIMMs or RDIMMs and may implement various memory technologies such as DDR4, DDR5, LPDDR4, LPDDR5, and future memory standards. A memory channel may include a memory controller integrated within the processor or it may be implemented as a separate component within the system and may support features such as ECC, memory interleaving, and channel bonding to improve performance and reliability.

[0133] In some system implementations, the CXL-based protocol utilizes CXL.mem, and CXL devices expose at least one Host Managed Device Memory (HDM) address region to entities. Operating according to CXL.mem, a CXL device (e.g., a CXL EP) may expose one or more HDM regions that appear to bound entities as memory-mapped regions. HDM regions can be configured with specific address ranges, access permissions, and memory attributes through an HDM decoder. Entities can access these HDM regions using standard memory load / store operations, which are translated into CXL.mem transactions by the entity's CXL root port. The system may support HDM regions with different characteristics, such as those backing up volatile memory regions into memory or persistent memory regions supporting storage class memory.

[0134] In some system implementations, CXL-based protocols utilize CXL.io, where inter-host physical address translation is performed by translating the physical address contained in the entity's CXL.io UIOMRd transaction layer packet (TLP) to a physical address in the HPA space. When operating according to CXL.io, the system may handle various types of TLPs, such as memory read / write TLPs, configuration TLPs, and message TLPs. UIOMRd TLPs have physical addresses within the entity's physical address space and may require translation to the local HPA space. The RPU can intercept these TLPs, extract the physical addresses, perform the corresponding translation, and generate the corresponding transactions in the local HPA space. It may also support other CXL.io transaction types for memory writes, such as UIOMWr, and implement flow control and credit management according to the CXL specification.

[0135] In some system implementations, the processor is composed of cores, at least one of which is a hidden core. Furthermore, the RPU is further configured to utilize hidden cores for internal tasks, which consist of at least one internal firmware processing, CXL Fabric Manager (FM) API processing, memory processing (PIM), near-memory processing, or housekeeping tasks. The RPU may utilize at least one hidden core for internal tasks such as processing internal firmware, CXL Fabric Manager (FM) API processing, memory processing (PIM), memory approximation processing, or housekeeping tasks. By leveraging hidden cores for these specific functions, the processor improves performance and enables efficient operation without overloading the non-hidden cores allocated to running user workloads. Additionally, leveraging hidden cores for RPU tasks allows CPU vendors to differentiate their processors from others while maintaining compatibility with existing or established designs, applications, and software codebases developed for existing CPUs.

[0136] In some system implementations, hidden cores are isolated from user access and visibility, achieving user-infrastructure isolation. Hidden processor cores are isolated from user access and visibility, enabling user-infrastructure isolation. This isolation prevents users from influencing code execution on hidden cores, improving system security and reliability. By separating visible user-controlled cores from hidden vendor-controlled cores, processors can effectively protect critical infrastructure functions from unwanted interference and tampering by malicious user code.

[0137] In some system implementations, the processor is composed of cores, at least one of which is hidden and used for memory telemetry collection. At least one of the processor's hidden cores may be used for memory telemetry collection. Performing memory telemetry on a hidden core allows the system to effectively monitor and manage memory resources, such as memory resources in the memory pool, without burdening user-accessible cores, enabling efficient resource utilization and preventing memory management tasks from interfering with user code execution.

[0138] In some system implementations, the processor is composed of cores, at least one of which is used as a hidden core for securely storing and managing keys for encrypting and decrypting data transmitted according to CXL-based protocols. This leverages user infrastructure isolation through the hidden core. At least one of the processor's hidden cores may be used specifically to protect the storage and management of keys for encrypting and decrypting data transmitted according to CXL-based protocols. By leveraging user infrastructure isolation through the hidden core, sensitive cryptographic keys used to secure data under CXL-based protocols are prevented from being accessed by user code. This isolation enhances the security of data transmitted between the processor and entities, protecting against potential breaches by malicious user code. The hidden core can perform cryptographic operations on the data itself, aiming to improve confidentiality, integrity, and replayability. Alternatively, the hidden core may utilize a hardware-accelerated cryptographic engine to perform at least some of the cryptographic operations on the data, while the hidden core is responsible for secure key management and control of the data processing flow. In this approach, the cryptographic accelerator handles data processing, the hidden core handles control, and there is a separation of the control plane and the data plane. Furthermore, the infrastructure code running on the hidden core may also play a role in enabling support for sensitive computations on the memory exposed and provisioned by the RPU through the CXL device.

[0139] Some implementations further include a hardware-accelerated cryptographic engine, configured so that a hidden core utilizes this engine to perform at least some cryptographic operations on data transmitted according to CXL-based protocols. The system may include one or more hardware-accelerated cryptographic engines that the hidden core can utilize to perform at least some cryptographic processing on data transmitted according to CXL-based protocols. The hidden core is responsible for secure key management and controlling the data processing flow, while the cryptographic engines handle the actual data processing. This approach features a separation of control and data planes, with the hidden core acting as the control plane and the cryptographic engines acting as the data plane. Offloading computationally intensive cryptographic processing to hardware accelerators can potentially improve the security and performance of data transmitted according to CXL-based protocols.

[0140] In some system implementations, hidden cores enable support for sensitive computing on memory exposed to the RPU via CXL devices. Sensitive computing protects data in use by performing calculations in a secure and isolated environment. Hidden cores in the processor may support sensitive computing on memory exposed and provisioned to the RPU via CXL devices. Sensitive computing is a security paradigm aimed at protecting data in use by performing calculations in a secure and isolated environment, such as a Trusted Execution Environment (TEE). In sensitive computing, data is encrypted and kept confidential even while being processed, protecting sensitive information from unauthorized access, alteration, and disclosure. This can be achieved using a combination of hardware-based security features such as encrypted memory areas and secure enclaves, and optional software-based logic to enforce access control and data isolation. By enabling calculations on encrypted data without exposing the plaintext content, sensitive computing provides a higher level of security and privacy compared to traditional computing models that only protect data in a stationary or transit state. Infrastructure code running on a hidden core participates in setting up and managing the secure environment necessary for sensitive computing, including provisioning encrypted memory areas, managing cryptographic keys, and protecting sensitive data from unauthorized access. By leveraging the isolation of user infrastructure by the hidden core, the system builds a reliable environment for executing sensitive computing, enabling the RPU to securely process sensitive data in exposed memory using CXL-based protocols.

[0141] In some system implementations, the processor is composed of cores, with at least one of them being a hidden core. Furthermore, the RPU is configured to leverage the hidden core for error handling and correction tasks within the memory pool, enhancing data integrity and reliability. Error handling and correction tasks performed by the hidden core include detecting and correcting single-bit and multi-bit errors, managing spare memory areas to replace faulty memory locations, and managing error logs for system analysis. The hidden core may implement scrubbing routines (e.g., patrol scrubs) that periodically read and correct memory contents to prevent error accumulation. The system may support various error correction codes and advanced ECC schemes suitable for large memory pools.

[0142] In some system implementations, error handling and / or correction tasks may include predictive failure analysis (PFA) operations, configured to predict and address impending failures of memory components in the memory pool, thereby preventing potential data loss and system downtime. Error handling and correction tasks may include predictive failure analysis operations to anticipate and address impending failures of memory components in the memory pool. By implementing PFA, the system can proactively identify potential failures before they escalate into actual failures, enabling timely interventions that mitigate the risk of data loss and system downtime. PFA not only enhances the reliability and data integrity of the memory system but can also improve the overall system resilience in high-performance computing architectures.

[0143] In some system implementations, memory consists of dynamic random access memory (DRAM), and the processor consists of cores, of which at least one is a hidden core. Furthermore, the RPU utilizes the hidden core to control or manage memory access scheduling within the memory pool that makes up the DRAM, and is configured to improve memory utilization and throughput. Memory access scheduling controlled or managed by the hidden core, such as using a hardware-based memory controller or a memory access scheduler managed by the hidden core, can optimize memory bandwidth utilization by sorting memory requests based on factors such as request priority, memory bank availability, and access patterns. The hidden core may implement and apply scheduling algorithms that take QoS (Quality of Service) requirements into consideration, minimizing memory access contention and maximizing row buffer hit rates. Scheduling can also take into account thermal constraints and power management goals while maintaining fair access for memory pool clients.

[0144] In some system implementations, the processor is composed of cores, of which at least one is a hidden core. Furthermore, the RPU is configured to utilize the hidden core to manage security protocols within the memory pool, including data encryption and access control. Security protocol management by the hidden core includes encryption algorithms for quiescent and in-transit data, management of security keys and certificates, and enforcement of access control policies. The hidden core may support various security standards such as CXL integrity and Data Encryption (IDE) to protect data transmitted over the CXL link. The memory pool may include secure enclaves and trusted execution environments to protect sensitive data and cryptographic operations from unauthorized access.

[0145] In some system implementations, the processor is composed of cores, of which at least one is a hidden core. Furthermore, the RPU is further configured to leverage the hidden core for configuration management tasks within the memory pool, including dynamic allocation and deallocation. In further examples, one or more hidden cores of the processor may be used for advanced infrastructure management tasks based on the processor and memory. These tasks include (i) error handling and correction (rapidly handling memory errors to improve data integrity and reliability), (ii) memory access scheduling (improving the allocation and utilization of memory resources based on current demand and operational priorities), (iii) security management to protect the memory pool by implementing robust encryption and access control, and / or (iv) configuration management to dynamically adjust memory settings to adapt to different workload requirements. One or more of these tasks are used to maintain the overall efficiency, security, and performance of the system, such as in environments requiring fast and highly consistent memory operations, enhancing the system's capabilities and distinguishing it from traditional CPU / GPU-based architectures (where CPU / GPU refers to the CPU and / or GPU).

[0146] In some system implementations, the processor is composed of cores, of which at least one is a hidden core. Furthermore, the RPU is further configured to leverage the hidden core for memory tiering tasks. Memory tiering by the hidden core involves classifying memory regions into different performance tiers based on their underlying technical characteristics. The hidden core can leverage hardware-based telemetry collectors and analyzers to monitor access patterns to different memory regions and dynamically adjust tier allocations to optimize overall system performance. The system may support various memory technologies and speeds at different tiers. For example, Tier 1 high-bandwidth DRAM (e.g., MRDIMM), Tier 2 standard DRAM (e.g., RDIMM), and lower tier persistent memory or storage-class memory (SCM).

[0147] In some system implementations, memory tiering operations include data movement between memory tiers based on the data's hotness level, improving the performance of memory access from entities to hot data. Hidden cores in the processor may enable support for memory tiering, which maps memory regions or subsets exposed to entities to memory resources based on parameters such as the data's hotness (e.g., data usage frequency). In some implementations, hidden cores may use memory telemetry to map hot data to high-performance memory tiers, while cold data is mapped to slower memory such as processor-coupled flash memory. In other implementations, hidden cores may utilize memory mapping based on data-related priorities or service level agreements (SLAs), for example, if the system is configured to prioritize specific workloads, virtual machines, users, or tenants. However, in other implementations, hidden cores may also migrate data between memory tiers, such as migrating hot data from a low-performance memory tier to a high-performance memory tier.

[0148] In some implementations, the system includes a direct memory access (DMA) engine, and a hidden core uses the DMA engine to move data between memory tiers. The processor's hidden core may use the DMA engine for data migration between memory tiers, and by offloading the data migration task from the hidden core to a dedicated engine, data migration becomes faster, allowing the hidden core to perform additional tasks.

[0149] In various examples, hidden cores are isolated from user access and visibility, while visible cores are available to the user. This isolation is achieved using a variety of techniques, including Type 1 hypervisors, Type 2 hypervisors, hardware partitioning, software partitioning, asymmetric multiprocessing (AMP), firmware configuration, CPU microcode updates, custom CPUs, security extensions, or a combination thereof.

[0150] In the first example, a Type 1 hypervisor can be used to create hidden and visible cores. Type 1 hypervisors, such as VMware ESXi and Microsoft Hyper-V, run on the hardware and manage virtual machines (VMs). Hypervisors can assign specific processing cores to VMs using techniques such as CPU affinity and core pinning. For example, certain cores may be designated as hidden and assigned to VMs that are inaccessible or invisible to users. These hidden cores may run specialized applications such as system administration tasks, CXL memory management, and memory pool operations, while visible cores are assigned to user-accessible VMs that run general-purpose operating systems (GPOS). The hypervisor maintains isolation by preventing users from directly accessing the hidden cores.

[0151] In a second example, a Type 2 hypervisor may be used to achieve similar isolation. Type 2 hypervisors, such as VMware Workstation and Oracle VirtualBox, run on the host OS and support guest OSs. The host OS manages the visible cores that the user can access. A Type 2 hypervisor uses hidden cores to create additional VMs, each running a different OS or specialized task. While Type 2 hypervisors have higher overhead than Type 1 hypervisors, they can offer users additional flexibility in managing visible and hidden cores.

[0152] In a third example, hardware partitioning (also known as hardware-assisted virtualization in some systems) can be used to divide processing cores into isolated partitions at the hardware level, with each isolated partition running a different operating system. It is used in various scenarios where isolation between partitions is necessary, especially in systems where high reliability and security are critical. For example, one partition with a hidden core might run an RTOS or embedded OS for critical system functions, while another partition runs a GPOS for user applications. Hardware partitioning enables isolation because the partitions are managed by hardware, preventing users from accessing the hidden core.

[0153] In the fourth example, software partitioning, such as that used in the Jailhouse hypervisor, can be leveraged to create isolated partitions with less overhead compared to full virtualization. This approach may involve assigning specific cores to different partitions, with hidden cores running dedicated tasks or specialized applications. For example, Jailhouse can be configured to run specific cores for an RTOS or bare-metal application, isolating them from user access. The visible cores can then run GPOS (Global Positioning System) applications available for user applications.

[0154] In the fifth example, asymmetric multiprocessing (AMP) can be used to run different operating systems on different cores without a hypervisor. In this configuration, certain cores may run an RTOS or an embedded OS, while other cores may run GPOS. Communication between operating systems can be achieved using shared memory or inter-process communication logic. For example, Linux® may run on a visible core for user applications, while the RTOS runs on a hidden core for real-time tasks. AMP provides a simple way to isolate the hidden core from user access while leveraging the unique strengths of different operating systems.

[0155] In the sixth example, hidden and visible cores can be achieved using firmware configuration. By accessing the Basic Input / Output System (BIOS) or Unified Extensible Firmware Interface (UEFI) settings, specific CPU cores can be disabled and made invisible to the OS. While this method prevents the OS from utilizing the disabled cores, in some cases these cores may still be accessible through other means, such as a hardware debugging interface, and these changes may not be permanent (e.g., a system reboot resets the BIOS / UEFI settings, making the hidden cores visible again). Therefore, depending on specific requirements, additional measures may be needed to completely isolate the hidden cores.

[0156] In the seventh example, updates to the CPU microcode provided by the hardware vendor may be used. These updates include specific instructions that disable or hide cores at the microcode level, preventing detection and use by the OS. This method provides a secure way to manage core visibility because the updates are controlled by the CPU manufacturer.

[0157] The eighth example involves custom CPUs designed by hardware vendors, including technologies and mechanisms that enable core partitioning and core visibility management. For example, Intel's Resource Director Technology (RDT) enables CPU resource partitioning, and ARM's Big.LITTLE architecture enables heterogeneous multiprocessing, allowing different types of cores to be used for different purposes. These vendor-specific examples provide control over core allocation and allow certain cores to be hidden from the user.

[0158] In the ninth example, security extensions such as Intel's Trusted Execution Technology (TXT) or ARM's TrustZone may be used. These technologies create a secure execution environment that isolates specific cores for security-sensitive operations. Hidden cores are only accessible within this secure environment, protected from user interference, and enabling the secure execution of critical tasks.

[0159] Various implementations employ the following methods: accessing memory connected to processors using physical addresses in the host physical address (HPA) space; processors are configured with coherent interconnections; mapping addresses in the virtual address space to physical addresses in the HPA space; addresses in the virtual address space are used by the operating systems (OS) of the devices that make up the processor; and communicating with devices-dependent entities via a Compute Express Link (CXL) device on a resource provisioning unit (RPU) according to a CXL-based protocol; the RPU is connected to a coherent connection. The RPU also performs inter-host physical address translation, which allows entities to access memory through the CXL device.

[0160] In some implementations of this method, an entity is comprised of a second host, which utilizes a second HPA space, and physical address translation between hosts translates physical addresses within the second HPA space to physical addresses within the HPA space.

[0161] In some implementations, this method further involves communicating with a CXL memory expander that utilizes the Device Physical Address (DPA) space via a CXL root port. Then, at least one operating system or system firmware maps physical addresses in the HPA space to physical addresses in the DPA space, and this mapping allows a second host to utilize the memory and CXL memory expander.

[0162] Various implementations include the following devices: a processor consisting of a coherent interconnect; the processor is coupled with memory with a capacity of at least 64GB; the processor is configured to access memory using physical addresses in the first host physical address (HPA) space and run an operating system (OS) that utilizes the virtual address space; a memory management unit (MMU) configured to enable memory access based on mapping addresses in the virtual address space to physical addresses in the first HPA space; a resource provisioning unit (RPU) is coupled to a Compute Express Link (CXL) device with a CXL-based protocol-compliant message exchange and utilizes a second HPA space; and the RPU is further coupled to a coherent interconnect and configured to translate physical addresses in the second HPA space to physical addresses in the first HPA space.

[0163] In various implementations, there are systems designed to function as a multi-head device (MHD), which include: a processor with coherent interconnections; the processor is coupled to dynamic random access memory (DRAM) with a capacity of at least 32GB; the processor is configured to access the DRAM using physical addresses in the host physical address space (HPA) and run an operating system (OS) that utilizes the virtual address space; a memory management unit (MMU) configured to map addresses in the virtual address space to physical addresses in the HPA space to enable access to the DRAM; first and second endpoints of Compute Express Link (CXL) configured to communicate with hosts connected to the system according to a CXL-based protocol; and a resource provisioning unit (RPU) configured to perform inter-host physical address translation, allowing the host to access the DRAM using messages compliant with the CXL-based protocol.

[0164] In the revised CXL specification 3.2, Section 2.5 defines a multi-head device (MHD) as a CXL type 3 device (referred to as a head) that has a CXL port. The CXL specification currently defines two types of MHDs, distinguished by their arrangement on each head: (i) MH-SLD (displays a single logical device (SLD) on the head, with a 1:1 mapping between the head and the LD), and (ii) MH-MLD (can display a multi-logical device (MLD) on any head; a multi-head device has at least one and up to 16 logical devices mapped to each head.

[0165] In some implementations of the system, DRAM is connected to the processor via at least four memory channels. The memory capacity of this DRAM exceeds 128GB, 256GB, 512GB, or 1TB. Furthermore, the DRAM consists of mainstream DRAM modules, exhibiting an average unit price / gigabyte that does not exceed three times the average unit price of the lowest-cost DRAM module technology mass-produced for data center servers.

[0166] Figure 6A shows an example of a system consisting of a memory switch, memory pool, global fabric-attached memory (GFAM) device (GFD), memory expander (ME), or memory expansion device, where a processor, memory (e.g., DRAM), and RPU are coupled to an entity such as a host. The processor may include processing cores and a cache hierarchy that utilize the initial HPA space for accessing system resources. Memory is connected to the processor via memory channels such as DDR4 or DDR5 channels, providing high-bandwidth memory access. The RPU may include or be coupled to a CXL device (e.g., CXL EP), which may be integrated within the same semiconductor device as the processor or implemented as a separate component. The RPU can perform physical address translation between the entity's HPA space and the processor's physical address space. Entities are connected to the memory pool via one or more CXL devices that support the CXL protocol, and can access memory based on address translation by the RPU.

[0167] Figure 6B shows an example of a system consisting of a host, a memory expander, and a connected memory pool, where the memory pool is based on a processor (e.g., MxPU) consisting of an RPU and a CXL device. The memory pool may include memory tiers. For example, the first memory tier (denoted "1") consists of DRAM connected to the MxPU via a memory channel, and the second memory tier (denoted "2") consists of DRAM associated with the memory expander. The MxPU may include a CXL RP for coupling to the memory expander, which allows the memory pool's capacity to be expanded beyond the directly connected DRAM. Multiple hosts are coupled to the memory pool via separate CXL devices (e.g., CXL EP) within the MxPU, with each host utilizing its own HPA space. The RPU within the MxPU can perform different inter-host physical address translations for each different coupled host, allowing simultaneous access to both memory tiers while maintaining isolation between the physical address spaces of different hosts.

[0168] Figure 7A shows an example of a system consisting of a memory pool comprising two or more MxPUs. The memory pool may utilize a chipset-based architecture in which a collection of electronic components such as MxPUs, xPUs, CPUs, and memory buffers work together on the platform to realize the memory pool function. The memory pool consists of a first memory layer ("1") consisting of DRAM connected to the first MxPU via a memory channel, a second memory layer ("2") consisting of DRAM associated with a memory expander, a third memory layer ("3") consisting of DRAM coupled to the second MxPU via a memory channel, and a fourth memory layer ("4") coupled to a memory buffer connected to the second MxPU. The MxPUs are interconnected via ISoLs such as UPI, Infinity Fabric, and CHI C2C, enabling consistent communication between MxPUs. Each MxPU may include its own RPU for inter-host physical address translation and coupling devices with external hosts such as CXL EP, allowing at least some external hosts to access distributed memory resources across memory tiers. Memory buffers provide additional memory capacity and may include buffer control logic that manages data flow between different memory tiers.

[0169] Figure 7B shows an example of a system consisting of a memory pool comprising at least one MxPU and at least one xPU (which may be a CPU). The memory pool may utilize a chipset-based architecture. The memory pool may include a first memory layer ("1") containing DRAM coupled to the MxPU, a second layer ("2") with DRAM associated with a memory expander coupled to the MxPU, a third layer ("3") consisting of DRAM coupled to the xPU / CPU, and a fourth memory layer ("4") coupled to a memory buffer. The MxPU may include a CXL device (such as a CXL EP) which serves as the primary interface for external hosts to access the memory pool via a CXL-based protocol. The xPU / CPU, on the other hand, may provide additional processing power or memory resources. The RPU within the MxPU coordinates address translation, allowing external hosts to access memory resources across layers, including memory tied to the xPU / CPU. In this example, cost and performance can be optimized by combining a dedicated MxPU for memory pooling with existing xPU / CPUs for processing tasks and additional memory capacity.

[0170] Figure 8A shows an example of a system consisting of a processor, DRAM, and a memory pool comprising an RPU. The RPU may include or be connected to a CXL device. The RPU performs inter-host physical address translation, allowing entities outside the memory pool to access the DRAM coupled to the processor. The processor may include cores, some of which are hidden from the user and used to perform infrastructure tasks related to the operation, management, and administration (OAM) of the memory pool.

[0171] Figure 8B shows an example of a system consisting of a memory pool including a CXL multi-head device (MHD), such as a multi-head single logic device (MH-SLD) or a multi-head multi-logic device (MH-MLD), which are composed of processors connected to DRAM. Each processor contains one or more processing cores, each of which may contain an MMU. The MHD is further composed of CXL endpoints, at least some of which are associated with logical devices such as SLDs and MLDs, and an RPU configured to perform inter-host physical address translation is configured to allow entities outside the MHD to access the DRAM. Optionally, some of the illustrated blocks can be omitted, combined, or implemented as discrete chiplets, IP blocks, or firmware-assisted logic. The number and types of cores depend on the implementation and may include general-purpose CPUs, vector engines, AI accelerators, or heterogeneous combinations thereof. As an alternative or additional example, one or more cores perform memory processing (PIM) operations, such as reduction, exploration, and machine learning kernels, on data residing in DRAM to reduce link bandwidth consumption. The RPU's address translation logic allows the MHD to expose DRAM as a shared or divisible pool, which entities can access via a CXL endpoint, enabling memory pooling, memory sharing, multi-tenant isolation, or dynamic capacity provisioning within a CXL-based system.

[0172] Figure 9 shows an example of a system consisting of an AI memory switch or memory pool, where a CXL multi-head device (MHD) is connected to two external entities. The memory pool may also include additional MHDs coupled with additional entities. The memory pool may utilize a chipset-based architecture in which a collection of electronic components such as MxPUs, xPUs, CPUs, and memory buffers work together on the platform to achieve memory pool functionality. The MHD is a structure in which the MxPU is connected to DRAM, which may be located inside the MHD, for example, mounted on a board next to the MxPU or mounted within the MHD enclosure. Alternatively, the DRAM may be located outside the MHD, and may be mounted in a pluggable memory module (e.g., EDSFF). The MxPU may also be derived from established processor designs, such as CPU designs that combine at least one compute die and at least one I / O die and can communicate with each other using on-package interconnects such as AMD Infinity Fabric, ARM CHI C2C, or NVIDIA NVLink-C2C. The RPU may optionally be implemented on a separate die / chiplet or integrated into the I / O die or compute die, performing physical address translation between hosts and allowing entities linked to the memory pool to access DRAM linked to the MxPU via CXL endpoints. The MxPU may include one or more inter-chip interfaces, such as ISoL, providing interconnection for multiple MxPU instances with different topologies, and creating memory pools that serve larger logical MHDs, distributed MHDs, or additional external entities, providing greater memory capacity. The inter-chip interfaces may utilize communication protocols used in on-package interconnect links, such as AMD Infinity Fabric, ARM CHI C2C, or NVIDIA NVLink-C2C.The processing cores within the MxPU are hidden cores used for infrastructure tasks and may provide Processing In Memory (PIM) services to data residing in DRAM.

[0173] Data center workloads in areas such as GenAI inference, LLM training, and distributed analytics may require memory resources spanning racks and pods within a data center fabric. Traditional CXL implementations rely on the PCIe physical layer, which can limit deployment distance and the ability to leverage existing IEEE 802.3-based data center network infrastructure for memory partitioning. Some implementations offer architectural solutions that go beyond the IEEE 802.3 PMA-based physical layer, where an RPU translates data received over the physical layer between the physical layer and the CXL request, allowing external entities to access memory resources across different physical address spaces. The RPU bridges the external physical address space used for transmission with the local physical address space used by processing cores and memory controllers, thereby facilitating memory pooling, sharing, and tiering at both the intra-rack and inter-pod scales on the data center network fabric.

[0174] In various implementations, the following devices exist: an integrated circuit package (IC package) consisting of a processing core with an instruction cache, the processing core connected to a memory controller via a coherent interconnect and configured to respond to snoop requests using physical addresses in the first physical address space; a memory management unit (MMU) coupled to the processing core, configured to translate virtual addresses to physical addresses in the first physical address space; a memory channel capable of supporting memory with a capacity of at least 64GB, a physical layer based on IEEE 802.3 physical media attachment (PMA), configured to receive transmissions containing Compute Express Link (CXL) opcodes and data indicating physical addresses in a second physical address space; and a resource provisioning unit (RPU) configured to translate the data into CXL requests, which enables entities outside the device to read memory through the physical layer based on IEEE 802.3 PMA, memory controller, and memory channel. This device may also represent a processor or system-on-a-chip (SoC) that combines host processing power with an RPU that exposes local memory resources externally through the IEEE 802.3 PMA-based physical layer. Coherent interconnects include mesh networks, ring interconnects, or other on-die fabrics that maintain cache coherence between processing cores. The RPU can store data indicating CXL opcodes and physical addresses in a second physical address space, translate the data into CXL requests using physical addresses in the first physical address space, and then forward the translated CXL requests to the memory controller via a memory channel to perform the service. In some examples, the device may also send responses to external entities via a physical layer based on IEEE 802.3 PMA to complete read transactions initiated by the external entities.

[0175] In some implementations of the device, when translating data into a CXL request, physical addresses in the second physical address space are translated to physical addresses in the first physical address space. Address translation may be necessary to accommodate scenarios where an external entity uses a different physical address space than that used by the processing core or memory controller. The RPU can implement address translation using lookup tables, page tables, base and offset calculations, hash-based mappings, or programmable translation functions. The first and second physical address spaces may differ in size, base address, and memory layout, and the translation must accommodate these differences while preserving the meaning of memory operations.

[0176] In some implementations of the device, the physical layer based on IEEE 802.3 PMA is configured as the UALink physical layer. The UALink physical layer can operate at the lane rate and link width defined in the UALink specification while utilizing the electrical and signal characteristics of IEEE 802.3 PMA.

[0177] In some implementations of the device, the IEEE 802.3 PMA-based physical layer is configured as the NVLink physical layer. This allows the device to participate in an NVLink-based GPU cluster while exposing local memory resources to external GPU entities via the RPU.

[0178] In some device implementations, the physical layer, based on IEEE 802.3 PMA, is configured for receiving and transmitting Ethernet frames. The physical layer can send and receive standard Ethernet frames, enabling CXL memory access operations to leverage existing Ethernet-based data center network infrastructure. Data indicating the CXL opcode and physical address is included within the Ethernet frame payload, allowing CXL semantics to pass through Ethernet switches, routers, and other network devices deployed within the data center fabric.

[0179] In some implementations of the device, the RPU is either contained within the IC package or in a second IC package coupled to the IC package. Integrating the RPU into the IC package can eliminate off-die communication overhead and potentially reduce latency and power consumption. Alternatively, placing the RPU in a second IC package, such as a companion chiplet or discrete bridge device, may allow for modularization or independent RPU functionality upgrades. When the RPU is contained in a second IC package, coupling may be implemented through die-to-die interconnects, package substrates, interposers, or other appropriate physical coupling mechanisms.

[0180] In some implementations of the device, memory consists of dynamic random access memory (DRAM), and the RPU is coupled to either a CXL device, a CXL endpoint, or a CXL port. The DRAM includes DDR5, DDR4, HBM, or other DRAM technologies and is coupled to the memory controller via a memory channel. The RPU is connected to a CXL device, CXL endpoint, or CXL port to facilitate communication between external entities and memory resources. The CXL device, CXL endpoint, or CXL port may expose a protocol interface that allows the RPU to send translated CXL requests to the CXL domain, where the memory controller can process them.

[0181] In some implementations of the device, the CXL port is configured as a switch port or root port, and the entities consist of GPUs, network devices, or storage devices. When the CXL port is configured as a switch port, the RPU may be coupled with a CXL switch fabric that routes translated requests to the memory controller. When the CXL port is configured as a root port, the RPU interfaces with the host's root complex, allowing external entities to access memory as if it were a local CXL device. GPUs can access memory for computational workloads such as AI / ML training and inference, network devices can access memory for packet buffer pooling and network function virtualization, and storage devices can access memory for caching and data staging operations.

[0182] In some implementations of the device, CXL devices consist of Global Fabric Attached Memory (G-FAM) devices (GFDs), and the RPU communicates with the GFDs via either CXL.mem or CXL.io UIO. The GFD provides scalable memory resources accessible to multiple requesters within the CXL fabric, and the GFD has a device physical address space common to all requesters. The RPU can communicate with the GFD using CXL.mem for memory load / store operations or CXL.io for configuration, management, or unordered I / O (UIO) operations. In some examples, the RPU may also translate data received via the physical layer based on IEEE 802.3 PMA into CXL.mem M2S requests targeting the GFD, allowing external entities to access fabric-attached memory within the data center.

[0183] In some implementations of the device, the second physical address space is identical to the first physical address space, and the RPU is configured to read memory-mapped I / O (MMIO) registers coupled to the device. When the second physical address space is identical to the first physical address space, the RPU can traverse physical addresses without translation, allowing external entities to target addresses within the device's physical address space. MMIO registers may include registers from peripherals coupled to the device, such as network interface cards (NICs), InfiniBand host bus adapters (HBAs), storage controllers, and other I / O devices. This configuration enables operation methods such as "direct GPU access," where an external GPU or accelerator reads the MMIO registers of NICs or HBAs connected to the device.

[0184] In some implementations of the device, the CXL opcode is composed of a UIOMRd transaction layer packet (TLP) type. By including UIOMRd within the range of CXL opcodes that the RPU translates, the device can support UIO-based access patterns, allowing external entities to perform unordered memory reads targeting MMIO registers and memory locations coupled to the device. This is beneficial in multipath fabric topologies where ordering flexibility improves bandwidth utilization.

[0185] In some implementations of the device, the second physical address space is a subset of the first physical address space, and the RPU is further configured to block entities from reading at least one predetermined address range within the first physical address space. The RPU may enforce access control policies that restrict external entities from accessing sensitive address ranges within the first physical address space, such as security controller MMIO registers, firmware storage areas, management engine registers, and other protected resources. Predetermined address ranges can be initialized or configured at runtime by firmware, BIOS, fabric manager, or system management software. The RPU may maintain access control lists, range registers, or bitmask structures that define the allowed and restricted address ranges for external entities.

[0186] In some implementations of the device, data indicating the CXL opcode and physical address is encapsulated within a carrier protocol-compliant message. Furthermore, the RPU is configured to extract the data indicating the CXL opcode and physical address from the carrier protocol-compliant message and to translate the extracted data into a CXL request. The carrier protocol can encapsulate the CXL payload data within its own framing, header, and trailer structure, enabling the transmission of CXL semantics over network infrastructure that natively supports the carrier protocol. The RPU parses the incoming carrier protocol message, identifies and extracts the encapsulated CXL opcode and physical address, and translates the extracted data into a CXL request for processing by the memory controller. Encapsulation allows CXL traffic to pass through standard data center network equipment such as switches and routers operating on the carrier protocol.

[0187] In some device implementations, the physical layer based on IEEE 802.3 PMA is configured to receive messages compliant with the carrier protocol. Furthermore, the RPU is further configured to translate physical addresses in the second physical address space to physical addresses in the first physical address space when processing CXL requests. Carrier protocol extraction and address translation may be performed as part of the integrated request processing pipeline, with both operations applied sequentially before the translated CXL request is forwarded to the memory controller. Address translation is implemented using lookup tables, base offset calculations, or programmable translation functions.

[0188] In some implementations of the device, the RPU is configured to further encapsulate the CXL response data into a response message compliant with a carrier protocol for transmission over the physical layer based on IEEE 802.3 PMA. The RPU can perform reverse address translation when constructing the response message, translating the physical address from the first physical address space to the second physical address space, ensuring that the response is properly correlated with the original request of the external entity.

[0189] In some implementations of the device, the carrier protocol is based on Ethernet or IEEE 802.3. Additionally, data indicating the CXL opcode and physical address are encapsulated within the Ethernet frame or IEEE 802.3 frame, respectively. A dedicated EtherType value and protocol identifier can be used to distinguish CXL-encapsulated frames from other Ethernet traffic sharing the same physical infrastructure. This allows CXL memory access traffic to be routed, switched, and managed within existing data center Ethernet infrastructure, such as top-of-rack switches, aggregation switches, and spine-leaf fabric elements.

[0190] In some implementations of the device, the carrier protocol is based on the Ultra Ethernet Transport (UET) protocol. Data indicating the CXL opcode and physical address is encapsulated in link-layer retry-eligible frames (LLR-eligible frames). UET's link-layer retry mechanism can potentially allow the retransmission of lost or corrupted frames, helping to maintain long-distance data integrity, which is more susceptible to transient errors compared to PCIe-based CXL connections.

[0191] In some implementations of the device, the carrier protocol is based on Scale Up Ethernet (SUE). Data indicating the CXL opcode and physical address is encapsulated in SUE-based protocol data units (PDUs). Examples of SUE-based PDUs include SUE PDUs, SUE Lite PDUs, or PDUs based on future SUE revisions. SUE provides low-latency, high-bandwidth transport optimized for scale-up interconnect topologies within data center pods, enabling tightly coupled memory access patterns suitable for distributed AI / ML training and HPC workloads.

[0192] In some implementations of the device, a CXL request consists of a master-to-sublossive request (M2S Req) message in CXL.mem. Furthermore, the RPU is configured to receive CXL responses that include a sublogue-to-master data response (S2M DRS) message in CXL.mem. The M2S Req message may contain memory read opcodes such as MemRd* or MemSpecRd. The RPU can correlate the corresponding M2S Req and S2M DRS messages using tags or other transaction identifiers, encapsulate the response data, and send it to an external entity via a carrier protocol.

[0193] In some implementations of the device, the RPU is further configured to: translate tags associated with data from the first tag space to the second tag space; the first tag space is used by entities outside the device, and the second tag space is used by hosts coupled to the device. The first and second tag spaces may have different sizes, allocation policies, or different encoding formats. The RPU may maintain a tag translation table, content-addressable memory, or algorithmic mapping logic to associate tags with the two tag spaces for each pending transaction. Optionally, the RPU translates tags in CXL responses associated with the second tag space to tags associated with the first tag space before encapsulating them in the carrier protocol, enabling end-to-end tag correlation across carrier protocol boundaries.

[0194] Various implementations include the following techniques: operating a processor consisting of a processing core and an instruction cache; translating virtual addresses to the processing core's physical addresses, in which case the processing core communicates with the memory controller via a coherent interconnect and responds to snoop requests using physical addresses in the first physical address space; communicating with memory with a capacity of at least 64GB through a memory channel; receiving transmissions in a second physical address space containing Compute Express Link (CXL) opcodes and data indicating physical addresses through a physical layer based on an IEEE 802.3 physical media attachment (PMA); and translating the data into CXL requests by a resource provisioning unit (RPU). This translation allows entities outside the processor to read memory through the physical layer based on the IEEE 802.3 PMA, memory controller, and memory channel. This technique may be performed by a processor, SoC, or other computing device. This method further includes sending CXL response data to an external entity through the physical layer based on the IEEE 802.3 PMA to complete a memory read transaction.

[0195] In some implementations of this technique, when translating data into a CXL request, physical addresses in the second physical address space are translated to physical addresses in the first physical address space. Address translation can be implemented using lookup tables, page tables, base offset calculations, or programmable translation functions managed by the RPU. The translation may accommodate different address space sizes, base addresses, and memory layouts between the external entity and the local memory subsystem.

[0196] In some implementations of this method, the second physical address space is identical to the first physical address space, and it also allows the RPU to read memory-mapped I / O (MMIO) registers coupled to the processor. This configuration can enable direct device control scenarios, such as direct access from the GPU to NIC registers or accelerator control of peripherals coupled to the processor.

[0197] In some implementations of this approach, the second physical address space is a subset of the first physical address space, and the RPU further prevents entities from reading at least one predetermined address region within the first physical address space. Blocked address regions include security-sensitive locations, firmware areas, management interfaces, or other protected resources. The RPU may maintain access control lists, range registers, or bitmask structures that define permitted and restricted address ranges. Access control policies may be set during system initialization or dynamically updated by system administration software or a fabric manager.

[0198] In some implementations of this method, data indicating the CXL opcode and physical address is encapsulated within a carrier protocol-compliant message, and the RPU further extracts the data indicating the CXL opcode and physical address from the carrier protocol-compliant message and converts the extracted data into a CXL request. Carrier protocols include Ethernet, IEEE 802.3, UET, ESUN, SUE, or other transport protocols that provide framing and delivery mechanisms on the physical layer based on IEEE 802.3 PMA. Encapsulation allows CXL traffic to pass through standard data center network equipment such as switches and routers that operate natively on the carrier protocol.

[0199] In some implementations, a carrier protocol-compliant message is received through a physical layer based on IEEE 802.3 PMA, the RPU translates the physical address in the second physical address space to the physical address in the first physical address space, the RPU encapsulates the CXL response data into a second message, and transmits it via the IEEE 802.3 PMA-based physical layer. Bidirectional translation can handle a complete request-response cycle across carrier protocol boundaries. The RPU may perform reverse address translation when constructing the second message.

[0200] In some implementations, this technique further involves the RPU converting tags from the first tag space to a second tag space, with the first tag space being used by entities outside the processor and the second tag space being used by a host coupled to the processor. Furthermore, the RPU converts tags associated with the second tag space back to those associated with the first tag space before encapsulating them in the carrier protocol. The RPU maintains tag mapping tables and uses algorithmic mapping to manage concurrent transactions between the two tag spaces. Tag mappings may be stored in hardware registers, content-addressable memory, or other suitable storage structures, allowing for efficient bidirectional retrieval of request processing and response correlations.

[0201] Various implementations include the following systems: an integrated circuit package (IC package) where the processing core is connected via a coherent connection to the memory controller, and the processing core responds to snoop requests using physical addresses in the host's physical address space; a memory management unit (MMU) configured to translate virtual addresses to physical addresses in the host's physical address space; a memory channel connected to memory with a capacity of at least 64GB and accessible from the memory controller; a physical layer based on IEEE 802.3 physical media connection (PMA) configured to communicate with each external entity, and the physical layer receives transmissions containing data for memory access requests, including physical addresses; and at least one resource provisioning unit (RPU) configured to translate between (i) the physical address associated with the transmission and (ii) a physical address in the host's physical address space. This translation allows external entities to access memory through the IEEE 802.3 PMA-based physical layer, memory controller, and memory channel. At least one RPU can simultaneously manage address translation for multiple external entities, enabling memory pooling and sharing scenarios when different entities access different or overlapping regions of memory. This system may be deployed in data center environments where GPUs, accelerators, network devices, and other computing elements access shared memory resources for distributed workloads such as AI / ML training, inference serving, and data analysis.

[0202] In some implementations, "data indicating a memory access request that includes a physical address" or "the physical address associated with the transmission" may include various transmission formats that convey the intent of the memory access. Some formats may include an explicit memory access request with the complete physical address embedded within the protocol message. Alternatively, certain protocols may use an implicit addressing scheme that derives addresses from an address cache, abbreviated address representations, delta coding, base and offset techniques, lookup tables, transaction identifiers that reference an already established address context, or other protocol fields, or that holds state information. The RPU or computer can reconstruct or derive the complete physical address from these representations and perform the appropriate translation.

[0203] In some implementations of the system, memory access requests comply with at least one protocol selected from Ultra Accelerator Link (UALink) requests, UALink Protocol-Level Interface (UPLI) requests, or NVLink requests. Additionally, at least one of the IEEE 802.3 PMA-based physical layers consists of the UALink physical layer, the NVLink physical layer, or an Ethernet physical layer operating at 100 Gbps or higher. The system may support heterogeneous interconnect environments where different external organizations utilize different protocols over the IEEE 802.3 PMA-based physical layer. UALink or NVLink may be used for accelerator-to-memory communication within an AI / ML cluster, while the Ethernet physical layer may be used for long-distance inter-pod memory access.

[0204] In some implementations of the system, at least one RPU consists of RPUs distributed across the entire IC package or between IC packages. Furthermore, the RPUs are configured to maintain separate address translation tables for each different external entity, enforce access control policies that define the permitted address ranges for the corresponding external entities, and create isolated security domains for memory access while sharing the same physical memory resources. Each RPU or RPU instance can maintain its own dedicated address translation table and access control policy for the relevant external entity. The address translation table maps different regions of the external entity's address space to different or overlapping regions of the host physical address space, enabling flexible memory partitioning and sharing. Access control policies may define permitted address ranges, read / write permissions, and bandwidth allocations for each external entity. This architecture is beneficial in multi-tenant data center environments and systems where different external entities have different levels of trust, enabling scalable memory access with quality and isolation assurances for each entity.

[0205] Figure 10A shows an example system in which entities are coupled to an RPU containing or connected to a CXL device via a physical layer based on IEEE 802.3 PMA, and the CXL device is coupled to an ARM architecture processor via one or more CCG nodes and possibly one or more RN-D nodes. A memory controller (MC) in the system is coupled to an SN-F node and interfaces with DRAM through DDR PHYs and memory channels. The example includes crosspoints (XPs) that act as routing elements in the ARM mesh interconnect, examining packet identifiers to determine the appropriate routing path and managing traffic flow between source and destination within the mesh structure.

[0206] Figure 10B shows an example of a TFD that can be executed in the example described in Figure 10A. This demonstrates the conversion from the first CXL.mem to the second CXL.mem by the RPU, and then shows the requester (if it is the CCG or CXL device in Figure 10A) initiating CHI and assigning the ReadShared request to the home node, with the memory controller acting as a subnode. The figure illustrates an optimization technique for the composite response from the subnode. The home node sends a ReadNoSnp request to the memory controller, and the memory controller returns the data to the requester using the CompData opcode, thus reducing the number of messages and eliminating the need for a data loop through the home node, thereby improving transaction latency.

[0207] Figure 11 shows an example of multi-host memory pooling or sharing using a switch-based topology with a physical layer based on IEEE 802.3 PMA. This system includes at least two different paths for hosts to access memory resources through the same memory channel. The first path involves an entity such as an xPU having a host (designated A.1), connected via CXL, connected via a carrier protocol interface, and sending to an RPU via a physical layer based on IEEE 802.3 PMA (sometimes utilizing an Ethernet PHY). The RPU is coupled to or embedded with a switch, which may be a PBR switch, enabling scalable connectivity. The switch includes downstream ports (DSPs) coupled to memory controllers and memory-connected CXL type 3 devices (designated A.2 and B.2), enabling distributed memory resources. The second path involves an upstream port (USP) of the switch connected to a host system (designated B.1) via a root port (RP), which includes a coherent interconnect, a processing core with MMU functionality, and an instruction cache. This dual-path connection allows host B.1 to access memory through connections via the switch's USP and RP, while simultaneously enabling external entities such as xPU (host A.1) to access the same memory resources through the RPU and switch infrastructure. The GPU can also connect to the switch via its own RPU and PHY based on IEEE 802.3 PMA, demonstrating the ability to integrate heterogeneous computing elements.

[0208] In heterogeneous computing architectures, a computer system may contain multiple hosts that utilize different address spaces while requiring coordinated access to shared resources. In such multi-host environments, the first host may operate in the first host physical address (HPA) space, the second host in the second HPA space, and both hosts may need to maintain cache consistency when communicating via CXL.cache. Translating CXL.cache messages associated with different hosts can maintain CXL.cache requirements while facilitating memory coherency operations, cache line invalidation, and data transfer between different address spaces.

[0209] In various implementations, there are methods for translating between Compute Express Link (CXL) messages, which include: receiving a CXL.cache Host-to-Device (H2D) request from the first entity; translating the CXL.cache H2D request into a CXL.cache Device-to-Host (D2H) request; and sending the CXL.cache D2H request to the second entity. The translation process may involve various aspects of the protocol message, such as opcodes, addresses, and transaction identifiers, enabling consistent communication between hosts that cannot communicate directly due to protocol limitations or direction mismatches. This technique can be implemented in hardware, firmware, software, or a combination thereof, and may be executed on various types of devices, such as semiconductor devices and RPUs.

[0210] In some implementations of this technique, the first entity consists of the first host, and the CXL.cache H2D request consists of the first address belonging to the first host's physical address (HPA) space. Then, the second entity consists of the second host, and the CXL.cache D2H request consists of the second address belonging to the second HPA space used by the second host. In some examples, address translation may be implemented using lookup tables, page tables, hash tables, base offset calculations, or programmable translation functions. The first and second HPA spaces may differ in size, base address, and memory layout, and the translation must accommodate these differences while preserving the meaning of memory operations.

[0211] In some implementations of this technique, a CXL.cache H2D request consists of a snoop invalidation (SnpInv) and a Unique Queue ID (UQID), while a CXL.cache D2H request consists of a CacheLine Flush (CLFlush) and a Command Queue Identifier (CQID). The conversion from SnpInv to CLFlush allows the first host to invalidate a cache line in the second host's cache hierarchy. The UQID in the H2D request is mapped to the CQID in the D2H request, and this mapping is maintained in a translation table, tracker entry, or similar data structure.

[0212] In some implementations of this technique, CXL.cache H2D requests consist of a snoop invalidation (SnpInv) and a unique queue ID (UQID), while CXL.cache D2H requests consist of a command queue identifier (CQID) and either a no-ownership read (RdOwnNoData) or ownership read (RdOwn). The conversion from SnpInv to RdOwnNoData or RdOwn may allow the first host to perform cross-orchestration of cache line states between the first and second host's cache coherency subsystems. This may optionally invalidate a cache line address in the second host's cache hierarchy, marking that cache line address in an exclusive state, and possibly before any write operations by the first host.

[0213] In some implementations, this method involves receiving CXL.cache H2D data from the second host, converting it into CXL.cache device-to-host data (D2H), and then sending the CXL.cache D2H data to the first host.

[0214] In some implementations of this technique, a CXL.cache H2D request consists of SnpData and a Unique Queue ID (UQID), while a CXL.cache D2H request consists of RdShared and a Command Queue Identifier (CQID). The conversion from SnpData to RdShared allows the first host to retrieve a cache line in a shared state from the second host's cache hierarchy. RdShared can request that the cache line be cached in a shared state, allowing both the first and second hosts to maintain a cached copy of the cache line. The UQID in the H2D request is mapped to the CQID in the D2H request, and this mapping is maintained in translation tables and tracker entries.

[0215] In some implementations of this technique, an H2D request for CXL.cache consists of SnpData and a Unique Queue ID (UQID), while a D2H request for CXL.cache consists of RdOwn and a Command Queue Identifier (CQID). The conversion from SnpData to RdOwn may allow the first host to acquire an exclusive cache line. SnpData may also indicate an intention to acquire a shared or exclusive state. The translation logic may select RdOwn based on additional factors such as system configuration, expected access patterns, and optimization policies. RdOwn may mean that the second host relinquishes ownership of the cache line, provides the cache line data to the translation logic, and has that data forwarded to the first host.

[0216] In some implementations of this technique, a CXL.cache H2D request consists of a Snoop Invalidation (SnpInv) and a Unique Queue ID (UQID), and a CXL.cache D2H request consists of an RdOwn and a Command Queue Identifier (CQID), and also receives a CXL.cache H2D Data message containing cache line data from the second host. The conversion from SnpInv to RdOwn may allow the first host to acquire exclusive ownership of the cache line while also receiving cache line data from the second host. The second host may return a GO-M or GO-E indication along with the cache line data, which may indicate that the second host previously held the cache line in a modified state or provided an exclusive cache line. Translation logic can complete the data transfer and cache state transition by converting the received H2D data message into a D2H data message and delivering it to the first host. As an example, a CXL.cache H2D request consists of an opcode selected from SnpData or SnpCur. A D2H request in CXL.cache consists of an opcode selected from one of the following: RdCurr, RdOwn, RdShared, RdAny, RdOwnNoData, ItoMWr, WrCur, CleanEvict, DirtyEvict, CleanEvictNoData, WOWrInv, WOWrInvF, WrInv, or CacheFlushed. The translation logic can select the appropriate D2H opcode while preserving the intent of the original operation and adapting to the receiver's protocol requirements. For example, in some implementations, the translation logic may translate an H2D request containing SnpCur into a D2H request containing RdCurr, and then respond to the H2D request with a D2H response consisting of RspSFwdM, RspIFwdM, and RspVFwdV, and further with D2H data containing the data obtained in the RdCurr D2H request. In other implementations, the translation logic may convert an H2D request containing SnpData into a D2H request containing RdShared.Furthermore, it is also possible to respond to an H2D request with a D2H response containing RspSFwdM and D2H data including data obtained by the RdShared D2H request.

[0217] In some implementations, this technique further receives a CXL.cache H2D data message including cache line data and a command queue identifier (CQID) from the second entity. Converting the CXL.cache H2D data message into a CXL.cache D2H data message including cache line data and a unique queue ID (UQID); and transmitting the CXL.cache D2H data message to the first entity. The translation of the data message allows cache line data to flow from the second host to the first host via the translation logic. The CQID in the H2D data message can be converted to the corresponding UQID using a previously stored mapping, enabling proper correlation with the originating request. The cache line data may include 64 bytes or other cache line sizes supported by CXL.cache, and may be transferred without modification, or may undergo additional processing such as address translation or data conversion.

[0218] In some implementations, this technique further includes receiving a CXL.cache H2D response consisting of a GO opcode and a command queue identifier (CQID) from the second entity. Converting the CXL.cache H2D response into a CXL.cache D2H response consisting of Rsp* and a Unique Queue ID (UQID); and transmitting the CXL.cache D2H response to the first entity. A GO response from the second host (e.g., GO-I) is converted to Rsp* (e.g., RspIHitI) for the first host, indicating that the cache line was not found in the device cache (i.e., the cache line is in an invalid state). The translation may utilize the previously stored mapping from UQID to CQID to correctly return the response to the original transaction.

[0219] In some implementations of this method, the CXL.cache H2D response consists of a GO opcode and a field indicating an invalid state (GO-I), in which the CXL.cache D2H response consists of RspIHitI.

[0220] In some implementations, the method further receives CXL.cache H2D data including a command queue identifier (CQID) from the second entity. Converting the CXL.cache H2D data into a CXL.cache D2H response consisting of *Fwd* and a Unique Queue ID (UQID); and transmitting the CXL.cache D2H response to the first entity. It indicates that the CXL.cache H2D data from the second host is converted into a CXL.cache D2H response containing *Fwd* such as RspSFwdM, RspIFwdM, RspVFwdV, and transmitted to the first host. This indicates that CXL.cache D2H data is sent after the CXL.cache D2H response, which may enable data transfer from the second host to the first host via the RPU.

[0221] In some implementations of this method, the CXL.cache D2H request consists of RdCurr and a command queue identifier (CQID). Further, it also includes converting the CXL.cache H2D data into a CXL.cache D2H response consisting of RspVFwdV and a Unique Queue ID (UQID).

[0222] In some implementations, this technique further includes receiving a CXL.cache H2D response from the second entity consisting of a GO opcode and a command queue identifier (CQID); converting the CXL.cache H2D response into CXL.cache D2H data including a unique queue ID (UQID); and sending the CXL.cache D2H data to the first entity. The CXL.cache H2D response from the second host may also be converted into CXL.cache D2H data and sent to the first host, for example, in error scenarios where synthetic data is generated based on an error response from the second host.

[0223] In some implementations, this technique further involves exposing a CXL type 1 or CXL type 2 device to the first entity via the first interface, and then exposing a CXL type 1 or CXL type 2 device to the second entity. The exposure of the device type may determine the types of CXL.cache transactions that can be initiated and received at each interface. By exposing the appropriate device type to each host, the transformation can accommodate different system configurations and use cases. Depending on the system configuration requirements, similar or different device types may be exposed to different hosts simultaneously.

[0224] In some implementations of this technique, the conversion includes conversions between CXL.cache H2D requests and CXL.cache D2H requests. Conversions include conversions between unique queue IDs (UQIDs) and command queue identifiers (CQIDs), conversions between reserved fields, and conversions between reserved and unreserved fields. UQIDs and CQIDs may function as transaction identifiers in their respective protocol directions. Reserved fields in one protocol direction may map to active fields in the other direction and may contain additional metadata or control information.

[0225] In some implementations, this technique further involves receiving a CXL.cache H2D response from the second entity, including a GO-S instruction and a Command Queue Identifier (CQID); translating the CXL.cache H2D response into a CXL.cache D2H response consisting of an RspSHitSE and a Unique Queue ID (UQID); and sending the CXL.cache D2H response to the first entity. The GO-S indication shows that the second host is providing the cache line in a shared state, which may allow simultaneous caching by multiple entities. The RspSHitSE enables cache line state orchestration by indicating to the first host that the cache line has reached a clean state and that its current state is shared. This allows both the first and second hosts to store the cache line in a shared state. The translation may utilize previously stored UQID-to-CQID mappings to correctly return the response to its original transaction.

[0226] In some implementations, this technique further involves receiving a CXL.cache H2D response from the second entity containing a GO-E instruction and a Command Queue Identifier (CQID); translating the CXL.cache H2D response into a CXL.cache D2H response consisting of an opcode selected from RspIHitI, RspIHitSE, or RspIFwdM, containing a Unique Queue ID (UQID); and sending the CXL.cache D2H response to the first entity. The GO-E instruction may indicate that the second host has granted exclusive ownership of the cache line address. The translated D2H response opcode indicates to the first host that the cache line no longer exists in the cache abstraction exposed by the translation logic, allowing the first host to exclusively transfer the cache line state. The choice of RspIHitI, RspIHitSE, or RspIFwdM depends on the prior state of the cache line and whether data transfer has occurred, and may allow the first host to complete the appropriate cache coherency protocol.

[0227] In some implementations, this technique further involves receiving a CXL.cache H2D response from the second entity consisting of a GO-M instruction and a Command Queue Identifier (CQID), and a CXL.cache H2D Data message consisting of the cache line data. The H2D response of CXL.cache is converted into a D2H response of CXL.cache consisting of an RspIFwdM and a Unique Queue ID (UQID); and the D2H response of CXL.cache is sent to the first entity. The GO-M indication may indicate that the second host previously held the cache line in a modified state and has relinquished ownership along with the modified data. The RSPIFwdM may indicate to the first host that the cache line has been found and transferred in a modified state, which may allow the first host to transition the cache line's state to a modified state. The modified cache line data is converted from an H2D data message to a D2H data message and transferred to the first host, completing the transfer of ownership and data delivery.

[0228] In some implementations, this technique further exposes the cache abstraction to the first entity through the CXL.cache interface, with the cache abstraction acting as a proxy for the cache contained within the second entity, and CXL.cache H2D requests targeting the cache abstraction. The cache abstraction may appear to the first host as a device cache accessible via CXL.cache transactions, but internally represents or proxies cache resources managed by the second host. The first host issues CXL.cache H2D requests (such as snoop requests) targeting the cache abstraction, and translation logic translates these requests into D2H requests to CXL.cache, which may affect the actual cache. This proxying scheme may enable cache coherence operations between hosts that cannot communicate directly due to protocol direction constraints or address space differences.

[0229] In some implementations of this technique, translation enables cache line state orchestration between a cache managed by the first entity and a second cache managed by the second entity. Cache line state orchestration allows transitions between cache states such as modified, exclusive, shared, and invalid (MESI) to be coordinated between the caches maintained by the first and second entities, respectively. Here, a transition from the first cache to the first cache state (e.g., Exclusive) corresponds to a transition from the second cache to the second cache state (e.g., Invalid). The translation logic allows the first entity to influence the cache line state of the second entity. This enables translating H2D requests into corresponding D2H requests, triggering appropriate cache state transitions in the second entity.

[0230] In some implementations of this technique, cache line state orchestration includes cross-invalidation of cache line states, or cache line state orchestration enables cache-coherent memory sharing between a first and second entity. Cross-invalidation allows the first host to cause invalidation of a cache line entry in the second host's cache hierarchy, or vice versa, maintaining cache consistency across a multi-host system. Cache line state orchestration may include transitions between various cache states such as modified, exclusive, shared, and invalid (MESI), or similar cache coherency protocols. Translation logic can track pending transactions and coordinate state transitions to maintain consistency invariance across both cache hierarchies. Furthermore, or alternatively, cache-coherent memory sharing allows the first and second hosts to access a shared memory area while maintaining data consistency through cache coherency protocols. Translation logic facilitates consistent access by translating snoop operations, read requests, and writes between hosts, and makes memory updates by one host visible to the other host according to a memory consistency model. Such cache-consistent memory sharing may be used in distributed memory systems, multi-GPU clusters, heterogeneous computing platforms, and other multi-host architectures.

[0231] Some implementations of this method consist of instructions in a non-temporary computer-readable medium that, when executed by a processor, cause the processor to execute the method. Some implementations of this technique involve one or more integrated circuits configured to execute the technique, and these one or more integrated circuits constitute one of the following: (i) a general-purpose processing unit configured or connected via a communication interface (configured to execute the method by software and / or firmware execution), (ii) firmware and / or hardware logic circuits integrated into electronic equipment, which utilize hardware acceleration or special processing capabilities not typically provided by general-purpose processing units, or (iii) an operation involving one or more chiplets within one or more integrated circuit packages. Some implementations of this method involve an active cable consisting of first and second pluggable modules coupled in a physical medium, where the active cable further constitutes hardware circuitry integrated into the active cable and configured to execute the method. Some implementations of this technique involve a device acting as a switch, which consists of switching circuitry and is configured to execute the method.

[0232] In various implementations, a computer is connected to a first and second interface based on Compute Express Link (CXL), and the computer is configured as follows: Through the first interface, it receives a CXL.cache host-to-device (H2D) request from the first entity, consisting of the first address belonging to the first host physical address (HPA) space; it translates the CXL.cache H2D request into a CXL.cache device-to-host (D2H) request containing the second address belonging to the second HPA space; and it sends the CXL.cache D2H request to the second entity via the second interface. The computer may include processing logic, memory to store translation tables and transaction state, and an interface controller to manage CXL.cache communication. The system may be implemented as a standalone device, integrated into a larger semiconductor component, or distributed among components within a computing platform. The translation function enables diverse system architectures, such as distributed memory systems, multi-GPU clusters, and heterogeneous computing platforms. Furthermore, the computer further exposes the cache abstraction to the first entity through the first interface, and the cache abstraction acts as a proxy for the cache contained in the second entity. The first host enumerates and operates on the cache abstraction as a local device cache, while the computer internally translates the cache operations into D2H requests for CXL.cache, which are then sent to the latter host. This enables transparent cache coherency operations between hosts, without requiring direct communication between hosts or protocol-level awareness of the multi-host topology.

[0233] In some implementations of the system, the first entity consists of the first host consuming the first HPA space, and the second entity consists of the second host consuming the second HPA space. Furthermore, the computer is configured to identify the intent indicated by the CXL.cache H2D request, which includes the intent of the cache line state at the cache line address, and the translation is performed at least partially based on the identified intent. The computer contains logic circuits, processing elements, or firmware that parse the CXL.cache H2D request and determine the cache line state intent from request attributes such as opcodes and address values. The identified intent can be used to select the appropriate translation mapping, D2H request opcode, and address translation. The computer manages a mapping table from intent to opcode or implements intent identification through combinational logic or a state machine.

[0234] In various implementations, based on Compute Express Link (CXL), a computer is connected to a first and second interface and configured as follows: It receives CXL.cache host-to-device (H2D) requests from the first entity through the first interface. It translates the CXL.cache H2D requests to CXL.cache device-to-host (D2H) requests. It sends the CXL.cache D2H requests to the second entity via the second interface. This translation enables cache line state orchestration between the cache managed by the first entity and the second cache managed by the second entity. The system may be implemented as a single semiconductor device, integrated into a large component such as an RPU, or distributed among components within a computing platform. Cache line state orchestration can coordinate transitions between cache states between both entities, such as coordinating transitions from the first cache to an exclusive state and transitions from the second cache to an invalid state. In this case, the translation of CXL.cache messages enables consistent operations that would otherwise be hindered by protocol constraints. The first and second interfaces expose CXL Type-1 or CXL Type-2 device interfaces to their respective entities, allowing them to interact with the system using CXL.cache transactions.

[0235] In some implementations of the system, cache line state orchestration includes cross-invalidation of cache line states, enabling cache-coherent memory sharing between the first and second entities. Cross-invalidation allows cache consistency to be maintained across multi-entity systems by either the first entity causing the second entity to invalidate a cache line entry in its cache hierarchy, or conversely, by the first cache transitioning to an Excluded state and the second cache transitioning to an Invalidated state. Cache-coherent memory sharing allows the first and second entities to access a shared memory area while maintaining data consistency through the cache coherency protocol. This enables consistent access by translating snoop operations, read requests, and write-back operations between entities and may be used in distributed memory systems, multi-GPU clusters, heterogeneous computing platforms, and other multi-entity architectures.

[0236] Figure 12A shows an example of a system in which a computer connects the first interface (Interface 1) and the second interface (Interface 2). The first interface exposes a CXL Type 1 or CXL Type 2 device and can communicate with the first entity (e.g., a host (Host.1)) according to the first CXL.cache. Optionally, communication can be via the first CXL root port (CXL RP.1) of the first entity. Similarly, the second interface exposes a CXL Type 1 or CXL Type 2 device and may communicate with a second entity (Entity.2), e.g., a second host (Host.2), according to the second CXL.cache. Optionally, communication can be via the second CXL root port (CXL RP.2) of the second entity. The computer then extracts the physical addresses of messages received through the first interface, which can be taken from the first HPA space used by the first entity. Translate these messages; and generate messages to send via the second interface, carrying the translated physical addresses. These translated addresses may correspond to the second HPA space used by the second entity. Optionally, the interfaces may also be implemented as internal and / or external interfaces of the semiconductor devices that make up the computer.

[0237] Figure 12B shows an example of a TFD transformation performed by the computer. This shows that the computer is performing a transformation between the first CXL.cache message received from the first entity (Entity.1) (e.g., the first host (Host.1)) and the second entity (Entity.2) sent to the second entity (Entity.2) (e.g., the second host). This allows the first entity to be preserved at least partially. For example, it enables memory sharing and memory consistency with the second entity by allowing the first entity to invalidate the cache line of the second entity. The first entity can initiate the first CXL.cache transaction, which includes an Opcode (SnpInv), UQID (t.1.1), and Address (AS.1.1) in the CXL.cache H2D request. The computer can convert the first CXL.cache transaction into a second CXL.cache transaction containing a CXL.cache D2H request consisting of Opcode (CLFlush), CQID (q.2.1), and Address (AS.2.1), and send the CXL.cache D2H request to the second entity. If the response from the second entity contains a CXL.cache H2D response (including Opcode (GO-I) and CQID (q.2.1)), the computer can convert the CXL.cache H2D response into a CXL.cache transaction consisting of Opcode (RspIHitI) and UQID (t.1.1). This converts to a D2H response. The computer may also perform opcode conversions, such as converting between H2D request opcodes in CXL.cache (Snp* (e.g., SnpData, SnpInv, SnpCur)) and D2H request opcodes in CXL.cache (RdCurr, RdOwn, RdShared, RdAny, RdOwnNoData, ItoMWr, WrCur, CLFlush, CleanEvict, DirtyEvict, CleanEvictNoData, WOWrInv, WOWrInv, WOWrInvF, WrInv, WrInv, CacheFlushed)).Furthermore, you can perform field conversions between messages that comply with the first CXL.cache transaction, as well as message conversions that comply with the second CXL.cache transaction (such as conversions between UQID and CQID, conversions between reserved fields, conversions between reserved fields, and conversions between reserved and unreserved fields).

[0238] Figure 12C illustrates an example of the transformations performed by the computer between TFDs (Transactions in the Data Function) and CXL.cache transactions, where a transaction may contain messages such as a request, response, and optionally a data message. The first entity (Entity.1), for example, the first host (Host.1), may send a CXL.cache H2D request consisting of Opcode (SnpCur), UQID (t.1.1), and Address (AS.1.1). Here, the CXL.cache H2D request may represent a snoop request for the current version of the cache line. The computer transforms the CXL.cache H2D request into a CXL.cache D2H request consisting of Opcode (RdCurr), CQID (q.2.1), and Address (AS.2.1), where the CXL.cache D2H request may represent a read request from the computer to a second entity for the current version of the cache line. The computer may identify the intent of the CXL.cache H2D request received from the first entity and provide intent-based translations, such as the intent to retrieve the latest version of the cache line, or to leverage that intent to translate between the CXL.cache H2D request and the CXL.cache D2H request. The second entity can respond to the CXL.cache D2H request, which includes the RdCurr opcode, with CXL.cache H2D Data, which includes the CQID (q.2.1) and Data (*Data.1*). The computer can then translate the CXL.cache H2D data into a CXL.cache D2H response consisting of the Opcode (RspVFwdV) and UQID (t.1.1), and send the CXL.cache D2H response to the first entity. Furthermore, the computer can convert the H2D data in CXL.cache into D2H data in CXL.cache (consisting of UQID(t.1.1) and Data(*Data.1*)) and send the D2H data in CXL.cache to the first entity.

[0239] FIG. 13A illustrates an example system consisting of an RPU and an optional third cache (Cache.3). The RPU may communicate with the first host (Host.1) using CXL.cache. Host.1 may include the first cache (Cache.1). The RPU may further communicate with the second host (Host.2) using CXL.cache. The host may include the second cache (Cache.2). The RPU is capable of conversion between CXL.cache transactions, or between CXL.cache messages (e.g., between a CXL.cache H2D request and a CXL.cache D2H request, between a CXL.cache H2D response and a CXL.cache D2H response, and even between a CXL.cache H2D Data message and a CXL.cache D2H Data message), which may enable cache line state orchestration among the first cache, the second cache, and optionally the third cache. The cache line state orchestration may be utilized, at least in part, to enable cache-coherent memory sharing between the first entity and the second entity. In some examples, the first entity may issue a CXL.cache H2D request (such as a snoop request) to target a cache managed by the RPU, such as the third cache (Cache.3) included in the RPU. In other examples, the first entity may issue a CXL.cache H2D request (such as a snoop request) to target a cache abstraction maintained by the RPU. In this case, the RPU may function as a proxy for a cache (e.g., the second cache) included in the second entity, and the RPU may use a CXL.cache D2H request to affect the second cache, causing a state transition of a cache line in the second cache.

[0240] Figure 13B shows an example TFD illustrating the RPU's transformation between CXL.cache transactions, where a transaction may include messages such as requests, responses, and optionally data messages. The first entity (Entity.1), for example, the first host (Host.1), can send a CXL.cache H2D request to the RPU. For example, it may intend to cache a snoop request for a cache line in a specific cache state of the first entity. In the first example, the first entity might send a CXL.cache H2D request to the RPU, which consists of SnpData, and this may indicate a snoop request for a cache line that is to be cached in a shared or exclusive state from the first entity. In the second example, the first entity might send a CXL.cache H2D request (SnpInv) to the RPU, which may indicate a snoop request for a cache line that is to be cached in an exclusive state from the first entity. The RPU can translate a CXL.cache H2D request into a CXL.cache D2H request (such as a cache line read request) and send the CXL.cache D2H request to a second entity, intending for it to be cached in a specific cache state. The RPU may provide intent-based translation, such as identifying the intent of the CXL.cache H2D request received from the first entity and using that intent to translate the CXL.cache H2D request into a CXL.cache D2H request.

[0241] In the first example, the RPU translates a CXL.cache H2D request containing SnpData into a CXL.cache D2H request consisting of RdShared, which requests that the cache line read be cached in a shared state. This translation allows the first entity to transition the cache line state to a shared state, creating cache line state orchestration between the cache managed by the first entity and the cache managed by the second entity. In the second example, the RPU might translate a CXL.cache H2D request containing SnpInv into a CXL.cache D2H request containing RdOwnNoData, requesting exclusive ownership of the cache line address from the latter entity. In this case, the cache line maintained by the second entity is invalidated, allowing the first entity to acquire exclusive ownership and exclusively transition the cache line state. As a result, cache line state orchestration occurs between the cache managed by the first entity and the cache managed by the second entity.

[0242] The second entity responds to the CXL.cache D2H request with a CXL.cache H2D response, and communicates the cache line status (e.g., via GO-S, GO-E) from the second entity to the RPU. The RPU converts the CXL.cache H2D response back to a CXL.cache D2H response, communicating the cache line status to the first entity. In some cases, the second entity may also respond to the CXL.cache D2H request with CXL.cache H2D Data. For example, this might occur if the second entity is transferring modified data, or if the CXL.cache D2H request consists of a read opcode such as RdOwn and returns data. The RPU may convert the CXL.cache H2D data back to CXL.cache D2H data and send the CXL.cache D2H data to the first entity. The RPU may also perform further conversions, such as conversions between CXL.cache's H2D request opcodes (Snoops (e.g., SnpData, SnpInv, SnpCur)) and CXL.cache's D2H request opcodes (RdCurr, RdOwn, RdShared, RdAny, RdOwnNoData, ItoMWr, WrCur, CLFlush, CleanEvict, DirtyEvict, CleanEvictNoData, WOWrInv, WOWrInv, WOWrInvF, WrInv, RcFlushed, CacheFlushed). The RPU may also perform other conversions, such as conversions between CXL.cache messages, conversions between reserved fields, and conversions between reserved and unreserved fields.

[0243] Figure 14A shows an example of a system consisting of an RPU communicating with the first entity (Entity.1) using CXL.cache. For example, the first host (Host.1) may contain the first cache (Cache.1). The RPU may also communicate with a second host (Host.2) using CXL.cache. The host may contain the second cache (Cache.2). The RPU may translate between CXL.cache transactions and CXL.cache messages, for example, between an H2D request of CXL.cache containing a snoop and a D2H request of CXL.cache containing a read opcode, and may enable cache line state orchestration between the first and second caches. In the latter, cache line state orchestration is used, at least in part, to enable memory sharing, such as cache-coherent memory sharing, between the first and second entities.

[0244] Figure 14B shows an example of a TFD (Transaction Function Diagram) where the RPU performs translations between CXL.cache transactions, and transactions may include messages such as requests, responses, and optionally data messages. The first entity (Entity.1), for example, the first host (Host.1), can send a CXL.cache H2D request to the RPU consisting of SnpData, Address (AS.1.1), and UQID (t.1.1). Here, the CXL.cache H2D request may represent a snoop request for a cache line from the first entity to the RPU. The cache line is to be cached in a shared or exclusive state by the first entity. The RPU translates the CXL.cache H2D request into a CXL.cache D2H request consisting of RdShared, Address (AS.2.1), and CQID (q.2.1), and the CXL.cache D2H request may represent a read request from the RPU to the second entity to cache the cache line in a shared state. The RPU may provide intent-based translation, such as identifying the intent of a CXL.cache H2D request received from the initial entity (e.g., the intent to obtain shared or exclusive cache line states) and using this information to translate between CXL.cache H2D and CXL.cache D2H requests.

[0245] The second entity responds to the CXL.cache D2H request, which includes RdShared, with a CXL.cache H2D response consisting of Opcode (GO), RspData (S), and CQID (q.2.1), and further CXL.cache H2D Data(CQID(q.2.1) and Data(64B) may respond, where Opcode(GO) and RspData(S) indicate the GO-S shared state of the cache line received by the RPU from the second entity. The RPU converts the H2D response of CXL.cache into a D2H response of CXL.cache consisting of Opcode(RspSHitSE) and UQID(t.1.1), where Opcode(RspSHitSE) indicates that the clean state has been reached and the current state is shared. This creates an orchestration of cache line states between the first and second caches, and both caches may be stored in a shared state. The RPU may also perform other conversions, such as conversions between CXL.cache messages, conversions between reserved fields, and conversions between reserved and unreserved fields. In some examples, the first entity is CXL.cache You might issue H2D requests (such as snoop requests) targeting cache abstractions maintained by the RPU. In this case, the RPU exposes the device cache on CXL.cache, which acts as a proxy for the cache contained in the second entity, and the RPU can use D2H requests on CXL.cache to influence the cache within the second entity and potentially trigger state transitions in the cache lines.

[0246] Figure 14C shows another example of a TFD illustrating the conversion between CXL.cache transactions. The first entity (Entity.1), for example, the first host (Host.1), can send a CXL.cache H2D request to the RPU consisting of SnpData, Address (AS.1.2), and UQID (t.1.2). Here, the CXL.cache H2D request may represent a snoop request for a cache line from the first entity to the RPU. The cache line is to be cached by the first entity in either a shared or exclusive state. The RPU converts the CXL.cache H2D request into a CXL.cache D2H request consisting of RdOwn, Address (AS.2.2), and CQID (q.2.2), and the CXL.cache D2H request may represent a read request to the second entity to cache the cache line in an exclusive state. The RPU may provide intent-based translation, such as identifying the intent of a CXL.cache H2D request received from the initial entity (e.g., the intent to obtain shared or exclusive cache line states) and using this information to translate between CXL.cache H2D and CXL.cache D2H requests. The second entity responds to a CXL.cache D2H request containing RdOwn with a CXL.cache H2D response consisting of Opcode(GO), RspData(E), and CQID(q.2.2), and may further respond with CXL.cache H2D Data(CQID(q.2.2) and Data(64B). Here, Opcode(GO) and RspData(E) may indicate the GO-E exclusive state of the cache line received by the RPU from the second entity. The RPU may convert the CXL.cache H2D response into a CXL.cache D2H response consisting of Opcode(RspIHitI) and UQID(t.1.2).In this case, Opcode(RspIHitI) could indicate that the cache line did not exist in the cache, which could lead to an orchestration of cache line states between the first and second caches, with the first cache potentially exclusively transferring its cache line state to the second cache if the cache line is not present.

[0247] Figure 14D shows an example of a TFD where the RPU performs conversions between CXL.cache transactions, which may include messages such as requests, responses, and optionally data messages. The first entity (Entity.1), for example, the first host (Host.1), can send a CXL.cache H2D request to the RPU consisting of SnpInv, Address (AS.1.1), and UQID (t.1.1). Here, the CXL.cache H2D request may indicate snoop invalidation of the exclusive cache line from the first entity to the RPU. The RPU converts the CXL.cache H2D request to a CXL.cache D2H request consisting of CLFlush, Address (AS.2.1), and CQID (q.2.1), which may indicate a cache line invalidation (flush) request from the RPU to the second entity. The RPU may provide intent-based translation, such as identifying the intent of the CXL.cache H2D request received from the first entity (e.g., the intent to obtain a shared or exclusive state of a cache line) and using it to translate between the CXL.cache H2D request and the CXL.cache D2H request. The second entity may respond to the CXL.cache D2H request consisting of CLFlush with a CXL.cache H2D response consisting of Opcode(GO), RspData(I), and CQID(q.2.1), where Opcode(GO) and RspData(I) indicate GO-I, confirming the invalidation of the cache line. The RPU can translate the CXL.cache H2D response into a CXL.cache D2H response consisting of Opcode(RspI*) and UQID(t.1.1). In this case, Opcode(RspI) could be RspIHitI, RspIHitSE, RspIFwdM, etc., indicating that the line is not in the cache. This can cause cache line state orchestration between the first and second caches, with the first cache exclusively transferring the cache line state.If the second cache does not have that cache line, in some cases the first entity may issue a CXL.cache H2D request (such as a snoop request) targeting a cache abstraction maintained by the RPU. In this case, the RPU exposes a device cache on CXL.cache, which acts as a proxy for the cache contained in the second entity, and the RPU may use the CXL.cache D2H request to influence the cache in the second entity and trigger a state transition of the cache line.

[0248] Figure 15A shows an example TFD illustrating the transformations performed by the RPU between CXL.cache transactions, where transactions may include messages such as requests, responses, and optionally data messages. The first entity (Entity.1), for example, the first host (Host.1), may send a CXL.cache H2D request to the RPU consisting of SnpInv, Address(AS.1.2), and UQID(t.1.2). Here, the CXL.cache H2D request may indicate snoop invalidation of a cache line from the first entity to the RPU, but the cache line is intended to be cached exclusively by the first entity. The RPU translates the CXL.cache H2D request into a CXL.cache D2H request consisting of RdOwnNoData, Address(AS.2.2), and CQID(q.2.2), where the CXL.cache D2H request may indicate an intention to acquire exclusive ownership of the cache line address indicated in the address field. The RPU may provide intent-based translations, such as identifying the intent of the CXL.cache H2D request received from the first entity (e.g., the intent to obtain the shared or exclusive state of the cache line) and using this information to translate between the CXL.cache H2D request and the CXL.cache D2H request. The second entity may respond to the CXL.cache D2H request containing RdOwnNoData with a CXL.cache H2D response consisting of Opcode(GO), RspData(E), and CQID(q.2.2). Here, Opcode(GO) and RspData(E) may indicate the GO-E exclusive state of the cache line received by the RPU from the second entity. The RPU may translate the CXL.cache H2D response into a CXL.cache D2H response consisting of Opcode(RspI*) and UQID(t.1.2).In this case, Opcodes such as RspIHitI, RspIHitSE, and RspIFwdM (RspI*) may indicate that the line does not exist in the cache, which can cause cache line state orchestration between the first and second caches, potentially leading to an exclusive transfer of the cache line state to the second cache if the cache line is not present there.

[0249] Figure 15B shows an example TFD illustrating the transformations performed by the RPU between CXL.cache transactions, where transactions may include messages such as requests, responses, and optionally data messages. The first entity (Entity.1), for example, the first host (Host.1), can send a CXL.cache H2D request to the RPU consisting of SnpInv, Address (AS.1.1), and UQID (t.1.1). Here, the CXL.cache H2D request may indicate snoop invalidation of the cache line from the first entity to the RPU for exclusive use. The RPU translates the CXL.cache H2D request into a CXL.cache D2H request consisting of RdOwn, Address (AS.2.1), and CQID (q.2.1), where the CXL.cache D2H request may indicate a read request from the RPU to the second entity to cache the cache line in an exclusive state. The RPU may provide intent-based translation, such as identifying the intent of a CXL.cache H2D request received from the initial entity (e.g., the intent to obtain shared or exclusive cache line states) and using this information to translate between CXL.cache H2D and CXL.cache D2H requests.

[0250] The second entity responds to a CXL.cache D2H request containing RdOwn with a CXL.cache H2D response consisting of Opcode(GO), RspData(M), and CQID(q.2.1), and may further respond with CXL.cache H2D Data(CQID(q.2.1) and Data(64B). Here, Opcode(GO) and RspData(M) indicate GO-M, and RPU is Modified. The cache line can be received in state. The RPU translates the H2D response of CXL.cache into a D2H response of CXL.cache consisting of Opcode (RspIFwdM) and UQID (t.1.1), and may indicate to the first entity that the snooped cache line has gone from the M (Modified) state to the I (Invalidated) state after reaching the M (Modified) state. This can result in cache line state orchestration between the first and second caches. The first cache can then transition the cache line state to the modified state if the second cache delivers the modified cache line and it is subsequently invalidated.

[0251] Figure 15C shows an example TFD illustrating the RPU's conversion between CXL.cache transactions. The first entity (Entity.1), for example, the first host (Host.1), may send a CXL.cache H2D request to the RPU consisting of SnpInv, Address (AS.1.2), and UQID (t.1.2). Here, the CXL.cache H2D request may indicate snoop invalidation of the cache line from the first entity to the RPU, but the cache line is intended to be cached exclusively by the first entity. The RPU converts the CXL.cache H2D request into a CXL.cache D2H request consisting of RdOwn, Address (AS.2.2), and CQID (q.2.2), and the CXL.cache D2H request may indicate a read request from the RPU to the second entity to cache the cache line exclusively. The RPU may provide intent-based translation, such as identifying the intent of a CXL.cache H2D request received from the initial entity (e.g., the intent to obtain shared or exclusive cache line states) and using this information to translate between CXL.cache H2D and CXL.cache D2H requests.

[0252] The second entity responds to a CXL.cache D2H request containing RdOwn with a CXL.cache H2D response consisting of Opcode(GO), RspData(E), and CQID(q.2.2), and may further respond with CXL.cache H2D Data(CQID(q.2.2) and Data(64B). Here, Opcode(GO) and RspData(E) may indicate the GO-E exclusive state of the cache line received by the RPU from the second entity. The RPU responds to the CXL.cache H2D response with CXL.cache consisting of Opcode(RspI*) and UQID(t.1.2) This may be converted to a D2H response. In this case, Opcodes (RspI*) such as RspIHitI, RspIHitSE, and RspIFwdM may indicate that the line does not exist in the cache, which can cause cache line state orchestration between the first and second caches, potentially leading to an exclusive transfer of the cache line state to the second cache if the cache line does not exist there.

[0253] Figure 16A shows an example of a system including an RPU that communicates with the first entity (Entity.1) using CXL.cache. For example, the first host (Host.1) may include the first cache (Cache.1). The RPU may also communicate with a second host (Host.2) using CXL.cache. The host may include the second cache (Cache.2). The RPU can translate between CXL.cache transactions and even between CXL.cache messages. For example, it may enable cache line state orchestration between the first and second caches between an H2D request of CXL.cache containing a snoop opcode (e.g., SnpData) and a D2H request of CXL.cache containing a read opcode (e.g., RdShared). This may be done, for example, when cross-coordination of cache line states is used. At least part of this is to enable memory sharing between the first and second entities, such as memory sharing for cache consistency. Optionally, the RPU may be implemented in an IC package with high-speed differential I / O balls arranged based on PCIe 5.0, 6.0, or 7.0 retimer supplementation and a ball grid array layout defined in the standard BGA footprint specification. The RPU can extract physical addresses from messages received via the first interface, and these addresses may correspond to the first HPA space used by the first entity. Translate these messages; and generate messages to send via the second interface, carrying the translated physical addresses. These translated addresses may correspond to the second HPA space used by the second entity. An optional CXL switch may be placed between the first interface and the first entity, or between the second interface and the second entity.

[0254] Figure 16B shows an example TFD illustrating the transformations performed by the RPU between CXL.cache transactions, which may include messages such as requests, responses, and optionally data messages. The first entity (Entity.1), for example, the first host (Host.1), can send a CXL.cache H2D request to the RPU consisting of SnpData, Address (AS.1.1), and UQID (t.1.1). Here, the CXL.cache H2D request may represent snooping a cache line from the first entity to the RPU. The cache line is only possible if it is cached in either a shared or exclusive state by the first entity (exclusive is only possible if all devices are cached by the first entity). Please respond with RspI*. The RPU translates a CXL.cache H2D request into a CXL.cache D2H request consisting of RdShared, Address (AS.2.1), and CQID (q.2.1), and the CXL.cache D2H request may indicate a read request from the RPU to a second entity to cache the cache line in a shared state. The RPU may provide intent-based translations, such as identifying the intent of the CXL.cache H2D request received from the first entity (e.g., intent to obtain a shared or exclusive state of the cache line) and using it to translate between the CXL.cache H2D request and the CXL.cache D2H request.

[0255] The second entity responds to a CXL.cache D2H request including RdShared with a CXL.cache H2D response consisting of Opcode(GO), RspData(S), and CQID(q.2.1), and may further respond with CXL.cache H2D Data(CQID(q.2.1) and Data(*Data.1*). Here, Opcode(GO) and RspData(S) indicate GO-S, and the RPU can receive the cache line in shared state. The RPU responds to the CXL.cache H2D response with CXL.cache consisting of Opcode(RspSFwdM) and UQID(t.1.1) The RPU may convert the data to a D2H response, indicating to the first entity that the snooped cache line has reached the M (Modified) state and then entered the S (Shared) state, potentially enabling explicit data delivery from the RPU to the first entity. The RPU may further convert the H2D data in CXL.cache to D2H data in CXL.cache, consisting of UQID(t.1.1) and Data(*Data.1*). When the RPU converts the H2D response in CXL.cache to a D2H response in CXL.cache, it may use *FwdM such as RspSFwdM. In this case, *FwdM enables data transfer from the RPU to the first entity using the CXL.cache D2H data, regardless of the actual GO-S response. Such a conversion would allow shared state cache line data to reach the first entity from the second entity via the RPU, but this path is not supported by the CXL specification.

[0256] Figure 17A shows an example system consisting of a computer connected to a first interface (Interface.1) that can communicate with a first entity (Entity.1) according to CXL.cache, and a second interface (Interface.2) that can communicate with a second entity (Entity.2) according to CXL.cache. This allows the first entity to communicate with the second entity, such as influencing the state transitions of the cache line. The computer can extract physical addresses in messages received through the first interface, and these addresses may point to the first physical address space used by the first entity. Translate these addresses; and generate messages to send via the second interface, carrying the translated physical addresses. These translated addresses may correspond to the second physical address space used by the second entity. In some examples, the first and second address spaces may be associated with a single HPA space. An optional CXL switch may be placed between the first interface and the first entity, or between the second interface and the second entity. In some examples, the computer may be implemented as a chiplet or as a functional unit within an IC such as an accelerator, processor, or switch. In other examples, computers can be implemented as discrete components, such as in IC packages with PCIe retimer supplementation and high-speed differential I / O balls arranged according to a ball grid array layout defined by the standard BGA footprint specification.

[0257] Figure 17B illustrates a transformation that TFD performs as an intent-based transformation by the computer, showing the transformation between the first CXL.cache message received from the first entity (Entity.1) (e.g., the first device (Device.1)) and the message sent to the second entity (Entity.2, e.g., the second device (Device.2)) to the second entity (Entity.2), which may allow the first entity to be maintained. At least part of this is cache line state orchestration, memory coherency, or memory sharing with the second entity (e.g., allowing the first entity to invalidate the cache lines of the second entity). The first entity may initiate the first CXL.cache transaction, which includes an Opcode (CLFlush), CQID (q.2.1), and Address (AS.2.1). The computer then converts the first CXL.cache transaction to a CXL.cache consisting of Opcode (SnpInv), UQID (t.1.1), and Address (AS.1.1). The first CXL.cache transaction can be converted into a second transaction containing the H2D request, and the CXL.cache H2D request can be sent to the second entity. Upon receiving a response from the second entity, if it contains a D2H response in CXL.cache containing Opcode(RspIHitI) and UQID(t.1.1), the computer converts the D2H response in CXL.cache into a CXL.cache H2D response consisting of Opcode(GO-I) and CQID(q.2.1).

[0258] The computer may also perform conversions between CXL.cache's D2H request opcodes (RdCurr, RdOwn, RdShared, RdAny, RdOwnNoData, ItoMWr, WrCur, CLFlush, CleanEvict, DirtyEvict, CleanEvictNoData, WOWrInv, WOWrInvF, WrInv, CacheFlushed, or CacheFlushed) and CXL.cache's H2D request opcodes (such as Snp*). Furthermore, it can perform field conversions between messages compliant with the first CXL.cache transaction, as well as messages compliant with the second CXL.cache transaction (such as conversions between UQID and CQID, conversions between reserved fields, conversions between reserved fields, and conversions between reserved and unreserved fields).

[0259] Figure 17C illustrates an example of the conversions performed by the computer between TFD and CXL.cache transactions, where transactions may include messages such as requests, responses, and optionally data messages. The first entity (Entity.1), for example, the first device (Device.1), may send a CXL.cache D2H request to the computer consisting of Opcode (RdCurr), CQID (q.2.1), and Address (AS.2.1). Here, the CXL.cache D2H request may represent a request from the first entity to the computer to read the latest version of the cache line. The computer may also convert the CXL.cache D2H request into a CXL.cache H2D request consisting of Opcode (SnpCur), UQID (t.1.1), and Address (AS.1.1), and send the CXL.cache H2D request to the second entity. In this case, the CXL.cache H2D request may represent a snoop request from the computer to the second entity. The computer may identify the intent of the CXL.cache D2H request received from the first entity and provide intent-based translations, such as the intent to retrieve the latest version of the cache line and leveraging that intent to translate between the CXL.cache D2H request and the CXL.cache H2D request. The second entity may respond to the CXL.cache H2D request containing SnpCur with a CXL.cache D2H response containing Opcode(RspVFwdV) and UQID(t.1.1), and a CXL.cache D2H Data response containing UQID(t.1.1) and Data(*Data.1*). The computer can translate the CXL.cache D2H data into CXL.cache H2D data (consisting of CQID(q.2.1) and Data(*Data.1*)) and send the CXL.cache H2D data to the first entity.

[0260] In various implementations, there are methods for translating between Compute Express Link (CXL) messages, which include: receiving the first CXL message from the first entity; identifying the cache line state intent based on either the first opcode, Snoop type (SnpType), metadata field (MetaField), or metadata value (MetaValue) of the first CXL message, where the cache line state intent is related to the cache line address specified in the first CXL message; translating the first CXL message into a second CXL message containing a second opcode selected based on the at least partially identified cache line state intent, where the second opcode is different from the first opcode; and sending the second CXL message to another entity. The first entity may include the first host, the second entity may include the second host, and both entities may communicate via CXL. The cache line state intent may indicate a cache state such as Edited, Exclusive, Shared, or Invalid (MESI), which the entity attempts to acquire or influence on the cache line address. For CXL.cache messages, intent can be derived from the opcode, such as SnpData, SnpInv, or SnpCur for H2D requests, or RdOwn, RdShared, or CLFlush for D2H requests. For CXL.mem messages, intent can be derived from the opcode, SnpType, or a combination thereof, where SnpType may indicate a snoop requirement such as No-Ops, SnpInv, or SnpData. By identifying intent from the incoming message, the translation logic selects the appropriate opcode for the outgoing message to achieve the desired cache state transition. This opcode may differ from the incoming opcode due to differences in protocol direction and protocol type. This technique may be performed by an RPU, semiconductor device, or other computing device located between the first and second entities.

[0261] In some implementations of this technique, the first CXL message consists of a Host-to-Device (H2D) request for CXL.cache, with the first opcode selected from SnpData, SnpInv, or SnpCur. The second CXL message then consists of a Request from the CXL.cache device to the host (D2H), with the second opcode selected from RdOwn, RdShared, RdOwnNoData, RdCurr, RdAny, or CLFlush. The conversion from an H2D request to a D2H request for CXL.cache may enable inter-host communication via a device placed between hosts, which can expose a CXL.cache interface to each host. The H2D request consists of snoops such as SnpData, SnpInv, and SnpCur, indicating different cache line state intentions. On the other hand, D2H requests consist of read or cache operation opcodes such as RdOwn, RdShared, RdOwnNoData, RdCurr, and CLFlush, which enable the corresponding cache state transitions.

[0262] In some implementations of this technique, the first opcode consists of SnpData, and the second opcode is selected from RdShared or RdOwn. RdShared is chosen to allow both the first and second entities to hold a shared copy of the cache line, while RdOwn causes the second entity to relinquish ownership, allowing the first entity to exclusively obtain the cache line data. SnpData indicates a snoop request for a cache line intended to be cached in either a shared or exclusive state by the first entity. Translation logic may determine whether it should be shared or exclusive based on additional context, system configuration, and bias state information. RdShared requests the cache line to be cached in a shared state, enabling simultaneous caching by multiple entities. RdOwn requests the cache line to be cached in an exclusive or modified state, with the second entity relinquishing ownership and providing the cache line data to the translation logic for transfer to the first entity.

[0263] In some implementations of this technique, the first opcode consists of SnpInv, and the second opcode is selected from CLFlush, RdOwnNoData, or RdOwn. In this case, CLFlush is selected to invalidate the cache line of the second entity without a data return, RdOwnNoData takes exclusive ownership of the cache line without a data return, and RdOwn takes exclusive ownership of the cache line with a data return from the second entity. SnpInv indicates a snoop request that invalidates the cache line of the receiving device and indicates an intention to obtain exclusive status at the first entity. The translation logic may determine whether data transfer is necessary based on the data cache policy related to the translation logic, the cache state of the first entity, pending write operations, or other context. CLFlush requests a flush of the cache line at the second entity, which may include a data return if the cache line is in the Modified state, and may be selected when the primary objective is to invalidate the cache copy at the second entity. RdOwnNoData requests exclusive ownership without data return and may be chosen when the first entity overwrites the entire cache line. RdOwn requests exclusive ownership with data return and may be chosen when the first entity needs the current cache line contents before modifying it.

[0264] In some implementations of this technique, the first opcode is SnpCur and the second opcode is RdCurr, and this conversion allows the first entity to retrieve the current cache line data from the second entity without changing the cache state of either the first or second entity. SnpCur indicates a snoop request to retrieve the current version of a cache line without changing the cache state in the hierarchy. RdCurr indicates a read request to retrieve the latest data without changing the existing state of the cache. The conversion from SnpCur to RdCurr may allow the first entity to read the current data from the second entity without affecting the cache's coherence state, which can be useful for I / O data manipulation, monitoring, debugging, or inference operations such as I / O coherence mode.

[0265] In some implementations of this technique, the first CXL message consists of a request from the CXL.cache device to the host (D2H), with the first opcode selected from RdOwn, RdOwnNoData, RdShared, RdCurr, or CLFlush. The second CXL message then consists of a request from the CXL.cache host to the device (H2D), with the second opcode selected from SnpData, SnpInv, or SnpCur based on the cache line state intent indicated by the first opcode. The conversion from a CXL.cache D2H request to a CXL.cache H2D request can enable inter-device communication; that is, a read request or cache operation from one device can be converted into a snoop request targeting another device. The translation logic may map RdShared to SnpData when there is a possibility of retrieving shared state and returning data, RdOwn, CLFlush, or RdOwnNoData to SnpInv when there is an intention to take or invalidate exclusive ownership, and RdCurr to SnpCur when there is an intention to access data without changing its state.

[0266] In some implementations of this method, the first CXL message consists of a Subordinate (M2S) request from the CXL.mem Master. The second CXL message consists of a CXL.cache device-to-host (D2H) request, with the second opcode selected from RdOwn, RdOwnNoData, RdShared, RdCurr, RdAny, or CLFlush. The conversion from a CXL.mem M2S request to a CXL.cache D2H request allows a memory access operation from the first entity using CXL.mem to be translated into a cache coherence operation targeting another entity using CXL.cache. Fields within the M2S request, such as SnpType, MetaField, and MetaValue, can indicate the state intent of the cache line and are used to select the appropriate D2H request opcode. The conversion logic can act as a bridge between the CXL.mem and CXL.cache domains while preserving the intent of the memory operation.

[0267] In some implementations of this technique, a CXL.mem M2S request consists of SnpType and MetaValue, and a CXL.cache D2H request consists of a second opcode, which is selected based on SnpType and MetaValue. The combination of SnpType and MetaValue in an M2S request may indicate the cache line's state intent more precisely than SnpType alone. For example, an M2S request where SnpType indicates SnpData and MetaValue indicates Shared (S) might lead to the selection of RdShared. Translation logic can use both fields to determine the appropriate D2H request opcode.

[0268] In some implementations of this technique, a CXL.mem M2S request consists of a second opcode that constitutes a SnpCur, and a CXL.cache D2H request consists of a second opcode that constitutes a RdCurr. The conversion from SnpCur to RdCurr may allow requests for values ​​of the current cache line that are not cacheable, such as I / O coherent reads. The M2S request also includes a MetaValue, which indicates that the requester does not cache that line by showing Invalid(I). The RdCurr can retrieve the latest data from a second entity without causing a cache state transition. Upon receiving a response such as CXL.cache H2D Data, the computer converts that response into a CXL.mem S2M DRS containing MemData and a CXL.mem S2M NDR containing Cmp.

[0269] In some implementations of this technique, a CXL.mem M2S request consists of a second opcode that constitutes SnpType(SnpData), and a CXL.cache D2H request consists of a second opcode that constitutes RdShared. The conversion from SnpData to RdShared may allow the requester to retrieve a cache line in shared state, enabling concurrent caching by multiple entities. The M2S request may also contain a MetaValue that can indicate Shared(S) to indicate that the host has at most a shared copy of that line. Upon receiving a response from the second entity (if it contains a CXL.cache H2D response with GO-S), the computer converts that response into a CXL.mem S2M NDR, providing an index of state that includes Cmp-S.

[0270] In some implementations of this technique, a CXL.mem M2S request consists of a second opcode that constitutes SnpType(SnpInv), and a CXL.cache D2H request consists of a second opcode that constitutes RdOwn. The conversion from SnpInv to RdOwn may allow the requester to acquire exclusive ownership of the cache line, resulting in the second entity relinquishing the cached copy. The M2S request may also contain a MetaValue which may indicate any (A) that the host has a shared, exclusive, or modified copy. Upon receiving a response from the second entity (if it contains a CXL.cache H2D response containing GO-E or GO-M), the computer converts that response to a CXL.mem S2M NDR which contains Cmp-E and may indicate exclusive ownership.

[0271] Some implementations of this technique select a second opcode based on SnpType. CLFlush is selected for SnpType indicating cache line invalidation without ownership, while RdShared is selected for SnpType indicating cache line invalidation without ownership, and SnpData (shared state acquisition). SnpType in an M2S request may indicate SnpInv for invalidation snoops and SnpData for data snoops. If SnpType indicates SnpInv, the translation logic may select RdOwn if data return is needed, RdOwnNoData if sufficient exclusive ownership without data is available, or CLFlush if invalidation is needed. If SnpType indicates SnpData, the translation logic may select RdShared to acquire a cache line in shared state. This mapping ensures that operations on CXL.mem are properly aligned with the integrity of CXL.cache.

[0272] In some implementations of this technique, the first CXL message consists of a device-to-host (D2H) request for CXL.cache, with the first opcode selected from RdOwn, RdShared, RdCurr, RdAny, or CLFlush. The second CXL message then consists of a Master-to-Subordinate (M2S) request for CXL.mem (MemRd* and a second SnpType). The conversion from a D2H request for CXL.cache to a M2S request for CXL.mem allows a cache coherency operation from the first entity using CXL.cache to be translated into a memory access operation targeting another entity using CXL.mem. The opcode of the D2H request can indicate the state intent of the cache line, which can be used to select the appropriate SnpType value for the M2S request. For example, a D2H request containing RdOwn can be converted to an M2S request with SnpType set to SnpInv, and a D2H request containing RdShared can be converted to an M2S request with SnpType set to SnpData.

[0273] In some implementations of this technique, a CXL.cache D2H request consists of a first opcode, and the second SnpType is set based on the cache line state intent indicated by the first opcode. Translation logic may parse the first opcode to identify the cache line state intent and set the second SnpType accordingly. For example, RdOwn indicates an intent to obtain exclusive state, and SnpType is set to SnpInv, while RdShared indicates an intent to obtain shared state, and SnpType is set to SnpData. This intent-based translation allows CXL.cache operations to be properly aligned with the coherence semantics of CXL.mem.

[0274] In some implementations of this technique, the CXL.cache D2H request consists of the first opcode (RdCurr), and the CXL.mem M2S request constitutes a second SnpType which consists of a SnpCur. The conversion from RdCurr to SnpCur may allow a request for the current cache line value which can be cached through the CXL.mem interface. The M2S request may also include a MetaField indicating MS0 and a MetaValue indicating Invalid(I) which indicates that the Request Player cannot cache the row. Upon receiving the response from the second entity, which includes a CXL.mem S2M DRS containing MemData and a CXL.mem S2M NDR containing Cmp, the computer converts the response into CXL.cache H2D data and delivers it to the first entity.

[0275] In some implementations of this technique, a CXL.cache D2H request consists of the first opcode (RdShared), and a CXL.mem M2S request consists of a second SnpType containing SnpData. The conversion from RdShared to SnpData allows the CXL.cache requester to retrieve the shared state cache line through the CXL.mem interface. The M2S request may also include a MetaField indicating MS0 and a MetaValue indicating Shared(S). Upon receiving a response from the second entity, if it contains a CXL.mem S2M NDR providing an indication of the shared state, the computer converts the response into a CXL.cache H2D response (GO-S) and CXL.cache H2D Data (cache line data).

[0276] In some implementations of this technique, a CXL.cache D2H request constitutes the first opcode, RdOwn, and a CXL.mem M2S request constitutes a second SnpType, which includes SnpInv. The conversion from RdOwn to SnpInv allows the CXL.cache requester to acquire exclusive ownership of the cache line through the CXL.mem interface, and typically receives a cache line in the Exclusive or Modified state. The M2S request may also include a MetaField indicating MS0 and a MetaValue indicating any (A). Upon receiving a response from the second entity, which includes a Cmp-E containing a CXL.mem S2M NDR and indicating exclusive ownership, the computer can convert that response into a CXL.cache H2D response containing a GO-E and CXL.cache H2D data containing the cache line data.

[0277] In some implementations, this technique further maintains a mapping between the cache line state intent and the opcode corresponding to the second CXL message. The cache line state intent indicates whether the first entity is attempting to acquire a cache state that allows concurrent caching by other entities, a cache state requiring exclusive ownership, or an attempt to invalidate the cache line by the second entity. The translation then uses the mapping to select the second opcode, achieving the cache state transition indicated by the second entity. The mapping can be implemented as a lookup table, combinatorial logic, or a programmable translation function that associates each cache line state intent with a candidate opcode for the second CXL message. Cache line state intents are categorized into shared state acquisition (allowing concurrent caching), exclusive state acquisition (the second entity is obligated to relinquish its cache copy), and invalidation (the second entity discards its cache copy without transferring ownership). The translation logic may utilize the mapping to select the opcode that achieves the desired cache state transition, taking into account additional context, such as whether a data return is required.

[0278] In some implementations, this technique further involves receiving a response message from a second entity indicating a cache state based on either a third opcode or cache line data contained in the response message. The response message is then translated into a translated response message containing a fourth opcode selected based on the cache state indicated in the response message; here, the fourth opcode is different from the third opcode. The translated response message is then sent to the first entity. The response message may include CXL.cache H2D responses such as GO-M, GO-E, GO-S, GO-I, indicating a modified, exclusive, shared, or invalid (MESI) cache line state, respectively. The translation logic may select an opcode for the translated response message, for example, a CXL.cache D2H response such as RspIHitI, RspSHitSE, or RspIFwdM, to convey the resulting cache line state to the first entity. The translation may utilize previously stored identifier mappings to correctly revert the response back to the original transaction.

[0279] In some implementations of this method, the cache state indicated in the response message can be shared, exclusive, or modified. The fourth opcode is then selected as follows: RspSHitSE is selected when the third opcode indicates GO-S (shared state) to indicate that the cache line is in a shared state; RspSHitSE is selected when the third opcode indicates GO-E (exclusive state) to indicate that the cache line no longer exists in the cache abstraction; RspIFwdM is selected when the third opcode indicates GO-M to indicate that modified data has been transferred; the GO-S indication indicates that the second entity has provided the cache line in a shared state, allowing simultaneous caching by multiple entities, and the translation logic can select RspSHitSE to indicate to the first entity that the cache line has been reached in a clean state and that the current state is shared; the GO-E indication indicates that the second entity has granted exclusive ownership, and the translation logic may select RspIHitI or RspIHitSE to indicate that the cache line no longer exists in the cache abstraction. The GO-M indicator shows that the second entity previously held the cache line in a modified state and has relinquished ownership along with the modified data, and the translation logic can select RspIFwdM to indicate the transfer of the modified data.

[0280] In some implementations, this technique further exposes the cache abstraction to the first entity, where it appears to the first entity as a device cache accessible via CXL transactions. The cache abstraction acts as a proxy for cache resources managed by the second entity, with the first entity issuing CXL messages targeting the cache abstraction, which are then translated into outgoing CXL messages that affect the second entity's actual cache based on the identified cache line state intent. The cache abstraction is exposed as a CXL type 1 or CXL type 2 device interface that the first entity can enumerate and operate as a local device cache. The first entity issues CXL messages, such as snoop requests, targeting the cache abstraction, and translation logic identifies the cache line state intent from these messages and translates it into an appropriate outgoing CXL message that affects the actual cache. This proxy scheme enables cache coherency operations between entities that cannot communicate directly due to protocol direction constraints or address space differences.

[0281] In some implementations of this technique, this translation enables cache line state orchestration between a cache managed by the first entity and a second cache managed by the second entity. Here, cache line state orchestration involves the first entity influencing the cache state of the second cache through a translated second CXL message, indicating transitions between at least two states selected from modified, exclusive, shared, and invalid (MESI) according to the cache coherency protocol. Cache line state orchestration may involve cooperative transitions between various cache states based on MESI or similar cache coherency protocols. The translation logic allows the first entity to invalidate, downgrade, or transfer ownership of a cache line in the second entity's cache hierarchy, translating the first entity's CXL message into the appropriate CXL message.

[0282] In some implementations of this technique, cache line state orchestration further enables cache-consistent memory sharing between the first and second entities, allowing both entities to access shared memory resources while maintaining data integrity. Translation logic may track pending transactions and coordinate state transitions to maintain consistency invariance between both cache tiers, enabling cache-coherent memory sharing in distributed memory systems, multi-GPU clusters, heterogeneous computing platforms, and other multi-entity architectures.

[0283] Some implementations of this method consist of instructions in a non-temporary computer-readable medium that, when executed by a processor, cause the processor to execute the method. Some implementations of this technique involve one or more integrated circuits configured to execute the technique, and these one or more integrated circuits constitute one of the following: (i) a general-purpose processing unit configured or connected via a communication interface (configured to execute the method by software and / or firmware execution), (ii) firmware and / or hardware logic circuits integrated into electronic equipment, which utilize hardware acceleration or special processing capabilities not typically provided by general-purpose processing units, or (iii) an operation involving one or more chiplets within one or more integrated circuit packages. Some implementations of this method involve an active cable consisting of first and second pluggable modules coupled in a physical medium, where the active cable further constitutes hardware circuitry integrated into the active cable and configured to execute the method. Some implementations of this technique involve a device acting as a switch, which consists of switching circuitry and is configured to execute the method.

[0284] In various implementations, the system includes the following: a computer receives the first Compute Express Link (CXL) message from the first entity; it identifies the cache line state intent based on either the first opcode, Snoop type (SnpType), metadata field (MetaField), or metadata value (MetaValue) of the first CXL message; and if the cache line state intent is related to the cache line address specified in the first CXL message, it translates the first CXL message into a second CXL message containing a second opcode selected based on the at least partially identified cache line state intent, where the second opcode differs from the first; and then sends the second CXL message to another entity. The computer includes processing logic, memory to store translation tables and transaction state, and an interface controller to manage CXL communication with each entity. The system may be implemented as a standalone device, integrated into a larger semiconductor component, or distributed across multiple components within a computing platform. Translation capabilities enable a variety of system architectures, including distributed memory systems, multi-host configurations, multi-GPU clusters, and heterogeneous computing platforms.

[0285] In some implementations of the system, the computer is contained within a semiconductor device, with the first CXL.cache interface configured to receive CXL.cache host-to-device (H2D) requests from the first entity, and the second CXL.cache interface configured to send requests from the CXL.cache device to the host. The semiconductor device sits between the first and second entities, translating snoops received from the first entity into translations that read or cache opcodes sent to the second entity based on the identified cache line state intent, thus enabling inter-entity communication. The semiconductor device may be implemented as an ASIC, FPGA, SoC, RPU, or other integrated circuit technology. The first CXL.cache interface can receive H2D requests containing snoops such as SnpData, SnpInv, and SnpCur, and the second CXL.cache interface can send D2H requests containing opcodes such as RdOwn, RdShared, RdOwnNoData, and CLFlush, selected based on the cache line state intent. Each interface includes a physical layer circuit, a link layer controller, and a protocol layer engine specifically designed for CXL.cache communication.

[0286] In some implementations of the system, the first CXL message consists of a subrosive (M2S) request from the CXL.mem master, and the second CXL message consists of a request (D2H) from the CXL.cache device to the host. The computer identifies the cache line state intent based on at least one SnpType or the first opcode and sets up a bridge between CXL.mem and CXL.cache. Then, for the D2H request, it selects an opcode that realizes the corresponding cache state transition in the second entity. This computer can be configured to bridge between the CXL.mem and CXL.cache domains, allowing memory access operations from entities using CXL.mem to be coordinated with cache coherency operations targeting entities using CXL.cache. The system parses the SnpType value to determine whether it indicates SnpInv or SnpData, and selects the corresponding D2H request opcode such as RdOwn, RdOwnNoData, CLFlush, or RdShared to realize the intended cache state transition. This bridging functionality could enable heterogeneous systems where different organizations utilize different CXL protocols.

[0287] Figure 18A shows an example of intent-based translation between a CXL.mem message received by TFD from the first entity (Entity.1) and a CXL.cache message sent to a second entity (Entity.2), such as a second host (Host.2). This allows the first entity to access resources mapped to the address space used by the second entity. The first entity can initiate a CXL.mem transaction, which includes a CXL.mem M2S request containing MemOpcode(MemRd*), SnpType(SnpCur), MetaField(MS0), MetaValue(I), Tag(p.1.1), and Address(AS.1.1), and the CXL.mem M2S request may indicate an intent to request the current value of a non-cacheable value in a cache line. A computer can perform intent-based translation between the CXL.mem domain and the CXL.cache domain, translating a CXL.mem transaction into a CXL.cache transaction. A D2H request for CXL.cache includes Opcode (RdCurr), CQID (q.2.1), and Address (AS.2.1), and may use RdCurr to indicate an intention to claim for a non-cacheable value in the cache line. A computer can send a D2H request for CXL.cache to a second entity. Upon receiving a response from the second entity, if the CXL.cache H2D data includes CQID (q.2.1) and Data (*Data.1*), the computer can convert the CXL.cache H2D data to CXL.mem S2M DRS, which includes Opcode (MemData), Tag (p.1.1), and Data (*Data.1*), and further convert the CXL.cache H2D data to CXL.mem S2M NDR, which includes Opcode (Cmp) and Tag (p.1.1).The computer further performs address translation, translation between CXL.mem tags and CXL.cache CQIDs, translation between reserved fields, opcode translation (e.g., CXL.mem M2S request opcodes (MemRd*, RdOwn, RdShared, RdAny, RdOwnNoData, ItoMWr, WrCur, CLFlush, CleanEvict, DirtyEvict, CleanEvictNoData, WOWrInv, WOWrInv, WOWrInvF, WrInv) or CacheFlushed).

[0288] Figure 18B illustrates an example where TFD demonstrates intent-based conversion, where the initial entity may initiate a CXL.mem transaction, which includes a CXL.mem M2S request containing MemOpcode(MemRd*), SnpType(SnpData), MetaField(MS0), MetaValue(S), Tag(p.4.1), and Address(AS.4.1). The CXL.mem M2S request may indicate an intent to request a shared copy of the cache line. Computers can perform intent-based conversions between the CXL.mem and CXL.cache domains, and can also convert CXL.mem transactions to CXL.cache transactions. A CXL.cache D2H request may include Opcode(RdShared), CQID(q.3.1), and Address(AS.3.1), and the CXL.cache D2H request may indicate an intent to cache the cache line in a shared state using RdShared. The computer can send a D2H request for CXL.cache to another entity. Upon receiving a response from the second entity, this may include CXL.cache H2D Data (CQID(q.3.1) and Data(*Data.2*)), and may also include CXL.cache H2D Response (CQID(q.3.1) and GO-S). The computer converts the CXL.cache H2D data to CXL.mem S2M DRS, which includes Opcode(MemData), Tag(p.4.1), and Data(*Data.2*). It then converts the CXL.cache H2D Response to CXL.mem S2M NDR, which includes Opcode(Cmp-S) and Tag(p.4.1), where Cmp-S can provide instructions from the shared state DCOH to the first entity.

[0289] Figure 18C illustrates an example where TFD demonstrates intent-based conversion, where the first entity initiates a CXL.mem transaction, which may include a CXL.mem M2S request containing MemOpcode(MemRd*), SnpType(SnpInv), MetaField(MS0), MetaValue(A), Tag(p.5.1), and Address(AS.5.1). Here, the CXL.mem M2S request may indicate an intent to request an exclusive copy of the cache line. A computer can perform intent-based conversions between the CXL.mem and CXL.cache domains, and may convert a CXL.mem transaction into a CXL.cache transaction. A CXL.cache D2H request may include Opcode(RdOwn), CQID(q.6.1), and Address(AS.6.1), and the CXL.cache D2H request may indicate a corresponding intent to request that the cache line be cached in an arbitrarily writable state using RdOwn. Typically, a cache line is received in an Exclusive (GO-E) or Modified (GO-M) state, and the computer can send a D2H request for CXL.cache to another entity. Upon receiving a response from the second entity, this may include CXL.cache H2D Data containing CQID (q.6.1) and Data (*Data.3*), and may also include a CXL.cache H2D response containing CQID (q.6.1) and GO-E. The computer can convert the CXL.cache H2D data into a CXL.mem S2M DRS containing Opcode (MemData), Tag (p.5.1), and Data (*Data.3*), and further convert the CXL.cache H2D response into a CXL.mem S2M NDR containing Opcode (Cmp-E) and Tag (p.5.1). Here, Cmp-E may provide instructions from DCOH to the first entity with exclusive ownership.

[0290] Figure 19A shows an example illustrating intent-based translation between a CXL.cache message received by TFD from the first entity (Entity.1) and a CXL.mem message sent to the second entity (Entity.2, e.g., the second device (Device.2)). This allows the first entity to access resources mapped to the address space used by the second entity. The first entity can initiate a CXL.cache transaction, which includes a CXL.cache D2H request containing Opcode (RdCurr), CQID (q.2.1), and Address (AS.2.1). The CXL.cache D2H request may indicate an intent to request a cacheable but current cache line value using RdCurr.

[0291] Computers can perform intent-based conversions between the CXL.cache and CXL.mem domains, and can also convert CXL.cache transactions to CXL.mem transactions. A CXL.mem M2S request includes MemOpcode (MemRd*), SnpType (SnpCur), MetaField (MS0), MetaValue (I), Tag (p.1.1), and Address (AS.1.1). A CXL.mem M2S request may indicate an intent to claim non-cacheable values ​​in a cache line. A computer can send a CXL.mem M2S request to a second entity. Upon receiving a response from the second entity, it may include a CXL.mem S2M DRS containing Opcode (MemData), Tag (p.1.1), and Data (*Data.1*), and possibly a CXL.mem S2M NDR containing Opcode (Cmp) and Tag (p.1.1). The computer converts the CXL.mem S2M DRS into a form consisting of CXL.cache H2D Data (CQID(q.2.1) and Data(*Data.1*)). The computer may also perform conversions such as address translation, conversion between CQID in CXL.cache and CXL.mem tags, conversion between reserved fields, opcode translation (e.g., conversion between D2H request opcodes in CXL.cache, such as RdCurr, RdOwn, RdShared, RdAny, RdOwnNoData, ItoMWr, WrCur, CLFlush, CleanEvict, DirtyEvict, Clean...

Claims

1. The methods for converting between Compute Express Link (CXL) messages include: The first entity receives the first CXL.mem master's subroch (M2S) request. The computer converts the first CXL.mem M2S request into a second CXL.mem M2S request, and if the value of at least one field selected from MemOpcode, tag, or address is different between the first and second CXL.mem M2S requests, Send the second CXL.mem M2S request to the second entity.

2. The method of claim 1 is such that the first entity constitutes a CXL host, and the first CXL.mem M2S request consists of the first physical address belonging to the first host physical address (HPA) space used by the CXL host. The second entity constitutes a CXL device, and the second CXL.mem M2S request consists of the second physical address in the address space exposed by the second entity.

3. The method of claim 1 comprises at least one field consisting of an address, wherein the first CXL.mem M2S request consists of the first physical address belonging to the first host physical address (HPA) space, and the second CXL.mem M2S request consists of the second physical address belonging to the second HPA space used by the second entity.

4. In the method of claim 1, at least one field consists of a MemOpcode and a tag, the first CXL.mem M2S request consists of the first opcode and the first tag, and the second CXL.mem M2S request consists of the second opcode and the second tag.

5. The method of claim 4, namely, selecting the first opcode from MemRd, MemRdData, MemRdTEE, and MemSpecRd. The second opcode is selected from MemInv, MemRd, MemRdData, MemRdTEE, MemRdDataTEE, MemInvTEE, MemSpecRd, MemInvNT, MemInvP, MemClnEvict, MemInvPTEE, MemSpecRdTEE, MemClnEvictTEE, or MemClnEvictU.

6. The method of claim 1 further initiated a third CXL.mem M2S request, sending a third CXL.mem M2S request to the second entity.

7. The method of claim 6, i.e., the third CXL.mem M2S request is MemSpecRd; or the third CXL.mem M2S request includes MemRd* and further includes receiving a CXL.mem S2M DRS from the second entity that includes MemData.

8. The method of claim 6 further includes detecting a sequential access pattern at the physical addresses of the CXL.mem M2S requests received from the first entity and initiating a third CXL.mem M2S request targeting the next consecutive physical addresses.

9. The method of claim 6 further includes detecting the stride access pattern of the physical addresses of the CXL.mem M2S request received from the first entity, calculating the stride distance between the accessed addresses, and initiating a third CXL.mem M2S request targeting the physical addresses offset by the stride distance.

10. The method of claim 1 includes at least one field consisting of an address, where the first CXL.mem M2S request consists of the first physical address and the first MemSpecRd, and the second CXL.mem M2S request consists of the second physical address and the second MemSpecRd. Furthermore, it also includes translating the first physical address to the second physical address.

11. The method of claim 10 is characterized in that at least one field further constitutes a mem opcode, the value of which differs in the first CXL.mem M2S request; or the first CXL.mem M2S request conforms to the first CXL specification revision and the second CXL.mem M2S request conforms to the second CXL specification revision; further comprising the computer exposing a CXL Type-2 or CXL Type-3 device to the first entity through the first interface and exposing the root port to the second entity via the second interface.

12. The method of claim 1 comprises at least one field consisting of an address and a tag; where the first CXL.mem M2S request consists of MemRd*, the first tag, and the first physical address. The second CXL.mem M2S request consists of the second tag and the second physical address. Furthermore, the first CXL.mem Sub-field to Master Data Response (S2M DRS) is received from the second entity, which includes the second tag. The first CXL.mem S2M DRS is translated into a second CXL.mem S2M DRS which comprises the first tag. The second CXL.mem S2M DRS is then sent to the first entity.

13. The method of claim 1, i.e., the second entity includes a global fabric-attached memory (G-FAM) or a global fabric-attached memory device (GFD), and there is no CXL switch between the computer and the second entity; further including receiving a third CXL.mem M2S request from a third party, converting the third CXL.mem M2S request into a fourth CXL.mem M2S request, and sending the fourth CXL.mem M2S request to the second entity, and there is no CXL switch between the computer and the third party.

14. In the method of claim 13, the third entity constitutes the second CXL host, the second entity constitutes the CXL device, and no CXL switch is placed between the third entity and the second entity.

15. The method of claim 1 further involves receiving a third CXL.mem M2S request from a third entity, translating the third CXL.mem M2S request into a fourth CXL.mem M2S request, and transmitting the fourth CXL.mem M2S request to the second entity; in this case, the second entity exposes its memory, and there is no CXL switch between the computer and the second entity.

16. It consists of instructions in a non-temporary computer-readable medium that, when executed by a processor, perform the method of claim 1.

17. One or more integrated circuits configured to perform the method of claim 1, wherein one or more integrated circuits include any of the following: (i) a general-purpose processing unit including a communication interface and configured to perform the method by software and / or firmware execution; (ii) firmware and / or hardware logic circuits integrated into an electronic device, which utilize hardware acceleration or special processing capabilities not typically provided by the general-purpose processing unit; or (iii) the operation of mounting one or more chiplets in one or more integrated circuit packages.

18. An active cable comprising first and second pluggable modules coupled by a physical medium; wherein the active cable further includes hardware circuitry integrated into the active cable and configured to perform the method of claim 1.

19. A device configured to operate as a switch, comprising a switching circuit, and configured to perform the method of claim 1.

20. This method enables multi-host access to Compute Express Link (CXL) devices and includes the following: The first entity receives a subrocket (M2S) request from the master of CXL.mem, which includes the first physical address; The second entity receives a second CXL.mem M2S request containing the second physical address, The process of translating the first and second physical addresses to the third and fourth physical addresses within the address space used by the CXL device. This generates the third and fourth CXL.mem M2S requests, consisting of the third and fourth physical addresses, and The third and fourth CXL.mem M2S requests are sent to the CXL device.

Images