Computing system and method for performing remote direct memory access therein

Abstract

Computing systems and methods for performing remote direct memory access in a computing system are provided. In some embodiments, the system includes a storage program processing circuit, a network interface circuit, a cache coherence switch, and a first memory module. In some embodiments, the first memory module is connected to the cache coherence switch, the cache coherence switch is connected to the network interface circuit, and the storage program processing circuit is connected to the cache coherence switch.

Description

Technical Field

[0001] One or more aspects of embodiments of this disclosure relate to computing systems, and more specifically, to systems and methods for managing memory resources in a system comprising one or more servers. Background Technology

[0002] This background section is intended to provide context only, and the disclosure of any embodiment or concept in this section does not constitute an admission that the embodiment or concept is prior art.

[0003] Some server systems may include a collection of servers connected by network protocols. Each server in such a system may include processing resources (e.g., processors) and memory resources (e.g., system memory). In some cases, it may be advantageous for the processing resources of one server to access the memory resources of another server, and it may be advantageous to make such access while minimizing the processing resources of either server.

[0004] Therefore, there is a need for improved systems and methods for managing memory resources in systems that include one or more servers. Summary of the Invention

[0005] In some embodiments, the server includes one or more processing circuitry, system memory, and one or more memory modules connected to the processing circuitry via a cache-coherent switch. The cache-coherent switch may also be connected to one or more network interface circuitry and may include a controller (e.g., a field-programmable gate array (FPGA) or application-specific integrated circuit (ASIC)) to provide enhanced capabilities. These capabilities may include virtualizing the memory modules, enabling the switch to use underlying technologies well-suited to the storage requirements (e.g., latency, bandwidth, persistence) of the data being stored. The cache-coherent switch may receive these storage requirements as a result of them being transmitted to it by the processing circuitry or as a result of monitoring access patterns. Enhanced capabilities may also include enabling the server to interact with the memory of another server without accessing the processor (e.g., by performing remote direct memory access (RDMA)).

[0006] According to an embodiment of the present invention, a system is provided, comprising: a first server, the first server comprising: a stored-program processing circuit, a network interface circuit, a cache coherence switch, and a first memory module, wherein: the first memory module is connected to the cache coherence switch, the cache coherence switch is connected to the network interface circuit, and the stored-program processing circuit is connected to the cache coherence switch.

[0007] In some embodiments, the system further includes a second memory module connected to a cache coherence switch, wherein the first memory module includes volatile memory and the second memory module includes persistent memory.

[0008] In some embodiments, the cache coherence switch is configured to virtualize a first memory module and a second memory module.

[0009] In some embodiments, the first memory module includes flash memory, and the cache coherence switch is configured to provide a flash translation layer to the flash memory.

[0010] In some embodiments, the cache coherence switch is configured to: monitor the access frequency of a first memory location in a first memory module; determine that the access frequency exceeds a first threshold; and copy the contents of the first memory location to a second memory location in a second memory module.

[0011] In some embodiments, the second memory module includes a high-bandwidth memory (HBM).

[0012] In some embodiments, the cache coherence switch is configured to maintain a table for mapping processor-side addresses to memory-side addresses.

[0013] In some embodiments, the system further includes a second server and a network switch connected to the first server and the second server.

[0014] In some embodiments, the network switch includes a top-of-rack (ToR) Ethernet switch.

[0015] In some embodiments, the cache coherence switch is configured to receive remote direct memory access (RDMA) requests and send RDMA responses.

[0016] In some embodiments, the cache coherence switch is configured to receive remote direct memory access (RDMA) requests via a ToR Ethernet switch and a network interface circuit, and to send RDMA responses via the ToR Ethernet switch and the network interface circuit.

[0017] In some embodiments, the cache-coherent switch is configured to support the ComputeExpress Link (CXL) protocol.

[0018] In some embodiments, the first server includes an expansion socket adapter connected to the first server, the expansion socket adapter including: the cache coherent switch; and a memory module socket, through which a first memory module is connected to the cache coherent switch.

[0019] In some embodiments, the memory module socket includes an M.2 socket.

[0020] In some embodiments, the network interface circuitry is on the expansion port adapter.

[0021] According to an embodiment of the present invention, a method for performing remote direct memory access in a computing system is provided, the computing system including: a first server and a second server, the first server including: a stored program processing circuit, a network interface circuit, a cache coherence switch and a first memory module, the method including: receiving a remote direct memory access (RDMA) request by the cache coherence switch and sending an RDMA response by the cache coherence switch.

[0022] In some embodiments, the computing system further includes an Ethernet switch, and receiving an RDMA request includes receiving an RDMA request through the Ethernet switch.

[0023] In some embodiments, the method further includes: receiving a read command for a first memory address from the stored procedure processing circuit by the cache coherence switch; converting the first memory address to a second memory address by the cache coherence switch; and retrieving data from the first memory module at the second memory address by the cache coherence switch.

[0024] In some embodiments, the method further includes: receiving data by the cache coherence switch, storing the data in the first memory module by the cache coherence switch, and sending a command to the stored procedure processing circuit by the cache coherence switch to invalidate a cache line.

[0025] According to an embodiment of the present invention, a system is provided, comprising: a first server, the first server including: a stored program processing circuit, a network interface circuit, a cache coherence switching device, and a first memory module, wherein: the first memory module is connected to the cache coherence switching device, the cache coherence switching device is connected to the network interface circuit, and the stored program processing circuit is connected to the cache coherence switching device. Attached Figure Description

[0026] The accompanying drawings provided herein are merely illustrative of certain embodiments; other embodiments not explicitly shown are not excluded from the scope of this disclosure.

[0027] These and other features and advantages of this disclosure will be understood and appreciated by referring to the specification, claims and drawings, in which:

[0028] Figure 1A This is a block diagram of a system for attaching memory resources to computing resources using cache-coherent joins according to embodiments of the present disclosure.

[0029] Figure 1B This is a block diagram of a system using an expansion socket adapter for attaching memory resources to computing resources using cache-coherent connections, according to embodiments of the present disclosure.

[0030] Figure 1C This is a block diagram of a system for using an Ethernet ToR switch to aggregate memory according to embodiments of the present disclosure;

[0031] Figure 1D This is a block diagram of a system for aggregating memory using an Ethernet ToR switch and an expansion socket adapter, according to embodiments of the present disclosure.

[0032] Figure 1E This is a block diagram of a system for a converged memory according to an embodiment of the present disclosure;

[0033] Figure 1F This is a block diagram of a system for aggregating memory using an expansion socket adapter, according to embodiments of the present disclosure.

[0034] Figure 1G This is a block diagram of a system for a disaggregate server according to embodiments of the present disclosure;

[0035] Figure 2A According to embodiments of this disclosure Figure 1A-1G A flowchart of an example method for performing remote direct memory access (RDMA) transfers by bypassing processing circuitry, as shown in the embodiment;

[0036] Figure 2B According to embodiments of this disclosure, such as Figure 1A-1D The flowchart shown is an example method for performing RDMA transfer with the participation of processing circuitry.

[0037] Figure 2C According to embodiments of this disclosure, such as Figure 1E and Figure 1F The flowchart shown is an example method for performing RDMA transfers via a Compute Express Link (CXL) switch, as illustrated in this embodiment; and

[0038] Figure 2D According to embodiments of this disclosure, such as Figure 1G The illustrated embodiment is a flowchart of an example method for performing RDMA transfers via a CXL switch. Detailed Implementation

[0039] The detailed description set forth below with reference to the accompanying drawings is intended as a description of exemplary embodiments of systems and methods for managing memory resources provided in accordance with this disclosure, and is not intended to represent the only form in which this disclosure may be constructed or utilized. This description illustrates features of the disclosure in conjunction with the illustrated embodiments. However, it should be understood that the same or equivalent functionality and structure may be implemented through different embodiments, which are also intended to be included within the scope of this disclosure. As shown elsewhere herein, the same element numbers are intended to indicate the same elements or features.

[0040] Peripheral Component Interconnect Express (PCIe) refers to a computer interface that can have relatively high and variable latency, limiting its use in connecting to memory. Compute Fast Link (CXL) is an open industry standard for communication over PCIe 5.0. It offers fixed, relatively short packet sizes, thus providing relatively high bandwidth and relatively low fixed latency. Therefore, CXL can support cache coherency, and it is well-suited for establishing connections to memory. CXL can also be used to provide connectivity between accelerators, memory devices, and network interface circuitry (or "network interface controller" or "network interface card, NIC") in hosts and servers.

[0041] Cache coherence protocols (such as CXL) can also be used for heterogeneous processing, such as in scalar, vector, and buffered memory systems. CXL can be used to provide a cache-coherent interface by leveraging channels, retimers, the system's PHY layer, logical aspects of the interface, and protocols from PCIe 5.0. The CXL transaction layer can include three multiplexed sub-protocols that run concurrently on a single link and can be referred to as CXL.io, CXL.cache, and CXL.memory. CXL.io can include I / O semantics, which can be similar to PCIe. CXL.cache can include cache semantics, and CXL.memory can include memory semantics; both cache and memory semantics are optional. Like PCIe, CXL can support: (i) native widths of x16, x8, and x4, which can be partitioned; (ii) a data rate of 32GT / s, which can be reduced to 8GT / s and 16GT / s, 128b / 130b; (iii) 300W (75W in x16 connectors); and (iv) plug and play. To support plug and play, a PCIe or CXL device link can begin training in PCIe under Gen1, negotiate CXL, complete Gen 1–5 training, and then begin a CXL transaction.

[0042] In some embodiments, as discussed in further detail below, the use of CXL connections that aggregate or “pool” memory (e.g., a number of memory units comprising multiple interconnected storage cells) in a system comprising multiple servers connected together via a network can offer various advantages. For example, in addition to providing packet-switching functionality for CXL packets, CXL switches with additional capabilities (referred to herein as “enhanced capability CXL switches”) can be used to connect aggregated memory to one or more central processing units (CPUs) (or “central processing circuitry”) and one or more network interface circuitry (which may have enhanced capabilities). Such a configuration allows (i) the aggregation of memory to include various types of memory with different characteristics, (ii) the enhanced capability CXL switch to virtualize the aggregation of memory and store data with different characteristics (e.g., access frequency) in appropriate types of memory, and (iii) the enhanced capability CXL switch to support Remote Direct Memory Access (RDMA), thereby enabling RDMA operations to be performed with little or no involvement of the server’s processing circuitry. As used herein, “virtualizing” memory means performing memory address translation between processing circuitry and memory.

[0043] CXL switches can (i) support dis-aggregation of memory and accelerators through single-level switching, (ii) enable resources to be offline and online across domains as needed, enabling time-division multiplexing across domains, and (iii) support virtualization of downstream ports. CXL can be used to implement aggregated memory, enabling one-to-many and many-to-one switching (e.g., it can (i) connect multiple root ports to one endpoint, (ii) connect one root port to multiple endpoints, or (iii) connect multiple root ports to multiple endpoints). In some embodiments, the aggregated devices are partitioned into multiple logical devices, each with a corresponding LD-ID (logical device identifier). In such embodiments, physical devices can be partitioned into multiple logical devices, each visible to its respective initiator. A device can have one physical function (PF) and multiple (e.g., 16) separate logical devices. In some embodiments, the number of logical devices (e.g., the number of partitions) may be limited (e.g., up to 16), and there may also be a control partition (which may be a physical function for controlling the devices).

[0044] In some embodiments, the fabric manager can be used to (i) perform device discovery and virtual CXL software creation, and (ii) bind virtual ports to physical ports. This fabric manager can operate via a connection on the SMBus sideband. The fabric manager can be implemented in hardware, software, firmware, or a combination thereof, and can reside, for example, in a host, in one of the memory modules 135, in the enhanced capability CXL switch 130, or elsewhere in the network. The fabric manager can issue commands, including commands issued via the sideband bus or via the PCIe tree.

[0045] refer to Figure 1AIn some embodiments, the server system includes multiple servers 105 connected together via a top-of-rack (ToR) Ethernet switch 110. While the switch is described as using the Ethernet protocol, any other suitable network protocol may be used. Each server includes one or more processing circuits 115, each connected to (i) system memory 120 (e.g., Double Data Rate (version 4) (DDR4) memory or any other suitable memory), (ii) one or more network interface circuits (NICs) 125, and (iii) one or more CXL memory modules 135. Each of the processing circuits 115 may be stored program processing circuitry, such as a central processing unit (CPU) (e.g., an x86 processor), a graphics processing unit (GPU), or an Advanced RISC-Machine (ARM) processor. In some embodiments, the network interface circuit 125 may be embedded within one of the memory modules 135 (e.g., embedded on the same semiconductor chip as one of the memory modules 135 or on the same module as one of the memory modules 135), or the network interface circuit 125 may be packaged separately from the memory module 135.

[0046] As used herein, a “memory module” is a package (e.g., a package including a printed circuit board and components connected thereto, or a package including a printed circuit board) comprising one or more memory dies, each memory die comprising multiple memory cells. Each memory die or group of memory dies in a set of memory dies may be in a package (e.g., an epoxy mold compound (EMC) package) soldered to (or connected to the printed circuit board of the memory module via a connector) a printed circuit board of the memory module. Each of memory modules 135 may have a CXL interface and may include a controller 137 (e.g., an FPGA, ASIC, processor, etc.) for translating between the CXL group and the memory interface of the memory die, for example, signals of a memory technology suitable for the memory in memory module 135. As used herein, a “memory interface” of a memory die is an interface inherent to the technology of the memory die; for example, in the case of DRAM, the memory interface may be word lines and bit lines. The memory module may also include the controller 137 providing enhanced capabilities, as described in further detail below. The controller 137 of each memory module 135 can be connected to the processing circuitry 115 via a cache coherence interface (e.g., via a CXL interface). The controller 137 can also facilitate data transfers (e.g., RDMA requests) between different servers 105, bypassing the processing circuitry 115. The ToR Ethernet switch 110 and network interface circuitry 125 may include an RDMA interface to facilitate RDMA requests between CXL memory devices on different servers (e.g., the ToR Ethernet switch 110 and network interface circuitry 125 can provide hardware offloading or hardware acceleration for RDMA over Converged Ethernet (RoCE), Infiniband, and iWARP packets).

[0047] The CXL interconnect in this system can conform to cache coherence protocols, such as the CXL 1.1 standard, or in some embodiments, the CXL 2.0 standard, a future version of CXL, or any other suitable protocol (e.g., a cache coherence protocol). As shown, memory module 135 can be directly attached to processing circuitry 115, and the rack-top Ethernet switch 110 can be used to scale the system to a larger size (e.g., with a larger number of servers 105).

[0048] In some embodiments, such as Figure 1AAs shown, each server may be equipped with multiple directly attached CXL attached memory modules 135. Each memory module 135 may expose a set of base address registers (BARs) as a memory range to the host's Basic Input / Output System (BIOS). One or more of the memory modules 135 may include firmware to transparently manage their memory space mapped by the host operating system (OS). Each of the memory modules 135 may include one or a combination of memory technologies, including, but not limited to, Dynamic Random Access Memory (DRAM), not-AND (NAND) flash memory, High Bandwidth Memory (HBM), and Low-Power Double Data Rate Synchronous Dynamic Random Access Memory (LPDDR SDRAM) technologies, and may also include a cache controller or separate individual controllers for different technology memory devices (for memory modules 135 that combine several memory devices of different technologies). Each memory module 135 may include different interface widths (x4 to x16) and may be constructed according to any of a variety of related form factors (e.g., U.2, M.2, half height, half length (HHHL), full height, half length (FHHL), E1.S, E1.L, E3.S, and E3.H).

[0049] In some embodiments, as described above, the enhanced capability CXL switch 130 includes an FPGA (or ASIC) controller 137 and provides additional features beyond CXL packet switching. The controller 137 of the enhanced capability CXL switch 130 can also act as a management device for the memory module 135 and assist host control plane processing, and it can implement rich control semantics and statistics. The controller 137 may include additional "backdoor" (e.g., gigabit Ethernet, GbE) network interface circuitry 125. In some embodiments, the controller 137 presents itself as a CXL type 2 device to the processing circuitry 115, enabling the issuance of a cache invalidation instruction to the processing circuitry 115 upon receiving a remote write request. In some embodiments, DDIO (Data Direct I / O) technology is enabled, and remote data is first pulled to the last level cache (LLC) of the processing circuitry and then written (from the cache) to the memory module 135. As used in this article, a “Type 2” CXL device is a device that is capable of initiating transactions and implementing optional consistent caching and host-managed device memory, and for the applicable transaction types, it includes all CXL.cache and all CXL.mem transactions.

[0050] As described above, one or more of the memory modules 135 may include persistent memory or "permanent storage" (i.e., storage in which data is not lost when external power is disconnected). If the memory module 135 is presented as a persistent device, the controller 137 of the memory module 135 can manage persistent domains; for example, it can (e.g., due to an application calling a corresponding operating system function) store data identified by the processing circuitry 115 as requiring persistent storage in persistent memory. In such embodiments, a software API can cache and flush data to persistent memory.

[0051] In some embodiments, direct memory transfer from network interface circuitry 125 to memory module 135 is enabled. This transfer can be a one-way transfer to remote memory for fast communication in a distributed system. In such embodiments, memory module 135 can expose hardware details to network interface circuitry 125 in the system to enable faster RDMA transfers. In such systems, two scenarios may occur depending on whether Direct Data Input / Output (DDIO) of processing circuitry 115 is enabled or disabled. DDIO can enable direct communication between the Ethernet controller or Ethernet adapter and the cache of processing circuitry 115. If DDIO of processing circuitry 115 is enabled, the destination of the transfer can be the last-level cache of the processing circuitry, from which data can then be automatically flushed to memory module 135. If DDIO of processing circuitry 115 is disabled, memory module 135 can operate in device-bias mode to force the destination memory module 135 to receive access directly (in the absence of DDIO). This RDMA transfer can be enabled using an RDMA-enabled network interface circuit 125 with a host channel adapter (HCA), buffers, and other processing, which can bypass the destination memory buffer transfer that may exist in other modes of RDMA transfer. For example, in such an embodiment, the use of a bounce buffer (e.g., a buffer in a remote server when the final destination in memory is in an address range not supported by the RDMA protocol) can be avoided. In some embodiments, RDMA uses an alternative physical media option besides Ethernet (e.g., a switch configured to handle other network protocols). Examples of inter-server connections that can enable RDMA include (but are not limited to) Infiniband, RDMA over Converged Ethernet (RoCe) (which uses the Ethernet User Datagram Protocol (UDP)), and iWARP (which uses the Transmission Control Protocol / Internet Protocol (TCP / IP)).

[0052] Figure 1B It shows the relationship with Figure 1AA system similar to the one described above, in which processing circuitry 115 is connected to network interface circuitry 125 via memory module 135. Memory module 135 and network interface circuitry 125 reside on expansion adapters 140. Each expansion adapter 140 can be plugged into an expansion socket 145 on the motherboard of server 105, such as an M.2 connector. Thus, the server can be any suitable (e.g., industry-standard) server, modified by installing expansion adapters 140 in the expansion socket 145. In such an embodiment, (i) each network interface circuit 125 can be integrated into one of the corresponding ones in memory module 135, or (ii) each network interface circuit 125 can have a PCIe interface (the network interface circuit 125 can be a PCIe endpoint (i.e., a PCIe slave device) such that the processing circuitry 115 to which it is connected (which can operate as a PCIe master device or "root port") can communicate with it via a root port to endpoint PCIe connection, and the controller 137 of memory module 135 can communicate with it via a peer-to-peer PCIe connection.

[0053] According to embodiments of the present invention, a system is provided, comprising: a first server, the first server including: a stored program processing circuit, a first network interface circuit, and a first memory module, wherein: the first memory module includes: a first memory die and a controller, the controller being connected to the first memory die via a memory interface, connected to the stored program processing circuit via a cache coherence interface, and connected to the first network interface circuit. In some embodiments: the first memory module further includes a second memory die, the first memory die including volatile memory, and the second memory die including persistent memory. In some embodiments, the persistent memory includes NAND flash memory. In some embodiments, the controller is configured to provide a flash translation layer for the persistent memory. In some embodiments, the cache coherence interface includes a compute fast link (CXL) interface. In some embodiments, the first server includes an expansion adapter connected to an expansion socket of the first server, the expansion adapter including: the first memory module; and the first network interface circuit. In some embodiments, the controller of the first memory module is connected to the stored program processing circuit via the expansion socket. In some embodiments, the expansion socket includes an M.2 socket. In some embodiments, the controller of the first memory module is connected to the first network interface circuit via a peer-to-peer peripheral interconnect fast (PCIe) connection. In some embodiments, the system further includes: a second server, and a network switch connected to the first server and the second server. In some embodiments, the network switch includes a top-of-rack (ToR) Ethernet switch. In some embodiments, the controller of the first memory module is configured to receive a direct remote direct memory access (RDMA) request and send a direct RDMA response. In some embodiments, the controller of the first memory module is configured to receive a direct remote direct memory access (RDMA) request through the network switch and through a first network interface circuit, and to send a direct RDMA response through the network switch and through the first network interface circuit. In some embodiments, the controller of the first memory module is configured to: receive data from the second server; store the data in the first memory module; and send a command to the stored program processing circuit for invalidating cache lines. In some embodiments, the controller of the first memory module includes a field-programmable gate array (FPGA) or an application-specific integrated circuit (ASIC). According to embodiments of the present invention, a method for performing remote direct memory access in a computing system is provided, the computing system including: a first server and a second server, the first server including: stored program processing circuitry, network interface circuitry, and a first memory module including a controller, the method including: receiving a remote direct memory access (RDMA) request by the controller of the first memory module; and sending a direct RDMA response by the controller of the first memory module.In some embodiments, the computing system further includes an Ethernet switch connected to a first server and a second server, and receiving a direct RDMA request includes receiving a direct RDMA request through the Ethernet switch. In some embodiments, the method further includes: receiving a read command for a first memory address from a stored-program processing circuit by a controller of the first memory module; converting the first memory address into a second memory address by the controller of the first memory module; and retrieving data from the first memory module at the second memory address by the controller of the first memory module. In some embodiments, the method further includes: receiving data by the controller of the first memory module; storing the data in the first memory module by the controller of the first memory module; and sending a command to the stored-program processing circuit to invalidate a cache line by the controller of the first memory module. According to embodiments of the present invention, a system is provided, including: a first server, the first server including: a stored-program processing circuit, a first network interface circuit, and a first memory module, wherein: the first memory module includes: a first memory die and a controller device, the controller device being connected to the first memory die via a memory interface, connected to the stored-program processing circuit via a cache coherence interface, and connected to the first network interface circuit.

[0054] refer to Figure 1C In some embodiments, the server system includes multiple servers 105 connected together via a top-of-rack (ToR) Ethernet switch 110. Each server includes one or more processing circuits 115, each connected to: (i) system memory 120 (e.g., DDR4 memory); (ii) one or more network interface circuits 125; and (iii) an enhancement capability CXL switch 130. The enhancement capability CXL switch 130 may be connected to multiple memory modules (MEMs) 135. That is, Figure 1C The system includes a first server 105, which includes a stored program processing circuit 115, a network interface circuit 125, a cache coherence switch 130, and a first memory module 135. Figure 1C In the system, the first memory module 135 is connected to the cache coherence switch 130, the cache coherence switch 130 is connected to the network interface circuit 125, and the stored program processing circuit 115 is connected to the cache coherence switch 130.

[0055] Memory modules 135 may be grouped by type, form factor, or technology type (e.g., DDR4, DRAM, LDPPR, High Bandwidth Memory (HBM), or NAND flash memory, or other persistent storage (e.g., solid-state drives containing NAND flash memory). Each memory module may have a CXL interface and include interface circuitry for translating signals between the CXL group and those suitable for the memory in memory module 135. In some embodiments, this interface circuitry is replaced by an enhanced capability CXL switch 130, and each of the memory modules 135 has an interface that serves as an inherent interface to the memory in memory module 135. In some embodiments, the enhanced capability CXL switch 130 is integrated into the memory module 135 (e.g., integrated into an M.2 form factor package having the memory module 135 or integrated into a single integrated circuit having other components of the memory module 135).

[0056] The ToR Ethernet switch 110 may include interface hardware to facilitate RDMA requests between aggregated memory devices on different servers. The enhanced capability CXL switch 130 may include one or more circuits (e.g., it may include an FPGA or ASIC) to (i) route data to different memory types based on workload, (ii) virtualize host addresses to device addresses, and / or (iii) facilitate RDMA requests between different servers to bypass the processing circuitry 115.

[0057] The memory modules 135 can be housed in an expansion box (e.g., in the same rack as the chassis housing the motherboard), which may include a predetermined number (e.g., more than 20 or more than 100) of memory modules 135, each inserted into a suitable connector. These modules may be M.2 form factors, and the connectors may be M.2 connectors. In some embodiments, the connections between servers are via different networks other than Ethernet; for example, they may be wireless connections such as WiFi or 5G. Each processing circuit may be an x86 processor or another processor, such as an ARM processor or a GPU. The PCIe link on which the CXL link is instantiated can be PCIe 5.0 or another version (e.g., an earlier version or a later (e.g., future) version, such as PCIe 6.0). In some embodiments, a different cache coherence protocol, either in place of CXL or other than CXL, is used in the system, and a different cache coherence switch, either in place of the enhanced capability CXL switch 130 or other than the enhanced capability CXL switch 130, can be used. This cache coherence protocol can be another standard protocol or a cache coherence variant of a standard protocol (in a manner similar to where CXL is a variant of PCIe 5.0). Examples of standard protocols include, but are not limited to, non-volatile dual in-line memory module (version P) (NVDIMM-P), Cache Coherent Interconnect for Accelerator (CCIX), and Open Coherent Accelerator Processor Interface (OpenCAPI).

[0058] System memory 120 may include, for example, DDR4 memory, DRAM, HBM, or LDPPR memory. Memory module 135 may be partitioned or contain a cache controller to handle various memory types. Memory module 135 may be in different form factors, examples of which include, but are not limited to, HHHL, FHHL, M.2, U.2, mezzanine card, daughter card, E1.S, E1.L, E1.L, and E3.S.

[0059] In some embodiments, the system implements an aggregated architecture comprising multiple servers, each aggregated with multiple memory modules 135 attached to the CXL. Each of the memory modules 135 may contain multiple partitions, which may be individually exposed as memory devices to multiple processing circuits 115. Each input port of the enhanced capability CXL switch 130 may independently access multiple output ports of the enhanced capability CXL switch 130 and the memory modules 135 connected thereto. As used herein, an “input port” or “upstream port” of the enhanced capability CXL switch 130 is a port connected to (or adapted to connect to) a PCIe root port, and an “output port” or “downstream port” of the enhanced capability CXL switch 130 is a port connected to (or adapted to connect to) a PCIe endpoint. Figure 1A In this embodiment, each memory module 135 may expose a set of base address registers (BARs) as a memory range to the host BIOS. One or more of the memory modules 135 may include firmware that transparently manages the memory space mapped by the host OS.

[0060] In some embodiments, as described above, the Enhanced Capability CXL switch 130 includes an FPGA (or ASIC) controller 137 and provides additional features beyond CXL packet switching. For example, it can (as described above) virtualize the memory module 135, i.e., as a translation layer operation, translate between circuit-side addresses (or “processor-side” addresses, i.e., addresses included in memory read and write commands issued by the processing circuitry 115) and memory-side addresses (i.e., addresses used by the Enhanced Capability CXL switch 130 to address storage locations within the memory module 135), thereby masking the physical addresses of the memory module 135 and presenting a virtual aggregation of memory. The controller 137 of the Enhanced Capability CXL switch 130 can also act as a management device for the memory module 135 and facilitate host control plane processing. The controller 137 can transparently move data without the involvement of the processing circuitry 115 and update the memory mapping (or “address translation table”) accordingly, ensuring that subsequent accesses function as expected. Controller 137 may include a switch management device that (i) can appropriately bind and unbind upstream and downstream connections during runtime, and (ii) can implement rich control semantics and statistics associated with data transfers to and from memory module 135. Controller 137 may include additional "backdoor" 100GbE or other network interface circuitry 125 (in addition to the network interface for connecting to a host) for connecting to other servers 105 or other networking devices. In some embodiments, controller 137 presents itself as a Type 2 device to processing circuitry 115, enabling the issuance of a cache invalidation instruction to processing circuitry 115 upon receiving a remote write request. In some embodiments, DDIO technology is enabled, and remote data is first pulled to the last-level cache (LLC) of processing circuitry 115 and then written (from the cache) to memory module 135.

[0061] As described above, one or more of the memory modules 135 may include persistent memory. If the memory module 135 is presented as a persistent device, the controller 137 of the enhanced capability CXL switch 130 can manage persistent domains via processing circuitry when persistent storage is required (e.g., it can store data recognized by using appropriate operating system functions in persistent memory). In such embodiments, a software API can flush cached data to persistent memory.

[0062] In some embodiments, direct memory transfer to memory module 135 can be performed in a manner similar to that described above. Figure 1A and Figure 1BThe operations performed by the controller of the memory module 135 are executed by the controller 137 of the enhanced capability CXL switch 130, as described in the embodiments.

[0063] As described above, in some embodiments, the memory modules 135 are organized into groups, for example, into one memory-intensive group, another HBM-heavy group, another density- and performance-limited group, and another capacity-intensive group. These groups may have different form factors or be based on different technologies. The controller 137 of the enhanced capability CXL switch 130 can intelligently route data and commands based on factors such as workload, tags, or quality of service (QoS). For read requests, there may not be routing based on these factors.

[0064] The controller 137 of the enhanced capability CXL switch 130 can also (as described above) virtualize the processing circuit-side address and memory-side address, enabling the controller 137 of the enhanced capability CXL switch 130 to determine where data will be stored. The controller 137 of the enhanced capability CXL switch 130 can make such a determination based on information or instructions it can receive from the processing circuitry 115. For example, the operating system can provide memory allocation features that allow applications to specify whether to allocate low-latency storage, high-bandwidth storage, or persistent storage, and such requests initiated by the application can then be considered by the controller 137 of the enhanced capability CXL switch 130 when determining where (e.g., which of the memory modules 35) memory is allocated. For example, high-bandwidth storage requested by the application can be allocated in a memory module 135 containing HBM, data persistence storage requested by the application can be allocated in a memory module 135 containing NAND flash memory, and other storage (not requested by the application) can be stored in a memory module 135 containing relatively inexpensive DRAM. In some embodiments, the controller 137 of the enhanced capability CXL switch 130 can determine where certain data will be stored based on network usage patterns. For example, the controller 137 of the enhanced capability CXL switch 130 can determine, by monitoring usage patterns, that data in a certain range of physical addresses is accessed more frequently than other data, and the controller 137 of the enhanced capability CXL switch 130 can then copy this data to the memory module 135 containing the HBM and modify its address translation table so that the data in the new location is stored in the same range of virtual addresses. In some embodiments, one or more of the memory modules 135 include flash memory (e.g., NAND flash memory), and the controller 137 of the enhanced capability CXL switch 130 implements a flash translation layer for this flash memory. The flash translation layer can support rewriting processor-side memory locations (by moving data to different locations and marking the previous location of the data as invalid), and it can perform garbage collection (e.g., erasing any valid data in a block after moving it to another block when the portion of data marked as invalid in the block exceeds a threshold).

[0065] In some embodiments, the controller 137 of the Enhanced Capability CXL switch 130 can facilitate physical function (PF) to PF transfers. For example, if one of the processing circuits 115 needs to move data from one physical address to another (which may have the same virtual address; this fact does not need to affect the operation of the processing circuit 115), or if the processing circuit 115 needs to move data between two virtual addresses (the processing circuit 115 needs to have both virtual addresses), the controller 137 of the Enhanced Capability CXL switch 130 can supervise the transfer without involving the processing circuit 115. For example, the processing circuit 115 may send a CXL request, and data may be sent from one memory module 135 to another memory module 135 after the Enhanced Capability CXL switch 130 has been established without reaching the processing circuit 115 (e.g., data can be copied from one memory module 135 to another). In this case, because the processing circuit 115 initiated the CXL request, the processing circuit 115 may need to flush its cache to ensure consistency. If, instead, a Type 2 memory device (e.g., one of the memory modules 135 or an accelerator that may also be connected to the CXL switch) initiates the CXL request and the switch is not virtualized, the Type 2 memory device may send a message to the processing circuitry 115 to invalidate the cache.

[0066] In some embodiments, the controller 137 of the enhanced capability CXL switch 130 can facilitate RDMA requests between servers. A remote server 105 can initiate such an RDMA request, which can be sent via the ToR Ethernet switch 110 and reach the enhanced capability CXL switch 130 in the server 105 (“local server”) responding to the RDMA request. The enhanced capability CXL switch 130 can be configured to receive such RDMA requests, and it can use a set of memory modules 135 in the receiving server 105 (i.e., the server receiving the RDMA request) as its own storage space. In the local server, the enhanced capability CXL switch 130 can receive the RDMA request as a direct RDMA request (i.e., an RDMA request not routed through the processing circuitry 115 in the local server), and it can send a direct response to the RDMA request (i.e., it can send a response without routing through the processing circuitry 115 in the local server). In the remote server, the response (e.g., data sent by the local server) can be received by the enhanced capability CXL switch 130 of the remote server without being routed through the processing circuit 115 in the remote server, and stored in the memory module 135 of the remote server.

[0067] Figure 1D It shows the relationship with Figure 1C A system similar to the one described above, in which processing circuitry 115 is connected to network interface circuitry 125 via an enhanced capability CXL switch 130. The enhanced capability CXL switch 130, memory module 135, and network interface circuitry 125 are located on an expansion adapter 140. The expansion adapter 140 may be a board or module that plugs into an expansion slot (e.g., PCIe connector 145) on the motherboard of server 105. Thus, the server can be any suitable server, modified only by installing the expansion adapter 140 in the PCIe connector 145. The memory module 135 may be installed in a connector (e.g., an M.2 connector) on the expansion adapter 140. In such an embodiment, (i) network interface circuitry 125 may be integrated into the enhanced capability CXL switch 130, or (ii) each network interface circuitry 125 may have a PCIe interface (the network interface circuitry 125 may be a PCIe endpoint) such that the processing circuitry 115 to which it is connected can communicate with the network interface circuitry 125 via a root port-to-endpoint PCIe connection. The controller 137 of the enhanced capability CXL switch 130 (which may have PCIe input ports connected to the processing circuitry 115 and the network interface circuitry 125) can communicate with the network interface circuitry 125 via a peer-to-peer PCIe connection.

[0068] According to embodiments of the present invention, a system is provided, comprising: a first server, the first server including: a stored program processing circuit, a network interface circuit, a cache coherence switch, and a first memory module, wherein: the first memory module is connected to the cache coherence switch, the cache coherence switch is connected to the network interface circuit, and the stored program processing circuit is connected to the cache coherence switch. In some embodiments, the system further includes a second memory module connected to the cache coherence switch, wherein the first memory module includes volatile memory, and the second memory module includes persistent memory. In some embodiments, the cache coherence switch is configured to virtualize the first memory module and the second memory module. In some embodiments, the first memory module includes flash memory, and the cache coherence switch is configured to provide a flash translation layer to the flash memory. In some embodiments, the cache coherence switch is configured to: monitor the access frequency of a first memory location in the first memory module; determine that the access frequency exceeds the first threshold; and copy the contents of the first memory location to a second memory location, the second memory location being in the second memory module. In some embodiments, the second memory module includes high-bandwidth memory (HBM). In some embodiments, the cache coherence switch is configured to maintain a table for mapping processor-side addresses to memory-side addresses. In some embodiments, the system further includes: a second server, and a network switch connected to the first server and the second server. In some embodiments, the network switch includes a top-of-rack (ToR) Ethernet switch. In some embodiments, the cache-coherent switch is configured to receive Direct Remote Direct Memory Access (RDMA) requests and send Direct RDMA responses. In some embodiments, the cache-coherent switch is configured to receive Remote Direct Memory Access (RDMA) requests via the ToR Ethernet switch and via network interface circuitry, and to send Direct RDMA responses via the ToR Ethernet switch and via network interface circuitry. In some embodiments, the cache-coherent switch is configured to support the Compute Fast Link (CXL) protocol. In some embodiments, the first server includes an expansion adapter connected to an expansion socket of the first server, the expansion adapter including: a cache-coherent switch; and a memory module socket, through which a first memory module is connected to the cache-coherent switch. In some embodiments, the memory module socket includes an M.2 socket. In some embodiments, the network interface circuitry is on the expansion adapter.According to embodiments of the present invention, a method for performing remote direct memory access in a computing system is provided. The computing system includes a first server and a second server. The first server includes a stored-program processing circuit, a network interface circuit, a cache coherence switch, and a first memory module. The method includes: receiving a direct remote direct memory access (RDMA) request by the cache coherence switch and sending a direct RDMA response by the cache coherence switch. In some embodiments, the computing system further includes an Ethernet switch, and receiving the direct RDMA request includes receiving the direct RDMA request through the Ethernet switch. In some embodiments, the method further includes: receiving a read command for a first memory address from the stored-program processing circuit by the cache coherence switch, converting the first memory address to a second memory address by the cache coherence switch, and retrieving data from the first memory module at the second memory address by the cache coherence switch. In some embodiments, the method further includes: receiving data by the cache coherence switch, storing the data in the first memory module by the cache coherence switch, and sending a command to the stored-program processing circuit to invalidate a cache line by the cache coherence switch. According to an embodiment of the present invention, a system is provided, comprising: a first server, the first server comprising: a stored program processing circuit, a network interface circuit, a cache coherence switching device, and a first memory module, wherein: the first memory module is connected to the cache coherence switching device, the cache coherence switching device is connected to the network interface circuit, and the stored program processing circuit is connected to the cache coherence switching device.

[0069] Figure 1E An embodiment is shown in which each of a plurality of servers 105 is connected to a ToR server-linking switch 112, which may be a PCIe 5.0 CXL switch with PCIe capability as shown. The server-linking switch 112 may include an FPGA or ASIC and may provide performance superior to that of an Ethernet switch (in terms of throughput and latency). Each of the servers 105 may include a plurality of storage modules 135 connected to the server-linking switch 112 via an enhanced capability CXL switch 130 and a plurality of PCIe connectors. As shown, each of the servers 105 may also include one or more processing circuitry 115 and system memory 120. The server-linking switch 112 may operate as a master device, and each of the enhanced capability CXL switches 130 may operate as a slave device, as discussed in further detail below.

[0070] exist Figure 1E In this embodiment, the server link switch 112 can group or batch multiple cache requests received from different servers 105, and it can group packets to reduce control overhead. The enhanced capability CXL switch 130 may include a controller (e.g., from an FPGA or an ASIC) to (i) route data to different memory types based on workload, (ii) virtualize processor-side addresses to memory-side addresses, and (iii) facilitate bypassing consistency requests between different servers 105 from the processing circuitry 115. Figure 1E The system shown can be based on CXL 2.0, it can include distributed shared memory within a rack, and it can use a ToR server to link switch 112 to connect natively to remote nodes.

[0071] The ToR server link switch 112 may have additional network connectivity (e.g., an Ethernet connection as shown, or another connection, such as a wireless connection, like a WiFi connection or a 5G connection) for connecting to other servers or clients. The server link switch 112 and the enhanced capability CXL switch 130 may each include a controller, which may be or include processing circuitry, such as an ARM processor. The PCIe interface may conform to the PCIe 5.0 standard or an earlier version, or a future version of the PCIe standard, or an interface conforming to a different standard (e.g., NVDIMM-P, CCIX, or OpenCAPI) may be used instead of the PCIe interface. The memory module 135 may include various memory types, including DDR4 DRAM, HBM, LDPPR, NAND flash memory, or solid-state drives (SSDs). The memory module 135 may be partitioned or contain a cache controller to handle multiple memory types, and they may have different form factors, such as HHHL, FHHL, M.2, U.2, mezzanine cards, daughter cards, E1.S, E1.L, E1.L, or E3.S.

[0072] exist Figure 1EIn this embodiment, the enhanced capability CXL switch 130 can implement one-to-many and many-to-one switching, and it can implement fine-grained load-to-store interfaces at the flit (64-byte) level. Each server can have aggregated memory devices, each partitioned into multiple logical devices, each logical device having a corresponding LD-ID. The ToR switch 112 (which may be referred to as a "server-linked switch") implements one-to-many functionality, and the enhanced capability CXL switch 130 in server 105 implements many-to-one functionality. The server-linked switch 112 can be a PCIe switch, a CXL switch, or both. In such a system, the requester can be the processing circuitry 115 of multiple servers 105, and the responder can be a number of aggregated memory modules 135. The hierarchical structure of the two switches (as described above, the master switch is the server-linked switch 112, and the slave switch is the enhanced capability CXL switch 130) enables any-any communication. Each of the memory modules 135 can have a physical function (PF) and up to 16 separate logical devices. In some embodiments, the number of logical devices (e.g., the number of partitions) may be limited (e.g., up to 16), and a control partition (which may be for controlling the physical functions of the devices) may also exist. Each of the memory modules 135 may be a Type 2 device with cxl.cache, cxl.mem, and cxl.io, as well as an address translation service (ATS) implementation, to process cache line copies that the processing circuitry 115 can hold. The enhanced capability CXL switch 130 and the structure manager can control the discovery of the memory modules 135 and (i) perform device discovery and virtual CXL software creation, and (ii) bind virtual ports to physical ports. Figure 1A-1D In this embodiment, the structure manager can operate via a connection on the SMBus sideband. The interface of the memory module 135 (which may be an Intelligent Platform Management Interface (IPMI)) or a Redfish-compliant interface (which may also provide additional features not required by the standard) can enable configurability.

[0073] As described above, some embodiments implement a hierarchical architecture where the master controller (which may be implemented in an FPGA or ASIC) is part of the server link switch 112, and the slave controllers are part of the enhanced capability CXL switch 130 to provide a load-store interface (i.e., an interface with cache line (e.g., 64-byte) granularity and operating within a consistency domain without software driver involvement). This load-store interface can extend the consistency domain beyond a single server, CPU, or host, and may include electrical or optical physical media (e.g., an optical connection with electro-optical transceivers at both ends). In operation, the master controller (in the server link switch 112) boots (or "reboots") and configures all servers 105 on the rack. The master controller has visibility across all hosts and can (i) discover each server and how many servers 105 and memory modules 135 exist in the server cluster, (ii) independently configure each of the servers 105, (iii) enable or disable some blocks of memory on different servers based on, for example, rack configuration (e.g., enable or disable any memory module 235), (iv) control access (e.g., which server can control which other server), (v) implement flow control (e.g., since all host and device requests pass through the master controller, it can transfer data from one server to another and perform flow control on the data), (vi) group or batch requests or packets (e.g., multiple cache requests received by the master controller from different servers 105), and (vii) receive remote software updates, broadcast communications, etc. In batching mode, the server link switch 112 can receive multiple packets destined for the same server (e.g., destined for the first server) and send them together (i.e., without any pauses between them) to the first server. For example, server link switch 112 can receive a first packet from a second server and a second packet from a third server, and send the first and second packets together to a first server. Each of servers 105 can expose (i) an IPMI network interface, (ii) a system event log (SEL), and (iii) a board management controller (BMC) to the master controller, enabling the master controller to measure performance, measure reliability dynamically (on the fly), and reconfigure server 105.

[0074] In some embodiments, a software architecture is used that facilitates a highly available load-store interface. Such a software architecture can provide reliability, replication, consistency, system consistency, hashing, caching, and persistence. The software architecture can provide reliability (in systems with a large number of servers) by performing periodic hardware checks on CXL device components via IPMI. For example, server-linked switch 112 can query the status of storage server 150 via its IPMI interface, querying for things like power status (whether the power supply to storage server 150 is working properly), network status (whether the interface to server-linked switch 112 is working properly), and error checking status (whether there are error conditions in any subsystem of storage server 150). The software architecture provides replication in that the master controller can replicate the data stored in storage module 135 and maintain data consistency between the replicas.

[0075] The software architecture provides consistency because the master controller can be configured with different consistency levels, and the server link switch 112 can adjust the packet format according to the consistency level to be maintained. For example, to maintain eventual consistency, the server link switch 112 can reorder requests, while to maintain strict consistency, the server link switch 112 can maintain a scoreboard of all requests with precise timestamps at the switch. The software architecture provides system consistency because multiple processing circuits 115 can read from or write to the same memory address, and to maintain consistency, the master controller can be responsible for (using directory lookup) reaching the home node at that address or broadcasting the request on the common bus.

[0076] The software architecture provides hashing because the server-linked switch 112 and the enhanced CXL switch can maintain a virtual mapping of addresses that can use consistent hashing with multiple hash functions to evenly map data across all CXL devices on all nodes at startup (or adjust when a server goes down or up). The software architecture provides caching because the master controller can designate certain memory partitions (e.g., those in memory modules 135 including HBM or similar technologies) to act as caches (e.g., using write-through caching or write-back caching). The software architecture provides persistence because the master and slave controllers can manage persistence domains and flushing.

[0077] In some embodiments, the capabilities of the CXL switch are integrated into the controller of the memory module 135. In such embodiments, the server link switch 112 can still act as the master controller and has enhanced features as discussed elsewhere herein. The server link switch 112 can also manage other memory devices in the system and can have Ethernet connectivity (e.g., a 100GbE connection) for connecting to client machines, for example, those not part of the PCIe network formed by the server link switch 112.

[0078] In some embodiments, server link switch 112 has enhanced capabilities and also includes an integrated CXL controller. In other embodiments, server link switch 112 is merely a physical routing device, and each server 105 includes a master CXL controller. In such embodiments, master devices across different servers can negotiate a master-slave architecture. The intelligent functions of (i) the enhanced capability CXL switch 130 and (ii) the server link switch 112 can be implemented in one or more FPGAs, one or more ASICs, one or more ARM processors, or one or more SSD devices with computing power. Server link switch 112 can perform flow control, for example, by reordering independent requests. In some embodiments, RDMA is optional because the interface is load-store, but there may be intermediate RDMA requests using PCIe physical media (instead of 100GbE). In such embodiments, a remote host can initiate an RDMA request that can be sent to enhanced capability CXL switch 130 via server link switch 112. Server link switch 112 and enhanced capability CXL switch 130 can prioritize RDMA 4KB requests or CXL chip (64-byte) requests.

[0079] As in Figure 1C and Figure 1D In one embodiment, the Enhanced Capability CXL switch 130 can be configured to receive such RDMA requests, and it can use a set of memory modules 135 in the receiving server 105 (i.e., the server receiving the RDMA request) as its own memory space. Furthermore, the Enhanced Capability CXL switch 130 can be virtualized across processing circuitry 115 and can initiate RDMA requests on remote Enhanced Capability CXL switches 130 to move data back and forth between servers 105 without involving processing circuitry 115.

[0080] Figure 1F It shows the relationship with Figure 1E A system similar to the one described above, in which processing circuitry 115 is connected to network interface circuitry 125 via an enhanced CXL switch 130. (As in...) Figure 1D In the embodiments, in Figure 1F The enhanced capability CXL switch 130, memory module 135, and network interface circuitry 125 are mounted on an expansion adapter 140. The expansion adapter 140 can be a board or module that plugs into an expansion slot (e.g., PCIe connector 145) on the motherboard of server 105. Thus, the server can be any suitable server, modified simply by installing the expansion adapter 140 in the PCIe connector 145. The memory module 135 can be mounted in a connector (e.g., an M.2 connector) on the expansion adapter 140. In such an embodiment, (i) the network interface circuit 125 may be integrated into the enhanced capability CXL switch 130, or (ii) each network interface circuit 125 may have a PCIe interface (the network interface circuit 125 may be a PCIe endpoint) such that the processing circuit 115 to which it is connected can communicate with the network interface circuit 125 via a root port to endpoint PCIe connection, and the controller 137 of the enhanced capability CXL switch 130 (which may have PCIe input ports connected to the processing circuit 115 and the network interface circuit 125) can communicate with the network interface circuit 125 via a peer-to-peer PCIe connection.

[0081] According to an embodiment of the present invention, a system is provided, comprising: a first server, the first server including: a stored program processing circuit, a cache coherence switch, and a first memory module; a second server; and a server link switch connected to the first server and the second server, wherein: the first memory module is connected to the cache coherence switch, the cache coherence switch is connected to the server link switch, and the stored program processing circuit is connected to the cache coherence switch. In some embodiments, the server link switch includes a Peripheral Component Interconnect Fast (PCIe) switch.

[0082] In some embodiments, the server link switch includes a compute fast link (CXL) switch. In some embodiments, the server link switch includes a top-of-rack (ToR) CXL switch. In some embodiments, the server link switch is configured to discover a first server. In some embodiments, the server link switch is configured to cause the first server to restart. In some embodiments, the server link switch is configured to cause a cache coherent switch to disable a first memory module. In some embodiments, the server link switch is configured to send data from a second server to the first server and perform flow control on that data. In some embodiments, the system further includes a third server connected to the server link switch, wherein the server link switch is configured to receive a first packet from the second server, receive a second packet from the third server, and send the first packet and the second packet to the first server. In some embodiments, the system further includes a second memory module connected to the cache coherent switch, wherein the first memory module includes volatile memory and the second memory module includes persistent memory. In some embodiments, the cache coherent switch is configured to virtualize the first memory module and the second memory module. In some embodiments, the first memory module includes flash memory, and the cache coherent switch is configured to provide a flash translation layer to the flash memory. In some embodiments, the first server includes an expansion adapter connected to an expansion port of the first server, the expansion adapter including: a cache coherent switch; and a memory module port, through which a first memory module is connected to the cache coherent switch. In some embodiments, the memory module port includes an M.2 port. In some embodiments, the cache coherent switch is connected to a server link switch via a connector, and the connector is on the expansion adapter. According to embodiments of the present invention, a method for performing remote direct memory access in a computing system is provided, the computing system including: a first server, a second server, a third server, and a server link switch connected to the first server, the second server, and the third server, the first server including: stored program processing circuitry, a cache coherent switch, and a first memory module, the method including: receiving a first packet from a second server by the server link switch, receiving a second packet from a third server by the server link switch, and sending the first packet and the second packet to the first server. In some embodiments, the method further includes: receiving a direct remote direct memory access (RDMA) request by the cache coherent switch, and sending a direct RDMA response by the cache coherent switch. In some embodiments, receiving a direct RDMA request includes receiving a direct RDMA request through the server link switch.In some embodiments, the method further includes: receiving a read command for a first memory address from a stored-program processing circuit by a cache coherence switch; converting the first memory address to a second memory address by the cache coherence switch; and retrieving data from the first memory module at the second memory address by the cache coherence switch. According to embodiments of the present invention, a system is provided, comprising: a first server including: a stored-program processing circuit, a cache coherence switching device, and a first memory module; a second server; and a server link switch connecting the first server and the second server, wherein: the first memory module is connected to the cache coherence switching device, the cache coherence switching device is connected to the server link switch, and the stored-program processing circuit is connected to the cache coherence switching device.

[0083] Figure 1G An embodiment is shown in which each of a plurality of storage servers 150 is connected to a ToR server link switch 112, which may be a PCIe 5.0 CXL switch as shown in the figure. As in Figure 1E and Figure 1F In some embodiments, the server link switch 112 may include an FPGA or ASIC and may provide performance superior to that of an Ethernet switch (in terms of throughput and latency). For example, in Figure 1E and Figure 1F In one embodiment, the storage server 150 may include multiple storage modules 135 connected to the server link switch 112 via multiple PCIe connectors. Figure 1G In some embodiments, the processing circuitry 115 and the system memory 120 may be absent, and the primary purpose of the memory server 150 may be to provide memory for use by other servers 105 with computing resources.

[0084] exist Figure 1G In this embodiment, the server link switch 112 can group or batch multiple cache requests received from different memory servers 150, and it can group packets to reduce control overhead. The enhanced capability CXL switch 130 may include composable hardware building blocks to (i) route data to different memory types based on workload, and (ii) virtualize processor-side addresses (translating such addresses into memory-side addresses). Figure 1G The system shown can be based on CXL 2.0, which can include composable and deaggregable shared memory within a rack, and can use a ToR server to link switch 112 to provide pooled (i.e., aggregated) memory to remote devices.

[0085] The ToR server link switch 112 may have additional network connectivity (e.g., an Ethernet connection as shown, or another connection, such as a wireless connection like WiFi or 5G) for connecting to other servers or clients. The server link switch 112 and the enhanced capability CXL switch 130 may each include a controller, which may be or include processing circuitry, such as an ARM processor. The PCIe interface may conform to the PCIe 5.0 standard or an earlier version, or a future version of the PCIe standard, or a different standard (e.g., NVDIMM-P, CCIX, or OpenCAPI) may be used instead of PCIe. The memory module 135 may include various memory types, including DDR4 DRAM, HBM, LPDDR, NAND flash memory, and solid-state drives (SSDs). The memory module 135 may be partitioned or contain a cache controller to handle multiple memory types, and these may be different form factors, such as HHHL, FHHL, M.2, U.2, mezzanine cards, daughter cards, E1.S, E1.L, E1.L, or E3.S.

[0086] exist Figure 1G In this embodiment, the enhanced capability CXL switch 130 can implement one-to-many and many-to-one switching, and it can implement fine-grained load-store interfaces at the microchip (64-byte) level. Each memory server 150 may have aggregated memory devices, each partitioned into multiple logical devices, each logical device having a corresponding LD-ID. The enhanced capability CXL switch 130 may include a controller 137 (e.g., an ASIC or FPGA) and circuitry for device discovery, enumeration, partitioning, and presentation of physical address ranges (which may be separate from or part of such an ASIC or FPGA). Each of the memory modules 135 may have a physical function (PF) and up to 16 separate logical devices. In some embodiments, the number of logical devices (e.g., the number of partitions) may be limited (e.g., up to 16), and a control partition (which may be a physical function for controlling the devices) may also exist. Each of the memory modules 135 may be a Type 2 device with cxl.cache, cxl.mem, and cxl.io, as well as an Address Translation Service (ATS) implementation, to process cache line copies that the processing circuitry 115 may hold.

[0087] The enhanced CXL switch 130 and the structure manager can control the discovery of the memory module 135, and (i) perform device discovery and virtual CXL software creation, and (ii) bind virtual ports to physical ports. (As in...) Figure 1A-1DIn this embodiment, the structure manager can operate via a connection on the SMBus sideband. Configurability can be achieved through an interface to the memory module 135 (which may be an Intelligent Platform Management Interface (IPMI)) or a Redfish-compliant interface (which may also provide additional features not required by the standard).

[0088] for Figure 1G In some embodiments, the building blocks may include (as described above) a CXL controller 137 implemented on an FPGA or ASIC, which exchanges to implement the aggregation of memory devices (e.g., in memory module 135), SSDs, accelerators (GPUs, NICs), CXL and PCIe5 connectors, and firmware to expose device details to the operating system’s advanced configuration and power interface (ACPI) tables, such as a heterogeneous memory attribute table (HMAT) or a static resource affinity table (SRAT).

[0089] In some embodiments, the system provides composability. The system can provide the ability to bring CXL devices and other accelerators online and offline based on software configuration, and it can be able to group accelerators, memory, and memory device resources and quantitatively allocate them to each memory server 150 in the rack. The system can hide the physical address space and provide transparent caching using faster devices such as HBM and SRAM.

[0090] exist Figure 1GIn this embodiment, the controller 137 of the Enhanced Capability CXL switch 130 can (i) manage the memory module 135, (ii) integrate and control heterogeneous devices such as NICs, SSDs, GPUs, and DRAMs, and (iii) enable dynamic reconfiguration of storage to the memory devices via power gating. For example, the ToR server link switch 112 can disable power to one of the memory modules 135 (i.e., cut off or reduce power supply) by instructing the Enhanced Capability CXL switch 130 to disable power to the memory module 135. When the server link switch 112 instructs to disable power to the memory module 135, the Enhanced Capability CXL switch 130 can disable power to the memory module 135. This disabling can save power and can improve the performance (e.g., throughput and latency) of other memory modules 135 in the memory server 150. Based on negotiation, each remote server 105 can see a different logical view of the memory module 135 and its connections. The controller 137 of the enhanced CXL switch 130 can maintain state, enabling each remote server to maintain its allocated resources and connections, and it can perform memory compression or deduplication (using configurable block sizes) to save memory capacity. Figure 1G The deaggregation rack can have its own BMC. It can also expose the IPMI network interface and System Event Log (SEL) to remote devices, enabling the host device (e.g., a remote server using storage provided by storage server 150) to dynamically measure performance and reliability and reconfigure the deaggregation rack. Figure 1G The depolymerization rack can be similar to the one described in this paper. Figure 1EThe embodiments described provide reliability, replication, consistency, system consistency, hashing, caching, and persistence, for example, by having multiple remote servers read from or write to the same memory address, each remote server configured with a different consistency level. In some embodiments, the server-linked switch maintains eventual consistency between data stored on a first memory server and data stored on a second memory server. The server-linked switch 112 can maintain different consistency levels for different server pairs; for example, the server-linked switch can also maintain strict consistency, sequential consistency, causal consistency, or processor consistency levels between data stored on a first memory server and data stored on a third memory server. The system can employ communication in "local band" (server-linked switch 112) and "global band" (de-aggregated server) domains. Write operations may flush to the "global band" so that new read operations from other servers are visible. The controller 137 of the enhanced capability CXL switch 130 can manage persistent domains and flushing separately for each remote server. For example, a cache-coherent switch can monitor the fullness of a first region of memory (volatile memory, operating as a cache), and when the fullness level exceeds a threshold, the cache-coherent switch can move data from the first region of memory to a second region of memory, which is in persistent memory. Flow control can be handled by enabling the controller 137 of the enhanced CXL switch 130 to establish priorities among remote servers, presenting different perceived latency and bandwidth.

[0091] According to embodiments of the present invention, a system is provided, comprising: a first memory server, including: a cache coherence switch and a first memory module; a second memory server; and a server link switch connected to the first memory server and the second memory server, wherein: the first memory module is connected to the cache coherence switch, and the cache coherence switch is connected to the server link switch. In some embodiments, the server link switch is configured to disable power supply to the first memory module. In some embodiments, the server link switch is configured to disable power supply to the first memory module by instructing the cache coherence switch to disable power supply to the first memory module, and the cache coherence switch is configured to disable power supply to the first memory module when instructed to do so by the server link switch. In some embodiments, the cache coherence switch is configured to perform deduplication within the first memory module. In some embodiments, the cache coherence switch is configured to compress data and store the compressed data in the first memory module. In some embodiments, the server link switch is configured to query the status of the first memory server. In some embodiments, the server link switch is configured to query the status of the first memory server via an Intelligent Platform Management Interface (IPMI). In some embodiments, querying the status includes querying a status selected from a group consisting of power status, network status, and error checking status. In some embodiments, the server link switch is configured to batch cache requests directed to a first memory server. In some embodiments, the system further includes a third memory server connected to the server link switch, wherein the server link switch is configured to maintain a consistency level selected from the group consisting of strict consistency, sequential consistency, causal consistency, and processor consistency between data stored on the first memory server and data stored on the third memory server. In some embodiments, a cache consistency switch is configured to: monitor the fullness of a first region of memory and move data from the first region of memory to a second region of memory, wherein the first region of memory is in volatile memory and the second region of memory is in persistent memory. In some embodiments, the server link switch includes a Peripheral Component Interconnect Fast (PCIe) switch. In some embodiments, the server link switch includes a Compute Fast Link (CXL) switch. In some embodiments, the server link switch includes a Top-of-Rack (ToR) CXL switch. In some embodiments, the server link switch is configured to send data from a second memory server to a first memory server and perform flow control on the data.In some embodiments, the system further includes a third memory server connected to a server link switch, wherein the server link switch is configured to receive a first packet from a second memory server, receive a second packet from the third memory server, and send the first packet and the second packet to the first memory server. According to embodiments of the present invention, a method for performing remote direct memory access in a computing system is provided, the computing system including: a first memory server; a first server; a second server; and a server link switch connected to the first memory server, the first server, and the second server, the first memory server including: a cache coherence switch and a first memory module; the first server including: stored program processing circuitry; the second server including: stored program processing circuitry; the method including: receiving a first packet from the first server via the server link switch; receiving a second packet from the second server via the server link switch; and sending the first packet and the second packet to the first memory server. In some embodiments, the method further includes: compressing data via a cache coherence switch and storing the compressed data in the first memory module. In some embodiments, the method further includes: querying the status of the first memory server via the server link switch. According to an embodiment of the present invention, a system is provided, comprising: a first memory server, the first memory server including: a cache coherence switch and a first memory module; a second memory server; and a server link switching device connected to the first memory server and the second memory server, wherein: the first memory module is connected to the cache coherence switch, and the cache coherence switch is connected to the server link switching device.

[0092] Figure 2A-2D These are flowcharts of various embodiments. In these flowchart embodiments, processing circuitry 115 is a CPU; in other embodiments, it may be other processing circuitry (e.g., a GPU). Reference Figure 2A , Figure 1A and Figure 1B The controller 137 of the memory module 135 in the embodiment, or Figure 1C-1GThe enhanced capability CXL switch 130 of any embodiment can be virtualized across processing circuitry 115 and initiate RDMA requests on the enhanced capability CXL switch 130 in another server 105 to move data back and forth between servers 105 without involving the processing circuitry 115 in either server (where virtualization is handled by the controller 137 of the enhanced capability CXL switch 130). For example, at 205, the controller 137 of memory module 135 or the enhanced capability CXL switch 130 generates an RDMA request for an additional remote memory (e.g., CXL memory or aggregated memory); at 210, network interface circuitry 125 bypasses the processing circuitry and sends the request to ToR Ethernet switch 110 (which may have an RDMA interface); at 215, ToR Ethernet switch 110 routes the RDMA request to remote server 105 for processing by the controller 137 of memory module 135 or by the remote enhanced capability CXL switch 130 bypassing the remote processing circuitry 115 via RDMA access to the remote aggregated memory; at 220, ToR Ethernet switch 110 receives the processed data and routes it to the local memory module 135 or the local enhanced capability CXL switch 130 via RDMA bypassing the local processing circuitry 115; and at 222, Figure 1A and Figure 1B The controller 137 of the memory module 135 in the embodiment, or the enhanced capability CXL switch 130, directly receives the RDMA response (e.g., directly receives the RDMA response if it is not forwarded by the processing circuitry 115).

[0093] In such an embodiment, the controller 137 of the remote memory module 135 or the enhanced capability CXL switch 130 of the remote server 105 is configured to receive direct remote direct memory access (RDMA) requests and send direct RDMA responses. As used herein, the controller 137 of the remote memory module 135 receiving or the enhanced capability CXL switch 130 receiving a “direct RDMA request” (or “directly” receiving such a request) means that the controller 137 of the remote memory module 135 or the enhanced capability CXL switch 130 receives such a request if such a request is not forwarded or otherwise processed by the processing circuitry 115 of the remote server, and the controller 137 of the remote memory module 135 or the enhanced capability CXL switch 130 sending a “direct RDMA response” (or “directly” sending such a response) means that such a response is sent if such a response is not forwarded or otherwise processed by the processing circuitry 115 of the remote server.

[0094] refer to Figure 2BIn another embodiment, RDMA can be performed with the processing circuitry of a remote server participating in data processing. For example, at 225, processing circuitry 115 can send data or workload requests via Ethernet; at 230, ToR Ethernet switch 110 can receive the request and route it to the appropriate server among multiple servers 105; at 235, the request can be received within the server via multiple ports (e.g., a 100GbE-enabled NIC) of network interface circuitry 125; at 240, processing circuitry 115 (e.g., x86 processing circuitry) can receive the request from network interface circuitry 125; and at 245, processing circuitry 115 can process the request via the CXL 2.0 protocol using DDR and additional memory resources (e.g., together) to share memory (which is in Figure 1A and Figure 1B In some embodiments, this could be a aggregated memory.

[0095] refer to Figure 2C ,exist Figure 1E In this embodiment, RDMA can be performed with the processing circuitry of the remote server participating in data processing. For example, at 225, processing circuitry 115 can send data or workload requests via Ethernet or PCIe; at 230, ToR Ethernet switch 110 can receive the request and route it to the appropriate server among multiple servers 105; at 235, the request can be received within the server via multiple ports of the PCIe connector; at 240, processing circuitry 115 (e.g., x86 processing circuitry) can receive the request from network interface circuitry 125; and at 245, processing circuitry 115 can process the request via the CXL 2.0 protocol using DDR and additional memory resources (e.g., together) to share memory (which is in... Figure 1A and Figure 1BIn some embodiments, this could be a aggregated memory. At 250, processing circuitry 115 can identify requests to access memory content (e.g., DDR or aggregated memory content) from different servers; at 252, processing circuitry 115 can send a request for memory content (e.g., DDR or aggregated memory content) from a server via the CXL protocol (CXL 1.1 or CXL 2.0); at 254, the request is propagated to server link switch 112 via the local PCIe connector, and server link switch 112 then sends the request to the second PCIe connector of the second server on the rack; at 256, second processing circuitry 115 (e.g., x86 processing circuitry) receives the request from the second PCIe connector; at 258, second processing circuitry 115 can process the request together (e.g., retrieval of memory content) using the second DDR and second additional memory resources via the CXL 2.0 protocol to share aggregated memory; and at 260, second processing circuitry (e.g., x86 processing circuitry) sends the result of the request back to the initial processing circuitry via the corresponding PCIe connector and through server link switch 112.

[0096] refer to Figure 2D ,exist Figure 1G In this embodiment, RDMA can be performed with the processing circuitry of a remote server participating in data processing. For example, at 225, the processing circuitry 115 can send data or workload requests via Ethernet; at 230, the ToR Ethernet switch 110 can receive the request and route it to the appropriate server among multiple servers 105; at 235, it can be done within the server via multiple ports of the network interface circuitry 125 (e.g., a 100GbE-enabled NIC). At 262, memory module 135 receives a request from the PCIe connector; at 264, the controller of memory module 135 processes the request using local memory; at 250, the controller of memory module 135 identifies a request to access memory content (e.g., aggregated memory content) from different servers; at 252, the controller of memory module 135 sends a request for memory content (e.g., aggregated memory content) from different servers via the CXL protocol; at 254, the request is propagated through the local PCIe connector to server link switch 112, which then forwards the request to the second PCIe connector of the second server on the rack; and at 266, the second PCIe connector provides access via the CXL protocol to share aggregated memory, thereby allowing the controller of memory module 135 to retrieve memory content.

[0097] As used herein, a "server" includes at least one stored program processing circuitry (e.g., processing circuitry 115), at least one memory resource (e.g., system memory 120), and at least one circuitry for providing network connectivity (e.g., network interface circuitry 25). As used herein, "part of something" means "at least a part of that thing," and therefore may mean less than all or all of that thing. Thus, "part of something" includes, as a special case, the whole thing, i.e., an example where the whole thing is a part of something.

[0098] The background provided in the background section of this disclosure is included only to set the context, and the content of that section is not considered prior art. Any component or combination of components described (e.g., as described in any system diagram included herein) may be used to perform one or more of the operations of any flowchart included herein. Furthermore, (i) the operations are example operations and may include various additional steps not explicitly covered, and (ii) the temporal order of the operations may vary.

[0099] As used herein, the terms "processing circuitry" or "controller device" refer to any combination of hardware, firmware, and software used to process data or digital signals. Processing circuitry hardware may include, for example, application-specific integrated circuits (ASICs), general-purpose or special-purpose central processing units (CPUs), digital signal processors (DSPs), graphics processing units (GPUs), and programmable logic devices such as field-programmable gate arrays (FPGAs). As used herein, in processing circuitry, each function is performed by hardware configured (i.e., hard-wired) to execute that function, or by more general-purpose hardware (such as a CPU) configured to execute instructions stored in a non-temporary storage medium. Processing circuitry may be fabricated on a single printed circuit board (PCB) or distributed across several interconnected PCBs. Processing circuitry may contain other processing circuitry; for example, processing circuitry may include two processing circuits, an FPGA and a CPU, interconnected on a PCB.

[0100] As used herein, "controller" includes circuitry, and a controller may also be referred to as "control circuitry" or "controller circuit." Similarly, "memory module" may also be referred to as "memory module circuitry" or "memory circuitry." As used herein, the term "array" refers to an ordered collection of numbers, regardless of how they are stored (e.g., whether stored in contiguous memory locations or in a linked list). As used herein, when a second number is "within Y%" of the first number, this means that the second number is at least (1-Y / 100) times the first number, and the second number is at most (1+Y / 100) times the first number. As used herein, the term "or" should be interpreted as "and / or," such that, for example, "A or B" means any one of "A" or "B" or "A and B."

[0101] As used herein, when a method (e.g., adjustment) or a first quantity (e.g., a first variable) is referred to as “based on” a second quantity (e.g., a second variable), it means that the second quantity is an input to the method or affects the first quantity. For example, the second quantity may be an input to a function that computes the first quantity (e.g., a unique input or one of several inputs), or the first quantity may be equal to the second quantity, or the first quantity may be the same as the second quantity (e.g., stored in the same location or multiple identical locations in memory).

[0102] It should be understood that although the terms "first," "second," "third," etc., may be used herein to describe various elements, components, regions, layers, and / or portions, these elements, components, regions, layers, and / or portions should not be limited by these terms. These terms are used only to distinguish one element, component, region, layer, or portion from another element, component, region, layer, or portion. Therefore, without departing from the spirit and scope of the invention, the first element, first component, first region, first layer, or first portion discussed herein may be referred to as a second element, second component, second region, second layer, or second portion.

[0103] Spatially related terms, such as “below,” “below,” “further below,” “under,” “above,” “above,” etc., may be used herein to describe the relationship of an element or feature to another element(s)(s)(s)(s) shown in the figures. It should be understood that, in addition to the orientations described in the figures, such spatially relative terms are intended to include different orientations of the device in use or operation. For example, if the device in the figure is flipped, an element described as “below other elements or features” or “below other elements or features” would be oriented “above other elements or features.” Thus, the example terms “below” and “below” can include orientations above and below. The device may be oriented in other ways (e.g., rotated 90 degrees or in other directions), and the spatially relative descriptors used herein should be interpreted accordingly. Furthermore, it should be understood that when a layer is referred to as “between two layers,” it can be the only layer between the two layers, or there may be one or more intermediate layers.

[0104] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the inventive concept. As used herein, the terms “substantially,” “about,” and similar terms are used as approximate terms rather than terms of degree and are intended to account for inherent deviations in measured or calculated values ​​that will be recognized by one of ordinary skill in the art. As used herein, the singular forms “a” and “an” are intended to also include the plural forms unless the context clearly indicates otherwise. It will also be understood that, when used in this specification, the terms “comprising” and / or “including” specify the presence of stated features, integers, steps, operations, elements, and / or components, but do not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof. As used herein, the term “and / or” includes any and all combinations of one or more of the associated listed items. Expressions (such as “at least one of”) following a list of elements modify the entire list of elements, not a single element in the list. Furthermore, when describing embodiments of the inventive concept, the use of “may” means “one or more embodiments of this disclosure.” Additionally, the term “exemplary” is intended to indicate or illustrate. As used herein, the terms “use,” “used,” and “being used” can be considered synonymous with the terms “exploitation,” “utilized,” and “being exploited,” respectively.

[0105] It should be understood that when a component or layer is referred to as "on another component or layer," "connected to," "coupled to," or "adjacent to" another component or layer, it can be directly on, connected to, coupled to, or adjacent to another component or layer, or one or more intermediate components or layers may exist. Conversely, when a component or layer is referred to as "directly on another component or layer," "directly connected to," "directly coupled to," or "directly adjacent to" another component or layer, no intermediate components or layers exist.

[0106] Any numerical range described herein is intended to include all subranges containing the same numerical precision within the stated range. For example, the range “1.0 to 10.0” or “between 1.0 and 10.0” is intended to include all subranges between (and inclusive of) the minimum value of 1.0 and the maximum value of 10.0, i.e., a minimum value equal to or greater than 1.0 and a maximum value equal to or less than 10.0, such as 2.4 to 7.6. Any maximum numerical limit described herein is intended to include all lower numerical limits contained therein, and any minimum numerical limit described in this specification is intended to include all higher numerical limits contained therein.

[0107] Although exemplary embodiments of systems and methods for managing memory resources have been specifically described and illustrated herein, many modifications and variations will be apparent to those skilled in the art. Therefore, it should be understood that systems and methods for managing memory resources constructed in accordance with the principles of this disclosure may be implemented in ways other than those specifically described herein. The invention is also defined in the appended claims and their equivalents.

[0108] This application claims priority to U.S. Provisional Patent Application No. 63 / 031,509, filed May 28, 2020; U.S. Provisional Patent Application No. 63 / 031,508, filed May 28, 2020; U.S. Provisional Patent Application No. 63 / 057,46, filed July 28, 2020; U.S. Provisional Patent Application No. 63 / 068,054, filed August 20, 2020; and U.S. Patent Application No. 17 / 026,082, filed September 18, 2020, the disclosures of which are incorporated herein by reference in their entirety.

Claims

1. A computing system, comprising: The first server includes: Stored program processing circuit, Network interface circuit, Cache-coherent switches, and First memory module, in: The first memory module is connected to the cache coherence switch. The cache coherence switch is connected to the network interface circuit, and The stored-program processing circuit is connected to the cache coherence switch, and the stored-program processing circuit and the network interface circuit are connected to the first memory module via the cache coherence switch. The cache coherence switch is configured to support the cache coherence memory protocol, which is configured to support cache coherence and connection to the first memory module.

2. The system of claim 1 further includes a second memory module connected to the cache coherence switch, wherein the first memory module includes volatile memory and the second memory module includes persistent memory.

3. The system according to claim 2, wherein the cache coherence switch is configured to virtualize the first memory module and the second memory module.

4. The system of claim 3, wherein the first memory module includes flash memory, and the cache coherence switch is configured to provide a flash translation layer to the flash memory.

5. The system according to claim 2, wherein the cache coherence switch is configured as follows: Monitor the access frequency of the first memory location in the first memory module; Determining that the access frequency exceeds a first threshold; and Copy the contents of the first memory location to the second memory location. The second memory location is within the second memory module.

6. The system of claim 5, wherein the second memory module comprises a high-bandwidth memory (HBM).

7. The system of claim 6, wherein the cache coherence switch is configured to maintain a table for mapping processor-side addresses to memory-side addresses.

8. The system according to claim 1, further comprising: The second server, and A network switch connected to the first server and the second server.

9. The system of claim 8, wherein the network switch comprises a top-of-rack ToR Ethernet switch.

10. The system of claim 8, wherein the cache coherence switch is configured to receive remote direct memory access (RDMA) requests and send RDMA responses.

11. The system of claim 10, wherein the cache coherence switch is configured to receive the Remote Direct Memory Access (RDMA) request via a top-of-rack (ToR) Ethernet switch and through the network interface circuit, and to send the RDMA response via the ToR Ethernet switch and through the network interface circuit.

12. The system of claim 1, wherein the cache coherent memory protocol is the compute fast link (CXL) protocol.

13. The system of claim 1, wherein the first server includes an expansion adapter connected to an expansion port of the first server, the expansion adapter comprising: The cache consistency switch; and Memory module socket, The first memory module is connected to the cache coherence switch via the memory module socket.

14. The system of claim 13, wherein the memory module socket includes an M.2 socket.

15. The system of claim 13, wherein the network interface circuit is on the expansion port adapter.

16. A method for performing remote direct memory access in a computing system, the computing system comprising: First server and second server. The first server includes: Stored program processing circuit, Network interface circuit, A cache-coherent switch is configured to support the cache-coherent memory protocol, and A first memory module is connected to the stored-program processing circuit and the network interface circuit via the cache coherence switch. The cache coherence memory protocol is configured to support cache coherence and the connection to the first memory module. The method includes: The cache coherence switch receives remote direct memory access (RDMA) requests, and The RDMA response is sent by the cache coherence switch.

17. The method of claim 16, wherein: The computing system also includes an Ethernet switch, and Receiving the RDMA request includes receiving the RDMA request through the Ethernet switch.

18. The method of claim 16, further comprising: The cache coherence switch receives a read command for the first memory address from the stored program processing circuit. The cache coherence switch translates the first memory address into a second memory address, and The cache coherence switch retrieves data from the first memory module at the second memory address.

19. The method of claim 16, further comprising: Data is received by the cache consistency switch. The cache coherence switch stores the data in the first memory module, and The cache coherence switch sends a command to the stored program processing circuit to invalidate cache lines.

20. A computing system, comprising: The first server includes: Stored program processing circuit, Network interface circuit, Cache coherence switching device, and First memory module, in: The first memory module is connected to the cache coherence switching device. The cache coherence switching device is connected to the network interface circuit, and The stored-program processing circuit is connected to the cache coherence switching device, and the stored-program processing circuit and the network interface circuit are connected to the first memory module via the cache coherence switching device. The cache coherence switching device is configured to support the cache coherence memory protocol, which is configured to support cache coherence and connection to the first memory module.

Images