System and method for managing memory resources

Abstract

A system and method for managing memory resources. In some embodiments, the system includes a first server including a storage program processing circuit, a first network interface circuit, and a first memory module. The first memory module includes a first memory die and a controller. The controller is connected to the first memory die through a memory interface, connected to the storage program processing circuit through a cache coherency interface, and connected to the first network interface circuit.

Description

[0001] This application claims priority and benefit to U.S. Provisional Application No. 63 / 031,508, filed May 28, 2020, entitled “Extending memory accesses with novel cache coherence connections”; U.S. Provisional Application No. 63 / 031,509, filed May 28, 2020, entitled “Pooling server memory resources for computational efficiency”; and U.S. Provisional Application No. 63 / 031,509, filed August 20, 2020, entitled “System with cache-coherent memory and server-linking switch domain”. The priorities and interests of U.S. Provisional Application No. 63 / 068,054 entitled “Disaggregated Memory Architecture with Novel Interconnects”, filed July 28, 2020; and U.S. Provisional Application No. 63 / 057,746 entitled “Disaggregated Memory Architecture with Novel Interconnects”, filed September 18, 2020; and U.S. Application No. 17 / 026,071 entitled “Memory with Cache-Coherent Interconnect”, filed September 18, 2020, the entire contents of which are incorporated herein by reference. Technical Field

[0002] One or more aspects of embodiments of this disclosure relate to computing systems, and more specifically, to systems and methods for managing memory resources in a system comprising one or more servers. Background Technology

[0003] This background section is intended to provide context only, and the disclosure of any embodiments or concepts in this section does not constitute an admission that the embodiments or concepts described are prior art.

[0004] Some server systems may include a collection of servers connected via network protocols. Each server in such a system may include processing resources (e.g., processors) and memory resources (e.g., system memory). In some cases, it may be advantageous for the processing resources of one server to access the memory resources of another server, and it may be advantageous for such access to occur while minimizing the processing resources of either server.

[0005] Therefore, there is a need for an improved system and method for managing memory resources in a system that includes one or more servers. Summary of the Invention

[0006] In some embodiments, the server includes one or more processing circuitry, system memory, and one or more memory modules connected to the processing circuitry via a cache coherence interface. The memory modules may also be connected to one or more network interface circuitry. Each memory module may include a controller (e.g., a field-programmable gate array (FPGA) or application-specific integrated circuit (ASIC)) that provides it with enhanced capabilities. These capabilities may include enabling the server to interact with the memory of another server without needing to access a processor (such as a central processing unit (CPU)) (e.g., by performing remote direct memory access (RDMA)).

[0007] According to an embodiment of the present invention, a system is provided, the system comprising: a first server, including a stored program processing circuit, a first network interface circuit, and a first memory module, wherein the first memory module includes a first memory die and a controller, the controller is connected to the first memory die through a memory interface, connected to the stored program processing circuit through a cache coherence interface, and connected to the first network interface circuit.

[0008] In some embodiments, the first memory module further includes a second memory die, the first memory die including volatile memory and the second memory die including persistent memory.

[0009] In some embodiments, the persistent memory includes NAND flash memory. In some embodiments, the controller is configured to provide a flash translation layer for the persistent memory.

[0010] In some embodiments, the cache coherence interface includes a compute fast link (CXL) interface.

[0011] In some embodiments, the first server includes an expansion socket adapter connected to an expansion socket of the first server, the expansion socket adapter including a first memory module and a first network interface circuit.

[0012] In some embodiments, the controller of the first memory module is connected to the stored program processing circuit via an expansion socket.

[0013] In some embodiments, the expansion socket includes an M.2 socket.

[0014] In some embodiments, the controller of the first memory module is connected to the first network interface circuit via a point-to-point peripheral component interconnect (PCIe) connection.

[0015] In some embodiments, the system further includes: a second server; and a network switch connected to the first server and the second server.

[0016] In some embodiments, the network switch includes a top-of-rack (ToR) Ethernet switch.

[0017] In some embodiments, the controller of the first memory module is configured to receive a remote direct memory access (RDMA) request and send an RDMA response.

[0018] In some embodiments, the controller of the first memory module is configured to receive a remote direct memory access (RDMA) request via a network switch and through a first network interface circuit, and to send an RDMA response via a network switch and through the first network interface circuit.

[0019] In some embodiments, the controller of the first memory module is configured to: receive data from a second server; store the data in the first memory module; and send a command to invalidate a cache line to the stored procedure processing circuit.

[0020] In some embodiments, the controller of the first memory module includes a field-programmable gate array (FPGA) or an application-specific integrated circuit (ASIC).

[0021] According to an embodiment of the present invention, a method for performing remote direct memory access in a computing system is provided, the computing system comprising: a first server and a second server, the first server comprising stored program processing circuitry, network interface circuitry, and a first memory module including a controller, the method comprising: receiving a remote direct memory access (RDMA) request by the controller of the first memory module; and sending an RDMA response by the controller of the first memory module.

[0022] In some embodiments, the computing system further includes: an Ethernet switch connected to a first server and a second server; and the step of receiving an RDMA request includes receiving an RDMA request via the Ethernet switch.

[0023] In some embodiments, the method further includes: receiving a read command for a first memory address from a stored program processing circuit by a controller of the first memory module; converting the first memory address to a second memory address by the controller of the first memory module; and retrieving data from the first memory module at the second memory address by the controller of the first memory module.

[0024] In some embodiments, the method further includes: receiving data by a controller of the first memory module; storing the data in the first memory module by the controller of the first memory module; and sending a command for invalidating a cache line to a stored procedure processing circuit by the controller of the first memory module.

[0025] According to an embodiment of the present invention, a system is provided, the system comprising: a first server, including a stored program processing circuit, a first network interface circuit, and a first memory module, wherein the first memory module includes a first memory die and a controller device, the controller device being connected to the first memory die via a memory interface, connected to the stored program processing circuit via a cache coherence interface, and connected to the first network interface circuit. Attached Figure Description

[0026] The accompanying drawings provided herein are for illustrative purposes only; other embodiments, which may not be explicitly shown, are not excluded from the scope of this disclosure.

[0027] These and other features and advantages of this disclosure will be appreciated and understood with reference to the specification, claims and drawings, in which:

[0028] FIG. 1A This is a block diagram of a system for attaching memory resources to computing resources using cache-coherent joins according to embodiments of the present disclosure;

[0029] FIG. 1B This is a block diagram of a system employing an expansion socket adapter for attaching memory resources to computing resources using cache-coherent connections, according to embodiments of the present disclosure.

[0030] FIG. 1C This is a block diagram of a system for aggregating memory using an Ethernet rack ToR switch, according to embodiments of the present disclosure.

[0031] FIG. 1D This is a block diagram of a system employing an Ethernet ToR switch and an expansion socket adapter for aggregated storage, according to embodiments of the present disclosure.

[0032] FIG. 1EThis is a block diagram of a system for a converged memory according to embodiments of the present disclosure;

[0033] FIG. 1F This is a block diagram of a system employing an expansion socket adapter aggregated memory according to embodiments of the present disclosure;

[0034] FIG. 1G This is a block diagram of a system for a de-aggregation server according to embodiments of the present disclosure;

[0035] FIG. 2A According to embodiments of this disclosure FIGS. 1A-1G The flowchart shown in the figure illustrates an example method for performing remote direct memory access (RDMA) transfers using a bypass processing circuitry in an embodiment.

[0036] FIG. 2B According to embodiments of this disclosure FIGS. 1A-1D The flowchart shown in the figure illustrates an example method for performing RDMA transfer with the participation of processing circuitry.

[0037] FIG. 2C According to embodiments of this disclosure FIG. 1E and FIG. 1F The flowchart shown in the figure illustrates an example method for performing RDMA transfers via a Compute Express Link (CXL) switch; and

[0038] FIG. 2D According to embodiments of this disclosure FIG. 1G The flowchart shown in the figure illustrates an example method for performing RDMA transfers via a CXL switch. Detailed Implementation

[0039] The specific embodiments described below with reference to the accompanying drawings are intended as exemplary descriptions of systems and methods for managing memory resources provided in this disclosure, and are not intended to represent the only forms in which this disclosure may be constructed or utilized. The description, in conjunction with the illustrated embodiments, illustrates the features of this disclosure. However, it will be understood that the same or equivalent functions and structures may be implemented by different embodiments, which are also intended to be included within the scope of the disclosure. As shown elsewhere herein, the same element reference numerals are intended to indicate the same elements or features.

[0040] Peripheral Component Interconnect Fast (PCIe) can represent a computer interface with relatively high and variable latency, which can limit the usefulness of the computer interface when connecting to memory. CXL is an open industry standard for communication via PCIe 5.0, which provides fixed, relatively short packet sizes, thus enabling relatively high bandwidth and relatively low fixed latency. Therefore, CXL may be able to support cache coherency, and CXL may be well-suited for connecting to memory. CXL can also be used to provide connectivity between the host and network interface circuitry (or "network interface controller" or "network interface card (NIC)") in accelerators, memory devices, and servers.

[0041] Cache coherence protocols (such as CXL) can also be used for heterogeneous processing (e.g., in scalar, vector, and buffer memory systems). CXL can be used to provide a cache-coherent interface by leveraging the system's PHY layer, retimers, channels, logical aspects of the interface, and protocols from PCIe 5.0. The CXL transaction layer may include three multiplexed sub-protocols running concurrently on a single link, and may be referred to as CXL.io, CXL.cache, and CXL.memory. CXL.io may include I / O semantics similar to PCIe. CXL.cache may include cache semantics, and CXL.memory may include memory semantics; both cache semantics and memory semantics may be optional. Similar to PCIe, CXL supports (i) local widths that can be partitioned x16, x8, and x4, (ii) a data rate of 32GT / s (which can be reduced to 8GT / s and 16GT / s, 128b / 130b), (iii) 300W (75W in x16 connectors), and (iv) plug-and-play. To support plug-and-play, PCIe or CXL device links can begin training in PCIe within Gen1, negotiate CXL, complete Gen1-5 training, and then begin a CXL transaction.

[0042] In some embodiments, as discussed further in detail below, using CXL connections for the aggregation or “pooling” of memory (e.g., a large number of memory units connected together) can provide various advantages in systems comprising multiple servers connected together via a network. For example, a CXL switch (referred to herein as an “enhanced CXL switch”) with capabilities other than providing packet switching for CXL packets can be used to connect the aggregation of memory to one or more central processing units (CPUs) (or “central processing circuits”) and one or more network interface circuits (which may have enhanced capabilities). Such a configuration makes it possible for: (i) the aggregation of memory to include various types of memory with different characteristics; (ii) the enhanced CXL switch to virtualize the aggregation of memory and store data with different characteristics (e.g., access frequency) in appropriate types of memory; and (iii) the enhanced CXL switch to support Remote Direct Memory Access (RDMA), enabling RDMA to be performed with little or no involvement from the processing circuitry of the servers. As used herein, “virtualizing” memory means performing memory address translation between the processing circuitry and the memory.

[0043] CXL switches can (i) support memory and accelerator deaggregation through single-level switching, (ii) enable resources to be offline and online across domains (which can enable time-division multiplexing across domains based on demand), and (iii) support the virtualization of downstream ports. CXL can be used to implement aggregated memory that enables one-to-many and many-to-one switching (e.g., it may be able to (i) connect multiple root ports to one endpoint, (ii) connect one root port to multiple endpoints, or (iii) connect multiple root ports to multiple endpoints), wherein, in some embodiments, the aggregated device is divided into multiple logical devices, each with a corresponding LD-ID (Logical Device Identifier). In such embodiments, physical devices can be divided into multiple logical devices, each visible to a corresponding initiator. A device may have one physical function (PF) and multiple (e.g., 16) isolated logical devices. In some embodiments, the number of logical devices (e.g., the number of partitions) may be limited (e.g., limited to 16), and a control partition (a control partition may be used to control the physical function of the device) may also exist.

[0044] In some embodiments, a fabric manager may be employed to: (i) perform device discovery and virtual CXL software creation, and (ii) bind virtual ports to physical ports. Such a fabric manager may operate via a connection through an SMBus sideband. The fabric manager may be implemented in hardware, software, firmware, or a combination thereof, and may reside, for example, in a host, in one of the memory modules 135, in an enhanced capability CXL switch (or cache coherent switch) 130, or elsewhere in the network. The fabric manager may issue commands (including commands issued via the sideband bus or via the PCIe tree).

[0045] Reference FIG. 1A In some embodiments, the server system includes multiple servers 105 connected together via a top-of-rack (ToR) Ethernet switch 110. While the switch is described as using the Ethernet protocol, any other suitable network protocol may be used. Each server includes one or more processing circuits 115, each processing circuit 115 being connected to (i) system memory 120 (e.g., Double Data Rate (Version 4) (DDR4) memory or any other suitable memory), (ii) one or more network interface circuits (e.g., 100GbE NIC) 125, and (iii) one or more CXL memory modules 135. Each processing circuit 115 may be a stored program processing circuit (e.g., a central processing unit (CPU (e.g., x86 CPU), graphics processing unit (GPU), or ARM processor)). In some embodiments, the network interface circuit 125 may be embedded in one of the memory modules 135 (e.g., on the same semiconductor chip as one of the memory modules 135, or in the same module as one of the memory modules 135), or the network interface circuit 125 may be packaged separately from the memory module 135.

[0046] As used herein, a “memory module” is a package (e.g., a package including a printed circuit board and components connected to the printed circuit board, or an encapsulation including a printed circuit board) comprising one or more memory dies, each memory die comprising multiple memory cells. Each memory die or each set of multiple sets of memory dies may be housed in a package (e.g., an epoxy molding compound (EMC) package) soldered to the printed circuit board of the memory module (or connected to the printed circuit board of the memory module via a connector). Each memory module 135 may have a CXL interface and may include a controller 137 (e.g., an FPGA, ASIC, and / or a processor, etc.) for translating between the CXL package and the memory interface of the memory die (e.g., signals of a memory technology suitable for the memory in memory module 135). As used herein, a “memory interface” of a memory die is an interface inherent to the technology of the memory die (e.g., in the case of DRAM, for example, the memory interface may be word lines and bit lines). The memory module may also include a controller 137 that provides enhancement capabilities as described further in detail below. The controller 137 of each memory module 135 can be connected to the processing circuitry 115 via a cache coherence interface (e.g., via a CXL interface) (e.g., PCIe 5). The controller 137 can also bypass the processing circuitry 115 to facilitate data transfers (e.g., RDMA requests) between different servers 105. The ToR Ethernet switch 110 and network interface circuitry 125 may include an RDMA interface for facilitating RDMA requests between CXL memory devices on different servers (e.g., the ToR Ethernet switch 110 and network interface circuitry 125 can provide hardware offloading or hardware acceleration of RDMA over Converged Ethernet (RoCE), Infiniband, and iWARP packets).

[0047] The CXL interconnect in the system may conform to a cache coherence protocol (such as the CXL 1.1 standard), or in some embodiments, the CXL interconnect in the system may conform to the CXL 2.0 standard, a future version of CXL, or any other suitable protocol (e.g., a cache coherence protocol). As shown, the memory module 135 may be directly attached to the processing circuitry 115, and the top-of-rack Ethernet switch 110 may be used to scale the system to a larger size (e.g., with a larger number of servers 105).

[0048] In some embodiments, such as FIG. 1AAs shown, each server may be populated with memory modules 135 to which multiple directly attached CXLs are attached. Each memory module 135 may expose a set of base address registers (BARs) to the host's basic input / output system (BIOS) as a memory domain. One or more of the memory modules 135 may include firmware for transparently managing their memory space behind the host OS mapping. Each memory module 135 may include one or a combination of memory technologies, such as (but not limited to) Dynamic Random Access Memory (DRAM) technology, NAND flash memory technology, High Bandwidth Memory (HBM) technology, and Low Power Double Data Rate Synchronous Dynamic Random Access Memory (LPDDR SDRAM) technology, and may also include a cache controller or a separate, corresponding discrete controller for different technology memory devices (for memory modules 135 that combine different technologies). Each memory module 135 may include different interface widths (x4-x16 or x48) and may be constructed according to any of a variety of relevant form factors (e.g., U.2, M.2, half-height half-length (HHHL), full-height half-length (FHHL), E1.S, E1.L, E3.S, and E3.H).

[0049] In some embodiments, as described above, the enhanced capability CXL switch 130 includes an FPGA (or ASIC) controller 137 and provides additional features beyond CXL packet switching. The controller 137 of the enhanced capability CXL switch 130 can also serve as a management device for the memory module 135 and assist host control plane processing, and the controller 137 of the enhanced capability CXL switch 130 can implement rich control semantics and statistics. The controller 137 may include additional "backdoor" (e.g., 100 Gigabit Ethernet (GbE)) network interface circuitry 125. In some embodiments, the controller 137 is presented as a Type 2 CXL device to the processing circuitry 115, enabling the issuance of a cache invalidation instruction to the processing circuitry 115 upon receiving a remote write request. In some embodiments, DDIO technology is enabled, where remote data is first pulled to the last-level cache (LLC) of the processing circuitry and then written to the memory module 135 (from the cache). As used herein, a "Type 2" CXL device is a device memory that can initiate transactions and implement optional consistency caching and host management, and the applicable transaction types include all CXL.cache and all CXL.memory transactions.

[0050] As described above, one or more of the memory modules 135 may include persistent memory or a "persistent storage device" (i.e., a storage device whose data is not lost when external power is disconnected). If the memory module 135 is presented as a persistent device, the controller 137 of the memory module 135 may manage a persistent domain (e.g., the controller 137 may store data identified by the processing circuitry 115 as requiring persistent storage in the persistent storage device (e.g., as a result of an application calling a corresponding operating system function)). In such an embodiment, a software accelerator processor interface (API) may cache and flush data to the persistent storage device.

[0051] In some embodiments, direct memory transfer from network interface circuitry 125 to memory module 135 is enabled. Such a transfer can be a one-way transfer to remote memory for fast communication in a distributed system. In such embodiments, memory module 135 can expose hardware details to network interface circuitry 125 in the system for faster RDMA transfers. In such systems, two scenarios can occur depending on whether data direct I / O (DDIO) of processing circuitry 115 is enabled or disabled. DDIO can enable direct communication between the Ethernet controller or Ethernet adapter and the cache of processing circuitry 115. If DDIO of processing circuitry 115 is enabled, the destination of the transfer can be the last-level cache of the processing circuitry, from which data can then be automatically flushed to memory module 135. If DDIO of processing circuitry 115 is disabled, memory module 135 can operate in device bias mode to force access to be received directly by the destination memory module 135 (without DDIO). A network interface circuit 125 with RDMA capability, including a host channel adapter (HCA), buffers, and other processing, can be used to implement such RDMA transfers, which can bypass destination memory buffer transfers that may exist in other RDMA transfer modes. For example, in such embodiments, the use of a bounce buffer (e.g., a buffer in a remote server when the final destination in memory is in an address range not supported by the RDMA protocol) can be avoided. In some embodiments, RDMA uses an alternative physical medium selection besides Ethernet (e.g., for use with a switch configured to handle other network protocols). Examples of RDMA-enabled inter-server connections include (but are not limited to) Infiniband, RDMA over Converged Ethernet (RoCE) (which uses Ethernet User Datagram Protocol (UDP)), and iWARP (which uses Transmission Control Protocol / Internet Protocol (TCP / IP)).

[0052] FIG. 1B Showing with FIG. 1AA similar system exists, in which processing circuitry 115 is connected to network interface circuitry 125 via memory module 135. Memory module 135 and network interface circuitry 125 reside on expansion socket adapters 140. Each expansion socket adapter 140 can be inserted into an expansion socket 145 (e.g., an M.2 connector) on the motherboard of server 105. Therefore, the server can be any suitable (e.g., industry-standard) server modified by installing expansion socket adapters 140 into expansion sockets 145. In such an embodiment, (i) each network interface circuit 125 may be integrated into a corresponding memory module in the memory module 135, or (ii) each network interface circuit 125 may have a PCIe interface (the network interface circuit 125 may be a PCIe endpoint (i.e., a PCIe slave device)) such that the processing circuit 115 to which the network interface circuit 125 is connected (which operates as a PCIe master device or "root port") can communicate with the network interface circuit 125 via a root port to endpoint PCIe connection, and the controller 137 of the memory module 135 can communicate with the network interface circuit 125 via a peer-to-peer PCIe connection.

[0053] According to embodiments of the present invention, a system (e.g., a system for managing memory resources) is provided, the system comprising: a first server including stored program processing circuitry, a first network interface circuitry, and a first memory module, wherein the first memory module includes a first memory die and a controller, the controller being connected to the first memory die via a memory interface, connected to the stored program processing circuitry via a cache coherence interface, and connected to the first network interface circuitry. In some embodiments, the first memory module further includes a second memory die, the first memory die including volatile memory and the second memory die including persistent memory. In some embodiments, the persistent memory includes NAND flash memory. In some embodiments, the controller is configured to provide a flash translation layer for the persistent memory. In some embodiments, the cache coherence interface includes a compute fast link (CXL) interface. In some embodiments, the first server includes an expansion socket adapter connected to an expansion socket of the first server, the expansion socket adapter including the first memory module and the first network interface circuitry. In some embodiments, the controller of the first memory module is connected to the stored program processing circuitry via an expansion socket. In some embodiments, the expansion socket includes an M.2 socket. In some embodiments, the controller of the first memory module is connected to the first network interface circuitry via a point-to-point peripheral component interconnect fast (PCIe) connection. In some embodiments, the system further includes a second server and a network switch connected to the first server and the second server. In some embodiments, the network switch includes a top-of-rack (ToR) Ethernet switch. In some embodiments, the controller of the first memory module is configured to receive a direct remote direct memory access (RDMA) request and send a direct RDMA response. In some embodiments, the controller of the first memory module is configured to receive a direct remote direct memory access (RDMA) request through the network switch and through a first network interface circuit, and to send a direct RDMA response through the network switch and through the first network interface circuit. In some embodiments, the controller of the first memory module is configured to: receive data from the second server; store data in the first memory module; and send a command for invalidating cache lines to the stored program processing circuit. In some embodiments, the controller of the first memory module includes a field-programmable gate array (FPGA) or an application-specific integrated circuit (ASIC). According to embodiments of the present invention, a method for performing remote direct memory access in a computing system is provided, the computing system including a first server and a second server, the first server including stored program processing circuitry, network interface circuitry, and a first memory module including a controller, the method including: receiving a direct remote direct memory access (RDMA) request by the controller of the first memory module; and sending a direct RDMA response by the controller of the first memory module.In some embodiments, the computing system further includes an Ethernet switch connected to a first server and a second server, and the step of receiving a direct RDMA request includes receiving a direct RDMA request via the Ethernet switch. In some embodiments, the method further includes: receiving a read command for a first memory address from a stored-program processing circuit by a controller of the first memory module; converting the first memory address to a second memory address by the controller of the first memory module; and retrieving data from the first memory module at the second memory address by the controller of the first memory module. In some embodiments, the method further includes: receiving data by the controller of the first memory module; storing data in the first memory module by the controller of the first memory module; and sending a command for invalidating cache lines to the stored-program processing circuit by the controller of the first memory module. According to embodiments of the present invention, a system (e.g., a system for managing memory resources) is provided, the system including: a first server including a stored-program processing circuit, a first network interface circuit, and a first memory module, wherein the first memory module includes a first memory die and a controller device, the controller device being connected to the first memory die via a memory interface, connected to the stored-program processing circuit via a cache coherence interface, and connected to the first network interface circuit.

[0054] Reference FIG. 1C In some embodiments, the server system includes multiple servers 105 connected together via a top-of-rack (ToR) Ethernet switch 110. Each server includes one or more processing circuits 115, each processing circuit 115 connected to (i) system memory 120 (e.g., DDR4 memory), (ii) one or more network interface circuits 125, and (iii) an enhancement capability CXL switch 130. The enhancement capability CXL switch 130 may be connected to multiple memory modules (MEMs) 135. That is, FIG. 1C The system includes a first server 105, which includes a stored program processing circuit 115, a network interface circuit 125, a cache coherence switch 130, and a first memory module 135. FIG. 1C In the system, the first memory module 135 is connected to the cache coherence switch 130, the cache coherence switch 130 is connected to the network interface circuit 125, and the stored program processing circuit 115 is connected to the cache coherence switch 130.

[0055] Memory modules 135 may be grouped by type, form factor, or technology type (e.g., DDR4, DRAM, LDPPR, High Bandwidth Memory (HBM), or NAND flash memory or other persistent storage devices (e.g., solid-state drives containing NAND flash memory)). Each memory module may have a CXL interface and include interface circuitry for translating signals between the CXL package and the memory suitable for use in memory module 135. In some embodiments, alternatively, these interface circuitries are in an enhanced capability CXL switch 130, and each memory module 135 has an interface that serves as a local interface to the memory in memory module 135. In some embodiments, the enhanced capability CXL switch 130 is integrated into memory module 135 (e.g., integrated in an M.2 form factor package along with other components of memory module 135, or integrated into a single integrated circuit along with other components of memory module 135).

[0056] The ToR Ethernet switch 110 may include interface hardware for facilitating RDMA requests between aggregated memory devices on different servers. The enhanced capability CXL switch 130 may include one or more circuits (e.g., the enhanced capability CXL switch 130 may include an FPGA or an ASIC) for (i) routing data to different memory types based on workload, (ii) virtualizing host addresses to device addresses, and / or (iii) bypassing the processing circuitry 115 to facilitate RDMA requests between different servers.

[0057] Memory modules 135 may reside in an expansion case (e.g., in the same rack as the motherboard housing the encapsulation) that may include a predetermined number (e.g., more than 20 or more than 100) of memory modules 135 (each memory module 135 inserted into a suitable connector). The memory modules may be in an M.2 form factor, and the connectors may be M.2 connectors. In some embodiments, the connection between servers is via a different network other than Ethernet (e.g., they may be wireless connections such as WiFi or 5G connections). Each processing circuit may be an x86 processor or another processor (e.g., an ARM processor or a GPU). The PCIe link instantiating the CXL link may be PCIe 5.0 or another version (e.g., an earlier version or a later (e.g., future) version (e.g., PCIe 6.0)). In some embodiments, a different cache coherence protocol may be used in the system instead of CXL, or a different cache coherence protocol other than CXL may be used in the system, and a different cache coherence switch may be used instead of the enhanced capability CXL switch 130, or a different cache coherence switch other than the enhanced capability CXL switch 130 may be used. Such a cache coherence protocol may be another standard protocol or a cache coherence variant of a standard protocol (in a manner similar to how CXL is a variant of PCIe 5.0). Examples of standard protocols include, but are not limited to, Non-volatile Dual In-line Memory Module (Version P) (NVDIMM-P), Cache Coherence Interconnect for Accelerators (CCIX), and Open Coherence Accelerator Processor Interface (OpenCAPI).

[0058] System memory 120 may include, for example, DDR4 memory, DRAM, HBM, or LDPPR memory. Memory module 135 may be partitioned or contain a cache controller to handle various memory types. Memory module 135 may be in different form factors, examples of which include, but are not limited to, HHHL, FHHL, M.2, U.2, mezzanine card, daughter card, E1.S, E1.1, E3.1, and E3.S.

[0059] In some embodiments, the system implements an architecture comprising the aggregation of multiple servers, wherein each server is aggregated with multiple memory modules 135 attached to the CXL. Each memory module 135 may contain multiple partitions that may be individually exposed as memory devices to multiple processing circuits 115. Each input port of the enhanced capability CXL switch 130 may independently access multiple output ports of the enhanced capability CXL switch 130 and the memory modules 135 connected thereto. As used herein, an “input port” or “upstream port” of the enhanced capability CXL switch 130 is a port connected to (or adapted to connect to) a PCIe root port, and an “output port” or “downstream port” of the enhanced capability CXL switch 130 is a port connected to (or adapted to connect to) a PCIe endpoint. FIG. 1A In the case of this embodiment, each memory module 135 may expose a set of base address registers (BARs) to the host BIOS as a memory range. One or more of the memory modules 135 may include firmware for transparently managing their memory space behind the host OS mapping.

[0060] In some embodiments, as described above, the Enhanced Capability CXL switch 130 includes an FPGA (or ASIC) controller 137 and provides additional features beyond the switching of CXL packets. For example, the Enhanced Capability CXL switch 130 may (as described above) virtualize memory modules 135 (i.e., operate as a translation layer) to translate between circuit-side addresses (or “processor-side” addresses, i.e., addresses included in memory read and write commands issued by the processing circuitry 115) and memory-side addresses (i.e., addresses used by the Enhanced Capability CXL switch 130 to address storage locations within memory modules 135), thereby masking the physical addresses of memory modules 135 and presenting a virtual aggregation of memory. The controller 137 of the Enhanced Capability CXL switch 130 may also serve as a management device for memory modules 135 and facilitate host control plane processing. The controller 137 may transparently move data without the involvement of the processing circuitry 115 and update the memory mapping (or “address translation table”) accordingly, ensuring that subsequent accesses function as intended. Controller 137 may include switch management devices that (i) can appropriately bind and unbind upstream and downstream connections during operation, and (ii) can implement rich control semantics and statistics associated with data transfers to and from memory module 135. Controller 137 may include additional “backdoor” 100GbE or other network interface circuitry 125 for connecting to other servers 105 or other networking devices. In some embodiments, controller 137 is presented as a Type 2 device to processing circuitry 115, enabling the issuance of a cache invalidation instruction to processing circuitry 115 upon receiving a remote write request. In some embodiments, DDIO technology is enabled, where remote data is first pulled to the last-level cache (LLC) of processing circuitry 115 and then written to memory module 135 (from the cache).

[0061] As described above, one or more of the memory modules 135 may include persistent storage devices. If the memory module 135 is presented as a persistent device, the controller 137 of the enhanced capability CXL switch 130 can manage the persistent domain (e.g., the controller 137 can store data identified as requiring persistent storage by the processing circuitry 115 (e.g., through the use of corresponding operating system functions) in the persistent storage device). In such an embodiment, a software API can cache and flush data to the persistent storage device.

[0062] In some embodiments, it can be similar to the above for... FIG. 1A and FIG. 1BThe embodiments described herein perform direct memory transfer to memory module 135 in a manner that is executed by the controller of memory module 135, wherein the operations performed by the controller of enhanced capability CXL switch 130 are performed by the controller 137.

[0063] As described above, in some embodiments, the memory modules 135 are organized into groups (e.g., a group with high memory density, another group with high HBM congestion, another group with limited density and performance, and another group with high capacity). Such groups may have different form factors or may be based on different technologies. The controller 137 of the enhanced capability CXL switch 130 may intelligently route data and commands based on, for example, workload, tags, or Quality of Service (QoS). For read requests, routing based on such factors may not exist.

[0064] The controller 137 of the enhanced capability CXL switch 130 can also (as described above) virtualize the processing circuit-side address and memory-side address, making it possible for the controller 137 of the enhanced capability CXL switch 130 to determine where data will be stored. The controller 137 of the enhanced capability CXL switch 130 can make such a determination based on information or instructions received from the processing circuitry 115. For example, the operating system may provide memory allocation features that allow an application to specify that a low-latency storage device, a high-bandwidth storage device, or a persistent storage device will be allocated, and such a request initiated by the application can then be taken into consideration by the controller 137 of the enhanced capability CXL switch 130 when determining where (e.g., in which memory module 135) memory is allocated. For example, a high-bandwidth storage device requested by the application can be allocated in a memory module 135 containing HBM, a data persistence storage device requested by the application can be allocated in a memory module 135 containing NAND flash memory, and other storage (not requested by the application) can be stored on a memory module 135 containing relatively inexpensive DRAM. In some embodiments, the controller 137 of the enhanced capability CXL switch 130 may determine where to store specific data based on network usage patterns. For example, the controller 137 of the enhanced capability CXL switch 130 may determine, by monitoring usage patterns, that data in a specific range of physical addresses is accessed more frequently than other data. The controller 137 of the enhanced capability CXL switch 130 may then copy this data to the memory module 135 containing the HBM and modify its address translation table so that the data in the new location is stored in the same range of virtual addresses. In some embodiments, one or more of the memory modules 135 include flash memory (e.g., NAND flash memory), and the controller 137 of the enhanced capability CXL switch 130 implements a flash translation layer for this flash memory. The flash translation layer may support overwriting of processor-side memory locations (by moving data to different locations and marking the previous location of the data as invalid), and the flash translation layer may perform garbage collection (e.g., erasing the block after moving any valid data in the block to another block when the amount of invalid data in the block exceeds a threshold).

[0065] In some embodiments, the controller 137 of the Enhanced Capability CXL switch 130 can facilitate Physical Function (PF) to PF transfers. For example, if one of the processing circuits 115 needs to move data from one physical address to another physical address (which may have the same virtual address; this fact does not need to affect the operation of the processing circuit 115), or if the processing circuit 115 needs to move data between two virtual addresses (the two virtual addresses that the processing circuit 115 will need to have), the controller 137 of the Enhanced Capability CXL switch 130 can supervise the transfer without involving the processing circuit 115. For example, the processing circuit 115 may send a CXL request, and data may be sent from one memory module 135 after the Enhanced Capability CXL switch 130 to another memory module 135 (e.g., data may be copied from one memory module 135 to another memory module 135) without going to the processing circuit 115. In such a scenario, because the processing circuit 115 initiated the CXL request, the processing circuit 115 may need to flush its cache to ensure consistency. If an alternative Type 2 memory device (e.g., one of memory modules 135, or an accelerator that may also be connected to the CXL switch) initiates a CXL request and the switch is not virtualized, the Type 2 memory device may send a message to the processing circuitry 115 to invalidate the cache.

[0066] In some embodiments, the controller 137 of the enhanced capability CXL switch 130 can facilitate RDMA requests between servers. A remote server 105 can initiate such an RDMA request, which can be sent via the ToR Ethernet switch 110 and reach the enhanced capability CXL switch 130 (“local server”) in the server 105 responding to the RDMA request. The enhanced capability CXL switch 130 can be configured to receive such RDMA requests, and it can treat a set of memory modules 135 in the receiving server 105 (i.e., the server receiving the RDMA request) as its own memory space. In the local server, the enhanced capability CXL switch 130 can receive the RDMA request as a direct RDMA request (i.e., an RDMA request not routed through the processing circuitry 115 in the local server), and the enhanced capability CXL switch 130 can send a direct response to the RDMA request (i.e., the enhanced capability CXL switch 130 can send a response without routing through the processing circuitry 115 in the local server). In the remote server, responses (e.g., data sent by the local server) can be received by the enhanced capability CXL switch 130 of the remote server and stored in the memory module 135 of the remote server, without being routed through the processing circuitry 115 in the remote server.

[0067] FIG. 1D Showing with FIG. 1CA system similar to the one described above, in which processing circuitry 115 is connected to network interface circuitry 125 via an enhanced capability CXL switch 130. The enhanced capability CXL switch 130, memory module 135, and network interface circuitry 125 reside on an expansion socket adapter 140. The expansion socket adapter 140 may be a board or module that plugs into an expansion socket (e.g., PCIe connector 145) on the motherboard of server 105. Therefore, the server can be any suitable server that can be modified simply by mounting the expansion socket adapter 140 in the PCIe connector 145. The memory module 135 may be mounted in a connector (e.g., an M.2 connector) on the expansion socket adapter 140. In such an embodiment, (i) network interface circuitry 125 may be integrated into the enhanced capability CXL switch 130, or (ii) each network interface circuitry 125 may have a PCIe interface (the network interface circuitry 125 may be a PCIe endpoint) such that the processing circuitry 115 to which each network interface circuitry 125 is connected communicates with the network interface circuitry 125 via a root port-to-endpoint PCIe connection. The controller 137 of the enhanced capability CXL switch 130 (which may have PCIe input ports connected to the processing circuitry 115 and to the network interface circuitry 125) can communicate with the network interface circuitry 125 via a point-to-point PCIe connection.

[0068] According to embodiments of the present invention, a system is provided, the system comprising: a first server including stored program processing circuitry, network interface circuitry, a cache coherence switch, and a first memory module, wherein the first memory module is connected to the cache coherence switch, the cache coherence switch is connected to the network interface circuitry, and the stored program processing circuitry is connected to the cache coherence switch. In some embodiments, the system further comprises a second memory module connected to the cache coherence switch, wherein the first memory module includes volatile memory and the second memory module includes persistent memory. In some embodiments, the cache coherence switch is configured to virtualize the first memory module and the second memory module. In some embodiments, the first memory module includes flash memory, and the cache coherence switch is configured to provide a flash translation layer for the flash memory. In some embodiments, the cache coherence switch is configured to monitor the access frequency of a first memory location in the first memory module; determine that the access frequency exceeds a first threshold; and copy the contents of the first memory location to a second memory location in the second memory module. In some embodiments, the second memory module includes high-bandwidth memory (HBM). In some embodiments, the cache coherence switch is configured to maintain a table for mapping processor-side addresses to memory-side addresses. In some embodiments, the system further comprises a second server and a network switch connected to the first server and the second server. In some embodiments, the network switch includes a top-of-rack (ToR) Ethernet switch. In some embodiments, the cache-coherent switch is configured to receive Direct Remote Direct Memory Access (RDMA) requests and send Direct RDMA responses. In some embodiments, the cache-coherent switch is configured to receive RDMA requests via the ToR Ethernet switch and via network interface circuitry, and to send Direct RDMA responses via the ToR Ethernet switch and via network interface circuitry. In some embodiments, the cache-coherent switch is configured to support the Compute Fast Link (CXL) protocol. In some embodiments, the first server includes an expansion socket adapter connected to an expansion socket of the first server, the expansion socket adapter including a cache-coherent switch and a memory module socket, the first memory module being connected to the cache-coherent switch via the memory module socket. In some embodiments, the memory module socket includes an M.2 socket. In some embodiments, the network interface circuitry is on the expansion socket adapter.According to embodiments of the present invention, a method for performing remote direct memory access in a computing system is provided. The computing system includes a first server and a second server. The first server includes a stored-program processing circuit, a network interface circuit, a cache coherence switch, and a first memory module. The method includes receiving a direct remote direct memory access (RDMA) request by the cache coherence switch and sending a direct RDMA response by the cache coherence switch. In some embodiments, the computing system further includes an Ethernet switch, and the step of receiving the direct RDMA request includes receiving the direct RDMA request via the Ethernet switch. In some embodiments, the method further includes receiving a read command for a first memory address from the stored-program processing circuit by the cache coherence switch, converting the first memory address to a second memory address by the cache coherence switch, and obtaining data from the first memory module at the second memory address by the cache coherence switch. In some embodiments, the method further includes receiving data by the cache coherence switch, storing the data in the first memory module by the cache coherence switch, and sending a command for invalidating a cache line to the stored-program processing circuit by the cache coherence switch. According to an embodiment of the present invention, a system is provided, the system comprising: a first server, including a stored program processing circuit, a network interface circuit, a cache coherence switching device, and a first memory module, wherein the first memory module is connected to the cache coherence switching device, the cache coherence switching device is connected to the network interface circuit, and the stored program processing circuit is connected to the cache coherence switching device.

[0069] FIG. 1E An embodiment of a plurality of servers 105 connected to a ToR server link switch (or server link switch) 112 is shown. As illustrated, the ToR server link switch 112 may be a PCIe 5.0 CXL switch with PCIe capability (or a ToR PCIe 5 switch). The server link switch 112 may include an FPGA or an ASIC and may provide performance superior to that of an Ethernet switch (in terms of throughput and latency). Each server 105 may include a plurality of memory modules 135 connected to the server link switch 112 via an enhanced capability CXL switch 130 and via a plurality of PCIe connectors. As illustrated, each server 105 may also include one or more processing circuits 115 and system memory 120. As discussed in further detail below, the server link switch 112 may operate as a master device and each enhanced capability CXL switch 130 may operate as a slave device.

[0070] exist FIG. 1EIn this embodiment, the server link switch 112 can group or batch multiple cache requests received from different servers 105, and the server link switch 112 can group packets to reduce control overhead. The enhanced capability CXL switch 130 may include a slave controller (e.g., from an FPGA or an ASIC) for (i) routing data to different memory types based on workload, (ii) virtualizing processor-side addresses to memory-side addresses, and (iii) bypassing the processing circuitry 115 to facilitate consistency requests between different servers 105. FIG. 1E The system shown may be based on CXL 2.0, may include distributed shared memory within a rack, and may use a ToR server to link switch 112 for local connection to remote nodes.

[0071] The ToR server link switch 112 may have additional network connectivity for connecting to other servers or clients (e.g., Ethernet connectivity as shown or other types of connectivity, such as wireless connectivity, such as WiFi or 5G connectivity). The server link switch 112 and the enhanced capability CXL switch 130 may each include a controller, which may be or include processing circuitry (such as an ARM processor). The PCIe interface may be compliant with the PCIe 5.0 standard or an earlier or future version of the PCIe standard, or an interface compliant with a different standard (e.g., NVDIMM-P, CCIX, or OpenCAPI) may be used instead of the PCIe interface. The memory module 135 may include various memory types, including DDR4 DRAM, HBM, LDPPR, NAND flash memory, or solid-state drives (SSDs). The memory module 135 may be partitioned or include a cache controller to handle multiple memory types, and they may be in different form factors (e.g., HHHL, FHHL, M.2, U.2, mezzanine card, daughter card, E1.S, E1.1, E3.1, or E3.S).

[0072] exist FIG. 1EIn this embodiment, the enhanced capability CXL switch 130 can implement one-to-many and many-to-one switching, and the enhanced capability CXL switch 130 can implement a fine-grained load-store interface at the flit (64-byte) level. Each server may have aggregated memory devices, each device being divided into multiple logical devices, each with its own corresponding LD-ID. The ToR switch 112 (which may be referred to as a "server-linked switch") implements one-to-many functionality, and the enhanced capability CXL switch 130 in server 105 implements many-to-one functionality. The server-linked switch 112 may be a PCIe switch or a CXL switch or both. In such a system, the requesting party may be the processing circuitry 115 of multiple servers 105, and the responding party may be a plurality of aggregated memory modules 135. The hierarchical structure of the two switches (as described above, the master switch is the server-linked switch 112, and the slave switch is the enhanced capability CXL switch 130) enables any communication. Each memory module 135 may have a physical function (PF) and up to 16 isolated logical devices. In some embodiments, the number of logical devices (e.g., the number of partitions) may be limited (e.g., limited to 16), and a control partition may also exist (this may be for controlling the physical functions of the devices). Each memory module 135 may be a Type 2 device with CXL.cache, CXL.memory, and CXL.io implementations, and an Address Translation Service (ATS), for processing cache line copies that can be stored by the processing circuitry 115. The enhanced capability CXL switch 130 and the structure manager can control the discovery of memory modules 135 and (i) perform device discovery and virtual CXL software creation, and (ii) bind virtual ports to physical ports. FIGS. 1A-1D As in the embodiments described, the structure manager can operate via a connection through the SMBus sideband. The interface of the memory module 135 is configurable and can be an Intelligent Platform Management Interface (IPMI) or a Redfish-compliant interface (and said interface may also provide additional features not required by the standard).

[0073] As described above, some embodiments implement a hierarchical architecture (in which the master controller (which may be implemented in an FPGA or an ASIC) is part of the server-linked switch 112, and the slave controllers are part of the enhanced capability CXL switch 130) to provide a load-store interface (i.e., an interface with cache line (e.g., 64-byte) granularity and operating within a consistency domain without the involvement of a software driver). Such a load-store interface extends the consistency domain beyond a single server, CPU, or host and may involve a physical medium that is either electrical or optical (e.g., an optical connection with an electrical-to-optical transceiver at both ends). In operation, the master controller (in the server-linked switch 112) boots up (or “reboots”) and configures all servers 105 on the rack. The master controller is visible on all hosts and can (i) discover each server and how many servers 105 and memory modules 135 exist in the server cluster, (ii) configure each server 105 independently, (iii) enable or disable some memory blocks on different servers based on, for example, rack configuration (e.g., enable or disable any memory module in memory module 135), (iv) control access (e.g., which server can control which other server), (v) implement flow control (e.g., since all host and device requests pass through the master device, the master controller can send data from one server to another and perform flow control on the data), (vi) group or batch requests or packets (e.g., the master device receives multiple cache requests from different servers 105), and (vii) receive remote software updates, broadcast communications, etc. In batch mode, the server link switch 112 can receive multiple packets destined for the same server (e.g., destined for the first server) and send the multiple packets together (i.e., without pauses between the multiple packets) to the first server. For example, server link switch 112 can receive a first packet from a second server, a second packet from a third server, and send the first and second packets together to the first server. Each server 105 can expose (i) an IPMI network interface, (ii) a System Event Log (SEL), and (iii) a Board Management Controller (BMC) to the main controller, enabling the main controller to measure performance, measure operational reliability, and reconfigure server 105.

[0074] In some embodiments, a software architecture that promotes high availability through a load-store interface is used. Such a software architecture provides reliability, replication, consistency, system coherence, hashing, caching, and persistence. The software architecture provides reliability (in systems with a large number of servers) by performing periodic hardware checks on CXL device components via IPMI. For example, server link switch 112 can query the status of storage server 150 via the IPMI interface, querying parameters such as power status (whether the power supply to storage server 150 is functioning correctly), network status (whether the interface to server link switch 112 is functioning correctly), and error checking status (whether there are error conditions in any subsystem of storage server 150). The software architecture provides replication because the master controller can replicate data stored in storage module 135 and maintain data consistency across replicas.

[0075] The software architecture provides consistency because the master controller can be configured with different consistency levels, and the server link switch 112 can adjust the packet format according to the consistency level to be maintained. For example, if eventual consistency is maintained, the server link switch 112 can reorder requests, while to maintain strict consistency, the server link switch 112 can maintain a scoreboard of all requests with precise timestamps on the switch. The software architecture provides system consistency because multiple processing circuits 115 can read from or write to the same memory address, and to maintain consistency, the master controller can be responsible for (using directory lookup) the home node of the address or broadcast requests on the common bus.

[0076] The software architecture provides hashing because the server link switch 112 and the enhanced CXL switch 130 maintain a virtual mapping of addresses, which can use consistent hashing with multiple hashing capabilities to evenly map data across all CXL devices across all nodes at startup (or adjust for server failures or occurrences). The software architecture provides caching because the master controller can (e.g., in memory modules 135 including HBM or similar technologies) designate specific memory partitions to act as caches (e.g., using write-through or write-back caching). The software architecture provides persistence because the master and slave controllers manage persistence domains and flushing.

[0077] In some embodiments, the capabilities of the CXL switch are integrated into the controller of the memory module 135. In such embodiments, the server link switch 112 can still be used as a master device and has the enhanced features discussed elsewhere here. The server link switch 112 can also manage other storage devices in the system, and the server link switch 112 may have Ethernet connectivity (e.g., 100GbE connectivity) for connecting to client machines, for example, those not part of the PCIe network formed by the server link switch 112.

[0078] In some embodiments, server link switch 112 has enhanced capabilities and also includes an integrated CXL controller. In other embodiments, server link switch 112 is merely a physical routing device, and each server 105 includes a master CXL controller. In such embodiments, the master devices on different servers can negotiate a master-slave architecture. The intelligent functions of (i) the enhanced capability CXL switch 130 and (ii) the server link switch 112 can be implemented with one or more FPGAs, one or more ASICs, one or more ARM processors, or one or more SSD devices with computing power. Server link switch 112 can perform flow control, for example, by reordering independent requests. In some embodiments, RDMA is optional because the interface is load-store, but intermediate RDMA requests using PCIe physical media (instead of 100GbE) may exist. In such embodiments, a remote host can initiate an RDMA request that can be sent to enhanced capability CXL switch 130 via server link switch 112. Server link switch 112 and enhanced capability CXL switch 130 can prioritize RDMA 4KB requests or CXL chip (64-byte) requests.

[0079] like FIG. 1C and FIG. 1D As in the embodiments described, the Enhanced Capability CXL switch 130 can be configured to receive such RDMA requests, and the Enhanced Capability CXL switch 130 can treat a set of memory modules 135 in the receiving server 105 (i.e., the server receiving the RDMA request) as its own memory space. Furthermore, the Enhanced Capability CXL switch 130 can be virtualized across processing circuitry 115 and can initiate RDMA requests to remote Enhanced Capability CXL switches 130 to move data back and forth between servers 105 without involving processing circuitry 115.

[0080] FIG. 1F Showing something similar to FIG. 1E The system, wherein processing circuitry 115 is connected to network interface circuitry 125 (e.g., a PCIe 5 connector) via an enhanced capability CXL switch 130.FIG. 1D As in the embodiments, in FIG. 1F In this configuration, the enhanced capability CXL switch 130, memory module 135, and network interface circuitry 125 are located on an expansion socket adapter 140. The expansion socket adapter 140 can be a board or module that plugs into an expansion socket (e.g., PCIe connector 145) on the motherboard of server 105. Therefore, the server can be any suitable server that can be modified simply by installing the expansion socket adapter 140 in the PCIe connector 145. The memory module 135 can be installed in a connector (e.g., an M.2 connector) on the expansion socket adapter 140. In such an embodiment, (i) the network interface circuit 125 may be integrated into the enhanced capability CXL switch 130, or (ii) each network interface circuit 125 may have a PCIe interface (the network interface circuit 125 may be a PCIe endpoint) such that the processing circuit 115 to which each network interface circuit 125 is connected can communicate with the network interface circuit 125 via a root port to endpoint PCIe connection, and the controller 137 of the enhanced capability CXL switch 130 (which may have PCIe input ports connected to the processing circuit 115 and the network interface circuit 125) can communicate with the network interface circuit 125 via a point-to-point PCIe connection.

[0081] According to embodiments of the present invention, a system is provided, the system comprising: a first server including stored program processing circuitry, a cache coherent switch, and a first memory module; a second server; and a server link switch connected to the first server and the second server, wherein the first memory module is connected to the cache coherent switch, the cache coherent switch is connected to the server link switch, and the stored program processing circuitry is connected to the cache coherent switch. In some embodiments, the server link switch includes a Peripheral Component Interconnect Fast (PCIe) switch. In some embodiments, the server link switch includes a Compute Fast Link (CXL) switch. In some embodiments, the server link switch includes a Top-of-Rack (ToR) CXL switch. In some embodiments, the server link switch is configured to discover the first server. In some embodiments, the server link switch is configured to restart the first server. In some embodiments, the server link switch is configured to disable the first memory module by the cache coherent switch. In some embodiments, the server link switch is configured to send data from the second server to the first server and perform flow control on the data. In some embodiments, the system further includes a third server connected to the server link switch, wherein the server link switch is configured to: receive a first packet from the second server, receive a second packet from the third server, and send the first packet and the second packet to the first server. In some embodiments, the system further includes a second memory module connected to a cache-coherent switch, wherein the first memory module includes volatile memory and the second memory module includes persistent memory. In some embodiments, the cache-coherent switch is configured to virtualize the first and second memory modules. In some embodiments, the first memory module includes flash memory, and the cache-coherent switch is configured to provide a flash translation layer for the flash memory. In some embodiments, the first server includes an expansion socket adapter connected to an expansion socket of the first server, the expansion socket adapter including a cache-coherent switch and a memory module socket, the first memory module being connected to the cache-coherent switch via the memory module socket. In some embodiments, the memory module socket includes an M.2 socket. In some embodiments, the cache-coherent switch is connected to a server link switch via a connector, and the connector is on the expansion socket adapter.According to embodiments of the present invention, a method for performing remote direct memory access in a computing system is provided. The computing system includes a first server, a second server, a third server, and a server link switch connected to the first server, the second server, and the third server. The first server includes a stored-program processing circuit, a cache coherent switch, and a first memory module. The method includes: receiving a first packet from a second server via the server link switch; receiving a second packet from a third server via the server link switch; and sending the first packet and the second packet to the first server. In some embodiments, the method further includes: receiving a direct remote direct memory access (RDMA) request via the cache coherent switch; and sending a direct RDMA response via the cache coherent switch. In some embodiments, the step of receiving the direct RDMA request includes receiving the direct RDMA request via the server link switch. In some embodiments, the method further includes: receiving a read command for a first memory address from the stored-program processing circuit via the cache coherent switch; converting the first memory address to a second memory address via the cache coherent switch; and obtaining data from the first memory module at the second memory address via the cache coherent switch. According to an embodiment of the present invention, a system is provided, the system comprising: a first server including a stored program processing circuit, a cache coherence switching device, a first memory module, a second server, and a server link switch connected to the first server and the second server, wherein the first memory module is connected to the cache coherence switching device, the cache coherence switching device is connected to the server link switch, and the stored program processing circuit is connected to the cache coherence switching device.

[0082] FIG. 1G An embodiment is shown in which each of a plurality of storage servers 150 is connected to a ToR server link switch 112, as shown, whereby the ToR server link switch 112 may be a PCIe 5.0 CXL switch (e.g., a ToR PCIe 5 CXL switch). FIG. 1E and FIG. 1F As in the embodiments described, the server link switch 112 may include an FPGA or ASIC and can provide performance superior to that of an Ethernet switch (in terms of throughput and latency). FIG. 1G and FIG. 1G As in the embodiments described, the storage server 150 may include multiple storage modules 135 connected to the server link switch 112 via multiple PCIe connectors. FIG. 1G In some embodiments, the processing circuitry 115 and system memory 120 may be absent, and the primary purpose of the memory server 150 may be to provide memory for use by other servers 105 with computing resources.

[0083] exist FIG. 1G In this embodiment, the server link switch 112 can group or batch multiple cache requests received from different memory servers 150, and the server link switch 112 can group packets to reduce control overhead. The enhanced capability CXL switch 130 may include composable hardware building blocks for (i) routing data to different memory types based on workload and (ii) virtualizing processor-side addresses (translating such addresses to memory-side addresses). FIGS. 1A-1D The system shown may be based on CXL 2.0, which may include composable and deaggregate shared memory within a rack, and may use a ToR server to link switch 112 to provide centralized (i.e., aggregated) memory to remote devices.

[0084] The ToR server link switch 112 may have additional network connectivity for connecting to other servers or clients (e.g., Ethernet connectivity as shown or other types of connectivity, such as wireless connectivity, such as WiFi or 5G connectivity). The server link switch 112 and the enhanced capability CXL switch 130 may each include a controller, which may be or include processing circuitry (such as an ARM processor). The PCIe interface may conform to the PCIe 5.0 standard or an earlier or future version of the PCIe standard, or a different standard (e.g., NVDIMM-P, CCIX, or OpenCAPI) may be adopted instead of PCIe. The memory module 135 may include various memory types, including DDR4 DRAM, HBM, LDPPR, NAND flash memory, and solid-state drives (SSDs). The memory module 135 may be partitioned or include a cache controller to handle multiple memory types, and they may be in different form factors (e.g., HHHL, FHHL, M.2, U.2, mezzanine card, daughter card, E1.S, E1.1, E3.1, or E3.S).

[0085] exist FIG. 1GIn this embodiment, the Enhanced Capability CXL switch 130 can implement one-to-many and many-to-one switching, and the Enhanced Capability CXL switch 130 can implement fine-grained load-store interfaces at the microchip (64-byte) level. Each memory server 150 may have aggregated memory devices, each device being divided into multiple logical devices, each with a corresponding LD-ID. The Enhanced Capability CXL switch 130 may include a controller 137 (e.g., an ASIC or FPGA) and circuitry for device discovery, enumeration, partitioning, and presentation of physical address ranges (this may be separate from or part of such an ASIC or FPGA). Each memory module 135 may have a physical function (PF) and up to 16 isolated logical devices. In some embodiments, the number of logical devices (e.g., the number of partitions) may be limited (e.g., limited to 16), and a control partition (which may be for controlling the physical function of the devices) may also exist. Each memory module 135 may be a Type 2 device with CXL.cache, CXL.memory, and CXL.io and Address Translation Service (ATS) implementations for processing cache line copies that can be stored by the processing circuitry 115.

[0086] The enhanced capability CXL switch 130 and the structure manager can control the discovery of the memory module 135, and (i) perform device discovery and virtual CXL software creation and (ii) bind virtual ports to physical ports. FIG. 1G As in the embodiments described, the structure manager can operate via a connection through the SMBus sideband. The interface of the memory module 135 is configurable and can be an Intelligent Platform Management Interface (IPMI) or a Redfish-compliant interface (which may also provide additional features not required by the standard).

[0087] against FIG. 1G In some embodiments, the building blocks may include (as described above) a CXL controller 137 implemented on an FPGA or ASIC and firmware, the CXL controller 137 being switched to implement memory devices (e.g., memory devices of memory module 135), SSDs, accelerators (GPUs, NICs), CXL and PCIe 5 connectors, and the firmware being used to expose device details to the operating system’s advanced configuration and power interface (ACPI) tables (such as the Heterogeneous Memory Attribute Table (HMAT) or Static Resource Association Table (SRAT)).

[0088] In some embodiments, the system provides composability. The system may provide the ability to bring CXL devices and other accelerators online or offline based on software configuration, and the system may be able to group accelerators, memory, and storage device resources and allocate them to each memory server 150 in the rack. The system may hide the physical address space and provide transparent caching using faster devices such as HBM and SRAM.

[0089] exist FIG. 1G In this embodiment, the controller 137 of the Enhanced Capability CXL switch 130 can (i) manage the memory module 135, (ii) integrate and control heterogeneous devices (such as NICs, SSDs, GPUs, DRAM), and (iii) dynamically reconfigure the storage of the memory devices via power gating. For example, the ToR server link switch 112 can disable the power of one of the memory modules 135 (i.e., turn off the power or reduce the power) by instructing the Enhanced Capability CXL switch 130 to disable the power of the memory module 135. The Enhanced Capability CXL switch 130 can then disable the power of the memory module 135 when instructed by the server link switch 112 to disable the power of the memory module. Such disabling saves power and improves the performance (e.g., throughput and latency) of other memory modules 135 in the memory server 150. Each remote memory server 150 can see a different logical view of the memory module 135 and its connections based on negotiation. The controller 137 of the enhanced capability CXL switch 130 can maintain state, allowing each remote storage server to retain allocated resources and connectivity, and the controller 137 of the enhanced capability CXL switch 130 can (using configurable block sizes) perform storage compression or deduplication to save storage capacity. FIG. 1E The depolymerization rack can have its own BMC. FIGS. 2A-2D The deaggregation rack can also expose the IPMI network interface and system event log (SEL) to remote devices, enabling the master device (e.g., a remote server using storage provided by storage server 150) to measure performance and reliability during operation and reconfigure the deaggregation rack. FIG. 2A The depolymerization rack can be similar to that used here for FIG. 1AThe embodiments described provide reliability, replication, consistency, system consistency, hashing, caching, and persistence, wherein, for example, multiple remote servers reading from or writing to the same memory address are provided with consistency, and each remote server is configured with a different consistency level. In some embodiments, the server link switch maintains eventual consistency between data stored on a first memory server and data stored on a second memory server. The server link switch 112 may maintain different consistency levels for different server pairs; for example, the server link switch may also maintain strict consistency, sequential consistency, causal consistency, or processor consistency levels between data stored on a first memory server and data stored on a third memory server. The system may employ communication in a "local band" (server link switch 112) domain and a "global band" (de-aggregating server) domain. Writes may be flushed to the "global band" to make new reads from other servers visible. The controller 137 of the enhanced capability CXL switch 130 may manage persistence domains and flushing individually for each remote server. For example, a cache-coherent switch can monitor the fullness of a first region of memory (volatile memory, operating as a cache), and when the fullness level exceeds a threshold, the cache-coherent switch can move data from the first region of memory to a second region of memory, which is in persistent memory. Flow control can be handled because priorities can be established in a remote server via the controller 137 of the enhanced capability CXL switch 130 to present different perceived latency and bandwidth.

[0090] According to embodiments of the present invention, a system is provided, the system comprising: a first memory server including a cache coherence switch and a first memory module; a second memory server; and a server link switch connected to the first memory server and the second memory server, wherein the first memory module is connected to the cache coherence switch, and the cache coherence switch is connected to the server link switch. In some embodiments, the server link switch is configured to disable power to the first memory module. In some embodiments, the server link switch is configured to disable power to the first memory module by instructing the cache coherence switch to disable power to the first memory module, and the cache coherence switch is configured to disable power to the first memory module when instructed to do so by the server link switch. In some embodiments, the cache coherence switch is configured to perform deduplication within the first memory module. In some embodiments, the cache coherence switch is configured to compress data and store the compressed data in the first memory module. In some embodiments, the server link switch is configured to query the status of the first memory server. In some embodiments, the server link switch is configured to query the status of the first memory server via an Intelligent Platform Management Interface (IPMI). In some embodiments, the step of querying the status includes querying a status selected from a group consisting of power status, network status, and error checking status. In some embodiments, the server link switch is configured to batch cache requests directed to a first memory server. In some embodiments, the system further includes a third memory server connected to the server link switch, wherein the server link switch is configured to maintain a consistency level selected from a group consisting of strict consistency, sequential consistency, causal consistency, and processor consistency between data stored on the first memory server and data stored on the third memory server. In some embodiments, the cache consistency switch is configured to: monitor the fullness of a first region of memory and move data from the first region of memory to a second region of memory, wherein the first region of memory is in volatile memory and the second region of memory is in persistent memory. In some embodiments, the server link switch includes a Peripheral Component Interconnect Fast (PCIe) switch. In some embodiments, the server link switch includes a Compute Fast Link (CXL) switch. In some embodiments, the server link switch includes a Top-of-Rack (ToR) CXL switch. In some embodiments, the server link switch is configured to send data from a second memory server to a first memory server and perform flow control on the data.In some embodiments, the system further includes a third memory server connected to a server link switch, wherein the server link switch is configured to: receive a first packet from a second memory server, receive a second packet from the third memory server, and send the first and second packets to the first memory server. According to embodiments of the present invention, a method for performing remote direct memory access in a computing system is provided, the computing system including a first memory server, a first server, a second server, and a server link switch connected to the first memory server, the first server, and the second server. The first memory server includes a cache coherence switch and a first memory module. The first server includes stored-program processing circuitry, and the second server includes stored-program processing circuitry. The method includes: receiving a first packet from the first server by the server link switch; receiving a second packet from the second server by the server link switch; and sending the first and second packets to the first memory server. In some embodiments, the method further includes: compressing data by the cache coherence switch and storing the compressed data in the first memory module. In some embodiments, the method further includes: querying the status of the first memory server by the server link switch. According to an embodiment of the present invention, a system is provided, the system comprising: a first memory server including a cache coherence switch and a first memory module; a second memory server; and a server link switching device connected to the first memory server and the second memory server, wherein the first memory module is connected to the cache coherence switch, and the cache coherence switch is connected to the server link switching device.

[0091] FIG. 1B These are flowcharts of various embodiments. In these flowchart embodiments, processing circuitry 115 is a CPU; in other embodiments, processing circuitry 115 may be other processing circuitry (e.g., a GPU). See also... FIGS. 1C-1G , FIG. 1A and FIG. 1B The controller 137 of the memory module 135 in the embodiment or FIG. 2BThe enhanced capability CXL switch 130 in any embodiment can be virtualized across processing circuitry 115 and initiate RDMA requests to the enhanced capability CXL switch 130 in another server 105 to move data back and forth between servers 105 without involving the processing circuitry 115 in any server (wherein, virtualization is handled by the controller 137 of the enhanced capability CXL switch 130). For example, at 205, the controller 137 of memory module 135 or the enhanced capability CXL switch 130 generates an RDMA request for an additional remote memory (e.g., CXL memory or aggregated memory); at 210, network interface circuitry 125 bypasses the processing circuitry and sends the request to ToR Ethernet switch 110 (which may have an RDMA interface); at 215, ToR Ethernet switch 110 bypasses the remote processing circuitry 115 and routes the RDMA request to a remote server 105 for processing by the controller 137 of memory module 135 or by the remote enhanced capability CXL switch 130 via RDMA access to the remote aggregated memory; at 220, ToR Ethernet switch 110 receives the processed data via RDMA bypass of the local processing circuitry 115 and routes the data to the local memory module 135 or to the local enhanced capability CXL switch 130; and at 222, FIG. 1A and FIG. 1B In the embodiment, the controller 137 of the memory module 135 or the enhanced capability CXL switch 130 directly receives the RDMA response (e.g., without the RDMA response being forwarded by the processing circuitry 115).

[0092] In such an embodiment, the controller 137 of the remote memory module 135 or the enhanced capability CXL switch 130 of the remote server 105 is configured to receive direct remote direct memory access (RDMA) requests and send direct RDMA responses. As used herein, the controller 137 of the remote memory module 135 receiving or the enhanced capability CXL switch 130 receiving a “direct RDMA request” (or “directly” receiving such a request) means that such a request is received by the controller 137 of the remote memory module 135 or by the enhanced capability CXL switch 130 without needing to be forwarded or otherwise processed by the processing circuitry 115 of the remote server, and the controller 137 of the remote memory module 135 or the enhanced capability CXL switch 130 sending a “direct RDMA response” (or “directly” sending such a request) means sending such a response without being forwarded or otherwise processed by the processing circuitry 115 of the remote server.

[0093] Reference FIG. 2CIn another embodiment, RDMA can be performed with the processing circuitry of a remote server involved in processing the data. For example, at 225, processing circuitry 115 can send data or workload requests via Ethernet; at 230, ToR Ethernet switch 110 can receive requests and route them to the appropriate server 105 among multiple servers 105; at 235, requests can be received within the server via one or more ports of network interface circuitry 125 (e.g., a 100GbE-enabled NIC); at 240, processing circuitry 115 (e.g., x86 processing circuitry) can receive requests from network interface circuitry 125; and at 245, processing circuitry 115 can process requests (e.g., together) using DDR and additional memory resources via the CXL 2.0 protocol to share memory (in FIG. 1E and FIG. 1A In some embodiments, the memory may be aggregated memory.

[0094] Reference FIG. 1B ,exist FIG. 2D In some embodiments, RDMA can be performed with the processing circuitry of a remote server participating in data processing. For example, at 225, processing circuitry 115 can send data or workload requests via Ethernet or PCIe; at 230, ToR Ethernet switch 110 can receive requests and route them to the appropriate server among multiple servers 105; at 235, requests can be received within the server via one or more ports of a PCIe connector; at 240, processing circuitry 115 (e.g., x86 processing circuitry) can receive requests from network interface circuitry 125; and at 245, processing circuitry 115 can process requests (e.g., together) using DDR and additional memory resources via the CXL 2.0 protocol to share memory (in FIG. 1G and FIG. 1GIn some embodiments, the memory may be aggregated memory. At 250, processing circuitry 115 can identify a need to access memory content (e.g., DDR or aggregated memory content) from different servers; at 252, processing circuitry 115 can send a request for the memory content (e.g., DDR or aggregated memory content) from the different servers via a CXL protocol (e.g., CXL 1.1 or CXL 2.0); at 254, the request is propagated to server link switch 112 via a local PCIe connector, and server link switch 112 then sends the request to a second PCIe connector of a second server on the rack; at 256, second processing circuitry 115 (e.g., x86 processing circuitry) receives the request from the second PCIe connector; at 258, second processing circuitry 115 can process the request (e.g., retrieve memory content) together with second DDR and second additional memory resources via a CXL 2.0 protocol to share aggregated memory; and at 260, second processing circuitry (e.g., x86 processing circuitry) sends the result of the request back to the original processing circuitry via the corresponding PCIe connector and through server link switch 112.

[0095] Reference ​ ,exist ​ In this embodiment, RDMA can be performed with the processing circuitry of the remote server involved in processing the data. At 225, the processing circuitry 115 can send data or workload requests via Ethernet; at 230, the ToR Ethernet switch 110 can receive requests and route them to the corresponding server 105 among a plurality of servers 105; at 235, requests can be received within the server via one or more ports of the network interface circuitry 125 (e.g., a 100GbE-enabled NIC). At 262, memory module 135 receives a request from the PCIe connector; at 264, the controller of memory module 135 processes the request using local memory; at 250, the controller of memory module 135 identifies a need to access memory content from different servers (e.g., aggregated memory content); at 252, the controller of memory module 135 sends a request for the memory content from the different servers (e.g., aggregated memory content) via the CXL protocol; at 254, the request is propagated through the local PCIe connector to server link switch 112, which then forwards the request to a second PCIe connector of a second server on the rack; and at 266, the second PCIe connector provides access via the CXL protocol to share the aggregated memory, thereby allowing the controller of memory module 135 to access the memory content.

[0096] As used herein, a "server" is a computing system that includes at least one stored program processing circuitry (e.g., processing circuitry 115), at least one memory resource (e.g., system memory 120), and at least one circuitry for providing network connectivity (e.g., network interface circuitry 125). As used herein, "part" of something means "at least some" of that thing, and therefore may mean less than or may mean the whole of that thing. Thus, "part" of something includes the whole thing as a special case (i.e., an example where the whole thing is part of the thing).

[0097] The background technology provided in the Background section of this disclosure is included only for setting the context, and the content of this section is not intended to be prior art. Any component or combination of components described (e.g., any component or combination of components described in any system diagram included herein) can be used to perform one or more operations of any flowchart included herein. Furthermore, (i) the operations are example operations and may involve various additional steps not explicitly covered, and (ii) the temporal order of the operations may be changed.

[0098] The terms "processing circuitry" or "controller device" are used herein to mean any combination of hardware, firmware, and software for processing data or digital signals. Processing circuitry hardware may include, for example, application-specific integrated circuits (ASICs), general-purpose or special-purpose central processing units (CPUs), digital signal processors (DSPs), graphics processing units (GPUs), and programmable logic devices (such as field-programmable gate arrays (FPGAs)). In processing circuitry, as used herein, each function is performed by hardware configured (i.e., hardwired) to perform that function, or by more general-purpose hardware (such as a CPU) configured to execute instructions stored in a non-transitory storage medium. Processing circuitry may be fabricated on a single printed circuit board (PCB) or distributed across several interconnected PCBs. Processing circuitry may include other processing circuitry; for example, processing circuitry may include two processing circuits (FPGA and CPU) interconnected on a PCB.

[0099] As used herein, "controller" includes circuitry, and a controller may also be referred to as "control circuitry" or "controller circuitry." Similarly, "memory module" may also be referred to as "memory module circuitry" or simply "memory circuitry." As used herein, the term "array" refers to an ordered collection of numbers, regardless of how they are stored (e.g., whether stored in contiguous memory locations or in a linked list). As used herein, "when the second number is within 'Y%' of the first number" means that the second number is at least (1-Y / 100) times the first number and at most (1+Y / 100) times the first number. As used herein, the term "or" should be interpreted as "and / or," such that, for example, "A or B" means either "A" or "B" or "A and B."

[0100] As used herein, when a method (e.g., adjustment) or a first quantity (e.g., a first variable) is referred to as “based on” a second quantity (e.g., a second variable), it means that the second quantity is an input to the method or affects the first quantity (e.g., the second quantity may be an input to a function that calculates the first quantity (e.g., a unique input or one of several inputs)), or the first quantity may be equal to the second quantity, or the first quantity may be the same as the second quantity (e.g., stored in the same one or more locations in memory).

[0101] It will be understood that although terms such as "first," "second," "third," etc., may be used herein to describe various elements, components, regions, layers, and / or portions, these elements, components, regions, layers, and / or portions should not be limited by these terms. These terms are used only to distinguish one element, component, region, layer, or portion from another element, component, region, layer, or portion. Therefore, without departing from the spirit and scope of the inventive concept, the first element, first component, first region, first layer, or first portion discussed herein can be referred to as a second element, second component, second region, second layer, or second portion.

[0102] For ease of description, spatial relative terms (such as "below", "under", "below", "below", "above", "above", etc.) may be used herein to describe the relationship between one element or feature as shown in the accompanying drawings and one or more other elements or features. It will be understood that, in addition to the orientations depicted in the accompanying drawings, such spatial relative terms are intended to encompass different orientations of the device in use or operation. For example, if the device in the accompanying drawings is flipped, an element described as "below" or "below" or "below" other elements or features will subsequently be positioned "above" that other element or feature. Thus, the example terms "below" or "below" may include both the orientations "above" and "below". The device may be otherwise positioned (e.g., rotated 90 degrees or in other orientations), and the spatial relative descriptions used herein should be interpreted accordingly. Furthermore, it will be understood that when a layer is referred to as "between" two layers, it may be the only layer between the two layers, or one or more intermediate layers may exist.

[0103] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the inventive concept. As used herein, the terms “substantially,” “about,” and similar terms are used as approximate terms rather than terms of degree and are intended to account for inherent deviations in measured or calculated values ​​that will be recognized by one of ordinary skill in the art. As used herein, the singular form is intended to also include the plural form unless the context clearly indicates otherwise. It will also be understood that when the terms “comprising” and / or “including” are used in this specification, they indicate the presence of the described features, integrals, steps, operations, elements, and / or components, but do not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components, and / or groups thereof. As used herein, the term “and / or” includes any and all combinations of one or more of the associated listed items. Expressions such as “at least one of…” when following an element in a list modify the entire list, not individual elements within the list. Furthermore, the use of “may” in describing embodiments of the inventive concept indicates “one or more embodiments of this disclosure.” Additionally, the term “exemplary” is intended to indicate an example or illustration. As used herein, the terms “use,” “being used,” and “being used” may be considered synonymous with the terms “exploitation,” “being exploited,” and “being exploited.”

[0104] It will be understood that when an element or layer is referred to as being "on" another element or layer, "connected to", "bonded to", or "adjacent to" another element or layer, the element or layer may be directly on, directly connected to, directly bonded to, or directly adjacent to the other element or layer, or one or more intermediate elements or layers may be present. Conversely, when an element or layer is referred to as being "directly on" another element or layer, "directly connected to", "directly bonded to", or "immediately adjacent to" another element or layer, no intermediate elements or layers are present.

[0105] Any numerical ranges listed herein are intended to include all subranges of the same numerical precision falling within the listed range. For example, the range “1.0 to 10.0” or “between 1.0 and 10.0” is intended to include all subranges between the listed minimum value 1.0 and the listed maximum value 10.0 (and includes both the listed minimum value 1.0 and the listed maximum value 10.0) (i.e., all subranges having a minimum value equal to or greater than 1.0 and a maximum value equal to or less than 10.0 (e.g., 2.4 to 7.6). Any maximum numerical limit listed herein is intended to include all lower numerical limits falling within it, and any minimum numerical limit listed in this specification is intended to include all higher numerical limits falling within it.

[0106] Although exemplary embodiments of systems and methods for managing memory resources have been specifically described and illustrated herein, many modifications and variations will be apparent to those skilled in the art. Therefore, it will be understood that systems and methods for managing memory resources constructed according to the principles of this disclosure can be implemented in ways different from those specifically described herein. The invention is also defined in the appended claims and their equivalents.

Claims

1. A system for managing memory resources, comprising: The first server includes: Stored program processing circuit, The first network interface circuit, and First memory module, in, The first memory module includes a first memory die and a controller. The controller is connected to the first memory die via a memory interface, to the stored program processing circuit via a cache coherence interface, and to the first network interface circuit. The system also includes: a second server, The controller of the first memory module is configured to: receive data from the second server; store the data in the first memory module; and send commands to invalidate cache lines to the stored program processing circuit.

2. The system according to claim 1, wherein, The first memory module also includes a second memory die. The first memory die includes volatile memory, and The second memory die includes persistent memory.

3. The system according to claim 2, wherein, Persistent storage includes NAND flash memory.

4. The system according to claim 3, wherein, The controller is configured to provide a flash translation layer for persistent memory.

5. The system according to claim 1, wherein, The cache consistency interface includes a compute fast link interface.

6. The system according to claim 1, wherein, The first server includes an expansion socket adapter connected to an expansion socket of the first server, the expansion socket adapter including: First memory module; and First network interface circuit.

7. The system according to claim 6, wherein, The controller of the first memory module is connected to the stored program processing circuit via an expansion socket.

8. The system according to claim 6, wherein, Expansion outlets include M.2 sockets.

9. The system according to claim 6, wherein, The controller of the first memory module is connected to the first network interface circuit via a point-to-point peripheral component interconnection.

10. The system according to any one of claims 1 to 9, further comprising: A network switch connects to the first and second servers.

11. The system according to claim 10, wherein, Network switches include top-of-rack Ethernet switches.

12. The system according to claim 10, wherein, The controller of the first memory module is configured to receive remote direct memory access (RDMA) requests and send RDMA responses.

13. The system according to claim 10, wherein, The controller of the first memory module is configured to receive remote direct memory access (RDMA) requests via a network switch and a first network interface circuit, and to send RDMA responses via a network switch and a first network interface circuit.

14. The system according to any one of claims 1 to 9, wherein, The controller for the first memory module includes a field-programmable gate array or an application-specific integrated circuit.

15. A method for performing remote direct memory access in a computing system, the computing system comprising: First server and second server. The first server includes a stored program processing circuit, a network interface circuit, and a first memory module including a controller. The method includes: The controller of the first memory module receives a remote direct memory access (RDMA) request; and The RDMA response is sent by the controller of the first memory module. The method further includes: Data is received by the controller of the first memory module; The controller of the first memory module stores the data in the first memory module; and The controller of the first memory module sends a command to invalidate a cache line to the stored program processing circuit.

16. The method according to claim 15, wherein, The computing system further includes: an Ethernet switch connected to the first server and the second server; and The steps for receiving an RDMA request include receiving the RDMA request via an Ethernet switch.

17. The method of claim 15, further comprising: The controller of the first memory module receives a read command for the first memory address from the stored program processing circuit; The controller of the first memory module translates the first memory address into a second memory address; and The controller of the first memory module obtains data from the first memory module at the second memory address.

18. A system for managing memory resources, comprising: The first server includes: Stored program processing circuit, The first network interface circuit, and First memory module, in, The first memory module includes a first memory die and a controller device. The controller device is connected to the first memory die via a memory interface, to the stored program processing circuit via a cache coherence interface, and to the first network interface circuit. The system also includes: a second server, The controller device of the first memory module is configured to: receive data from the second server; store the data in the first memory module; and send commands for invalidating cache lines to the stored program processing circuit.

Images