System and method for managing memory resources

Abstract

To provide improved system and method for managing a memory resource in a system including one or more servers.SOLUTION: A system for managing a memory resource according to the present invention includes a first server including a processing circuit for a stored program, a cache-coherent switch, and a first memory module, a second server, and a server link switch connected to the first server and the second server. The first memory module is connected to the cache-coherent switch, the cache-coherent switch is connected to the server link switch, and the processing circuit for the stored program is connected to the cache-coherent switch.SELECTED DRAWING: Figure 1A

Description

[Technical Field]

[0001] The present invention relates to computing systems, and more particularly to a system and method for managing memory resources in a system that includes one or more servers. [Background technology]

[0002] Some server systems may include a collection of servers connected by a network protocol. Each of the servers in such a system may include processing resources (eg, a processor) and memory resources (eg, system memory). In some environments, it is advantageous for the processing resources of one server to access the memory resources of another server, and it is advantageous for such access to occur while minimizing the processing resources of either one of those servers.

[0003] Therefore, there is a need and a challenge for development of improved systems and methods for managing memory resources in systems that include one or more servers. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] U.S. Patent No. 9,619,389 [Patent Document 2] US Patent Application Publication No. 2015 / 0258437 [Patent Document 3] US Patent Application Publication No. 2016 / 0299767 [Patent Document 4] US Patent Application Publication No. 2019 / 0179805 [Patent Document 5] US Patent Application Publication No. 2019 / 0235777 [Patent Document 6] US Patent Application Publication No. 2019 / 0384733 [Patent Document 7] US Patent Application Publication No. 2019 / 0391936 [Patent Document 8] US Patent Application Publication No. 2020 / 0021540 [Patent Document 9] US Patent Application Publication No. 2020 / 0050403 [Patent Document 10] US Patent Application Publication No. 2020 / 0050570 [Patent Document 11] US Patent Application Publication No. 2020 / 0104275 [Patent Document 12] U.S. Patent Application Publication No. 2020 / 0125503 [Non-patent literature]

[0005] [Non-Patent Document 1] AWS Summit,Seoul,Korea,2017,36pages,https: / / www.slideshare.net / awskorea / awscloud-game-architecture?from_action=save),Amazon Web Services,Inc. [Non-patent document 2] Unpublished US application no. 17 / 026082 (filed September 18, 2020). [Non-patent document 3] Unpublished US application no. 17 / 026071 (filed September 18, 2020). [Non-patent document 4] Unpublished US application no. 17 / 026087 (filed September 18, 2020). Summary of the Invention [Problem to be solved by the invention]

[0006] The present invention has been made in consideration of the problems with the conventional server systems described above, and an object of the present invention is to provide an improved system and method for managing memory resources in a system including one or more servers. [Means for solving the problem]

[0007] In order to achieve the above object, the present invention provides a system for managing memory resources, comprising: a first server including a stored program processing circuit, a cache-coherent switch, and a first memory module; a second server; and a server link switch connected to the first server and the second server, wherein the first memory module is connected to the cache-coherent switch, the cache-coherent switch is connected to the server link switch, and the stored program processing circuit is connected to the cache-coherent switch. the server link switch comprises a Peripheral Component Interconnect Express (PCIe) switch or a Compute Express Link (CXL) switch, the first memory module comprises a controller for converting signals to conform to a protocol of the first memory module, the controller further comprising a switch management device for appropriately binding and unbinding upstream and downstream connections during runtime and enabling control semantics and statistics related to data transfers to and from the first memory module. It is characterized by:

[0008] In some embodiments, the data storage and processing system includes multiple servers connected by a server link switch. Each server may include multiple processing circuits, system memory, and one or more memory modules connected to the processing circuits via a cache coherent switch. The cache coherent switch may be connected to a connection switch to the servers and may include a controller (e.g., a field-programmable gate array (FPGA) or an application specific integrated circuit (ASIC)) that provides improved functionality. These features include virtualizing memory modules, allowing data to be stored on the memory modules using an underlying technology that better matches the storage requirements (e.g., latency, bandwidth, or durability) of the data to be stored on the switch. The cache coherent switch may receive storage requirements as a result of these requirements being forwarded by processing circuitry or as a result of monitoring access patterns. Improved functionality may further include enabling servers to interact with the memory of other servers without accessing a processor such as a central processing unit (CPU) (e.g., by performing remote direct memory access (RDMA)).

[0009] before The server link switches preferably include top of rack (ToR) CXL switches. The server link switch preferably discovers the first server. The server link switch preferably reboots the first server. The server link switch preferably causes the cache coherent switch to deactivate the first memory module. The server link switch preferably transfers data from the second server to the first server and performs flow control on the data. It is preferable that the network further includes a third server connected to the server link switch, wherein the server link switch receives a first packet from the second server, receives a second packet from the third server, and forwards the first packet and the second packet to the first server.

[0010] It is preferable that the system further comprises a second memory module connected by the cache coherent switch, the first memory module including a volatile memory, and the second memory module including a persistent memory. The cache coherent switch preferably virtualizes the first memory module and the second memory module. Preferably, the first memory module includes a flash memory, and the cache coherent switch provides a flash translation layer for the flash memory. Preferably, the first server includes an expansion socket adapter connected to an expansion socket of the first server, the expansion socket adapter including the cache coherent switch and a memory module socket, and the first memory module is connected to the cache coherent switch via the memory module socket. The memory module socket preferably includes an M.2 socket. The cache coherent switch is connected to the server link switch via a connector, which is preferably on the expansion socket adapter.

[0011] In order to achieve the above object, the system for managing memory resources according to the present invention is a method for performing remote direct memory access (RDMA) in a computing system, the computing system comprising: a first server, a second server, a third server, and a server link switch connected to the first server, the second server, and the third server, the first server including a processing circuit for stored programs, a cache coherent switch, and a first memory module; the server link switch comprises a Peripheral Component Interconnect Express (PCIe) switch or a Compute Express Link (CXL) switch, the first memory module comprises a controller for converting signals to conform to a protocol of the first memory module, the controller further comprising a switch management device for appropriately binding and unbinding upstream and downstream connections during runtime and enabling control semantics and statistics related to data transfers to and from the first memory module;The method for performing the remote direct memory access includes receiving a first packet from the second server by the server link switch, receiving a second packet from the third server by the server link switch, and forwarding the first packet and the second packet to the first server.

[0012] Preferably, the method further comprises the steps of receiving a remote direct memory access (RDMA) request by the cache coherent switch, and forwarding an RDMA response by the cache coherent switch. Preferably, receiving the RDMA request includes receiving the RDMA request via the server link switch. Preferably, the method further includes the steps of receiving a read command to a first memory address from the processing circuit of the stored program by the cache coherent switch, converting the first memory address to a second memory address by the cache coherent switch, and retrieving data from the first memory module at the second memory address by the cache coherent switch.

[0013] In order to achieve the above object, a system for managing memory resources according to the present invention includes a first server including a stored program processing circuit, a cache coherent switching means, and a first memory module, a second server, and a server link switch connected to the first server and the second server, wherein the first memory module is connected to the cache coherent switching means, the cache coherent switching means is connected to the server link switch, and the stored program processing circuit is connected to the cache coherent switching means. the server link switch comprises a Peripheral Component Interconnect Express (PCIe) switch or a Compute Express Link (CXL) switch, the first memory module comprises a controller for converting signals to conform to a protocol of the first memory module, the controller further comprising a switch management device for appropriately binding and unbinding upstream and downstream connections during runtime and enabling control semantics and statistics related to data transfers to and from the first memory module. It is characterized by: [Effects of the Invention]

[0014] The system and method for managing memory resources according to the present invention provides a CXL 2.0 based system that provides faster communication between servers in a system that includes one or more servers. This improves the speed of data transfer between shared memories distributed within the rack. [Brief explanation of the drawings]

[0015] [Figure 1A] 1 is a block diagram of a system for connecting memory resources to computing resources using cache-coherent connections according to an embodiment of the present invention. [Figure 1B] 1 is a block diagram illustrating a schematic configuration of a system using an enhanced socket adapter to connect memory resources to computing resources using cache coherent connections according to an embodiment of the present invention. [Figure 1C] 1 is a block diagram of a system for aggregating memory using an Ethernet ToR switch according to an embodiment of the present invention. [Figure 1D] 1 is a block diagram of a system for aggregating memory using an Ethernet ToR switch and an expansion socket adapter according to an embodiment of the present invention. [Figure 1E] 1 is a block diagram of a system for collecting memory according to an embodiment of the present invention. [Figure 1F] 1 is a block diagram of a system for collecting memory using an expansion socket adapter according to an embodiment of the present invention. [Figure 1G] FIG. 1 is a block diagram of a system for disaggregating servers according to an embodiment of the present invention. [Figure 2A]1A-1G is a flowchart illustrating an exemplary method for performing remote direct memory access (RDMA) transfers by bypassing processing circuitry for the embodiments shown in FIGS. 1A-1G. [Figure 2B] 1A-1D is a flowchart illustrating an example method for performing an RDMA transfer with participation of processing circuitry for the embodiment shown in FIGS. 1A-1D. [Figure 2C] 1E-1F is a flowchart illustrating an example method for performing RDMA transfers through a Compute Express Link (CXL) switch for the embodiment illustrated in FIGS. 1E-1F. [Figure 2D] 1G is a flowchart illustrating an example method for performing an RDMA transfer through a CXL switch for the embodiment shown in FIG. 1G. DETAILED DESCRIPTION OF THE INVENTION

[0016] Next, a specific example of an embodiment of a system and method for managing memory resources according to the present invention will be described with reference to the drawings.

[0017] The following detailed description, taken in conjunction with the accompanying drawings, is intended as a description of example embodiments of memory resource management systems and methods provided in accordance with the present disclosure and is not intended to represent the only manner in which the present disclosure may be configured or utilized. The description illustrates features of the present disclosure in connection with the illustrated embodiments. However, it is to be understood that the same or equivalent functions and structures accomplished by different embodiments are also intended to be within the scope of the present disclosure. As shown elsewhere in this specification, like drawing numerals are intended to indicate like elements or features.

[0018] The terms "processing circuitry" or "controller means" are used herein to mean any combination of hardware, firmware, and software used to process data or digital signals. Processing circuitry hardware may include, for example, application specific integrated circuits (ASICs), general-purpose or special-purpose central processing units (CPUs), digital signal processors (DSPs), graphics processing units (GPUs), and programmable logic devices such as field programmable gate arrays (FPGAs). As used herein, in a processing circuit, each function is performed by hardware configured to perform that function, i.e., hardwired, or by more general hardware, such as a CPU configured to execute commands stored on a non-transitory storage medium. A processing circuit may be fabricated on a single printed circuit board (PCB) or distributed across multiple interconnected PCBs. A processing circuit may include other processing circuits. For example, a processing circuit may include two processing circuits, i.e., an FPGA and a CPU, interconnected on a PCB.

[0019] As used herein, a "controller" includes circuitry, and a controller may also be referred to as a "control circuit" or a "controller circuit." Similarly, a "memory module" may be referred to as a "circuit of a memory module" or a "memory circuit." As used herein, the term "array" means an ordered series of numbers, regardless of storage method (e.g., whether stored in contiguous memory locations or in a linked list). Here, if the second number is "within Y%" of the first number, then the second number is at least (1-Y / 100) times the first, and the second number is at most (1+Y / 100) times the first. As used herein, the term "or" should be interpreted as "and / or," e.g., "A or B" means any one of "A," "B," or "A and B."

[0020] As used herein, when a method (e.g., adjustment) or a first quantity (e.g., a first variable) is referred to as being "based on" a second quantity (e.g., a second variable), it means that the second quantity is an input to the method or influences the first quantity. For example, the second quantity may be an input (e.g., the only input or any one of many inputs) to a function that calculates the first quantity, or the first quantity may be equal to or the same as the second quantity (e.g., stored in the same location or positions in memory).

[0021] For example, the terms "first," "second," "third," etc. may be used herein to describe various elements, components, regions, layers, and / or sections, but these elements, components, regions, layers, and / or sections should not be limited to these terms. These terms are only used to distinguish one element, component, region, layer or section from another element, component, region, layer or section. Thus, a first element, component, region, layer or section described herein could be termed a second element, component, region, layer or section without departing from the spirit and scope of the inventive concept.

[0022] Spatially relative terms such as "beneath / below / lower / under," "above / upper," etc. may be used herein to easily describe one element or feature relative to other elements or features as shown in the drawings. It will be understood that such spatially relative terms are intended to encompass different orientations of the device in use or operation in addition to the orientation depicted in the drawings. For example, if a device in the figures is turned over, elements described as "below" other elements or features would now be oriented "above" the other elements or features. Thus, the example term "below" can encompass both an orientation of above and below. The device may be otherwise oriented (e.g., rotated 90 degrees or at another orientation) and the spatially relative modifiers used herein should be interpreted accordingly. It will also be understood that when a layer is said to exist between two layers, it may be the only layer between the two layers, or there may be one or more intervening layers.

[0023] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. As used herein, the terms "substantially," "about," and similar terms are used as terms of approximation, rather than as terms of degree, and are intended to account for inherent variations in measured or calculated values ​​that would be recognized by a person of ordinary skill in the art. As used herein, the singular forms "a," "an," and "the" are intended to include the plural forms unless the context clearly indicates otherwise. It will still be understood that as used herein, the terms "comprises" and / or "comprising" specify the presence of stated features, integers, steps, operations, elements, and / or components, but do not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof. As used herein, the term "and / or" includes any and all combinations of one or more of the associated and listed items. A phrase such as "at least one or more," when placed before a list of elements, modifies the elements of the entire list, and not the individual elements of the list. It should be noted that, when describing embodiments of the present invention, the term "may" refers to "one or more embodiments of the present disclosure." Additionally, the term "by way of example" is intended to indicate an example or illustration. As used herein, the terms "use," "using," and "used" may be considered synonymous with the terms "utilize," "utilizing," and "utilized," respectively.

[0024] When an element or layer is described as being "connected on," "coupled to," or "adjacent to" another element or layer, it will be understood that the other element or layer can be immediately above, directly connected to, or adjacent to the other element or layer, or there can be one or more intervening elements or layers. In contrast, when an element or layer is referred to as being "directly on," "directly connected to," "directly coupled to," or "immediately adjacent to" another element or layer, there are no intervening elements or layers present.

[0025] Any numerical range recited herein is intended to include all sub-ranges of the same numerical precision contained within the recited range. For example, a range "from 1.0 to 10.0" or a range "between 1.0 and 10.0" is intended to include all subranges between (and including) the stated minimum of 1.0 and the stated maximum of 10.0, i.e., having minimums equal to or greater than 1.0 and maximums equal to or less than 10.0 (e.g., 2.4 to 7.6). Any maximum numerical limitation given herein is intended to include every lower numerical limitation subsumed therein, and any minimum numerical limitation given herein is intended to include every higher numerical limitation subsumed therein.

[0026] Peripheral Component Interconnect Express (PCIe) represents a computer interface that can have relatively high and variable latency, which can limit its usefulness for connecting to memory. CXL is an open industry standard for communication over PCIe 5.0 that provides fixed, relatively short packet sizes, and as a result, can provide relatively high bandwidth and relatively low, fixed latency. Therefore, CXL can support cache coherence and is suitable for connecting to memory. CXL is also used on servers to provide connections between hosts and accelerators, storage devices and network interface circuits (network interface circuits, "network interface controllers" or network interface cards (NICs)).

[0027] Cache coherent protocols such as CXL can be used for heterogeneous processing, for example, in scalar, vector, and buffered memory systems. To provide a cache coherent interface, CXL is used to leverage the channels, retimers, PHY layer of the system, logical aspects of the interface, and the PCIe 5.0 protocol. The CXL transaction layer can include three multiplexed sub-protocols running simultaneously over a single link, called "CXL.io," "CXL.cache," and "CXL.memory."

[0028] "CXL.io" includes I / O semantics similar to PCIe. "CXL.cache" may include a caching semantic, and "CXL.memory" may include a memory semantic. Both cache and memory semantics may be options. Like PCIe, CXL is (i) Partionable x16, x8, and x4 basic widths, (ii) 8 GT / s and 16 GT / s, 32 GT / s data rates degradable to 128b / 130b; (iii) 300W (75W with x16 connectors), (iv) It can support plug and play. To support plug and play, a PCIe or CXL device link can start training on PCIe Gen1, negotiate CXL, and initiate CXL transactions after completing Gen1-5 training.

[0029] In some embodiments, as described in more detail below, in a system including multiple servers connected together through a network, using a CXL connection to a collection or "pool" of memory (e.g., a memory quantity including multiple memory cells connected together) can provide various advantages. For example, a CXL switch that has additional functionality in addition to providing packet switching functionality for CXL packets (hereinafter, an "enhanced capability CXL switch") may be used to connect a collection of memory to one or more central processing units (CPUs) (or "central processing circuits") and one or more network interface circuits (which may have enhanced functionality). Such a configuration is (i) the collection of memories includes various types of memories having different characteristics; (ii) an improved CXL switch virtualizes a collection of memories, allowing data with different characteristics (e.g., access frequency) to be stored in the appropriate type of memory; (iii) Improved functionality of the CXL switch allows it to support remote direct memory access (RDMA) so that RDMA can be performed with little or no involvement from the server's processing circuitry. As used herein, "virtualizing" memory means performing memory address translation between processing circuitry and memory.

[0030] The CXL switch is (i) It can support memory and accelerator isolation through a single level of switching; (ii) allowing resources to be offline and online between domains, allowing time multiplexing between domains as needed; (iii) It can support downstream port virtualization. CXL is used to implement aggregated memories, which can activate one-to-many and many-to-one switching (e.g., (i) connecting multiple root ports to one endpoint; (ii) connecting one Root Port to multiple Endpoints; or (iii) Allows multiple root ports to be connected to multiple endpoints).

[0031] In some embodiments, the aggregated device is divided into multiple logical devices with their own LD-IDs (Logical Device Identifiers). In such an embodiment, the physical device is divided into multiple logical devices that are visible to each initiator. A device may have one physical function (PF) and multiple (eg, 16) separate logical devices. In some embodiments, the number of logical devices (e.g., the number of partitions) may be limited (e.g., at 16), and there may also be one control partition (which may be a physical function used to control the devices).

[0032] In some embodiments, the fabric manager comprises: (i) performing device detection and virtualized CXL software generation; (ii) It is used to bind a virtual port to a physical port. Such fabric management devices can operate via an SMBus sideband connection. The fabric management device may be implemented in hardware, software, firmware, or a combination thereof and may reside, for example, in a host, in one of memory modules 135, in an enhanced CXL switch 130, or elsewhere on the network. A management device in the fabric can issue commands, including commands issued over a sideband bus or PCIe tree.

[0033] FIG. 1A is a block diagram illustrating a schematic configuration of a system for connecting memory resources to computing resources using cache-coherent connections according to an embodiment of the present invention. Referring to FIG. 1A, a server system according to an embodiment of the present invention includes multiple servers 105 connected together by a Top of Rack (ToR) Ethernet switch 110.

[0034] Such switches are described as using the Ethernet protocol, although other suitable network protocols may be used. Each server is (i) system memory 120 (e.g., Double Data Rate (DDR4) (version 4) memory or any other suitable memory); (ii) one or more network interface circuits 125; (iii) includes one or more processing circuits 115 each connected to one or more (CXL) memory modules 135; Each processing circuit 115 may be a stored program processing circuit, for example, a central processing unit (CPU (eg, an x86 CPU)), a graphics processing unit (GPU), or an ARM processor. In some embodiments, the network interface circuitry 125 may be embedded in any one of the memory modules 135 (e.g., on the same semiconductor chip or in the same module), or the network interface circuitry 125 may be packaged separately from the memory modules 135.

[0035] As used herein, a "memory module" is a package (e.g., a package containing a printed circuit board and components connected thereto, or an enclosure containing a printed circuit board) that contains one or more memory dies, each of which contains a plurality of memory cells. Each memory die or group of memory dies may reside in a package (e.g., an epoxy mold compound (EMC) package) that is soldered to (or connected to) the printed circuit board of the memory module via a connector.

[0036] Each memory module 135 may have a CXL interface and may include a controller 137 (e.g., an FPGA, ASIC, processor, etc.) for converting signals between CXL packets and the memory interface of the memory die, e.g., appropriate for the memory technology of the memory within memory module 135. As used herein, the "memory interface" of a memory die is an interface specific to the technology of the memory die; for example, in the case of DRAM, the memory interface may be word lines and bit lines.

[0037] The memory module may include a controller 137 that may provide improved functionality, as described in more detail below. The controller 137 of each memory module 135 is connected to the processing circuitry 115 via a cache coherent interface, for example via a CXL interface. The controller 137 may also bypass the processing circuitry 115 to facilitate data transfers (eg, RDMA requests) between different servers 105 . The ToR Ethernet switch 110 and network interface circuitry 125 may include an RDMA interface to facilitate RDMA requests between CXL memory devices on different servers (e.g., the ToR Ethernet switch 110 and network interface circuitry 125 may provide hardware offload or acceleration of RoCE (RDMA over Converged Ethernet), Infiniband, and iWARP packets).

[0038] The CXL interconnect of the system may conform to a cache coherent protocol such as the CXL 1.1 standard, or in some embodiments, the CXL 2.0 standard, a future version of CXL, or any other suitable protocol (eg, a cache coherent protocol). The memory modules 135 may also be attached directly to the processing circuitry 115 as shown, and the ToR Ethernet switch 110 used to scale the system to a larger size (e.g., a larger number of servers 105).

[0039] In some embodiments, each server can be filled with multiple direct-attach (CXL) memory modules 135, as shown in FIG. 1A. Each memory module 135 exposes a set of base address registers (BARs) to the host's Basic Input / Output System (BIOS) as a memory range. Any one or more of the memory modules 135 may include firmware for transparently managing memory space behind a host OS map.

[0040] Each memory module 135 may include any one or combination of memory technologies, including, but not limited to, Dynamic Random Access Memory (DRAM), Not-AND (NAND) flash, High Bandwidth Memory (HBM), and Low-Power Double Data Rate Synchronous Dynamic Random Access Memory (LPDDR SDRAM) technologies, and may also include a cache controller (in the case of a memory module 135 that combines multiple memory devices of different technologies) or individual split controllers for the memory devices of different technologies. Each memory module 135 can include different interface widths (x4-x16) and can be configured according to any of a variety of associated form factors, such as, for example, "U.2," "M.2," half height half length (HHHL), full height half length (FHHL), "E1.S," "E1.L," "E3.S," and "E3.H."

[0041] In some embodiments, as previously described, the enhanced functionality CXL switch 130 includes an FPGA (or ASIC) controller 137 to provide additional features beyond switching CXL packets. The controller 137 of the enhanced CXL switch 130 can also act as a management device for the memory module 135, facilitating host control plane processing and enabling rich control semantics and statistics.

[0042] The controller 137 may include an additional "backdoor" (eg, 100 Gigabit Ethernet (GbE)) network interface circuit 125. In some embodiments, controller 137 resides in processing circuitry 115 as a CXL Type 2 device, which receives remote write requests and enables processing circuitry 115 to issue cache invalidation commands. In some embodiments, DDIO technology is enabled, and remote data is first pulled to the last level cache (LLC) of the processing circuit and later recorded in memory module 135 (from the cache). As used herein, a "Type 2" CXL device can initiate transactions and embodies a selective coherent cache and host-managed device memory, and applicable transaction types may include all "CXL.cache" and all "CXL.mem" transactions.

[0043] As previously mentioned, one or more of the memory modules 135 may include persistent memory or "persistent storage" (i.e., storage in which data is not lost when external power is interrupted). If memory module 135 is provided as a persistent device, controller 137 of memory module 135 may manage the persistent domain (e.g., store data in persistent storage that is identified by processing circuitry 115 as requiring persistent storage (e.g., as a result of an application invoking a corresponding operating system function)). In such an embodiment, the software API can flush the cache and data to persistent storage.

[0044] In some embodiments, direct memory transfers from the network interface circuit 125 to the memory module 135 are possible. Such transfers may be one-way transfers to remote memory for high speed communication in a distributed system. In such an embodiment, the memory module 135 exposes hardware details to the system's network interface circuitry 125 to enable faster RDMA transfers. In such a system, two scenarios can occur depending on whether the data direct I / O (DDIO) of processing circuit 115 is enabled or disabled. The DDIO allows direct communication between an Ethernet controller or adapter and the cache of the processing circuit 115. When the DDIO of the processing circuit 115 is enabled, the target of the transfer may be the last level cache of the processing circuit, from which the data may subsequently be automatically flushed to the memory module 135. When the DDIO of the processing circuit 115 is disabled (deactivated), the memory module 135 operates in device bias mode to force the access to be received directly (without the DDIO) by the destination memory module 135.

[0045] An RDMA-capable network interface circuit 125 having a host channel adapter (HCA), buffers, and other processing may be used to enable such RDMA transfers, which may bypass the target memory buffer transfers that may be present in other RDMA transfer modes. For example, in such an embodiment, the use of bounce buffers (e.g., buffers at a remote server when the final destination in memory is in an address range not supported by the RDMA protocol) may be avoided. In some embodiments, RDMA uses other physical media options besides Ethernet (eg, for use with switches configured to handle other network protocols). Examples of server-to-server connections that can enable RDMA include, but are not limited to, Infiniband, RoCE (RDMA over Converged Ethernet) (which uses Ethernet User Datagram Protocol (UDP)), and iWARP (which uses TCP / IP (transmission control protocol / Internet protocol)).

[0046] FIG. 1B is a block diagram illustrating a schematic configuration of a system using an expansion socket adapter to connect memory resources to computing resources using cache coherent connections according to an embodiment of the present invention. FIG. 1B shows a system similar to FIG. 1A, in which processing circuitry 115 is connected to network interface circuitry 125 via memory module 135 .

[0047] The memory module 135 and the network interface circuit 125 reside on an expansion socket adapter 140 . Each expansion socket adapter 140 plugs into an expansion socket 145, such as an M.2 connector, on the motherboard of server 105. Thus, the server may be any suitable (eg, industry standard) server that is modified by installation of expansion socket adapter 140 in expansion socket 145 . In such an embodiment, (i) each network interface circuit 125 may be integrated into a respective one of the memory modules 135; or (ii) Each network interface circuit 125 may have a PCIe interface (a network interface circuit 125 may be a PCIe endpoint (ie, a PCIe slave device)). Thus, processing circuit 115 connected to network interface circuit 125 (which can act as a PCIe master device or "root port") can communicate with network interface circuit 125 over an endpoint PCIe connection via a root point, and controller 137 of memory module 135 can communicate with network interface circuit 125 over a P2P (peer-to-peer) PCIe connection.

[0048] According to one embodiment of the present invention, a system is provided comprising a stored program processing circuit, a first network interface circuit, and a first server including a first memory module, wherein the first memory module includes a first memory die and a controller, the controller connected to the first memory die via a memory interface, to the stored program processing circuit via a cache coherent interface, and to the first network interface circuit.

[0049] In some embodiments, the first memory module further includes a second memory die, the first memory die including volatile memory and the second memory die including persistent memory. In some embodiments, the persistent memory includes NAND flash. In some embodiments, the controller is configured to provide a flash translation layer for the persistent memory. In some embodiments, the cache coherent interface includes a Compute Express Link (CXL) interface. In some embodiments, the first server includes an expansion socket adapter connected to an expansion socket of the first server, the expansion socket adapter including a first memory module and a first network interface circuit. In some embodiments, the controller of the first memory module is connected to the processing circuitry of the stored program via an expansion socket. In some embodiments, the expansion socket includes an M.2 socket.

[0050] In some embodiments, the controller of the first memory module is connected to the first network interface circuit by a peer-to-peer Peripheral Component Interconnect Express (PCIe) connection. In some embodiments, the system further includes a second server and a network switch connected to the first server and the second server. In some embodiments, the network switch comprises a top of rack (ToR) Ethernet switch. In some embodiments, the controller of the first memory module is configured to receive straight remote direct memory access (RDMA) requests and forward straight RDMA responses. In some embodiments, the controller of the first memory module receives a straight RDMA request through the network switch and the first network interface circuit, and forwards a straight RDMA response through the network switch and the first network interface circuit. In some embodiments, the controller of the first memory module receives data from the second server, stores the data in the first memory module, and forwards a command to the stored program processing circuit to invalidate the cache line. In some embodiments, the controller of the first memory module includes a field programmable gate array (FPGA) or an application-specific integrated circuit (ASIC).

[0051] According to one embodiment of the present invention, there is provided a method for performing remote direct memory access in a computing system, the computing system including a first server and a second server, the first server having a first memory module including a processing circuit for stored programs, a network interface circuit, and a controller, the method for performing remote direct memory access comprising: receiving a straight remote direct memory access (RDMA) request by the controller of the first memory module; and forwarding a straight RDMA response by the controller of the first memory module.

[0052] In some embodiments, the computing system further includes an Ethernet switch connected to the first server and the second server, and receiving the straight RDMA request includes receiving the straight RDMA request via the Ethernet switch. In some embodiments, a method for performing remote direct memory access includes receiving, by a controller of a first memory module, a read command for a first memory address from a stored program processing circuit; translating, by the controller of the first memory module, the first memory address to a second memory address; and retrieving, by the controller of the first memory module, data from the first memory module at the second memory address. In some embodiments, the method for performing remote direct memory access further includes receiving data by a controller of a first memory module; storing the data in the first memory module by the controller of the first memory module; and forwarding a command to the stored program processing circuitry by the controller of the first memory module to invalidate the cache line.

[0053] According to one embodiment of the present invention, there is provided a system comprising a first server including a stored program processing circuit, a first network interface circuit, and a first memory module, wherein the first memory module includes a first memory die and controller means, the controller means being connected to the first memory die via a memory interface and to the stored program processing circuit via a cache coherent interface, and to the first network interface circuit.

[0054] FIG. 1C is a block diagram illustrating a schematic configuration of a system for aggregating memory using an Ethernet ToR switch according to an embodiment of the present invention. Referring to FIG. 1C, in some embodiments, a server system includes multiple servers 105 connected together by a Top of Rack (ToR) Ethernet switch 110.

[0055] Each server includes one or more processing circuits 115, each of which: (i) system memory 120 (e.g., DDR4 memory); (ii) one or more network interface circuits 125; and (iii) It can be connected to a CXL switch 130 for improved functionality. The improved functionality CXL switch 130 may be connected to multiple memory modules 135 . That is, the system of FIG. 1C comprises a (first) server 105 including a (stored program) processing circuit 115, a network interface circuit 125, an improved functionality CXL switch (cache coherent switch) 130, and a (first) memory module 135. In the system of FIG. 1C, first memory module 135 is connected to cache coherent switch 130, cache coherent switch 130 is connected to network interface circuitry 125, and stored program processing circuitry 115 is connected to cache coherent switch 130.

[0056] The memory modules 135 are grouped by type, form factor, or technology type (e.g., DDR4, DRAM, LDPPR, high bandwidth memory (HBM), NAND flash, or other persistent storage (e.g., solid state drives (SSDs) that integrate NAND flash). Each of the memory modules may have a CXL interface and may include interface circuitry for converting between CKL packets and signals appropriate for the memory of memory module 135. In some embodiments, these interface circuits exist in place of the improved functionality CXL switch 130, and each memory module 135 includes a unique interface to the memory located in the memory module 135. In some embodiments, the improved functionality CXL switch 130 is integrated into the memory module 135 (e.g., with an M.2 form factor package or in a single integrated circuit with the other components of the memory module 135).

[0057] The ToR Ethernet switch 110 may include interface hardware that facilitates RDMA requests between aggregated memory devices on different servers. The improved CXL Switch 130 (i) routing data to different memory types based on workload; (ii) virtualizing host addresses into device addresses; and / or (iii) may include one or more circuits (which may include, for example, FPGAs or ASICs) for bypassing processing circuitry 115 to facilitate RDMA requests between different servers.

[0058] The memory modules 135 may reside in an expansion box (e.g., in the same rack as the enclosure that houses the enclosure's motherboard), which may contain a pre-determined number (e.g., 20 or more or 100 or more) of memory modules 135, each plugged into an appropriate connector. The module may be in an M.2 form factor and the connector may be an M.2 connector. In some embodiments, the connection between the servers is over a different network rather than Ethernet, and may be, for example, a wireless connection such as a WiFi or 5G connection.

[0059] Each processing circuit may be an x86 processor or other processor, for example, an ARM processor or a GPU. The PCIe link on which the CXL link is instantiated may be PCIe 5.0 or another version (e.g., an earlier version or a later (e.g., future) version (e.g., PCIe 6.0)). In some embodiments, a different cache coherent protocol may be used instead of or in addition to CXL, and a different cache coherent switch may be used instead of or in addition to CXL switch 130 for improved functionality. Such a cache coherent protocol may be another standard protocol or a cache coherent variant of a standard protocol (similar to how CXL is a variant of PCIe 5.0). Examples of standard protocols include, but are not limited to, NVDIMM-P (non-volatile dual in-line memory module (version P)), CCIX (Cache Coherent Interconnect for Accelerators), and OpenCAPI (Open Coherent Accelerator Processor Interface).

[0060] System memory 120 may include, for example, DDR4 memory, DRAM, HBM, or LDPPR memory. The memory module 135 may include a cache controller or may be partitioned to handle multiple memory types. The memory module 135 can exist in different form factors, examples of which include, but are not limited to, HHHL, FHHL, M.2, U.2, mezzanine card, daughter card, "E1.S", "E1.L", "E3.L", and "E3.S".

[0061] In some embodiments, the system implements a clustered architecture including multiple servers, each server clustered with multiple CXL-attached memory modules 135 . Each memory module 135 may include multiple partitions that may be individually exposed as memory devices to multiple processing circuits 115 . Each input port of the improved CXL switch 130 can independently access multiple output ports of the improved CXL switch 130 and the memory module 135 connected thereto. As used herein, an "input port" or "upstream port" of the improved functionality CXL switch 130 is a port that is connected (or suitable for connection) to a PCIe root port, and an "output port" or "downstream port" of the improved functionality CXL switch 130 is a port that is connected (or suitable for connection) to a PCIe endpoint. This is the same as in the embodiment of FIG. 1A. As in the embodiment of FIG. 1A, each memory module 135 can expose a set of base address registers (BARs) to the host BIOS as memory ranges. One or more memory modules 135 may contain firmware to transparently manage its memory space behind a host OS map.

[0062] In some embodiments, as previously described, the enhanced functionality CXL switch 130 includes an FPGA (or ASIC) controller 137 to provide additional functionality beyond switching CXL packets. For example, (as previously described) memory module 135 may be virtualized, i.e., recorded as a translation layer between processing circuitry-side addresses (or "processor-side addresses," i.e., addresses contained in read and write commands issued by processing circuitry 115) and memory-side addresses (i.e., addresses used by enhanced CXL switch 130 to address storage locations within the memory module), masking the physical addresses of memory module 135 accordingly and providing a virtual aggregation of memory.

[0063] The controller 137 of the enhanced CXL switch 130 may also act as a management device for the memory module 135 and facilitate host control plane processing. Controller 137 can transparently move data without the participation of processing circuitry 115, and can therefore update the memory map (or "address translation table") so that subsequent accesses work as expected.

[0064] The controller 137 (i) It is possible to bind and unbind upstream and downstream connections appropriately during runtime; (ii) It may include a switch management facility that allows for rich control semantics and statistics related to the transfer of data into and out of memory module 135 . The controller 137 may include additional "backdoor" 100GbE or other network interface circuitry 125 (in addition to the network interface used to connect to the host) for connecting to other servers 105 or other network equipment.

[0065] In some embodiments, controller 137 appears to processing circuit 115 as a Type 2 device, which enables it to issue cache invalidation commands to processing circuit 115 when it receives a remote write request. In some embodiments, DDIO technology is enabled and remote data is first pulled to a last level cache (LLC) of the processing circuit and later stored in memory module 135 (from the cache).

[0066] As previously mentioned, any one or more of the memory modules 135 may include persistent storage devices. If the memory module 135 is designated as a persistent device, the controller 137 of the improved functionality CXL switch 130 can request persistent storage and manage the persistent domain with the processing circuitry (e.g., identified as a persistent storage device (e.g., by using corresponding operating system functions)). In such an embodiment, the software API can flush the cache and data to persistent storage.

[0067] In some embodiments, direct memory transfer to memory module 135 is performed by controller 137 of improved functionality CXL switch 130 . The operations performed by the controller of memory module 135 are performed in a manner similar to that described above for the embodiment of Figures 1A and 1B.

[0068] As previously mentioned, in some embodiments, memory modules 135 are organized into groups, e.g., a memory-aggregated group, another group weighted toward HBM, another group with limited density and performance, and another group with high-density capacity. Such groups may have different form factors or be based on different technologies. The controller 137 of the improved CXL switch 130 intelligently routes data and commands based on, for example, workload, tagging, or quality of service (QoS). For read requests, there may be no routing based on such factors.

[0069] The controller 137 of the improved CXL switch 130 also virtualizes the processing circuit side addresses and memory side addresses (as described above), allowing the controller 137 of the improved CXL switch 130 to determine where data is stored. The controller 137 of the enhanced CXL switch 130 can make such decisions based on information or commands that may be received from the processing circuitry 115 .

[0070] For example, the operating system may provide memory allocation functionality to allow an application to specify low latency storage or high bandwidth storage devices, or persistent storage devices may be allocated and considered by the controller 137 of the CXL switch 130 of the improved functionality when determining where to allocate memory (e.g., any memory within memory module 135) after such a request is initiated by an application. For example, storage that requires high bandwidth by an application may be allocated to memory module 135 including HBM, storage that requires data persistence by an application may be allocated to memory module 135 including NAND flash, and other storage (not requested by the application) may be stored on memory module 135 including relatively inexpensive DRAM.

[0071] In some embodiments, the controller 137 of the improved CXL switch 130 makes decisions about where to store particular data based on network usage patterns. For example, the controller 137 of the improved CXL switch 130 may monitor usage patterns to determine that certain data at physical addresses is accessed more frequently than other data, and then the controller 137 of the improved CXL switch 130 may copy these data to the memory module 135 containing the HBM and modify the address translation table so that the data is stored in the same range of virtual addresses at the new location.

[0072] In some embodiments, one or more memory modules 135 include flash memory (e.g., NAND flash), and the controller 137 of the improved functionality CXL switch 130 embodies a flash translation layer for such flash memory. The flash translation layer can support overwriting of processor-side memory locations (by moving data to a different location and marking the data's previous location as invalid) and can perform garbage collection (e.g., erasing a block after moving any valid data in it to another block when the percentage of data in the block that is marked as invalid exceeds a threshold).

[0073] In some embodiments, the controller 137 of the improved functionality CXL switch 130 facilitates a physical function (PF) to PF transfer. For example, if one of the processing circuits 115 needs to move data from one physical address to another physical address (which may have the same virtual address, and this fact need not affect the operation of the processing circuits 115), or if the processing circuits 115 need to move data between two virtual addresses (as required by the processing circuits 115), the controller 137 of the improved CXL switch 130 manages the transfer without intervention from the processing circuits 115.

[0074] For example, processing circuit 115 can forward a CXL request, and data can be transferred from one memory module 135 to another memory module 135 behind the improved function CXL switch 130 without going to processing circuit 115 (e.g., data can be copied from one memory module 135 to another memory module 135). In such a situation, because processing circuit 115 initiates a CXL request, processing circuit 115 may need to flush its cache to ensure consistency. If a Type 2 memory device (e.g., any one of memory modules 135 or an accelerator that may be connected to the CXL switch) instead initiates a CXL request, and the switch is not virtualized, the Type 2 memory device sends a message to the processing circuitry to invalidate its cache.

[0075] In some embodiments, the controller 137 of the improved CXL switch 130 facilitates RDMA requests between servers. The remote server 105 initiates such an RDMA request, which may be forwarded through the ToR Ethernet switch 110 and arrive at the CXL switch 130 of the improved functionality of the server ("local server") 105, which responds to the RDMA request. An improved functionality CXL switch 130 may be configured to receive such RDMA requests and treat the group of memory modules 135 of the receiving server 105 (i.e., the server receiving the RDMA request) as its own memory space.

[0076] At the local server, the improved functionality CXL switch 130 may receive the RDMA request as a direct RDMA request (i.e., an RDMA request that is not routed through the processing circuitry 115 at the local server) and may forward a direct response to the RDMA request (i.e., a response that is not routed through the processing circuitry 115 at the local server). The response at the remote server (e.g., data forwarded by the local server) may be received by the remote server's improved function CXL switch 130 and stored in the remote server's memory module 135 without being routed through the remote server's processing circuitry 115.

[0077] FIG. 1D shows a system similar to FIG. 1C in which processing circuitry 115 is connected to network interface circuitry 125 via an improved CXL switch 130. The improved functionality CXL switch 130 , memory module 135 , and network interface circuit 125 reside on an expansion socket adapter 140 .

[0078] Expansion socket adapter 140 may be a circuit board or module that plugs into an expansion socket (eg, PCIe connector 145 ) present on the motherboard of server 105 . Thus, the server may be any suitable server that is modified only by the installation of expansion socket adapter 140 in PCIe connector 145 . The memory module 135 may be installed in a connector (e.g., an M.2 connector) on the expansion socket adapter 140.

[0079] In such an embodiment, (i) the network interface circuitry 125 may be integrated into the CXL switch 130 for improved functionality, or (ii) Each network interface circuit 125 may have a PCIe interface (the network interface circuit 125 may be a PCIe endpoint), so that the processing circuit 115 to which the network interface circuit 125 is connected may communicate with the network interface circuit 125 via a root port to end point PCIe connection. The controller 137 of the improved functionality CXL switch 130 (which may have PCIe input ports connected to the processing circuitry 115 and the network interface circuitry 125) communicates with the network interface circuitry 125 via a peer-to-peer PCIe connection.

[0080] According to one embodiment of the present invention, a system is provided that includes a first server including a stored program processing circuit, a network interface circuit, a cache coherent switch, and a first memory module, wherein the first memory module is connected to the cache coherent switch, the cache coherent switch is connected to the network interface circuit, and the stored program processing circuit is connected to the cache coherent switch.

[0081] In some embodiments, the system further includes a second memory module coupled to the cache coherent switch, the first memory module including volatile memory and the second memory module including persistent memory. In some embodiments, the cache coherent switch is configured to virtualize the first memory module and the second memory module. In some embodiments, the first memory module includes a flash memory, and the cache coherent switch is configured to provide a flash translation layer for the flash memory. In some embodiments, the cache coherent switch monitors an access frequency of a first memory location in a first memory module, determines that the access frequency exceeds a first threshold, and copies the contents of the first memory location to a second memory location, the second memory location residing in the second memory module. In some embodiments, the second memory module includes high bandwidth memory (HBM).

[0082] In some embodiments, the cache coherent switch is configured to maintain a table for mapping processor-side addresses to memory-side addresses. In some embodiments, the system further includes a second server and a network switch connected to the first server and the second server. In some embodiments, the network switch comprises a top of rack (ToR) Ethernet switch. In some embodiments, the cache coherent switch is configured to receive straight remote direct memory access (RDMA) requests and forward straight RDMA responses. In some embodiments, the cache coherent switch is configured to receive RDMA requests via the ToR Ethernet switch and via the network interface circuitry, and to forward straight RDMA responses via the ToR Ethernet switch and via the network interface circuitry.

[0083] In some embodiments, the cache coherent interface is configured to support the Compute Express Link (CXL) protocol. In some embodiments, the first server includes an expansion socket adapter connected to an expansion socket of the first server, the expansion socket adapter including a cache coherent switch and a memory module socket, and the first memory module is connected to the cache coherent switch via the memory module socket. In some embodiments, the memory module socket includes an M.2 socket. In some embodiments, the network interface circuitry is on an expansion socket adapter.

[0084] According to one embodiment of the present invention, there is provided a method for performing remote direct memory access in a computing system, the computing system including a first server and a second server, the first server including stored program processing circuitry, a network interface circuitry, a cache coherent switch, and a first memory module, the method comprising: receiving a straight RDMA request by the cache coherent switch; and forwarding a straight RDMA response by the cache coherent switch.

[0085] In some embodiments, the computing system further includes an Ethernet switch, and receiving the straight RDMA request includes receiving the straight RDMA request via the Ethernet switch. In some embodiments, the method includes receiving a read command for a first memory address from a program processing circuit stored by the cache coherent switch, translating the first memory address to a second memory address by the cache coherent switch, and retrieving data from the first memory module at the second memory address by the cache coherent switch. In some embodiments, the method further includes receiving the data by the cache coherent switch, storing the data in the first memory module by the cache coherent switch, and forwarding a command to invalidate the cache line by the cache coherent switch to the stored program processing circuit.

[0086] According to one embodiment of the present invention, a system is provided comprising a first server including a stored program processing circuit, a network interface circuit, a cache coherent switching means, and a first memory module, wherein the first memory module is connected to the cache coherent switching means, the cache coherent switch is connected to the network interface circuit, and the stored program processing circuit is connected to the cache coherent switching means.

[0087] FIG. 1E illustrates an embodiment in which multiple servers 105 are each connected to a (ToR) server-linking switch 112, which may be a PCIe 5.0 CXL switch with PCIe functionality, as shown. The server link switch 112 may include an FPGA or an ASIC and may provide better performance (in terms of throughput and latency) than an Ethernet switch. Each of the servers 105 includes an improved CXL switch 130 and multiple memory modules 135 connected to the server link switch 112 via multiple PCIe connectors. Each server 105 also includes one or more processing circuits 115 and system memory 120, as shown. The server link switch 112 acts as a master and each of the improved CXL switches 130 acts as a slave, as described in more detail below.

[0088] In the embodiment of FIG. 1E, the server link switch 112 can group or batch multiple cache requests received from different servers 105, grouping packets to reduce control overhead. The improved CXL Switch 130 (i) routing data to different memory types based on workload; (ii) virtualizing processor-side addresses to memory-side addresses; (iii) may include a slave controller (e.g., a slave FPGA or a slave ASIC) to facilitate coherent requests between different servers 105 by bypassing the processing circuitry 115; The system shown in FIG. 1E may be CXL2.0 based, may include shared memory distributed within racks, and may use ToR server link switches 112 for native connectivity with remote nodes.

[0089] The ToR server link switch 112 may have additional network connections (e.g., an Ethernet connection as illustrated or other types of connections, e.g., a wireless connection such as a WiFi connection or a 5G connection) to connect to other servers or clients. The server link switch 112 and the improved functionality CXL switch 130 may each include a controller that is or includes processing circuitry such as an ARM processor. The PCIe interface may comply with the PCIe 5.0 standard, previous versions, or future versions, or an interface complying with other standards (e.g., NVDIMM-P, CCIX, or OpenCAPI) may be used instead of the PCIe interface. The memory module 135 may include a variety of memory types, including DDR4 DRAM, HBM, LDPPR, NAND flash, or solid state drives (SSDs). Memory module 135 may be partitioned or include a cache controller to handle multiple memory types, which may be in different form factors such as HHHL, FHHL, "M.2", "U.2", mezzanine card, daughter card, "E1.S", "E1.L", "E3.L", or "E3.S", etc.

[0090] In the embodiment of FIG. 1E, the improved functionality CXL switch 130 allows one-to-many and many-to-one switching and allows a fine grain load-store interface at the flit (64 byte) level. Each server may have a collection of memory devices, each divided into multiple logical devices with individual "LD-IDs." The (ToR) server link switch 112 (sometimes referred to as the "server link switch") enables one-to-many functionality, and the enhanced functionality CXL switch 130 of the server 105 enables many-to-one functionality. The server link switch 112 can be a PCIe switch, a CXL switch, or both. In such a system, the requesters may be the processing circuits 115 of multiple servers 105 and the responders may be many aggregated memory modules 135 .

[0091] Two layers of switches (as previously mentioned, the master switch is the server link switch 112 and the slave switch is the enhanced CXL switch 130) facilitate any-any communication. Each memory module 135 can have one physical function (PF) and up to 16 independent logical devices. In some embodiments, the number of logical devices (e.g., the number of partitions) may be limited (e.g., 16), and there may also be one control partition (which may be a physical function used to control the devices). Each memory module 135 may be a Type 2 device having a "CXL.cache," "CXL.mem," "CXL.io," and address translation services (ATS) implementation to handle cache line copies that the processing circuit 115 may hold.

[0092] The improved CXL switch 130 and fabric manager control the discovery of memory modules 135, (i) Perform device discovery and virtual CXL software generation; (ii) Binding virtualization to physical ports. As in the embodiment of Figures 1A-1D, the fabric manager operates via a connection on the SMBus sideband. The interface to memory module 135, which may be an interface that conforms to the Intelligent Platform Management Interface (IPMI) or Redfish standards (and may also provide additional functionality not required by the standards), enables configurability.

[0093] As previously mentioned, some embodiments implement a hierarchical structure in which a master controller (which may be implemented in an FPGA or ASIC) is part of the server link switch 112 and a slave controller is part of the enhanced functionality CXL switch 130 to provide a load-store interface (i.e., that has cache line (e.g., 64 byte) granularity and operates within the coherent domain without software driver intervention). Such load-store interfaces can extend the coherent domain beyond an individual server, CPU, or host and may involve physical media that are electrical or optical (e.g., optical connections with electrical-to-optical transceivers at both ends). In operation, the master controller (located in the server link switch 112) starts (or "reboots") and configures all the servers 105 in the rack.

[0094] The master controller can have visibility to all hosts, (i) discovering each server and discovering the number of servers 105 and memory modules 135 present in the server cluster; (ii) independently configuring each server 105; (iii) for example, enabling or disabling some blocks of memory on different servers based on the rack configuration (e.g., enabling or disabling any of the memory modules 135); (iv) control access (e.g., one server controls another); (v) implementing flow control (e.g., all host and device requests go through the master, which transfers data from one server to another and performs flow control on the data); (vi) grouping or batching requests or packets (e.g., multiple cache requests received by the master from different servers 105); (vii) Receive remote software updates, broadcast communications, etc.

[0095] In batch mode, the server link switch 112 receives multiple packets destined for the same server (eg, destined for the first server) and forwards them together (ie, without pause between them) to the first server. For example, the server link switch 112 receives a first packet from a second server and a second packet from a third server, and forwards both the first packet and the second packet to the first server. Each of the servers 105 is configured to communicate with the master controller: (i) IPMI network interface, (ii) Exposes the system event log (SEL) and baseboard management controller (BMC) to allow the master controller to measure performance, measure reliability on the fly, and reconfigure the server 105 .

[0096] In some embodiments, a software architecture is used that facilitates a high availability load-store interface. Such a software architecture may provide reliability, replication, consistency, system coherence, hashing, caching, and persistence. The software architecture provides reliability (in systems with large server populations) by performing periodic hardware checks on the CXL device components via IPMI. For example, the server link switch 112 queries the status of the memory server 150 via the IPMI interface, such as the power status (whether the power supply of the memory server 150 is working properly), the network status (whether the interface to the server link switch 112 is working properly), and the error check status (whether an error condition exists in the subsystem of the memory server 150). The software architecture provides replication in that the master controller can replicate data stored in memory modules 135 and maintain data consistency across the replicas.

[0097] The software architecture provides consistency in that the master controllers may be configured with different levels of consistency and the server link switch 112 can adjust the packet format depending on the level of consistency being maintained. For example, if eventual consistency is maintained, the server link switch 112 rearranges requests, while to maintain strict consistency, the server link switch 112 maintains a scoreboard of all requests with accurate timestamps at the switch. The software architecture can provide system coherence in that multiple processing circuits 115 can read from or write to the same memory address, and a master controller can be responsible for reaching the home node of the address (using a directory lookup) or broadcasting the request on a common bus to maintain coherence.

[0098] The software architecture provides hashing in that the server link switch 112, and with improved functionality the CXL switch 130, can maintain a virtual mapping of addresses that can use consistent hashing with multiple hash functions to map data evenly to all CXL devices across all nodes at startup (or to adjust when one server goes down or up). The software architecture provides caching in that the master controller can designate a particular memory partition (e.g., in a memory module 135 that includes HBM or similar technology) to act as a cache (using a write-through or write-back cache). The software architecture provides persistence in that the master and slave controllers can manage the persistence domains and the flash.

[0099] In some embodiments, the capabilities of the CXL switch are integrated into the controller of the memory module 135. In such an embodiment, the server link switch 112 may nevertheless act as a master and may have improved features as described elsewhere herein. The server link switch 112 may also manage other storage devices in the system, and may, for example, have an Ethernet connection (e.g., a 100 GbE connection) for connecting to client machines that are not part of the PCIe network formed by the server link switch 112.

[0100] In some embodiments, the server link switch 112 has improved functionality and also includes an integrated CXL controller. In other embodiments, the server link switch 112 is only a physical routing device, and each server 105 includes a master CXL controller. In such an embodiment, masters across different servers negotiate a master-slave architecture. (i) an improved function CXL switch 130; and (ii) The intelligent functionality of the server link switch 112 may be embodied in one or more FPGAs, one or more ASICs, one or more ARM processors, or one or more SSD devices with computing capabilities.

[0101] The server link switch 112 performs flow control, for example, by reordering independent requests. In some embodiments, RDMA is optional since the interface is load-store, although there may be intervening RDMA requests that use PCIe physical media (instead of 100GbE). In such an embodiment, a remote host initiates an RDMA request, which may be forwarded via the server link switch 112 to the enhanced CXL switch 130 . The server link switch 112 and the enhanced CXL switch 130 prioritize RDMA 4KB requests or CXL fleet (64 byte) requests.

[0102] As in the embodiment shown in Figures 1C and 1D, an improved CXL switch 130 may be configured to receive such RDMA requests and treat a group of memory modules 135 of the receiving server 105 (i.e., the server receiving the RDMA request) as its own memory space. Additionally, the enhanced CXL switch 130 virtualizes across the processing circuitry 115 and initiates RDMA requests to the remote enhanced CXL switch 130 to move data back and forth between the servers 105 without the processing circuitry 115 having to be involved.

[0103] FIG. 1F shows a system similar to that of FIG. 1E, in which processing circuitry 115 is connected to network interface circuitry 125 via an improved CXL switch 130. As in the embodiment of FIG. 1D, in FIG. 1F the improved functionality CXL switch 130, memory module 135, and network interface circuit 125 reside on an expansion socket adapter 140. Expansion socket adapter 140 may be a circuit board or module that plugs into an expansion socket, such as a PCIe connector 145, on the motherboard of server 105. Thus, the server may be any suitable server that is modified only by the installation of expansion socket adapter 140 on PCIe connector 145.

[0104] The memory module 135 may be installed in a connector (e.g., an M.2 connector) on the expansion socket adapter 140. In such an embodiment, (i) the network interface circuitry 125 is integrated into the CXL switch 130 for improved functionality; or (ii) each network interface circuit 125 may have a PCIe interface (the network interface circuit 125 may be a PCIe endpoint), such that the processing circuit 115 connected thereto can communicate with the network interface circuit 125 via a root port-to-endpoint PCIe connection, and the controller 137 of the improved functionality CXL switch 130 (which may have PCIe input ports connected to the processing circuit 115 and the network interface circuit 125) communicates with the network interface circuit 125 via a peer-to-peer PCIe connection;

[0105] According to one embodiment of the present invention, there is provided a system comprising: a first server including a stored program processing circuit, a cache coherent switch and a first memory module; a second server; and a server link switch connected to the first server and the second server, wherein the first memory module is connected to the cache coherent switch, the cache coherent switch is connected to the server link switch, and the stored program processing circuit is connected to the cache coherent switch.

[0106] In some embodiments, the server link switch comprises a Peripheral Component Interconnect Express (PCIe) switch. In some embodiments, the server link switch comprises a Compute Express Link (CXL) switch. In some embodiments, the server link switches include top-of-rack (ToR) CXL switches. In some embodiments, the server link switch is configured to find the first server. In some embodiments, the server link switch is configured to cause the first server to be restarted (rebooted). In some embodiments, the server link switch is configured to cause the cache coherent switch to disable the first memory module. In some embodiments, the server link switch is configured to forward data from the second server to the first server and to perform flow control on the data.

[0107] In some embodiments, the system includes a third server connected to a server link switch, the server link switch configured to receive a first packet from the second server, receive a second packet from the third server, and forward the first packet and the second packet to the first server. In some embodiments, the system further comprises a second memory module coupled to the cache coherent switch, the first memory module including volatile memory and the second memory module including persistent memory. In some embodiments, the cache coherent switch is configured to virtualize the first memory module and the second memory module.

[0108] In some embodiments, the first memory module includes a flash memory, and the cache coherent switch is configured to provide a flash translation layer for the flash memory. In some embodiments, the first server includes an expansion socket adapter connected to an expansion socket of the first server, the expansion socket adapter including a cache coherent switch and a memory module socket, and the first memory module is connected to the cache coherent switch via the memory module socket. In some embodiments, the memory module socket includes an "M.2" socket. In some embodiments, the cache coherent switch is connected to the server link switch via a connector, the connector being on an expansion socket adapter.

[0109] According to one embodiment of the present invention, there is provided a method for performing remote direct memory access in a computing system, the computing system including a first server, a second server, a third server, and a server link switch connected to the first server, the second server, and the third server, the first server including a processing circuit for stored programs, a cache coherent switch, and a first memory module, the method for performing remote direct memory access comprising: receiving a first packet from the second server by the server link switch; receiving a second packet from the third server by the server link switch; and forwarding the first packet and the second packet to the first server.

[0110] In some embodiments, the method further comprises receiving, by the cache coherent switch, a straight RDMA request; and forwarding, by the cache coherent switch, a straight RDMA response. In some embodiments, receiving the straight RDMA request includes receiving the straight RDMA request via a server link switch. In some embodiments, the method includes receiving a read command for a first memory address from a program processing circuit stored by the cache coherent switch, translating the first memory address to a second memory address by the cache coherent switch, and retrieving data from the first memory module at the second memory address by the cache coherent switch.

[0111] According to one embodiment of the present invention, there is provided a system comprising: a first server including a stored program processing circuit, a cache coherent switching means, and a first memory module; a second server; and a server link switch connected to the first server and the second server, wherein the first memory module is connected to the cache coherent switching means, the cache coherent switching means is connected to the server link switch, and the stored program processing circuit is connected to the cache coherent switching means.

[0112] FIG. 1G illustrates an embodiment in which multiple memory servers 150 are each connected to a ToR server link switch 112, which may be a PCIe 5.0 CXL switch. As in the embodiments of Figures 1E and 1F, the server link switch 112 can include an FPGA or an ASIC and can provide performance (in terms of throughput and latency) superior to that of an Ethernet switch. As in the embodiment of FIGS. 1E and 1F, memory server 150 may include multiple memory modules 135 connected to server link switch 112 via multiple PCIe connectors. In the embodiment of FIG. 1G, processing circuitry 115 and system memory 120 are absent, and the primary purpose of memory server 150 is to provide memory for use by other servers 105 with computing resources.

[0113] In the embodiment of FIG. 1G, the server link switch 112 groups or batches multiple cache requests received from different memory servers 150, grouping packets to reduce control overhead. The improved CXL Switch 130 (i) routing data to different memory types based on workload; (ii) It may include configurable hardware building blocks for virtualizing processor-side addresses (translating such addresses into memory-side addresses). The system shown in FIG. 1G may be CXL 2.0 based, includes configurable and isolated shared memory within a rack, and uses a ToR server link switch 112 to provide pooled (i.e., aggregated) memory to remote devices.

[0114] The ToR server link switch 112 may have additional network connections (e.g., an Ethernet connection as shown or other types of connections, e.g., a wireless connection such as a WiFi connection or a 5G connection) to connect to other servers or clients. The server link switch 112 and the improved functionality CXL switch 130 may each include a processing circuit such as an ARM processor, or a controller including the same. The PCIe interface may follow the PCIe 5.0 standard, an earlier version, a future version of the PCIe standard, or may follow other standards used instead of PCIe (e.g., NVDIMM-P, CCIX, or OpenCAPI). The memory module 135 may include a variety of memory types, including DDR4 DRAM, HBM, LDPPR, NAND flash, and SSDs (Solid State Drives). Memory modules 135 can be partitioned and include cache controllers to handle multiple memory types, which can be in different form factors such as HHHL, FHHL, "M.2", "U.2", mezzanine card, daughter card, "E1.S", "E1.L", "E3.L", or "E3.S".

[0115] In the embodiment of FIG. 1G, the improved functionality of the CXL switch 130 can enable one-to-many and many-to-one switching and can enable a fine grain load-store interface at the flit (64-byte) level. Each memory server 150 may have a collection of memory devices, each of which is divided into multiple logical devices with their own LD-ID. An improved functionality CXL switch 130 may include a controller 137 (e.g., an ASIC or FPGA), circuitry (which may be separate from or part of the ASIC or FPGA) for device discovery, enumeration, partitioning, and indicating ranges of physical addresses.

[0116] Each memory module 135 can have one physical function (PF) and up to 16 isolated logical units. In some embodiments, the number of logical devices (e.g., the number of partitions) may be limited (e.g., up to 16), and there may also be one control partition (which may be a physical function used to control the device). Each of the memory modules 135 may be a Type 2 device having an implementation of "CXL.cache," "CXL.mem," "CXL.io," and address translation services (ATS) to handle cache line copies that the processing circuit 115 may hold.

[0117] The improved functionality of the CXL switch 130 and fabric manager allows for control of the discovery of memory modules 135; (i) discovering devices and generating virtual CXL software; (ii) Binding virtualization to physical ports. As in the embodiment of Figures 1A-1D, the fabric manager operates over an SMBus sideband connection. An interface to memory module 135 that is an interface that conforms to the Intelligent Platform Management Interface (IPMI) or Redfish standard (and may also provide additional functionality not required by the standard) can enable configurability.

[0118] A building block for the embodiment of FIG. 1G (as previously described) may include a CXL controller 137 implemented on an FPGA or ASIC that can activate a collection of memory devices (e.g., memory modules 135), SSDs, accelerators (GPUs, NICs), CXL, and PCIe5 connectors, and firmware to expose device details to the operating system's advanced configuration and power interface (ACPI) tables, such as a heterogeneous memory attribute table (HMAT) or static resource affinity table (SRAT).

[0119] In some embodiments, the system provides composability. The system can provide functionality for online and offline CXL devices and other accelerators based on software configuration, and can group accelerator, memory, and storage device resources and distribute them to each memory server 150 in the rack. The system can hide the physical address space and use faster devices such as HBM and SRAM to provide a transparent cache.

[0120] In the embodiment of FIG. 1G, the controller 137 of the improved CXL switch 130 includes: (i) managing the memory module 135; (ii) Integrate and control heterogeneous devices such as NICs, SSDs, GPUs, and DRAM; (iii) It can affect the dynamic reconfiguration of storage to memory devices through power gating. For example, the ToR server link switch 112 (by instructing the enhanced CXL switch 130 to disable (deactivate) power to the memory modules 135) disables (i.e., cuts off or reduces) power to any one of the memory modules 135. When instructed by the server link switch 112 to disable power to the memory module 135, the improved CXL switch 130 disables power to the memory module. Such deactivation can conserve power and improve the performance (eg, throughput and latency) of other memory modules 135 in memory server 150 .

[0121] Each remote server 105 may know a different logical view of the memory modules 135 and their connections based on negotiation. The controller 137 of the improved CXL switch 130 can maintain state so that each remote server maintains its assigned resources and connections, and can perform memory compression or deduplication to conserve memory capacity (using configurable chunk sizes). The unpopulated racks in FIG. 1G may have their own BMC. It also exposes the IPMI network interface and system event log (SEL) to remote devices, enabling a master (e.g., a remote server using storage provided by memory server 150) to measure performance and reliability on the fly and reconfigure ungrouped wrappers.

[0122] The ungrouped racks of FIG. 1G can provide reliability, replication, consistency, system coherence, hashing, caching, and durability in a manner similar to that described herein for the embodiment of FIG. 1E. For example, coherence can be provided by multiple remote servers reading from or writing to the same memory address, with each remote server configured with a different level of consistency. In some embodiments, the server link switch maintains eventual consistency between data stored in the first memory server and data stored in the second memory server. The server link switch 112 maintains different levels of consistency for different pairs of servers. For example, the server link switch also maintains a level of consistency between data stored in the first memory server and data stored in the third memory server, which may be strict consistency, sequential consistency, causal consistency, or processor consistency.

[0123] The system can use communications in the "local band" (server link switch 112) and "global band" (non-aggregated server) domains. Writes can be flushed to the "global band" so that they become visible to new reads from other servers. The controller 137 of the improved CXL switch 130 manages the persistence domain and flushes it separately for each remote server. For example, a cache coherent switch may monitor the fullness of a first region of memory (volatile memory, acting as a cache), and when the fullness level exceeds a threshold, the cache coherent switch moves from the first region of memory to a second region of memory, which is persistent memory. The improved functionality of the controller 137 of the CXL switch 130 allows flow control to be handled so that priorities can be set to account for different perceived latencies and bandwidths among remote servers.

[0124] According to one embodiment of the present invention, there is provided a system comprising: a first memory server including a cache coherent switch and a first memory module; a second memory server; and a server link switch connected to the first memory server and the second memory server, wherein the first memory module is connected to the cache coherent switch and the cache coherent switch is connected to the server link switch.

[0125] In some embodiments, the server link switch is configured to disable power to the first memory module. In some embodiments, the server link switch is configured to disable power to the first memory module by instructing the cache coherent switch to disable power for the first memory module, and the cache coherent switch is configured to disable power to the first memory module when instructed by the server link switch to disable power to the first memory module. In some embodiments, the cache coherent switch is configured to perform deduplication within the first memory module. In some embodiments, the cache coherent switch is configured to compress the data and store the compressed data in the first memory module.

[0126] In some embodiments, the server link switch is configured to query the state of the first memory server. In some embodiments, the server link switch is configured to query the state of the first memory server via an intelligent platform management interface (IPMI). In some embodiments, querying a state includes querying a state selected from the group consisting of a power state, a network state, and an error check state. In some embodiments, the server link switch is configured to batch cache requests destined for the first memory server.

[0127] In some embodiments, the system further comprises a third memory server connected to the server link switch, the server link switch configured to maintain a level of consistency between data stored in the first memory server and data stored in the third memory server selected from the group consisting of strict consistency, sequential consistency, causal consistency and processor consistency. In some embodiments, the cache coherent switch is configured to monitor the fullness of a first region of memory and move data from the first region of memory to a second region of memory, where the first region of memory resides in volatile memory and the second region of memory resides in persistent memory.

[0128] In some embodiments, the server link switch comprises a Peripheral Component Interconnect Express (PCIe) switch. In some embodiments, the server link switch comprises a Compute Express Link (CXL) switch. In some embodiments, the server link switches include top-of-rack (ToR) CXL switches. In some embodiments, the server link switch is configured to transfer data from the second memory server to the first memory server and to perform flow control on the data. In some embodiments, the system further includes a third memory server connected to the server link switch, the server link switch configured to receive a first packet from the second memory server, receive a second packet from the third memory server, and forward the first packet and the second packet to the first memory server.

[0129] According to one embodiment of the present invention, there is provided a method for performing remote direct memory access in a computing system, the computing system comprising: a first memory server, the first server, a second server, and a server link switch connected to the first memory server, the first server, and the second server, the first memory server comprising a cache coherent switch and a first memory module, the first server comprising a processing circuit for a stored program, and the second server comprising a processing circuit for a stored program, the method comprising: receiving a first packet from the first server by the server link switch; receiving a second packet from the second server by the server link switch; and forwarding the first packet and the second packet to the first memory server.

[0130] In some embodiments, the method further comprises compressing the data with a cache coherent switch and storing the data in the first memory module. In some embodiments, the method further comprises querying the state of the first memory server via the server link switch.

[0131] According to one embodiment of the present invention, there is provided a system including a first memory server including a cache coherent switch and a first memory module, a second memory server, and a server link switching means connected to the first memory server and the second memory server, wherein the first memory module is connected to the cache coherent switch and the cache coherent switch is connected to the server link switching means.

[0132] 2A-2D are flow charts for various embodiments. In an embodiment of these flowcharts, processing circuitry 115 is a CPU. In other embodiments, the processing circuit 115 may be another processing circuit (eg, a GPU).

[0133] Referring to FIG. 2A, the controller 137 of the memory module 135 of the embodiment of FIGS. 1A and 1B, or the improved CXL switch 130 of any of the embodiments of FIGS. 1C-1G, virtualizes across the processing circuitry and initiates RDMA requests on the improved CXL switch 130 of the other server 105 to move data back and forth between servers 105 without involving processing circuitry in any of the servers (the virtualization is handled by the controller 137 of the improved CXL switch 130).

[0134] For example, in step S205, the controller 137 of the memory module 135 or the enhanced CXL switch 130 generates an RDMA request to additional remote memory (eg, CXL memory or aggregated memory). In step S210, the network interface circuitry 125 forwards the request to the ToR Ethernet switch 110 (which may have an RDMA interface) by bypassing the processing circuitry. In step S215, the ToR Ethernet switch 110 routes the RDMA request to the remote server 105 for processing by the controller 137 of the memory module 135 or the remote enhanced functionality CXL switch 130 via RDMA access to the remote aggregated memory, bypassing the remote processing circuitry 115. In step S220, the ToR Ethernet switch 110 receives the processed data and routes the data to the local memory module 135 or the local enhanced functionality CXL switch 130 via RDMA, bypassing the local processing circuitry 115. In step S222, the controller 137 of the memory module 135 in the embodiment of FIGS. 1A and 1B or the CXL switch 130 in the improved functionality receives the RDMA response directly (eg, without being forwarded by the processing circuit 115).

[0135] In such an embodiment, the controller 137 of the remote memory module 135 or the CXL switch 130 of the improved functionality of the remote server 105 is configured to receive straight remote direct memory access (RDMA) requests and forward straight RDMA responses. As used herein, a controller 137 of a remote memory module 135 or an improved CXL switch 130 receiving a "straight RDMA request" (or receiving such a request "straight") means receiving such a request by the controller 137 of the remote memory module or by the improved CXL switch without being forwarded or otherwise processed by the processing circuitry 115 of the remote server, and a controller 137 of a remote memory module 135 or an improved CXL switch 130 forwarding a "straight RDMA response" (or forwarding such a request "straight") means forwarding such a response without being forwarded or otherwise processed by the processing circuitry 115 of the remote server.

[0136] Referring to FIG. 2B, in another embodiment, RDMA is performed with processing circuitry of a remote server involved in data processing. For example, in step S225, the processing circuitry 115 transfers the data or workload request over Ethernet. In step S230, the ToR Ethernet switch 110 receives the request and routes it to a corresponding server 105 among the plurality of servers 105. In step S235, the request is received in the server via a port of the network interface circuit 125 (eg, a 100GbE-enabled NIC). In step S240, the processing circuit 115 (eg, an x86 processing circuit) receives a request from the network interface circuit 125. In step S245, processing circuit 115 processes the request using (e.g., together) the DDR and additional memory resources via the CXL 2.0 protocol to share memory (which may be aggregated memory in the embodiments of Figures 1A and 1B).

[0137] Referring to FIG. 2C, in the embodiment of FIG. 1E, RDMA occurs with processing circuitry in the remote server involved in processing the data. For example, in step S225C, the processing circuitry 115 transfers the data or workload request via Ethernet or PCIe. In step S230C, the ToR Ethernet switch 110 receives the request and routes it to a corresponding one of the plurality of servers 105. In step S235C, the request is received in the server via a port of the PCIe connector. In step S240C, the processing circuit 115 (eg, an x86 processing circuit) receives the request from the network interface circuit 125. In step S245C, processing circuit 115 processes the request (e.g., jointly) using the DDR and additional memory resources via the CXL 2.0 protocol to share the memory (which may be an aggregated memory in the embodiment of Figures 1A and 1B).

[0138] In step S250, processing circuit 115 identifies requirements for accessing memory contents (eg, DDR or DDR memory contents) from other servers. In step S252, processing circuit 115 forwards a request for memory contents (eg, DDR or aggregated memory contents) from another server via the CXL protocol (eg, CXL 1.1 or CXL 2.0). In step S254, the request is propagated via a local PCIe connector to the server link switch 112, which then forwards the request to a second PCIe connector of a second server in the rack. In step S256, the second processing circuit 115 (eg, an x86 processing circuit) receives the request from the second PCIe connector. In step S258, the second processing circuit 115 processes the request (e.g., retrieving the contents of the memory) together using the second DDR and the second additional memory resource via the CXL2.0 protocol to share the aggregated memory. In step S260, the second processing circuit (eg, the x86 processing circuit) transfers the result of the request back to the original processing circuit via the respective PCIe connectors and the server link switch 112.

[0139] Referring to FIG. 2D, in the embodiment of FIG. 1G, RDMA is performed using, for example, processing circuitry of the remote server involved in processing the data. In step S225D, processing circuitry 115 transfers the data and workload request over Ethernet. In step S230D, the ToR Ethernet switch 110 receives the request and routes it to a corresponding one of the plurality of servers 105. In step S235D, the request is received in the server via a port of the network interface circuit 125 (eg, a 100 GbE-enabled NIC). In step S262, the memory module 135 receives the request from the PCIe connector. In step S264, the controller of the memory module 135 processes the request using its local memory.

[0140] In step S250D, the controller of the memory module 135 identifies requirements for accessing the contents of the memory (eg, the contents of the aggregated memory) from other servers. In step S252D, the controller of memory module 135 forwards the request for memory contents (eg, aggregate memory contents) from other servers via the CXL protocol. In step S254D, the request is propagated via the local PCIe connector to the server link switch 112, which forwards the next request to a second PCIe connector of a second server in the rack. In step S266, the second PCIe connector shares the aggregated memory and provides access through the CXL protocol so that the controller of the memory module 135 can retrieve the contents of the memory.

[0141] As used herein, a "server" includes at least one stored program processing circuit (e.g., processing circuit 115), at least one memory resource (e.g., system memory 120), and at least one circuit for providing network connectivity (e.g., network interface circuit 125). As used herein, "a portion of" means "at least a portion" of something, and thus can mean all or less than all of something. In this way, a "part" of a thing includes the whole thing as a special case. That is, the whole thing is an example of a part of the thing.

[0142] The background provided in the Background Technology section of this disclosure is included solely to set the context, and the contents of this section are not admitted to be prior art. Any component or combination of components illustrated (e.g., in any system diagram included herein) may be used to perform any one or more of the operations of any flowchart included herein. In addition, (i) the operations are by way of example and may include various additional steps not explicitly addressed; (ii) the temporal order of the operations may be varied;

[0143] Exemplary embodiments of systems and methods for managing memory resources are specifically illustrated and described herein. The present invention is not limited to the above-described embodiment, and various modifications can be made without departing from the technical scope of the present invention. [Explanation of symbols]

[0144] 105 (1st) Server 110 ToR Ethernet Switch 112 (ToR) Server Link Switch 115 Processing circuit (stored program processing circuit) 120 system memory 125 Network Interface Circuit 130 Improved functionality of CXL switches (cache coherent switches) 135 (first) memory module 137 Controller 140 Expansion Socket Adapter 145 Expansion Socket

Claims

1. a first server including stored program processing circuitry, a cache-coherent switch, and a first memory module; A second server; a server link switch connected to the first server and the second server; the first memory module is connected to the cache coherent switch; the cache coherent switch is connected to the server link switch; the stored program processing circuitry is connected to the cache coherent switch; The server link switch includes a PCIe (Peripheral Component Interconnect Express) switch or a CXL (Compute Express Link) switch; the first memory module includes a controller for converting signals to conform to a protocol of the first memory module; The controller further comprises: Properly binding and unbinding upstream and downstream connections during runtime; A system for managing memory resources, comprising a switch management device that enables control semantics and statistics related to data transfers to and from the first memory module.

2. 2. The system for managing memory resources according to claim 1, wherein the server link switch comprises a top of rack (ToR) CXL switch.

3. 2. The system of claim 1, wherein the server link switch discovers the first server.

4. 2. The system of claim 1, wherein the server link switch reboots the first server.

5. 2. The system of claim 1, wherein the server link switch causes the cache coherent switch to deactivate the first memory module.

6. 2. The system for managing memory resources of claim 1, wherein the server link switch transfers data from the second server to the first server and performs flow control on the data.

7. a third server connected to the server link switch; the server link switch receives a first packet from the second server and a second packet from the third server; 2. The system for managing memory resources according to claim 1, further comprising: forwarding the first packet and the second packet to the first server.

8. a second memory module coupled to the cache coherent switch; the first memory module includes a volatile memory; 2. The system of claim 1, wherein the second memory module includes persistent memory.

9. 9. The system for managing memory resources of claim 8, wherein the cache coherent switch virtualizes the first memory module and the second memory module.

10. the first memory module includes a flash memory; 10. The system of claim 9, wherein the cache coherent switch provides a flash translation layer for the flash memory.

11. the first server includes an expansion socket adapter connected to an expansion socket of the first server; the expansion socket adapter includes the cache coherent switch and a memory module socket; 2. The system for managing memory resources according to claim 1, wherein the first memory module is connected to the cache coherent switch via the memory module socket.

12. 12. The system for managing memory resources of claim 11, wherein the memory module socket comprises an M.2 socket.

13. the cache coherent switch is connected to the server link switch via a connector; 12. The system for managing memory resources of claim 11, wherein the connector is on the expansion socket adapter.

14. 1. A method for performing remote direct memory access (RDMA) in a computing system, comprising: The computing system includes a first server; A second server; A third server; a server link switch connected to the first server, the second server, and the third server; the first server includes a stored program processing circuit, a cache coherent switch, and a first memory module; The server link switch includes a PCIe (Peripheral Component Interconnect Express) switch or a CXL (Compute Express Link) switch; the first memory module includes a controller for converting signals to conform to a protocol of the first memory module; The controller further comprises: Properly binding and unbinding upstream and downstream connections during runtime; a switch management unit that enables control semantics and statistics related to data transfers to and from the first memory module; The method for performing remote direct memory access includes receiving, by the server link switch, a first packet from the second server; receiving, by the server link switch, a second packet from the third server; forwarding the first packet and the second packet to the first server.

15. receiving a remote direct memory access (RDMA) request by the cache coherent switch; 15. The method for managing memory resources of claim 14, further comprising: forwarding an RDMA response through the cache coherent switch.

16. 16. The method of claim 15, wherein receiving the RDMA request comprises receiving the RDMA request through the server link switch.

17. receiving, by the cache coherent switch, a read command from the stored program processing circuitry to a first memory address; translating the first memory address to a second memory address by the cache coherent switch; 16. The method of claim 15, further comprising retrieving data from the first memory module at the second memory address by the cache coherent switch.

18. a first server including stored program processing circuitry, cache coherent switching means, and a first memory module; A second server; a server link switch connected to the first server and the second server; the first memory module is connected to the cache coherent switching means; the cache coherent switching means is connected to the server link switch; said stored program processing circuitry is connected to said cache coherence switching means; The server link switch includes a PCIe (Peripheral Component Interconnect Express) switch or a CXL (Compute Express Link) switch; the first memory module includes a controller for converting signals to conform to a protocol of the first memory module; The controller further comprises: Properly binding and unbinding upstream and downstream connections during runtime; A system for managing memory resources, comprising a switch management device that enables control semantics and statistics related to data transfers to and from the first memory module.

Images