System and method for scalable and consistent memory device

Abstract

Systems and methods are provided for scalable and consistent memory devices. The methods can include operating a first controller coupled to a network interface in accordance with a cache coherency protocol, performing at least one operation on data associated with a cache using a second controller coupled to the first controller and coupled to a first memory, and storing the data on a second memory coupled to one of the first controller or the second controller.

Description

[0001] This application claims priority and benefit to U.S. Provisional Application No. 63 / 031,508, filed May 28, 2020, entitled “Extending memory accesses with novel cache coherence connections”; U.S. Provisional Application No. 63 / 031,509, filed May 28, 2020, entitled “Pooling server memory resources for computational efficiency”; and U.S. Provisional Application No. 63 / 031,509, filed August 20, 2020, entitled “System with cache-coherent memory and server-linking switch”. The priority and benefits of U.S. Provisional Application No. 63 / 068,054 entitled “Disaggregated Memory Architecture with Novel Interconnects”, filed July 28, 2020, and U.S. Provisional Application No. 63 / 057,746 entitled “Disaggregated Memory Architecture with Novel Interconnects”, filed April 30, 2021, are incorporated herein by reference in their entirety. Technical Field

[0002] This disclosure relates generally to cache coherence, and more specifically to systems and methods for scalable and coherent memory devices. Background Technology

[0003] Some server systems may include a collection of servers connected via network protocols. Each server in such a system may include processing resources (e.g., processors) and memory resources (e.g., system memory). In some cases, it may be advantageous for the processing resources of one server to access the memory resources of another server, and it may be advantageous to make such access while minimizing the processing resources of either server.

[0004] Therefore, there is a need for an improved system and method for managing memory resources in a system that includes one or more servers.

[0005] The information disclosed above in this background section is only intended to enhance the understanding of the disclosed background technology, and therefore may contain information that does not constitute prior art. Summary of the Invention

[0006] In various embodiments, the description herein includes systems, methods, and apparatuses for providing storage resources. Specifically, an apparatus is described comprising: a first controller coupled to a network interface, wherein the first controller is capable of operating using a cache coherence protocol; a second controller coupled to the first controller and coupled to a first memory, wherein the second controller performs at least one operation associated with a cache of the apparatus; and a second memory coupled to at least one of the first controller or the second controller.

[0007] In some embodiments, the apparatus may receive configuration information from a host; configure the apparatus to operate as either a software-based cache or a hardware-based cache in an operation startup mode; receive data via the network interface using the cache coherence protocol via the first controller; perform at least one second operation on the data to generate second data; and store the second data on a second memory based on the cache coherence protocol. In another embodiment, the cache coherence protocol may include a Fast Compute Link (CXL) protocol, and wherein the apparatus further includes a profiler that determines at least one capability associated with the apparatus and provides the capability to the host.

[0008] In other embodiments, the apparatus further includes a third controller coupled to a third memory, wherein the third memory includes non-volatile memory. In various embodiments, the first memory includes a first volatile memory, and the second memory includes a second volatile memory. In other embodiments, the first volatile memory includes double data rate memory or low-power double data rate memory. In some embodiments, the first memory includes a first non-volatile memory, and the second memory includes a second non-volatile memory. In an embodiment, the first non-volatile memory includes 3D flash memory.

[0009] In various embodiments, the device includes at least one of the following: M.2 shape factor, E1.L shape factor, E1.S shape factor, E3S / L shape factor, U.2 shape factor, NF1 shape factor, full height half length (FHHL) card (AIC) shape factor, or half height half length (HHHL) AIC shape factor.

[0010] In other embodiments, the second controller includes at least one of a flow control module, a cache addressing module, or a cache policy module. In some embodiments, the second controller includes at least one of a channel request queue, a volatile memory request scheduler, or a volatile memory command scheduler. In other embodiments, the apparatus further includes an accelerator or a network interface card (NIC).

[0011] Similarly, systems and methods for performing the essentially the same or similar operations as described above are further disclosed.

[0012] Therefore, specific embodiments of the subject matter described herein can be implemented to achieve one or more of the following advantages: reduced network latency, improved network stability and operational data transmission rates, thereby improving user experience; reduced costs associated with routing network traffic, network maintenance, network upgrades, etc. Furthermore, in some aspects, the disclosed system can be used to reduce the power consumption and / or bandwidth of devices on a network, and can be used to improve the speed and / or efficiency of communication between devices. Attached Figure Description

[0013] The foregoing and other aspects of this application will be better understood when considering the following drawings, in which the same reference numerals denote similar or identical elements:

[0014] Figure 1A This is a block diagram of a system for attaching memory resources to computing resources using cache-coherent joins according to embodiments of the present disclosure;

[0015] Figure 1B This is a block diagram of a system employing an expansion slot adapter for attaching memory resources to computing resources using cache coherent connections, according to embodiments of the present disclosure.

[0016] Figure 1C This is a block diagram of a system for aggregating memory using an Ethernet top-of-rack (ToR) switch, according to embodiments of the present disclosure.

[0017] Figure 1D This is a block diagram of a system for aggregated memory employing an Ethernet ToR switch and an expansion slot adapter according to embodiments of the present disclosure;

[0018] Figure 1E This is a block diagram of a system for a converged memory according to embodiments of the present disclosure;

[0019] Figure 1F This is a block diagram of a system for aggregated memory employing an expansion slot adapter according to an embodiment of the present disclosure;

[0020] Figure 1GThis is a block diagram of a system for a decomposed server according to embodiments of the present disclosure;

[0021] Figure 2 A diagram illustrating a representative system architecture according to disclosed example embodiments is provided, wherein aspects of the disclosed embodiments can be combined to enable communication and configuration. Figures 1A to 1G The description describes the operation of various server management computing entities.

[0022] Figure 3A A first diagram depicts a representative system architecture according to a disclosed example embodiment, wherein aspects of the disclosed embodiments can be combined to enable communication and configuration. Figures 1A to 1G The description describes the operation of various server management computing entities.

[0023] Figure 3B A second diagram depicts a representative system architecture according to the disclosed example embodiments, wherein aspects of the disclosed embodiments can be combined to enable communication and configuration. Figures 1A to 1G The description describes the operation of various server management computing entities.

[0024] Figure 3C A third diagram depicts a representative system architecture according to the disclosed example embodiments, wherein aspects of the disclosed embodiments can be combined to enable communication and configuration. Figures 1A to 1G The description describes the operation of various server management computing entities.

[0025] Figure 3D A fourth diagram depicts a representative system architecture according to the disclosed example embodiments, wherein aspects of the disclosed embodiments can be combined to enable communication and configuration. Figures 1A to 1G The description describes the operation of various server management computing entities.

[0026] Figure 4 A characterizable combination according to the disclosed example embodiments is described. Figures 1A to 1G The diagram illustrates representative parameter tables for various aspects of the described servers, where the management computing entity configures the various servers based on these parameter tables.

[0027] Figure 5 The diagram illustrates different configurations of a memory device according to exemplary embodiments of the present disclosure.

[0028] Figure 6 This is an illustration of an exemplary table associated with device-related parameters according to an example embodiment of this disclosure.

[0029] Figure 7 This is a diagram of an exemplary cache coherency device microarchitecture according to an example embodiment of the present disclosure.

[0030] Figure 8 This is an illustration of an exemplary apparatus for resource management according to an example embodiment of the present disclosure.

[0031] Figure 9 This is an illustration of an exemplary cache coherence apparatus according to an example embodiment of the present disclosure.

[0032] Figure 10 This is an illustration of an exemplary flow including example operations associated with the disclosed system, according to an example embodiment of this disclosure.

[0033] Figure 11 An example schematic diagram of a system that can be used to practice embodiments of the present disclosure is shown.

[0034] Figure 12 An example schematic diagram of a management computing entity is shown according to a disclosed example embodiment.

[0035] Figure 13 An example schematic diagram of a user device according to a disclosed example embodiment is shown.

[0036] While this technology is readily adaptable to various modifications and alternatives, specific embodiments of the technology are illustrated by way of example in the accompanying drawings and will be described herein. The drawings may be non-to scale. However, it should be understood that the drawings and their detailed description are not intended to limit the technology to the specific forms disclosed, but rather are intended to cover all modifications, equivalents, and alternatives falling within the spirit and scope of the technology as defined by the appended claims. Detailed Implementation

[0037] Details of one or more embodiments of the subject matter described herein are set forth in the accompanying drawings and the following description. Further features, aspects, and advantages of the subject matter will become apparent from the specification, drawings, and claims.

[0038] Various embodiments of this disclosure will now be described more fully below with reference to the accompanying drawings, which illustrate some, but not all, of the embodiments. In fact, the disclosure may be implemented in many different forms and should not be construed as limited to the embodiments set forth herein; rather, these embodiments are provided so that this disclosure will satisfy applicable legal requirements. Unless otherwise indicated, the term “or” is used herein in an optional and combined sense. The terms “illustrative” and “example” are used as examples that do not indicate a level of quality. The same reference numerals always denote the same elements. Arrows in each figure depict bidirectional data flow and / or bidirectional data flow capabilities. The terms “path,” “pathway,” and “route” are used interchangeably herein.

[0039] Embodiments of this disclosure can be implemented in various ways, including as a computer program product containing an article of manufacture. A computer program product may include a non-transitory computer-readable storage medium storing applications, programs, program components, scripts, source code, program code, object code, bytecode, compiled code, interpreted code, machine code, executable instructions, etc. (also referred to herein as executable instructions, instructions for execution, computer program product, program code, and / or similar terms used interchangeably herein). Such non-transitory computer-readable storage media include all computer-readable media (including volatile and non-volatile media).

[0040] In one embodiment, a non-volatile computer-readable storage medium may include floppy disks, flexible disks, hard disks, solid-state storage devices (SSS) (e.g., solid-state drives (SSDs)), solid-state cards (SSCs), solid-state components (SSMs), enterprise-class flash drives, magnetic tape, or any other non-transitory magnetic media. Non-volatile computer-readable storage media may also include punched cards, paper tape, optically marked sheets (or any other physical medium having a perforated pattern or other optically identifiable markings), optical disc read-only memories (CD-ROMs), rewritable optical discs (CD-RWs), digital universal discs (DVDs), Blu-ray discs (BDs), and any other non-transitory optical media. Such non-volatile computer-readable storage media may also include read-only memories (ROMs), programmable read-only memories (PROMs), erasable programmable read-only memories (EPROMs), electrically erasable programmable read-only memories (EEPROMs), flash memory (e.g., serial, NAND, NOR, etc.), multimedia memory cards (MMCs), secure digital storage (SD) cards, smart media cards, compact flash memory (CF) cards, memory sticks, etc. In addition, non-volatile computer-readable storage media may also include conductive bridged random access memory (CBRAM), phase change random access memory (PRAM), ferroelectric random access memory (FeRAM), non-volatile random access memory (NVRAM), magnetoresistive random access memory (MRAM), resistive random access memory (RRAM), silicon-oxide-nitride-oxide-silicon memory (SONOS), floating junction gate random access memory (FJGRAM), Millipede memory, racetrack memory, etc.

[0041] In one embodiment, a volatile computer-readable storage medium may include random access memory (RAM), dynamic random access memory (DRAM), static random access memory (SRAM), fast page mode dynamic random access memory (FPM DRAM), extended data output dynamic random access memory (EDO DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), double data rate type II synchronous dynamic random access memory (DDR2 SDRAM), double data rate type III synchronous dynamic random access memory (DDR3 SDRAM), Rambus dynamic random access memory (RDRAM), dual transistor RAM (TTRAM), thyristor RAM (T-RAM), zero capacitor (Z-RAM), Rambus through-hole memory module (RIMM), dual in-line memory module (DIMM), single in-line memory module (SIMM), video random access memory (VRAM), cache memory (including various levels), flash memory, register memory, etc. It will be understood that, where the embodiments are described as using computer-readable storage media, other types of computer-readable storage media may be used in place of the computer-readable storage media described above, or other types of computer-readable storage media may be used in addition to the computer-readable storage media described above.

[0042] It should be understood that the various embodiments of this disclosure can also be implemented as methods, devices, systems, computing devices, computing entities, etc. Therefore, embodiments of this disclosure can take the form of devices, systems, computing devices, computing entities, etc., that execute instructions stored on a computer-readable storage medium to perform specific steps or operations. Therefore, embodiments of this disclosure can also take the form of entirely hardware embodiments, entirely computer program product embodiments, and / or embodiments including combinations of hardware and computer program products that perform specific steps or operations.

[0043] Embodiments of this disclosure are described below with reference to block diagrams and flowcharts. Therefore, it should be understood that each block of the block diagrams and flowcharts can be implemented as a computer program product, a complete hardware embodiment, a combination of hardware and computer program products, and / or an apparatus, system, computing device, computing entity, etc., that executes instructions, operations, steps, and interchangeable similar terms (e.g., executable instructions, instructions for execution, program code, etc.) on a computer-readable storage medium. For example, code retrieval, loading, and execution can be performed sequentially, such that one instruction is retrieved, loaded, and executed at a time. In some example embodiments, retrieval, loading, and / or execution can be performed in parallel, such that multiple instructions are retrieved, loaded, and / or executed together. Thus, such embodiments can produce machines specifically configured to perform the steps or operations specified in the block diagrams and flowcharts. Therefore, the block diagrams and flowcharts support various combinations of embodiments for performing specified instructions, operations, or steps.

[0044] In several respects, networked computing and storage face challenges as data demands increase. Specifically, hyperscale workloads are becoming increasingly demanding due to their diverse memory and input / output (I / O) latency requirements, in addition to high bandwidth allocation needs. Furthermore, some existing systems exhibit reduced resource resilience without requiring reconfiguration of hardware rack systems, leading to inefficiencies that hinder data processing and storage requirements. Additionally, compute and memory resources are becoming increasingly intertwined, and increased demands on one can impact demands on the other. Moreover, the industry as a whole faces a shortage of viable distributed shared memory and large address space systems. In some respects, fixed resources can increase ownership costs (e.g., for data center-based environments) and limit the peak performance of subsystems. In others, hardware used in such environments may have different replacement cycles and associated timelines, potentially further complicating the upgrading of such systems. Therefore, improvements in resource sharing and workload matching within networked computing systems are needed.

[0045] In some embodiments, cache coherence protocols (such as Fast Compute Link (CXL)) can implement memory expansion and coherence accelerators. In various embodiments, the disclosed systems can use cache coherence protocols (such as CXL) to implement a class of memory systems and accelerators, adapting to different workloads requiring unique configurations. Therefore, the disclosed systems can implement composable cache-coherent (e.g., CXL) memory and accelerator resources by leveraging structures and architectures that present a system view to each workload running across racks in one or more clusters in a data center, for example. In some aspects, the disclosed systems can be used to extend cache coherence beyond a single server, provide heterogeneous rack management based on workload requirements, and provide resource composability. Furthermore, in some examples, CXL via a PCIe architecture can be used as a counterpart to another protocol (such as architecture-based Non-Volatile Memory Fast (NVMeoF), which can be used for the composability of remote I / O devices). As used herein, composability can represent an attribute by which a given device (e.g., a cache-coherent device in a particular cluster) can request and / or obtain resources (e.g., memory, compute, and / or network resources) (e.g., for performing at least a portion of a workload) from different parts of the network (e.g., at least one other cache-coherent device in a second cluster). In some embodiments, as used herein, composability can include using a fluid pool of physical and virtual compute, storage, and infrastructure resources in any suitable configuration to run any application or workload.

[0046] In various embodiments, the disclosed system may include one or more architectural components, including cache-coherent CXL modules having one or more processors (e.g., RISC-V processors) that can be configured to perform various operations associated with the control plane. Furthermore, the disclosed system may be able to utilize one or more homogeneous pools of cache-coherent CXL resources, which will be discussed further below. Specifically, the disclosed system may feature a management computing device for exposing and utilizing the performance and capacity, as well as acceleration features, of cache-coherent resources for use by various network devices. Specifically, the management computing device may determine one or more parameters associated with the system on which the management computing device operates, and route workloads to different clusters based on these parameters.

[0047] In various embodiments, the disclosed system may be able to use multiple homogeneous resource pools, each dedicated to a specific cache coherence architecture. Specifically, the disclosed system may use type A clusters (which may represent a collection of servers with direct-attached memory expansion devices (SCM, DRAM, DRAM-ZNAND hybrid), type B clusters (which may represent a collection of CXL 2 composable coherence accelerators), type C clusters (which may include CXL devices connected in a distributed memory system architecture with backdoor PCIe interconnects, whereby processes share the same address space), and type D clusters (which include physical cluster memory and accelerator expansions in the same structure (e.g., chassis).

[0048] In various embodiments, the disclosed system, including the management of computing devices, may be characterized by an intelligent device architecture. Specifically, the disclosed system may be characterized by a device that inserts a cache coherence interface (e.g., a CXL / PCIe5 interface) and may implement various cache and memory protocols (e.g., type-2 device based CXL.cache and CXL.memory protocol). Furthermore, in some examples, the device may include a programmable controller or processor (e.g., a RISC-V processor) that can be configured to present remote coherent devices as part of a local system negotiated using a cache coherence protocol (e.g., the CXL.IO protocol).

[0049] In various embodiments, the disclosed system enables cluster-level performance-based control and management capabilities, thereby automatically routing workloads (e.g., via algorithmic and / or machine learning-based methods) based on remote architectural configurations and device performance, power characteristics, etc. In some examples, the disclosed system can be programmed at least partially via ASIC circuitry, FPGA units, etc. Furthermore, such devices can implement AI-based techniques (e.g., machine learning-based methods) to route workloads as shown and described herein. Additionally, the disclosed system can use management computing entities to perform discovery and / or workload partitioning and / or resource binding based on predetermined criteria (e.g., optimal performance per unit of currency or power). Furthermore, the management computing entity can perform such operations based on various system parameters, including but not limited to round-trip time based on cache coherence protocols (e.g., CXL-based), determination of whether the device is host-biased or device-biased, switch hierarchy and / or host upstream port to device downstream port binding based on cache coherence protocols (e.g., CXL-based), switch fabrication manager configuration based on cache coherence protocols (e.g., CXL-based), protocol packets or physical media packets based on cache coherence protocols (e.g., CXL.IO or PCIe intervention bulk 4KB packets), network latency, memory technology based on cache coherence protocols (e.g., CXL-based) (e.g., memory type), and combinations thereof.

[0050] In various embodiments, the management computing entity may operate at the rack and / or cluster level and / or at least partially within a given device (e.g., a device with cache coherence enabled) that is part of a given cluster architecture (e.g., type A, B, C, and / or D clusters). In various embodiments, a device within a given cluster architecture may perform a first portion of the operations of the management computing entity, while another portion of the operations of the management computing entity may be implemented on the rack and / or at the cluster level. In some embodiments, the two portions of the operations may be performed in a coordinated manner (e.g., a device in the cluster sends a coordination message to and receives a coordination message from the management computing entity implemented on the rack and / or at the cluster level). In some embodiments, the first portion of the operations associated with a device in the cluster may include, but is not limited to, operations for determining current or future resource requirements of the device or cluster, announcing current or future resource availability of the device or cluster, synchronizing specific parameters associated with algorithms running at the device or cluster level, training one or more machine learning modules associated with the operation of the device or rack / cluster, recording corresponding data associated with routing workloads, combinations thereof, etc.

[0051] Fast Peripheral Component Interconnect (PCIe) can represent a computer interface with relatively high and variable latency, which can limit the usefulness of the computer interface when connecting to memory. CXL is an open industry standard for communication via PCIe 5.0, offering fixed, relatively short packet sizes, thus enabling relatively high bandwidth and relatively low fixed latency. Therefore, CXL supports cache coherency and is well-suited for connection to memory. CXL can also be used to provide connectivity between the host and network interface circuitry (or "network interface controller" or "network interface card (NIC)") in accelerators, memory devices, and servers.

[0052] Cache coherence protocols (such as CXL) can also be used for heterogeneous processing (e.g., in scalar, vector, and buffer memory systems). CXL can be used to provide a cache-coherent interface by leveraging channels, retimers, the system's PHY layer, logical aspects of the interface, and protocols from PCIe 5.0. The CXL transaction layer may include three multiplexed sub-protocols running simultaneously on a single link and may be referred to as CXL.io, CXL.cache, and CXL.memory. CXL.io may include I / O semantics similar to PCIe. CXL.cache may include cache semantics, and CXL.memory may include memory semantics; both cache semantics and memory semantics may be optional. Like PCIe, CXL may support (i) local widths that can be partitioned x16, x8, and x4, (ii) data rates of 32GT / s, downgradable to 8GT / s and 16GT / s, 128b / 130b, (iii) 300W (75W in x16 connectors), and (iv) plug-and-play. To support plug-and-play, PCIe or CXL device links can begin training in the PCIe of Gen1, negotiate CXL, complete Gen1-5 training, and then begin CXL transactions.

[0053] In some embodiments, as discussed in further detail below, using CXL connections for the aggregation or “pooling” of memory (e.g., multiple memories, including multiple memory cells connected together) can provide various advantages in systems comprising multiple servers connected together via a network. For example, a CXL switch (referred to herein as an “enhanced capability CXL switch”) with capabilities other than providing packet switching functionality for CXL packets can be used to connect the aggregation of memory to one or more central processing units (CPUs) (or “central processing circuitry”) and one or more network interface circuitry (which may have enhanced capabilities). Such a configuration allows (i) the aggregation of memory to include various types of memory with different characteristics, (ii) the enhanced capability CXL switch to virtualize the aggregation of memory and store data with different characteristics (e.g., access frequency) in appropriate types of memory, and (iii) the enhanced capability CXL switch to support Remote Direct Memory Access (RDMA), enabling RDMA to be performed with little or no involvement from the processing circuitry of the servers. As used herein, “virtualizing” memory means performing memory address translation between the processing circuitry and the memory.

[0054] CXL switches can (i) support memory and accelerator decomposition through single-level switching, (ii) enable resources to be offline and online across domains, which can enable time-division multiplexing across domains based on demand, and (iii) support the virtualization of downstream ports. CXL can be used to implement aggregated memory, which enables one-to-many and many-to-one switching (e.g., it can be able to (i) connect multiple root ports to one endpoint, (ii) connect one root port to multiple endpoints, or (iii) connect multiple root ports to multiple endpoints), wherein, in some embodiments, the aggregated device is divided into multiple logical devices, each with its own LD-ID (Logical Device Identifier). In such embodiments, physical devices can be divided into multiple logical devices, each visible to its corresponding initiator. The device may have one physical function (PF) and multiple (e.g., 16) isolated logical devices. In some embodiments, the number of logical devices (e.g., the number of partitions) can be limited (e.g., limited to 16), and a control partition (which may be for controlling the physical function of the device) may also exist.

[0055] In some embodiments, a schema manager may be employed for (i) performing device discovery and virtual CXL software creation and (ii) binding virtual ports to physical ports. Such a schema manager operates via an SMBus sideband connection. The schema manager may be implemented in hardware, software, firmware, or a combination thereof, and may reside, for example, in a host, in one of the memory modules 135, in an enhanced capability cache coherent switch 130, or elsewhere in the network. In some embodiments, the cache coherent switch may be a CXL switch 130. The schema manager may issue commands (including commands issued via the sideband bus or via the PCIe tree).

[0056] Reference Figure 1A In some embodiments, the server system includes multiple servers 105 connected together via a top-of-rack (ToR) Ethernet switch 110. While the switch is described as using the Ethernet protocol, any other suitable network protocol may be used. Each server includes one or more processing circuits 115, each processing circuit 115 connected to (i) system memory 120 (e.g., Double Data Rate (Version 4) (DDR4) memory or any other suitable memory), (ii) one or more network interface circuits 125, and (iii) one or more CXL memory modules 135. Each of the processing circuits 115 may be stored program processing circuitry (e.g., a central processing unit (CPU (e.g., x86 CPU), graphics processing unit (GPU), or ARM processor)). In some embodiments, the network interface circuit 125 may be embedded in one of the memory modules 135 (e.g., on the same semiconductor chip as one of the memory modules 135, or in the same module as one of the memory modules 135), or the network interface circuit 125 may be packaged separately from the memory module 135.

[0057] In various embodiments, the management computing entity 102 (described in detail below) may be configured to include processing elements (e.g., processors, FPGAs, ASICs, controllers, etc.) that can monitor one or more parameters associated with any part of the network (e.g., Ethernet traffic, data center parameters, ToR Ethernet switch 110 parameters, parameters associated with server 105, associated parameters of network interface circuitry (NIC) 125, associated parameters of one or more CXL memory modules 135, combinations thereof, etc.) to route workloads and / or portions of workloads to different parts of the network (including those described herein). Figures 1A to 1G(Any suitable component). Furthermore, as described above, in various embodiments, the disclosed system enables control and management capabilities based on cluster-level performance, whereby workloads can be automatically routed based on remote architectural configurations and device performance, power characteristics, etc. (e.g., via algorithmic methods and / or machine learning-based methods). In some examples, the disclosed system can be programmed at least partially via ASIC circuitry, FPGA units, etc. Furthermore, such a device can implement AI-based techniques (e.g., machine learning-based methods) to route workloads as shown and described herein. Additionally, the disclosed system can use a management computing entity to perform discovery and / or workload partitioning and / or resource binding based on predetermined criteria (e.g., optimal performance per unit of currency or power). Furthermore, the management computing entity can perform such operations based on various system parameters, including but not limited to: round-trip time based on cache coherence protocols (e.g., CXL-based), determination of whether the device is host-biased or device-biased, switch hierarchy and / or host upstream port to device downstream port binding based on cache coherence protocols (e.g., CXL-based), switch fabrication manager configuration based on cache coherence protocols (e.g., CXL-based), protocol packets or physical media packets based on cache coherence protocols (e.g., CXL.IO or PCIe intermediate batch 4KB packets), network latency, memory technology based on cache coherence protocols (e.g., CXL-based) (e.g., memory type), and combinations thereof.

[0058] As used herein, a “memory module” is a package (e.g., a package including a printed circuit board and components connected thereto, or an encapsulation including a printed circuit board) comprising one or more memory dies, each memory die comprising multiple memory cells. Each of each memory die or set of multiple memory dies may be housed in a package (e.g., an epoxy molding compound (EMC) package) soldered to (or connected to the printed circuit board of the memory module via a connector) the printed circuit board of the memory module. Each memory module 135 may have a CXL interface and may include a controller 137 (e.g., an FPGA, ASIC, processor, etc.) for translating between CXL packets and the memory interface of the memory die (e.g., signals of a memory technology suitable for the memory in memory module 135). As used herein, the “memory interface” of the memory die is an interface inherent to the technology of the memory die (e.g., in the case of DRAM, the memory interface may be word lines and bit lines). The memory module may also include the controller 137, which provides enhanced capabilities, as described further in detail below. Each memory module 135's controller 137 can be connected to the processing circuitry 115 via a cache coherence interface (e.g., via a CXL interface). The controller 137 can also facilitate data transfers (e.g., RDMA requests) between different servers 105, bypassing the processing circuitry 115. The ToR Ethernet switch 110 and network interface circuitry 125 may include RDMA interfaces to facilitate RDMA requests between CXL memory devices on different servers (e.g., the ToR Ethernet switch 110 and network interface circuitry 125 can provide hardware offloading or hardware acceleration via RDMA over Converged Ethernet (RoCE), Infiniband, and iWARP packets).

[0059] The CXL interconnect in the system may conform to a cache coherence protocol (such as the CXL 1.1 standard), or in some embodiments, to the CXL 2.0 standard, a future version of CXL, or any other suitable protocol (e.g., a cache coherence protocol). The memory module 135 may be directly connected to the processing circuitry 115 as shown, and the top-of-rack Ethernet switch 110 may be used to scale the system to a larger size (e.g., with a larger number of servers 105).

[0060] In some embodiments, such as Figure 1AAs shown, each server can be populated with multiple directly-attached CXL memory modules 135. Each memory module 135 can expose a set of base address registers (BARs) to the host's basic input / output system (BIOS) as a memory range. One or more of the memory modules 135 may include firmware to transparently manage their memory space behind the host OS mapping. Each memory module 135 may include one or a combination of memory technologies, including, but not limited to, dynamic random access memory (DRAM), NAND flash memory, high-bandwidth memory (HBM), and low-power double data rate synchronous dynamic random access memory (LPDDR SDRAM), and may also include a cache controller or a separate, separate controller for different technology memory devices (for memory modules 135 that combine several memory devices of different technologies). Each memory module 135 may include different interface widths (x4-x16) and may be constructed according to any of a variety of relevant form factors (e.g., U.2, M.2, half-height half-length (HHHL), full-height half-length (FHHL), E1.S, E1.L, E3.S, and E3.H).

[0061] In some embodiments, as described above, the enhanced capability CXL switch 130 includes an FPGA (or ASIC) controller 137 and provides additional features beyond the switching of CXL packets. The controller 137 of the enhanced capability CXL switch 130 can also serve as a management device for the memory module 135 and assists in host control plane processing, and it enables rich control semantics and statistics. The controller 137 may include additional "backdoor" (e.g., 100 Gigabit Ethernet (GbE)) network interface circuitry 125. In some embodiments, the controller 137 is presented as a Type 2 CXL device to the processing circuitry 115, enabling a cache invalidation instruction to be issued to the processing circuitry 115 upon receiving a remote write request. In some embodiments, DDIO technology is enabled, and remote data is first fetched to the last-level cache (LLC) of the processing circuitry and then later (from the cache) written to the memory module 135. As used herein, a “Type 2” CXL device is a CXL device capable of initiating transactions and implementing optional consistent caching and host-managed device memory, and the applicable transaction types include all CXL.cache and all CXL.memory transactions.

[0062] As described above, one or more of the memory modules 135 may include persistent memory or a "persistent storage device" (i.e., a storage device whose data is not lost when external power is disconnected). If the memory module 135 is presented as a persistent device, the controller 137 of the memory module 135 may manage a persistent domain (e.g., it may store data identified by the processing circuitry 115 as requiring persistent storage in the persistent storage device (e.g., as a result of an application calling a corresponding operating system function)). In such an embodiment, a software API may flush cached data to the persistent storage device.

[0063] In some embodiments, direct memory transfer from network interface circuitry 125 to memory module 135 is enabled. Such a transfer can be a one-way transfer to remote memory for fast communication in a distributed system. In such embodiments, memory module 135 may expose hardware details to network interface circuitry 125 in the system to enable faster RDMA transfers. In such systems, two scenarios can occur depending on whether data direct I / O (DDIO) of processing circuitry 115 is enabled or disabled. DDIO can enable direct communication between the Ethernet controller or Ethernet adapter and the cache of processing circuitry 115. If DDIO of processing circuitry 115 is enabled, the destination of the transfer can be the last-level cache of the processing circuitry, from which data can then be automatically flushed to memory module 135. If DDIO of processing circuitry 115 is disabled, memory module 135 can operate in device bias mode to force the destination memory module 135 (without DDIO) to receive direct access. Such RDMA transfers can be implemented using a network interface circuit 125 with RDMA capabilities, including a host channel adapter (HCA), buffers, and other processing, which can bypass target memory buffer transfers that may exist in other RDMA transfer modes. For example, in such embodiments, the use of bounce buffers (e.g., buffers in remote servers when the final destination in memory is in an address range not supported by the RDMA protocol) can be avoided. In some embodiments, RDMA uses an alternative physical media option besides Ethernet (e.g., for use with switches configured to handle other network protocols). Examples of server-to-server connections that can enable RDMA include (but are not limited to) Infiniband, RDMA over Converged Ethernet (RoCE) (using Ethernet User Datagram Protocol (UDP)), and iWARP (using Transmission Control Protocol / Internet Protocol (TCP / IP)).

[0064] Figure 1B Showing with Figure 1AA similar system exists, in which processing circuitry 115 is connected to network interface circuitry 125 via memory module 135. Memory module 135 and network interface circuitry 125 reside on expansion slot adapters 140. Each expansion slot adapter 140 can be inserted into an expansion slot 145 (e.g., an M.2 connector) on the motherboard of server 105. Thus, the server can be any suitable (e.g., industry-standard) server modified by installing expansion slot adapters 140 into expansion slots 145. In such an embodiment, (i) each network interface circuit 125 may be integrated into a corresponding memory module in the memory module 135, or (ii) each network interface circuit 125 may have a PCIe interface (the network interface circuit 125 may be a PCIe endpoint (i.e., a PCIe slave device)) such that the processing circuit 115 to which the network interface circuit 125 is connected (which may operate as a PCIe master device or a “root port”) can communicate with the network interface circuit 125 via a root port to endpoint PCIe connection, and the controller 137 of the memory module 135 can communicate with the network interface circuit 125 via a point-to-point PCIe connection.

[0065] According to embodiments of the present invention, a system is provided, the system comprising: a first server including stored program processing circuitry, a first network interface circuitry, and a first memory module, wherein the first memory module includes a first memory die and a controller, the controller being connected to the first memory die via a memory interface, and the controller being connected to the stored program processing circuitry via a cache coherence interface and connected to the first network interface circuitry. In some embodiments, the first memory module further includes a second memory die, the first memory die including volatile memory, and the second memory die including persistent memory. In some embodiments, the persistent memory includes NAND flash memory. In some embodiments, the controller is configured to provide a flash translation layer for the persistent memory. In some embodiments, the cache coherence interface includes a Fast Compute Link (CXL) interface. In some embodiments, the first server includes an expansion slot adapter connected to an expansion slot of the first server, the expansion slot adapter including the first memory module and the first network interface circuitry. In some embodiments, the controller of the first memory module is connected to the stored program processing circuitry via an expansion slot. In some embodiments, the expansion slot includes an M.2 slot. In some embodiments, the controller of the first memory module is connected to the first network interface circuitry via a Point-to-Point Peripheral Component Interconnect Fast (PCIe) connection. In some embodiments, the system further includes a second server and a network switch connected to the first server and the second server. In some embodiments, the network switch includes a top-of-rack (ToR) Ethernet switch. In some embodiments, the controller of the first memory module is configured to receive a Remote Direct Memory Access (RDMA) request and send an RDMA response. In some embodiments, the controller of the first memory module is configured to receive a Remote Direct Memory Access (RDMA) request through the network switch and through a first network interface circuit, and to send an RDMA response through the network switch and through the first network interface circuit. In some embodiments, the controller of the first memory module is configured to: receive data from a second server; store data in the first memory module; and send a command to the stored program processing circuit for invalidating cache lines. In some embodiments, the controller of the first memory module includes a field-programmable gate array (FPGA) or an application-specific integrated circuit (ASIC). According to embodiments of the present invention, a method for performing Remote Direct Memory Access in a computing system is provided, the computing system including a first server and a second server, the first server including stored program processing circuitry, network interface circuitry, and a first memory module including a controller, the method including: receiving a Remote Direct Memory Access (RDMA) request by the controller of the first memory module; and sending an RDMA response by the controller of the first memory module.In some embodiments, the computing system further includes an Ethernet switch connected to a first server and a second server, and the step of receiving an RDMA request includes receiving the RDMA request through the Ethernet switch. In some embodiments, the method further includes: receiving a read command for a first memory address from a stored-program processing circuit by a controller of the first memory module; converting the first memory address to a second memory address by the controller of the first memory module; and retrieving data from the first memory module at the second memory address by the controller of the first memory module. In some embodiments, the method further includes: receiving data by the controller of the first memory module; storing the data in the first memory module by the controller of the first memory module; and sending a command to the stored-program processing circuit to invalidate a cache line by the controller of the first memory module. According to an embodiment of the present invention, a system is provided, the system including: a first server including a stored-program processing circuit, a first network interface circuit, and a first memory module, wherein the first memory module includes a first memory die and a controller device, the controller device being connected to the first memory die via a memory interface, the controller device being connected to the stored-program processing circuit via a cache coherence interface, and being connected to the first network interface circuit.

[0066] Reference Figure 1C In some embodiments, the server system includes multiple servers 105 connected together via a top-of-rack (ToR) Ethernet switch 110. Each server includes one or more processing circuits 115, each processing circuit 115 connected to (i) system memory 120 (e.g., DDR4 memory), (ii) one or more network interface circuits 125, and (iii) an enhancement capability CXL switch 130. The enhancement capability CXL switch 130 may be connected to multiple memory modules 135. That is, Figure 1C The system includes a first server 105, which includes a stored program processing circuit 115, a network interface circuit 125, a cache coherence switch 130, and a first memory module 135. Figure 1C In the system, the first memory module 135 is connected to the cache coherence switch 130, the cache coherence switch 130 is connected to the network interface circuit 125, and the stored program processing circuit 115 is connected to the cache coherence switch 130.

[0067] Memory modules 135 may be grouped by type, form factor, or technology type (e.g., DDR4, DRAM, LDPPR, High Bandwidth Memory (HBM), or NAND flash memory or other persistent storage devices (e.g., solid-state drives containing NAND flash memory)). Each memory module may have a CXL interface and include interface circuitry for translating between CXL packets and signals suitable for the memory in memory module 135. In some embodiments, alternatively, these interface circuitries are in an enhanced capability CXL switch 130, and each of the memory modules 135 has an interface that serves as a local interface to the memory in memory module 135. In some embodiments, the enhanced capability CXL switch 130 is integrated into the memory module 135 (e.g., integrated in an M.2 form factor package along with other components of the memory module 135, or integrated into a single integrated circuit along with other components of the memory module 135).

[0068] The ToR Ethernet switch 110 may include interface hardware to facilitate RDMA requests between aggregated memory devices on different servers. The enhanced capability CXL switch 130 may include one or more circuits (e.g., FPGAs or ASICs) to (i) route data to different memory types based on workload, (ii) virtualize host addresses to device addresses, and / or (iii) facilitate RDMA requests between different servers, thereby bypassing the processing circuitry 115.

[0069] Memory modules 135 may reside in an expansion enclosure (e.g., in the same rack as the motherboard enclosure housing the memory modules 135) that may include a predetermined number (e.g., more than 20 or more than 100) of memory modules 135, each memory module 135 being inserted into a suitable connector. Modules may have an M.2 form factor, and connectors may be M.2 connectors. In some embodiments, connections between servers are via different networks other than Ethernet (e.g., they may be wireless connections such as WiFi or 5G connections). Each processing circuit may be an x86 processor or another processor (e.g., an ARM processor or a GPU). The PCIe link instantiating the CXL link may be PCIe 5.0 or another version (e.g., an earlier version or a later (e.g., future) version such as PCIe 6.0). In some embodiments, different cache coherence protocols are used in the system to replace or supplement CXL, and different cache coherence switches may be used to replace or supplement the enhanced capability CXL switch 130. Such a cache coherence protocol can be another standard protocol or a cache coherence variant of a standard protocol (in a similar way to how CXL is a variant of PCIe 5.0). Examples of standard protocols include, but are not limited to, Non-Volatile Dual In-line Memory Module (Version P) (NVDIMM-P), Cache Coherent Interconnect for Accelerators (CCIX), and Open Coherent Accelerator Processor Interface (OpenCAPI).

[0070] System memory 120 may include, for example, DDR4 memory, DRAM, HBM, or LDPPR memory. Memory module 135 may be partitioned or include a cache controller to handle various memory types. Memory module 135 may have different form factors, examples of which include, but are not limited to, HHHL, FHHL, M.2, U.2, mezzanine card, daughter card, E1.S, E1.1, E3.1, and E3.S.

[0071] In some embodiments, the system implements an aggregated architecture comprising multiple servers, wherein each server is aggregated with multiple CXL-attached memory modules 135. Each of the memory modules 135 may contain multiple partitions, which may be individually exposed as memory devices to multiple processing circuits 115. Each input port of the enhanced capability CXL switch 130 may independently access multiple output ports of the enhanced capability CXL switch 130 and the memory modules 135 connected thereto. As used herein, an "input port" or "upstream port" of the enhanced capability CXL switch 130 is a port connected to (or adapted to connect to) a PCIe root port, and an "output port" or "downstream port" of the enhanced capability CXL switch 130 is a port connected to (or adapted to connect to) a PCIe endpoint. Figure 1A In the case of this embodiment, each memory module 135 may expose a set of base address registers (BARs) to the host BIOS as a memory range. One or more of the memory modules 135 may include firmware to transparently manage their memory space behind the host OS mapping.

[0072] In some embodiments, as described above, the Enhanced Capability CXL Switch 130 includes an FPGA (or ASIC) controller 137 and provides additional features beyond the exchange of CXL packets. For example, it can (as described above) virtualize the memory module 135 (i.e., operate as a translation layer) to translate between circuit-side addresses (or “processor-side” addresses, i.e., addresses included in memory read and write commands issued by the processing circuitry 115) and memory-side addresses (i.e., addresses used by the Enhanced Capability CXL Switch 130 to address storage locations within the memory module 135), thereby masking the physical addresses of the memory module 135 and presenting a virtual aggregation of memory. The controller 137 of the Enhanced Capability CXL Switch 130 can also serve as a management device for the memory module 135 and facilitate host control plane processing. The controller 137 can transparently move data without the involvement of the processing circuitry 115 and update the memory mapping (or “address translation table”) accordingly, ensuring that subsequent accesses function as expected. Controller 137 may include a switch management device that (i) can appropriately bind and unbind upstream and downstream connections during operation and (iii) can implement rich control semantics and statistics associated with data transfers to and from memory module 135. Controller 137 may include additional "backdoor" 100GbE or other network interface circuitry 125 (in addition to the network interface for connecting to the host) for connecting to other servers 105 or other networked devices. In some embodiments, controller 137 is presented as a Type 2 device to processing circuitry 115, enabling the issuance of a cache invalidation instruction to processing circuitry 115 upon receiving a remote write request. In some embodiments, DDIO technology is enabled, and remote data is first fetched to the last-level cache (LLC) of processing circuitry 115 and then written to memory module 135 (from the cache).

[0073] As described above, one or more of the memory modules 135 may include persistent storage devices. If the memory module 135 is presented as a persistent device, the controller 137 of the enhanced capability CXL switch 130 can manage persistent domains (e.g., it can store data identified as requiring persistent storage by the processing circuitry 115 (e.g., through the use of corresponding operating system functions) in the persistent storage device). In such embodiments, a software API can cache and flush data to the persistent storage device.

[0074] In some embodiments, it can be similar to the above for... Figure 1A and Figure 1B The embodiments described herein perform direct memory transfers to memory module 135 in a manner that is executed by the controller of memory module 135, wherein the operations performed by the controller of enhanced capability CXL switch 130 are performed by the controller 137.

[0075] As described above, in some embodiments, the memory modules 135 are organized into groups (e.g., one group with memory density, another group with HBM congestion, another group with limited density and performance, and yet another group with high capacity). Such groups may have different form factors or be based on different technologies. The controller 137 of the enhanced capability CXL switch 130 can intelligently route data and commands based on factors such as workload, tags, or Quality of Service (QoS). For read requests, routing based on such factors may not be necessary.

[0076] The controller 137 of the enhanced capability CXL switch 130 can also (as described above) virtualize the processing circuit-side address and memory-side address, enabling the controller 137 of the enhanced capability CXL switch 130 to determine where data will be stored. The controller 137 of the enhanced capability CXL switch 130 can make such a determination based on information or instructions that can be received from the processing circuitry 115. For example, the operating system may provide memory allocation features that allow an application to specify whether to allocate low-latency storage, high-bandwidth storage, or persistent storage. Such a request initiated by the application can then be taken into consideration by the controller 137 of the enhanced capability CXL switch 130 when determining where (e.g., in which memory module 135) memory should be allocated. For example, a high-bandwidth storage device requested by the application can be allocated in a memory module 135 containing HBM, a data persistence storage device requested by the application can be allocated in a memory module 135 containing NAND flash memory, and other storage (not requested by the application) can be stored on a memory module 135 containing relatively inexpensive DRAM. In some embodiments, the controller 137 of the enhanced capability CXL switch 130 may determine where to store specific data based on network usage patterns. For example, the controller 137 of the enhanced capability CXL switch 130 may determine, by monitoring usage patterns, that data in a specific range of physical addresses is accessed more frequently than other data. The controller 137 of the enhanced capability CXL switch 130 may then copy this data to the memory module 135 containing the HBM and modify its address translation table so that the data in the new location is stored in the same range of virtual addresses. In some embodiments, one or more of the memory modules 135 include flash memory (e.g., NAND flash memory), and the controller 137 of the enhanced capability CXL switch 130 implements a flash translation layer for this flash memory. The flash translation layer may support overwriting of processor-side memory locations (by moving data to different locations and marking the previous location of the data as invalid), and it may perform garbage collection (e.g., erasing the block after moving any valid data in the block to another block when the portion of data in the block marked as invalid exceeds a threshold).

[0077] In some embodiments, the controller 137 of the enhanced capability CXL switch 130 can facilitate physical function (PF) to PF transfers. For example, if one of the processing circuits 115 needs to move data from one physical address to another physical address (which may have the same virtual address; this fact does not need to affect the operation of the processing circuit 115), or if the processing circuit 115 needs to move data between two virtual addresses (the processing circuit 115 will need to have both virtual addresses), the controller 137 of the enhanced capability CXL switch 130 can supervise the transfer without involving the processing circuit 115. For example, the processing circuit 115 may send a CXL request and may send data from one memory module 135 to another memory module 135 after the enhanced capability CXL switch 130 (e.g., data may be copied from one memory module 135 to another memory module 135) without going to the processing circuit 115. In this scenario, because the processing circuit 115 initiated the CXL request, the processing circuit 115 may need to flush its cache to ensure consistency. If an alternative Type 2 memory device (e.g., one of memory modules 135, or an accelerator that may also be connected to the CXL switch) initiates a CXL request and the switch is not virtualized, the Type 2 memory device may send a message to the processing circuitry 115 to invalidate the cache.

[0078] In some embodiments, the controller 137 of the enhanced capability CXL switch 130 can facilitate RDMA requests between servers. A remote server 105 can initiate such an RDMA request, which can be sent via the ToR Ethernet switch 110 and reach the enhanced capability CXL switch 130 (“local server”) in the server 105 responding to the RDMA request. The enhanced capability CXL switch 130 can be configured to receive such RDMA requests, and it can treat a set of memory modules 135 in the receiving server 105 (i.e., the server receiving the RDMA request) as its own memory space. In the local server, the enhanced capability CXL switch 130 can receive the RDMA request as a direct RDMA request (i.e., an RDMA request not routed through the processing circuitry 115 in the local server), and it can send a direct response to the RDMA request (i.e., it can send a response without routing through the processing circuitry 115 in the local server). In the remote server, responses (e.g., data sent by the local server) can be received by the enhanced capability CXL switch 130 of the remote server and stored in the memory module 135 of the remote server, without being routed through the processing circuitry 115 in the remote server.

[0079] Figure 1D Showing with Figure 1CA system similar to the one described above, in which processing circuitry 115 is connected to network interface circuitry 125 via an enhanced capability CXL switch 130. The enhanced capability CXL switch 130, memory module 135, and network interface circuitry 125 reside on an expansion slot adapter 140. The expansion slot adapter 140 may be a board or module that inserts into an expansion slot (e.g., PCIe connector 145) on the motherboard of server 105. Therefore, the server can be any suitable server that can be modified simply by installing the expansion slot adapter 140 in the PCIe connector 145. The memory module 135 may be mounted in a connector (e.g., an M.2 connector) on the expansion slot adapter 140. In such an embodiment, (i) network interface circuitry 125 may be integrated into the enhanced capability CXL switch 130, or (ii) each network interface circuitry 125 may have a PCIe interface (the network interface circuitry 125 may be a PCIe endpoint) such that the processing circuitry 115 to which it is connected communicates with the network interface circuitry 125 via a root port-to-endpoint PCIe connection. The controller 137 of the enhanced capability CXL switch 130 (which may have PCIe input ports connected to the processing circuitry 115 and to the network interface circuitry 125) can communicate with the network interface circuitry 125 via a point-to-point PCIe connection.

[0080] According to embodiments of the present invention, a system is provided, the system comprising: a first server including stored program processing circuitry, network interface circuitry, a cache coherence switch, and a first memory module, wherein the first memory module is connected to the cache coherence switch, the cache coherence switch is connected to the network interface circuitry, and the stored program processing circuitry is connected to the cache coherence switch. In some embodiments, the system further comprises a second memory module connected to the cache coherence switch, wherein the first memory module includes volatile memory, and the second memory module includes persistent memory. In some embodiments, the cache coherence switch is configured to virtualize the first memory module and the second memory module. In some embodiments, the first memory module includes flash memory, and the cache coherence switch is configured to provide a flash translation layer for the flash memory. In some embodiments, the cache coherence switch is configured to monitor the access frequency of a first memory location in the first memory module; determine that the access frequency exceeds a first threshold; and copy the contents of the first memory location to a second memory location, the second memory location being in the second memory module. In some embodiments, the second memory module includes high-bandwidth memory (HBM).

[0081] In some embodiments, the cache coherence switch is configured to maintain a table for mapping processor-side addresses to memory-side addresses. In some embodiments, the system further includes a second server and a network switch connected to the first and second servers. In some embodiments, the network switch includes a top-of-rack (ToR) Ethernet switch. In some embodiments, the cache coherence switch is configured to receive Remote Direct Memory Access (RDMA) requests and send RDMA responses. In some embodiments, the cache coherence switch is configured to receive RDMA requests via the ToR Ethernet switch and via network interface circuitry, and to send RDMA responses via the ToR Ethernet switch and via network interface circuitry. In some embodiments, the cache coherence switch is configured to support the Fast Compute Link (CXL) protocol. In some embodiments, the first server includes an expansion slot adapter connected to an expansion slot of the first server, the expansion slot adapter including a cache coherence switch and a memory module slot, the first memory module being connected to the cache coherence switch via the memory module slot. In some embodiments, the memory module slot includes an M.2 slot. In some embodiments, the network interface circuitry is on the expansion slot adapter. According to embodiments of the present invention, a method for performing Remote Direct Memory Access (RDMA) in a computing system is provided. The computing system includes a first server and a second server. The first server includes a stored-program processing circuit, a network interface circuit, a cache coherence switch, and a first memory module. The method includes receiving a RDMA request by the cache coherence switch and sending an RDMA response by the cache coherence switch. In some embodiments, the computing system further includes an Ethernet switch, and the step of receiving the RDMA request includes receiving the RDMA request via the Ethernet switch. In some embodiments, the method further includes receiving a read command for a first memory address from the stored-program processing circuit by the cache coherence switch, converting the first memory address to a second memory address by the cache coherence switch, and retrieving data from the first memory module at the second memory address by the cache coherence switch. In some embodiments, the method further includes receiving data by the cache coherence switch, storing the data in the first memory module by the cache coherence switch, and sending a command to the stored-program processing circuit to invalidate a cache line by the cache coherence switch.According to an embodiment of the present invention, a system is provided, the system comprising: a first server, including a stored program processing circuit, a network interface circuit, a cache coherence switching device, and a first memory module, wherein the first memory module is connected to the cache coherence switching device, the cache coherence switching device is connected to the network interface circuit, and the stored program processing circuit is connected to the cache coherence switching device.

[0082] Figure 1E An embodiment of a plurality of servers 105 connected to a ToR server link switch 112 is illustrated. As shown, the ToR server link switch 112 may be a PCIe 5.0 CXL switch (also known as a ToRPCIe5 switch) with PCIe capability. The server link switch 112 may include an FPGA or ASIC and may provide superior performance (in terms of throughput and latency) compared to Ethernet switches. Each of the servers 105 may include a plurality of memory modules 135 connected to the server link switch 112 via an enhanced capability CXL switch 130 and via a plurality of PCIe connectors. As shown, each of the servers 105 may also include one or more processing circuits 115 and system memory 120. As discussed in further detail below, the server link switch 112 may operate as a master device, and each of the enhanced capability CXL switches 130 may operate as a slave device.

[0083] exist Figure 1E In this embodiment, the server link switch 112 can group or batch multiple cache requests received from different servers 105, and it can group packets to reduce control overhead. The enhanced capability CXL switch 130 may include a controller (e.g., from an FPGA or an ASIC) to (i) route data to different memory types based on workload, (ii) virtualize processor-side addresses to memory-side addresses, and (iii) facilitate consistency requests between different servers 105, thereby bypassing the processing circuitry 115. Figure 1E The system shown may be based on CXL 2.0, may include distributed shared memory within a rack, and may use a ToR server to link switch 112 for local connection to remote nodes.

[0084] The ToR server link switch 112 may have additional network connectivity for connecting to other servers or clients (e.g., an Ethernet connection as shown, or another type of connectivity (e.g., a wireless connection, such as a WiFi connection or a 5G connection)). The server link switch 112 and the enhanced capability CXL switch 130 may each include a controller, which may be or include processing circuitry such as an ARM processor. The PCIe interface may conform to the PCIe 5.0 standard or an earlier or future version of the PCIe standard, or may use an interface conforming to a different standard (e.g., NVDIMM-P, CCIX, or OpenCAPI) instead of the PCIe interface. The memory module 135 may include various memory types, including DDR4 DRAM, HBM, LDPPR, NAND flash memory, or solid-state drives (SSDs). The memory module 135 may be partitioned or contain a cache controller to handle multiple memory types, and they may have different form factors (e.g., HHHL, FHHL, M.2, U.2, mezzanine card, daughter card, E1.S, E1.1, E3.1, or E3.S).

[0085] exist Figure 1EIn this embodiment, the enhanced capability CXL switch 130 enables one-to-many and many-to-one switching, and it implements fine-grained load-to-store interfaces at the microchip (64-byte) level. Each server may have aggregated memory devices, each device being partitioned into multiple logical devices, each with its own corresponding LD-ID. The ToR switch 112 (which may be referred to as a “server-linked switch”) implements one-to-many functionality, and the enhanced capability CXL switch 130 in server 105 implements many-to-one functionality. The server-linked switch 112 may be a PCIe switch or a CXL switch, or both. In such a system, the requesting party may be the processing circuitry 115 of multiple servers 105, and the responding party may be a plurality of aggregated memory modules 135. The hierarchical structure of the two switches (as described above, the master switch is the server-linked switch 112, and the slave switch is the enhanced capability CXL switch 130) enables any communication. Each memory module 135 may have a physical function (PF) and up to 16 isolated logical devices. In some embodiments, the number of logical devices (e.g., the number of partitions) may be limited (e.g., limited to 16), and a control partition (which may be used to control the physical functions of the devices) may also exist. Each of the memory modules 135 may be a Type 2 device with CXL.cache, CXL.memory, and CXL.io, as well as an Address Translation Service (ATS) implementation, to process cache line copies that can be stored by the processing circuitry 115. The enhanced capability CXL switch 130 and the structure manager can control the discovery of the memory modules 135 and (i) perform device discovery and virtual CXL software creation, and (ii) bind virtual ports to physical ports. Figures 1A to 1D In one embodiment, the structure manager can operate via an SMBus sideband connection. The interface to the memory module 135 is configurable and can be an Intelligent Platform Management Interface (IPMI) or a Redfish-compliant interface (and may also provide additional features not required by the standard).

[0086] As described above, some embodiments implement a hierarchical architecture where the master controller (which may be implemented as an FPGA or ASIC) is part of the server link switch 112, and the slave controllers are part of the enhanced capability CXL switch 130 to provide a load-store interface (i.e., an interface with cache line (e.g., 64-byte) granularity and operating within a consistency domain without software driver involvement). Such a load-store interface extends the consistency domain beyond a single server, CPU, or host and may involve an electrical or optical physical medium (e.g., an optical connection with electro-optical transceivers at both ends). In operation, the master controller (in the server link switch 112) boots up (or “reboots”) and configures all servers 105 on the rack. The master controller is visible on all hosts and can (i) discover each server and how many servers 105 and memory modules 135 exist in the server cluster, (ii) configure each server 105 independently, (iii) enable or disable some memory blocks on different servers based on, for example, rack configuration (e.g., enable or disable any memory module in memory module 135), (iv) control access (e.g., which server can control which other server), (v) implement flow control (e.g., since all host and device requests pass through the master device, data can be sent from one server to another, and flow control can be performed on the data), (vi) group or batch requests or packets (e.g., the master device receives multiple cache requests from different servers 105), and (vii) receive remote software updates, broadcast communications, etc. In batch mode, the server link switch 112 can receive multiple packets destined for the same server (e.g., destined for the first server) and send them together (i.e., without pauses between them) to the first server. For example, server link switch 112 can receive a first data packet from a second server and a second data packet from a third server, and send the first and second data packets together to the first server. Each of servers 105 can expose (i) an IPMI network interface, (ii) a System Event Log (SEL), and (iii) a Board Management Controller (BMC) to the master controller, enabling the master controller to measure performance, measure operational reliability, and reconfigure server 105.

[0087] In some embodiments, a software architecture that facilitates a highly available load-store interface is used. Such a software architecture provides reliability, replication, consistency, system consistency, hashing, caching, and persistence. The software architecture provides reliability (in systems with a large number of servers) by performing periodic hardware checks on CXL device components via IPMI. For example, server link switch 112 can query the status of storage server 150 via its IPMI interface (e.g., querying power status (whether the power supply to storage server 150 is functioning correctly), network status (whether the interface to server link switch 112 is functioning correctly), and error checking status (whether there are error conditions in any subsystem of storage server 150)). Replication is provided by the software architecture because the master controller can replicate data stored in storage module 135 and maintain data consistency across replicas.

[0088] Because the master controller can be configured with different consistency levels, and the server link switch 112 can adjust the packet format according to the consistency level to be maintained, the software architecture provides consistency. For example, if eventual consistency is being maintained, the server link switch 112 can reorder requests, while to maintain strict consistency, the server link switch 112 can maintain a scoreboard at the switch with precise timestamps for all requests. Because multiple processing circuits 115 can read from or write to the same memory address, the software architecture provides system consistency, and to maintain consistency, the master controller can be responsible for the home node of the address (using directory lookup) or broadcasting requests on the common bus.

[0089] Because the server-linked switch 112 and the enhanced CXL switch can maintain a virtual mapping of addresses, which can use a consistent hash (i.e., consistent hashing) with multiple hash functions to uniformly map data across all CXL devices across all nodes at startup (or adjust when a server goes down or appears), the software architecture provides hashing. Because the master controller can designate specific memory partitions (e.g., in memory module 135 including HBM or similar technologies) as caches (e.g., using write-through or write-back caches), the software architecture provides caching. Because the master and slave controllers can manage persistence domains and flushing, the software architecture provides persistence.

[0090] In some embodiments, the capabilities of the CXL switch are integrated into the controller of the memory module 135. In such embodiments, the server link switch 112 can still be used as a master device and has enhanced features as discussed elsewhere herein. The server link switch 112 can also manage other storage devices in the system and may have Ethernet connectivity (e.g., a 100GbE connection) for connecting to client machines, for example, those not part of the PCIe network formed by the server link switch 112.

[0091] In some embodiments, server link switch 112 has enhanced capabilities and also includes an integrated CXL controller. In other embodiments, server link switch 112 is merely a physical routing device, and each server 105 includes a master CXL controller. In such embodiments, master devices across different servers can negotiate a master-slave architecture. The intelligent functions of (i) the enhanced capability CXL switch 130 and (ii) the server link switch 112 can be implemented in one or more FPGAs, one or more ASICs, one or more ARM processors, or one or more SSD devices with computing power. Server link switch 112 can perform flow control, for example, by reordering independent requests. In some embodiments, RDMA is optional because the interface is load-store, but intermediate RDMA requests using PCIe physical media (instead of 100GbE) may exist. In such embodiments, a remote host can initiate an RDMA request that can be sent to enhanced capability CXL switch 130 via server link switch 112. Server link switch 112 and enhanced CXL switch 130 can prioritize RDMA 4KB requests or CXL chip (64-byte) requests.

[0092] like Figure 1C and Figure 1D In one embodiment, the Enhanced Capability CXL switch 130 can be configured to receive such RDMA requests, and it can treat a set of memory modules 135 in the receiving server 105 (i.e., the server receiving the RDMA request) as its own memory space. Furthermore, the Enhanced Capability CXL switch 130 can be virtualized across the processing circuitry 115, and RDMA requests can be initiated on remote Enhanced Capability CXL switches 130 to move data back and forth between servers 105 without involving the processing circuitry 115.

[0093] Figure 1F Showing something similar Figure 1E In this system, processing circuitry 115 is connected to network interface circuitry (e.g., a PCIe 5 connector) 125 via an enhanced capability CXL switch 130. Figure 1D In the embodiments, Figure 1F In this configuration, the enhanced capability CXL switch 130, memory module 135, and network interface circuitry 125 are located on an expansion slot adapter 140. The expansion slot adapter 140 can be a board or module that inserts into an expansion slot (e.g., PCIe connector 145) on the motherboard of server 105. Therefore, the server can be any suitable server that can be modified simply by installing the expansion slot adapter 140 in the PCIe connector 145. The memory module 135 can be installed in a connector (e.g., an M.2 connector) on the expansion slot adapter 140. In such an embodiment, (i) the network interface circuit 125 may be integrated into the enhanced capability CXL switch 130, or (ii) each network interface circuit 125 may have a PCIe interface (the network interface circuit 125 may be a PCIe endpoint) such that the processing circuit 115 to which it is connected can communicate with the network interface circuit 125 via a root port to endpoint PCIe connection, and the controller 137 of the enhanced capability CXL switch 130 (which may have PCIe input ports connected to the processing circuit 115 and the network interface circuit 125) can communicate with the network interface circuit 125 via a point-to-point PCIe connection.

[0094] According to embodiments of the present invention, a system is provided, the system comprising: a first server including a stored program processing circuit, a cache coherent switch, and a first memory module; a second server; and a server link switch connected to the first server and the second server, wherein the first memory module is connected to the cache coherent switch, the cache coherent switch is connected to the server link switch, and the stored program processing circuit is connected to the cache coherent switch. In some embodiments, the server link switch includes a Fast Peripheral Component Interconnect (PCIe) switch.

[0095] In some embodiments, the server link switch includes a Fast Compute Link (CXL) switch. In some embodiments, the server link switch includes a Top-of-Rack (ToR) CXL switch. In some embodiments, the server link switch is configured to discover a first server. In some embodiments, the server link switch is configured to cause the first server to restart. In some embodiments, the server link switch is configured to cause a cache coherent switch to disable a first memory module. In some embodiments, the server link switch is configured to send data from a second server to a first server and perform flow control on the data. In some embodiments, the system further includes a third server connected to the server link switch, wherein the server link switch is configured to: receive a first data packet from the second server, receive a second data packet from the third server, and send the first data packet and the second data packet to the first server. In some embodiments, the system further includes a second memory module connected to the cache coherent switch, wherein the first memory module includes volatile memory and the second memory module includes persistent memory. In some embodiments, the cache coherent switch is configured to virtualize the first memory module and the second memory module. In some embodiments, the first memory module includes flash memory, and the cache coherent switch is configured to provide a flash translation layer for the flash memory. In some embodiments, the first server includes an expansion slot adapter connected to an expansion slot of the first server. The expansion slot adapter includes a cache coherent switch and a memory module slot, and the first memory module is connected to the cache coherent switch via the memory module slot. In some embodiments, the memory module slot includes an M.2 slot. In some embodiments, the cache coherent switch is connected to a server link switch via a connector, and the connector is on the expansion slot adapter. According to an embodiment of the present invention, a method for performing remote direct memory access in a computing system is provided. The computing system includes a first server, a second server, a third server, and a server link switch connected to the first server, the second server, and the third server. The first server includes stored program processing circuitry, a cache coherent switch, and a first memory module. The method includes: receiving a first data packet from a second server by the server link switch; receiving a second data packet from a third server by the server link switch; and sending the first data packet and the second data packet to the first server. In some embodiments, the method further includes: receiving a remote direct memory access (RDMA) request by the cache coherent switch and sending an RDMA response by the cache coherent switch. In some embodiments, the step of receiving an RDMA request includes receiving the RDMA request via the server link switch.In some embodiments, the method further includes: receiving a read command for a first memory address from a stored-program processing circuit by a cache-coherent switch; converting the first memory address to a second memory address by the cache-coherent switch; and retrieving data from the first memory module at the second memory address by the cache-coherent switch. According to an embodiment of the present invention, a system is provided, the system comprising: a first server including a stored-program processing circuit, a cache-coherent switching device, a first memory module, a second server, and a server link switch connected to the first server and the second server, wherein the first memory module is connected to the cache-coherent switching device, the cache-coherent switching device is connected to the server link switch, and the stored-program processing circuit is connected to the cache-coherent switching device.

[0096] Figure 1G An embodiment of a plurality of storage servers 150 connected to a ToR server link switch 112 is shown, as illustrated. The ToR server link switch 112 may be a PCIe 5.0 CXL switch (also known as a ToR PCIe 5 CXL switch). Figure 1E and Figure 1F In some embodiments, the server link switch 112 may include an FPGA or ASIC and can provide performance superior to that of an Ethernet switch (in terms of throughput and latency). Figure 1E and Figure 1F In one embodiment, the storage server 150 may include multiple storage modules 135 connected to the server link switch 112 via multiple PCIe connectors. Figure 1G In some embodiments, the processing circuitry 115 and system memory 120 may be absent, and the primary purpose of the memory server 150 may be to provide memory for use by other servers 105 with computing resources.

[0097] exist Figure 1G In this embodiment, the server link switch 112 can group or batch multiple cache requests received from different memory servers 150, and it can group packets to reduce control overhead. The enhanced capability CXL switch 130 may include composable hardware building blocks to (i) route data to different memory types based on workload and (ii) virtualize processor-side addresses (translating such addresses to memory-side addresses). Figure 1G The system shown may be based on CXL 2.0, which may include composable and decomposable shared memory within a rack, and may use a ToR server to link switch 112 to provide pooled (i.e., aggregated) memory to remote devices.

[0098] The ToR server link switch 112 may have additional network connectivity for connecting to other servers or clients (e.g., an Ethernet connection as shown, or another type of connectivity (e.g., a wireless connection, such as a WiFi connection or a 5G connection)). The server link switch 112 and the enhanced capability CXL switch 130 may each include a controller, which may be or include processing circuitry such as an ARM processor. The PCIe interface may conform to the PCIe 5.0 standard or an earlier or future version of the PCIe standard, or may use a different standard (e.g., NVDIMM-P, CCIX, or OpenCAPI) instead of PCIe. The memory module 135 may include various memory types, including DDR4 DRAM, HBM, LDPPR, NAND flash memory, and solid-state drives (SSDs). The memory module 135 may be partitioned or contain a cache controller to handle multiple memory types, and they may have different form factors (e.g., HHHL, FHHL, M.2, U.2, mezzanine card, daughter card, E1.S, E1.1, E3.1, or E3.S).

[0099] exist Figure 1G In this embodiment, the enhanced capability CXL switch 130 can implement one-to-many and many-to-one switching, and it can implement fine-grained load-memory interfaces at the microchip (64-byte) level. Each memory server 150 may have aggregated memory devices, each device being divided into multiple logical devices, each with a corresponding LD-ID. The enhanced capability CXL switch 130 may include a controller 137 (e.g., an ASIC or FPGA) and circuitry for device discovery, enumeration, partitioning, and presentation of physical address ranges (which may be separate from or part of such an ASIC or FPGA). Each memory module 135 may have a physical function (PF) and up to 16 isolated logical devices. In some embodiments, the number of logical devices (e.g., the number of partitions) may be limited (e.g., limited to 16), and a control partition (which may be for controlling the physical function of the devices) may also exist. Each of the memory modules 135 may be a Type 2 device with CXL.cache, CXL.memory, and CXL.io and an Address Translation Service (ATS) implementation to process cache line copies that can be stored by the processing circuitry 115.

[0100] The enhanced capability CXL switch 130 and the structure manager can control the discovery of the memory module 135, and (i) perform device discovery and virtual CXL software creation and (ii) bind virtual ports to physical ports. Figures 1A to 1DIn one embodiment, the structure manager can be operated via a connection on the SMBus sideband. The interface to the memory module 135 enables configurability; the interface can be an Intelligent Platform Management Interface (IPMI) or a Redfish-compliant interface (which may also provide additional features not required by the standard).

[0101] against Figure 1G In some embodiments, building blocks may include (as described above) a CXL controller 137 implemented on an FPGA or ASIC that switches to enable (e.g., the aggregation of memory devices, SSDs, accelerators (GPUs, NICs), CXL and PCIe5 connectors, and firmware of memory module 135, thereby exposing device details to the operating system’s high-level configuration and power interface (ACPI) tables (such as heterogeneous memory attribute tables (HMAT) or static resource association tables (SRAT)).

[0102] In some embodiments, the system provides scalability. The system may provide the ability to bring CXL devices and other accelerators online or offline based on software configuration, and it may be able to group accelerators, memory, and storage device resources and allocate them to each memory server 150 in the rack. The system may hide the physical address space and provide transparent caching using faster devices such as HBM and SRAM.

[0103] exist Figure 1G In this embodiment, the controller 137 of the Enhanced Capability CXL switch 130 can (i) manage the memory module 135, (ii) integrate and control heterogeneous devices (such as NICs, SSDs, GPUs, DRAM), and (iii) enable dynamic reconfiguration of the storage of the memory devices via power gating. For example, the ToR server link switch 112 can disable the power supply of one of the memory modules 135 (i.e., turn off the power or reduce the power) by instructing the Enhanced Capability CXL switch 130 to disable the power supply of the memory module 135. The Enhanced Capability CXL switch 130 can then disable the power supply of the memory module 135 when instructed by the server link switch 112 to disable the power supply of the memory module. Such disabling can save power and improve the performance (e.g., throughput and latency) of other memory modules 135 in the memory server 150. Each remote server 105 can see a different logical view of the memory module 135 and its connections based on negotiation. The controller 137 of the enhanced CXL switch 130 is maintainable, enabling each remote server to maintain allocated resources and connections, and to perform memory compression or deduplication to save memory capacity (using configurable block sizes). Figure 1GThe disassembled rack can have its own BMC. It can also expose the IPMI network interface and System Event Log (SEL) to remote devices, enabling the master device (e.g., a remote server using storage provided by storage server 150) to measure performance and reliability during operation and reconfigure the disassembled rack. Figure 1G The disassembled rack can be similar to that used here. Figure 1E The embodiments described provide reliability, replication, consistency, system consistency, hashing, caching, and persistence, wherein, for example, multiple remote servers reading from or writing to the same memory address are provided with consistency, and each remote server is configured with a different consistency level. In some embodiments, the server-linked switch maintains eventual consistency between data stored on a first memory server and data stored on a second memory server. The server-linked switch 112 may maintain different consistency levels for different server pairs; for example, the server-linked switch may also maintain strict consistency, sequential consistency, causal consistency, or processor consistency levels between data stored on a first memory server and data stored on a third memory server. The system may employ communication in "local band" (server-linked switch 112) and "global band" (decomposed server) domains. Writes may be flushed to the "global band" to make new reads from other servers visible. The controller 137 of the enhanced capability CXL switch 130 may manage persistent domains and flushing individually for each remote server. For example, a cache-coherent switch can monitor the fullness of a first region of memory (volatile memory, operating as a cache), and when the fullness level exceeds a threshold, the cache-coherent switch can move data from the first region of memory to a second region of memory, which is in persistent memory. Flow control can be handled because priorities can be established between remote servers by the controller 137 of the enhanced capability CXL switch 130 to present different perceived latency and bandwidth.

[0104] According to embodiments of the present invention, a system is provided, the system comprising: a first memory server including a cache coherence switch and a first memory module; a second memory server; and a server link switch connected to the first memory server and the second memory server, wherein the first memory module is connected to the cache coherence switch, and the cache coherence switch is connected to the server link switch. In some embodiments, the server link switch is configured to disable power to the first memory module. In some embodiments, the server link switch is configured to disable power to the first memory module by instructing the cache coherence switch to disable power to the first memory module, and the cache coherence switch is configured to disable power to the first memory module when instructed to do so by the server link switch. In some embodiments, the cache coherence switch is configured to perform deduplication within the first memory module. In some embodiments, the cache coherence switch is configured to compress data and store the compressed data in the first memory module. In some embodiments, the server link switch is configured to query the status of the first memory server. In some embodiments, the server link switch is configured to query the status of the first memory server via an Intelligent Platform Management Interface (IPMI). In some embodiments, the step of querying the status includes querying a status selected from a group consisting of power status, network status, and error checking status. In some embodiments, the server link switch is configured to batch cache requests directed to a first memory server. In some embodiments, the system further includes a third memory server connected to the server link switch, wherein the server link switch is configured to maintain a consistency level selected from a group consisting of strict consistency, sequential consistency, causal consistency, and processor consistency between data stored on the first memory server and data stored on the third memory server. In some embodiments, the cache consistency switch is configured to: monitor the fullness of a first region of memory and move data from the first region of memory to a second region of memory, wherein the first region of memory is in volatile memory and the second region of memory is in persistent memory. In some embodiments, the server link switch includes a Fast Peripheral Component Interconnect (PCIe) switch. In some embodiments, the server link switch includes a Fast Compute Link (CXL) switch. In some embodiments, the server link switch includes a Top-of-Rack (ToR) CXL switch. In some embodiments, the server link switch is configured to send data from a second memory server to a first memory server and perform flow control on the data.In some embodiments, the system further includes a third memory server connected to a server link switch, wherein the server link switch is configured to: receive a first data packet from a second memory server, receive a second data packet from the third memory server, and send the first data packet and the second data packet to the first memory server. According to embodiments of the present invention, a method for performing remote direct memory access in a computing system is provided. The computing system includes a first memory server, a first server, a second server, and a server link switch connected to the first memory server, the first server, and the second server. The first memory server includes a cache coherence switch and a first memory module. The first server includes stored program processing circuitry, and the second server includes stored program processing circuitry. The method includes: receiving a first data packet from the first server via the server link switch; receiving a second data packet from the second server via the server link switch; and sending the first data packet and the second data packet to the first memory server. In some embodiments, the method further includes: compressing data via the cache coherence switch and storing the data in the first memory module. In some embodiments, the method further includes: querying the status of the first memory server via the server link switch. According to an embodiment of the present invention, a system is provided, the system comprising: a first memory server including a cache coherence switch and a first memory module; a second memory server; and a server link switching device connected to the first memory server and the second memory server, wherein the first memory module is connected to the cache coherence switch, and the cache coherence switch is connected to the server link switching device.

[0105] Figure 2 Figure 200 illustrates a representative system architecture according to a disclosed example embodiment, wherein aspects of the disclosed embodiments can be combined to enable communication and configuration. Figures 1A to 1GThe various servers described operate as management computing entities. In some embodiments, the disclosed system may include a management computing entity 202 that can be configured to operate in conjunction with multiple clusters. As shown, the clusters may include a type A pool cluster 204, a type B pool cluster 206, a type C pool cluster 208, and a type D pool cluster 210. In one embodiment, type A pool cluster 204 may include direct-attached memory (e.g., CXL memory), type B pool cluster 206 may include accelerators (e.g., CXL accelerators), type C pool cluster 208 may include pooled / distributed memory (e.g., CXL memory), and type D pool cluster 210 may include decomposed memory (e.g., CXL memory). Furthermore, each cluster may include, but is not limited to, plug-in modules 212 that may include computing elements 214 (such as processors (e.g., RISC-V-based processors) and / or programmable controllers (e.g., FPGA-based controllers)) and corresponding media 216.

[0106] In various embodiments, the management computing entity 202 may be configured to direct I / O and memory storage and retrieval operations to various clusters based on one or more predetermined parameters (e.g., parameters associated with the corresponding workloads processed by hosts or devices on a network communicating with the management computing entity 202).

[0107] In various embodiments, the management computing entity 202 may operate at the rack and / or cluster level, or at least partially within a given device (e.g., a device with cache coherence enabled) that is part of a given cluster architecture (e.g., pool A cluster 204, pool B cluster 206, pool C cluster 208, and pool D cluster 210). In various embodiments, a device within a given cluster architecture may perform a first portion of the management computing entity's operations, while another portion of the management computing entity's operations may be implemented on the rack and / or at the cluster level. In some embodiments, both portions of the operations may be performed in a coordinated manner (e.g., a device in the cluster sends a coordination message to and receives a coordination message from the management computing entity implemented on the rack and / or at the cluster level). In some embodiments, the first portion of the operations associated with a device in the cluster may include, but is not limited to, operations for determining current or future resource requirements of the device or cluster, announcing current or future resource availability of the device or cluster, synchronizing specific parameters associated with algorithms running at the device or cluster level, training one or more machine learning modules associated with the operation of the device or rack / cluster, recording corresponding data associated with routing workloads, combinations thereof, etc.

[0108] Figure 3A Another diagram 300 depicts a representative system architecture according to a disclosed example embodiment, wherein aspects of the disclosed embodiments can be combined to enable communication and configuration. Figures 1A to 1G The various servers described are operated by management computing entities. In some embodiments, management computing entity 302 may be combined with the above. Figure 2 The management computing entity 202 shown and described is similar but not necessarily the same. Furthermore, the management computing entity 202 may communicate with a Type A pool. In various embodiments, the Type A pool cluster 312 may include several servers. Furthermore, the Type A pool cluster 312 may feature a direct-attached cache-coherent (e.g., CXL) device, for example, that can be configured to operate using RCIEP. In another embodiment, the Type A pool cluster 312 may feature a cache-coherent protocol-based memory (such as CXL memory) to reduce any limitations on CPU pins. In one embodiment, the Type A pool cluster 312 may include a direct-attached device with various form factor options (e.g., E1, E3 form factors capable of conforming to the EDSFF standard and / or add-in card (AIC) form factors). In another embodiment, the disclosed system may include a switch 304 (such as a cache-coherent (e.g., CXL) based switch and / or a silicon photonics based switch). In one embodiment, the switch 304 may feature a top-of-rack (ToR) Ethernet-based switch capable of scaling the system to the rack level.

[0109] In various embodiments, such as Figure 3B As shown, the Type B pool cluster 314 may also include several servers. Furthermore, the Type B pool cluster 314 may utilize cache-coherent (e.g., CXL 2.0 based) switches and accelerators (hereinafter referred to as ACCs) capable of pooling within one of the servers. Additionally, the Type B pool cluster 314 may feature a workload-based switch (VCS) hierarchy capability based on a virtual cache-coherent protocol (e.g., the CXL protocol). Specifically, a VCS may be identified as part of a switch and a connectivity component behind a specific root port (e.g., a PCIe root port). In another embodiment, the disclosed system may include switch 306 (such as cache-coherent (e.g., CXL) switches and / or silicon photonics-based switches).

[0110] In various embodiments, such as Figure 3CAs shown, the C-type pool cluster 316 may also include several servers. Furthermore, the C-type pool cluster 316 may utilize a CXL 2.0 switch within one of the servers. Additionally, the C-type pool cluster 316 may utilize a PCIe-based architecture and / or a Gen-Z-based system to scale cache-coherent memory across servers. Furthermore, the C-type pool cluster 316 may introduce at least three coherent memory pools within the cluster: local DRAM, local CXL memory, and remote memory. In another embodiment, the disclosed system may include a switch 308 (such as a cache-coherent (e.g., CXL) based switch and / or a silicon photonics based switch).

[0111] In various embodiments, such as Figure 3D As shown, the D-type pool cluster 318 may also include several servers. In one embodiment, the D-type pool cluster 318 may include physically decomposed CXL memory. Furthermore, partitions may be assigned to each server, allowing for limited or no sharing across servers. In some embodiments, the D-type pool cluster 318 may initially be limited to a predetermined number (e.g., 16) of multiple logical device (MLD) partitions and hosts. Specifically, memory devices based on a Type 3 cache coherence protocol (e.g., CXL) may be partitioned to appear as multiple devices, each presenting a unique logical device ID. Furthermore, the D-type pool cluster 318 may use a PCIe-based architecture and / or a Gen-Z-based system to extend cache-coherent memory across servers. In another embodiment, the disclosed system may include a switch 310 (such as a cache-coherent (e.g., CXL) based switch and / or a silicon photonics based switch).

[0112] Figure 4 A characterizable combination according to the disclosed example embodiments is described. Figures 1A to 1GA diagram illustrating representative parameter tables for various aspects of the described servers, whereby the management computing entity configures various servers based on these parameter tables. Specifically, Table 400 shows various example parameters that can be considered by the disclosed system and specifically by the management computing entity described herein in various ways, to route portions of the workload to different clusters based on comparisons of the values ​​of these parameters (or similar parameters) for different pool cluster types. Specifically, Table 400 shows parameters 402 corresponding to the different cluster types shown in the columns (i.e., direct-attached 406 memory clusters (similar to type A pool clusters), pooled 408 memory clusters (similar to type B pool clusters), distributed 410 memory clusters (similar to type C pool clusters), and decomposed 412 memory clusters (similar to type D pool clusters)). Non-limiting examples of such parameters 402 include direct memory capacity, remote memory capacity (e.g., for cache coherence protocols such as CXL), remote memory capacity (e.g., per server), remote memory performance, total cost of ownership (TCO), total power (amortized), and total area (e.g., with an E1 form factor). In various embodiments, as further described below, the disclosed system may use machine learning algorithms associated with the management computing entity to determine at least a portion of the workload to be routed to different clusters. Although Figure 4 Some example parameters are shown, but the disclosed system can be configured to monitor any suitable parameters to route workloads or portions of workloads to different devices associated with the cluster. Furthermore, the management computing entity can perform such operations based on a variety of system parameters, including but not limited to round-trip time based on cache coherence protocols (e.g., CXL-based), determination of whether a device is host-biased or device-biased, switch hierarchy and / or host upstream port to device downstream port binding based on cache coherence protocols (e.g., CXL-based), switch fabrication manager configuration based on cache coherence protocols (e.g., CXL-based), protocol packets or physical media packets based on cache coherence protocols (e.g., CXL.IO or PCIe intermediate batch 4KB packets), network latency, memory technology based on cache coherence protocols (e.g., CXL-based) (e.g., memory type), combinations thereof, etc.

[0113] In various aspects, there may be a variety of hyperscale user workload requirements that may have specific characteristics that pose challenges to existing data center and server architectures. For example, such workloads may exhibit a variety of memory and I / O latency requirements, bandwidth demands, and may be resource-constrained in terms of computing and / or memory resources.

[0114] As mentioned, the disclosed systems may include cache coherence protocols such as CXL. Specifically, CXL enables other types of memory expansion and coherence accelerators to work alongside conventional storage and compute. However, in some respects, protocols such as those related to CXL may not describe the system design and / or microarchitecture required to meet the specific needs of an application. Furthermore, due to the numerous implementation challenges that distributed and large-scale CXL memory systems may present, various different types of CXL systems and associated microarchitectures may exist.

[0115] In some aspects, cache coherence protocols such as CXL enable memory expansion and heterogeneous computing architectures. Furthermore, cache coherence operations within a CPU-centric ecosystem enable accelerator and memory tier design and use cases. In various embodiments, the disclosed system may include a CXL device microarchitecture, which may include CXL Type 1, Type 2, and / or Type 3 memory device microarchitectures as further described herein. In another embodiment, the disclosed system may be configured to co-locate different memory types (e.g., volatile and non-volatile memory types) within a given device and communicate using a cache coherence protocol. In yet another embodiment, the disclosed system may reconfigure memory resources at startup. Optionally or additionally, the disclosed system may disable and / or enable specific memories based on predetermined parameters (e.g., bandwidth or memory usage parameters).

[0116] As mentioned, the disclosed system enables resource reconfiguration, for example, at startup of a cache-coherent compatible device. For instance, the disclosed system can be reconfigured to disable / enable a specific memory type (e.g., volatile or non-volatile memory) based on changed network parameters. In another embodiment, the disclosed system can be reconfigured to use hardware or software caching based on network parameters. In some embodiments, the parameters can be determined by an on-device profiling module that can determine the capabilities of the device (the device's capabilities may change over time based on usage, etc.). The parameters can be sent to another device, such as a host, that can use this information, for example, at runtime, to reconfigure the device. In another embodiment, the disclosed system can be used to partition a given interface, such as a PCIe interface, into dedicated interfaces (e.g., using a portion corresponding to multiple channels as a first interface, for example, in conjunction with a first memory type, and a second portion corresponding to a second number of channels as a second interface, for example, in conjunction with a second memory type). In another embodiment, the disclosed system can partition the device into different domains, such as via a CXL-based logical domain (LD-LD).

[0117] As mentioned, the disclosed systems may include Type 1, Type 2, and / or Type 3 CXL device microarchitectures. In various embodiments, a Type 1 CXL device may refer to a device that implements a coherent cache but may not include host-managed device memory. Furthermore, a Type 1 CXL device can extend PCIe protocol capabilities (e.g., atomic operation capabilities). In various embodiments, a Type 2 CXL device can implement both a coherent cache and host-managed device memory. Furthermore, a Type 2 CXL device can support applications including devices with attached high-bandwidth memory. Furthermore, applicable transaction types for such a Type 2 CXL device may include various CXL.cache and CXL.mem transactions. In various embodiments, a Type 3 CXL device may include host-managed device memory. Furthermore, in some embodiments, the disclosed systems can support applications for such devices, such as host-based memory expansion. Furthermore, a Type 3 CXL device may include applicable transaction types such as CXL.mem, memory read transactions, and memory write transactions.

[0118] In another embodiment, the disclosed system may include, but is not limited to, various architectural blocks. In some embodiments, the disclosed system may include, but is not limited to, the cache controller, CXL controller, intelligent memory controller, device coherence engine (DCOH) and interleaving decoder, error correction control (ECC) and security block, etc., which are further described herein.

[0119] In various embodiments, the DCOH and interleaving decoder can be used as device consistency bridges, simplified home agents, and consistency bridges on the device rather than the host. Furthermore, in some embodiments, a device (e.g., a Type 3 device) can be interleaved with other CXL devices, for example, within a given physical address range. In various embodiments, the disclosed system can perform memory interleaving at the master bridge, root port, or switch level. Additionally, the disclosed system can implement a management layer for performing interleaving (e.g., by assembling leaf nodes into interleaved sets, etc.).

[0120] In various embodiments, the ECC and security blocks may include various hardware and software blocks that implement secure (e.g., cryptographic) algorithms. In another embodiment, the ECC and security blocks include modules that configure a cryptographic engine (e.g., via the Advanced Cryptographic Standard Galois / Counter Mode (AES-GCM) for CXL). In yet another embodiment, the ECC and security blocks may include any other blocks that communicate directly or indirectly with the two blocks described above. In various embodiments, the ECC and security blocks can be used to protect transactions (both data and metadata) exchanged between two devices on a physical link. In the case of cryptographic security, the ECC and security blocks may implement symmetric encryption keys (e.g., a 256-bit key for AES-GCM). In various embodiments, the ECC and security blocks may operate according to the CXL.io and CXL.cachemem protocols. Specifically, the CXL.io path may use a PCIe-defined IDE, while CXL.cachemem may include additional updates from CXL 2.0.

[0121] In various embodiments, the ECC and security block can perform an ECC generation and verification method including one or more of the following methods. The ECC code can be generated by the disclosed system based on written data. The disclosed system can store both the written data and the ECC code. Furthermore, during a read operation, the disclosed system can read both the data and the ECC code. The disclosed system can then regenerate the ECC code from the received data and can compare the regenerated ECC code with the received ECC code. Therefore, if the disclosed system finds a match, it can determine that no error has occurred. However, if a mismatch exists, the disclosed system can correct the error. In various embodiments, the ECC and security block, for example, use double data rate (DDR) memory (such as DDR4 and DDR5) to implement various schemes for ECC (including side-band ECC). Specifically, the disclosed system can send the ECC code as side-band data along with the actual data to the memory. In another embodiment, since sideband ECC can be a more complex implementation utilizing low-power DDR (LPDDR), the ECC and security block can, for example, be implemented using LPDDR memory to achieve inline ECC. In various embodiments, the ECC and security block can, for example, be implemented using LPDDR5 memory to achieve link-ECC. Link-ECC provides additional protection against errors on the LPDDR5 link or channel. Furthermore, the ECC and security block can determine the ECC for write data and can transmit the ECC for specific bits along with the data. DRAM generates the ECC for received data, checks it against the received ECC data, and corrects any errors. This operation can also be implemented using a role opposite to that of DRAM, ECC, and security blocks for read data.

[0122] In various embodiments, the disclosed system may include combining Figure 5 , Figure 7 , Figure 8 and Figure 9The CXL device microarchitecture is further described and may include reconfigurable architectures (e.g., between different memory types such as DDR, LPDDR5, Z-NAND, and combinations thereof). In some embodiments, the disclosed system may include a cache coherent device (e.g., a CXL device) that can use a first protocol for memory management (e.g., the CXL.mem protocol) and a second protocol for managing remote regions (e.g., the CXL.cache protocol). In one embodiment, the disclosed system enables the device to be configured at boot time using metadata (e.g., CXL metadata). In another embodiment, the disclosed system enables data and memory tier remapping while satisfying DRAM timing requirements. In one embodiment, the disclosed system provides acceleration and deduplication at the predetermined byte level (e.g., the 64-byte level).

[0123] In some embodiments, the disclosed system may include an intelligent controller. In one embodiment, the intelligent controller may include logic for tiering memory across different technologies. In another embodiment, the intelligent controller may include logic for partitioning PCIe interfaces into different domains, supporting multiple logical IDs for pooling, and other CXL features. The intelligent controller may include interface logic and a global host for communicating with a DRAM / NAND controller. The intelligent controller may include address mapping for remapping incoming read / write transactions. The intelligent controller may interface directly with the global memory controller and receive memory addresses and read / write transactions. The intelligent controller may include a cache addressing module that identifies addresses and translates them into indexes / tags and other cache fields. The intelligent controller may be optimized, including demultiplexing the same cache set to different DRAM banks to minimize lookup time. The intelligent controller may include a cache policy module that implements one or more alternatives, associative policies, bank policies, replication, and placement to manage the movement and organization of data in the DRAM cache. The intelligent controller may also include a large write buffer to aggregate most writes to downstream NAND. The intelligent controller may include a transaction queue within the controller, wherein the transaction queue identifies individual memory transactions and pushes them to each channel request queue.

[0124] In some embodiments, the disclosed system may include one or more media controllers. In various embodiments, the media controller may include a per-channel module to efficiently schedule requests while satisfying all DRAM timing policies and power management. Furthermore, additional memory controller policies from a standard DDR controller may be implemented in association with the media controller(s).

[0125] In some embodiments, the apparatus including the media controller may include a flash memory (e.g., Z-NAND) controller, also referred to herein as a NAND controller. In some embodiments, the system or apparatus may communicate with the flash memory controller when it needs to read data from or write data to the flash memory. In some embodiments, the flash memory controller may use a technique called wear leveling to distribute writes as evenly as possible across all flash memory blocks in the storage device, so that each block can be written to its maximum lifetime. In another embodiment, the flash memory controller may also include a flash translation layer (FTL) (a layer below the file system) that maps host-side or file system logical block addresses to physical addresses in the flash memory (logical-to-physical mapping). In some embodiments, the physical location of the LBA may change dynamically as the flash memory controller implements wear leveling and other flash memory management algorithms (bad block management, read interference management, secure flash handling, etc.). Therefore, the disclosed system can differentiate the mapping units of the FTL so that the LBA is mapped based on blocks, pages, or subpages. Therefore, the disclosed system can implement finer mapping units to reduce flash memory wear and maximize the durability of flash-based storage media. In another embodiment, the flash memory controller can implement garbage collection, in which specific blocks can be prepared for use after they no longer have current data (e.g., old blocks). The data in these blocks is replaced with newly written blocks and thus queued for erasure, so that new data can be written into these blocks.

[0126] In some embodiments, the disclosed system may include a prefetch module that performs cache line prefetching and efficiently accesses the data stream from a DRAM cache after a data stream is detected. In various embodiments, cache prefetching can refer to a technique that improves execution performance by fetching instructions or data from their original storage in slower memory to faster local memory before the instructions or data are needed. In another embodiment, the prefetch module may fetch data or instructions into a cache. As mentioned, data prefetching fetches data before it is needed; however, accurate data prefetching can be more complex than instruction prefetching because data access patterns exhibit less regularity than instruction patterns. On the other hand, instruction prefetching fetches instructions before they need to be executed. In another embodiment, hardware-based prefetching may be performed by the prefetch module using a dedicated hardware mechanism that monitors the instruction or data stream requested by the executor, identifies the next few elements the program may need based on the stream, and prefetches them into a cache. In another embodiment, software-based prefetching can be performed by a prefetch module using a software mechanism, wherein additional prefetch instructions are inserted into the program. In some embodiments, the disclosed system may include a deduplication module that provides a model for looking up and modifying data by value. Therefore, the disclosed system ensures that only one copy of the data exists in the memory system and that all other addresses with the same data point to the same data.

[0127] Figure 5 The diagrams illustrate different configurations of memory devices according to exemplary embodiments of the present invention. Specifically, the disclosed systems may include different types of controllers and memory devices that can be configured to operate using cache coherence protocols such as CXL. Figure 501 illustrates a first microarchitecture configuration including an example DDR-based memory. Specifically, Figure 501 illustrates a configuration 502 of DRAM including DDR5506 (or DDR4 or any other suitable DDR memory) and DRAM controller 504. DRAM controller 504 may be configured to communicate with a processor 508 (e.g., an x86-based CPU) via a PCIe interface through CXL. In some embodiments, DRAM controller 504 may be configured to communicate with processor 508 via a predetermined number of channels (e.g., 16 channels). Processor 508 itself may be coupled to DRAM (e.g., DDR5 510 or any suitable memory). In various embodiments, such a microarchitecture configuration 502 may be configured to accelerate and / or adapt address decoding and / or interleaving mechanisms.

[0128] Figure 503 illustrates a second microarchitecture configuration 512 including example DDR and Z-NAND memory. Specifically, Figure 503 illustrates a configuration where the DRAM includes DDR5 520 (or DDR4 or any other suitable DDR memory) and a DRAM controller 514, which may also include cache controller functionality. Furthermore, Figure 503 illustrates a configuration 512 where flash memory (e.g., Z-NAND 518) may be present, which may have a corresponding controller (e.g., a solid-state device (SSD) controller 516). The DRAM controller 514 may be configured to communicate with the SSD controller 516 to coordinate I / O via any suitable protocol such as PCIe and / or CXL. Additionally, the DRAM controller 514 and / or the SSD controller 516 may communicate with the PCIe interface of the processor 522 (e.g., an x86-based CPU) via CXL. In some embodiments, the DRAM controller 514 and / or the SSD controller 516 may be configured to communicate with the processor 522 via a predetermined number of channels (e.g., 16 channels). The processor 522 itself may be coupled to DRAM (e.g., DDR5 524 or any suitable memory). In various embodiments, such a microarchitectural configuration 512 may be configured to provide the processor with separate and / or dedicated interfaces for the two memory media (e.g., volatile and non-volatile memory).

[0129] Figure 505 illustrates a third microarchitecture configuration 532 including example DDR5 538 and LPDDR5 536 memory. Specifically, Figure 505 illustrates a configuration 532 of DRAM including LPDDR5 536 (or LPDDR4 or any other suitable LPDDR memory) and a DRAM controller 534. The DRAM controller 534 may be configured to communicate with the PCIe interface of the processor 540 (e.g., an x86-based CPU) via CXL. In some embodiments, the DRAM controller 534 may be configured to communicate with the processor 540 via a predetermined number of channels (e.g., 16 channels). The processor 540 itself may be coupled to the DRAM (e.g., DDR5 543 or any suitable memory). In various embodiments, such a microarchitecture configuration 532 may be configured to accelerate and / or adapt address decoding and / or interleaving mechanisms.

[0130] Figure 507 illustrates a fourth microarchitecture configuration 542 including example-separate DDR and Z-NAND memories (e.g., eight such memories). Furthermore, Figure 507 illustrates a configuration 542 where flash memory (e.g., Z-NAND 548) may be present, each with a corresponding controller (e.g., SSD controller 546). The DRAM controller 544 may be configured to communicate with the SSD controller 546 to coordinate I / O on a predetermined number of channels. Furthermore, the DRAM controller 544 and / or the SSD controller 546 may communicate with the processor 552 (e.g., an x86-based CPU) via a CXL PCIe interface. Additionally, the DRAM controller 544 may communicate with the processor 552 via a first number of channels (e.g., eight CXL channels). Furthermore, the SSD controller 546 may communicate with the processor 552 via a second number of channels (e.g., eight PCIe channels) via a CXL. In some embodiments, the DRAM controller 544 and / or the SSD controller 546 may be configured to communicate with the processor 552 via a predetermined number of channels (e.g., 16 channels). The processor 552 itself may be coupled to DRAM (e.g., DDR5554 or any suitable memory). In various embodiments, such a microarchitectural configuration 542 may be configured to provide the processor with separate and / or dedicated interfaces for the two memory media (e.g., volatile and non-volatile memory).

[0131] In various embodiments, as mentioned, the disclosed system may include low-power double data rate (low-power DDR SDRAM or LPDDR SDRAM) memory. In another embodiment, LPDDR may include low-power (e.g., below a predetermined threshold amount of power) and double data rate synchronous dynamic random access memory that can be used in conjunction with a mobile device. In various embodiments, the disclosed system may use LPDDR to achieve increased data transfer rates (e.g., up to 6400 MT / s). Furthermore, the disclosed system may use LPDDR and employ differential clocking. Additionally, the number of memory banks may be increased (e.g., increased to 16) and may be partitioned into DDR4-like memory bank groups. In various embodiments, LPDDR may allow various power-saving mechanisms, such as data copying and write-X (all 1s or all 0s) commands, wherein the commands may also reduce data transfers and enable dynamic frequency and voltage adjustment.

[0132] In another embodiment, as mentioned, the disclosed system may use flash memory. In one embodiment, flash memory is an electrically erasable and reprogrammable electronic non-volatile computer memory storage medium. In another embodiment, the disclosed system may use three-dimensional flash memory, such as Z-NAND. In one embodiment, Z-NAND may include 3D single-cell (SLC) NAND or any other suitable flash memory. Specifically, the disclosed system may include, but is not limited to, SLC NAND, multi-cell (MLC) NAND (such as three-cell (TLC) NAND, four-cell (QLC) NAND), combinations thereof, etc.

[0133] Figure 6 This is a diagram illustrating an exemplary table associated with device-related parameters according to an example embodiment of this disclosure. In various embodiments, Figure 601 illustrates a table of various modular architecture constraints for dynamically reconfiguring a microarchitecture based on predetermined parameters. Specifically, the table shows parameters including form factor 602, size 604, power 606, interface 608, BW ratio 610, chip configuration 612, power-limited capacity 614 for DDR5, and area-limited capacity 616 for DDR5. Specifically, the disclosed system can inform different selections of the type of CXL-based device microarchitecture for different applications in data centers or other environments based on such parameters. In various embodiments, form factor 602 may include, but is not limited to, M.2, E1.1, E1.S, E3 S / L, U.2, NF1, full-height half-length (FHHL) add-in card (AIC), half-height half-length (HHHL) AIC, combinations thereof, etc.

[0134] In one embodiment, dimension 604 may be one of the dimensions corresponding to each form factor as shown in Table 1. For example, M.2 may have a form factor of approximately 110 mm × 22 mm, while NF1 may have a form factor of approximately 110 mm × 33 mm. In one embodiment, power requirement 606 may be one of the power requirements corresponding to each form factor and dimension as shown in the table. For example, for M.2, the power requirement may be approximately 8 W, while for U.2, the power requirement may be approximately 25 W. In various embodiments, each module may have a corresponding PCIe interface 608 corresponding to each form factor as shown in the table. For example, for an M.2 module, the interface may include four channels with a transmission rate of 16 Gb / s, while for an E1.S module, the interface may include any of four to eight channels operating at a transmission rate of approximately 16 to 32 Gb / s.

[0135] Furthermore, each device may have a corresponding BW ratio 610 (compared to the DDR5 channel BW) for each form factor, as shown in the table. For example, for an E1.L form factor module, the BW ratio may be approximately 0.5 to approximately 1 times the DDR5 BW ratio, while for a U.2 form factor module, the BW ratio may be approximately 0.5 times the DDR5 BW ratio. In various embodiments, example chip configurations are provided for each module. For example, an E1.S module may have a 55mm... 2 The chip incorporates approximately 12GB of LPDDR5 memory. The NF1 form factor module can have a diameter of 46mm. 2 The chip contains approximately 64GB of Z-NAND flash memory. Furthermore, it shows example power-limited DDR5 614 and area-limited DDR5 616 capacities that can be used in each module. For example, for an M.2 module using power-limited DDR5, the capacity could be approximately 25GB, while the area-limited DDR5 (non-3D) memory capacity could be approximately 16GB.

[0136] Note that these parameters are merely examples that may change over time with technological advancements in various underlying memories (e.g., DRAM, DDR memory, and / or flash memory). Furthermore, other types of memory may be used in modules with different form factors and / or sizes. In some embodiments, the disclosed system may include various device-level characteristics that manage the device microarchitecture. Specifically, a given microarchitecture may be determined individually or in combination based on a number of factors, such as the given system architecture using the device, the behavior of the high-level CXL controller, the volatile memory (e.g., DRAM) cache controller and organization, and / or the non-volatile memory (e.g., Z-NAND) controller and characteristics.

[0137] In some embodiments, the microarchitectural parameters of the CXL controller may include, but are not limited to, DRAM cache size, line size, and / or set organization. Furthermore, the microarchitecture of the CXL controller may be configured to implement a given eviction policy, detailed miss paths (e.g., write allocation, no allocation, etc.), and / or support prefetching. In another embodiment, the microarchitecture of the CXL controller may be configured to use included and excluded parameters. In one embodiment, the disclosed system may enable a cache coherent device to use one or more workload-dependent runtime parameters. Optionally or additionally, the disclosed system may include a device-on-device profiler module capable of profiling space / time access at the DRAM level. Thus, the disclosed system can profile capacity and BW and latency sensitivity at runtime and route data for processing based on such sensitivity. In various embodiments, the disclosed system may include a software architecture supporting the device microarchitecture. Specifically, the S / W architecture may support OS-aware data placement and migration. Furthermore, the disclosed system may implement an S / W architecture to utilize one or more Non-Uniform Memory Access (NUMA) mechanisms.

[0138] In various embodiments, the disclosed system can determine boot-time events and reconfigure itself accordingly. Specifically, the disclosed system can configure or reconfigure logic on the device to use DRAM as, for example, a software or hardware cache. In some embodiments, the disclosed system can, for example, use DVSEC to advertise its capabilities to the host device via CXL. Furthermore, the disclosed system can, for example, receive host commands at the aforementioned boot time to reconfigure the device. In various embodiments, the device may include an on-device profiler module capable of performing various operations, such as determining memory addresses, R / W, access frequencies, one or more address modes, etc. In various embodiments, the profiler can provide this information to the host as described above, so that when the system boots, the host can reconfigure the device based on previous usage patterns identified by the profiler.

[0139] In various embodiments, the disclosed system can be extended to a Type 2 CXL device microarchitecture implementation. In some embodiments, the disclosed system may include a Type 1 device implementing a cache coherence protocol, such as the CXL.io and CXL.cache modules in a cache controller. Furthermore, the disclosed system may include device-attached memory, which may also be cache-coherent.

[0140] Figure 7This is an illustration of an exemplary cache coherence device microarchitecture according to an example embodiment of the present invention. In various embodiments, Figure 702 may include a CXL 3 type device, but is not necessarily limited to such a protocol. In one embodiment, the device may include a CXL controller 704 (or other cache coherence protocol-based controller), a DRAM cache controller 706 (or other volatile memory cache controller), and a NAND controller 708 (or other non-volatile memory controller). In another embodiment, the CXL controller 704 may be connected to a root complex 714. Specifically, the root complex 714 may connect the cache coherence device to a PCI Express switch fabric consisting of one or more switch devices. The root complex 714 may be connected to a DRAM memory controller 716 and an associated DDR4 memory 718 that perform substantially similar operations on the host side.

[0141] In another embodiment, the DRAM cache controller 706 can be connected to the DDR4 710 and NAND controller 708 using any suitable protocol (such as CXL and / or PCIe). The NAND controller 708 can be connected to an SSD 712 device (e.g., a flash memory chip) for non-volatile storage.

[0142] Figure 8 This is an illustration of an exemplary apparatus for resource management according to an example embodiment of the present disclosure. In various embodiments, Figure 801 may include means that can be further coupled to a CXL controller 802. In another embodiment, the means may include intelligent components (e.g., circuitry and corresponding firmware and / or software), such as prefetch logic 804. In one embodiment, the means may include a cache controller 806, which may have a flow control module, a write buffer module, a cache addressing module, a cache policy implementation module, and a timing module. Furthermore, the cache controller may be connected to an SSD controller 808.

[0143] In some embodiments, the disclosed system may include device components that are CXL-compatible and can be used as accelerators for compute-intensive applications, and therefore include a local cache and thus have a cache controller 806. Thus, if a host component wants to access the same location in memory, it can force the device to evict a line from its local cache and update the corresponding memory location based on the cache line state. In various embodiments, the disclosed system may be configured to operate using the CXL.cache protocol, which may notify interactions between the disclosed device and the host as multiple requests, each with at least one associated response message and sometimes data transfers. In some embodiments, the disclosed system may communicate with the host and other devices via an interface, wherein the interface includes three channels in each direction: request, response, and data.

[0144] In one embodiment, the flow control module of the cache controller 806 can implement flow control to manage the data transfer rates of the CXL controller and any other devices (inputs or outputs) to prevent a fast transmitter from overwhelming a slow receiver. Therefore, the flow control module can provide the CXL controller with a mechanism to control the transmission speed so that the CXL controller 802 is not overwhelmed by data from auxiliary devices (e.g., an SSD controller or a host).

[0145] In another embodiment, the write buffer module of the cache controller 806 may include a type of data buffer that can be used to hold data being written from the cache to memory or the next cache in the memory hierarchy. In some embodiments, the write buffer module can release the cache to serve read requests when a write occurs. Therefore, when the write buffer is full (causing the buffer to be occupied), subsequent writes must still wait until the time slot is released; thus, subsequent reads can be served by the write buffer.

[0146] In some aspects, memory blocks may not be randomly placed in the cache, but may need to be restricted to a single cache line or a set of cache lines through a cache placement strategy. In one embodiment, the cache addressing module may implement such a cache placement strategy. Specifically, the cache in the main memory hierarchy may include cache lines grouped into sets. Furthermore, a data request has an address specifying the location of the requested data. Data blocks of each cache line size from lower levels can only be placed in one set; furthermore, the set of blocks that can be placed of the cache line size may depend on its address. Therefore, the cache addressing module may generate a request address that may include an offset portion identifying a specific location within the cache line. Additionally, the request may include a set portion identifying the set containing the requested data. Furthermore, the request may include a tag portion that can be stored with its data in each cache line to distinguish different addresses that can be placed in the set.

[0147] In another embodiment, the cache strategy implementation module of the cache controller 806 can implement different strategies for placing memory blocks in the cache, such as direct mapping, fully associative and set-associative placement strategies known to those skilled in the art.

[0148] In one embodiment, the timing module of the cache controller 806 may determine the timing of writes based on a write policy. In some aspects, the disclosed system may include a write-through cache, wherein writes to the cache trigger writes to memory based on a write policy determined by the timing module. Optionally, the disclosed system may include a write-back or copy-back cache, wherein writes are not immediately mirrored to main memory, but the cache tracks which locations have been overwritten, thereby marking them as dirty based on a write policy determined by the timing module. Furthermore, when data in these locations is evicted from the cache, the data is written back to main memory. Additionally, writes to main memory locations not yet mapped in the write-back cache may evict dirty locations, thereby freeing up cache space for new memory locations based on a write policy determined by the timing module. The disclosed system may also implement an intermediate policy based on a write policy determined by the timing module. For example, the disclosed system may include a cache that can be write-through, but writes may be temporarily held in a storage data queue, and thus multiple stores can typically be processed together based on a write strategy determined by the timing module (e.g., to reduce bus turnaround and improve bus utilization).

[0149] In various embodiments, the apparatus may include a DRAM controller. The DRAM controller may include multiple channels. Each channel may be coupled to a respective channel request queue (e.g., channel request queues 810 and 812) and may include respective controller submodules 814 and 816, wherein each controller submodule 814 and 816 includes a DRAM timing / policy repository, a DRAM request queue, a DRAM request scheduler, and a DRAM command scheduler. In one embodiment, channel request queues 814 and 816 enable the DRAM memory and the memory controller to communicate via the specified channel when the specified channel becomes available via a request.

[0150] In another embodiment, the DRAM timing / policy repository can be configured to control various timing parameters. For example, the DRAM timing / policy repository can be configured to specify the minimum time that must elapse between row activation and a read or write command. The DRAM timing / policy repository can control CAS latency, which may include the number of clock cycles allowed for the internal operation between a read command and the appearance of a first data word on the corresponding data bus. The DRAM timing / policy repository can determine other configurable parameters, including but not limited to the lengths of read and write bursts, wherein the lengths of the read and write bursts include the number of words transmitted by each read or write command.

[0151] In various embodiments, a DRAM request scheduler can schedule commands associated with a controller for processing by the controller. In one embodiment, a DRAM request queue can enqueue a sequence of commands associated with a transaction that has won arbitration and entered the memory controller. The disclosed system can map such sequences to memory address locations and translate them into sequences of DRAM commands. In some embodiments, the DRAM request queues can be arranged as pools, and the CXL controller 802 can select from the commands to be executed. Optionally or additionally, the queues can be arranged such that one queue exists for each bank or row of memory. A DRAM command scheduler can determine a DRAM command scheduling policy and can schedule commands to DRAM devices via a telecommunications interface.

[0152] Figure 9This is an illustration of an exemplary cache coherence device 902 according to an example embodiment of the present disclosure. In various embodiments, the cache coherence device 902 may include a Type 1 device. In another embodiment, the cache coherence device 902 may include a CXL controller 904, an accelerator and / or network interface card (NIC) device 906, a DDR4 908 (or other volatile memory), a processor 910 (optional), and a second DDR4 912 (or other volatile memory, which may be optional). The cache coherence device 902 may also be connected to a root complex 914 via the CXL controller 904. The root complex 914 may be coupled to a DRAM controller 916, which may also be connected to the DDR4 918.

[0153] In various embodiments, the CXL controller 904 may include soft IP designed for application-specific integrated circuit (ASIC) and / or field-programmable gate array (FPGA) implementations. In another embodiment, the CXL controller 904 may include a PCIe 5.0 (or other version) architecture for the CXL.10 path and may add CXL-specific CXL.cache and CXL.mem paths. In various embodiments, the CXL controller 904 may be implemented to support the CXL protocol (e.g., the CXL 2.0 protocol or any other version). In another embodiment, the CXL controller 904 may be configured to be backward compatible with older cache coherence protocols such as CXL 1.1. As described above, the CXL controller 904 may be configured to implement the CXL.io, CXL.mem, and CXL.cache protocols or other suitable cache coherence protocols. In some embodiments, the CXL controller 904 may be configured to support different CXL device types, such as Type 1, Type 2, and / or Type 3 CXL devices. In various embodiments, the CXL controller 904 may be configured to support PCIe protocols, such as the PCIe 5.0 protocol. In another embodiment, the CXL controller 904 may be configured to support the PIPE 5.x protocol using any suitable PIPE interface width (e.g., configurable PIPE interface widths of 8, 16, 32, 64, and 128 bits). In yet another embodiment, the CXL controller 904 may be configured to support various CXL device configurations, such as those combined with the above. Figure 5 The CXL device configurations shown and described.

[0154] In one embodiment, the accelerator / NIC device 906 may perform hardware acceleration, i.e., perform some functions more efficiently than could be done in software running on a general-purpose central processing unit (CPU). In another embodiment, the accelerator / NIC device 906 may perform such acceleration as an implementation of computational tasks in hardware to reduce latency and increase throughput associated with processing workloads or other tasks. Furthermore, the accelerator / NIC device 906 may be configured to improve the execution of specific algorithms by allowing greater concurrency, having specific data paths for its temporary variables, and reducing the overhead of instruction control in the fetch-decode-execution cycle.

[0155] In another embodiment, DDR4 908 is an example of a DRAM device that can be used as a memory for storing data and performing associated tasks on the data. In another embodiment, the disclosed system may include any suitable memory, and DDR4 is discussed only as an example. Specifically, such memory may include any DRAM, wherein the operation of the external pin interface of the DRAM is coordinated by an externally provided clock signal. In some embodiments, the memory may include synchronous dynamic random access memory with a high-bandwidth (e.g., double data rate) interface, such as DDR5 or any suitable DRAM. Furthermore, the memory may include high-bandwidth memory (HBM), wherein the high-bandwidth memory (HBM) may include a high-performance RAM interface for DRAMs in 3D stacking.

[0156] In another embodiment, the second DDR4 912 may also be an example of a DRAM device that can be used as a memory for storing data and performing associated tasks on the data. In another embodiment, the disclosed system may include any suitable memory, and DDR4 is discussed only as an example. Specifically, such memory may include any DRAM, wherein the operation of the external pin interface of the DRAM is coordinated by an externally provided clock signal. In some embodiments, the memory may include synchronous dynamic random access memory with a high-bandwidth (e.g., double data rate) interface, such as DDR5 or any suitable DRAM. Furthermore, the memory may include HBM.

[0157] In one embodiment, processor 910 may include electronic circuitry that executes instructions comprising a computer program. The CPU may be configured to perform arithmetic, logic, control, and input / output (I / O) operations specified by the instructions in the program.

[0158] Figure 10This is an illustration of an exemplary flow including example operations associated with the disclosed system according to an example embodiment of the present disclosure. At block 1002, the disclosed system may operate a first controller coupled to a network interface according to a cache coherence protocol. At block 1004, the disclosed system may use a second controller coupled to the first controller and coupled to a first memory to perform at least one operation on data associated with a cache. At block 1006, the disclosed system may store data on a second memory coupled to one of the first controller or the second controller. In various embodiments, the disclosed system may also receive configuration information from a host and operate the second controller as one of a software-based cache or a hardware-based cache in a startup-time mode of operation. Furthermore, the disclosed system may receive data via the network interface using a cache coherence protocol via the first controller, perform at least one second operation on the data to generate second data, and store the second data on the second memory based on the cache coherence protocol. In some aspects, the cache coherence protocol may include the CXL protocol, and the disclosed system may include a profiler that determines at least one capability associated with the device and provides that capability to a host.

[0159] Figure 11 Example schematic diagrams are shown of systems that can be used to practice embodiments of this disclosure. For example... Figure 11 As illustrated, specific embodiments may include one or more management computing entities 1100, one or more networks 1105, and one or more user devices 1110. Each of these components, entities, devices, systems, and similar terms used interchangeably herein may communicate directly or indirectly with each other via the same or different wired or wireless networks. Furthermore, management computing entity 1100 may include the machine learning components described herein. As mentioned above, any suitable protocols further described herein may be used to perform communication.

[0160] Figure 12 An example schematic diagram of a management computing entity according to a disclosed example embodiment is shown. Furthermore, the management computing entity 1200 may include a content component, a processing component, and a transmission component (not shown). Specifically, the content component may be used to determine signals indicating data (e.g., video, audio, text, data, combinations thereof, etc.) to be transmitted through the architecture described herein. In another embodiment, the determination of the signals for transmission may be based, for example, on user input from the device, a predetermined schedule of data transmission on the network, changes in network conditions, etc. In one embodiment, the signal may include: data may be encapsulated in data frames configured to be transmitted from the device to one or more devices on the network.

[0161] In another embodiment, processing element 1205 can be used to determine various parameters associated with data transmitted over the network and / or parameters associated with clusters of parts of the network. For example, processing element 1205 can be used to run models against network data, run machine learning techniques against network data, determine the distribution of workloads to be processed by various parts of the network architecture, combinations thereof, etc. As another example, processing element 1205 can be used to run models against network data, run machine learning techniques against parameters associated with different performance capabilities of network clusters, determine the distribution of workloads to be processed by various clusters of parts of the network architecture, combinations thereof, etc.

[0162] In one embodiment, a transmitting component (not shown) can be used to transmit a signal from one device to another device on a network (e.g., from a first device on a first cluster to a second device on a second cluster, e.g., using a cache coherence protocol). For example, the transmitting component can be used to prepare a transmitter (e.g., as shown below). Figure 12 The transmitter 1204 can transmit signals over a network. For example, the transmitting component can queue data in one or more buffers, determine that the transmitting device and associated transmitter are working properly and have sufficient power to transmit signals over the network, and adjust one or more parameters associated with data transmission (e.g., modulation type, signal amplification, signal power level, noise suppression, combinations thereof, etc.).

[0163] Generally, the terms computing entity, computer, entity, device, system, and / or similar terms used interchangeably herein can refer to, for example, one or more computers, computing entities, desktop computers, mobile phones, tablet computers, phablets, laptop computers, distributed systems, game consoles (e.g., Xbox, PlayStation, Wii), watches, glasses, iBeacons, proximity beacons, keychains, radio frequency identification (RFID) tags, headphones, scanners, televisions, dongles, cameras, wristbands, wearable items / devices, automated service terminals, input terminals, servers or server networks, blades, gateways, switches, processing devices, processing entities, set-top boxes, repeaters, routers, network access points, base stations, etc., and / or any combination of devices or entities suitable for performing the functions, operations, and / or processes described herein. Such functions, operations, and / or processes may include, for example, sending, receiving, operating, processing, displaying, storing, determining, creating / generating, monitoring, evaluating, comparing, and / or similar terms used interchangeably herein. In one embodiment, these functions, operations, and / or processes may be performed on data, content, information, and / or similar terms used interchangeably herein.

[0164] As indicated, in one embodiment, the management computing entity 1100 may also include one or more communication interfaces 1220 for communicating with various computing entities (e.g., by transmitting data, content, information, and / or similar terms that may be sent, received, manipulated, processed, displayed, stored, etc.). For example, the management computing entity 1100 may communicate with user device 1110 and / or various other computing entities.

[0165] like Figure 12 As shown, in one embodiment, the management computing entity 1100 may include one or more processing elements 1205 (also referred to as processors, processing circuitry, and / or similar terms used interchangeably herein) or communicate with one or more processing elements 1205, which may communicate with other elements within the management computing entity 1100, for example, via a bus. As will be understood, the processing element 1205 may be implemented in a variety of different ways. For example, the processing element 1205 may be implemented as one or more complex programmable logic devices (CPLDs), microprocessors, multi-core processors, coprocessor entities, application-specific instruction set processors (ASIPs), microcontrollers, and / or controllers. Furthermore, the processing element 1205 may be implemented as one or more other processing means or circuits. The term "circuit" may refer to a completely hardware embodiment or a combination of hardware and computer program products. Thus, the processing element 1205 may be implemented as an integrated circuit, an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), a programmable logic array (PLA), a hardware accelerator, other circuitry, etc. Therefore, as will be understood, processing element 1205 may be configured for a particular purpose or configured to execute instructions stored in volatile or non-volatile media or otherwise accessible by processing element 1205. Thus, whether configured by hardware or computer program products, or by a combination thereof, when configured accordingly, processing element 1205 may be able to perform steps or operations according to embodiments of this disclosure.

[0166] In one embodiment, the management computing entity 1100 may also include or communicate with a non-volatile medium (also referred to as a non-volatile storage device, memory, memory storage device, memory circuit, and / or similar terms used interchangeably herein). In one embodiment, the non-volatile storage device or memory may include one or more non-volatile storage devices or storage media 1210, including but not limited to hard disks, ROMs, PROMs, EPROMs, EEPROMs, flash memory, MMC, SD memory cards, Memory Sticks, CBRAMs, PRAMs, FeRAMs, NVRAMs, MRAMs, RRAMs, SONOS, FJG RAMs, Millipede memory, track memory, etc. As will be appreciated, the non-volatile storage device or memory media may store databases, database instances, database management systems, data, applications, programs, program components, scripts, source code, object code, bytecode, compiled code, interpreted code, machine code, executable instructions, etc. The terms database, database instance, database management system, and / or similar terms used interchangeably herein can refer to a collection of records or data stored in a computer-readable storage medium using one or more database models (such as hierarchical database models, network models, relational models, entity-relationship models, object models, document models, semantic models, graph models, etc.).

[0167] In one embodiment, the management computing entity 1100 may further include or communicate with volatile media (also referred to as volatile storage device, memory, memory storage device, memory circuitry, and / or similar terms used interchangeably herein). In one embodiment, the volatile storage device or memory may further include one or more volatile storage devices or storage media 1215, including but not limited to RAM, DRAM, SRAM, FPMDRAM, EDO DRAM, SDRAM, DDR SDRAM, DDR2 SDRAM, DDR3 SDRAM, RDRAM, TTRAM, T-RAM, Z-RAM, RIMM, DIMM, SIMM, VRAM, cache memory, register memory, etc. As will be appreciated, the volatile storage device or storage media may be used to store, for example, at least a portion of databases, database instances, database management systems, data, applications, programs, program components, scripts, source code, object code, bytecode, compiled code, interpreted code, machine code, executable instructions, etc., executed by the processing element 1205. Therefore, databases, database instances, database management systems, data, applications, programs, program components, scripts, source code, object code, bytecode, compiled code, interpreted code, machine code, executable instructions, etc., can be used to control and manage certain aspects of the operation of computing entity 1100 with the help of processing element 1205 and operating system.

[0168] As indicated, in one embodiment, the management computing entity 1100 may also include one or more communication interfaces 1220 for communicating with various computing entities (e.g., by transmitting data, content, information, and / or similar terms that may be sent, received, manipulated, processed, displayed, stored, etc., and / or used interchangeably herein). Such communication may be performed using wired data transmission protocols such as Fast Peripheral Component Interconnect (PCIe), Fiber Distributed Data Interface (FDDI), Digital Subscriber Line (DSL), Ethernet, Asynchronous Transfer Mode (ATM), Frame Relay, Cable Data Service Interface Specification (DOCSIS), or any other wired transmission protocol. Similarly, the management computing entity 1100 can be configured to communicate via a wireless external communication network using any of a variety of protocols, such as General Packet Radio Service (GPRS), Universal Mobile Telecommunications System (UMTS), Code Division Multiple Access 2000 (CDMA2000), CDMA2000 1X (1xRTT), Wideband Code Division Multiple Access (WCDMA), Time Division Synchronous Code Division Multiple Access (TD-SCDMA), Long Term Evolution (LTE), Evolved Universal Terrestrial Radio Access Network (E-UTRAN), Evolved Data Optimized (EVDO), High Speed ​​Packet Access (HSPA), High Speed ​​Downlink Packet Access (HSDPA), IEEE 802.11 (Wi-Fi), Wi-Fi Direct, 802.16 (WiMAX), Ultra Wideband (UWB), Infrared (IR) protocol, Near Field Communication (NFC) protocol, ZigBee, Bluetooth protocol, 5G protocol, Wireless Universal Serial Bus (USB) protocol, and / or any other wireless protocol.

[0169] Although not shown, the management computing entity 1100 may include one or more input elements (such as keyboard input, mouse input, touchscreen / display input, motion input, movement input, audio input, indicator input, joystick input, keypad input, etc.) or communicate with one or more input elements. The management computing entity 1100 may also include one or more output elements (not shown) (such as audio output, video output, screen / display output, motion output, movement output, etc.) or communicate with one or more output elements.

[0170] As will be understood, one or more components of the management computing entity 1100 may be configured to be located remotely from other management computing entity 1100 components (e.g., in a distributed system). Furthermore, one or more components may be combined, and additional components performing the functions described herein may be included in the management computing entity 1100. Therefore, the management computing entity 1100 can be adapted to various needs and situations. As will be appreciated, these architectures and descriptions are provided for illustrative purposes only and are not limited to various embodiments.

[0171] Users can be individuals, families, companies, organizations, entities, departments within an organization, representatives of organizations and / or individuals, etc. In one example, a user can be an employee, resident, customer, etc. For example, a user can operate user device 1110, which includes one or more components that are functionally similar to those used to manage computing entity 1100.

[0172] In various aspects, the processing components, transmitting components, and / or receiving components (not shown) may be configured to operate on one or more user devices 1110, and they may include, as described herein, in combination with other components. Figure 11 and Figure 12 Aspects of the functionality of the management computing entity 1100 are shown and described. Specifically, the processing components, transmitting components, and / or receiving components may be configured to communicate with one or more processing elements 1205, memory 1210, volatile memory 1215, and may include a communication interface 1220 (e.g., for facilitating communication between devices).

[0173] Figure 13 An example schematic diagram of a user device according to a disclosed example embodiment is shown. Figure 13 A user device 1110 (in conjunction with embodiments of this disclosure) is provided, which can be used in conjunction with embodiments of this disclosure. Figure 11 (Illustrative diagram shown). Generally, the terms apparatus, system, computing entity, entity, and / or similar terms used interchangeably herein can refer to, for example, one or more computers, computing entities, desktop computers, mobile phones, tablet computers, phablets, laptop computers, laptop computers, distributed systems, game consoles (e.g., Xbox, PlayStation, Wii), watches, glasses, keychains, radio frequency identification (RFID) tags, headphones, scanners, cameras, wristbands, automated service terminals, input terminals, servers or server networks, blades, gateways, switches, processing devices, processing entities, set-top boxes, repeaters, routers, network access points, base stations, etc., and / or any combination of apparatuses or entities suitable for performing the functions, operations, and / or processes described herein. User apparatus 1110 can be operated by the parties. Figure 13 As shown, user equipment 1110 may include antenna 1312, transmitter 1304 (e.g., radio), receiver 1306 (e.g., radio), and processing elements (also referred to as processing devices) 1308 (e.g., CPLD, FPGA, microprocessor, multi-core processor, coprocessor entity, ASIP, microcontroller, and / or controller) that provide signals to transmitter 1304 and receive signals from receiver 1306, respectively.

[0174] The signals provided to transmitter 1304 and received from receiver 1306 may include signaling information according to the air interface standard of the applicable wireless system. In this respect, user equipment 1110 may be able to operate with one or more air interface standards, communication protocols, modulation types, and access types. More specifically, user equipment 1110 may operate according to multiple wireless communication standards and protocols (such as those mentioned above regarding...). Figure 10 The user equipment 1110 can operate according to any of the wireless communication standards and protocols described in the management computing entity 1100. In a particular embodiment, the user equipment 1110 can operate according to a plurality of wireless communication standards and protocols, such as the publicly disclosed IoT DOCSIS protocol, UMTS, CDMA2000, 1xRTT, WCDMA, TD-SCDMA, LTE, E-UTRAN, EVDO, HSPA, HSDPA, 5G, Wi-Fi, Wi-Fi Direct, WiMAX, UWB, IR, NFC, Bluetooth, USB, etc. Similarly, the user equipment 1110 can operate via network interface 1320 according to a plurality of wired communication standards and protocols, such as the wired communication standards and protocols described above with respect to management computing entity 1100.

[0175] Through these communication standards and protocols, user equipment 1110 can communicate with various other entities using concepts such as Unstructured Supplemental Service Data (USSD), Short Message Service (SMS), Multimedia Messaging Service (MMS), Dual-Tone Multi-Frequency Signaling (DTMF), and / or Subscriber Identity Component Dialer (SIM Dialer). User equipment 1110 can also download changes, add-ons, and updates to, for example, its firmware, software (e.g., including executable instructions, applications, program components), and operating system.

[0176] According to one embodiment, user device 1110 may include location determination aspects, devices, components, functions, and / or similar terms used interchangeably herein. Location determination aspects may be used to inform one or more models used by an administrative computing entity, as well as models and / or machine learning techniques described herein. For example, user device 1110 may include outdoor positioning aspects (such as location components adapted to acquire, for example, latitude, longitude, altitude, geocoding, route, direction, heading, speed, UTC, date, and / or various other information / data). In one embodiment, the location component may acquire data, sometimes referred to as ephemeris data, by identifying the number of satellites in the field of view and the relative positions of these satellites. Satellites may be a variety of different satellites, including Low Earth Orbit (LEO) satellite systems, U.S. Department of Defense (DOD) satellite systems, European Union Galileo positioning system, China's Compass Navigation System, India's Regional Navigation Satellite System, etc. Optionally, location information may be determined by triangulation of the location of user device 1110 in conjunction with various other systems, including cell towers, Wi-Fi access points, etc. Similarly, user device 1110 may include indoor positioning aspects (such as location components adapted to acquire, for example, latitude, longitude, altitude, geocoding, route, direction, heading, speed, time, date, and / or various other information / data). Some indoor systems may use various location or positioning technologies (including RFID tags, indoor beacons or transmitters, Wi-Fi access points, cellular towers, nearby computing devices (e.g., smartphones, laptops), etc.). For example, such technologies may include iBeacon, gimbal proximity beacons, Bluetooth Low Energy (BLE) transmitters, NFC transmitters, etc. These indoor positioning aspects can be used in various setups to pinpoint the location of a person or object within inches or centimeters.

[0177] User device 1110 may also include a user interface (which may include a display 1316 connected to processing element 1308) and / or a user input interface (connected to processing element 1308). For example, as described herein, the user interface may be a user application, browser, user interface, and / or similar terms used interchangeably herein that execute on and / or are accessible via user device 1110 for interacting with management computing entity 1100 and / or displaying information from management computing entity 1100. The user input interface may include any of a plurality of means or interfaces that allow user device 1110 to receive data (such as a keypad 1318 (hard or soft), a touch display, a voice / voice or motion interface, or other input means). In embodiments that include keypad 1318, keypad 1318 may include (or be able to display) conventional numbers (0-9) and related keys (#, *) and other keys for operating user device 1110, and may include a full set of letter keys or a set of keys that can be activated to provide a full set of alphanumeric keys. In addition to providing input, the user input interface can be used to activate or deactivate specific functions (such as screen saver and / or sleep mode).

[0178] User device 1110 may also include volatile storage devices or memories 1322 and / or non-volatile storage devices or memories 1324, which may be embedded and / or removable. For example, non-volatile memory may be ROM, PROM, EPROM, EEPROM, flash memory, MMC, SD memory card, Memory Stick, CBRAM, PRAM, FeRAM, NVRAM, MRAM, RRAM, SONOS, FJG RAM, Millipede memory, track memory, etc. Volatile memory may be RAM, DRAM, SRAM, FPMDRAM, EDO DRAM, SDRAM, DDR SDRAM, DDR2 SDRAM, DDR3 SDRAM, RDRAM, TTRAM, T-RAM, Z-RAM, RIMM, DIMM, SIMM, VRAM, cache memory, register memory, etc. Volatile and non-volatile storage devices or memories may store databases, database instances, database management systems, data, applications, programs, program components, scripts, source code, object code, bytecode, compiled code, interpreted code, machine code, executable instructions, etc., to implement the functions of user device 1110. As indicated, this may include user applications that exist on the physical entity or can be accessed through a browser or other user interface for communicating with management computing entity 1100 and / or various other computing entities.

[0179] In another embodiment, as described in more detail above, user device 1110 may include one or more components or functions that are the same as or similar to those of the management computing entity 1100. As will be appreciated, these architectures and descriptions are provided for illustrative purposes only and are not limited to the various embodiments.

[0180] Specific embodiments may be implemented as hardware, firmware, and software, or a combination thereof. Other embodiments may also be implemented as instructions stored on a computer-readable storage device that can be read and executed by at least one processor to perform the operations described herein. A computer-readable storage device may include any non-transitory memory mechanism for storing information in a machine-readable (e.g., computer) form. For example, a computer-readable storage device may include read-only memory (ROM), random access memory (RAM), magnetic disk storage media, optical storage media, flash memory devices, and other storage devices and media.

[0181] The term "exemplary" is used herein to mean "serving as an example, instance, or illustration." Any embodiment described herein as "exemplary" is not necessarily to be construed as preferred or advantageous over other embodiments. The terms "computing device," "user device," "communication station," "station," "handheld device," "mobile device," "wireless device," and "user equipment" (UE) are used herein to refer to wireless communication devices (such as cellular phones, smartphones, tablet computers, netbooks, wireless terminals, laptop computers, home base stations, high data rate (HDR) user stations, access points, printers, point-of-sale devices, access terminals, or other personal communication system (PCS) devices). Devices may be mobile or stationary.

[0182] As used herein, the term "communication" is intended to include sending, or receiving, or both. This is particularly useful in the claims when describing an organization of data sent by one device and received by another device, but where only the functionality of one of these devices is required. Similarly, a bidirectional data exchange between two devices (where both devices send and receive during the exchange) can be described as "communication" when only the functionality of one of these devices is required. The term "communication" as used herein in relation to wireless communication signals includes sending and / or receiving wireless communication signals. For example, a wireless communication unit capable of transmitting wireless communication signals may include a wireless transmitter for sending wireless communication signals to at least one other wireless communication unit, and / or a wireless communication receiver for receiving wireless communication signals from at least one other wireless communication unit.

[0183] Some embodiments can be used with a variety of devices and systems, such as personal computers (PCs), desktop computers, mobile computers, laptop computers, notebook computers, tablet computers, server computers, handheld computers, handheld devices, personal digital assistant (PDA) devices, handheld PDA devices, in-vehicle devices, non-in-vehicle devices, hybrid devices, vehicle devices, non-vehicle devices, mobile or portable devices, consumer devices, non-mobile or non-portable devices, wireless communication stations, wireless communication devices, wireless access points (APs), wired or wireless routers, wired or wireless modems, video devices, audio devices, audio-video (A / V) devices, wired or wireless networks, wireless local area networks, wireless video local area networks (WVANs), local area networks (LANs), wireless LANs (WLANs), personal area networks (PANs), wireless PANs (WPANs), etc.

[0184] Some embodiments may be used in conjunction with one-way and / or two-way radio communication systems, cellular wireless telephone communication systems, mobile phones, cell phones, wireless phones, personal communication system (PCS) devices, PDA devices including wireless communication devices, mobile or portable global positioning system (GPS) devices, devices including GPS receivers or transceivers or chips, devices including RFID elements or chips, multiple-input multiple-output (MIMO) transceivers or devices, single-input multiple-output (SIMO) transceivers or devices, multiple-input single-output (MISO) transceivers or devices, devices having one or more internal antennas and / or external antennas, digital video broadcasting (DVB) devices or systems, multi-standard radio devices or systems, wired or wireless handheld devices (e.g., smartphones), Wireless Application Protocol (WAP) devices, etc.

[0185] Some embodiments may be compatible with one or more wireless communication protocols (e.g., radio frequency (RF), infrared (IR), frequency division multiplexing (FDM), orthogonal FDM (OFDM), time division multiplexing (TDM), time division multiple access (TDMA), extended TDMA (E-TDMA), General Packet Radio Service (GPRS), extended GPRS, code division multiple access (CDMA), wideband CDMA (WCDMA), CDMA 2000, single-carrier CDMA, multi-carrier CDMA, multi-carrier modulation (MDM), discrete multi-tone (DMT), Bluetooth). TM Global Positioning System (GPS), Wi-Fi, Wi-Max, ZigBee TMIt can be used in combination with one or more types of wireless communication signals and / or systems, such as Ultra Wideband (UWB), Global System for Mobile Communications (GSM), 2G, 2.5G, 3G, 3.5G, 4G, 5G mobile networks, 3GPP, Long Term Evolution (LTE), LTE Advanced, and Enhanced Data Rate GSM Evolution (EDGE). Other embodiments can be used in a variety of other devices, systems, and / or networks.

[0186] Although an example processing system has been described above, embodiments of the subject matter and functional operation described herein may be implemented in other types of digital electronic circuits, or in computer software, firmware, or hardware (including the structures disclosed herein and their equivalents), or in one or more combinations thereof.

[0187] The embodiments of the subject matter and operation described herein may be implemented in digital electronic circuits, or in computer software, firmware, or hardware (including the structures disclosed herein and their equivalents), or a combination thereof. Embodiments of the subject matter described herein may be implemented as one or more computer programs (i.e., one or more components of computer program instructions encoded on a computer storage medium for execution by or control of the operation of an information / data processing device). Optionally or additionally, the program instructions may be encoded on artificially generated propagation signals (e.g., machine-generated electrical, optical, or electromagnetic signals) generated to encode information / data for transmission to a suitable receiver device for execution by the information / data processing device. The computer storage medium may be a computer-readable storage device, a computer-readable storage substrate, a random or serial access memory array or device, or a combination thereof, or the computer storage medium may be included in a computer-readable storage device, a computer-readable storage substrate, a random or serial access memory array or device, or a combination thereof. Furthermore, although computer storage media are not propagating signals, they can be a source or destination of computer program instructions encoded in artificially generated propagating signals. Computer storage media can also be one or more separate physical components or media (e.g., multiple CDs, disks, or other storage devices), or be included within one or more separate physical components or media (e.g., multiple CDs, disks, or other storage devices).

[0188] The operations described herein can be implemented as operations performed by an information / data processing device on information / data stored on one or more computer-readable storage devices or received from other sources.

[0189] The term "data processing device" includes all types of devices, apparatuses, and machines for processing data (including, for example, programmable processors, computers, systems-on-a-chip, or multiple or combinations of the foregoing). Devices may include special-purpose logic circuitry (e.g., FPGAs (Field-Programmable Gate Arrays) or ASICs (Application-Specific Integrated Circuits)). In addition to hardware, devices may also include code that creates the execution environment for the computer program in question (e.g., code constituting processor firmware, protocol stacks, database management systems, operating systems, cross-platform runtime environments, virtual machines, or one or more combinations thereof). Devices and execution environments can implement a wide variety of computing model infrastructures (such as web services, distributed computing, and grid computing infrastructures).

[0190] Computer programs (also known as programs, software, software applications, scripts, or code) can be written in any programming language (including compiled or interpreted languages, declarative or procedural languages) and can be deployed in any form (including as standalone programs or as components, subroutines, objects, or other units suitable for use in a computing environment). A computer program may, but does not necessarily, correspond to a file in a file system. A program may be stored as a part of a file containing other programs or information / data (e.g., stored in one or more scripts in a markup language document), in a single file dedicated to the program in question, or in multiple coordinating files (e.g., a file storing one or more components, subroutines, or code sections). A computer program may be deployed to execute on a single computer or on multiple computers located at a single site or distributed across multiple sites and interconnected by a communication network.

[0191] The processing and logical flow described herein can be executed by one or more programmable processors that execute one or more computer programs to perform actions by manipulating input information / data and generating output. As an example, processors suitable for executing computer programs include both general-purpose microprocessors and special-purpose microprocessors, as well as any type of digital computer and any one or more processors. Typically, the processor receives instructions and information / data from read-only memory or random access memory, or both. The basic elements of a computer are a processor for performing actions according to instructions and one or more memory devices for storing instructions and data. Typically, a computer will also include one or more mass storage devices (e.g., magnetic disks, magneto-optical disks, or optical disks) for storing data, or operatively connected to receive information / data from one or more mass storage devices or transfer information / data to one or more mass storage devices, or both. However, a computer does not need to have such devices. Suitable devices for storing computer program instructions and information / data include all forms of non-volatile memory, media, and memory devices (e.g., semiconductor memory devices (e.g., EPROM, EEPROM, and flash memory devices); magnetic disks (e.g., internal hard disks or removable hard disks); magneto-optical disks; and CD-ROM and DVD-ROM discs). Processors and memory may be supplemented by or contained within dedicated logic circuitry.

[0192] To provide interaction with the user, embodiments of the subject matter described herein can be implemented on a computer having a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor for displaying information / data to the user) and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices may also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback), and input from the user can be received in any form (including sound, speech, or tactile input). Furthermore, the computer can interact with the user by sending documents to and receiving documents from the device used by the user (e.g., by sending web pages to a web browser on the user's client device in response to a request received from a web browser).

[0193] Embodiments of the subject matter described herein can be implemented in a computing system that includes back-end components (e.g., as an information / data server), middleware components (e.g., an application server), front-end components (e.g., a client computer with a graphical user interface or web browser through which a user can interact with embodiments of the subject matter described herein), or any combination of one or more such back-end, middleware, or front-end components. Components of the system can be interconnected via digital information / data communication (e.g., a communication network) of any form or medium. Examples of communication networks include local area networks (“LANs”) and wide area networks (“WANs”), interconnected networks (e.g., the Internet) and peer-to-peer networks (e.g., self-organizing peer-to-peer networks).

[0194] A computing system may include clients and servers. Clients and servers are typically geographically separated and usually interact via a communication network. The client-server relationship is established by means of computer programs running on respective computers and having a client-server relationship with each other. In some embodiments, the server sends information / data (e.g., HTML pages) to the client device (e.g., for the purpose of displaying information / data to a user interacting with the client device and receiving user input from the user interacting with the client device). The server may receive information / data generated on the client device (e.g., the result of user interaction) from the client device.

[0195] While this specification contains numerous details of specific embodiments, these details should not be construed as limiting any embodiment or the scope of the claim, but rather as descriptions of features specific to particular embodiments. Specific features described herein in the context of separate embodiments may also be implemented in combination in a single embodiment. Conversely, various features described in the context of a single embodiment may also be implemented separately or in any suitable sub-combination in multiple embodiments. Furthermore, although features may be described above as functioning in a particular combination and even initially claimed in this way, in some cases, one or more features from the claimed combination may be removed from the combination, and the claimed combination may be for sub-combinations or variations thereof.

[0196] Similarly, although the operations are depicted in a specific order in the accompanying drawings, this should not be construed as requiring that these operations be performed in the specific order shown or sequentially, or that all of the shown operations be performed to achieve the desired result. In certain situations, multitasking and parallel processing may be advantageous. Furthermore, the separation of the various system components in the above embodiments should not be construed as requiring such separation in all embodiments, and it should be understood that the described program components and systems can generally be integrated together in a single software product or packaged into multiple software products.

[0197] Therefore, specific embodiments of the subject matter have been described. Other embodiments are within the scope of the claims. In some cases, the actions recited in the claims may be performed in a different order and still achieve the desired result. Furthermore, the processes depicted in the drawings do not necessarily require the specific order or sequential order shown to achieve the desired result. In certain embodiments, multitasking and parallel processing may be advantageous.

[0198] Benefiting from the teachings presented in the foregoing description and the associated accompanying drawings, those skilled in the art will conceive of many modifications and other embodiments of the disclosure set forth herein. Therefore, it should be understood that the embodiments are not limited to the specific embodiments disclosed, and that modifications and other embodiments are intended to be included within the scope of the appended claims. Although specific terminology is used herein, it is used only in a general and descriptive sense and not for limiting purposes.

Claims

1. An apparatus for providing storage resources, the apparatus comprising: The first controller is coupled to the network interface, and the first controller is capable of operating using a cache coherence protocol; A second controller, coupled to the first controller and coupled to the first memory, wherein the second controller performs at least one operation associated with the cache of the device; and A second memory is coupled to at least one of the first controller and the second controller. The device further includes a profiler that determines at least one capability associated with the device and provides the at least one capability to another device.

2. The apparatus of claim 1, wherein, The device performs the following operations: Receive configuration information from the host; The device is configured to operate as either a software-based cache or a hardware-based cache in an operation startup mode. Data is received via the network interface using the cache coherence protocol via the first controller; Perform at least one second operation on the data to generate second data, and The second data is stored in the second memory based on the cache coherence protocol.

3. The apparatus of claim 1, wherein, The cache coherence protocol includes the Fast Compute Link (CXL) protocol, and the other device includes a host.

4. The apparatus of claim 1, wherein, The device further includes a third controller coupled to a third memory, wherein the third memory includes non-volatile memory.

5. The apparatus of claim 1, wherein, The first memory includes a first volatile memory, and the second memory includes a second volatile memory.

6. The apparatus of claim 5, wherein, The first type of volatile memory includes double data rate memory or low-power double data rate memory.

7. The apparatus of claim 1, wherein, The first memory includes a first non-volatile memory, and the second memory includes a second non-volatile memory.

8. The apparatus of claim 7, wherein, The first non-volatile memory includes 3D flash memory.

9. The apparatus of claim 1, wherein, The device includes at least one of the following: M.2 shape factor, E1.L shape factor, E1.S shape factor, E3 S / L shape factor, U.2 shape factor, NF1 shape factor, full-height half-length FHHL insert card AIC shape factor, and half-height half-length HHHL AIC shape factor.

10. The apparatus of claim 1, wherein, The second controller includes at least one of a flow control module, a cache addressing module, and a cache policy module.

11. The apparatus of claim 1, wherein, The second controller includes at least one of a channel request queue, a volatile memory request scheduler, and a volatile memory command scheduler.

12. The apparatus of claim 1, wherein, The device also includes an accelerator or a network interface card (NIC).

13. A system for providing storage resources, the system comprising means, the means comprising: The first controller is coupled to the network interface, and the first controller is capable of operating using a cache coherence protocol; A second controller, coupled to the first controller and coupled to the first memory, wherein the second controller performs at least one operation associated with the cache of the device; and A second memory is coupled to at least one of the first controller and the second controller. The device further includes a profiler that determines at least one capability associated with the device and provides the at least one capability to another device.

14. The system of claim 13, wherein, The system performs the following operations: Data is received via the network interface using the cache coherence protocol via the first controller. Perform at least one second operation on the data to generate second data, and The second data is stored in the first memory or the second memory.

15. The system of claim 13, wherein, The cache coherence protocol includes the CXL protocol, and the other device includes a host.

16. The system of claim 13, wherein, The first memory includes volatile memory, and the second memory includes volatile memory.

17. The system of claim 16, wherein, The volatile memory includes double data rate memory or low-power double data rate memory.

18. The system of claim 13, wherein, The first memory includes non-volatile memory, and the second memory includes non-volatile memory.

19. A method performed by an apparatus for providing storage resources, the method comprising: A first controller coupled to a network interface operates the device according to a cache coherence protocol; A second controller, which is coupled to a first controller and a first memory, performs at least one operation on data associated with the cache of the device. and The data is stored in the device and coupled to a second memory of one of the first and second controllers. The method further includes: determining at least one capability associated with the device by a profiler of the device and providing the at least one capability to another device.

20. The method of claim 19, wherein, The method further includes: Receive configuration information from the host; In the startup time mode of operation, the second controller is operated as either a software-based cache or a hardware-based cache. The second data is received via the network interface using the cache coherence protocol via the first controller. Perform at least one second operation on the second data to generate the third data, and The third data is stored in the second memory based on the cache coherence protocol.

Images