System and method for sharing memory between clusters of directly connected nodes

Abstract

An exemplary system includes a cluster of nodes communicatively coupled to each other via at least one direct link and collectively including a plurality of memory devices. The example system also includes at least one system memory manager communicatively coupled to the cluster of nodes. In one example, the system memory manager is configured to allocate a plurality of sharable memory pools between memory devices. Various other systems, methods, and computer readable media are also disclosed.

Description

BACKGROUND OF THE DISCLOSURE

[0001] Memory is often one of the most expensive resources across data fabrics and / or node clusters. Scaling memory resources across data fabrics and / or node clusters can provide, support, and / or facilitate a number of improvements and / or advantages, such as improved performance, expanded service offerings, and / or increased storage potential. Unfortunately, the memory hardware costs required to do so are often prohibitively high and / or unwise. Accordingly, the present disclosure identifies and addresses the need for additional and improved systems and methods that facilitate the beneficial effects of memory scaling without incurring high hardware costs. BRIEF DESCRIPTION OF THE DRAWINGS

[0002] The drawings illustrate multiple exemplary embodiments and are part of the specification. Together with the following description, the drawings demonstrate and explain various principles of the present disclosure.

[0003] Figure 1 is a diagram of an exemplary system for sharing memory between clusters of directly connected nodes in accordance with one or more embodiments of the present disclosure.

[0004] Figure 2 is a diagram of an exemplary system for sharing memory between clusters of directly connected nodes in accordance with one or more embodiments of the present disclosure.

[0005] Figure 3 is a diagram of an exemplary system for sharing memory between clusters of directly connected nodes in accordance with one or more embodiments of the present disclosure.

[0006] Figure 4 is a diagram of an exemplary memory map that facilitates access to shared memory addresses and / or pools memory addresses in accordance with one or more embodiments of the present disclosure.

[0007] Figure 5 is a diagram of an exemplary system for sharing memory between clusters of directly connected nodes in accordance with one or more embodiments of the present disclosure.

[0008] Figure 6 is a flowchart of an exemplary method for sharing memory between clusters of directly connected nodes in accordance with one or more embodiments of the present disclosure.

[0009] In all the figures, the same reference numerals and descriptions indicate like but not necessarily identical elements. While the exemplary specific implementations described herein are susceptible to various modifications and alternative forms, specific specific implementations have been shown by way of example in the figures and will be described in detail herein. However, the exemplary specific implementations described herein are not intended to be limited to the particular forms disclosed. Rather, the present disclosure covers all modifications, equivalents, and alternatives falling within the scope of the appended claims. Detailed Description

[0010] The present disclosure describes various apparatuses, systems, and methods for sharing memory among clusters of directly connected nodes. As will be described in more detail below, multiple nodes can be clustered together by being directly connected to each other. In some examples, a cluster of nodes implements a memory lending and / or borrowing scheme that accommodates and / or normalizes the differential workloads among the memory resources of the nodes in the cluster of nodes. For example, one or more of the cluster nodes can share and / or pool memory blocks for use and / or access by other nodes in the cluster.

[0011] In some examples, at startup, each node scans its local memory range and broadcasts those local memory ranges to other nodes in the cluster. In one example, each node deploys and / or implements a system memory manager that constructs a memory map based on those broadcast memory ranges to support a shareable memory pool distributed among the clusters. In this example, the system memory manager and / or pool manager of each node can request to add data to, remove data from, and / or access data in the shareable memory pool distributed among the clusters. Such a memory lending and / or borrowing scheme can enable some nodes to utilize underutilized and / or available memory located on other nodes via the memory map. Under this scheme, for each cluster node, the lent and / or borrowed memory can appear the same as the local memory.

[0012] In some examples, a system includes a cluster of nodes communicatively coupled to each other via at least one direct link and collectively including a plurality of memory devices. In such examples, the system also includes at least one system memory manager communicatively coupled to the cluster of nodes. In one example, the system memory manager is configured to allocate a plurality of shareable memory pools among the memory devices.

[0013] In some examples, the system memory manager is configured to identify the workload of a node and / or allocate a shareable memory pool among memory devices at least in part based on the workload of the node. Additionally or alternatively, the system memory manager is configured to reserve at least a portion of the shareable memory pool for exclusive use by a first node included in the node.

[0014] In some examples, a first node included in a node hosts a shareable memory pool included in the shareable memory pool. In such examples, a second node included in the node is configured to access, via a direct link, a portion of the shareable memory pool at least in part because the second node is authorized to use at least a portion of the shareable memory pool connected to at least one application running on the second node. In one example, a third node included in the node is configured to access a shareable memory pool connected to an additional application running on the third node.

[0015] In some examples, a third node included in a node hosts an additional shareable memory pool included in the shareable memory pool. In such examples, the second node is configured to access the portion of the additional shareable memory pool at least in part because the second node is authorized to use at least a portion of the additional shareable memory pool connected to an application running on the second node.

[0016] In some examples, the system memory manager is configured to generate at least one memory map that includes an address range corresponding to the shareable memory pool so that a node can access the shareable memory pool. In one example, each node includes dedicated local memory that is not accessible to each other. In this example, the system memory manager is configured to generate a memory map to include an address range corresponding to the dedicated local memory normalized among the nodes.

[0017] In some examples, the direct link communicatively coupling the nodes includes a physical communication link that does not include a switch between the nodes. In one example, the system memory manager is configured to receive a memory management request from a remote device. In this example, the system memory manager is also configured to increase or decrease the amount of memory allocated to at least one shareable memory pool in the shareable memory pool in response to the memory management request.

[0018] In some examples, the system memory manager is configured to detect the addition of at least one memory device to multiple memory devices during node operation. Additionally or alternatively, the system memory manager is configured to detect the removal of at least one memory device from the multiple memory devices during node operation. In one example, the system memory manager is configured to reallocate a shareable memory pool among the memory devices to account for the addition or removal of memory devices during node operation. In another example, the system memory manager is configured to detect the addition or removal of memory devices via the Advanced Configuration and Power Interface (ACPI).

[0019] In some examples, the node includes a security processor that implements a memory fence across the shareable memory pool to ensure that attempts to access any shareable memory pool without authorization are denied. In one example, the system memory manager includes multiple system memory controllers that are implemented by the node and communicate with each other to coordinate the allocation of the shareable memory pool.

[0020] In some examples, the node scans the memory devices to find address ranges to broadcast to each other. In such examples, the node constructs a memory map based at least in part on the address ranges broadcast to each other. In one example, the system memory manager is centralized to manage the shareable memory pool.

[0021] In some examples, a method includes identifying, by at least one system memory manager, a range of memory addresses of multiple memory devices corresponding to a cluster of nodes. In such examples, the method includes allocating, by the system memory manager, multiple shareable memory pools among the memory devices based at least in part on the range of memory addresses. Additionally or alternatively, the method includes enabling, by the system memory manager, a remote node included in the node to access a portion of the shareable memory pool at least because the remote node is authorized to use the portion of the shareable memory pool hosted by a local node included in the node.

[0022] In some examples, the method also includes identifying, by the system memory manager, the workload of the node. In such examples, the method further includes allocating the shareable memory pool based at least in part on the workload of the node. In one example, the method additionally includes denying, by the system memory manager, a node access to a portion of the shareable memory pool at least because at least one node included in the node is unauthorized to use the portion of the shareable memory pool hosted by the local node.

[0023] In some examples, a non-transitory computer-readable medium includes one or more computer-executable instructions. In such examples, when executed by at least one processing device implementing at least one system memory manager, the computer-executable instructions cause the system memory manager to identify a range of memory addresses of a plurality of memory devices corresponding to a cluster of nodes. In one example, the computer-executable instructions also cause the system memory manager to allocate a plurality of shareable memory pools among the memory devices at least in part based on the range of memory addresses. In this example, the computer-executable instructions further cause the system memory manager to enable a remote node included in a node to access a portion of the shareable memory pool at least in part because the remote node is authorized to use a portion of the shareable memory pool hosted by a local node included in the node.

[0024] Reference will be made below to Figures 1 to 5 , for a detailed description of exemplary devices, systems, components, and / or corresponding specific implementations for sharing memory among clusters of directly-connected nodes. It will be described in conjunction with Figure 6 a detailed description of exemplary methods for sharing memory among clusters of directly-connected nodes.

[0025] Figure 1 An exemplary system 100 is illustrated that includes and / or characterizes a cluster of nodes 102(1) to 102(N) and / or one or more system memory managers 110(1) to 110(N). In some examples, nodes 102(1) to 102(N) are communicatively coupled to each other via at least one direct link 108. In such examples, nodes 102(1) to 102(N) respectively include and / or are attached to memory devices 104(1) to 104(N). In one example, system memory managers 110(1) to 110(N) are communicatively coupled to the cluster of nodes 102(1) to 102(N). In this example, system memory managers 110(1) to 110(N) allocate, assign, and / or distribute shareable memory pools 106(1) to 106(N) among memory devices 104(1) to 104(N).

[0026] In some examples, nodes 102(1) to 102(N) also respectively include and / or are attached to processing devices 114(1) to 114(N). In such examples, processing devices 114(1) to 114(N) respectively execute and / or initiate application programs 116(1) to 116(N).

[0027] In some examples, nodes 102(1) through 102(N) may each include and / or represent any type or form of computing device capable of performing computational tasks, facilitating communication, and / or sharing memory with other nodes in a cluster configuration. Examples of nodes 102(1) through 102(N) each include but are not limited to: network devices, servers, routers, switches, data fabric devices, data centers, host devices, client devices, laptops, tablets, desktop computers, personal computers, cellular phones, personal digital assistants (PDAs), multimedia players, embedded systems, wearable devices (e.g., smartwatches, smart glasses, etc.), gaming consoles, hubs, modems, bridges, repeaters, gateways, variations or combinations of one or more of the foregoing, portions of one or more of the foregoing, and / or any other suitable nodes.

[0028] In some examples, memory devices 104(1) through 104(N) may each include and / or represent any type or form of storage device connected to one or more applications and / or features installed and / or running on one or more nodes to maintain, store, hold, and / or buffer data. For example, memory devices 104(1) through 104(N) may each include and / or represent volatile and / or non-volatile storage devices and / or media capable of storing data and / or computer-readable instructions. In one example, memory devices 104(1) through 104(N) facilitate, support, and / or implement Compute Express Link (CXL) connections and / or interfaces for accessing and / or sharing data between nodes 102(1) through 102(N). In certain implementations, each of memory devices 104(1) through 104(N) constitutes and / or represents multiple discrete memory devices and / or components. Examples of memory devices 104(1) through 104(N) include but are not limited to: random access memory (RAM) devices, dynamic RAM (DRAM) devices, read-only memory (ROM) devices, flash memory devices, hard disk drives (HDDs), solid state drives (SSDs), CXL-compatible memory devices, optical disc drives, caches, main memories, variations or combinations of one or more of the foregoing, portions of one or more of the foregoing, and / or any other suitable memory devices.

[0029] In some examples, in addition to the shareable memory pools 106(1) through 106(N), the memory devices 104(1) through 104(N) also include and / or characterize a range of dedicated and / or only local memory. In such examples, the range of dedicated and / or only local memory is accessible to the corresponding node and / or inaccessible to remote nodes. For example, node 102(1) can access data in the dedicated local memory on memory device 104(1), but cannot access data in the dedicated local memory on memory device 104(N). Additionally or alternatively, node 102(N) can access data in the dedicated local memory on memory device 104(N), but cannot access data in the dedicated local memory on memory device 104(1). Thus, the dedicated local memory, the system memory managers 110(1) through 110(N), and / or the corresponding memory mappings can be configured to prevent unauthorized guests and / or hypervisors from accessing.

[0030] In some examples, the processing devices 114(1) through 114(N) may each include and / or characterize any type or form of hardware-implemented device capable of interpreting and / or executing computer-readable instructions. Examples of the processing devices 114(1) through 114(N) include, but are not limited to: central processing unit (CPU), graphics processing unit (GPU), parallel acceleration processor, microprocessor, multi-core processor, microcontroller, field-programmable gate array (FPGA), application-specific integrated circuit (ASIC), system-on-a-chip (SoC), variations or combinations of one or more of the foregoing, portions of one or more of the foregoing, and / or any other suitable processing device.

[0031] In some examples, the system memory managers 110(1) through 110(N) may each include and / or characterize any type or form of controller capable of allocating the shareable memory pools among the memory devices in the cluster nodes. In one example, the system memory managers 110(1) through 110(N) may be integrated in and / or implemented by the nodes 102(1) through 102(N). In this example, the system memory managers 110(1) through 110(N) communicate with each other to coordinate the allocation, release, expansion, and / or contraction of the shareable memory pools 106(1) through 106(N). In another example, a single centralized system memory manager manages and / or coordinates the allocation, release, expansion, and / or contraction of the shareable memory pools 106(1) through 106(N) among all of the nodes in the nodes 102(1) through 102(N). In certain embodiments, the system memory managers 110(1) through 110(N) for each node may include a baseboard management controller (BMC), a system firmware manager, and / or a system software manager.

[0032] In some examples, the system memory managers 110(1) through 110(N) may include and / or represent one or more portions of hardware devices, firmware, and / or software. In one example, one or more of the system memory managers 110(1) through 110(N) may be implemented as a stand-alone computing device. Additionally or alternatively, one or more of the system memory managers 110(1) through 110(N) may be implemented by a computing device (e.g., nodes 102(1) through 102(N)) that performs additional computing tasks and / or communication operations.

[0033] In some examples, the shareable memory pools 106(1) through 106(N) may each include and / or represent ranges of memory addresses that have been allocated for sharing among the nodes 102(1) through 102(N) of the cluster. In one example, these ranges of memory addresses may constitute and / or represent an allocation that the nodes 102(1) through 102(N) can reserve and / or share, at least in part, based on their respective workload requirements. In certain implementations, the shareable memory pools 106(1) through 106(N) may include and / or represent a homogeneous distribution and / or amount of memory among the memory devices 104(1) through 104(N). In other implementations, the shareable memory pools 106(1) through 106(N) may include and / or represent a heterogeneous distribution and / or amount of memory.

[0034] In some examples, the direct link 108 includes and / or represents a physical cable and / or connection between nodes 102(1) and 102(N). In one example, the direct link 108 does not include and / or omits switches and / or intermediate devices between nodes 102(1) and 102(N). Examples of the direct link 108 include, but are not limited to: fiber optic cables, Ethernet cables, coaxial cables, twisted pair cables, cable wires, network cables, variations or combinations of one or more of the foregoing, and / or any other suitable direct link.

[0035] In some examples, system memory managers 110(1) through 110(N) measure, determine, and / or identify the workloads of nodes 102(1) through 102(N), memory devices 104(1) through 104(N), and / or processing devices 114(1) through 114(N). In these examples, a workload can constitute and / or characterize the amount of resources, processing, and / or time required to execute one or more scheduled computing tasks on nodes 102(1) through 102(N). For example, the amount of memory required to run one or more applications, processes, programs, and / or virtual machines on node 102(1) can be added to and / or counted towards the workload of node 102(1). Additionally or alternatively, the amount of memory required to run one or more applications, processes, programs, and / or virtual machines on node 102(N) can be added to and / or counted towards the workload of node 102(N).

[0036] In some examples, the workload requirements of nodes 102(1) through 102(N) can consider and / or involve the latency, bandwidth, memory capacity, and / or termination of virtual machines or threads. In one example, system memory managers 110(1) through 110(N) allocate shareable memory pools 106(1) through 106(N) among memory devices 104(1) through 104(N) at least in part based on the workloads of nodes 102(1) through 102(N). In a specific example, the entire working set of a node's memory requirements can be implemented and / or placed in one or more of shareable memory pools 106(1) through 106(N).

[0037] In some examples, system memory managers 110(1) through 110(N) detect, discover, and / or sense topology changes within the cluster (e.g., links in an isolated, disconnected, and / or powered-off state). Additionally or alternatively, system memory managers 110(1) through 110(N) initiate and / or perform updates to the memory mappings of nodes 102(1) through 102(N).

[0038] Figure 2 Illustrating exemplary system 200, the exemplary system includes and / or characterizes a cluster of nodes 102(1), 102(2), 102(3), and 102(4) communicatively coupled to each other via direct links 108(1), 108(2), 108(3), 108(4), 108(5), and 108(6). In some examples, system 200 can include and / or characterize certain components and / or features that perform and / or provide the same functions as those described above in connection with Figure 1Described functional similarities and / or identical functionality. In one example, node 102(1) includes and / or represents memory device 104(1) and system memory manager 110(1), and node 102(2) includes and / or represents memory device 104(2) and system memory manager 110(2). In this example, node 102(3) includes and / or represents memory device 104(3) and system memory manager 110(3), and node 102(4) includes and / or represents memory device 104(4) and system memory manager 110(4).

[0039] In some examples, memory device 104(1) includes and / or hosts shareable memory pool 106(1) and dedicated local memory 206(1), and memory device 104(2) includes and / or hosts shareable memory pool 106(2) and dedicated local memory 206(2). In such examples, memory device 104(3) includes and / or hosts shareable memory pool 106(3) and dedicated local memory 206(3), and memory device 104(4) includes and / or hosts shareable memory pool 106(4) and dedicated local memory 206(4).

[0040] In some examples, system memory managers 110(1) through 110(4) may each reserve portions (e.g., certain address ranges) of shareable memory pools 106(1) through 106(4) for exclusive use by one of nodes 102(1) through 102(4) and / or a corresponding application. For example, system memory managers 110(1) through 110(N) may provide, support, and / or facilitate memory fences across shareable memory pools 106(1) through 106(4) to ensure that all attempts by unauthorized nodes and / or applications to access such portions of shareable memory pools 106(1) through 106(4) are denied. In one example, such access attempts may include and / or represent any type or form of read, write, and / or update operation. Additionally or alternatively, such access attempts may be performed in conjunction with one or more applications running on nodes 102(1) through 102(4).

[0041] In some examples, node 102(1) may access at least a portion of shareable memory pool 106(2) via direct link 108(1), access at least a portion of shareable memory pool 106(3) via direct link 108(3), and / or access at least a portion of shareable memory pool 106(4) via direct link 108(4). Additionally or alternatively, node 102(2) may access at least a portion of shareable memory pool 106(1) via direct link 108(1), access at least a portion of shareable memory pool 106(3) via direct link 108(2), and / or access at least a portion of shareable memory pool 106(4) via direct link 108(5).

[0042] In some examples, node 102(3) may access at least a portion of shareable memory pool 106(1) via direct link 108(3), access at least a portion of shareable memory pool 106(2) via direct link 108(2), and / or access at least a portion of shareable memory pool 106(4) via direct link 108(6). Additionally or alternatively, node 102(4) may access at least a portion of shareable memory pool 106(1) via direct link 108(4), access at least a portion of shareable memory pool 106(2) via direct link 108(5), and / or access at least a portion of shareable memory pool 106(3) via direct link 108(6).

[0043] In some examples, if a memory fence is implemented, a successful access attempt to shareable memory pools 106(1) to 106(4) may occur and / or be generated at least in part because the node is authorized to use that portion of shareable memory pools 106(1) to 106(4). In one example, portions within each of shareable memory pools 106(1) to 106(4) may be allocated for use by a specific node and / or restricted from use by a specific node. For example, nodes 102(1) and 102(2) may be authorized to access one or more portions of shareable memory pools 106(1) to 106(4) that are not accessible to nodes 102(3) and 102(4).

[0044] In some examples, the system memory managers 110(1) to 110(4) generate, create, and / or build a memory map that includes address ranges corresponding to the shareable memory pools 106(1) to 106(4) and / or address ranges corresponding to the dedicated local memory of the nodes in question. For example, the nodes 102(1) to 102(4) and / or the system memory managers 110(1) to 110(4) scan the memory devices 104(1) to 104(4) respectively to find the address ranges broadcast to each other, with the aim of building the corresponding memory map and / or allocating address ranges for the shareable memory pools 106(1) to 106(4). In this example, the nodes 102(1) to 102(4) and / or the system memory managers 110(1) to 110(4) receive the broadcast memory ranges and then build their own memory maps that are consistent with each other. These memory maps may appear the same to each other with respect to the addresses of the shareable memory pool and the dedicated local memory, but the addresses of the dedicated local memory in each memory map will only correspond to the nodes and / or memory devices in question.

[0045] As a specific example, the system memory managers 110(1) to 110(4) may generate, create, and / or build a memory map that includes address ranges corresponding to the shareable memory pools 106(1) to 106(4) such that the nodes 102(1) to 102(4) can access the shareable memory pools 106(1) to 106(4). In this example, the address ranges are statically partitioned across the nodes 102(1) to 102(N). Additionally, these memory maps may each include and / or characterize address ranges corresponding to the dedicated local memory. In one example, for each memory map in the cluster, the address ranges corresponding to the dedicated local memory are normalized to zero. Thus, the dedicated local memory may start at the same memory address (e.g., zero) in each memory map of the nodes 102(1) to 102(N).

[0046] In some examples, one or more of the system memory managers 110(1) to 110(4) may receive a memory management request from a remote device (e.g., a device external to the cluster). In one example, one or more of the system memory managers 110(1) to 110(4) may increase and / or decrease the amount of memory allocated to the shareable memory pools 106(1) to 106(4) in response to the memory management request.

[0047] Additionally or alternatively, one or more of the system memory managers 110(1) to 110(4) may detect and / or discover that new memory devices have been added to the cluster during the operation or startup of one or more of the nodes 102(1) to 102(4). Similarly, one or more of the system memory managers 110(1) to 110(4) may detect and / or discover that one of the memory devices 104(1) to 104(4) has been removed from the cluster during the operation or startup of one or more of the nodes 102(1) to 102(4). In one example, one or more of the system memory managers 110(1) to 110(4) may detect and / or discover the addition or removal of a memory device via the Advanced Configuration and Power Interface (ACPI). In some embodiments, one or more of the system memory managers 110(1) to 110(4) may reallocate the shareable memory pool among the memory devices to account for the addition or removal of a memory device during the operation or startup of one or more of the nodes 102(1) to 102(4).

[0048] Figure 3 Illustrative exemplary system 300, which includes and / or characterizes a cluster of nodes 102(1) to 102(4) communicatively coupled to each other via direct links 108(1) to 108(6). In some examples, system 300 may include and / or characterize certain components and / or features that perform and / or provide functionality similar and / or identical to the functionality described above in connection with Figure 1 and Figure 2 any one of them. In one example, system 300 includes and / or characterizes a system memory manager 110 communicatively coupled to nodes 102(1) to 102(4). In this example, the system memory manager 110 serves as a single centralized unit for managing, allocating, and / or releasing the shareable memory pools 106(1) to 106(4) for the entire cluster, rather than each node implementing its own system memory manager. In certain embodiments, the system memory manager 110 may issue and / or send Intelligent Platform Management Interface (IPMI) requests to establish the shared memory configuration of the cluster and / or facilitate the distribution to nodes 102(1) to 102(4).

[0049] In some examples, the system memory manager 110 generates, creates, and / or constructs a memory map that includes address ranges corresponding to the shareable memory pools 106(1) through 106(4) and / or address ranges corresponding to the dedicated local memory of the nodes 102(1) through 102(4). For example, the nodes 102(1) through 102(4) and / or the system memory manager 110 may scan the memory devices 104(1) through 104(N) to find the address ranges for constructing the corresponding memory map and / or allocate address ranges for the shareable memory pools 106(1) through 106(4). In this example, the system memory manager 110 compiles the memory ranges scanned from the memory devices 104(1) through 104(4) and then constructs a memory map for distribution to the nodes 102(1) through 102(4). In one example, the same memory map is implemented, applied, and / or used by each of the nodes 102(1) through 102(4). In this example, the addresses of the dedicated local memory in the memory map correspond only to the node and / or memory device in question.

[0050] In some examples, the system memory manager 110 may receive a memory management request from a remote device 302. In such examples, the system memory manager 110 then makes decisions regarding the system memory requirements of each node. In one example, the system memory manager 110 may increase and / or decrease the amount of memory allocated to the shareable memory pools 106(1) through 106(4) in response to the memory management request.

[0051] Figure 4 Illustrating an exemplary memory map 400 that includes and / or characterizes local memory ranges 416 and / or shareable memory ranges 402(1), 402(2), 402(3), and / or 402(4). In some examples, the memory map 400 is distributed and / or provided to each of the nodes 102(1) through 102(4) by Figure 3 the system memory manager 110 in. In other examples, each of the system memory managers 110(1) through 110(4) in Figure 2 generates, creates, and / or constructs the memory map 400 based at least in part on the memory addresses scanned by the nodes 102(1) through 102(4).

[0052] In some examples, the local memory range 416 includes and / or represents memory addresses 404(1) through 404(N). In one example, the local memory range 416 is normalized to zero. In other words, the memory addresses 404(1) through 404(N) start at zero and move upward. As a specific example, the local memory range 416 may include and / or represent approximately 512 gigabytes of memory and / or data.

[0053] In some examples, the shareable memory range 402(1) includes and / or represents memory addresses 406(1) through 406(N). In one example, the shareable memory range 402(1) corresponds to and / or is located in memory device 104(1) and / or shareable memory pool 106(1). In this example, the shareable memory range 402(1) is after the local memory range 416 and / or before the shareable memory range 402(2) in the memory map 400. As a specific example, the shareable memory range 402(1) may include and / or represent approximately 512 gigabytes of memory and / or data.

[0054] In some examples, the shareable memory range 402(2) includes and / or represents memory addresses 408(1) through 408(N). In one example, the shareable memory range 402(2) corresponds to and / or is located in memory device 104(2) and / or shareable memory pool 106(2). In this example, the shareable memory range 402(2) is after the shareable memory range 402(1) and / or before the shareable memory range 402(3) in the memory map 400. As a specific example, the shareable memory range 402(2) may include and / or represent approximately 512 gigabytes of memory and / or data.

[0055] In some examples, the shareable memory range 402(3) includes and / or represents memory addresses 410(1) through 410(N). In one example, the shareable memory range 402(3) corresponds to and / or is located in memory device 104(3) and / or shareable memory pool 106(3). In this example, the shareable memory range 402(3) is after the shareable memory range 402(2) and / or before the shareable memory range 402(4) in the memory map 400. As a specific example, the shareable memory range 402(3) may include and / or represent approximately 512 gigabytes of memory and / or data.

[0056] In some examples, the shareable memory range 402(4) includes and / or characterizes memory addresses 412(1) through 412(N). In one example, the shareable memory range 402(4) corresponds to and / or is located in memory device 104(4) and / or shareable memory pool 106(4). In this example, the shareable memory range 402(4) follows the shareable memory range 402(3) and / or terminates the memory map 400. As a specific example, the shareable memory range 402(4) may include and / or characterize approximately 512 gigabytes of memory and / or data.

[0057] Figure 5 Illustrative exemplary system 500 includes and / or characterizes a deconstructed, abstracted, and / or flattened form of a cluster of directly connected nodes capable of sharing memory with each other. In some examples, system 500 may include and / or characterize certain components and / or features that perform and / or provide functionality similar and / or identical to the functionality described above in connection with Figures 1 to 4 any of the foregoing. In one example, system 500 includes and / or characterizes a memory allocation layer 502 that generates and / or transmits a new memory allocation for each node based at least in part on the requirements of applications running on nodes 102(1) through 102(4) and / or the amount of memory required by virtual machines running on nodes 102(1) through 102(4).

[0058] In some examples, system 500 also includes and / or characterizes a system memory manager 110 communicatively coupled to the memory allocation layer 502. In one example, system memory manager 110 may include and / or characterize a fabric manager that transmits the new memory allocations to host software 504(1), 504(2), 504(3), and 504(4), which respectively correspond to and / or run on nodes 102(1) through 102(4). Additionally or alternatively, the fabric manager may adjust and / or modify certain fabric settings.

[0059] In some examples, host software 504(1) through 504(4) may invoke and / or apply hot-add and / or hot-remove features to dynamically increase and / or decrease the amount of memory allocated to hosts and / or virtual machines running on nodes 102(1) through 102(4). In one example, host software 504(1) through 504(4) may directly invoke and / or boot security processors 506(1), 506(2), 506(3), and / or 506(4) and / or a root of trust (RoT) device, respectively, to adjust and / or modify the memory size allocated to the host and / or virtual machine. In certain embodiments, security processors 506(1) through 506(4) and / or the RoT device may be used by nodes 102(1) through 102(4) to ensure exclusive access to certain memory ranges in shared memory pools 106(1) through 106(4) in a memory fencing scheme.

[0060] In some examples, a node and / or a system memory manager responsible for lending and / or donating memory to another node in a cluster may enable memory fencing for incoming attempts to access such memory. In such examples, incoming attempts to access the lent and / or donated memory must reach a specific address range with authorization to succeed and / or complete.

[0061] In some examples, in combination with Figures 1 to 5 the various systems and / or devices described may include and / or characterize one or more additional components, devices, and / or features that are not necessarily shown and / or labeled in Figures 1 to 5 . In such examples, one or more of these additional components, devices, and / or features may be inserted and / or applied between any of the components and / or devices illustrated in Figures 1 to 5 that is consistent with the purposes and / or objectives provided herein. Thus, one or more of the communication couplings and / or electrical couplings described with reference to Figures 1 to 5 may be a direct connection without intermediate components, devices, and / or nodes or an indirect connection with one or more intermediate components, devices, and / or nodes.

[0062] In some examples, the term "coupled" as used herein may refer to a direct connection and / or an indirect connection. For example, a direct communication coupling between two components may constitute and / or characterize a coupling in which those two components are directly connected to each other to achieve communication continuity from one of those two components to the other. In other words, a direct coupling may not include and / or omit any additional components between those two components.

[0063] Additionally or alternatively, an indirect communication coupling between two components may constitute and / or characterize a coupling in which the two components are indirectly connected to each other through one or more intermediate devices to achieve electrical continuity from one of the two components to the other. In other words, an indirect coupling may include and / or incorporate at least one additional component between the two components.

[0064] Figure 6 is a flowchart of an exemplary method 600 for sharing memory between clusters of directly connected nodes. In one example, the steps shown may be performed and / or executed during the process and / or program of starting to share memory between cluster nodes. Additionally or alternatively, Figure 6 the steps shown may also incorporate and / or involve various sub-steps and / or variations consistent with the description provided above in connection with Figure 6 the steps shown may also incorporate and / or involve various sub-steps and / or variations consistent with the description provided above in connection with Figures 1 to 5 the description provided above.

[0065] As Figure 6 illustrated, exemplary method 600 includes and / or involves a step (610) of identifying a range of memory addresses of a plurality of memory devices corresponding to a plurality of nodes. Step 610 may be performed in various ways, including any of the ways described above in connection with Figures 1 to 5 the description provided above. For example, at least one system memory manager identifies a range of memory addresses of a plurality of memory devices corresponding to a plurality of nodes.

[0066] Exemplary method 600 also includes a step (620) of allocating a plurality of sharable memory pools among the memory devices based at least in part on the range of memory addresses. Step 620 may be performed in various ways, including any of the ways described above in connection with Figures 1 to 5 the description provided above. For example, the system memory manager allocates a plurality of sharable memory pools among the memory devices based at least in part on the range of memory addresses.

[0067] Exemplary method 600 further includes a step of enabling a remote node included in a node to access a portion of the sharable memory pool at least in part because the remote node is authorized to use a portion of the sharable memory pool hosted by a local node included in the node (630). Step 630 may be performed in various ways, including any of the ways described above in connection with Figures 1 to 5 the description provided above. For example, the system memory manager enables a remote node included in a node to access a portion of the sharable memory pool at least in part because the remote node is authorized to use a portion of the sharable memory pool hosted by a local node included in the node.

[0068] Although the foregoing disclosure has illustrated various specific implementations using specific block diagrams, flowcharts, and examples, each block diagram component, flowchart step, operation, and / or component described and / or shown herein can be implemented individually and / or jointly using a variety of hardware, software, or firmware (or any combination thereof) configurations. Additionally, any disclosure of components contained within other components should be considered exemplary in nature, as many other architectures can be implemented to achieve the same functionality. Furthermore, the various steps, events, and / or features performed by such components should be considered exemplary in nature, as many alternatives and / or variations that achieve the same functionality are possible within the scope of this disclosure.

[0069] The devices, systems, and methods described herein can be implemented using any number of software, firmware, and / or hardware configurations. For example, one or more of the exemplary specific implementations disclosed herein can be encoded as a computer program (also referred to as computer software, software application, computer-readable instructions, and / or computer control logic) on a computer-readable medium. In one example, when executed by at least one processor, the encoding of the computer-readable medium causes the processor to generate and / or produce a computer-readable representation of an integrated circuit configured to perform, execute, and / or carry out any of the tasks, features, and / or actions described herein in connection with Figures 1 to 6 the description. The term "computer-readable medium" generally refers to any form of device, carrier, or medium capable of storing or carrying computer-readable instructions. Examples of computer-readable media include, but are not limited to, transmission-type media such as carrier waves and non-transitory-type media such as magnetic storage media (e.g., hard disk drives and floppy disks), optical storage media (e.g., compact discs (CDs) and digital video discs (DVDs)), electronic storage media (e.g., solid-state drives and flash media), and / or other distribution systems.

[0070] The order of process parameters and steps described and / or illustrated herein is given by way of example only and can vary as needed. For example, although the steps illustrated and / or described herein are shown or discussed in a particular order, these steps do not necessarily need to be performed in the order illustrated or discussed. The various exemplary methods described and / or illustrated herein can also omit one or more of the steps described or illustrated herein, or include additional steps other than those disclosed.

[0071] The foregoing description has been provided to enable other technicians in the art to best utilize the various aspects of the exemplary embodiments disclosed herein. This exemplary description is not intended to be exhaustive or limited to any precise form. Many modifications and variations are possible without departing from the spirit and scope of the disclosure. The embodiments disclosed herein should be considered illustrative in all respects and not restrictive. In determining the scope of the disclosure, reference should be made to the appended claims and their equivalents.

[0072] Unless otherwise indicated, as used in the specification and claims, the terms "connected to" and "coupled to" (and their derivatives) will be regarded as allowing both direct and indirect (i.e., via other elements or components) connections. Additionally, as used in the specification and claims, the term "a" or "an" will be regarded as meaning "at least one". Finally, for ease of use, the terms "comprising" and "having" (and their derivatives) used in the specification and claims may be interchanged with the word "including" and have the same meaning.

Claims

1. A system, the system comprising: A cluster of nodes, the nodes being communicatively coupled to each other via at least one direct link and collectively including a plurality of memory devices; And At least one system memory manager communicatively coupled to the cluster of nodes, the system memory manager being configured to allocate a plurality of shareable memory pools among the memory devices.

2. The system according to claim 1, wherein, The system memory manager is configured to: Identify the workload of the nodes; and Allocate the shareable memory pools among the memory devices at least in part based on the workload of the nodes.

3. The system according to claim 2, wherein The system memory manager is configured to reserve at least a portion of the shareable memory pools for exclusive use by a first node included in the nodes.

4. The system according to claim 1, wherein: A first node included in the nodes hosts a shareable memory pool included in the shareable memory pools; and A second node included in the nodes is configured to access a portion of the shareable memory pool via the direct link at least in part because the second node is authorized to use at least a portion of the shareable memory pool connected to at least one application running on the second node.

5. The system according to claim 4, wherein, A third node included in the nodes is configured to access the shareable memory pool connected to an additional application running on the third node.

6. The system according to claim 4, wherein: A third node included in the nodes hosts an additional shareable memory pool included in the shareable memory pools; and The second node is configured to access a portion of the additional shareable memory pool at least in part because the second node is authorized to use at least a portion of the additional shareable memory pool connected to the application running on the second node.

7. The system according to claim 1, wherein The system memory manager is configured to generate at least one memory map, the at least one memory map including an address range corresponding to the shareable memory pools, so that the nodes can access the shareable memory pools.

8. The system according to claim 7, wherein: The nodes each include dedicated local memories that are inaccessible to each other; and the system memory manager is configured to generate the memory map to include an address range corresponding to the dedicated local memories normalized among the nodes.

9. The system according to claim 1, wherein, The direct link communicatively coupling the nodes to each other includes a physical communication link that does not include a switch between the nodes.

10. The system according to claim 1, wherein, The system memory manager is configured to: Receive a memory management request from a remote device; and In response to the memory management request, increase or decrease the amount of memory allocated to at least one of the shareable memory pools in the shareable memory pools.

11. The system according to claim 1, wherein, The system memory manager is configured to perform at least one of the following: Detect the addition of at least one memory device to the plurality of memory devices during operation of the nodes; or Detect the removal of at least one of the plurality of memory devices during operation of the node.

12. The system according to claim 11, wherein, The system memory manager is configured to reallocate the shareable memory pool among the memory devices to account for the addition or removal of the memory devices during operation of the node.

13. The system according to claim 11, wherein The system memory manager is configured to detect the addition or removal of the memory devices via an Advanced Configuration and Power Interface (ACPI).

14. The system according to claim 1, wherein, The node includes a security processor configured to implement a memory fence across the shareable memory pool to ensure that attempts to access any of the shareable memory pools without authorization are denied.

15. The system according to claim 1, wherein, The system memory manager includes a plurality of system memory controllers implemented by the node and communicating with each other to coordinate the allocation of the shareable memory pool.

16. The system according to claim 15, wherein The node is configured to: Scan the memory devices for address ranges to be broadcast to each other; and build a memory map at least in part based on the address ranges broadcast to each other.

17. The system according to claim 1, wherein The system memory manager includes a centralized memory manager that manages the shareable memory pool.

18. A method, the method comprising: Identifying, by at least one system memory manager, a range of memory addresses of a plurality of memory devices corresponding to a cluster of nodes; Allocating, by the system memory manager, a plurality of shareable memory pools among the memory devices at least in part based on the range of memory addresses; and And Causing, by the system memory manager, a remote node included in the node to be able to access a portion of the shareable memory pool at least because the remote node is authorized to use a portion of the shareable memory pool hosted by a local node included in the node.

19. The method according to claim 18, the method further comprising identifying, by the system memory manager, the workload of the node; and Among them, Allocating the shareable memory pool among the memory devices includes allocating the shareable memory pool at least in part based on the workload of the node.

20. A non-transitory computer-readable medium, the non-transitory computer-readable medium including one or more computer-executable instructions that, when executed by at least one processing device implementing at least one system memory manager, cause the system memory manager to: Identify a range of memory addresses of a plurality of memory devices corresponding to a cluster of nodes; allocate a plurality of shareable memory pools among the memory devices at least in part based on the range of memory addresses; and Cause a remote node included in the node to be able to access a portion of the shareable memory pool at least because the remote node is authorized to use a portion of the shareable memory pool hosted by a local node included in the node.

Images