Split memory pool allocation

Abstract

Examples involving a split memory pool are disclosed. One example provides a memory system including a memory controller and a memory attached to the memory controller and forming at least a portion of a split memory pool, the split memory pool including a plurality of slices that are each dynamically allocated to a respective compute node. The memory system is configured to receive a request to adjust an allocation of the memory pool to a requesting compute node, allocate at least a portion of an unallocated portion of the memory pool to the requesting compute node in an instance in which the portion of the memory pool includes an unallocated slice that can satisfy the request, and cause the request to be directed to another compute node to release at least one slice to the another compute node in an instance in which the portion of the memory pool does not include an unallocated slice that can satisfy the request.

Description

Background Technology

[0001] Data centers typically consist of a large number of discrete computing nodes, such as server computers or other suitable computing devices. Such devices can work independently and / or collaboratively to meet a variety of computing workloads. Attached Figure Description

[0002] Figure 1 Multiple compute nodes are schematically depicted and communicatively coupled to a discrete memory pool.

[0003] Figure 2 An example list depicting slices of a split memory pool.

[0004] Figure 3 A flowchart is shown of a method for bringing one or more slices of a split memory pool online for performing memory transactions.

[0005] Figure 4 An example physical address mapping is shown.

[0006] Figure 5 A flowchart illustrating a method for allocating slices of a split memory pool to compute nodes is shown.

[0007] Figure 6 A flowchart illustrating a method for deallocating slices of a split memory pool allocated to a compute node is shown.

[0008] Figure 7 A block diagram of an example computing system is shown. Detailed Implementation

[0009] As explained above, data centers typically comprise a large number of discrete computing nodes, such as server computers or other suitable computing devices. These computing nodes can be referred to as "host computing devices" or "hosts" because, in some cases, they can be used to host virtual machines. However, it should be understood that computing nodes can be used for any suitable computing purpose and do not need to be specifically designed for hosting virtual machines. Furthermore, in some examples, computing nodes can be implemented as virtual machines.

[0010] Depending on the specific implementation, each individual compute node can have any suitable set of computer hardware. For example, a traditional server can be largely self-sufficient, including processing resources, data storage, volatile / non-volatile memory, network interface components, power supply, cooling solutions, etc. In contrast, some "blade servers" or rack-mount systems omit internal power supplies, cooling systems, and / or network interfaces, instead relying on the enclosure / rack structure to provide this type of infrastructure functionality for the compute node cluster.

[0011] Regardless, each individual compute node typically includes a set of local / native hardware resources, including data storage, memory, processing resources, etc. However, compute workloads (e.g., those associated with data center customers) are often unevenly distributed across compute nodes in a data center. Instead, in a common scenario, a subset of compute nodes in a data center may be assigned resource-intensive workloads, while other nodes are idle or processing relatively few resource-intensive tasks. Therefore, the overall resource utilization of the data center may be relatively low, and because such workloads are localized to individual nodes with limited resources, the completion of other resource-intensive workloads may be resource-constrained. In other cases, compute nodes are equipped with only memory and / or other resources that substantially exceed the statistically normal operating requirements. These scenarios represent inefficient use of available compute resources, sometimes referred to as "resource stifling," because native resources that could potentially be applied to non-native workloads are instead left idle or underutilized on hosts.

[0012] This problem can be mitigated when hardware resources are removed from a single compute node and instead fragmented into disaggregated resource pools that can be flexibly accessed by multiple attached compute nodes. For example, volatile memory hardware (e.g., random access memory (RAM)) can be collected as part of a disaggregated memory pool that can be used by any of multiple compute nodes (e.g., in a data center). This service helps alleviate resource staging because compute nodes can freely request memory when needed and release it when no longer required.

[0013] In some examples, the amount of memory collectively allocated to multiple compute nodes may exceed the amount of memory actually provided in a discrete memory pool. This is sometimes referred to as "thin provisioning." Generally, in data center environments without thin provisioning, it can be observed that over time, a single compute node (and / or the virtual machines implemented on that compute node) is statistically configured with more resources (e.g., storage space, memory) than the compute node ultimately uses. For example, in most cases, the amount of memory installed for a particular compute node may be significantly higher than the amount of memory actually used by the compute node. When combined across multiple compute nodes, the average amount of free memory (or other resources) can represent a large portion of the total memory in the data center.

[0014] In an example scenario without thin provisioning, the split memory pool can include a total of 1TB (1024GB) of memory, which can be evenly distributed across eight compute nodes. Additionally, each compute node can include 128GB of natively attached memory. Therefore, each compute node can be allocated 128GB of memory from the split memory pool, while having a total of 256GB of provisioned memory between the natively attached memory and the pooled memory. In total, also between the natively attached memory and the pooled memory, the eight compute nodes can access a total of 2TB of memory. In this example, due to the 128GB of native memory and 128GB of pooled memory, from the perspective of the operating system (OS) running on the node and memory system, each node is allocated 256GB of memory. That is, 256GB of available memory is "visible" to the nodes.

[0015] However, it is unlikely that every compute node will fully utilize its memory allocation, especially since the system is not thinly configured. Instead, in more common scenarios, each compute node may only use 50% of its allocated memory during normal use, and some compute nodes may use significantly less than 50%. Therefore, even if a 1TB discrete memory pool is fully allocated to multiple compute nodes, only a relatively small portion of the pooled memory may be used at any given time. This means inefficient use of available resources.

[0016] With this in mind, the amount of actual available memory (i.e., the amount of memory "pre-allocated" in the memory pool) can be reduced without significantly impacting the performance of multiple compute nodes. For example, the memory space for each compute node could still be configured such that its pooled portion of memory allocation is 128GB (thus reaching 1TB when summed over eight nodes), for example, by providing a 128GB address range of remote memory; however, the memory pool could actually be equipped with only a total of 256GB. Thus, the amount of allocated memory exceeds the amount of memory actually provided, but the OS and applications running on the compute nodes will be aware of and able to use 256GB, including the 128GB of native memory attached to the compute node and the 128GB of remote memory from the memory pool as needed.

[0017] In this example, while each specific compute node is allocated 128GB of pooled memory, statistically, many compute nodes may not use all or even most of that 128GB at any given time. Therefore, any unused memory allocated to a compute node can be reallocated to one or more other nodes. In this way, any particular compute node can choose to use up to 128GB of pooled memory if needed, while still retaining memory in the separate pool, since each compute node typically does not use 128GB at any given time. Therefore, "allocation" as used herein can refer to the amount or region of memory visible and accessible to a compute node and the software executing on it, where the overall allocation can come from local, locally attached, and remote pooled memory. As mentioned above, if the memory pool is evenly distributed across compute nodes, the amount of memory allocated from the memory pool to each of the multiple compute nodes may be greater than the amount of physical memory available from the memory pool to each compute node.

[0018] Thin provisioning represents a potential incentive to adjust how discrete memory pools are allocated to compute nodes. Adjustments to memory pool allocation may be desirable in other situations, including those where thin provisioning is not adopted—for example, compute nodes may need to be provided with additional memory to perform workloads requiring more memory than currently allocated to the nodes, even if the allocated amount equals the amount actually available to the nodes.

[0019] One method of increasing memory allocation to a compute node simply involves attaching additional physical memory to the node. An example of being able to add physical memory to a compute node during node operation is called "hot-plugging," where memory physically added during node operation is called "hot-added" memory, and memory physically removed during node operation is called "hot-removed" memory. As an example, these operations might involve physically inserting or removing a dual in-line memory module (DIMM) from the motherboard. Hot-plugging also supports mechanisms for logically making memory "online" (i.e., making the memory available / usable to the node after installation) and "offline" (i.e., making the installed memory unavailable / unusable to the node).

[0020] However, hot-swappable memory management methods have various drawbacks. For example, the hot-swappable protocol supported by the Advanced Configuration and Power Interface (ACPI) standard is based on allocating physical memory devices to nodes. This protocol specifies that the entire memory device (e.g., a memory expansion board) and therefore all the memory provided by that memory device are attached to a single host. While this allows for flexible allocation of the entire memory provided by the memory device to different nodes via online / offline connections, it does not allow for the allocation of smaller, more granular memory. Depending on the memory provided by the memory device, a large amount of memory (e.g., tens of gigabytes) can be allocated to the host. This larger allocation of memory increases memory fragmentation and significantly limits the ability to share pooled memory across multiple nodes.

[0021] Hot-plugging examples can introduce additional drawbacks. For instance, existing mechanisms for detecting hot-addition / removal of memory are based on custom hardware circuitry. This requires custom firmware to interface with ACPI. Consequently, the operating system needs to interact with the system firmware during hot-addition / removal. Reliability and service issues can lead to implementations that are not transparent to the firmware. Furthermore, hot-plugging implementations may require custom hardware, firmware, and drivers to support different central processing unit (CPU) architectures, increasing cost and complexity. In summary, existing hot-plugging-based technologies are limited in their ability to dynamically allocate, manage, and share memory across nodes.

[0022] Some methods allow physical memory devices to be partitioned into logical devices. For example, the mechanism based on the Peripheral Component Interconnect Fast (PCIe) Plug and Play framework allows physical memory devices to be partitioned into logical devices, but requires additional hardware such as (multiple) Compute Fast Link (CXL) switches to present the logical devices as physical devices. Introducing (multiple) switches into the CXL hierarchy also increases memory access latency. Furthermore, PCIe Plug and Play requires re-enumerating the PCIe bus and loading drivers for newly added memory devices. This results in uncertain latency for hot-adding / removing memory. In other words, from a memory latency perspective, these methods are suboptimal.

[0023] Therefore, an example involving providing a discrete memory pool comprising multiple memory slices dynamically allocated to compute nodes is disclosed. As part of the initialization process, the compute node discovers one or more memory controllers, each attached to memory located remotely from the compute node's perspective. The memory attached to each memory controller collectively forms a memory pool comprising a discrete memory pool, the scope (e.g., address range) and topology of which are also discovered during the compute node's initialization. Although this environmental information is known at the node and exposed to the node, the compute node's use of the memory pool is initially disabled. Instead, the compute node queries the memory controller to discover memory pool slices allocated to that compute node. Typically, the number of slices in the pooled memory ecosystem significantly exceeds the number of slices a node might use. For example, a pooled system might comprise many terabytes of memory slices, but a single node might only be able to access 512 GB of memory.

[0024] The request-based framework allows compute nodes to request the allocation of additional slices and the release of previously allocated slices. In some examples, unallocated slices from a memory pool can be used to serve requests for additional slices or slice extensions. In other examples, such requests can be served by requesting the compute node to release allocated slices, where the compute node can evaluate the slice's usage to determine whether the slice can be released.

[0025] The method described herein provides a distributed memory pool that can be distributed across any appropriate number of compute nodes at any suitable fine-grained level through slicing. This increases memory pool utilization and reduces resource staging, thereby enabling the provision of smaller memory pools that meet the computational requirements of compute nodes. Therefore, the described method supports and enhances thin provisioning and is also applicable to examples that do not use thin provisioning. Furthermore, the described method provides these and other advantages without requiring device re-enumeration or the inclusion of switches, thus reducing latency and latency uncertainty in memory transactions. Additionally, mechanisms are provided to protect the slice allocation to individual compute nodes, preventing compute nodes from accessing slices not allocated to them.

[0026] Figure 1 An example computing system 100 is schematically illustrated, in which multiple computing nodes 102A-N are communicatively coupled to a memory system 104 including a discrete memory pool 106. N is any suitable positive integer such that any suitable number of computing nodes 100 can utilize the memory pool 106 when performing computing workloads.

[0027] Memory system 104 includes multiple memory controllers 108A-N, where N is any suitable positive integer. Attached to each memory controller 108 is a corresponding set of memories 110, such as memories 110A attached to memory controller 108A. In general, the memories 110A-N attached to each memory controller 108 form a memory including a discrete memory pool 106. As described below, each memory controller 108 manages requests to allocate portions of its attached memory 110 to compute nodes 102, manages requests to release allocated portions of its attached memory, and can implement per-node allocation. In some examples, each compute node 102 can access memory controller 108 relatively consistently, while in other examples, different memory controllers can be provided to different compute nodes with different qualities of service.

[0028] In general, the memory controller 108 collaborates with other components in system 100 to allocate slices of memory pool 106 to appropriate compute nodes. As used herein, a “slice” of memory pool 106 refers to at least a portion of memory 110, which includes the memory pool. Memory 110 may take any suitable form, including but not limited to Double Data Rate Synchronous Dynamic RAM (DDR SDRAM).

[0029] Memory 110, forming a discrete memory pool 106, is remotely attached to compute node 100. Therefore, memory 110 and memory pool 106 can be referred to as “extended,” “remote,” or “pooled” memory. Similarly, when memory controller 108 manages such pooled memory, the memory controller can be referred to as an “extended,” “remote,” or “pooled” memory controller. Conversely, each compute node 102 is associated with a respective native memory controller 112 (e.g., native memory controller 112A associated with compute node 102A). Each native memory controller 112 manages the native memory natively attached to the corresponding compute node 100.

[0030] In this way, the native memory controller 112 can facilitate the execution of memory transactions directed to the natively attached memory. The native memory controller 112 can perform other functions, including but not limited to prioritizing the routing of memory transactions to the natively attached memory, and performing functions in response to communications from the memory controller 108. For example, the native memory controller 112 may receive from the memory controller 108 an indication that a portion of the memory pool 106 available for allocation to the compute node 102 is low, and in response, delay the request to allocate a portion of the memory pool to the compute node.

[0031] In some examples, each native memory controller 112 may include a component that processes memory transactions to determine whether these transactions should be routed to natively attached memory or discrete memory pool 106, and routes the memory transactions to the appropriate location. For example, this component may utilize an address mapping that identifies address ranges corresponding to natively attached memory and address ranges corresponding to memory pool 106. This component may also handle communication from memory controller 108.

[0032] Conversely, the multiple operating systems running on compute node 102 that cause such memory transactions to occur may lack awareness of the division between natively attached memory and pooled memory. Instead, these two sets of memory may appear to the multiple operating systems as a contiguous address range. Therefore, native memory controller 112 can route memory transactions to memory pool 106 in a manner transparent to compute node 102 and the software executing on it. However, in some examples described below, multiple operating systems executing on compute node 102 may be configured with class drivers and application programming interfaces (APIs) that engage with memory controller 108 to enable the multiple operating systems to utilize the separate memory pool 106.

[0033] Each memory controller 108 may include one or more ports, each port communicatively coupling the memory controller to one or more compute nodes 102. When the native memory controller 112 is configured to be aware of the memory controller 108 as described above, the ports(s) in the memory controller 108 may be communicatively coupled to the native memory controller. In the depicted example, the memory controller 108 is directly attached to the compute node 102 without an intermediate switch. However, implementations using switches to communicatively couple the memory controller 108 to the compute node 102 are conceivable. When a switch is used, each memory controller 108 may include a single port communicating with the switch and multiple compute nodes 102. Furthermore, in some examples, each memory controller 108 may be a CXL-compliant memory controller (e.g., a Type 3 CXL device supporting the CXL.io and CXL.mem protocols). In such examples, and where the memory controller 108 is directly attached to the compute node 102 without a CXL switch, each memory controller may present itself to the compute node 102 as multiple single logical devices on multiple CXL links. In the example where memory controller 108 is a CXL-compliant memory controller but uses a switch to facilitate communication between memory controller nodes, each memory controller can present itself to compute node 102 as a multi-logical memory device on CXL(multiple) links.

[0034] The computing system 100 also includes a structure manager 114 that collaborates with the compute node 102 and memory controller 108 to serve requests regarding slices of the discrete memory pool 106. The structure manager essentially has a larger global view of the memory pool 106 (as opposed to visibility only to slices managed by a single pooled memory controller 108) and provides coordinated global functionality. For this purpose, the structure manager 114 can receive descriptions of the slice(s) of memory(s) 110 attached to each memory controller 108. The structure manager 114 can then assemble the descriptions from each memory controller 108 into a manifest that collectively describes the slice(s) of the entire discrete memory pool 106.

[0035] Figure 2 Example listing 200 shows a slice of memory pool 106 divided for an example scenario where a portion of the memory pool is allocated to various compute nodes 102, and another portion is not allocated to any compute nodes. Listing 200 includes a description of each slice of memory pool 106, indicating the compute node 102 to which the slice is allocated (for slices allocated to compute nodes) and the size of the slice. However, listing 200 may include any suitable additional or alternative information, including but not limited to the address range of each slice. Furthermore, in some examples, structure manager 114 may provide the listing to advanced managers, allocators, and / or hypervisors involved in allocating slices of memory pool 106.

[0036] In addition to maintaining a list of how to allocate slices in the discrete memory pool 106, the structure manager 114 can guide the allocation of the memory pool. For example, the structure manager 114 can receive a request to allocate a slice to compute node 102 and consult its list to identify slices in memory pool 106 that satisfy the request. If an unallocated slice that can satisfy the request (i.e., a slice not allocated to any compute node 102) is identified, the structure manager 114 can request or instruct the corresponding memory controller 108 attached to the memory 110 containing the slice to allocate at least a portion of the unallocated slice to the requesting node. Specifically, a portion of the unallocated slice that satisfies the request can be allocated to the requesting compute node 102 (e.g., if the size of a portion is larger than the portion that satisfies the request, the entire unallocated slice is not allocated).

[0037] Conversely, if no unallocated slices are available anywhere in memory pool 106, structure manager 114 may request one or more memory controllers 108 to release the slice(s) currently allocated to compute node 102. However, examples are possible where this functionality is performed at least partially at memory controller 108. For example, memory controller 108 may receive a request for a slice and determine whether memory 110 attached to the memory controller includes an unallocated portion capable of satisfying the request. If such an unallocated portion exists, memory controller 108 may allocate at least a portion of that portion as a slice to the requesting compute node 102. If no such unallocated portion exists, memory controller 108 may request compute node 102 to release the allocated slice stored by memory 110 attached to the memory controller.

[0038] In some examples, a portion of memory pool 106 may be maintained as a buffer to accommodate such requests if there are not enough unallocated portions (other than the buffer) to serve them, and / or if compute node 102, despite being requested to do so, cannot release enough slices to serve them. In examples where memory pool 106 includes such a buffer, requests for slices may be rejected even if the buffer size is sufficient to serve the request. This can be considered a scenario where memory pool 106 does not include unallocated slices capable of satisfying slice requests. In such examples, instead of using a buffer to serve requests, unallocated slice(s) that are unlikely to cause problems can be identified based on slice usage information, and compute node 102 that has been allocated these slice(s)(s)(s)(s)(s)(s)(s)(s)) can be requested to release these slice(s)(s).

[0039] The slice usage information utilized by the structure manager 114 in guiding slice allocation and slice deallocation may include per-slice usage information (e.g., the time each slice was used, the frequency of each slice's use, the identifier of the application using each slice, and the priority or importance of the data in each slice). Such information may be included in the list described above or provided in any other suitable manner. For example, the structure manager 114 may use the per-slice usage information to allocate a slice from a specific unit of memory 110 to the requesting compute node 102, which in turn accommodates another slice allocated to the same compute node. As another example, the structure manager 114 may use the per-slice usage information to prioritize candidate slices to be released that are considered to have lower priority and / or are less frequently or recently used.

[0040] The memory controller 108 that receives a request to release an allocated slice can use the allocated slice to query the operating system running on the compute node 102. Figure 1 An exemplary method is illustrated in which memory controller 108A generates an interrupt (e.g., Message Signaled Interrupt (MSI), MSI-X, System Management Interrupt (SMI), or Non-Maskable Interrupt (NMI)) to compute node 102A to attempt to release slice(s) allocated to compute node 102A and used by operating system 116 executing on compute node 102B. In this example, a request to release the allocated slice is stored in one or more registers 118 of memory controller 108A. Class driver 120, implemented by operating system 116, reads register(s) 118 to discover the request and invokes the operating system's API to request the release of the allocated slice(s). Similarly, requests to allocate free slices, as well as other requests regarding slice allocation, may also be stored in register(s) 118 for discovery by operating system 116 via class driver 120.

[0041] Operating system 116 can determine whether the slice(s) allocated to compute node 102A can be released in any suitable manner. For example, if the operating system is not currently using the slice or expects to use it in the future, operating system 116 can accept a request to release the slice, and if the operating system is currently using the slice or may use it in the future, then operating system 116 can reject the request. Operating system 116 can consider alternative or additional criteria, including but not limited to strategies that make slice relinquishment less likely for some applications / loads than for others, such as mission-critical workloads or workloads that need to be completed to fulfill a service level agreement. As another example, operating system 116 can specify the allocation of slice(s) to compute node 102A as fixed, wherein such fixed-allocation slices cannot be reassigned to another compute node.

[0042] In some examples, the operating system 116 releases enough slices to satisfy the compute node 102's request for slices. In other examples, the operating system 116 may not be able to release a sufficient number and / or size of slices to satisfy the request. In this case, the operating system 116 may release as many slices as possible and (from compute node 102A or memory controller 108A) indicate to the structure manager 114 that other slices have not been released. The structure manager 114 may then request one or more other operating systems running on other compute nodes 102 to release the slices that the operating system 116 cannot release. In some examples, the operating system 116 may not be able to release any slices, in which case the compute node 102A or memory controller 108A may output an indication to the structure manager 114 that the request to release slices has been rejected.

[0043] Requests to adjust the allocation of the split memory pool 106 can originate from any suitable location, including requests to allocate (multiple) slices and release (multiple) allocated slices. For example, such requests could originate from compute node 102 or an operating system (e.g., operating system 116) running on it. These requests can be forwarded directly to one or more memory controllers 108, or forwarded to structure manager 114, which can then send the request to the selected memory controller. As another example, structure manager 114 can instruct one or more memory controllers 108 to adjust (multiple) slice allocations—for example, based on requests or instructions to adjust slice allocations from a higher-level allocator. In cases where structure manager 114 is involved in slice allocation, the structure manager can consult a list (e.g., list 200) to determine which memory controllers 108 and / or compute node 102 have requested adjustments to (multiple) slice allocations. To establish slice allocations, in some examples, structure manager 114 can command one or more memory controllers 108 to self-program to reallocate memory addresses. The structure manager 114 can receive confirmation from the memory controller 108 that the requested slice allocation operation has been successfully executed. At this time, the structure manager can send a message to the operating system to indicate the slice allocation and the potential memory address range(s) associated with the allocated slice(s).

[0044] The memory controller 108 can implement functions that help manage the categorized memory pool 106 in situations where the memory pool experiences memory pressure (i.e., when the unallocated portion of the memory pool is relatively small). For example, the memory controller 108 can detect the presence of memory pressure (e.g., by analyzing the allocation of its attached memory 110) and, in response to issuing a request to one or more compute nodes 102 to release the allocated slice(s) without consulting the compute nodes, issue a warning to the compute nodes so that they can proactively release the allocated slice(s), delay or disable requests for the slice(s), and / or take any other appropriate action. Furthermore, the memory controller 108 can evaluate the usage of the attached memory 110 in identifying slices for release, for example, by identifying the least recently used slice, the least frequently used slice, or by considering any other suitable usage patterns.

[0045] The allocation of slices in the split memory pool 106 can be enforced such that compute node 102 cannot access slices not allocated to it. This per-node allocation can be implemented in any suitable manner. In some examples, each memory controller 108 can perform per-node allocation—knowing which compute node 102 is allocated to each slice of memory 110 attached to that controller, the memory controller can prevent another compute node not allocated to that slice from accessing a slice allocated to a compute node. Alternatively or additionally, per-node allocation can be implemented by providing compute node 102 only with the address ranges of the slice(s) ...

[0046] Structure manager 114 can be implemented in any suitable manner. As an example, structure manager 114 can be implemented in code executed on one or more compute nodes 102 within computing system 100. In other examples, structure manager 114 can be implemented by one or more compute nodes or any other suitable device outside computing system 100. Furthermore, structure manager 114 can have any suitable communication relationship with components within computing system 100 to achieve its functionality. For example, structure manager 114 can communicate directly with compute node 102 to establish a desired slice allocation, or in other examples, it can communicate with the compute node via memory controller 108 and / or native memory controller 112. Where a switch (e.g., a CXL switch) is used to facilitate communication between memory system 104 and compute node 102, structure manager 114 can be programmed to facilitate communication between memory system nodes.

[0047] Although Figure 1An implementation including a structure manager 114 is described, but it is possible for the structure manager to be omitted. For example, in the case where the computing system 100 includes a single compute node 102 attached to multiple memory controllers 108, the structure manager 114 can be omitted. In such an example, the memory controller 108 may receive a request from the compute node 102 to adjust slice allocation, determine whether such a request can be fulfilled by the attached memory 110, and, if unallocated slices are not available to fulfill such a request, issue a request to the compute node to release the slice(s)(s). Typically, at least some of the functions described herein in conjunction with memory controller 108 can be alternatively or additionally implemented at the structure manager 114 (or any other suitable component(s)), and at least some of the functions described herein in conjunction with the structure manager can be alternatively or additionally implemented at the memory controller(s)(s). In some examples, the structure manager 114 may be included in the implementation of two or more compute nodes 102 attached to a common memory controller 108 included in the computing system 100. In other words, the inclusion of the structure manager 114 can be a result of whether the computing system 100 includes multiple computing nodes 102.

[0048] It should be understood that releasing an allocated slice can involve any suitable operation(s). For example, a released slice can be deallocated from compute node 102 to which it was previously allocated and then allocated to the requesting compute node, making the released slice subsequently available for running memory transactions (e.g., read, write, memory consistency operations). Once the released slice is allocated, the data stored in the released slice can be overwritten by the requesting compute node 102. In other examples, the data stored by the released slice can be copied to another location before the slice becomes available to the requesting compute node 102. For example, such data can be copied from a separate memory pool 106 to a mass storage device (e.g., a memory with lower performance than the memory in the memory pool).

[0049] Figure 3 A flowchart illustrating method 300 for bringing one or more slices of a discrete memory pool online at the operating system level to perform memory transactions. For example, method 300 can be implemented at compute node 102 to bring slices of discrete memory pool 106 online. Furthermore, method 300 can be performed as part of the compute node's initialization (e.g., boot process).

[0050] In method 300, the method includes discovering one or more memory controllers having memory forming a discrete memory pool via the compute node's Basic Input / Output System (BIOS). For example, compute node 102A may discover one or more of memory controllers 108A-108N, each memory controller attached to a corresponding memory 110 in the memory collectively forming memory pool 106. In some examples, the compute node may use the CXL enumeration method to discover the memory controller(s). Discovering the memory controller may include, for each discovered memory controller, adding 304 to the physical address mapping of the extended memory attached to that memory controller to the compute node. Figure 4 An example physical address mapping 400 is shown, which maps the extended memories of two pooled memory controllers (e.g., extended memory 110A attached to memory controller 108A and extended memory 110B attached to pooled memory controller 108B). Address mapping 400 also includes address information about memory-mapped inputs / outputs (MMIOs) and memories (e.g., DDR) natively attached to the compute node. The extent of the entire discrete memory pool can also be discovered by finding the extent of each extended memory cell attached to each pooled memory controller.

[0051] At 306, method 300 includes updating the ACPI table via the BI operating system using the base address and size of the extended memory for each discovered memory controller. Updating the ACPI table may include designating the extended memory at 308 as hot-pluggable. Alternatively or additionally, updating the ACPI table may include designating the extended memory at 310 as offline. Here, the extended memory is disabled during compute node initialization, where initialization refers to a portion of the node startup process during which the BIOS is active but the operating system is not yet running. Although a range of extended memory (e.g., a memory address range) is visible and known to the compute node, it cannot yet be used because it is designated as hot-pluggable and / or offline and is therefore disabled. Similarly, because the address range of the slice(s) is not yet known, the compute node is not yet aware of the extended memory(s) that the node can use. At this point, the extended memory and its(s) slice(s) will not be used by the BIOS or boot code of the operating system already executing on the compute node.

[0052] In 312, method 300 includes invoking the operating system via the BIOS. In 314, method 300 includes reading the ACPI table via the operating system to discover the extended memory of each discovered memory controller. Here, and as described above, the extent of the extended memory is visible and known to the operating system, but the extended memory is not yet accessible because it may be disabled due to being designated as offline. In 316, method 300 includes initializing the discovered extended memory controllers via the operating system. Initializing the discovered extended memory controllers may include enumerating the extended memory controllers.

[0053] At 318, method 300 includes initializing the extended memory of each discovered extended memory controller via a class driver implemented by the operating system. At 320, method 300 includes outputting a request for one or more slices of the discrete memory pool to each discovered extended memory controller via the class driver. At 322, method 300 includes receiving one or more address ranges for the slice(s). At 324, method 300 includes calling an operating system API via the class driver to bring the slice(s) online at the operating system. This may include designating the slice(s) online. The slice(s) may be designated offline before being designated online. Here, the memory addresses of the slice(s) are known to and accessible by the operating system. At 326, method 300 includes performing memory transactions using the slice(s). This may include bootstrapping memory-related instructions (e.g., store, load) to the address range(s) associated with the slice. In view of the above, the boot or initialization process of the compute node described above can provide addressability of the discrete memory pool, while the online process described above can provide accessibility to the memory pool.

[0054] Figure 5 A flowchart illustrating a method 500 for allocating slices of a split memory pool to compute nodes is shown. Method 500 can be implemented, for example, to allocate slices of a split memory pool 106 to compute node 102.

[0055] At 502, method 500 includes receiving, at a pooled memory controller (e.g., memory controller 108A), a request to allocate slices of a discrete memory pool to a compute node. In some examples, the 504 request may be received from a structure manager (e.g., structure manager 114). The structure manager may select a memory controller from a plurality of memory controllers, for example, based on a list describing the allocation of slices of individual memory attached to each memory controller. In some examples, the structure manager may receive the request from another entity (e.g., a higher-level allocator or hypervisor).

[0056] In method 506, method 500 includes generating an interrupt to the compute node at the memory controller. The interrupt may be of the form MSI-X, SIX, SMI, NMI, or any other suitable form. In method 508, method 500 includes reading one or more registers of the memory controller via a class driver implemented by an operating system running on the compute node to discover a request to allocate a slice to the compute node. This allows the operating system to access the memory controller to discover requests regarding the slice. In method 510, method 500 includes calling an API of the operating system via the class driver to add the slice to the operating system. Adding the slice to the operating system may include adding the slice (e.g., the memory address range of the slice) to a physical address mapping maintained by the operating system. In some examples, the native memory controller (e.g., native memory controller 112A) may store the address range of the slice, allowing the native memory to route instructions for the slice to an extended memory controller attached to the memory including the slice. Knowing the memory address information corresponding to the slice, the operating system can use the slice to perform memory transactions.

[0057] In 514, method 500 includes generating a response via the operating system indicating that the allocation of a slice to a compute node is complete. In 516, method 500 includes receiving a response at the memory controller. As an example, the response may be sent via a CXL port of the memory controller associated with the compute node. In 518, method 500 includes sending a response from the memory controller to the structure manager.

[0058] Figure 6 A flowchart illustrating a method 600 for deallocating slices allocated to compute nodes in a split memory pool is shown. Method 600 can be implemented, for example, to deallocate slice(s) of split memory pool 106.

[0059] At 602, method 600 includes receiving a request from a compute node to cancel a slice of a discrete memory pool at a pooled memory controller (e.g., memory controller 108A). As an example, a 604 request may be received from a structure manager. In some examples, the structure manager may select a memory controller from multiple memory controllers to handle the request. In other examples, another entity (e.g., a higher-level allocator or compute node) may identify the memory controller as the recipient of the request. Furthermore, the compute node from which the slice will be allocated may be identified in any suitable manner, such as by the structure manager. In another example, the memory controller may analyze its slices to identify slices whose cancellation is more likely to be accepted—for example, the memory controller may prioritize slices for cancellation based on a least recently used strategy, or it may determine that the data held in the slice is parked in another memory / storage location.

[0060] At 606, method 600 includes generating an interrupt to the compute node at the memory controller. At 608, method 600 includes reading one or more registers of the memory controller to discover a request via a class driver implemented by an operating system running on the compute node. At 610, method 600 includes calling an API of the operating system via the class driver to remove the slice from the operating system. Removing the slice from the operating system may include removing slice 612 from a physical address mapping maintained by the operating system. Before removing the slice, the operating system may be queried to determine whether the slice can be released, for example, based on whether the operating system or an application is using or may use the slice in the future. If the operating system accepts the request, the slice can be released, and the operating system can stop performing memory transactions using the slice. However, if the operating system rejects the request, the slice will not be released.

[0061] In 614, method 600 includes generating a response via the operating system indicating whether the request is accepted or rejected. As described above, the operating system may accept the request if the slice is currently not in use, is unlikely to be used in the future, contains low-priority data, and / or for any other suitable reason. The operating system may reject the request if the slice is currently in use, is likely to be used in the future, contains high-priority or mission-critical data, and / or for any other suitable reason. The response may be output to the memory controller. If the operating system accepts the request, the response may indicate that the slice is released. If the operating system rejects the request, the response may indicate that the slice is not released. In 616, method 600 includes receiving a response at the memory controller. In 618, method 600 includes sending a response from the memory controller to the structure manager.

[0062] The methods described in this paper provide a decentralized memory pool that can be distributed across multiple compute nodes through flexible and dynamically allocated slices. As described above, the extent of the memory pool can be notified to each compute node without initial knowledge or access to its constituent slices. By using a message-passing framework, slices are made online and available to multiple operating systems running on the compute nodes, thus avoiding the complexities of hot-adding processes. Similarly, the message-passing framework allows for the deallocation of allocated slices and their provision to requesting compute nodes while taking into account their usage, ensuring that compute workloads performed using allocated slices are not excessively interrupted and do not undergo hot removal processes. Therefore, these methods enhance thin-provisioning implementations but are also applicable to implementations that do not employ thin-provisioning.

[0063] In some embodiments, the methods and processes described herein can be attached to a computing system of one or more computing devices. In particular, such methods and processes can be implemented as computer applications or services, application programming interfaces (APIs), libraries, and / or other computer program products.

[0064] Figure 7 A non-limiting embodiment of a computing system 700 that can implement one or more of the methods and processes described above is illustrated schematically. The computing system 700 is shown in a simplified form. The computing system 700 may take the form of one or more personal computers, server computers, tablet computers, home entertainment computers, network computing devices, gaming devices, mobile computing devices, mobile communication devices (e.g., smartphones), and / or other computing devices.

[0065] The computing system 700 includes a logic subsystem 702 and a storage subsystem 704. The computing system 700 may optionally include a display subsystem 706, an input subsystem 708, a communication subsystem 710, and / or... Figure 7 Other components not shown.

[0066] The logic subsystem 702 includes one or more physical devices configured to execute instructions. For example, a logic unit may be configured to execute instructions that are part of one or more applications, services, programs, routines, libraries, objects, components, data structures, or other logical constructs. Such instructions may be implemented to perform tasks, implement data types, change the state of one or more components, achieve technical effects, or otherwise obtain desired results.

[0067] A logic unit may include one or more processors configured to execute software instructions. Additionally or alternatively, a logic unit may include one or more hardware or firmware logic units configured to execute hardware or firmware instructions. The processor of the logic unit may be single-core or multi-core, and the instructions executed thereon may be configured for sequential, parallel, and / or distributed processing. Components of the logic unit may optionally be distributed across two or more separate devices that may be remotely located and / or configured for coordinated processing. Aspects of the logic unit may be virtualized and executed by remotely accessible, networked computing devices configured in a cloud computing configuration.

[0068] The storage device subsystem 704 includes one or more physical devices configured to store instructions executable by a logic unit. When such methods and processes are implemented, the state of the storage device subsystem 704 can be transformed, for example, to store different data.

[0069] Storage device subsystem 704 may include removable and / or built-in devices. Storage device subsystem 704 may include optical disc storage (e.g., CD, DVD, HD-DVD, Blu-ray disc, etc.), semiconductor memory (e.g., RAM, EPROM, EEPROM, etc.), and / or magnetic memory (e.g., hard disk drive, floppy disk drive, magnetic tape drive, MRAM, etc.). Storage device subsystem 704 may include volatile, non-volatile, dynamic, static, read / write, read-only, random access, sequential access, location-addressable, file-addressable, and / or content-addressable devices.

[0070] It should be understood that the storage device subsystem 704 includes one or more physical devices. However, aspects of the instructions described herein may alternatively be propagated by communication media (e.g., electromagnetic signals, optical signals, etc.) that are not held by physical devices for a finite duration.

[0071] Various aspects of the logic subsystem 702 and the storage device subsystem 704 can be integrated together into one or more hardware logic components. For example, such hardware logic components may include field-programmable gate arrays (multiple FPGAs), application-specific integrated circuits (PASICs / multiple ASICs), application-specific standard products (PSSPs / multiple ASPs), system-on-a-chip (SOCs), and complex programmable logic devices (multiple CPLDs).

[0072] The terms "module," "program," and "engine" can be used to describe an aspect of a computing system 700 implemented to perform a specific function. In some cases, a module, program, or engine can be instantiated via a logical subsystem 702 that executes instructions stored in a storage device subsystem 704. It should be understood that different modules, programs, and / or engines can be instantiated from the same applications, services, code blocks, objects, libraries, routines, APIs, functions, etc. Similarly, the same module, software, and / or engine can be instantiated from different applications, services, code blocks, objects, routines, APIs, functions, etc. The terms "module," "program," and "engine" can include single or grouped executable files, data files, libraries, drivers, scripts, database records, etc.

[0073] It should be understood that the term "service" as used herein refers to an application that can execute across multiple user sessions. A service can be used for one or more system components, programs, and / or other services. In some implementations, a service can run on one or more server computing devices.

[0074] When included, the display subsystem 706 can be used to present a visual representation of the data stored by the storage device subsystem 704. This visual representation may take the form of a graphical user interface (GUI). Since the methods and processes described herein change the data held by the storage machine and thus change the state of the storage computer, the state of the display subsystem 706 can also be translated into a visual representation of the changes in the underlying data. The display subsystem 706 may include one or more display devices utilizing virtually any type of technology. Such display devices may be combined with the logical subsystem 702 and / or the storage device subsystem 704 within a shared enclosure, or such display devices may be peripheral display devices.

[0075] When included, the input subsystem 708 may include or interface with one or more user input devices, such as a keyboard, mouse, touchscreen, or game controller, and the conversion and / or processing of input actions may be performed on-board or off-board. In some embodiments, the input subsystem may include or interface with a selected native user input (NUI). Such components may be integrated or peripheral, and the transmission and / or processing of input actions may be handled on-board or off-board. Example NUI components may include microphones for voice and / or voice identification; infrared, color, stereo, and / or depth cameras for machine vision and / or gesture identification; head trackers, eye trackers, accelerometers, and / or gyroscopes for motion detection and / or intent identification; and electric field sensing components for assessing brain activity.

[0076] When included, the communication subsystem 710 can be configured to communicatively couple the computing system 700 to one or more other computing devices. The communication subsystem 710 may include wired and / or wireless communication devices compatible with one or more different communication protocols. As a non-limiting example, the communication subsystem can be configured to communicate via a wireless telephone network or a wired or wireless local area network or wide area network. In some embodiments, the communication subsystem may allow the computing system 700 to send and / or receive messages from other devices via a network such as the Internet.

[0077] Another example provides a memory system including a memory controller and memory attached to the memory controller and forming at least a portion of a discrete memory pool. The discrete memory pool includes multiple slices, each dynamically allocated to a corresponding one of two or more compute nodes. The memory system is configured to receive a request to adjust the allocation of the discrete memory pool to a requesting compute node; in response to determining that a portion of the discrete memory pool includes unallocated slices capable of satisfying the request, allocate at least a portion of the unallocated portion to the requesting compute node; and in response to determining that a portion of the discrete memory pool does not include unallocated slices capable of satisfying the request, output a request to another compute node to release at least one slice of the plurality of slices allocated to the compute node. In such an example, the address range of the discrete memory pool is visible to each of the two or more compute nodes. In such an example, the memory system may alternatively or additionally be configured to output a corresponding address range for each of the plurality of slices. In such an example, the request to release at least one slice may be handled by an operating system running on another compute node. In such an example, the memory system alternative or additional location can be configured to output the address range of at least one slice for the requesting compute node to receive in response to another compute node releasing at least one slice, and to output an indication that the at least one slice has not been released in response to the other compute node refusing to release the at least one slice. In such an example, for each of two or more compute nodes, each of a plurality of slices can be disabled for that node during initialization. In such an example, the memory system alternative or additional location can be configured to prevent the requesting compute node from accessing slices allocated to another compute node. In such an example, the memory controller alternative or additional location can be configured to describe the allocation of a portion of a discrete memory pool to the structure manager, and to adjust the allocation of that portion of the discrete memory pool in response to the structure manager. In such an example, the memory controller alternative or additional location may include one or more registers, and can be configured to store instructions for adjusting the allocation of a portion of the discrete memory pool in one or more registers, wherein such instructions are discoverable by the operating system. In such an example, memory can be mapped in the physical address space of each compute node. In such an example, the memory controller may alternatively or additionally be configured to: in response to another compute node refusing a request to release at least one slice, output an indication to the structure manager that the request to release at least one slice has been refused, and the structure manager may be configured to make a request to another memory controller to allocate a slice of memory attached to the other memory controller to the requesting compute node.

[0078] Another example provides a method at a compute node that includes discovering one or more memory controllers attached to corresponding memory, the corresponding memory controllers forming a discrete memory pool, the discrete memory pool including multiple slices, the multiple slices being dynamically assigned to corresponding compute nodes in two or more compute nodes, the multiple slices being initially disabled, outputting a request for one or more slices to at least one of the one or more memory controllers, receiving address ranges of one or more slices assigned to the compute node, and performing memory transactions using the one or more slices. In such an example, the method may alternatively or additionally include discovering the range of the discrete memory pool. In such an example, the range of the discrete memory pool may be discovered during the initialization of the compute node. In such an example, the method may alternatively or additionally include initializing the discrete memory pool via a class driver implemented by the compute node. In such an example, the method may alternatively or additionally include adding the corresponding memory of each of the one or more memory controllers attached to a memory map. In such an example, the request may be output via a class driver implemented by the compute node. In such an example, alternatively or additionally, the method may include designating one or more slices as online.

[0079] Another example provides a method at a compute node that includes accessing a memory controller to discover a request for a slice of a discrete memory pool, wherein the request is used to: allocate the slice to the compute node, add the address range of the slice to a memory map and perform memory transactions using the slice, and query the operating system executed by the compute node to determine if the slice can be released. In such an example, the method may alternatively or additionally include: in response to the operating system accepting the request, outputting a response to the memory controller indicating that the slice has been released and ceasing memory transactions using the slice, and in response to the operating system rejecting the request, outputting a response to the memory controller indicating that the slice has not been released.

[0080] It should be understood that the configurations and / or methods described herein are exemplary in nature, and these particular embodiments or examples should not be considered limiting, as many variations are possible. The specific routines or methods described herein may represent one or more of any number of processing strategies. Thus, the various actions shown and / or described may be performed in the shown and / or described order, in a different order, in parallel, or omitted. Similarly, the order of the above processing may be changed.

[0081] The subject matter of this invention includes all novel and non-obvious combinations and sub-combinations of various processes, systems and configurations, as well as other features, functions, behaviors and / or properties disclosed herein, and any and all equivalents.

Claims

1. A memory system, comprising: Two or more compute nodes, which are configured to output memory allocation requests; Memory controller; as well as A memory, attached to the memory controller and forming at least a portion of a discrete memory pool, the discrete memory pool comprising multiple slices, each dynamically allocated to a corresponding compute node among the two or more compute nodes. The memory system is configured as follows: Receive requests to adjust the allocation of the discrete memory pool to the two or more compute nodes; In response to determining that the portion of the split memory pool does not include an unallocated slice capable of satisfying the request, a request to release at least one slice of the plurality of slices allocated to the compute node is output to another compute node of the two or more compute nodes; In response to the other compute node refusing the request to release the at least one slice, an indication that the request to release the at least one slice has been rejected is output to the structure manager, wherein the structure manager is configured to receive a description of a memory slice attached to each memory controller and assemble a memory list of the discrete memory pool based on the description of the memory slice; and wherein the structure manager is configured to issue a request to another memory controller to allocate a slice of memory attached to the other memory controller to the requesting compute node. The address range of the slice allocated to the requesting compute node is transmitted to the requesting compute node; and The request computing node is configured as follows: Receive the address range of the slice allocated to the requesting compute node; and Memory transactions are performed using the slice.

2. The memory system of claim 1, wherein the address range of the discrete memory pool is visible to each of the two or more compute nodes.

3. The memory system of claim 1, wherein the memory system is further configured to output a corresponding address range for each of the plurality of slices.

4. The memory system of claim 1, wherein the request to release the at least one slice is processed by an operating system running on the other computing node.

5. The memory system of claim 1, wherein for each of the two or more compute nodes, each of the plurality of slices is disabled for the node in the process of initialization.

6. The memory system of claim 1, wherein the memory system is further configured to prevent the requesting compute node from accessing a slice allocated to the other compute node.

7. The memory system of claim 1, wherein the memory controller is further configured to: describe the allocation of the portion of the discrete memory pool to a structure manager, and adjust the allocation of the portion of the discrete memory pool in response to the structure manager.

8. The memory system of claim 1, further comprising one or more registers, the memory controller being configured to store instructions for adjusting the allocation of said portion of the discrete memory pool in said one or more registers, said instructions being discoverable by an operating system.

9. The memory system of claim 1, wherein the memory is mapped in the physical address space of each computing node.

10. A method at a computation node, comprising: Discover one of two or more memory controllers attached to a corresponding memory, the corresponding memory forming a split memory pool, the split memory pool comprising multiple slices, each of which is dynamically assigned to one of two or more compute nodes, the multiple slices being initially disabled; The system outputs a request for one of the plurality of slices to the memory controller, wherein in response to the memory controller rejecting the request for the slice, the memory controller outputs an indication that the request for the slice has been rejected to the structure manager, and wherein the structure manager is configured to receive a description of a memory slice attached to each of the two or more memory controllers, and assemble a memory list for the discrete memory pool based on the description of the memory slice, and wherein the structure manager is configured to issue a request to a second memory controller of the two or more memory controllers to allocate a slice of memory attached to the second memory controller to the compute node; Receive the address range of the slice allocated to the computing node; as well as Memory transactions are performed using the slice.

11. The method of claim 10, further comprising discovering the range of the discrete memory pool.

12. The method of claim 11, wherein the extent of the discrete memory pool is discovered during the initialization of the computing node.

13. The method of claim 10, further comprising initializing the discrete memory pool via a class driver implemented by the compute node.

14. The method of claim 10, further comprising adding the respective memory attached to each of the two or more memory controllers to the memory map.

15. The method of claim 10, wherein the request for the slice is output via a class driver implemented by the compute node.

16. The method of claim 10, further comprising designating the slice as online.

17. A method at a computation node, comprising: Output a request to the memory controller to allocate a split memory pool slice to the compute node; If the request is rejected by the memory controller, the memory controller outputs an indication to the structure manager that the request for the slice has been rejected, wherein the structure manager is configured to receive descriptions of memory slices attached to two or more memory controllers and assemble a memory list for the discrete memory pool based on the descriptions of the memory slices; and wherein the structure manager is configured to issue a request to a second memory controller among the two or more memory controllers to allocate a slice of memory attached to the second memory controller to the compute node; as well as The address range of the slice is added to the memory map, and memory transactions are performed using the slice.

Images