Memory reduction in a system by oversubscribing the physical memory shared by the computational entities that the system supports

Abstract

A method and system are provided for reducing memory in a system by oversubscribing physical memory shared among computing entities, the portion of memory including a combination of a portion of a first physical memory of a first type and a logical pool memory associated with the system, the logical pool memory controller configured to (1) track both a state of whether a page of the logical pool memory assigned to any of the multiple computing entities is a known pattern page and a relationship between the physical memory addresses and the logical memory addresses associated with any assigned logical pool memory, and (2) allow write operations to write data to any available space in a second physical memory of the first type only up to an extent of the physical memory corresponding to the portion of the logical pool memory previously assigned to the computing entity.

Description

[Background technology]

[0001] Multiple users or tenants may share a system, including computing systems and communication systems. The computing system may include a public cloud, a private cloud, or a hybrid cloud having both public and private portions. A public cloud includes a global network of servers that perform various functions, including storing and managing data, running applications, and delivering content or services, such as streaming video, provisioning email, providing office productivity software, or engaging with social media. While a public cloud provides services to the public over the Internet, a business may use a private cloud or a hybrid cloud. Both private and hybrid clouds also include a network of servers housed in a data center.

[0002] Multiple tenants may use the computational, storage, and networking resources associated with a server in the cloud. By way of example, computing entities associated with different tenants may be allocated a certain amount of computational and memory resources. In many such situations, the allocated resources to the various computing entities, including memory resources, may be underutilized. In other systems, such as communication systems including base stations, similar underutilization of memory resources may occur. Summary of the Invention

[0003] In one example, the disclosure relates to a method comprising allocating a portion of memory associated with a system to a computing entity, the portion of memory including a combination of a portion of a first physical memory of a first type and a portion of a logical pool memory for use with a plurality of computing entities associated with the system. As used herein, the term "computing entity" encompasses, but is not limited to, any executable code (in the form of hardware, firmware, software, or any combination thereof) that implements functionality, a virtual machine, an application, a service, a microservice, a container, a unikernel for serverless computing, or portions of the foregoing. As used herein, the term "logical pool memory" refers to memory including overcommitted physical memory shared by a plurality of computing entities, which may correspond to a single host or multiple hosts. The logical pool memory may be mapped to a second physical memory of the first type, and an amount of the logical pool memory designated as available for allocation to the plurality of computing entities may be greater than the amount of the second physical memory of the first type. The method may further include indicating to a logical pool memory controller associated with the logical pool memory that all pages associated with the logical pool memory that are initially allocated to any of the plurality of computing entities are known pattern pages. The method may further include the logical pool memory controller tracking both a state of whether a page of the logical pool memory assigned to any of the plurality of computing entities is a known pattern page and a relationship between physical memory addresses and logical memory addresses associated with any assigned logical pool memory to any of the plurality of computing entities. The method may further include, in response to a write operation initiated by the computing entity, the logical pool memory controller allowing writing of data to any available space in the second physical memory of the first type only up to an extent of the physical memory corresponding to a portion of the logical pool memory previously assigned to the computing entity.

[0004] In another example, the disclosure relates to a system having a memory for use with a plurality of computing entities, the portion of the memory including a combination of a portion of a first physical memory of a first type and a portion of a logical pool memory associated with the system. The logical pool memory may be mapped to a second physical memory of the first type, and an amount of the logical pool memory designated as available for allocation to the plurality of computing entities may be greater than the amount of the second physical memory of the first type. The system may further include a logical pool memory controller coupled to the logical pool memory and configured to: (1) track both a state of whether a page of the logical pool memory assigned to any of the plurality of computing entities is a known pattern page and a relationship between a physical memory address and a logical memory address associated with any assigned logical pool memory to any of the plurality of computing entities; and (2) in response to a write operation initiated by a computing entity to write any data other than the known pattern, allow the write operation to write data to any available space in the second physical memory of the first type only up to an extent of the physical memory corresponding to the portion of the logical pool memory previously assigned to the computing entity.

[0005] In yet another example, the present disclosure relates to a system including a plurality of host servers configured to execute one or more of a plurality of computing entities. The system may further include a memory, where the portion of the memory includes a combination of a portion of a first physical memory of a first type and a portion of a logical pool memory shared among the plurality of host servers. The logical pool memory may be mapped to a second physical memory of the first type, and an amount of the logical pool memory designated as available for allocation to the plurality of computing entities may be greater than the amount of the second physical memory of the first type. The system may further include a logical pool memory controller coupled to a logical pool memory associated with the system and configured to: (1) track both a state of whether a page of the logical pool memory assigned to any of the plurality of computing entities is a known pattern page and a relationship between physical memory addresses and logical memory addresses associated with any assigned logical pool memory to any of the plurality of computing entities; and (2) in response to a write operation initiated by the computing entity, executed by a processor associated with any of the plurality of host servers to write any data other than the known pattern, enable the write operation to write the data to any available space in a second physical memory of the first type only up to an extent of the physical memory corresponding to a portion of the logical pool memory previously assigned to the computing entity.

[0006] This Summary is provided to introduce some concepts in a simplified form that are further described below in the Detailed Description. This Summary is not intended to identify key features or essential features of the claimed aspects, nor is it intended to be used to limit the scope of the claimed subject matter.

[0007] The present disclosure is illustrated by way of example and not limitation in the accompanying drawings, in which like reference numerals indicate similar elements and in which elements in the figures are illustrated for simplicity and clarity and have not necessarily been drawn to scale. [Brief description of the drawings]

[0008] [Figure 1] 1 is a block diagram of a system environment including a host server coupled with a logical pool memory system according to one example. [Diagram 2] 1 illustrates an oversubscription process according to an example. [Diagram 3] FIG. 1 is a block diagram of a system including a host server including a logical pool memory system configured to allow oversubscription, a machine learning (ML) system, and a VM scheduler, according to one example. [Figure 4] 1 is a block diagram of a logical pool memory system configured to allow memory oversubscription, according to one example. [Diagram 5] 1 illustrates an embodiment of a mapping table using a translation lookaside buffer (TLB), according to an example. [Figure 6] 1 illustrates a block diagram of an example system including multiple host servers coupled with a logical pool memory system configured to allow memory oversubscription. [Figure 7] 1 illustrates a block diagram of an example system that implements at least a portion of a method for allowing oversubscription of pool memory allocated to a computing entity. [Figure 8] 1 illustrates a flowchart of an example method for oversubscribing physical memory used with a computing entity. [Figure 9] 13 illustrates a flowchart of another example method for oversubscribing physical memory used with a computing entity. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0009] Examples described in this disclosure relate to reducing memory in a system by oversubscribing physical memory shared among the computing entities that the system supports. A particular example relates to oversubscribing physical memory used with virtual machines in a multitenant computing system. The multitenant computing system may be a public cloud, a private cloud, or a hybrid cloud. A public cloud includes a global network of servers that perform various functions including storing and managing data, running applications, and delivering content or services such as streaming video, email, office productivity software, or social media. The servers and other components may be located in data centers around the world. While a public cloud provides services to the public over the Internet, a business may use a private cloud or a hybrid cloud. Both private and hybrid clouds also include a network of servers housed in a data center. A computing entity may run using the computing and memory resources of a data center. As used herein, the term "computing entity" encompasses, but is not limited to, any executable code (in the form of hardware, firmware, software, or any combination thereof) that implements functionality, a virtual machine, an application, a service, a microservice, a container, a unikernel for serverless computing, or any portion of the foregoing. Alternatively, the computing entity may be executed in hardware associated with an edge computing device, an on-premise server, or another type of system, including a communications system such as a base station (5G or 6G base station).

[0010] In accordance with examples of the present disclosure, a computing entity is assigned a combination of local physical memory and logical pool memory under the assumption that memory usage will typically be equal to or less than the memory allocated to the computing entity. This allows for oversubscription of installed memory in the system by the computing entity, since more computing entities can be supported by the system without the need to allocate additional physical memory. As an example, in a multi-tenant computing or communication system, each tenant is assigned some portion of the total system memory. It is observed that not all tenants use their entire allocation of memory at the same time. Thus, as an example, a host server in a data center may be assigned logical pool memory exposed by a pool memory system. Each of the VMs may be assigned a combination of local physical memory on the host server and a portion of the logical pool memory available to the host server. Logical pool memory may be a combination of known pattern pages that are not backed by any physical memory and physical pool memory. In short, as used herein, the term "logical pool memory" refers to memory that includes overcommitted physical memory shared by multiple computing entities, which may correspond to a single host or multiple hosts. In this example, overcommitted means that the logical pool memory has an amount of physical memory that is less than the total amount of memory designated as available to the computing entities. As used herein, the term "known pattern page" refers to any page that contains only zeros, only ones, or other known patterns of values ​​or symbols. A VM first uses local physical memory, and as it is used, the VM may access portions of the logical pool memory. In a multi-tenant computing system, if all VMs are allocated memory in this manner, they are unlikely to exhaust their entire allocation of physical pool memory backing the logical pool memory.This is because the actual physical memory behind a logical pool memory controller can be less than the logical pool memory exposed to the host servers, which in turn can allow for a reduction in the amount of physical memory (e.g., DRAM) that needs to be deployed as part of a multi-tenant computing or communication system.

[0011] According to the above implementation of logical pool memory, some portion of the logical pool memory remains unused, but it is difficult to predict which portion of the logical pool memory will be unused at a given time. Therefore, a mechanism is needed to flexibly map the pooled physical memory to the logical pool memory exposed to the computing entity such that the unused physical pool memory remains unmapped while the logical pool memory space that is in use is backed by the physical pool memory. In one example, the mapping function may be performed using a page mapping table. Other mechanisms may also be used. Furthermore, in some cases, memory usage may be further optimized by compressing data into the physical pool memory before storing the data. If data compression is used, the mapping function may be further modified to keep track of the compressed memory pages.

[0012] FIG. 1 is a block diagram of a system environment 100 including host servers (e.g., host servers 110, 120, and 130) coupled with a logical pool memory system 180, according to one example. Each host server may be configured to execute a number of computing entities. In this example, host server 110 may be executing computing entities 112 (e.g., CE-0, CE-1, and CE-2), host server 120 may be executing computing entities 122 (e.g., CE-3, CE-4, and CE-5), and host server 130 may be executing computing entities 132 (e.g., CE-6, CE-7, and CE-7). Logical pool memory system 180 may include logical pool memory 190, which may include a number of memory modules. Although not shown in FIG. 1, logical pool memory system 180 may include a logical pool memory controller (described below). As will be described in more detail below, logical pool memory 190 may include an address space larger than the physical memory configured to support the logical address space. In this example, logical pool memory 190 may include memory modules: PM-1, PM-2, and PM-M, where M is an integer greater than 1. As used herein, the term "memory module" includes any memory based on a memory technology that can operate as a cacheable memory. Examples of such memory modules include, but are not limited to, dual in-line memory modules (DIMMs) or single in-line memory modules (SIMMs). As used herein, the term "cacheable memory" includes any type of memory in which data obtained by a read operation associated with the memory can be copied to a memory cache so that the next time the same data is accessed, it can be obtained from the memory cache. The memory may be dynamic random access memory (DRAM), flash memory, static random access memory (SRAM), phase change memory, magnetic random access memory, or any other type of memory technology that allows the memory to operate as a cacheable memory.

[0013] 1, each of host servers 110, 120, and 130 may be coupled by links to memory modules included in logical pool memory 190. By way of example, host server 110 is shown coupled to memory module PM-1 via link 142, to memory module PM-2 via link 144, and to memory module PM-M via link 146. Host server 120 is shown coupled to memory module PM-1 via link 152, to memory module PM-2 via link 154, and to memory module PM-M via link 156. As another example, host server 130 is shown coupled to memory module PM-1 via link 162, to memory module PM-2 via link 164, and to memory module PM-M via link 166.

[0014] Still referring to FIG. 1 , in this example, fabric manager 170 may manage the allocation of memory ranges to each host server. A processor (e.g., a CPU) servicing a computation entity may issue a load or store instruction. The load or store instruction may result in a read or write to a local memory (e.g., a DRAM associated with the CPU of the host server configured to execute the computation entity) or a read / write to a memory module associated with logical pool memory 190. In this example, a load or store instruction that results in a read / write to a memory module associated with logical pool memory 190 may be translated by fabric manager 170 into a transaction that is completed using one of the links (e.g., links 142, 144, or 146). In one example, fabric manager 170 may be implemented using a fabric manager based on the Compute Express Link standard.

[0015] Continuing with reference to FIG. 1, the computing entities 112, 122, and 132 may be provided with computing, storage, and networking resources using a set of clusters that may be included as part of a data center. As used in this disclosure, the term "data center" may include, but is not limited to, some or all of a data center owned by a cloud service provider, some or all of the data centers owned and operated by a cloud service provider, some or all of the data centers owned by a cloud service provider operated by a customer of the service provider, any other combination of data centers, a single data center, or several clusters within a particular data center. In one example, each cluster may include several identical servers. Thus, a cluster may include servers that include a certain number of CPU cores and a certain amount of memory. Alternatively, the computing entities 112, 122, and 132 may be executed in hardware associated with edge computing devices, on-premise servers, or other types of systems, including communication systems such as base stations (e.g., 5G or 6G base stations). Although FIG. 1 illustrates system environment 100 as having a particular number of components, including computing entities and memory components, arranged in a particular manner, system environment 100 may include additional or fewer components arranged otherwise.

[0016] FIG. 2 illustrates an example of an oversubscription process flow. The example illustrates an oversubscription process using a virtual machine (VM) as an example computational entity that runs using hardware associated with a host server. The example further assumes that the host server is located in a data center that includes a cluster of host servers. The example further assumes that the cluster of host servers is part of a multi-tenant cloud. However, the oversubscription process may be used in other settings, including when the computational entity runs on hardware associated with an edge computing device, an on-premise server, or other types of systems, including communication systems such as base stations (e.g., 5G or 6G base stations). In response to a request from a user to initialize a VM, a VM scheduler may schedule the VM for execution using hardware resources associated with the host server. As part of this process, the VM may be assigned specific computational resources (e.g., CPU) and memory (e.g., DRAM associated with the CPU). The memory associated with the host server may be better utilized by using oversubscription alone or in combination with pooling. As an example, memory may be oversubscribed with the expectation that most of the time, VMs will only use memory that is typically utilized by or predicted for use by such VMs. In one example, the average peak usage by a VM may be assigned to the VM as local physical memory. Thus, a certain amount of memory may be assigned to a VM while recognizing that such memory may not always be available to such VMs, e.g., during peak usage or other high memory usage scenarios. Pooling can aid in oversubscription, and oversubscription can aid in pooling when the two techniques are used together, since memory from the pool may be used for VMs that are assigned oversubscribed memory.

[0017] Still referring to FIG. 2, an exemplary oversubscription process for a request for 32 GB of memory (210) by a VM is shown. In this example, in response to this request, the hypervisor (or other coordinator for allocating resources to VMs) may allocate 19 GB of physical memory and 13 GB of memory that is mapped to logical pool memory. In this example, the allocation of the 19 GB of memory may be based on the projected usage of memory by the requesting VM. In this example, the pooled memory may have 256 GB of logical pool memory along with 128 GB of physical pool memory. In this manner, the 13 GB of memory that is mapped to the pool may or may not be available, since the 256 GB of logical pool memory is shared with other host servers that support additional VMs. A VM that is allocated 19 GB of physical memory and 13 GB of memory that is mapped to pooled memory may run without making any requests to the pooled memory, since in most cases, less than 19 GB of memory is actually used by the VM (220). Alternatively, in rare cases, a VM's memory usage may exceed 19 GB (230). In such cases, additional memory may be allocated from physical pool memory 242 in logical pool memory 240. As part of the oversubscription process, a high water mark may be associated with the use of physical pool memory 242, such that certain mitigation processes may be initiated when the VM's use of this memory approaches the high water mark. As an example, when the high water mark is reached, the logical pool memory controller may begin compressing pages before storing them in physical pool memory 242. In another example, physical pool memory 242 may be backed by additional physical memory that may have higher latency and therefore likely be less expensive to allocate than the memory associated with physical pool memory 242. In such situations, when the high water mark is reached, pages may be written to the slower memory.

[0018] To ensure that a compute entity first uses its portion of the physical pool memory (e.g., 19 GB) before using its portion of the logical pool memory (e.g., 13 GB), software memory may be used that allows the compute entity to distinguish between physical and logical pool memory. In one example, to ensure better performance in a host server that supports Non-Uniform Memory Access (NUMA) nodes, the pooled memory may be exposed to the VM as a compute-less virtual NUMA node. This allows the VM to distinguish between local physical memory and logical pool memory, which has higher latency. Thus, the VM may continue to use the local physical memory first, and then, when it is exhausted, start using the pooled memory that is exposed to the VM as a compute-less virtual NUMA node. Other mechanisms that allow the VM to distinguish between local physical memory and logical pool memory may also be used. Although FIG. 2 describes the oversubscription process with a total of 32 GB requested by the VMs and 256 GB of logical pool memory, the VMs may request additional or lesser amounts of memory (e.g., anywhere between 4 GB and 128 GB). Similarly, the size of the logical pool memory may be smaller or larger depending on the number of computing entities serviced by the computing system. Additionally, while FIG. 2 describes the amount of local physical memory as 19 GB and the amount of logical pool memory as 13 GB, these amounts may vary between computing entities. Additionally, the oversubscription process does not require pooling of memory between host servers (or other systems), it only requires multiple computing entities to share the logical pool memory.

[0019] 3 is a block diagram of a system 300 including a host server 310, a machine learning (ML) system 304, and a VM scheduler 302, including resources 330 configured to allow oversubscription, according to one example. The host server 310 may also include a hypervisor 320, compute resources (e.g., CPU-0 332 and CPU-1 338), local memory resources (e.g., local memory-0 340 and local memory-1 342), a fabric manager 350, and a logical pool memory system 360. The VM scheduler 302 may be configured to receive requests for VMs from any of the tenants associated with the multi-tenant cloud and schedule placement of the VMs (e.g., VM 312, VM 314, VM ​​316, and VM 318 on the host server 310). In this example, each of the resources 330 may be implemented as a hardware component, such as using one or more of a SoC, a controller, a processing core, a memory module, an ASIC, an FPGA, a CPLD, a PLS, a silicon IP, or the like. Further, in this example, functionality associated with each of VM scheduler 302, ML system 304, and hypervisor 320 may be implemented by instructions stored on a non-transitory medium when executed by a processor. Although FIG. 3 describes system 300 with respect to VMs, other computing entities may be used by system 300 as well. Additionally, VM scheduler 302 may be configured to process requests from computing entities included in other types of systems, including edge computing devices, on-premise servers, or communication systems such as base stations (e.g., 5G or 6G base stations).

[0020] The ML system 304 may be configured to provide resource predictions, including predicted memory usage, to the VM scheduler 302. A request to place a set of VMs may arrive at the VM scheduler 302. In this example, the VM scheduler 302 may then query the ML system 304 to request the predicted memory usage. The ML system 304 may provide memory usage predictions. Using these predictions, the VM scheduler 302 may allocate memory resources to the VMs. Alternatively, the VM scheduler 302 may be provided with the memory usage predictions or other memory usage information so that the VM scheduler 302 does not need to query the ML system 304 every time a VM needs to be placed or scheduled for execution. As previously described in connection with FIG. 2, the allocated memory may be a combination of physical memory (e.g., local DRAM) and a mapping to a logical pool memory.

[0021] Continuing with reference to FIG. 3, the ML system 304 may include several components having instructions configured to perform various functions associated with the ML system 304. The ML system 304 may include both offline training and online prediction components. The offline training component may be responsible for training various machine learning models, validating the models, and publishing the validated models. The online prediction component may generate predictions related to various aspects including predictions regarding memory usage by VMs and any behavioral patterns associated with the VMs. Prior to training the ML models, features may be selected that enable the ML models to predict metrics based on inputs. The training phase may include the use of backward propagation or other techniques that enable ML to learn relationships between particular input parameters and particular predictions based on the parameters. As an example, a neural network model trained using stochastic gradient descent may be used to predict memory usage for a computational entity such as a VM. Other types of ML models may be used, including Bayesian models.

[0022] In general, supervised learning algorithms can be implemented that can be trained based on input data, and once trained, predictions or prescriptions can be made based on the training. Learning and inference techniques such as linear regression, support vector machines (SVMs) configured for regression, random forests configured for regression, gradient boosting trees configured for regression, and neural networks can be used. Linear regression can involve modeling past relationships between independent variables and dependent output variables. A neural network can include an input layer, one or more hidden layers, and artificial neurons used to generate an output layer. Each layer can be encoded as a matrix or vector of weights represented in the form of coefficients or constants obtained by offline training of the neural network. A neural network may be implemented as a recurrent neural network (RNN), a long short-term memory (LSTM) neural network, or a gated recurrent unit (GRU). All information required by a model based on supervised learning can be converted into a vector representation corresponding to any of these techniques.

[0023] Taking LSTM as an example, an LSTM network may have a series of repeating RNN layers or other types of layers. Each layer of an LSTM network may consume input at a given time step, e.g., the state of the layer from the previous time step, and generate a new set of outputs or states. When using LSTMs, a single chunk of content may be encoded into a single vector or multiple vectors. As an example, a word or combination of words (e.g., a phrase, sentence, or paragraph) may be encoded as a single vector. Each chunk may be encoded into a separate layer (e.g., a particular time step) of the LSTM network. An LSTM layer may be described using a set of equations such as:

number

[0024] In this example, within each LSTM layer, the inputs and hidden states may be processed using vector operations (e.g., dot products, inner products, or vector addition) or a combination of nonlinear operations, as appropriate. Instructions corresponding to the machine learning system may be encoded as hardware corresponding to the A / I processor. In this case, some or all of the functionality associated with the ML system 304 may be hard-coded or otherwise provided as part of the A / I processor. By way of example, the A / I processor may be implemented using a feature-rich FPGA.

[0025] In other examples, memory allocated to a VM may be dynamically changed or determined. As an example, behavioral changes may correspond to the workload processed by the VM over time, which can be used to dynamically change the memory allocation. An example of behavioral changes may be associated with a VM serving a web search workload or other similar workload. Such a VM may have a diurnal pattern where the workload requires more resources during the day, but less resources at night. In one example, a Fast Fourier Transform (FFT) algorithm may be used to detect periodicity in behavior associated with the VM. While an FFT may be used to detect periodicity at multiple time scales, the memory allocation of the VM may be dynamically changed only if the workload handled by the VM has a periodicity that matches a particular pattern, e.g., a diurnal pattern. The periodicity analysis may generate ground truth labels that may also be used in training ML models to predict that a VM is likely to execute a diurnal or another pattern. In other examples, other features (e.g., cloud subscription ID, user who created the VM, VM type, VM size, guest operating system) may also be used to statically or dynamically change the range of physical and pooled memory allocated to the VM.

[0026] In yet another example, instead of predicting memory usage or determining behavior / usage patterns, a VM may be exclusively allocated local physical memory in response to a request to schedule the VM. At a later point in time, when the local physical memory associated with the host server (or collection of host servers) begins to reach a certain utilization level, the scheduled VM may be allocated a combination of local physical memory and mapped logical pool memory. In this manner, memory may be dynamically allocated to currently running VMs.

[0027] Still referring to FIG. 3, hypervisor 320 may manage virtual machines including VMs 312, 314, 316, and 318. CPU-0 332 may include one or more processing cores, and CPU-1 338 may include one or more processing cores. Instructions corresponding to hypervisor 320, when executed by any of the processing cores, may provide management functions provided by hypervisor 320. Hypervisor 320 may ensure that any thread associated with any given VM is scheduled only to the logical cores of that group. CPU-0 332 may be coupled to local memory-0 340, and CPU-1 338 may be coupled to another local memory-1 342. Each of local memory-0 340 and local memory-1 342 may include one or more memory modules. Examples of such memory modules include, but are not limited to, dual in-line memory modules (DIMMs) or single in-line memory modules (SIMMs). Any memory type that is a "cacheable memory" may be used to implement a memory module. As used herein, the term "cacheable memory" includes any type of memory in which data obtained by a read operation associated with the memory can be copied to a memory cache so that the next time the same data is accessed it can be obtained from the memory cache. The memory may be dynamic random access memory (DRAM), flash memory, static random access memory (SRAM), phase change memory, magnetic random access memory, or any other type of memory technology that allows the memory to operate as a cacheable memory.

[0028] 3, in this example, CPU-0 332 may be coupled to fabric manager 350 via fabric interface-0 334, and CPU-1 338 may be coupled to fabric manager 350 via another fabric interface-1 336. Fabric manager 350 may allow any of the VMs executed by processing cores associated with CPU-0 332 and CPU-1 338 to access logical pool memory system 360. In one example, any of the CPUs associated with host server 310 may issue a load or store instruction to a logical address space established by hypervisor 320. The load or store instruction may result in a read or write to local memory (e.g., either local memory-0 340 or local memory-1 342 depending on the CPU issuing the load or store instruction), or a read / write transaction associated with logical pool memory system 360. In one example, if the local memory is DRAM, then any of the relevant double data rate (DDR) protocols associated with accessing the DRAM may be used by the CPU to read / write the local memory. In this example, load or store instructions resulting in a read / write to a memory module associated with logical pool memory system 360 may be translated by fabric manager 350 (or other similar subsystem) into a transaction that is completed using any of the links coupling the VMs with the memory associated with logical pool memory system 360. The read / write transaction may be performed using a protocol associated with fabric manager 350 and logical pool memory system 360. The memory modules associated with logical pool memory system 360, when implemented as DRAM modules, may be accessed by a controller associated with logical pool memory system 360 using any of the relevant DDR protocols.

[0029] In one example, fabric manager 350 may be implemented using a fabric manager based on the Compute Express Link (CXL) standard. In this example, memory modules associated with logical pool memory system 360 may be configured as Type 3 CXL devices. The logical address space exposed by hypervisor 320 to virtual machines may be at least a subset of the address range exposed by the controller associated with the CXL bus / link. Thus, as part of this example, transactions associated with the CXL.io protocol, which is a PCIe-based non-coherent I / O protocol, may be used to configure memory devices and links between memory modules and CPUs included in logical pool memory system 360. The CXL.io protocol may also be used by CPUs associated with host server 310 for device discovery, enumeration, error reporting, and management. Alternatively, any other I / O protocol that supports such configuration transactions may be used. Memory access to memory modules may be handled via transactions associated with the CXL.mem protocol, which is a memory access protocol that supports memory transactions. As an example, load and store instructions associated with any of the CPUs of the host server 310 may be processed via the CXL.mem protocol. Alternatively, any other protocol that allows for the translation of CPU load / store instructions into read / write transactions associated with memory modules included in the logical pool memory system 360 may be used. Although FIG. 3 illustrates the host server 310 with certain components arranged in a particular manner, the host server 310 may include additional or fewer components arranged otherwise. As an example, while FIG. 3 illustrates local memory coupled to the CPU, non-local memory may be coupled to the CPU. Instead, the CPU may only access memory associated with the logical pool memory system 360.3 shows the VMs hosted by a single host server, VMs 312, 314, 316, and 318 may be hosted using a group of host servers (e.g., a cluster of host servers in a data center). Logical pool memory system 360 may be shared among the host servers or may be dedicated to each host server.

[0030] FIG. 4 is a block diagram of a logical pool memory system 400 configured to allow memory oversubscription, according to one example. In one example, logical pool memory system 360 of FIG. 3 may be implemented as logical pool memory system 400. In this example, logical pool memory system 400 may include logical pool memory 410, logical pool memory controller 450, and pool memory control structure 480. Logical pool memory 410 may include physical pool memory 412 and known pattern pages 414. Physical pool memory 412 may include memory modules with physical memory (e.g., DRAM). Physical pool memory 412 may correspond to a logical address space. Known pattern pages 414 may represent a logical address space that is not backed by any physical memory. Logical pool memory controller 450 may manage access to logical pool memory 410 by interacting with pool memory data structure 480. Logical pool memory controller 450 may be implemented as any combination of hardware, firmware, or software instructions. Instructions corresponding to logical pool memory controller 450 may be stored in a memory associated with logical pool memory system 400. Such instructions, when executed by a processor associated with logical pool memory system 400, may provide at least a portion of the functionality associated with logical pool memory controller 450. Other functionality may be provided by control logic associated with logical pool memory controller 450, including finite state machines and other logic.

[0031] In this example, pool memory control structure 480 may include a mapping table 482 and a physical memory free page list 484. Both mapping table 482 and physical memory free page list 484 may be stored in physical pool memory 412 or other memory (e.g., a cache associated with logical pool memory controller 450). Mapping table 482 may hold mappings between physical address spaces and logical address spaces associated with physical pool memory 412. Mapping table 482 may also track the state of a page with respect to whether or not the page is a known pattern page. A known pattern page may not have any allocated physical space as part of physical pool memory 412. In this example, physical memory addresses associated with known pattern pages are considered invalid (identified as not applicable (N / A)) in mapping table 482. If compression is used to free up additional space in physical pool memory 412, that can also be tracked using mapping table 482. Table 1 below illustrates an example mapping table 482. [Table 1]

[0032] As shown above in Table 1, the mapping table 482 may be indexed by page number. A known pattern page indicator (e.g., a known pattern page flag) may be associated with each page. A logical high value (e.g., 1) may indicate to the logical pool memory controller 450 that no physical memory space has been allocated in the physical pool memory 412 corresponding to the page number. A logical low value (e.g., 0) may indicate to the logical pool memory controller 450 that physical memory space has been allocated in the physical pool memory 412 corresponding to the page number. The logical pool memory controller 450 may change the known pattern page indicator based on any status change associated with the page number. At the time of scheduling or creation of a VM, the hypervisor (or host operating system) may map a portion of the memory for the VM to the physical memory associated with the host server and the remainder to the logical pool memory 410. At this point, the hypervisor (or operating system) may issue a command to the logical pool memory controller 450 to set the known pattern page indicator to a logical high value (e.g., 1) for the page associated with the logical pool memory assigned to the VM. A VM may also write a known pattern to the logical pool memory that was allocated to it when the VM was initialized. Logical pool memory controller 450 may detect a write operation writing a known pattern as an indication that physical memory does not need to be allocated to the VM at this time. Additionally, logical pool memory controller 450 may ensure that the known pattern page's indicator for the corresponding page is accurately reflected in mapping table 482.

[0033] Initially, logical pool memory controller 450 may map all of the logical pool memory for the VM to known pattern pages 414. A newly scheduled VM may also work with the hypervisor to ensure that any pages corresponding to logical pool memory 410 that are assigned to the VM are identified as known pattern pages in mapping table 482. At a later point in time, when the VM uses up all of its assigned local physical memory and begins writing unknown patterns (e.g., non-zeros) to the logical memory address space, logical pool memory controller 450 may begin allocating pages to the VM from within the logical address space (exposed by logical pool memory controller 450) that correspond to physical pool memory 412. Physical memory free page list 484 may include a list of pages corresponding to free physical pool memory 412 that are not mapped by mapping table 482. Table 2 below illustrates an example physical memory free page list 484. This example lists pages corresponding to physical pool memory 412 that are free to allocate. [Table 2]

[0034] Although the address translation is described as a single level address translation, the address translation may be a multi-level address translation, and therefore may require the use of additional mapping tables. In either case, the logical pool memory controller 450 may be configured to maintain the mapping table and ensure proper tracking of known pattern pages to enable pooling and oversubscription of memory assigned to VMs, as described above. Additionally, while the mapping table 482 is described in terms of pages, other groupings of memory, including memory blocks, may also be used with the mapping table 482. Additionally, the functionality of the mapping table 482 may be implemented using registers or other similar hardware structures associated with the processor and controller. Additionally, while the mapping table 482 is described as being stored in the physical pool memory 412, at least a portion of the mapping table may be stored in a cache associated with the logical pool memory controller 450 to accelerate the address translation process.

[0035] FIG. 5 illustrates a diagram of an embodiment of a mapping table using a translation lookaside buffer (TLB), according to one example. As previously described in connection with FIG. 4, the mapping table 482 may be used by the logical pool memory controller 450 to translate logical addresses received from the hypervisor (or other source) to physical addresses associated with the physical pool memory 412. In this example, the logical pool memory controller 520 may include a cache implemented as a translation lookaside buffer (TLB) 530. The TLB 530 may be implemented as a content addressable memory (CAM), a fully associative cache, a two-way set associative cache, a direct-mapped cache, or other type of suitable cache. The TLB 530 may be implemented to accomplish both the translation from logical pool addresses to physical pool memory addresses and to check the status of known pattern pages of the page. For each address translation request, the logical pool memory controller 520 may first check the TLB 530 to determine if the address translation is already cached. If the address translation is found to be cached in TML 530, a TLB HIT signal may be generated. Additionally, a physical address corresponding to the page number that caused the TLB HIT signal may be provided to physical pool memory 560 to enable access to the data or instructions stored at that physical address. Alternatively, if the address translation is not cached, a TLB MISS signal may be generated. This may then result in a table walk across mapping table 562 unless the flag of the known pattern page is set to a logical true value. As shown in FIG. 5, a physical memory free page list 564 may also be stored in physical pool memory 560.

[0036] FIG. 6 illustrates a block diagram of a system 600 including multiple host servers coupled to a logical pool memory system 670 configured to allow memory oversubscription. The system 600 may include a host server 610, a host server 620, and a host server 630 coupled to the logical pool memory system 670. As previously described, each host server may host multiple computing entities (e.g., VMs), which may be purchased or otherwise paid for by different tenants of the system 600. Each server may include local memory (e.g., memory modules described in connection with the host server 310 of FIG. 3) and may have access to logical pool memory, which may be a portion of the logical pool memory included in the logical pool memory system 670. As previously described, the fabric manager (implemented based on CXL) may expose a subset of the logical pool memory associated with the logical pool memory system 670 to each of the hosts 610, 620, and 630. The logical pool memory system 670 may be implemented similarly as previously described.

[0037] Each of the computing entities that are scheduled or otherwise made available to a tenant via the host server 610 may be assigned a portion of the local memory 612 and a portion of the pooled memory 614. As described above, the assignment to each computing entity may be based on an oversubscription to the logical pool memory managed by the logical pool memory system 670. In this example, each computing entity hosted by the host server 610 may be coupled to the logical pool memory system 670 via a link 616 (or a collection of links). Each of the computing entities that are scheduled or otherwise made available to a tenant via the host server 620 may be assigned a portion of the local memory 622 and a portion of the pooled memory 624. As described above, the assignment to each computing entity hosted by the host server 620 may be based on an oversubscription to the logical pool memory managed by the logical pool memory system 670. In this example, each computing entity hosted by the host server 620 may be coupled to the logical pool memory system 670 via a link 626 (or a collection of links). Each of the computing entities that are scheduled or otherwise made available to a tenant via host server 630 may be assigned a portion of local memory 632 and a portion of pooled memory 634. As previously described, the allocation to each computing entity hosted by host server 630 may be based on an oversubscription to a logical pool memory managed by logical pool memory system 670. In this example, each computing entity hosted by host server 630 may be coupled to logical pool memory system 670 via link 636 (or a collection of links). By way of example, load and store instructions associated with any of the CPUs of each of host servers 610, 620, and 630 may be processed by the CXL.mem protocol.Alternatively, any other protocol may be used that allows for the translation of CPU load / store instructions into read / write transactions associated with memory modules included in logical pool memory system 670. Although Figure 6 illustrates system 600 as including certain components arranged in a particular manner, system 600 may include additional or fewer components arranged otherwise.

[0038] 7 illustrates a block diagram of a system 700 that implements at least a portion of a method for allowing oversubscription of pooled memory allocated to a computational entity. The system 700 may include a processor 702, an I / O component 704, a memory 706, a presentation component 708, a sensor 710, a database 712, a networking interface 714, and an I / O port 716, which may be interconnected via a bus 720. The processor 702 may execute instructions stored in the memory 706. The I / O component 704 may include components such as a keyboard, a mouse, a speech recognition processor, or a touch screen. The memory 706 may be any combination of non-volatile or volatile storage (e.g., flash memory, DRAM, SRAM, or other types of memory). The presentation component 708 may include a display, a holographic device, or other presentation device. The display may be any type of display, such as an LCD, LED, or other type of display. The sensors 710 may include telemetry or other types of sensors configured to detect and / or receive information (e.g., collected data). The sensors 710 may include telemetry or other types of sensors configured to detect and / or receive information (e.g., memory usage by various computing entities running on host servers in a data center). The sensors 710 may include sensors configured to detect conditions associated with CPUs, memory, or other storage components, FPGAs, motherboards, board management controllers, etc. The sensors 710 may also include sensors configured to detect conditions associated with racks, chassis, fans, power supply units (PSUs), etc. The sensors 710 may include sensors configured to detect conditions associated with network interface controllers (NICs), top-of-rack (TOR) switches, middle-of-rack (MOR) switches, routers, power distribution units (PDUs), rack-level uninterruptible power supply (UPS) systems, etc.

[0039] Still referring to FIG. 7, database 712 may be used to store any of the collected or recorded data necessary for the execution of the methods described herein. Database 712 may be implemented as a collection of distributed databases or as a single database. Network interface 714 may include a communication interface such as Ethernet, cellular radio, Bluetooth radio, UWB radio, or other types of wireless or wired communication interfaces. I / O ports 716 may include Ethernet ports, fiber optic ports, wireless ports, or other communication or diagnostic ports. Although FIG. 7 illustrates system 700 as including a particular number of components arranged and coupled in a particular manner, it may include fewer or additional components arranged and coupled otherwise. Additionally, functionality associated with system 700 may be distributed as desired.

[0040] FIG. 8 illustrates a flowchart 800 of an example method for oversubscribing physical memory used with a computing entity. In one example, steps associated with the method may be performed by various components of the system described above. Step 810 may include allocating a portion of memory associated with the system to the computing entity, the portion of memory having a combination of a portion of a first physical memory of a first type and a portion of a logical pool memory associated with the system. The logical pool memory is mapped to a second physical memory of the first type, and an amount of the logical pool memory designated as available for allocation to the multiple computing entities is greater than the amount of the second physical memory of the first type. As described above with respect to FIG. 2, a virtual machine, which is a type of computing entity, may be allocated 19 GB of physical memory (e.g., a memory module including DRAM configured as local memory for a CPU) and 13 GB of logical pool memory, which may be a portion of logical pool memory 240 of FIG. 2, which includes physical pool memory 242 of FIG. 2, which may also be a memory module including DRAM. As used herein, the term "type" used in the phrase "first type" refers to whether the memory is cacheable memory or not. As previously described, the term "cacheable memory" includes any type of memory in which data obtained by a read operation associated with the memory can be copied to a memory cache so that the next time the same data is accessed, it can be obtained from the memory cache. The memory may be a dynamic random access memory (DRAM), a flash memory, a static random access memory (SRAM), a phase change memory, a magnetic random access memory, or any other type of memory technology that allows the memory to operate as a cacheable memory. In short, the first physical memory of the first type and the second physical memory of the first type must be of a cacheable type, but they may be of different types on the other side (e.g., DRAM vs. phase change memory, or SRAM vs. DRAM).

[0041] Step 820 may include indicating to a logical pool memory controller associated with the logical pool memory that all pages associated with the logical pool memory that are initially allocated to any of the multiple computing entities are known pattern pages. By way of example, a hypervisor (e.g., hypervisor 320 of FIG. 3) may indicate the status of the initially allocated pages as known pattern pages (e.g., known pattern pages 414) to a logical pool memory controller (e.g., logical pool memory controller 450 of FIG. 4).

[0042] Step 830 may include the logical pool memory controller tracking both the status of whether a page of logical pool memory assigned to any of the plurality of computing entities is a known pattern page and the relationship between the physical memory address and the logical memory address associated with any assigned logical pool memory to any of the plurality of computing entities. In this example, as part of this step, as described above in connection with Figures 4 and 5, the logical pool memory controller (e.g., logical pool memory controller 450 of Figure 4) may track the status of the page (known pattern page or not) and the relationship between the physical address and the page number in the physical memory (e.g., physical pool memory 412 of Figure 4) using a mapping table (e.g., mapping table 482 of Figure 4). As described above, at least a portion of the mapping table may be implemented as a translation lookaside buffer to accelerate access to the logical pool memory. Additionally, the mapping table (or similar structure) may be implemented using registers.

[0043] Step 840 may include, in response to a write operation initiated by the compute entity, allowing the logical pool memory controller to write data to any available space in the second physical memory of the first type only up to a range of physical memory corresponding to a portion of the logical pool memory previously allocated to the compute entity. As previously described, when a compute entity (e.g., a VM) uses up all of the local physical memory allocated to it and begins writing unknown patterns to the logical memory address space, the hypervisor may begin allocating pages to the compute entity from within the logical address space (exposed by logical pool memory controller 450) that correspond to the physical memory (e.g., physical pool memory 412 of FIG. 4). The logical pool memory controller may access a free page list (e.g., physical memory free page list 484 of FIG. 4) to access free pages that are not mapped by the mapping table. Furthermore, as previously described, the compute entity may not be allocated more physical memory managed by the logical pool memory controller than was previously allocated to the compute entity (e.g., 13 GB in the example described with respect to FIG. 2). This method allows oversubscription of installed memory in a system by computing entities, since more computing entities can be supported by the system without the need to allocate additional physical memory (e.g., DRAM).

[0044] FIG. 9 illustrates a flowchart 900 of an example of another method for oversubscribing physical memory used with a computing entity. In one example, steps associated with the method may be performed by various components of the system described above. Step 910 may include allocating a portion of memory associated with the system to the computing entity, the portion of memory having a combination of a portion of a first physical memory of a first type and a portion of a logical pool memory associated with the system. The logical pool memory may be mapped to a second physical pool memory of the first type, and the amount of logical pool memory designated as available for allocation to the multiple computing entities may be greater than the amount of the second physical pool memory of the first type. As described above with respect to FIG. 2, a virtual machine, which is a type of computing entity, may be allocated 19 GB of physical memory (e.g., a memory module including DRAM configured as local memory for a CPU) and 13 GB of logical pool memory, which may be a portion of logical pool memory 240 of FIG. 2, which includes physical pool memory 242 of FIG. 2, which may also be a memory module including DRAM. By way of example, the logical pool memory may be logical pool memory included as part of the logical pool memory system 670 of FIG. 6, and the physical pool memory may correspond to physical memory (e.g., DRAM) included as part of the logical pool memory system 670 of FIG. 6.

[0045] Step 920 may include translating any load or store instructions directed to the logical pool memory into memory transactions for completion over respective links between each of the plurality of computing entities and each of the physical memory devices included as part of the second physical pool memory of the first type. In one example, a fabric manager implemented using a fabric manager based on the Compute Express Link (CXL) specification (e.g., fabric manager 350 of FIG. 3 or fabric manager 650 of FIG. 6 ), as previously described, may translate the load and store instructions into transactions related to the CXL specification. The logical address space exposed to the computing entities may be at least a subset of the address range exposed by the controller associated with the CXL bus / link.

[0046] Step 930 may include indicating to a logical pool memory controller associated with the logical pool memory that all pages associated with the logical pool memory that are initially allocated to any of the multiple computing entities are known pattern pages. By way of example, a hypervisor (e.g., hypervisor 320 of FIG. 3 or a hypervisor associated with any of host servers 610, 620, and 630 of FIG. 6) may indicate the status of the initially allocated pages as known pattern pages (e.g., known pattern pages 414) to a logical pool memory controller (e.g., logical pool memory controller 450 of FIG. 4). Combinations of hypervisors may also cooperate with each other to manage memory allocations to computing entities supported by a computing or communication system having multiple hypervisors managing a shared logical pool memory.

[0047] Step 940 may include the logical pool memory controller tracking both the status of whether a page of logical pool memory assigned to any of the plurality of computing entities is a known pattern page and the relationship between the physical memory address and the logical memory address associated with any assigned logical pool memory to any of the plurality of computing entities. In this example, as part of this step, as described above in connection with FIGS. 4-6, the logical pool memory controller (e.g., logical pool memory controller 450 of FIG. 4) may track the status of the page (known pattern page or not) and the relationship between the physical address and the page number in the physical memory (e.g., physical pool memory 412 of FIG. 4) using a mapping table (e.g., mapping table 482 of FIG. 4). As described above, at least a portion of the mapping table may be implemented as a translation lookaside buffer to accelerate access to the logical pool memory. Additionally, the mapping table (or similar structure) may be implemented using registers.

[0048] Step 950 may include, in response to a write operation initiated by the computing entity to write any data other than the known pattern, the logical pool memory controller permits the writing of data to any available space in the second physical pool memory of the first type only up to a range of physical memory corresponding to a portion of the logical pool memory previously allocated to the computing entity. As previously described, when a computing entity (e.g., a VM) uses up all of its assigned local physical memory and begins writing unknown patterns to the logical memory address space, the hypervisor may begin allocating pages from within the logical address space (exposed by the logical pool memory controller 450) corresponding to the physical memory (e.g., physical pool memory 412 of FIG. 4 or physical pool memory associated with the logical pool memory system 670 of FIG. 6) to the computing entity. The logical pool memory controller may access a free page list (e.g., physical memory free page list 484 of FIG. 4) to access free pages that are not mapped by the mapping table. Additionally, as explained above, a compute entity may not be allocated more physical memory managed by the logical pool memory controller than was previously allocated to the compute entity (e.g., 13 GB in the example described with respect to FIG. 2). This method also allows for oversubscription of installed memory in a system by compute entities, as more compute entities may be supported by the system without having to allocate additional physical memory (e.g., DRAM).

[0049] In summary, the present disclosure relates to a method comprising allocating a portion of memory associated with a system to a computing entity, the portion of memory comprising a combination of a portion of a first physical memory of a first type and a portion of a logical pool memory used with a plurality of computing entities associated with the system. The logical pool memory may be mapped to a second physical memory of the first type, and an amount of the logical pool memory designated as available for allocation to the plurality of computing entities may be greater than the amount of the second physical memory of the first type. The method may further include indicating to a logical pool memory controller associated with the logical pool memory that all pages associated with the logical pool memory that are initially allocated to any of the plurality of computing entities are known pattern pages. The method may further include the logical pool memory controller tracking both a state of whether pages of the logical pool memory allocated to any of the plurality of computing entities are known pattern pages and a relationship between physical memory addresses and logical memory addresses associated with any allocated logical pool memory to any of the plurality of computing entities. The method may further include, in response to a write operation initiated by the computing entity, the logical pool memory controller permitting writing of data to any available space in the second physical memory of the first type only up to an extent of the physical memory corresponding to a portion of the logical pool memory previously allocated to the computing entity.

[0050] In this manner, an amount of logical pool memory associated with the system may be equal to an amount of the second physical memory of the first type combined with an amount of memory corresponding to known pattern pages not backed by any type of physical memory. Tracking both the state of whether a page of logical pool memory assigned to any of the plurality of computing entities is a known pattern page and the relationship between physical and logical memory addresses may include maintaining a mapping table. In one example, the mapping table may be implemented using a translation lookaside buffer.

[0051] The method may further include allocating a portion of the first physical memory of the first type based on a predicted usage of the first physical memory of the first type by the computing entity. The method may further include dynamically varying an amount of the portion of the first physical memory of the first type allocated to the computing entity based on a usage pattern associated with usage of the first physical memory of the first type allocated to the computing entity.

[0052] The method may further include exposing the portion of the logical pool memory allocated to the compute entity via a software mechanism to enable the compute entity to distinguish between the portion of the logical pool memory allocated to the compute entity and the portion of the first physical memory of the first type allocated to the compute entity. The logical pool memory may be coupled to a processor executing any of a number of compute entities that have been assigned at least a portion of the logical pool memory associated with the system via respective links managed by the fabric manager.

[0053] In another example, the disclosure relates to a system having a memory for use with a plurality of computing entities, the portion of the memory comprising a combination of a portion of a first physical memory of a first type and a portion of a logical pool memory associated with the system. The logical pool memory may be mapped to a second physical memory of the first type, and an amount of the logical pool memory designated as available for allocation to the plurality of computing entities may be greater than the amount of the second physical memory of the first type. The system may further include a logical pool memory controller coupled to the logical pool memory and configured to: (1) track both a state of whether a page of the logical pool memory assigned to any of the plurality of computing entities is a known pattern page and a relationship between a physical memory address and a logical memory address associated with any assigned logical pool memory to any of the plurality of computing entities; and (2) in response to a write operation initiated by a computing entity to write any data other than the known pattern, allow the write operation to write data to any available space in the second physical memory of the first type only up to an extent of the physical memory corresponding to the portion of the logical pool memory previously assigned to the computing entity.

[0054] As part of the system, an amount of logical pool memory associated with the system may be equal to an amount of a second physical memory of the first type combined with an amount of memory corresponding to known pattern pages not backed by any type of physical memory. The logical pool memory controller may be further configured to maintain a mapping table to track both a state of whether a page of logical pool memory assigned to any of the plurality of computing entities is a known pattern page and a relationship between physical memory addresses and logical memory addresses associated with any assigned logical pool memory to any of the plurality of computing entities.

[0055] As part of the system, in one example, a scheduler may be configured to allocate portions of the first physical memory of the first type based on a predicted usage of the first physical memory of the first type by the computing entity. The scheduler may also be configured to dynamically vary an amount of the portion of the first physical memory of the first type allocated to the computing entity based on a usage pattern associated with usage of the first physical memory of the first type allocated to the computing entity. The logical pool memory controller may be further configured to indicate to each of the plurality of computing entities that all pages associated with the logical pool memory initially allocated to any of the plurality of computing entities are known pattern pages.

[0056] In yet another example, the present disclosure relates to a system including a plurality of host servers configured to execute one or more of a plurality of computing entities. The system may further include a memory, the portion of the memory having a combination of a portion of a first physical memory of a first type and a portion of a logical pool memory shared among the plurality of host servers. The logical pool memory may be mapped to a second physical memory of the first type, and an amount of the logical pool memory designated as available for allocation to the plurality of computing entities may be greater than the amount of the second physical memory of the first type. The system may further include a logical pool memory controller coupled to the logical pool memory associated with the system and configured to track both a state of whether a page of the logical pool memory assigned to any of the plurality of computing entities is a known pattern page and a relationship between physical memory addresses and logical memory addresses associated with any assigned logical pool memory to any of the plurality of computing entities, and (2) in response to a write operation initiated by the computing entity, executed by a processor associated with any of the plurality of host servers to write any data other than the known pattern, enable the write operation to write the data to any available space in a second physical memory of the first type only up to an extent of the physical memory corresponding to a portion of the logical pool memory previously assigned to the computing entity.

[0057] As part of the system, an amount of logical pool memory associated with the system may be equal to an amount of a second physical memory of the first type combined with an amount of memory corresponding to known pattern pages not backed by any type of physical memory. The logical pool memory controller may be further configured to maintain a mapping table to track both a state of whether a page of logical pool memory assigned to any of the plurality of computing entities is a known pattern page and a relationship between physical memory addresses and logical memory addresses associated with any assigned logical pool memory to any of the plurality of computing entities.

[0058] As part of the system, in one example, a scheduler may be configured to allocate portions of the first physical memory of the first type based on a predicted usage of the first physical memory of the first type by the computing entity. The scheduler may also be configured to dynamically vary an amount of the portion of the first physical memory of the first type allocated to the computing entity based on a usage pattern associated with usage of the first physical memory of the first type allocated to the computing entity. The logical pool memory controller may be further configured to indicate to each of the plurality of computing entities that all pages associated with the logical pool memory initially allocated to any of the plurality of computing entities are known pattern pages.

[0059] It should be understood that the methods, modules, and components described herein are merely exemplary. Alternatively, or in addition, the functions described herein may be performed, at least in part, by one or more hardware logic components. For example, without limitation, exemplary types of hardware logic components that may be used include field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), and the like. In an abstract, yet definite sense, any arrangement of components to achieve the same functionality is effectively "associated" such that the desired functionality is achieved. Thus, any two components combined herein to achieve a particular functionality may be considered to be "associated" with each other such that the desired functionality is achieved, regardless of architecture or intermediate components. Similarly, any two components so associated may also be considered to be "operably connected" or "coupled" with each other to achieve the desired functionality. Just because a component, which may be any other implementation of a device, structure, system, or function, is described herein as being coupled to another component does not mean that the component is necessarily a separate component. As an example, a component A described as being coupled to another component B may be a subcomponent of component B, or component B may be a subcomponent of component A, or components A and B may be coupled subcomponents of another component C.

[0060] Functions related to some examples described in this disclosure may also include instructions stored on non-transitory media. The term "non-transitory media" as used herein refers to any media that stores data and / or instructions that cause a machine to operate in a particular manner. Illustrative non-transitory media include non-volatile media and / or volatile media. Non-volatile media include, for example, hard disks, solid state drives, magnetic disks or tapes, optical disks or tapes, flash memory, EPROM, NVRAM, PRAM, or other such media, or networked versions of such media. Volatile media include, for example, dynamic memory such as DRAM, SRAM, cache, or other such media. Non-transitory media are distinct from, but can be used in conjunction with, transmission media. Transmission media are used to transfer data and / or instructions to and from a machine. Illustrative transmission media include coaxial cable, fiber optic cable, copper wire, and wireless media such as radio waves.

[0061] Moreover, those skilled in the art will recognize that boundaries between the functionality of the operations described above are merely illustrative. Functionality of multiple operations may be combined into a single operation and / or functionality of a single operation may be distributed among additional operations. Moreover, alternative embodiments may include multiple instances of a particular operation, and the order of operations may be changed in various other embodiments.

[0062] While the present disclosure provides specific examples, various modifications and changes may be made without departing from the scope of the present disclosure, as set forth in the claims below. Accordingly, the specification and drawings should be regarded in an illustrative rather than a restrictive sense, and all such modifications are intended to be included within the scope of the present disclosure. Any advantages, benefits, or solutions to problems described herein with respect to the specific examples are not intended to be construed as critical, necessary, or essential features or elements of any or all claims.

[0063] Moreover, the terms "a" and "an" as used herein are defined as one or more. Also, the use of introductory phrases such as "at least one" or "one or more" in the claims should not be construed as implying that the introduction of other claim elements with an indefinite article (e.g., a or an) limits any particular claim containing such introduced claim element to an invention containing at least one such element, even if the same claim contains the introductory phrase "one or more" or "at least one" and an indefinite article (e.g., a or an). The same is true for the use of definite articles.

[0064] Unless otherwise stated, terms such as "first" and "second" are used to arbitrarily distinguish between the elements that such terms describe, and as such, these terms are not necessarily intended to indicate a temporal or other priority therebetween.

Claims

1. allocating to a computing entity a portion of memory associated with the system, the portion of memory including a combination of a portion of a first physical memory of a first type and a portion of a logical pool memory for use with a plurality of computing entities associated with the system, the logical pool memory being mapped to a second physical memory of the first type, and an amount of the logical pool memory designated as available for allocation to the plurality of computing entities being greater than an amount of the second physical memory of the first type; indicating to a logical pool memory controller associated with said logical pool memory that all pages associated with said logical pool memory that are initially allocated to any of said plurality of computing entities are known pattern pages; said logical pool memory controller tracking both a state of whether a page of said logical pool memory assigned to any of said plurality of computing entities is a known pattern page and a relationship between physical and logical memory addresses associated with any assigned logical pool memory to any of said plurality of computing entities; in response to a write operation initiated by the computing entity, the logical pool memory controller permits writing of data to any available space in the second physical memory of the first type only up to an extent of physical memory corresponding to the portion of the logical pool memory previously allocated to the computing entity; The method according to claim 1,

2. an amount of said logical pool memory associated with said system equal to an amount of said second physical memory of said first type combined with an amount of memory corresponding to known pattern pages not backed by any type of physical memory; The method of claim 1.

3. tracking both the state of whether the page of the logical pool memory assigned to any of the plurality of computing entities is a known pattern page and the relationship between the physical memory address and the logical memory address includes maintaining a mapping table. The method of claim 1.

4. The mapping table is implemented using a translation lookaside buffer. The method according to claim 3.

5. and allocating the portion of the first physical memory of the first type based on a predicted usage of the first physical memory of the first type by the computing entity. The method of claim 1.

6. dynamically varying an amount of the portion of the first physical memory of the first type allocated to the computing entity based on a usage pattern associated with a usage of the first physical memory of the first type allocated to the computing entity. The method of claim 1.

7. exposing, via a software mechanism, the portion of the logical pool memory allocated to the computing entity to enable the computing entity to distinguish between the portion of the logical pool memory allocated to the computing entity and the portion of the first physical memory of the first type allocated to the computing entity. The method of claim 1.

8. the logical pool memory is coupled to a processor executing any of the plurality of computing entities that are assigned at least a portion of the logical pool memory associated with the system via respective links managed by a fabric manager; The method of claim 1.

9. 1. A system comprising: a memory, the portion of the memory including a combination of a portion of a first physical memory of a first type and a portion of a logical pool memory associated with the system, the logical pool memory being mapped to a second physical memory of the first type, and an amount of the logical pool memory designated as available for allocation to a plurality of computing entities being greater than an amount of the second physical memory of the first type; a logical pool memory controller coupled to the logical pool memory associated with the system and configured to: (1) track both a state of whether a page of the logical pool memory assigned to any of the plurality of computing entities is a known pattern page and a relationship between physical memory addresses and logical memory addresses associated with any assigned logical pool memory to any of the plurality of computing entities; and (2) in response to a write operation initiated by a computing entity to write any data other than a known pattern, enable the write operation to write the data to any available space in the second physical memory of the first type only up to an extent of physical memory corresponding to a portion of the logical pool memory previously assigned to the computing entity; A system having

10. an amount of said logical pool memory associated with said system equal to an amount of said second physical memory of said first type combined with an amount of memory corresponding to known pattern pages not backed by any type of physical memory; The system of claim 9.

11. the logical pool memory controller is further configured to maintain a mapping table to track both the state of whether the page of the logical pool memory assigned to any of the plurality of computing entities is the known pattern page and the relationship between the physical memory address and the logical memory address associated with any assigned logical pool memory to any of the plurality of computing entities. The system of claim 9.

12. a scheduler configured to allocate the portions of the first physical memory of the first type to the computing entities based on a predicted usage of the first physical memory of the first type by the computing entities. The system of claim 9.

13. the scheduler is configured to dynamically vary an amount of the portion of the first physical memory of the first type allocated to the computing entity based on a usage pattern associated with a usage of the first physical memory of the first type allocated to the computing entity. The system of claim 12.

14. the logical pool memory controller is further configured to indicate to each of the plurality of computing entities that all pages associated with the logical pool memory that are initially allocated to any of the plurality of computing entities are known pattern pages. The system of claim 9.

15. 1. A system comprising: a plurality of host servers configured to execute one or more of the plurality of computing entities; a memory, the portion of memory including a combination of a portion of a first physical memory of a first type and a portion of a logical pool memory shared among the plurality of host servers, the logical pool memory being mapped to a second physical memory of the first type, and an amount of the logical pool memory designated as available for allocation to the plurality of computing entities being greater than an amount of the second physical memory of the first type; a logical pool memory controller coupled to the logical pool memory associated with the system and configured to: (1) track both a state of whether a page of the logical pool memory assigned to any of the plurality of computing entities is a known pattern page and a relationship between physical memory addresses and logical memory addresses associated with any assigned logical pool memory to any of the plurality of computing entities; and (2) in response to a compute entity-initiated write operation executed by a processor associated with any of the plurality of host servers to write any data other than a known pattern, enable the write operation to write the data to any available space in the second physical memory of the first type only up to an extent of physical memory corresponding to a portion of the logical pool memory previously assigned to the compute entity; A system having

Images