Shared Memory Pool Caching for Multi-CPU Thin Provisioning
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Computing platforms face inefficiencies in memory utilization due to workloads using less memory than allocated, leading to idle resources and high costs from overprovisioning of memory, which is a costly component in data centers.
Innovation Solution
Implement memory thin provisioning using a shared memory pool across multiple compute nodes, dynamically allocating and deallocating memory at page or cache line granularity, with inclusive or exclusive cache configurations, and address translation for shared memory transactions.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If memory is overprovisioned to ensure adequate resources for all workloads, then reliability of memory availability is improved, but cost and resource efficiency deteriorate due to idle memory resources
Solution Approach 1:
Multiple compute nodes are merged to share a common memory pool through the fabric interconnect. Instead of each compute node having dedicated memory, the memory resources are consolidated into a shared pool that serves multiple compute nodes, improving utilization efficiency while maintaining adequate availability for all workloads.
Solution Approach 2:
Memory allocation is made dynamic rather than static. The system continuously monitors workload demands and dynamically allocates memory from the shared pool to compute nodes based on actual needs. This allows memory resources to be flexibly reassigned between workloads, ensuring reliability while reducing the total quantity of physical memory required.
2Ease of operation
If memory is allocated statically to each compute node, then ease of operation is improved, but memory resource efficiency deteriorates due to idle resources when workloads use less allocated memory
Solution Approach 1:
The memory management system operates autonomously to monitor workload demands and automatically allocate memory from the shared pool to compute nodes as needed. This self-service approach maintains ease of operation for users while dramatically improving memory utilization efficiency, as memory is dynamically assigned based on actual workload requirements rather than static allocation.
Solution Approach 2:
The system implements continuous feedback monitoring of workload memory consumption patterns. Based on this feedback, the memory management controller dynamically adjusts allocations from the shared pool, ensuring that compute nodes receive adequate memory while minimizing idle resources. This feedback mechanism maintains operational simplicity while optimizing utilization.
3Adaptability or versatility
If memory capacity is increased to handle growing workload demands, then adaptability is improved, but cost deteriorates due to expensive memory components
Solution Approach 1:
The shared memory pool serves multiple compute nodes simultaneously, making the memory resources universal rather than dedicated. This multi-functionality allows the same physical memory capacity to support a larger total workload demand across the system, improving adaptability without proportionally increasing the quantity of memory required.
Solution Approach 2:
Memory capacity allocation is dynamic and adaptable to changing workload demands. When workload requirements increase, the system can allocate additional memory from the shared pool to affected compute nodes. When demands decrease, memory is freed back to the pool for reallocation. This dynamic approach provides adaptability while optimizing the total memory capacity needed.
Data Source
AI summary
Examples described herein relate to memory thin provisioning in a memory pool of one or more dual in-line memory modules or memory devices. At any instance, any central processing unit (CPU) can request and receive a full virtual allocation of memory in an amount that exceeds the physical memory attached to the CPU (near memory). A remote pool of additional memory can be dynamically utilized to fill the gap between allocated memory and near memory. This remote pool is shared between multiple CPUs, with dynamic assignment and address re-mapping provided for the remote pool. To improve performance, the near memory can be operated as a cache of the pool memory. Inclusive or exclusive content storage configurations can be applied. An inclusive cache configuration can include an entry in a near memory cache also being stored in a memory pool whereas an exclusive cache configuration can provide an entry in either a near memory cache or in a memory pool but not both. Near memory cache management includes current data location tracking, access counting and other caching heuristics, eviction of data from near memory cache to pool memory and movement of data from pool memory to memory cache.