Dynamic Memory Pooling via Accelerator Fabric Manager
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current volatile memory allocation in computing systems is inefficient, as a significant portion of allocated memory remains unused due to static allocations per CPU, leading to wasted resources, especially in cloud networks where memory usage is often below 50%.
Innovation Solution
Implementing a dynamic memory allocation system using an accelerator that manages volatile memory through a memory fabric manager, allowing on-the-fly allocation and optimization of memory resources across multiple processors, and enabling the pooling of both memory and compute resources to reduce waste and improve utilization.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If static memory allocation is used for each CPU, then memory allocation is simple and guaranteed, but memory utilization efficiency deteriorates significantly
Solution Approach 1:
The patent implements dynamic memory allocation where the accelerator can allocate, deallocate, and reallocate memory to CPUs based on actual usage patterns. The system transitions from static predetermined allocations to dynamic on-demand allocations, allowing memory to be efficiently redistributed among CPUs according to their current workload requirements, thereby resolving the contradiction between allocation reliability and utilization efficiency.
Solution Approach 2:
The system changes the allocation parameters from fixed static sizes to variable dynamic sizes. The accelerator monitors memory usage patterns and adjusts allocation parameters in real-time, allowing each CPU to receive the precise amount of memory needed based on actual usage rather than predetermined fixed allocations, thus improving utilization while maintaining allocation guarantees through controlled dynamic distribution.
2Speed
If volatile memory is dedicated to each CPU, then access performance is guaranteed, but resource waste increases as cloud networks grow
Solution Approach 1:
The patent merges the memory pools of multiple CPUs into a unified memory space managed by the accelerator. Instead of each CPU having dedicated isolated memory, the system combines all volatile memory resources into a shared pool that can be dynamically allocated to any CPU needing it, thereby eliminating memory waste while maintaining fast access performance through the accelerator's coordinated management.
Solution Approach 2:
The accelerator serves as an intermediary between the memory subsystem and the CPUs. It mediates memory access by managing the pooled memory resources, allocating them to requesting CPUs, and coordinating access to ensure performance guarantees. This intermediary layer enables efficient resource sharing while maintaining the performance benefits of fast memory access.
3Device complexity
If memory is statically allocated per CPU, then system complexity is low, but adaptability to workload changes deteriorates
Solution Approach 1:
The accelerator implements self-service memory management by autonomously monitoring memory usage patterns, detecting workload changes, and automatically reallocating memory resources without requiring manual intervention. The system serves itself by making intelligent allocation decisions based on observed usage patterns, thereby achieving high adaptability while keeping the overall system architecture relatively simple.
Solution Approach 2:
The system incorporates feedback mechanisms where the accelerator continuously monitors memory usage patterns from multiple CPUs and uses this information to adjust future allocations. This feedback loop enables the system to adapt to changing workloads by learning from past usage patterns and making more accurate predictions about future memory needs, thereby improving adaptability without significantly increasing complexity.
Data Source
AI summary
Apparatuses, systems, and techniques that allocate volatile memory associated with a particular processor and/or device to one or more other processors. At least one embodiment pertains to maintaining a memory map of volatile memory allocated to a plurality of processors.