Virtual Memory Pooling for GPU Resource Management
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing information handling systems face challenges in efficiently managing memory resources, particularly in scenarios where computational tasks exceed the capacity of internal memory allocations, leading to inefficiencies and potential bottlenecks.
Innovation Solution
The method involves identifying multiple graphic processing units (GPUs) within information handling systems, partitioning their internal memory into a first allocation accessible only by each GPU and a second allocation pooled to form a virtual memory pool accessible by all GPUs. This allows for dynamic memory allocation and sharing among GPUs when processing computational tasks.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If internal memory allocation is increased for each GPU, then processing capacity is improved, but memory resource utilization efficiency deteriorates
Solution Approach 1:
The patent merges the second memory allocations from multiple GPUs into a shared virtual memory pool. This allows memory resources to be consolidated and shared across GPUs, improving overall utilization efficiency while maintaining sufficient processing capacity for individual computational tasks.
Solution Approach 2:
The patent implements dynamic memory allocation where GPUs can access additional memory from the virtual memory pool when needed. This dynamic approach allows memory resources to be flexibly allocated based on actual task requirements rather than static pre-allocation, resolving the contradiction between having enough memory for processing and efficient utilization.
2Reliability
If memory is partitioned into separate allocations for each GPU, then data security and isolation are improved, but memory accessibility and sharing capability deteriorate
Solution Approach 1:
The patent segments memory into two distinct parts: first memory allocation that is exclusively accessible by each GPU for security and isolation, and second memory allocation that is pooled and shareable. This segmentation allows both data security through isolated first allocations and memory sharing through the pooled second allocations.
Solution Approach 2:
Different parts of the memory system have different accessibility properties. The first memory allocation has exclusive local access for security, while the second memory allocation has shared access for versatility. This local quality differentiation resolves the contradiction between isolation and sharing.
3Loss of energy
If a virtual memory pool is created for sharing, then memory resource utilization is improved, but system complexity deteriorates
Solution Approach 1:
The system implements self-service mechanisms where GPUs can automatically access the virtual memory pool when their local memory is insufficient. The memory management operates autonomously without requiring complex external intervention, reducing system complexity while maintaining high resource utilization.
4Productivity
If GPUs access external memory pool when internal memory is exceeded, then processing continuity is improved, but access time and latency worsen
Solution Approach 1:
The system performs preliminary action by pre-establishing the virtual memory pool and configuring memory allocations before computational tasks begin. This preparation ensures that when GPUs need additional memory, they can access it immediately from the pre-configured pool rather than waiting for memory allocation, reducing access time and latency while maintaining processing continuity.
Data Source
AI summary
Virtual memory pooling, including identifying GPUs of respective IHSs, wherein each of the GPUs is associated with a respective internal memory allocation; partitioning, for each GPU, the internal memory allocation associated with the GPU into a first memory allocation and a second memory allocation; allocating, for each GPU, the first memory allocation of the internal memory allocation associated with the GPU as accessible only by the associated GPU; pooling, for each GPU, the second memory allocation of the internal memory allocation associated with the GPU to define a virtual memory pool, the virtual memory pool accessible by each GPU; processing, at a first GPU, a computational task, including: accessing the first memory allocation associated with the first GPU; determining that processing of the computational task exceeds a capacity of the first memory allocation of the first GPU and in response, requesting access to the virtual memory pool.