GPU Memory Allocator With Direct Block Addressing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional memory management techniques for Graphics Processing Units (GPUs) incur significant overhead due to lookup functions and memory management operations, slowing down processes and compromising security by allowing cross-processor access.
Innovation Solution
A memory allocator using a power of 2 addressing scheme with block sizes allocates memory directly to GPUs, employing hardware registers to secure memory access and prevent unauthorized access, and includes defragmentation to maintain contiguous blocks.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If conventional memory management techniques (CUDA Unified Memory, HMM, virtual memory mapping) are used, then memory can be shared between CPU and GPU, but significant overhead is incurred due to lookup functions and memory management operations
Solution Approach 1:
The patent extracts the memory management overhead (lookup functions, virtual memory mapping) from the memory access path by implementing direct physical address mapping. GPU processes receive direct physical addresses to memory blocks, eliminating the need for virtual-to-physical address translation and lookup operations, thereby resolving the contradiction between memory sharing capability and access speed
Solution Approach 2:
The patent introduces a lightweight memory manager as an intermediary that allocates contiguous physical memory blocks directly to GPU processes without requiring full virtual memory management. This mediator enables efficient direct access while maintaining controlled sharing, balancing versatility and productivity
2Adaptability or versatility
If conventional memory management techniques are used, then memory can be allocated flexibly, but security is compromised by allowing cross-processor access
Solution Approach 1:
The patent segments memory into distinct contiguous blocks and assigns each block to a specific GPU process with dedicated physical addresses. This segmentation isolates memory access boundaries, preventing cross-processor access while maintaining allocation flexibility through the memory manager's ability to allocate and deallocate blocks as needed
Solution Approach 2:
The patent implements local quality control by giving each GPU process exclusive access rights to its allocated memory block. The memory manager enforces boundary control at the local block level, ensuring that each process can only access its designated memory region, thereby improving security without sacrificing overall system flexibility
3Productivity
If power of 2 block sizes are used with direct addressing, then memory access overhead is reduced, but memory fragmentation occurs requiring defragmentation operations
Solution Approach 1:
The patent applies preliminary action by implementing a defragmentation mechanism that proactively consolidates scattered memory blocks into contiguous regions. The memory manager monitors fragmentation and performs defragmentation operations to maintain contiguous block structures, ensuring that the benefits of direct addressing are preserved over time
Solution Approach 2:
The patent introduces dynamics by making the memory allocation system adaptive through defragmentation operations. The system dynamically adjusts memory block positions and consolidates fragmented regions, transforming the static memory layout into a dynamic one that maintains contiguity and efficiency as allocation patterns change
Data Source
AI summary
A computer system with GPUs or other special-purpose or general-purpose processors may allocate processor-specific memory in variable-sized blocks based on 2{circumflex over ( )}n sizing. Once a block has been allocated to a processor's process, that process may access the block directly without additional lookups or other overhead, by using a memory address composed of a block's allocated prefix stored in a set of registers, along with the sequential address in memory. A memory allocator may receive a request for memory of a given size, identify a contiguous block within physical memory, and populate a set of registers with the memory prefix. The processor may store and retrieve data using the memory prefix and the sequential address suffix. When addresses may be managed and stored in a secure operating system-level allocator, memory may be secured against cross-processor access.