Unified Memory GPU Localized Mode Latency Reduction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
High latency, increased energy consumption, and reduced memory utilization efficiency occur when graphics processing units (GPUs) access memory allocated to other GPUs in a multi-GPU system, due to the need for remote memory access and data duplication in the last level cache.
Innovation Solution
The implementation of a memory localization feature that allows the GPU to selectively confine software function execution and associated data storage resources to locally-connected processing/storage components, using affinity masks and address mapping hardware to restrict memory access and perform localized memory mapping.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If unified memory addressing is used to enable all processors to access any memory, then memory access flexibility and parallelism are improved, but memory access latency and energy consumption increase when accessing remote memory
Solution Approach 1:
The patent segments the unified memory space into local memory regions associated with each GPC. Each GPC has a local memory region that can be accessed with low latency, while other GPCs can access this memory through the unified addressing mechanism with higher latency. This segmentation allows the system to maintain unified memory flexibility while enabling faster local access for computations that benefit from data locality.
Solution Approach 2:
The patent implements local quality by providing each GPC with preferential access rights to its associated local memory region. The memory management unit (MMU) is configured to route memory access requests to local memory regions when possible, providing faster access for local computations while maintaining the ability to access remote memory through the unified addressing mechanism when necessary.
2Ease of operation
If unified memory addressing is used to enable cross-GPU memory access, then programming simplicity is improved, but data duplication in last level cache increases energy consumption
Solution Approach 1:
The patent implements preliminary action by having the memory management unit (MMU) pre-configure memory region associations with GPCs during system initialization or memory allocation. This pre-configured mapping allows the MMU to automatically route memory access requests to appropriate local memory regions without requiring explicit programming instructions, thereby maintaining programming simplicity while reducing energy-consuming remote memory accesses.
3Speed
If fully connected crossbar is implemented between each GPC and memory subsystem, then memory access speed is improved, but device complexity and cost increase
Solution Approach 1:
The patent segments the memory subsystem into local memory regions served by individual GPCs through dedicated crossbar connections. Each GPC has its own crossbar connection to its local memory region, providing fast access. The unified memory addressing mechanism provides a secondary path for accessing other GPCs' memory regions. This segmentation reduces the complexity of any single crossbar while maintaining overall system performance.
Data Source
AI summary
A GPU can selectively confine software function execution and associated data storage resources to locally-connected processing/storage components, thereby minimizing latency and other overhead that would otherwise be needed to access more remote resources. The GPU can selectively permit other software function execution and associated data storage resources to range across non-locally-connected processing/storage components when more processing and/or storage resources are required.