GPU Hardware Support for Generic Pointer Memory Access
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The use of generic pointers in GPU workloads incurs a significant performance penalty due to compiler emulation, leading to increased code size and runtime, as conditionals are generated to handle memory access messages, resulting in a slowdown of up to 10% or more.
Innovation Solution
Implementing generic pointers in GPU hardware by using memory access messages with a memory type identifier, allowing the processing resource to compute addresses at runtime, eliminating the need for compiler-generated conditionals and improving performance.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If compiler emulation is used to handle generic pointers, then code compatibility is maintained, but performance deteriorates due to increased code size and runtime
Solution Approach 1:
The patent replaces the compiler-based mechanical emulation system with a hardware-based memory access control system. Instead of using compiler-generated conditionals to handle generic pointers, the hardware uses a memory access control unit that receives memory access messages with memory type identifiers and directly routes them to the appropriate memory controller, eliminating the need for runtime conditional logic.
Solution Approach 2:
The patent introduces a memory access control unit as an intermediary between the processing resource and memory controllers. This intermediary receives memory access messages containing memory type identifiers and translates them into appropriate routing decisions, enabling generic pointers to be handled efficiently without requiring the processing resource to execute complex conditional logic.
2Adaptability or versatility
If compiler-generated conditionals are used to handle memory access messages, then generic pointer functionality is achieved, but code size and runtime increase
Solution Approach 1:
The patent replaces compiler-generated conditional code with a hardware-based memory access control mechanism. The processing resource simply issues memory access messages with memory type identifiers, and the hardware automatically routes these to the correct memory controller, eliminating the need for large blocks of conditional code that would otherwise be generated by the compiler.
3Ease of operation
If compiler emulation handles generic pointers, then programming simplicity is maintained, but execution time increases due to conditional logic
Solution Approach 1:
The patent replaces runtime conditional execution with hardware-based memory access control. The processing resource continues to use simple generic pointer syntax in high-level languages, but instead of executing complex conditionals at runtime, the hardware automatically handles the memory type routing based on memory type identifiers in the access messages, significantly reducing execution time.
Data Source
AI summary
Systems and methods for supporting generic pointers in hardware of a graphics processing unit (GPU) are provided. In various examples, a GPU includes multiple sub-cores each having a processing resource and a load/store pipeline. The processing resource is operable to receive a memory access message including a generic pointer. The processing resource is further operable to output a load or store operation to the load/store pipeline based on the memory access message, in which an address for the load or store operation is determined based on a base address of a named memory type of a plurality of named memory types referenced by the generic pointer and an offset into a memory of the named memory type. The load/store pipeline is operable to, responsive to receipt of the load or store operation, access the memory at the address.