Automatic Compute Kernel Generation via Template Metaprogramming
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Manually generating compute kernels for mathematical and logical operations on GPUs is challenging due to the complexity of programming and the vast number of possible operations, especially for domain scientists unfamiliar with advanced programming techniques.
Innovation Solution
A software library that uses type system overloading and template metaprogramming to automatically generate compute kernels by combining operators through simple mathematical and logical expressions, allowing users to define operations without specifying inputs, outputs, or types, and compiles them for execution on GPUs.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If compute kernels are manually created by users for each operation, then the operations can be performed on GPU, but the task becomes challenging and time-consuming due to complexity of programming and unlimited sequence of operations
Solution Approach 1:
The system enables self-service by allowing the kernel generation component to automatically generate compute kernels from user-provided mathematical expressions without requiring manual programming. The user simply defines the desired computation, and the system handles the complex kernel generation process automatically.
Solution Approach 2:
A kernel generation component acts as an intermediary between the user's mathematical expressions and the GPU execution. This intermediary automatically translates high-level mathematical operations into optimized compute kernels, eliminating the need for users to directly program complex kernel code.
2Adaptability or versatility
If domain scientists create kernels manually, then they can perform domain-specific operations, but they struggle due to unfamiliarity with complex programming techniques
Solution Approach 1:
The system empowers domain scientists to perform their domain-specific operations by automatically generating the necessary compute kernels from their mathematical expressions, without requiring them to learn complex programming techniques. The kernel generation component serves their needs directly based on their domain knowledge.
Solution Approach 2:
The kernel generation component serves as an intermediary that translates domain scientists' mathematical expressions into executable GPU kernels, bridging the gap between domain knowledge and implementation without requiring the scientists to become programmers.
3Productivity
If operators are combined into a single function, then execution efficiency improves and memory access is reduced, but the automatic generation process becomes more complex
Solution Approach 1:
The system merges multiple operators into a single compute kernel function, which improves execution efficiency by reducing the number of function calls and memory accesses. The kernel generation component automatically combines operators that can be executed together in a single GPU kernel.
Solution Approach 2:
The kernel generation component performs preliminary analysis of the mathematical expressions and operator combinations before generating the final kernel code. This preliminary action includes determining which operators can be combined and optimizing their arrangement for efficient GPU execution.
Data Source
AI summary
Apparatuses, systems, and techniques to receive, by a first processor, a multi-dimensional data structure and an operation to be performed on at least a first element of the multi-dimensional data structure; generate, by the first processor and using at least one of template metaprogramming or operator overloading, one or more operators to perform the operation on at least the first element of the multi-dimensional data structure; and generate, by the first processor, a kernel that comprises the one or more operators, wherein the kernel is executable by a second processor.