Kernel Fusion via JIT Compilation for Hidden GPU Optimization
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing applications targeting specific processor architectures face difficulties in maximizing performance due to the need to expose specialized implementations, which can be undesirable for hardware providers.
Innovation Solution
Implementing kernel fusion techniques through separate compilation and just-in-time compilation of compute-intensive kernels, such as those for GPUs, using an active component to fuse kernels compiled separately and distribute them in low-level portable representations, while maintaining proprietary functionality hidden from users.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If kernel fusion is implemented through separate compilation and just-in-time compilation, then performance and energy efficiency are enhanced, but device complexity and compilation overhead increase
Solution Approach 1:
The compilation process is segmented into separate compilation phases for different kernels, allowing independent optimization and compilation of compute-intensive operations. This enables the system to compile kernels separately and then fuse them at runtime through just-in-time compilation, improving performance without requiring complete recompilation of entire applications.
Solution Approach 2:
Kernels are pre-compiled into low-level portable representations (such as intermediate representation or bytecode) in advance, but the actual fusion and final optimization occurs just-in-time before execution. This preliminary action allows the system to have compilation-ready components without bearing the full overhead of complete compilation at runtime.
2Productivity
If specialized implementations are exposed for specific processor architectures, then performance is maximized, but hardware provider proprietary functionality is disclosed
Solution Approach 1:
An intermediate representation layer is introduced between the high-level application code and the specialized processor implementations. This intermediary allows kernels to be compiled once into a portable low-level representation that can be fused and executed on different architectures without exposing the proprietary implementation details of specific hardware providers.
Solution Approach 2:
The compilation system is designed to produce universal, architecture-independent intermediate representations that can be adapted to multiple processor architectures. This allows the same fused kernel to run on different hardware platforms without requiring architecture-specific code exposure, maintaining both performance optimization and proprietary functionality protection.
3Adaptability or versatility
If multiple separate kernels are compiled and executed, then functionality is maintained, but code size and overhead increase
Solution Approach 1:
Multiple separate kernels are merged into a single fused kernel through just-in-time compilation. The system identifies adjacent compute-intensive kernels in the computation graph, fuses their functionality into one unified kernel, and compiles them together. This reduces the total number of kernel launches and associated overhead while maintaining all required functionality.
Data Source
AI summary
Apparatuses, systems, and techniques are presented to compile code. In at least one embodiment, one or more compilers are to compile one or more compiled portions of code with one or more intermediate representations of one or more portions of code.