Runtime System for Dynamic Compute Kernel Selection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current parallel-processing computer systems lack an efficient, stable, and user-friendly software development and execution platform that provides easy-to-use program interfaces, rich library resources, and supports program debugging and profiling, making it difficult to execute programs across different types of parallel-processing computer systems.
Innovation Solution
A runtime system that dynamically identifies and prepares compute kernels for operation requests on various processing elements, using a Language-Specific Interface, compilation scheduler, and program generator to execute operations on graphics processing units (GPUs) and multi-core CPUs, with features like trace and macro caching to optimize performance.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Power
If a parallel-processing computer system uses multiple processing elements with different architectures, then computing capacity and performance are improved, but software compatibility and ease of operation deteriorate
Solution Approach 1:
The patent implements a universal software development platform that can execute programs on multiple types of processing elements (CPUs, GPUs, FPGAs, etc.) through a common interface. The runtime system automatically identifies available processing elements and selects appropriate compute kernels, making the system universally compatible across different hardware architectures without requiring separate software versions for each platform.
Solution Approach 2:
The patent introduces a runtime system as an intermediary layer between the user program and the diverse processing elements. This runtime system includes a compilation scheduler, program generator, and execution manager that translate high-level program requests into architecture-specific compute kernels, shielding users from hardware complexity while enabling multi-architecture execution.
2Speed
If a parallel-processing computer system is designed for high performance, then processing speed is improved, but device complexity increases
Solution Approach 1:
The patent segments the software system into distinct functional modules: a language-specific interface for user programming, a compilation scheduler for task management, a program generator for kernel creation, and a runtime execution system. This segmentation allows each component to be optimized independently while maintaining overall system performance and managing complexity through modular design.
Solution Approach 2:
The patent implements dynamic processing element identification and compute kernel selection that adapts to available hardware resources in real-time. The runtime system dynamically determines which processing elements are available and selects appropriate compute kernels based on current system state, enabling high performance across varying hardware configurations without requiring static, complex hardware designs.
3Adaptability or versatility
If compute kernels are dynamically generated for different processing elements, then adaptability is improved, but loss of time in compilation and preparation increases
Solution Approach 1:
The patent implements a compilation scheduler that performs compute kernel generation and compilation in advance before runtime execution. By pre-compiling compute kernels for multiple processing element types and caching them, the system reduces runtime compilation overhead while maintaining adaptability to different hardware configurations.
Solution Approach 2:
The patent uses program generation techniques that create compute kernels with configurable parameters, allowing the same kernel template to be adapted to different processing elements by changing parameters rather than recompiling entire kernels. This approach maintains hardware adaptability while significantly reducing compilation time through parameter-based customization.
Data Source
AI summary
A runtime system implemented in accordance with the present invention provides an application platform for parallel-processing computer systems. Such a runtime system enables users to leverage the computational power of parallel-processing computer systems to accelerate/optimize numeric and array-intensive computations in their application programs. This enables greatly increased performance of high-performance computing (HPC) applications.