Unified API for Cross-Vendor CPU GPU Parallel Computing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current data processing systems struggle to leverage processing resources from both CPUs and GPUs effectively, as they are configured through separate programming environments that are not compatible with each other, limiting the ability to utilize the combined processing capabilities of these devices, especially when applications are designed for specific GPUs or CPUs.
Innovation Solution
The development of APIs that allow for data parallel computing across CPUs and GPUs, enabling applications to generate program objects and executable codes that can be executed concurrently across multiple compute units, including CPUs and GPUs, by selecting and allocating available processing resources based on capability requirements and optimizing thread group sizes for efficient execution.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If separate programming environments are used for CPUs and GPUs, then each processor type can be optimized for its specific architecture, but the ability to leverage processing resources from both CPUs and GPUs is limited
Solution Approach 1:
The patent implements a universal programming interface that allows applications to execute on multiple processor types (CPUs and GPUs) through a common API. The compute shader program can be dynamically dispatched to appropriate processing resources based on availability and capability, enabling one program to serve multiple processor architectures without requiring separate programming environments for each.
Solution Approach 2:
The patent introduces an intermediary layer (the unified API and runtime system) that sits between the application and the diverse processing resources. This intermediary handles the complexity of processor-specific optimizations and resource management, allowing applications to interact with a standardized interface while the underlying system manages the diversity of CPU and GPU architectures.
2Productivity
If vendor-specific interfaces are used for GPUs, then GPU performance can be maximized, but applications cannot leverage processing resources across different GPU vendors or between GPUs and CPUs
Solution Approach 1:
The unified compute shader API enables a single program to target multiple GPU vendors and CPU architectures. The system automatically adapts the compute shader execution to the specific processor being used, allowing applications to leverage processing resources from Intel CPUs, NVIDIA GPUs, AMD GPUs, and other vendors without requiring vendor-specific code paths.
3Productivity
If applications are designed to run on specific GPUs or CPUs, then optimization for that specific hardware is achieved, but the application cannot utilize other available processing resources
Solution Approach 1:
The system dynamically determines at runtime which processing resources (CPU cores or GPU compute units) are available and suitable for executing the compute shader program. This dynamic resource selection allows applications to automatically utilize whatever processing resources are available without requiring static compilation for specific hardware targets, enabling flexible resource utilization while maintaining hardware optimization.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
A method and an apparatus for a parallel computing program calling APIs (application programming interfaces) in a host processor to perform a data processing task in parallel among compute units are described. The compute units are coupled to the host processor including central processing units (CPUs) and graphic processing units (GPUs). A program object corresponding to a source code for the data processing task is generated in a memory coupled to the host processor according to the API calls. Executable codes for the compute units are generated from the program object according to the API calls to be loaded for concurrent execution among the compute units to perform the data processing task.