Unified Executable Runtime for Cross-Vendor CPU GPU Scheduling
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current data processing systems struggle to leverage both CPUs and GPUs effectively for data parallel computations, as applications are typically designed for specific vendors and cannot utilize available processing resources across different vendors or types, limiting their ability to perform tasks efficiently.
Innovation Solution
The system allows applications to dynamically load and execute executables on either CPUs or GPUs based on API requests, compiling sources for target processing units and managing execution across multiple processors to optimize resource usage, enabling parallel execution of threads across CPUs and GPUs.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If applications are designed for specific vendors and processing units, then execution reliability is improved, but adaptability and resource utilization deteriorate
Solution Approach 1:
The patent implements a universal executable format that can be executed on multiple types of processing units including CPUs and GPUs from different vendors. The system uses a common executable interface that abstracts the underlying hardware differences, allowing the same application to run on diverse processing resources without vendor-specific modifications, thereby achieving both reliability and adaptability
2Productivity
If separate programming environments are used for CPUs and GPUs, then processing capability is improved, but device complexity and ease of operation worsen
Solution Approach 1:
The patent merges previously separate CPU and GPU programming environments into a unified system. By implementing a common executable format and unified API interface, the system combines the strengths of both processing units while eliminating the need for separate programming environments, thus maintaining high processing capability while reducing complexity
3Manufacturing precision
If vendor-specific interfaces are used for GPU execution, then execution precision is improved, but adaptability and ease of operation deteriorate
Solution Approach 1:
The patent introduces an intermediary layer in the form of a unified executable format and common API interface that sits between the application and the underlying hardware. This intermediary translates generic execution requests into hardware-specific operations, maintaining execution precision for each vendor's processing units while enabling broad vendor compatibility through the standardized interface
Data Source
AI summary
A method and an apparatus that schedule a plurality of executables in a schedule queue for execution in one or more physical compute devices such as CPUs or GPUs concurrently are described. One or more executables are compiled online from a source having an existing executable for a type of physical compute devices different from the one or more physical compute devices. Dependency relations among elements corresponding to scheduled executables are determined to select an executable to be executed by a plurality of threads concurrently in more than one of the physical compute devices. A thread initialized for executing an executable in a GPU of the physical compute devices are initialized for execution in another CPU of the physical compute devices if the GPU is busy with graphics processing threads. Sources and existing executables for an API function are stored in an API library to execute a plurality of executables in a plurality of physical compute devices, including the existing executables and online compiled executables from the sources.