Dynamic CPU-GPU Task Scheduling via Runtime Compilation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current data processing systems struggle to leverage both CPUs and GPUs effectively for data parallel computations, as applications are typically designed for specific vendors and cannot utilize both processing resources simultaneously or across different vendors.
Innovation Solution
The system allows applications to dynamically load and execute executables on either CPUs or GPUs based on available processing resources, using APIs to manage the distribution of tasks and compile sources for optimal execution across multiple processing units.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If applications are designed to run on specific GPUs using vendor specific interfaces, then they can execute on dedicated hardware, but they cannot leverage CPUs or GPUs from different vendors
Solution Approach 1:
The patent implements a universal programming environment that allows applications to execute on multiple types of processors (CPUs and GPUs from different vendors) through a common interface. The system translates vendor-specific instructions into a universal intermediate representation that can be executed on any supported processor type, eliminating the need for separate vendor-specific programming environments.
Solution Approach 2:
The patent introduces an intermediate representation layer that acts as a mediator between the application and the underlying hardware. This intermediate layer translates high-level application code into processor-specific instructions at runtime, allowing the same application to run on different vendor-specific processors without modification.
2Productivity
If applications are designed for specific vendors, then they can optimize for that vendor's hardware, but they cannot utilize both CPUs and GPUs simultaneously
Solution Approach 1:
The patent implements dynamic processor selection and task distribution that adapts to available hardware resources at runtime. The system can dynamically allocate tasks to CPUs or GPUs based on workload characteristics, resource availability, and performance requirements, allowing flexible utilization of multiple processor types simultaneously.
Solution Approach 2:
The patent divides the application workload into segments that can be executed on different processor types. The system automatically partitions tasks between CPU and GPU based on their respective strengths, allowing simultaneous execution on multiple processors while maintaining optimization for each processor type.
3Ease of manufacture
If separate programming environments are used for GPUs and CPUs, then each can be optimized independently, but it becomes very difficult to leverage both for processing resources
Solution Approach 1:
The patent provides a single universal programming environment that maintains the ability to independently optimize for different processor types through a unified interface. Developers write application code once in the universal environment, and the system handles the complex task of generating and managing vendor-specific optimized code for different processors.
Data Source
AI summary
A method and an apparatus that schedule a plurality of executables in a schedule queue for execution in one or more physical compute devices such as CPUs or GPUs concurrently are described. One or more executables are compiled online from a source having an existing executable for a type of physical compute devices different from the one or more physical compute devices. Dependency relations among elements corresponding to scheduled executables are determined to select an executable to be executed by a plurality of threads concurrently in more than one of the physical compute devices. A thread initialized for executing an executable in a GPU of the physical compute devices are initialized for execution in another CPU of the physical compute devices if the GPU is busy with graphics processing threads. Sources and existing executables for an API function are stored in an API library to execute a plurality of executables in a plurality of physical compute devices, including the existing executables and online compiled executables from the sources.