Adaptive Task Scheduling for Heterogeneous CPU-GPU Systems
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing techniques for task assignment in heterogeneous processor systems, such as CPU-GPU systems, face challenges in load balancing and overhead reduction, particularly due to offline training dependencies, inaccurate performance modeling, and limitations in work-stealing approaches that incur CPU execution overhead.
Innovation Solution
The implementation of adaptive scheduling through online profiling, where a GPU proxy profiling thread and CPU profiling threads measure execution rates and metrics to calculate a workload distribution ratio, allowing for dynamic assignment of tasks between CPU and GPU without prior training, and utilizing work-stealing queues for efficient distribution.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If offline training is used to train a runtime scheduling algorithm, then the scheduling performance can be improved based on training data, but the system requires repeated training for each new platform and the accuracy depends on how well the training reflects real runtime execution
Solution Approach 1:
The patent performs preliminary profiling of execution rates during an initial runtime execution phase. This profiling data is then used to calculate a distribution ratio that guides subsequent task assignments. By performing the profiling action first and reusing the results, the system avoids repeated offline training for each new platform while maintaining scheduling performance.
2Reliability
If work-stealing with restrictions on stealing is used to address load imbalance, then the GPU cannot initiate communication with the CPU, but the CPU has to act on behalf of the GPU workers or threads incurring overheads on CPU execution
Solution Approach 1:
The patent implements a feedback mechanism where the CPU monitors GPU execution progress and task completion status. Based on this feedback, the CPU dynamically assigns new tasks to the GPU and adjusts the distribution ratio. This feedback loop enables the CPU to efficiently manage GPU work without requiring the GPU to initiate communication, maintaining load balance while minimizing CPU overhead.
Solution Approach 2:
The patent introduces a CPU-based scheduling mechanism that acts as an intermediary between the GPU workers and the task pool. The CPU retrieves tasks from the shared pool and assigns them to the GPU based on the calculated distribution ratio, eliminating the need for direct GPU-to-CPU communication initiation while maintaining efficient load balancing.
3Measurement precision
If accurate performance models are constructed for irregular workloads, then the runtime behavior can be predicted, but constructing accurate models is difficult particularly where distribution of work can vary significantly between processors
Solution Approach 1:
The patent employs a self-service approach where the system profiles its own execution characteristics during runtime. By measuring actual execution rates of the GPU and CPU during an initial phase, the system automatically determines the optimal distribution ratio without requiring external performance models. This self-profiling mechanism handles irregular workloads effectively without the complexity of constructing accurate predictive models.
Data Source
AI summary
Generally, this disclosure provides systems, devices, methods and computer readable media for adaptive scheduling of task assignment among heterogeneous processor cores. The system may include any number of CPUs, a graphics processing unit (GPU) and memory configured to store a pool of work items to be shared by the CPUs and GPU. The system may also include a GPU proxy profiling module associated with one of the CPUs to profile execution of a first portion of the work items on the GPU. The system may further include profiling modules, each associated with one of the CPUs, to profile execution of a second portion of the work items on each of the CPUs. The measured profiling information from the CPU profiling modules and the GPU proxy profiling module is used to calculate a distribution ratio for execution of a remaining portion of the work items between the CPUs and the GPU.