Multi-GPU Kernel Dependency Scheduling for Local Memory Access
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing technologies face challenges in efficiently utilizing multiple GPUs for image processing due to unbalanced workloads, idle time, and bottlenecks caused by data copying and synchronization, especially when GPUs are connected via different bus speeds and memory configurations.
Innovation Solution
A method is introduced where kernels are divided into portions assigned to specific GPUs, with dependency data generated and checked before executing subsequent portions, allowing for delayed execution until dependencies are met, thereby optimizing workload distribution and reducing idle time.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If multiple GPUs are used to process data in parallel, then processing throughput should increase, but workload distribution becomes unbalanced causing idle time and synchronization bottlenecks
Solution Approach 1:
The patent segments the data processing workload into distinct portions that can be independently assigned to different GPUs. Each GPU executes specific portions of the processing pipeline, allowing fine-grained control over workload distribution and eliminating idle time by ensuring continuous productive work on each device.
Solution Approach 2:
The patent implements preliminary dependency tracking before GPU execution begins. By pre-determining which portions must be executed on which GPUs and establishing dependency relationships in advance, the system eliminates synchronization bottlenecks and ensures optimal workload distribution from the start of processing.
2Reliability
If data copying is performed between GPUs for synchronization, then data consistency is maintained, but copying time and bandwidth consumption increase
Solution Approach 1:
The patent segments data into portions that are processed independently on different GPUs. By dividing the data rather than copying it, the system maintains data consistency through dependency tracking while eliminating the time-consuming copying process entirely. Each GPU works on its assigned data portions without requiring synchronization copies.
3Adaptability or versatility
If GPUs with different bus speeds are used, then system flexibility increases, but performance disparity and bottleneck effects worsen
Solution Approach 1:
The patent implements dynamic workload assignment where portions of data can be dynamically assigned to any available GPU regardless of its bus speed. The system adapts to heterogeneous hardware configurations by assigning work based on availability and capability rather than assuming uniform performance, thereby maintaining flexibility while optimizing processing efficiency through dynamic scheduling.
Data Source
AI summary
Methods for graphics processing are provided. One example method includes executing a plurality of kernels using a plurality of graphics processing units (GPUs), wherein responsibility for executing a corresponding kernel is divided into one or more portions each of which being assigned to a corresponding GPU. The method includes generating a plurality of dependency data at a first kernel as each of a first plurality of portions of the first kernel completes processing. The method includes checking dependency data from one or more portions of the first kernel prior to execution of a portion of a second kernel. The method includes delaying execution of the portion of the second kernel as long as the corresponding dependency data of the first kernel has not been met.