Multi-GPU Kernel Dependency Tracking for Local-Memory Access
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing technologies face challenges in evenly distributing workload across multiple GPUs, leading to inefficiencies such as idle time and increased latency due to unbalanced processing and memory copy bottlenecks, especially when using GPUs connected via lower speed buses.
Innovation Solution
The method involves dividing kernel responsibility into portions assigned to individual GPUs, generating dependency data, and delaying execution of subsequent kernel portions until dependencies are met, thereby enabling balanced workload distribution and minimizing synchronization overhead.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If multiple GPUs are used to process data, then processing capacity is improved, but workload distribution becomes unbalanced causing idle time and increased latency
Solution Approach 1:
The patent divides the kernel execution into multiple portions that can be independently assigned to different GPUs. Each GPU executes specific portions of the kernel, allowing fine-grained control over workload distribution and eliminating idle time by ensuring each GPU has appropriate work to do.
Solution Approach 2:
The patent implements dynamic dependency tracking where the system monitors which portions of the kernel have completed and uses this information to dynamically determine which portions can be executed next. This dynamic approach allows the system to adapt to varying execution times and balance the workload in real-time.
2Quantity of substance
If data is copied between GPUs, then data availability is improved, but copy size increases creating a bottleneck
Solution Approach 1:
The patent enables each GPU to access data from the local memory of other GPUs through the dependency mechanism. Instead of requiring large data copies, GPUs can access specific dependency data locally, reducing the amount of data that needs to be transferred across the bus.
Solution Approach 2:
The patent extracts only the necessary dependency information from the full dataset and makes it available through the dependency tracking mechanism. This allows GPUs to access only the specific data they need without copying the entire dataset, reducing bandwidth requirements.
3Reliability
If GPUs wait for synchronization, then data consistency is improved, but processing speed is reduced
Solution Approach 1:
The patent performs preliminary dependency tracking during kernel execution, recording which portions need to wait for which other portions. This preliminary action allows the system to optimize synchronization by only waiting for actual dependencies rather than requiring full synchronization, enabling faster processing while maintaining data consistency.
Solution Approach 2:
The patent implements feedback through dependency tracking, where the system continuously monitors which kernel portions have completed and uses this feedback to determine when it is safe to execute dependent portions. This feedback mechanism ensures data consistency is maintained only where necessary, allowing parallel execution to proceed at full speed.
Data Source
AI summary
Methods for graphics processing are provided. One example method includes executing a plurality of kernels using a plurality of graphics processing units (GPUs), wherein responsibility for executing a corresponding kernel is divided into one or more portions each of which being assigned to a corresponding GPU. The method includes generating a plurality of dependency data at a first kernel as each of a first plurality of portions of the first kernel completes processing. The method includes checking dependency data from one or more portions of the first kernel prior to execution of a portion of a second kernel. The method includes delaying execution of the portion of the second kernel as long as the corresponding dependency data of the first kernel has not been met.