GPU Neural Network Scheduling for Irregular Kernels
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current graphics processing units (GPUs) face challenges in efficiently processing diverse workloads, including machine learning tasks, due to limitations in scheduling mechanisms and parallel processing capabilities, particularly in handling irregular kernels and sparse data structures.
Innovation Solution
A scheduling mechanism is introduced that performs direct scheduling of precompiled neural network (NN) kernel client workloads, utilizing a gather unit to process irregular kernels and a relative address table to manage memory locations, facilitating efficient processing of sparse data structures.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If traditional fixed function computational units are used for graphics processing, then processing reliability is maintained, but processing efficiency and adaptability to diverse workloads deteriorate
Solution Approach 1:
The patent implements a unified computational unit that can execute both traditional graphics processing operations and machine learning neural network operations. The scheduling mechanism directs different workload types to the same physical resources, allowing one computational unit to serve multiple functions - rendering graphics data and processing neural network kernels, thereby improving adaptability without proportionally increasing device complexity
2Productivity
If parallel processing techniques are implemented to increase throughput, then processing speed improves, but scheduling complexity and difficulty of managing diverse kernels increases
Solution Approach 1:
The patent segments the scheduling mechanism into specialized components: a first scheduler for graphics processing workloads and a second scheduler for machine learning workloads. Each scheduler is optimized for its specific workload type, managing parallel processing threads independently. This segmentation allows high throughput for each workload type while reducing the overall scheduling complexity by avoiding a single monolithic scheduler that would need to handle all diversity
3Productivity
If regular SIMD architecture is used for parallel processing, then processing efficiency for regular operations is improved, but handling of irregular kernels and sparse data structures deteriorates
Solution Approach 1:
The patent implements dynamic scheduling that adapts to the specific characteristics of each workload. The scheduler can dynamically allocate processing threads and adjust execution strategies based on whether the workload is regular graphics data or irregular neural network kernels with sparse structures. This dynamic adaptation allows the system to maintain high processing efficiency for regular operations while becoming versatile enough to handle irregular kernels effectively
Data Source
AI summary
An apparatus to facilitate workload scheduling is disclosed. The apparatus includes one or more clients, one or more processing units to processes workloads received from the one or more clients, including hardware resources and scheduling logic to schedule direct access of the hardware resources to the one or more clients to process the workloads.