Accelerated Processing Device Task Partitioning for GPU Utilization
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current computing systems face limitations in utilizing graphics processing units (GPUs) due to constrained programming environments, separate memory systems, inefficient scheduling, and power consumption issues, which hinder the efficient execution of workloads that require data parallel operations and heavy register resources.
Innovation Solution
The implementation of a method that allows simultaneous processing of multiple tasks within an accelerated processing device (APD) by managing resource utilization, enabling full GPU resource utilization through techniques such as temporary context switching and prioritized scheduling.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If single task launch system is used, then programming simplicity is maintained, but GPU resource utilization is insufficient
Solution Approach 1:
The patent segments the task execution model into multiple independent tasks that can be launched concurrently. Instead of a single monolithic task, the system divides work into discrete task units that can be managed independently, allowing multiple tasks to coexist in the task queue and utilize GPU resources simultaneously without complex programming overhead.
Solution Approach 2:
The patent implements preliminary action by pre-launching multiple tasks into the task queue before GPU execution begins. Tasks are prepared and staged in advance, allowing the GPU to continuously process work without idle periods. This eliminates wait states and ensures full utilization of GPU resources while maintaining simple task submission semantics for developers.
2Power
If heavy register resource tasks are executed, then computational capability is utilized, but occupancy of compute units decreases
Solution Approach 1:
The patent merges multiple tasks with different resource requirements into a single execution environment. By combining tasks that use heavy register resources with tasks that use light register resources, the system achieves balanced occupancy of compute units. The task queue manages this merging automatically, allowing diverse workloads to share GPU resources efficiently without manual intervention.
Solution Approach 2:
The patent introduces dynamic task scheduling that adapts to current GPU resource availability. The system dynamically adjusts which tasks are launched based on real-time occupancy metrics and resource utilization. When heavy register tasks are detected, the scheduler dynamically introduces lighter tasks to maintain optimal occupancy, creating a dynamic balance between computational intensity and resource utilization.
3Productivity
If multiple tasks are concurrently scheduled, then GPU resource utilization improves, but scheduling complexity increases
Solution Approach 1:
The patent implements self-service scheduling where the task queue autonomously manages task launch, monitoring, and resource allocation without external intervention. The system automatically tracks GPU utilization, manages task priorities, and makes scheduling decisions based on predefined policies. This self-service approach handles the complexity internally while presenting a simple interface to developers, maintaining high resource utilization without increasing perceived scheduling complexity.
4Productivity
If fine grain offload is implemented, then processing efficiency improves, but memory overhead becomes unacceptable
Solution Approach 1:
The patent transitions from a single-dimension memory model to a multi-dimensional memory hierarchy. Instead of relying solely on external memory, the system introduces multiple memory dimensions including local memory, shared memory, and global memory. This dimensional expansion allows fine-grained offload operations to access data from closer memory layers, reducing the overhead and energy cost of memory access while maintaining high processing efficiency.
Data Source
AI summary
Embodiments describe herein provide an apparatus, a computer readable medium and a method for simultaneously processing tasks within an APD. The method includes processing a first task within an APD. The method also includes reducing utilization of the APD by the first task to facilitate simultaneous processing of the second task, such that the utilization remains below a threshold.