Parallel Processing Subsystem Priority Mapping
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing parallel processing subsystems face challenges in scheduling work components, leading to unacceptable latency and throughput issues due to strict issue-order execution and inter-stream dependencies, particularly in latency-sensitive and high-performance simulation applications.
Innovation Solution
Implementing a method to prioritize computational work components within parallel processing subsystems by associating device priorities with CUDA streams, allowing for flexible scheduling and kernel preemption, thereby optimizing resource allocation and reducing latency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Device complexity
If work components are executed in strict issue-order on the least recently used parallel processing subsystem resources, then resource allocation is simplified, but latency-sensitive work components experience unacceptable latency and throughput
Solution Approach 1:
The patent introduces dynamic priority-based scheduling that allows the execution order to change based on priority levels rather than following a fixed issue-order pattern. The scheduler dynamically selects work components from different streams based on their priority assignments, enabling latency-sensitive work to be executed ahead of less time-critical work while maintaining manageable scheduling complexity through structured priority levels.
Solution Approach 2:
The patent changes the scheduling parameter from strict issue-order to priority-based selection. By assigning priority values to different work components and streams, the system transforms the scheduling mechanism to select work components based on their priority rather than their issuance sequence, thereby reducing latency for high-priority work while maintaining overall system manageability.
2Productivity
If internal work components occupy most of the subsystem resources, then resource utilization is maximized, but halo work components get stalled blocking dependent work components in other streams
Solution Approach 1:
The patent applies local quality by assigning different priority levels to different work components based on their characteristics and dependencies. Halo work components that are critical for enabling other streams are assigned higher priorities, while internal work components with fewer dependencies receive lower priorities. This localized priority assignment ensures that resource allocation is optimized for each specific work component's needs rather than treating all work uniformly.
Solution Approach 2:
The priority-based scheduling system provides feedback mechanisms where the scheduler continuously evaluates the state of different work components and streams, adjusting execution selection based on current system state. When halo work components are ready and have high priority, the scheduler responds by allocating resources to them, preventing blocking of dependent work while maintaining high overall resource utilization through dynamic adaptation.
Data Source
AI summary
One embodiment sets forth a method for assigning priorities to kernels launched by a software application and executed within a stream of work on a parallel processing subsystem. First, the software application assigns a desired priority to a stream using a call included in the API. The API receives this call and passes it to a driver. The driver maps the desired priority to an appropriate device priority associated with the parallel processing subsystem. Subsequently, if the software application launches a particular kernel within the stream, then the driver assigns the device priority associated with the stream to the kernel before adding the kernel to the stream for execution on the parallel processing subsystem. Advantageously, by assigning priorities to streams and, subsequently, strategically launching kernels within the prioritized streams, an application developer may fine-tune the software application to increase the overall processing efficiency of the software application.