Auto-Throttling Compute Tasks for GPU Memory Optimization
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional memory allocation techniques in parallel processor architectures, such as GPUs, are inefficient as they allocate the same amount of memory to each thread regardless of usage, leading to wasted resources and limited concurrency due to physical memory constraints.
Innovation Solution
The method involves auto-throttling encapsulated compute tasks by configuring the processor to activate a varying number of parallel processing units based on the memory requirements of scheduled tasks, allowing for dynamic allocation of shader local memory across discrete throttling modes.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of manufacture
If the same amount of local memory is allocated to each thread, then memory allocation is simplified, but memory usage efficiency deteriorates
Solution Approach 1:
The system dynamically adjusts the number of active processing units based on task requirements. Instead of static memory allocation, the processor configures different numbers of active processing units (first number in non-throttled mode, second number in throttled mode) to match the memory needs of scheduled tasks, thereby improving memory efficiency while maintaining allocation simplicity
Solution Approach 2:
The system changes the operational parameters of the processor by switching between different throttling modes. Each mode corresponds to a different configuration of active processing units, allowing the system to adapt memory allocation to task requirements by changing the number of active units rather than changing individual thread allocations
2Productivity
If more threads are allocated to execute concurrently, then processor productivity increases, but physical memory requirements increase
Solution Approach 1:
The system dynamically configures the number of active processing units based on the memory requirements of scheduled tasks. When tasks with high memory requirements are scheduled, the system switches to throttled mode with fewer active units, reducing total memory consumption while maintaining productive task execution
Solution Approach 2:
The processor is segmented into multiple independent processing units that can be selectively activated. This segmentation allows the system to activate only the necessary number of units based on task requirements, thereby maintaining productivity for scheduled tasks while reducing overall memory consumption by keeping some units inactive
3Speed
If the processor operates in non-throttled mode with maximum processing units active, then task execution speed increases, but memory consumption increases
Solution Approach 1:
The system dynamically switches between non-throttled and throttled modes based on task requirements. When tasks are scheduled that can execute with fewer memory resources, the system transitions to throttled mode, reducing active processing units and thereby reducing memory consumption while maintaining adequate execution speed for the scheduled tasks
Data Source
AI summary
Systems and methods for auto-throttling encapsulated compute tasks. A device driver may configure a parallel processor to execute compute tasks in a number of discrete throttled modes. The device driver may also allocate memory to a plurality of different processing units in a non-throttled mode. The device driver may also allocate memory to a subset of the plurality of processing units in each of the throttling modes. Data structures defined for each task include a flag that instructs the processing unit whether the task may be executed in the non-throttled mode or in the throttled mode. A work distribution unit monitors each of the tasks scheduled to run on the plurality of processing units and determines whether the processor should be configured to run in the throttled mode or in the non-throttled mode.