Multi-threaded Processing Unit Pre-emption via Task Segmentation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing multi-threaded processing systems, particularly GPUs, face significant context switch latency due to large context state sizes and long pipelines, leading to inefficiencies in executing tasks with varying priorities.
Innovation Solution
A method and system that divide tasks into smaller subtasks, allowing for preemptive execution of higher-priority tasks without the need for complex scheduling logic or context switching, by using a pre-processor to manage task execution and prioritize tasks based on size and priority.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Device complexity
If the GPU waits for current work to complete processing through a pipeline (WFI approach), then context state saving is eliminated, but context switch latency increases due to long pipeline completion time
Solution Approach 1:
The patent segments a task into multiple subtasks, each small enough to complete within a threshold number of clock cycles. This segmentation allows the GPU to issue one subtask, complete it quickly, and then switch to another task without waiting for long pipeline completion, thereby reducing context switch latency while maintaining a relatively simple context switching mechanism.
2Adaptability or versatility
If the GPU stops processing of all work currently in the pipeline and saves context state to memory, then a different context can be loaded, but context switch latency increases due to large context state size and saving time
Solution Approach 1:
By dividing tasks into small subtasks that can be completed quickly, the patent enables frequent context switches without requiring saving of large context states. The segmentation allows the system to switch contexts at subtask boundaries, avoiding the need to save and restore large GPU context states, thereby reducing context switch latency while maintaining adaptability.
Solution Approach 2:
The patent preemptively divides tasks into small subtasks before execution, ensuring that each subtask can be completed within a threshold time. This preliminary action enables the GPU to switch contexts quickly at predetermined boundaries without requiring complex context saving operations, thus reducing context switch latency while preserving context switching capability.
3Ease of operation
If tasks are executed without division into subtasks, then programming is simpler, but context switch latency increases due to long task execution time
Solution Approach 1:
The patent automatically segments tasks into subtasks without requiring programmer intervention. The segmentation is performed by the system, which divides a task into multiple subtasks and manages their execution and preemption. This maintains programming simplicity while enabling reduced context switch latency through automatic task division and management.
4Adaptability or versatility
If complex scheduling logic and context switching capabilities are implemented at each processing engine, then task preemption is more flexible, but device complexity increases
Solution Approach 1:
The patent introduces an intermediary mechanism that manages task segmentation and preemption at a higher level, rather than implementing complex scheduling logic at each processing engine. The system acts as an intermediary between task submission and execution, dividing tasks into subtasks and managing preemption points, thereby reducing device complexity while maintaining task preemption flexibility.
Data Source
AI summary
A multi-threaded processing unit includes a hardware pre-processor coupled to one or more processing engines (e.g., copy engines, GPCs, etc.) that implement pre-emption techniques by dividing tasks into smaller subtasks and scheduling subtasks on the processing engines based on the priority of the tasks. By limiting the size of the subtasks, higher priority tasks may be executed quickly without switching the context state of the processing engine. Tasks may be subdivided based on a threshold size or by taking into account other consideration such as physical boundaries of the memory system.