Thread Scheduler Dispatch for Graph Streaming Processors
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing graph streaming processors face challenges in efficiently dispatching and managing threads for neural network computations, particularly in meeting resource requirements and optimizing resource utilization across multiple processor arrays.
Innovation Solution
The proposed solution involves a thread scheduler that receives a group of threads operating on an input tensor and a weight tensor, calculates resource requirements, and dispatches these threads to processors with sufficient resource availability. The scheduler also schedules a group load instruction to synchronize threads and load inputs into a group load register, ensuring efficient processing across multiple processor arrays.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If threads are dispatched to processors with sufficient resource availability, then thread execution reliability is improved, but resource utilization efficiency deteriorates due to potential idle processors
Solution Approach 1:
The thread scheduler dynamically adjusts thread dispatch decisions based on real-time resource availability and processor state. When processors become available, the scheduler quickly redistributes threads to maximize utilization while ensuring each thread receives sufficient resources for reliable execution.
Solution Approach 2:
The system changes the resource allocation parameters dynamically by monitoring resource availability metrics and adjusting dispatch thresholds. This allows the system to balance between ensuring sufficient resources for reliability and maximizing resource utilization by dispatching threads to processors with adequate but not excessive resources.
2Stability of the object's composition
If group load instructions are used to synchronize threads, then thread coordination is improved, but processing throughput deteriorates due to synchronization overhead
Solution Approach 1:
The patent segments the thread group into smaller sub-groups that can execute independently with minimal synchronization. This reduces the impact of group load instructions by limiting the scope of synchronization overhead to smaller, more manageable thread sets while maintaining necessary coordination.
Solution Approach 2:
The thread scheduler performs preliminary actions by pre-loading required data into processor caches and preparing thread contexts before actual computation begins. This reduces the frequency and impact of synchronization operations during execution, as much of the coordination is established in advance.
Data Source
AI summary
Methods, systems, and apparatuses for graph streaming processing are disclosed. One method includes receiving, by a thread scheduler, a group of threads, calculating a resource requirement for execution of the group of threads, calculating resource availability in a plurality of processors of each of a plurality of processor arrays, dispatching the group of threads to a selected one of plurality of processors of processor arrays, scheduling a group load instruction for all threads of the group of threads, including loading into a group load register a subset of inputs of the input tensor for processing of each thread of the group of threads, wherein the group load register provides the subset of the inputs of the input tensor to the group of threads of the selected one of the plurality of processors, wherein all threads of the group of threads are synchronized when executing the group load instruction.