Neural Core Task Manager for Parallel NPU Scheduling
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing neural processing units (NPUs) face challenges in efficiently managing tasks due to limitations in distributing and scheduling computations across multiple computation devices, leading to increased latency and overhead from synchronization signals and CPU intervention.
Innovation Solution
A neural processing device with a task manager that includes a neural core, core global, and task manager to efficiently distribute and manage tasks, minimizing CPU intervention by directly handling task dependencies and synchronization signals among neural cores, and selectively generating completion reports.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If neural processing units use multiple computation devices operating in parallel, then computation efficiency is enhanced, but task management complexity increases
Solution Approach 1:
The task manager is segmented into distinct functional units: a task passage for generating and transmitting task information, a done passage for receiving completion signals and generating reports, and a check-in buffer for storing task descriptors. This segmentation allows each unit to handle specific aspects of task management independently, reducing overall management complexity while maintaining parallel computation efficiency
Solution Approach 2:
The task manager acts as an intermediary between the command processor and neural cores, and between neural cores and the core global. By introducing this intermediate layer, the system manages the complexity of coordinating multiple parallel computation devices without requiring direct complex interactions between all components
2Reliability
If the task manager generates completion reports for all tasks, then task monitoring is comprehensive, but CPU overhead increases
Solution Approach 1:
The system changes the parameter of completion report generation from a constant (always generate) to a conditional operation (generate only when needed). The task manager selectively generates completion reports based on task requirements and system state, reducing unnecessary CPU overhead while maintaining comprehensive monitoring where required
Solution Approach 2:
Instead of generating completion reports for all tasks without distinction, the system applies partial action by generating reports only for tasks that require monitoring or have not yet been reported. This selective approach reduces CPU overhead while maintaining necessary monitoring coverage
3Measurement precision
If the system uses check-in and check-out procedures for task descriptors, then task tracking accuracy is improved, but processing latency increases
Solution Approach 1:
The check-in procedure performs preliminary action by registering task descriptors in the check-in buffer before tasks are executed. This advance registration establishes accurate tracking from the outset, ensuring task tracking accuracy without requiring extensive verification during and after execution, thereby reducing overall latency
Data Source
AI summary
A neural processing device and a method for managing tasks thereof are provided. The neural processing device includes a neural core configured to perform a task and generate a completion signal for completion of the task, a core global configured to transfer task information for the task to the neural core and receive the completion signal of the task from the neural core, and a task manager configured to generate and transmit the task information to the core global, receive the completion signal from the core global, generate a completion report, and transmit the completion report.