Task Preemption in Deep Learning Accelerator Hardware
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The conventional task preemption scheme in deep learning accelerators is highly inefficient, leading to increased overhead and quality degradation in deep learning tasks due to manually created breakpoints and overlapping regions in input data.
Innovation Solution
A method for task preemption in deep learning accelerators that involves executing a first task using a neural network, sending an interrupt request to the DLA driver upon completion of a neural network operation, receiving and executing a second task to completion before resuming the first task, thereby dynamically managing task execution and reducing overhead.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If manually created breakpoints are used for task preemption, then task preemption capability is provided, but overhead increases and efficiency decreases
Solution Approach 1:
The system uses hardware-level interrupt mechanisms that automatically trigger preemption without requiring manual breakpoint configuration. The DLA driver monitors task completion events and autonomously manages preemption, eliminating the need for programmer-defined breakpoints and reducing overhead.
Solution Approach 2:
The patent replaces the software-based manual breakpoint mechanism with a hardware-level interrupt system. Instead of relying on programmed breakpoints in the computation graph, the system uses automatic interrupt requests generated by the DLA hardware when tasks complete, enabling more efficient preemption management.
2Adaptability or versatility
If overlapping regions are added to input data for preemption, then task switching is enabled, but quality degradation occurs
Solution Approach 1:
The system segments tasks at the operation level rather than requiring overlapping data regions. By dividing the computation into discrete operations that can be independently tracked and interrupted, the system enables task switching without needing to replicate or overlap input data, thus maintaining output quality.
3Adaptability or versatility
If more breakpoints are created for preemption opportunities, then task preemption flexibility increases, but overhead increases
Solution Approach 1:
The system dynamically manages preemption opportunities through hardware interrupts that are triggered only when tasks actually complete. This dynamic approach provides preemption flexibility when needed while avoiding the overhead of continuously monitoring or pre-configuring numerous fixed breakpoints throughout the computation graph.
Data Source
AI summary
Deep learning accelerator (DLA) hardware performs task preemption. The DLA hardware executes a first task by using a neural network of multiple layers on a given input. In response to a stop command from a DLA driver to stop execution of the first task, the DLA hardware completes a current operation of the neural network and sending an interrupt request (IRQ) to the DLA driver. The DLA hardware then receives a second task from the DLA driver. The DLA hardware executes the second task to completion before resuming the execution of the first task.