Neural Processor Branching via Task Manager Enqueue
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing neural processor circuits face challenges in efficiently managing branching operations in neural networks, leading to increased latency and power consumption when transitioning between different neural networks.
Innovation Solution
A neural processor circuit is designed with a task manager circuit that selects a branch based on the output of a preceding task, enqueueing the selected branch and transmitting tasks from the selected branch to a data processor circuit for execution, thereby reducing latency and improving operational efficiency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of time
If a neural processor circuit uses a traditional sequential execution approach for neural network tasks, then the circuit structure is simpler, but the latency in transitioning between different neural networks increases
Solution Approach 1:
The patent applies preliminary action by pre-enqueuing multiple segment branches (first branch and second branch) in the task queue before execution is needed. The task manager circuit prepares alternative task segments in advance, so when a branch decision is made, the selected segment is already ready for immediate execution, eliminating transition latency between neural networks.
Solution Approach 2:
The patent segments the neural network processing into multiple independent task segments (first segment branch, second segment branch) that can be independently enqueued and executed. This segmentation allows the system to prepare multiple possible execution paths in parallel, reducing the time penalty associated with switching between different neural network configurations.
2Productivity
If the neural processor circuit dynamically selects branches based on task outputs, then the operational efficiency improves, but the power consumption increases
Solution Approach 1:
The patent implements partial action by only fully executing one selected branch while the other branch remains enqueued but unexecuted. This allows the system to maintain readiness for alternative paths without committing full resources to multiple simultaneous executions, thereby improving operational efficiency while controlling power consumption through selective execution.
3Loss of time
If the circuit enqueues multiple segment branches in advance, then the transition latency is reduced, but the task queue management complexity increases
Solution Approach 1:
The task manager circuit performs preliminary enqueuing of multiple segment branches before execution decisions are made. By preparing the first and second segment branches in advance and placing them in the task queue, the system eliminates transition latency when switching between neural network segments, as the selected segment is already positioned for immediate execution.
Data Source
AI summary
A neural processor includes neural engines for performing convolution operations on input data corresponding to one or more tasks to generate output data. The neural processor circuit also includes a data processor circuit that is coupled to one or more neural engine. The data processor circuit receives the output data from the neural engine and generates a branching command from the output data. The neural processor circuit further includes a task manager that is coupled to the data processor circuit. The task manager receives the branching command from the data processor circuit. The task manager enqueues one of two or more segment branches according to the received branching command. The two or more segment branches are subsequent to a pre-branch task segment that includes the pre-branch task. The task manager transmits a task from the selected one of the segment branches to data processor circuit to perform the task.