Neural Processing Command Processor Branch Execution
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current neural processing devices face inefficiencies due to repeated and duplicated communication with host systems, which hampers the speed and efficiency of command branch processing in deep learning and inference tasks.
Innovation Solution
A neural processing device and system that includes a neural processor, shared memory, and a command processor capable of executing branch commands independently, generating on-device decisions without host system intervention, thereby reducing the need for repeated communication and accelerating computing speed.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If the neural processing device communicates frequently with the host system for command branch processing, then the host system can make centralized decisions, but the communication overhead increases and processing efficiency decreases
Solution Approach 1:
The command processor in the neural processing device is empowered to independently execute branch commands and generate on-device decisions without requiring continuous host system intervention. This self-service capability allows the device to autonomously handle command branch processing, eliminating repeated communication cycles with the host system and significantly reducing communication overhead while maintaining processing efficiency
Solution Approach 2:
The system divides the decision-making authority into segments: the host system provides high-level command streams, while the neural processing device's command processor handles specific branch command execution and on-device decisions. This segmentation allows localized processing of branch commands within the device, reducing the need for centralized host system involvement in every decision point
2Speed
If the neural processing device executes branch commands independently without host system intervention, then processing speed increases, but device complexity increases
Solution Approach 1:
The command processor is designed with multi-functionality to handle various types of branch commands (conditional branches, loop control, jump commands) using a unified execution framework. This universal design allows the processor to independently execute diverse branch operations without requiring separate specialized circuits for each command type, thereby increasing computing speed while controlling device complexity through functional consolidation
Data Source
AI summary
An apparatus comprising neural processors, a command processor, and a shared memory is provided. The command processor, in response to receiving a context start signal indicating a start of a context of a neural network model from a host system, directly accesses a memory in the host system to read command stream data for the neural network model. The command processor selects a current command and determines whether the current command is a branch command or a command describing neural network model tasks. The command processor selects a command among commands in the command stream data as a next command to be executed, based on a determination on whether the current command is the branch command or the command describing neural network model tasks.