AI Chip Dedicated Execution Unit Locking Mechanism
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing neural network accelerators require frequent communication with the CPU for task execution and frequent locking/unlocking of dedicated execution units, which hampers performance due to increased idle time and communication overhead.
Innovation Solution
An artificial intelligence chip with an instruction memory, general execution units, and dedicated execution units, where the general execution unit locks the dedicated execution unit associated with a code block and sends instructions for execution, allowing the dedicated execution unit to store results in data memory without frequent unlocking, thus minimizing CPU interaction during kernel code execution.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If the CPU frequently sends operation tasks to the GPU and waits for completion, then the neural network training can be performed, but the communication overhead and idle time increase significantly
Solution Approach 1:
The patent segments the execution units into two distinct types: general execution units for control and coordination tasks, and dedicated execution units for specific neural network operations. This segmentation allows the CPU to offload computation-intensive tasks to dedicated units while maintaining minimal communication overhead, as each dedicated unit can execute tasks independently once initialized by the general execution units.
Solution Approach 2:
The patent introduces general execution units as intermediaries between the CPU and dedicated execution units. These general execution units receive tasks from the CPU, lock the appropriate dedicated execution units, and send instructions to them. This intermediary layer reduces direct CPU-GPU communication frequency, allowing the CPU to continue other tasks while dedicated units execute computations independently.
2Ease of operation
If the dedicated execution unit is frequently locked and unlocked for each operation task, then the CPU can control the execution flow, but the locking overhead reduces overall performance
Solution Approach 1:
The patent implements preliminary locking of dedicated execution units before task execution begins. The general execution unit locks a dedicated execution unit in advance and maintains this locked state throughout the execution of multiple operation tasks. This preliminary action eliminates the need for repeated locking and unlocking operations, reducing overhead and improving execution efficiency while maintaining CPU control through the general execution unit.
3Reliability
If the CPU waits for each operation task completion before sending the next task, then task execution can be properly coordinated, but the overall processing speed decreases
Solution Approach 1:
The patent enables continuous operation of dedicated execution units by having general execution units maintain locked states and send multiple instructions sequentially without releasing the lock. This continuity allows dedicated execution units to process multiple operation tasks in an uninterrupted manner, significantly improving throughput. The CPU can continue sending tasks through general execution units without waiting for each task completion, as the locked dedicated units maintain their execution state continuously.
Data Source
AI summary
Embodiments of the present disclosure disclose an artificial intelligence chip and an instruction execution method for an artificial intelligence chip. A specific embodiment of the artificial intelligence chip includes: an instruction memory, a data memory, at least one general execution unit, and at least one dedicated execution unit. The instruction memory is configured to: receive a kernel code including at least one code block. The general execution unit is configured to: receive the code block, lock the dedicated execution unit associated with the received code block, and send an instruction in the received code block to the locked dedicated execution unit. The dedicated execution unit is configured to: execute the received instruction, and store an execution result in the data memory. The data memory is configured to: store the execution result sent by the dedicated execution unit.