Neural Network Chip Online Parameter Update via Mixed Instruction Set
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current neural network model computing systems face inefficiencies when dealing with models that require online parameter updates, as they necessitate frequent interactions between the neural network model computing chip and the general purpose processor, leading to delayed computing power utilization and reduced efficiency due to bus bottlenecks and time-consuming data migrations.
Innovation Solution
A neural network model computing chip is designed with an instruction processing unit, instruction parsing unit, and scheduling unit that enables online generation of update instructions within the chip, using a mixed instruction set that includes both original instructions and control information, allowing for dynamic updates without relying on external processors, thereby reducing interactions and optimizing computing power.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If the neural network model computing chip frequently interacts with the general purpose processor for online parameter updates, then the model can adapt to changing data, but the computing efficiency is reduced due to bus bottlenecks and data migration time
Solution Approach 1:
The instruction processing unit enables the computing chip to self-update instructions internally using control information from the pre-compiled mixed instruction set, eliminating the need for external processor intervention during runtime. This self-service mechanism resolves the contradiction by allowing online adaptation without external data migration, thus maintaining high computing efficiency while enabling parameter updates.
2Productivity
If the system uses a static instruction set pre-compiled for the entire neural network model, then computing efficiency is high, but the system cannot handle models requiring online parameter updates
Solution Approach 1:
The system transitions from a static instruction set to a dynamic instruction generation mechanism. The instruction processing unit dynamically generates updated instructions by combining original instructions with control information from the mixed instruction set at runtime, enabling the system to adapt to online parameter updates while maintaining high computing efficiency through the pre-compiled structure.
Solution Approach 2:
The mixed instruction set is pre-compiled to include both original instructions and control information necessary for online updates. This preliminary preparation allows the instruction processing unit to quickly generate updated instructions during runtime without external intervention, resolving the contradiction between static efficiency and dynamic adaptability.
3Reliability
If the chip relies on external general purpose processor for instruction updates, then instruction accuracy is maintained, but waiting time increases and computing power utilization is delayed
Solution Approach 1:
The computing chip performs instruction updates internally through the instruction processing unit, which autonomously generates updated instructions using control information from the pre-compiled mixed instruction set. This eliminates waiting time for external processor intervention while maintaining instruction accuracy through the structured control information design.
Solution Approach 2:
All necessary control information for instruction updates is pre-compiled into the mixed instruction set before runtime. This preliminary preparation enables the instruction processing unit to immediately generate accurate updated instructions without external communication, eliminating waiting time while ensuring instruction correctness.
Data Source
AI summary
A method, applied to a neural network model computing chip, includes: obtaining a current instruction from a mixed instruction set including N instructions and obtained through pre-compiling based on model data of a target neural network model, the N instructions including an original instruction and control information for updating a target original instruction of the target neural network model; determining a target instruction based on the current instruction, where when the current instruction is control information, the target instruction is an update instruction obtained after the target original instruction is updated based on the control information; and parsing the target instruction, and scheduling a target engine based on a parsing result to perform a computing operation or a data migration operation indicated by the target instruction, the target engine being one of a plurality of pre-configured engines in the neural network model computing chip.