Neural Network Operation Instructions for Edge Microcontrollers
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing deep neural networks (DNNs) face challenges in implementing convolution and batch normalization operations efficiently at edge nodes, particularly in low-end microcontrollers, due to resource constraints and the need for separate hardware accelerators.
Innovation Solution
A neural network instruction is added to the CPU's instruction set architecture (ISA) to perform convolution or batch normalization operations directly, utilizing shared data paths and sub-circuits within the CPU, eliminating the need for multiple hardware accelerators.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If separate hardware accelerators are used for convolution and batch normalization operations, then operation performance is improved, but device complexity and cost increase
Solution Approach 1:
The patent combines separate hardware accelerators for convolution and batch normalization into a single integrated neural network operation unit. This unit can perform both convolution operations (including 16×8 and 8×8 modes) and batch normalization operations (including scale, shift, and clamp functions) using shared hardware resources, thereby reducing device complexity while maintaining operational performance.
Solution Approach 2:
The neural network operation unit is designed with multi-functionality to handle diverse neural network operations. It can dynamically switch between convolution operations with different precision modes (16×8, 8×8) and batch normalization operations (scale, shift, clamp), allowing a single hardware unit to replace multiple specialized accelerators.
2Speed
If multiple hardware accelerators are deployed, then computational speed is improved, but power consumption increases
Solution Approach 1:
By merging multiple hardware accelerators into a single integrated unit with shared computational resources, the patent reduces the total power consumption while maintaining computational speed. The shared resources include arithmetic logic units, data paths, and control logic that can be dynamically allocated to different operations.
Solution Approach 2:
The hardware accelerator employs dynamic resource allocation and reconfiguration capabilities. The same hardware resources can be dynamically switched between different neural network operations (convolution with different precision modes, batch normalization with scale/shift/clamp), optimizing power efficiency by avoiding idle resources while maintaining high computational speed.
3Productivity
If convolution operations use 16×8 precision mode, then performance is improved, but hardware resource requirements increase
Solution Approach 1:
The patent implements dynamic precision mode switching capability in the neural network operation unit. The hardware can be reconfigured between 16×8 precision mode (higher performance) and 8×8 precision mode (lower resource requirements) based on the specific neural network operation being performed, allowing optimal balance between performance and resource utilization.
Solution Approach 2:
The hardware accelerator utilizes parameter changes in data precision representation. By supporting both 16×8 and 8×8 precision modes, the system can adjust the numerical precision parameters to match the requirements of different neural network layers and operations, optimizing both performance and hardware resource usage.
Data Source
AI summary
Disclosed herein are improvements to instructions and hardware for performing neural network operations. In an embodiment, a processing device includes instruction fetch circuitry, decoder circuitry, and neural network operation circuitry. The instruction fetch circuitry is configured to fetch a neural network instruction from memory that specifies an operation and a set of values that enable sub-circuits of the neural network operation circuitry for use with one or more of the operations of the group of operations and provide the neural network instruction to the decoder circuitry. The decoder circuitry is configured to cause the neural network operation circuitry to perform, based on the operation, a convolution operation using a first sub-circuit of the neural network operation circuitry and a first subset of the set of values or a batch normalization operation using a second sub-circuit of the neural network operation circuitry and a second subset of the set of values.