Variable-Bit Neural Engine Circuit for Lower Power AI Compute
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing neural network computations consume significant CPU bandwidth and power due to extensive operations, affecting the performance and speed of electronic devices.
Innovation Solution
A neural engine circuit with multiple multiply circuits operating in different modes to adjust bit width dynamically, allowing parallel and combined computations for efficient power and bandwidth management.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If CPU and main memory are used to instantiate and execute machine learning systems, then ease of configuration and instantiation is improved, but CPU bandwidth consumption and power consumption increase significantly
Solution Approach 1:
The system divides the machine learning execution into two segments: configuration management handled by the CPU (maintaining ease of configuration) and actual computational operations handled by the neural engine circuit (reducing CPU bandwidth and power consumption). The neural engine circuit includes dedicated multiply circuits that perform computations independently of the CPU.
Solution Approach 2:
A buffer circuit is introduced as an intermediary between the CPU and the neural engine circuit. The buffer circuit receives configuration data from the CPU and feeds input data to the neural engine circuit, allowing the CPU to offload computational tasks while maintaining configuration flexibility through software updates.
2Adaptability or versatility
If CPU and main memory are used to execute machine learning operations, then flexibility in model instantiation is improved, but overall device performance and computation speed deteriorate
Solution Approach 1:
The system segments functionality so that the neural engine circuit is dedicated to high-speed computation while the CPU maintains flexibility for configuration. The neural engine circuit includes multiple multiply circuits that can process data in parallel, significantly increasing computation speed compared to CPU-only execution.
Solution Approach 2:
The neural engine circuit is designed with multiple multiply circuits that can operate in different modes (first mode with first bit width, second mode with second bit width), providing versatility for different computational requirements while maintaining high performance. This multi-functionality allows the same hardware to handle various machine learning operations efficiently.
3Productivity
If multiple multiply circuits operate in parallel to increase computation speed, then productivity is improved, but device complexity increases
Solution Approach 1:
Multiple multiply circuits are merged into a single neural engine circuit that can operate in different modes. In the first mode, multiple multiply circuits operate independently in parallel to process multiple data elements simultaneously. In the second mode, the same circuits operate as a combined computation circuit for higher bit width operations. This merging approach achieves high productivity without proportionally increasing overall device complexity.
Data Source
AI summary
Embodiments relate to an electronic device that includes a neural processor having multiple neural engine circuits that operate in multiple modes of different bit width. A neural engine circuit may include a first multiply circuit and a second multiply circuit. The first and second multiply circuits may be combined to work as a part of a combined computation circuit. In a first mode, the first multiply circuit generates first output data of a first bit width by multiplying first input data with a first kernel coefficient. The second multiply circuit generates second output data of the first bit width by multiplying second input data with a second kernel coefficient. In a second mode, the combined computation circuit generates third output data of a second bit width by multiplying third input data with a third kernel coefficient.