Neural Engine Circuit for Mixed-Precision Processing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing neural network operations on CPUs consume significant bandwidth and increase power consumption due to extensive computing requirements for machine learning tasks like convolutional neural networks.
Innovation Solution
A neural engine circuit with an input buffer, kernel extract circuit, and multiply-accumulator (MAC) circuit that performs neural networking operations using fixed-point and floating-point precision, optimizing power usage by turning off unused devices in fixed-point mode and efficiently processing data through MAD circuits and accumulators.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If CPU and main memory are used to instantiate and execute machine learning systems, then ease of configuration is improved, but power consumption increases
Solution Approach 1:
The system divides the neural network execution into two segments: configuration management handled by the CPU and actual computational operations handled by the neural network processor. This segmentation allows the CPU to remain idle during computation while the neural processor handles power-intensive operations, thus reducing overall power consumption while maintaining ease of configuration through CPU-based model instantiation.
Solution Approach 2:
A neural network processor is introduced as an intermediary between the CPU and the computational tasks. This intermediary device takes over the power-consuming computational operations from the CPU, allowing the CPU to maintain ease of configuration through high-level model instantiation while the intermediary handles the power-intensive low-level computations.
2Adaptability or versatility
If CPU is used for neural network operations, then adaptability to different configurations is improved, but processing speed decreases
Solution Approach 1:
The system segments the neural network processing into configuration phase (CPU) and execution phase (neural processor). The CPU handles adaptability through model instantiation and configuration updates, while the neural processor handles high-speed execution. This segmentation allows both adaptability and high processing speed to coexist by assigning different responsibilities to different components.
Solution Approach 2:
The system implements dynamic configuration capabilities where the neural network processor can be reconfigured through software updates from the CPU. This allows the system to adapt to different neural network architectures and configurations dynamically while maintaining high processing speed during execution, as the hardware remains optimized for computation while the software adapts to different tasks.
3Measurement precision
If floating-point precision is used for all operations, then measurement precision is improved, but device complexity increases
Solution Approach 1:
The neural network processor implements local quality by providing different precision modes (fixed-point and floating-point) for different computational needs. The system can selectively use fixed-point precision for operations where high precision is not critical, reducing device complexity, and use floating-point precision only where measurement precision is essential, thus optimizing the balance between precision and complexity.
Solution Approach 2:
The system changes the precision parameter dynamically based on the specific computational task. The neural network processor can switch between fixed-point and floating-point modes depending on the requirements of the neural network layer and operation. This parameter change allows the system to achieve high measurement precision when needed while maintaining lower device complexity for operations where full precision is not required.
Data Source
AI summary
Embodiments relate to a neural engine circuit that includes an input buffer circuit, a kernel extract circuit, and a multiply-accumulator (MAC) circuit. The MAC circuit receives input data from the input buffer circuit and a kernel coefficient from the kernel extract circuit. The MAC circuit contains several multiply-add (MAD) circuits and accumulators used to perform neural networking operations on the received input data and kernel coefficients. MAD circuits are configured to support fixed-point precision (e.g., INT8) and floating-point precision (FP16) of operands. In floating-point mode, each MAD circuit multiplies the integer bits of input data and kernel coefficients and adds their exponent bits to determine a binary point for alignment. In fixed-point mode, input data and kernel coefficients are multiplied. In both operation modes, the output data is stored in an accumulator, and may be sent back as accumulated values for further multiply-add operations in subsequent processing cycles.