Neural Network Operation Device Mixed Precision Controller
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current neural network processing devices face challenges in efficiently handling large quantities of input data in real-time, particularly in extracting desired information through complex operations, as they require significant computational resources and struggle with varying bit lengths.
Innovation Solution
The proposed solution involves a neural network operation device with an input feature map buffer, weight buffer, and controller that enables mixed precision operations and data parallelism by strategically mapping input feature maps and weights to adder tree units based on bit length, allowing for flexible bit precision and parallel processing.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If the bit length of input data is reduced to half the reference bit length, then the operation performance speed increases, but the number of channels that can be processed decreases
Solution Approach 1:
The input feature map is segmented into multiple channels, and the controller selectively increases the number of channels mapped to the operator when the bit length is reduced. This allows the system to process more channels in parallel at lower precision, compensating for the reduced precision while maintaining overall productivity.
Solution Approach 2:
The system dynamically changes the bit length parameter of the input data based on the required operation performance. By allowing flexible adjustment of bit length, the system can optimize between speed (lower bit length) and precision (higher bit length), resolving the contradiction between operation speed and processing capability.
2Measurement precision
If the bit length of input data is increased to double the reference bit length, then the measurement precision improves, but the operation performance speed decreases
Solution Approach 1:
The system dynamically adjusts the bit length of input data according to the specific operation requirements. The controller can flexibly map input feature maps with varying bit lengths to the operator, allowing the system to adapt between high precision (double bit length) and high speed (reference bit length) modes as needed.
3Productivity
If mixed precision operation is implemented by increasing the number of channels, then the data processing capacity increases, but the device complexity increases
Solution Approach 1:
The operator is designed to handle multiple precision formats (different bit lengths) universally. The controller implements a unified mapping mechanism that can accommodate various bit lengths and channel configurations, allowing the same hardware operator to perform mixed precision operations without requiring separate processing paths for different precision levels.
Data Source
Figure 1A
Figure 1B
Figure 1C
AI summary
A neural network operation device includes an input feature map buffer to store an input feature map, a weight buffer to store a weight, an operator including an adder tree unit to perform an operation between the input feature map and the weight by a unit of a reference bit length, and a controller to map the input feature map and the weight to the operator to provide one or both of a mixed precision operation and data parallelism.