Neural Network Accelerator Online Number Format Selection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current neural network accelerators face challenges in efficiently processing neural network parameters due to the complexity of floating point number formats, which increases silicon area, power consumption, and bandwidth requirements, while fixed point formats compromise accuracy with fewer bits.
Innovation Solution
The implementation of a neural network accelerator with a statistics collection hardware unit to dynamically select and convert number formats for network parameters based on collected statistics, allowing for online format selection and conversion during neural network passes.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If floating point number formats are used for network parameters, then classification accuracy is improved, but silicon area, power consumption, and bandwidth requirements increase
Solution Approach 1:
The patent implements dynamic number format selection where the format (precision) of network parameters is adjusted in real-time based on the specific layer and operation being performed. Different layers of the neural network use different number formats - higher precision for layers requiring accuracy and lower precision for layers where accuracy is less critical. This dynamic adaptation allows the system to maintain high classification accuracy where needed while reducing silicon area and power consumption in other regions.
Solution Approach 2:
The patent applies different number formats to different parts (layers) of the neural network based on their specific requirements. Each layer can be configured with an appropriate number format locally, rather than using a uniform format throughout. This local optimization ensures that precision is allocated efficiently - high precision only where necessary for maintaining classification accuracy, and lower precision elsewhere to reduce overall device complexity.
2Measurement precision
If floating point number formats are used for network parameters, then classification accuracy is improved, but power consumption increases
Solution Approach 1:
The system dynamically selects number formats based on the operational requirements of each neural network layer. By using lower precision formats in layers where high accuracy is not critical, the system reduces the computational complexity and power consumption of arithmetic operations in those layers, while maintaining sufficient accuracy in layers where it matters most for classification.
Solution Approach 2:
Different power consumption characteristics are applied locally to different layers by selecting appropriate number formats for each. Layers that require high classification accuracy use floating point formats with higher power consumption, while other layers use fixed point or lower precision formats with reduced power consumption, optimizing the overall energy efficiency of the system.
3Device complexity
If fixed point number formats are used for network parameters, then silicon area is reduced, but classification accuracy deteriorates
Solution Approach 1:
The patent implements a dynamic format selection mechanism that chooses between fixed point and floating point formats based on the specific requirements of each neural network layer. This allows the system to use compact fixed point representations where sufficient precision can be achieved, reducing silicon area, while switching to floating point formats in layers where higher precision is necessary to maintain classification accuracy.
Solution Approach 2:
The system applies different number format qualities to different layers locally. Layers where classification accuracy is most sensitive use higher precision floating point formats, while layers with lower sensitivity use lower precision fixed point formats. This local differentiation optimizes the trade-off between silicon area and classification accuracy for the overall system.
4Device complexity
If uniform number format is used for all network parameters, then device complexity is reduced, but processing efficiency deteriorates
Solution Approach 1:
The patent implements dynamic number format selection that adapts to the specific requirements of each layer and operation. This dynamic approach allows the system to optimize processing efficiency by using appropriate precision levels for different layers, improving overall throughput and performance while managing device complexity through automated format selection rather than manual configuration.
Solution Approach 2:
The system employs a universal format selection mechanism that can automatically determine and apply the most appropriate number format for any given layer or operation. This multi-functional approach handles both fixed point and floating point formats within a single system architecture, allowing the system to optimize for processing efficiency in different contexts without requiring separate hardware paths for each format.
Data Source
AI summary
Methods and neural network accelerator for online selection of number formats for network parameters of a neural network. The neural network accelerator comprises at least one network processing hardware unit configured to receive network parameters for layers of the neural network and perform one or more neural network operations on the received network parameters in accordance with the neural network; a statistics collection hardware unit configured to collect one or more statistics on a first set of network parameters for a layer while the neural network accelerator is performing a pass of the neural network; and a format conversion hardware unit configured to convert a second set of network parameters to a number format selected based on the collected one or more statistics, the second set of network parameters comprising (i) the first set of network parameters and/or another set of network parameters for the layer, or (ii) a set of network parameters for a subsequent pass of the neural network corresponding to the first set of network parameters.