Deep Neural Network Acceleration via Adaptive Fixed-Point Precision
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing deep learning methods are limited by energy consumption, speed, and memory occupation due to the use of single-precision and dual-precision floating-point formats, and there is a need for alternative methods to accelerate deep neural network training and inference.
Innovation Solution
The method involves randomly initializing weights and biases as n-bit fixed-point numbers and reducing data in layers with high calculation amounts to m-bit fixed-point numbers, while maintaining other layers at n-bits, and training the network until convergence, with automatic adjustment of fixed-point formats to optimize resource usage and precision.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If single-precision or dual-precision floating-point format is used for deep neural network operations, then processing precision is maintained, but hardware resource usage, energy consumption, and processing speed are limited
Solution Approach 1:
The patent changes the numerical representation parameter from floating-point format to fixed-point format for deep neural network operations. This parameter change enables the system to achieve faster processing speeds and reduced hardware resource usage while maintaining sufficient processing precision through careful design of the fixed-point number format (e.g., Q-format specifications for different data types).
Solution Approach 2:
The patent dynamically adjusts the fixed-point number format (precision and bit allocation) for different layers and operations within the deep neural network. By analyzing the actual numerical ranges and precision requirements of different layers, the system assigns optimal fixed-point formats adaptively, thereby maximizing processing speed while maintaining necessary precision where required.
2Measurement precision
If single-precision or dual-precision floating-point format is used for deep neural network operations, then processing precision is maintained, but hardware resource usage and energy consumption increase
Solution Approach 1:
The patent changes the numerical representation parameter from floating-point format to fixed-point format, which requires simpler hardware circuits without floating-point units. This parameter change significantly reduces energy consumption while maintaining adequate processing precision through optimized fixed-point number formats tailored to the specific requirements of deep neural network operations.
3Productivity
If data precision is reduced to accelerate deep learning, then processing speed improves, but data precision is lost
Solution Approach 1:
The patent applies different fixed-point precision levels to different parts of the deep neural network based on their specific requirements. Input data, weights, biases, and intermediate calculations use appropriately tailored fixed-point formats (e.g., Q15, Q31) that provide sufficient precision for their specific functions. This local optimization ensures that precision is maintained where necessary while allowing speed optimization where lower precision is acceptable.
4Productivity
If fixed-point bit numbers are reduced to lower fixed-point bit redundancy, then hardware resource usage decreases, but processing precision may be affected
Solution Approach 1:
The patent optimizes the fixed-point number format parameters (integer bits and fractional bits allocation) for each data type in the deep neural network. By carefully selecting parameters such as Q15 format for weights and Q31 format for activations, the system achieves optimal balance between hardware efficiency and processing precision, eliminating bit redundancy while preserving necessary precision for accurate computations.
Data Source
AI summary
A method and apparatus of accelerating deep learning, and a deep neural network are provided. The method comprises: randomly initializing weights and biases of a deep neural network as n-bit fixed-point numbers; reducing data in a plurality of layers in the deep neural network that have calculation amounts are greater than a first predetermined threshold as m-bit fixed-point numbers, where m and n are integers and m<n, and maintaining data in remaining layers among the plurality of layers as n-bit fixed-point numbers; and training the deep neural network after the reducing, until convergence.