ANN Fixed-Point Format Control via Virtual Overflow Detection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Artificial neural networks (ANNs) face challenges with fixed-point operations due to the risk of overflow and underflow, which can lead to precision loss and increased power consumption, especially when dealing with large data sizes exceeding processor register sizes.
Innovation Solution
An ANN system and method that employs a dynamic fixed-point policy by using a processor to perform node operations, quantization, and virtual overflow detection to adjust the fixed-point format, preventing overflow and underflow through virtual overflow information, thereby enhancing performance and reducing power consumption.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Use of energy by moving object
If fixed-point operation is used for ANN training, then power consumption and operation time are reduced, but overflow risk increases due to narrow range of real number
Solution Approach 1:
The patent implements dynamic adjustment of the fractional length in fixed-point format based on the distribution of quantization values. The system monitors valid bit numbers and adjusts the fractional length dynamically to prevent overflow while maintaining efficiency. This resolves the contradiction by making the fixed-point format adaptable rather than static, allowing the system to maintain narrow range benefits when data is small while expanding range when needed.
Solution Approach 2:
The patent changes the parameter of fractional length in the fixed-point format dynamically. By adjusting the fractional length based on the distribution of quantization values from previous operations, the system optimizes the range to prevent overflow. This parameter change allows the system to maintain the efficiency of fixed-point operations while adapting to prevent overflow risks.
2Reliability
If fractional length is reduced to prevent overflow, then overflow risk is reduced, but precision or value resolution is sacrificed due to underflow
Solution Approach 1:
The system dynamically adjusts the fractional length based on the actual distribution of quantization values. Instead of using a fixed reduced fractional length that would always sacrifice precision, the system monitors valid bit numbers and adjusts the fractional length to match the actual data requirements. This prevents overflow when data is small while maintaining precision when data is large.
Solution Approach 2:
The patent implements a feedback mechanism where the system monitors the distribution of quantization values and uses this information to adjust the fractional length for the next operation. The virtual overflow detection circuit provides feedback about valid bit numbers, which the data format controller uses to optimize the fixed-point format. This feedback loop ensures that precision is maintained while preventing overflow.
3Measurement precision
If floating-point operation is used, then accuracy and range of expression are improved, but operation time and power consumption increase
Solution Approach 1:
The patent dynamically changes the parameters of the fixed-point format to approximate floating-point precision when needed. By adjusting the fractional length based on the distribution of quantization values, the system achieves high accuracy for small numbers while maintaining the efficiency of fixed-point operations for larger numbers. This resolves the contradiction by making the fixed-point format adaptable to achieve floating-point-like precision without the constant overhead.
Data Source
AI summary
An artificial neural network (ANN) system includes a processor, a virtual overflow detection circuit and a data format controller. The processor performs node operations with respect to a plurality of nodes included in each layer of an ANN to obtain a plurality of result values of the node operations and performs a quantization operation on the obtained plurality of result values based on a k-th fixed-point format for a current quantization of the each layer to obtain a plurality of quantization values. The virtual overflow detection circuit generates a virtual overflow information indicating a distribution of valid bit numbers of the obtained plurality of quantization values. The data format controller determines a (k+1)-th fixed-point format for a next quantization of the each layer based on the generated virtual overflow information. An overflow and/or an underflow are prevented efficiently by controlling the fixed-point format using the virtual overflow.