N-Bit Accelerator Bias Scaling for Fixed-Point CNN Accuracy
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Bias saturation occurs in deep learning models due to the conversion of floating point weights and biases to fixed point, leading to accuracy loss, as the scale applied to bias values is not synchronized with feature and weight scales, causing overflow and propagation of errors through subsequent layers.
Innovation Solution
An iterative approach is used to refine feature and weight scales in convolution layers, ensuring they are synchronized by adjusting scales to prevent bias saturation, allowing for accurate conversion to fixed point operations using an N-bit matrix multiplication accelerator.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Use of energy by moving object
If floating point weights and biases are converted to fixed point for N-bit hardware acceleration, then power consumption is reduced and processing speed is improved, but bias saturation occurs causing accuracy loss
Solution Approach 1:
The patent applies parameter changes by dynamically adjusting the bias scale parameter based on the relationship between feature scale and weight scale. When the product of feature scale and weight scale exceeds the bias scale, the bias scale is increased to prevent saturation, thereby maintaining model accuracy while using fixed-point representation for energy-efficient hardware acceleration
Solution Approach 2:
The patent implements preliminary action by pre-calculating and storing optimal bias scale values in lookup tables before deployment. This allows the system to quickly retrieve appropriate bias scales during inference without real-time computation, preventing bias saturation before it occurs while maintaining the power efficiency of fixed-point operations
2Ease of manufacture
If feature scale and weight scale are not synchronized with bias scale, then conversion to fixed point is simplified, but bias saturation occurs leading to error propagation through subsequent layers
Solution Approach 1:
The patent implements feedback mechanisms where the system continuously monitors the relationship between feature scale, weight scale, and bias scale during model execution. When imbalance is detected, the bias scale is adjusted accordingly, creating a closed-loop system that prevents bias saturation and error propagation while maintaining relatively simple fixed-point conversion processes
3Speed
If N-bit fixed point matrix operations accelerator is used, then processing speed is improved, but precision is limited by bit depth constraints
Solution Approach 1:
The patent applies dimensionality change by introducing scale parameters as an additional dimension to the fixed-point representation system. Instead of simply increasing bit depth, the system uses scale factors to effectively expand the dynamic range and precision of N-bit fixed-point operations, allowing high-speed hardware acceleration while maintaining accuracy through mathematical scaling rather than brute-force bit expansion
Data Source
AI summary
In described examples, an integrated circuit includes a memory storing weights and biases, an N-bit fixed point matrix operations accelerator, and a processor. Starting with a first convolution layer, a convolution layer modeled using the processor receives input feature values. A feature scale and weight scale are reduced if an accumulator scale is greater than a maximum bias scale. The input feature values are rescaled using the feature scale, the weights are quantized using the weight scale, and the biases are quantized using the feature scale and weight scale. The rescaled input feature values and quantized weights and biases are convolved using the N-bit fixed point matrix operations accelerator to generate output feature values. The process repeats from the receive action using the output feature values as the input feature values of the next convolution layer. The process then repeats for all layers, feeding back an output feature range.