Neural Network Bias Correction for Approximation Errors
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Deep neural networks trained on high-resource devices face challenges when deployed on mobile devices with limited processing resources, as approximation techniques like quantization and compression introduce biases and errors, which conventional solutions like retraining or fine-tuning cannot effectively mitigate without significant computational resources.
Innovation Solution
The method involves generating an approximation of a neural network by performing approximation operations on weights tensors, determining expected output errors, and adjusting bias parameters to correct for these errors, allowing the neural network to function accurately on resource-constrained devices without retraining or fine-tuning.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Use of energy by moving object
If approximation operations (quantization/compression) are applied to weights tensor to reduce computational resources, then processing efficiency and energy consumption are improved, but manufacturing precision and reliability deteriorate due to introduced biases and errors
Solution Approach 1:
The patent applies preliminary action by computing expected output errors from approximation operations in advance and using these errors to adjust bias parameters before the neural network is deployed. This pre-computation of correction terms allows the system to compensate for quantization errors without requiring retraining or fine-tuning on the target device, thus resolving the contradiction between energy efficiency and precision.
Solution Approach 2:
The patent changes parameters by adjusting the bias parameter based on computed expected output errors from approximation operations. By modifying the bias parameter to compensate for quantization effects, the system maintains output accuracy while using lower-precision weights, thereby resolving the contradiction between weight precision and energy consumption.
2Adaptability or versatility
If approximation operations are applied to weights tensor to enable deployment on mobile devices, then adaptability is improved, but manufacturing precision deteriorates due to rounding errors and biases
Solution Approach 1:
The patent performs preliminary computation of expected output errors and bias adjustments on high-resource devices before deployment. This allows the approximated model with adjusted bias parameters to be directly deployed on mobile devices with limited resources while maintaining accuracy, thus improving adaptability without sacrificing precision.
Solution Approach 2:
The patent introduces an intermediary correction mechanism where expected output errors are computed as intermediate values to bridge the gap between high-precision training weights and low-precision deployment weights. By using this intermediary error term to adjust bias parameters, the system maintains precision across different device types.
3Manufacturing precision
If conventional solutions (retraining or fine-tuning) are used to mitigate approximation errors, then manufacturing precision is improved, but productivity and use of energy worsen due to significant computational resources required
Solution Approach 1:
The patent extracts the essential correction information (expected output errors) from the approximation operations and separates it from the full retraining process. By computing only the necessary error terms and using them to adjust bias parameters, the system achieves precision improvement without requiring resource-intensive retraining or fine-tuning, thus improving deployment efficiency.
Solution Approach 2:
The patent applies partial action by performing only the necessary bias parameter adjustment based on expected output errors, rather than conducting complete retraining or fine-tuning. This partial correction approach achieves sufficient precision improvement for deployment while avoiding the excessive computational resources required by conventional methods.
Data Source
AI summary
Various embodiments include methods and neural network computing devices implementing the methods, for generating an approximation neural network. Various embodiments may include performing approximation operations on a weights tensor associated with a layer of a neural network to generate an approximation weights tensor, determining an expected output error of the layer in the neural network due to the approximation weights tensor, subtracting the expected output error from a bias parameter of the layer to determine an adjusted bias parameter and substituting the adjusted bias parameter for the bias parameter in the layer. Such operations may be performed for one or more layers in a neural network to produce an approximation version of the neural network for execution on a resource limited processor.