Neural Network Weight Quantization via Ensemble Kalman Filter
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods for implementing neural networks on edge devices face challenges due to hardware limitations, requiring a reduction in computation load and memory usage, especially during online learning and weight quantization.
Innovation Solution
A learning program that employs an ensemble Kalman filter method to update weights in a neural network, incorporating bit quantization and changing the bit expression to reduce computational load and memory usage, while allowing online learning.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If online learning with Kalman filter is applied to update weights, then learning speed is improved, but computation load and memory usage increase
Solution Approach 1:
The patent applies parameter changes by transitioning from floating-point weight representations to fixed-point quantized representations. Specifically, it quantizes weights to 8-bit integers, reducing the numerical precision from typical floating-point (32-64 bits) to 8 bits. This parameter change maintains the functional behavior of the neural network while dramatically reducing computation load and memory requirements, making the system suitable for edge devices with limited hardware resources.
2Quantity of substance
If weight quantization is performed at the time of learning, then memory usage is reduced, but existing techniques are limited to discrimination tasks and require offline operation
Solution Approach 1:
The patent implements dynamics by enabling online quantization recognition learning, where the quantization process occurs dynamically during the learning procedure rather than offline beforehand. The system continuously quantizes weights during training iterations, allowing the neural network to adapt to prediction tasks in real-time. This dynamic approach extends quantization recognition learning from static offline processing to active online operation, making it applicable to both discrimination and prediction tasks.
3Power
If bit expression is changed to reduce word length and decimal part length, then computation load is reduced, but precision may be affected
Solution Approach 1:
The patent applies copying by maintaining multiple representations of weights with different precision levels. It stores both the original floating-point weights and the quantized fixed-point weights, allowing the system to copy and switch between representations as needed. During inference, the system uses the quantized 8-bit weights to reduce computation load, while the original high-precision weights are retained for accuracy when needed. This copying strategy enables the system to optimize between computation efficiency and precision based on operational requirements.
Data Source
AI summary
A learning program that performs an operation of updating a weight or an estimated value of a state variable in a neural network or a dynamical system is provided. The learning program includes: a first operation of calculating a Kalman gain using an ensemble Kalman filter method on the basis of a pre-update weight; a second operation of estimating a post-update weight in a first bit expression by adding the pre-update weight to a result obtained by multiplying an error between an inference result using the pre-update weight and a training signal by the Kalman gain; and a third operation of performing bit quantization of the post-update weight expressed in the first bit expression and changing the first bit expression to a second bit expression in which a word length and a length of a decimal part are shorter than those in the first bit expression.