Kalman Gain Compression for Edge Learning Load
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing online learning methods for neural networks on edge devices face challenges in reducing operation load and memory usage, particularly when implementing the Kalman filter, which requires more computation and memory.
Innovation Solution
The method involves compressing and expanding the range of possible values of the Kalman gain using nonlinear functions, specifically through a compressor and expander in the online learning device, to optimize memory usage and reduce operation load during the learning process.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If the Kalman filter is applied for online learning, then the learning speed and stability are improved, but the computation load and memory usage increase
Solution Approach 1:
The patent applies parameter changes by dynamically adjusting the precision (decimal places) of the Kalman gain values based on their magnitude. When the Kalman gain is large, the system uses lower precision (e.g., 2 decimal places), and when the Kalman gain is small, the system uses higher precision (e.g., 4 decimal places). This adaptive parameter adjustment reduces unnecessary computation while maintaining learning stability and speed.
Solution Approach 2:
The patent implements dynamics by making the precision of Kalman gain calculations variable rather than fixed. The system dynamically switches between different precision levels based on the current state of the Kalman gain, allowing the computation load to adapt to the learning process requirements in real-time.
2Reliability
If the Kalman filter is applied for online learning, then the learning stability is improved, but the memory usage increases
Solution Approach 1:
The patent reduces memory usage by changing the precision parameter of stored Kalman gain values. Instead of always storing high-precision values, the system stores values with adjusted precision based on their magnitude, thereby reducing memory consumption while maintaining the stability required for effective learning.
3Quantity of substance
If the range of Kalman gain values is compressed, then memory usage is reduced, but the precision of weight update may be affected
Solution Approach 1:
The patent resolves this contradiction by dynamically changing the precision parameter based on the magnitude of Kalman gain values. The system uses lower precision when values are large (reducing memory usage) and higher precision when values are small (maintaining weight update accuracy), thus balancing memory efficiency with computational precision.
Data Source
AI summary
An online learning method includes: compressing a range of possible values of a Kalman gain before an update; obtaining a Kalman gain after the update from the compressed Kalman gain before the update using an expanded Kalman filter method; expanding the range of possible values of the Kalman gain after the update, and updating a weight by adding a weight before the update to a result obtained by multiplying the Kalman gain in which the range of the possible values of the Kalman gain is expanded by an error between a training signal and an inference result in which a weight before the update is used.