Learnable Quantization Thresholds for Fixed-Point Neural Networks
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing multilayer neural networks face challenges in reducing processor resource usage on embedded devices due to high memory overhead and cumulative errors from fixed-point conversions, limiting their application on devices with limited resources.
Innovation Solution
A method to generate multilayer neural networks by acquiring networks with convolutional and quantization layers, generating learnable quantization threshold parameters based on bit parameters and interval parameters, and updating them to create fixed-point networks, allowing for efficient operation on embedded devices.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Use of energy by moving object
If fixed-point operation is used to convert floating-point parameters into fixed-point parameters for embedded devices, then processor resource usage is reduced, but cumulative errors occur and system performance deteriorates
Solution Approach 1:
The patent applies dynamics by making the quantization threshold parameters learnable rather than fixed. The quantization layer learns optimal threshold parameters during training, allowing the system to adapt to different input data distributions dynamically. This resolves the contradiction by enabling the system to maintain high accuracy (reliability) while using fixed-point operations (low processor resource usage) on embedded devices.
Solution Approach 2:
The patent changes the parameter representation by introducing learnable quantization threshold parameters that are optimized during training. Instead of using fixed predefined thresholds, the system learns parameter values that minimize quantization errors. This parameter change enables accurate fixed-point inference on resource-constrained devices without cumulative performance degradation.
2Productivity
If predefined quantization threshold parameters are used in quantization layers, then joint fixed-point technology can be applied to reduce computation overheads, but generalization capability and overall performance are limited
Solution Approach 1:
The patent transforms static predefined quantization thresholds into dynamic learnable parameters. During training, the quantization layer adapts the threshold parameters to match the actual data distribution, enabling the model to generalize better to unseen data while maintaining computational efficiency through fixed-point operations during inference.
Solution Approach 2:
The quantization layer performs self-adjustment by learning optimal threshold parameters from the training data itself. This self-service mechanism eliminates the need for manual predefined thresholds and allows the system to automatically adapt to different data distributions, improving both generalization capability and computational efficiency.
3Reliability
If learnable quantization interval parameters are used to improve system performance, then accuracy is greatly improved, but computation overheads cannot be reduced due to floating-point operations
Solution Approach 1:
The patent segments the quantization process into two distinct phases: training phase where learnable parameters are optimized using floating-point operations, and inference phase where fixed-point operations with learned parameters are used. This segmentation allows the system to achieve high accuracy during training while maintaining low computation overhead during deployment on resource-constrained devices.
Solution Approach 2:
The patent performs preliminary learning of optimal quantization parameters during the training phase before deployment. By pre-learning the quantization thresholds and interval parameters, the system prepares the fixed-point representation in advance, enabling accurate inference with minimal computation overhead on embedded devices without requiring floating-point operations during runtime.
Data Source
AI summary
A method for generating a multilayer neural network including acquiring a multilayer neural network, wherein the multilayer neural network includes at least convolutional layers and quantization layers; generating, for each of the quantization layers in the multilayer neural network, quantization threshold parameters based on a quantization bit parameter and a learnable quantization interval parameter in the quantization layer; and updating the multilayer neural network to obtain a fixed-point neural network based on the generated quantization threshold parameters and operation parameters for each layer in the multilayer neural network.