Neural Network Quantization Adjustment for Input-Adaptive Inference
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing neural network models face challenges in maintaining precision while improving computational efficiency due to fixed quantized weight data that is not optimized for varying input data attributes.
Innovation Solution
An electronic apparatus and method that optimize quantized weight data by acquiring latent vectors representing weight and input data attributes, combining them to adjust quantization levels, and updating weight data using quantization adjustment values.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If weight data is quantized to low precision to improve computational efficiency, then computational efficiency is improved, but manufacturing precision (weight data precision) deteriorates
Solution Approach 1:
The patent applies dynamics by making the quantization level adjustable rather than fixed. The quantization adjustment value acquisition module dynamically determines quantization adjustment values based on input data attributes, allowing the system to adapt the precision level of weight data according to the specific input being processed. This resolves the contradiction by enabling the system to use lower precision when computationally efficient and higher precision when accuracy is critical, for different input scenarios.
Solution Approach 2:
The patent changes the parameter of quantization level from a fixed value to a variable that can be adjusted based on input data characteristics. By acquiring quantization adjustment values that modify the base quantized weight data, the system can dynamically change the precision parameter to optimize both computational efficiency and precision requirements for different inference scenarios.
2Device complexity
If fixed quantized weight data is used to simplify the system, then device complexity is reduced, but adaptability deteriorates
Solution Approach 1:
The patent introduces quantization adjustment values as an intermediary between the fixed quantized weight data and the inference process. These adjustment values act as a mediator that allows the system to adapt to different input data attributes without fundamentally changing the weight data structure. The intermediary enables flexibility and adaptability while maintaining the simplicity of the underlying quantized weight data representation.
Solution Approach 2:
The system transitions from a static, fixed quantization approach to a dynamic one where quantization adjustment values are acquired based on input data attributes. This dynamic adaptation allows the system to respond to different input characteristics while maintaining a relatively simple overall structure, resolving the contradiction between simplicity and adaptability.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
Disclosed are an electronic device and a method for controlling the electronic device. In particular, the electronic device according to the present disclosure comprises: a memory storing quantized first weight data of a neural network model; and a processor which acquires a first latent vector compressively representing an attribute of the first weight data, acquires a second latent vector compressively representing an attribute of input data of the neural network model, acquires a third latent vector by combining the first latent vector and the second latent vector, acquires a plurality of quantization adjustment values for changing a quantization level of each of a plurality of weights included in the first weight data by inputting the third latent vector to a quantization adjustment value acquisition module, and acquires second weight data changed such that the first weight data is optimized for the input data, on the basis of the first weight data and the plurality of quantization adjustment values.