Neural Network Weight Quantization Adapted to Input Attributes
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing neural network models face challenges in maintaining precision while improving computational efficiency due to fixed quantized weight data that is not optimized for varying input attributes.
Innovation Solution
An electronic apparatus and method that optimize quantized weight data by acquiring latent vectors representing weight and input data attributes, combining them to adjust quantization levels, and training a quantization adjustment module to generate optimized weight data for specific inputs.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If weight data is quantized to improve computational efficiency, then computational efficiency is improved, but precision is degraded
Solution Approach 1:
The patent applies dynamics by transforming fixed quantized weight data into dynamically adjustable weight data. The system generates multiple candidate weight data sets with different quantization levels and selectively applies them based on input data characteristics, enabling the model to adapt its precision level dynamically rather than being constrained to a fixed quantization level.
Solution Approach 2:
The patent changes the quantization level parameter dynamically. Instead of using a single fixed quantization level, the system generates candidate weight data at multiple quantization levels (e.g., 8-bit, 16-bit, 32-bit) and selects or combines them based on input characteristics, effectively changing the precision parameter adaptively to balance computational efficiency and model accuracy.
2Device complexity
If fixed quantized weight data is used to simplify the model, then device complexity is reduced, but adaptability to different input attributes is worsened
Solution Approach 1:
The patent applies preliminary action by pre-generating multiple candidate weight data sets at different quantization levels before inference. This preparation work is done in advance, so during actual inference, the system can quickly select or combine appropriate weight data based on input characteristics without performing complex real-time quantization, thus maintaining low operational complexity while achieving high adaptability.
Solution Approach 2:
The system transitions from static fixed weight data to dynamic adaptive weight data. By maintaining multiple candidate weight data sets and selecting/combining them based on input attributes, the model gains the ability to adapt its structure dynamically while keeping the overall system manageable through pre-computation and efficient selection mechanisms.
Data Source
AI summary
An electronic apparatus including a memory storing quantized first weight data of a neural network model and at least one processor. The at least one processor is configured to acquire a first latent vector that compressively represents an attribute of the first weight data. The at least one processor is configured to acquire a second latent vector that compressively represents an attribute of input data of the neural network model. The at least one processor is configured to acquire a third latent vector by combining the first latent vector with the second latent vector. The at least one processor is configured to acquire a plurality of quantization adjustment values. The at least one processor is configured to acquire second weight data in which the first weight data is changed to be optimized for the input data based on the first weight data and the plurality of quantization adjustment values.