Quantized Neural Network Evaluation via Feature Map Importance
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods for evaluating quantized artificial neural network models face challenges such as the need for separate validation datasets, privacy issues, and weak correlation between mean square error (MSE) and model performance.
Innovation Solution
A method that generates original and quantized feature maps using respective artificial neural network models, determines the importance of each feature map element, and calculates an evaluation value based on these maps and their importance, allowing for precise evaluation of quantized models without requiring separate validation datasets.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If evaluation is performed using a separate validation dataset, then evaluation accuracy can be improved, but data privacy issues arise and additional data security requirements are imposed
Solution Approach 1:
The patent uses feature maps generated during the training process as a substitute (copy) for separate validation datasets. Instead of requiring additional private data, the method evaluates quantized models by comparing their feature maps against those from the original model, thereby achieving evaluation accuracy without exposing sensitive data
Solution Approach 2:
The patent introduces feature maps as an intermediary representation that bridges the original model and quantized model for comparison. By evaluating through this intermediate representation rather than directly comparing raw data or requiring separate validation sets, the method maintains evaluation accuracy while avoiding data privacy issues
2Ease of manufacture
If Mean Square Error (MSE) is used to evaluate quantized models, then calculation simplicity is improved, but evaluation accuracy deteriorates due to weak correlation between MSE and actual model performance
Solution Approach 1:
The patent applies local quality by computing MSE separately for different feature map channels and layers, then weighting these local errors according to their importance. This allows critical features to have greater impact on the overall evaluation, improving accuracy while maintaining computational efficiency through selective focus rather than uniform processing
3Measurement precision
If feature map comparison is performed for all elements, then evaluation accuracy is improved, but computation complexity increases
Solution Approach 1:
The patent computes importance weights for different feature map elements (channels, layers) and uses these weights to prioritize which elements require precise comparison. High-importance elements receive fuller evaluation while low-importance elements can be processed more simply, maintaining overall accuracy while reducing total computation
Solution Approach 2:
The patent changes the evaluation parameter from uniform MSE across all feature map elements to importance-weighted MSE. By introducing importance weights as a new parameter, the method achieves better accuracy-computation tradeoff, focusing computational resources on the most critical features
Data Source
AI summary
In a method for evaluating a quantized artificial neural network according to an embodiment, at least one original feature map for input data is generated using a first artificial neural network model, importance of each element of the at least one original feature map is determined, at least one quantized feature map for the input data is generated using a second artificial neural network model that is a quantized artificial neural network model for the first artificial neural network model, an evaluation value for the second artificial neural network model is calculated based on at least one original feature map, the at least one quantized feature map, and the importance.