Neural Network Filter Elimination for Quantization
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The challenge lies in efficiently quantizing neural networks to reduce computation costs while minimizing the impact on inference performance, particularly due to the dynamic range expansion caused by Batch Normalization (BN) when combined with Depthwise Convolution (DC), which requires reconstruction of the learning model, increasing processing load and time.
Innovation Solution
An information processing apparatus that includes a filter elimination processing section to remove filters with small variance in neural networks with Depthwise convolution, Pointwise convolution, and Batch normalization layers, followed by quantization, along with adjustment and incorporation processing to maintain parameter relationships and reduce quantization errors.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Manufacturing precision
If Batch Normalization is used after Depthwise Convolution to maintain inference performance, then the dynamic range of parameters expands making quantization difficult, but if Batch Normalization is removed to enable quantization, then the processing load for model reconstruction increases and quantization time increases
Solution Approach 1:
The patent extracts and eliminates specific filters with small variance from the Batch Normalization layer, separating the problematic components from the overall structure. This allows quantization to proceed on the remaining parameters while removing only the filters that cause dynamic range expansion issues
Solution Approach 2:
The patent changes the parameter representation by eliminating filters based on variance thresholds, transforming the parameter distribution to enable effective quantization. By modifying which parameters are retained (those with sufficient variance) versus eliminated (those with small variance), the patent makes the remaining parameters suitable for quantization without requiring complete model reconstruction
2Ease of manufacture
If all filters in Batch Normalization layer are eliminated to enable quantization, then quantization can be performed, but the inference performance degrades significantly
Solution Approach 1:
The patent applies local quality by differentiating between filters based on their individual variance characteristics. Instead of uniformly eliminating all filters or keeping all filters, it selectively eliminates only those filters with small variance (below a threshold), while preserving filters with larger variance that contribute more significantly to inference performance. This localized approach ensures quantization ease while maintaining reliability
3Manufacturing precision
If model reconstruction is performed to remove Batch Normalization layer, then quantization becomes feasible, but the processing time and computational load increase significantly
Solution Approach 1:
The patent performs preliminary filtering of variance-based importance metrics before the actual quantization process. By pre-identifying and eliminating insignificant filters (those with small variance) in advance, the patent prepares the model in a state that is already suitable for quantization, avoiding the need for complete model reconstruction and reducing the time required for the quantization process
Data Source
AI summary
The information processing apparatus is configured to have a filter elimination processing section for eliminating a filter whose dispersion is less than a threshold value in a neural network having a Depthwise convolution layer, a Pointwise convolution layer, and a Batch normalization layer, and have a quantization processing section for performing quantization of the neural network.