Neural Network Filter Compression via Layer-Wise Normalization
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing neural network model compression techniques fail to effectively reduce latency while minimizing accuracy loss and user complexity, often causing information distortion and significant accuracy reductions.
Innovation Solution
A method for compressing neural network models by determining the importance of filters using a metric, normalizing their importance layer by layer, and removing filters based on a compression ratio and redundancy, ensuring that at least one filter remains in each layer to prevent layer removal and maintain accuracy.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of time
If filters are removed to compress the neural network model, then latency is reduced and model size is decreased, but model accuracy is rapidly reduced and information is distorted
Solution Approach 1:
The patent changes the parameter of filter importance evaluation by introducing a normalization process that transforms raw importance values into normalized values within a specific range. This allows for more precise control over which filters are removed, enabling the system to achieve the desired compression ratio while maintaining model accuracy by making informed decisions about filter removal based on normalized importance metrics.
Solution Approach 2:
The patent applies different treatment to different filters within the same layer based on their individual importance scores. Instead of uniformly removing filters or keeping all filters, the system evaluates each filter's contribution locally and makes targeted removal decisions. This localized approach ensures that only filters with low importance are removed while preserving critical filters, thus maintaining model accuracy while achieving compression.
2Quantity of substance
If the entire layer is removed to achieve compression, then model size is reduced, but information contained in the model is distorted and accuracy is significantly reduced
Solution Approach 1:
The patent extracts only the necessary and least important filters from each layer for removal, rather than removing entire layers. By selectively extracting individual filters based on their normalized importance scores, the system achieves model compression while preserving the structural integrity of layers and retaining essential information, thus avoiding the information loss that would result from removing entire layers.
3Productivity
If a user directly configures compression ratio for each layer, then compression control is achieved, but user convenience is reduced due to high entry barrier requiring lightweighting knowledge
Solution Approach 1:
The patent implements a self-service mechanism where the system automatically calculates normalized importance values for all filters and performs intelligent filter selection and removal without requiring user intervention. The system autonomously determines which filters to remove based on the overall compression ratio target, eliminating the need for users to manually configure compression ratios for each layer and thus improving user convenience while maintaining effective compression control.
Data Source
AI summary
A method for compressing a neural network model is disclosed. The method for compressing a neural network model includes receiving, at a processor of the electronic apparatus, an original model including a plurality of layers each including a plurality of filters, a compression ratio to be applied to the original model, and a metric for determining an importance of the plurality of filters, determining the importance of the plurality of filters using the metric, normalizing the importance of the plurality of filters layer by layer, and compressing the original model by removing at least one filter among the plurality of filters based on the normalized importance and the compression ratio.