Neural Network Filter Redundancy Detection for Model Compression
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current AI technologies face challenges in deploying Deep Neural Network (DNN) models on resource-constrained devices like smartphones and drones due to high computational costs and memory constraints, which leads to rapid battery discharge and reduced accuracy when compressing neural networks.
Innovation Solution
A method and system for compressing neural network models using multi-criteria based filter redundancy detection, where a computing device identifies and removes redundant filters based on predefined criteria, such as frequency of occurrence and feature extraction types, to reduce computational complexity without compromising accuracy.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If DNN models are deployed on resource-constrained devices, then real-time AI applications can be executed, but computational cost and memory usage increase leading to rapid battery discharge
Solution Approach 1:
The patent extracts and removes redundant filters from neural network layers based on multiple redundancy detection criteria. By identifying and eliminating filters that provide duplicate or minimal value to the network's output, the model size and computational requirements are reduced, enabling deployment on resource-constrained devices while maintaining accuracy.
Solution Approach 2:
The patent changes the parameter configuration of the neural network by adjusting the number of filters in each layer. Through iterative pruning and redundancy detection, the filter counts are optimized to achieve the minimal necessary configuration for maintaining accuracy, thereby reducing computational cost and energy consumption.
2Use of energy by moving object
If DNN models are compressed to reduce computational cost, then energy consumption decreases, but model accuracy deteriorates
Solution Approach 1:
The patent systematically changes the network parameters (filter counts) through controlled pruning while using multiple redundancy detection criteria to guide the process. This ensures that filters are removed in a way that minimizes impact on accuracy, achieving the right balance between compression and performance.
Solution Approach 2:
The patent employs feedback mechanisms where the impact of filter removal on model accuracy is continuously evaluated. The redundancy detection criteria provide feedback signals that guide subsequent pruning decisions, allowing the system to adapt and maintain accuracy thresholds while maximizing compression.
3Measurement precision
If multiple redundancy detection criteria are used to identify redundant filters, then filter selection accuracy improves, but computational complexity of the compression process increases
Solution Approach 1:
The patent segments the redundancy detection process into multiple independent criteria (e.g., filter response similarity, gradient importance, activation patterns). Each criterion operates independently to evaluate different aspects of filter redundancy, making the overall process more manageable and systematic while improving detection accuracy.
Solution Approach 2:
The patent introduces multiple dimensions of analysis for redundancy detection beyond traditional single-metric approaches. By evaluating filters across multiple criteria dimensions simultaneously, the system achieves more accurate identification of redundant filters while organizing the complexity into structured, independent evaluation categories.
Data Source
AI summary
A method and system of compressing a neural network model (NNM) is disclosed. The method includes determining a set of criteria based redundant filters from a set of filters, in a corresponding layer, based on each of the plurality of predefined redundancy detection criteria. A set of redundant filters is identified from the set of criteria based redundant filters based on a first intersection score among the set of criteria based redundant filters. Further, the set of redundant filters is identified from the set of criteria based redundant filters based on a normalized minimum or a normalized maximum. The NNM is compressed based on the set of redundant filters for each of the layer.