Pruned CNN Intrusion Detection for Faster Edge Security Inference
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
As convolutional neural networks (CNNs) grow deeper, the computational costs of inference increase, posing challenges for embedded sensors and mobile devices with limited resources, and existing methods for reducing costs through weight pruning are inefficient and require sparse libraries or retraining.
Innovation Solution
The method involves pruning filters with the smallest summed weights from CNNs, reducing computational costs without introducing sparsity, and using one-shot pruning across multiple layers to minimize retraining time, thereby reducing matrix multiplications and maintaining accuracy.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If CNN depth and number of filters are increased to improve classification accuracy, then classification accuracy is improved, but computational cost and inference time increase
Solution Approach 1:
The patent extracts and removes redundant filters from the CNN that contribute minimally to classification accuracy. By calculating the L1 norm of filter weights and removing filters with smallest norms, the system eliminates unnecessary computational operations while preserving the essential filtering capabilities needed for accurate classification, thus improving inference speed without significantly compromising accuracy
Solution Approach 2:
The patent changes the parameter configuration of the CNN by dynamically adjusting the number of filters based on their contribution to accuracy. Filters are pruned based on their L1 norm values, transforming the network from a fixed-depth architecture to a dynamically optimized one that maintains accuracy while reducing computational complexity and inference time
2Measurement precision
If CNN depth and number of filters are increased to improve classification accuracy, then classification accuracy is improved, but storage cost increases
Solution Approach 1:
The patent extracts and removes redundant filters that occupy storage space but contribute minimally to classification performance. By calculating the L1 norm of each filter and removing those with smallest norms, the system reduces the total number of parameters and weights that need to be stored, thereby reducing storage cost while maintaining classification accuracy through the preservation of essential filters
3Productivity
If weight pruning is applied to reduce computational cost, then computational cost is reduced, but model accuracy deteriorates and retraining is required
Solution Approach 1:
The patent performs preliminary evaluation of filter importance by calculating the L1 norm of filter weights before pruning. This preliminary assessment identifies which filters can be safely removed without significantly impacting accuracy, allowing the system to prune filters in advance while maintaining model performance without requiring extensive retraining
Solution Approach 2:
The patent treats certain filters as disposable components that can be removed without critically affecting model performance. By identifying and removing filters with minimal contribution (smallest L1 norms), the system eliminates computational overhead while maintaining accuracy, avoiding the need for costly retraining of the entire model
4Quantity of substance
If existing weight pruning methods are used to reduce computational cost, then storage cost is reduced, but the resulting sparse models lack optimizations for practical computation
Solution Approach 1:
The patent extracts and removes entire filters rather than creating sparse weight patterns within filters. This approach results in a dense, compact model structure that is easier to optimize and deploy on practical hardware systems, avoiding the complexity of sparse matrix operations while still achieving significant compression and computational cost reduction
Solution Approach 2:
The patent maintains homogeneity in the remaining filter structures after pruning, ensuring that all retained filters have uniform dimensions and properties. This homogeneous structure enables efficient memory access patterns and optimization opportunities that sparse, heterogeneous models lack, making the pruned model more practical for deployment on resource-constrained devices
Data Source
AI summary
Security systems and methods for detecting intrusion events include one or more sensors configured to monitor an environment. A pruned convolutional neural network (CNN) is configured process information from the one or more sensors to classify events in the monitored environment. CNN filters having the smallest summed weights have been pruned from the pruned CNN. An alert module is configured to detect an intrusion event in the monitored environment based on event classifications. A control module is configured to perform a security action based on the detection of an intrusion event.