CNN Filter Optimization via Inner Product Search
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Deep neural networks, particularly Convolutional Neural Networks (CNNs), face high computational, memory, and power requirements, making them inefficient for deployment on low-cost and low-power edge devices like smartphones and IoT sensors, where high responsiveness and low latency are needed.
Innovation Solution
A method and system that optimize CNN operations by using a maximum inner product search to query an index of filters, selecting only relevant filters for each block, and applying them to reduce computational costs, while maintaining accuracy through sparse feature maps and efficient indexing algorithms like Locality-sensitive hashing.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If all filters are applied to each block in CNN operations, then comprehensive feature extraction is achieved, but computational cost increases significantly
Solution Approach 1:
The patent extracts and applies only the most relevant filters to each block based on maximum inner product search results, rather than applying all filters. This selective extraction reduces computational operations while maintaining the essential feature extraction capability, directly resolving the contradiction between comprehensive feature extraction and computational efficiency
Solution Approach 2:
The patent applies partial action by selecting a subset of top-k filters (where k < total filters) for each block based on relevance scoring through inner product computation. This partial application of filters maintains sufficient feature extraction accuracy while significantly reducing the computational burden of applying all filters
2Measurement precision
If more filters are used in CNN layers, then detection precision improves, but power consumption increases
Solution Approach 1:
The patent extracts only the necessary filters for each block through maximum inner product search, eliminating redundant filter applications. This reduces the number of computational operations directly proportional to power consumption while retaining sufficient detection precision through selective filter application
Solution Approach 2:
The patent dynamically changes the number of active filters per block based on the relevance scoring from inner product computations. By adjusting the filter application parameter (from all filters to top-k filters) based on input characteristics, the system optimizes the trade-off between detection precision and power consumption adaptively
3Reliability
If comprehensive filter application is performed across all blocks, then processing accuracy is maintained, but processing time increases
Solution Approach 1:
The patent performs preliminary computation of inner products between filters and blocks to determine relevance scores before actual filter application. This preliminary action identifies and prioritizes relevant filters, enabling subsequent selective application that maintains accuracy while reducing inference time by avoiding unnecessary filter operations
Solution Approach 2:
The patent applies partial action by limiting filter application to only the top-k most relevant filters for each block, determined through preliminary inner product computation. This partial application maintains processing accuracy for critical features while significantly reducing the time-consuming operations on less relevant filters
Data Source
AI summary
There are provided a system and method of optimization of operating a Convolutional Neural Network (CNN), the method including: for each given layer of the CNN, obtaining at least one input feature map, retrieving an index of one or more filters in the given layer; generating one or more output feature maps corresponding to the one or more filters, including: for each given block, querying the index to find a number of selected filters meeting a predetermined criterion related to respective inner products between the selected filters and the given block, applying the selected filters on the given block to obtain a number of output values for a given output position in a number of output feature maps corresponding to the number of selected filters, and filling in the given output position in the rest of output feature maps with a value of zero, thereby reducing the computational cost of operating the CNN.