Convolutional Basis Filter Layer for Neural Network Efficiency
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Deep learning neural networks face complexity and resource inefficiency due to the large number of redundant kernels in convolutional layers, which hinders performance and resource utilization.
Innovation Solution
Implementing a convolutional basis filter layer that uses a bank of basis kernels and scaling coefficients to reduce the number of operations and parameters, allowing for faster and less resource-intensive neural network processing without retraining the network.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If traditional convolutional layers with many small 2D kernels are used, then the neural network can process data effectively, but the computational complexity and resource requirements increase significantly
Solution Approach 1:
The patent segments the traditional single large kernel into multiple smaller basis kernels that work together. Instead of using one large 2D kernel directly, the invention decomposes it into several smaller kernels that can be processed separately and then combined, reducing the computational burden while maintaining the same filtering effect.
Solution Approach 2:
The patent combines multiple scaled convolution results from different basis kernels to produce the final output. By merging the results of several simpler operations rather than performing one complex operation, the system achieves the same computational goal with reduced complexity and better resource utilization.
2Productivity
If traditional convolutional layers with tens of thousands to millions of kernels are used, then the neural network maintains high processing capability, but the number of redundant kernels increases
Solution Approach 1:
The patent extracts only the essential computational components from the traditional convolutional layer. By identifying and removing redundant kernels while retaining the core functionality through a smaller set of basis kernels, the system maintains processing capability with significantly fewer parameters.
Solution Approach 2:
The patent changes the parameters of the convolutional operation by introducing scaling factors that allow smaller basis kernels to achieve the equivalent effect of many larger kernels. This parameter transformation enables the system to reduce the number of kernels while preserving processing power through adaptive scaling of convolution results.
3Adaptability or versatility
If more kernels are used in convolutional layers, then the neural network can capture more features, but the resource efficiency decreases
Solution Approach 1:
The patent introduces dynamic scaling factors that adaptively adjust the contribution of each basis kernel based on the input data characteristics. This dynamic approach allows the system to capture diverse features through a smaller set of kernels by varying their weights and scaling, rather than relying on a fixed large number of kernels, thereby improving resource efficiency.
Data Source
AI summary
An electronic device, method, and computer readable medium for 3D association of detected objects are provided. The electronic device includes a memory and at least one processor coupled to the memory. The at least one processor configured to convolve an input to a neural network with a basis kernel to generate a convolution result, scale the convolution result by a scalar to create a scaled convolution result, and combine the scaled convolution result with one or more of a plurality of scaled convolution results to generate an output feature map.