Convolutional Neural Network Filter Integration for Inference
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The convolution operation in CNN inference processing is computationally intensive due to the large number of layers, and existing methods to reduce calculation, such as layer deletion, lack versatility and may compromise accuracy.
Innovation Solution
An integration device, method, and program that delete and integrate activation function processing between convolutional layers, specifically combining filters from multiple layers, such as those using 1×1 and 3×3 filters, to reduce the number of convolution operations while maintaining accuracy.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If multiple convolutional layers are used in CNN inference processing, then recognition accuracy is improved, but calculation amount increases enormously
Solution Approach 1:
The patent merges multiple convolutional layers into a single integrated convolutional layer by combining their filters. Specifically, it integrates the filter parameters from multiple layers into one layer, reducing the number of convolution operations while preserving the hierarchical feature extraction capability. This merging approach maintains recognition accuracy by keeping the combined filter parameters that capture essential features from all original layers, while dramatically reducing computational burden.
2Productivity
If the number of convolutional layers is reduced to decrease calculation amount, then processing speed is improved, but recognition accuracy deteriorates
Solution Approach 1:
Instead of simply reducing the number of layers, the patent combines the functional parameters (filters) from multiple layers into a single layer. The integrated layer uses a set of filter parameters that are derived from the parameters of multiple original layers, thereby preserving the feature extraction capabilities of the deeper network architecture while operating with a shallower structure. This approach maintains recognition accuracy by ensuring that the combined filters capture the essential features that would have been extracted by the multiple separate layers.
3Measurement precision
If activation function processing is retained between convolutional layers, then feature extraction quality is improved, but calculation amount increases
Solution Approach 1:
The patent extracts and eliminates the activation function processing steps from the network architecture. By removing these non-linear transformation operations between convolutional layers, the calculation amount is significantly reduced. The integration process absorbs the feature extraction functionality that would have been performed by the activation functions into the combined filter parameters of the integrated convolutional layer, thereby maintaining feature extraction quality while eliminating computationally expensive operations.
Data Source
AI summary
The integration unit 26, using configuration information of the convolutional neural network model and each filter used in each convolutional layer of the convolutional neural network model as inputs, deletes one or more pieces of activation function processing performed between the plurality of convolutional layers and integrates a plurality of filters used in the plurality of convolutional layers.