Pooling Layer Parameter Minimization in CNN Feature Learning
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing image feature extraction methods using deep learning face challenges in optimizing the pooling layer parameters, leading to poor performance due to undiscriminating image regions and excessive parameter numbers.
Innovation Solution
An image feature learning device and method that incorporates a convolutional neural network with a fully convolutional layer, a weighting matrix estimation layer, and a pooling layer to learn feature vectors based on training data, minimizing the number of parameters and enhancing robustness by estimating weightings for each element of the feature tensor.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If a fully connected layer is used as a pooling layer in CNN, then the flexibility of parameter adjustment is improved, but the number of parameters becomes enormous and optimization becomes difficult
Solution Approach 1:
The patent segments the pooling layer into multiple independent pooling units, where each unit processes a specific region of the feature map. This segmentation reduces the overall parameter count while maintaining adaptability, as each pooling unit has its own learnable parameters that can be independently optimized.
Solution Approach 2:
The patent applies local quality by allowing different pooling operations with distinct parameters to be applied to different spatial regions of the feature map. This enables the network to adapt to local variations in the data while keeping the total number of parameters manageable through regional specialization.
2Productivity
If traditional pooling operations are used, then the computational efficiency is improved, but the performance is adversely affected by undiscriminating image regions
Solution Approach 1:
The patent introduces dynamic pooling by making pooling parameters learnable through backpropagation, allowing the pooling operation to adapt dynamically to the input data characteristics. This dynamic adjustment enables the network to handle undiscriminating regions effectively while maintaining computational efficiency.
Solution Approach 2:
The patent changes the pooling parameters from fixed values to learnable parameters that can be optimized during training. This parameter change allows the pooling layer to adapt to different image regions and conditions, improving robustness without sacrificing computational efficiency.
Data Source
AI summary
The purpose of the present invention is to enable learning of a neural network for extracting features of images having high robustness from an undiscriminating image region while minimizing the number of parameters of a pooling layer. A parameter learning unit 130 learns parameters of each layer in a convolutional neural network configured by including a fully convolutional layer for performing convolution of an input image to output a feature tensor of the input image, a weighting matrix estimation layer for estimating a weighting matrix indicating a weighting of each element of the feature tensor, and a pooling layer for extracting a feature vector of the input image based on the feature tensor and the weighting matrix. The parameter learning unit 130 learns the parameters such that a loss function value obtained by calculating a loss function expressed by using a distance between a first feature vector of a first image and a second feature vector of a second image, which are relevant images and are obtained by applying the convolutional neural network, becomes smaller.