CNN Pooling Layer Features for Object Localization
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing object localization methods, such as data-driven detection (DDD), while efficient, can be improved by better incorporating spatial information into global image representations while maintaining a compact representation for effective prominent object detection in images.
Innovation Solution
The method employs a deep convolutional neural network (CNN) to generate image representations based on activations from specific layers, allowing for similarity-based retrieval and transfer of object location information from annotated images to query images, with optional metric learning for improved precision.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If Fisher vectors with dense spatial pyramids or probability maps are used for localized representation, then detection precision is improved, but device complexity and computation time increase
Solution Approach 1:
The patent changes the parameter of feature encoding from traditional Fisher vectors with dense spatial pyramids to CNN-based feature representations. This parameter change maintains localized representation capabilities while simplifying the encoding process and reducing computational complexity, as CNNs automatically learn hierarchical features that capture spatial information more efficiently than hand-crafted features.
Solution Approach 2:
The patent replaces the mechanical system of hand-crafted feature extraction (Fisher vectors, spatial pyramids) with a learned system using convolutional neural networks. This substitution allows the system to automatically learn optimal feature representations that capture spatial information, eliminating the need for complex manual feature engineering while improving detection precision.
2Measurement precision
If metric learning algorithms are used to find optimal projections, then retrieval accuracy is improved, but computation time increases
Solution Approach 1:
The patent applies metric learning algorithms during the offline training phase to learn optimal projection matrices that map CNN features to a lower-dimensional space optimized for retrieval. By performing this computationally intensive optimization in advance, the system achieves high retrieval accuracy while keeping online query processing fast, as the learned projections can be applied efficiently during actual detection tasks.
3Measurement precision
If high-dimensional feature vectors are used for comprehensive image representation, then detection accuracy is improved, but compression efficiency decreases
Solution Approach 1:
The patent extracts only the most relevant features from high-dimensional CNN representations by using metric learning to identify and retain the most discriminative dimensions. This extraction process reduces feature vector dimensionality while maintaining detection accuracy, as the learned projections focus on the most informative features for the specific detection task at hand.
Solution Approach 2:
The patent creates a composite representation by combining CNN features with learned metric projections. This composite approach merges the advantages of deep features (comprehensive representation) with the efficiency of dimensionality reduction (fast retrieval), resulting in a feature space that is both accurate and compact for similarity-based detection.
Data Source
AI summary
A system and method provide object localization in a query image based on a global representation of the image generated with a model derived from a convolutional neural network. Representations of annotated images and a query image are each generated based on activations output by a layer of the model which precedes the fully-connected layers of the neural network. A similarity is computed between the query image representation and each of the annotated image representations to identify a subset of the annotated images having the highest computed similarity. Object location information from at least one of the subset of annotated images is transferred to the query image and information is output, based on the transferred object location information.