Image Feature Representation via Selective Pooling Vector
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional image classification methods, such as the Bag-of-Visual-Words (BOV) model and its extensions, are suboptimal in distinguishing finer details in fine-grained image recognition tasks due to the discarding of spatial order and averaging of local descriptors, which can smear important visual patterns.
Innovation Solution
The selective pooling vector (SPV) method uses a Gaussian Mixture Model to encode local descriptors, eliminating those with low similarity scores to form a more discriminative image feature representation by selecting only the most representative local features, thereby avoiding the averaging issues present in Super vector and Fisher vector representations.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If local descriptors are averaged to form image representations (as in Super vector and Fisher vector coding), then computational efficiency is improved, but fine visual patterns are smeared and classification performance on fine-grained tasks deteriorates
Solution Approach 1:
The patent extracts only the most representative local descriptors by comparing each descriptor to Gaussian mixture model components and selecting those with the highest similarity scores. This extraction process removes non-discriminative descriptors that would otherwise be averaged and cause smearing of fine visual patterns, while maintaining computational efficiency by working with a reduced set of selected descriptors rather than all descriptors.
2Speed
If the Bag-of-Visual-Words model is used to represent images, then processing speed is improved, but spatial order information is discarded and descriptive power is limited
Solution Approach 1:
The patent changes the representation parameters by using Gaussian mixture model components to encode local descriptors with continuous weights based on similarity scores, rather than using discrete visual words. This parameter change allows the model to retain more information about descriptor characteristics and spatial relationships while maintaining the computational efficiency of histogram-based methods.
3Quantity of substance
If all local descriptors are retained in the image feature vector, then completeness of representation is improved, but computational complexity and dimensionality increase
Solution Approach 1:
The patent applies partial action by selecting only a subset of local descriptors that have the highest similarity scores to Gaussian mixture model components. Rather than using all local descriptors, this selective approach retains the most discriminative features while reducing the dimensionality of the feature vector, thereby decreasing computational complexity without sacrificing representation quality.
Data Source
AI summary
Techniques are disclosed for image feature representation. The techniques exhibit discriminative power that can be used in any number of classification tasks, and are particularly effective with respect to fine-grained image classification tasks. In an embodiment, a given image to be classified is divided into image patches. A vector is generated for each image patch. Each image patch vector is compared to the Gaussian mixture components (each mixture component is also a vector) of a Gaussian Mixture Model (GMM). Each such comparison generates a similarity score for each image patch vector. For each Gaussian mixture component, the image patch vectors associated with a similarity score that is too low are eliminated. The selectively pooled vectors from all the Gaussian mixture components are then concatenated to form the final image feature vector, which can be provided to a classifier so the given input image can be properly categorized.