Real-Time Video Object Recognition Using CNN Feature Matching
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current computer vision approaches for object recognition are computationally intensive and do not perform well in real-time, relying on hand-crafted local image descriptors and spatial relationships that are difficult to optimize for real-time video object recognition.
Innovation Solution
A system utilizing a pre-trained convolutional neural network (CNN) as a feature extractor to generate high-dimensional feature vectors, which are matched against a feature dictionary to generate probability distributions for object recognition, allowing for real-time video object recognition on mobile platforms.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If hand-crafted local image descriptors (SIFT, SURF) are used for object recognition, then feature extraction can be performed, but the matching process becomes computationally intensive and cannot achieve real-time performance
Solution Approach 1:
The patent extracts only the essential matching operation from the traditional pipeline by using pre-trained CNN features as fixed descriptors, eliminating the need for complex hand-crafted feature extraction and optimization processes. The CNN features are extracted once during training and reused during inference, separating the heavy computation from real-time operation.
Solution Approach 2:
The patent performs feature extraction and optimization in advance by using pre-trained CNN models to generate feature descriptors before the actual object recognition task. These pre-computed CNN features are stored and reused during real-time inference, eliminating the need for repeated heavy computation during runtime.
2Measurement precision
If deformable parts model with optimization process is used to add spatial relationships, then object recognition accuracy improves, but computational intensity increases and real-time performance is lost
Solution Approach 1:
The patent replaces the iterative optimization process of traditional methods with a direct CNN-based feature matching approach. Instead of mechanically optimizing spatial relationships through iterative algorithms, the CNN automatically learns and encodes spatial relationships during training, eliminating the need for runtime optimization computations.
3Ease of operation
If traditional feature matching approaches are used, then object recognition can be performed, but the system cannot operate on low-power mobile platforms in real-time
Solution Approach 1:
The patent uses pre-computed CNN features that can be stored and reused multiple times without degradation, replacing the need for continuous heavy computation. The feature descriptors are extracted once and then used repeatedly during real-time inference on mobile devices, making the system feasible for low-power platforms.
Data Source
AI summary
Described is a system for real-time object recognition. During operation, the system extracts convolutional neural network (CNN) feature vectors from an input image. The input image reflects a scene proximate the system, with the feature vector representing an object in the input image. The CNN feature vector is matched against feature vectors stored in a feature dictionary to identify k nearest neighbors for each object class stored in the feature dictionary. The matching results in a probability distribution over object classes stored in the feature dictionary. The probability distribution provides a confidence score that each of the object classes in the feature dictionary are representative of the object in the input image. Based on the confidence scores, the object in the input image is then recognized as being a particular object class when the confidence score for the particular object class exceeds a threshold.