Silhouette-Based 3D Object Recognition for Mobile Devices
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current object recognition technologies are limited in recognizing multiple objects simultaneously and are computationally intensive, making them unsuitable for real-time tracking on mobile devices, especially for 3D objects, which lack scale invariance and practicality on resource-constrained devices.
Innovation Solution
The method generates object-image alignment data by creating a digital 3D model of an object, obtaining 2D images from various viewpoints, generating silhouettes, registering images with these silhouettes, and deriving descriptors to create an object-image alignment packet that can be used for real-time recognition and tracking on mobile devices.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If computationally intensive approaches are used to generate large numbers of possible poses for 3D object recognition, then recognition accuracy is improved, but processing speed and suitability for mobile devices deteriorate
Solution Approach 1:
The system performs preliminary actions by pre-computing and storing silhouette templates for multiple object poses in advance. During real-time recognition, the mobile device only needs to compare captured images against these pre-generated silhouettes, avoiding computationally intensive pose generation during operation. This resolves the contradiction by shifting computational load from runtime to offline preparation.
Solution Approach 2:
The invention extracts only the essential silhouette information from full 3D models, removing unnecessary computational complexity. By representing objects as simple 2D silhouettes rather than complete 3D geometries, the system achieves adequate recognition accuracy with significantly reduced processing requirements suitable for mobile devices.
2Measurement precision
If traditional object recognition technologies are used, then single object recognition capability is achieved, but the ability to recognize multiple objects simultaneously deteriorates
Solution Approach 1:
The silhouette-based recognition system is universally applicable to multiple object types without requiring object-specific processing pipelines. The same silhouette comparison algorithm works for diverse objects (toys, people, graphic images, etc.), enabling multi-object recognition while maintaining single-object accuracy. This resolves the contradiction by creating a versatile system that handles both single and multiple objects with the same mechanism.
3Manufacturing precision
If scale-invariant methods are not used, then 3D object modeling accuracy is maintained, but applicability to handheld devices deteriorates
Solution Approach 1:
The system changes the critical parameter from 3D geometric precision to 2D silhouette matching. By transforming the recognition problem from three-dimensional space to two-dimensional image space, the system achieves adequate modeling accuracy while becoming computationally feasible for handheld devices with limited processing power and memory.
Data Source
AI summary
An object-image alignment data generating method for use in an object recognition system is presented. The method obtains a 3D model and a set of 2D images of the object. Each 2D image from the set is captured based on a particular camera point of view. The method then uses the 3D model of the object to generate multiple silhouettes of the object according to different camera point of views. Each silhouette is then matched and aligned with a 2D image based on the corresponding camera point of view. The method also derives at least one descriptor from the 2D images and compiles feature points that correspond to the descriptors. Each feature point includes a 2D location and a 3D location. The method then generates an object-image alignment packet by packaging the 2D images, the descriptors, and the feature points.