Real-time Pose Estimation for Unseen Objects via Displacement Maps
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods for detecting the presence, location, and orientation of objects in environments, especially unseen objects not represented in training data, are computationally expensive and less accurate, particularly in real-time applications like augmented reality and robotics.
Innovation Solution
A method using multiple artificial neural networks (ANNs) to generate heat maps and displacement maps from images, allowing for the determination of object location and orientation, even for objects not in the training data, by applying images to a first ANN for intermediate output, then to a second ANN for heat maps, and further to third ANNs for displacement maps to define bounding box vertices.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If existing methods for object detection and pose estimation are used, then detection capability is achieved, but computational cost is high and accuracy is reduced for unseen objects
Solution Approach 1:
The patent segments the pose estimation task into multiple independent components: (1) detecting object presence and generating a bounding box, (2) estimating pose parameters independently within that bounding box. This segmentation allows each component to be processed separately and efficiently, reducing overall computational complexity while maintaining accuracy for unseen objects.
Solution Approach 2:
The patent introduces an intermediary bounding box as a mediator between object detection and pose estimation. The bounding box serves as an intermediate representation that simplifies the pose estimation problem by providing a constrained search space, thereby reducing computational requirements while improving accuracy for objects not in the training data.
2Productivity
If existing object detection methods are applied, then object presence is detected, but processing speed is slow for real-time applications
Solution Approach 1:
The patent performs preliminary object detection and bounding box generation before pose estimation. By pre-establishing the bounding box that constrains the search space, the system prepares the data in advance, enabling faster pose parameter calculation while maintaining high accuracy through the constrained search region.
Solution Approach 2:
The patent divides the processing pipeline into segmented stages: fast object detection to locate the bounding box, followed by more precise but computationally lighter pose estimation within that box. This segmentation enables real-time processing by performing computationally intensive operations only where needed.
3Measurement precision
If traditional pose estimation methods are used, then pose is determined, but accuracy is reduced for objects not represented in training data
Solution Approach 1:
The patent transitions from traditional 2D image space to a constrained search space defined by the detected bounding box. This dimensional constraint allows the system to generalize to unseen objects by searching for pose parameters within the already-detected spatial boundaries, improving accuracy without requiring extensive training data for each object type.
Solution Approach 2:
The patent changes the approach from learning object-specific features to estimating pose parameters (rotation, translation, scale) within a detected bounding box. This parameter-based approach is universally applicable to any object type, enabling high accuracy for unseen objects by focusing on geometric transformations rather than object-specific recognition.
Data Source
AI summary
Example embodiments allow for fast, efficient determination of bounding box vertices or other pose information for objects based on images of a scene that may contain the objects. An artificial neural network or other machine learning algorithm is used to generate, from an input image, a heat map and a number of pairs of displacement maps. The location of a peak within the heat map is then used to extract, from the displacement maps, the two-dimensional displacement, from the location of the peak within the image, of vertices of a bounding box that contains the object. This bounding box can then be used to determine the pose of the object within the scene. The artificial neural network can be configured to generate intermediate segmentation maps, coordinate maps, or other information about the shape of the object so as to improve the estimated bounding box.