Depth-Based Object Pose Estimation Without Manual Annotation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Training deep neural networks (DNNs) to detect objects in images requires a large number of annotated visual or range images, which is time-consuming and resource-intensive, especially for applications like vehicle guidance, security, and product tracking.
Innovation Solution
The use of registered visual and range image data (RGB-D data) as ground truth for training DNNs, reducing the need for manual annotation and minimizing the time and computer resources required, by determining a four degree-of-freedom pose of objects and optimizing network parameters using loss functions based on silhouette comparisons.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If manual annotation of visual or range images is used to train deep neural networks, then training data accuracy is improved, but training time and computational resources increase significantly
Solution Approach 1:
The system uses the depth sensor data itself to automatically generate ground truth annotations for training images. The depth data provides inherent 3D spatial information that can be projected onto 2D images to create accurate object masks and annotations without requiring manual labeling, thus the system serves itself to produce training data
Solution Approach 2:
The patent creates synthetic training data by copying and transforming depth information from real scenes to generate corresponding annotated 2D images. The depth data serves as a template that is projected and rendered to produce multiple views with automatic annotations, eliminating the need for manual annotation of each image
2Measurement precision
If manual annotation of visual or range images is used to train deep neural networks, then training data accuracy is improved, but computational resources increase significantly
Solution Approach 1:
The depth sensor data itself is used to automatically generate ground truth annotations for training images. The depth data provides inherent 3D spatial information that can be projected onto 2D images to create accurate object masks and annotations without requiring manual labeling, thus the system serves itself to produce training data
Solution Approach 2:
The patent creates synthetic training data by copying and transforming depth information from real scenes to generate corresponding annotated 2D images. The depth data serves as a template that is projected and rendered to produce multiple views with automatic annotations, eliminating the need for manual annotation of each image
3Productivity
If registered visual and range image data (RGB-D data) is used as ground truth for training, then training efficiency is improved, but data processing complexity increases
Solution Approach 1:
The patent extracts the essential 3D spatial information from depth sensor data and separates it from the visual data. By projecting depth data onto 2D image planes and extracting object masks directly from depth thresholds, the complex RGB-D data is decomposed into simpler components that can be processed independently and combined to form training annotations
Data Source
AI summary
A depth image of an object can be input to a deep neural network to determine a first four degree-of-freedom pose of the object. The first four degree-of-freedom pose and a three-dimensional model of the object can be input to a silhouette rendering program to determine a first two-dimensional silhouette of the object. A second two-dimensional silhouette of the object can be determined based on thresholding the depth image. A loss function can be determined based on comparing the first two-dimensional silhouette of the object to the second two-dimensional silhouette of the object. Deep neural network parameters can be optimized based on the loss function and the deep neural network can be output.