Depth Training Data Generation Using Synthetic 3D Viewpoints
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional depth sensors for range imaging are expensive and structurally difficult to implement, making the collection of a large set of depth values for different objects in a real environment complex and costly.
Innovation Solution
A system that uses a depth sensor to capture initial images and depth values, combined with a 3D graphics engine to generate synthetic data from different viewpoints, creating a training dataset for a neural network model that includes both real and synthetic data pairs.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If depth sensors are deployed to capture depth values from multiple positions or viewpoints, then the quantity and diversity of depth data increases, but the system complexity and cost increase significantly
Solution Approach 1:
The patent creates synthetic depth images by rendering 3D models from multiple virtual viewpoints, copying the effect of physical depth sensor deployments without the associated complexity. The system generates training data by virtually positioning a camera around a 3D model and synthesizing depth images, thereby obtaining diverse depth data without deploying multiple physical sensors.
2Quantity of substance
If depth sensors are deployed to capture depth values from multiple positions or viewpoints, then the quantity and diversity of depth data increases, but the implementation cost increases
Solution Approach 1:
The system copies the functional outcome of multiple depth sensor deployments through synthetic data generation. By rendering depth images from virtual viewpoints around a 3D model, the system obtains diverse training data without the cost of purchasing and deploying multiple physical depth sensors.
Solution Approach 2:
The patent replaces expensive, complex depth sensor hardware with computationally-generated synthetic data. The 'cheap' alternative is the software-based synthesis process that generates unlimited training samples without physical hardware costs.
3Measurement precision
If a large set of depth sensors is deployed for collecting depth values, then the quality and coverage of depth measurement improves, but the structural complexity of the system increases
Solution Approach 1:
The system achieves comprehensive depth measurement coverage by virtually rendering the 3D model from multiple angles and positions. This synthetic approach replicates the measurement quality that would require numerous physical depth sensors, but without the structural complexity of deploying and coordinating multiple sensors.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
A system for generation of training dataset is provided. The system controls a depth sensor to capture, from a first viewpoint, a first image a first depth value associated with the first object. The system receives tracking information from a handheld device associated with the depth sensor, based on a movement of the handheld device and the depth sensor in a 3D space. The system generates graphic information corresponding to the first object based on the received tracking information. The graphic information includes the first object from a second viewpoint. The system calculates a second depth value associated with the first object, based on the graphic information. The system generates, for a neural network model, a training dataset which includes a first combination of the first image and the first depth value, and a second combination of second images corresponding to the graphic information and the second depth value.