Synthetic Depth Image Pairs for Missing-Value Restoration
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Depth images obtained by sensors often suffer from missing depth values due to shadow effects and the presence of highly absorbing objects, which adversely affect applications relying on accurate depth imaging.
Innovation Solution
Generate synthetic depth images with missing depth data by selectively removing portions of depth data using algorithms that simulate shadow effects and highly absorbing objects, then train a machine learning model with these synthetic images to restore missing depth data.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If depth images are captured using sensors, then depth data is obtained for imaging applications, but missing depth values occur in shadow regions and areas with highly absorbing objects
Solution Approach 1:
The patent creates synthetic depth images that copy the characteristics of real depth images including shadow effects and absorbing objects. By rendering virtual scenes with known ground truth depth values and applying the same shadow and absorption effects that occur in real imaging, the system generates training data that teaches the machine learning model to recover missing depth values in real images.
Solution Approach 2:
The patent performs preliminary action by pre-computing synthetic depth images with controlled missing data patterns before actual depth image processing is needed. The system renders virtual scenes, intentionally creates shadow regions and absorbing object effects, and stores these as training examples, so that when real depth images need processing, the model is already trained to handle such missing data scenarios.
2Ease of manufacture
If real depth images are used for training machine learning models, then training data is obtained, but it is difficult to obtain complete depth data for training due to shadow effects
Solution Approach 1:
Instead of relying on difficult-to-obtain real complete depth images, the patent copies the essential characteristics of real depth imaging scenarios through synthetic rendering. Virtual scenes are created with objects that cast shadows and have absorbing properties, replicating the challenging conditions found in real depth imaging without requiring actual physical scenes with perfect depth coverage.
Solution Approach 2:
The patent changes the parameter space by moving from physical real-world scenes to virtual rendered scenes. This allows precise control over lighting conditions, object materials, and camera parameters to systematically generate training data with various shadow and absorption scenarios, making training data acquisition easier and more systematic.
3Measurement precision
If machine learning models are trained to restore missing depth data, then accuracy of depth image processing is improved, but large datasets are required for effective training
Solution Approach 1:
The patent creates a universal training framework where a single synthetic scene rendering pipeline can generate diverse training examples by varying object positions, lighting conditions, and material properties. This multi-functional system produces many different shadow and absorption patterns from a core set of virtual scenes, reducing the need for an excessively large quantity of training data while maintaining high restoration accuracy.
Data Source
Figure 1
Figure 2
Figure 3A~3B
AI summary
Proposed concepts aim to provide a method for generating pairs of synthetic depth images by first rendering a synthetic reference depth image and then processing this rendered image with a depth data removal algorithm to generate a depth-omitted image that is missing at least a portion of depth data.