CG Training Data Generation for Multi-Sensor Image Estimation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The generation of training data for machine learning models is challenging due to the difficulty in acquiring diverse imaging conditions and sensor configurations, leading to suboptimal estimation results, especially when using stereo cameras and TOF images.
Innovation Solution
A method for generating training data using a computer graphics (CG) model to simulate various imaging conditions, including different camera positions, angles, optical systems, and sensor types, to create diverse input and output images for machine learning.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If actual landscapes are photographed and pictures are drawn to collect training data, then diverse imaging conditions can be achieved, but time and effort are required and it may be difficult to acquire desired teaching data
Solution Approach 1:
The patent uses CG (computer graphics) images to copy and simulate real-world imaging conditions. Instead of photographing actual landscapes under various conditions, the system generates synthetic training data that replicates diverse imaging scenarios including different camera positions, angles, and sensor types, thereby eliminating time-consuming field data collection while maintaining data diversity
Solution Approach 2:
The patent performs preliminary generation of training data through CG simulation before actual model training. By pre-generating diverse training images with known ground truth information in a virtual environment, the system prepares comprehensive training datasets without requiring subsequent physical data collection efforts
2Measurement precision
If images are captured from the center position of a stereo camera, then accurate depth estimation can be achieved, but it may be physically difficult to acquire the image due to camera size constraints
Solution Approach 1:
The patent uses CG simulation to copy the ideal center-positioned camera view that would be difficult to obtain physically. The virtual environment allows placement of a reference camera at the exact center position between stereo cameras, generating teaching data that reflects true depth relationships without being constrained by physical camera dimensions
Solution Approach 2:
The patent introduces a CG virtual environment as an intermediary between the physical stereo camera system and the desired center-positioned teaching data. This virtual mediator enables generation of accurate depth estimation training data by simulating optical paths and geometric relationships that would be impossible to capture directly with physical sensors
3Quantity of substance
If a TOF image is used to acquire distance information, then distance data can be obtained, but it is difficult to accurately acquire the distance from the plane of the sensor that has acquired the RGB image
Solution Approach 1:
The patent uses CG simulation to copy and align the optical reference planes of different sensor types. By virtually positioning cameras and sensors at identical locations with matched optical characteristics in the simulation environment, the system generates training data where RGB and depth information are perfectly registered, eliminating misalignment errors that occur when using separate physical TOF and RGB cameras
Solution Approach 2:
The patent changes the parameter of sensor positioning from physical separation to virtual coincidence. By adjusting the virtual camera parameters in CG simulation to place multiple sensor viewpoints at the exact same spatial location, the system achieves perfect alignment between color and depth data without the physical constraints that cause misalignment in real-world multi-sensor systems
Data Source
AI summary
To generate training data setting various sensor states.A method for generating data is a method for generating training data used for machine learning from a CG model using a processor. The method for generating data includes: generating, through simulation, a first image acquired under a first imaging condition and a second image acquired under a second imaging condition that is different from the first imaging condition, in the CG model, and acquiring at least the first image and the second image as input image data in the training data.