3D Reconstruction Model Training Without Background Masking
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional photogrammetry techniques using neural networks require background removal or masking during training due to background interference in image data, which is cumbersome and affects training accuracy.
Innovation Solution
A machine learning model is trained with constraints on background and virtual viewpoint densities, allowing direct training on images with background, using a multi-layer perceptron to define object surfaces and densities, eliminating the need for background masking.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If conventional photogrammetry techniques using neural networks are used, then three-dimensional shape estimation can be performed, but background removal or masking is required during training which is cumbersome and affects training accuracy
Solution Approach 1:
The patent extracts and isolates the background density as a separate controllable parameter from the image data. By extracting background information and representing it through density values at background points, the system separates background interference from object information, allowing the neural network to process both simultaneously without requiring manual background removal or masking operations
Solution Approach 2:
The patent changes the parameter representation of background from binary mask values (0 or 1) to continuous density values. This parameter transformation allows the background to be represented in the same numerical space as the object, enabling the neural network to handle both together. The density parameter at background points provides a smooth gradient that helps the network distinguish between object and background regions without explicit masking
2Measurement precision
If background removal is performed using foreground masks, then training accuracy can be maintained, but additional processing steps and time are required
Solution Approach 1:
The patent merges the background representation into the same processing framework as the object representation. Instead of handling background and object separately through masking, both are processed together through the neural network using density values. This unified approach eliminates the need for separate background removal steps while maintaining training accuracy
Solution Approach 2:
The neural network automatically learns to distinguish between object and background regions through the density parameter constraints without requiring external masking operations. The network self-regulates by learning appropriate density distributions, making the system self-sufficient and eliminating manual or pre-processing intervention for background removal
Data Source
AI summary
Viewpoint image DB 120 stores a plurality of viewpoints and images obtained by capturing images of an object from the plurality of viewpoints in association with each other. Data acquisition means 111 reads out viewpoint image DB 120 and supplies it to generation means 112. Generation means 112 uses machine learning model 121 in an initial state or in the middle of training, to generate a virtual image that can be obtained when an image of a virtual object is captured from a virtual viewpoint. Training means 113 compares the images stored in viewpoint image DB 120 with the virtual images generated by generation means 112, and updates machine learning model 121 that is constrained so that the densities at the background point and the virtual viewpoint are each a predetermined constant. The present invention eliminates the need to remove the background from each image when training a machine learning model that uses images obtained by capturing an object from a plurality of viewpoints as training data to estimate a three-dimensional model of the object in a virtual space.