Depth Estimation Neural Network Motion Segmentation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing technologies for estimating depth using neural networks fail to accurately model the motion of moving objects, leading to inaccurate depth estimation.
Innovation Solution
An information processing apparatus that includes a depth calculation unit, a motion calculation unit, and a correspondence calculation unit, which use two input images and camera parameters to calculate depth information, motion information, and correspondence information, and update the parameters of estimation models to optimize loss functions for accurate depth and motion estimation.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Device complexity
If only stationary background motion is modeled in egomotion, then the neural network can be trained with simpler geometric constraints, but depth estimation accuracy for moving objects deteriorates
Solution Approach 1:
The patent segments the scene into stationary background and moving object components. The motion modeling is divided into separate treatments: stationary background uses standard egomotion constraints, while moving objects are handled with additional motion parameters. This segmentation allows the system to maintain simplicity for the majority of the scene while improving accuracy for moving objects without requiring complete re-modeling of the entire scene.
Solution Approach 2:
The patent introduces dynamic motion parameters that can adapt to different scene conditions. Instead of using a fixed rigid egomotion model, the system allows motion parameters to vary dynamically based on whether a region is identified as stationary or moving. This enables the model to transition between simple and complex representations as needed, improving depth estimation accuracy for moving objects while maintaining computational efficiency for stationary regions.
2Measurement precision
If moving objects are included in the motion model, then depth estimation accuracy for moving objects improves, but the complexity of the neural network and training process increases
Solution Approach 1:
The patent applies different quality levels of motion modeling to different regions of the image. Stationary background regions use simpler motion constraints, while moving object regions receive more sophisticated motion modeling with additional parameters. This local differentiation allows the system to improve depth estimation accuracy where needed (for moving objects) without increasing complexity across the entire system.
Solution Approach 2:
The patent implements partial action by applying complex motion modeling only to the extent necessary for moving objects rather than to the entire scene. The system identifies moving object regions and applies enhanced motion constraints only to those specific areas, leaving the majority of the stationary scene to use simpler models. This partial application of complex modeling improves moving object depth estimation while limiting the overall increase in system complexity.
Data Source
AI summary
According to an embodiment, an information processing apparatus includes one or more hardware processors. The one or more hardware processors update parameters of the first estimation model and the second estimation model so as to optimize a first loss function including a term indicating a difference between the correspondence information and correspondence training data that is training data concerning correspondence between the first pixel and the second pixel, a second loss function including a term concerning a depth, and a third loss function including a term indicating a difference in pixel value between the first pixel and the second pixel whose correspondence is indicated by the correspondence information, and generate the first estimation model and the second estimation model represented by the updated parameters.