Monocular Depth Estimation for Dynamic Scenes
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing monocular depth estimation methods struggle with jittering due to camera shaking and moving objects, leading to inaccurate depth data and low model accuracy, as they fail to effectively filter out moving objects and optimize model parameters.
Innovation Solution
The method involves acquiring and calibrating image frames containing dynamic objects, reconstructing frames using a preset depth estimation model, calculating reconstruction errors, and fusing point cloud and instance segmentation data to create mask data, which is used to train a depth estimation model with a loss function that excludes moving objects and corrects system errors, improving estimation accuracy.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If SFM principle is used to reconstruct images from monocular video frames, then depth estimation accuracy is improved for static scenes, but moving objects cause reconstruction errors and cannot be correctly reconstructed
Solution Approach 1:
The patent segments the video frames into static background regions and moving object regions using optical flow analysis and RANSAC plane fitting. By separating the scene into these components, the system can apply different processing strategies: SFM for static regions and exclusion/masking for moving regions, thereby resolving the contradiction between achieving accurate depth for static scenes and handling moving objects appropriately.
Solution Approach 2:
The patent extracts and identifies moving objects from the video sequence using optical flow magnitude thresholds and RANSAC plane fitting. Once moving objects are extracted/identified, they are excluded from the SFM reconstruction process or masked out in the loss function. This allows the system to maintain high depth estimation accuracy for static scenes while properly handling moving objects by removing them from the reconstruction constraint.
2Measurement precision
If monocular depth estimation is performed on video frames with camera shaking and moving objects, then depth data is obtained, but large jitter occurs and depth data between adjacent frames becomes inconsistent
Solution Approach 1:
The patent performs preliminary actions by pre-identifying moving objects using optical flow analysis and RANSAC plane fitting before the depth estimation process. By detecting and marking moving object regions in advance, the system can then exclude these regions from the SFM reconstruction and depth loss calculation, preventing the jitter and inconsistency that would otherwise be introduced by moving objects and camera shaking.
Solution Approach 2:
The patent implements a feedback mechanism where the optical flow magnitude and RANSAC plane fitting results continuously identify moving objects, which then feed back into the loss function to dynamically mask out moving regions. This closed-loop approach ensures that depth estimation consistently excludes moving objects across adjacent frames, maintaining depth data consistency and reliability even in the presence of camera shaking and object motion.
3Measurement precision
If existing training methods include moving objects in the loss function, then model training proceeds, but model parameters cannot be optimized and accuracy remains low
Solution Approach 1:
The patent segments the loss function calculation into static region loss and excludes moving object regions. By using optical flow and RANSAC to identify moving objects, the system creates a masked loss function that only computes depth estimation error for static regions. This segmentation allows model parameters to be optimized based on reliable static scene data while ignoring the noisy moving object regions, thereby improving accuracy without requiring complex alternative training approaches.
Data Source
AI summary
An image depth prediction method acquires image frames of containing a dynamic object by a monocular camera, extracts a continuous of object frames and reference frames from the image frames, reconstructs the object frames to obtain reconstructed frames according to the reference frames and a preset depth estimation model, obtains a reconstruction error between the object frames and the reconstructed frames, processes the image frames to obtain point cloud data and instance segmentation data, fuses the point cloud data with the instance segmentation data to obtain mask data, obtains a loss function according to the reconstruction error and the mask data, and trains the depth estimation model based on the loss function until the loss function converges. The method can obtain more accurate depth estimation results for dynamic scenes. An electronic device and a non-transitory storage recording the method are also disclosed.