Monocular Depth Estimation for Dynamic Scene Error Reduction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing monocular depth estimation models struggle with inaccuracies due to uncontrollable factors like camera shaking and object movement, leading to large jitter and inability to accurately filter out moving objects, resulting in low accuracy and suboptimal model parameters.
Innovation Solution
A method involving the use of monocular images and point cloud data to correct and calibrate object frame images, determine reconstruction errors, calculate inertia probabilities, and train a second depth estimation model using a loss function to exclude moving objects, thereby improving depth estimation accuracy.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If SFM principle is used to reconstruct images from depth estimation, then the reconstructed image can be closer to the original object image, but moving objects cannot be correctly reconstructed because they do not conform to SFM's viewpoint pose transformation
Solution Approach 1:
The patent extracts and identifies moving objects from the scene using optical flow analysis and masks, separating them from static objects. This allows the system to apply different processing strategies: SFM-based reconstruction for static objects and optical flow-based tracking for moving objects, thereby resolving the contradiction between reconstruction accuracy and adaptability to moving objects
Solution Approach 2:
The patent introduces dynamic adaptive weighting that adjusts the contribution of different depth estimation methods based on object motion characteristics. For static regions, SFM-based depth is weighted higher; for moving regions, optical flow-based depth is weighted higher. This dynamic adaptation resolves the contradiction by making the system flexible to different object types
2Productivity
If monocular depth estimation is performed on videos with camera shaking and object movement, then depth data can be obtained, but large jitter occurs and depth data of adjacent frames becomes quite different
Solution Approach 1:
The patent employs iterative optimization where depth estimates from adjacent frames are used to refine the current frame's depth estimation. The system uses feedback from optical flow consistency and reprojection errors to adjust depth values, reducing jitter while maintaining productivity in depth data acquisition
Solution Approach 2:
The patent performs preliminary registration and alignment of adjacent frames using feature matching and camera pose estimation before depth estimation. This preliminary action compensates for camera shaking and establishes a consistent coordinate system, thereby reducing jitter in the final depth data while maintaining acquisition efficiency
3Ease of manufacture
If existing technology trains monocular depth estimation model without filtering moving objects, then training can proceed, but the accuracy of model depth estimation is low and model parameters cannot be optimized
Solution Approach 1:
The patent segments the training data into static and moving object regions using optical flow analysis and masks. During training, different loss functions and weighting schemes are applied to different segments: static regions use reconstruction error-based loss while moving regions use optical flow consistency loss. This segmentation allows the model to learn appropriate depth estimation strategies for different object types, improving accuracy while maintaining training feasibility
Solution Approach 2:
The patent applies different training strategies to different spatial regions of the image based on motion characteristics. Static regions are trained with higher emphasis on geometric consistency, while moving regions are trained with emphasis on temporal consistency. This local quality approach enables the model to achieve high accuracy across diverse scene contents without complicating the overall training process
Data Source
AI summary
A method for reducing the error of depth estimation model comprises: obtaining a plurality of monocular images and a point cloud data of each of the plurality of monocular images, wherein each of the plurality of monocular images comprises an object frame image and a reference frame image; reconstructing the object frame image to obtain a reconstructed frame image according to the reference frame image and a first depth estimation model; determining a reconstructed error between the object frame image and the reconstructed frame image; and obtaining an inertia probability of each pixel of the object frame image according to speed information of the point cloud data and pixel information of the object frame image. This application provides more accurate depth estimation results for dynamic scenes. An electronic device and a non-transitory storage recording the method are also disclosed.