Monocular Voxel Flow Estimation for Dynamic Object Velocity
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing computer vision techniques for estimating object velocities in autonomous systems, such as autonomous vehicles, struggle to accurately handle dynamic objects and often result in ghosting and blurring artifacts, particularly when using voxel-based approaches that concatenate voxel grid maps over time without change detection, and require costly depth sensors like LIDAR or radar.
Innovation Solution
The method employs monocular cameras to capture images at two points in time, converts them into voxel grids, warps and estimates voxel flow using ego pose data, clusters the dynamic voxel flow to identify objects, and determines velocity estimates without relying on stereo vision or depth sensors, using a self-supervised learning approach to train networks for 3D voxel motion.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Stability of the object's composition
If voxel-based approaches concatenate voxel grid maps over time without change detection, then temporal consistency is improved, but ghosting and blurring artifacts occur
Solution Approach 1:
The patent extracts and isolates dynamic objects from the scene by detecting changes in voxel grids over time. By separating moving objects from static background, the system eliminates ghosting artifacts that occur when concatenating full voxel grids, while maintaining temporal consistency for both dynamic and static elements independently.
Solution Approach 2:
The patent implements dynamic voxel grid representation where the voxel grid structure adapts to capture moving objects. Instead of using a fixed concatenated voxel grid, the system dynamically updates voxel grids to reflect current scene state, allowing accurate velocity estimation while preventing artifacts from stale object representations.
2Measurement precision
If depth sensors like LIDAR or radar are used for velocity estimation, then measurement precision is improved, but device complexity and cost increase
Solution Approach 1:
The patent replaces mechanical depth sensing systems (LIDAR, radar) with a computational approach using monocular camera images. By converting 2D image data into voxel grid representations and analyzing temporal changes, the system achieves velocity estimation without requiring expensive depth sensors, substituting physical sensing with image processing and geometric reasoning.
Solution Approach 2:
The patent creates a virtual 3D representation (voxel grid) as a copy of the physical scene captured by monocular cameras. This virtual model allows velocity estimation through computational analysis of the copied scene representation, eliminating the need for direct physical depth measurement instruments while preserving the essential spatial and temporal information.
3Device complexity
If monocular cameras are used instead of depth sensors, then device complexity is reduced, but velocity estimation accuracy deteriorates
Solution Approach 1:
The patent transforms 2D monocular camera images into 3D voxel grid representations by introducing a depth dimension through geometric reasoning and temporal analysis. By stacking multiple 2D frames and analyzing pixel displacement over time, the system recovers 3D motion information, enabling accurate velocity estimation from inherently 2D monocular data without requiring depth sensors.
Solution Approach 2:
The patent implements a feedback mechanism where velocity estimates from temporal voxel grid changes are continuously refined. By comparing predicted object positions based on estimated velocity with actual observed positions in subsequent frames, the system adjusts and improves velocity estimates, compensating for the limitations of monocular vision and achieving precision comparable to depth sensor-based methods.
Data Source
AI summary
Estimating a dynamic object velocity includes warping a first voxel grid generated from camera output images of a scene captured at a first time to a third voxel grid representing the scene at a second time; generating a voxel flow from the first time to the second time based at least in part on a second voxel grid generated from camera output images of the scene captured at the second time and the third voxel flow; determining a dynamic voxel flow based at least in part on the voxel flow and an ego motion flow; clustering the dynamic voxel flow to identify one or more object instances; and determining a velocity estimate for a dynamic object of the scene from motion of the one or more object instances in the dynamic voxel flow.