Monocular Camera Depth Estimation Using Asynchronous Vehicle Frames
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Monocular camera systems for autonomous vehicles face challenges in accurately estimating the depth and height of objects due to their limited capability in converting 2D images into 3D views compared to stereo vision systems, and introducing additional sensors like LiDAR or radars increases complexity.
Innovation Solution
A monocular camera system comprising a front mono camera and a side mono camera, with controllers that determine an ideal lagged distance and number of delayed frames to perform feature matching and triangulation, optimizing the area of overlap between camera views to estimate 3D coordinates without additional depth sensors.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If additional depth sensors like LiDAR or radars are introduced to improve depth estimation accuracy, then measurement precision is improved, but device complexity increases
Solution Approach 1:
The patent combines two monocular cameras with non-overlapping fields of view into a unified depth estimation system. By merging the data from both cameras and using temporal synchronization with lagged distance compensation, the system achieves depth estimation capability comparable to multi-sensor systems while maintaining the simplicity of monocular camera technology.
Solution Approach 2:
The patent transitions from spatial separation (stereo vision with simultaneous capture) to temporal separation (monocular with lagged distance compensation). By capturing images at different time steps and compensating for vehicle movement, the system creates a virtual baseline similar to stereo vision without requiring spatial separation of cameras.
2Device complexity
If a monocular camera system is used to reduce device complexity, then device complexity is reduced, but measurement precision of depth and height deteriorates
Solution Approach 1:
The system performs preliminary actions by capturing images at multiple time steps before processing. By storing and retrieving previous time step images with appropriate lagging based on detected distance, the system prepares the necessary data for accurate depth estimation before the actual measurement is performed.
Solution Approach 2:
The system uses feedback from detected object distances to dynamically adjust the lagged distance for previous time step images. By continuously monitoring distance and adjusting the temporal offset accordingly, the system maintains accurate depth estimation despite variations in object distance and vehicle speed.
3Measurement precision
If asynchronous camera frames from two mono cameras are used to improve depth estimation, then measurement precision is improved, but difficulty of detecting and measuring increases
Solution Approach 1:
The patent applies local quality by focusing feature matching efforts on regions within the overlapping field of view of both cameras. By identifying and prioritizing matching features in the overlap region, the system reduces the computational burden of comparing entire image frames while maintaining depth estimation accuracy.
Solution Approach 2:
The system performs partial action by using only the necessary portion of captured images - specifically, previous time step images lagged by the detected distance. Rather than processing all available frames, the system selectively uses only the relevant historical data needed for accurate depth calculation.
Applied Scientific Principles
This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.
Function Achieved in This Case
Enables accurate depth estimation of objects using asynchronous camera frames from two mono cameras with non-overlapping views, reducing complexity by eliminating the need for additional sensors like LiDAR or radars.
Implementation Method 1
The controllers perform triangulation of the pixel positions captured by the front mono camera and the side mono camera to determine the three-dimensional coordinates of the object being evaluated
Data Source
AI summary
A monocular camera system for a vehicle includes a front mono camera, a side mono camera, and one or more controllers in electronic communication with the front mono camera and the side mono camera. The one or more controllers execute instructions to determine an ideal lagged distance the vehicle travels between a current time step and a previous time step as two asynchronous camera frames are captured by the front mono camera and the side mono camera, determine a number of delayed frames captured by either the front mono camera or the side mono camera between the current time step and the previous time step based on the ideal lagged distance. The controllers determine a direction of travel of the vehicle that indicates which mono camera is selected to provide a previous camera frame captured at the previous time step.