Monocular Video Scale Recovery via Object Height Estimation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Monocular video cameras in autonomous driving systems face challenges in efficiently recovering camera scaling factors without prior knowledge of object class parameters, limiting their application in traffic simulation and car trajectory prediction.
Innovation Solution
A scale recovery system that processes monocular video using machine learning models, such as Convolutional Neural Networks, to estimate camera parameters like height and pitch, and calculates absolute scale without requiring object class information, utilizing image segmentation, lane detection, and object detection techniques.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of manufacture
If monocular video cameras are used for autonomous driving, then cost and ease of operation are improved, but the ability to recover absolute scale without prior knowledge of object class parameters deteriorates
Solution Approach 1:
The system performs preliminary actions by detecting objects and estimating their heights before scale recovery. It uses object detection to identify vehicles, pedestrians, and cyclists, then estimates their heights using machine learning models. These preliminary estimations are then used to calculate the camera's absolute scale, allowing the system to recover scale information without requiring prior knowledge of object class parameters.
Solution Approach 2:
The system serves itself by using the monocular camera's own video feed to recover absolute scale. It processes the video frames to detect objects, estimates their heights, and calculates the camera parameters needed for scale recovery. This self-service approach eliminates the need for external calibration data or prior knowledge of object dimensions, allowing the system to independently recover absolute scale from monocular video.
2Device complexity
If monocular video cameras are used for autonomous driving, then device complexity is reduced, but measurement precision of camera scaling factors deteriorates
Solution Approach 1:
The system changes parameters by using machine learning models to estimate object heights and camera parameters dynamically. Instead of relying on fixed calibration data, the system continuously estimates object heights and uses these estimates to calculate camera scaling factors. This parameter change approach allows the system to maintain measurement precision while using simpler monocular cameras, as the scaling factors are adaptively determined from the video content itself.
3Measurement precision
If object class parameters are required for scale recovery, then measurement precision of scale is improved, but adaptability to different traffic scenarios deteriorates
Solution Approach 1:
The system achieves universality by developing a scale recovery method that works across multiple traffic scenarios without requiring scenario-specific calibration data. The machine learning models detect various objects (vehicles, pedestrians, cyclists) and estimate their heights universally, allowing the system to recover absolute scale in diverse traffic situations. This multi-functional approach enables the system to adapt to different traffic scenarios while maintaining measurement precision, as the same core methodology applies regardless of the specific scenario.
Data Source
AI summary
Methods, apparatus, systems and articles of manufacture are disclosed for scale recovery from monocular video. An example non-transitory computer readable medium comprises instructions that, when executed, cause a machine to at least segment an input image from a monocular video to detect an object in the camera field, estimate camera parameters from the segmented input image, iteratively refine the estimated camera parameters using known object heights, calculate a scale for the video, iteratively refine the scale based on a user input, and report the scaling results for visualization.