Monocular Camera Velocity Estimation via Ground Plane Depth
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing systems for estimating the velocity of an autonomous vehicle and state information of surrounding vehicles face challenges with monocular cameras, as they lack distance information and are less reliable compared to multi-sensor systems like LiDAR or stereo cameras, which are costly and require precise calibration.
Innovation Solution
A system that uses a monocular camera to estimate velocity and state information by performing ground plane estimation, determining a 3D bounding box, and calculating velocity based on immovable objects, employing neural networks for depth prediction and 3D bounding box detection, and correcting ground plane coefficients in real-time using RANSAC algorithms.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If LiDAR or multiple sensors are used to estimate velocity and state information, then measurement precision and reliability are improved, but device complexity and cost increase
Solution Approach 1:
The patent extracts and utilizes immovable objects (such as road markings, signs, or natural features) from the environment as reference points for velocity estimation. By taking out these naturally available reference objects and using them for ego-motion compensation, the system achieves accurate velocity estimation without requiring additional active sensors like LiDAR or radar, thus reducing device complexity while maintaining measurement precision
Solution Approach 2:
The patent creates a virtual 3D model of the environment by projecting 2D image data onto a ground plane and constructing 3D bounding boxes of surrounding vehicles. This virtual copying of the physical environment allows the system to estimate velocity and state information from visual data alone, replacing the need for expensive multi-sensor setups while maintaining estimation accuracy
2Measurement precision
If LiDAR or stereo cameras are used to estimate 3D features, then measurement precision is improved, but cost and calibration complexity increase
Solution Approach 1:
The patent replaces the mechanical/optical calibration requirements of stereo cameras and LiDAR with a computational approach using monocular vision. By substituting the need for precise physical calibration with algorithmic ground plane estimation and 3D bounding box construction from single-camera 2D data, the system achieves 3D feature estimation without the manufacturing and calibration complexity of multi-sensor systems
Solution Approach 2:
The patent transforms the problem from estimating absolute 3D positions (which requires calibrated multi-sensor systems) to estimating relative 3D positions on a ground plane. By changing the reference frame and using pixel-level depth information combined with ground plane geometry, the system derives 3D features from monocular data, eliminating the need for complex calibration while maintaining precision
3Device complexity
If monocular camera is used to reduce cost and simplify the system, then device complexity and cost are reduced, but measurement precision and reliability deteriorate
Solution Approach 1:
The patent introduces ground plane estimation as an intermediary step between 2D monocular image data and 3D velocity estimation. By projecting 2D pixel coordinates onto a 3D ground plane and using the geometric relationships, the system recovers depth information that would normally require active sensors. This intermediary transformation allows the monocular camera to achieve measurement precision comparable to multi-sensor systems
Solution Approach 2:
The patent transitions from 2D image space to 3D physical space by constructing a ground plane model and projecting 2D features onto it. By adding the depth dimension through computational geometry rather than through additional physical sensors, the system enables a monocular camera to provide accurate distance and velocity information, resolving the precision limitation while maintaining system simplicity
Data Source
AI summary
Systems and methods for estimating velocity of an autonomous vehicle and state information of a surrounding vehicle are provided. In some aspects, the system includes a memory that stores instructions for executing processes for estimating velocity of an autonomous vehicle and state information of the surrounding vehicle and a processor configured to execute the instructions. In various aspects, the processes include: receiving image data from an image capturing device; performing a ground plane estimation by predicting a depth of points on a road surface based on an estimated pixel-level depth; determining a three-dimensional (3D) bounding box of the surrounding vehicle; determining the state information of the surrounding vehicle based on the ground plane estimation and the 3D bounding box; and determining the velocity of the autonomous vehicle based on an immovable object relative to the autonomous vehicle. In some aspects, an operation of the autonomous vehicle may be controlled based on at least one of the state information or the velocity of the autonomous vehicles.