Monocular 3D Object Detection With Future Position Prediction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current sensing systems face challenges in accurately predicting the future locations of objects in environments using monocular RGB cameras, as they require expensive sensors and significant computing resources, especially when compared to stereo RGB cameras or lidar sensors.
Innovation Solution
A deep neural network (DNN) is trained to process images from a monocular RGB camera, determining three-dimensional object data by using a Gaussian heatmap to estimate object locations and predict future positions, which reduces the need for expensive sensors and computing resources, and allows for vehicle path determination based on object tracking.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of manufacture
If monocular RGB cameras are used for object detection, then cost is reduced, but measurement precision deteriorates
Solution Approach 1:
The patent introduces an intermediary computational process (deep neural network with Gaussian heatmap generation and 3D coordinate transformation) that mediates between the limited data from monocular RGB cameras and the requirement for precise 3D object detection, enabling accurate measurements without expensive stereo cameras or LiDAR sensors
Solution Approach 2:
The patent replaces mechanical/optical systems (stereo RGB cameras, LiDAR sensors) with a computational system (deep neural network processing monocular RGB images), substituting physical measurement mechanisms with algorithmic processing to achieve 3D object detection at lower cost
2Device complexity
If monocular RGB cameras are used for object detection, then device complexity is reduced, but measurement precision deteriorates
Solution Approach 1:
The patent replaces complex mechanical/optical systems (stereo cameras requiring synchronized capture, LiDAR requiring precise timing and calibration) with a simpler monocular camera system augmented by computational algorithms, reducing device complexity while maintaining measurement precision through software processing
3Productivity
If deep neural network processing is used, then productivity is improved, but use of energy worsens
Solution Approach 1:
The patent performs preliminary actions by pre-training the deep neural network offline and preparing Gaussian heatmap templates in advance, so that during real-time operation, the system only needs to perform forward propagation on new images, significantly reducing the computational energy required for each detection while maintaining high productivity
Data Source
AI summary
An image can be input to a deep neural network to determine a point in the image based on a center of a Gaussian heatmap corresponding to an object included in the image. The deep neural network can determine an object descriptor corresponding to the object and include the object descriptor in an object vector attached to the point. The deep neural network can determine object parameters including a three-dimensional location of the object in global coordinates and predicted pixel offsets of the object. The object parameters can be included in the object vector, and the deep neural network can predict a future location of the object in global coordinates based on the point and the object vector.