3D Object Detection via 2D-to-3D Coordinate Transformation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional 2D object detection models in autonomous driving systems lack the ability to provide absolute 3D location and size of objects, which is crucial for accurate motion planning and control, leading to potential safety hazards.
Innovation Solution
A 3D image processing system that includes a deep learning module for learning 2D bounding box coordinates and a fitting module using geological and camera information to calculate 3D attributes such as length, height, width, and orientation of objects, enabling real-time 3D object detection.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If conventional 2D object detection models are used, then the system can detect objects and provide category labels with location information in pixels, but the system cannot provide absolute 3D location and size of objects in meters
Solution Approach 1:
The patent transitions from 2D image plane detection to 3D spatial detection by introducing depth estimation through stereo vision. The system processes images from multiple cameras at different positions and depths, transforming 2D pixel coordinates into 3D spatial coordinates (x, y, z) to provide absolute location and size measurements in meters rather than relative pixel values.
Solution Approach 2:
The patent introduces camera calibration matrices and geometric transformation models as intermediaries between the 2D image data and 3D spatial information. These calibration parameters serve as mediators that enable the conversion from pixel coordinates to real-world 3D coordinates, allowing the system to provide absolute measurements without requiring complex direct 3D sensing hardware.
2Measurement precision
If 3D object detection is implemented using multiple cameras and calibration matrices, then the system can provide accurate 3D attributes, but the computational complexity and processing requirements increase
Solution Approach 1:
The patent performs camera calibration and computes transformation matrices in advance before actual object detection occurs. By pre-calculating the geometric relationships between multiple cameras and establishing the coordinate transformation models beforehand, the system reduces the computational burden during real-time detection, as the complex calibration computations are completed once rather than for each detection task.
3Loss of information
If 2D bounding box coordinates are used for object location, then the system can provide location information quickly, but the system loses exact location and car dimensionality information
Solution Approach 1:
The patent segments the detection process into two stages: first detecting objects in 2D image planes using fast 2D detection algorithms, then separately computing 3D attributes through coordinate transformation. This segmentation allows the system to maintain the speed advantage of 2D detection while adding 3D information computation, as the transformation step is more efficient than performing full 3D detection from scratch.
Data Source
AI summary
A system and method for three-dimensional (3D) object detection is disclosed. A particular embodiment can be configured to: receive image data from a camera associated with a vehicle, the image data representing an image frame; use a machine learning module to determine at least one pixel coordinate of a two-dimensional (2D) bounding box around an object in the image frame; use the machine learning module to determine at least one vertex of a three-dimensional (3D) bounding box around the object; obtain camera calibration information associated with the camera; and determine 3D attributes of the object using the 3D bounding box and the camera calibration information.