Neural Network 3D Bounding Box for Single Camera 6DoF Pose
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current object detection systems require multiple and costly sensors like stereo cameras, lidar, and radar, which are computationally expensive and inefficient in determining six degree-of-freedom (6DoF) poses of objects, especially in vehicle guidance, robot operation, and security applications.
Innovation Solution
A single camera system using deep neural networks processes image data to determine 6DoF poses of objects by outputting a 3D bounding box and transforming pixel coordinates into real-world coordinates, reducing the need for multiple sensors and computational resources.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If multiple sensors (stereo cameras, lidar, radar) are used to determine 6DoF poses, then measurement precision is improved, but device complexity and cost increase
Solution Approach 1:
The patent combines multiple sensor types (stereo cameras, lidar, radar) into an integrated multi-sensor system that shares common processing infrastructure and coordinate transformation frameworks, reducing overall system complexity while maintaining 6DoF pose determination accuracy
Solution Approach 2:
The patent creates a universal processing framework that can handle data from multiple sensor types using common algorithms for 3D bounding box generation and coordinate transformations, allowing a single system to perform multiple sensing functions efficiently
2Measurement precision
If multiple sensors are used to determine 6DoF poses, then measurement precision is improved, but computational cost increases
Solution Approach 1:
The patent uses 2D bounding boxes from single-camera views as simplified copies or projections of the full 3D bounding box, allowing the system to work with lower-dimensional data representations that require less computational energy while still enabling accurate 6DoF pose determination through coordinate transformations
3Device complexity
If a single camera is used to determine 6DoF poses, then device complexity is reduced, but measurement precision deteriorates
Solution Approach 1:
The patent transforms 2D image data from a single camera into 3D spatial information by generating 3D bounding boxes and applying coordinate transformations between pixel coordinates and real-world coordinates, effectively adding dimensional information through mathematical transformations rather than additional physical sensors
4Measurement precision
If multiple sensors are used to determine 6DoF poses, then measurement precision is improved, but loss of time increases
Solution Approach 1:
The patent performs preliminary processing by generating 3D bounding boxes and establishing coordinate transformation relationships in advance, so that when 6DoF pose determination is needed, the system can quickly apply pre-computed transformations rather than processing raw sensor data from multiple sources in real-time
Data Source
AI summary
A camera is positioned to obtain an image of an object. The image is input to a neural network that outputs a three-dimensional (3D) bounding box for the object relative to a pixel coordinate system and object parameters. Then a center of a bottom face of the 3D bounding box is determined in pixel coordinates. The bottom face of the 3D bounding box is located in a ground plane in the image. Based on calibration parameters for the camera that transform pixel coordinates into real-world coordinates, a) a distance from the center of the bottom face of the 3D bounding box to the camera relative to a real-world coordinate system and b) an angle between a line extending from the camera to the center of the bottom face of the 3D bounding box and an optical axis of the camera are determined. The calibration parameters include a camera height relative to the ground plane, a camera focal distance, and a camera tilt relative to the ground plane. A six degree-of-freedom (6DoF) pose for the object is determined based on the object parameters, the distance, and the angle.