6DoF Pose Estimation in Fisheye Images Using 3D Bounding Boxes
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Fisheye cameras introduce convex distortion in images, making it challenging to extract quantitative data from traffic scenes, which complicates the determination of object locations and poses necessary for autonomous vehicle navigation and other applications.
Innovation Solution
A method involving a deep neural network and geometric approaches to determine a six degree-of-freedom pose of objects in fisheye images by segmenting the image, projecting three-dimensional models onto a roadway plane, and using a convolutional neural network to refine bounding box calculations, thereby overcoming distortion issues.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Area of stationary object
If fisheye lens is used to acquire panoramic images, then field of view is improved, but image distortion increases
Solution Approach 1:
The patent transforms the 2D fisheye image coordinates into 3D world coordinates by introducing a vertical dimension (Z-axis) and using camera height and pitch angle information. This dimensional transformation allows the system to compensate for fisheye distortion by mapping distorted 2D coordinates to undistorted 3D spatial positions, thereby resolving the contradiction between wide field of view and measurement precision.
2Measurement precision
If deep neural network is used to determine 6DoF pose, then pose estimation accuracy is improved, but computational resources increase
Solution Approach 1:
The patent segments the pose estimation task into multiple independent components: 2D bounding box detection, 3D bounding box estimation, and 6DoF pose calculation. Each segment can be processed separately using appropriate methods (geometric approaches for 3D bounding box, DNN for 6DoF refinement), allowing the system to achieve high accuracy while optimizing computational resource usage by applying heavy computation only where necessary.
Solution Approach 2:
The patent performs preliminary geometric calculations to estimate 3D bounding box and initial pose parameters before applying the deep neural network. This preliminary action provides the DNN with pre-processed features and initial guesses, significantly reducing the computational burden and training time required for the neural network while maintaining high pose estimation accuracy.
3Productivity
If geometric approach is used to determine 3D bounding box, then computational efficiency is improved, but pose accuracy for obscured objects decreases
Solution Approach 1:
The patent merges the geometric approach and deep neural network approach into a hybrid system. The geometric approach is used to efficiently estimate the 3D bounding box and initial pose parameters, while the DNN is applied to refine these estimates and handle obscured or partially visible objects. This combination allows the system to achieve both computational efficiency and high pose accuracy by leveraging the strengths of both methods.
Data Source
AI summary
A two-dimensional image segment that includes an outline of an object can be determined in a top-down fisheye image. A six degree of freedom (DoF) pose for the object can be determined based on determining a three-dimensional bounding box determined by one or more of (1) an axis of the two-dimensional image segment in a ground plane included in the top-down fisheye image and a three-dimensional model of the object and (2) inputting the two-dimensional image segment to a deep neural network trained to determine a three-dimensional bounding box for the object.