Pseudo-3D Box Regression for 2D Bounding Box Depth Estimation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional methods for estimating 3D bounding boxes from 2D images are computationally intensive, require exhaustive searches, and suffer from accuracy issues due to varying boundary conditions and single template limitations, especially when handling symmetric objects.
Innovation Solution
A method using a pseudo-3D box regression approach with a convolutional neural network (CNN) to simplify the 3D bounding box construction by leveraging 2D coordinates of vertices, reducing computational resources and improving accuracy through joint optimization of orientation classification and vertex regression.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If conventional methods (sliding windows, anchor boxes with exhaustive search) are used to construct 3D bounding boxes, then detection coverage is achieved, but computational time and resources increase significantly
Solution Approach 1:
The patent segments the 3D bounding box construction into two independent parts: (1) detecting 2D bounding boxes using conventional methods, and (2) generating pseudo-3D boxes by extending 2D vertices along the depth axis. This segmentation eliminates the need for exhaustive 3D template matching while preserving detection coverage.
Solution Approach 2:
The patent transitions from 2D to 3D by adding a depth dimension to 2D bounding box vertices. By defining pseudo-3D boxes through simple depth extension rather than full 3D object detection, the method reduces computational complexity from exhaustive 3D search to efficient 2D detection plus dimensional extension.
2Device complexity
If a single template is used for determining 3D bounding box surfaces, then device complexity is reduced, but manufacturing precision (detection accuracy) deteriorates due to varying boundary conditions
Solution Approach 1:
The patent introduces dynamic adaptation by classifying objects into categories (vehicle, pedestrian, cyclist) and applying category-specific depth extension parameters. This dynamic adjustment of depth parameters based on object type improves detection accuracy while maintaining a single template framework.
Solution Approach 2:
The patent changes the depth parameter dynamically based on object category. Different object types receive different depth extension values, allowing the system to adapt to varying boundary conditions without requiring multiple complex templates, thus improving precision while controlling complexity.
3Reliability
If exhaustive search methods are used to match 3D templates, then detection reliability improves, but productivity decreases due to redundant computation
Solution Approach 1:
The patent extracts only the essential 2D bounding box information from images and generates pseudo-3D representations by adding depth parameters. This extraction approach eliminates the need for exhaustive 3D template matching while retaining sufficient information for reliable detection, thereby improving productivity.
Solution Approach 2:
The patent performs preliminary 2D object detection and classification before generating pseudo-3D boxes. By pre-processing to identify object categories and obtain accurate 2D bounding boxes, the system prepares optimal input for depth extension, reducing subsequent computational requirements and improving overall processing speed.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
A method for acquiring a pseudo-3D box from a 2D bounding box in a training image is provided. The method includes steps of: (a) a computing device acquiring the training image including an object bounded by the 2D bounding box; (b) the computing device performing (i) a process of classifying a pseudo-3D orientation of the object, by referring to information on probabilities corresponding to respective patterns of pseudo-3D orientation and (ii) a process of acquiring 2D coordinates of vertices of the pseudo-3D box by using regression analysis; and (c) the computing device adjusting parameters thereof by backpropagating loss information determined by referring to at least one of (i) differences between the acquired 2D coordinates of the vertices of the pseudo-3D box and 2D coordinates of ground truth corresponding to the pseudo-3D box, and (ii) differences between the classified pseudo-3D orientation and ground truth corresponding to the pseudo-3D orientation.