Pseudo-3D Bounding Box Detection via CNN Instance Segmentation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional methods for detecting 3D bounding boxes in autonomous vehicles are inefficient due to excessive computational resources, low accuracy, and the need for exhaustive searches, especially when dealing with varying 3D orientations of objects.
Innovation Solution
A method using a CNN with instance segmentation to detect pseudo-3D bounding boxes, which simplifies the detection process by utilizing 2D coordinates of vertices and reducing computational requirements, while improving accuracy through a binary mask and regression information generation.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If conventional template-based methods (sliding windows or anchor boxes) are used to detect 3D bounding boxes, then object detection can be performed, but computational resources are excessively consumed and detection accuracy is low
Solution Approach 1:
The patent segments the 3D bounding box detection task into two distinct stages: first detecting 2D bounding boxes in the image plane, then projecting these 2D boxes to generate 3D bounding boxes. This segmentation separates the complex 3D detection problem into simpler 2D detection and projection steps, reducing computational complexity while maintaining accuracy.
Solution Approach 2:
The patent transitions from direct 3D bounding box detection to a two-dimensional approach by first detecting objects in 2D image space and then projecting these 2D detections to 3D space. This dimensionality change simplifies the detection process by leveraging the easier 2D detection task as an intermediate step.
2Reliability
If exhaustive search methods are used to find 3D bounding boxes for objects with six surfaces, then complete coverage is achieved, but computational time increases significantly
Solution Approach 1:
The patent extracts only the necessary 2D bounding box information from the image and uses this extracted data to generate 3D bounding boxes through projection. By taking out only the essential 2D detection results and deriving 3D information from them, the method avoids exhaustive searches while maintaining detection completeness.
Solution Approach 2:
The patent performs preliminary 2D bounding box detection before proceeding to 3D bounding box generation. This preliminary action in 2D space simplifies the subsequent 3D detection task, as the 2D boxes provide a foundation that reduces the search space for 3D bounding boxes.
3Ease of operation
If a single template is used for determining 3D bounding boxes, then the detection process is simplified, but accuracy decreases due to varying boundary conditions with different 3D orientations
Solution Approach 1:
The patent employs dynamic pose estimation to adapt to different 3D orientations of objects. Instead of using a static single template, the system dynamically adjusts the bounding box parameters based on the detected object's orientation and position, allowing accurate detection across varying boundary conditions while maintaining operational simplicity.
Data Source
Figure 1
Figure 2
Figure 3(A)~3(H)
AI summary
A method for detecting a pseudo-3D bounding box based on a CNN capable of converting modes according to poses of detected objects using an instance segmentation is provided to be used for realistic rendering in virtual driving. Shade information of each of surfaces of the pseudo-3D bounding box can be reflected on the learning according to this method. The pseudo-3D bounding box may be obtained through a lidar or a rader, and the surface may be segmented by using a camera. The method includes steps of: a learning device instructing a pooling layer to apply pooling operations to a 2D bounding box region, thereby generating a pooled feature map, and instructing an FC layer to apply neural network operations thereto; instructing a convolutional layer to apply convolution operations to surface regions; and instructing a FC loss layer to generate class losses and regression losses.