3D Object Detection Using Weighted Channel Features
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current three-dimensional object detection methods in self-driving systems face challenges due to computationally intensive 3D convolution operations and sparse point clouds, leading to inaccurate segmentation and interference points that affect the estimation of 3D boxes.
Innovation Solution
A method and system that extract targets from two-dimensional images using a pre-trained deep convolutional neural network, determine point cloud frustums, segment the point clouds using a point cloud segmentation network, and estimate 3D box parameters with weighted channel features to reduce interference and enhance key point weights.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If three-dimensional convolution operation is used to extract features from point cloud, then feature extraction capability is improved, but computational complexity increases significantly
Solution Approach 1:
The point cloud processing is segmented into multiple stages: first projecting 3D points to 2D image space, then using 2D convolution on the projected points, and finally mapping back to 3D space. This segmentation replaces direct 3D convolution with a combination of projection and 2D convolution operations, reducing computational complexity while maintaining feature extraction capability.
Solution Approach 2:
The patent replaces the mechanical 3D convolution operation with an alternative computational approach involving projection to 2D space, 2D convolution, and back-projection. This substitution uses geometric transformation and 2D processing to achieve what would otherwise require computationally intensive 3D convolution.
2Measurement precision
If point cloud segmentation is performed to improve detection accuracy, then object detection precision is improved, but interference points still affect the final 3D box estimation
Solution Approach 1:
The patent applies local quality by differentiating between foreground points (belonging to target objects) and background points (interference points) within the point cloud. By assigning different weights and processing differentials to these local regions, the method enhances the quality of relevant features while suppressing interference, improving both segmentation accuracy and final 3D box estimation.
Solution Approach 2:
The method performs preliminary point cloud segmentation and foreground-background differentiation before final 3D box estimation. By pre-identifying and separating interference points from target object points, the system prepares a cleaner, more accurate point cloud for subsequent 3D parameter estimation, reducing the impact of interference on final results.
3Productivity
If standard point cloud processing is used, then processing speed is maintained, but detection accuracy decreases due to sparse point clouds
Solution Approach 1:
The patent introduces a 2D image projection as an intermediary step between 3D point cloud data and feature extraction. By projecting sparse 3D points onto a 2D image plane, the method creates a denser representation that preserves spatial relationships while enabling more effective feature extraction, thereby improving detection accuracy without sacrificing processing efficiency.
Data Source
AI summary
A three-dimensional object detection method includes: extracting a target in a two-dimensional image by a pre-trained deep convolutional neural network to obtain a plurality of target objects; determining a point cloud frustum in a corresponding three-dimensional point cloud space based on each target object; segmenting the point cloud in the frustum based on a point cloud segmentation network to obtain a point cloud of interest; and estimating parameters of a 3D box in the point cloud of interest based on a network with the weighted channel features to obtain the parameters of the 3D box for three-dimensional object detection. According to the present invention, the features of the image can be learned more accurately by the deep convolutional neural network and the parameters of the 3D box in the point cloud of interest are estimated based on the network with the weighted channel features.