Target Detection via Image and Point Cloud Fusion
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional 3D target detection methods in self-driving and robot navigation scenarios face challenges in accurately detecting targets outside the field of view or under adverse environmental conditions such as dark nights, insufficient illumination, or rapid movement, due to their reliance on image sensors and environmental factors.
Innovation Solution
A method that combines image feature analysis and voxel-based analysis of three-dimensional point clouds from laser sensors, projecting point clouds into bird's eye and perspective views, performing voxelization, and fusing voxel features to obtain comprehensive 2D detection boxes and construct 3D models of targets, thereby enhancing detection accuracy.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If image sensor-based 2D detection is used, then detection process is simple, but detection accuracy deteriorates for targets outside field of view or under adverse environmental conditions
Solution Approach 1:
The patent merges image sensor-based 2D detection with laser sensor-based 3D point cloud detection. The system obtains both 2D detection boxes from image features and 3D detection boxes from point cloud data, then fuses these results through association analysis to produce final detection outcomes. This combination allows the system to leverage the simplicity of image processing while compensating for its limitations using 3D spatial information.
Solution Approach 2:
The patent transitions from two-dimensional image detection to three-dimensional point cloud detection. By processing 3D point cloud data alongside 2D image data, the system adds a spatial dimension that enables detection of targets outside the image sensor's field of view and improves accuracy under adverse environmental conditions where image-based methods fail.
2Device complexity
If image sensor is used for target detection, then device complexity is low, but reliability deteriorates under dark night, insufficient illumination, or rapid movement conditions
Solution Approach 1:
The system combines image sensor data with laser sensor data to create a hybrid detection approach. The laser sensor provides reliable 3D spatial information that is independent of lighting conditions, while the image sensor provides contextual information. This merging maintains relatively low device complexity while significantly improving detection reliability under adverse environmental conditions.
Solution Approach 2:
The patent introduces 3D point cloud data as an intermediary that bridges the gap between image sensor limitations and detection requirements. The point cloud serves as a mediator that provides accurate spatial information regardless of lighting conditions, enabling reliable detection in dark nights or insufficient illumination scenarios where image-based methods alone would fail.
3Productivity
If 2D detection box is obtained from image feature, then processing speed is fast, but detection completeness deteriorates for targets outside field of view
Solution Approach 1:
The patent merges 2D detection results from image features with 3D detection results from point cloud data. The system obtains 2D detection boxes quickly from image processing, then uses 3D point cloud data to complement and expand the detection results. This merging maintains high processing speed while improving detection completeness by adding targets that would be missed by 2D image analysis alone.
Solution Approach 2:
The patent extends detection from two dimensions to three dimensions by incorporating 3D point cloud data. This dimensional expansion allows the system to detect targets outside the image sensor's field of view while maintaining the speed benefits of 2D processing. The 3D spatial information fills in gaps and completes detection for targets that 2D image analysis alone would miss.
Data Source
AI summary
A target detection method and apparatus are provided. A first image of a target scenario collected by an image sensor is analyzed to obtain one or more first 2D detection boxes of the target scenario, and a three-dimensional point cloud of the target scenario collected by a laser sensor is analyzed to obtain one or more second 2D detection boxes of the target scenario in one or more views (for example, a BEV and/or a PV). Then, comprehensive analysis is performed on a matching degree and confidence of the one or more first 2D detection boxes, and a matching degree and confidence of the one or more second 2D detection boxes, to obtain a 2D detection box of a target. Finally, a 3D model of the target is obtained based on a three-dimensional point corresponding to the 2D detection box of the target.