Multi-Sensor Fusion Target Detection Reducing Computational Load
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing target detection methods in autonomous driving face challenges with high computational resource consumption due to large numbers of anchor points generated by vision cameras and inaccuracies in detecting obstacles of varying sizes, especially when using single-sensor systems like millimeter wave radar.
Innovation Solution
A multi-sensor fusion method combining vision, lidar, and millimeter wave radar data, where calculated millimeter wave radar data is used to generate three-dimensional regions of interest, which are then projected onto feature maps and fused with lidar and camera data to reduce computational burden and improve accuracy.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Area of stationary object
If a large number of anchor points are generated by the vision camera to ensure detection coverage, then the detection range is improved, but the computational resource consumption increases significantly
Solution Approach 1:
The detection space is segmented into multiple regions of interest based on millimeter wave radar detection results. Instead of processing the entire image with uniform anchor points, the system divides the search space into relevant segments where targets are likely to be located, reducing the number of anchor points needed while maintaining detection coverage.
Solution Approach 2:
The millimeter wave radar performs preliminary detection to identify potential target locations and generate regions of interest before the vision camera processes the image. This preliminary action filters the search space, allowing the vision system to focus computational resources only on relevant areas rather than processing the entire image uniformly.
2Device complexity
If fixed-size anchor points are used in the vision camera detection stage, then the device complexity is reduced, but the accuracy of detecting obstacles of different sizes decreases
Solution Approach 1:
The anchor point size and density are made dynamic rather than fixed. The system adjusts anchor point parameters based on the detected obstacle size and distance, allowing larger anchors for distant objects and smaller anchors for nearby objects. This dynamic adaptation improves detection accuracy across different scales without requiring multiple fixed configurations.
Solution Approach 2:
Different regions of the image receive different anchor point configurations based on local characteristics. Areas with detected targets receive denser and appropriately sized anchor points, while empty regions use fewer anchors. This local adaptation allows the system to optimize detection accuracy for each region without uniformly increasing complexity across the entire image.
3Speed
If millimeter wave radar is used to generate candidate regions, then the detection speed is improved, but the accuracy decreases when the radar misses the target
Solution Approach 1:
The system merges the results from millimeter wave radar detection with vision camera detection. The radar provides fast preliminary candidate regions, while the vision system performs detailed verification. By combining both detection results through fusion algorithms, the system maintains the speed advantage of radar while compensating for its reliability limitations through visual confirmation.
Solution Approach 2:
The system implements feedback mechanisms where vision detection results are used to verify and correct radar detections. When radar detects a target, the vision system verifies it; when radar misses a target, the vision system can still detect it and provide feedback to adjust radar parameters. This feedback loop improves reliability while maintaining the overall detection speed through the primary radar-based approach.
Data Source
AI summary
A target detection method based on fusion of vision, lidar and millimeter wave radar comprises: obtaining original data detected by a camera, a millimeter wave radar, and a lidar, and synchronizing the millimeter wave radar, the lidar, and the camera in time and space; performing a calculation on the original data detected by the millimeter wave radar according to a radar protocol; generating a region of interest by using a position, a speed, and a radar reflection area obtained from the calculation; extracting feature maps of a point cloud bird's-eye view and the original data detected by the camera; projecting the region of interest onto the feature maps of the point cloud bird's-eye view and the original data detected by the camera; fusing the feature maps of the point cloud bird's-eye view and the original data detected by the camera, and processing a fused image through a fully connected layer.