Multi-View LiDAR Perception Using Motion Cues for Occlusion Detection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional perception methods for autonomous vehicles face limitations in detecting objects and environmental conditions due to limited resolution and accuracy, leading to increased collision risks and safety issues, particularly in complex urban environments with occlusions and weather artifacts.
Innovation Solution
A multi-view LiDAR perception system using a DNN processes multiple 2D views of 3D environments, incorporating motion cues from multiple time slices, and employs a lossless projection technique to preserve multiple returns, enhancing detection accuracy through geometric projection and caching of features.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If conventional DNNs are used for object detection, then the system is simple to implement, but detection accuracy and resolution are limited
Solution Approach 1:
The perception system is divided into multiple independent modules: multi-view projection module, temporal feature extraction module, motion cue generation module, and DNN detection module. Each module processes specific aspects of the data independently, allowing for improved accuracy through specialized processing while maintaining manageable system complexity through modular architecture.
Solution Approach 2:
The system transitions from conventional single-view 2D image processing to multi-view 3D point cloud processing with temporal dimensions. By incorporating multiple viewing angles and time slices, the system creates a higher-dimensional feature space that significantly improves detection accuracy and resolution beyond traditional approaches.
2Measurement precision
If conventional single-view LiDAR processing is used, then processing speed is maintained, but small hazards and occluded objects are missed
Solution Approach 1:
The system merges data from multiple LiDAR views and multiple time slices into a unified feature representation. By combining spatial information from different angles and temporal information from successive frames, the system achieves comprehensive detection of small hazards and occluded objects that would be invisible in single-view processing.
Solution Approach 2:
The system performs preliminary feature extraction and motion cue generation from multiple views and time slices before the main detection process. This preliminary processing prepares enhanced feature representations that make subsequent detection more effective, allowing the system to identify small hazards earlier in the processing pipeline.
3Measurement precision
If sensor data from multiple time slices is processed, then motion cues are provided for improved detection, but computational load and processing time increase
Solution Approach 1:
The system extracts only the essential motion cues and temporal features from the multi-time slice data, rather than processing all raw sensor data. By selectively extracting relevant temporal patterns and motion information, the system achieves improved detection accuracy while minimizing the computational burden of processing multiple time slices.
4Loss of information
If conventional projection techniques that bin 3D points are used, then data is simplified for processing, but multiple returns and important characteristics are lost
Solution Approach 1:
The system implements a nested data structure where multiple LiDAR returns are contained within hierarchical layers of the projection. Each projection layer preserves information from different return levels, creating a nested representation that maintains the full complexity of the original multi-return data while organizing it in a structured format suitable for neural network processing.
Data Source
AI summary
Embodiments of the present disclosure relate to multi-view LIDAR perception with motion cues for autonomous and semi-autonomous machines and applications. A DNN may be used to detect objects, a navigable space, weather or surface conditions, artifacts, and/or other parts or features of an environment based on multiple views of LIDAR data from multiple time slices. The DNN may include multiple input channels for processing multiple views of sensor data from multiple time slices to provide motion cues, and the extracted features from the different time slices may be geometrically projected from a first 2D view to a second 2D view, combined with features that were extracted from the second 2D view, and applied to a subsequent stage of the DNN. The data generated by the DNN may be provided to the drive stack of an autonomous vehicle or other ego-machine to enable safe planning and control of the vehicle.