Radar Neural Network Obstacle Detection in Cluttered 3D Scenes
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional perception systems for autonomous vehicles rely heavily on cameras and LIDAR, which are unreliable in occluded scenes and inclement weather, and RADAR systems struggle to distinguish between obstacles and background noise, especially in cluttered environments.
Innovation Solution
A deep neural network (DNN) using RADAR data is trained with ground truth labels from LIDAR data to detect moving and stationary obstacles, employing a common trunk with feature extraction and separate heads for class confidence and instance regression, enabling accurate object detection even in challenging conditions.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If conventional RADAR detection techniques are used, then moving reflective objects can be detected, but stationary objects cannot be distinguished from background noise
Solution Approach 1:
The detection system segments objects based on their motion characteristics, separating moving objects detection (using traditional RADAR techniques) from stationary objects detection (using deep learning classification of RADAR point clouds). This allows each segment to be optimized for its specific detection challenge.
Solution Approach 2:
A deep learning classifier acts as an intermediary between raw RADAR point cloud data and object detection results. This intermediary processes the point cloud data to distinguish stationary objects from background noise, enabling accurate detection without requiring additional sensors.
2Reliability
If conventional perception systems use cameras or LIDAR, then obstacle detection is possible, but the systems fail in occluded scenes and inclement weather
Solution Approach 1:
The system changes the operating parameters by using RADAR frequency bands that penetrate occlusions and weather conditions differently than optical sensors. The deep learning model is trained on RADAR-specific features, allowing the system to maintain reliability across varied environmental conditions.
3Measurement precision
If conventional detection techniques are used, then obstacle detection is achieved, but heavy post-processing is required increasing run-time and computational demands
Solution Approach 1:
Traditional mechanical post-processing algorithms are replaced with a deep learning neural network that performs classification and 3D bounding box estimation in an end-to-end manner. This substitution reduces computational overhead by integrating multiple processing steps into a single trained model inference.
4Speed
If traditional RADAR detection techniques are used, then moving objects are detected well, but object classification, dimension, and orientation prediction accuracy is limited
Solution Approach 1:
The system transitions from traditional 2D RADAR imagery to 3D point cloud representation, adding spatial dimensionality. This enables the deep learning model to predict 3D bounding boxes with accurate dimensions and orientations by leveraging the three-dimensional structure of the point cloud data.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
In various examples, a deep neural network(s) (e.g., a convolutional neural network) may be trained to detect moving and stationary obstacles from RADAR data of a three dimensional (3D) space. In some embodiments, ground truth training data for the neural network(s) may be generated from LIDAR data. More specifically, a scene may be observed with RADAR and LIDAR sensors to collect RADAR data and LIDAR data for a particular time slice. The RADAR data may be used for input training data, and the LIDAR data associated with the same or closest time slice as the RADAR data may be annotated with ground truth labels identifying objects to be detected. The LIDAR labels may be propagated to the RADAR data, and LIDAR labels containing less than some threshold number of RADAR detections may be omitted. The (remaining) LIDAR labels may be used to generate ground truth data.