Radar-Lidar Neural Network Training for All-Weather Object Detection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Radar sensors provide lower data resolution compared to lidar sensors, while lidar sensors' accuracy is inhibited by adverse weather conditions, affecting the accuracy of localization routines in autonomous vehicles.
Innovation Solution
A method and system for training a student neural network using radar and lidar data, involving intensity map generation, augmentation routines, and weight updates based on loss values and moving averages to enhance object detection accuracy.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If radar sensors are used for object detection, then the system can operate in adverse weather conditions, but the data resolution is lower compared to lidar sensors
Solution Approach 1:
The patent combines radar and lidar sensors into a unified sensor system that processes both data types simultaneously. The neural network is trained to fuse radar data (which works in adverse weather) with lidar data (which provides high resolution), allowing the system to maintain high object detection reliability in adverse weather while preserving measurement precision through multi-sensor data fusion
2Measurement precision
If lidar sensors are used for object detection, then higher data resolution is achieved, but accuracy is inhibited by adverse weather conditions
Solution Approach 1:
The patent introduces radar sensors as an intermediary data source that compensates for lidar's weakness in adverse weather. The neural network learns to weight and fuse radar and lidar data dynamically, using radar as a reliable mediator when weather conditions degrade lidar performance, thereby maintaining both high resolution and reliability across all weather conditions
3Adaptability or versatility
If a neural network is trained using only radar data, then the training can be performed in various weather conditions, but the detection accuracy is limited due to lower data resolution
Solution Approach 1:
The patent performs preliminary action by training the neural network in advance using a diverse dataset that includes both radar and lidar data from multiple weather conditions. This pre-training with augmented data (including simulated adverse weather scenarios) prepares the network to effectively fuse sensor data and maintain high detection accuracy when deployed in various real-world weather conditions
Data Source
Figure 1
Figure 2
Figure 3
AI summary
A method includes generating a radar-based intensity map and a lidar-based intensity map and performing one or more augmentation routines on the radar-based intensity map and the lidar-based intensity map to generate a radar input and a lidar input. The method includes generating a plurality of teacher-based bounding boxes and a plurality of student-based bounding boxes based on the radar input and the lidar input. The method includes determining a loss value of the plurality of student-based bounding boxes based on the plurality of teacher-based bounding boxes and a plurality of ground truth bounding boxes, updating one or more weights of the student neural network (80) based on the loss value, and updating one or more weights of the teacher neural network (70) based on a moving average associated with the one or more weights of the student neural network (80).