Multi-Sensor Trajectory Prediction With Radar-LIDAR Map Fusion
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Autonomous vehicles face challenges in accurately predicting trajectories of objects in their environment due to limitations in early fusion of radar, LIDAR, and map data, leading to inefficiencies in object detection and motion planning.
Innovation Solution
A computing system that integrates LIDAR, radar, and map data into a network model to generate fused feature data, which is then used for predicting trajectories, enabling improved object detection and motion planning through a multi-scale fusion backbone and prediction head.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If traditional separate processing of radar and LIDAR data is used, then device complexity is reduced, but measurement precision and trajectory prediction accuracy deteriorate
Solution Approach 1:
The patent combines radar data processing and LIDAR data processing into a unified network model that performs early fusion of both sensor types. The model integrates multiple input channels for radar and LIDAR data, processes them through shared and specialized neural network layers, and produces combined trajectory predictions, thereby improving accuracy through fused multi-modal information.
Solution Approach 2:
The network model is designed as a multi-functional system that simultaneously performs object detection, trajectory prediction, and data fusion for both radar and LIDAR inputs. The unified architecture handles multiple sensor types and multiple prediction tasks within a single model framework, reducing overall system complexity despite the increased processing capabilities.
2Measurement precision
If early fusion of radar and LIDAR data is implemented, then trajectory prediction accuracy improves, but computational time and processing complexity increase
Solution Approach 1:
The network model performs preliminary processing of radar and LIDAR data through dedicated feature extraction layers before fusion. By pre-processing each sensor type's data independently through optimized neural network layers, the model prepares fused features in advance, reducing the computational burden during the final prediction stage and accelerating overall processing.
3Adaptability or versatility
If multi-scale fusion backbone is used, then feature extraction capability improves, but device complexity increases
Solution Approach 1:
The network model segments the feature extraction process into multiple scales through a multi-scale fusion backbone. This involves dividing the processing into parallel branches that operate at different spatial or feature scales, with each branch specialized for extracting specific types of features. The segmented approach enables comprehensive feature extraction while maintaining manageable complexity through modular architecture.
Data Source
AI summary
Systems and methods for trajectory prediction are provided. A method can include obtaining LIDAR data, radar data, and map data; inputting the LIDAR data, the radar data, and the map data into a network model; transforming, by the network model, the radar data into a coordinate frame associated with a most recent radar sweep in the radar data; generating, by the network model, one or more features for each of the LIDAR data, the transformed radar data, and the map data; combining, by the network model, the one or more generated features to generate fused feature data; generating, by the network model, prediction data based at least in part on the fused feature data; and receiving, as an output of the network model, the prediction data. The prediction data can include a respective predicted trajectory for a future time period for one or more detected objects.