Autonomous Distance Detection Using Multi-Sensor Fusion
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional systems for detecting distances to objects or obstacles in autonomous vehicles assume a flat ground plane, leading to inaccurate predictions when the road surface is curved, and rely on unity between training and deployment cameras, causing safety and performance issues.
Innovation Solution
A deep neural network (DNN) is trained using sensor data from LIDAR, RADAR, and image data to accurately predict distances, leveraging depth sensors for ground truth generation and incorporating a camera adaptation algorithm to account for varying camera models, ensuring accurate predictions regardless of road topology and camera parameters.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Device complexity
If conventional systems assume a flat ground plane for distance calculation, then the system complexity is reduced and calculation is simplified, but measurement precision deteriorates when the road surface is curved
Solution Approach 1:
The patent transitions from 2D image data to 3D spatial understanding by introducing depth maps and utilizing the vertical dimension information from LIDAR point clouds. This allows the system to account for curved road surfaces by measuring actual vertical distances and angles, resolving the contradiction between simple flat-plane assumptions and accurate curved surface measurements.
Solution Approach 2:
The patent introduces an intermediary processing system that combines data from multiple sensors (LIDAR, cameras, RADAR) and uses deep neural networks to generate accurate distance measurements. This intermediary layer reconciles the simplicity of conventional calculation methods with the complexity of real-world curved surfaces by learning from multi-sensor data.
2Measurement precision
If a deep neural network is trained with multi-sensor data (LIDAR, RADAR, image data), then measurement precision improves for curved surfaces, but device complexity increases
Solution Approach 1:
The patent creates a universal deep neural network model that processes multiple sensor types (LIDAR, RADAR, image data) through a unified architecture. This multi-functional system handles various sensor inputs and road conditions through a single trained model, achieving high precision while managing complexity through data fusion rather than separate processing systems.
Solution Approach 2:
The patent changes the parameters and representation of input data by transforming LIDAR point clouds into depth maps and using standardized feature representations. This parameter transformation allows the neural network to efficiently process diverse sensor data without proportionally increasing computational complexity.
3Ease of operation
If conventional systems use simple geometry calculations based on 2D images, then ease of operation is improved, but reliability deteriorates on curved road surfaces
Solution Approach 1:
The patent replaces traditional mechanical geometry calculation methods with a data-driven deep learning approach. Instead of relying on explicit geometric formulas that assume flat surfaces, the system uses a trained neural network that has learned distance estimation patterns from multi-sensor data, maintaining operational simplicity while improving reliability on curved surfaces.
4Manufacturing precision
If manual annotation is used for training data preparation, then manufacturing precision of training data is improved, but productivity decreases
Solution Approach 1:
The patent implements a self-service training data generation system where LIDAR and RADAR sensors automatically provide ground truth distance measurements that are directly used to supervise the training of the deep neural network. This eliminates the need for manual annotation while maintaining high training data accuracy, as the sensors themselves generate the reference labels.
Solution Approach 2:
The patent creates accurate copies of real-world sensor data through simulated training environments and uses data augmentation techniques to generate diverse training samples. This copying approach allows the system to train on large volumes of high-quality data without manual intervention, improving both productivity and training data precision.
Applied Scientific Principles
This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.
Function Achieved in This Case
The solution provides robust and accurate distance predictions to objects or obstacles, enhancing safety and performance by reducing manual annotation requirements and adapting to different camera models, thus improving the reliability of autonomous vehicle operations.
Implementation Method 1
a DNN may be trained—using one or more depth sensors, such as LIDAR sensors
Implementation Method 2
RADAR data to aid in ground truth generation
Data Source
AI summary
In various examples, a deep neural network (DNN) is trained to accurately predict, in deployment, distances to objects and obstacles using image data alone. The DNN may be trained with ground truth data that is generated and encoded using sensor data from any number of depth predicting sensors, such as, without limitation, RADAR sensors, LIDAR sensors, and/or SONAR sensors. Camera adaptation algorithms may be used in various embodiments to adapt the DNN for use with image data generated by cameras with varying parameters—such as varying fields of view. In some examples, a post-processing safety bounds operation may be executed on the predictions of the DNN to ensure that the predictions fall within a safety-permissible range.