Camera-Only Obstacle Distance Estimation on Curved Roads
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional systems for detecting distances to objects or obstacles in autonomous vehicles rely on the assumption of a flat ground plane, leading to inaccurate predictions when the road surface is curved, and are also affected by variations in camera parameters during training and deployment, resulting in safety and performance issues.
Innovation Solution
A deep neural network (DNN) is trained using sensor data from LIDAR, RADAR, and other depth sensors to predict distances, leveraging ground truth data for accurate image-based predictions, and a camera adaptation algorithm is used to account for camera model variations, with safety bounds computation to ensure accurate and reliable distance estimation.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Device complexity
If conventional systems assume a flat ground plane for distance calculation, then the system complexity is reduced and calculation is simplified, but measurement precision deteriorates when the road surface is curved
Solution Approach 1:
The patent applies curvature by modeling the ground plane as a curved surface rather than a flat plane. The system estimates road curvature parameters and uses them to adjust distance calculations, accounting for the actual curved geometry of the road surface. This resolves the contradiction by introducing curvature (increasing complexity slightly) to maintain measurement precision on curved roads without requiring full 3D reconstruction.
Solution Approach 2:
The system changes parameters by introducing curvature estimation parameters (such as road grade and curvature radius) to the distance calculation model. Instead of using fixed flat-ground assumptions, the system dynamically adjusts calculation parameters based on estimated road conditions, thereby maintaining precision across varying road geometries while keeping the overall system architecture relatively simple.
2Adaptability or versatility
If deep neural network is trained with image data from cameras with different parameters, then adaptability is improved, but measurement precision deteriorates due to scale variations
Solution Approach 1:
The system performs preliminary action by pre-processing image data to normalize scale variations before feeding them to the neural network. This includes applying camera-specific transformation parameters and scale corrections during data preparation, so that the network receives standardized inputs regardless of the original camera parameters. This maintains precision while allowing training on diverse camera hardware.
Solution Approach 2:
The patent introduces an intermediary processing layer that acts as a mediator between raw image data from different cameras and the neural network. This intermediary applies camera parameter-specific corrections and normalizations, translating various camera outputs into a unified representation that the network can process accurately, thereby decoupling adaptability from precision loss.
3Ease of operation
If conventional systems use simple geometry calculations based on flat ground assumption, then ease of operation is improved, but reliability deteriorates on curved road surfaces
Solution Approach 1:
The system applies dynamics by making the ground plane model adaptive rather than static. Instead of assuming a fixed flat ground, the system dynamically estimates road curvature and adjusts the ground plane model accordingly. This allows the calculation method to automatically adapt to different road conditions, maintaining reliability on curved surfaces while preserving the simplicity of using a ground-plane-based approach rather than full 3D reconstruction.
Applied Scientific Principles
This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.
Function Achieved in This Case
The solution provides robust and accurate distance predictions to objects or obstacles regardless of road topology and camera variations, enhancing the safety and reliability of autonomous vehicle operations by reducing manual annotation and training time.
Implementation Method 1
a deep neural network (DNN) trained with sensor data—such as LIDAR data
Implementation Method 2
a deep neural network (DNN) trained with sensor data—such as LIDAR data, RADAR data
Data Source
AI summary
In various examples, a deep neural network (DNN) is trained to accurately predict, in deployment, distances to objects and obstacles using image data alone. The DNN may be trained with ground truth data that is generated and encoded using sensor data from any number of depth predicting sensors, such as, without limitation, RADAR sensors, LIDAR sensors, and/or SONAR sensors. Camera adaptation algorithms may be used in various embodiments to adapt the DNN for use with image data generated by cameras with varying parameters—such as varying fields of view. In some examples, a post-processing safety bounds operation may be executed on the predictions of the DNN to ensure that the predictions fall within a safety-permissible range.