Camera-Based Obstacle Distance Estimation on Curved Road Surfaces
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional systems for detecting distances to objects or obstacles in autonomous vehicles rely on the assumption of a flat ground plane, leading to inaccurate predictions when the road surface is curved, and are also affected by variations in camera parameters during training and deployment, resulting in safety and performance issues.
Innovation Solution
A deep neural network (DNN) is trained using sensor data from LIDAR, RADAR, and other depth sensors to predict distances, leveraging these sensors for ground truth generation and incorporating a camera adaptation algorithm to account for camera model variations, ensuring accurate distance estimation regardless of road topology and camera parameters.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Device complexity
If conventional systems assume a flat ground plane for distance calculation, then the system complexity is reduced and calculation is simplified, but measurement precision deteriorates when the road surface is curved
Solution Approach 1:
The patent transitions from 2D image data to 3D point cloud data from LIDAR, adding a depth dimension to accurately represent curved road surfaces. This dimensional change enables the system to capture the actual 3D geometry of the environment, resolving the contradiction between simplified flat-plane assumptions and accurate curved surface measurement.
Solution Approach 2:
The patent changes the fundamental parameter representation from 2D image coordinates to 3D spatial coordinates with depth information. By using LIDAR range data to provide accurate depth values, the system transforms the measurement parameter space to inherently account for road curvature without requiring complex computational corrections.
2Measurement precision
If manual annotation is used for training data preparation, then measurement precision may be improved, but productivity deteriorates due to time-consuming annotation processes
Solution Approach 1:
The system uses LIDAR sensors to automatically provide accurate depth information and ground truth data for training, eliminating the need for manual annotation. The sensors self-generate the training data by capturing real-world scenes with known geometric relationships, thereby improving productivity while maintaining precision through the inherent accuracy of LIDAR measurements.
Solution Approach 2:
The patent replaces the manual mechanical annotation process with automated sensor-based data generation. LIDAR and other depth sensors automatically capture and process environmental data to create training datasets, substituting human labor with automated sensing and computational processing systems.
3Measurement precision
If a single camera model is used during training and deployment, then measurement precision is maintained, but adaptability deteriorates when camera parameters vary
Solution Approach 1:
The patent creates a universal training framework using LIDAR point cloud data that can be paired with images from any camera model. The LIDAR provides model-agnostic depth information that serves as ground truth for training, enabling the system to adapt to different camera parameters and models while maintaining prediction accuracy through the consistent geometric reference provided by LIDAR.
Data Source
AI summary
In various examples, a deep neural network (DNN) is trained to accurately predict, in deployment, distances to objects and obstacles using image data alone. The DNN may be trained with ground truth data that is generated and encoded using sensor data from any number of depth predicting sensors, such as, without limitation, RADAR sensors, LIDAR sensors, and/or SONAR sensors. Camera adaptation algorithms may be used in various embodiments to adapt the DNN for use with image data generated by cameras with varying parameters—such as varying fields of view. In some examples, a post-processing safety bounds operation may be executed on the predictions of the DNN to ensure that the predictions fall within a safety-permissible range.