Image-Based Depth Data for Autonomous Vehicle Localization
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Autonomous vehicles face challenges in accurately detecting and navigating through environments due to limited sensor range and low density of sensor data, which can lead to inaccuracies in localization and object detection.
Innovation Solution
A machine-learning model is trained using image data and lidar data to generate depth data, allowing for the determination of a vehicle's location and the creation of three-dimensional bounding boxes for objects, even in areas with sparse lidar data, by integrating image-based depth estimation techniques.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If sensors are used to capture sensor data for object detection, then object detection capability is enabled, but sensor range is limited and data density is low
Solution Approach 1:
The patent combines multiple sensor types (lidar, radar, ultrasonic sensors, and cameras) into a unified sensor system that captures data across different modalities. This merging allows the system to overcome the limitations of individual sensors by compensating for their respective weaknesses - for example, using radar to supplement lidar when lidar data is sparse, or using camera data to enhance detection when other sensors have limited range.
Solution Approach 2:
The sensor system is designed to perform multiple functions simultaneously - detection, classification, ranging, and tracking - using a multi-functional array of sensors. Each sensor type contributes different capabilities to the overall system, creating a universal detection platform that can handle various detection scenarios with a single integrated system rather than requiring separate specialized systems.
2Measurement precision
If multiple sensors are deployed to improve data density, then measurement precision improves, but device complexity increases
Solution Approach 1:
The patent introduces a computing device that acts as an intermediary between the multiple sensors and the control system. This computing device receives, processes, and fuses data from all sensor types, managing the complexity of coordinating multiple sensors. By centralizing the data fusion and processing functions in a dedicated computing device, the system avoids the complexity of distributed processing across multiple independent units.
3Area of stationary object
If sensor range is extended to cover more environment, then detection coverage improves, but data density decreases
Solution Approach 1:
The patent applies local quality by using different sensor types with different effective ranges and data densities for different spatial zones. Near-field objects are detected using high-density sensors like lidar and ultrasonic sensors, while far-field objects are detected using long-range sensors like radar. This creates a layered detection architecture where each sensor type operates optimally in its appropriate range zone, maintaining high data density locally while extending overall coverage area.
Data Source
AI summary
A vehicle can use an image sensor to both detect objects and determine depth data associated with the environment the vehicle is traversing. The vehicle can capture image data and lidar data using the various sensors. The image data can be provided to a machine-learned model trained to output depth data of an environment. Such models may be trained, for example, by using lidar data and/or three-dimensional map data associated with a region in which training images and/or lidar data were captured as ground truth data. The autonomous vehicle can further process the depth data and generate additional data including localization data, three-dimensional bounding boxes, and relative depth data and use the depth data and/or the additional data to autonomously traverse the environment, provide calibration/validation for vehicle sensors, and the like.