Autonomous Vehicle Depth Map Fusion for Camera View Calibration
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing neural network models for measuring the distance between external objects and vehicles using cameras face significant errors when the camera view is not included in the learning database, necessitating accurate distance measurement in various environments.
Innovation Solution
A vehicle control device and method utilizing a first sensor, a second sensor, and a neural network model to obtain and compare depth maps, perform online calibration, and train the model to reduce differences between depth maps, incorporating intrinsic and extrinsic parameters and distortion coefficients to enhance accuracy.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If a neural network model is used to measure distance between external objects and vehicle using camera, then distance measurement can be performed, but very large error occurs when camera view is not included in the learning database
Solution Approach 1:
The system integrates multiple sensors (camera, LIDAR, radar) to create a universal sensing platform that can handle various camera views and environmental conditions. The depth map fusion technology allows the system to function effectively whether using familiar or unfamiliar camera views by combining multiple sensor data sources.
Solution Approach 2:
The patent introduces depth maps as an intermediary representation that bridges camera images and point cloud data. By converting both camera views and LIDAR data into depth map format, the system can compare and fuse them effectively, reducing errors even when camera views are not in the learning database.
2Measurement precision
If online calibration is performed to realign sensor reference lines, then measurement accuracy is improved, but system complexity increases
Solution Approach 1:
The system performs automatic online calibration without requiring manual intervention or external calibration tools. The calibration process uses the sensor data itself to automatically realign reference lines, making the system self-calibrating and reducing operational complexity despite the sophisticated calibration algorithms involved.
Solution Approach 2:
The calibration process is performed preliminarily before main operations to ensure sensors are properly aligned. By pre-aligning the reference lines of cameras and LIDAR, the system avoids accumulation of alignment errors during operation, maintaining measurement precision without requiring continuous complex calibration.
3Measurement precision
If depth maps from multiple sensors are compared and fused, then three-dimensional object detection precision is improved, but processing time increases
Solution Approach 1:
The system segments the depth map processing into distinct stages: individual sensor depth map generation, depth map comparison, and fused depth map generation. This segmentation allows parallel processing of different sensor data and optimizes the fusion process, reducing overall processing time while maintaining high three-dimensional detection accuracy.
Data Source
AI summary
An apparatus for controlling autonomous driving of a vehicle comprises a first sensor, a second sensor, a memory configured to store a neural network model, and a processor. The processor obtains coordinates of an object from an image acquired by the first sensor, based on intrinsic and extrinsic parameters or a distortion coefficient of the first sensor. It then inputs the image or coordinates into the neural network model to generate a first depth map. A second depth map is obtained based on a cluster of points acquired by the second sensor. By comparing the first and second depth maps, the processor determines any difference between them, outputs a signal indicating this difference, and controls the vehicle's autonomous driving based on the signal.