Vehicle Depth Mapping for LiDAR Blind Spot Recognition
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing neural network models for vehicle autonomous driving struggle with recognizing external objects in LiDAR blind spots due to the lack of a comprehensive database, leading to suboptimal recognition performance.
Innovation Solution
A vehicle control apparatus that utilizes a LiDAR device and a camera to generate virtual points in LiDAR blind spots, creating a dense depth map by synthesizing sparse depth maps from both systems, and trains a neural network model using supervised learning with the dense depth map as ground truth to improve object distance prediction.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If a neural network model is trained using only LiDAR data, then the model can process LiDAR input efficiently, but the model fails to recognize external objects in LiDAR blind spots due to incomplete coverage
Solution Approach 1:
The patent combines LiDAR point cloud data with camera image data to create a fused depth map. The LiDAR provides accurate depth information where available, while the camera fills in the blind spot regions through monocular depth estimation, creating a comprehensive depth representation that covers the entire field of view.
Solution Approach 2:
The patent introduces an intermediary process that generates virtual depth information for LiDAR blind spots using camera images and a monocular depth estimation network. This intermediary depth map acts as a bridge to fill the information gap in regions where LiDAR cannot directly measure depth.
2Productivity
If a neural network model is trained with unrealistic datasets, then the training process is simpler and faster, but the recognition performance deteriorates due to lack of real-world blind spot data
Solution Approach 1:
The patent performs preliminary action by pre-processing LiDAR and camera data to generate fused depth maps with virtual points for blind spots before training the neural network. This preparation creates a comprehensive dataset that includes realistic blind spot scenarios, enabling the model to learn accurate distance prediction without requiring extensive manual annotation of real-world blind spot data.
Solution Approach 2:
The patent creates virtual depth points that copy the structure and characteristics of real LiDAR points but are generated algorithmically for blind spot regions. These virtual points replicate the appearance and distribution patterns of actual depth data, providing realistic training examples without requiring physical measurement of every possible blind spot scenario.
3Area of stationary object
If sparse depth maps from LiDAR and camera are used separately, then each sensor operates independently, but the overall depth coverage remains incomplete due to blind spots
Solution Approach 1:
The patent merges the sparse depth map from LiDAR with the depth map generated from camera images through monocular depth estimation. The fusion process combines the reliable LiDAR depth measurements with the estimated camera depth, creating a complete depth map that covers both LiDAR-visible and blind spot regions.
Solution Approach 2:
The patent transitions from relying solely on direct LiDAR measurement to incorporating monocular depth estimation from camera images. This adds another dimension of depth information acquisition, using visual cues from the camera to infer depth in regions where LiDAR cannot directly measure, thereby completing the depth coverage.
Applied Scientific Principles
This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.
Function Achieved in This Case
Enhances the accuracy of predicting distances between vehicles and external objects by effectively training the neural network model, addressing blind spots and improving overall recognition performance.
Implementation Method 1
a light detection and ranging (LiDAR) device
Implementation Method 2
obtain, via the camera, an image
Data Source
AI summary
The present disclosure relates to a vehicle control apparatus and a method thereof. The vehicle control apparatus may include a memory in which neural network models are stored, a light detection and ranging (LiDAR) device, a camera, and a processor. The processor may obtain a first point cloud, generate a first sparse depth map based on the first point cloud, generate segmentation information by classifying a type of at least one pixel included in an image, generate a second point cloud by forming virtual points corresponding to a ground included in a blind spot of the LiDAR; generate a second sparse depth map and a third sparse depth map, generate a first dense depth map and a second dense depth map, train a second neural network model based on the second dense depth map, and control a vehicle based on the second dense depth map.