LiDAR-Camera Depth Mapping for Moving Region and Occlusion Detection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing technologies face challenges in accurately detecting moving bodies and generating high-density point clouds and depth images, particularly with nonrigid objects and occlusions, using LiDAR and camera data, leading to inefficiencies in existing systems.
Innovation Solution
An information processing device and method that projects LiDAR point clouds onto a camera image plane, forms depth images using optical flow, and compares these images to detect moving and non-moving body regions, generating high-density point clouds and depth images by merging LiDAR data on a common coordinate system and removing occlusions.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If LiDAR point clouds are used for high-precision observation, then measurement precision is improved, but device complexity increases and high-density observation capability deteriorates
Solution Approach 1:
The patent merges LiDAR point cloud data with camera image data to create a fused depth map. The LiDAR provides high-precision depth measurements while the camera provides high-density pixel coverage. By combining these two data sources, the system achieves both high precision and high density without requiring a more complex LiDAR system.
Solution Approach 2:
The patent uses camera images as an intermediary to transfer LiDAR depth information to regions where LiDAR points are sparse. The camera image serves as a mediator that allows high-precision LiDAR data to be propagated to areas lacking direct LiDAR measurements, achieving high-density observation without increasing LiDAR complexity.
2Productivity
If camera images are used for high-density observation, then productivity is improved, but measurement precision deteriorates
Solution Approach 1:
The patent combines camera image data with LiDAR point cloud data to create a fused depth map. The camera provides high-density pixel coverage while the LiDAR provides high-precision depth measurements. By merging these data sources, the system achieves both high productivity (density) and high measurement precision.
Solution Approach 2:
The patent uses optical flow as an intermediary to transfer depth information from LiDAR-measured regions to camera regions lacking direct depth data. The optical flow field serves as a mediator that propagates precise depth information across the high-density camera image grid, maintaining precision while achieving high density.
3Ease of operation
If semantic segmentation is used for moving body detection, then ease of operation is improved, but measurement precision deteriorates due to labeling range limitations
Solution Approach 1:
The patent uses depth comparison as an intermediary method to detect moving bodies without relying on semantic segmentation labels. By comparing depth maps from different time points, the system can identify moving objects regardless of whether they fall within predefined labeling categories, improving recognition accuracy while maintaining operational simplicity.
4Productivity
If cluster analysis is used for LiDAR point cloud processing, then productivity is improved, but measurement precision deteriorates in sparse LiDAR or small target scenarios
Solution Approach 1:
The patent uses camera images as an intermediary to supplement sparse LiDAR data. The camera provides additional spatial information that compensates for the sparsity of LiDAR points, enabling accurate moving body detection even when LiDAR cluster analysis would fail due to insufficient points.
Applied Scientific Principles
This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.
Function Achieved in This Case
Enables accurate detection of moving and non-moving body regions without labeling, constructing high-density three-dimensional environments, and forming high-density depth images by removing occlusions, improving environmental recognition for advanced driver-assistance systems and automated driving.
Implementation Method 1
LiDAR (Light Detection And Ranging)
Implementation Method 2
LiDAR point cloud
Implementation Method 3
forming a second depth image using a camera image according to an optical flow
Data Source
AI summary
Detection of a moving body region, generation of a high-density point cloud, and the like can be achieved in a preferable manner.A processing unit performs a process that forms a first depth image by projecting a LiDAR point cloud on a camera image plane, a process that forms a second depth image by using a camera image according to an optical flow, and a process that compares the first depth image and the second depth image to detect a moving body region or a non-moving body region. For example, the processing unit further performs a process that generates a high-density point cloud by projecting LiDAR point clouds corresponding to non-moving body regions of a plurality of frames on an identical coordinate system and sequentially merging the LiDAR point clouds.