LiDAR Projection Image Generation for Sparse Data
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
LiDAR systems face challenges in identifying or detecting objects due to sparse reflection intensity data, necessitating the integration with cameras for effective object identification, especially in varying lighting conditions.
Innovation Solution
A method and apparatus that reconstructs two-dimensional reflection intensity images from three-dimensional data using LiDAR and applies them to a deep learning network to generate color images, enhancing object detection capabilities regardless of lighting conditions.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If LiDAR is used to measure distance and obtain reflected light information, then measurement reliability is improved, but reflection intensity data becomes sparse making object detection difficult
Solution Approach 1:
The patent combines LiDAR's reliable distance measurement capability with camera's rich visual information capability. The LiDAR projection image generation unit creates a two-dimensional reflection intensity image from three-dimensional LiDAR data, which is then integrated with camera images through a deep learning network. This merging allows the system to maintain LiDAR's measurement reliability while overcoming its sparsity limitation by supplementing with camera data.
Solution Approach 2:
The patent introduces a LiDAR projection image as an intermediary representation that bridges LiDAR point cloud data and camera images. The LiDAR projection image generation unit transforms three-dimensional LiDAR reflection intensity data into a two-dimensional projection image that matches the camera's field of view. This intermediary structure enables the deep learning network to effectively fuse LiDAR and camera data, resolving the object detection difficulty caused by LiDAR data sparsity.
2Difficulty of detecting and measuring
If cameras are used for imaging, then object identification capability is improved, but image quality deteriorates under varying light conditions
Solution Approach 1:
The patent uses LiDAR projection images as an intermediary that provides stable structural information independent of lighting conditions. The LiDAR projection image generation unit creates this intermediary representation from LiDAR data, which is then fed into the deep learning network alongside camera images. This intermediary structure allows the system to maintain consistent object identification capability across varying light conditions by relying on LiDAR's illumination-independent measurements.
Solution Approach 2:
The patent creates a composite information structure by fusing LiDAR projection images with camera images through a deep learning network. The LiDAR projection image provides stable geometric and spatial information, while the camera image provides rich texture and color information. This composite approach enables the system to achieve reliable object identification under varying light conditions by combining the strengths of both sensing modalities.
3Measurement precision
If LiDAR data is processed directly, then measurement accuracy is maintained, but image generation capability deteriorates due to three-dimensional data structure
Solution Approach 1:
The patent applies dimensionality transformation by converting three-dimensional LiDAR reflection intensity data into a two-dimensional projection image. The LiDAR projection image generation unit performs this transformation by projecting 3D points onto a 2D plane that corresponds to the camera's field of view. This dimensional change maintains measurement accuracy by preserving the spatial relationships in LiDAR data while making the data compatible with 2D image processing and deep learning networks.
Solution Approach 2:
The patent transforms the three-dimensional point cloud structure of LiDAR data into a two-dimensional projection image that aligns with the camera's image plane. This dimensional transformation is achieved by the LiDAR projection image generation unit, which maps 3D coordinates to 2D coordinates while preserving spatial relationships. The resulting 2D projection image maintains the measurement precision of LiDAR data while enabling effective integration with 2D camera images through the deep learning network.
Applied Scientific Principles
This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.
Function Achieved in This Case
This approach enables the generation of clear and consistent images day and night, reducing environmental influence and improving object detection in autonomous vehicles and crime prevention applications.
Implementation Method 1
the distance to the object is measured by using the time taken to radiate light to an object and receive the light back
Implementation Method 2
the amount of reflected light
Data Source
AI summary
According to an aspect of embodiments, a method of generating an image by using LiDAR includes performing a reconstruction of a two-dimensional reflection intensity image, the performing of the reconstruction including projecting three-dimensional reflection intensity data that are measured by using the LiDAR as the two-dimensional reflection intensity image, and the method includes generating a color image by applying a projected two-dimensional reflection intensity image to a Fully Convolutional Network (FCN).