LiDAR Point Cloud and Camera Image Synchronization for Color Visibility

Resolve Bottlenecks,
Find Innovative Solutions
Generate Solutions

Solution Overview

Problem

Existing LiDAR-based point group data displays face challenges in enhancing abnormal position visibility and acquiring color information at distant or sky locations where reflected light is not reachable, leading to inconvenience in data visualization.

Innovation Solution

A data processing apparatus and method that integrates point group data processing and image data processing to display point group data from a specific viewpoint, alongside corresponding image data from multiple positions, allowing synchronized display and improved visibility and browsability.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Loss of information

If RGB information is imparted to each point of point group data, then color information is acquired, but positional information cannot be enhanced and abnormal positions cannot be highlighted

Engineering Contradiction:
Improvecolor informationVSAvoidpositional information enhancement
Core Design Contradiction:
Loss of informationVSEase of operation

Solution Approach 1:

The patent separates the display into two independent parts: point group data display for spatial/positional information and image data display for color information. This segmentation allows each component to serve its specific function without interference, enabling both positional enhancement and color information acquisition simultaneously

Inventive Principle:
Principle #1Segmentation

Solution Approach 2:

The patent introduces image data as an intermediary element that complements the point group data. By displaying image data corresponding to the point group data, the system enables color information acquisition without compromising positional information, as the image data serves as an additional layer of information rather than replacing the point group data

Inventive Principle:
Principle #24Intermediary (Mediator)

2Measurement precision

If point group data is displayed using LiDAR scanning, then spatial structure is acquired, but color information cannot be acquired at distant places or sky where reflected light is not reachable

Engineering Contradiction:
Improvespatial structureVSAvoidcolor information at inaccessible locations
Core Design Contradiction:
Measurement precisionVSLoss of information

Solution Approach 1:

The patent merges point group data (from LiDAR) with image data (from cameras) into a unified display system. This combination allows the system to leverage the strengths of both data types: LiDAR provides accurate spatial structure while cameras provide color information, even for distant locations and sky areas where LiDAR alone cannot acquire color data

Inventive Principle:
Principle #5Merging (Combining)

Solution Approach 2:

The display system is designed to handle multiple types of information simultaneously - spatial structure from LiDAR and color information from cameras. This multi-functionality enables the system to acquire both spatial and color data across the entire scene, including distant places and sky regions, without being limited by the constraints of single-data-type systems

Inventive Principle:
Principle #6Universality (Multi-functionality)

3Measurement precision

If only point group data is displayed, then spatial information is shown, but convenience and browsability are reduced

Engineering Contradiction:
Improvespatial informationVSAvoiddisplay convenience
Core Design Contradiction:
Measurement precisionVSEase of operation

Solution Approach 1:

The patent adds another dimension to the display by incorporating image data alongside the point group data. This dimensional expansion transforms the display from a single-type visualization to a multi-type presentation, enhancing both spatial information and color information while improving overall convenience and browsability

Inventive Principle:
Principle #17Another dimension (Dimensionality change)

Applied Scientific Principles

This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.

Function Achieved in This Case

Enhances the convenience and visibility of point group data by displaying image data alongside LiDAR-scanned data, enabling color information acquisition at previously inaccessible locations and improving the inspection results' clarity.

Implementation Method 1

The LiDAR includes a light emission unit emitting the irradiation light, and a detection unit detecting reflected light that is the irradiation light reflected by the object. The LiDAR detects the reflected light reflected by the object while scanning the object with the irradiation light at a predetermined angle of view. Further, the LiDAR calculates a distance D to the object from an expression D=(t2−t1)/2×(light velocity) by using a time t1 until the irradiation light reaches the object and a time t2 until the reflected light reaches the detection unit.

Methodology Applied
Scientific EffectTime of Flight: Time of Flight

Implementation Method 2

a detection unit detecting reflected light that is the irradiation light reflected by the object

Methodology Applied
Scientific EffectLight Reflection: Reflection

Data Source

PatentUS20230080973A1Data processing apparatus, data processing system, and data processing method
Publication Date: 2023.03.16 NEC CORP
  • US20230080973A1 patent drawing
  • US20230080973A1 patent drawing
  • US20230080973A1 patent drawing

AI summary

There are provided a data processing apparatus, a data processing system, and a data processing method that can improve convenience of display of point group data. A data processing apparatus (1) includes a point group data processing unit (10) configured to display, when a viewpoint for display of point group data obtained by scanning a display object is input, the point group data viewed from the viewpoint, and an image data processing unit (20) configured to determine image data corresponding to the point group data to be displayed, from a plurality of pieces of image data obtained by imaging the display object from different positions, and to display the determined image data.