LiDAR Sensor Data Inference Using Labels for Missing Intensity
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Machine learning models face challenges in accurately training due to the inefficiency and incompleteness of sensor data, particularly for LiDAR sensors, which lack comprehensive data sets, leading to gaps in understanding and inaccurate determinations.
Innovation Solution
A simulation system using a machine learning model that infers missing attributes in sensor data by leveraging partial sensor data and semantic labels, such as range information, to generate simulated sensor data, including intensity data, through a convolutional neural network architecture.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If actual sensor data is collected through manual driving and labeling, then training data quantity increases, but data collection efficiency decreases and labeling accuracy deteriorates due to manual errors
Solution Approach 1:
The patent uses camera images as a source to generate synthetic LiDAR point cloud data through rendering algorithms. This copying approach creates virtual training data that mimics real LiDAR measurements without requiring physical data collection, thereby increasing training data quantity while maintaining high productivity and eliminating manual labeling errors
Solution Approach 2:
The system employs automated pipelines where camera images are automatically converted to LiDAR point clouds through rendering, and automatically labeled through object detection algorithms. This self-service mechanism eliminates manual intervention entirely, achieving both high data quantity and high collection efficiency simultaneously
2Measurement precision
If LiDAR sensors are used to capture range data, then depth information is obtained, but intensity information is lost or has lower fidelity
Solution Approach 1:
The patent introduces camera images as an intermediary that bridges the gap between LiDAR range data and intensity information. The rendering process uses camera-derived semantic labels and material properties to generate synthetic intensity values for LiDAR points, thereby recovering lost intensity information while preserving accurate range measurements
Solution Approach 2:
The system merges LiDAR range data with camera image data by projecting camera-derived semantic information onto the LiDAR point cloud. This combination allows the recovered intensity information to be integrated with the high-precision range measurements, creating comprehensive sensor data that retains both depth accuracy and intensity completeness
3Loss of information
If camera images are used to capture visual information, then color and texture data are obtained, but depth and range information is limited or absent
Solution Approach 1:
The patent adds the depth dimension to camera images by rendering LiDAR point clouds from camera viewpoints. This dimensional enhancement transforms 2D visual information into 3D spatial data, allowing the system to retain complete visual information while gaining accurate depth and range measurements through the synthesized point cloud structure
Data Source
AI summary
System, methods, and other embodiments described herein relate to simulating sensor data. In one embodiment, a method includes, in response to receiving a request to generate simulated information corresponding to the sensor data, acquiring the sensor data that includes at least range information about a perceived environment. The simulated information includes one or more attributes of the sensor data that are absent from the sensor data in a current format. The method includes computing simulated information of the sensor data using a machine learning model that accepts the sensor data and labels as an input and produces the simulated information as an output. The labels identify at least objects in the perceived environment that are depicted by the sensor data. The method includes providing the simulated information with the sensor data.