LiDAR Range Image Object Detection via Cross-Domain Annotation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional autonomous vehicle perception systems using LiDAR data face challenges in accurately detecting and classifying objects due to limited precision and recall, and reliance on manual annotation of LiDAR data, which is time-consuming and costly.
Innovation Solution
The system propagates ground truth data from the image domain to the LiDAR domain using a combined point cloud segmentation and bounding box regression network (PCSNet), enabling accurate object detection and classification without manual LiDAR annotation, and uses virtual LiDAR sensors to increase training data efficiency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If traditional LiDAR processing methods are used to classify points based on geometric and dynamic features, then basic object classification is achieved, but precision and recall remain limited and robust semantic class assignment fails
Solution Approach 1:
The patent introduces an intermediary representation called 'range image' that transforms complex 3D LiDAR point cloud data into a 2D image format with depth information encoded in pixel values. This intermediary form enables the use of成熟的 2D image processing techniques and deep learning models to achieve accurate object detection and semantic classification, resolving the limitation of traditional methods while avoiding direct complexity of raw point cloud processing
Solution Approach 2:
The patent creates a 2D copy of the 3D LiDAR data by projecting point cloud information onto a range image plane. This copying process preserves essential spatial and depth information in a format that can be efficiently processed by standard computer vision algorithms, achieving both precision and computational feasibility
2Reliability
If manual annotation of LiDAR data is performed to train DNNs, then accurate ground truth data is obtained, but the process becomes time-consuming and costly
Solution Approach 1:
The patent leverages existing 2D image annotations by copying and adapting them for LiDAR range image training. Since 2D images and range images share the same spatial structure and object locations, annotations created for conventional images can be directly transferred to range images, eliminating the need for separate manual LiDAR annotation while maintaining ground truth accuracy
Solution Approach 2:
The patent makes annotation data universal by demonstrating that 2D image annotations serve multiple purposes: they can annotate both conventional images and LiDAR range images. This multi-functionality allows a single annotation effort to train models for both visual and LiDAR-based perception systems, significantly reducing annotation time and cost
3Adaptability or versatility
If DNNs are trained on ground truth data directly labeled over LiDAR range images or point clouds, then LiDAR-specific perception is achieved, but accuracy decreases due to visual difficulty in interpretation
Solution Approach 1:
The patent uses 2D images as an intermediary reference during the annotation process. Annotators work with visually interpretable 2D images to create ground truth, which is then transferred to LiDAR range images. This intermediary approach ensures high accuracy in the source domain while enabling accurate LiDAR domain adaptation without direct manual annotation of difficult-to-interpret LiDAR data
4Measurement precision
If a large training data set is generated for accurate DNN training, then detection accuracy improves, but data generation time and resources increase
Solution Approach 1:
The patent efficiently generates large training datasets by copying and transforming existing 2D image data into range image format. This copying process is computationally efficient compared to manual LiDAR annotation, enabling the creation of extensive training datasets that improve detection accuracy without proportionally increasing data generation resources
Solution Approach 2:
The patent changes the parameter representation of training data by converting 2D image coordinates and annotations into range image parameters (depth-encoded pixel values). This parameter transformation enables the reuse of existing large-scale image datasets for LiDAR training, multiplying available training data without additional data collection or annotation efforts
Data Source
AI summary
In various examples, a deep neural network (DNN) may be used to detect and classify animate objects and/or parts of an environment. The DNN may be trained using camera-to-LiDAR cross injection to generate reliable ground truth data for LiDAR range images. For example, annotations generated in the image domain may be propagated to the LiDAR domain to increase the accuracy of the ground truth data in the LiDAR domain—e.g., without requiring manual annotation in the LiDAR domain. Once trained, the DNN may output instance segmentation masks, class segmentation masks, and/or bounding shape proposals corresponding to two-dimensional (2D) LiDAR range images, and the outputs may be fused together to project the outputs into three-dimensional (3D) LiDAR point clouds. This 2D and/or 3D information output by the DNN may be provided to an autonomous vehicle drive stack to enable safe planning and control of the autonomous vehicle.