Lidar Point Cloud Segmentation via 2D Feature Distillation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing lidar point cloud segmentation methods consume excessive computing resources and suffer from low segmentation accuracy due to the limitations of sensor fusion, including different field of views and high computational burdens.
Innovation Solution
A method involving block processing of two-dimensional images to generate multi-scale features, fusion with three-dimensional features using a GRU-inspired fusion mechanism, and unidirectional modal preservation for distilling a single-modal semantic segmentation model, which reduces computational load and enhances accuracy.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If fusion-based methods are used to improve segmentation accuracy by combining camera and lidar data, then segmentation accuracy is improved, but computing resources consumption increases significantly
Solution Approach 1:
The patent extracts and utilizes only the necessary depth information from lidar point clouds, rather than processing complete high-resolution images. By selectively extracting depth features and using them to guide segmentation on lower-resolution images, the method achieves accurate segmentation while significantly reducing computational resource consumption compared to full fusion-based approaches.
2Measurement precision
If fusion-based methods process both images and point clouds simultaneously, then segmentation accuracy is improved, but real-time processing capability deteriorates
Solution Approach 1:
The patent segments the processing task into distinct stages: first processing lidar point clouds to extract depth features, then using these features to guide image segmentation. This segmentation of processing steps allows for optimized computation at each stage, improving real-time processing capability while maintaining segmentation accuracy through the coordinated use of both sensor data types.
3Adaptability or versatility
If camera and lidar have different field of views, then each sensor captures unique information, but establishing point-to-pixel mapping becomes difficult and limits fusion application
Solution Approach 1:
The patent transitions from attempting direct two-dimensional point-to-pixel mapping to utilizing three-dimensional spatial relationships. By processing lidar data in 3D space to extract depth features and then projecting or aligning these features with the image plane, the method effectively handles different field of views without requiring complex direct mapping relationships, thus reducing system complexity while maintaining adaptability.
Data Source
AI summary
The present disclosure provides a lidar point cloud segmentation method, device, apparatus, and storage medium. In the method, the three-dimensional point cloud and the two-dimensional image of the target scene are obtained, and multiple image blocks are obtained by performing block processing on the two-dimensional image; one image block is randomly selected from the multiple image blocks and is outputted to the preset two-dimensional feature extraction network to generate multi-scale two-dimensional features; the feature extraction is performed based on the three-dimensional point cloud to generate multi-scale three-dimensional features; the multi-scale three-dimensional and two-dimensional features are fused to obtain fused features; the fused features are distilled to obtain a single-modal semantic segmentation model. The three-dimensional point cloud is taken as the input of the single-modal semantic segmentation model for semantic discrimination to obtain a semantic segmentation label, thus, the target scene can be segmented based on the semantic segmentation label.