Instance Segmentation Model Training Using Projected 3D Point Cloud Data
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional techniques for generating instance segmentation training data are time-intensive and laborious, requiring manual annotation of full pixel-level segmentation masks, especially for objects with irregular boundaries.
Innovation Solution
A computer-implemented method for training an instance segmentation model using sensor data of different dimensionalities, involving capturing 3D point cloud data and 2D image data, projecting the 3D data onto the 2D data, and generating sparse instance segmentation masks for loss propagation during training.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If manual annotation of full pixel-level segmentation masks is used, then training data quality is improved, but time consumption and labor requirements increase significantly
Solution Approach 1:
The patent segments the segmentation task by using 3D bounding boxes to define object regions in point cloud data, which then serve as masks for corresponding 2D image regions. This divides the complex pixel-level annotation task into simpler 3D object-level annotations, reducing manual labor while maintaining training quality
Solution Approach 2:
The patent creates synthetic training data by projecting 3D labeled point cloud data onto 2D image data, effectively copying the 3D annotations into the 2D domain. This allows the system to generate pixel-level segmentation masks automatically from 3D bounding boxes, eliminating the need for manual pixel-level annotation
2Measurement precision
If manual annotation of full pixel-level segmentation masks is used, then training data quality is improved, but labor requirements increase significantly
Solution Approach 1:
The patent segments the segmentation task by using 3D bounding boxes to define object regions in point cloud data, which then serve as masks for corresponding 2D image regions. This divides the complex pixel-level annotation task into simpler 3D object-level annotations, reducing manual labor while maintaining training quality
Solution Approach 2:
The patent creates synthetic training data by projecting 3D labeled point cloud data onto 2D image data, effectively copying the 3D annotations into the 2D domain. This allows the system to generate pixel-level segmentation masks automatically from 3D bounding boxes, eliminating the need for manual pixel-level annotation
3Productivity
If 3D point cloud data is projected onto 2D image data, then training efficiency is improved, but data processing complexity increases
Solution Approach 1:
The patent leverages the dimensional relationship between 3D point cloud data and 2D image data by projecting 3D objects onto their corresponding 2D projections. This dimensional transformation allows automatic generation of 2D segmentation masks from 3D bounding boxes, improving training efficiency while managing processing complexity through established projection algorithms
Data Source
AI summary
Described herein are systems, methods, and non-transitory computer readable media for using 3D point cloud data such as that captured by a LiDAR as ground truth data for training an instance segmentation deep learning model. 3D point cloud data captured by a LiDAR can be projected on a 2D image captured by a camera and provided as input to a 2D instance segmentation model. 2D sparse instance segmentation masks may be generated from the 2D image with the projected 3D data points. These 2D sparse masks can be used to propagate loss during training of the model. Generation and use of the 2D image data with the projected 3D data points as well as the 2D sparse instance segmentation masks for training the instance segmentation model obviates the need to generate and use actual instance segmentation data for training, thereby providing an improved technique for training an instance segmentation model.