Epipolar-Projected 3D Image Labeling via 2D Region Mapping
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Manual labeling of regions of interest in 3D images acquired by 3D sensors is complex and less reliable due to the interpretation of distance-based pixel colors and sparser measurement points, making it difficult for human operators.
Innovation Solution
A method that utilizes a 2D image of the same scene acquired by a camera to identify regions of interest, determining corresponding points in the 3D image based on relative position and orientation data, and assigning depth coordinates to delineate the region of interest in the 3D image.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If manual labeling is performed directly on 3D images, then the labeling process can be completed, but the reliability and accuracy of labeling deteriorates due to difficulty in interpreting distance-based pixel colors and sparser measurement points
Solution Approach 1:
The patent introduces a 2D image as an intermediary medium between the operator and the 3D image for the labeling operation. Operators perform labeling on the 2D image which is easier to interpret, and then the labeling results are automatically transferred to the corresponding 3D image through coordinate transformation, thus improving both ease of operation and labeling reliability
Solution Approach 2:
The patent creates a 2D copy (projection) of the 3D image scene captured by a camera. This 2D copy preserves the spatial layout and allows operators to label regions of interest more easily, and then these labels are mapped back to the original 3D image through coordinate transformation, ensuring accurate labeling without requiring direct interpretation of the difficult 3D image
2Loss of information
If labeling is performed on 3D images with distance-based color encoding, then 3D spatial information is preserved, but the ease of interpretation by human operators deteriorates compared to 2D images with actual object colors
Solution Approach 1:
The patent segments the labeling task into two separate stages: first performing labeling on a 2D projection image where color interpretation is intuitive, then separately handling the 3D coordinate transformation. This segmentation allows each stage to be optimized independently - the 2D stage for ease of interpretation and the transformation stage for preserving 3D spatial information
Solution Approach 2:
The 2D image serves as an intermediary that bridges the gap between human visual interpretation capabilities and 3D spatial data. The 2D image maintains correspondence with the 3D scene through coordinate transformation, allowing operators to work with familiar 2D visual information while the system preserves and utilizes the 3D spatial structure
Data Source
AI summary
A method for labeling a 3D image of a scene acquired by a 3D sensor including: receiving: a 2D image of the same scene acquired by a camera, coordinates, in the 2D image, of a set of pixels delineating the region of interest, and of a reference point belonging to the region of interest, and determining the depth of the reference point in a coordinate system associated with the camera; assigning, to the pixels delineating the region of interest in the 2D image, a depth corresponding to the depth of the reference point; computing the coordinates, in the 3D image, of the pixels delineating the region of interest, based on the coordinates of the pixels delineating the region of interest in the 2D image and on the depth assigned to the pixels delineating the region of interest.