Pseudo 3D Bounding Box Generation for LiDAR-Image Labeling
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional labeling tools for generating 3D bounding boxes are cumbersome and prone to errors due to repetitive tasks, leading to increased costs for inspection and longer processing times, especially when transitioning from 2D to 3D bounding box generation.
Innovation Solution
A method and computing device that automatically create pseudo 3D bounding boxes by matching LiDAR and 2D bounding boxes using regression processes, incorporating calibration and average size/rotational information to generate accurate pseudo 3D bounding boxes for undetected objects.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If conventional labeling tools with pointing devices are used to generate 3D bounding boxes, then labeling workers can create bounding boxes for objects, but the repetitive tasks cause distraction and mistakes, increasing inspection costs and processing time
Solution Approach 1:
The system performs self-labeling by automatically generating 3D bounding boxes using LiDAR data and regression models, eliminating the need for manual labeling operations. The algorithm independently completes the labeling task by matching projected 3D bounding boxes with 2D ground truth and iteratively optimizing parameters, thereby removing human workers from the repetitive labeling process and eliminating associated errors and time losses.
Solution Approach 2:
The patent replaces the mechanical manual operation of pointing devices with an automated computational system. The regression model and coordinate transformation algorithms substitute human manual labeling actions, automatically calculating 3D bounding box parameters from LiDAR point clouds and image data, thereby eliminating the time loss and errors associated with manual repetitive tasks.
2Ease of operation
If manual labeling with pointing devices is used to generate 3D bounding boxes, then bounding boxes can be created, but the complexity of setting size and rotational information makes the process much slower than generating 2D bounding boxes
Solution Approach 1:
The system automatically determines all 3D bounding box parameters (position, size, rotation) through regression modeling and coordinate transformations, eliminating the need for operators to manually set complex parameters. The algorithm self-completes the labeling task by iteratively optimizing bounding box parameters to match ground truth, thereby simplifying the operation while dramatically increasing productivity.
Solution Approach 2:
The patent transforms the complex manual parameter-setting task into an automated parameter-optimization process. By using regression models that automatically adjust bounding box parameters (x, y, z, width, height, length, roll, pitch, yaw) to minimize loss functions, the system changes the approach from manual parameter input to automated parameter optimization, thereby simplifying operation and increasing speed.
3Productivity
If automated pseudo 3D bounding box generation is implemented, then labeling efficiency is improved and human error is reduced, but the system requires complex processing of LiDAR data, image data, and calibration data
Solution Approach 1:
The patent segments the complex automated labeling system into distinct functional modules: LiDAR data processing module, image data processing module, coordinate transformation module, regression modeling module, and bounding box generation module. Each module handles a specific aspect of the process, making the overall complex system manageable and maintainable while achieving high labeling efficiency.
Solution Approach 2:
The patent introduces calibration data as an intermediary that bridges LiDAR data and image data, enabling accurate coordinate transformations between different coordinate systems. This intermediary element facilitates the integration of multi-source data without requiring direct complex interactions between all data types, thereby managing system complexity while maintaining high productivity.
Data Source
Figure 1
Figure 2
Figure 3(A)~3(C)
AI summary
There is provided a method for labeling one or more specific objects by automatically creating one or more specific pseudo 3D bounding boxes. The method includes steps of: (a) acquiring each of projected 3D LiDAR bounding boxes on an image coordinate system using raw data and determining whether each of the projected 3D LiDAR bounding boxes is matched with each of 2D GT bounding boxes on the image coordinate system; and (b) if matched, performing regression process to fit each of the 1-st projected 3D LiDAR bounding boxes into each of the 1-st 2D GT bounding boxes; and if not matched, generating each of the specific pseudo 3D bounding boxes using each of the 2-nd 2D GT bounding boxes and its average size, and rotational information, and performing the regression process to fit each of the pseudo 3D bounding boxes into each of the 2-nd 2D GT bounding boxes.