3D Object Localization Using Bounding Box Weighting
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods for localizing three-dimensional objects relative to a vehicle are inaccurate, require extensive manual labeling, and struggle to distinguish between similar objects, leading to errors and the need for complicated tracking algorithms.
Innovation Solution
A system that combines two-dimensional data and three-dimensional point clouds using a bounding box and weighting algorithm to filter and generate accurate 3D locations of objects, reducing errors and eliminating the need for specialized algorithms for each object type by using a lidar system and cameras to create a 3D point cloud and 2D bounding box.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If generic multi-view geometry algorithms are used for localization, then the system can handle general objects, but the measurement precision deteriorates with errors over one meter
Solution Approach 1:
The patent transitions from 2D image data to 3D point cloud data by introducing depth information through LiDAR sensors. This dimensional enhancement allows the system to achieve accurate localization (error < 0.1 meters) by utilizing spatial coordinates (x, y, z) instead of only 2D pixel coordinates, thereby resolving the precision limitation of generic multi-view geometry algorithms.
Solution Approach 2:
The system combines data from multiple sensor types (cameras for 2D images and LiDAR for 3D point clouds) to create a composite representation of the environment. This fusion of 2D and 3D data leverages the strengths of each modality: the object recognition capability of 2D images and the precise depth measurement of 3D point clouds, achieving both versatility and high precision.
2Measurement precision
If deep learning networks are used to directly regress object location, then localization accuracy improves, but the device complexity increases due to extensive manual labeling and data storage requirements
Solution Approach 1:
Instead of using complex deep learning networks that require extensive labeled training data, the patent creates a simplified 3D representation (bounding box) that copies the essential spatial information from the point cloud. This bounding box approximation serves as a lightweight surrogate that achieves accurate localization without the computational and data infrastructure requirements of deep learning models.
Solution Approach 2:
The system uses temporary, computationally inexpensive bounding boxes as intermediate representations rather than maintaining complex deep learning models. These bounding boxes are generated on-the-fly from the point cloud data and discarded after use, avoiding the need for persistent, resource-intensive deep learning infrastructure while achieving comparable localization accuracy.
3Measurement precision
If custom algorithms are created for specific object characteristics, then localization accuracy for that object type improves, but the device complexity increases and the system cannot distinguish between similar shaped objects
Solution Approach 1:
The patent develops a universal 3D localization algorithm based on point cloud processing and bounding box generation that works for all object types without requiring custom algorithms. The method uses general geometric operations (clustering points, fitting bounding boxes) that are applicable to any object shape, thereby achieving both high precision and low complexity while naturally distinguishing between similar objects through their unique 3D spatial configurations.
Data Source
Figure 1~2
Figure 3
Figure 4~5
AI summary
Disclosed herein are system, method, and computer program product embodiments for localizing three-dimensional objects relative to a vehicle. The system includes: at least one sensor for generating two-dimensional (2D) data and a three-dimensional (3D) point cloud of an environment external to a vehicle. The 3D point cloud includes object points associated with an object. The localization system also includes a memory and at least one processor coupled to the memory. The processor is configured to: provide a bounding box associated with the object from the memory based on the 2D data; arrange the bounding box proximate to the object points in the 3D point cloud; assign a weight to each point of the 3D point cloud based on a position of the point relative to the bounding box; filter the weighted points; and generate a 3D location of the object based on the filtered points.