3D Bounding Box Inference from Image and Point Cloud Fusion

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Existing systems struggle to accurately estimate three-dimensional bounding boxes for objects in an environment using two-dimensional image and point cloud data, which is crucial for autonomous systems like autonomous vehicles and drones for navigation and collision avoidance.

Innovation Solution

A machine learning algorithm, specifically an artificial neural network (ANN), is applied to combine feature vectors from image data and point cloud data to estimate the parameters of a three-dimensional bounding box, including geometric and appearance features, with techniques like PointNet for point cloud processing and ResNet for image processing, and using confidence scoring to determine the bounding box.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Measurement precision

If machine learning algorithms combine feature vectors from image data and point cloud data to estimate three-dimensional bounding boxes, then measurement precision is improved, but device complexity increases

Engineering Contradiction:

Improvethree-dimensional bounding box estimation accuracyVSAvoidsystem complexity

Core Design Contradiction:

Measurement precisionVSDevice complexity

Solution Approach 1:

The system segments the feature extraction process into separate modules: image data processing (using ResNet) and point cloud data processing (using PointNet), then combines them through feature vector concatenation. This modular approach maintains high measurement precision while managing device complexity through specialized processing paths.

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

The patent merges image data and point cloud data by concatenating their respective feature vectors into a unified feature representation. This combination enables the system to leverage complementary information from both data sources to achieve accurate three-dimensional bounding box estimation.

Inventive Principle:

Principle #5Merging (Combining)

2Reliability

If multiple sensors and processing algorithms are used to extract feature vectors, then reliability is improved, but manufacturing precision requirements increase

Engineering Contradiction:

Improveobject detection reliabilityVSAvoidsensor calibration precision

Core Design Contradiction:

ReliabilityVSManufacturing precision

Solution Approach 1:

The patent introduces feature vectors as intermediary representations that mediate between raw sensor data and final bounding box parameters. By processing sensor data through learned feature extraction networks (ResNet for images, PointNet for point clouds), the system reduces direct sensitivity to sensor calibration precision while maintaining high reliability through multiple processing paths.

Inventive Principle:

Principle #24Intermediary (Mediator)

Data Source

PatentEP3685306B1Three-dimensional bounding box from two-dimensional image and point cloud data

Publication Date: 2026.03.18 ZOOX INC

Figure 1
Figure 2
Figure 3

AI summary

A three-dimensional bounding box is determined from a two-dimensional image and a point cloud. A feature vector associated with the image and a feature vector associated with the point cloud may be passed through a neural network to determine parameters of the three- dimensional bounding box. Feature vectors associated with each of the points in the point cloud may also be determined and considered to produce estimates of the three-dimensional bounding box on a per-point basis.