3D Point Cloud Fusion for Precise Vehicle Object Detection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing computer-based navigation systems for autonomous vehicles face challenges in accurately detecting objects around the vehicle, particularly in correlating 2D image data with 3D point cloud data, leading to imprecise projections and noise in object detection, which can result in false negatives and delayed corrective actions.
Innovation Solution
The use of two convolutional neural networks (CNNs) to process 2D image and 3D point cloud data, calibrating them together to identify relevant information, and generating an amalgamated tensor that enriches the 3D data with 2D image information, which is then used by a machine learning algorithm to determine the presence and characteristics of objects around the vehicle.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If 2D image data and 3D point cloud data are correlated for object detection, then measurement precision is improved, but device complexity increases
Solution Approach 1:
The patent segments the object detection process into distinct stages: 2D image processing to identify candidate objects, 3D point cloud processing to verify presence, and fusion of results. This segmentation allows each module to be optimized independently, improving overall detection precision while managing system complexity through modular architecture.
Solution Approach 2:
The patent introduces an intermediary fusion module that correlates 2D image data with 3D point cloud data. This intermediary layer reconciles the two different data types by matching spatial coordinates and features, enabling precise object detection without requiring direct complex integration of the sensor systems themselves.
2Reliability
If multiple sensors are used to capture image and point cloud data, then reliability of object detection is improved, but device complexity increases
Solution Approach 1:
The patent merges the functionality of multiple sensors (2D camera and 3D LiDAR) into a unified object detection system. By combining the strengths of both sensors—2D image recognition capabilities and 3D spatial mapping—the system achieves higher reliability than either sensor could provide alone, while the fusion architecture manages the inherent complexity through coordinated data processing.
Solution Approach 2:
The patent creates a multi-functional sensor system where the 2D camera and 3D LiDAR work together to perform multiple detection tasks simultaneously (object presence, distance measurement, classification). This universal approach improves reliability by cross-validating detections across different sensor modalities while sharing common processing infrastructure to manage complexity.
3Measurement precision
If 3D point cloud data is enriched with 2D image information, then measurement precision is improved, but loss of time increases
Solution Approach 1:
The patent performs preliminary processing of 2D image data to identify candidate objects and extract their spatial coordinates before processing the 3D point cloud data. This preliminary action allows the system to focus computational resources on relevant regions of interest in the 3D data, improving object characterization precision while reducing overall processing time through selective enrichment.
Solution Approach 2:
The patent efficiently integrates 2D image information into 3D point cloud data by adding dimensional enrichment only where needed—using 2D detected object coordinates to select and enhance corresponding 3D point clusters. This selective dimensionality approach improves measurement precision for object characterization while minimizing processing time by avoiding complete 3D cloud reconstruction.
Data Source
AI summary
A computer-implemented method for processing a 3-D point cloud data and an associated image data to enrich the 3-D point cloud data with relevant portions of the image date. The method comprises generating a 3-D point cloud data tensor representative of information contained in the 3-D point cloud data and generating an image tensor representative of information contained in the image data; and then analyzing the image tensor to identify a relevant data portion of the image information relevant to the at least one object candidate. The method further includes amalgamating the 3-D point cloud data tensor with a relevant portion of the image tensor associated with the relevant data portion of the image information to generate an amalgamated tensor associated with the surrounding area and storing the amalgamated tensor to be used by a machine learning algorithm (MLA) to determine presence of the object in the surrounding area.