Cross-Domain Spatial Matching for Monocular 3D Detection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods for monocular 3D object detection and sensor fusion are computationally intensive and not suitable for embedded systems due to the complexity of deriving 3D data from 2D images, particularly in vehicles where computing resources are limited.
Innovation Solution
The implementation of Cross-Domain Spatial Matching (CDSM) techniques that transform 2D image features into 3D features using a CDSM rotation and aggregation process, aligning them with a Birds-Eye-View (BEV) grid, enabling direct 3D object detection and low-level sensor fusion without explicit projections or depth estimations, thus reducing computational requirements.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If explicit projections or depth estimations are used to transform 2D image features into 3D features, then measurement precision is improved, but device complexity and computational requirements increase
Solution Approach 1:
The patent extracts only the essential spatial relationships needed for 3D detection by using a CDSM layer that directly transforms 2D image features into 3D BEV features without performing full explicit projections or depth estimations. This selective extraction of necessary spatial information reduces computational complexity while maintaining adequate measurement precision for detection tasks.
Solution Approach 2:
The patent introduces a CDSM (Cross-Domain Spatial Mapping) layer as an intermediary component that bridges 2D image features and 3D BEV features. This intermediary performs a simplified transformation that avoids the computational burden of traditional explicit projection methods while still enabling accurate 3D object detection through learned spatial mappings.
2Reliability
If complex sensor fusion algorithms are implemented to combine camera and radar data, then reliability is improved, but ease of operation and implementation difficulty worsen
Solution Approach 1:
The patent merges camera image features and radar point cloud features into a unified 3D BEV feature space using the CDSM transformation. By combining both sensor modalities in a common coordinate system, the system achieves reliable sensor fusion that leverages the complementary strengths of camera (visual detail) and radar (depth and velocity information) while simplifying the fusion process through unified spatial representation.
Solution Approach 2:
The CDSM layer serves multiple functions simultaneously: it performs spatial transformation from 2D to 3D, enables sensor fusion between different modalities, and provides a unified feature representation for detection. This multi-functionality simplifies the overall system architecture and ease of implementation while maintaining reliable fusion performance.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
A method of performing Cross-Domain Spatial matching (CDSM) for monocular object detection in 3D free space surrounding a vehicle includes receiving into an image processing network, from a camera of the vehicle, at least one input image, determining from the input image, a set of 2D image features of potential targets in surrounding the vehicle, transforming the set of 2D image features into a set of 3D image features of the potential targets by aligning a lateral axis and a vertical axis of the 2D image features with, respectively, a tensor height axis and a tensor width axis of a 3D birds eye view grid, applying a CDSM aggregation to the 3D image features to extrapolate depth information, and generating a set of aggregated 3D features to the potential targets, and detecting, based on the aggregated 3D features, one or more objects associated with the potential targets.