Object Sensing via Image-Point Cloud Feature Fusion
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing object sensing technologies face issues of poor robustness and accuracy, particularly in autonomous driving and monitoring scenarios where accurate identification of objects is crucial.
Innovation Solution
The method involves collecting image data and point cloud data, generating fusion features by combining image features and point cloud features through feature fusion, using techniques such as neural field networks and sampling point subsets to enhance sensing accuracy and robustness.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If single-source sensing (image or point cloud) is used, then device complexity is reduced, but sensing accuracy and robustness deteriorate
Solution Approach 1:
The patent combines image data from cameras and point cloud data from LiDAR into a unified sensing system. The multi-source data fusion module integrates these different data types to create comprehensive object representations, improving sensing accuracy while managing system complexity through structured fusion processes.
Solution Approach 2:
The patent creates composite feature representations by fusing image features and point cloud features. The fusion feature generation module combines visual appearance information with geometric structure information to form enhanced object features that leverage the strengths of both sensing modalities.
2Measurement precision
If feature fusion is performed without mapping to common feature space, then processing speed is improved, but feature fusion accuracy deteriorates
Solution Approach 1:
The patent performs feature mapping to a common feature space as a preliminary step before feature fusion. The feature mapping module transforms image features and point cloud features into a unified feature space, enabling accurate fusion while optimizing subsequent processing efficiency through pre-aligned feature representations.
3Measurement precision
If all sampling points are processed globally, then sensing completeness is improved, but processing efficiency deteriorates
Solution Approach 1:
The patent divides the set of sampling points into multiple subsets and processes each subset independently through local feature mapping. The sampling point subset processing module segments the global processing task into manageable local tasks, improving processing efficiency while maintaining sensing completeness through subsequent feature aggregation.
Solution Approach 2:
The patent transitions from global batch processing to localized subset processing, adding a spatial dimension to the processing organization. By processing features locally for each sampling point subset and then aggregating results, the system achieves both efficiency and completeness.
Data Source
Figure 1~2
Figure 3~4
Figure 5~7
AI summary
The present invention provides an object sensing method and apparatus, a vehicle, an electronic device, and a computer-readable storage medium, and relates to an autopilot technology. The present method includes: collecting (S 101) image data and point cloud data of the object, and acquiring an image feature of the image data and a point cloud feature of the point cloud data; generating (S102) a fusion feature by performing feature fusion on the image feature and the point cloud feature; and generating (S 103) a sensing result of the object according to the fusion feature.