Object Recognition via Multi-Channel 2D and BEV Feature Fusion
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing vehicle monitoring technologies face challenges in accurately and quickly recognizing objects in the external environment, particularly in classifying objects based on size and distance, leading to errors in object detection and classification.
Innovation Solution
An apparatus and method utilizing a combination of a camera for 2D image capture, lidar for 3D image generation, and a processor that generates bird's-eye view and multi-channel 2D image feature maps, mixing these to create a complex feature map learned by artificial intelligence using convolutional neural networks and the YOLO model for accurate object recognition.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If only 2D image processing is used, then processing speed is fast, but object recognition accuracy is insufficient
Solution Approach 1:
The patent merges 2D image data from cameras with 3D point cloud data from LiDAR to create a comprehensive feature representation. The processor integrates these multi-source data to generate enhanced feature maps that combine the computational efficiency of 2D processing with the spatial accuracy of 3D data, resolving the contradiction between speed and accuracy.
Solution Approach 2:
The patent transforms 2D image data into 3D spatial representations by fusing camera images with LiDAR point clouds. This dimensional transformation enables the system to maintain fast 2D processing speeds while incorporating 3D spatial information for improved object recognition accuracy, particularly for objects at different distances.
2Measurement precision
If only 3D LiDAR data is used, then object classification by size is improved, but detection speed decreases
Solution Approach 1:
The patent segments the object detection task into multiple processing stages: first processing 2D images for rapid initial detection, then using 3D LiDAR data for size-based classification. This segmentation allows the system to maintain fast detection speeds while achieving accurate classification by processing data in an optimized sequence rather than simultaneously.
Solution Approach 2:
The patent applies 3D LiDAR data selectively for size-based classification of objects that require it, rather than processing all objects through full 3D analysis. This partial application of 3D processing maintains detection speed while improving classification accuracy for objects where size differentiation is critical.
3Measurement precision
If multi-channel 2D image with 3D information is generated, then object recognition accuracy is improved, but computational complexity increases
Solution Approach 1:
The patent extracts and processes only the essential features from the multi-channel 2D image data and 3D point cloud data to create optimized feature maps. By extracting only the necessary spatial and appearance features rather than processing all raw data, the system reduces computational complexity while maintaining high recognition accuracy.
Solution Approach 2:
The patent transforms and normalizes data parameters to optimize processing efficiency. The processor adjusts the representation of 3D information when mapping it to 2D images, transforming coordinates and intensities into optimized formats that reduce computational complexity during feature map generation while preserving recognition accuracy.
4Measurement precision
If bird's-eye view feature map is generated from 3D information, then spatial understanding is improved, but processing time increases
Solution Approach 1:
The patent performs preliminary processing of 3D LiDAR data to create pre-transformed feature representations before final object recognition. By pre-processing and organizing 3D spatial information into structured feature maps, the system reduces the computational time required during the actual recognition phase while maintaining accurate spatial understanding.
Data Source
AI summary
An apparatus for recognizing an object includes a camera that obtains a 2D image, a lidar that obtains a 3D image, and a processor. The processor generates a bird's-eye view (BEV) feature map by extracting features from a two-dimensional plane BEV generated based on 3D information. The processor also generates an image feature map by extracting features of a multi-channel 2D image in which the 3D information is added to the 2D image. The processor also generates a complex feature map by mixing the image feature map and the BEV feature map. The processor also recognizes the object by artificial intelligence learning the complex feature map.