Camera-Radar RoI Fusion for Reliable 3D Object Detection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current object recognition technologies in intelligent vehicle systems face accuracy limitations due to inherent sensor limitations and potential sensor failures, and combining data from different sensors only marginally improves accuracy without addressing these limitations.
Innovation Solution
An electronic device and method that fuse region of interest (RoI) data from camera and radar sensors to enhance 3D object detection accuracy, overcoming individual sensor limitations by extracting and combining feature maps while assigning weights to improve detection reliability and stability.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If object recognition technology uses one sensor, then device complexity is low, but measurement precision deteriorates due to inherent sensor limitations
Solution Approach 1:
The patent combines data from multiple sensor types (camera and radar) to perform object recognition. The system integrates image data from the camera with detection data from the radar sensor, allowing the object recognition unit to leverage complementary strengths of different sensors to overcome individual sensor limitations and improve overall recognition accuracy.
Solution Approach 2:
The object recognition system is designed to handle multiple sensor types and data formats universally. The system can process data from camera sensors, radar sensors, or combinations thereof, making it adaptable to different sensor configurations and failure scenarios while maintaining robust object recognition capability.
2Measurement precision
If object recognition technology uses different types of sensors, then measurement precision improves, but device complexity increases
Solution Approach 1:
The patent divides the object recognition process into distinct functional units: an image acquisition unit for camera data, a detection unit for radar data, and an object recognition unit that integrates both. This segmentation allows each component to specialize in processing specific sensor types while maintaining manageable system complexity through modular architecture.
Solution Approach 2:
The object recognition unit serves as an intermediary that receives and processes data from multiple sensor types. It acts as a mediator that compares and integrates results from different sensors, resolving conflicts and combining information to produce accurate object recognition outcomes while managing the complexity of multi-sensor integration.
3Reliability
If sensor fusion is performed to overcome sensor limitations, then reliability improves, but computational load increases
Solution Approach 1:
The system performs sensor fusion selectively rather than continuously processing all sensor data at full capacity. The object recognition unit compares results from different sensors and processes only the necessary information needed to overcome individual sensor limitations, achieving improved reliability while avoiding excessive computational energy consumption.
Data Source
AI summary
An electronic device and an operating method thereof according to various embodiments are intended to detect a three-dimensional (3D) object based on camera module and radar sensor fusion, and may be configured to extract a first region of interest (RoI) from a first feature map extracted from first sensing data obtained through a camera module, extract a second RoI from a second feature map extracted from second sensing data obtained through a radar sensor, generate a third RoI by fusing the first RoI and the second RoI, and detect a 3D object based on the third RoI.