Multi-Sensor Fusion Processing for Robust Object Recognition
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current automated driving and advanced driver assistance systems face challenges in accurately recognizing objects in varying environmental conditions due to the strengths and weaknesses of individual sensors like cameras, millimeter wave radars, and LiDARs, which are affected by weather and lighting conditions.
Innovation Solution
An information processing device and method that performs fusion processing on data from multiple sensors, including millimeter wave radars and cameras, to enhance recognition accuracy by combining detection signals, distance, speed, and angle data before and after recognition, and using early or late fusion techniques to supplement recognition results.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If multiple sensors are combined for sensor fusion, then object recognition accuracy is improved, but device complexity increases
Solution Approach 1:
The patent segments the sensor fusion process into multiple distinct units: a recognition unit that performs recognition processing on objects based on sensor detection signals, and a processing unit that performs fusion processing on data before recognition. This segmentation allows each unit to specialize in specific tasks, improving overall recognition accuracy while managing system complexity through modular architecture.
Solution Approach 2:
The processing unit performs fusion processing on first data before the recognition by the recognition unit and another data. By performing preliminary fusion processing on raw sensor data before recognition, the system integrates information from multiple sensors early in the processing chain, enhancing recognition accuracy while avoiding the need for complex post-recognition fusion operations.
2Reliability
If fusion processing is performed on data before recognition, then recognition accuracy in adverse conditions is improved, but processing complexity increases
Solution Approach 1:
The processing unit performs fusion processing on first data before the recognition by the recognition unit and another data. This preliminary fusion action combines sensor data early in the processing chain, ensuring that recognition operations work with already-integrated information, which improves reliability in adverse conditions while managing processing complexity through structured data flow.
Solution Approach 2:
The processing unit acts as an intermediary between the sensor and the recognition unit, performing fusion processing on data before it reaches the recognition unit. This intermediary function integrates information from multiple sensors and prepares it for recognition, improving reliability in adverse conditions while isolating the complexity of fusion operations in a dedicated processing layer.
3Loss of information
If multiple fusion processing operations are performed, then data integration completeness is improved, but processing time increases
Solution Approach 1:
The patent segments fusion processing into distinct operations: fusion processing on first data before recognition, fusion processing on third data before recognition by a second recognition unit, fusion processing on fourth data after recognition by the second recognition unit, and fusion processing on first and second data after recognition. This segmentation allows each fusion operation to handle specific data types at appropriate processing stages, ensuring data integration completeness while managing processing time through parallel and staged operations.
Data Source
AI summary
There is provided an information processing device that processes detection information of an external recognition sensor.The information processing device includes: a recognition unit that performs recognition processing on an object on the basis of a detection signal of the sensor; and a processing unit that performs fusion processing on first data before the recognition by the recognition unit and another data. The information processing device further includes a second recognition unit that performs recognition processing on the object on the basis of a detection signal of a second sensor. The processing unit performs fusion processing on third data before the recognition by the second recognition unit and the first data, fusion processing on fourth data after the recognition by the second recognition unit and the first data, and the like.