Multi-Sensor Fusion Network for Overlap-Region Object Detection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional systems using deep neural networks (DNNs) for autonomous driving face challenges in accurately detecting static and dynamic objects due to noisy or inconsistent data at sensor field of view borders, leading to inaccurate object tracking and system performance issues.
Innovation Solution
A multi-sensor fusion machine learning model, such as a deep neural network, integrates data from multiple DNNs to generate a fused output, accounting for learned associations between sensor fields of view, using additional channels like location priors and velocity images to enhance accuracy.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If hand crafted or rule-based algorithms are used to stitch together outputs from multiple DNNs, then the system can integrate data from multiple sensors, but the fused predictions along borders or overlap regions become inconsistent and inaccurate
Solution Approach 1:
The patent merges multiple DNNs into a unified end-to-end trainable system where sensors with overlapping fields of view are processed together through shared architectural components. This allows the system to learn consistent representations across sensor boundaries, eliminating the discontinuities that arise from separate processing pipelines.
Solution Approach 2:
The patent introduces additional input channels representing different sensor modalities and spatial perspectives into a unified neural network architecture. By processing multi-sensor data in a higher-dimensional joint space rather than stitching 2D outputs, the system learns coherent predictions across overlap regions through shared feature extraction layers.
2Ease of operation
If individual DNNs are used for each sensor, then each sensor can be processed independently, but duplicate objects may be detected in overlap regions leading to false determinations
Solution Approach 1:
The patent combines multiple sensor processing pipelines into a unified end-to-end trainable system that shares architectural components and learns joint representations. This allows the system to recognize and eliminate duplicate object detections in overlap regions by learning consistent object representations across all sensors simultaneously.
Solution Approach 2:
The patent implements end-to-end training where detection results from multiple sensors provide feedback to jointly optimize the shared network parameters. This feedback mechanism allows the system to learn from duplicate detections in overlap regions, adjusting weights to suppress false positives while maintaining independent sensor processing capabilities.
3Area of stationary object
If DNNs process objects near image borders, then the field of view can be maximized, but noisy results occur as objects disappear and reappear causing quick changes in detection
Solution Approach 1:
The patent merges information from multiple sensors with overlapping fields of view to provide continuous object tracking across border regions. By combining detections from adjacent sensors, the system maintains object presence information even when individual sensors experience objects entering or exiting their fields of view, reducing detection noise and improving tracking stability.
Data Source
AI summary
In various examples, a multi-sensor fusion machine learning model-such as a deep neural network (DNN)—may be deployed to fuse data from a plurality of individual machine learning models. As such, the multi-sensor fusion network may use outputs from a plurality of machine learning models as input to generate a fused output that represents data from fields of view or sensory fields of each of the sensors supplying the machine learning models, while accounting for learned associations between boundary or overlap regions of the various fields of view of the source sensors. In this way, the fused output may be less likely to include duplicate, inaccurate, or noisy data with respect to objects or features in the environment, as the fusion network may be trained to account for multiple instances of a same object appearing in different input representations.