Vehicle Vision System Sensor Fusion via Cost Matrix Assignment
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current vehicle vision systems face challenges in effectively fusing data from multiple sensors to accurately detect and track objects and lanes, particularly in transitioning between sensor views, which affects the reliability of trajectory and velocity estimation.
Innovation Solution
A vehicle vision system that utilizes CMOS cameras to capture exterior images, performs data transformation through core fusion (synced frame-hypotheses fusion, temporal hypotheses propagation, and temporal smoothing), and higher-level tasks like trajectory/velocity estimation, using a cost matrix calculation and optimal assignment algorithm to generate fused object hypotheses, while integrating data from various sensors like cameras, ultrasonic, radar, and lidar.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If data from multiple sensors is fused to improve object detection accuracy, then measurement precision is improved, but device complexity increases
Solution Approach 1:
The patent segments the complex sensor fusion process into distinct functional modules: a cost matrix generation module that creates cost matrices from sensor data, an optimal assignment module that processes these matrices, and a fused object hypothesis generation module. This segmentation allows each module to handle specific aspects of the fusion process independently, managing complexity while maintaining detection accuracy.
Solution Approach 2:
The patent introduces cost matrices as an intermediary data structure that mediates between raw sensor inputs and final object detection results. The cost matrix serves as a computational bridge that systematically evaluates and combines hypotheses from multiple sensors, reducing the complexity of direct multi-sensor fusion while preserving measurement precision.
2Reliability
If multiple sensors are integrated for comprehensive object tracking, then reliability is improved, but device complexity increases
Solution Approach 1:
The patent implements a universal optimal assignment framework that can process and fuse data from multiple different sensor types (cameras, radar, lidar, ultrasonic sensors) through a common cost matrix methodology. This multi-functional approach allows the system to integrate diverse sensors reliably without requiring separate processing pipelines for each sensor type, thereby improving reliability while controlling complexity.
Solution Approach 2:
The patent transforms sensor data into a standardized parameter representation through cost matrices, where different sensor inputs are converted into comparable cost values. This parameter transformation allows heterogeneous sensor data to be integrated systematically, improving tracking reliability across multiple sensor types while managing the complexity of sensor integration through uniform parameter handling.
3Productivity
If sensor data is processed in real-time for trajectory estimation, then productivity is improved, but use of energy increases
Solution Approach 1:
The patent implements partial processing by focusing computational resources on generating and processing only the necessary cost matrices for current object hypotheses, rather than exhaustively processing all possible sensor data combinations. This selective processing approach maintains real-time productivity for critical trajectory estimation while reducing overall energy consumption by avoiding unnecessary computational operations.
Solution Approach 2:
The patent performs preliminary actions by pre-generating cost matrices from sensor data before final object hypothesis formation. This preliminary processing organizes sensor data into structured cost matrices in advance, enabling faster real-time trajectory estimation and velocity calculation while reducing the computational energy required during critical real-time processing phases.
Data Source
AI summary
A vision system of a vehicle includes at least one camera configured to be disposed at a vehicle so as to have a field of view exterior and forward of the vehicle, and at least one other sensor configured to be disposed at a vehicle so as to have a sensing field exterior of the vehicle. A processor is operable to process data captured by the camera and the sensor. Responsive to processing of captured data, the processor determines an object present in the sensing field of the sensor. Responsive to processing of image data captured by the camera, the processor determines lane markings present in the field of view of the camera and forward of the vehicle. The processor receives object hypotheses, sensor properties and vehicle data as inputs and performs a data transformation to generate fused object hypotheses.