Sensor Fusion Using Conversion Model for Temporal Alignment
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing sensor fusion technologies face challenges in harmonizing data from multiple sensors with different data formats and sampling rates, leading to inconsistencies in temporal and spatial relationships.
Innovation Solution
A method of sensor fusion that uses a computing device with a conversion model to align data from image capturing devices and lidar sensors. The method involves selecting candidate images, generating two-dimensional data sets from point clouds, superimposing these data sets on images, calculating derived distance inconsistencies, and using a conversion model to derive time differences, thereby selecting target images that match point clouds in time.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If data from multiple sensors with different sampling rates are combined, then comprehensive environmental perception is improved, but temporal and spatial consistency deteriorates
Solution Approach 1:
The system performs preliminary actions by selecting candidate images and point clouds based on timestamp matching before fusion. It pre-processes the data by generating two-dimensional representations and superimposing them to detect inconsistencies, then uses a conversion model to calculate time differences and select target images that align temporally with point clouds, ensuring temporal consistency before comprehensive fusion occurs.
Solution Approach 2:
The conversion model acts as an intermediary that transforms distance inconsistencies into time differences, enabling the system to bridge the temporal gap between sensors with different sampling rates. This intermediary mechanism allows the system to harmonize data from cameras and lidars while maintaining both comprehensive perception and temporal-spatial consistency.
2Quantity of substance
If images and point clouds from different time instants are fused, then data availability is improved, but temporal alignment deteriorates
Solution Approach 1:
The system implements feedback by calculating distance inconsistencies between superimposed two-dimensional point cloud representations and candidate images, then feeding these inconsistencies into a conversion model to derive time differences. This feedback loop enables continuous temporal alignment adjustment, allowing the system to maintain accurate temporal correspondence while maximizing data availability from multiple time instants.
Solution Approach 2:
The system changes parameters by transforming spatial distance inconsistencies into temporal time differences through the conversion model. This parameter transformation allows the system to adjust temporal alignment dynamically based on observed spatial discrepancies, enabling effective fusion of data from different time instants while maintaining temporal accuracy.
3Reliability
If a conversion model is used to harmonize sensor data, then temporal consistency is improved, but computational complexity increases
Solution Approach 1:
The system extracts only the essential temporal alignment information by using the conversion model to calculate time differences from distance inconsistencies. Rather than performing complex full-state optimization, it extracts and utilizes only the necessary temporal correction data, reducing computational complexity while maintaining temporal consistency in the fused sensor data.
Data Source
AI summary
A method includes: selecting, from among a series of images, a candidate image that was captured at an image-capturing time instant corresponding to a point-cloud-generating time instant at which a point cloud was generated; generating a two-dimensional data set from the point cloud; superimposing the two-dimensional data set on the candidate image to result in a superimposed image; obtaining a derived distance inconsistency between the candidate image and the two-dimensional data set in the superimposed image; feeding the derived distance inconsistency into a conversion model to obtain a derived time difference; calculating a target time instant based on the derived time difference and the image-capturing time instant of the candidate image; and selecting, from among the series of images that have been received from the image capturing device, a target image that was captured at a time instant the closest to the target time instant.