Multi-Sensor Fusion Using Two-Stage Transform Learning
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional multi-sensor fusion techniques are computationally complex and do not perform well in all scenarios due to data imperfections and diversity of sensing mechanisms, often requiring complex architectures that are not universally effective.
Innovation Solution
A processor-implemented method for joint optimization of sensor-specific transforms, fusing transforms, coefficients, and weight matrices using a two-stage Transform Learning approach, where sensor-specific transforms and coefficients are learned individually and then fused using a common transform and coefficient to capture correlations between sensor representations.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If conventional multi-sensor fusion techniques are used, then data from multiple sensors can be combined, but computational complexity increases significantly
Solution Approach 1:
The patent segments the sensor fusion process into two distinct stages: (1) individual transform learning for each sensor type to capture sensor-specific characteristics, and (2) joint optimization to learn fusion weights and correlations. This segmentation reduces computational complexity by breaking down the complex fusion problem into manageable sub-problems that can be solved independently and then combined.
Solution Approach 2:
The patent performs preliminary transform learning for each sensor type before performing the joint optimization. By pre-learning sensor-specific transforms and capturing individual sensor characteristics in advance, the system prepares data in a standardized form that simplifies the subsequent fusion process, reducing the computational burden during actual multi-sensor fusion operations.
2Reliability
If complex fusion architectures are adopted to handle data imperfections and diversity, then fusion accuracy may improve, but the system becomes less universally effective and more complex
Solution Approach 1:
The patent develops a universal transform learning framework that can handle multiple sensor types (accelerometers, gyroscopes, magnetometers, barometers, cameras, microphones) through a common two-stage optimization process. The method learns sensor-specific transforms that are tailored to each sensor type's characteristics while using a unified fusion architecture, making the system universally effective across diverse sensing mechanisms without requiring architecture-specific modifications.
Solution Approach 2:
The patent applies local quality by learning sensor-specific transforms that are customized to each sensor type's unique characteristics and data imperfections. Each sensor undergoes individual transform learning to capture its specific patterns and noise characteristics, while the joint optimization stage learns fusion weights that account for local sensor correlations. This approach handles data imperfections and diversity effectively while maintaining a relatively simple universal fusion architecture.
Data Source
Figure 1
Figure 2
Figure 3A
AI summary
This disclosure relates to multi-sensor fusion using Transform Learning (TL) that provides a compact representation of data in many scenarios as compared to Dictionary Learning (DL) and Deep network models that may be computationally intensive and complex. A two-stage approach for better modeling of sensor data is provided, wherein in the first stage, representation of the individual sensor time series is learnt using dedicated transforms and their associated coefficients and in the second stage, all the representations are fused together using a fusing (common) transform and its associated coefficients to effectively capture correlation between the different sensor representations for deriving an inference. The method and system of the present disclosure can find application in areas employing multiple sensors that are mostly heterogeneous in nature.