Asynchronous Sensor Fusion Using Dual Dead Reckoning for Low Latency
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing sensor fusion methods struggle with latency issues when combining asynchronous sensor data from systems operating at different frequencies, particularly in applications requiring real-time or low-latency state estimation, such as inertial navigation and digital pens.
Innovation Solution
Implement interrelated first and second dead reckoning algorithms to integrate relative measurement samples, using a sensor fusion algorithm at a lower frequency to correct and update the state estimation, with buffer mechanisms to synchronize and reset the system state, reducing latency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If sensor data from multiple sources are combined using sensor fusion, then the estimation of the entity's pose and/or location is improved, but latency increases due to time delay in data delivery
Solution Approach 1:
The patent applies preliminary action by predicting the state at a previous time point before the actual measurement arrives. The prediction mechanism pre-calculates what the state should have been, allowing the system to work with predicted values in advance rather than waiting for delayed measurements, thus reducing the effective latency in the fusion process
Solution Approach 2:
The patent introduces an intermediary prediction mechanism that mediates between the delayed absolute measurements and the continuous relative measurements. This prediction intermediary bridges the time gap by providing estimated state values at previous time points, enabling synchronous processing without directly transmitting the delayed data
2Stability of the object's composition
If buffering mechanism is implemented to synchronize sensor data, then synchronization between sensor data is improved, but output latency increases
Solution Approach 1:
Instead of passively buffering data and waiting for synchronization, the system performs preliminary prediction of the state at the previous time point. This active prediction approach provides synchronized data immediately rather than waiting for the buffer to fill, reducing output latency while maintaining synchronization
Solution Approach 2:
The patent makes the synchronization mechanism dynamic by using prediction that adapts to the system's motion characteristics. Rather than a static buffer with fixed delay, the prediction dynamically adjusts to provide the most current estimated state, reducing latency while maintaining synchronization accuracy
3Productivity
If sensor fusion algorithm is executed at lower frequency to reduce computational complexity, then processing speed is improved, but measurement precision deteriorates
Solution Approach 1:
The patent segments the processing into two parts: a lightweight prediction step that runs at high frequency using only relative measurements, and a heavier correction step that runs at lower frequency using absolute measurements. This segmentation allows the system to maintain high update rates for basic functionality while performing comprehensive fusion less frequently, balancing processing speed and precision
Solution Approach 2:
The patent applies partial action by using only relative measurements for the prediction step, which is computationally lighter than full sensor fusion. This partial processing runs continuously at high frequency, providing timely estimates, while the complete fusion algorithm runs at lower frequency to correct these estimates, achieving both speed and accuracy
Data Source
Figure 1
Figure 2
Figure 3
AI summary
There is described a method of estimating a state of a system (100) based on sensor fusion of sensor data provided by a first sensing system (S1) delivering absolute measurement samples at a first frequency F1 and sensor data provided by at least a second sensing system (S2) delivering relative measurement samples at a second frequency F2 higher than the first frequency F1, the absolute measurement samples being delivered with a time delay Δt relative to the relative measurement samples. The method includes applying interrelated first and second dead reckoning algorithms (ALG1H, ALG1T) to integrate over time the relative measurement samples to respectively provide a current estimation and a prior estimation of the state of the system, a state (State - Head) of the first dead reckoning algorithm (ALG1H) representing the estimated state of the system (100) at a current time t, while a state (State - Tail) of the second dead reckoning algorithm (ALG1T) represents the estimated state of the system (100) at a prior time t - Δt. Sensor fusion is carried out by means of a sensor fusion algorithm (ALG2) that is executed at the first frequency F1 based on the absolute measurement samples and samples of the prior estimation of the state of the system (100) provided by the second dead reckoning algorithm (ALG1T), the state (State - Tail) of the second dead reckoning algorithm (ALG1T) being periodically corrected based on the fused output of the sensor fusion algorithm (ALG2) and used to periodically update the state (State - Head) of the first dead reckoning algorithm (ALG1H). The method is designed to provide a current estimation of the state of the system (100) at high frequency and with minimal output latency despite the time delay Δt and asynchronous nature of the sensor data.