Serial Fusion of Eulerian and Lagrangian Heart Rate Estimation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing heart rate estimation systems using face videos face challenges with noise from facial expressions, respiration, and environmental factors, and are limited by the computational expense of Lagrangian approaches and the accuracy issues of Eulerian approaches under improper illumination or camera focus.
Innovation Solution
A method that serially fuses Eulerian and Lagrangian approaches by dividing face videos into intervals, computing heart rate using both methods, and identifying the heart rate with the lower poorness measure to mitigate noise and improve accuracy, employing a poorness measure to switch between approaches based on the efficacy of temporal signals.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If Lagrangian approach is used to track ROI explicitly over time, then measurement precision of heart rate estimation is improved, but device complexity and computational cost increase significantly
Solution Approach 1:
The system dynamically switches between Eulerian and Lagrangian approaches based on detected facial variations. When facial expressions or movements exceed a threshold, the system transitions from Eulerian to Lagrangian tracking, making the measurement approach adaptive rather than static. This resolves the contradiction by applying the computationally intensive Lagrangian method only when necessary for accuracy.
Solution Approach 2:
The patent introduces an intermediate Eulerian approach as a preliminary step before Lagrangian tracking. The system first attempts Eulerian analysis on fixed ROI, and only when this proves insufficient (detected through poorness measure) does it proceed to explicit Lagrangian feature tracking. This intermediary approach reduces overall computational complexity while maintaining measurement precision when needed.
2Productivity
If Eulerian approach is used to fix ROI and analyze variations, then computational cost is reduced, but measurement precision deteriorates under improper illumination or camera focus
Solution Approach 1:
The system employs a feedback mechanism through the poorness measure that continuously evaluates the quality of Eulerian temporal signals. When the measure indicates poor signal quality (due to illumination or focus issues), the system triggers a switch to Lagrangian approach. This feedback loop ensures measurement precision is maintained while preserving computational efficiency of Eulerian method during normal conditions.
Solution Approach 2:
The patent changes the measurement approach parameter from fixed Eulerian to dynamic Lagrangian based on detected conditions. By monitoring signal quality parameters and switching methods accordingly, the system maintains high measurement precision across varying illumination and focus conditions while preserving computational efficiency during stable conditions.
3Measurement precision
If Lagrangian approach is used for all conditions, then measurement precision is maintained, but productivity decreases due to computational expense
Solution Approach 1:
Instead of applying Lagrangian tracking excessively to all video segments, the patent applies it partially and selectively only when the poorness measure indicates Eulerian approach is insufficient. This partial application maintains measurement precision when needed while preserving real-time processing speed during normal conditions where Eulerian suffices.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
Traditional electrocardiography (ECG) and photo-plethysmography (PPG) based HR estimation require human skin contact which is not only user uncomfortable, but also infeasible when multiple user monitoring is required or extreme sensitive conditions is a prime concern as in the case of monitoring neonates, sleeping human and skin damaged patients. Temporal signals depicting the motion or color variations in the frames across time, are estimated from a Region of Interest using Eulerian or Lagrangian approaches. However, the Eulerian approach fails under improper illumination, inappropriate camera focus or human factors like skin color. Likewise, Lagrangian approach is highly time-consuming and may fail when few or less discriminatory features are available for tracking. The present disclosure provides a poorness measure that is indicative of when an approach fails and facilitates serial fusion of the two approaches. Switching to an appropriate approach results in accurate heart rate estimation.