Video-Based ECG Reconstruction from Remote PPG Signals
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing health monitoring devices that rely on electrocardiogram (ECG) measurements using bioelectrodes are cumbersome and limit user mobility due to skin adherence and wired connections, while methods to reconstruct ECG signals from photoplethysmogram (PPG) signals are noisy and not generalizable.
Innovation Solution
A system and method that generates remote PPG signals from video stream data using blind source separation and wavelet-based filtering, followed by a prediction model to reconstruct ECG signals, simulating multiple lead ECG measurements without physical contact, using a 1D CNN-based encoder-decoder architecture.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If bioelectrode-based ECG measurements are used, then measurement precision is improved, but device complexity and ease of operation deteriorate due to skin adherence and wired connections
Solution Approach 1:
The patent replaces the mechanical/electrical contact system of bioelectrodes with an optical sensing system using a camera to capture facial blood volume changes. This substitution eliminates the need for skin adherence and wired connections, thereby improving ease of operation and user mobility while maintaining measurement capability through optical detection of physiological signals
Solution Approach 2:
The patent introduces remote PPG signals as an intermediary between the optical video data and the final ECG reconstruction. The remote PPG serves as a mediator that captures physiological information non-invasively through facial blood volume changes, enabling ECG-like measurements without direct body contact, thus resolving the contradiction between measurement precision and ease of operation
2Ease of operation
If PPG signal reconstruction methods are used, then ease of operation is improved, but measurement precision deteriorates due to noise and lack of generalizability
Solution Approach 1:
The patent employs a feedback mechanism where the predicted ECG signals are continuously refined through a neural network model that processes remote PPG signals. The model learns from the relationship between optical facial blood volume changes and actual ECG patterns, progressively improving measurement precision through iterative prediction and adjustment, thereby overcoming the noise and generalizability issues of traditional PPG methods
3Device complexity
If traditional PPG methods are used, then device complexity is reduced, but measurement precision worsens due to noisy signals
Solution Approach 1:
The patent creates a computational copy of the ECG signal generation process by using a neural network model that replicates the relationship between remote PPG signals and ECG waveforms. This copying approach allows the system to generate clean, noise-free ECG predictions from simple optical sensors, improving measurement precision without significantly increasing device complexity as the computational model replaces the need for complex hardware
Applied Scientific Principles
This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.
Function Achieved in This Case
Enables non-contact, mobile, and accurate detection of user vitals such as heart rate and rhythm, overcoming the limitations of bioelectrode-based ECG systems by providing synchronized and generalized ECG signal reconstruction from video-derived PPG signals.
Implementation Method 1
generate a remote photoplethysmogram (PPG) signal based on the image data set
Implementation Method 2
determine a recovered PPG signal by generating a frequency response based on a frequency-tuned filter bank and the remote PPG signal
Data Source
AI summary
Systems and methods of detecting user vitals based on video stream data. The system includes a processor and a memory. The memory may store processor-executable instructions that, when executed, configure the processor to receive an image data set representing a user face over an evaluation period; generate a remote photoplethysmogram (PPG) signal based on the image data set; determine a recovered PPG signal by generating a frequency response based on a frequency-tuned filter bank and the remote PPG signal, the recovered PPG signal generated based on a peak wavelet magnitude of the frequency response; generate a predicted electrocardiogram (ECG) signal based on a prediction model where one or more prediction model decoders tuned based on semantic features of the prior identified remote PPG signal; and determine, for display at a user device, user vitals data associated with the evaluation period.