Physiological Signal Prediction via Spatio-Temporal Graphs
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods for predicting physiological signals using remote photoplethysmography (rPPG) face challenges in accuracy and robustness, particularly with ambient noise and motion artifacts such as head rotation and changes in illumination, which affect the stability and reliability of the predicted signals.
Innovation Solution
A method involving video segmentation into short-term clips, feature extraction from specific regions like the cheeks, conversion to spatio-temporal graphs, and training with deep learning models like 3D convolutional neural networks to predict physiological parameters, enhancing stability and robustness by handling noise and motion artifacts.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If traditional signal processing methods (ICA, FFT, WT) are used for rPPG, then the measurement process is simple, but the prediction accuracy is low due to sensitivity to motion artifacts and ambient noise
Solution Approach 1:
The patent segments long-term videos into multiple short-term video clips, each corresponding to a specific physiological signal tag. This segmentation allows the system to process manageable time windows independently, reducing the impact of long-term motion artifacts and illumination changes while maintaining computational feasibility through localized analysis.
Solution Approach 2:
The patent transforms video data into spatio-temporal graphs by converting RGB color space to YUV color space and constructing graphs with temporal and spatial dimensions. This dimensional transformation enables deep learning models to capture both spatial relationships within frames and temporal relationships across frames, significantly improving physiological signal extraction accuracy.
Solution Approach 3:
The patent replaces traditional mechanical signal processing methods (ICA, FFT, WT) with deep learning-based computational approaches. By using 3D convolutional neural networks and graph neural networks, the system achieves superior robustness to motion artifacts and ambient noise while automatically learning optimal feature representations without manual feature engineering.
2Measurement precision
If deep learning models are used to improve prediction accuracy, then the measurement precision improves, but the computational complexity and processing time increase
Solution Approach 1:
By segmenting videos into short-term clips and processing them independently, the system enables parallel computation and reduces the computational burden on each processing step. This segmentation strategy allows deep learning models to converge faster and reduces overall processing time while maintaining high accuracy through focused local analysis.
Solution Approach 2:
The patent performs preliminary processing by converting RGB to YUV color space and extracting spatio-temporal features before feeding data into deep learning models. This pre-processing prepares the data in an optimal format, reducing the computational complexity during model inference and accelerating the overall processing pipeline.
3Reliability
If feature extraction from interested regions is performed, then the signal quality improves, but the processing complexity increases
Solution Approach 1:
The patent focuses feature extraction on interested regions (such as facial regions) rather than processing the entire video frame. By concentrating computational resources on regions most relevant to physiological signal extraction, the system improves signal quality and reliability while reducing overall processing complexity through selective analysis.
Solution Approach 2:
The construction of spatio-temporal graphs transforms local region features into a structured representation that captures both spatial relationships and temporal dynamics. This graph-based representation simplifies subsequent processing by organizing features in a manner that naturally reflects the underlying physiological patterns, reducing the need for complex feature engineering.
Applied Scientific Principles
This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.
Function Achieved in This Case
The method improves the accuracy and stability of physiological signal prediction by converting video clips into spatio-temporal graphs and using deep learning models to predict physiological parameters, effectively mitigating the impact of noise and motion artifacts, resulting in more reliable predictions.
Implementation Method 1
Considering that hemoglobin absorbs light into the skin, diffuse reflection waves will slightly vary with changes in the blood volume
Data Source
AI summary
A physiological signal prediction method includes: collecting a video file, the video file containing long-term videos, and contents of the video file containing data for a face of a single person and true physiological signal data; segmenting a single long-term video into multiple short-term video clips; extracting, by using each frame of image in each of the short-term video clips, features of interested regions for identifying physiological signals so as to form features of interested regions of a single frame; splicing, for each of the short-term video clips, features of interested regions of all fixed frames corresponding to the short-term video clip into features of interested regions of a multi-frame video, and converting the features of the interested regions of the multi-frame video into a spatio-temporal graph; inputting the spatio-temporal graph into a deep learning model for training, and using the trained deep learning model to predict physiological signal parameters.