Contactless Video Pulse Waveform Detection with 3D CNNs
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing video-based techniques for detecting pulse waveform and heart rate are limited in accuracy, require controlled subject posture and camera positioning, and struggle with movement artifacts.
Innovation Solution
A system utilizing a 3-dimensional convolutional neural network (3DCNN) processes video streams to spatially and temporally analyze frames, producing accurate pulse waveforms without physical contact and with minimal constraints on subject movement or posture.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If video based techniques are used for pulse detection, then physical contact is eliminated, but measurement accuracy deteriorates
Solution Approach 1:
The patent transitions from traditional 2D image processing to 3D volumetric processing by incorporating depth information from stereo cameras or time-of-flight sensors. This dimensional enhancement allows the system to isolate the subject's face from the background more effectively, extract blood volume changes more accurately, and maintain measurement precision while preserving contactless operation.
Solution Approach 2:
The system employs multiple wavelength lights (red, green, blue, infrared) to illuminate the subject and captures reflectance variations at different wavelengths. By analyzing the differential absorption characteristics of hemoglobin at these wavelengths, the system can accurately extract pulse waveform parameters without physical contact, resolving the accuracy-precision tradeoff.
2Device complexity
If existing video processing methods are used, then processing complexity is reduced, but reliability deteriorates under subject movement
Solution Approach 1:
The patent implements dynamic background subtraction and adaptive region-of-interest tracking that automatically adjusts to subject movement. The system continuously updates the reference background model and re-identifies the subject's facial region in each frame, maintaining reliable pulse detection even when the subject moves, talks, or changes expression.
Solution Approach 2:
The system employs feedback mechanisms where the detected pulse signal quality is continuously monitored, and processing parameters are dynamically adjusted based on signal reliability metrics. When movement artifacts are detected, the system adapts by adjusting temporal filtering parameters or selecting alternative spatial regions for analysis, thereby maintaining reliability without excessive complexity.
3Adaptability or versatility
If ambient lighting conditions are utilized, then illumination control requirements are reduced, but measurement precision deteriorates due to lighting variations
Solution Approach 1:
The system uses periodic modulation of LED illumination sources at frequencies distinct from ambient lighting variations. By synchronizing the detection to these known modulation frequencies, the system can isolate the reflected light signal from the subject's skin from ambient lighting changes, maintaining precision while accepting flexible ambient lighting conditions.
Solution Approach 2:
The system captures images at multiple wavelengths and uses the differential absorption properties of hemoglobin across these wavelengths to compensate for ambient lighting variations. By analyzing the spectral signature of blood oxygenation at different wavelengths, the system can extract accurate pulse waveform data even when ambient lighting conditions are uncontrolled or varying.
Applied Scientific Principles
This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.
Function Achieved in This Case
The system achieves accurate and reliable pulse waveform capture, enabling effective heart rate and heart rate variability estimation, even with subject movement, thus improving upon existing remote pulse estimation methods.
Implementation Method 1
The change in reflected light from the skin's surface, because of light absorption of blood, is very minor compared to those caused by changes in illumination
Data Source
AI summary
The video based detection of pulse waveform includes systems, devices, methods, and computer-readable instructions for capturing a video stream including a sequence of frames, processing each frame of the video stream to spatially locate a region of interest, cropping each frame of the video stream to encapsulate the region of interest, processing the sequence of frames, by a 3-dimensional convolutional neural network, to determine the spatial and temporal dimensions of each frame of the sequence of frames and to produce a pulse waveform point for each frame of the sequence of frames, and generating a time series of pulse waveform points to generate the pulse waveform of the subject for the sequence of frames.