Camera-Based PPG Reconstruction for Driver Drowsiness Detection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing driver drowsiness detection systems face challenges such as costly integration of dedicated sensors, low tolerance to non-ideal measurement conditions, and high complexity, especially in vehicles, where accurate biometric data acquisition is hindered by motion artifacts and environmental factors.
Innovation Solution
A method employing a computer-implemented system that uses Long Short-Term Memory (LSTM) artificial neural networks for predicting PPG signals from camera-recorded face landmark points, combined with optional Convolutional Neural Network processing for face expression recognition, allowing for 'sensor-less' biometric data acquisition using general-purpose devices like smartphones, which can process low-frame-rate images to estimate HRV and indicate driver drowsiness or attention levels.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If dedicated PPG sensors are integrated in vehicles for driver drowsiness detection, then measurement precision of biometric data is improved, but device complexity and cost increase
Solution Approach 1:
The patent uses a camera to capture optical images of the driver's face, which serve as a copy or surrogate for direct PPG sensor measurements. The camera records light variations reflected from facial blood vessels, creating an optical copy of the physiological signal that can be processed to extract PPG waveforms without requiring physical contact with dedicated sensors
Solution Approach 2:
The patent replaces the mechanical/optical PPG sensor system with an imaging-based system. Instead of using dedicated photodetectors and light sources in direct contact with the driver, the system uses a camera to capture reflected light from the face, substituting a complex sensor assembly with a simpler imaging device that can process visual information to derive physiological data
2Measurement precision
If ECG signals are used for heart rate variability measurement, then measurement precision is improved, but reliability decreases due to motion artifacts
Solution Approach 1:
The patent introduces facial blood vessels as an intermediary medium for signal acquisition. Instead of directly measuring electrical heart activity with ECG electrodes that are sensitive to motion, the system uses the facial vasculature as a natural optical pathway where blood volume changes modulate reflected light, providing a motion-resistant indirect measurement of cardiac activity
Solution Approach 2:
The patent substitutes electrical measurement (ECG) with optical measurement (reflectance imaging). By replacing electrode-based electrical signal detection with camera-based optical reflectance measurement from facial blood vessels, the system achieves heart rate variability detection that is less susceptible to motion artifacts while maintaining measurement precision
3Reliability
If PPG sensors are integrated in vehicles for driver monitoring, then reliability of drowsiness detection is improved, but ease of manufacture decreases due to integration complexity
Solution Approach 1:
The patent makes the camera serve multiple functions: it captures both visual imagery for driver monitoring and physiological signals for drowsiness detection. This multi-functionality eliminates the need for separate dedicated PPG sensors, allowing the same imaging device to perform both tasks and significantly simplifying vehicle integration while maintaining reliable drowsiness detection
Solution Approach 2:
The patent uses the camera's optical capture capability to create a copy of the PPG signal from facial blood vessels. This copying approach allows the system to derive physiological measurements from a general-purpose imaging device rather than requiring specialized sensors, making the system easier to manufacture and integrate into vehicles while preserving detection reliability
4Measurement precision
If high-frame-rate cameras are used for PPG signal reconstruction, then measurement precision is improved, but use of energy increases
Solution Approach 1:
The patent changes the temporal sampling parameter by using low-frame-rate cameras instead of high-frame-rate cameras. The system processes images captured at lower frame rates (e.g., 30 fps or lower) and applies signal processing techniques to reconstruct PPG waveforms, thereby reducing the energy consumption of the camera while maintaining sufficient measurement precision through computational methods
Applied Scientific Principles
This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.
Function Achieved in This Case
This approach provides a scalable, accurate, and cost-effective solution for detecting driver drowsiness using reconstructed PPG signals from camera images, improving accuracy and integrating with existing vehicle systems without the need for complex hardware, while maintaining robustness against errors and motion artifacts.
Implementation Method 1
A camera is configured to record a series of images of a face of the driver
Implementation Method 2
A method employing a computer-implemented system that uses Long Short-Term Memory (LSTM) artificial neural networks for predicting PPG signals from camera-recorded face landmark points
Implementation Method 3
PPG (PhotoPlethysmoGraphy), InterBeat Intervals (IBIs) may be measured to compute HRV in place of R-R intervals
Data Source
AI summary
A method includes receiving a video signal that comprises a time series of images of a face of a human, wherein the images in the time series of images comprise a set of landmark points in the face, applying tracking processing to the video signal to reveal variations over time of at least one image parameter at the set of landmark points in the human face, generating a set of variation signals indicative of variations revealed at respective landmark points in the set of landmark points, applying processing to the set of variation signals, the processing comprising artificial neural network processing to produce a reconstructed PhotoPletysmoGraphy (PPG) signal, and estimating a heart rate variability of a variable heart rate of the human as a function of the reconstructed PPG signal.