Systems, devices, and methods for vital sign monitoring

Abstract

Devices, systems, and methods herein relate to non-invasive monitoring of a patient. These systems and methods may receive a plurality of images corresponding to one or more skin regions of the patient, separate the one or more skin regions into a plurality of spatial regions using the plurality of images, determine a plurality of vascular regions based on a spectral graph of the plurality of spatial regions and a cardiovascular parameter, and predict a physiological parameter based on the plurality of vascular regions.

Description

SYSTEMS, DEVICES, AND METHODS FOR VITAL SIGN MONITORINGCROSS-REFERENCE TO RELATED APPLICATIONS

[0001] This application claims priority to U.S. Provisional Patent Application Serial No. 63 / 614,880 filed December 26, 2023, the contents of which is incorporated herein by reference in their entirety for all purposes.TECHNICAL FIELD

[0002] Devices, systems, and methods herein relate to non-invasive monitoring of a physiological parameter of a patient.BACKGROUND

[0003] Vital signs such as blood pressure and heart rate are commonly used to indicate the status and health of a subject. For example, measurement of blood pressure is commonly performed in a clinical setting using a sphygmomanometer and pressure cuff, which may be cumbersome and impractical for continuous and / or ambulatory blood pressure monitoring. However, vital sign monitoring outside a clinical setting is generally limited due to equipment cost, procedural complexity, and poor patient compliance. As such, additional devices, systems, and methods for physiological parameter estimation may be desirable.SUMMARY

[0004] Described here are patient monitoring devices, systems, and methods for providing realtime, non-invasive prediction of one or more physiological parameters of a patient, such as vital signs and related metrics. In some variations, a method of predicting a physiological parameter of a patient may comprise receiving a plurality of images corresponding to one or more skin regions of the patient, separating the one or more skin regions into a plurality of spatial regions using the plurality of images, determining a plurality of vascular regions based on a spectral graph of the plurality of spatial regions and a cardiovascular parameter, and predicting the physiological parameter based on the plurality of vascular regions.

[0005] In some variations, the plurality of images may be generated by one or more cameras. In some variations, the plurality of images may comprise a video. In some variations, higher bitratemay be prioritized over one or more of higher frame rate and higher resolution of the video. In some variations, a bitrate value of the video may be maximized.

[0006] In some variations, a plurality of patient motion regions may be determined based on the plurality of images and a predetermined motion threshold. The plurality of patient motion regions may be excluded from the plurality of spatial regions. In some variations, the plurality of patient motion regions may be determined based on one or more of landmark tracking, optical flow, and difference imaging.

[0007] In some variations, a plurality of patient illumination regions may be determined based on an illumination gradient of the plurality of images and a predetermined illumination threshold. The plurality of patient illumination regions may be excluded from the plurality of spatial regions.

[0008] In some variations, one or more skin regions may be separated based on a neural network. In some variations, determining the plurality of vascular regions may comprise generating the spectral graph comprising nodes and edges. The nodes correspond to the plurality of spatial regions and the edges correspond to similarity to the cardiovascular parameter. A plurality of time-series signals may be generated based on the spectral graph. A plurality of periodic heart rate signals may be identified corresponding to the plurality of time-series signals. The plurality of vascular regions may correspond to the identified periodic heart rate signals.

[0009] In some variations, similarity to the cardiovascular parameter may comprise one or more of a maximum value of time-series cross-correlation and a time shift that maximizes timeseries cross-correlation. In some variations, the plurality of time-series signals may be transmitted to a remote processing device. Predicting the physiological parameter may be performed by the remote processing device. In some variations, determining the plurality of vascular regions may be based on correlation to a set of predetermined spatial regions.

[0010] In some variations, the plurality of vascular regions may be determined based on one or more of a dominant frequency, maximal variation, a correlation coefficient, a cross-correlation among a set of cardiac cycles within a predetermined time period, cycle-by-cycle validation, bandpass filtering, smoothness, motion artifact removal, session filtering, and power spectrum.

[0011] In some variations, a plurality of color signals from the plurality of images may be generated based on the plurality of vascular regions. In some variations, the plurality of color signals may comprise a red signal, a green signal, and a blue signal for each vascular region of the plurality of vascular regions. In some variations, a photoplethysmogram (PPG) signal may be generated based on the plurality of color signals. In some variations, the PPG signal may comprise an absorption PPG signal and a reflection PPG signal. In some variations, the physiological parameter may comprise one or more of blood pressure and blood glucose.

[0012] In some variations, predicting the physiological parameter may comprise generating a plurality of pulse transit times (PTT) based on the PPG signals corresponding to the set of vascular regions. In some variations, a set of PTT values between a first vascular region, a second vascular region, and a third vascular region may be determined. The set of PTT values may be excluded based on a predetermined threshold.

[0013] In some variations, the plurality of PPG signals corresponding to a set of vascular regions of the plurality of vascular regions may be aggregated. In some variations, aggregating the plurality of PPG signals may comprise averaging the plurality of PPG signals.

[0014] In some variations, predicting the physiological parameter uses a first machine learning model. In some variations, the first machine learning model may comprise one or more of linear ridge regression, gradient boosted decision trees, Gaussian process regression, and combinations thereof. In some variations, a vascular map of the patient may be generated based on the plurality of vascular regions.

[0015] In some variations, a calibration value of the physiological parameter may be estimated using a second machine learning model. In some variations, the predicted physiological parameter may be updated using the calibration value. In some variations, a reference physiological parameter signal may be measured using one or more measurement devices while synchronously receiving a plurality of reference images corresponding to one or more skin regions of the patient. The second machine learning model may be trained on the measured reference physiological parameter signal and the synchronously recorded plurality of reference images.

[0016] In some variations, the reference physiological signal may be measured and synchronously recording the plurality of reference images is performed at a first time period anda second time period different from the first time period. In some variations, one or more of the measurement devices may comprise one or more of a blood pressure measurement device, an optical sensor, a pulse oximeter, and an ECG measurement device. In some variations, the optical sensor may comprise one or more of an infrared sensor, a thermal sensor, and an RGB sensor.

[0017] In some variations, a plurality of pulse transit times (PTT) may be generated based on the PPG signals corresponding to the set of vascular regions. In some variations, a set of PTT values between a first vascular region, a second vascular region, and a third vascular region may be determined. The set of PTT values may be excluded based on a predetermined threshold.

[0018] In some variations, the physiological parameter prediction may comprise a confidence interval. In some variations, the physiological parameter may comprise blood pressure or blood glucose. In some variations, predicting the physiological parameter may comprise calculating one or more of systolic amplitude, pulse area, pulse interval, heart rate, time between systolic peak and end of a cardiac cycle, ratio of time before and after a systolic peak in a cardiac cycle, pulse width, maximum upslope, absorbance, Kaiser-Teager energy, signal energy, magnitude, phase, crest time, pulse interval, pulse width at half height (PWHH), Dicrotic Notch time (Tn), A2 time (A2T), diastolic time (DT), first derivative peak time (FDPT), pulse area (PA), area 1, area 2, pulse height (PH), ratio of b peak to a peak of a second derivative (b / a), ratio of e peak to a peak of the second derivative (e / a), modified Normalized Pulse Volume (mNPV), mean arterial pressure (MAP), cardiac output (CO), and total peripheral resistance (TPR). In some variations, the blood pressure may comprise a continuous arterial blood pressure. In some variations, a vascular map of the patient may be generated based on the plurality of vascular regions.

[0019] In some variations, a video conference may be established using a communication device, and the predicted physiological parameter may be output using a display during the video conference. In some variations, the predicted physiological parameter may be transmitted to a communication device.

[0020] Also described here are methods of predicting a physiological parameter of a patient, comprising receiving a plurality of images corresponding to a body of the patient, separating the body into a plurality of non-skin spatial regions using the plurality of images, generatingballistocardiogram data based on the non-skin spatial regions, and predicting the physiological parameter based on at least the ballistocardiogram data.

[0021] In some variations, the body may be separated into a plurality of skin spatial regions using the plurality of images. A plurality of vascular regions may be determined based on a spectral graph of the plurality of skin spatial regions and a cardiovascular parameter. The physiological parameter may be predicted based on the plurality of vascular regions and the ballistocardiograph data.

[0022] In some variations, the plurality of images may comprise a video. In some variations, higher bitrate may be prioritized over one or more of higher frame rate and higher resolution of the video. In some variations, a bitrate value of the video may be maximized.

[0023] In some variations, predicting the physiological parameter may use a first machine learning model. In some variations, the first machine learning model may comprise one or more of linear ridge regression, gradient boosted decision trees, Gaussian process regression, and combinations thereof. In some variations, a calibration value of the physiological parameter may be estimated using a second machine learning model. In some variations, the predicted physiological parameter may be updated using the calibration value.

[0024] In some variations, a reference physiological parameter signal may be measured using one or more measurement devices while synchronously receiving a plurality of reference images corresponding to one or more skin regions of the patient. The second machine learning model may be trained on the measured reference physiological parameter signal and the synchronously recorded plurality of reference images.

[0025] In some variations, measuring the reference physiological signal and synchronously recording the plurality of reference images may be performed at a first time period and a second time period different from the first time period. In some variations, one or more of the measurement devices may comprise one or more of a blood pressure measurement device, an optical sensor, a pulse oximeter, and an ECG measurement device. In some variations, the optical sensor may comprise one or more of an infrared sensor, a thermal sensor, and an RGB sensor. In some variations, the physiological parameter prediction may comprise a confidence interval. In some variations, the physiological parameter may comprise blood pressure or blood glucose.

[0026] In some variations, the physiological parameter may comprise one or more of blood pressure and blood glucose. In some variations, predicting the physiological parameter may comprise calculating one or more of systolic amplitude, pulse area, pulse interval, heart rate, time between systolic peak and end of a cardiac cycle, ratio of time before and after a systolic peak in a cardiac cycle, pulse width, maximum upslope, absorbance, Kaiser-Teager energy, signal energy, magnitude, phase, crest time, pulse interval, pulse width at half height (PWHH), Dicrotic Notch time (Tn), A2 time (A2T), diastolic time (DT), first derivative peak time (FDPT), pulse area (PA), area 1, area 2, pulse height (PH), ratio of b peak to a peak of a second derivative (b / a), ratio of e peak to a peak of the second derivative (e / a), modified Normalized Pulse Volume (mNPV), mean arterial pressure (MAP), cardiac output (CO), and total peripheral resistance (TPR).

[0027] In some variations, the blood pressure may comprise a continuous arterial blood pressure. In some variations, a video conference may be established using a communication device. The predicted physiological parameter may be output using a display during the video conference. In some variations, the predicted physiological parameter may be transmitted to a communication device.

[0028] Also described here are methods of predicting a physiological parameter of a patient comprising receiving a plurality of images corresponding to a body of the patient, separating the body into a plurality of spatial regions using the plurality of images, generating ballistocardiogram (BCG) data and photoplethysmogram (PPG) data based on the plurality of spatial regions, and predicting the physiological parameter based on the PPG data and the BCG data.

[0029] In some variations, generating the BCG data and the PPG data may comprise one or more of independent component analysis principal component analysis. In some variations, the PPG data may be processed based on the BCG data, and the BCG data may be processed based on the PPG data.

[0030] Also described here are methods of predicting a physiological parameter of a patient comprising receiving a plurality of images corresponding to a finger of the patient, generating one or more of a plurality of overlapping spatial regions and a plurality of blood flow vectors using the plurality of images, and predicting the physiological parameter based on one or moreof the plurality of overlapping spatial regions and the plurality of blood flow vectors. In some variations, the plurality of overlapping spatial regions may comprise a region size and a stride size. In some variations, a plurality of color signals may be generated from one or more of the plurality of overlapping spatial regions and the plurality of blood flow vectors.

[0031] In some variations, the plurality of color signals may comprise a red signal, a green signal, and a blue signal for each overlapping spatial region of the plurality of overlapping spatial regions. In some variations, a color intensity for each overlapping spatial region of the plurality of overlapping spatial regions may be calculated.

[0032] In some variations, the plurality of blood flow vectors may be processed based on one or more of histograms of oriented optical flow, principal component analysis, Gaussian mixture models, and Fisher vectors.

[0033] In some variations, a photoplethysmogram (PPG) signal may be generated based on the plurality of color signals. In some variations, the PPG signal may comprise an absorption PPG signal and a reflection PPG signal.

[0034] In some variations, processing the PPG signal may be based on one or more of color channel, normalization, standardization, cropping, clipping, bandpass filtering, and mean / median filtering.

[0035] In some variations, the physiological parameter may comprise one or more of blood pressure and blood glucose. In some variations, the plurality of images may comprise a video. In some variations, higher bitrate may be prioritized over one or more of higher frame rate and higher resolution of the video. In some variations, a bitrate value of the video may be maximized.

[0036] In some variations, a plurality of patient illumination regions may be determined based on an illumination gradient of the plurality of images and a predetermined illumination threshold. The plurality of patient illumination regions may be excluded from the plurality of overlapping spatial regions.

[0037] In some variations, a camera configured to generate the plurality of images may be calibrated. In some variations, camera calibration may comprise one or more of exposure, sensitivity, focus, focus distance, white balance, and lighting.

[0038] In some variations, a plurality of patient illumination regions may be determined based on an illumination gradient of the plurality of images and a predetermined illumination threshold, and the plurality of patient illumination regions from the plurality of overlapping spatial regions may be excluded. In some variations, a camera configured to generate the plurality of images may be calibrated. In some variations, calibrating the camera may comprise one or more of exposure, sensitivity, focus, focus distance, white balance, and lighting.

[0039] In some variations, a physiological parameter may be measured using a measurement device while recording the plurality of images. The predicted physiological parameter may be calibrated based on the measured physiological parameter.

[0040] In some variations, the physiological parameter prediction may comprise a confidence interval. In some variations, the physiological parameter may be predicted using a first machine learning model. In some variations, the first machine learning model may comprise one or more of convolutional neural networks, feed forward neural networks, UNets, variational autoencoders, transformer networks, and combinations thereof. In some variations, the first machine learning model may comprise one or more of a ResNet model, transformer decoder, feed forward neural network, MLP mixer, variational autoencoder decoder, Siamese networks, and combinations thereof. In some variations, the first machine learning model may comprise one or more of XGBoost, SVM, linear regression, logistic regression, decision trees, random forests, K-nearest neighbor, Gaussian process, and combinations thereof.

[0041] In some variations, a calibration value of the physiological parameter may be estimated using a second machine learning model. In some variations, the predicted physiological parameter may be updated using the calibration value. In some variations, a reference physiological parameter signal may be measured using one or more measurement devices while synchronously receiving a plurality of reference images corresponding to one or more skin regions of the patient. The second machine learning model may be trained on the measured reference physiological parameter signal and the synchronously recorded plurality of reference images.

[0042] In some variations, measuring the reference physiological signal and synchronously recording the plurality of reference images may be performed at a first time period and a second time period different from the first time period. In some variations, one or more of themeasurement devices may comprise one or more of a blood pressure measurement device, an optical sensor, a pulse oximeter, and an ECG measurement device. In some variations, the optical sensor may comprise one or more of an infrared sensor, a thermal sensor, and an RGB sensor.BRIEF DESCRIPTION OF THE DRAWINGS

[0043] The patent or application file contains at least one drawing executed in color. Copies of this patent or patent application publication with color drawing(s) will be provided by the Office upon request and payment of the necessary fee.

[0044] FIGS. 1 A and IB are flowcharts of an illustrative variation of a method of predicting a physiological parameter of a patient using a face.

[0045] FIG. 2 is a flowchart of an illustrative variation of a method of predicting a physiological parameter of a patient using skin and non-skin regions.

[0046] FIG. 3 is a flowchart of an illustrative variation of a method of predicting a physiological parameter of a patient using skin and non-skin regions.

[0047] FIGS. 4 A and 4B are flowcharts of an illustrative variation of a method of predicting blood pressure of a patient using a finger.

[0048] FIG. 5 is a plot of an illustrative variation of video bitrate relative to frame rate.

[0049] FIG. 6A is an image of an illustrative variation of a face of a patient. FIG. 6B is an image of an illustrative variation of a face of a patient separated into a plurality of spatial regions.

[0050] FIG. 7 are plots of an illustrative variation of heart rate signals based on unsupervised region selection.

[0051] FIGS. 8A-8D are images of illustrative variations of vascular regions of a face of a patient.

[0052] FIGS. 9A-9C are images of illustrative variations of a face of a patient separated into a plurality of spatial regions based on supervised region selection.

[0053] FIG. 10A is an image of an illustrative variation of a face of a patient separated into a plurality of vascular regions. FIGS. 10B and 10C are plots of an illustrative variation of a timeseries signal of a respective first and second vascular region.

[0054] FIG. 11 is a plot of an illustrative variation of a PPG signal.

[0055] FIGS. 12A-12C are images of illustrative variations of a face of a patient separated into a plurality of spatial regions and vascular regions. FIG. 12D is schematic depiction of an illustrative variation of a PPG signals of vascular regions.

[0056] FIG. 13 A is a plot of an illustrative variation of a plurality of PPG signals. FIG. 13B is a detailed view of the PPG signal plot of FIG. 13 A.

[0057] FIG. 14A is an image of an illustrative variation of a face of a patient separated into a plurality of spatial regions. FIG. 14B is an image of an illustrative variation of vascular time delays of a face of a patient.

[0058] FIG. 15 is a plot of an illustrative variation of an inverted PPG signal.

[0059] FIGS. 16A and 16B are images of illustrative variations of a plurality of spatial regions of a face of a patient. FIGS. 16C and 16D are images of illustrative variations of a plurality of quality values corresponding to FIGS. 16A and 16B.

[0060] FIG. 17A is an image of an illustrative variation of a face of a patient. FIG. 17B and 17C are images of illustrative variations of a vascular map of a face of a patient. FIG. 17D is a schematic diagram of an illustrative variation of a vascular map of a patient.

[0061] FIGS. 18A and 18B are schematic flowcharts of illustrative variations of a method of predicting a physiological parameter using a finger of a patient.

[0062] FIG. 19 is a schematic flowchart of an illustrative variation of a method of predicting a physiological parameter using a finger of a patient.

[0063] FIG. 19 is a schematic diagram of an illustrative variation of a spatio-temporal grid.

[0064] FIGS. 20A-20C are images of illustrative variations of blood flow vectors.

[0065] FIG. 21 depicts plots of illustrative variations of a set of color channels.

[0066] FIG. 22 is a schematic flowchart of an illustrative variation of applying dimensionality reduction to a method of predicting a physiological parameter using a finger of a patient.

[0067] FIGS. 23 A and 23B are plots of illustrative variations of pre-processed PPG signals.

[0068] FIGS. 24A-24F are schematic flowcharts of illustrative variations of machine learning models used to predict a physiological parameter .

[0069] FIG. 25 is a block diagram of an illustrative variation of a computing device.

[0070] FIG. 26 is an illustrative variation of a graphical user interface relating to physiological parameter prediction using a finger.

[0071] FIG. 27 is an illustrative variation of a graphical user interface relating to physiological parameter prediction using a face.

[0072] FIG. 28 is an illustrative variation of a graphical user interface relating to physiological parameter prediction using a face during a video conference.

[0073] FIGS. 29A and 29B are plots of illustrative variations of PTT values for normotensive and hypertensive patients.

[0074] FIGS. 30A-30D are plots of illustrative variations of ground truth and predicted systolic blood pressure.

[0075] FIGS. 31A-31F are plots of illustrative variations of ground truth and predicted diastolic blood pressure.

[0076] FIG. 32 is a schematic flowchart of an illustrative variation of a method of predicting a physiological parameter of a patient.

[0077] FIG. 33 is a flowchart of an illustrative variation of a method of predicting a physiological parameter of a patient by estimating a relative difference in measurements.DETAILED DESCRIPTION

[0078] Described here are systems, devices, and methods for non-invasively predicting a physiological parameter (e.g., characteristic, biomarker) of a patient, such as a vital sign orrelated metric. These systems, devices, and methods may receive and process patient data (e.g., image data, audio data) using one or more signal processing, image processing, computer vision, and machine learning (e.g., deep learning, reinforcement learning) techniques for predicting one or more physiological parameters (e.g., vital signs). One or more of the physiological parameters may be predicted using, for example, a machine learning model with medical-grade accuracy using commonly available hardware.

[0079] In some variations, patient data may be monitored non-invasively using a computing device such as a smartphone, tablet, portable computer, and the like. For example, a smartphone camera may be used to record image data corresponding to one or more of a finger and a face of the patient for processing and vital sign prediction. This may, for example, allow insight into a set of vital signs of a patient on a continuous or semi-continuous, real-time basis. Furthermore, the physiological parameter predictions may be generated securely without patient-identifiable information (e.g., does not include the recorded image data of a face), thereby enhancing the privacy of patient data. The patient data may be processed in a computationally efficient manner such that older and / or less capable hardware (e.g., processor, camera) may be utilized (e.g., without processing using a communication channel and a remote server). As described in more detail herein, data processing (e.g., region identification, region selection, signal extraction, time-series signal calculation) may be performed natively on-device to avoid the time and cost of streaming video data. The systems, devices, and methods described herein may thereby increase adoption among patients otherwise hindered by financial, geographic, cultural, and / or structural barriers. About one in five Americans (e.g., about 60 million people) live in rural areas that traditionally face access challenges to hospital-level care. Thus, the devices described herein for use in predicting a physiological parameter may be portable and utilize existing hardware such that they may allow for continuous or semi-continuous, real-time monitoring with intuitive operation by the patient. Improved prediction and patient compliance may lead to earlier and / or predictive diagnosis, preventative care, and treatment that improve patient outcomes.

[0080] By contrast, conventional methods of determining blood pressure often require use of, for example, a blood pressure cuff and a stethoscope, that can be cumbersome, and difficult to use outside of a clinical setting for a non-medical professional. These conventional methods also fail to provide continuous or semi-continuous monitoring and require training. The systems, devices, and methods described herein are advantageous relative to conventional methods in several ways. For example, the devices, systems, and methods, are intuitive for a non-medicalprofessional to use, and provide an efficient and portable way to predict and monitor health indicators over time. Moreover, the devices, systems, and methods may be configured for a wide range of hardware specifications and environmental conditions (e.g., lighting conditions). Moreover, conventional algorithms suffer from one or more of poor accuracy, high minimum hardware requirements, and high computational load that may reduce their usefulness and / or adoption.

[0081] In some variations, a physiological parameter (e.g., vital sign) may comprise one or more of a heart rate, heart rate variability, respiratory rate, oxygen saturation, blood pressure, and blood glucose. As used herein, patient data may refer to one or more image signals and audio signals measured over a predetermined time period.I. Methods

[0082] Also described here are methods for non-invasively predicting a physiological parameter (e.g., characteristic) of a patient using the systems and devices described herein. In particular, the systems, devices, and methods described herein may be used to accurately predict and monitor values of a physiological parameter, such as, for example, heart rate, blood pressure, blood glucose, heart rate variability, respiratory rate, and oxygen saturation. The predicted physiological parameter may be used in a variety of ways. For example, as will be described herein, the physiological parameter may be output (e.g., displayed) to one or more of a patient and health care professional on a computing device and / or may be stored on the computing device or on a server for later viewing on the computing device. For example, the predicted physiological parameter may be estimated and displayed in real-time during a telehealth meeting (e.g., video conference) between the patient and their health care professional. That is, one or more of the image data and the audio data of the patient may be simultaneously used for estimating a set of patient vital signs for display on a video conference. For example, as described in more detail with respect to FIGS. 26-28, the estimated vital signs may be displayed in real-time. The methods described herein may promote the use of telehealth visits by facilitating high-quality care while reducing the risk of disease transmission (e.g., COVID-19) that may occur during in-person visits. Additionally or alternatively, the patient may selfmonitor themselves independently of a meeting or videoconference with a health care professional. For example, the patient may track a set of their vital signs on a predetermined schedule (e.g., daily before breakfast).

[0083] In some variations, the values of the predicted physiological parameter may be used to establish trends for clinical assessments of the patient. Additionally or alternatively, in some variations, the physiological parameters may be utilized to remotely monitor and / or manage a user. For example, users with known risk factors may be more actively and comprehensively monitored using an application. Health care providers, for example, primary care physicians and / or specialists, may use the information to prescribe a medication regimen and / or to inform therapy decision-making.

[0084] In some instances, the predicted physiological parameter may be used in conjunction with data from other devices or applications (e.g., an activity or fitness tracker, a sleep tracker, a glucometer, an internet-enabled scale and / or body composition device, a meditation tracker) to provide a patient or a health care professional with a more comprehensive view of the patient’s health. In some variations, the predicted physiological parameter may be exported to or used by applications or devices that may analyze them in conjunction with other health related data (e.g., activity data, fitness data, sleep data, weight, body fat percentage, temperature, blood glucose, or the like).Overview

[0085] In some variations, clinical insights into patient health may be derived using a computing device having a sensor including one or more of an optical sensor, microphone, and / or pressure sensor. For example, a smartphone camera may be configured to generate patient image data for vital sign prediction using one or more signal processing and explainable machine learning models. The prediction may be performed by the computing device for remote patient monitoring and / or telehealth applications. It should be appreciated that any of systems and devices described herein may be used in any of the methods described here.

[0086] In some variations, one or more of a reference physiological signal and a plurality of images corresponding to a skin region of a patient may be used to predict a physiological parameter. For example, FIG. 32 is a flowchart depicting an illustrative variation of a method of predicting a physiological parameter (3200) including one or more signal sources. For example, a reference physiological parameter signal may be measured using one or more measurement devices including an optical sensor (3210, 3220), an ECG measurement device (3230), and a pulse oximeter (3240). An optical sensor may include an infrared / thermal sensor (3210)configured to generate infrared image data and an RGB sensor (3220) configured to generate RGB image data. The infrared image data may be processed to generate a corresponding PPG signal. The ECG measurement device (3230) may be configured to generate a reference ECG signal and the pulse oximeter (3240) may be configured to generate a reference PPG signal.

[0087] The method (3200) may include a plurality of data sources (3210, 3220, 3230, and 3240) to increase the accuracy of parameter prediction, but may provide a prediction with a single data source. One or more of the PPG signals and ECG signals derived from the data sources (3210, 3220, 3230, and 3240) may be processed to output (e.g., extract) one or more shape-related features (3250) (e.g., pulse interval, dicrotic notch, systolic peak rising slope, etc.) and to estimate one or more one-dimensional time series signals (3250) (e.g., PTT signal, BCG signal), as described herein. For example, a PTT signal may be generated using a plurality of PPG signals derived from image data from the RGB camera (3220) (e.g., a plurality of time synchronized PPG signals). In some variations, dimensionality reduction (e.g., partial least squares regression) may be applied to the PTT signal to generate a corresponding representation vector.

[0088] Next, one or more of the representation vectors corresponding to the extracted features (3250) and the estimated PTT and BCG signals may be processed using representation learning (3272) (e.g., encoder architecture including a CNN, a residual network, a feed forward neural network, a UNet neural network). Signal similarity (3274) (e.g., cosine similarity) may be calculated between pairs of representation vectors to generate a similarity matrix. An initial prediction of the physiological parameter (e.g., blood pressure) may be estimated based on the similarity matrix. For example, a weighted average of corresponding ground truth blood pressure values (when available) may be calculated as an initial blood pressure prediction.

[0089] The output representation vector of the representation learning (3272) may be input with a calibration representation vector, if available, to a regression model (3276) configured to estimate a relative difference of a physiological parameter value of the current measurement with respect to a previous measurement. The estimated physiological parameter prediction (3280) may include the relative difference of the physiological parameter value added to the initial physiological parameter prediction from the signal similarity (3274) calculation.Face image signal

[0090] FIGS. 1 A and IB are flowcharts depicting an illustrative variation of a method of predicting a physiological parameter using a face of a patient (100). In the variation depicted in FIG. 1A, the method (100) may optionally comprise measuring a reference physiological parameter signal using a measurement device while synchronously receiving a plurality of reference images corresponding to one or more skin regions of the patient (102). In some variations, a calibration machine learning model may be trained on the measured reference physiological parameter signal and the synchronously recorded plurality of reference images. For example, reference physiological parameter measurements (e.g., ground truth values from a gold standard measurement device) may be correlated to respective reference images, used to train a machine learning model, and used to improve physiological parameter prediction. The calibration machine learning model may be used optionally and additionally to a machine learning model configured to predict a physiological parameter value.

[0091] In some variations, a reference physiological parameter signal may be measured using one or more measurement devices while synchronously receiving the plurality of reference images corresponding to one or more skin regions of the patient. For example, the measurement device may comprise one or more of a blood pressure measurement device (e.g., sphygmomanometer), an optical sensor (e.g., infrared sensor, thermal sensor, RGB sensor), a pulse oximeter, and an ECG measurement device.

[0092] In some variations, measuring the reference physiological signal and synchronously recording the plurality of reference images is performed at a first time period and a second time period different from the first time period. For example, the calibration measurements may be improved by measuring a reference physiological parameter on different days or periodically as a patient’s health condition changes over time.

[0093] When a physiological parameter prediction is desired, a user may use an optical sensor to generate a plurality of images corresponding to the patient. For example, a plurality of images corresponding to one or more skin regions of the patient may be received (110). In some variations, the skin corresponds to any portion of the body (e.g., face, finger, neck, wrist) having a sufficient blood vessel structure beneath the skin that may be sufficiently imaged. For example, the image signal may comprise a face and a portion of the patient’s body, as well as any corresponding background. In some variations, the plurality of image signals may be generated by an optical sensor. For example, the optical sensor may be configured to generate one or moreof RGB image data and infrared image data. As used herein, a plurality of images may refer to a video having a predetermined duration (e.g., about 8 seconds, 10, second, 15 seconds, 30 seconds).Bitrate

[0094] Some computing devices (e.g., processor, memory, optical sensor) among a patient population may have limited capabilities such that the quality of the received images may be unsuitable for conventional physiological parameter prediction algorithms. By contrast, the methods described herein may provide medical-grade accuracy using lower quality (e.g., noisy, low bandwidth) image signals. In some variations, higher bitrate may be prioritized over one or more of higher frame rate and higher resolution of the video. A higher bitrate relative to frame rate and resolution may improve physiological parameter prediction. For example, by receiving video at a lower frame rate with a higher bitrate, time series signals may be generated that have minimal reconstruction error due to one or more of noise, quantization error, and compression loss. Consequently, time delays between such signals may be calculated with higher accuracy. That is, higher bitrate image signals may be used to generate higher quality pulse transit time (PTT) signals. In some variations, optical sensor (e.g., camera) settings may be set to maximize a bitrate value of the image signals (e.g., video).

[0095] A video frame rate of 30 frames per second provides about 33 ms resolution for a time series signal. However, pulse transit times may be below 10 ms for regions of the face including and between the cheek and the forehead. Therefore, the frame rate of a video need not be maximized and may be reduced while prioritizing bitrate. FIG. 5 is a plot 500 of bit rate relative to frame rate. A video signal may have a bitrate and frame rate at 510 that enables a smaller region size and provides less correlated noise (e.g., highest bitrate and lowest frame rate). In some variations, PTT may be accurately determined by interpolating a time series signal from low frame rate video, provided that the bit rate of the video recording is high. A higher bitrate provides higher spatial resolution in each video frame, ensuring that periodic signals up to the Nyquist frequency of about 15 Hz may be reconstructed with minimal distortion using spatial noise cancelation techniques. Physiological signals such as a photoplethysmogram (PPG) signal are periodic with a frequency of between about 0.5 Hz and about 2 Hz, and may therefore be reconstructed with enough accuracy to robustly measure PTT from 30 frames per second video.In some variations, image signal quality may further depend on lighting (e.g., wide spectrum lighting, uniform patient lighting) and patient motion, as described in more detail herein.Region selection

[0096] In some variations, one or more skin regions may be separated (e.g., segmented) into a plurality of spatial regions using the plurality of images (120). The plurality of spatial regions may be used to determine a plurality of vascular regions, which in turn may be used to predict a physiological parameter. For example, a plurality of time-series signals may be generated based on the plurality of vascular regions. A plurality of periodic heart rate signals may be identified corresponding to the plurality of time-series signals. The plurality of vascular regions may correspond to the identified periodic heart rate signals.

[0097] FIG. 6A is an image 600 of a face of a patient and FIG. 6B is an image 650 where the skin regions of the patient have been separated into a plurality of spatial regions using a plurality of landmarks 602 corresponding to predetermined facial features. In some variations, one or more skin regions may be separated into a plurality of spatial regions using the plurality of images and a machine learning model (e.g., neural network). For example, each spatial region of a given frame of an image signal may be defined by a set of pixel coordinates corresponding to a predetermined skin region (e.g., cheek, forehead, mouth, eyes, hair).

[0098] In some variations, similarity to the cardiovascular parameter may comprise one or more of a maximum value of time-series cross-correlation and a time shift that maximizes time-series cross-correlation. In some variations, the plurality of time-series signals may be transmitted to a remote processing device. Predicting the physiological parameter may be performed by the remote processing device. In some variations, determining the plurality of vascular regions may be based on correlation to a set of predetermined spatial regions.

[0099] In some variations, the received image signals may be optionally processed to improve the image signal for physiological parameter prediction by, for example, removing extraneous features (e.g., patient body, background objects, overexposure, uneven illumination, patient motion). Signal processing as described herein may enable lower quality data to be used for parameter prediction while maintaining or improving prediction accuracy.

[0100] In some variations, processing one or more of the image signals selects one or more spatial and temporal portions of the one or more image signals. For example, portions of an image signal that do not contain a finger or a face may be identified and removed to reduce computational load and / or increase prediction accuracy.Motion detection

[0101] Patient motion during image data (e.g., video) capture may negatively affect the quality of the received images and subsequent processing and physiological parameter prediction. In some variations, the patient may be encouraged to maintain stillness (e.g., motion of the face and / or body) while recording video. Furthermore, the received plurality of images may be processed to determine and exclude patient motion regions of the plurality of images having motion above a predetermined motion threshold (e.g., indicating involuntary shaking). For example, with respect to method (100) of FIGS. 1 A and IB, a plurality of patient motion regions may be determined based on the plurality of images and a predetermined motion threshold (122). In some variations, the plurality of patient motion regions may be excluded from the plurality of spatial regions (124). In some variations, the plurality of patient motion regions may be determined based on one or more of landmark tracking, optical flow, difference imaging, and motion blur. In some variations, the entire video may be excluded if the number of patient motion regions is above a predetermined exclusion threshold. For example, the image data capture process may be stopped and the patient may be prompted to repeat the process. In some variations, an amount of involuntary motion may be determined with respect to a percentage width of a face. For example, no involuntary motion corresponds to zero percent width of a face, while a maximum amount of involuntary motion may be below about 10% width of the face.Uneven illumination

[0102] Uneven illumination during image data (e.g., video) capture may negatively affect the quality of the received images and subsequent processing and physiological parameter prediction. In some variations, an illumination gradient across the skin of a patient may be used to determine regions of uneven illumination. By tracking the illumination gradient over time (e.g., across sequential frames), a duration (e.g., temporary, persistent) of uneven illumination may be determined.

[0103] With respect to method (100) of FIG. 1 A, a plurality of patient illumination regions may be determined based on an illumination gradient of the plurality of images and a predetermined illumination threshold (126). In some variations, the predetermined illumination threshold may comprise an illumination gradient value and a time duration (e.g., from above 0 to 1 second, above 0 to 2 seconds, above 0 to 3 seconds, above 0 to 4 seconds, above 0 to 5 seconds, above 0 to 10 seconds). In some variations, the plurality of patient illumination regions may be excluded from the plurality of spatial regions (128).

[0104] In some variations, the patient may be encouraged to adjust their lighting conditions while recording video when uneven illumination is determined. In some variations, the entire video may be excluded if the illumination gradient is above the predetermined exclusion threshold. For example, the image data capture process may be stopped and the patient may be prompted to repeat the process after adjusting the lighting. Once the plurality of image signals have been processed to exclude patient motion regions and patient illumination regions, the remaining spatial regions may be used to generate a physiological parameter prediction.Unsupervised region selection

[0105] The vascular regions of the plurality of spatial regions may contain the data useful for physiological parameter prediction and may be determined using unsupervised region selection or supervised region selection. With respect to unsupervised region selection, spectral graph embedding may be used to determine vascular regions corresponding to time series signals with strong Fourier components at or nearby the heart beat frequency, indicating a likely relationship with blood flow. A time series signal may correspond to each spatial region of the determined plurality of spatial regions.

[0106] With respect to method (100) of FIG. 1 A, a plurality of vascular regions may be determined based on a spectral graph of the plurality of spatial regions and a cardiovascular parameter (130). For example, a spectral graph comprising nodes and edges may be generated. The nodes may correspond to the plurality of spatial regions. The edges may correspond to similarity to the cardiovascular parameter. For example, the edges may correspond to a physiologically-relevant measure of time-series similarity such as a maximum value of timeseries cross-correlation and a time shift that maximizes a time-series cross-correlation. That is,the similarity to the cardiovascular parameter may comprise one or more of a maximum value of time-series cross-correlation and a time shift that maximizes time-series cross-correlation.

[0107] In some variations, spectral graph embeddings may be determined (e.g., using a Laplacian Eigenmap). A plurality of time-series signals based on the spectral graph may be generated. For example, a respective linear combination of time-series signals may be used to generate a time-series signal corresponding to each graph embedding component of the spectral graph.

[0108] In some variations, a plurality of periodic heart rate signals corresponding to the plurality of time-series signals may be identified where the plurality of vascular regions correspond to the identified periodic heart rate signals. For example, FIG. 7 are respective plots 700, 710, 720, 730 of periodic heart rate signals corresponding to respective time-series signals. FIGS. 8A-8D are images 800, 810, 820, 830 of vascular regions of a face of a patient corresponding to respective plots 700, 710, 720, 730. Plot 700 may be selected from among plots 700, 710, 720, 730 for providing a higher quality heart rate signal.Supervised region selection

[0109] Supervised region selection may be used to determine a plurality of vascular regions from the plurality of spatial regions of received images. FIGS. 9A-9C are respective images 900, 910, 920 of a plurality of spatial regions of a face of a patient based on supervised region selection. For example, image 900 of FIG. 9A includes a plurality of predetermined (e.g., user selected) landmarks 902. Similarly, image 910 of FIG. 9B includes a plurality of predetermined (e.g., user selected) spatial regions 904 based on the landmarks 902. Additionally or alternatively, time-series signals corresponding to a plurality of spatial regions may be compared against a set of reference physiological signals (e.g., ECG, PPG) measured by a measurement device. For example, if the time-series signals extracted from a spatial region has a correlation to a reference PPG signal above a predetermined threshold, then the corresponding vascular region may be determined. For example, image 920 of FIG. 9C includes a reference spatial region 906.

[0110] As another example, a predetermined spatial region 904 (having a high likelihood of a high-quality time-series signal) may be selected and a correlation of the time-series signal of the spatial region 904 to a time-series signal of every other spatial region may be calculated and compared to a predetermined threshold to determine the vascular region.

[0111] In some variations, the plurality of vascular regions may be determined based on one or more of a dominant frequency, maximal variation, a correlation coefficient, a cross-correlation among a set of cardiac cycles within a predetermined time period, cycle-by-cycle validation, bandpass filtering, smoothness, motion artifact removal, session filtering, and power spectrum.Time-series signal extraction

[0112] From the time-series signal extracted from a corresponding vascular region, one or more physiologically relevant features (e.g., PPG) may be computed and used to predict a physiological parameter (e.g., blood pressure, blood glucose). Generally, photoplethysmography (PPG) is the optical measurement of a change in light absorption corresponding to a change in blood volume associated with heart contraction. When the heart pumps blood, the arteries distend to accommodate the new volume of blood. This increased blood volume increases the amount of light absorbed by the blood, which can be measured by an optical sensor (e.g., photodetector, camera).

[0113] In some variations, the time-series signals corresponding to the received images may include aggregated color pixel values (e.g., red, green, blue). For example, a color signal (e.g., channel) may correspond to an average pixel value of a color within a vascular region. As shown in method (100) of FIG. 1A, a plurality of color signals from the plurality of images may be generated based on the plurality of vascular regions (132). In some variations, the plurality of color signals may comprise a red signal, a green signal, and a blue signal for each vascular region of the plurality of vascular regions. FIG. 10A is an image 1000 of a face of a patient separated into a plurality of vascular regions including forehead region 1002 and cheek region 1004. FIGS. 10B and 10C are plots of time-series signals 1010, 1020 of a respective forehead region 1002 and check region 1004. The time-series signals 1010 and 1020 represent the average green signal value of the pixels within the respective forehead region 1002 and cheek region 1004.Photoplethysmogram (PPG) signal

[0114] As shown in method (100) of FIG. 1 A, a photoplethysmogram (PPG) signal may be generated based on the plurality of color signals (134). For example, baseline wander and signal filtering to remove noise may be performed on the plurality of color signals to generate the PPG signal. In some variations, the PPG signal may comprise an absorption PPG signal and areflection PPG signal. The absorption PPG may correspond to light absorbed by blood and a green signal, an example of which is shown in PPG signal 1100 of FIG. 11. The reflection PPG signal may correspond to light reflected by blood, a red color signal, and a blue color signal. A PPG signal may be generated for each vascular region. In some variations, a PPG signal may be generated based on a single color signal or a weighted combination of a plurality of color signals. In some variations, the PPG signal may be upsampled to a higher frequency. In some variations, a bandpass filter may be applied to the PPG signal. In some variations, polynomial fitting may be applied to the PPG signal.

[0115] While each PPG signal of a vascular region corresponds to patient vasculature (e.g., blood vessels), a single vascular region may be noisy and / or corrupted. In some variations, a plurality of PPG signals may be aggregated (e.g., combined, averaged, summed) to reduce noise and improve PPG signal quality. For example, the PPG signals of a set of vascular regions may be aggregated. FIG. 12A is an image 1200 of a plurality of skin regions of a face of a patient, similar to images 610, 910 as described with respect to FIGS. 6B and 9A. FIG. 12B is an image 1210 of a plurality of vascular regions, similar to images 920, 1000 as described with respect to FIGS. 9C and 10 A. For example, the face may include a forehead region, left cheek region, nose region, right cheek region, and chin region. FIG. 12C is an image 1220 of a schematic depiction of a plurality of PPG signals corresponding to the plurality of vascular regions. For example, each facial region may include four PPG signals per face region. FIG. 12D is schematic depiction of aggregated PPG signals 1230, 1232, 1234. For example, any number of the PPG signals within a face region may be aggregated (e.g., averaged) to generate a higher quality PPG signal for its respective face region (e.g., forehead, left cheek, nose, right cheek, chin). In some variations, PPG signals from different face regions may be aggregated.Pulse Transit Time

[0116] In some variations, PPG signals corresponding in different regions of the face (e.g., forehead, cheek) may be used to generate one or more physiological parameters such as pulse transit time (PTT). In some variations, PTT may be determined based on a time delay (e.g., phase difference) between PPG signals from different regions of the face. FIG. 13 A is a plot 1300 of PPG signals from a forehead, cheek, and wrist of a patient. FIG. 13B is a detailed view 1310 of the PPG signal plot 1300 including a time delay At between the PPG obtained from theforehead and the PPG obtained from the wrist. For example, a pulse may arrive in the lips first, followed by the cheeks.

[0117] In some variations, a time delay between time-series signals may be determined by calculating a difference between Fast Fourier Transform (FFT) phases for each signal at the FFT frequency of maximum signal amplitude (e.g., heart rate frequency), identifying a time shift that maximizes a cross-correlation between time-series signals, or identifying the peaks in each periodic signal and calculating the average time difference between corresponding signal peaks. FIG. 14A is an image 1400 having a plurality of spatial regions. FIG. 14B is an image 1410 of vascular time delays 1402, 1404 of a face of a patient. For example, a blue time delay 1402 may correspond a positive delay relative to a reference region, and a red time delay 1404 may correspond to a negative delay (e.g., signal arriving earlier than the reference signal) relative to the reference region.

[0118] In some variations, noise in a PTT measurements may be reduced by applying least squares fitting to pulse arrival times to enforce consistency with an arrival time of a single heart beat with a common origin. In some variations, a set of PTT values may be calculated between a set of three vascular regions. A PTT value between a first vascular region and a third vascular region should equal the sum of the PTT values between the first vascular region and the second vascular region, and between the second vascular region and the third vascular region. That is, the PTT values may be confirmed via triangulation. A significant difference between the calculated PTT values between the three vascular regions indicates a noisy (e.g., outlier) set of PTT values. The PTT values between those three vascular regions may be excluded based on a predetermined threshold (e.g., PTT values not within 5% of each other).Cross-Correlation

[0119] Generally, signal similarity may be used to assess signal quality and improve physiological parameter prediction. For example, noisy and / or low-quality signals may be filtered based on signal similarity. The shape of a blood pulse wave may change as it travels to different regions of the body. In some variations, maximum signal cross-correlation between physiological signals (e.g., PPG signals) from different regions may correspond to signal similarity. In some variations, signal peaks of a PPG signal may be identified and peak distances may be calculated. Furthermore, maximum values may be physiologically constrained (e.g.,limited to within a time duration between two heart beats). Similarly, peak values may be physiologically constrained (e.g., limited to within about 40 beats per minute to about 140 beats per minute). In some variations, known blood flow patterns may be used for time delay calculations. For example, blood flow arrives first in the head through the neck. Therefore, time delay at the forehead is positive relative to the neck.

[0120] In some variations, a quality value of a PPG signal may be determined by applying a Fourier transform to the PPG signal and identifying the largest frequency components corresponding to the heart rate. The energy of the identified frequency components may be compared to the signal’s total energy. A higher quality signal is found where the frequency components have a higher proportion of energy relative to the total energy. The quality value may be compared to a predetermined threshold to exclude lower quality PPG signals.

[0121] FIGS. 16A and 16B are images 1600, 1610 of a plurality of spatial regions 1602, 1604 of a face of a patient. FIG. 16C is an image 1620 including an overlay of quality scores 1606 corresponding to the FIG. 16B. Quality values below a predetermined threshold may be excluded from physiological parameter prediction, thereby reducing computational load. FIG. 16D is an image 1630 including an overlay of maximum correlation values corresponding to FIG. 16B. Similar to the quality values, maximum correlation values below a predetermined threshold may be excluded from physiological parameter prediction. Removing PPG signals in this manner may lead to an insufficient number of PPG signals. Accordingly, in some variations, the highest quality values and / or maximum correlation values in the predetermined spatial regions 1602 may be included for physiological parameter estimation.Dimensionality reduction

[0122] In some variations, applying dimensionality reduction to a set of PTT values may reduce both noise and computational load such that the PTT values are more robust to outliers and less prone to overfitting. For example, a set of 40 vascular regions (e.g., 12 regions in each of the forehead, left cheek, right cheek, and 4 regions in the chin) may be aggregated to generate 204 PPG signals. These PPG signals may be used to generate 20,708 PTT values. For example, the set of PTT values may be grouped based on vascular region (e.g., first vascular region of forehead) and then averaged in order to reduce the impact of any outlier (e.g., noisy) PTT values within the vascular region.Inverted PPG signal detection

[0123] In some variations, PPG signals may be analyzed to determine an inverted (e.g., flipped) PPG signal. Depending on the vascular tone of blood vessels, adjacent blood vessels may exhibit an elastic-like effect where the expansion of a first blood vessel leads to a contraction of an adjacent second vessel. Frequent PPG flipping indicates a high arterial elasticity, which in turn indicates low arterial stiffness and low blood pressure. Therefore, the absence of flipped PPG signals is a strong indicator of a hypertensive patient. This phenomenon is expressed in a PPG signal as flipping where the phase is shifted by half a period. PTT values cannot be determined for flipped PPG signals.

[0124] FIG. 15 is a plot 1500 of an inverted PPG signal 1502 and a non-inverted PPG signal 1504. In particular, PPG signal 1502 inverts at around 4000 samples. In some variations, a flipped PPG signal may be identified by comparing the PPG signals to each other. For example, a mean squared difference between the PPG signals may be calculated and compared to a predetermined threshold. A relatively large mean squared difference corresponds to a flipped PPG signal. The proportion of flipped PPG signals to the total number of PPG signals may be compared to a predetermined threshold to determine a risk of hypertension. For example, a relatively low proportion of flipped PPG signals corresponds to an increased risk of hypertension.Vascular map

[0125] As shown in method (100) of FIG. 1 A, a vascular map of the patient may be generated based on the plurality of vascular regions (136). For example, the locations of subcutaneous blood vessels (e.g., veins, arteries) may be mapped and used to select vascular regions for PPG calculations and / or detect irregularities in blood flow. FIG. 17A is an image 1700 of a face of a patient and FIGS. 17B and 17C are corresponding vascular maps 1710, 1720. FIG. 17D is a schematic diagram 1730 of reference vascular map of a patient.Physiological parameter prediction

[0126] As shown in method (100) of FIG. IB, a physiological parameter may be predicted based on the plurality of vascular regions (140). Physiological parameter prediction may be useful for one or more of personalized vital sign monitoring, hypertension screening, and vitalsign prediction. In some variations, the physiological parameter (e.g., blood pressure, blood glucose) may be predicted using a first machine learning model trained on, for example, a set of reference PTT values. In some variations, the machine learning model may comprise one or more of linear ridge regression, gradient boosted decision trees, Gaussian process regression, convolutional neural network, and combinations thereof. The inputs to the machine learning model may comprise one or more of PTT, BCG, correlation values, patient demographic data, and previous blood pressure values (e.g., reference physiological parameter signals). In some variations, a confidence interval (e.g., uncertainty measurement) may be generated and output with the physiological parameter prediction.

[0127] Parameter prediction methods and machine learning models suitable for use in the systems and methods here are described in U.S. Patent Application No. 18 / 068,463, filed December 19, 2022, and titled “SYSTEMS, DEVICES, AND METHODS FOR VITAL SIGN MONITORING,” which is hereby incorporated by reference in its entirety.

[0128] In some variations, physiological parameter prediction may comprise calculating one or more of systolic amplitude, pulse area, pulse interval, heart rate, time between systolic peak and end of a cardiac cycle, ratio of time before and after a systolic peak in a cardiac cycle, pulse width, maximum upslope, absorbance, Kaiser-Teager energy (KTE), signal energy, magnitude, phase, crest time, pulse interval, pulse width at half height (PWHH), Dicrotic Notch time (Tn), A2 time (A2T), diastolic time (DT), first derivative peak time (FDPT), pulse area (PA), area 1, area 2, pulse height (PH), ratio of b peak to a peak of a second derivative (b / a), ratio of e peak to a peak of the second derivative (e / a), modified Normalized Pulse Volume (mNPV), mean arterial pressure (MAP), cardiac output (CO), and total peripheral resistance (TPR).

[0129] Systolic amplitude (SA) may comprise an amplitude of the systolic peak. Pulse area (PA) may comprise a total area under a PPG curve. Pulse interval (PI) may comprise a distance between a beginning and an end of the image signal waveform. Pulse width (PW) may comprise an elapsed time between sampling points at about 0.25, about 0.5 and about 0.75 of the height of a systolic peak. Max upslope (MU) may comprise a highest value of the first derivative of the image signal. Absorbance may comprise a minus logarithm of a ratio of incident and outgoing light intensities which may be represented by a peak and valley of the image signal. KTE may comprise an instantaneous energy of a PPG signal. In some variations, the blood pressure may comprise arterial blood pressure.

[0130] Optionally, personalized vital sign monitoring may include calibrating physiological parameter predictions to a set of reference measurements. As discussed herein with respect to step 102, a physiological parameter (e.g., blood pressure) may be measured using a measurement device (e.g., sphygmomanometer) while recording the plurality of images. The predicted physiological parameter may be calibrated based on the measured physiological parameter. The calibration measurements facilitate the use of simplified models with a reduced number of parameters while increasing accuracy. For example, where a calibration machine learning model (e.g., second machine learning model) is available from step 102, a calibration value of the physiological parameter may be estimated using the second machine learning model. For example, the second machine learning model may be configured to estimate a difference between the initial physiological parameter prediction (e.g., output from a first machine learning model) and the calibrated physiological parameter value. The predicted physiological parameter may be updated using the calibration value. In some variations, personalized vital sign monitoring may include processing one or more of the representation vectors and PTT signals using representation learning (e.g., encoder architecture including a CNN, a residual network, a feed forward neural network, a UNet neural network) (137). Signal similarity (e.g., cosine similarity) may be calculated between pairs of representation vectors to generate a similarity matrix (138). An initial prediction of the physiological parameter (e.g., blood pressure) may be estimated based on the similarity matrix. For example, a weighted average of corresponding ground truth blood pressure values (when available) may be calculated as an initial blood pressure prediction (139).

[0131] The output representation vector of the representation learning may be input with a calibration representation vector, if available, to a machine learning (e.g., regression) model configured to estimate a relative difference of a physiological parameter value of the current measurement with respect to a previous measurement (141). In some variations, the physiological parameter prediction (140) may optionally include the relative difference of the physiological parameter value added to the initial physiological parameter prediction from the signal similarity calculation.

[0132] FIGS. 30A-30D are plots 3000-3030 of ground truth systolic blood pressure values for a patient and predicted systolic blood pressure values using the first and second machine learning models described herein. Similarly, FIGS. 31A-31F are plots 3100-3150 of ground truth diastolic blood pressure values for a patient and predicted diastolic blood pressure values usingthe first and second machine learning models described herein. Each plot includes a respective calibration point 3002, 3102. The upper-right quadrant and lower-left quadrants of the plots correspond to predicted blood pressure values where the fluctuations with respect to the calibration measurements are correctly estimated. The respective error lines 3004, 3104 indicate where the predicted blood pressure values have an error of less than 10 mmHg.

[0133] Additionally or alternatively, one or more of the predicted physiological parameters may be processed using one or more signal processing techniques including motion detection and outlier detection. For example, a patient may involuntarily shake or move due to high blood pressure, and may be determined to predict one or more of a risk of hypertension and a blood pressure value. In some variations, a risk of hypertension and / or blood pressure may be determined based on a rate of motion (e.g., motion blur) in the plurality of received images. In some variations, the predicted blood pressure value may be increased proportionally to the amount of patient motion and / or number of outliers detected. For example, an amount of motion (e.g., involuntary motion of about 5% of a face width) may be weighted by a predetermined value and added to a predicted blood pressure value (e.g., increase systolic blood pressure by about 5 mm Hg).

[0134] In some variations, an outlier value may be generated based on a distribution of PTT values relative to a training data set, and used to determine a risk of hypertension. For example, a higher outlier value may correspond to a higher risk of hypertension and / or higher blood pressure as the training data set generally includes a normotensive population. In some variations, an outlier value may be generated using one or more distance-based methods, clustering-based methods, and autoencoder-based methods.

[0135] FIG. 29A is a plot 2900 of PTT delay distributions for normotensive and hypertensive patients values. A difference between the normotensive PTT delay distribution 2910 and the hypertensive PTT delay distribution 2920 may be relatively small. However, a trained machine learning model may be configured to identify a shift towards lower values indicating higher blood pressure.

[0136] FIG. 29A is a plot 2950 of PTT delay distributions for a normotensive patient and a set of hypertensive patients. A difference between the normotensive PTT delay distribution 2960 and the hypertensive PTT delay distributions 2970, 2908 may be relatively larger than thedifference in FIG. 29A. For example, a degree of novelty (e.g., a distance, difference between two distributions) may be used to predict a higher blood pressure.

[0137] In some variations, a risk of hypertension may be determined based on the determined PTT values and a predetermined threshold. For example, PTT values over a first predetermined threshold may correspond to a normal blood pressure while PTT values over a second predetermined threshold may correspond to high blood pressure.Ballistocardiogram (BCG)

[0138] The physical force (e.g., impulse) of a heart beat and corresponding blood flow through the body applies physical motion to the body that may be measured using a ballistocardiogram (BCG). A physiological parameter such as heart rate and blood pressure may be determined based on a BCG signal. In some variations, a BCG signal may be obtained from both skin and non-skin (e.g., hair) regions of the body including portions covered by clothing or other objects (e.g., tattoos). Some regions of the body such as the eyebrows contain both skin and non-skin regions having both a PPG signal and a BCG signal, which may be separated using, for example, independent component analysis or principal component analysis. Higher quality BCG signals may be obtained from non-skin regions such as the eyebrows, nostrils, and lips. BCG signals are not limited to the face and may comprise non-face regions of the body such as the arm, elbow, wrist, ear, neck, chest, leg, foot, and the like. Physiological parameters may be estimated based on a local body region or a plurality of body regions.

[0139] In some variations, a BCG signal may be generated by obtaining a signal that contains both a PPG signal and a BCG signal (e.g., non-skin regions and skin regions of the head) and subtracting the PPG signal. Alternatively, a PPG signal may be processed by subtracting a BCG signal. For example, a BCG signal is substantially the same across a head of a patient such that an improved (e.g., cleaner) PPG signal may be obtained by removing the frequency components corresponding to the BCG signal. In some variations, one or more of the PPG signal and BCG signal may be used for physiological parameter prediction.

[0140] In some variations, BCG data may be used to predict a physiological parameter. FIG. 2 is a flowchart depicting an illustrative variation of a method of predicting a physiological parameter using skin and non-skin regions of a patient (200). In the variation depicted in FIG. 2, the method (200) may optionally comprise measuring a reference physiological parameter signalusing a measurement device while synchronously receiving a plurality of reference images corresponding to one or more skin regions of the patient (202) in a similar manner as described with respect to FIG. 1 A. For example, reference physiological parameter measurements (e.g., ground truth values from a gold standard measurement device) may be correlated to respective reference images, used to train a machine learning model, and used to improve physiological parameter prediction. The calibration machine learning model may be used optionally and additionally to a machine learning model configured to predict a physiological parameter value. The method (200) may comprise receiving a plurality of images corresponding to a body of the patient (210). In some variations, the body may be separated into a plurality of non-skin spatial regions using the plurality of images (220). Optionally, the body may be separated into a plurality of skin spatial regions using the plurality of images (230). Ballistocardiogram data may be generated based on the non-skin spatial regions (240). Optionally, a plurality of vascular regions may be determined based on a spectral graph of the plurality of skin spatial regions and a cardiovascular parameter (250). Photoplethysmogram data may be generated based on the skin spatial regions.

[0141] In some variations, personalized vital sign monitoring may include processing one or more of the representation vectors and PTT signals using representation learning (e.g., encoder architecture including a CNN, a residual network, a feed forward neural network, a UNet neural network) (252). Signal similarity (e.g., cosine similarity) may be calculated between pairs of representation vectors to generate a similarity matrix (254). An initial prediction of the physiological parameter (e.g., blood pressure) may be estimated based on the similarity matrix. For example, a weighted average of corresponding ground truth blood pressure values (when available) may be calculated as an initial blood pressure prediction (256).

[0142] The output representation vector of the representation learning may be input with a calibration representation vector, if available, to a machine learning (e.g., regression) model configured to estimate a relative difference of a physiological parameter value of the current measurement with respect to a previous measurement (258). In some variations, the physiological parameter prediction (260) may optionally include the relative difference of the physiological parameter value added to the initial physiological parameter prediction from the signal similarity calculation.

[0143] In some variations, a physiological parameter may be predicted based on at least the ballistocardiogram data (260). Optionally, the physiological parameter may be predicted based on both the ballistocardiogram data and photoplethysmogram (PPG) data. In some variations, the physiological parameter may comprise one or more of blood pressure and blood glucose. In some variations, the blood pressure may comprise a continuous arterial blood pressure.

[0144] In some variations, both PPG data and BCG data may be used to predict a physiological parameter. FIG. 3 is a flowchart depicting an illustrative variation of a method of predicting a physiological parameter using skin and non-skin regions of a patient (300). In the variation depicted in FIG. 3, the method (300) may optionally comprise measuring a reference physiological parameter signal using a measurement device while synchronously receiving a plurality of reference images corresponding to one or more skin regions of the patient (302) in a similar manner as described with respect to FIG. 1 A. The method (300) may comprise receiving a plurality of images corresponding to a body of the patient (310). In some variations, the body may be separated into a plurality of spatial regions using the plurality of images (320). BCG data and PPG data may be generated based on the spatial regions (330). In some variations, generating the BCG data and the PPG data may comprise one or more of independent component analysis principal component analysis. Optionally, the PPG data may be processed based on the BCG data (340). Optionally, the BCG data may be processed based on the PPG data (350). In some variations, personalized vital sign monitoring may include processing one or more of the representation vectors and PTT signals using representation learning (e.g., encoder architecture including a CNN, a residual network, a feed forward neural network, a UNet neural network) (352). Signal similarity (e.g., cosine similarity) may be calculated between pairs of representation vectors to generate a similarity matrix (354). An initial prediction of the physiological parameter (e.g., blood pressure) may be estimated based on the similarity matrix (356). For example, a weighted average of corresponding ground truth blood pressure values (when available) may be calculated as an initial blood pressure prediction. The output representation vector of the representation learning may be input with a calibration representation vector, if available, to a machine learning (e.g., regression) model configured to estimate a relative difference of a physiological parameter value of the current measurement with respect to a previous measurement (358).

[0145] In some variations, a physiological parameter may be predicted based on at least the BCG data and the PPG data (360). In some variations, the physiological parameter prediction(360) may optionally include the relative difference of the physiological parameter value added to the initial physiological parameter prediction from the signal similarity calculation. For each of the methods (200, 300) described in FIGS. 2 and 3, the methods and techniques described with respect to one more of bitrate, region selection (e.g., supervised, unsupervised), motion detection, time-series signal extraction, pulse transit time, cross-correlation, dimensionality reduction, inverted PPG signal detection, vascular map, reference measurement calibration, and physiological parameter prediction may be similarly applied as described with respect to method (100) of FIGS. lA and IB.Finger image signal

[0146] Also described here are methods of predicting a physiological parameter of a patient based on video of a finger of the patient. FIG. 18A is a schematic flowchart depicting a method of predicting a physiological parameter using a finger of a patient (1800). The method (1800) may include calibrating a camera (1810), receiving a plurality of images (1820), applying dimensionality reduction (1830), preprocessing the data (1840), inputting the data into a regression model (1850), and predicting a physiological parameter (1860) such as blood pressure and blood glucose. In some variations, the received plurality of images may be processed based on spatio-temporal or optical flow techniques. For example, FIG. 18B depicts processing the plurality of images to generate spatio-temporal data (1830) and / or optical flow data (1840), as described in more detail herein.

[0147] FIG. 4 is a flowchart depicting an illustrative variation of a method of predicting a physiological parameter using a finger of a patient (400). In the variation depicted in FIG. 4, the method (400) may optionally comprise measuring a reference physiological parameter signal using a measurement device while synchronously receiving a plurality of reference images corresponding to one or more skin regions of the patient (401) in a similar manner as described with respect to FIG. 1 A. The method (400) may optionally comprise calibrating a camera configured to generate a plurality of images (402). For example, one or more of auto exposure, sensor sensitivity, sensor exposure, auto focus, lens focus distance, auto white balance, and flash mode may be set to predetermined values. In some variations, an ISO value may be adjusted during fingertip measurement based on a predetermined threshold. In some variations, a plurality of images corresponding to a body of the patient may be received (410). As used herein, a plurality of images may refer to a video having a predetermined duration (e.g., about 8 seconds,10, second, 15 seconds, 30 seconds). Optionally, a plurality of patient illumination regions may be determined based on an illumination gradient of the plurality of images and a predetermined illumination threshold (412). In some variations, one or more of a plurality of overlapping spatial regions and a plurality of blood flow vectors may be generated using the plurality of images (420).Overlapping spatial regions

[0148] In some variations, the plurality of received images may be processed to generate a plurality of overlapping spatial regions. For example, FIG. 19 is a schematic diagram of a set of spatio-temporal grids 1900 where each temporal frame 1910, 1912, 1914, etc. includes a plurality of spatial regions 1920, 1922, 1924. In some variations, the plurality of overlapping spatial regions may comprise a two-dimensional region size and a stride size (e.g., stride length). For example, the stride size may be based on a size of the original image and a desired grid size. With respect to each received image, a two-dimensional window based on the region size may be applied across the image in increments of the stride size. Where the stride size is less than a length of the region, then the windows will overlap each other. At each window, an average value of the RGB values within the window of the image is calculated and is used as the value of a corresponding spatial region (e.g., spatial region 1920). The window is subsequently advanced by the stride size across the image and an average of the RGB values within this adjacent window is used as the value of an adjacent spatial region (e.g., spatial region 1922). These averaged RGB values are arranged in spatio-temporal grids 1910, 1912, 1914, etc. to generate a plurality of color signals (e.g., RGB video). In some variations, the plurality of color signals may comprise a red signal, a green signal, and a blue signal for each overlapping spatial region of the plurality of overlapping spatial regions. In some variations, a predetermined border (e.g., 10%) of the images may be cropped to reduce noise. With respect to method (400) of FIG. 4, the plurality of patient illumination regions may optionally be excluded from the plurality of overlapping spatial regions (422) where, for example, portions of the image are unsatisfactorily illuminated.Optical Flow

[0149] In some variations, the harmonic motion of blood flow and diffusion between capillaries and interstitial fluid may be determined from the determined using one or moreoptical flow techniques including the Lucas-Kanade Method, RAFT (Recurrent All-Pairs Field Transforms), FlowFormer, and deep equilibrium (DEQ) optical flow. For example, FIGS. 20A and 20B show images 2000, 2010 of blood flow vectors generated based on the received images. In some variations, a two scalar flow vector is generated each pixel of the plurality of images. For example, the flow vector comprises a horizontal component and a vertical component. In some variations, the flow vectors may be converted into 24 bit representations for each pixel. The 24 bit representations may be divided into three color channels (e.g., signals) of 8 bit length corresponding to RGB video. In this manner, a plurality of color signals may be generated from the plurality of blood flow vectors. Accordingly, a plurality of color signals may be generated from one or more of the plurality of overlapping spatial regions and the plurality of blood flow vectors (430) as shown in FIG. 4.Dimensionality reduction

[0150] In some variations, applying dimensionality reduction to the plurality of color signals generated from the plurality of images may reduce computational load. In some variations, the plurality of blood flow vectors may apply dimensional reduction based on one or more of averaging color signals, histograms of oriented optical flow, principal component analysis, Gaussian mixture models, and Fisher vectors (440).

[0151] With respect to method (400) of FIG. 4, a color intensity for each overlapping spatial region of the plurality of overlapping spatial regions may optionally be calculated (424). Optionally, a photopl ethy smogram (PPG) signal may be generated based on the plurality of color signals (432). For example, the color values for every spatial regions (e.g., 1920, 1922, 1924, etc.) may be averaged for each color channel at each time step to generate a PPG signal for each color channel. FIG. 21 depicts plots of color channels 2100 (red), 2110 (green), 2120 (blue). For red channel plot 2100, the red color value is plotted over time for a set of five spatial regions. These red color values may be averaged to generate a red color PPG signal.

[0152] In some variations, dimensionality reduction may be applied by computing histograms of oriented optical flow (HOOF). For example, HOOF descriptors may be generated based on blood flow vectors of two consecutive frames. HOOF descriptors represent the motion of blood and may be binned and weighted to the orientation and magnitude vectors of the blood flowvectors. For example, FIG. 20C depicts a HOOF 2020 corresponding to blood flow vectors 2000, 2010.

[0153] In some variations, Fisher Vectors (FVs) may be used to encode local features and pool as global descriptors to extract dense and compact representations from local image features. FIG. 22 is a schematic flowchart 2200 of applying dimensionality reduction to a method of predicting a physiological parameter using a finger of a patient. As shown in FIG. 22, Principal Component Analysis (PCA) may be applied to HOOF descriptors to reduce dimensionality. Gaussian Mixture Models (GMMs) may be used to cluster the reduced HOOF descriptors. For example, a GMM with k=8 components for D-dimensional HOOF descriptors may be generated after performing PCA. The PCA and GMM parameters may be used to extract a single 2Dk dimensional Fisher Vector.

[0154] With respect to method (400) of FIG. 4, the PPG signal may be optionally processed based on one or more of color channel, normalization, standardization, cropping, clipping, bandpass filtering, and mean / median filtering (434). These data pre-processing steps may improve machine learning training and modeling. For example, FIGS. 23A and 23B depict a raw PPG signal 2300 and a pre-processed PPG signal 2310.Physiological parameter prediction

[0155] In some variations, a physiological parameter may be predicted based on one or more of the plurality of overlapping spatial regions and the plurality of blood flow vectors (450). In some variations, predicting the physiological parameter (e.g., blood pressure, blood glucose) may use a machine learning model. For example, FIGS. 24A-24F are schematic flowcharts 2400, 2410, 2420, 2430, 2440, 2450 of machine learning models used to predict a physiological parameter. The inputs to the machine learning model may comprise one or more of mean RGB signals, PPG signals, patient demographic data, and previous blood pressure values. In some variations, a confidence interval may be generated and output with the physiological parameter prediction. In some variations, the physiological parameter may comprise one or more of blood pressure and blood glucose. In some variations, the physiological parameter may be output as one or more of point estimates of physiological parameter (e.g., blood pressure, blood glucose) values, point estimates with corresponding confidence intervals, binary classification output (e.g., normal, high), binary classification output with corresponding probabilities for each class,multiclass classification output (e.g., low, normal, elevated, high), and multiclass classification output with corresponding probabilities for selected classes. In some variations, the machine learning model may optionally be improved by applying on one or more of representation learning, regression model training, and model ensembling.

[0156] In some variations, personalized vital sign monitoring may include processing one or more of the representation vectors and PTT signals using representation learning (e.g., encoder architecture including a CNN, a residual network, a feed forward neural network, a UNet neural network) (442). Signal similarity (e.g., cosine similarity) may be calculated between pairs of representation vectors to generate a similarity matrix (444). An initial prediction of the physiological parameter (e.g., blood pressure) may be estimated based on the similarity matrix (446). For example, a weighted average of corresponding ground truth blood pressure values (when available) may be calculated as an initial blood pressure prediction.

[0157] The output representation vector of the representation learning may be input with a calibration representation vector, if available, to a machine learning (e.g., regression) model configured to estimate a relative difference of a physiological parameter value of the current measurement with respect to a previous measurement (448). In some variations, the physiological parameter prediction (450) may optionally include the relative difference of the physiological parameter value added to the initial physiological parameter prediction from the signal similarity calculation.Machine learning model trainingRepresentation learning

[0158] In some variations, an autoencoder neural network may be trained. For example, the machine learning model may comprise one or more of convolutional neural networks, residual networks, feed forward neural networks, UNets, variational autoencoders, transformer networks, and combinations thereof. In some variations, the weights of an encoder of the network may be weighted in a self-supervised manner where an input (e.g., mean RGB) signal is fed into the autoencoder (FIG. 24A). The input signal may include noise and error, and the network may be trained to remove the noise and output the clear ground truth signal. In variations where ground truth vital measurements (e.g., PPG, arterial blood pressure (ABP) signal, electrocardiogram (ECG) signal) from a reference measurement device are input, the autoencoder may be trained tooutput those additional signals in a supervised manner, as shown in the schematic flowchart 2440 in FIG. 24E. Furthermore, contrastive learning and vector quantization techniques may improve the quality of learned representations.Regression model training

[0159] In some variations, the machine learning model may comprise one or more of a ResNet model (ID, 2D, 3D), transformer decoder, feed forward neural network, MLP mixer, variational autoencoder decoder, Siamese networks, and combinations thereof. These machine learning models may be used to predict the absolute values of a physiological parameter (e.g., blood pressure, blood glucose). For example, an absolute blood pressure value for the current measurement may be predicted. As shown in the schematic flowcharts 2410, 2420 in respective FIGS. 24B and 24C, the input to these models may include preprocessed video features from their learned representations as discussed herein.

[0160] In some variations, a relative difference of a blood pressure value of the current measurement with respect to a previous measurement (e.g., reference measurements, ground truth measurements) may be predicted. The flowchart 2430 of FIG. 24D corresponds to a Siamese network where multiple instantiations of a machine learning model (e.g., Resnet, UNet) are synchronously trained while maintaining the same weights across each model. The input may be, for example, RGB frames generated from an optical sensor. Each model instance is fed with the data from a different session, and the final prediction of the current session is calculated based on the difference (e.g., cosine similarity) of the representations from the previous sessions. The flowchart 2450 of FIG. 24F corresponds to another variation of a Siamese network where multiple instantiations of a machine learning model (e.g., UNet) are synchronously trained while maintaining the same weights across each model. The input may be a raw (e.g., native) PPG signal such as measured by an ECG measurement device, a pulse oximeter, etc. (e.g., measurement device that does not generate RGB frames). Each model instance is fed with the data from a different session, and the final prediction of the current session is calculated based on the difference (e.g., cosine similarity) of the representations from the previous sessions.

[0161] FIG. 33 is a flowchart depicting an illustrative variation of a method of predicting a physiological parameter (3300) based on a relative difference of physiological parameter signals not including images such as RGB frames. For example, a reference physiological parametersignal may be received from non-image-based measurement device(s) to generate a reference PPG signal (e.g., from a pulse oximeter) and / or a reference ECG signal (e.g., from an ECG measurement device) (3310). Then, when a physiological parameter prediction is desired (e.g., for a current session or time period), a physiological parameter signal (e.g., PPG signal, ECG signal) may be received (3320). The received physiological parameter signal may be processed (3330) and input to a Siamese network comprising a plurality of machine learning models in order to generate corresponding representation vectors (3340). For example, respective machine learning models (e.g., UNet encoders having the same weights) may receive the current physiological parameter signal and the reference physiological parameter signal as shown in FIG. 24F where each model instance is fed with the data from a different session. Signal similarity (e.g., cosine similarity) may be calculated using the representation vectors (3350), which may then be input to a regression model to predict the physiological parameter (3360).Model ensembling

[0162] In some variations, the accuracy of a machine learning model may be further improved using model ensembling techniques. In some variations, representation vectors calculated in the middle layers of the trained models may be concatenated in a single vector to obtain a single representation entity of the input. Then, a partial least squares regression transformation may be applied to the vector to maintain the dimensionality at a constant length. In some variations, the resulting vectors may be combined with patient demographic data and input to the ecnsemble model. In some variations, a machine learning model (e.g., ensemble model) may comprise one or more of XGBoost, SVM, linear regression, logistic regression, decision trees, random forests, K-nearest neighbor, Gaussian process, and combinations thereof.II. DevicesOverview

[0163] Also described here are systems that may include one or more of the components used to predict a physiological parameter. Generally, described herein is an artificial intelligence (Al) environment configured to process patient data and predict a set of vital signs. The Al environment may be accessible from a plurality of configurations such as a mobile platform (e.g., accessible through a mobile application executable on a mobile computing device such as a smartphone) as well as a web-based platform (e.g., accessible through a web browser on a laptopor desktop computing device). In these variations, a user may interact with the mobile and webbased platforms interchangeably. Furthermore, the Al environment may include a system of applications that allows services (e.g., web conference, web browser, telehealth) to integrate physiological parameter predictions in real-time for a set of users (e.g., patients, doctors).

[0164] FIG. 25 is a block diagram of a computing device (2510) comprising the computing device (2510) may comprise one or more of a display (2512), processor (2514), memory (2516), machine learning model(s) (2518), optical sensor (2520), audio sensor (2522), pressure sensor (2524), input device (2528), communication device (2530), and optional illumination source (2532).Display

[0165] Patient data and physiological parameter predictions may be output on a display (e.g., display (2512)) of a computing device. In some variations, a display may include at least one of a light emitting diode (LED), liquid crystal display (LCD), electroluminescent display (ELD), plasma display panel (PDP), thin film transistor (TFT), organic light emitting diodes (OLED), electronic paper / e-ink display, laser display, and / or holographic display.Processor

[0166] The processor (e.g., processor (2514)) described here may process data and / or other signals to control one or more components of the computing device. The processor may be configured to receive, process, compile, compute, predict, store, access, read, write, and / or transmit data and / or other signals. Additionally, or alternatively, the processor may be configured to control one or more components of a device and / or one or more components of computing device (e.g., laptop, tablet, personal computer).

[0167] In some variations, the processor may be configured to access or receive patient data, machine learning model training model set, and / or sensor signals from one or more of a computing device, and a storage medium (e.g., memory, flash drive, memory card). In some variations, the processor may be any suitable processing device configured to run and / or execute a set of instructions or code and may include one or more data processors, image processors, graphics processing units (GPU), physics processing units, digital signal processors (DSP), analog signal processors, mixed-signal processors, machine learning processors, deep learningprocessors, finite state machines (FSM), compression processors (e.g., data compression to reduce data rate and / or memory requirements), encryption processors (e.g., for secure wireless data transfer), and / or central processing units (CPU). The processor may be, for example, a general purpose processor, Field Programmable Gate Array (FPGA), an Application Specific Integrated Circuit (ASIC), a processor board, and / or the like. The processor may be configured to run and / or execute application processes and / or other modules, processes and / or functions associated with the system. The underlying device technologies may be provided in a variety of component types (e.g., metal-oxide semiconductor field-effect transistor (MOSFET) technologies like complementary metal-oxide semiconductor (CMOS), bipolar technologies like emitter-coupled logic (ECL), polymer technologies (e.g., silicon-conjugated polymer and metal- conjugated polymer-metal structures), mixed analog and digital, and the like.

[0168] The systems, devices, and / or methods described herein may be performed by software (executed on hardware), hardware, or a combination thereof. Hardware modules may include, for example, a general-purpose processor (or microprocessor or microcontroller), a field programmable gate array (FPGA), and / or an application specific integrated circuit (ASIC). Software modules (executed on hardware) may be expressed in a variety of software languages (e.g., computer code), including C, C++, Java®, Python, Ruby, Visual Basic®, and / or other object-oriented, procedural, or other programming language and development tools. Examples of computer code include, but are not limited to, micro-code or micro-instructions, machine instructions, such as produced by a compiler, code used to produce a web service, and files containing higher-level instructions that are executed by a computer using an interpreter. Additional examples of computer code include, but are not limited to, control signals, encrypted code, and compressed code.Memory

[0169] The computing devices described here may include a memory (e.g., memory (2516)) configured to store data and / or information. In some variations, the memory may include one or more of a random access memory (RAM), static RAM (SRAM), dynamic RAM (DRAM), a memory buffer, an erasable programmable read-only memory (EPROM), an electrically erasable read-only memory (EEPROM), a read-only memory (ROM), flash memory, volatile memory, non-volatile memory, combinations thereof, and the like. In some variations, the memory may store instructions to cause the processor to execute modules, processes, and / or functionsassociated with the device, such as image processing, image display, data and / or signal transmission, data and / or signal reception, and / or communication. Some variations described herein may relate to a computer storage product with a non-transitory computer-readable medium (also may be referred to as a non-transitory processor-readable medium) having instructions or computer code thereon for performing various computer-implemented operations. The computer-readable medium (or processor-readable medium) is non-transitory in the sense that it does not include transitory propagating signals per se (e.g., a propagating electromagnetic wave carrying information on a transmission medium such as space or a cable). The media and computer code (also may be referred to as code or algorithm) may be those designed and constructed for the specific purpose or purposes.

[0170] In some variations, the memory may be configured to store any received data and / or data generated by the device. In some variations, the device may be configured to store graph data (e.g., second graph data nodes, user graph data, user activity, user preferences, and user input. In some variations, the memory may be configured to store data temporarily or permanently.Optical Sensor

[0171] In some variations, an optical sensor may comprise one or more of a camera, photodetector, a photodiode, charged coupled device (CCD) or complementary metal-oxide semiconductor (CMOS) optical sensor, and an optical lens assembly. In some embodiments, the optical sensor may be configured to generate an image signal having a resolution of at least about 640 by 360 pixels and at least about 24 frames per second video.

[0172] In some variations, an illumination source may include one or more of a light emitter and / or an optical waveguide. Non-limiting examples of a light emitter include incandescent, electric discharge (e.g., excimer lamp, fluorescent lamp, electrical gas-discharge lamp, plasma lamp, etc.), electroluminescence (e.g., light-emitting diodes, organic light-emitting diodes, laser, etc.), induction lighting, and fiber opticsInput Device

[0173] In some variations, the display may include and / or be operatively coupled to an input device (2528) (e.g., touch screen) configured to receive input data from a user. For example, userinput to an input device (2528) (e.g., keyboard, buttons, touch screen) may be received and processed by a processor (e.g., processor (2514)) and memory (e.g., memory (2516)) of the visualization system. The input device may include at least one switch configured to generate a user input. For example, an input device may include a touch surface for a user to provide input (e.g., finger contact to the touch surface) corresponding to a user input. An input device including a touch surface may be configured to detect contact and movement on the touch surface using any of a plurality of touch sensitivity technologies including capacitive, resistive, infrared, optical imaging, dispersive signal, acoustic pulse recognition, and surface acoustic wave technologies. In variations of an input device including at least one switch, a switch may have, for example, at least one of a button (e.g., hard key, soft key), touch surface, keyboard, analog stick (e.g., joystick), directional pad, mouse, trackball, jog dial, step switch, rocker switch, pointer device (e.g., stylus), motion sensor, image sensor, and microphone. A motion sensor may receive user movement data from an optical sensor and classify a user gesture as a user input. An audio sensor 2523 such as a microphone may receive audio data and recognize a user voice as a user input.

[0174] In some variations, the computing system may optionally include one more output devices in addition to the display, such as, for example, an audio device and haptic device. An audio device may audibly output any system data, alarms, and / or notifications. For example, the audio device may output an audible alarm when a malfunction is detected. In some variations, an audio device may include at least one of a speaker, piezoelectric audio device, magnetostrictive speaker, and / or digital speaker. In some variations, a user may communicate with other users using the audio device and a communication channel. For example, a user may form an audio communication channel (e.g., VoIP call).

[0175] Additionally or alternatively, the system may include a haptic device configured to provide additional sensory output (e.g., force feedback) to the user. For example, a haptic device may generate a tactile response (e.g., vibration) to confirm user input to an input device (2528) (e.g., touch surface). As another example, haptic feedback may notify that user input is overridden by the processor.Communication Device

[0176] In some variations, the computing device may include a communication device (e.g., communication device (2530)) configured to communicate with another computing device and one or more databases. The communication device may be configured to connect the computing device to another system (e.g., Internet, remote server, graph database, media database) by wired or wireless connection. In some variations, the system may be in communication with other devices via one or more wired and / or wireless networks. In some variations, the communication device may include a radiofrequency receiver, transmitter, and / or optical (e.g., infrared) receiver and transmitter configured to communicate with one or more devices and / or networks. The communication device may communicate by wires and / or wirelessly.

[0177] The communication device may include RF circuitry configured to receive and send RF signals. The RF circuitry may convert electrical signals to / from electromagnetic signals and communicate with communications networks and other communications devices via the electromagnetic signals. The RF circuitry may include well-known circuitry for performing these functions, including but not limited to an antenna system, an RF transceiver, one or more amplifiers, a tuner, one or more oscillators, a digital signal processor, a CODEC chipset, a subscriber identity module (SIM) card, memory, and so forth.

[0178] Wireless communication through any of the devices may use any of plurality of communication standards, protocols and technologies, including but not limited to, Global System for Mobile Communications (GSM), Enhanced Data GSM Environment (EDGE), highspeed downlink packet access (HSDPA), high-speed uplink packet access (HSUPA), Evolution, Data-Only (EV-DO), HSPA, HSPA+, Dual-Cell HSPA (DC-HSPDA), long term evolution (LTE), near field communication (NFC), wideband code division multiple access (W-CDMA), code division multiple access (CDMA), time division multiple access (TDMA), Bluetooth, Wireless Fidelity (WiFi) (e.g., IEEE 802.11a, IEEE 802.11b, IEEE 802.11g, IEEE 802.1 In, and the like), voice over Internet Protocol (VoIP), Wi-MAX, a protocol for e-mail (e.g., Internet message access protocol (IMAP) and / or post office protocol (POP)), instant messaging (e.g., extensible messaging and presence protocol (XMPP), Session Initiation Protocol for Instant Messaging and Presence Leveraging Extensions (SIMPLE), Instant Messaging and Presence Service (IMPS)), and / or Short Message Service (SMS), or any other suitable communication protocol. In some variations, the devices herein may directly communicate with each other without transmitting data through a network (e.g., through NFC, Bluetooth, WiFi, RFID, and the like).

[0179] In some variations, the systems, devices, and methods described herein may be in communication with other wireless devices via, for example, one or more networks, each of which may be any type of network (e.g., wired network, wireless network). The communication may or may not be encrypted. A wireless network may refer to any type of digital network that is not connected by cables of any kind. Examples of wireless communication in a wireless network include, but are not limited to cellular, radio, satellite, and microwave communication. However, a wireless network may connect to a wired network in order to interface with the Internet, other carrier voice and data networks, business networks, and personal networks. A wired network is typically carried over copper twisted pair, coaxial cable and / or fiber optic cables. There are many different types of wired networks including wide area networks (WAN), metropolitan area networks (MAN), local area networks (LAN), Internet area networks (IAN), campus area networks (CAN), global area networks (GAN), like the Internet, and virtual private networks (VPN). Hereinafter, network refers to any combination of wireless, wired, public and private data networks that are typically interconnected through the Internet, to provide a unified networking and information access system.

[0180] Cellular communication may encompass technologies such as GSM, PCS, CDMA or GPRS, W-CDMA, EDGE or CDMA2000, LTE, WiMAX, and 5G networking standards. Some wireless network deployments combine networks from multiple cellular networks or use a mix of cellular, Wi-Fi, and satellite communication.

[0181] In some variations, a system may comprise an optical sensor configured to generate one or more image signals corresponding to a skin of the patient, a memory, a processor operatively coupled to the memory and the optical sensor. The processor may be configured to receive one or more image signals corresponding to a skin of the patient using the optical sensor, process the one or more image signals using a first machine learning model, and predict a physiological parameter based on the processed one or more image signals using a second machine learning model.

[0182] In some variations, a pressure sensor may be configured to measure finger pressure against the optical sensor. In some variations, an audio sensor may be configured to measure patient audio. In some variations, the system may comprise a handheld housing. Processing the one or more image signals and predicting the physiological parameter may be performed within the handheld housing. In some variations, a communication device and a display may beoperatively coupled to the processor. The processor may be configured to establish a video conference using the communication device, and output the predicted physiological parameter using the display during the video conference. In some variations, a communication device may be operatively coupled to the processor. The processor may be configured to transmit the predicted physiological parameter to a predetermined device using the communication device.Graphical User Interface

[0183] FIGS. 26-28 illustrate variations of graphical user interfaces comprising real-time physiological parameter predictions. FIG. 26 depicts a graphical user interface (GUI) (2600) relating to patient monitoring using image data of a finger of a patient. In some variations, the patient may be instructed to place their finger over an optical sensor so as to cover a smartphone camera (e.g., front-facing camera, rear-facing camera). In some variations, an illumination source (e.g., flashlight) of a computing device (e.g., smartphone) may be configured to illuminate the patient (e.g., finger, face) based on light conditions. In some variations, the GUI (2600) may comprise an optical sensor display region (2610) configured to output the image signal (e.g., video) recorded by the optical sensor in real-time. In some variations, the GUI (2600) may comprise one or more physiological parameter predictions (2620, 2630, 2640, 2650, 2660, 2670). For example, the predictions may be overlaid above the optical sensor display region (2610). As shown in FIG. 26, the GUI (2600) may include, but is not limited to, a heart rate display region (2620), a heart rate variability display region (2630), a respiratory rate display region (2640), an oxygen saturation display region (2650), a blood pressure display region (2660), and a cough display region (2670). Furthermore, any of the GUIs described herein may include a glucose display region (not shown). In some variations, the physiological parameter predictions may be output in the GUI (2600) using one or more alphanumeric characters, symbols, colors, image, and graphics. For example, a patient’s oxygen saturation level may be output as a percentage value in the oxygen saturation display region (2650) with color coding (e.g., green, yellow, red) indicating corresponding health status.

[0184] In some variations, a contact pressure display region (2612) may be configured to guide the patient to apply finger pressure against the optical sensor within a predetermined range. For example, the contact pressure display region (2612) may output measured contact pressure against the optical sensor relative to a predetermined scale (e.g., too low, low, optimum, high, too high) in real-time.

[0185] Additionally or alternatively, a contact pressure and / or a predicted physiological parameter may be communicated using a set of light patterns. The light patterns described herein may, for example, comprise one or more of flashing light, occulting light, isophase light, etc., and / or light of any suitable light / dark pattern. Light pulse patterns may include one or more colors (e.g., different color output per pulse), light intensities, and frequencies. Additionally or alternatively, a measured contact pressure may be output using respective audio and haptic devices. For example, a speaker of a computing device may audibly beep when the finger pressure applied to the optical sensor is within a predetermined optimal range of contact pressure. A haptic motor of the computing device may vibrate when the finger pressure applied to the optical sensor is outside the predetermined optimal range. Contact pressure measurements outside the predetermined range may include noise that reduce the accuracy of a physiological parameter prediction. In some variations, a physiological parameter prediction may be performed only when the measured contact pressure against the optical sensor is within a predetermined range. Additionally or alternatively, one or more of the predicted physiological parameters may be output using respective audio and haptic devices.

[0186] FIG. 27 depicts a graphical user interface (2700) relating to patient monitoring using a face of a patient. In some variations, the patient may be instructed to position their face within range of an optical sensor such as a smartphone camera, web cam, and the like. In some variations, the GUI (2700) may comprise an optical sensor display region (2710) configured to output the image signal (e.g., video) recorded by the optical sensor in real-time. In some variations, the GUI (2700) may comprise one or more physiological parameter predictions (2720, 2730, 2740, 2750, 2760, 2770). For example, the predictions may be overlaid above the optical sensor display region (2710). As shown in FIG. 27, the GUI (2700) may comprise a heart rate display region (2720), a heart rate variability display region (2730), a respiratory rate display region (2740), an oxygen saturation display region (2750), a blood pressure display region (2760), and a cough display region (2770). In some variations, the physiological parameter predictions may be output in the GUI (2700) using one or more alphanumeric characters, symbols, colors, image, and graphics in a similar manner as described with respect to GUI (2600).

[0187] In some variations, a face distance display region (2712) may be configured to guide the patient to position their face within a predetermined distance of the optical sensor. For example, a predetermined portion (e.g., between about 30% and 60%) of an image can includethe face. For example, the face distance display region (2712) may output a scale of distances relative to the optical sensor (e.g., too far, far, optimum, close, too close) in real-time. Additionally or alternatively, a face distance and / or a predicted physiological parameter may be communicated using one or more visual, audio, and haptic methods in a similar manner as described with respect to GUI (600). In some variations, face distance measurements outside the predetermined range may include noise or not contain sufficient information so as to reduce the accuracy of a physiological parameter prediction. In some variations, a physiological parameter prediction may be performed only when the measured face distance is within a predetermined range. As described in more detail herein, the methods described herein may be configured to differentiate between a face (2713) of the patient and background (2711).

[0188] FIG. 28 depicts a graphical user interface (2800) relating to patient monitoring on a video conference. In some variations, the patient may be instructed to position their face within range of an optical sensor such as a web cam, smartphone camera, and the like. In some variations, the GUI (2800) may comprise an optical sensor display region (2810) configured to output the image signal (e.g., video) recorded by the optical sensor in real-time. In some variations, the GUI (2800) may comprise one or more physiological parameter predictions (2820, 2830, 2840, 2850, 2860, 2870). For example, the predictions may be overlaid above the optical sensor display region (2810). As shown in FIG. 28, the GUI (2800) may comprise a heart rate display region (2820), a heart rate variability display region (2830), a respiratory rate display region (2840), an oxygen saturation display region (2850), a blood pressure display region (2860), and a cough display region (2870). In some variations, the physiological parameter predictions may be output in the GUI (2800) using one or more alphanumeric characters, symbols, colors, image, and graphics in a similar manner as described with respect to GUIs (2600, 2700).

[0189] In some variations, a face distance display region (2812) may be configured to guide the patient to position their face within a predetermined range of the optical sensor. For example, the face distance display region (2812) may output a scale of face distances relative to the optical sensor (e.g., too far, far, optimum, close, too close) in real-time. Additionally or alternatively, a face distance and / or a predicted physiological parameter may be communicated using one or more visual, audio, and haptic methods in a similar manner as described with respect to GUIs (600, 700, 800). In some variations, a physiological parameter prediction may be performed only when the measured face distance is within a predetermined range. As described in more detailherein, the methods described herein may be configured to differentiate between a face (2813) of the patient and a background (2811). In some variations, a web conference application such as a web browser may perform a majority (e.g., about 80%) of the computational load locally. Additionally or alternatively, one or more portions of processing and / or prediction may be encrypted and transmitted for remote processing and / or prediction.

[0190] Optionally, a trend of one or more of the physiological parameters may be generated based on a set of physiological parameters predicted over time. For example, a plot of a patient’s blood pressure over time may be generated and output on a display. This may enable a patient’s health status to be monitored and analyzed over time for one or more of diagnosis and treatment. In some variations, the predicted physiological parameter data may be merged with other data sets (e.g., nutrition, drug, activity, etc.). Trend analysis of one or more data sets may provide holistic insight for one or more of a patient, health care professional, family, support group of the patient’s health over time. For example, a thirty second selfie video of the face may be used to assess a change in a patient’s real-time health status or if there is an increased risk for an adverse event that may be notified to the patient’s health care professional. In some variations, a GUI may be configured to output one or more of a real-time physiological parameter prediction, historical physiological parameter prediction, and physiological parameter trends (e.g., lower blood pressure).

[0191] In some cases, the patient’s predicted data and trends may be used by one or more of the patient and health care professional to take action to improve health outcomes. The results of the analysis may be used to generate one or more prompts to output to the patient and / or health care professional. For example, data analysis showing that a patient is exhibiting a potentially dangerous trend may be the basis to output a GUI notification advising them to schedule an appointment with a health care professional, change diet habits, and / or add an activity notification through their computing device to encourage more physical activity. Additionally or alternatively (e.g., concurrently), a GUI notification may be output to the patient’s health care professional, family members, and / or other support group notifying them of the patient’s condition and optionally suggesting appropriate intervention steps. As another example, data analysis showing that a patient is on a positive trend may be used to generate a prompt providing positive reinforcement to the patient.

[0192] While various inventive variations have been described and illustrated herein, those of ordinary skill in the art will readily envision a variety of other means and / or structures for performing the function and / or obtaining the results and / or one or more of the advantages described herein, and each of such variations and / or modifications is deemed to be within the scope of the inventive embodiments / variations described herein. More generally, those skilled in the art will readily appreciate that all parameters, dimensions, materials, and configurations described herein are meant to be exemplary and that the actual parameters, dimensions, materials, and / or configurations will depend upon the specific application or applications for which the inventive teachings is / are used. Those skilled in the art will recognize, or be able to ascertain using no more than routine experimentation, many equivalents to the specific inventive variations described herein. It is, therefore, to be understood that the foregoing variations are presented by way of example only and that, within the scope of the appended claims and equivalents thereto, inventive variations may be practiced otherwise than as specifically described and claimed. Inventive variations of the present disclosure are directed to each individual feature, system, article, material, kit, and / or method described herein. In addition, any combination of two or more such features, systems, articles, materials, kits, and / or methods, if such features, systems, articles, materials, kits, and / or methods are not mutually inconsistent, is included within the inventive scope of the present disclosure.

Claims

CLAIMS1. A method of predicting a physiological parameter of a patient, comprising: receiving a plurality of images corresponding to one or more skin regions of the patient; separating the one or more skin regions into a plurality of spatial regions using the plurality of images; determining a plurality of vascular regions based on a spectral graph of the plurality of spatial regions and a cardiovascular parameter; and predicting the physiological parameter based on the plurality of vascular regions.

2. The method of claim 1, wherein the plurality of images are generated by one or more cameras.

3. The method of claim 1, wherein the plurality of images comprise a video.

4. The method of claim 3, further comprising prioritizing higher bitrate over one or more of higher frame rate and higher resolution of the video.5 The method of claim 3, further comprising maximizing a bitrate value of the video.

6. The method of claim 1, further comprising: determining a plurality of patient motion regions based on the plurality of images and a predetermined motion threshold; and excluding the plurality of patient motion regions from the plurality of spatial regions.

7. The method of claim 6, wherein determining the plurality of patient motion regions is based on one or more of landmark tracking, optical flow, and difference imaging.

8. The method of claim 1, further comprising: determining a plurality of patient illumination regions based on an illumination gradient of the plurality of images and a predetermined illumination threshold; andexcluding the plurality of patient illumination regions from the plurality of spatial regions.

9. The method of claim 1, wherein separating the one or more skin regions is based on a neural network.

10. The method of claim 1, wherein determining the plurality of vascular regions comprises: generating the spectral graph comprising nodes and edges, the nodes corresponding to the plurality of spatial regions and the edges corresponding to similarity to the cardiovascular parameter; generating a plurality of time-series signals based on the spectral graph; and identifying a plurality of periodic heart rate signals corresponding to the plurality of time-series signals, wherein the plurality of vascular regions correspond to the identified periodic heart rate signals.

11. The method of claim 10, wherein the similarity to the cardiovascular parameter comprises one or more of a maximum value of time-series cross-correlation and a time shift that maximizes time-series cross-correlation.

12. The method of claim 10, further comprising transmitting the plurality of time-series signals to a remote processing device, wherein predicting the physiological parameter is performed by the remote processing device.

13. The method of claim 1, wherein determining the plurality of vascular regions is based on correlation to a set of predetermined spatial regions.

14. The method of claim 1, wherein determining the plurality of vascular regions is based on one or more of a dominant frequency, maximal variation, a correlation coefficient, a crosscorrelation among a set of cardiac cycles within a predetermined time period, cycle-by-cycle validation, bandpass filtering, smoothness, motion artifact removal, session filtering, and power spectrum.

15. The method of claim 1, further comprising generating a plurality of color signals from the plurality of images based on the plurality of vascular regions.

16. The method of claim 15, wherein the plurality of color signals comprises a red signal, a green signal, and a blue signal for each vascular region of the plurality of vascular regions.

17. The method of claim 15, further comprising generating a photoplethysmogram (PPG) signal based on the plurality of color signals for each vascular region of the plurality of vascular regions.

18. The method of claim 17, wherein the PPG signal comprises an absorption PPG signal and a reflection PPG signal.

19. The method of claim 17, wherein predicting the physiological parameter comprises generating a plurality of pulse transit times (PTT) based on the PPG signals corresponding to the set of vascular regions.

20. The method of claim 19, further comprising determining a set of PTT values between a first vascular region, a second vascular region, and a third vascular region; and excluding the set of PTT values based on a predetermined threshold.

21. The method of claim 17, further comprising aggregating the plurality of PPG signals corresponding to a set of vascular regions of the plurality of vascular regions.

22. The method of claim 21, wherein aggregating the plurality of PPG signals comprises averaging the plurality of PPG signals.

23. The method of claim 1, wherein predicting the physiological parameter uses a first machine learning model.

24. The method of claim 23, wherein the first machine learning model comprises one or more of linear ridge regression, gradient boosted decision trees, Gaussian process regression, and combinations thereof.

25. The method of claim 1, further comprising estimating a calibration value of the physiological parameter using a second machine learning model.

26. The method of claim 25, further comprising updating the predicted physiological parameter using the calibration value.

27. The method of claim 26, further comprising measuring a reference physiological parameter signal using one or more measurement devices while synchronously receiving a plurality of reference images corresponding to one or more skin regions of the patient, wherein the second machine learning model is trained on the measured reference physiological parameter signal and the synchronously recorded plurality of reference images.

28. The method of claim 27, further wherein measuring the reference physiological signal and synchronously recording the plurality of reference images is performed at a first time period and a second time period different from the first time period.

29. The method of claim 27, wherein the one or more measurement devices comprise one or more of a blood pressure measurement device, an optical sensor, a pulse oximeter, and an ECG measurement device.

30. The method of claim 29, wherein the optical sensor comprises one or more of an infrared sensor, a thermal sensor, and an RGB sensor.

31. The method of claim 1, wherein the physiological parameter prediction comprises a confidence interval.

32. The method of claim 1, wherein the physiological parameter comprises blood pressure or blood glucose.

33. The method of claim 32, wherein predicting the physiological parameter comprises calculating one or more of systolic amplitude, pulse area, pulse interval, heart rate, time between systolic peak and end of a cardiac cycle, ratio of time before and after a systolic peak in a cardiac cycle, pulse width, maximum upslope, absorbance, Kaiser-Teager energy, signal energy, magnitude, phase, crest time, pulse interval, pulse width at half height (PWHH), Dicrotic Notch time (Tn), A2 time (A2T), diastolic time (DT), first derivative peak time (FDPT), pulse area (PA), area 1, area 2, pulse height (PH), ratio of b peak to a peak of a second derivative (b / a), ratio of e peak to a peak of the second derivative (e / a), modified Normalized Pulse Volume (mNPV), mean arterial pressure (MAP), cardiac output (CO), and total peripheral resistance (TPR).

34. The method of claim 32, wherein the blood pressure comprises a continuous arterial blood pressure.

35. The method of claim 1, further comprising generating a vascular map of the patient based on the plurality of vascular regions.

36. The method of claim 1, further comprising: establishing a video conference using a communication device; and outputting the predicted physiological parameter using a display during the video conference.

37. The method of claim 1, further comprising transmitting the predicted physiological parameter to a communication device.

38. A method of predicting a physiological parameter of a patient, comprising: receiving a plurality of images corresponding to a body of the patient; separating the body into a plurality of non-skin spatial regions using the plurality of images; generating ballistocardiogram data based on the non-skin spatial regions; and predicting the physiological parameter based on at least the ballistocardiogram data.

39. The method of claim 38, further comprising: separating the body into a plurality of skin spatial regions using the plurality of images; determining a plurality of vascular regions based on a spectral graph of the plurality of skin spatial regions and a cardiovascular parameter, wherein predicting the physiological parameter is based on the plurality of vascular regions and the ballistocardiograph data.

40. The method of claim 38, wherein the plurality of images comprise a video.

41. The method of claim 40, further comprising prioritizing higher bitrate over one or more of higher frame rate and higher resolution of the video.

42. The method of claim 40, further comprising maximizing a bitrate value of the video.

43. The method of claim 38, wherein predicting the physiological parameter uses a first machine learning model.

44. The method of claim 43, wherein the first machine learning model comprises one or more of linear ridge regression, gradient boosted decision trees, Gaussian process regression, and combinations thereof.

45. The method of claim 38, further comprising estimating a calibration value of the physiological parameter using a second machine learning model.

46. The method of claim 45, further comprising updating the predicted physiological parameter using the calibration value.

47. The method of claim 46, further comprising measuring a reference physiological parameter signal using one or more measurement devices while synchronously receiving a plurality of reference images corresponding to one or more skin regions of the patient, wherein the second machine learning model is trained on the measured reference physiological parameter signal and the synchronously recorded plurality of reference images.

48. The method of claim 47, further wherein measuring the reference physiological signal and synchronously recording the plurality of reference images is performed at a first time period and a second time period different from the first time period.

49. The method of claim 47, wherein the one or more measurement devices comprise one or more of a blood pressure measurement device, an optical sensor, a pulse oximeter, and an ECG measurement device.

50. The method of claim 48, wherein the optical sensor comprises one or more of an infrared sensor, a thermal sensor, and an RGB sensor.

51. The method of claim 38, wherein the physiological parameter prediction comprises a confidence interval.

52. The method of claim 38, wherein the physiological parameter comprises blood pressure or blood glucose.

53. The method of claim 52, wherein predicting the physiological parameter comprises calculating one or more of systolic amplitude, pulse area, pulse interval, heart rate, time between systolic peak and end of a cardiac cycle, ratio of time before and after a systolic peak in a cardiac cycle, pulse width, maximum upslope, absorbance, Kaiser-Teager energy, signal energy, magnitude, phase, crest time, pulse interval, pulse width at half height (PWHH), Dicrotic Notch time (Tn), A2 time (A2T), diastolic time (DT), first derivative peak time (FDPT), pulse area (PA), area 1, area 2, pulse height (PH), ratio of b peak to a peak of a second derivative (b / a), ratio of e peak to a peak of the second derivative (e / a), modified Normalized Pulse Volume (mNPV), mean arterial pressure (MAP), cardiac output (CO), and total peripheral resistance (TPR).

54. The method of claim 52, wherein the blood pressure comprises a continuous arterial blood pressure.

55. The method of claim 38, further comprising:establishing a video conference using a communication device; and outputting the predicted physiological parameter using a display during the video conference.

56. The method of claim 38, further comprising transmitting the predicted physiological parameter to a communication device.

57. A method of predicting a physiological parameter of a patient, comprising: receiving a plurality of images corresponding to a body of the patient; separating the body into a plurality of spatial regions using the plurality of images; generating ballistocardiogram (BCG) data and photoplethysmogram (PPG) data based on the plurality of spatial regions; and predicting the physiological parameter based on the PPG data and the BCG data.

58. The method of claim 57, wherein generating the BCG data and the PPG data comprises one or more of independent component analysis principal component analysis.

59. The method of claim 57, further comprising: processing the PPG data based on the BCG data; and processing the BCG data based on the PPG data.

60. A method of predicting a physiological parameter of a patient, comprising: receiving a plurality of images corresponding to a finger of the patient; generating one or more of a plurality of overlapping spatial regions and a plurality of blood flow vectors using the plurality of images; and predicting the physiological parameter based on one or more of the plurality of overlapping spatial regions and the plurality of blood flow vectors.

61. The method of claim 60, wherein the plurality of overlapping spatial regions comprise a region size and a stride size.

62. The method of claim 60, further comprising generating a plurality of color signals from one or more of the plurality of overlapping spatial regions and the plurality of blood flow vectors.

63. The method of claim 62, wherein the plurality of color signals comprise a red signal, a green signal, and a blue signal for each overlapping spatial region of the plurality of overlapping spatial regions.

64. The method of claim 60, further comprising calculating a color intensity for each overlapping spatial region of the plurality of overlapping spatial regions.

65. The method of claim 60, further comprising processing the plurality of blood flow vectors based on one or more of histograms of oriented optical flow, principal component analysis, Gaussian mixture models, and Fisher vectors.

66. The method of claim 62, further comprising generating a photoplethysmogram (PPG) signal based on the plurality of color signals.

67. The method of claim 66, wherein the PPG signal comprises an absorption PPG signal and a reflection PPG signal.

68. The method of claim 67, further comprising processing the PPG signal based on one or more of color channel, normalization, standardization, cropping, clipping, bandpass filtering, and mean / median filtering.

69. The method of claim 68, wherein the physiological parameter comprises blood pressure or blood glucose.

70. The method of claim 60, wherein the plurality of images comprise a video.

71. The method of claim 70, further comprising prioritizing higher bitrate over one or more of higher frame rate and higher resolution of the video.

72. The method of claim 70, further comprising maximizing a bitrate value of the video.

73. The method of claim 60, further comprising: determining a plurality of patient illumination regions based on an illumination gradient of the plurality of images and a predetermined illumination threshold; and excluding the plurality of patient illumination regions from the plurality of overlapping spatial regions.

74. The method of claim 60, further comprising calibrating a camera configured to generate the plurality of images.

75. The method of claim 74, wherein calibrating the camera comprises one or more of exposure, sensitivity, focus, focus distance, white balance, and lighting.

76. The method of claim 74, wherein the physiological parameter prediction comprises a confidence interval.

77. The method of claim 74, wherein predicting the physiological parameter uses a first machine learning model.

78. The method of claim 77, wherein the first machine learning model comprises one or more of convolutional neural networks, residual networks, feed forward neural networks, UNets, variational autoencoders, transformer networks, and combinations thereof.

79. The method of claim 77, wherein the first machine learning model comprises one or more of a ResNet model, transformer decoder, feed forward neural network, MLP mixer, variational autoencoder decoder, Siamese networks, and combinations thereof.

80. The method of claim 77, wherein the first machine learning model comprises one or more of XGBoost, SVM, linear regression, logistic regression, decision trees, random forests, K- nearest neighbor, Gaussian process, and combinations thereof.

81. The method of claim 60, further comprising estimating a calibration value of the physiological parameter using a second machine learning model.

82. The method of claim 81, further comprising updating the predicted physiological parameter using the calibration value.

83. The method of claim 82, further comprising measuring a reference physiological parameter signal using one or more measurement devices while synchronously receiving a plurality of reference images corresponding to one or more skin regions of the patient, wherein the second machine learning model is trained on the measured reference physiological parameter signal and the synchronously recorded plurality of reference images.

84. The method of claim 83, further wherein measuring the reference physiological signal and synchronously recording the plurality of reference images is performed at a first time period and a second time period different from the first time period.

85. The method of claim 83, wherein the one or more measurement devices comprise one or more of a blood pressure measurement device, an optical sensor, a pulse oximeter, and an ECG measurement device.

86. The method of claim 85, wherein the optical sensor comprises one or more of an infrared sensor, a thermal sensor, and an RGB sensor.

Images