Remote blood pressure monitoring system and method based on camera

Abstract

In the present invention, in order to extract systolic and diastolic pressure estimates of a subject, the system first begins to capture a color video of the subject. Then, computer vision techniques are applied to the video frames to locate regions of interest of the face and continuously track these regions for a period of time. And then, inputting the positioned image of the region of interest into a processing flow with image processing, signal processing and a machine learning algorithm, so that data can be created based on the rPPG signal. Wherein the rPPG signal, its derivatives of different orders, and the heart rate estimated from the rPPG signal, can both be input into a machine learning model to estimate / predict systolic pressure and diastolic pressure of the subject.

Description

Cross-reference of related applications

[0001] This application claims priority to U.S. Provisional Patent Application No. 63 / 685,266, filed August 21, 2024, the disclosure of which is incorporated herein by reference in its entirety. Technical Field

[0002] This invention relates to the field of computer vision, and is used to extract physiologically relevant information from images, and then use machine learning models to estimate the systolic and diastolic blood pressure of subjects. Background Technology

[0003] Blood pressure (BP) is a key physiological factor for identifying any potential cardiovascular disease in a subject. Blood pressure is typically assessed using systolic blood pressure (SBP) and diastolic blood pressure (DBP). Systolic blood pressure represents the pressure in the arteries when the heart is beating, while diastolic blood pressure represents the pressure in the arteries when the heart is at rest between two beats.

[0004] For decades, blood pressure has typically been measured using cuff-based oscillation measurements and sphygmomanometers. However, these methods are unsuitable for long-term monitoring as they can cause discomfort and a feeling of foreign body irritation in individuals with sensitive skin. To overcome these limitations, cuffless methods for continuous blood pressure monitoring have been developed, particularly utilizing signals generated by photoplethysmography (PPG) and machine learning techniques. PPG signals are also frequently used to assess characteristics of other living organisms, including heart rate (HR), respiratory rate, and blood oxygen saturation. These involve using light emitters and sensors to measure fluctuations in the vascular region beneath the skin. For example, when biological tissue is illuminated by light, photodetectors can record subtle changes in light intensity caused by blood flow.

[0005] However, these methods are still contact-based and cannot eliminate the discomfort caused by contact. Therefore, there is high hope in the current field for non-contact blood pressure measurement methods, with specific needs focused on non-contact blood pressure measurement systems that aim to improve comfort while achieving highly accurate health monitoring. Summary of the Invention

[0006] The purpose of this invention is to provide a system and method for solving the aforementioned technical problems by extracting physiologically relevant information from images.

[0007] In general, to extract a subject's systolic blood pressure (SBP) and diastolic blood pressure (DBP), the system first captures color video of the subject. Then, computer vision techniques are applied to the video frames to locate regions of interest (ROIs) on the subject's face and continuously track these regions for a period of time. Next, the images of the located ROIs are fed into a framework incorporating image processing, signal processing, and machine learning algorithms to create a signal based on remote photoplethysmography (rPPG). The rPPG signal, its derivatives of different orders, and the heart rate (HR) estimated from the rPPG signal are then input into the machine learning model to estimate / predict the subject's SBP and DBP.

[0008] According to a first aspect of the present invention, a camera-based remote blood pressure monitoring system is provided. The system includes: a camera, a color image and signal processing system, a remote photoplethysmography (rPPG) signal extraction system, a blood pressure estimator, and a report output module. The camera is used to capture color image frames of a subject. The color image and signal processing system is used to extract physiological signals from the color image frames of the subject and to: perform light intensity analysis on the color image frames of the subject; identify the subject's face and the location of facial landmarks; trace regions of interest (ROIs) based on approximate facial landmarks; select one of the ROIs; and extract the physiological signal from the selected ROI. The rPPG signal extraction system is used to generate an rPPG signal using the extracted physiological signals, which embeds cardiovascular activity information of the subject, and to: form a 1D (one-dimensional) signal from the extracted physiological signals; apply a machine learning or image processing algorithm to obtain the rPPG signal using the 1D signal; apply bandpass filtering to the extracted rPPG signal; and calculate derivatives of different orders of the rPPG signal to generate derivative information. The blood pressure estimator receives the rPPG signal and the derivative information, and uses a trained machine learning model to output the estimated systolic blood pressure (SBP) and estimated diastolic blood pressure (DBP) of the subject based on the rPPG signal and the derivative information. A report output module receives the estimated SBP and estimated DBP of the subject and provides a readable report on the estimated SBP and DBP values ​​of the subject.

[0009] According to a second aspect of the present invention, a camera-based remote blood pressure monitoring method is provided. The method includes the following steps: capturing color image frames of a subject using a camera; extracting physiological signals from the color image frames of the subject using a color image and signal processing system, including the following steps: (1) performing light intensity analysis on the color image frames of the subject; (2) identifying the subject's face and the location of facial landmarks; (3) tracing regions of interest (ROIs) based on approximate facial landmarks; (4) selecting one of the ROIs; and (5) extracting the physiological signals from the selected ROIs. The method further includes the step of generating an rPPG signal from the extracted physiological signals using an rPPG signal extraction system, which embeds cardiovascular activity information of the subject, including the following steps: (1) forming a 1D (one-dimensional) signal from the extracted physiological signals; (2) applying a machine learning or image processing algorithm to obtain the rPPG signal using the 1D signal; (3) applying bandpass filtering to the extracted rPPG signal; and (4) calculating derivatives of different orders of the rPPG signal to generate derivative information. The method further includes the steps of: receiving the rPPG signal and the derivative information via a blood pressure estimator; using a trained machine learning model via the blood pressure estimator, outputting the estimated systolic blood pressure (SBP) and estimated diastolic blood pressure (DBP) of the subject based on the rPPG signal and the derivative information; and receiving the estimated SBP and estimated DBP of the subject via a report output module, and providing a readable report on the estimated SBP and DBP values ​​of the subject.

[0010] With the above configuration, the system equipped with a machine learning model offers the following technical benefits: enabling non-contact blood pressure monitoring using a camera. The approach provides a painless, non-invasive solution for continuous blood pressure monitoring, overcoming the limitations of traditional cuff-based methods. Furthermore, by leveraging computer vision and machine learning, the proposed system can accurately estimate blood pressure from video frames, making it suitable for long-term blood pressure monitoring and improving user comfort. Attached Figure Description

[0011] The embodiments of the present invention will now be described in more detail with reference to the accompanying drawings, wherein:

[0012] Figure 1 A schematic diagram of the architecture of a camera-based remote blood pressure monitoring system according to an embodiment of the present invention is shown.

[0013] Figure 2 A schematic diagram illustrating the process of estimating systolic blood pressure (SBP) and diastolic blood pressure (DBP) according to an embodiment of the present invention is shown;

[0014] Figure 3A schematic diagram illustrating the process of converting raw color frames into physiological signals using a color image and signal processing system according to an embodiment of the present invention is shown.

[0015] Figure 4 A schematic diagram illustrating the process of generating rPPG signals from extracted physiological signals using an rPPG signal extraction system according to some embodiments of the present invention is shown.

[0016] Figure 5 A schematic diagram illustrating the process of forming an rPPG signal from an captured subject video according to some embodiments of the present invention is shown.

[0017] Figure 6 This illustration shows a flowchart of a process for training a machine learning model for a blood pressure estimator using captured subject videos, according to an embodiment of the present invention; and

[0018] Figure 7 A schematic diagram of the architecture of a camera-based remote blood pressure monitoring system according to an embodiment of the present invention is shown. Detailed Implementation

[0019] In the following description, camera-based remote blood pressure monitoring systems and methods, etc., are described as preferred examples. Those skilled in the art will understand that modifications, including additions and / or substitutions, can be made without departing from the scope and spirit of the invention. Specific details may be omitted so as not to obscure the invention; however, this disclosure is prepared to enable those skilled in the art to practice the teachings herein without undue experimentation.

[0020] Figure 1 A schematic diagram of the architecture of a camera-based remote blood pressure monitoring system 100 according to an embodiment of the present invention is shown. The system 100 includes a camera 110, a color image and signal processing system 120, a remote photoplethysmography (rPPG) signal extraction system 130, a blood pressure estimator 140, and a report output module 150.

[0021] Camera 110 is configured to capture video with color image frames of the subject to be analyzed. The term "subject" as used herein refers to a person undergoing blood oxygen saturation monitoring. Camera 110 serves as the primary input device for system 100, providing raw visual data for subsequent processing steps. In some embodiments, camera 110 may record one or more color videos with color image frames for the subject.

[0022] The color image and signal processing system 120 communicates with the camera 110, wherein the color image frames captured by the camera 110 serve as the input source for the color image and signal processing system 120. The color image and signal processing system 120 is configured to extract physiological signals from the subject's color image frames. Specifically, during the processing of the color image frames, the color image and signal processing system 120 can perform several key functions. The color image and signal processing system 120 is configured to: perform light intensity analysis on the subject's color image frames; identify the subject's face and the location of its facial feature points; trace regions of interest (ROIs) based on approximate facial feature points; select the optimal ROI based on certain metrics; and extract physiological signals from the selected ROI.

[0023] The rPPG signal extraction system 130 communicates with the color image and signal processing system 120. After extracting physiological signals, the physiological signals can be transmitted from the color image and signal processing system 120 to the rPPG signal extraction system 130. The rPPG signal extraction system 130 is configured to generate an rPPG signal using the extracted physiological signals, which embeds cardiovascular activity information of the subject. Specifically, in the process of generating the rPPG signal, the rPPG signal extraction system 130 can perform several key functions. The rPPG signal extraction system 130 is configured to: form a 1D (one-dimensional) signal from the extracted physiological signals; apply machine learning or image processing algorithms to obtain the rPPG signal using the 1D signal; perform bandpass filtering on the extracted rPPG signal; and calculate different orders of derivatives of the rPPG signal to generate derivative information.

[0024] The blood pressure estimator 140 communicates with the rPPG signal extraction system 130 to receive rPPG signals and derivative information. The rPPG signals and their derivatives of different orders (i.e., derivative information), as well as the estimated heart rate (HR) calculated from the rPPG signals, are input into the blood pressure estimator 140 for inference, thereby outputting the estimated systolic blood pressure (SBP) and diastolic blood pressure (DBP) for the subject. Specifically, the blood pressure estimator 140 includes a machine learning model configured for SBP and DBP estimation. The machine learning model can be trained using a blood pressure (BP) training set combining the PPG signal set and the rPPG signal set to perform the SBP and DBP estimation procedure. Once the machine learning model is trained, the blood pressure estimator 140 is configured to use the trained machine learning model to estimate the subject's SBP and DBP based on the rPPG signals and derived information from the rPPG signal extraction system 130.

[0025] The report output module 150 communicates with the blood pressure estimator 140 to receive inference results containing SBP and DBP estimates. The report output module 150 is configured to compile and present the results in a user-friendly format, thereby providing a readable report on the subject's cardiovascular activity status.

[0026] Figure 2 A schematic diagram of a process 200 for estimating systolic blood pressure (SBP) and diastolic blood pressure (DBP) according to an embodiment of the present invention is shown. Figure 2 The process 200 shown can be used as before. Figure 1 The aforementioned camera-based remote blood pressure monitoring system 100 performs the procedure. Process 200 includes steps S201, S202, S203, S204, S205, S206, and S207, which are used to estimate the subject's SBP and DBP.

[0027] The process begins with step S201, which involves capturing a color video of the subject. In this step, camera 110 is activated to record a color video of the subject, thereby obtaining the raw color frames required for the subsequent physiological signal extraction stage.

[0028] In step S202, the raw color frame captured by camera 110 is transmitted to color image and signal processing system 120 for feature extraction and region recognition. Color image and signal processing system 120 can locate and track pixels on the subject's face and identify regions of interest (ROIs) within the raw color frame to extract physiological information.

[0029] Once the Region of Interest (ROI) of the subject's face has been selected / determined, the process proceeds to step S203. The color image and signal processing system 120 extracts physiological signals from the ROI based on color image frames and records these signals. At this stage, the operation of "converting the raw color frame into physiological signals" performed by the color image and signal processing system 120 helps to minimize noise in the frame and provide higher quality data for the physiological signal extraction procedure.

[0030] Next, the process proceeds to step S204. In step S204, the extracted physiological signals are transmitted from the color image and signal processing system 120 to the rPPG signal extraction system 130. The rPPG signal extraction system 130 processes the physiological signals to form an rPPG signal, which embeds a single one-dimensional (1D) input source of the subject's physiological information. At this stage, the rPPG signal extraction system 130 also calculates derivatives of different orders of the rPPG signal, which are then input into the machine learning model of the blood pressure estimator 140.

[0031] After the rPPG signal is formed / generated, the process continues to step S205. In step S205, the subject's heart rate (HR) is estimated from the rPPG signal using a machine learning algorithm or frequency analysis performed by the rPPG signal extraction system 130, and is input into the machine learning model of the blood pressure estimator 140 as an input source.

[0032] The process then proceeds to step S206. The rPPG signal and its derivatives of different orders, as well as the HR estimated from the corresponding rPPG signal, are input into the machine learning model of the blood pressure estimator 140. The blood pressure estimator 140 uses the input data to estimate SBP and DBP. This stage employs machine learning and / or deep learning techniques, with the blood pressure estimator 140 applying an optimized pre-trained model for the SBP and DBP estimation tasks.

[0033] Finally, in step S207, the SBP and DBP estimation results made by the blood pressure estimator 140 are output to the report output module 150. The report output module 150 generates and outputs a report, completing the estimation process of the subject's SBP and DBP.

[0034] The following section will describe in detail how each component processes data.

[0035] Figure 3 A schematic diagram illustrating the process of converting raw color frames into physiological signals using a color image and signal processing system 120 according to an embodiment of the present invention is shown. The process includes steps S301, S302, S303, S304, and S305.

[0036] Once the color image and signal processing system 120 receives the raw color frame of the subject from the camera 110, the process can begin from step S301. In step S301, the color image and signal processing system 120 applies face detection and facial landmark detection to the raw color frame. In one embodiment, face detection and facial landmark detection can be performed using computer vision techniques to identify the subject's face and its facial landmarks in the color frame. The detection process can utilize various face detection and landmark recognition methods to determine the location of the subject's facial features. For example, the color image and signal processing system 120 includes a feature extraction and feature recognition-based model or a face detection model (e.g., a face detection model based on a pre-trained convolutional neural network) for face detection and facial landmark detection.

[0037] After detecting the subject's face and its facial landmarks, regions of interest (ROIs) in the facial representation can be identified. These designated ROIs can then serve as the basis for extracting physiological signals in subsequent processing. In facial representations, computer vision algorithms that approximate facial landmarks are used to determine the ROIs, where facial landmarks serve as descriptors of the locations of key facial features such as the nose, eyes, and mouth.

[0038] For example, in the process of face detection and facial landmark detection, firstly, the original color frame is processed using a face detection model, which detects the bounding boxes of facial representations within the original color frame. Once facial features are localized, the facial landmark predictor applies detection algorithms to identify key facial points, such as the area around the eyes, the tip of the nose, and the area near the mouth, and determines facial landmarks, defining ROIs on the facial features for subsequent analysis.

[0039] The process moves to step S302. This step performs ROI tracking. Throughout the captured video (i.e., consecutive color frames), ROIs identified on facial representations are continuously tracked to minimize noise and improve the quality of the expected generated physiological signals to be extracted. The tracking process is based on approximate facial landmarks.

[0040] In one embodiment, tracking a Region of Interest (ROI) based on detected facial landmarks includes tracking any ROI on the facial representation using any type of ROI tracking technique. For example, optical flow methods can be used to estimate the motion of facial features between consecutive frames (i.e., the first frame and the next frame). Alternatively, filters can be applied to predict and correct the localization outcome of the ROI over time, thereby enhancing robustness / robustness to slight occlusion or sudden movement.

[0041] The process moves to step S303. ROI selection is performed to determine the optimal region of interest (optimal ROI) for physiological signal extraction. The selection process can be based on predefined metrics, such as signal-to-noise ratio (SNR), to minimize noise and improve signal quality. Various selection methods can be employed depending on the specific requirements. For example, a maximum SNR method can be used, which evaluates multiple candidate ROIs and selects the ROI with the highest SNR. In some embodiments, machine learning-based methods (e.g., classification models trained on labeled data) can predict ROIs with high reliability based on historical signal quality.

[0042] The process moves to the final step, S304. A pixel intensity extraction process is performed, including calculating the average intensity value of each color channel (i.e., different color channels) within the selected region of interest. Since the average intensity value is a core feature for extracting physiological signals, it is recorded and analyzed. Specifically, the average intensity value can be considered a physiological signal because it captures color changes caused by variations in blood volume. Then, in subsequent processing, the average intensity value is used to generate the rPPG signal.

[0043] To generate the rPPG signal, extracting the average pixel intensity value of the green channel was given high priority because the green channel is highly sensitive to changes in blood volume under typical lighting conditions. However, in low-light environments, the red or blue channels can also provide high signal fidelity. In some embodiments, the average pixel intensity values ​​of all three color channels (red, green, and blue) can be obtained simultaneously, forming a representation of the physiological signal with three channels.

[0044] Figure 4 A schematic diagram illustrating the process of generating an rPPG signal from an extracted physiological signal using an rPPG signal extraction system 130 according to some embodiments of the present invention is shown. The extraction process includes steps S401, S402, S403, S404, and S405, which are used to generate the rPPG signal.

[0045] Once the rPPG signal extraction system 130 receives the extracted physiological signal from the color image and signal processing system 120, the process can begin from step S401. Based on the physiological signal, the rPPG signal extraction system 130 acquires the average pixel intensity values ​​of different color channels within the selected region of interest (ROI) to form a 1D signal. In this regard, a 1D signal can be generated by processing the average pixel intensity values ​​of a single color channel over time. For each video frame, the average pixel intensity value is calculated within the selected ROI. All average intensity values ​​recorded frame by frame form a continuous time series for each color channel. In this way, three independent 1D signals are generated, corresponding to the red, green, and blue channels, respectively, which collectively encode physiological information. The 1D signals of all color channels are then input into a machine learning or image processing algorithm.

[0046] The process moves to step S402. A machine learning or image processing algorithm is used to process the 1D signal to extract the rPPG signal. For example, the machine learning algorithm in this step may include a feature extraction model utilizing principal component analysis (PCA). The feature extraction model can process 1D signals obtained from different color channels and extract core components representing physiological changes. By identifying the main signal features, the feature extraction model is able to separate periodic fluctuations associated with changes in blood volume. Subsequently, the optimized signal components are combined to generate the rPPG signal, which can be further used for heart rate estimation and other physiological index analyses.

[0047] The process proceeds to step S403. The rPPG signal extraction system 130 has a bandpass filter, and the extracted rPPG signal can be processed by the bandpass filter to remove noise and retain physiologically relevant information. In some embodiments, the bandpass filtering performed by the rPPG signal extraction system 130 can use bandpass filters of arbitrary order, low cutoff frequency, and high cutoff frequency to remove noise and retain physiologically relevant information.

[0048] The process proceeds to step S404. After bandpass filtering, a clean rPPG signal is obtained, which means that unwanted noise components, such as motion artifacts and ambient light fluctuations, are removed, while the physiologically relevant frequency range corresponding to heart rate changes is preserved.

[0049] The process proceeds to step S405. Different orders of derivatives can be calculated for the clean rPPG signal. These derivative calculations help capture rates of change and dynamic changes in the clean rPPG signal, thus providing features for physiological analysis. For example, the first derivative can highlight rapid fluctuations related to heart rate, while the second derivative emphasizes acceleration and deceleration characteristics in blood volume changes.

[0050] In some implementations, the order of the derivative calculation is not limited to first and second order; it can also include derivatives of any order. In this regard, the derivatives of any order can include first-order derivatives (the rate of signal change), second-order derivatives (the acceleration of signal change), or even higher-order derivatives, to achieve more advanced signal representation.

[0051] Furthermore, after calculating the derivative, machine learning algorithms or frequency analysis can be used to estimate the heart rate (HR) from the clean rPPG signal. In some implementations, frequency analysis methods, such as Fast Fourier Transform (FFT) or wavelet transform, can be used to identify the dominant frequency corresponding to the heartbeat and filter out non-cardiac noise. In some embodiments, the machine learning model can estimate the heart rate by learning signal features while also compensating for motion artifacts and illumination variations.

[0052] Figure 5 A schematic diagram illustrating the process of forming an rPPG signal starting from an captured subject video according to some embodiments of the present invention is shown. Figure 5 Able to Figure 3 and Figure 4 The intermediate and final products in the steps are specifically visualized.

[0053] Steps S501 and S502: The process of forming a clean rPPG signal begins with applying face detection and facial landmark detection to the captured video of the subject. Step S503: Then, based on the detected facial landmarks, the region of interest (ROI) is continuously traced along the captured video. Step S504: The optimal ROI with the least noise is selected using certain metrics (e.g., signal-to-noise ratio). Step S505: A 1D signal is formed by averaging the pixel intensity values ​​of different color channels of the selected ROI. For example, there are three color channels, which can be red, green, and blue, respectively. Step S506: All these 1D signals are processed together with machine learning or image processing algorithms to extract the rPPG signal. Step S507: Bandpass filtering is applied to the extracted rPPG signal to obtain a clean rPPG signal.

[0054] Figure 6 A schematic diagram illustrating the process of training a machine learning model for a blood pressure estimator using captured subject videos according to an embodiment of the present invention is shown. Stages (a), (b), and (c) can be executed before training the machine learning model using rPPG signals and other real-world benchmark parameters.

[0055] The training process for the machine learning model used to estimate systolic blood pressure (SBP) and diastolic blood pressure (DBP) can begin in stage (a), which involves acquiring facial video using camera 110. Then, in stage (b), a color image and signal processing system 120 is applied to extract physiological signals from the facial video. In stage (c), the extracted physiological signals are passed to an rPPG signal extraction system 130 to form a clean rPPG signal and its derivatives of different orders.

[0056] Before training, a training set can be prepared containing real baseline parameters corresponding to the same group of training subjects, including HR values, SBP values, and DBP values, as well as processed PPG and rPPG signals and their derivatives. In some embodiments, the machine learning model integrates convolutional neural networks (CNNs), recurrent neural networks (RNNs), deep neural networks (DNNs), multilayer perceptrons (MLPs), transformer models, or combinations thereof.

[0057] During training, the machine learning model is first pre-trained using clean PPG signals, different orders of derivatives of the PPG signals, and true baseline values ​​for HR, SBP, and DBP to learn general features. To prepare the training data, clean PPG signals are first collected using contact-based sensors and processed with bandpass filtering to eliminate noise. Then, the first and second derivatives of the PPG signals are calculated to capture rate changes and acceleration features. True baseline values ​​for HR can be obtained from physical measurement devices and synchronized with the PPG signals. True baseline values ​​for SBP and DBP are recorded using physical monitoring devices and matched with corresponding PPG signal segments. In this way, the machine learning model can be pre-trained using correctly aligned physiological signals.

[0058] The pre-trained machine learning model is further fine-tuned using clean rPPG signals, derivatives of different orders of the rPPG signals, and true baseline values ​​for HR, SBP, and DBP. The fine-tuning process uses clean rPPG signals and their derivatives, where the derivatives can be obtained through the same steps described above. Furthermore, the true baseline values ​​for HR, SBP, and DBP are collected from actual physiological measurements, similar to the pre-training phase, thus ensuring that the clean rPPG signals and their derivatives are aligned with real physiological data.

[0059] Machine learning can automatically extract features from rPPG signals, their derivatives of different orders, and HR, thereby learning and forming a function to map these inputs to estimated SBP and DBP values. In some embodiments, the machine learning model can use provided true baseline values ​​of HR, SBP, and DBP to guide its own training process and improve the proposed function. Once the machine learning model has been trained, it can be used to estimate the subject's SBP and DBP values.

[0060] Therefore, once the model is trained, the system of this invention can estimate a subject's blood pressure by simply recording facial video. The recorded video is processed by a color image and signal processing system, and then physiological signals are extracted by an rPPG signal extraction system. These signals are then fed into the trained machine learning model, enabling the model to analyze the data and output a report containing SBP and DBP values.

[0061] Figure 7 A schematic diagram of the architecture of a camera-based remote blood pressure monitoring system 700 according to an embodiment of the present invention is shown. System 700 includes a camera 710, a color image and signal processing system 720, an rPPG signal extraction system 730, a blood pressure estimator 740, and a report output module 750. These components can perform various steps and execute the aforementioned instructions, including using the camera 710 to capture or photograph a subject's face.

[0062] Similarly, the captured video is further processed by the color image and signal processing system 720 to extract relevant physiological features. The rPPG signal extraction system 730 further processes these features to generate a clean rPPG signal and its derivatives. These processed signals are then input into the blood pressure estimator 740, where machine learning algorithms or signal analysis techniques estimate SBP and DBP values. Finally, the report output module 750 generates and outputs a blood pressure measurement report.

[0063] In addition, system 700 includes a feedback module 760 that analyzes video captured or extracted from camera 710 to assess the subject's physiological state. For example, feedback module 760 can detect whether the subject has engaged in physical activity in a short period of time, or whether the subject is under physical conditions that may distort the quality of rPPG signals, potentially affecting the accuracy of estimated SBP and DBP values.

[0064] In one embodiment, the feedback module 760 includes a post-exercise state detection model that analyzes multiple physiological and visual indicators to assess whether the subject has engaged in physical activity over a short period of time.

[0065] The post-exercise state detection model utilizes computer vision and machine learning techniques to detect changes in respiratory rate, skin reflectivity, facial expression, and multimodal fusion-based classification to improve detection accuracy. During exercise, breathing patterns change, resulting in a faster and more pronounced respiratory rate. The feedback module 760 can use facial micro-movement analysis of the subject to estimate respiratory rate conditions and detect any abnormalities. On the other hand, after physical activity, increased sweating makes the skin appear brighter, especially on the forehead and nose. The feedback module 760 can apply specular reflection analysis and machine learning models to detect changes in skin brightness.

[0066] In one embodiment, the feedback module 760 can detect foreign objects on the subject's face, such as glasses, masks, or face coverings, which may interfere with the accurate rPPG signal extraction process. By utilizing computer vision algorithms and machine learning models, the feedback module 760 can identify and assess obstacles that may affect signal quality.

[0067] In one embodiment, the feedback module 760 can detect whether the subject has incorrect posture during the evaluation process, thereby ensuring optimal conditions for rPPG signal extraction. By using posture estimation technology and facial landmark tracking analysis of head position, tilt angle, and stability, the feedback module 760 can determine whether the subject is misaligned, has excessive motion interference, or is not correctly facing the camera.

[0068] The report output module 750 is also configured to improve the reliability of blood pressure reporting. When the estimated SBP or DBP value exceeds the normal physiological range or exhibits an abnormally high rate of change, the report output module 750 can interact with the feedback module 760 to verify whether the subject is experiencing physiological conditions that may distort the quality of the rPPG signal. If the feedback module 760 detects a potential source of measurement distortion, such as brief physical activity, facial obstruction, incorrect posture, or excessive motion interference, the report output module 750 includes a description of this in the report, indicating the potential distortion factors.

[0069] This method efficiently utilizes video data from multiple angles, thereby enhancing the reliability of non-contact blood pressure measurement. By integrating the feedback module 760 and the report output module 750, the system 700 can not only extract rPPG signals for blood pressure estimation, but also verify the current assessment conditions and detect potential distortions.

[0070] Functional units and modules of the apparatus and methods according to the embodiments disclosed herein can be implemented using computing devices, computer processors, or electronic circuits, including but not limited to application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), microcontrollers, and other programmable logic devices configured or programmed according to the teachings of this disclosure. Those skilled in the art of software or electronics can readily prepare computer instructions or software code executable in computing devices, computer processors, or programmable logic devices based on the teachings of this disclosure.

[0071] All or part of the methods according to the embodiments can be performed in one or more computing devices, including server computers, personal computers, laptops, and mobile computing devices (such as smartphones and tablets).

[0072] Embodiments may include computer storage media, transient and non-transient memory devices storing computer instructions or software code, which can be used to program or configure computing devices, computer processors, or electronic circuits to perform any of the processes of the present invention. Storage media, transient and non-transient memory devices may include, but are not limited to, floppy disks, optical disks, Blu-ray discs, DVDs, CD-ROMs, magneto-optical disks, ROMs, RAMs, flash memory devices, or any type of medium or device suitable for storing instructions, code, and / or data.

[0073] Each functional unit and module according to various embodiments can also be implemented in a distributed computing environment and / or cloud computing environment, wherein all or part of the machine instructions are executed in a distributed manner by one or more processing devices interconnected through communication networks, such as intranets, wide area networks (WANs), local area networks (LANs), the Internet, and other forms of data transmission media.

[0074] The foregoing description of the present invention is provided for illustrative purposes. It is not intended to be exhaustive or to limit the invention to the precise forms disclosed. Many modifications and variations will be apparent to those skilled in the art.

[0075] These embodiments were chosen and described in order to best explain the principles of the invention and its practical application, thereby enabling those skilled in the art to understand the various embodiments of the invention and the various modifications suitable for the intended particular use.

Claims

1. A camera-based remote blood pressure monitoring system, characterized in that, include: A camera used to capture color image frames of the subject; A color image and signal processing system for extracting physiological signals from the color image frames of the subject. And used in: Perform light intensity analysis on the color image frames of the subject; Identify the subject's face and the location of facial landmarks; Regions of interest (ROIs) are traced based on approximate facial landmarks; Select one of the regions of interest; as well as Extract the physiological signal from the selected region of interest; A remote photoplethysmography (rPPG) signal extraction system is used to generate an rPPG signal from the extracted physiological signal, which embeds cardiovascular activity information of the subject, and is used for: From the extracted physiological signals, a 1D (one-dimensional) signal is formed; The rPPG signal is obtained using the 1D signal by applying machine learning or image processing algorithms. A bandpass filter is applied to the extracted rPPG signal; as well as Calculate the derivatives of the rPPG signal of different orders to generate derivative information; A blood pressure estimator receives the rPPG signal and the derivative information, and uses a trained machine learning model to output the estimated systolic blood pressure (SBP) and estimated diastolic blood pressure (DBP) of the subject based on the rPPG signal and the derivative information. as well as The report output module is used to receive the estimated SBP and estimated DBP of the subject and provide a readable report on the estimated SBP and DBP values ​​of the subject.

2. The system according to claim 1, characterized in that, The color image and signal processing system includes models based on feature extraction and feature recognition or face detection models for face detection and facial landmark detection.

3. The system according to claim 2, characterized in that, The region of interest is identified by the color image and signal processing system, which also includes a facial landmark predictor that applies detection algorithms to identify key facial points, including the area around the eyes, the tip of the nose, and the area near the subject's mouth.

4. The system according to claim 3, characterized in that, The color image and signal processing system continuously tracks the identified region of interest throughout the entire sequence of captured color image frames.

5. The system according to claim 1, characterized in that, The color image and signal processing system selects the region of interest based on the maximum signal-to-noise ratio metric by evaluating multiple candidate regions of interest and selecting the region of interest with the highest signal-to-noise ratio.

6. The system according to claim 1, characterized in that, By applying the bandpass filter to the rPPG signal, unwanted noise components are removed while the physiologically relevant frequency range is preserved, thereby obtaining a clean rPPG signal.

7. The system according to claim 1, characterized in that, Also includes: The feedback module is used to analyze the video captured by the camera to assess the physiological state of the subject. The feedback module detects whether the subject has engaged in physical activity in a short period of time or whether the subject is under physical conditions that may cause distortion of rPPG signal quality.

8. The system according to claim 7, characterized in that, When the estimated values ​​of SBP and DBP exceed the normal physiological range or exhibit an abnormally high rate of change, the report output module interacts with the feedback module to verify whether the subject is experiencing physiological conditions that may distort the quality of the rPPG signal.

9. The system according to claim 8, characterized in that, If the feedback module detects a potential source of measurement distortion, the report output module is further configured to add descriptions to the readable report, indicating potential distortion factors that may occur.

10. The system according to claim 1, characterized in that, The trained machine learning model of the blood pressure estimator is trained to estimate SBP and DBP using a blood pressure (BP) training set, which includes a PPG signal set, a rPPG signal set, different order derivative sets, and true baseline values ​​for heart rate, SBP, and DBP, all corresponding to the same group of training subjects.

11. A remote blood pressure monitoring method based on a camera, characterized in that, include: Use a camera to capture color image frames of the subjects; Physiological signals are extracted from the subject's color image frames using a color image and signal processing system, including the following steps: Perform light intensity analysis on the color image frames of the subject; Identify the subject's face and the location of facial landmarks; Regions of interest (ROIs) are traced based on approximate facial landmarks; Select one of the regions of interest; and Extract the physiological signal from the selected region of interest; Using a remote photoplethysmography (rPPG) signal extraction system, an rPPG signal is generated from the extracted physiological signals, which embeds the subject's cardiovascular activity information. The steps are as follows: From the extracted physiological signals, a 1D (one-dimensional) signal is formed; The rPPG signal is obtained using the 1D signal by applying machine learning or image processing algorithms. A bandpass filter is applied to the extracted rPPG signal; and Calculate the derivatives of the rPPG signal of different orders to generate derivative information; The rPPG signal and the derivative information are received by a blood pressure estimator; Using a trained machine learning model, the blood pressure estimator outputs the estimated systolic blood pressure (SBP) and estimated diastolic blood pressure (DBP) of the subject based on the rPPG signal and the derivative information; and The report output module receives the estimated SBP and estimated DBP of the subject and provides a readable report on the estimated SBP and DBP values ​​of the subject.

12. The method according to claim 11, characterized in that, The color image and signal processing system includes models based on feature extraction and feature recognition or face detection models for face detection and facial landmark detection.

13. The method according to claim 12, characterized in that, The region of interest is identified by the color image and signal processing system, which also includes a facial landmark predictor that applies detection algorithms to identify key facial points, including the area around the eyes, the tip of the nose, and the area near the subject's mouth.

14. The method according to claim 13, characterized in that, Also includes: The color image and signal processing system continuously tracks the identified region of interest throughout the entire sequence of captured color image frames.

15. The method according to claim 11, characterized in that, The color image and signal processing system selects the region of interest based on the maximum signal-to-noise ratio metric by evaluating multiple candidate regions of interest and selecting the region of interest with the highest signal-to-noise ratio.

16. The method according to claim 11, characterized in that, By applying the bandpass filter to the rPPG signal, unwanted noise components are removed while the physiologically relevant frequency range is preserved, thereby obtaining a clean rPPG signal.

17. The method according to claim 11, characterized in that, Also includes: The feedback module analyzes the video captured by the camera to assess the subject's physiological state. The feedback module detects whether the subject has engaged in physical activity in a short period of time or whether the subject is under physical conditions that may cause distortion of rPPG signal quality.

18. The method according to claim 17, characterized in that, Also includes: When the estimated values ​​of SBP and DBP exceed the normal physiological range or exhibit an abnormally high rate of change, the report output module interacts with the feedback module to verify whether the subject is experiencing physiological conditions that may distort the quality of the rPPG signal.

19. The method according to claim 8, characterized in that, Also includes: If the feedback module detects a potential source of measurement distortion, the report output module adds a description to the readable report, indicating the potential distortion factors that may occur.

20. The method according to claim 11, characterized in that, The trained machine learning model of the blood pressure estimator is trained to estimate SBP and DBP using a blood pressure (BP) training set, which includes a PPG signal set, a rPPG signal set, different order derivative sets, and true baseline values ​​for heart rate, SBP, and DBP, all corresponding to the same group of training subjects.

Images