Non-contact blood pressure monitoring apparatus, method and system for enhancing robustness by using spatial redundancy

Abstract

A non-contact blood pressure monitoring apparatus, method and system for enhancing robustness by using spatial redundancy, which relate to the technical field of non-contact blood pressure monitoring. The method comprises the following main steps: processing an original signal by means of a quality metricImage denoising algorithm, so as to obtain a relatively pure rPPG signal; then, extracting a multi-site PTT and a multi-wavelength PTT from the rPPG signal by means of an adaptive PTT extraction algorithm; and finally, combining feature information of the multi-site PTT with feature information of the multi-wavelength PTT and using a multi-feature model to estimate blood pressure.

Description

Non-contact blood pressure monitoring device, method and system using spatial redundancy to enhance robustness Technical Field

[0001] The present invention discloses a blood pressure monitoring method, in particular a non-contact blood pressure monitoring device, method and system that utilize spatial redundancy to enhance robustness, belonging to the technical field of non-contact blood pressure monitoring methods and monitoring equipment. Background Art

[0002] Hypertension, a common chronic disease, has become a serious global health concern, with its morbidity and mortality rates increasing year by year. Nearly half of hypertensive patients are unaware of their condition, significantly increasing the risk of life-threatening complications such as cerebral hemorrhage, heart failure, and renal failure. Therefore, real-time blood pressure monitoring is of great clinical value in preventing and managing hypertension. Providing accurate and convenient home blood pressure monitoring is crucial for preventing the onset of hypertension and reducing its associated risks.

[0003] Cuff-type electronic blood pressure monitors are currently widely used blood pressure monitoring devices. They have relatively good performance, but they also have obvious drawbacks. For example, the tightness and position of the cuff can have a significant impact on the measurement results, and cuff contraction can also cause discomfort to the user. Due to the above-mentioned drawbacks of cuff-type electronic blood pressure monitors, researchers have conducted research and development on cuffless blood pressure monitoring. Early cuffless blood pressure monitoring technology mainly used contact blood pressure monitoring technology, which used photoelectric sensors to collect the human pulse wave signal and then extract physiological signals related to blood pressure. However, due to the limitations of photoelectric sensors, it has high requirements for the sensor measurement position and has certain limitations.

[0004] With the continuous development of rPPG technology (remote photoplethysmography, rPPG remote photoplethysmography refers to the technology of using sensors such as cameras to capture the periodic changes in skin color caused by the cardiac cycle. By using a multi-wavelength RGB camera to detect subtle color changes caused by pulses on the surface of human skin, non-contact monitoring of human cardiac activity is achieved), it has achieved very robust performance in heart rate and respiratory monitoring. People have also begun to use rPPG to achieve non-contact blood pressure monitoring.

[0005] At present, there are two main types of non-contact blood pressure monitoring methods based on rPPG: (1) using deep neural networks to directly extract blood pressure from rPPG signals. However, deep neural networks require a large amount of data training, and because they are not based on physiological foundations, they do not have good interpretability; (2) using physiological features that are highly correlated with blood pressure to extract blood pressure based on physiological models. Currently, such methods can be mainly divided into two types: pulse wave transmission time method and pulse wave waveform feature method. Among them, the pulse wave transmission time method (i.e., pluse transmit time, abbreviated as PTT) can be divided into multi-point PTT, which usually uses multiple cameras to capture PPG signals from different parts of the human body, and the operation is relatively difficult; multi-wavelength PTT is to calculate PTT by extracting PPG signals at different skin depths using light of different wavelengths. Since rPPG signals of different bands are required to extract PTT, it has high signal quality requirements and poor anti-interference ability. The pulse wave waveform feature method has extremely high requirements on the quality of the pulse wave signal, and has large individual differences, and does not have good generalization ability and robustness. Technical issues

[0006] In response to the technical problems mentioned above in the prior art, such as the high difficulty in operation, high signal quality requirements, poor anti-interference ability of the pulse wave transit time method, and the lack of good generalization and robustness of the pulse wave waveform feature method, the present invention provides a non-contact blood pressure monitoring device, method and system that utilize spatial redundancy to enhance robustness. Based on rPPG technology, the method utilizes the spatial redundancy of the image to extract multi-point PTT using a single camera, further combines multi-wavelength PTT, establishes a multi-feature model between PTT and blood pressure, and combines the quality metric Image denoising algorithm and the adaptive PTT extraction algorithm to improve the data quality of PTT, further improving the robustness and generalization ability of non-contact blood pressure monitoring, and solving the problems of the prior art being susceptible to noise, low robustness, weak generalization ability, and complex operation. Technical Solutions

[0007] The technical solution adopted by the present invention to solve its technical problems is: a device that uses spatial redundancy to enhance the robustness of non-contact blood pressure monitoring, the device includes a light source module and an optical sensor module, the light source module is a light source that can provide continuous visible light and near-infrared light, and the optical sensor module adopts a multi-band narrowband image acquisition module.

[0008] A method for enhancing the robustness of non-contact blood pressure monitoring by utilizing spatial redundancy, the method comprising the following steps:

[0009] Step S1, noise reduction processing: the original signal collected by the multi-band narrowband camera is processed by the quality metricImage noise reduction algorithm to obtain a pure rPPG;

[0010] Step S2, signal extraction: extracting the muti-site PTT and muti-wavelength PTT from the rPPG signal using an adaptive PTT extraction algorithm;

[0011] Step S3, blood pressure estimation: combining the feature information of multi-point PTT and multi-band PTT to estimate blood pressure using a multi-feature model.

[0012] A system for enhancing the robustness of non-contact blood pressure monitoring by utilizing spatial redundancy, the system comprising:

[0013] Noise reduction processing unit: used to process the original signal collected by the multi-band narrowband camera through the quality metricImage noise reduction algorithm to obtain a pure rPPG;

[0014] Signal extraction unit: Adopts adaptive PTT extraction algorithm to extract muti-site PTT and muti-wavelength PTT from rPPG signals respectively;

[0015] Blood pressure estimation unit: used to estimate blood pressure using a multi-feature model by combining the feature information of multi-point PTT and multi-band PTT.

[0016] The technical solution adopted by the present invention to solve the technical problem further includes:

[0017] The light source module uses a DC full-spectrum incandescent lamp.

[0018] The light source module can provide continuous visible light and near-infrared light within the range of 440nm-940nm.

[0019] The multi-band narrowband image acquisition module is a camera that uses a narrowband three-bandpass filter that allows near-infrared light to pass through, so that the camera can simultaneously capture information carried by visible light with a wavelength of 550nm, visible light with a wavelength of 660nm, and near-infrared light with a wavelength of 850nm.

[0020] In step S1, the quality metric Image denoising algorithm used includes the following sub-steps:

[0021] Step S1-1, identifying the skin area captured by the camera and quantitatively extracting the light intensity of the skin area, selecting an area with appropriate light intensity as a selected region ROI, defined as ROI1;

[0022] Step S1-2: Use the spatial redundancy of the image to divide ROI1 into several boxes of equal area, extract the rPPG signals of the g channel and ir channel in each box, perform spectrum analysis on them, and extract the frequencies with the largest amplitudes of the g channel and ir channel in the frequency range of [0.5, 6.0] Hz and record them as , select The smallest or equal number of boxes are used as the final ROI. This area is the metricImage and is defined as ROI2.

[0023] Step S1-3: extract the rPPG signal in ROI2, perform DC removal and bandpass filtering on it, and obtain a pure rPPG signal after noise reduction.

[0024] In step S2, the adaptive PTT extraction algorithm includes the following sub-steps:

[0025] Step S2-1: Detect the peaks and valleys in the terminated rPPG signal using a peak detection algorithm, and calculate the first-order derivative of the rising phase of the signal to obtain the point where the first-order derivative is the largest;

[0026] Step S2-2, extracting the time interval between two adjacent pulses in the termination signal, i.e., IBI, and determining the PTT threshold based on the IBI;

[0027] Step S2-3: Use the same method to calculate the steepest point of the initial rPPG signal, calculate the time difference between the two points, and determine whether it is a valid PTT through a threshold.

[0028] In step S2, the process of muti-site PTT extraction includes the following sub-steps:

[0029] Step S2-1-1, select two or more boxes with the longest vertical distance in the metricImage in step S1 and determine the starting point and ending point of the pulse wave signal according to the direction of the blood vessel;

[0030] Step S2-1-2: extract the rPPG signals of the starting point and the ending point, and extract the muti-site PTT using an adaptive PTT extraction algorithm.

[0031] In step S2, the process of extracting the multi-wavelength PTT includes the following sub-steps:

[0032] Step S2-2-1, extracting the rPPG signals of the G channel and IR channel after noise reduction in step S1, denoted as g_rPPG and ir_rPPG, respectively, which carry the physiological information of the arterioles and large arteries;

[0033] Step S2-2-2: Extract the muti-wavelength PTT using an adaptive PTT extraction algorithm.

[0034] In step S3, the feature model algorithm includes the following sub-steps:

[0035] Step S3-1, extract the muti-site PTT and its starting point and ending point rPPG signals, respectively, marked as s_rPPG and e_rPPG, and extract the muti-wavelength PTT and g_rPPG and ir_rPPG signals respectively;

[0036] Step S3-2, calculate the difference between the muti-site PTT and the muti-wavelength PTT, marked as dif1; calculate the difference between s_rPPG and g_rPPG, marked as dif2; calculate the difference between s_rPPG and ir_rPPG, marked as dif3; calculate the difference between e_rPPG and g_rPPG, marked as dif4; calculate the difference between e_rPPG and ir_rPPG, marked as dif5;

[0037] Step S3-3: using muti-site PTT, muti-wavelength PTT, dif1, dif2, dif3, dif4, and dif5 as input features, and inputting them into a polynomial regression to fit a multi-feature regression model between multi-feature PTT and blood pressure.

[0038] The noise reduction processing unit includes:

[0039] Non-skin part removal subunit: used to identify the skin area captured by the camera and quantitatively extract the light intensity of the skin area, and select the area with appropriate light intensity as the selected area ROI, which is defined as ROI1;

[0040] Region division subunit: It is used to divide ROI1 into several boxes of equal area by utilizing the spatial redundancy of the image, extract the rPPG signals of the g channel and ir channel in each box, perform spectrum analysis on them, and extract the frequencies with the largest amplitudes of the g channel and ir channel in the frequency range of [0.5, 6.0] Hz and record them as , select The smallest or equal number of boxes are used as the final ROI. This area is the metricImage and is defined as ROI2.

[0041] Denoising subunit: It is used to extract the rPPG signal in ROI2 and perform DC removal and bandpass filtering operations on it to obtain the pure rPPG signal after noise reduction.

[0042] The signal extraction unit includes:

[0043] Rising signal calculation subunit: used to detect the peaks and troughs in the terminal rPPG signal through the peak detection algorithm, calculate the first-order derivative of the rising period of the signal to obtain the point with the maximum first-order derivative;

[0044] IBI calculation subunit: used to extract the time interval between two adjacent pulses in the termination signal, i.e., IBI, and determine the PTT threshold based on the IBI;

[0045] PTT determination subunit: used to calculate the steepest point of the starting rPPG signal using the same method, calculate the time difference between the two points, and determine whether it is a valid PTT through a threshold.

[0046] In the signal extraction unit, the muti-site PTT extraction includes the following subunits:

[0047] The pulse wave signal determination subunit is used to select two or more boxes with the longest vertical distance in the metricImage in the noise reduction processing unit and determine the starting point and ending point of the pulse wave signal according to the direction of the blood vessel;

[0048] The muti-site PTT extraction subunit is used to extract the rPPG signals of the starting and ending points, and extract the muti-site PTT using an adaptive PTT extraction algorithm.

[0049] In the signal extraction unit, the process of multi-wavelength PTT extraction includes the following subunits:

[0050] rPPG signal extraction subunit: used to extract the denoised rPPG signals of the G channel and IR channel using the noise reduction processing unit, recorded as g_rPPG and ir_rPPG respectively, which carry the physiological information of the small arteries and large arteries respectively;

[0051] muti-wavelength PTT extraction subunit: used to extract muti-wavelength PTT using an adaptive PTT extraction algorithm.

[0052] The blood pressure estimation unit includes the following subunits:

[0053] Component information extraction subunit: used to extract the muti-site PTT and its starting point and ending point rPPG signals, marked as s_rPPG and e_rPPG respectively, and extract the muti-wavelength PTT and g_rPPG and ir_rPPG signals respectively;

[0054] Difference calculation subunit: used to calculate the difference between muti-site PTT and muti-wavelength PTT, marked as dif1; calculate the difference between s_rPPG and g_rPPG, marked as dif2; calculate the difference between s_rPPG and ir_rPPG, marked as dif3; calculate the difference between e_rPPG and g_rPPG, marked as dif4; calculate the difference between e_rPPG and ir_rPPG, marked as dif5;

[0055] Input subunit: used to take muti-site PTT, muti-wavelength PTT, dif1, dif2, dif3, dif4 and dif5 as input features, and input polynomial regression to fit a multi-feature regression model between multi-feature PTT and blood pressure. Beneficial effects

[0056] The beneficial effects of the present invention are: 1. The present invention can effectively capture images of the user's skin in the visible light and near-infrared light bands by using a light source module that can provide continuous visible light in combination with a multi-band narrowband image acquisition module, providing a prerequisite for subsequent reliable image processing.

[0057] 2. Based on rPPG technology, the present invention utilizes the spatial redundancy of images to extract multi-point PTT using a single camera. This is further combined with multi-wavelength PTT to establish a multi-feature model between PTT and blood pressure. This constructs an rPPG signal array and implements a quality metric image (quality metric image) noise reduction algorithm to reduce the impact of noise on blood pressure monitoring and improve the robustness of the non-contact blood pressure monitoring method.

[0058] 3. The present invention proposes an adaptive PTT extraction algorithm to improve the reliability and accuracy of PTT features.

[0059] 4. The present invention uses a single camera to extract the Multi-site PTT feature, reducing the complexity of Multi-sit PTT extraction.

[0060] 5. The present invention utilizes the complementary physiological characteristics of multi-site PTT and multi-wavelength and combines them with the spatial redundancy of images. The differences between the two PTTs and the differences between their corresponding rPPG signals are used to constrain and calibrate each other, thereby realizing a multi-feature model of PTT and blood pressure, reducing the variability between individuals and improving the robustness and generalization ability of non-contact blood pressure monitoring. BRIEF DESCRIPTION OF THE DRAWINGS

[0061] FIG1 is a flow chart of blood pressure monitoring according to the present invention.

[0062] FIG2 is a flow chart of the Quality MetricImage denoising algorithm of the present invention.

[0063] FIG3 is a schematic diagram of a non-contact blood pressure monitoring device according to the present invention.

[0064] In the figure: 1. Image acquisition module; 2. Signal analysis module; 3. Model training module; 4. Blood pressure prediction module. Best Mode for Carrying Out the Invention

[0065] This embodiment is a preferred implementation manner of the present invention. Other embodiments whose principles and basic structures are the same or similar to those of this embodiment are within the scope of protection of the present invention.

[0066] With reference to Figures 1 through 3 , the present invention relates to a device for enhancing the robustness of non-contact blood pressure monitoring using spatial redundancy. The device primarily comprises a light source module and an optical sensor module. In this embodiment, the light source module utilizes a DC full-spectrum incandescent lamp (i.e., an incandescent lamp) as a light source, which provides continuous visible and near-infrared light within a wavelength range of 440 nm to 940 nm. In practice, other full-spectrum (540 nm to 950 nm) and flicker-free lighting devices may also be used as the light source, such as white light LEDs or combinations of multiple LEDs with different wavelengths, as long as the light emitted can be used for detection illumination. Considering size, ease of use, and cost, the present invention preferably utilizes a DC full-spectrum incandescent lamp. Its wavelength range must at least cover the wavelength range captured by the multi-band narrowband camera to meet illumination requirements. In this embodiment, the optical sensor module utilizes a multi-band narrowband image acquisition module. This module utilizes a modified camera. Specifically, the RGB filter of a conventional camera is replaced with a narrowband triple-bandpass filter (GR-IR) that allows near-infrared light to pass. This allows the camera to simultaneously capture information carried by visible light (550nm and 660nm wavelengths) and near-infrared light (850nm wavelength). In specific implementations, the narrowband triple-bandpass filter (GR-IR) can be directly incorporated into the camera to create a custom camera. In addition to the aforementioned camera, the multi-band narrowband camera can also be replaced with a camera, video camera, or camera module integrated with an optical system or CCD chip, capable of simultaneously capturing visible light (540nm-670nm) and near-infrared light (840nm-950nm). The optical sensor module can capture continuous video of the user's forehead as raw data.

[0067] In the present invention, the above device can also be defined as an image acquisition module. The device for enhancing the robustness of non-contact blood pressure monitoring by utilizing spatial redundancy can also include a signal analysis module, a model training module and a blood pressure prediction module, wherein:

[0068] Signal analysis module, used to obtain rPPG signals in continuous video frames, perform noise reduction on them, and extract muti-site PTT and muti-wavelength PTT;

[0069] Model training module, used to train the model between PTT features and BP;

[0070] The blood pressure prediction module uses the model trained in the model training module to predict and display the blood pressure based on the PTT features extracted in the image acquisition module and signal analysis module.

[0071] The signal analysis module, model training module and blood pressure prediction module are operations implemented by software, not actual hardware devices.

[0072] The present invention also protects a method for enhancing the robustness of non-contact blood pressure monitoring by utilizing spatial redundancy. This method is implemented using the aforementioned apparatus and can also be understood as processing and applying the optical signal collected by the optical sensor module. The method primarily comprises the following steps:

[0073] Step S1: Noise reduction: Since the data acquired by the camera may be affected by noise such as motion and lighting, the raw signal collected by the multi-band narrowband camera needs to be processed by the quality metric image (i.e., quality metric image) noise reduction algorithm to obtain a pure rPPG;

[0074] In this embodiment, the Quality Metric Image denoising algorithm used includes the following sub-steps:

[0075] Step S1-1: Identify the skin area captured by the camera, segment the skin and non-skin parts in the image captured by the camera, and remove the non-skin parts. There are many algorithms for identifying skin and non-skin areas, and they are relatively mature, such as: (1) Segment the skin and non-skin areas by skin color characteristics. Skin color varies among different people, but is usually within a narrow color range (usually in the RGB color space). A color model (such as YCbCr or HSV) can be used to identify skin pixels and segment the skin area by setting an appropriate threshold; (2) Build a skin color model based on known skin samples, which is then used to identify images or video frames. Skin regions in the image can be identified by using machine learning algorithms (e.g., support vector machines or deep learning models); (3) Skin generally has specific texture and shape features that can be used to distinguish skin from non-skin regions. These features are calculated and used to segment the image; (4) Deep learning techniques such as convolutional neural networks (CNNs) can be used for end-to-end skin region recognition. A CNN model is trained to automatically learn the skin regions in the image. Although this method requires a large amount of labeled data for training, it can generally provide high-precision results; (5) Skin regions can be identified by detecting small movements caused by the pulse, which can be achieved by optical flow estimation or motion tracking technology. After removing the non-skin region, it is also necessary to quantify the illumination of the skin region, that is, to extract the brightness of the skin region, segment the regions with obvious illumination changes, and the regions that are too dark or too bright, and select the region with appropriate illumination intensity as the preliminary selected region ROI. In the present invention, it is defined as ROI1. In this embodiment, the appropriate illumination intensity should comprehensively consider multiple factors such as signal quality, contrast between the skin region and the background, and human safety. The selected illumination intensity threshold is specifically set according to actual needs.

[0076] Step S1-2: Use the spatial redundancy of the image to divide ROI1 into several boxes of equal area. The number of boxes is subject to individual differences, device differences, box size, and whether overlap is required. It is automatically adjusted according to the area of ​​the final ROI. Extract the rPPG signals of the g channel and ir channel in each box, and perform spectrum analysis on them. In the frequency domain, extract the frequencies with the largest amplitudes of the g channel and ir channel in the frequency range of [0.5, 6.0] Hz and record them as , select The minimum or equal (if there are several equal minimum values) boxes are used as the final ROI. This area is the metricImage, which is defined as ROI2 in the present invention. In this embodiment, the box size selected is 100×100 pixels with 1 / 2 overlap. Correspondingly, in specific implementation, the size of 80×80 pixels, 120×120 pixels, 160×160 pixels, etc. can also be selected according to the usage scenario, and 1 / 3, 1 / 4 overlap or no overlap can be selected.

[0077] Step S1-3: extract the rPPG signal in ROI2, perform DC removal and bandpass filtering on it, and then obtain a pure rPPG signal after noise reduction.

[0078] In the above embodiment, the Quality Metric Image denoising algorithm is preferably used to process the image. During specific implementation, other conventional algorithms may also be used, such as: Mean Filter, Median Filter, Bilateral Filter, Wavelet Denoising, Total Variation Denoising, Non-local Means Denoising, Adaptive Denoising, etc. The main purpose is to process the image and highlight the skin area to make the detection result more accurate.

[0079] Step S2, signal extraction: extracting the muti-site PTT and muti-wavelength PTT from the rPPG signal using an adaptive PTT extraction algorithm;

[0080] In this embodiment, the adaptive PTT extraction algorithm used includes the following sub-steps:

[0081] Step S2-1: The original rPPG signal is subjected to quality metric image noise reduction to obtain a pure rPPG signal. Furthermore, a peak detection algorithm (i.e., peak detction) is used to detect peaks and troughs in the terminal rPPG signal (i.e., the terminal rPPG in multi-site PTT or the G-channel rPPG in multi-wavelength PTT. Since the present invention is typically performed on the user's forehead, and human blood vessels flow from bottom to top, blood flows from bottom to top, several boxes at the bottom are selected as the starting point, and several boxes vertically farthest from them are selected as the ending point). The first-order derivative of the signal during the rising phase (which corresponds to the heart's contraction process) is calculated to obtain the point with the maximum first-order derivative, i.e., the steepest point (i.e., the point with the most obvious feature change).

[0082] Step S2-2: extracting the time interval between two adjacent pulses in the terminated rPPG signal (i.e., the inter-beat interval, abbreviated as IBI), and determining the PTT threshold based on the IBI;

[0083] Step S2-3: Use the same method to calculate the steepest point of the starting rPPG signal (the starting point rPPG for muti-site PTT, the IR channel rPPG for muti-wavelength PTT). The time difference between the two points is calculated and a threshold is used to determine whether it is a valid PTT. The average of the valid PTT over several heartbeat cycles is used as the final PTT. In this embodiment, the PTT threshold is 1 / 2 IBI, but 1 / 3 IBI or 1 / 4 IBI can also be used in different usage scenarios and settings. The average of the valid PTT over 10 heartbeat cycles is used as the final PTT. Other appropriate numbers of heartbeat cycles can also be used in different usage scenarios and on different devices.

[0084] In this embodiment, the process of muti-site PTT extraction includes the following sub-steps:

[0085] Step S2-1-1. Select the two or more boxes with the longest vertical distance in the metricImage in step S1 (i.e., the two or more boxes with the longest vertical distance in the valid ROI) and determine the starting and ending points of the pulse wave signal based on the direction of the blood vessels. In this embodiment, since the application is to the forehead, the direction of human blood vessels is from bottom to top, and thus blood flows from bottom to top, so several boxes at the bottom are selected as the starting points, and several boxes at the farthest vertical distance from them are selected as the ending points.

[0086] Step S2-1-2: Extract the rPPG signals at the start and end points, and extract the mutual-site PTT using an adaptive PTT extraction algorithm. In this embodiment, the G channel rPPG signal is extracted because the G channel signal is the most stable. The rPPG signals in three boxes are selected for both the start and end points. The number of boxes can be adjusted to 2, 4, 5, 6, or more depending on the box size and image size.

[0087] In this embodiment, the process of extracting the multi-wavelength PTT includes the following sub-steps:

[0088] Step S2-2-1, using the metricImage extracted in step S1 after denoising (i.e., the rPPG signals of the G channel and IR channel in the valid ROI are selected), recorded as g_rPPG and ir_rPPG, respectively, which carry the physiological information of the arterioles and arteries, respectively;

[0089] Step S2-2-2: Extract the muti-wavelength PTT using an adaptive PTT extraction algorithm.

[0090] In the above embodiment, an adaptive PTT extraction algorithm is adopted. In specific implementation, other algorithms in the prior art may also be adopted, such as a multi-position (multi-point) PTT extraction algorithm, a multi-wavelength (multi-spectrum) PTT extraction algorithm, etc.

[0091] Step S3, blood pressure estimation: combining the feature information of multi-site PTT (i.e., multi-wavelength PTT) and multi-band PTT (i.e., multi-feature model) to estimate blood pressure.

[0092] In this paper, muti-site PTT is defined as the time difference between PPG signals at different locations within the same blood vessel. This means that muti-site PTT can be extracted from a single-band PPG signal. This definition shows that it is highly dependent on the distance between the two points and less dependent on vessel wall thickness. Muti-wavelength PTT is defined as the time difference between PPG signals at the same location but at different blood vessels. This is highly dependent on vessel wall thickness, but can be measured from only one point. Based on the complementary characteristics of these two PTT methods, the present invention proposes a multi-feature model for estimating blood pressure. This feature model algorithm includes the following substeps:

[0093] Step S3-1, extracting the muti-site PTT and its starting point and ending point rPPG signals (respectively labeled as s_rPPG and e_rPPG in the present invention), the muti-wavelength PTT, and the g_rPPG and ir_rPPG signals;

[0094] Step S3-2, respectively calculating the difference between the muti-site PTT and the muti-wavelength PTT (labeled as dif1 in the present invention), the difference between s_rPPG and g_rPPG and ir_rPPG (labeled as dif2 and dif3 in the present invention), and the difference between e_rPPG and g_rPPG and ir_rPPG (labeled as dif4 and dif5 in the present invention);

[0095] Step S3-3: Using muti-site PTT, muti-wavelength PTT, and dif (including dif1, dif2, dif3, dif4, and dif5) as input features, a multi-feature regression model is fitted between the multi-feature PTT and blood pressure (including systolic, diastolic, and mean pressure) using polynomial regression. The differences between the two PTT features and their corresponding PPG signals are used to constrain and calibrate each other, thereby improving the robustness and generalization of non-contact blood pressure monitoring.

[0096] The present invention also protects a system for enhancing the robustness of non-contact blood pressure monitoring by utilizing spatial redundancy, the system mainly comprising:

[0097] Noise reduction processing unit: used to perform noise reduction processing on the raw signal collected by the multi-band narrowband camera. Since the data obtained by the camera will be interfered by noise such as motion and lighting, the raw signal collected by the multi-band narrowband camera needs to be processed by the quality metric image (i.e., quality metric image) noise reduction algorithm to obtain a pure rPPG;

[0098] In this embodiment, the Quality Metric Image denoising algorithm used includes the following subunits:

[0099] Non-skin part removal subunit: used to identify the skin area captured by the camera, segment the skin and non-skin parts in the image captured by the camera, and remove the non-skin part. There are many algorithms for identifying skin and non-skin areas, and they are relatively mature, such as: (1) Segmenting skin and non-skin areas by skin color features. Skin color varies among different people, but is usually within a narrow color range (usually in RGB color space). Color models (such as YCbCr or HSV) can be used to identify skin pixels and segment skin areas by setting appropriate thresholds; (2) Establishing a skin color model based on known skin samples, which is then used to identify images or Skin regions in video frames can be identified by using machine learning algorithms (e.g., support vector machines or deep learning models); (3) Skin generally has specific texture and shape features that can be used to distinguish skin from non-skin regions. These features are calculated and used to segment images; (4) Deep learning techniques such as convolutional neural networks (CNNs) can be used for end-to-end skin region recognition. A CNN model is trained to automatically learn the skin regions in the image. Although this method requires a large amount of labeled data for training, it can generally provide high-precision results; (5) Skin regions can be identified by detecting small movements caused by the pulse, which can be achieved by optical flow estimation or motion tracking technology. After removing the non-skin region, it is also necessary to quantify the illumination of the skin region, that is, to extract the brightness of the skin region, segment the regions with obvious illumination changes, and the regions that are too dark or too bright, and select the region with appropriate illumination intensity as the preliminary selected region ROI. In the present invention, it is defined as ROI1. In this embodiment, the appropriate illumination intensity should comprehensively consider multiple factors such as signal quality, contrast between the skin region and the background, and human safety. The selected illumination intensity threshold is specifically set according to actual needs.

[0100] Region division subunit: It is used to divide ROI1 into several boxes of equal area by utilizing the spatial redundancy of the image. The number of boxes is subject to individual differences, device differences, box size, and whether overlap is required. It is automatically adjusted according to the area of ​​the final ROI. The rPPG signals of the g channel and ir channel in each box are extracted and spectrally analyzed. In the frequency domain, the frequencies with the largest amplitudes of the g channel and ir channel are extracted within the frequency range of [0.5, 6.0] Hz and recorded as , select The minimum or equal (if there are several equal minimum values) boxes are used as the final ROI. This area is the metricImage, which is defined as ROI2 in the present invention. In this embodiment, the box size selected is 100×100 pixels with 1 / 2 overlap. Correspondingly, in specific implementation, the size of 80×80 pixels, 120×120 pixels, 160×160 pixels, etc. can also be selected according to the usage scenario, and 1 / 3, 1 / 4 overlap or no overlap can be selected.

[0101] Denoising subunit: It is used to extract the rPPG signal in ROI2 and perform operations such as DC removal and bandpass filtering to obtain the pure rPPG signal after noise reduction.

[0102] Signal extraction unit: Adopts adaptive PTT extraction algorithm to extract muti-site PTT and muti-wavelength PTT from rPPG signals respectively;

[0103] In this embodiment, the signal extraction unit includes the following subunits:

[0104] Rising signal calculation subunit: The original rPPG signal is subjected to quality metric image denoising to obtain a pure rPPG signal. Furthermore, a peak detection algorithm (i.e., peak detction) is used to detect peaks and troughs in the ending rPPG signal (i.e., the ending point rPPG in muti-site PTT, or the G-channel rPPG in muti-wavelength PTT. Since the present invention is usually performed on the user's forehead, and human blood vessels flow from bottom to top, blood flows from bottom to top, several boxes at the bottom are selected as the starting point, and several boxes vertically farthest away from them are selected as the ending point). The first-order derivative of the signal during the rising period (which corresponds to the heart's contraction process) is calculated to obtain the point with the maximum first-order derivative, i.e., the steepest point (i.e., the point where the characteristic change is most obvious).

[0105] IBI calculation subunit: used to extract the time interval between two adjacent pulses in the termination rPPG signal (i.e., Inter-Beat interval, abbreviated as IBI) and determine the PTT threshold based on the IBI;

[0106] The PTT determination subunit uses the same method to calculate the steepest point of the starting rPPG signal (the starting point rPPG for muti-site PTT and the IR channel rPPG for muti-wavelength PTT), calculates the time difference between the two points, and uses a threshold to determine whether it is a valid PTT. The average of the valid PTT over several heartbeat cycles is used as the final PTT. In this embodiment, the PTT threshold is 1 / 2 IBI, but 1 / 3 IBI or 1 / 4 IBI can also be used in different usage scenarios and settings. The average of the valid PTT over 10 heartbeat cycles is used as the final PTT. Other appropriate numbers of heartbeat cycles can also be used in different usage scenarios and on different devices.

[0107] In this embodiment, the process of muti-site PTT extraction includes the following sub-steps:

[0108] The pulse wave signal determination subunit is used in the noise reduction processing unit to select the two or more boxes with the longest vertical distance in the metricImage (i.e., the two or more boxes with the longest vertical distance in the valid ROI) and determine the starting and ending points of the pulse wave signal based on the direction of the blood vessels. In this embodiment, since the application is in the forehead, human blood vessels flow from bottom to top, and blood flows from bottom to top, the lower boxes are selected as the starting points, and the boxes with the longest vertical distance from them are selected as the ending points.

[0109] The multi-site PTT extraction subunit is used to extract the rPPG signals at the start and end points, and to extract the multi-site PTT using an adaptive PTT extraction algorithm. In this embodiment, the rPPG signal is extracted from the G channel, as the G channel signal is the most stable. The rPPG signals in three boxes are selected for both the start and end points. The number of boxes can be adjusted to 2, 4, 5, 6, or more depending on the box size and image size.

[0110] In this embodiment, the process of multi-wavelength PTT extraction includes the following subunits:

[0111] The rPPG signal extraction subunit is used to extract the rPPG signals of the G channel and IR channel after metricImage denoising (i.e., the rPPG signals of the G channel and IR channel in the effective ROI are selected) using the denoising processing unit. These signals are recorded as g_rPPG and ir_rPPG, respectively, and carry the physiological information of the arterioles and arteries, respectively.

[0112] muti-wavelength PTT extraction subunit: used to extract muti-wavelength PTT using an adaptive PTT extraction algorithm.

[0113] Blood pressure estimation unit: used to estimate blood pressure using a multi-feature model by combining the feature information of multi-point PTT (i.e., muti-site PTT) and multi-band PTT (i.e., muti-wavelength PTT).

[0114] In this paper, muti-site PTT is defined as the time difference between PPG signals at different locations within the same blood vessel. This means that muti-site PTT can be extracted from a single-band PPG signal. This definition shows that it is highly dependent on the distance between the two points and less dependent on vessel wall thickness. Muti-wavelength PTT is defined as the time difference between PPG signals at the same location but at different blood vessels. This is highly dependent on vessel wall thickness, but can be measured from only one point. Based on the complementary characteristics of these two PTT methods, this paper proposes a multi-feature model for estimating blood pressure. This feature model algorithm includes the following subunits:

[0115] Component information extraction subunit: used to extract the muti-site PTT and its starting point and ending point rPPG signals (respectively labeled as s_rPPG and e_rPPG in the present invention), the muti-wavelength PTT, and the g_rPPG and ir_rPPG signals respectively;

[0116] Difference calculation subunit: used to calculate the difference between muti-site PTT and muti-wavelength PTT (labeled as dif1 in the present invention), the difference between s_rPPG and g_rPPG and ir_rPPG (labeled as dif2 and dif3 in the present invention), and the difference between e_rPPG and g_rPPG and ir_rPPG (labeled as dif4 and dif5 in the present invention);

[0117] Input subunit: used to take muti-site PTT, muti-wavelength PTT and dif (including dif1, dif2, dif3, dif4 and dif5) as input features, and use polynomial regression to fit a multi-feature regression model between multi-feature PTT and blood pressure (including systolic pressure, diastolic pressure and mean pressure). The difference between the two PTT features and their corresponding PPG signals is used to constrain and calibrate each other, thereby achieving the purpose of improving the robustness and generalization of non-contact blood pressure monitoring.

[0118] The present invention also protects a storage medium which stores a program capable of implementing the above method when running.

[0119] It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above and that the invention can be embodied in other specific forms without departing from the spirit or essential characteristics of the invention. Therefore, the embodiments should be considered in all respects as illustrative and non-restrictive, and the scope of the invention is defined by the appended claims, not the foregoing description, and all variations within the meaning and range of equivalents of the claims are intended to be included therein. Any reference sign in a claim should not be construed as limiting the claim to which it relates.

[0120]

Claims

1. A device for enhancing the robustness of non-contact blood pressure monitoring by using spatial redundancy, characterized by: The device comprises a light source module and an optical sensor module. The light source module is a light source capable of providing continuous visible light and near-infrared light, and the optical sensor module adopts a multi-band narrow-band image acquisition module.

2. The device for enhancing the robustness of non-contact blood pressure monitoring by utilizing spatial redundancy according to claim 1, characterized in that: The light source module uses a DC full-spectrum incandescent lamp that can provide continuous visible light and near-infrared light in the range of 440nm-940nm. The multi-band narrowband image acquisition module is a camera that uses a narrowband three-bandpass filter that allows near-infrared light to pass through, so that the camera can simultaneously capture information carried by visible light with a wavelength of 550nm, visible light with a wavelength of 660nm, and near-infrared light with a wavelength of 850nm.

3. A method for enhancing the robustness of non-contact blood pressure monitoring by utilizing spatial redundancy, implemented by the device for enhancing the robustness of non-contact blood pressure monitoring by utilizing spatial redundancy as claimed in claim 1 or 2, characterized in that: The method comprises the following steps: Step S1, noise reduction processing: the original signal collected by the multi-band narrowband camera is processed by the quality metricImage noise reduction algorithm to obtain a pure rPPG; Step S2, signal extraction: extracting muti-site PTT and muti-wavelength PTT from the rPPG signal respectively by using an adaptive PTT extraction algorithm; Step S3, blood pressure estimation: combining the feature information of multi-point PTT and multi-band PTT to estimate blood pressure using a multi-feature model.

4. The method for enhancing the robustness of non-contact blood pressure monitoring by utilizing spatial redundancy according to claim 3, characterized in that: In step S1, the quality metricImage denoising algorithm used includes the following sub-steps: Step S1-1, identifying the skin area captured by the camera and quantifying the light intensity of the skin area, selecting an area with appropriate light intensity as a selected area ROI, defined as ROI1; Step S1-2: Use the spatial redundancy of the image to divide ROI1 into several boxes of equal area, extract the rPPG signals of the g channel and ir channel in each box, perform spectrum analysis on them, and extract the frequencies with the largest amplitudes of the g channel and ir channel in the frequency range of [0.5, 6.0] Hz and record them as , select The smallest or equal boxes are used as the final ROI. This area is the metricImage and is defined as ROI2. Step S1-3, extract the rPPG signal in ROI2, perform DC removal and bandpass filtering operations on it, and obtain a pure rPPG signal after noise reduction.

5. The method for enhancing the robustness of non-contact blood pressure monitoring by utilizing spatial redundancy according to claim 3, characterized in that: In step S2, the adaptive PTT extraction algorithm includes the following sub-steps: Step S2-1, using a peak detection algorithm to detect the peaks and valleys in the terminated rPPG signal, and calculating the first-order derivative of the rising period of the signal to obtain the point where the first-order derivative is the largest; Step S2-2, extracting the time interval between two adjacent pulses in the termination signal, i.e., IBI, and determining the threshold of PTT according to IBI; Step S2-3, using the same method to calculate the steepest point of the starting rPPG signal, calculate the time difference between the two points, and determine whether it is a valid PTT through a threshold; In step S2, the process of muti-site PTT extraction includes the following sub-steps: Step S2-1-1, selecting two or more boxes with the longest vertical distance in the metricImage in step S1 and determining the starting point and the ending point of the pulse wave signal according to the direction of the blood vessel; Step S2-1-2, extracting the rPPG signals of the starting point and the ending point, and extracting the muti-site PTT using an adaptive PTT extraction algorithm; Step S2-2-1, extracting the rPPG signals of the G channel and the IR channel after noise reduction in step S1, respectively denoted as g_rPPG and ir_rPPG, which carry the physiological information of the arterioles and the aortas, respectively; Step S2-2-2: Extract the muti-wavelength PTT using an adaptive PTT extraction algorithm.

6. The method for enhancing the robustness of non-contact blood pressure monitoring by utilizing spatial redundancy according to claim 3, characterized in that: In step S3, the feature model algorithm includes the following sub-steps: Step S3-1, extract the muti-site PTT and its starting point and ending point rPPG signals respectively, marked as s_rPPG and e_rPPG respectively, and extract the muti-wavelength PTT and g_rPPG and ir_rPPG signals respectively; Step S3-2, calculating the difference between the muti-site PTT and the muti-wavelength PTT, marked as dif1; Calculate the difference between s_rPPG and g_rPPG, marked as dif2; calculate the difference between s_rPPG and ir_rPPG, marked as dif3; calculate the difference between e_rPPG and g_rPPG, marked as dif4; calculate the difference between e_rPPG and ir_rPPG, marked as dif5; Step S3-3, using muti-sitePTT, muti-wavelength PTT, dif1, dif2, dif3, dif4 and dif5 as input features, and inputting polynomial regression to fit a multi-feature regression model between multi-feature PTT and blood pressure.

7. A system for enhancing the robustness of non-contact blood pressure monitoring using spatial redundancy, characterized by: The system comprises: Noise reduction processing unit: used to process the original signal collected by the multi-band narrowband camera through the quality metricImage noise reduction algorithm to obtain a pure rPPG; Signal extraction unit: Adopts adaptive PTT extraction algorithm to extract muti-site PTT and muti-wavelength PTT from rPPG signals respectively; Blood pressure estimation unit: used to estimate blood pressure using a multi-feature model by combining the feature information of multi-point PTT and multi-band PTT.

8. The system for enhancing the robustness of non-contact blood pressure monitoring by using spatial redundancy according to claim 7, characterized in that: The noise reduction processing unit comprises: Non-skin part removal subunit: used to identify the skin area captured by the camera and quantify the light intensity of the skin area, and select the area with appropriate light intensity as the selected area ROI, which is defined as ROI1; Region division subunit: It is used to divide ROI1 into several boxes of equal area by utilizing the spatial redundancy of the image, extract the rPPG signals of the g channel and ir channel in each box, perform spectrum analysis on them, and extract the frequencies with the largest amplitudes of the g channel and ir channel in the frequency range of [0.5, 6.0] Hz, which are recorded as , select The smallest or equal boxes are used as the final ROI. This area is the metricImage and is defined as ROI2. Denoising subunit: used to extract the rPPG signal in ROI2, perform DC removal and bandpass filtering operations on it, and thus obtain the pure rPPG signal after noise reduction.

9. The system for enhancing the robustness of non-contact blood pressure monitoring by using spatial redundancy according to claim 7, characterized in that: The signal extraction unit comprises: Rising signal calculation subunit: used to detect the peaks and valleys in the termination rPPG signal through the peak detection algorithm, and calculate the first-order derivative of the rising period of the signal to obtain the point with the maximum first-order derivative; IBI calculation subunit: used to extract the time interval between two adjacent pulses in the termination signal, i.e., IBI, and determine the PTT threshold according to IBI; PTT determination subunit: used to calculate the steepest point of the starting rPPG signal using the same method, calculate the time difference between the two points, and determine whether it is a valid PTT through a threshold; In the signal extraction unit, the muti-site PTT extraction includes the following subunits: The pulse wave signal determination subunit is used to select two or more boxes with the longest vertical distance in the metricImage in the noise reduction processing unit and determine the starting point and the ending point of the pulse wave signal according to the direction of the blood vessel; muti-site PTT extraction subunit: used to extract the rPPG signals of the starting and ending points, and extract the muti-site PTT using an adaptive PTT extraction algorithm; In the signal extraction unit, the process of muti-wavelength PTT extraction includes the following subunits: rPPG signal extraction subunit: used to extract the denoised rPPG signals of the G channel and IR channel using the denoising processing unit, which are recorded as g_rPPG and ir_rPPG, respectively, and carry the physiological information of the arterioles and large arteries, respectively; muti-wavelength PTT extraction subunit: used to extract muti-wavelength PTT using an adaptive PTT extraction algorithm.

10. The method for enhancing the robustness of non-contact blood pressure monitoring by utilizing spatial redundancy according to claim 7, characterized in that: The blood pressure estimation unit includes the following subunits: Component information extraction subunit: used to extract the muti-site PTT and its starting point and end point rPPG signals, marked as s_rPPG and e_rPPG, respectively, and extract the muti-wavelength PTT and g_rPPG and ir_rPPG signals respectively; Difference calculation subunit: used to calculate the difference between muti-site PTT and muti-wavelength PTT, marked as dif1; Calculate the difference between s_rPPG and g_rPPG, marked as dif2; calculate the difference between s_rPPG and ir_rPPG, marked as dif3; calculate the difference between e_rPPG and g_rPPG, marked as dif4; calculate the difference between e_rPPG and ir_rPPG, marked as dif5; Input subunit: used to take muti-sitePTT, muti-wavelength PTT, dif1, dif2, dif3, dif4 and dif5 as input features, and input polynomial regression to fit a multi-feature regression model between multi-feature PTT and blood pressure.

Images