A blood pressure prediction method for face videos based on multi-scale attention constraints

Abstract

The present invention belongs to the field of computer vision technology, and discloses a method for predicting blood pressure from a face video based on multi-scale attention constraints, comprising the following steps: Step 1: extracting a remote photoplethysmogram rPPG signal; firstly, the face is detected in the video by the MTCNN face detection algorithm and the region of interest ROI is divided, and the effective face area is extracted, and then the rPPG signal carrying noise is preliminarily extracted by combining optical flow estimation and median filtering. In the blood pressure prediction part, the rPPG signal is subjected to baseline drift removal and re-filtering to obtain a relatively clean signal, and then the rPPG signal is windowed; Step 2: designing an EBP-Net model, and inputting the rPPG signal into the EBP-Net model to predict systolic and diastolic pressure. The present invention simplifies the types of signals required for blood pressure measurement, and only requires rPPG signals, and the measurement error is minimized, which reduces the cost of signal acquisition.

Description

Technical Field

[0001] The present invention belongs to the field of computer vision technology, and in particular relates to a method for predicting blood pressure from a face video based on multi-scale attention constraints. Background Art

[0002] In daily home blood pressure monitoring, in addition to traditional blood pressure meters, the mainstream method currently used by researchers is still to use some non-end-to-end measurement methods such as the combination of multiple physiological signals. In the field of blood pressure measurement using machine learning, the combination of photoplethysmography and deep learning is the most common method. In recent years, some researchers have also begun to explore the use of PPG signals, ECG, BVP and other methods to predict blood pressure. Among them, Liu et al. proposed a representative method by extracting 21 traditional PPG features and 14 second-order derivative features, combining the support vector regression algorithm to train the estimation module, and achieved good results. Since the linear relationship between the pulse wave transmission time and arterial blood pressure remains relatively stable for an individual due to the stability of the individual's physiological state over a period of time, Xu et al. used this feature to propose a method for capturing dynamic blood pressure using a high-speed camera. This method uses a high-speed camera to shoot a video of the forearm, capture the pulse wave, and calculate the time difference between the pulse reaching different positions of the arm to obtain the pulse transmission time (PTT), thereby achieving blood pressure measurement. However, in actual application scenarios, it is obviously unrealistic to use the arm to measure blood pressure, especially affected by wearable products. Similarly, Secerbegovic et al. used the same principle to simultaneously acquire the PPG signals of the face and palm of a person using a camera during the signal acquisition process, and calculated PTT based on the arrival time difference of the PPG signal. However, the blood pressure measurement method based on pulse time difference has certain limitations, because the parameters are different under different physiological states, and the established linear model lacks universality. In addition to using PPG signals and their second-order derivatives, researchers such as Lin proposed a new parameter index-the first-order derivative wave intensity ratio of PPG, that is, the amplitude ratio between the first trough and the peak in a cardiac cycle. After introducing this parameter into the model, the absolute mean and standard deviation of systolic and diastolic pressures were reduced by 0.34±0.27mmHg and 0.33±0.44mmHg, respectively. Although the results show significant effects, its disadvantage is still the difficulty in acquiring signals, and more importantly, the contact pressure of the PPG sensor affects the pulse morphology. In addition to the above methods, dual-channel PPG signals also show good performance. For example, Kim et al. developed a small finger-cuff blood pressure measurement device that obtains two 940nm PPG signals from different positions of the same finger. This is similar to the non-invasive continuous blood pressure measurement method of pulse wave phase difference used by Li et al. Li et al. measured the blood pressure values ​​in the recovery phase and resting phase after exercise respectively. This method does not require the measurement of ECG signals, and the error is stable at about 5% compared with the pneumatic automatic blood pressure measurement device. However, the common problem with these methods is that the measurement process is cumbersome, requiring multiple instruments and strict time synchronization requirements.

[0003] Since 2018, end-to-end blood pressure measurement methods have gradually emerged. The earliest article proposed the concept of rPPG, laying the foundation for research in this field. Schrumpf et al. compared the non-invasive blood pressure prediction of PPG and rPPG signals. The results showed that the method of measuring blood pressure using rPPG signals has great application prospects. In recent years, some researchers have begun to use end-to-end methods to measure blood pressure. For example, Yan et al. extracted pulse wave signals from the face ROI and used a deep learning model to extract features and predict blood pressure. Similarly, Wang et al. designed a deep network model based on the attention fusion mechanism, using a bidirectional long short-term memory neural network to automatically extract waveform features, and combined self-attention mechanism and personal characteristics to measure blood pressure. What these two may overlook is the selection of the region of interest (ROI). Although the blood flow signal of the cheek part is more obvious, it is obviously not comparable to select only one ROI. Moreover, with the development of deep learning, LSTM has gradually popped up in the public eye; Eom et al. proposed an end-to-end deep learning architecture for continuous blood pressure assessment based on the attention mechanism. Although the average error of the experimental results is relatively low, the experimental method requires the use of three signals. In addition, Zhuang et al. used a different ROI segmentation method to divide the face into four regions and predicted blood pressure through the designed blood pressure classifier and calculator. However, the problem with these methods is that the blood pressure measurement process requires the person's personal information (such as BMI, age, gender, etc.).

[0004] In summary, the existing technology has the following problems: 1. Most blood pressure measurement methods require the combination of multiple signals: such as PPG, ECG and other multiple signals to measure, which will lead to cumbersome measurement process and difficulty in collecting signals. 2. The existing blood pressure measurement methods based on face video are generally less accurate. Summary of the invention

[0005] The purpose of the present invention is to provide a face video blood pressure prediction method based on multi-scale attention constraints to solve the above-mentioned technical problems.

[0006] In order to solve the above technical problems, the specific technical solution of the face video blood pressure prediction method based on multi-scale attention constraint of the present invention is as follows:

[0007] A method for predicting blood pressure from face videos based on multi-scale attention constraints, comprising the following steps:

[0008] Step 1: Extract remote photoplethysmography (rPPG) signals. First, the face is detected in the video using the MTCNN face detection algorithm and the region of interest (ROI) is divided to extract the effective face area. Then, the rPPG signal with noise is initially extracted by combining optical flow estimation and median filtering. In the blood pressure prediction part, the rPPG signal is baseline-drifted and filtered again to obtain a relatively clean signal. The rPPG signal is then window-segmented.

[0009] Step 2: Design the blood pressure level detection model EBP-Net, and input the rPPG signal into the EBP-Net model to predict systolic and diastolic blood pressure.

[0010] Furthermore, the step 1 includes the following specific steps:

[0011] Step 1.1: Extracting remote photoplethysmography (rPPG) signals from face videos in visible light scenes;

[0012] Step 1.2: For the extracted ROI, use the optical flow estimation method to obtain the motion information of the pixels in the face area, and then calculate the optical flow vector of each pixel in the image to describe the changes of the image over time.

[0013] Furthermore, in the step 1.1, in the remote signal extraction rPPG signal stage, for each frame of the video, the face detection algorithm MTCNN is used. To perform face detection, MTCNN uses multi-scale sliding window detection and applies the CNN model to classify and regress each candidate window to determine the position and size of the face frame; secondly, for each detected face frame, the corresponding face image area is cropped from the original image, that is, the part containing the face is extracted.

[0014] Furthermore, the step 1.2 extracts the microvascular motion signal caused by the heartbeat by analyzing the change of the optical flow vector. I(x,y,t) , calculate the spatial gradient of the image I x 、I y and time gradient I t To estimate, in the optical flow estimation, the Lucas-Kanade method is used to calculate the optical flow vector by minimizing the gray value change of the pixels in the window (v x ,v y ) , the change of the gray value of each pixel in the window over time is represented by a vector, and the equation group for solving the optical flow vector is shown in formula (1):

[0015] ,

[0016] in, I x , I y and I t Respectively represent the gradient of the image in the x direction, y direction and time direction, Ω represents the set of pixels in the window, ( v x ,v y ) represents the optical flow vector. By solving the above equations, the optical flow vector of each pixel is obtained, thereby obtaining the motion information in the face area. The linear filtering technology of median filtering and mean filtering is used to reduce the noise level in the signal by eliminating the high-frequency components in the signal.

[0017] Furthermore, the step 2 comprises the following steps:

[0018] Step 2.1: Propose a signal processing method model EBP-Net that includes multi-task and multi-scale;

[0019] Step 2.2: Design an improved multi-scale fusion module MSF;

[0020] Step 2.3: Design an improved multi-scale attention module EMA.

[0021] Furthermore, the model EBP-Net of step 2.1 selects the facial rPPG signal and its first-order and second-order derivatives as the model input; selects to use the multi-scale fusion module MSF to combine information of different scales to capture the hierarchical structure and complex patterns in the signal; introduces and improves the EMA module to help the model predict blood pressure values ​​by capturing the long-term change trend of the features.

[0022] Furthermore, the step 2.2 includes the following specific steps:

[0023] First, the facial rPPG signal and its derivative are concatenated as input. The input features are first passed through the first PW convolution to change the channel of the input features to adapt to the subsequent DW convolution. The two DW convolutions use convolution kernels of sizes 5x1 and 3x1 respectively. Secondly, the results of the DW convolution are average pooled and fully connected, and the processing results are multiplied with the output features of the DW convolution. Finally, the input features are residually linked with the results of the initial PW convolution. One-dimensional convolution is used, and the attention mechanism of the softmax weighted operation is used to dynamically extract larger features. Residual connection is used to compensate for the differences in rPPG signals and retain the original features.

[0024] Furthermore, the step 2.3 includes the following specific steps:

[0025] By introducing a dynamic weighting mechanism, the importance of features is adjusted according to changes in time, and the changes in features are smoothed using an exponential moving average. The multi-scale attention module EMA receives the output features of the multi-scale fusion module MSF as input, and first divides them into several sub-features in the depth direction. For each sub-feature, it is a one-dimensional signal. The multi-scale attention module EMA divides this signal into two steps to operate on it. In the first step, the input features are pooled and one-dimensionally convolved, and the output of the activation function sigmoid is multiplied by the starting input. In the second step, in another branch, the input features are firstly convolved and pooled with a 3x1 one-dimensional convolution, and then connected and multiplied with several output branches in the first step. Finally, an output feature with the same size as the input feature is obtained.

[0026] Furthermore, step 2.3 includes the step of enhancing the multi-scale attention module EMA to recognize the time series features of the one-dimensional rPPG signal: first, the grouping factor used is increased to 32; second, a nonlinear transformation of the output is added to the activation function; and finally, group normalization is used to improve stability and convergence speed.

[0027] Furthermore, the specific steps of group normalization are:

[0028] First, the rPPG signal is converted into a tensor and the tensor is reshaped and grouped. The purpose is to calculate the mean and variance within each group. The formula for calculating the mean for each group is as follows:

[0029] ,

[0030] in, H and W are the height and width within the group; the variance is calculated as follows:

[0031] ,

[0032] Finally, the transformed tensor is normalized using the mean and variance, as follows:

[0033] ,

[0034] in, tensor is the original tensor and ϵ is a constant.

[0035] The face video blood pressure prediction method based on multi-scale attention constraint of the present invention has the following advantages:

[0036] The types of signals required for blood pressure measurement are simplified to only rPPG signals.

[0037] In the field of blood pressure measurement using facial video, the measurement error is minimal.

[0038] The acquisition method used in the present invention only requires a consumer-grade camera and a finger-clip blood oximeter, which reduces the cost of signal acquisition. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] Figure 1 It is a schematic diagram of the overall process of the present invention;

[0040] Figure 2 is the EBP-Net flow chart of the present invention;

[0041] Figure 3 Schematic diagram of the multi-scale fusion module (MSF) of the present invention;

[0042] Figure 4 Schematic diagram of the multi-scale attention module (EMA) of the present invention. DETAILED DESCRIPTION

[0043] In order to better understand the purpose, structure and function of the present invention, the following is a further detailed description of a face video blood pressure prediction method based on multi-scale attention constraints in conjunction with the accompanying drawings.

[0044] A method for predicting blood pressure from a face video based on multi-scale attention constraints of the present invention comprises the following steps:

[0045] Step 1: If Figure 1 As shown, extract the remote photoplethysmogram rPPG signal; first, detect the face in the video through the MTCNN face detection algorithm and divide the ROI (region of interest), extract the effective face area, and then combine the optical flow estimation and median filtering to preliminarily extract the rPPG signal with noise. The rPPG signal extracted by the optical flow method is more robust than the rPPG signal extracted manually. Due to the influence of ambient light and facial pseudo-motion, the extracted signal has more noise. Therefore, in the blood pressure prediction part, the rPPG signal is processed by removing the baseline drift and filtering again to obtain a relatively clean signal. In order to adapt to the model input, the signal is segmented into a 7-second window. The specific steps of step 1 are as follows:

[0046] Step 1.1: Extract remote photoplethysmography (rPPG) signals from face videos in visible light scenes.

[0047] In the remote signal extraction rPPG signal stage, for each frame of the video, the face detection algorithm MTCNN is used Perform face detection. MTCNN uses a multi-scale sliding window detection and a CNN model to classify and regress each candidate window, thereby determining the position and size of the face frame. Secondly, for each detected face frame, the present invention crops the corresponding face image area from the original image, that is, extracts the part containing the face, and the image size is 620*880.

[0048] Step 1.2: For the extracted ROI, the present invention uses the optical flow estimation method to obtain the motion information of the pixels in the face area, that is, the displacement information of the pixels in the same scene over time in the continuous frame images. Then, the optical flow vector of each pixel in the image is calculated to describe the change of the image over time. In this process, the microvascular motion signal caused by the heartbeat is extracted by analyzing the change of the optical flow vector. I(x,y,t) , calculate the spatial gradient of the image I x 、I y and time gradient I t In optical flow estimation, the Lucas-Kanade method is used to calculate the optical flow vector by minimizing the gray value change of the pixels in the window. (v x ,v y ) , the change of the gray value of each pixel in the window over time can be expressed by a vector, and the equation group for solving the optical flow vector is shown in formula (1):

[0049] ,

[0050] in, I x 、I y and I t Respectively represent the images in x direction, y The gradient in the direction and time direction, Ω represents the set of pixels in the window, ( v x ,v y) represents the optical flow vector. By solving the above equations, the optical flow vector of each pixel can be obtained, thereby obtaining the motion information in the face area. In addition, during the shooting process, the facial light and facial movements will be affected by the outside world. In order to eliminate these noises while retaining the original characteristics of the signal as much as possible to obtain a robust rPPG signal, the present invention adopts linear filtering techniques such as median filtering and mean filtering in noise filtering, which reduces the noise level in the signal by eliminating the high-frequency components in the signal.

[0051] Step 2: Design the blood pressure level detection model EBP-Net, and input the rPPG signal into the EBP-Net model to predict systolic and diastolic blood pressure.

[0052] Step 2.1: In order to improve the accuracy of the model in predicting blood pressure levels, the present invention proposes a multi-task and multi-scale signal processing method model (EBP-Net), which mainly includes a multi-scale fusion module MSF and a multi-scale attention module EMA. The model architecture diagram is shown in the attached Figure 2 . Ziegler and other researchers have shown that there is a close relationship between blood pressure and atherosclerosis, and the first-order derivative and second-order derivative of the photoplethysmogram just reflect the condition of atherosclerosis. Therefore, the present invention chooses to superimpose the facial rPPG signal and its first-order and second-order derivatives as the model input. In order to extract multi-scale feature representations from the input physiological signals, the present invention chooses to use the multi-scale fusion module MSF to combine information of different scales to capture the hierarchical structure and complex patterns in the signal, thereby improving the model's abstraction and expression capabilities for data features. In addition, the multi-scale attention module EMA is introduced and improved. By more accurately capturing the long-term change trend of features, the multi-scale attention module EMA can help the model predict blood pressure values ​​more accurately, especially in applications under long-term monitoring and unstable environments.

[0053] Step 2.2: Design an improved multi-scale fusion module MSF;

[0054] The multi-scale fusion module MSF is mainly composed of point convolution, depth convolution and dilation convolution, see the attached Figure 3. Using these technical means, the input signal is feature extracted and fused at multiple scales to improve the model's sensitivity and understanding of features at different scales. First, the facial rPPG signal and its first-order and second-order derivatives are concatenated as input features. The input features are first passed through the first PW convolution to change the channel of the input features to adapt to the subsequent DW convolution. The two DW convolutions use convolution kernels of sizes 5x1 and 3x1 respectively. The DW convolution is used because the key to blood pressure measurement lies in rPPG signal analysis. The spatial information contained in the pulse waveform at different positions can reflect the changes in blood pressure, and the DW convolution can just independently process the spatial features of each channel. The purpose of using convolution kernels of different sizes is to allow the model to capture multi-scale spatial features, especially focusing on local and global blood pressure features. Secondly, the results of the DW convolution are processed by average pooling, full connection, etc., and the processing results are multiplied by the output features of the DW convolution, mainly to realize the channel and space attention mechanism and dynamically adjust the weight of the features. Finally, the input features are residually linked with the results of the initial PW convolution, mainly to alleviate the gradient vanishing problem and promote the effective training and stability of the model.

[0055] In order to improve the adaptability of the module to one-dimensional time signals, the present invention adopts one-dimensional convolution in the multi-scale fusion module MSF, and dynamically extracts larger features through the attention mechanism of softmax weighted operation, thereby improving the robustness of the model. In addition, in the process of blood pressure prediction, individual rPPG signals also have differences. The present invention adopts residual connection to make up for this difference and retain the original features, further promotes gradient flow, and improves training efficiency. As an important component of the blood pressure estimation model, the multi-scale fusion module MSF significantly improves the model's ability to understand and predict physiological signals through the capture and hierarchical expression of multi-scale features, providing strong technical support for accurate estimation of blood pressure.

[0056] Step 2.3: Design an improved multi-scale attention module EMA;

[0057] like Figure 4As shown in the figure, the main purpose of the designed and improved multi-scale attention module EMA is to enhance the stability of the model and improve the prediction accuracy. By introducing a dynamic weighting mechanism, the importance of the features is adjusted according to the changes in time. This mechanism enables the model to process time series data more effectively and adapt to the changes and importance fluctuations of features in different time periods. At the same time, the exponential moving average method is used to smooth the changes in features and reduce the impact of noise and emergencies on the model. The multi-scale attention module EMA receives the output features of the multi-scale fusion module MSF as input. First, it is divided into several sub-features in the depth direction. For each sub-feature, it is a one-dimensional signal. The module divides this signal into two steps to operate on it. In the first step, the input features are pooled and one-dimensionally convolved, and the output of the activation function sigmoid is multiplied by the starting input, which is similar to the same operation in the multi-scale fusion module MSF. In the second step, in another branch, the input features are firstly convolved and pooled with 3x1 one-dimensional convolution, and then connected and multiplied with several output branches in the first step, and finally the output features with the same size as the input features are obtained.

[0058] The characteristic of this structure is the ability to capture the long-term changing trend of features, and the improved multi-scale attention module EMA can help the model predict blood pressure values ​​more accurately. Because the facial rPPG signal will inevitably be affected by the ambient light and its facial movement during the extraction process, the improved multi-scale attention module EMA can enhance the adaptability and generalization ability of the model in different time periods, so that it can perform well in different environments and application scenarios. The difference between the improved multi-scale attention module EMA of the present invention and the traditional convolution is that the traditional convolution, such as SE-Net, needs to increase the dimension first and then reduce the dimension during the convolution process, which may destroy the one-to-one correspondence between channels and weights. The original intention of designing this module is to improve the interaction strategy, perform information aggregation, and abandon traditional operations such as addition and splicing. In addition, in order to enhance the time series feature recognition of the one-dimensional rPPG signal by the multi-scale attention module EMA, firstly, the grouping factor used in the present invention is increased to 32, which not only reduces the computational complexity, but also enables more fine-grained processing of the channel during feature extraction; secondly, the activation function is added with a nonlinear transformation of the output. Considering the long sequence and relatively large degree of transformation of the pulse wave characteristics, this nonlinear transformation enables the model to adapt to such a complex input feature distribution; finally, because the rPPG signal is divided into small segments of 7 seconds, if one-dimensional batch normalization is used, it will produce inadaptability, so the present invention uses group normalization to improve stability and convergence speed. The specific process of group normalization is:

[0059] First, the rPPG signal is converted into a tensor and the tensor is reshaped and grouped. The purpose is to calculate the mean and variance within each group. The formula for calculating the mean for each group is as follows:

[0060] ,

[0061] in, H and W is the height and width of the group (the dimension after the number of channels is flattened). The variance is calculated as follows:

[0062] ,

[0063] Finally, the transformed tensor is normalized using the mean and variance, as follows:

[0064] ,

[0065] in, tensor is the original tensor, is a small constant used to prevent division by zero. In addition to improving stability and convergence speed, group normalization can better maintain feature consistency during small batch signal training.

[0066] Experimental process and results:

[0067] 1. Processing Data

[0068] The present invention collects clinical data (MIMIC-Ⅲ) from a hospital. In this data set, data containing both pulse wave signals and blood pressure signals are selected, and the average values ​​of systolic and diastolic pressures are calculated through the peak detection algorithm. During the processing, samples with systolic pressure greater than 200 or diastolic pressure less than 40 are excluded, and finally about 100,000 valid samples are extracted.

[0069] Preprocessing process: First, the extracted rPPG signal and MIMIC-Ⅲ data were divided into windows with an interval of 7 seconds. In view of the fact that the camera sensor may be interfered by various external factors such as environmental noise, movement, light changes, etc., which may cause baseline drift problems in the signal. In order to ensure the integrity of capturing individual pulsation information in the rPPG signal, the present invention adopts processing methods such as baseline drift removal. Secondly, considering that the heart rate pulsation signal usually presents a frequency range of 0.5 Hz to 8 Hz, for this reason, the present invention chooses to apply a Butterworth bandpass filter to filter the rPPG signal. The cutoff frequency of the filter is set to 0.5 Hz and 8 Hz to effectively suppress the interference of high-frequency noise and non-pulsation signals, and ensure the accuracy and stability of subsequent analysis.

[0070] After the above preprocessing, a relatively clean rPPG signal can be obtained. However, it is found in practice that the signal itself still has some noise. In order to filter out the excess noise, the present invention processes the signal by Morse wavelet transform, and selects the scale range as (1,31) to provide better localization in frequency and time and improve the reliability and robustness of wavelet transform. Morse wavelet transform can be calculated as follows:

[0071]

[0072] ψ( t ) is the Morse wavelet function, t is the time variable, w 0 is the frequency parameter, which controls the center frequency of the Morse wavelet.

[0073] 2. Experimental Design

[0074] In order to compare with previous work and verify the performance of the improved model, the experiment includes 5 groups, and these 5 groups of experiments use the common public dataset MIMIC.

[0075] 3. Comparative Experimental Results

[0076] Four relatively representative experiments were selected for comparison. The data sets of these experiments are the same as the data sets used in the present invention, which are more comparable. The experimental results are shown in Table 1, where '-' indicates that the corresponding indicator is not given.

[0077]

[0078] Note: The bold font is the optimal value in each column.

[0079] The methods of these groups of experiments can be roughly summarized as directly using pulse wave signals to measure blood pressure. Since pulse wave signals are closer to the true value than pulse waves extracted from the face, and can better reflect the real pulsation of the human body, the effect is more impressive than the first group of experimental results. From the results of these groups of experiments, the model designed by the present invention has smaller errors in both systolic and diastolic pressure than the results of the control group, especially the diastolic pressure with an error within 5 mmHg, which has increased by 10.48%. However, the MAE of diastolic pressure in this experiment is higher than that in the first experiment. The reason is that the data set used in the first group is the second version of MIMIC, and the data comes from the CareVue bedside monitor, while the data set used in this experiment is the third version of MIMIC, and the data comes from Metavision and CareVue bedside monitors. The data volume of the third version is richer. Although the data volume is relatively large, the MAE of diastolic pressure in this experiment is only 1.00 higher than that of the control group, which shows that the model of the present invention has a strong generalization ability.

[0080] It is to be understood that the present invention is described by some embodiments, and it is known to those skilled in the art that various changes or equivalent substitutions may be made to these features and embodiments without departing from the spirit and scope of the present invention. In addition, under the teachings of the present invention, these features and embodiments may be modified to adapt to specific circumstances and materials without departing from the spirit and scope of the present invention. Therefore, the present invention is not limited by the specific embodiments disclosed herein, and all embodiments falling within the scope of the claims of this application are within the scope of protection of the present invention.

Claims

1. A method for predicting blood pressure from face videos based on multi-scale attention constraints, characterized in that: The steps include: Step 1: Extract remote photoplethysmography (rPPG) signals. First, the face is detected in the video using the MTCNN face detection algorithm and the region of interest (ROI) is divided to extract the effective face area. Then, the rPPG signal with noise is initially extracted by combining optical flow estimation and median filtering. In the blood pressure prediction part, the rPPG signal is baseline-drifted and filtered again to obtain a relatively clean signal. The rPPG signal is then window-segmented. Step 2: Design the blood pressure level detection model EBP-Net, input the rPPG signal into the EBP-Net model to predict systolic and diastolic blood pressure; Step 2.1: Propose a signal processing method model EBP-Net that includes multi-task and multi-scale; The model EBP-Net selects the facial rPPG signal and its first-order and second-order derivatives as the model input; selects the multi-scale fusion module MSF to combine information of different scales to capture the hierarchical structure and complex patterns in the signal; introduces and improves the EMA module to help the model predict blood pressure values ​​by capturing the long-term change trend of the features; Step 2.2: Design an improved multi-scale fusion module MSF; First, the facial rPPG signal and its derivatives are concatenated as input. The input features are first passed through the first PW convolution to change the channel of the input features to adapt to the subsequent DW convolution. The two DW convolutions use convolution kernels of sizes 5x1 and 3x1 respectively. Secondly, the results of the DW convolution are average pooled and fully connected, and the processing results are multiplied with the output features of the DW convolution. Finally, the input features are residually linked with the results of the initial PW convolution. One-dimensional convolution is used, and the attention mechanism of the softmax weighted operation is used to dynamically extract larger features. Residual connection is used to compensate for the differences in rPPG signals and retain the original features. Step 2.3: Design an improved multi-scale attention module EMA; By introducing a dynamic weighting mechanism, the importance of features is adjusted according to changes in time, and the exponential moving average is used to smooth the changes in features; the multi-scale attention module EMA receives the output features of the multi-scale fusion module MSF as input, and first divides it into several sub-features in the depth direction. For each sub-feature, it is a one-dimensional signal. The multi-scale attention module EMA divides this signal into two steps to operate on it. In the first step, the input features are pooled and one-dimensionally convolved, and the output of the activation function sigmoid is multiplied by the starting input. In the second step, in another branch, the input features are firstly convolved and pooled with a 3x1 one-dimensional convolution, and then connected and multiplied with several output branches in the first step, and finally the output features with the same size as the input features are obtained; The step 2.3 includes the steps of enhancing the multi-scale attention module EMA to recognize the time series features of the one-dimensional rPPG signal: first, the grouping factor used is increased to 32; second, a nonlinear transformation of the output is added to the activation function; and finally, group normalization is used to improve stability and convergence speed.

2. The method for predicting blood pressure from face videos based on multi-scale attention constraints according to claim 1, characterized in that: The step 1 comprises the following specific steps: Step 1.1: Extracting remote photoplethysmography (rPPG) signals from face videos in visible light scenes; Step 1.2: For the extracted ROI, use the optical flow estimation method to obtain the motion information of the pixels in the face area, and then calculate the optical flow vector of each pixel in the image to describe the changes of the image over time.

3. The method for predicting blood pressure from face videos based on multi-scale attention constraints according to claim 2, characterized in that: In the step 1.1, at the stage of extracting rPPG signals from remote signals, for each frame of the video, face detection is performed using the face detection algorithm MTCNN. MTCNN uses multi-scale sliding window detection and a CNN model to classify and regress each candidate window, thereby determining the position and size of the face frame. Secondly, for each detected face frame, the corresponding face image area is cropped from the original image, that is, the part containing the face is extracted.

4. The method for predicting blood pressure from facial videos based on multi-scale attention constraints according to claim 2, characterized in that: The step 1.2 extracts the microvascular motion signal caused by the heartbeat by analyzing the change of the optical flow vector, and calculates the spatial gradient I of the image for the continuous frame image I (x, y, t) x ,I y and time gradient I t To estimate, in the optical flow estimation, the Lucas-Kanade method is used to calculate the optical flow vector (v x ,v y ), the change of the gray value of each pixel in the window over time is represented by a vector, and the equation group for solving the optical flow vector is shown in formula (1): Among them, I x ,I y and I t Respectively represent the gradient of the image in the x direction, y direction and time direction, Ω represents the set of pixels in the window, (v x ,v y ) represents the optical flow vector. By solving the above equations, the optical flow vector of each pixel is obtained, thereby obtaining the motion information in the face area. The linear filtering technology of median filtering and mean filtering is used to reduce the noise level in the signal by eliminating the high-frequency components in the signal.

5. The method for predicting blood pressure from face videos based on multi-scale attention constraints according to claim 1, characterized in that: The specific steps of group normalization are: First, the rPPG signal is converted into a tensor and the tensor is reshaped and grouped. The purpose is to calculate the mean and variance within each group. The formula for calculating the mean for each group is as follows: Where H and W are the height and width of the group; the variance is calculated as follows: Finally, the transformed tensor is normalized using the mean and variance, as follows: Here, tensor is the original tensor and ε is a constant.

Images