A facial video-based spatiotemporal domain weighted photoplethysmography recovery method

Abstract

The application discloses a kind of based on face video's space-time field weighting photoelectric volume pulse wave recovery method, the model involved mainly includes signal extraction, signal quality score and signal enhancement three modules.First face video is detected and cut to face, obtain only containing face video as input;Signal extraction module calculates the frame difference graph of input, to extract spatial domain weight from input feature with weighting, from spatial domain attention face non-motion area, and further from the weighted signal, preliminary extraction is carried out to pulse wave signal;Signal quality score module is segmented to pulse wave signal score;Signal enhancement module assigns higher attention weight value to the signal section with low score, from time domain attention signal quality poor part, output recovered high-quality pulse wave signal.The application can alleviate the motion interference in imaging photoelectric volume pulse wave tracing method, extract effective signal from time domain and spatial domain, provide high-quality pulse wave signal for subsequent physiological parameter estimation.

Description

Technical Field

[0001] The present invention relates to the fields of digital image processing technology and digital signal processing technology, and in particular to a spatiotemporal domain weighted photoplethysmography recovery method based on facial video. Background Art

[0002] Physiological parameters such as heart rate, blood oxygen saturation, respiratory rate, and blood pressure reflect the body's physiological state, providing crucial information about health status and potential risks. Real-time and accurate monitoring of these parameters facilitates early disease detection and personalizes individual health management to prevent disease. Traditional physiological signal measurement relies on direct contact between the sensor and the skin. However, this is inconvenient for long-term monitoring and can cause discomfort for patients with sensitive skin or trauma. Image photoplethysmography (IPPG), a non-contact method, is emerging as a promising future development direction for physiological parameter monitoring devices. Its principle is similar to that of IPPG: the cyclical contraction and relaxation of the heart causes changes in the blood filling level within the blood vessels. When ambient light strikes the skin's surface, the skin's absorption of light also generates corresponding pulsations due to changes in blood filling, resulting in corresponding changes in the intensity of the reflected light. By analyzing the changes in light intensity in each frame captured by the camera, physiological characteristics of the human body can be extracted. Compared with traditional contact methods, this technology has the advantages of easy operation, low cost, and no need for direct contact with the skin, thereby improving the real-time and convenience of monitoring. It has practical value and important prospects for telemedicine and daily monitoring.

[0003] However, IPPG measurement is susceptible to various interference factors, such as motion, and performs poorly in reconstructing photoplethysmography. It can only be used to estimate heart rate, with poor results for detecting other physiological parameters. Currently, most applications focus on extracting raw signals from a limited number of facial regions of interest. However, due to irregular head movements, signal quality varies across different facial regions at different times. Selecting a fixed number of regions of interest cannot fully utilize all high-quality signal areas on the face.

[0004] To address this issue, a spatiotemporal weighted photoplethysmography (PPE) recovery method based on facial videos was developed. This method employs both spatial and temporal approaches. It uses a frame difference map to weight the input video in the spatial domain, assigning low weights to facial parts with noticeable motion, allowing the network to focus more on stable facial areas when recovering the signal. Furthermore, it uses a signal quality score to weight the input video in the temporal domain, assigning high weights to sections with low signal quality, allowing the network to prioritize restoration of these segments during signal enhancement. This method can dynamically restore PPE signals, helping to improve the accuracy of various physiological indicator estimates. Summary of the Invention

[0005] In order to solve the problems existing in the background technology, the purpose of the present invention is to provide a spatiotemporal domain weighted photoplethysmography recovery method based on facial video, which can effectively reduce the influence of motion interference and improve the reliability of the recovered photoplethysmography signal.

[0006] The specific technical solution for achieving the purpose of the present invention is:

[0007] A spatiotemporal weighted photoplethysmography recovery method based on facial video, the method comprising the following steps:

[0008] Step 1: Build a pulse wave recovery model

[0009] The constructed pulse wave recovery model consists of a signal extraction module, a signal quality scoring module and a signal enhancement module; wherein the signal extraction module is composed of a convolution block and average pooling, calculates the frame difference map of the input video, performs feature extraction on the input video and the calculated frame difference map respectively, uses the frame difference map features as spatial domain weights to weight the features of the input video, and preliminarily extracts a one-dimensional pulse wave signal; the signal quality scoring module is composed of a linear layer and an LSTM network, which is responsible for giving the quality score of the one-dimensional pulse wave signal in different time periods and using this as the time domain attention weight; the signal enhancement module is an encoder block and a decoder block structure, which is responsible for multiplying the attention weight with the preliminarily extracted one-dimensional pulse wave signal, that is, performing time domain weighting on the preliminarily extracted one-dimensional pulse wave signal, and assigning a high attention weight value to the signal segment with a low quality score, thereby improving the quality of the time domain weighted pulse wave signal;

[0010] Step 2: Train the pulse wave recovery model

[0011] The training pulse wave recovery model is divided into two stages. Firstly, a signal extraction module, a signal quality scoring module and a signal enhancement module are separately pre-trained; a video input signal extraction module containing only a face is trained to obtain a motion-weighted signal extraction model, outputting a one-dimensional pulse wave signal extracted initially; the one-dimensional pulse wave signal extracted initially is input into a signal quality scoring module for training to obtain a signal quality scoring model, outputting a signal quality score; the signal quality score and the one-dimensional pulse wave signal extracted initially are input into a signal enhancement module for training to obtain a signal enhancement model, outputting a recovered pulse wave signal with higher quality;

[0012] Secondly, the pre-trained motion-weighted signal extraction model, signal quality scoring model and signal enhancement model are trained together and fine-tuned to obtain a usable pulse wave recovery model.

[0013] Step 3: The face position in the collected face video is detected using a Mediapipe Face Detection method, and the face video is cropped to obtain a video containing only a face, which is input into the usable pulse wave recovery model to obtain a recovered final pulse wave signal.

[0014] Further, the signal extraction module is trained, and the specific process is as follows:

[0015] After the video containing only a face is input into the signal extraction module, the frame difference map DIF thereof is first calculated, and the process is as follows: for the input video containing only a face, a green channel image corresponding to each frame thereof is extracted, and the i-th frame green channel image frame i is calculated, and a pixel-by-pixel normalized difference value between the adjacent frame green channel image frame i+1 is calculated, denoted as dif i , and the calculation formula is as follows:

[0016]

[0017] Another motion suppression normalized difference value is calculated. Since the value of dif i is larger in the area with more intense motion, the motion suppression normalized difference value is calculated so that the weight of the part with intense motion is closer to 0.

[0018] The motion suppression normalized difference value The image is scaled to the range of [0, 255] and then binarized. The pixel values ​​exceeding 220 are set to 255, indicating a region with high spatial attention weight, and the pixel values ​​less than or equal to 220 are set to 0, indicating a region with low spatial attention weight. Thus, the binary frame difference image DIF of the i-th frame image and the adjacent image in the input video is obtained. i ;

[0019] The binary frame difference map corresponding to each frame of the input video is calculated in sequence. If the i-th frame image is the last frame image of the input video, its binary frame difference map is the same as the i-1-th frame binary frame difference map, thereby forming a frame difference map DIF. If the input video contains m frames, then DIF = {DIF1, DIF2, ..., DIF m-1 ,DIF m-1};

[0020] The signal extraction module extracts the input video features containing only human faces;

[0021] The signal extraction module extracts the frame difference image DIF feature as a spatial domain weight, weights the video feature containing only the face, focuses on the non-moving area of ​​the face from the spatial domain, obtains a spatially weighted signal, and preliminarily extracts a one-dimensional pulse wave signal;

[0022] The signal extraction module is trained to obtain a motion weighted signal extraction model. The training process is specifically as follows:

[0023] The signal extraction module uses the photoplethysmography (PPG) collected synchronously with the input video as a label for training, and the loss function is Pearson Loss, which is as follows:

[0024]

[0025] Where N is the number of samples, y is the i-th signal value of a certain segment of PPG signal, is the mean value of a certain segment of PPG signal, is the i-th signal value of a certain pulse wave signal, is the mean value of a certain pulse wave signal, and n is the number of signal values ​​in a signal segment; when training the signal extraction module, a video containing only a face is used as input, and Pearson Loss is used as a loss function to control the training, thereby obtaining the motion weighted signal extraction model.

[0026] Furthermore, the signal quality scoring module is trained, and the specific process is as follows:

[0027] For each one-dimensional pulse wave signal, the standard deviation of the difference between the frame numbers corresponding to each adjacent maximum peak point is calculated. width, the standard deviation of the difference between the signal value of each minimum point and the signal value of the adjacent maximum point height , both of which are normalized to 0-1, then the label point label The calculation formula is as follows:

[0028] point label = 0.3 * point height + 0.7 * point width ;

[0029] The training loss function is L1_Loss, and the formula is as follows:

[0030]

[0031] In the formula, N is the number of samples, y is the actual label of the i-th sample, is the predicted value of the i-th sample;

[0032] The signal quality scoring module uses a one-dimensional pulse wave signal as input during training, and L1_Loss as a loss function to control training, thereby obtaining the signal quality scoring model.

[0033] Further, the signal enhancement model is trained, and the process is specifically as follows:

[0034] The signal enhancement module uses the signal quality score and the preliminarily extracted one-dimensional pulse wave signal as input, and the PPG waveform collected synchronously with the input video as a label, and uses Pearson Loss as a loss function to control training, thereby obtaining the signal enhancement model.

[0035] Further, the available pulse wave recovery model is trained, and the process is specifically as follows:

[0036] The weights of the motion weighted signal extraction model, the signal quality scoring model, and the signal enhancement model are respectively imported into the corresponding signal extraction module, the signal quality scoring module, and the signal enhancement module;

[0037] Except for the decoder block part of the signal enhancement module, the weights of the remaining parts are frozen, all modules are jointly trained, and the decoder block part is fine-tuned;

[0038] The joint training loss function includes two parts: signal quality loss and heart rate loss. The signal quality loss uses the PPG waveform collected synchronously with the input video as a label, and compares the difference between the final pulse wave signal and the PPG waveform collected synchronously with the input video. The loss function is Pearson Loss. The heart rate loss uses the heart rate data collected synchronously with the input video as a label, and compares the difference between the heart rate calculated from the final pulse wave signal and the synchronously collected heart rate data. The loss function is L1_Loss.

[0039] When training the pulse wave recovery model, a video containing only a human face is used as input, and the sum of Pearson Loss and L1_Loss is used as a loss function to control the training, thereby obtaining the usable pulse wave recovery model and outputting the final recovered pulse wave signal.

[0040] The technical solution provided by the present invention has the following beneficial effects: The proposed method for spatiotemporal weighted photoplethysmography (PPW) recovery based on facial video calculates frame difference maps corresponding to an input facial video frame sequence, extracts their features as spatial weights, and weights the input facial video, assigning lower attention weights to regions with significant inter-frame motion. However, the size and number of ROIs are not restricted, effectively improving the problem of limited source signal extraction quality caused by selecting fixed ROIs in motion. Furthermore, from the perspective of improving signal recovery quality, the method assigns one-dimensional pulse wave signal quality scores by time period, assigning higher attention weights to signal segments with lower quality scores, effectively improving the quality and reliability of pulse wave signal recovery. The present invention can dynamically recover PPG signals, helping to improve the accuracy of various physiological indicator estimates. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] Figure 1 is a flow chart of the present invention;

[0042] Figure 2 Schematic diagram of a facial frame and a frame difference map in the present invention;

[0043] Figure 3 A neural network structure diagram of the present invention;

[0044] Figure 4 This is a diagram of the signal quality scoring network structure of the present invention. DETAILED DESCRIPTION

[0045] In order to more clearly illustrate the technical means, technical improvements and beneficial effects of the present invention, the present invention is described in detail below with reference to the accompanying drawings.

[0046] Example

[0047] This embodiment takes a facial video of an adult male with a duration of 4 seconds and a frame rate of 30 frames per second as an example.

[0048] See also Figure 1-4 The present invention provides a method for recovering weighted photoplethysmography in spatiotemporal domain based on facial video, comprising the following specific steps:

[0049] Use the Mediapipe Face Detection method to detect the face position for each frame in the video. If a face is detected in the current frame, extract the face bounding box information and calculate the face position according to the bounding box coordinates x. min 、y min and width, height to crop the face image from the current frame, see Figure 2 (a)(b) Traverse all frames in the video to perform cropping, and scale all cropped face images to 198*198 to obtain a video frame sequence containing only faces as the input video F = {frame1,frame2,……,frame 120};

[0050] The Mediapipe Face Detection method is a well-known algorithm [Lugaresi C, Tang J, Nash H, et al. Mediapipe: A framework for building perception pipelines[J]. arXiv preprint arXiv:1906.08172, 2019.] and will not be introduced in detail here.

[0051] Calculate the frame difference map of the input video. The specific process is as follows:

[0052] After the video containing only human faces is input into the signal extraction module, the frame difference map DIF is first calculated. The process is as follows: for the input video containing only human faces, the green channel image corresponding to each frame is extracted. For the i-th frame green channel image frame i , calculate the green channel image frame of the adjacent frame i+1 The pixel-by-pixel normalized difference value is recorded as dif i , which is calculated as follows:

[0053]

[0054] For dif i The more intense the movement, the closer the corresponding normalized difference value is to 1. In order to suppress movement, the movement suppression normalized difference value is calculated. Make the weight of the parts corresponding to intense exercise closer to 0;

[0055] normalizing the motion suppression difference value scaled to the range of [0, 255], and then binarized, with the part where the pixel value is greater than 220 set to 255, representing a region with high spatial attention weight, and the pixel value less than or equal to 220 set to 0, representing a region with low spatial attention weight, to obtain a binarized frame difference map DIF of the i-th image and the adjacent image in the input video i , see Figure 2 (c);

[0056] The binarized frame difference map corresponding to each frame of the input video is calculated in turn. If the i-th image is the last image of the input video, its binarized frame difference map is the same as the binarized frame difference map of the (i-1)-th image, thereby forming a frame difference map DIF. If the input video contains m frames, then DIF = {DIF1, DIF2, …, DIF m-1 , DIF m-1};

[0057] The signal extraction module includes two groups of convolution blocks, an average pooling (AvgPool3D), and a layer of three-dimensional convolutional neural network (Conv3D). The convolution block mainly includes a layer of maximum pooling (MaxPool3D) and two layers of three-dimensional convolutional neural network (Conv3D), see Figure 3 (a);

[0058] The signal extraction module extracts the features of the input video;

[0059] The signal extraction module extracts the frame difference map DIF feature as a spatial domain weight weight spatial , and weights the features of the input video, focusing on the non-motion region of the face in the spatial domain, to obtain a spatially weighted signal, and preliminarily extract a one-dimensional pulse wave signal IPPG pre ;

[0060] The signal extraction module is trained to obtain a motion-weighted signal extraction model. The training process is as follows:

[0061] The signal extraction module is trained using a PPG waveform collected synchronously with the input video as a label. The loss function is Pearson Loss, and the formula is as follows:

[0062]

[0063] In the formula, N is the number of samples, y i is the i-th signal value of a certain segment of PPG signal, is the mean value of a certain segment of PPG signal, is the i-th signal value of a certain segment of pulse wave signal, is the mean value of a certain pulse wave signal, and n is the number of signal values ​​in a signal segment;

[0064] The signal extraction module is trained using a video containing only a human face as input, using Pearson Loss as a loss function to control the training, and outputting a preliminary extracted one-dimensional pulse wave signal IPPG. Ore , and obtain the motion weighted signal extraction model.

[0065] Furthermore, the signal quality scoring module is trained, and the specific process is as follows:

[0066] For each one-dimensional pulse wave signal, the standard deviation of the difference between the frame numbers corresponding to each adjacent maximum peak point is calculated. width , the standard deviation of the difference between the signal value of each minimum point and the adjacent maximum point heighi , normalize both to between 0 and 1, then the label point label The calculation formula is as follows:

[0067] point label =0.3*point height +0.7*point width ;

[0068] The signal quality scoring module extracts the one-dimensional pulse wave signal features;

[0069] The signal quality scoring network is a Bi-LSTM structure, which mainly includes three linear layer blocks and one LSTM block. Except for the last linear layer block, the remaining linear layer blocks include a linear layer (Linear) and a SELU activation function. The last linear layer block includes a linear layer (Linear) and a Sigmoid activation function. Figure 4 , the training loss function is L1_Loss, the formula is as follows:

[0070]

[0071] Where N is the number of samples, y i is the actual label of the i-th sample, is the predicted value of the i-th sample;

[0072] The 120-point signal is cut into three 60-point segments, signal1, signal2, and signal3, using a sliding window of 30 data points. These three overlapping segments are input into the quality scoring module to obtain three normalized scores, point1, point2, and point3.

[0073] Split the original signal into three non-overlapping signal segments of 40 data points each. 1* 、signal 2* 、signal 3* , the score of each signal segment is the mean of the covered scores, recorded as point 1* 、point 2* 、point 3* ,Right now

[0074] Use 1-score average to get each signal segment signal 1* 、signal 2* 、signal 3* The signal quality scores are weight1, weight2, and weight3.

[0075] The signal quality scoring module uses a one-dimensional pulse wave signal as input during training, uses L1_Loss as a loss function to control training, outputs a signal quality score for each signal segment, and obtains the signal quality scoring model.

[0076] Furthermore, the signal enhancement model training process is specifically as follows:

[0077] The signal quality score weight time With the preliminary extracted pulse wave signal IPPG pre Input signal enhancement module, which is specifically:

[0078] The signal enhancement network structure mainly includes six encoder blocks and six decoder blocks. Each encoder block contains a one-dimensional convolutional neural network (Conv1D) and a PReLU activation function. Each decoder block contains a one-dimensional deconvolutional neural network (ConvTranspose1D), a PReLU activation function and a skip connection layer. The last decoder contains a one-dimensional deconvolutional neural network (ConvTranspose1D) and a Tanh activation function. Figure 3 (b);

[0079] The signal enhancement module uses the signal quality score and the initially extracted one-dimensional pulse wave signal as input, extracts the time-domain weighted one-dimensional pulse wave signal features, uses the PPG waveform collected synchronously with the input video as a label, uses Pearson Loss as a loss function to control training, and outputs a restored pulse wave signal with higher quality to obtain the signal enhancement model.

[0080] Furthermore, the training process of the available pulse wave recovery model is specifically as follows:

[0081] Importing the motion-weighted signal extraction model, signal quality scoring model, and signal enhancement model weights into their corresponding signal extraction module, signal quality scoring module, and signal enhancement module, respectively;

[0082] Except for the decoder block part of the signal enhancement module, the weights of the rest are frozen, all modules are trained jointly, and the decoder block part is fine-tuned;

[0083] The joint training loss function consists of two parts: signal quality loss and heart rate loss. Signal quality loss uses the PPG waveform collected synchronously with the input video as a label, comparing the difference between the final pulse wave signal and the synchronously collected PPG waveform. The loss function is Pearson Loss. Heart rate loss uses the heart rate data collected synchronously with the input video as a label, comparing the heart rate calculated from the final pulse wave signal with the simultaneously collected heart rate data. The loss function is L1_Loss. The sum of Pearson Loss and L1_Loss is used to jointly control training to obtain the usable pulse wave recovery model.

[0084] Input the video containing only the face into the available pulse wave recovery model to obtain the final recovered pulse wave signal IPPG pd .

[0085] In summary, the present invention proposes a spatiotemporal weighted photoplethysmography (PP) recovery method based on facial video. This method extracts frame difference images corresponding to an input facial video frame sequence as spatial weights and weights the input facial video, assigning lower attention weights to regions with significant inter-frame motion. However, the size and number of ROIs are not restricted. This effectively improves the limited source signal extraction quality caused by selecting fixed ROIs in the presence of motion. Furthermore, from the perspective of improving signal recovery quality, the method assigns one-dimensional pulse wave signal quality scores by time segment, assigning higher attention weights to signal segments with lower quality scores, effectively improving the quality and reliability of pulse wave signal recovery. This method provides effective data for non-contact physiological signal measurement.

[0086] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A spatiotemporal weighted photoplethysmography recovery method based on facial video, characterized in that: The method comprises the following steps: Step 1: Build a pulse wave recovery model The constructed pulse wave recovery model consists of a signal extraction module, a signal quality scoring module and a signal enhancement module; wherein the signal extraction module is composed of a convolution block and average pooling, calculates the frame difference map of the input video, performs feature extraction on the input video and the calculated frame difference map respectively, uses the frame difference map features as spatial domain weights to weight the features of the input video, and preliminarily extracts a one-dimensional pulse wave signal; the signal quality scoring module is composed of a linear layer and an LSTM network, which is responsible for giving the quality score of the one-dimensional pulse wave signal in different time periods and using this as the time domain attention weight; the signal enhancement module is an encoder block and a decoder block structure, which is responsible for multiplying the attention weight with the preliminarily extracted one-dimensional pulse wave signal, that is, performing time domain weighting on the preliminarily extracted one-dimensional pulse wave signal, and assigning a high attention weight value to the signal segment with a low quality score, thereby improving the quality of the time domain weighted pulse wave signal; Step 2: Train the pulse wave recovery model The training of the pulse wave recovery model is divided into two stages. First, the signal extraction module, the signal quality scoring module, and the signal enhancement module are pre-trained separately. The video input signal extraction module containing only the face is trained to obtain a motion-weighted signal extraction model, which outputs a preliminarily recovered one-dimensional pulse wave signal. The preliminarily recovered one-dimensional pulse wave signal is input into the signal quality scoring module for training to obtain a signal quality scoring model, which outputs a signal quality score. The signal quality score and the preliminarily recovered one-dimensional pulse wave signal are input into the signal enhancement module for training to obtain a signal enhancement model, which outputs a recovered pulse wave signal of higher quality. Secondly, the pre-trained motion weighted signal extraction model, signal quality scoring model, and signal enhancement model are trained and fine-tuned together to obtain a usable pulse wave recovery model; Step 3: Using the Mediapipe Face Detection method to detect the face position in the collected facial video, cropping the facial video to obtain a video containing only the face, and inputting the available pulse wave recovery model to obtain a final recovered pulse wave signal; wherein: The signal extraction module is trained, and the specific process is as follows: After the video input signal containing only human faces is extracted into the module, the frame difference map DIF is first calculated. The process is as follows: For the input video containing only human faces, extract the green channel image corresponding to each frame, the i-th frame green channel image , calculate the green channel image of the adjacent frame The pixel-by-pixel normalized difference value is recorded as , which is calculated as follows: ； Also calculate the motion suppression normalized difference value , because the more intense the movement, The larger the value, the more motion suppression normalized difference value is calculated. =1- Make the weight of the parts corresponding to intense exercise closer to 0; The motion suppression normalized difference value The image is scaled to the range of [0, 255] and then binarized. The pixel values ​​exceeding 220 are set to 255, indicating a region with high spatial attention weight, and the pixel values ​​less than or equal to 220 are set to 0, indicating a region with low spatial attention weight. Thus, the binary frame difference image DIF of the i-th frame image and the adjacent image in the input video is obtained. i ; Calculate the binary frame difference map corresponding to each frame of the input video in sequence. If the i-th frame image is the last frame image of the input video, then its binary frame difference map is the same as the i-1-th frame binary frame difference map, thereby forming a frame difference map DIF. If the input video contains m frames, then ; The signal extraction module extracts the input video features containing only human faces; The signal extraction module extracts the frame difference image DIF feature as a spatial domain weight, weights the video feature containing only the face, focuses on the non-moving area of ​​the face from the spatial domain, obtains a spatially weighted signal, and preliminarily extracts a one-dimensional pulse wave signal; The signal extraction module is trained to obtain a motion weighted signal extraction model. The training process is specifically as follows: The signal extraction module uses the photoplethysmography (PPG) collected synchronously with the input video as a label for training, and the loss function is Pearson Loss, which is as follows: ， Where N is the number of samples, is the i-th signal value of a certain segment of PPG signal, is the mean value of a certain segment of PPG signal, is the i-th signal value of a certain pulse wave signal, is the mean value of a certain pulse wave signal, and n is the number of signal values ​​in a signal segment; During training, the signal extraction module uses a video containing only human faces as input, and uses Pearson Loss as a loss function to control training, thereby obtaining the motion weighted signal extraction model.

2. The spatiotemporal domain weighted photoplethysmography recovery method according to claim 1, characterized in that: The signal quality scoring module is trained, and the specific process is as follows: For each of the initially extracted one-dimensional pulse wave signals, the standard deviation of the difference between the frame numbers corresponding to each adjacent maximum peak point is calculated. , the standard deviation of the difference between the signal values ​​of each minimum point and the adjacent maximum point , normalize both to between 0 and 1, then the label The calculation formula is as follows: ； The training loss function is L1_Loss, and the formula is as follows: ， Where N is the number of samples, is the actual label of the i-th sample, is the predicted value of the i-th sample; The signal quality scoring module uses a one-dimensional pulse wave signal as input during training, and L1_Loss is used as a loss function to control training to obtain the signal quality scoring model.

3. The spatiotemporal domain weighted photoplethysmography recovery method according to claim 1, characterized in that: The signal enhancement model training process is specifically as follows: The signal enhancement module uses the signal quality score and the initially extracted one-dimensional pulse wave signal as input, the PPG waveform collected synchronously with the input video as a label, and Pearson Loss as a loss function to control training to obtain the signal enhancement model.

4. The spatiotemporal weighted photoplethysmography recovery method according to claim 1, wherein: The training process of the available pulse wave recovery model is as follows: Importing the motion-weighted signal extraction model, signal quality scoring model, and signal enhancement model weights into their corresponding signal extraction module, signal quality scoring module, and signal enhancement module, respectively; Except for the decoder block part of the signal enhancement module, the weights of the rest are frozen, all modules are trained jointly, and the decoder block part is fine-tuned; The joint training loss function includes two parts: signal quality loss and heart rate loss. The signal quality loss uses the PPG waveform collected synchronously with the input video as a label, and compares the difference between the final pulse wave signal and the PPG waveform collected synchronously with the input video. The loss function is Pearson Loss. The heart rate loss uses the heart rate data collected synchronously with the input video as a label, and compares the difference between the heart rate calculated from the final pulse wave signal and the synchronously collected heart rate data. The loss function is L1_Loss. When training the pulse wave recovery model, a video containing only a human face is used as input, and the sum of Pearson Loss and L1_Loss is used as a loss function to control the training, thereby obtaining the usable pulse wave recovery model and outputting the final recovered pulse wave signal.

Images