Remote physiological signal estimation method and system based on wavelet transform

Abstract

The invention discloses a remote physiological signal estimation method and system based on wavelet transform, and relates to the technical field of non-contact physiological signal detection.The method comprises the steps that an original face video is preprocessed, including face recognition, cutting and segmentation of a long video into a plurality of segments; diffusion fusion is carried out on the preprocessed video frames, and spatial information is fully fused into a channel to preliminarily extract features; performing multi-stage discrete wavelet transform on the preliminarily extracted features to obtain an approximate coefficient sequence and a detail coefficient sequence, and filtering the coefficient sequence of each frequency band according to the frequency band where the target prediction signal is located; local and global feature learning is carried out on each coefficient sequence along the channel dimension and the feature dimension, and corresponding approximation coefficient and detail coefficient sequences are predicted; and performing inverse wavelet transform on each predicted coefficient sequence to realize feature reconstruction, and finally outputting a predicted rPPG signal. According to the invention, the accuracy of rPPG signal estimation is improved.

Description

Technical Field

[0001] This invention relates to the field of non-contact physiological signal detection technology, and in particular to a remote physiological signal estimation method and system based on wavelet transform. Background Technology

[0002] Currently, in the field of physiological signal detection, the main technologies can be divided into contact and non-contact methods. Most of the most common methods for detecting heart rate on the market are contact-based, such as pulse taking, stethoscopes, electrocardiographs, and smart bracelets. While these methods have high accuracy, contact-based measurements still have certain limitations. Against this backdrop, non-contact methods are gradually attracting more attention, such as remote photoplethysmography (rPPG), which shows broad application prospects. This technology infers physiological parameters such as heart rate by capturing subtle color changes on the surface of facial skin in a video, i.e., changes in light absorption. The advantage of this method is that it does not require direct contact with the subject, resulting in lower cost and a wide range of applications. However, its detection accuracy is relatively low and it is affected by external factors such as light and movement.

[0003] Physiological signal detection techniques based on facial videos mainly include the following different model architectures: 1. Non-end-to-end model: This type of model typically requires complex preprocessing steps to generate feature maps, and the quality of these feature maps directly affects the final detection results. Deep learning methods are then used to learn from these feature maps to predict heart rate values. While this type of model has high accuracy, it is complex and has poor real-time performance.

[0004] 2. End-to-end models: In recent years, end-to-end models such as the visual Mamba architecture have emerged. These models can adaptively aggregate local and global spatiotemporal features, thereby enhancing the representation of rPPG and reducing intermediate preprocessing steps. However, these models often have high computational requirements and demand high computing power from the devices used.

[0005] 3. Unsupervised models based on contrastive learning: These models generate different views of the same video through data augmentation, guiding the model to ignore interference and focus on heart rate features, thus narrowing the distance between similar signals in the feature space and predicting physiological signals. Unsupervised learning does not require labeled data, enhancing the model's robustness against interference, but its stability and accuracy still need improvement.

[0006] Although existing non-contact physiological signal detection technologies have made significant progress, some shortcomings still exist. First, their anti-interference capabilities are limited; changes in illumination (such as variations in light color and intensity), facial occlusion (such as glasses and beards), and head movements (such as head movements in all directions) can significantly affect the accuracy of the detection results. Second, they are affected by different ages and skin characteristics; existing technologies have varying detection effects on rPPG signals depending on the optical properties of different skin types, thus the generalization ability of the models needs improvement. Furthermore, they require high computational resources, especially end-to-end models based on large models, which have extremely high computational complexity and are not suitable for resource-constrained scenarios. Summary of the Invention

[0007] To overcome the shortcomings of the prior art, the present invention provides a remote physiological signal estimation method and system based on wavelet transform, which improves the accuracy of rPPG signal estimation.

[0008] To achieve the above objectives, the present invention adopts the following technical solution, including: A method for estimating long-range physiological signals based on wavelet transform is shown below: Perform video preprocessing on facial images; The preprocessed video frames are diffused and fused to incorporate spatial information into the channel in order to initially extract features; The features initially extracted are processed by multi-level discrete wavelet transform to obtain wavelet components in each frequency band. Based on the frequency band of the target prediction signal, i.e. the physiological signal, interference signals outside the physiological signal frequency band are filtered out, and only wavelet components related to the physiological signal are retained, i.e. only wavelet components within the physiological signal frequency band are retained. Multi-resolution branch independent processing is performed on wavelet components of different frequency bands to obtain predicted wavelet components. The multi-resolution branch independent processing includes: for each wavelet coefficient sequence in the wavelet component, firstly, the wavelet coefficient sequence is divided into blocks; then, the block-shaped wavelet coefficient sequence is extended in the time dimension through a linear embedding layer; then, the wavelet coefficient sequence after time dimension extension is aggregated in the channel dimension and time dimension in sequence to realize local information aggregation and global information capture; finally, the processed wavelet coefficient sequence is flattened and linearly projected to output the predicted wavelet coefficient sequence. The predicted wavelet components are subjected to inverse wavelet transform to reconstruct the signal feature values, and finally the predicted rPPG signal is output.

[0009] Preferably, video preprocessing includes face recognition, image segmentation, and video segmentation. Specifically, face recognition is performed on the first frame of the face video, the coordinates of the selected face region are recorded, and the coordinates are used to crop subsequent video frames. The cropped video frames are then divided into video segments of the same length.

[0010] The preferred method for diffusion fusion is as follows: First, backbone network 1 is used to perform coarse-grained feature extraction on the original frame and the difference frame respectively to generate the original features. and motion features containing contextual information : ; ; in, It is the preprocessed video frame sequence, i.e., the original frame sequence; It is a sequence of differential frames calculated from video frames; Indicates splicing; This represents backbone network 1, which consists of sequentially connected two-dimensional convolutions, batch normalization, activation functions, and max pooling. Then the original features output by backbone network 1 and motion features containing contextual information Input the backbone network 2 for fusion to obtain fused features. : ; Where α and β are the proportions of the original frame and the difference frame in feature extraction, respectively; This represents backbone network 2, which consists of sequentially connected two-dimensional convolutions, batch normalization, activation functions, and max pooling. Then the fused features output by backbone network 2 Input backbone network 3, and use backbone network 3 to generate high-level features. : ; in, This represents backbone network 3, which consists of sequentially connected two-dimensional convolutions and batch normalizations. Finally, self-attention modules and global average pooling are used to enhance the ROI regions in facial skin, fully integrating spatial information into the channels and outputting the initially extracted features. .

[0011] Preferably, the expression for wavelet transform is as follows: ; in, These are the features initially extracted after diffusion fusion; m is the number of wavelet decomposition layers. Indicates the type of wavelet; These are the wavelet components generated by wavelet decomposition, i.e., the wavelet coefficient sequence, including: It is the approximate coefficient sequence of the m-th layer. These are the detail coefficient sequences for layers 1 to m, respectively. The filter removes interference signals outside the frequency range of the physiological signal, retaining only the wavelet components related to the physiological signal.

[0012] Preferably, multi-resolution branches are processed independently, as shown below: First, each wavelet coefficient sequence in the wavelet component is divided into overlapping blocks. To ensure that each value of the wavelet coefficient sequence is retained, the last value is used for padding, resulting in the divided wavelet coefficient sequence. : ; ; in, This represents one of the approximate coefficient sequence or detail coefficient sequence obtained after wavelet decomposition at level i. It is the number of pieces obtained by cutting; It is the length of the wavelet coefficient sequence; S is the length of each block; S is the step size; C is the number of channels; Indicates cutting into chunks; The segmented wavelet coefficient sequence is then processed through a linear embedding layer. Expanding the time dimension (i.e., length d) yields the feature values ​​after expanding the dimension. : ; Then, the feature values ​​after expanding the dimensions The Mixer module is used to obtain the wavelet coefficient sequence after processing. : ; Finally, the wavelet coefficient sequence processed by the Mixer module is passed to the Head module for flattening. and linear projection Output the predicted wavelet coefficient sequence: ; ; in, The result after flattening; This is the result after linear projection, i.e., the predicted wavelet coefficient sequence; This represents the predicted length of the wavelet coefficient sequence.

[0013] Preferably, the Mixer module consists of a Patch Mixer block and an Embedding Mixer block, as shown below: First, consider the eigenvalues ​​after expanding the dimensions. Perform Patch Mixer block processing to obtain the Patch Mixer block processing result. : ; Where BN represents batch normalization, This represents the activation function. The Patch Mixer block processes data along the channel dimension, starting with the first linear layer. First, use the set channel factor Increase the channel dimension, that is The second linear layer Then restore the channel dimension, that is ; The processing results of the Patch Mixer block The Embedding Mixer block is processed to obtain the result of the Embedding Mixer block, which is also the result of the Mixer module. : ; Where BN represents batch normalization, This represents the activation function. The Embedding Mixer block processes data along the time dimension, and the first linear layer... First, use the set time factor Increase the time dimension, that is The second linear layer Then restore the time dimension, that is Furthermore, the Embedding Mixer block also performs residual joins.

[0014] Preferably, the predicted wavelet components are subjected to inverse wavelet transform to reconstruct the signal feature values, and finally the predicted rPPG signal is output, as shown below: ; in, For the predicted approximate coefficient sequence, For the predicted sequence of detailed coefficients; The wavelet type is represented by Y; Y is the reconstructed signal feature value. Y is processed for prediction, and the predicted rPPG signal is output.

[0015] This invention provides a remote physiological signal estimation system based on wavelet transform, which is applied to the aforementioned remote physiological signal estimation method based on wavelet transform. The system model includes: a preprocessing module, a diffusion fusion module, a wavelet transform and denoising module, a multi-resolution module, and a reconstruction prediction module. The preprocessing module is used to perform video preprocessing on face videos; The diffusion fusion module is used to perform diffusion fusion on the preprocessed video frames, integrating spatial information into the channel to initially extract features. The wavelet transform and denoising module is used to obtain wavelet components of each frequency band from the initially extracted features through multi-level discrete wavelet transform, and to filter out interference signals outside the frequency band of the physiological signal based on the frequency band of the target prediction signal, i.e. the physiological signal, and retain only the wavelet components related to the physiological signal. The multi-resolution module is used to perform multi-resolution branch independent processing on wavelet components of different frequency bands to obtain the predicted wavelet components; wavelet components of different frequency bands are processed independently in the multi-resolution module and share the same weights. The reconstruction prediction module is used to perform inverse wavelet transform on the predicted wavelet components, reconstruct the signal feature values, and finally output the predicted rPPG signal. The system model is trained using a sample set, which includes face videos and corresponding labels (real rPPG signals) to minimize the error between the predicted and real rPPG signals. During training, the number of wavelet transform decomposition levels and the type of wavelet are also optimized. The trained system model is then used to predict the face videos to be tested.

[0016] The present invention provides an electronic device comprising a processor, a memory, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the aforementioned remote physiological signal estimation method based on wavelet transform.

[0017] This invention provides a computer program product comprising a computer program / instruction that, when executed by a processor, implements the aforementioned remote physiological signal estimation method based on wavelet transform.

[0018] The advantages of this invention are: (1) This invention introduces wavelet transform for remote physiological signal detection. Multi-resolution wavelet transform can effectively extract features in both the frequency domain and the time domain simultaneously, avoiding frequent data conversion between the time domain and the frequency domain, and effectively separating heart rate-related physiological fluctuations from noise such as motion artifacts and illumination changes, thereby more accurately estimating heart rate and heart rate variability.

[0019] (2) The present invention improves the anti-interference capability of the model. Wavelet transform generates wavelet components of different frequency bands, separating useful information and interference noise in the signal, making it easier for the model to learn to focus on those task-related and stable frequency patterns, thereby improving the stability and robustness of detection.

[0020] (3) The present invention improves computational efficiency and the method is more optimized in terms of computational resource consumption, making it suitable for more application scenarios.

[0021] (4) The present invention enhances the generalization ability of the model. The interference noise of different sample sets is different and specific. Wavelet transform filters out these signals that are not related to the core frequency band of heart rate through time-frequency denoising, so that the model can learn the periodic fluctuation features that are stable across sample sets and related to heart rate, thereby improving the model's tolerance to input.

[0022] (5) The present invention can achieve prediction of arbitrary time length. For videos of different time lengths, after preprocessing, different batches of video frames of the same length are generated. The models are processed according to the specified batch size to achieve prediction results of the corresponding length of video input and output for arbitrary time length. Attached Figure Description

[0023] Figure 1 This is a schematic diagram of a remote physiological signal estimation system based on wavelet transform according to the present invention.

[0024] Figure 2 This is a flowchart of a remote physiological signal estimation method based on wavelet transform according to the present invention. Detailed Implementation

[0025] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0026] The relevant prior art involved in this invention is described below: 1. Fundamental Principles of Wavelet Transform Wavelet transform uses scaling and wavelet functions to decompose the original signal into wavelet components of different scales (frequency bands) and locations (time). It can analyze the global overview and local details of a signal at different resolutions, while revealing the time-domain and frequency-domain characteristics of the signal. Wavelet transform is divided into two main categories based on whether the parameters are discrete: continuous wavelet transform and discrete wavelet transform. It includes two main stages: wavelet decomposition and signal reconstruction.

[0027] Wavelet decomposition process: Continuous wavelet transform performs inner product operation on the original signal and continuous wavelet basis functions to obtain continuous wavelet transform coefficients; Discrete wavelet transform usually performs high-pass and low-pass filtering and downsampling on the signal in a cascaded manner, decomposing the original signal into approximate coefficients and detail coefficients at different resolutions.

[0028] Signal reconstruction process: After processing the wavelet components obtained from the previous decomposition according to actual needs, the inverse wavelet components are used to gradually synthesize them to recover the signal, thereby achieving the purpose of denoising and feature extraction of the original signal.

[0029] 2. Mathematical Representation of Wavelet Transform

[0030] Wavelet decomposition: The formula for continuous wavelet transform is as follows: ; in, Represents the original time signal. This represents the continuous wavelet basis function, obtained by scaling and shifting the mother wavelet, where s is the scale parameter. These are translation parameters. Indicates conjugate. Indicates the signal at scale Translation The continuous wavelet transform coefficients reflect the similarity between the signal and the wavelet basis.

[0031] Discrete wavelet transform is an iterative operation of filtering and downsampling. The i-th level decomposition process is shown below: ; ; in, This represents the approximate coefficients (or the original signal) obtained from the previous level of decomposition. It is a low-pass filter. It is a high-pass filter. These are the approximate coefficients obtained from the i-th level decomposition. The i-th level decomposition yields the detail coefficients, k represents the index of the decomposed coefficients, and n-2k represents the sliding window of the filter, which moves by 2 samples each time as k moves, thus completing convolution and downsampling in one step.

[0032] Signal reconstruction: The continuous wavelet transform superimposes the continuous wavelet coefficients with the dual wavelet to recover the original signal. The specific expression is as follows: ; in, The admissibility constant of the mother wavelet must satisfy the admissibility condition to ensure distortion-free reconstruction. It is a dual wavelet.

[0033] The discrete wavelet transform first upsamples the coefficients at each stage by a factor of 2, then filters them using the corresponding reconstruction filters, and finally superimposes them onto the previous stage signal until the original signal is recovered. The i-th stage reconstruction process is as follows: ; in, These are the coefficients of the low-pass reconstruction filter. These are the coefficients of the Qualcomm reconstructed filter. This represents the approximate signal of the (i-1)th layer (or the original signal of the first layer).

[0034] 3. Application of wavelet transform in physiological signal detection The application of wavelet transform in the detection of remote physiological signals (rPPG signals) is based on its ability to process non-stationary time signals and perform multi-resolution analysis. By applying wavelet transform to video data, physiological signals (rPPG signals) and noise interference can be effectively distinguished.

[0035] Signal decomposition: The initially extracted features are decomposed into wavelet components to obtain wavelet components in different frequency bands. Thresholding and other methods can effectively distinguish the true features of the signal from noise. Then, more accurate feature values ​​are extracted from the wavelet components.

[0036] Signal reconstruction: The processed wavelet components are subjected to inverse wavelet transform to obtain the desired physiological signal (rPPG signal).

[0037] Depend on Figure 1 As shown, the present invention provides a remote physiological signal estimation system based on wavelet transform, comprising: a data (video) preprocessing module, a diffusion fusion module, a wavelet transform and denoising module, a multi-resolution module, and a reconstruction prediction module. The entire system's operation is divided into two parts: a training process and an estimation process.

[0038] During training: Data preprocessing module: After performing face recognition on the face video, the face portion is cropped (segmented) and background noise interference is removed; then the video frames are cut into blocks according to the specified length, so rPPG signal estimation can be achieved by inputting videos of any length.

[0039] Diffusion module: It uses differential frames and original frames to initially extract features and fully fuse spatial information into the channel.

[0040] Wavelet transform and denoising module: The wavelet transform is used to decompose the initially extracted features to obtain wavelet components of different frequency bands. Based on the frequency band characteristics of the required rPPG signal, incoherent frequency bands are filtered using thresholding, and useful frequency bands related to the rPPG signal are retained for subsequent learning.

[0041] Multi-resolution module: Using patching, embedding, and deep learning networks, this module learns how to decompose the wavelet components to obtain the most suitable values, and further extracts features from these wavelet components to make the reconstructed rPPG signal as close as possible to the true value. Wavelet components of different frequency bands are processed independently in this module, while sharing the same weights.

[0042] Reconstruction and prediction module: The signal feature values ​​are reconstructed using wavelet components learned previously. After processing by the Predictor, the predicted rPPG signal is output and compared with the real rPPG signal to ensure that the predicted value is as close as possible to the real physiological state.

[0043] During the estimation process: Data preprocessing module: Performs face recognition on the input face video, cuts out the face region, and processes the input video into blocks according to the specified requirements.

[0044] Diffusion module: Calculates differential frames for preprocessed video frames, integrates inter-frame differences into the original frames, realizes the representation and perception of frame-level rPPG signal changes, and performs preliminary extraction of main features.

[0045] Wavelet transform and denoising module: Based on the trained model, select an appropriate wavelet decomposition method to obtain wavelet components of different frequency bands, and then filter out irrelevant frequency bands other than the rPPG signal to obtain denoised wavelet components of different frequency bands.

[0046] Multi-resolution module: performs more refined feature extraction on wavelet components of different frequency bands after denoising, and obtains wavelet components that are only related to the rPPG signal.

[0047] Reconstruction and prediction module: Reconstructs the processed wavelet components, i.e., inverse wavelet transform, reconstructs the signal feature values, and outputs the predicted rPPG signal after processing by the Predictor.

[0048] Depend on Figure 2 As shown, the present invention provides a remote physiological signal estimation method based on wavelet transform. First, the model is trained by inputting a pre-collected face video during the training process. Then, the trained model is used to estimate the remote physiological signal. During the estimation process, the input is the face video of the person to be detected, and the output is the corresponding predicted rPPG signal.

[0049] like Figure 2 As shown, the processing methods for the model during training are as follows: S11, Obtain the sample set, which includes face videos and labels corresponding to the face videos, i.e., real rPPG signals; S12 preprocesses the original face video, unifying the format of sample sets from different sources to ensure that video models of different durations can be processed; the Viola-Jones face detection algorithm is used to quickly calculate Haar-like features using integral images, and after recognizing faces, the video is cut into 128×128 blocks so that the face region occupies most of the space in the video frame; the video frames are then divided into blocks of 160 frames each for easier batch processing of video frames later.

[0050] S13, perform diffusion fusion on the preprocessed video frames, and integrate spatial information into the channel to initially extract features.

[0051] The specific method is as follows: First, utilize backbone network 1 (by...) It consists of kernel-based 2D convolution (Conv), batch normalization (BN), the ReLU activation function, and max pooling (MaxPool), and is called... Preliminary, coarse-grained feature extraction is performed on the original frame and the difference frame to generate the original features. and motion features containing contextual information : ; ; in, It is a sequence of preprocessed video frames (original frames); It is a differential frame sequence calculated from the original frame. The four differential frames in the differential frame sequence represent the four differences between the current frame t and the adjacent frames. As t changes, the differential frame sequence will also change. Indicates splicing; This represents backbone network 1.

[0052] The coarse-grained features are then processed through backbone network 2 (with the same structure). , is called ), to the original features and motion features containing contextual information Fusion, resulting in fusion characteristics : ; Where α and β represent the proportions of the original frame and the difference frame in feature extraction; This represents backbone network 2.

[0053] Then utilize backbone network 3 (from The kernel-based 2D convolution and batch normalization (referred to as Stem3) further process the fused features to generate higher-level features with a larger receptive field. : ; Finally, self-attention modules and global average pooling are used to enhance the ROI regions (forehead, cheeks, chin) in the facial skin, fully integrating spatial information into the channels and outputting the initially extracted features. .

[0054] Attention masks used by self-attention modules The calculation formula is: ; in, This represents the image resolution (pixel size) after transformation by the network layers, corresponding to the height and width respectively; H represents the Sigmoid activation function; H and W represent the original image resolution, corresponding to the height and width, respectively.

[0055] S14. The initially extracted features are decomposed into wavelets and then denoised to obtain wavelet components of different frequency bands after denoising.

[0056] The specific method is as follows: This embodiment uses multi-level discrete wavelet transform to initially extract features. The wavelet decomposition process involves convolving the wavelet with low-pass and high-pass filters in the time domain to generate approximate and detail coefficient sequences, respectively. This process is called the first-level wavelet decomposition. The approximate coefficient sequences generated in the first level are then convolved with low-pass and high-pass filters in the time domain again to generate new approximate and detail coefficient sequences, which is called the second-level wavelet decomposition. During training, the number of decomposition levels and the type of wavelet need to be optimized to achieve the best prediction results. Specifically, the Optuna method and the default tree structure Parzen Estimator (TPE) are used to optimize hyperparameters, such as the number of decomposition levels and the type of wavelet, to achieve the best prediction results. The final result retains only the approximate coefficient sequence obtained from the last decomposition and the detail coefficient sequences from all levels. ; in, These are the features initially extracted after the diffusion fusion module; m is the number of wavelet decomposition layers. More wavelet layers are not necessarily better; the specific value of this parameter should be obtained through training based on the specific situation. This indicates the wavelet type, and the optimal value is selected from db2, db3, db5, sym2, sym3, sym4, sym5, coif4, and coif5. The wavelet components generated by wavelet decomposition, It is the approximate coefficient sequence of the m-th layer. These are detail coefficient sequences from layer 1 to layer m; then, a filter is used to remove interference signals outside the physiological signal frequency range. In this embodiment, the predicted rPPG signal is mainly for heart rate information, so the retained frequency range is [0.75 Hz, 2.5 Hz].

[0057] S15 performs multi-resolution branch independent processing on wavelet components of different frequency bands to obtain the predicted wavelet components.

[0058] The specific method is as follows: First, each wavelet coefficient sequence in the wavelet component is patched with overlapping blocks. To ensure that each value of the wavelet coefficient sequence (approximate coefficient sequence or detail coefficient sequence) is retained, the last value is repeated for a length equal to the stride to fill in the coefficient sequence, resulting in the patched wavelet coefficient sequence. : ; ; in, This represents one of the approximate coefficient sequence or detail coefficient sequence obtained from the i-th level decomposition. It is the number of pieces obtained by cutting. It is the length of each wavelet coefficient sequence. It is the length of each piece. Set to 16, S is the step size, S is set to 8, and C represents the number of channels; This indicates cutting into chunks.

[0059] The segmented wavelet coefficient sequence is then processed through a linear embedding layer to expand the time dimension to d=256, extracting more hidden feature information and obtaining the feature values ​​after expanding the dimension. : ; Then, the feature values ​​after expanding the dimensions The Mixer module is used to obtain the wavelet coefficient sequence after processing. ; The Mixer module consists of a Patch Mixer block and an Embedding Mixer block, both of which are similar to a 2-layer MLP. First, process the Patch Mixer blocks. The Patch Mixer blocks are batch normalized from 2D. Dimensional permutation, first linear layer GELU activation function Second linear layer The process involves processing the channel dimension, aggregating the local information (local information) contained in the patching into the global information context, and obtaining the processing result of the Patch Mixer block. : ; Among them, the first linear layer With the set channel factor Expanding channel dimensions To capture more comprehensive feature information, the second linear layer Then restore the original channel dimension, that is In this embodiment, C=96. Next, the Embedding Mixer block processes the data. The Embedding Mixer block (mixer) has the same structure as the PatchMixer block, but it processes the time dimension, capturing global information in a high-dimensional space to obtain the processing result. That is, the processing result of the Mixer module. : ; Among them, the first linear layer With the set time factor Expanding the time dimension, i.e. The second linear layer Then restore the original time dimension, that is Furthermore, the Embedding Mixer block also performs residual joins here.

[0060] Finally, the wavelet coefficient sequence processed by the Mixer module is passed to the Head module for flattening. and linear projection Output the predicted wavelet coefficient sequence : ; ; in, The result after flattening; This is the result after linear projection, i.e., the predicted wavelet coefficient sequence; The predicted length of the wavelet coefficient sequence is assigned during model initialization. In this embodiment, it is set to be the same as the length of each batch of input, i.e., 160.

[0061] S16, perform inverse wavelet transform on the predicted wavelet coefficient sequence. Reconstruction is performed to obtain the predicted rPPG signal. The reconstruction expression in the estimation process is shown below: ; in, For the predicted approximate coefficient sequence, For the predicted sequence of detailed coefficients; Indicates the type of wavelet.

[0062] Y represents the reconstructed signal feature values. Y is processed by a Predictor, which involves first upsampling to restore the temporal resolution of the signal, then convolution to project the rPPG features into an rPPG waveform, and finally dimensionality compression to remove redundant channel dimensions, outputting a one-dimensional predicted rPPG signal.

[0063] like Figure 2 As shown, the trained model is used for remote physiological signal estimation. The estimation process is as follows: S21, Input a face video. First, perform face recognition on the face video, crop out the face region. If no face can be recognized, then select the maximum bounding box, adjust the resolution to 128×128, and then divide the input video into blocks every 160 frames. The output sequence is as follows. B is the number of blocks, D=160 is the length of each block, H and W are the resolution of each frame, and C=3 is the number of channels.

[0064] S22, following the method of step S13, perform preliminary feature extraction on the preprocessed video frames and output the preliminary extracted features, with a feature dimension of (B, (T / 2).

[0065] S23, Following the method in step S14, perform wavelet decomposition on the initially extracted features to obtain wavelet components of different frequency bands. The parameters of wavelet decomposition (such as the number of wavelet decomposition layers, the selection of wavelet basis, etc.) are set according to the parameters of the trained model, and then a bandpass filter is used to filter out wavelet components outside the range of [0.75 Hz, 2.5Hz].

[0066] S24. Following the method in step S15, the denoised wavelet components are input into the multi-resolution branches with the same weight for independent processing to obtain the predicted clean wavelet components.

[0067] S25, following the method in step S16, perform inverse wavelet transform on the predicted wavelet components according to the same standard to obtain the reconstructed signal feature values, the dimension of which is (B, (T / 2), and finally processed by the Predictor, first upsampled to recover the time resolution of the signal. Then, convolution projects the rPPG features into an rPPG waveform (B, 1, T). Finally, dimensionality compression removes redundant channel dimensions, outputting a one-dimensional predicted rPPG signal.

[0068] The wavelet transform used in this invention produces the following effects: Firstly, wavelet transform allows for the selection of the optimal transform strategy for different practical situations, such as continuous wavelet transform and discrete wavelet transform. For discrete wavelet transform, the number of decomposition levels and the selection of wavelet basis functions can be determined based on the input, while continuous wavelet transform allows for the selection of different scale ranges. This provides high flexibility and adaptability.

[0069] Secondly, wavelet components of different frequency bands obtained by wavelet transform are used to replace downsampling or directly cut the video to generate multi-resolution signals, preventing the information correlation between different resolution layers from being severed. Each component retains the characteristics of the original signal in different frequency ranges, and also retains the information independence and reconstructability between each component.

[0070] Finally, and more importantly, the frequency components generated after wavelet transform contain information in both the time and frequency dimensions. This allows for clear differentiation of different frequency components (such as changes in lighting conditions, head movements, and facial movements) and precise correspondence to their locations on the time axis. This enables the prediction model to learn specifically based on "when and what frequency changes occur," thereby improving model accuracy.

[0071] The above are merely preferred embodiments of the present invention and are not intended to limit the scope of the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for estimating long-range physiological signals based on wavelet transform, characterized in that, As shown below: Perform video preprocessing on facial videos; The preprocessed video frames are diffused and fused to incorporate spatial information into the channel in order to initially extract features; The features initially extracted are processed by multi-level discrete wavelet transform to obtain wavelet components in each frequency band. Based on the frequency band of the target prediction signal, i.e. the physiological signal, interference signals outside the physiological signal frequency band are filtered out, and only wavelet components related to the physiological signal are retained, i.e. only wavelet components within the physiological signal frequency band are retained. Multi-resolution branch independent processing is performed on wavelet components of different frequency bands to obtain the predicted wavelet components. The multi-resolution branch independent processing includes: for each wavelet coefficient sequence in the wavelet component, first cutting the wavelet coefficient sequence into blocks; The segmented wavelet coefficient sequence is then extended in the time dimension through a linear embedding layer; then, the wavelet coefficient sequence after time dimension extension is aggregated in the channel dimension and time dimension in turn to achieve local information aggregation and global information capture; finally, the processed wavelet coefficient sequence is flattened and linearly projected to output the predicted wavelet coefficient sequence. The predicted wavelet components are subjected to inverse wavelet transform to reconstruct the signal feature values, and finally the predicted rPPG signal is output.

2. The method for estimating remote physiological signals based on wavelet transform according to claim 1, characterized in that, Video preprocessing includes face recognition, image segmentation, and video segmentation. Specifically, face recognition is performed on the first frame of the face video, and the coordinates of the selected face region are recorded. These coordinates are then used to crop subsequent video frames. The cropped video frames are then divided into segments of equal length.

3. The method for estimating remote physiological signals based on wavelet transform according to claim 1, characterized in that, The specific methods of diffusion fusion are as follows: First, backbone network 1 is used to perform coarse-grained feature extraction on the original frame and the difference frame respectively to generate the original features. and motion features containing contextual information : in, It is the preprocessed video frame sequence, i.e., the original frame sequence; It is a sequence of differential frames calculated from video frames; Indicates splicing; This represents backbone network 1, which consists of sequentially connected two-dimensional convolutions, batch normalization, activation functions, and max pooling. Then the original features output by backbone network 1 and motion features containing contextual information Input the backbone network 2 for fusion to obtain fused features. : Where α and β are the proportions of the original frame and the difference frame in feature extraction, respectively; This represents backbone network 2, which consists of sequentially connected two-dimensional convolutions, batch normalization, activation functions, and max pooling. Then the fused features output by backbone network 2 Input backbone network 3, and use backbone network 3 to generate high-level features. : in, This represents backbone network 3, which consists of sequentially connected two-dimensional convolutions and batch normalizations. Finally, self-attention modules and global average pooling are used to enhance the ROI regions in facial skin, fully integrating spatial information into the channels and outputting the initially extracted features. .

4. The method for estimating remote physiological signals based on wavelet transform according to claim 1, characterized in that, The expression for wavelet transform is as follows: in, These are the features initially extracted after diffusion fusion; m is the number of wavelet decomposition layers. Indicates the type of wavelet; These are the wavelet components generated by wavelet decomposition, i.e., the wavelet coefficient sequence, including: It is the approximate coefficient sequence of the m-th layer. These are the detail coefficient sequences for layers 1 to m, respectively. The filter removes interference signals outside the frequency range of the physiological signal, retaining only the wavelet components related to the physiological signal.

5. The method for estimating remote physiological signals based on wavelet transform according to claim 4, characterized in that, Multi-resolution branches are handled independently, as detailed below: First, the wavelet coefficient sequence of each wavelet component is divided into overlapping blocks. To ensure that each value of the wavelet coefficient sequence is retained, the last value is used for padding, resulting in the divided wavelet coefficient sequence. : in, This represents one of the approximate coefficient sequence or detail coefficient sequence obtained after wavelet decomposition at the i-th level; It is the number of pieces obtained by cutting; It is the length of the wavelet coefficient sequence; S is the length of each block; S is the step size; C is the number of channels; Indicates cutting into chunks; The segmented wavelet coefficient sequence is then processed through a linear embedding layer. Expanding the time dimension (i.e., length d) yields the feature values ​​after expanding the dimension. : Then, the feature values ​​after expanding the dimensions The Mixer module is used to obtain the wavelet coefficient sequence after processing. : Finally, the wavelet coefficient sequence processed by the Mixer module is passed to the Head module for flattening. and linear projection Output the predicted wavelet coefficient sequence: in, The result after flattening; This is the result after linear projection, i.e., the predicted wavelet coefficient sequence; This represents the predicted length of the wavelet coefficient sequence.

6. The method for estimating remote physiological signals based on wavelet transform according to claim 5, characterized in that, The Mixer module consists of a Patch Mixer block and an Embedding Mixer block, as shown below: First, consider the eigenvalues ​​after expanding the dimensions. Perform Patch Mixer block processing to obtain the Patch Mixer block processing result. : Where BN represents batch normalization, This represents the activation function. The Patch Mixer block processes data along the channel dimension, starting with the first linear layer. First, use the set channel factor Increase the channel dimension, that is The second linear layer Then restore the channel dimension, that is ; The processing results of the Patch Mixer block The Embedding Mixer block is used for processing, resulting in the processing result of the EmbeddingMixer block, which is the processing result of the Mixer module. : Where BN represents batch normalization, This represents the activation function. The Embedding Mixer block processes data along the time dimension, and the first linear layer... First, use the set time factor Increase the time dimension, that is The second linear layer Then restore the time dimension, that is Furthermore, the Embedding Mixer block also performs residual joins.

7. The method for estimating remote physiological signals based on wavelet transform according to claim 4, characterized in that, Perform inverse wavelet transform on the predicted wavelet components to reconstruct the signal eigenvalues, and finally output the predicted rPPG signal, as shown below: in, For the predicted approximate coefficient sequence, For the predicted sequence of detailed coefficients; The wavelet type is represented by Y; Y is the reconstructed signal feature value. Y is processed for prediction, and the predicted rPPG signal is output.

8. A remote physiological signal estimation system based on wavelet transform, characterized in that, The system model of the remote physiological signal estimation method based on wavelet transform, applied to any one of claims 1 to 7, includes: a preprocessing module, a diffusion fusion module, a wavelet transform and denoising module, a multi-resolution module, and a reconstruction prediction module; The preprocessing module is used to preprocess face videos; The diffusion fusion module is used to perform diffusion fusion on the preprocessed video frames, integrating spatial information into the channel to initially extract features. The wavelet transform and denoising module is used to obtain wavelet components of each frequency band from the initially extracted features through multi-level discrete wavelet transform, and to filter out interference signals outside the frequency band of the physiological signal based on the frequency band of the target prediction signal, i.e. the physiological signal, and retain only the wavelet components related to the physiological signal. The multi-resolution module is used to perform multi-resolution branch independent processing on wavelet components of different frequency bands to obtain the predicted wavelet components; wavelet components of different frequency bands are processed independently in the multi-resolution module and share the same weights. The reconstruction prediction module is used to perform inverse wavelet transform on the predicted wavelet components, reconstruct the signal feature values, and finally output the predicted rPPG signal. The system model is trained using a sample set, which includes face videos and corresponding labels (real rPPG signals) to minimize the error between the predicted and real rPPG signals. During training, the number of wavelet transform decomposition levels and the type of wavelet are also optimized. The trained system model is then used to predict the face videos to be tested.

9. An electronic device, characterized in that, It includes a processor, a memory, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements a remote physiological signal estimation method based on wavelet transform as described in any one of claims 1 to 7.

10. A computer program product, characterized in that, It includes a computer program / instruction that, when executed by a processor, implements a remote physiological signal estimation method based on wavelet transform as described in any one of claims 1 to 7.

Images