Remote physiological signal detection method and system based on state space and dual-path interaction

Abstract

The invention relates to the technical field of non-contact physiological signal detection, and discloses a remote physiological signal detection method and system based on state space and dual-path interaction. The method comprises the steps of obtaining a sample set; frame difference preprocessing is carried out on the original face video, and time sequence features are extracted; performing time sequence dependence modeling on the extracted time sequence features through a state space model and a state space duality mechanism; performing frequency domain feature enhancement on the time sequence features through a frequency domain feed-forward network; respectively inputting the time sequence characteristics after the frequency domain characteristics are enhanced into a self-attention path and a cross attention path, then fusing the time sequence characteristics after the two paths are processed, and taking the fused data as a predicted rPPG signal; performing model training by minimizing the difference between the predicted rPPG signal and the real rPPG signal; and inputting a face video of a to-be-detected person into the trained prediction model, and outputting a corresponding prediction rPPG signal. The detection method provided by the invention has good accuracy and robustness.

Description

Technical Field

[0001] The present invention relates to the technical field of non-contact physiological signal detection, and in particular to a remote physiological signal detection method and system based on state space and dual-path interaction. Background Art

[0002] Remote photoplethysmography (rPPG) is a technology that uses a camera to capture tiny changes in facial blood flow to non-contactly extract physiological indicators such as heart rate and blood oxygen saturation. This technology does not require the wearing of any sensors and has been widely used in scenarios such as psychological state monitoring, telemedicine, and anti-counterfeiting identification. In recent years, with the development of deep learning and computer vision, methods based on convolutional neural networks and attention mechanisms (such as PhysFormer) have significantly improved the accuracy of rPPG in static or slightly moving scenes. In addition, multispectral imaging (such as the GREEN method) effectively alleviates the impact of skin pigment differences and ambient light changes on signal quality by selecting specific band illumination and spatiotemporal fusion technology. An emerging research direction is to combine the state-space model with the dual-path interaction architecture of the attention mechanism to achieve collaborative modeling of long-term and short-term dependencies.

[0003] However, existing rPPG methods based on deep CNN or self-attention Transformer still have difficulty in balancing efficiency and robustness in practical applications: they rely on O(T 2 ) complexity of the attention mechanism or multi-layer convolutional network, resulting in the number of parameters and multiplication and addition operations growing quadratically with the length of the video, making it impossible to achieve real-time monitoring on resource-constrained embedded devices; at the same time, most models are only trained within a fixed frame length range, have poor adaptability to video clips that are too short or too long, and their performance is easily limited by video memory and computing delays; in addition, these methods often focus on unidirectional enhancement in the time domain or frequency domain, lack frame global pooling and dual-domain interaction design, and cannot simultaneously capture the long-term temporal dependence and key frequency components of physiological signals such as heart rate; coupled with the limited information interaction capabilities of a single path or shallow fusion structure, signal loss and noise residue problems are still very prominent in real scenarios such as large-scale motion, complex lighting, and skin color diversity. Summary of the Invention

[0004] In order to solve the technical problems existing in the prior art, the present invention provides a remote physiological signal detection method and system based on state space and dual-path interaction. The physiological signal detection method of the present invention has good accuracy and robustness while reducing computational complexity.

[0005] To achieve the above object, the present invention provides the following technical solutions:

[0006] The present invention discloses a remote physiological signal detection method based on state space and dual-path interaction, comprising:

[0007] S1. Obtain a sample set, the sample set including a face video and a real rPPG signal corresponding to the face video;

[0008] S2. Perform frame difference preprocessing on the original face video to extract temporal features;

[0009] S3. Model the temporal dependency of the extracted temporal features through the state-space model and state-space duality mechanism;

[0010] S4. Enhance the frequency domain features of the time series features through the frequency domain feedforward network;

[0011] S5. Input the time series features after frequency domain feature enhancement into the self-attention path and the cross-attention path respectively, then fuse the time series features processed by the two paths and use the fused data as the predicted rPPG signal;

[0012] S6. Performing model training by minimizing the difference between the predicted rPPG signal and the true rPPG signal to obtain a trained prediction model;

[0013] S7. Input the face video of the person to be detected into the trained prediction model, and output the corresponding predicted rPPG signal with reference to the processing method of steps S2 to S5.

[0014] As a further improvement of the above solution, in step S2, the expression of frame difference preprocessing is as follows:

[0015] X Stem =Stem(X origin )+Stem(X origin +X diff )

[0016] Where, X origin is the original face video data; X diff is the differential data between adjacent frames; X Stem is the output feature map; Stem(·) represents the Frame Stem function.

[0017] As a further improvement of the above solution, in step S3, the expression of the state space model is as follows:

[0018] h t =Ah t-1 +B x t

[0019]

[0020] Where A, B and C are learnable parameter matrices; h t and h t-1are the hidden state vectors at time t and t-1 respectively; x t is the input feature at time t, namely X Stem ;y t is the output feature vector at time t; T is the transpose symbol.

[0021] As a further improvement to the above scheme, in step S3, the expression of the state space duality mechanism SSD is as follows:

[0022]

[0023] Where Q, K, and V are query, key, and value matrices, respectively; L is the structured mask matrix; is the element-wise multiplication symbol.

[0024] As a further improvement of the above solution, in step S4, the processing method of the frequency domain feedforward network is:

[0025] First, perform fast Fourier transform on the time series features, and the expression is as follows:

[0026]

[0027] Where f is the time series feature; H(f) is the expression of the time series feature in the frequency domain; h(t) is the expression of the time series feature in the time domain; dt is the time integral operator; e is a natural constant; j is an imaginary unit;

[0028] Then, the frequency domain enhanced signal is restored through inverse Fourier transform, and the expression is as follows:

[0029]

[0030] As a further improvement to the above solution, in step S5, the expression of the self-attention path processing is as follows:

[0031]

[0032] The expression of cross attention path processing is as follows:

[0033]

[0034] The concatenation expression of the two path features in the channel dimension is as follows:

[0035] X fusion =Concat(FSA,FCA)

[0036] Where, X SA is the output feature of the self-attention path; L S is the structured mask matrix of the self-attention path; Q S , K S and VS are the query, key, and value matrices corresponding to the self-attention path respectively; is the element-by-element multiplication symbol; T is the transpose symbol; X CA is the output feature of the cross attention path; L C is the structured mask matrix of the cross attention path; Q C , K C and V C are the query, key, and value matrices corresponding to the cross-attention path respectively; X fusion is the fusion feature of the output features of the self-attention path and the cross-attention path after splicing in the channel dimension; FSA is the output feature of the self-attention path after being enhanced by the frequency domain feedforward network; FCA is the output feature of the cross-attention path after being enhanced by the frequency domain feedforward network; Concat(·) is the splicing process.

[0037] As a further improvement to the above solution, in step S6, the overall loss function expression during the model training process is as follows:

[0038] L overall =L Time +L Freq

[0039] Where, L overall is the overall loss, L Time Used to measure the matching degree between the predicted rPPG signal and the real rPPG signal in the time domain; L Freq Used to measure the similarity between the rPPG signal and the real rPPG signal in the frequency domain.

[0040] The present invention further discloses a remote physiological signal detection system based on state space and dual-path interaction, which applies the remote physiological signal detection method based on state space and dual-path interaction as described above; the detection system comprises:

[0041] A frame difference preprocessing module is used to obtain a sample set, which includes a face video and a real rPPG signal corresponding to the face video; the frame difference preprocessing module is also used to perform frame difference preprocessing on the original face video to extract temporal features;

[0042] The temporal dependency modeling module is used to perform temporal dependency modeling on the extracted temporal features through the state space model and the state space duality mechanism;

[0043] The frequency domain feature enhancement module is used to enhance the frequency domain features of the time series features through the frequency domain feedforward network;

[0044] The prediction module is used to input the time series features after frequency domain feature enhancement into the self-attention path and the cross-attention path respectively, and then fuse the time series features processed by the two paths and use the fused data as the predicted rPPG signal. The model is trained by minimizing the difference between the predicted rPPG signal and the true rPPG signal to obtain a trained prediction model.

[0045] The sampling module is used to collect facial videos of the person to be detected and input them into the trained prediction model; the corresponding predicted rPPG signal is output according to the processing method of the above modules.

[0046] Compared with the prior art, the present invention has the following beneficial effects:

[0047] 1. The present invention extracts spatiotemporal features from the original facial video through frame difference preprocessing, and then uses the state-space model to model the timing information. The state-space model can capture long-term dependencies and adapt to subtle and continuous dynamic changes in physiological signals; at the same time, the state-space duality mechanism is introduced to perform weighted processing on the features of different time segments and spatial regions, thereby enhancing the capture of key physiological signals. Furthermore, a frequency domain feedforward network is used to perform a fast Fourier transform (FFT) on the signal after time series modeling to extract the important components in the frequency domain, and then the enhanced signal is restored through an inverse Fourier transform (IFFT). This two-pronged processing method in the time domain and frequency domain can not only accurately model the time evolution of the signal globally, but also highlight the frequency components related to physiological indicators such as heart rate and respiration, thereby improving the accuracy and robustness of monitoring.

[0048] 2. The prediction model of this invention is based on the principles of collaborative state-space duality and dual-path interaction. It adopts an end-to-end architecture. Through multi-timescale collaborative state-space duality module learning and dual-path interaction, it can effectively extract weak rPPG signals from videos containing a large amount of spatiotemporal redundancy and understand the periodic patterns of their long-range context. It also maintains stable performance for video inputs of arbitrary length. This detection method has broad application prospects in healthcare, affective computing, anti-fraud and other fields.

[0049] 3. The dual-path interaction structure designed in this invention processes temporal features through self-attention and cross-attention paths respectively, and then fuses the information obtained by the two paths in the channel dimension. The self-attention path focuses on capturing the internal relationships and local consistency of the input data, while the cross-attention path focuses on exploring the complementary information and global dependencies between different features. This design enables the model to fully explore and integrate spatiotemporal information at different levels, avoiding the information loss that may occur when processing through a single path, and suppressing noise and interference, thereby greatly enhancing the model's sensitivity and stability to long-range physiological signals in complex environments.

[0050] 4. The state space model (SSM) used in this paper has a computational complexity of O(T), while the traditional attention-based Transformer has a complexity of O((HWT) 2 In this model, thanks to the linear complexity of the collaborative state-space duality mechanism and frame-by-frame global average pooling, its computational cost grows linearly with video length. In terms of the number of parameters (Param.) and multiplication-addition operations (MACs), this model (1.065M parameters, 12.943GMACs) is far lower than other methods, such as DeepPhys with 7.504M parameters and 120,000GMACs, which enables the present invention to run efficiently even on resource-constrained devices. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] Figure 1 This is a flow chart of a remote physiological signal detection method based on state space and dual-path interaction according to embodiment 1 of the present invention.

[0052] Figure 2 This is a framework diagram of a remote physiological signal detection system based on state space and dual-path interaction according to embodiment 2 of the present invention. DETAILED DESCRIPTION

[0053] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0054] Example 1

[0055] See also Figure 1 This embodiment provides a remote physiological signal detection method based on state space and dual-path interaction. First, during the training process, the collected sample data is input to train the model, and then the trained model is used to perform remote physiological signal monitoring. During the monitoring process, the input is the face video of the person to be detected, and the output is the corresponding predicted rPPG signal.

[0056] The training process is as follows:

[0057] S1. Obtain a sample set, which includes face videos and the corresponding real rPPG signals. This step ensures data diversity and provides sufficient and authentic labeled data for subsequent model training.

[0058] S2. Perform frame difference preprocessing on the original face video to extract temporal features. The expression of frame difference preprocessing is as follows:

[0059] X Stem =Stem(X origin )+Stem(X origin +X diff )

[0060] Where, X origin is the original face video data; X diff is the differential data between adjacent frames; X Stem is the output feature map; Stem(·) represents the Frame Stem module function.

[0061] The Stem(·) function is introduced as follows:

[0062] First, the difference between adjacent frames is calculated to highlight the motion information:

[0063] ΔX t =X t+1 -X t ,t=1,…,D-1

[0064] Then use two "Stem" modules to extract static texture and motion features respectively:

[0065] Y raw =Stem 11 (X t )

[0066] Y diff =Stem 12 (concat(ΔX t ))

[0067] Finally, the output is obtained by fusing the learnable weights α and β at two levels and downsampling:

[0068] F1=αY raw +βY diff

[0069] Y out =αStem 21 (F1)+βStem 22 (Y diff ).

[0070] In the above formula, is the original image data of the tth frame; ΔX t The difference between the tth frame and the t+1th frame highlights the time change; Stem 11 and Stem 12 Both are feature extraction modules that include Conv-BN-ReLU-Pool (i.e., convolutional layer-batch normalization-ReLU-pooling); Y rawis the static texture feature output, Y diff is the differential motion feature output, α and β are the learnable weights of the two-way feature fusion; Stem 21 and Stem 22 is the second-level down-sampling and feature processing module; F1 is the intermediate feature after the first-level fusion; Y out This is the final output feature, which has been downsampled to H / 8×W / 8 and the number of channels is the set number of dimensions.

[0071] S3. Temporal dependency modeling is performed on the extracted temporal features through the state space model and state space duality mechanism.

[0072] In step S3, the expression of the state space model is as follows:

[0073] h t =Ah t-1 +B x t

[0074] y t =C T h t

[0075] Where A, B and C are learnable parameter matrices; h t and h t-1 are the hidden state vectors at time t and t-1 respectively; x t is the input feature at time t, namely X Stem ;y t is the output feature vector at time t; is the transpose symbol.

[0076] The expression of the state space duality SSD mechanism is as follows:

[0077]

[0078] Where Q, K, and V are query, key, and value matrices, respectively; L is the structured mask matrix; is the element-wise multiplication symbol.

[0079] S4. Enhance the frequency domain features of the time series features through the frequency domain feedforward network (FDF).

[0080] In step S4, the processing method of the frequency domain feedforward network is:

[0081] First, perform fast Fourier transform on the time series features, and the expression is as follows:

[0082]

[0083] Where f is the time series feature; H(f) is the expression of the time series feature in the frequency domain; h(t) is the expression of the time series feature in the time domain; dt is the time integral operator; e is a natural constant; j is an imaginary unit;

[0084] Then, the frequency domain enhanced signal is restored through inverse Fourier transform, and the expression is as follows:

[0085]

[0086] S5. Input the time series features after frequency domain feature enhancement into the self-attention path and the cross-attention path respectively, then fuse the time series features processed by the two paths, and use the fused data as the predicted rPPG signal.

[0087] In step S5, the expression of the self-attention path processing is as follows:

[0088]

[0089] The expression of cross attention path processing is as follows:

[0090]

[0091] The concatenation expression of the two path features in the channel dimension is as follows:

[0092] X fusion =Concat(FSA,FCA)

[0093] Where, X SA is the output feature of the self-attention path; L S is the structured mask matrix of the self-attention path; Q S , K S and V S are the query, key, and value matrices corresponding to the self-attention path respectively; ° is the element-by-element multiplication symbol; X CA is the output feature of the cross attention path; L C is the structured mask matrix of the cross attention path; Q C , L C and V C are the query, key, and value matrices corresponding to the cross-attention path respectively; X fusion is the fusion feature of the output features of the self-attention path and the cross-attention path after splicing in the channel dimension; FSA is the output feature of the self-attention path after being enhanced by the frequency domain feedforward network; FCA is the output feature of the cross-attention path after being enhanced by the frequency domain feedforward network; Concat(·) is the splicing process.

[0094] S6. Perform model training by minimizing the difference between the predicted rPPG signal and the true rPPG signal to obtain a trained prediction model.

[0095] In step S6, the overall loss function expression during the model training process is as follows:

[0096] L overall =L Time +L Freq

[0097] Where, L overall is the overall loss, L Time Used to measure the matching degree between the predicted rPPG signal and the real rPPG signal in the time domain; L Freq Used to measure the similarity between the rPPG signal and the real rPPG signal in the frequency domain.

[0098] The monitoring process is as follows:

[0099] S7. Input the trained prediction model into the facial video of the person to be detected, and output the corresponding predicted rPPG signal using the same processing methods as steps S2-S5. Specifically, the facial video of the person to be detected is preprocessed using frame difference to extract spatiotemporal features; the temporal features are modeled using temporal dependency and enhanced in the frequency domain; the temporal features are processed through a dual-path to generate fused data; and the fused data, processed by the model, is output as the final rPPG signal.

[0100] The present invention constructs a complete end-to-end system from preprocessing, time series modeling, frequency domain enhancement, to final signal prediction. Each link is collaboratively optimized during the overall training process, which not only reduces the redundancy and error accumulation in the information transmission process, but also ensures that the overall system reaches a high level in real-time and resource efficiency, making deployment and practical application more convenient. The frame difference method is adopted in the preprocessing stage to effectively suppress static background noise and highlight the dynamic changes between frames. This method enables the subsequently extracted spatiotemporal features to more accurately reflect the subtle dynamics of human physiological activities, providing richer basic information for the model to capture key physiological signals.

[0101] Unlike previous remote physiological signal detection methods that often use network structures such as RNN, LSTM, or CNN to model temporal dependencies, this invention introduces a state-space model, enabling the capture of long-term dependencies with lower computational complexity. Furthermore, the introduction of a state-space duality mechanism enables dynamic weighting of features across different time periods and spatial regions. This allows the model to not only adaptively adjust the importance of each feature but also effectively suppress noise interference on signal prediction, significantly improving detection accuracy and stability.

[0102] When focusing on time domain processing, important frequency information related to specific physiological parameters in physiological signals may be overlooked. After time series modeling, the present invention converts the signal to the frequency domain through fast Fourier transform (FFT), then uses a frequency domain feedforward network to enhance the key frequency domain components, and finally restores the enhanced signal through inverse Fourier transform (IFFT). This strategy of combining time domain and frequency domain can more comprehensively extract and enhance the characteristics of physiological signals, especially in environments with more interference, significantly improving the robustness and recognition accuracy of the signal.

[0103] In the past, a single-path attention mechanism was typically used during the feature fusion stage, which can make it difficult to balance local details with global information. However, the model of this invention employs two paths: self-attention and cross-attention, extracting information from the perspectives of internal consistency within the data and cross-feature complementarity, respectively. After individual processing, the information is then fused at the channel level, forming a richer feature representation. This multi-angle information fusion strategy enables the model to more comprehensively understand spatiotemporal dynamics, further improving the accuracy of long-range physiological signal prediction.

[0104] Example 2

[0105] This embodiment provides a remote physiological signal detection system based on state space and dual-path interaction, which applies the remote physiological signal detection method based on state space and dual-path interaction described in Example 1. The detection system includes: a frame difference preprocessing module, a time series dependency modeling module, a frequency domain feature enhancement module, a prediction module, and a sampling module. The working process of the detection system is divided into a training process and an estimation process.

[0106] During training:

[0107] The frame difference preprocessing module is used to obtain a sample set, which includes a face video and a real rPPG signal corresponding to the face video; the frame difference preprocessing module is also used to perform frame difference preprocessing on the original face video to extract temporal features;

[0108] The temporal dependency modeling module is used to perform temporal dependency modeling on the extracted temporal features through the state space model and the state space duality mechanism;

[0109] The frequency domain feature enhancement module is used to enhance the frequency domain features of the time series features through the frequency domain feedforward network;

[0110] The prediction module is used to input the time series features after frequency domain feature enhancement into the self-attention path and the cross-attention path respectively, and then fuse the time series features processed by the two paths and use the fused data as the predicted rPPG signal;

[0111] The model is trained by minimizing the difference between the predicted rPPG signal and the true rPPG signal to obtain a trained prediction model;

[0112] During the estimation process:

[0113] The sampling module is used to collect the face video to be estimated and input it into the trained prediction model; the corresponding predicted rPPG signal is output according to the processing method of the above modules.

[0114] The above description is only a preferred specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any technician familiar with the technical field, within the technical scope disclosed by the present invention, who makes equivalent replacements or changes based on the technical solution and inventive concept of the present invention, should be covered by the scope of protection of the present invention.

Claims

1. A remote physiological signal detection method based on state space and dual-path interaction, characterized in that: include: S1. Obtain a sample set, the sample set including a face video and a real rPPG signal corresponding to the face video; S2. Perform frame difference preprocessing on the original face video to extract temporal features; S3. Model the temporal dependency of the extracted temporal features through the state-space model and state-space duality mechanism; S4. Enhance the frequency domain features of the time series features through the frequency domain feedforward network; S5. Input the time series features after frequency domain feature enhancement into the self-attention path and the cross-attention path respectively, then fuse the time series features processed by the two paths and use the fused data as the predicted rPPG signal; S6. Performing model training by minimizing the difference between the predicted rPPG signal and the true rPPG signal to obtain a trained prediction model; S7. Input the face video of the person to be detected into the trained prediction model, and output the corresponding predicted rPPG signal with reference to the processing method of steps S2 to S5.

2. The remote physiological signal detection method based on state space and dual-path interaction according to claim 1 is characterized in that: In step S2, the expression of frame difference preprocessing is as follows: X Stem =Stem(X origin )+Stem(X origin +X diff ) Where, X origin is the original face video data; X diff is the differential data between adjacent frames; X Stem is the output feature map; Stem(·) represents the Frame Stem function.

3. The remote physiological signal detection method based on state space and dual-path interaction according to claim 2, characterized in that: In step S3, the expression of the state space model is as follows: h t =Ah t-1 +Bx t Where A, B and C are learnable parameter matrices; h t and h t-1 are the hidden state vectors at time t and t-1 respectively; x t is the input feature at time t, namely X Stem ;y t is the output feature vector at time t; is the transpose symbol.

4. The remote physiological signal detection method based on state space and dual-path interaction according to claim 3 is characterized in that: In step S3, the expression of the state space duality mechanism SSD is as follows: Where Q, K, and V are query, key, and value matrices, respectively; L is the structured mask matrix; and ° is the element-wise multiplication symbol.

5. The remote physiological signal detection method based on state space and dual-path interaction according to claim 1, characterized in that: In step S4, the processing method of the frequency domain feedforward network is: First, perform fast Fourier transform on the time series features, and the expression is as follows: Where f is the time series feature; H(f) is the expression of the time series feature in the frequency domain; h(t) is the expression of the time series feature in the time domain; dt is the time integral operator; e is a natural constant; j is an imaginary unit; Then, the frequency domain enhanced signal is restored through inverse Fourier transform, and the expression is as follows:

6. The remote physiological signal detection method based on state space and dual-path interaction according to claim 1, characterized in that: In step S5, the expression of the self-attention path processing is as follows: The expression of cross attention path processing is as follows: The concatenation expression of the two path features in the channel dimension is as follows: X fusion =Concat(FSA,FCA) Where, X SA is the output feature of the self-attention path; L S is the structured mask matrix of the self-attention path; Q S , K S and V S are the query, key, and value matrices corresponding to the self-attention path respectively; ° is the element-by-element multiplication symbol; T is the transpose symbol; X CA is the output feature of the cross attention path; L C is the structured mask matrix of the cross attention path; Q C , K C and V C are the query, key, and value matrices corresponding to the cross-attention path respectively; X fusion is the fusion feature of the output features of the self-attention path and the cross-attention path after splicing in the channel dimension; FSA is the output feature of the self-attention path after being enhanced by the frequency domain feedforward network; FCA is the output feature of the cross-attention path after being enhanced by the frequency domain feedforward network; Concat(·) is a concatenation process.

7. The remote physiological signal detection method based on state space and dual-path interaction according to claim 1, characterized in that: In step S6, the overall loss function expression during the model training process is as follows: L overall =L Time +L Freq Where, L overall is the overall loss, L Time Used to measure the matching degree between the predicted rPPG signal and the real rPPG signal in the time domain; L Freq Used to measure the similarity between the rPPG signal and the real rPPG signal in the frequency domain.

8. A remote physiological signal detection system based on state space and dual-path interaction, characterized in that: The remote physiological signal detection method based on state space and dual-path interaction according to any one of claims 1 to 7 is applied; the detection system comprises: A frame difference preprocessing module is used to obtain a sample set, which includes a face video and a real rPPG signal corresponding to the face video; the frame difference preprocessing module is also used to perform frame difference preprocessing on the original face video to extract temporal features; The temporal dependency modeling module is used to perform temporal dependency modeling on the extracted temporal features through the state space model and the state space duality mechanism; The frequency domain feature enhancement module is used to enhance the frequency domain features of the time series features through the frequency domain feedforward network; The prediction module is used to input the time series features after frequency domain feature enhancement into the self-attention path and the cross-attention path respectively, and then fuse the time series features processed by the two paths and use the fused data as the predicted rPPG signal. The model is trained by minimizing the difference between the predicted rPPG signal and the true rPPG signal to obtain a trained prediction model. The sampling module is used to collect facial videos of the person to be detected and input them into the trained prediction model; the corresponding predicted rPPG signal is output according to the processing method of the above modules.

Images