A non-contact real-time heart rate measurement method, system, and medium

Abstract

The application discloses a kind of non-contact real-time heart rate measurement method, system and medium, it is related to computer vision and physiological signal processing technical field, including: obtaining face video continuous frame and processing generation physiological space-time feature map, along the time axis of the physiological space-time feature map Division multiple cuttings And projected as feature vector, input to by multiple bidirectional Mamba block stacked feature coding layer, after coding, output by prediction layer with the time length of the physiological space-time feature map Equal length rPPG signal;In real-time inference stage, the reverse Mamba stream is shielded, only the forward Mamba stream is retained, and the rPPG signal of the current time is output frame by frame according to the current frame and the historical frame;The output rPPG signal is normalized and frequency domain transformed, and the main frequency in the physiological band is extracted to calculate the real-time heart rate value;The method solves the delay problem of current heart rate detection method requiring video cache detection.

Description

Technical Field

[0001] This invention relates to the fields of computer vision and physiological signal processing technology, and in particular to a non-contact real-time heart rate measurement method, system and medium. Background Technology

[0002] While existing contact-based health monitoring technologies (such as electrocardiograms (ECG) and photoplethysmography (PPG) on smart bracelets are mature, they require prolonged wear, leading to poor comfort, low compliance, and risks of skin damage. These limitations make them unsuitable for long-term monitoring needs in various scenarios, including home care and remote health monitoring. To overcome these contact limitations, remote photoplethysmography (rPPG) has emerged. The core mechanism of rPPG technology is that the heartbeat causes periodic changes in blood volume in the subcutaneous blood vessels of the face, resulting in fluctuations in the absorption of specific light wavelengths by hemoglobin and causing extremely weak periodic changes in the intensity of reflected light from the skin. Through video acquisition and signal modeling, physiological signals can be separated and reconstructed, enabling the measurement of key physiological indicators such as heart rate (HR), respiratory rate (RR), and heart rate variability (HRV). This technology overcomes the limitations of physical contact and shows great potential in scenarios such as telemedicine and home care.

[0003] Existing research on heart rate estimation based on facial videos largely relies on deep learning models such as Convolutional Neural Networks (CNNs) and Transformers. However, physiological signals on a face are typically extremely weak, and consecutive frames of facial video data contain a large amount of redundant information. This means that methods that directly extract physiological features from video data using deep learning models require a significant amount of parameters to handle this redundancy, increasing the model's workload. Furthermore, existing CNN- or Transformer-based methods require caching the entire video to obtain contextual information before decoding physiological features. In real-world applications, this makes it impossible to simultaneously capture video and output predictions, thus failing to meet real-time requirements. Summary of the Invention

[0004] Based on the technical problems existing in the background technology, the present invention proposes a non-contact real-time heart rate measurement method, system and medium, which solves the time delay problem of current heart rate detection methods that require video caching before detection.

[0005] The present invention proposes a non-contact real-time heart rate measurement method, comprising: Acquire consecutive frames of face video, detect facial key points in each frame and segment multiple skin regions, perform spatial pooling and color space transformation on each skin region, arrange the average pixel values ​​of each frame in time series, and input the generated physiological spatiotemporal feature map as the training set into the causal model. The physiological spatiotemporal feature map is divided into multiple blocks along its temporal axis and projected into feature vectors. These vectors are then input into a feature encoding layer consisting of multiple bidirectional Mamba blocks stacked together. Each bidirectional Mamba block contains a forward Mamba stream, a reverse Mamba stream, and a feature fusion unit. After encoding, the rPPG signal, which has the same temporal length as the physiological spatiotemporal feature map, is output through a prediction layer. The model parameters are then trained using labeled data and negative Pearson loss. During the real-time inference phase, the reverse Mamba stream is blocked, and only the forward Mamba stream is retained. The rPPG signal at the current moment is output frame by frame according to the current frame and historical frames. The output rPPG signal is normalized and frequency domain transformed to extract the dominant frequency within the physiological frequency band to calculate the real-time heart rate value.

[0006] Furthermore, the step of detecting facial key points and segmenting multiple skin regions in each frame specifically involves: A video segment of length T is cropped from consecutive frames of a face video using a sliding window. Facial landmark detection is performed on each frame of the video segment to segment multiple skin regions.

[0007] Furthermore, the horizontal axis of the physiological spatiotemporal feature map is the number of video frames, and the vertical axis is the number of skin region blocks segmented. The size of the physiological spatiotemporal feature map is the number of channels × the number of skin region blocks × the number of video frames, where the number of channels includes 6 channels: R, G, B, Y, U, and V.

[0008] Furthermore, the physiological spatiotemporal feature map is divided into multiple blocks along its temporal axis, wherein the spatial axis of the physiological spatiotemporal feature map remains intact, and each block contains complete spatial features and a single temporal segment.

[0009] Furthermore, the forward Mamba stream is configured to have true causality, utilizing only past time-series information; the reverse Mamba stream is configured to have false causality, and a dropout layer is added at the output.

[0010] Furthermore, the feature fusion unit is used to fuse the positive features of the forward Mamba stream output and the reverse features of the reverse Mamba stream output, and capture more complex feature representations through linear projection and the GELU activation function, thereby enhancing the expressive power of the model.

[0011] Furthermore, the prediction layer consists of two linear layers and a ReLU activation layer, which maps the encoded feature sequence into an rPPG signal of length T.

[0012] A non-contact real-time heart rate measurement system includes a training set construction module, a model training module, a model execution module, and a signal calculation module; The training set construction module is used to acquire continuous frames of face video, detect facial key points in each frame and segment multiple skin regions, perform spatial pooling and color space transformation on each skin region, arrange the average pixel values ​​of each frame in time series, and input the generated physiological spatiotemporal feature map as the training set into the causal model. The model training module is used to divide the physiological spatiotemporal feature map into multiple blocks along the temporal axis and project them into feature vectors. These vectors are then input into a feature encoding layer consisting of multiple bidirectional Mamba blocks stacked together. Each bidirectional Mamba block contains a forward Mamba stream, a reverse Mamba stream, and a feature fusion unit. After encoding, the module outputs an rPPG signal with the same temporal length as the physiological spatiotemporal feature map through a prediction layer. The module then uses labeled data to train the model parameters using negative Pearson loss. The model execution module is used to block the reverse Mamba stream during the real-time inference stage, retain only the forward Mamba stream, and output the rPPG signal at the current moment frame by frame according to the current frame and historical frames. The signal calculation module is used to normalize and frequency-domain transform the output rPPG signal, and extract the main frequency within the physiological frequency band to calculate the real-time heart rate value.

[0013] Furthermore, the training set construction module includes a skin region segmentation unit, which is used to cut out a video segment of length T from consecutive frames of a face video using a sliding window, perform facial key point detection on each frame of the video segment, and segment multiple skin regions.

[0014] A computer-readable storage medium storing a plurality of computer programs for being invoked by a processor and executing the method described above.

[0015] The advantages of the non-contact real-time heart rate measurement method, system, and medium provided by this invention are: it solves the latency problem of current heart rate detection methods that require video caching before detection; specifically, it extracts smooth skin regions by locating facial key points, transforms long video data into a feature image, and then uses a lightweight Mamba network structure for model training. Notably, during the inference phase, the Mamba structure can predict current physiological characteristics based solely on the current frame and past frame data, thus achieving real-time physiological indicator prediction capabilities. Attached Figure Description

[0016] Figure 1 This is a schematic diagram of the structural process of the present invention; Figure 2 This is a schematic diagram of the structure of a bidirectional Mamba block; Figure 3 The graph shows the rPPG signal obtained through the causal model. Figure 4 According to Figure 3 A diagram showing the calculated real-time heart rate. Detailed Implementation

[0017] The technical solution of the present invention will now be described in detail through specific embodiments. Many specific details are set forth in the following description to provide a thorough understanding of the invention. However, the present invention can be implemented in many other ways different from those described herein, and those skilled in the art can make similar modifications without departing from the spirit of the invention. Therefore, the present invention is not limited to the specific embodiments disclosed below.

[0018] like Figures 1 to 4 As shown, the present invention proposes a non-contact real-time heart rate measurement method, comprising: Step 1: Acquire consecutive frames of face video, detect facial key points in each frame and segment multiple skin regions, perform spatial pooling and color space transformation on each skin region, arrange the average pixel values ​​of each frame in time series, and input the generated physiological spatiotemporal feature map as the training set into the causal model. Step 2: Divide the physiological spatiotemporal feature map into multiple blocks along the temporal axis and project them into feature vectors. Input the vectors into a feature encoding layer consisting of multiple bidirectional Mamba blocks stacked together. Each bidirectional Mamba block contains a forward Mamba stream, a reverse Mamba stream, and a feature fusion unit. After encoding, the rPPG signal with the same temporal length as the physiological spatiotemporal feature map is output through the prediction layer. The model parameters are trained using labeled data through negative Pearson loss. Step 3: In the real-time inference stage, the reverse Mamba stream is blocked, and only the forward Mamba stream is retained. The rPPG signal at the current moment is output frame by frame according to the current frame and historical frames. Step 4: Normalize and frequency-domain transform the output rPPG signal to extract the dominant frequency within the physiological frequency band to calculate the real-time heart rate value.

[0019] This embodiment utilizes a temporal Mamba structure to achieve real-time heart rate detection, solving the latency problem of current heart rate detection methods that require video caching before detection. Specifically, it extracts smooth skin regions by locating facial key points, transforming long video data into a feature image, and then uses a lightweight Mamba network structure for model training. Notably, during the inference phase, the Mamba structure can predict current physiological characteristics based solely on current and past frame data, thus achieving real-time physiological indicator prediction capabilities.

[0020] The overall process of this embodiment is as follows: input of facial video data → generation of physiological spatiotemporal feature map → training of causal model → fixing of model parameters → real-time inference of physiological signals and heart rate.

[0021] (1) Input and preset parameters; Input data: includes labeled datasets D= in, Represents facial video data, This represents the pulse wave signal corresponding to the face video, i.e., the PPG signal. Represents the total sample size. For sample index.

[0022] Video slice length (time window): A slice of video using T frames; Physiological frequency band (candidate heart rate frequency range): 0.7Hz-4.0Hz, corresponding to a heart rate range of 42-240bpm.

[0023] (2) Construct the training set; From consecutive frames of a face video, a video segment of length T is cropped using a sliding window. Facial landmark detection is performed on each frame of the video segment, multiple skin regions are segmented, and spatial pooling and color space transformation are applied to generate a physiological spatiotemporal feature map. The horizontal axis represents the number of video frames, and the vertical axis represents the number of segmented skin region blocks. The specific steps are as follows: (a1) Facial landmark detection; For each frame of a face image in a continuous video, the coordinates of 68 key points of the face are detected using the Dlib library in OpenCV.

[0024] (a2) Skin region segmentation; The face is segmented into H independent skin regions based on the coordinates of 68 key points. It is worth noting that H can be customized according to the task.

[0025] (a3) Region pooling and color transformation; Each pixel in the skin region is pooled to obtain H vectors, each containing three channels: R, G, and B. Then, the RGB values ​​are mapped to the YUV color space according to a color transformation, resulting in a 6-channel vector containing R, G, B, Y, U, and V. The transformation formula is as follows: Y = 0.299R + 0.587G + 0.114B; U = -0.169R - 0.331G + 0.5B + ​​128; V = 0.5R - 0.419G -0.081B + 128.

[0026] (a4) Generation of physiological spatiotemporal feature maps; The average pixel values ​​of different frames are concatenated into a time series, and the time series of H skin regions and color channels are arranged in rows to obtain the final physiological spatiotemporal feature map, which has the size of C x H x T, where C is the number of color channels.

[0027] (3) Causal model; Overall architecture of the causal model: The model consists of 3 core modules, from top to bottom: a temporal axis patch embedding layer, a feature encoding layer consisting of L bidirectional Mamba blocks stacked together, and a prediction layer.

[0028] (b1) Timing axis Patch embedding layer; Patch partitioning: The patch is divided only along the temporal axis of the physiological spatiotemporal feature map, while the spatial axis remains intact. This ensures that each patch contains complete spatial features and a single temporal segment. The dimension of each patch is (C, H, t_patch), where t_patch is the number of consecutive frames contained in each patch.

[0029] Embedding operation: Project each slice as a 1D vector, and perform feature mapping on all slices using a 2D convolution operation. The convolution kernel parameters are: kernel size (H, t_patch), stride (H, t_patch), and the dimension of the embedded feature vector is (B, N, D), where B is the batch size, N is the number of slices, and D is the embedding dimension.

[0030] (b2) Feature coding layer; The feature encoding layer consists of L stacked bidirectional Mamba blocks, each containing a forward Mamba stream, a reverse Mamba stream (with path-level dropout constraints), and a feature fusion unit.

[0031] The forward Mamba stream and the reverse Mamba stream (with path-level dropout constraints) are two independent and structurally identical submodules. Each submodule is a complete Mamba block in itself. The path-level dropout constraints only apply to the output of the reverse Mamba stream.

[0032] In both forward and reverse Mamba streams, the specific structure of a Mamba block includes: Linear projection (fully connected layer, which upscales features and splits them into u and z branches), local convolution (depthseparable one-dimensional convolution + SiLU activation function), and selective state space scanning (generating...). Gating and output projection. For the SSM backbone, this refers to the core computational part of the selective state-space model, a key component in the Mamba block responsible for modeling long-term temporal dependencies.

[0033] Linear projection expands the original features into a higher dimension, and then splits them into two branches along the channel dimension: the u branch is fed into the SSM backbone, and the z branch is used as a gating mechanism.

[0034] Local convolution passes the u-branch through local convolution (depthseparable one-dimensional convolution + SiLU activation function) to capture local temporal relationships.

[0035] Selective state-space scanning: The u-branch, after local convolution, is fed into the SSM backbone, starting from the current input. Dynamically generated .in, It controls the state update step size, which can be understood as the strength of the memory for updating the current token / frame. It controls how the current input is written to the state. It controls how the current state is read out and output.

[0036] Gating: The resulting z-branch will act as a gating mechanism to enhance the nonlinear expressive power.

[0037] Output projection: The z-branch does not pass through the SSM, but is only used as a gating signal to multiply with the SSM output, and then the dimension is projected back to the original D through a linear layer, so that Mamba blocks can be stacked.

[0038] The forward Mamba stream is configured to have true causality, utilizing only past time-series information; its function is to learn the forward periodic dependencies of rPPG by utilizing only past time-series information (causality), adapting to streaming scenarios during inference.

[0039] The reverse Mamba stream is configured with causality set to false, and a path-level dropout layer is added to the output with a dropout probability in the range [0.0, 1.0]. Its function is to assist the learning of the forward Mamba stream. By using dropout, the information from the entire reverse Mamba stream disappears, ensuring the output consists entirely of the forward Mamba stream and the residual. This prevents the forward Mamba stream from over-relying on future information and forces it to autonomously capture periodicity. Dropout is a regularization technique.

[0040] In this embodiment, due to the randomness of dropout, the features output by the reverse Mamba stream are not exactly the same in each training iteration (the reverse features are randomly zeroed out). This uncertainty leads to a lack of information provided by the reverse Mamba stream to the feature fusion unit, resulting in lower reliability compared to the stable and complete information of the forward Mamba stream. Unlike the traditional use of Dropout to randomly discard reverse stream information during inference, the method in this embodiment aims to completely discard and not rely on reverse stream information during inference. By adding path-level dropout to randomly discard the entire reverse path, the causal model is forced to train during training to perfectly reconstruct features using only the forward stream without reverse context information, thereby achieving the goal of reducing the weight of reverse stream information.

[0041] (b3) Feature fusion unit; In each bidirectional Mamba block, the forward features (B, N, D) output from the forward Mamba stream and the reverse features (B, N, D) output from the reverse Mamba stream are fused by a feature fusion unit. The fused features are obtained by adding them together. Then, the input features before entering the bidirectional Mamba block are added to the current fused features as residuals and then processed by a LayerNorm layer (layer normalization).

[0042] The fused features are nonlinearly enhanced using an MLP enhancement layer, preserving temporal dependencies while introducing local feature interactions. The MLP enhancement layer consists of a first linear layer, a GELU activation layer, and a second linear layer.

[0043] In one embodiment of the MLP enhancement layer, the normalized fused features (B, N, D) are passed through a first linear layer (e.g., a fully connected layer) to map the embedding dimension from D to a higher dimension (usually 4×D or as set by the design), denoted as (B, N, 4D). Its function is to expand the feature space and provide richer representation capabilities for subsequent nonlinear transformations.

[0044] The GELU activation function (Gaussian error linear unit) is applied element-wise to the output of the first linear layer. This embodiment introduces nonlinearity, enabling the causal model to learn more complex feature interactions. At the same time, it is smoother than the ReLU activation function, which is conducive to gradient flow.

[0045] The activated features (B, N, 4D) are passed through a second linear layer (e.g., a fully connected layer) and mapped back to the original embedding dimension D, outputting (B, N, D). In this embodiment, the enhanced features are compressed back to the original dimension for easy connection with residuals (if any) or subsequent use.

[0046] Therefore, the MLP enhancement layer in this embodiment is essentially two fully connected layers with a GELU activation function inserted in between. Without changing the temporal order, it independently performs nonlinear transformations on the feature vectors of each slice, thereby enhancing the expressive power of the features and introducing local inter-channel interactions.

[0047] (b4) Prediction layer; The feature sequence (B, N, D) encoded by the feature encoding layer is mapped to an rPPG signal of the same length as the time sequence T of the physiological spatiotemporal feature map. Specifically, the time sequence length N is upsampled by t_patch times to the length T through linear upsampling, and then the feature vector is mapped through two linear layers and a ReLU activation layer to obtain an rPPG signal (B, T) of the same length as the time sequence T.

[0048] (b5) Training of the causal model; The NegPearson loss function is used as the loss function. This negative Pearson loss ensures that the predicted rPPG signal is consistent with the temporal pattern of the real signal, significantly improving the quality of rPPG signal recovery. Backpropagation is used to update model parameters, with the learning rate, number of training epochs, and batch size adjusted based on actual training performance. For dropout probability settings, a course learning strategy is adopted. For example, in the early stages of training, the dropout probability is set to 0.0, allowing the bidirectional Mamba to fully learn the complete temporal representation. Subsequently, p is progressively and linearly increased from 0.0 to 1.0, forcing the forward flow to seamlessly take over all prediction tasks, where p refers to the dropout probability of the path-level dropout layer added at the output of the reverse Mamba flow.

[0049] (b6) Real-time reasoning for causal models; During the inference phase, this embodiment does not require caching the face video before predicting the rPPG signal; instead, it enables real-time video acquisition and rPPG signal prediction. Specifically, the reverse Mamba stream channels in all bidirectional Mamba blocks of the trained causal model are masked, retaining only the forward Mamba stream.

[0050] Based on this, for an input at time t, the model relies only on information from past (1-t) times and predicts the rPPG signal at the current time.

[0051] The specific inference flow of a forward Mamba stream is as follows: When stacking L layers of Mamba blocks, the layers are sequentially concatenated, and each layer is a forward Mamba stream (causal pattern): Layer 1: Receives feature vectors from the Patch embedding layer and outputs features. And save the internal state of that layer; Layer 2: The features output from Layer 1 As input, output features And save the internal state of that layer; And so on, Layer L: Features output from layer L-1 As input, output features .

[0052] Finally, the features The signal is fed into the prediction layer to obtain the rPPG signal of the current frame.

[0053] The advantage of this embodiment in retaining only the positive Mamba stream during the real-time inference stage is (c1) to (c4): (c1) Achieve true real-time streaming inference: The forward Mamba stream is configured with causal=True, meaning its computation depends only on the current and historical frames, not on any future frames. Therefore, upon acquiring a new video frame, the model can immediately output the rPPG signal point corresponding to the current frame, without waiting for subsequent frames to arrive.

[0054] The reverse Mamba stream uses future frame information (causality is false) during training. If this information is retained during the inference phase, the entire video or at least several future frames need to be cached to calculate the current output. This results in a fixed latency (at least equal to the backtracking length of the reverse stream), which cannot meet the requirements for real-time monitoring.

[0055] (c2) Significantly reduces computational load and memory usage; The single-step inference complexity of a forward Mamba stream is O(1×D) (where D is the embedding dimension), which is independent of the total length of the video, and the memory only needs to maintain a fixed-size state vector.

[0056] Reverse Mamba streaming requires scanning from the end of the sequence backwards. In streaming scenarios, this either requires caching the entire history or makes computation impossible. By masking this, inference computation is reduced by approximately 50% (compared to twice the computation for bidirectional streaming), memory usage is lower, and it is more suitable for deployment on resource-constrained platforms such as mobile phones and embedded devices.

[0057] (c3) Maintain consistency with the training objective (avoid performance degradation); During the training phase, dropout was used to suppress the information weights of the reverse Mamba stream, which forced the forward Mamba stream to autonomously learn to capture the periodic dependence of the rPPG signal, rather than relying on future information.

[0058] Experimental verification (such as) Figure 3 As shown in the figure, the model using only the forward Mamba stream can still output high-quality periodic rPPG signals during inference. Forcibly retaining the untrained reverse Mamba stream (or simply removing it without fine-tuning) may introduce noise or disrupt causality, leading to inaccurate heart rate estimation.

[0059] (c4) Meets the physical constraints of the actual application scenario; In scenarios such as remote health monitoring and home care, the system cannot acquire future frames (because the future has not yet occurred). Only causal models can achieve simultaneous data acquisition and output.

[0060] When used in hospitals or at home, users expect to see heart rate changes in real time (such as a recovery curve after exercise). Any non-causal delay will reduce the user experience and may delay warnings.

[0061] (b7) Real-time heart rate output; For the rPPG signal y output in real time by the causal model, the heart rate value is calculated once per second as the real-time heart rate monitoring frequency. Specifically, the initial time window is set to 5 seconds (configurable), then the signal length is 150. After 5 seconds of real-time acquisition of face video, the real-time heart rate value of the rPPG signal with the latest window length is calculated every 1 second (configurable).

[0062] Heart rate is calculated as follows: rPPG signal normalization: Normalize the predicted rPPG signal y.

[0063] Frequency domain analysis of rPPG signal: The spectrum of the normalized rPPG signal y is obtained by applying Fast Fourier Transform. Based on the frequency range of normal heart rate (0.7Hz-4.0Hz), bandpass filtering is used to filter out noise signals in the spectrum that are not within the heart rate range.

[0064] Calculate heart rate: Find the frequency f with the largest amplitude in the spectrum within the physiological frequency band, and calculate the corresponding heart rate value. .

[0065] The physiological spatiotemporal feature map technology used in this embodiment effectively filters redundant information in face video data, achieving an information compression ratio greater than 90%. The Mamba network structure used in this embodiment has a complexity of O(L×D) during the training phase, compared to the Transformer structure's O(L×D) complexity. 2 The method has lower complexity (1×D); during the inference phase, the computational complexity of this embodiment is O(1×D), where L is the video length and D is the embedding dimension. Compared to other methods that require caching the video before detection, this method can output the corresponding heart rate while inputting the video, achieving real-time monitoring.

[0066] Compared to the Transformer architecture, the Mamba network in this embodiment addresses the limitations of existing Transformer-based rPPG methods, which cannot provide real-time, low-latency, and long-duration continuous monitoring. This allows non-contact heart rate measurement to move from post-hoc offline analysis to a phase of on-the-fly data acquisition and output. The advantages of this embodiment are as follows: Meeting real-time requirements: Scenarios such as home care and telemedicine require simultaneous data collection and output of heart rate. The Transformer's buffer-then-calculate mode can cause delays of several seconds or even tens of seconds, while Mamba can update the heart rate estimate immediately after each frame of input.

[0067] Reduce the computing power requirements of terminal devices: Mamba's O(1) inference complexity and fixed memory usage enable it to run smoothly on edge devices such as mobile phones and Raspberry Pi, while Transformer's video memory and computing power will quickly exceed the terminal's capacity as the video length increases.

[0068] Supports continuous monitoring for unlimited duration: Transformer is limited by the maximum context length (usually 512~2048 frames), and long videos must be segmented and inter-segment dependencies lost; Mamba can theoretically handle continuous video streams of any length, making it suitable for long-term scenarios such as nighttime sleep monitoring.

[0069] Example 1 This embodiment verifies the performance of the proposed real-time heart rate measurement method in monitoring heart rate when a face video is captured by a mobile phone in an indoor environment. The experiment uses data from the VIPL dataset, captured by the front-facing camera of a mobile phone (e.g., Huawei P9) at 30fps with a resolution of 1920×1080, retaining only the facial area. The face is extracted frame-by-frame using 68 facial key points, including the forehead, cheeks, and chin, and divided into 64 non-overlapping regions. The test uses a 300-frame video time window, resulting in a final physiological spatiotemporal feature map of 6x64x300 pixels. The spatiotemporal feature map is calculated frame-by-frame and input into a causal model to obtain the corresponding rPPG signal (…). Figure 3 ), and calculate the heart rate value every 1 second (30 frames). Figure 4 ).from Figure 3 As can be seen, the rPPG signal obtained using the causal model of this embodiment has significant periodicity, and the real-time heart rate signal of the human body can be effectively calculated according to step (b7).

[0070] Example 2 This embodiment uses the MMPD dataset for experiments, and the results are shown in Table 1. MMPD is a highly challenging dataset containing data from 33 subjects under different lighting conditions, skin color, and head movement complexity. This dataset contains more than 11 hours of video footage captured by mobile phones, covering diverse environmental factors, including significant noise and motion artifacts, realistically simulating the complexity of real-world scenes.

[0071] Table 1

[0072] In Table 1, ICA stands for Independent Component Analysis, CHROM stands for Chromaticity Method (or Color Method), POS stands for Planar Orthogonal Skin Method, DeepPhys (ECCV18) stands for Deep Physiology Network, TS-CAN (NIPS2020) stands for Temporal Exchange Attention Network, PhysFormer (CVPR2022) stands for Physiology Transformer based on Transformer architecture, EfficientPhys (WACA2023) stands for Efficient Physiology Network, PhysNet (BMVC2018) stands for Physiology Network, and RhythmMamba (AAAI2025) stands for Rhythm Mamba.

[0073] For the arrows indicating MAE (bpm) ↓, RMSE (bpm) ↓, and Pearson (ρ) ↑, ↓ indicates that the lower the value, the better the algorithm performance (for error-related indicators), and ↑ indicates that the higher the value, the better the algorithm performance (for correlation-related indicators).

[0074] As shown in Table 1, the causal model in this embodiment exhibits excellent accuracy, significantly outperforming traditional methods and most deep learning methods. Compared to traditional methods (ICA, CHROM, POS), this embodiment reduces error by approximately 60%–70% and improves correlation by 4–8 times. Compared to earlier deep learning methods (DeepPhys, TS-CAN, PhysFormer, EfficientPhys), this embodiment significantly outperforms in all metrics. For example, it reduces absolute error by approximately 50% compared to TS-CAN (MAE=8.97); and it has a lower error (4.46 < 4.81) and higher correlation (0.84 vs 0.60) compared to PhysNet (currently recognized as a better benchmark).

[0075] Compared to the state-of-the-art RhythmMamba (AAAI2025), the performance of the method in this embodiment is extremely close, almost on par. The method in this embodiment has a slightly higher MAE (1.3 bpm) and a slightly higher RMSE (0.46 bpm), but the Pearson score is identical (0.84). Statistically, this slight difference may not be significant, and the method in this embodiment possesses causal real-time inference capabilities, while RhythmMamba (AAAI2025)'s Mamba module employs a multi-timescale strategy, requiring caching of the entire video length, and therefore does not support real-time streaming inference. Therefore, the method in this embodiment achieves real-time performance while maintaining accuracy comparable to state-of-the-art methods.

[0076] The method in this embodiment demonstrates robustness on the highly challenging MMPD dataset, which contains realistic noise such as complex lighting, skin color variations, and head movements. The method still achieves a MAE < 5 bpm, proving its strong resistance to motion artifacts and lighting changes. Compared to PhysNet (BMVC2018) (MAE = 4.81), it performs better on the same dataset, indicating that the design incorporating physiological spatiotemporal feature maps and causal Mamba effectively improves generalization ability in complex scenes.

[0077] Therefore, none of the comparison methods in Table 1 emphasize real-time inference capabilities (most require caching the entire video). In addition, the method in this embodiment uses only forward flow during the inference stage, can output heart rate frame by frame, has constant latency (O(1)), and does not significantly sacrifice accuracy (almost on par with RhythmMamba (AAAI2025)). This is the core innovation of this embodiment.

[0078] Therefore, the causal model proposed in this embodiment achieves better heart rate estimation accuracy (MAE=4.46bpm, Pearson=0.84) than most existing methods on the MMPD dataset, and is comparable to the current state-of-the-art RhythmMamba (AAAI2025). It also has real-time streaming inference capability, achieving the best balance between accuracy and real-time performance, and verifying the effectiveness of the joint spatiotemporal feature map and causal Mamba structure.

[0079] Based on the above description of the embodiments, those skilled in the art will understand that the non-contact real-time heart rate measurement method, system, and medium described in this embodiment can be implemented in pure software or deployed and run on a general-purpose or dedicated computing hardware platform. Based on this essence, the technical solution of this embodiment can be specifically implemented in the form of a software product containing program instructions. This software product can be stored on various non-volatile storage media or directly deployed as a local or cloud service. The program instructions are used to cause a computer device with processing capabilities—including but not limited to personal computers, server clusters, mobile terminals, or other network devices—to execute the steps described in this embodiment.

[0080] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.

Claims

1. A non-contact real-time heart rate measurement method, characterized by, include: Acquire consecutive frames of face video, detect facial key points in each frame and segment multiple skin regions, perform spatial pooling and color space transformation on each skin region, arrange the average pixel values ​​of each frame in time series, and input the generated physiological spatiotemporal feature map as the training set into the causal model. The physiological spatiotemporal feature map is divided into multiple blocks along its temporal axis and projected into feature vectors. These vectors are then input into a feature encoding layer consisting of multiple bidirectional Mamba blocks stacked together. Each bidirectional Mamba block contains a forward Mamba stream, a reverse Mamba stream, and a feature fusion unit. After encoding, the rPPG signal, which has the same temporal length as the physiological spatiotemporal feature map, is output through a prediction layer. The model parameters are then trained using labeled data and negative Pearson loss. During the real-time inference phase, the reverse Mamba stream is blocked, and only the forward Mamba stream is retained. The rPPG signal at the current moment is output frame by frame according to the current frame and historical frames. The output rPPG signal is normalized and frequency domain transformed to extract the dominant frequency within the physiological frequency band to calculate the real-time heart rate value.

2. The method of claim 1, wherein, The process of detecting facial landmarks and segmenting multiple skin regions in each frame specifically involves: A video segment of length T is cropped from consecutive frames of a face video using a sliding window. Facial landmark detection is performed on each frame of the video segment to segment multiple skin regions.

3. The method of claim 1, wherein, The horizontal axis of the physiological spatiotemporal feature map is the number of video frames, and the vertical axis is the number of skin region blocks segmented. The size of the physiological spatiotemporal feature map is the number of channels × the number of skin region blocks × the number of video frames, where the number of channels includes 6 channels: R, G, B, Y, U, and V.

4. The method of claim 1, wherein, The physiological spatiotemporal feature map is divided into multiple segments along its temporal axis, wherein the spatial axis of the physiological spatiotemporal feature map remains intact, and each segment contains complete spatial features and a single temporal segment.

5. The method of claim 1, wherein, The forward Mamba stream is configured to have true causality, utilizing only past time-series information; the reverse Mamba stream is configured to have false causality, and a dropout layer is added at the output.

6. The method according to claim 1, characterized in that, The feature fusion unit is used to fuse the positive features of the forward Mamba stream output and the reverse features of the reverse Mamba stream output, and capture more complex feature representations through linear projection and the GELU activation function.

7. The method according to claim 2, characterized in that, The prediction layer consists of two linear layers and a ReLU activation layer, which maps the encoded feature sequence into an rPPG signal of length T.

8. A non-contact real-time heart rate measurement system, characterized in that, It includes a training set construction module, a model training module, a model execution module, and a signal calculation module; The training set construction module is used to acquire continuous frames of face video, detect facial key points in each frame and segment multiple skin regions, perform spatial pooling and color space transformation on each skin region, arrange the average pixel values ​​of each frame in time series, and input the generated physiological spatiotemporal feature map as the training set into the causal model. The model training module is used to divide the physiological spatiotemporal feature map into multiple blocks along the temporal axis and project them into feature vectors. These vectors are then input into a feature encoding layer consisting of multiple bidirectional Mamba blocks stacked together. Each bidirectional Mamba block contains a forward Mamba stream, a reverse Mamba stream, and a feature fusion unit. After encoding, the module outputs an rPPG signal with the same temporal length as the physiological spatiotemporal feature map through a prediction layer. The module then uses labeled data to train the model parameters using negative Pearson loss. The model execution module is used to block the reverse Mamba stream during the real-time inference stage, retain only the forward Mamba stream, and output the rPPG signal at the current moment frame by frame according to the current frame and historical frames. The signal calculation module is used to normalize and frequency-domain transform the output rPPG signal, and extract the main frequency within the physiological frequency band to calculate the real-time heart rate value.

9. The system according to claim 8, characterized in that, The training set construction module includes a skin region segmentation unit, which is used to cut out a video segment of length T from consecutive frames of a face video using a sliding window, perform facial key point detection on each frame of the video segment, and segment multiple skin regions.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a plurality of computer programs, which are used to be invoked by a processor and to execute the method as described in any one of claims 1-7.

Images