Remote heart rate estimation method and system based on curriculum learning

Abstract

The present application relates to the technical field of computer vision, and discloses a remote heart rate estimation method and system based on curriculum learning, which comprises the following steps: acquiring a face video and constructing a multi-scale space-time graph; constructing a physiological signal estimation network; calculating a physiological difficulty index score composed of an objective physiological quality item and a model feedback difficulty item for each training sample; sorting the training samples from easy to difficult according to the physiological difficulty index score, and intercepting the data usage proportion of the sorted training samples with the training round; in each training round, performing boundary sampling based on the intercepted training samples to construct a training subset; training the physiological signal estimation network using the training subset, updating the feedback information according to the training loss, and recalculating the physiological difficulty index score to form an iterative closed loop. The present application improves the remote heart rate estimation accuracy and training efficiency based on remote photoplethysmography.

Description

Technical Field

[0001] This invention relates to the field of computer vision technology, specifically to a remote heart rate estimation method and system based on course learning. Background Technology

[0002] Remote heart rate estimation is a non-contact measurement method based on remote photoplethysmography (rPPG). It acquires facial video using ordinary camera equipment and recovers the pulse wave signal from the video using rPPG to estimate physiological parameters such as heart rate. Compared with contact sensing methods, this method has advantages such as lower cost, non-contact acquisition, and ease of deployment, making it highly valuable in health monitoring, human-computer interaction, and mobile healthcare. Existing methods mainly include traditional methods based on color space transformation and blind source separation, as well as deep learning methods based on convolutional networks, Transformers, state-space models, and diffusion models. Previous research has improved performance by suppressing head movement and illumination interference, enhancing long-range temporal modeling capabilities, reducing model complexity, and using iterative denoising to recover the pulse wave. However, in real-world scenarios, rPPG signals remain very weak and easily affected by head movement, illumination changes, occlusion, and skin color differences, resulting in significant differences in the extractability of physiological signals between different samples. Current training processes primarily employ random sampling or uniform scheduling, lacking evaluation and organization mechanisms tailored to differences in physiological signal quality. Furthermore, the model's learning process is unstable in the early stages of training, easily affecting convergence stability and signal learning efficiency. Therefore, how to construct a curriculum learning strategy around physiological quality differences, while ensuring modeling efficiency and improving heart rate estimation accuracy, remains a pressing issue in this field. Summary of the Invention

[0003] This invention provides a remote heart rate estimation method and system based on course learning, which solves the problems in the prior art such as the ineffective utilization of the extractability differences of physiological signals between samples, the relatively random organization of training data, and the easy occurrence of sample sorting bias due to relying solely on model feedback in the early stages of training. This improves the accuracy and training efficiency of remote heart rate estimation based on remote photoplethysmography.

[0004] This invention constructs a multi-dimensional physiological signal quality assessment and curriculum learning framework based on the differences in physiological signal quality contained in training samples. On the one hand, it extracts three types of objective physiological quality indicators—motion amplitude, frequency band signal-to-noise ratio, and regional consistency—from multi-scale spatiotemporal maps, and combines them with the model feedback difficulty item during training to form a four-component unified scoring mechanism, enabling training samples to be organized and scheduled according to physiological signal quality. On the other hand, the above scoring results are used for curriculum learning scheduling and boundary sampling, allowing the model to learn step by step in order of difficulty. Furthermore, the Mamba state-space model is introduced into the signal estimation backbone to balance long-range time series modeling capabilities and computational efficiency, further improving pulse wave reconstruction performance.

[0005] To solve the above-mentioned technical problems, the present invention adopts the following technical solution: In a first aspect, the present invention provides a remote heart rate estimation method based on course learning, comprising the following steps: S1. Acquire facial video, extract multi-channel spatiotemporal features from each region of interest in the facial area, and construct a multi-scale spatiotemporal map; S2. Construct a physiological signal estimation network. The physiological signal estimation network takes a multi-scale spatiotemporal map, a noisy pulse wave, and the corresponding dynamic conditions as input. It reconstructs the pulse wave signal through Mamba spatiotemporal modeling and diffusion denoising. The reconstructed pulse wave signal is then subjected to spectral analysis to obtain the heart rate estimation result. S3, calculates a physiological difficulty index score for each training sample, consisting of an objective physiological quality item and a model feedback difficulty item; S4. The training samples are sorted according to the physiological difficulty index score, and the data of the sorted training samples are truncated according to the proportion of training rounds. S5, in each training round, perform boundary sampling based on the truncated training samples to construct a training subset; S6: Train the physiological signal estimation network using a training subset, update the feedback information based on the training loss, and return to S3 to recalculate the physiological difficulty index score; the feedback information includes smoothing loss, uncertainty, learning progress, and learning progress difference. S7, repeat S3 to S6 until the physiological signal estimation network converges or reaches the maximum number of training rounds.

[0006] In one embodiment, the objective physiological quality items include a motion amplitude index, a frequency band signal-to-noise ratio index, and a regional consistency index; Amplitude of motion index for: ; Indicates the first The training sample in the main channel is the nth training sample. The first region of interest The spatiotemporal graph signal value at each moment, Indicates the first Index of regions of interest, Indicates the first Index of each moment; Bandwidth signal-to-noise ratio for: ; Indicates the first The spatiotemporal map signal of the main channel of the multi-scale spatiotemporal map of the training samples at the frequency Power spectral density at that point Indicates the frequency range corresponding to the effective physiological components; Regional Consistency Indicators for: ; Indicates the first The set of region pairs between regions of interest of each training sample. This indicates a correlation calculation; Indicates by the first The first training sample in the main channel The spatiotemporal plot signal values ​​of a region of interest at each time point are arranged in chronological order to form a time-series signal. Indicates by the first The first training sample in the main channel The spatiotemporal plot signal values ​​of a region of interest at each time point are arranged in chronological order to form a time-series signal. and These represent the first and second parts of the region pair, respectively. The region of interest and the first Index of regions of interest, Indicates timing length; Objective physiological quality items Represented as: ; , and They represent the first The normalized values ​​of motion amplitude index, frequency band signal-to-noise ratio index, and region consistency index corresponding to each training sample are: , , These are the corresponding weight coefficients. This indicates that the input value will be truncated to... A function on an interval.

[0007] In one embodiment, the model feedback difficulty term The calculation method is as follows: ; Indicates the first The normalized smooth loss for each training sample. This represents the difference in learning progress, which is the complement of normalized learning progress. and This represents the weighting coefficient.

[0008] In one embodiment, the physiological difficulty index score for: ; in, For objective physiological quality items, This is the difficulty level of the model feedback. Indicates the current learning progress of the course. This is a physiological quality correction factor that varies with the progress of course learning. Physiological quality threshold, This indicates that the input value will be truncated to... A function on an interval.

[0009] In one embodiment, during the warm-up phase of training, the physiological signal estimation network is trained using all training samples. An objective physiological quality term is calculated based on the multi-scale spatiotemporal graph of each training sample. This objective physiological quality term is obtained solely from the training samples themselves and does not depend on the training feedback of the physiological signal estimation network. Simultaneously, a smoothing loss is obtained based on the loss of each training sample, and the learning progress, learning progress difference, and uncertainty are further calculated to form feedback information and update the model feedback difficulty term. After the warm-up phase, a physiological difficulty index score is calculated based on the objective physiological quality term and the model feedback difficulty term, and the training samples are sorted and courses are selected.

[0010] In one embodiment, the step of proportionally truncating the data of the sorted training samples with each training round specifically includes: Data usage ratio Adopting an exponential expansion strategy: ; Alternatively, a linear expansion strategy can be adopted: ; in, Use the proportion for the initial data. For the final data usage ratio, This represents the current course learning progress; the data usage ratio applies to the course learning phase after the warm-up phase of training, where all training samples are used for training.

[0011] In one embodiment, the step of performing boundary sampling based on the truncated training samples to construct a training subset specifically includes: The physiological difficulty index scores and objective physiological quality items of each training sample in the same video directory are aggregated to obtain a directory-level difficulty score. and directory-level objective quality score : , ; in, Indicates the first The video directory number The physiological difficulty index score corresponding to each training sample Indicates the first The video directory number The objective physiological quality item corresponding to each training sample Indicates the first The number of training samples contained in each video directory; Training samples are sorted from low to high according to their physiological difficulty index scores. Within the candidate range determined by the current data usage ratio, training samples with objective physiological quality items not lower than a set threshold are selected, and a set number of training samples are selected as easy-to-learn samples according to the sorting. When the number of training samples meeting the set threshold is insufficient, the remaining training samples within the candidate range are supplemented according to the physiological difficulty index scores from low to high. A boundary candidate pool is constructed from training samples that have not been selected as easy-to-learn samples and whose sorting position is close to the current course boundary. Boundary samples are selected from the boundary candidate pool by combining the directory-level difficulty score, directory-level objective quality score, learning progress, and uncertainty. When the preset exploration conditions are met, training samples with objective physiological quality items not lower than the exploration quality threshold are selected from the remaining training samples that have not been selected. A set number of samples are selected as exploration samples according to the weighted result of uncertainty and normalized smoothing loss from high to low. The easy-to-learn samples, boundary samples, and exploration samples are combined to form the training subset of the current training round.

[0012] In one embodiment, updating the feedback information based on the training loss specifically includes: The loss values ​​of the training samples that have already participated in training are updated using an exponential moving average. The training sample at the th ... Smooth loss over each training epoch Represented as: ; in, This indicates the first training round. The current loss value for each training sample. For smoothing coefficients; The learning progress is calculated and updated smoothly based on the non-negative decrease in the smoothing loss between two consecutive smoothing losses; The learning progress difference is the complementary quantity of normalized learning progress; Uncertainty is calculated based on the absolute value of the difference between the current loss and the smoothed loss, and then updated smoothly.

[0013] In one embodiment, the dynamic conditions include: amplitude conditions and trend conditions calculated from the noisy pulse wave, and the corresponding diffusion time step encoding.

[0014] In a second aspect, the present invention provides a remote heart rate estimation system based on course learning, for use in any embodiment of the first aspect, comprising: The multi-scale spatiotemporal map preprocessing module acquires facial videos, extracts multi-channel spatiotemporal features from each region of interest in the facial region, and constructs a multi-scale spatiotemporal map. The physiological signal estimation network takes multi-scale spatiotemporal maps, noisy pulse waves and corresponding dynamic conditions as inputs, and reconstructs pulse wave signals through Mamba spatiotemporal modeling and diffusion denoising. The heart rate estimation unit performs spectral analysis on the reconstructed pulse wave signal to obtain the heart rate estimation result; The physiological difficulty index scoring module calculates a physiological difficulty index score for each training sample, consisting of objective physiological quality items and model feedback difficulty items. The course learning scheduling module sorts the training samples according to the physiological difficulty index score, and extracts data from the sorted training samples according to the usage ratio in each training round. The boundary sampling module performs boundary sampling based on the truncated training samples in each training round to construct a training subset; The training execution module trains the physiological signal estimation network using a training subset, updates the feedback information based on the training loss, and returns to the physiological difficulty index scoring module to recalculate the physiological difficulty index score. The feedback information includes smoothing loss, uncertainty, learning progress, and learning progress difference. Repeatedly execute the physiological difficulty index scoring module, the course learning scheduling module, the boundary sampling module, and the training execution module until the physiological signal estimation network converges or reaches the maximum number of training rounds.

[0015] The system and method in this invention correspond to each other; the specific technical solutions applicable to the method are also applicable to the system.

[0016] Compared with the prior art, the beneficial technical effects of the present invention are: First, this invention establishes a physiological signal quality curriculum learning mechanism for supervised learning scenarios. By combining motion amplitude, frequency band signal-to-noise ratio, regional consistency, and model feedback difficulty to form a physiological difficulty index score, the organization of training samples shifts from traditional random sampling to an ordered scheduling consistent with the extractability of physiological signals. This allows for the priority use of high-quality, easily learnable samples in the early stages of training, improving the stability of training convergence.

[0017] Second, this invention reduces the impact of unstable model feedback signals on sample organization during the initial training phase by coordinating the design of objective physiological indicators and model feedback. During the warm-up phase, using all training samples, the three objective components—motion amplitude, frequency band signal-to-noise ratio, and regional consistency—can be directly calculated from the multi-scale spatiotemporal graph, and model feedback information is accumulated simultaneously. After the warm-up phase, the objective physiological quality component provides a quality basis for the initial sample ranking that is independent of the network state, while the model feedback difficulty component is dynamically updated as training progresses, thereby achieving adaptive adjustment of the sample learning difficulty.

[0018] Third, the boundary sampling mechanism proposed in this invention does not perform a simple hard truncation on the ranked samples. Instead, it first constructs a base of easy-to-learn samples, and then selects boundary samples from the subsequent boundary candidate pool based on the directory-level boundary utility score and quality bonus. Furthermore, it mixes in a small number of exploration samples selected based on a weighted result of uncertainty and normalized smoothing loss. This mechanism balances the stability of course learning, the learning value of boundary samples, and the diversity of training data. It can reduce training oscillations caused by introducing a large number of difficult samples too early, and also avoid learning stagnation caused by using only easy samples.

[0019] Fourth, this invention embeds the Mamba state-space modeling module into the diffusion denoising and reconstruction process. While retaining the ability of the diffusion model to progressively denoise and reconstruct high-quality pulse wave signals, it completes long-range time series modeling and spatiotemporal feature interaction with linear complexity. Compared with estimation methods that only use direct regression, this invention is more conducive to extracting physiological rhythm information from complex noise backgrounds; compared with high-complexity spatiotemporal interaction backbones, this invention reduces the computational burden while ensuring modeling capabilities. Attached Figure Description

[0020] Figure 1 This is a flowchart of the method of the present invention; Figure 2 This is the overall system framework diagram of the present invention. Detailed Implementation

[0021] A preferred embodiment of the present invention will now be described in detail with reference to the accompanying drawings.

[0022] like Figure 1 As shown, this invention provides a remote heart rate estimation method based on course learning, including the following steps: S1. Acquire facial video, extract multi-channel spatiotemporal features from each region of interest in the facial area, and construct a multi-scale spatiotemporal map; S2. Construct a physiological signal estimation network. The physiological signal estimation network takes a multi-scale spatiotemporal map, a noisy pulse wave, and the corresponding dynamic conditions as input. It can reconstruct the pulse wave signal through Mamba spatiotemporal modeling and diffusion denoising. Perform spectral analysis on the reconstructed pulse wave signal to obtain the heart rate estimation result. S3, calculates a physiological difficulty index score for each training sample, consisting of an objective physiological quality item and a model feedback difficulty item; S4. The training samples are sorted from easy to difficult according to the physiological difficulty index score, and the data of the sorted training samples are truncated according to the proportion of training rounds. S5, in each training round, perform boundary sampling based on the truncated training samples to construct a training subset; S6, use the training subset to train the physiological signal estimation network, update the feedback information according to the training loss, return to step S3 to recalculate the physiological difficulty index score, and form an iterative closed loop; S7. In the inference phase, the multi-scale spatiotemporal graph of the test video is input into the trained physiological signal estimation network, and the reconstructed pulse wave signal is obtained through the denoising process; the reconstructed pulse wave signal is subjected to spectral analysis to obtain the heart rate estimation result.

[0023] Figure 2 The overall system framework of this invention is illustrated. The input facial video is preprocessed to generate a multi-scale spatiotemporal map. The estimation network takes the multi-scale spatiotemporal map, noisy pulse wave signal, and their dynamic conditions as input, reconstructs the pulse wave signal through a combination of Mamba spatiotemporal modeling and diffusion denoising, and performs spectral analysis on the reconstructed pulse wave signal to obtain the heart rate estimation result. The physiological difficulty index scoring and course data selection module generates a unified physiological difficulty index score based on motion amplitude, frequency band signal-to-noise ratio, regional consistency, and model feedback difficulty. Training samples are sorted from easy to difficult, and boundary sample selection and exploration sample mixing are completed under course scheduling control. Feedback information generated during training is then used to update the physiological difficulty index scores in subsequent rounds, thus forming a closed loop of scoring, sorting, sampling, and feedback updates.

[0024] 1. Obtain facial videos from the training dataset, perform face detection, keypoint alignment, and region of interest (ROI) extraction, and divide the facial regions into... The dataset is divided into training and testing sets according to a preset subject segmentation method. The dataset is then divided into regions of interest (ROIs), and multi-channel spatiotemporal features are extracted from each ROI.

[0025] 2. Constructing a physiological signal estimation network. The network takes a multi-scale spatiotemporal map, a noisy pulse wave signal, and its dynamic conditions as input. It reconstructs the target pulse wave signal through Mamba spatiotemporal modeling and diffusion denoising, and obtains the final heart rate estimate based on the pulse wave signal. The training results and feedback information of this network are further used in subsequent physiological difficulty index scoring, course scheduling, and boundary sampling processes.

[0026] 3. Calculate the Physiological Difficulty Index (PDI) for each training sample in the training set. The score is composed of motion amplitude, frequency band signal-to-noise ratio, region consistency, and model feedback difficulty, and is used to uniformly characterize the extractability of physiological signals and the current learning difficulty of the sample.

[0027] The main channel with a high power ratio within the heart rate frequency band is selected from the multi-scale spatiotemporal map, and the motion amplitude index, frequency band signal-to-noise ratio index, and regional consistency index are calculated accordingly.

[0028] Amplitude of motion index Represented as: ; in, Indicates the first The training sample on the main channel The spatiotemporal graph signal value at each moment.

[0029] Motion amplitude index is used to characterize the degree of dispersion of signal changes between adjacent time points; when the value is large, it usually indicates that there are more obvious head movements, facial expression disturbances or unstable changes in local areas in the sample, and the corresponding sample is more difficult to learn.

[0030] Bandwidth signal-to-noise ratio Represented as: ; in, Indicates the main channel signal at frequency Power spectral density at that point This indicates the frequency range corresponding to the effective physiological components. The frequency band signal-to-noise ratio (SNR) is used to measure the proportion of effective physiological components in the overall spectrum; a higher value indicates that the pulse wave signal contained in the sample is clearer and the physiological signal is more extractable.

[0031] Regional Consistency Indicators Represented as: ; in, This represents a set of region pairs between several high-quality regions of interest. This indicates a correlation calculation. The regional consistency index is used to measure whether the physiological rhythms extracted from different regions of interest remain synchronized; a higher value indicates that the physiological changes among multiple regions are more consistent, and the sample quality is usually higher.

[0032] Indicates by the first The first training sample in the main channel The spatiotemporal plot signal values ​​of a region of interest at each time point are arranged in chronological order to form a time-series signal. Indicates by the first The first training sample in the main channel The spatiotemporal plot signal values ​​of each region of interest at various times are arranged in chronological order to form a time-series signal. , ; Indicates the first The training sample in the main channel is the nth training sample. The first region of interest The spatiotemporal graph signal value at each moment, Indicates the first The training sample in the main channel is the nth training sample. The first region of interest The spatiotemporal graph signal value at each moment.

[0033] The model feedback difficulty term consists of the smoothed loss and learning progress-related quantities of the sample during training, used to characterize the learning difficulty of the sample at the current training stage. To reduce the impact of single-round training fluctuations on the scoring results, the sample loss is updated exponentially, where... This represents the smoothing coefficient, used to balance the proportion of historical loss information to current loss information. The smoothing loss can be expressed as: .

[0034] After obtaining the above four types of components, the objective physiological components are normalized and oriented to construct the objective physiological quality term. The motion amplitude is converted in the direction of smaller value and higher quality, while the frequency band signal-to-noise ratio and regional consistency are converted in the direction of larger value and higher quality.

[0035] Simultaneously, a model feedback difficulty term is constructed based on smoothing loss and learning progress. It can be represented as: ; in, This represents the normalized smoothing loss. The difference in learning progress for the i-th training sample is obtained by normalizing the smooth loss decrease of the training sample in adjacent training rounds and taking the complementary value. The larger the value of , the less the smooth loss of the training sample decreases, and the less progress the model makes in learning from that training sample. and This represents the corresponding weight coefficient. The model feedback difficulty term reflects the model's current learning progress on the samples; when the smoothing loss is high or the learning progress is poor, this term's value increases accordingly.

[0036] The standardized physiological difficulty score is expressed as follows: ; in, Indicates the current course progress. This represents the physiological quality correction factor that changes with the progress of the course. This represents the physiological quality threshold. The resulting physiological difficulty score takes into account both the quality of the sample's physiological signals and the model's current learning status with respect to that sample. When the sample's objective physiological quality is low, its physiological difficulty score is increased accordingly, and it is introduced later in subsequent lessons. When the sample's physiological quality is high and the model has learned from it sufficiently, its physiological difficulty score is lower, and it can be prioritized for the current training round.

[0037] 4. Training samples are ranked from easiest to hardest based on physiological difficulty scores, with lower scores indicating easier learning. During the warm-up phase, the network is trained using all training samples. Objective physiological quality indicators are extracted, and feedback information such as sample smoothing loss, learning progress, and uncertainty is accumulated. This establishes the statistical basis for subsequent scoring without sample selection, preventing large fluctuations in feedback volume in the early training stages from prematurely affecting sample ranking and selection. After the warm-up phase, a complete four-component score is calculated based on objective physiological quality information and model feedback information to perform the initial ranking of training samples, which is continuously updated in subsequent training rounds.

[0038] 5. After the warm-up phase, for each training round, determine the data usage ratio for the current round based on the selected course scheduling strategy. The data usage proportion is used to control the sample range participating in course selection in the current round, and to determine what proportion of samples are retained from the sorted sample sequence to enter the candidate set in the current round. As the training rounds progress, The training process gradually increases the sample size, allowing the model to progress from prioritizing low-difficulty samples to learning more complex ones. For large datasets, a strategy of rapid initial expansion followed by slower expansion can be used to quickly establish stable physiological rhythm modeling capabilities in the early stages of training, gradually introducing complex samples later. For smaller datasets, a more stable linear strategy can be employed to avoid excessively rapid changes in sample coverage that could lead to increased training fluctuations. At the end of training, course data selection can be discontinued, and the entire training data can be used for full-scale fine-tuning to further improve the model's adaptability to different subjects and different data collection conditions.

[0039] 6. Based on the current data usage ratio Boundary sampling is performed on the directory-level sorted sequence. Boundary sampling does not involve random sampling from all samples; instead, it divides the current training subset into three parts based on the sorting results: easy-to-learn samples, boundary samples, and exploration samples. Easy-to-learn samples are those with low physiological difficulty scores and high objective physiological quality scores within the current cutoff range, used to maintain training stability in the current round. Boundary samples are selected from the adjacent samples with higher scores immediately following the easy-to-learn samples, and re-sorted according to directory-level difficulty score, directory-level objective quality score, learning progress, and uncertainty. Exploration samples are not additional data, but rather a small number of samples added from the unselected more difficult samples when the course learning progress reaches a preset threshold and the exploration mixing ratio is greater than zero, weighted from high to low based on the uncertainty and normalized smoothing loss. This maintains the training order from easy to difficult while supplementing samples near the course boundaries or with high learning uncertainty, balancing sample diversity and subsequent generalization ability. Exploration samples are mainly used to supplement less covered but potentially valuable sample types in the current round. This forms the current training subset.

[0040] 7. Train the network using a selected training subset. During training, the network prioritizes using easily learned samples to establish stable physiological rhythm representations in the current round, while gradually learning more complex sample patterns by combining boundary samples and exploration samples. After training is complete, update the smoothing loss, learning progress, and uncertainty of the samples used in training based on the loss results of the current round to reflect the model's latest learning state with the samples.

[0041] 8. Based on the updated feedback information, the physiological difficulty index score of the training samples is recalculated, and the training samples are re-ranked accordingly to serve as the basis for selecting course data in the next round. This re-scoring process allows the sample difficulty assessment to dynamically change with the model's learning state, thereby avoiding the long-term use of the same ranking result during training. By repeatedly executing the scoring update, sample selection, and network training processes, a course learning closed loop that dynamically evolves with each training round is formed.

[0042] 9. Repeat the training process until the preset total number of training rounds is reached. In the final stage of training, the data selection mechanism of the course learning can be disabled, and full fine-tuning can be performed using all training data to further improve the model's generalization ability under different subjects and different collection conditions. Full fine-tuning is used to allow the model to reuse all training samples for overall parameter correction after the course learning has been basically completed, thereby reducing the local bias caused by course selection.

[0043] 10. During the inference phase, the same data preprocessing procedure as in the training phase is performed on the test video. The multi-scale spatiotemporal graph is input into the trained network, and the pulse wave signal is reconstructed through a multi-step denoising process. Subsequently, spectral analysis is performed on the reconstructed signal, and the final heart rate estimate is obtained based on its main peak frequency. This enables a complete estimation process from facial video input to pulse wave reconstruction and then to heart rate output.

[0044] Example: This embodiment uses the VIPL-HR dataset as an example to illustrate the specific process of the method of the present invention. The dataset contains 2378 facial video samples from 107 subjects, covering real-world acquisition scenarios with varying lighting, movement, and distances, and can be used to demonstrate application scenarios where there are significant differences in the quality of physiological signals between different samples.

[0045] In this implementation, facial video samples from the training dataset are first acquired. Face detection and 68-point keypoint alignment are performed on each video, and the facial region is divided into 63 regions of interest (ROIs). Multi-channel spatiotemporal features are then extracted from each ROI to construct a 6-channel multi-scale spatiotemporal map representation, with input dimensions set to [6, 63, 300]. Data partitioning uses Fold 1 cross-validation, with videos from approximately 80 subjects serving as the training set and videos from approximately 27 subjects serving as the test set. To improve initial convergence efficiency, model weights pre-trained on the UBFC-rPPG dataset are used to initialize the network parameters.

[0046] In terms of network construction, a multi-scale spatiotemporal map, the current noisy pulse wave signal, and its dynamic conditions are jointly input into the physiological signal estimation network. As a preferred implementation, the network can adopt a configuration with a hidden feature dimension of 256, a network depth of 6, a head count of 4, a feedforward dimension of 256, a region aggregation count of 63, and a dropout rate of 0.1. During training, a multi-GPU parallel approach can be used, with the batch size set to 160 or 288 depending on available GPU memory. The optimizer uses Adam, with an initial learning rate of 0.001, which decays after a preset number of rounds. The loss function employs a dynamic decoupling loss composed of Pearson correlation coefficient loss, amplitude loss, and trend loss, where the balance coefficient can be set to 0.1. After training, the network outputs the reconstructed pulse wave signal, and spectral analysis is performed on the reconstructed pulse wave signal to obtain the heart rate estimation result. The dynamic conditions include amplitude and trend conditions calculated from the noisy pulse wave signal, as well as the corresponding diffusion time step encoding.

[0047] During the training phase, a physiological difficulty index score is calculated for each sample in the training set. This score is composed of motion amplitude, frequency band signal-to-noise ratio, region consistency, and model feedback difficulty. Motion amplitude, frequency band signal-to-noise ratio, and region consistency are calculated from the main channel of the multi-scale spatiotemporal graph, while the model feedback difficulty is extracted from the smoothing loss and learning progress-related quantities during training. An objective physiological quality term is constructed after uniformly normalizing and aligning the objective physiological components; a model feedback difficulty term is constructed after fusing the model feedback-related quantities; and a unified physiological difficulty index score is formed based on these. As a preferred implementation, the effective physiological component frequency range in the frequency band signal-to-noise ratio index is... The Hz values ​​can be set to [0.66, 3.0] Hz; the weighting coefficients for motion amplitude, frequency band signal-to-noise ratio, and regional consistency can be set to [values ​​to be specified]. Internal weighting coefficients of the model feedback difficulty term , The values ​​can be 0.70 and 0.30 respectively; physiological quality threshold. A value of 0.35 can be used; the course-related correction factor can be adopted. In this embodiment, the objective physiological quality item, the model feedback difficulty item, and the physiological difficulty index score can be represented as follows: ; ; ; in, , and They represent the first The normalized values ​​of motion amplitude, frequency band signal-to-noise ratio, and region consistency for each training sample; the smaller the motion amplitude, the better. The larger the value, the higher the bandwidth signal-to-noise ratio and regional consistency. and The larger; This represents the normalized smoothing loss. This indicates the degree of learning progress.

[0048] Regarding sample ranking, training samples are ranked from easiest to hardest based on physiological difficulty index scores, with lower scores indicating easier samples to learn. In this implementation, the first few training rounds after the start of training are designated as a warm-up phase, such as the first three training rounds. During the warm-up phase, the network is trained using all training samples, while objective physiological quality indicators are extracted and model feedback information is accumulated. Sample ranking and course selection are not performed during this phase. After the warm-up phase, a complete four-component score is calculated based on the objective physiological quality information and model feedback information, and the training samples are ranked for the first time.

[0049] Regarding course scheduling, the data usage ratio for the current round is dynamically determined based on the course scheduling strategy. For datasets with large sample sizes, an exponential course expansion strategy can be adopted to quickly expand the range of data available for training in the early stages and smoothly transition to more challenging samples in the later stages of training. For datasets with relatively small sample sizes, a more stable linear expansion strategy can be used. In the later stages of training, course data selection can be deactivated as needed, and full fine-tuning can be performed using all training data. Let... Use the proportion for the initial data. For the final data usage ratio, In this implementation method, to represent the current course progress, 0.50 is acceptable. A value of 1.0 can be used; the exponentially expanded course strategy can be represented as: .

[0050] Furthermore, let the total number of samples currently available for course selection be... The minimum retention ratio is As a preferred implementation, A value of 0.50 can be used. Therefore, the effective retention rate and the cutoff quantity for the current round can be expressed as: .

[0051] This indicates rounding down to the nearest integer.

[0052] After determining the data usage ratio for the current round, boundary sampling is performed on the directory-level sorted sequence. First, the block-level scores of each training segment under the same video directory are aggregated to obtain the directory-level difficulty score and the directory-level objective quality score. In this embodiment, if the same directory contains a total of For each training segment, the directory-level difficulty score and directory-level objective quality score can be expressed as: , ; in, Indicates the first The video directory number The physiological difficulty index score corresponding to each training sample Indicates the first The video directory number The objective physiological quality item corresponding to each training sample Indicates the first The number of training samples contained in each video directory; Based on this, within the cutoff range, samples with lower physiological difficulty scores and higher objective physiological quality scores are prioritized as the base of easy-to-learn samples. Then, a boundary candidate pool is constructed from the more difficult samples immediately following the easy-to-learn samples. Combining the directory-level difficulty score, directory-level objective quality score, learning progress, and uncertainty, boundary samples for the current round are selected from the boundary candidate pool. When the course learning progress reaches a preset threshold and the exploration mixing ratio is greater than zero, a small number of exploration samples, selected from high to low based on the weighted result of uncertainty and normalized smoothing loss, are added to the unselected more difficult samples. As a preferred implementation, the directory score can use a mean mode, and the boundary sampling can use a boundary sampling configuration; for datasets with large sample sizes, the boundary sample ratio can be adjusted to 0.10, 0.18, and 0.25 in segments according to the course progress. Let the proportion of boundary samples change with the course progress be represented. Then the number of boundary samples and the number of easy-to-learn samples can be expressed as: ; The current training subset can be represented as: ; in, This represents the current set of easy-to-learn samples. This represents the set of boundary samples for the current round. This represents the set of samples explored in the current round.

[0053] In each training round, the system trains the network using a subset of the current training set. After training, the loss values ​​of the trained samples are updated using an exponential moving average, and feedback information such as learning progress, learning progress difference, uncertainty, and sampling utility are updated simultaneously. Learning progress is calculated and smoothly updated based on the non-negative decrease in the smoothed loss between two consecutive iterations. The learning progress difference is the complement of the normalized learning progress; a larger value indicates insufficient learning progress. Uncertainty is calculated and smoothly updated based on the absolute value of the difference between the current loss and the smoothed loss. Sampling utility is calculated jointly by normalized learning progress, objective physiological quality, the closeness of the normalized smoothed loss to a preset moderate loss reference value, and uncertainty, and is used for ranking the boundary candidate pool. The weighted result of uncertainty and normalized smoothed loss is used for ranking the exploration samples. Among these, smoothed loss and learning progress difference are used to construct the model feedback difficulty term. The smoothed loss of the samples can be updated as follows: ; in, Indicates the first The training sample at the th ... Smooth loss over each training epoch, This indicates the first training round. The current loss value for each training sample. This represents the smoothing coefficient. As a preferred implementation, A value of 0.9 can be chosen. Then, based on the updated feedback information, the physiological difficulty index score of the training samples is recalculated, and the next round of sample re-ranking is completed, thus forming an iterative closed loop of "scoring—ranking—boundary sampling—training—feedback update". The above process of course scheduling, boundary sampling, network training, and feedback update is repeated until the preset total number of training rounds is reached. In the later stages of training, the course data selection mechanism can be disabled, and full fine-tuning can be performed using all training data to further improve the model's generalization ability. Parameters such as the exploration mixing ratio, exploration enabling progress threshold, total number of course training rounds, number of rounds to disable course selection, and warm-up rounds can all be set according to the experimental objectives and dataset size.

[0054] During the inference phase, the same face detection, keypoint alignment, region of interest segmentation, and multi-scale spatiotemporal graph construction processes as those in the training phase are performed on the test video. The generated multi-scale spatiotemporal graph is then input into the trained network. The network uses random Gaussian noise as initial input and gradually reconstructs a clean pulse wave signal through multi-step denoising updates. Spectral analysis is then performed on the pulse wave signal, and the final heart rate estimate is obtained based on the peak frequency.

[0055] When implementing this invention on datasets of different sizes and under different collection conditions, parameters such as batch size, course scheduling strategy, initial data ratio, exploratory mixing ratio, and the number of rounds for course deactivation can be adaptively set without changing the overall technical process. For large-scale real-world datasets like VIPL-HR, an exponential course expansion strategy is preferred; for datasets with relatively small sample sizes like PURE and UBFC-rPPG, a more stable linear expansion strategy is preferred, which can be expressed as: ; The catalog scoring can adopt the mean model, and the boundary sampling configuration and exploration sample ratio can be adjusted according to the data scale and training objectives. However, its implementation process still includes multi-scale spatiotemporal graph construction, physiological difficulty index scoring, course scheduling, boundary sampling, feedback updates, and heart rate output.

[0056] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the invention. The terms “comprising,” “including,” etc., as used herein indicate the presence of the stated features, steps, operations, and / or components, but do not exclude the presence or addition of one or more other features, steps, operations, or components.

[0057] It should be understood that although the steps in the flowcharts of the accompanying drawings are shown sequentially as indicated by the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some of the steps in the flowcharts of the accompanying drawings may include multiple steps or stages, which are not necessarily completed at the same time, but may be executed at different times, and the execution order of these steps or stages is not necessarily sequential, but may be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0058] This invention also provides a system. The system can be a system that uses software (applications), modules, components, servers, clients, etc., employing the methods described in the embodiments of this specification, combined with necessary hardware implementation. Since the implementation schemes and methods for solving the problem are similar, the specific system implementations in the embodiments of this specification can be found in the implementations of the foregoing methods, and repeated details will not be elaborated upon. Although the system is preferably implemented in software, hardware implementation, or a combination of software and hardware, is also possible and contemplated.

[0059] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0060] It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above, and that the invention can be implemented in other specific forms without departing from its spirit or essential characteristics. Therefore, the embodiments should be considered in all respects as exemplary and non-limiting, and the scope of the invention is defined by the appended claims rather than the foregoing description. Thus, all variations falling within the meaning and scope of equivalents of the claims are intended to be included within the present invention, and no reference numerals in the claims should be construed as limiting the scope of the claims.

[0061] Furthermore, it should be understood that although this specification describes embodiments, not every embodiment contains only one independent technical solution. This narrative style is merely for clarity. Those skilled in the art should consider the specification as a whole, and the technical solutions in each embodiment can also be appropriately combined to form other embodiments that can be understood by those skilled in the art.

Claims

1. A remote heart rate estimation method based on course learning, characterized in that, Includes the following steps: S1. Acquire facial video, extract multi-channel spatiotemporal features from each region of interest in the facial area, and construct a multi-scale spatiotemporal map; S2, construct a physiological signal estimation network. The physiological signal estimation network takes a multi-scale spatiotemporal map, a noisy pulse wave and the corresponding dynamic conditions as inputs, and reconstructs the pulse wave signal through Mamba spatiotemporal modeling and diffusion denoising. Spectral analysis was performed on the reconstructed pulse wave signal to obtain the heart rate estimation result; S3, calculates a physiological difficulty index score for each training sample, consisting of an objective physiological quality item and a model feedback difficulty item; S4. The training samples are sorted according to the physiological difficulty index score, and the data of the sorted training samples are truncated according to the proportion of training rounds. S5, in each training round, perform boundary sampling based on the truncated training samples to construct a training subset; S6: Train the physiological signal estimation network using a training subset, update the feedback information based on the training loss, and return to S3 to recalculate the physiological difficulty index score; the feedback information includes smoothing loss, uncertainty, learning progress, and learning progress difference. S7, repeat S3 to S6 until the physiological signal estimation network converges or reaches the maximum number of training rounds.

2. The remote heart rate estimation method based on course learning according to claim 1, characterized in that, The objective physiological quality items include motion amplitude index, frequency band signal-to-noise ratio index, and regional consistency index; Amplitude of motion index for: ； Indicates the first The training sample in the main channel is the nth training sample. The first region of interest The spatiotemporal graph signal value at each moment, Indicates the first Index of regions of interest, Indicates the first Index of each moment; Bandwidth signal-to-noise ratio for: ； Indicates the first The spatiotemporal map signal of the main channel of the multi-scale spatiotemporal map of the training samples at the frequency Power spectral density at that point Indicates the frequency range corresponding to the effective physiological components; Regional Consistency Indicators for: ； Indicates the first The set of region pairs between regions of interest of each training sample. This indicates a correlation calculation; Indicates by the first The first training sample in the main channel The spatiotemporal plot signal values ​​of a region of interest at each time point are arranged in chronological order to form a time-series signal. Indicates by the first The first training sample in the main channel The spatiotemporal plot signal values ​​of a region of interest at each time point are arranged in chronological order to form a time-series signal. and These represent the first and second parts of the region pair, respectively. The region of interest and the first Index of regions of interest, Indicates timing length; Objective physiological quality items Represented as: ； , and They represent the first The normalized values ​​of motion amplitude index, frequency band signal-to-noise ratio index, and region consistency index corresponding to each training sample are: , , These are the corresponding weight coefficients. This indicates that the input value will be truncated to... A function on an interval.

3. The remote heart rate estimation method based on course learning according to claim 1, characterized in that, Model feedback difficulty item The calculation method is as follows: ； Indicates the first The normalized smooth loss for each training sample. This represents the difference in learning progress, which is the complement of normalized learning progress. and This represents the weighting coefficient.

4. The remote heart rate estimation method based on course learning according to claim 1, characterized in that, Physiological Difficulty Index Score for: ； in, For objective physiological quality items, This is the difficulty level of the model feedback. Indicates the current learning progress of the course. This is a physiological quality correction factor that varies with the progress of course learning. Physiological quality threshold This indicates that the input value will be truncated to... A function on an interval.

5. The remote heart rate estimation method based on course learning according to claim 1, characterized in that, During the warm-up phase of training, the physiological signal estimation network is trained using all training samples. An objective physiological quality term is calculated based on the multi-scale spatiotemporal graph of each training sample. This objective physiological quality term is obtained solely from the training samples themselves and does not depend on the training feedback of the physiological signal estimation network. Simultaneously, a smoothing loss is obtained based on the loss of each training sample, and the learning progress, learning progress difference, and uncertainty are further calculated to form feedback information and update the model feedback difficulty term. After the warm-up phase, a physiological difficulty index score is calculated based on the objective physiological quality term and the model feedback difficulty term, and the training samples are sorted and courses are selected.

6. The remote heart rate estimation method based on course learning according to claim 1, characterized in that, The process of truncating data from the sorted training samples according to a set ratio during each training round specifically includes: Data usage ratio Adopting an exponential expansion strategy: ； Alternatively, a linear expansion strategy can be adopted: ； in, Use the proportion for the initial data. For the final data usage ratio, This represents the current course learning progress; the data usage ratio applies to the course learning phase after the warm-up phase of training, where all training samples are used for training.

7. The remote heart rate estimation method based on course learning according to claim 1, characterized in that, The step of performing boundary sampling based on the truncated training samples to construct a training subset specifically includes: The physiological difficulty index scores and objective physiological quality items of each training sample in the same video directory are aggregated to obtain a directory-level difficulty score. and directory-level objective quality score : ， ； in, Indicates the first The video directory number The physiological difficulty index score corresponding to each training sample Indicates the first The video directory number The objective physiological quality item corresponding to each training sample Indicates the first The number of training samples contained in each video directory; Training samples are sorted from low to high according to their physiological difficulty index scores. Within the candidate range determined by the current data usage ratio, training samples with objective physiological quality items not lower than a set threshold are selected, and a set number of training samples are selected as easy-to-learn samples according to the sorting. When the number of training samples meeting the set threshold is insufficient, the remaining training samples within the candidate range are supplemented according to the physiological difficulty index scores from low to high. A boundary candidate pool is constructed from training samples that have not been selected as easy-to-learn samples and whose sorting position is close to the current course boundary. Boundary samples are selected from the boundary candidate pool by combining the directory-level difficulty score, directory-level objective quality score, learning progress, and uncertainty. When the preset exploration conditions are met, training samples with objective physiological quality items not lower than the exploration quality threshold are selected from the remaining training samples that have not been selected. A set number of samples are selected as exploration samples according to the weighted result of uncertainty and normalized smoothing loss from high to low. The easy-to-learn samples, boundary samples, and exploration samples are combined to form the training subset of the current training round.

8. The remote heart rate estimation method based on course learning according to claim 1, characterized in that, The step of updating the feedback information based on the training loss specifically includes: The loss values ​​of the training samples that have already participated in training are updated using an exponential moving average. The training sample at the th ... Smooth loss over each training epoch Represented as: ； in, This indicates the first training round. The current loss value for each training sample. For smoothing coefficients; The learning progress is calculated and updated smoothly based on the non-negative decrease in the smoothing loss between two consecutive smoothing losses; The learning progress difference is the complementary quantity of normalized learning progress; Uncertainty is calculated based on the absolute value of the difference between the current loss and the smoothed loss, and then updated smoothly.

9. The remote heart rate estimation method based on course learning according to claim 1, characterized in that, The dynamic conditions include: amplitude conditions and trend conditions calculated from the noisy pulse wave, and the corresponding diffusion time step encoding.

10. A remote heart rate estimation system based on course learning, used to implement the method of any one of claims 1 to 9, characterized in that, include: The multi-scale spatiotemporal map preprocessing module acquires facial videos, extracts multi-channel spatiotemporal features from each region of interest in the facial region, and constructs a multi-scale spatiotemporal map. The physiological signal estimation network takes multi-scale spatiotemporal maps, noisy pulse waves and corresponding dynamic conditions as inputs, and reconstructs pulse wave signals through Mamba spatiotemporal modeling and diffusion denoising. The heart rate estimation unit performs spectral analysis on the reconstructed pulse wave signal to obtain the heart rate estimation result; The physiological difficulty index scoring module calculates a physiological difficulty index score for each training sample, consisting of objective physiological quality items and model feedback difficulty items. The course learning scheduling module sorts the training samples according to the physiological difficulty index score, and extracts data from the sorted training samples according to the usage ratio in each training round. The boundary sampling module performs boundary sampling based on the truncated training samples in each training round to construct a training subset; The training execution module trains the physiological signal estimation network using a training subset, updates the feedback information based on the training loss, and returns to the physiological difficulty index scoring module to recalculate the physiological difficulty index score. The feedback information includes smoothing loss, uncertainty, learning progress, and learning progress difference. Repeatedly execute the physiological difficulty index scoring module, the course learning scheduling module, the boundary sampling module, and the training execution module until the physiological signal estimation network converges or reaches the maximum number of training rounds.

Images