A heart rate detection method and system based on feature orthogonal constraint and RGB time sequence reconstruction

Abstract

The application discloses a heart rate detection method and system based on feature orthogonal constraint and RGB time sequence reconstruction. The heart rate detection method carries out a region of interest processing on an original training video segment containing a sample object face region, and generates a target RGB time sequence signal. A pulse branch and a noise branch which are independent of each other are used to extract pulse-related high-level features and noise-related high-level features, respectively, and a corresponding time regression head is used to output a predicted pulse signal and a non-physiological noise signal. The RGB time sequence signal is reconstructed based on the two types of signals, and a pulse supervision loss, a maximum cross-correlation loss, a reconstruction loss, an orthogonal loss and a frequency domain separation loss are combined to train the model. During detection, only the preprocessed video segment of the measured object is input into the trained pulse branch and the corresponding time regression head to obtain the predicted pulse signal, and the heart rate of the measured object is calculated through frequency domain main peak search or time domain peak value detection. The application can reduce the interference of motion, illumination change and imaging noise on pulse representation, and improve the stability of non-contact heart rate detection.

Description

Technical Field

[0001] This invention belongs to the fields of computer vision, biomedical signal processing and non-contact vital sign monitoring technology, specifically involving a heart rate detection method and system based on feature orthogonal constraints and RGB temporal reconstruction. Background Technology

[0002] rPPG technology can use ordinary camera equipment to capture videos of exposed skin areas of the human body and reconstruct pulse signals related to changes in blood volume based on subtle color changes in the video. Compared with contact-based physiological signal measurement methods, rPPG technology has the advantages of being non-contact and easy to deploy.

[0003] The skin color changes caused by heartbeats are relatively small and easily affected by head movements, facial expressions, posture changes, ambient light fluctuations, local shadows, camera auto-exposure, video compression, and sensor noise. These non-physiological factors cause the physiological pulse component to mix with the non-physiological noise component in the RGB observation domain, resulting in peak-to-valley shifts, local abnormal oscillations, or increased spectral noise in the recovered rPPG waveform.

[0004] Existing deep learning methods typically establish an end-to-end mapping from input video to pulse waveforms or heart rate, with the training objective primarily constraining the consistency between the predicted results and the reference pulse signal. These methods often lack explicit modeling of physiological pulse components and non-physiological noise components, potentially allowing non-physiological noise to enter the pulse representation. Furthermore, contact-based reference pulse signals can be affected by differences in acquisition location, contact instability, or sampling delays. Therefore, a technical solution is needed that can separate the two types of components and utilize video observation information to form closed-loop supervision. Summary of the Invention

[0005] The purpose of this invention is to provide a heart rate detection method and system based on feature orthogonal constraints and RGB time-series reconstruction.

[0006] In a first aspect, the present invention provides a heart rate detection method based on feature orthogonal constraints and RGB temporal reconstruction, the method comprising:

[0007] The original training video clip containing the facial region of the sample object is acquired using an RGB camera device; the original training video clip is processed to obtain a region of interest (ROI) for the face, and the target RGB time-series signal is obtained based on the temporal changes of each RGB channel in the ROI of the face;

[0008] A dual-branch network consisting of a pulse branch and a noise branch is constructed. Training video clips are input into the pulse branch and the noise branch in parallel to extract pulse-related high-level features and noise-related high-level features, respectively. The pulse-related high-level features and noise-related high-level features are processed by a time regression head to obtain the predicted pulse signal and non-physiological noise signal.

[0009] Pulse-driven RGB components and noise-driven RGB components are constructed based on the predicted pulse signal and non-physiological noise signal, respectively. The pulse-driven RGB components and noise-driven RGB components are superimposed to obtain the reconstructed RGB time-series signal. The reconstruction loss is constructed based on the consistency between the target RGB time-series signal and the reconstructed RGB time-series signal.

[0010] An orthogonal loss is constructed based on the absolute value of the cosine similarity between the feature vectors corresponding to pulse-related high-level features and noise-related high-level features.

[0011] Construct a total loss that includes reconstruction loss and orthogonality loss, and update the learnable parameters of the pulse branch, noise branch, and time regression head based on the total loss;

[0012] The facial video of the subject is acquired by an RGB camera device, and the facial video is processed by the region of interest to obtain a preprocessed video segment. The preprocessed video segment is input into a trained pulse branch and time regression head to output a predicted pulse signal. The heart rate of the subject is determined based on the predicted pulse signal.

[0013] Preferably, the orthogonal loss is constructed as follows:

[0014] Global average pooling is performed on the high-level features related to pulse and noise along the spatial dimension to obtain pulse-related pooling features and noise-related pooling features. The channel dimension and time dimension of the pulse-related pooling features and noise-related pooling features are expanded to obtain the pulse feature vector and noise feature vector corresponding to each training sample. After aligning the pulse feature vector and noise feature vector corresponding to each training sample, the absolute value of the cosine similarity between the two is calculated, and the average value of the absolute value of the cosine similarity corresponding to all training samples is used as the orthogonal loss.

[0015] Preferably, the method for constructing the pulse-driven RGB components and the noise-driven RGB components is as follows:

[0016] Learnable amplitude coefficients and pulse projection vectors corresponding to the predicted pulse signal, and learnable amplitude coefficients and noise projection vectors corresponding to the non-physiological noise signal are set respectively; the pulse projection vector and noise projection vector are both learnable parameters; the predicted pulse signal is multiplied by its corresponding learnable amplitude coefficients and pulse projection vectors to obtain the pulse-driven RGB component; the non-physiological noise signal is multiplied by its corresponding learnable amplitude coefficients and noise projection vectors to obtain the noise-driven RGB component.

[0017] Preferably, a channel relative weight order constraint is applied to the three channel components of the pulse projection vector; the channel relative weight order constraint is that the green channel component is greater than the blue channel component, and the blue channel component is greater than the red channel component.

[0018] Preferably, the total loss also includes a pulse supervision loss; the pulse supervision loss is constructed based on the Pearson correlation coefficient between the predicted pulse signal and the reference pulse signal; the reference pulse signal is a photoplethysmography pulse waveform or a blood volume pulse waveform acquired synchronously with the original training video clip.

[0019] Preferably, the total loss also includes the maximum cross-correlation loss; the method for constructing the maximum cross-correlation loss is as follows:

[0020] Within a preset time offset range, the reference pulse signal is relatively time-shifted; the normalized cross-correlation value between the predicted pulse signal and the reference pulse signal corresponding to each time offset is calculated; the maximum normalized cross-correlation value is determined from each normalized cross-correlation value, and the maximum cross-correlation loss is determined based on the maximum normalized cross-correlation value.

[0021] Preferably, the total loss also includes frequency domain separation loss; the method for constructing the frequency domain separation loss is as follows:

[0022] The predicted pulse signal and non-physiological noise signal are subjected to mean removal and one-sided fast Fourier transform respectively. The power spectral density of the predicted pulse signal and non-physiological noise signal are extracted in the preset physiological frequency band and normalized to obtain the pulse power spectrum and noise power spectrum. The average value of the squared difference between the pulse power spectrum and the noise power spectrum at each frequency point is obtained to obtain the spectral difference degree. If the spectral difference degree is less than the preset interval threshold, the frequency domain separation loss is the difference between the preset interval threshold and the spectral difference degree; otherwise, the frequency domain separation loss is zero.

[0023] Preferably, the pulse branch includes seven convolutional blocks connected in sequence; the noise branch includes five convolutional blocks connected in sequence; each convolutional block includes a convolutional layer, a batch normalization layer, and a nonlinear activation layer connected in sequence.

[0024] Preferably, the facial region of interest processing method is as follows: perform facial detection, facial tracking or key point localization on the original training video clip or the facial video of the test subject to determine at least one region among the whole face skin region, forehead region, cheek region or multiple facial partitions, and perform image cropping according to the determined region.

[0025] Secondly, the present invention provides a heart rate detection system based on feature orthogonal constraints and RGB time-series reconstruction, which is used to execute the above-mentioned heart rate detection method; the heart rate detection system includes an RGB camera device, a processor and a memory, wherein the processor is configured with a data acquisition module, a region of interest processing module, an RGB time-series signal generation module, a dual-branch processing module, a loss construction module, a parameter update module and a vital signs output module;

[0026] The data acquisition module is used to acquire original training video clips containing the facial regions of the sample objects and facial videos of the test objects through RGB camera equipment, and to acquire reference pulse signals synchronously acquired with the original training video clips; the region of interest processing module is used to perform facial region of interest processing on the original training video clips to obtain training video clips, and to perform facial region of interest processing on the facial videos of the test objects to obtain preprocessed video clips; the RGB timing signal generation module is used to obtain target RGB timing signals based on the timing changes of each RGB channel in the facial region of interest; the dual-branch processing module includes parameter phase... Independent pulse and noise branches, along with a time regression head, are used to output predicted pulse signals and non-physiological noise signals. The loss construction module is used to construct a total loss based on the predicted pulse signal, non-physiological noise signal, target RGB time-series signal, pulse-related high-level features, and noise-related high-level features. The parameter update module is used to update the learnable parameters used in the dual-branch network, time regression head, and RGB time-series reconstruction based on the total loss. The vital signs output module is used to input preprocessed video clips into the trained pulse branch and time regression head to obtain the predicted pulse signal of the tested object and determine the heart rate of the tested object based on the predicted pulse signal.

[0027] Thirdly, the present invention provides a computer device including a memory, a processor, and a computer program stored in the memory and capable of running on the processor; the processor executes the computer program to implement the aforementioned heart rate detection method.

[0028] Fourthly, the present invention provides a computer-readable storage medium storing a computer program; when the computer program is executed by a processor, it implements the aforementioned heart rate detection method.

[0029] The beneficial effects of this invention are:

[0030] 1. This invention uses pulse branches and noise branches with independent parameters to represent physiological pulse components and non-physiological noise components respectively, and uses orthogonal constraints to reduce the linear correlation and representation redundancy between the two types of high-level features, which can reduce the interference of non-physiological factors such as movement, facial expression changes and illumination changes on pulse representation.

[0031] 2. This invention utilizes the predicted pulse signal and non-physiological noise signal to jointly reconstruct the RGB temporal changes of the facial region of interest, enabling the model to obtain closed-loop supervision from the video observation domain. At the same time, this invention combines maximum cross-correlation constraints and frequency domain separation constraints, which can reduce the influence of the reference pulse signal time offset and the absorption of physiological main frequency components by noise branches, thereby improving the stability of pulse waveform recovery and heart rate detection in complex scenarios.

[0032] 3. This invention can achieve non-contact heart rate detection using ordinary RGB camera equipment. During the detection phase, only a trained pulse branch and time regression head need to be run. It can be deployed on mobile terminals, edge computing devices, servers or monitoring equipment, and has good device compatibility and ease of application. Attached Figure Description

[0033] Figure 1 This is a flowchart illustrating the training process of the heart rate detection model provided in an embodiment of the present invention.

[0034] Figure 2 This is a schematic diagram of a dual-branch network with mutually independent parameters and its output relationship, provided for an embodiment of the present invention.

[0035] Figure 3 This is a schematic diagram of orthogonal constraints between pulse-related features and noise-related features provided in an embodiment of the present invention.

[0036] Figure 4 This is a schematic diagram of the RGB observation domain reconstruction process provided in an embodiment of the present invention.

[0037] Figure 5 This is a schematic diagram illustrating the relationship between the model training phase and the heart rate detection phase provided in an embodiment of the present invention.

[0038] Figure 6 This is a schematic diagram of the heart rate detection system provided in an embodiment of the present invention. Detailed Implementation

[0039] The present invention will be further described below with reference to the accompanying drawings and embodiments. The embodiments are used to illustrate the technical solutions of the present invention and are not intended to limit the scope of protection of the present invention to a single network structure, parameter settings, or device form. Where there is no conflict, the technical features of the various embodiments can be combined with each other.

[0040] Example 1

[0041] A heart rate detection method based on feature orthogonal constraints and RGB temporal reconstruction includes the following steps:

[0042] Step 1: Training data and target RGB time series signal

[0043] like Figure 1 As shown, a dataset is constructed by acquiring the original training video segment V and its corresponding reference pulse signal s using an RGB camera device. The original training video segment V contains the facial region of the sample object. The reference pulse signal s is preferably a PPG waveform (photoplethysmography waveform) or a BVP waveform (blood volume pulse waveform) acquired synchronously with the original training video segment V. The reference pulse signal s is interpolated and resampled according to the frame timestamp of the original training video segment V, so that the reference pulse signal s corresponds one-to-one with the video segment in the time dimension.

[0044] Face detection, face tracking, or keypoint localization are performed on the original training video segment V to determine the entire face skin region, forehead region, cheek region, or multiple facial partitions, thus obtaining the region of interest (ROI). Based on the ROI, cropping, size standardization, and pixel normalization are performed to construct a video tensor X of size B×3×T×H×W; where B is the number of mini-batch samples; 3 represents the number of RGB channels; T is the time length; and H and W are the spatial height and spatial width, respectively. The spatial mean of each RGB channel is calculated frame-by-frame for the ROI to obtain a three-channel time series; the temporal mean of each channel is removed, and channel normalization is performed to obtain the target RGB time series signal. Time-mean removal is used to reduce the impact of static skin tone baseline, while channel normalization is used to reduce amplitude differences between different objects and acquisition devices.

[0045] The dataset was divided into training, validation, and test sets in a 6:2:2 ratio, with the mini-batch size B set to 4 and the number of training epochs set to 10. The validation set was used to monitor the model training status, and the model parameters were saved according to the preset training termination conditions.

[0046] Step 2: Construct a dual-branch network with independent parameters.

[0047] like Figure 2 As shown, the dual-branch network comprises a pulse branch and a noise branch, both configured in parallel and with independent parameters. The pulse branch extracts low-amplitude, periodic local color changes; the noise branch extracts non-physiological temporal changes caused by motion, facial expression changes, lighting changes, shadow changes, automatic exposure, video compression, or sensor noise. The noise branch synchronously outputs non-physiological noise signals to characterize the temporal components that the model assigns to the non-physiological noise path from the input video.

[0048] The video clip X is input in parallel into the pulse branch and the noise branch. The two branches do not share trainable parameters, thus obtaining high-level pulse-related features. High-rise characteristics related to noise It is represented as follows:

[0049]

[0050]

[0051] in, This indicates the processing of pulse branches; This indicates the processing procedure for noise branches.

[0052] The pulse branch consists of seven convolutional blocks connected in sequence; the noise branch consists of five convolutional blocks connected in sequence. Each convolutional block consists of a three-dimensional convolutional layer, a batch normalization layer, and a ReLU nonlinear activation layer connected in sequence.

[0053] In some embodiments, the three-dimensional convolutional layer in the convolutional block can be replaced by a spatiotemporal Transformer, a temporal convolutional network, a recurrent neural network, or a hybrid spatiotemporal encoder.

[0054] High-level features related to pulse were analyzed using time regression heads. High-rise characteristics related to noise The signal is processed to obtain the predicted pulse signal. Non-physiological noise signals The time regression heads are positioned at the ends of the pulse and noise branches, respectively. Each time regression head includes three sets of spatiotemporal convolutional processing blocks connected in sequence. Each set of spatiotemporal convolutional processing blocks includes a 3D convolutional layer, a batch normalization layer, and a ReLU nonlinear activation layer. At the end, the processing results are aggregated in spatial dimensions and compressed in channel dimensions. Dimensional smoothing or temporal dimension restoration is then used to make the output time equal to the input video window time. Correspondingly, the predicted pulse signals were obtained respectively. Non-physiological noise signals Both are the same size The kernel size, number of channels, stride, and time dimension recovery method can be set according to the input video window and the deployment device.

[0055] Step 3: Construct joint loss and optimize the model

[0056] The dual-branch network and temporal regression head were optimized using the dataset, and the total loss was constructed. Guide the updating of parameters in the dual-branch network and the time regression head. Total loss. Represented as:

[0057]

[0058] in, For pulse monitoring loss The weights are used to constrain the consistency between the predicted pulse signal and the reference pulse signal; For maximum cross-correlation loss The weights are used to reduce the impact of the time offset between the predicted signal and the reference signal; RGB reconstruction loss The weights are used to constrain the consistency between the reconstructed RGB timing signal and the target RGB timing signal; For characteristic orthogonal loss The weights are used to reduce redundant coupling between pulse features and noise features; Frequency domain separation loss The weights are used to enhance the separability of pulse components from non-physiological noise components in the frequency domain. to All are non-negative weight coefficients, used to balance the contributions of different training constraints to the joint optimization objective.

[0059] The methods for constructing each loss in the total loss are as follows:

[0060] (1) Loss of pulse monitoring

[0061] Using predictive pulse signals The Pearson correlation coefficient between the pulse signal and the reference pulse signal s is used to construct the pulse supervision loss. It is represented as follows:

[0062]

[0063] in, This represents the Pearson correlation coefficient.

[0064] (2) Maximum cross-correlation loss

[0065] The maximum cross-correlation (MCC) loss is calculated within the range of [-Δ, Δ] to account for a small time offset between the facial rPPG waveform and the contact reference pulse signal. Normalized cross-correlation with reference pulse signal s And based on normalized cross-correlation Construct the maximum cross-correlation loss It is represented as follows:

[0066]

[0067] .

[0068] in, This is the time offset; The maximum time offset allowed for the search; This is the normalized cross-correlation function.

[0069] (3) Reconstruction loss

[0070] like Figure 4 As shown, based on the predicted pulse signal Non-physiological noise signals Constructing pulse-driven RGB components and noise-driven RGB components And combined with pulse-driven RGB components and noise-driven RGB components Constructing and reconstructing RGB timing signals It is represented as follows:

[0071]

[0072]

[0073]

[0074] in, These are learnable amplitude coefficients; and These are the pulse projection vector and the noise projection vector, respectively. The pulse projection vector and noise projection vector are learnable parameters.

[0075] Based on the target RGB timing signal With reconstructed RGB timing signals The Pearson correlation coefficient between them is used to construct the reconstructed loss. It is represented as follows:

[0076]

[0077] In some embodiments, the pulse projection vector The relative weights of the three channel components are subject to an order constraint of G>B>R, where G, B, and R represent the green, blue, and red channel components, respectively. This order constraint applies only to the pulse projection vector. Noise projection vector To maintain learnability, a fixed relative weight order for channels is not set, in order to adapt to non-physiological noise caused by changes in objects, scenes, and acquisition times.

[0078] (4) Orthogonal loss

[0079] like Figure 3 As shown, orthogonal loss This is based on constructing orthogonal constraints on high-level features; orthogonal constraints are used to decouple features by reducing the linear correlation between two types of high-level features. These constraints are applied to pulse-related high-level features. High-rise characteristics related to noise Perform global average pooling to obtain pulse correlation pooling features. Noise-related pooling features It is represented as follows:

[0080]

[0081]

[0082] in, and These represent the spatial height and spatial width of pulse-related high-level features, respectively. and These represent the spatial height and spatial width of noise-related high-rise features, respectively.

[0083] Pulse-related pooling features were respectively Noise-related pooling features The channel dimension and time dimension are expanded to obtain the pulse feature vector and noise feature vector corresponding to multiple samples.

[0084] After making the pulse feature vector and noise feature vector have the same dimension through linear projection or dimension alignment operations, an orthogonal loss is constructed based on the aligned pulse feature vector and noise feature vector. Its expression is:

[0085]

[0086] in, Let be the pulse feature vector corresponding to the b-th sample; Let be the noise feature vector corresponding to the b-th sample.

[0087] Orthogonal loss, by taking the absolute value of cosine similarity, can simultaneously penalize both positive and negative correlations between two types of features, thereby reducing the linear redundancy of the two branches in the high-level representation space.

[0088] (5) Frequency domain separation loss

[0089] Predicted pulse signals Non-physiological noise signals Mean reduction and one-sided fast Fourier transform were performed to extract the power spectral density within the preset physiological frequency band from 0.6 Hz to 4.0 Hz and then normalized to obtain the normalized pulse power spectrum. and noise power spectrum According to the pulse power spectrum and noise power spectrum Obtain spectral difference And based on spectral differences Constructing frequency domain separation loss It is represented as follows:

[0090]

[0091]

[0092] in, The number of frequency points within the preset physiological frequency band; The preset interval threshold is 0.5 in this embodiment.

[0093] Step 4: Heart Rate Monitoring

[0094] like Figure 5 As shown, facial videos of the test subject are acquired. After face detection, region of interest cropping, size standardization, time window segmentation, and pixel normalization, a preprocessed video segment is obtained. During the detection phase, only the trained pulse branch and its corresponding pulse time regression head are run to output the predicted pulse signal. Heart rate estimation is performed on the predicted pulse signal: when using the frequency domain method, the dominant frequency corresponding to the maximum power spectrum value is searched within a preset physiological frequency band. and in accordance with Calculate heart rate; when using a time-domain method, detect the effective peak position in the predicted pulse signal. , … and in accordance with Calculate heart rate, where The sampling rate.

[0095] Example 2

[0096] like Figure 6 As shown, a heart rate detection system based on feature orthogonal constraints and RGB time-series reconstruction includes an RGB camera device, a processor, and a memory; the processor is equipped with a data acquisition module, a region of interest processing module, an RGB time-series signal generation module, a dual-branch processing module, a loss construction module, a parameter update module, and a vital signs output module.

[0097] The RGB camera device is used to acquire raw training video clips containing the facial region of the sample subject and facial videos of the test subject; the memory is used to store the computer program, training video clips, reference pulse signal, model parameters, and intermediate data generated during processing; the processor is connected to both the RGB camera device and the memory to execute the computer program to realize the functions of each module. The reference pulse signal can be synchronously acquired by a contact-type photoplethysmography (PPG) sensor that communicates with the processor, or it can be imported from an external acquisition device through a data interface.

[0098] During model training, the data acquisition module acquires raw training video clips via RGB camera equipment or video input interface, and simultaneously acquires reference pulse signals. The region of interest (ROI) processing module performs face detection, face tracking, region cropping, size unification, and pixel normalization on the processor to obtain training video clips. The RGB temporal signal generation module calculates the spatial mean of each RGB channel in the ROI frame by frame and performs temporal normalization to obtain the target RGB temporal signal. The dual-branch processing module consists of a pulse branch and a noise branch running on the processor, along with corresponding pulse time regression heads and noise time regression heads, used to output predicted pulse signals and non-physiological noise signals. The loss construction module constructs a total loss based on the predicted pulse signal, non-physiological noise signal, target RGB temporal signal, reference pulse signal, pulse-related high-level features, and noise-related high-level features, including pulse supervision loss, maximum cross-correlation loss, RGB reconstruction loss, feature orthogonality loss, and frequency domain separation loss. The parameter update module updates the parameters of the dual-branch network, time regression head, learnable amplitude coefficients, and projection vectors based on the total loss, and stores the updated model parameters in memory.

[0099] During heart rate detection, the vital signs output module performs frequency domain peak search or time domain peak detection on the predicted pulse signal, and determines and outputs the heart rate of the tested object according to the preset heart rate calculation method; when the signal quality of one detection method does not meet the conditions, another detection method can be used to supplement it.

[0100] The processor and memory can be integrated into a mobile terminal, personal computer, edge computing device, server, or monitoring device; model training and heart rate detection can be performed on the same device, or the model training can be completed by the server, and the trained pulse branch and time regression head can be deployed to the mobile terminal, edge computing device, or monitoring device to perform heart rate detection.

[0101] Example 3

[0102] A computer device includes a processor, a memory, and a computer program stored in the memory and capable of running on the processor. The memory also stores pulse branches, noise branches, a time regression head, and learnable parameters used for RGB time-series reconstruction; when the processor executes the computer program, it implements the aforementioned heart rate detection method based on feature orthogonal constraints and RGB time-series reconstruction.

[0103] Example 4

[0104] A computer-readable storage medium storing a computer program; when the computer program is read and executed by a processor, it implements the above-mentioned heart rate detection method based on feature orthogonal constraints and RGB temporal reconstruction.

[0105] The heart rate detection method and system provided by this invention collect user data only when it is legal, compliant, and with the user's consent.

[0106] All user-related data (including but not limited to facial videos) is collected with user authorization and de-identified, retaining only the minimum scope required to achieve the processing purpose.

[0107] This invention does not require facial features, voiceprints, fingerprints or other biometric information as necessary input.

Claims

1. A heart rate detection method based on feature orthogonal constraints and RGB temporal reconstruction, characterized in that: The method includes: The original training video clip containing the facial region of the sample object is acquired using an RGB camera device; the original training video clip is processed to obtain a region of interest (ROI) for the face, and the target RGB time-series signal is obtained based on the temporal changes of each RGB channel in the ROI of the face; A dual-branch network consisting of a pulse branch and a noise branch is constructed. Training video clips are input into the pulse branch and the noise branch in parallel to extract pulse-related high-level features and noise-related high-level features, respectively. The pulse-related high-level features and noise-related high-level features are processed by a time regression head to obtain the predicted pulse signal and non-physiological noise signal. Pulse-driven RGB components and noise-driven RGB components are constructed based on the predicted pulse signal and non-physiological noise signal, respectively. The pulse-driven RGB components and noise-driven RGB components are superimposed to obtain the reconstructed RGB time-series signal. The reconstruction loss is constructed based on the consistency between the target RGB time-series signal and the reconstructed RGB time-series signal. An orthogonal loss is constructed based on the absolute value of the cosine similarity between the feature vectors corresponding to pulse-related high-level features and noise-related high-level features. Construct a total loss that includes reconstruction loss and orthogonality loss, and update the learnable parameters of the pulse branch, noise branch, and time regression head based on the total loss; The facial video of the subject is acquired by an RGB camera device, and the facial video is processed by the region of interest to obtain a preprocessed video segment. The preprocessed video segment is input into a trained pulse branch and time regression head to output a predicted pulse signal. The heart rate of the subject is determined based on the predicted pulse signal.

2. The heart rate detection method based on feature orthogonal constraints and RGB temporal reconstruction according to claim 1, characterized in that: The method for constructing the orthogonal loss is as follows: Global average pooling is performed along the spatial dimension on the pulse-related high-level features and the noise-related high-level features to obtain pulse-related pooling features and noise-related pooling features; The channel dimension and time dimension of the pulse-related pooling feature and the noise-related pooling feature are expanded respectively to obtain the pulse feature vector and noise feature vector corresponding to each training sample. After aligning the pulse feature vector and noise feature vector corresponding to each training sample, the absolute value of the cosine similarity between the two is calculated, and the average value of the absolute value of the cosine similarity corresponding to all training samples is used as the orthogonal loss.

3. The heart rate detection method based on feature orthogonal constraints and RGB temporal reconstruction according to claim 1, characterized in that: The methods for constructing the pulse-driven RGB components and the noise-driven RGB components are as follows: Learnable amplitude coefficients and pulse projection vectors corresponding to the predicted pulse signal, and learnable amplitude coefficients and noise projection vectors corresponding to the non-physiological noise signal are set respectively; the pulse projection vector and noise projection vector are both learnable parameters; the predicted pulse signal is multiplied by its corresponding learnable amplitude coefficients and pulse projection vectors to obtain the pulse-driven RGB component; the non-physiological noise signal is multiplied by its corresponding learnable amplitude coefficients and noise projection vectors to obtain the noise-driven RGB component.

4. The heart rate detection method based on feature orthogonal constraints and RGB temporal reconstruction according to claim 3, characterized in that: A relative channel weight order constraint is applied to the three channel components of the pulse projection vector; the relative channel weight order constraint is that the green channel component is greater than the blue channel component, and the blue channel component is greater than the red channel component.

5. The heart rate detection method based on feature orthogonal constraints and RGB temporal reconstruction according to claim 1, characterized in that: The total loss also includes pulse supervision loss; the pulse supervision loss is constructed based on the Pearson correlation coefficient between the predicted pulse signal and the reference pulse signal; the reference pulse signal is a photoplethysmography pulse waveform or a blood volume pulse waveform acquired synchronously with the original training video clip.

6. The heart rate detection method based on feature orthogonal constraints and RGB temporal reconstruction according to claim 5, characterized in that: The total loss also includes the maximum cross-correlation loss; the method for constructing the maximum cross-correlation loss is as follows: Within a preset time offset range, the reference pulse signal is relatively time-shifted; the normalized cross-correlation value between the predicted pulse signal and the reference pulse signal corresponding to each time offset is calculated. The maximum normalized cross-correlation value is determined from all normalized cross-correlation values, and the maximum cross-correlation loss is determined based on the maximum normalized cross-correlation value.

7. The heart rate detection method based on feature orthogonal constraints and RGB temporal reconstruction according to claim 1, characterized in that: The total loss also includes frequency domain separation loss; the method for constructing the frequency domain separation loss is as follows: The predicted pulse signal and non-physiological noise signal are subjected to mean removal and one-sided fast Fourier transform respectively. The power spectral density of the predicted pulse signal and non-physiological noise signal are extracted in the preset physiological frequency band and normalized to obtain the pulse power spectrum and noise power spectrum. The average value of the squared difference between the pulse power spectrum and the noise power spectrum at each frequency point is obtained to obtain the spectral difference degree. If the spectral difference degree is less than the preset interval threshold, the frequency domain separation loss is the difference between the preset interval threshold and the spectral difference degree; otherwise, the frequency domain separation loss is zero.

8. The heart rate detection method based on feature orthogonal constraints and RGB temporal reconstruction according to claim 1, characterized in that: The pulse branch includes seven convolutional blocks connected in sequence, and the noise branch includes five convolutional blocks connected in sequence. Each convolutional block includes a three-dimensional convolutional layer, a batch normalization layer, and a nonlinear activation layer connected in sequence. The time regression heads are respectively located at the ends of the pulse branch and the noise branch. Each time regression head includes three sets of spatiotemporal convolutional processing blocks connected in sequence. Each set of spatiotemporal convolutional processing blocks includes a three-dimensional convolutional layer, a batch normalization layer, and a ReLU nonlinear activation layer.

9. The heart rate detection method based on feature orthogonal constraints and RGB temporal reconstruction according to claim 1, characterized in that: The facial region of interest processing method is as follows: perform facial detection, facial tracking or key point localization on the original training video clip or the facial video of the test subject to determine at least one region among the whole face skin region, forehead region, cheek region or multiple facial partitions, and perform image cropping according to the determined region.

10. A heart rate detection system based on feature orthogonal constraints and RGB temporal reconstruction, characterized in that: The system is used to execute the heart rate detection method based on feature orthogonal constraints and RGB time-series reconstruction as described in claim 1. The heart rate detection system includes an RGB camera device, a processor, and a memory. The processor is configured with a data acquisition module, a region of interest processing module, an RGB time-series signal generation module, a dual-branch processing module, a loss construction module, a parameter update module, and a vital signs output module. The data acquisition module is used to acquire the original training video segment containing the facial region of the sample object and the facial video of the test object through the RGB camera device, and to acquire the reference pulse signal synchronously acquired with the original training video segment; the region of interest processing module is used to perform facial region of interest processing on the original training video segment to obtain the training video segment, and to perform facial region of interest processing on the facial video of the test object to obtain the preprocessed video segment. The RGB timing signal generation module is used to obtain the target RGB timing signal based on the timing changes of each RGB channel in the region of interest of the face; The dual-branch processing module includes a pulse branch and a noise branch with independent parameters, a pulse time regression head set at the end of the pulse branch, and a noise time regression head set at the end of the noise branch, used to output the predicted pulse signal and non-physiological noise signal; the loss construction module is used to construct a total loss including pulse supervision loss, maximum cross-correlation loss, RGB reconstruction loss, feature orthogonality loss, and frequency domain separation loss based on the predicted pulse signal, non-physiological noise signal, target RGB time series signal, pulse-related high-level features, and noise-related high-level features; the parameter update module is used to update the dual-branch network, time regression head, and learnable parameters used for RGB time series reconstruction based on the total loss; the vital signs output module is used to perform frequency domain main peak search or time domain peak detection based on the predicted pulse signal to determine and output the heart rate of the tested object.

Images