A non-contact heart rate detection method for face occlusion scene

Abstract

The application discloses a kind of non-contact heart rate detection methods for face has sheltered scene, steps are as follows: input training video, face picture sequence is intercepted by face recognition;Training is based on the three-dimensional residual neural network of block;Extract the block weight based on correlation coefficient;Extract the block frequency domain feature clustering center;Input real video, face picture sequence is intercepted by face recognition;Extract the block depth feature by the three-dimensional residual neural network of block based on the training;Extract the block weight based on clustering;Extract block detection heart rate, the block weight based on correlation coefficient and the block weight based on clustering are weighted summation, obtain final detection heart rate.The application assigns different weights to the heart rate detection results of different facial regions, overcoming the problem of accuracy decline in non-contact heart rate detection method under the scene of partially occluded face.

Description

Technical Field

[0001] This invention relates to the fields of information technology and smart health technology, specifically to a non-contact heart rate detection method for scenarios where the face is obscured. Background Technology

[0002] Non-contact heart rate detection has significant application value in the field of human health monitoring. Traditional contact-based heart rate detection methods require the user to wear specialized equipment, which is complex to operate and has limited applicability. In contrast, non-contact heart rate detection methods based on facial video can detect the corresponding heart rate simply by capturing a video of the user's face with a camera, making it convenient to use and applicable to a wide range of scenarios.

[0003] With the development of deep learning, non-contact heart rate detection technology based on deep neural network models has become increasingly mature. These methods typically extract the region of interest (ROI) on the face first, and then use a neural network model to detect the heart rate. Compared to traditional methods, they offer advantages such as high versatility and accuracy. However, few of these methods address scenarios involving partial facial occlusion. When the face is partially occluded, the occluding object is treated as part of the facial area and participates in heart rate detection, introducing irrelevant information that leads to larger errors and lower reliability in the heart rate detection results. Therefore, a non-contact heart rate detection method is urgently needed to solve the problem of partial facial occlusion and expand the application scenarios of non-contact heart rate detection. Summary of the Invention

[0004] The purpose of this invention is to solve the problems of increased detection error and low reliability of detection results caused by external objects obstructing the face of the person being detected in existing non-contact heart rate detection technologies. This invention provides a non-contact heart rate detection method for scenarios with facial occlusion. By combining a block-based three-dimensional residual neural network, different weights are assigned to the heart rate detection results of different face blocks, thereby reducing the negative impact of the occluded area on the heart rate detection results and improving the accuracy of non-contact heart rate detection.

[0005] The objective of this invention can be achieved by adopting the following technical solutions:

[0006] A non-contact heart rate detection method for scenarios where the face is obscured, the non-contact heart rate detection method comprising the following steps:

[0007] S1. Input multiple training videos and extract face image sequences through face recognition;

[0008] S2. Train a block-based 3D residual neural network;

[0009] S3. Extract block weights based on correlation coefficients;

[0010] S4. Extract the cluster centers of the block frequency domain features;

[0011] S5. Input a real video and extract a sequence of face images through face recognition;

[0012] S6. Extract block depth features using a trained block-based 3D residual neural network;

[0013] S7. Extract cluster-based block weights;

[0014] S8. Extract the detected heart rate by segmentation, and sum the segmentation weights based on correlation coefficient and clustering to obtain the final detected heart rate.

[0015] Further, step S1 is as follows:

[0016] S101. For each training video, including face videos and PPG signals collected by a standard fingertip pulse oximeter, extract the first T frames of each training video as training samples. The input video must be a face video because this method can extract the corresponding heart rate changes based on the changes in skin color caused by the dilation and contraction of blood vessels. Since the face has more exposed skin and richer capillaries, it is easier to detect heart rate through video compared to other parts of the human body.

[0017] S102. Perform face recognition on T video frames using the publicly available face recognition algorithm library Dlib. Obtain the face region coordinates of each video frame to obtain the face region and obtain a sequence of T face images. Extracting the face region can remove irrelevant information such as video background and reduce the interference of irrelevant information on heart rate detection.

[0018] S103. Use bilinear interpolation to scale the sequence of T face images to the same specified dimension. The reason for unifying the image dimension is that the face regions cropped in S102 may be of different sizes, while the block-based 3D residual neural network in step S2 requires an input sequence of images with the same dimension. Therefore, it is necessary to use bilinear interpolation to scale the images to facilitate the subsequent training of the network.

[0019] S104. Repeat S101-S103 until all training videos have extracted T sequences of face images of the same specified dimension.

[0020] Furthermore, the block-based three-dimensional residual neural network structure is as follows:

[0021] A deep feature extraction module is connected to four parallel block signal extraction modules; the structure of the deep feature extraction module is as follows:

[0022] The layers connected from input to output are as follows: 3D convolutional layer Conv3d-1, average pooling layer AvgPool-2, 3D convolutional layer Conv3d-3, BN layer BatchNorm3d-4, ReLU layer ReLU-5, 3D convolutional layer Conv3d-6, BN layer BatchNorm3d-7, ReLU layer ReLU-8, 3D convolutional layer Conv3d-9, BN layer BatchNorm3d-10, ReLU layer ReLU-11, 3D convolutional layer Conv3d-12, BN layer BatchNorm3d-13, ReLU layer ReLU-14, average pooling layer AvgPool-15, 3D convolutional layer Conv3d-16, BN layer BatchNorm3d-17, ReLU layer ReLU-18, 3D convolutional layer Conv3d-19, BN layer BatchNorm3d-20, ReLU... U-layer ReLU-21, 3D convolutional layer Conv3d-22, BN layer BatchNorm3d-23, ReLU layer ReLU-24, 3D convolutional layer Conv3d-25, BN layer BatchNorm3d-26, ReLU layer ReLU-27, Average pooling layer AvgPool-28, 3D convolutional layer Conv3d-29, BN layer BatchNorm3d-30, ReLU layer ReLU-31, 3D convolutional layer Conv3d-32, BN layer BatchNorm3d-33, ReLU layer ReLU-34, 3D convolutional layer Conv3d-35, BN layer BatchNorm3d-36, ReLU layer ReLU-37, 3D convolutional layer Conv3d-38, BN layer BatchNorm3d-39, ReLU layer ReLU-40, Average pooling layer AvgPool-41;

[0023] The signal extraction module has the following structure: from the input layer to the output layer, it is connected in sequence as follows: Adaptive AvgPool3d-42 and 3D convolutional layer Conv3d-43.

[0024] Furthermore, step S2 is as follows:

[0025] The face image sequence extracted in step S1 is input into the block-based three-dimensional residual neural network in batches, and the output is 4 block rPPG signals. The Pearson correlation coefficient is calculated between the output 4 block rPPG signals and the PPG signals collected by the training video through a standard finger clip pulse oximeter. The results are then accumulated as the loss function Loss, as shown in the following formula.

[0026]

[0027] Where Pearson() represents the calculation of the Pearson correlation coefficient between two signals, S i S represents the block rPPG signal output by the 3D convolutional layer Conv3d-43 of the i-th block signal extraction module in a block-based 3D residual neural network. gt The training video uses PPG signals collected by a standard finger-clip pulse oximeter. During training, the parameters of a block-based 3D residual neural network are iteratively adjusted according to the loss function (Loss) until the absolute difference between the loss functions of two adjacent iterations does not exceed 0.1. The Pearson correlation coefficient is used as the loss function because it reflects the correlation between two signals. When the output signal of the neural network differs significantly from the PPG signal, the Pearson correlation coefficient will be small. In this case, the loss function (Loss) can be used to promptly feed back to the network for parameter adjustment, improving the network's stability. The block-based 3D residual neural network is trained because the network parameters are randomized during initialization, and training is needed to enable the network to extract heart rate information from the face region.

[0028] Furthermore, step S3 is as follows:

[0029] Record the Pearson correlation coefficients of each block calculated in step S2 for each training video. After training, sum the correlation coefficients recorded for each block, and then normalize the sums of the four blocks to obtain four block weights based on the correlation coefficients. The calculation formula is as follows:

[0030]

[0031] Where N is the number of training videos. This represents the block rPPG signal output by the 3D convolutional layer Conv3d-43 of the ith block signal extraction module in the block-based 3D residual neural network for the j-th training video. This represents the PPG signal collected by a standard finger clip pulse oximeter for the j-th training video.

[0032] Block weights based on correlation coefficients are used to distinguish the strength of different blocks in extracting rPPG signals. Since different facial regions contain different numbers of capillaries, the quality of the block rPPG signals output by the 3D convolutional layer Conv3d-43 of different block signal extraction modules also varies. By recording the correlation coefficients of different blocks during the training process, when the signal quality extracted by a certain block is high, the correlation coefficient of that signal is relatively large, and the corresponding block weight based on the correlation coefficient will also be larger, and its proportion in the final heart rate detection result will also be higher. Therefore, using block weights based on correlation coefficients can improve the reliability of the final heart rate detection.

[0033] Furthermore, step S4 is as follows:

[0034] For the segmented rPPG signals extracted in step S2, a Fast Fourier Transform (FFT) is used to transform the rPPG signals into segmented frequency domain signals. K-means clustering is then used to extract K cluster centers from these frequency domain signals. The conversion to frequency domain before clustering is because the original segmented rPPG signals are time domain signals, and signals extracted from different training videos do not have strong correlations in the time domain. However, in the frequency domain, since heart rates are concentrated within a certain frequency range, the frequency domain signals are more concentrated, allowing for the extraction of cluster centers. The purpose of extracting cluster centers is to distinguish between segmented rPPG signals and irrelevant signals based on the characteristic that segmented rPPG signals are closer to the cluster centers, while irrelevant signals are farther away.

[0035] Furthermore, step S5 is as follows:

[0036] For the input real video, a sequence of T-frame face images is extracted through steps S101-S103. Because the block-based 3D residual neural network has fixed the dimension of the input image sequence during training, inputs of the same dimension are required during the testing phase to obtain better output results.

[0037] Furthermore, step S6 is as follows:

[0038] The face image sequence extracted in step S5 is input into a block-based 3D residual neural network, and the depth feature map is extracted by the depth feature extraction module in the 3D residual neural network. Where T represents the number of video frames, and C, H, and W represent the number of channels, feature map height, and feature map width of the depth feature map X, respectively; the depth feature map X is divided into 4 blocks with equal height and width, resulting in the following block-wise depth feature map: Dividing the feature map equally along its height and width preserves local spatial features without segmenting the temporal dimension, thus retaining complete temporal information, which is beneficial for subsequent block-based rPPG signal extraction.

[0039] Furthermore, step S7 is as follows:

[0040] S701. Input the four segmented depth block feature maps Xi into the four block signal extraction modules of the trained block-based three-dimensional residual neural network, and output four block rPPG signals. The purpose of this step is to ensure that even when the face is partially occluded by an obstacle, the other unoccluded facial areas captured by the camera can still have their rPPG signals extracted by the block signal extraction modules, thereby reducing the negative impact of occlusion on heart rate detection.

[0041] S702. Use Fast Fourier Transform to transform the four block rPPG signals into four block frequency domain signals;

[0042] S703. Calculate the Euclidean distance between the frequency domain signal of each block and the K cluster centers extracted in step S4, and calculate the block weights based on clustering using the following formula:

[0043]

[0044] in, The clustering-based block weights represent the i-th block, where i = 1, 2, 3, 4; d i This represents the Euclidean distance between the i-th block's frequency domain signal and the nearest cluster center among the K cluster centers. By comparing the distances between the signals and cluster centers, the quality of each block's rPPG signal can be distinguished; higher quality rPPG signals are closer to the cluster centers. When a block is significantly affected by facial occlusion, the extracted block's frequency domain signal will contain more irrelevant signals, thus resulting in a greater distance from the cluster centers. Consequently, the corresponding cluster-based block weights are smaller, which has less impact on the final weighted summation of heart rate detection results, thereby improving the accuracy of heart rate detection.

[0045] Furthermore, step S8 is as follows:

[0046] S801. For the four segmented frequency domain signals extracted in step S7, extract the amplitude values ​​with frequencies between 0.8Hz and 3Hz, and select the frequency corresponding to the largest amplitude value as the segmented heart rate detection frequency (hr). i , i = 1, 2, 3, 4; the cutoff frequency range of 0.8 Hz to 3 Hz can filter out high-frequency and low-frequency noise that is unrelated to heart rate;

[0047] S802. Multiply the correlation coefficient-based block weights and cluster-based block weights of the four blocks respectively, and then normalize them to obtain the total block weight w. i , i = 1, 2, 3, 4; This step combines the block weights based on correlation coefficients extracted in step S3 and the block weights based on clustering extracted in step S7. The resulting total block weights simultaneously include the heart rate detection capability of different blocks and the degree of influence of occlusion, reflecting the reliability of heart rate detection in each block.

[0048] S803, Detect the heart rate (hr) from the four segments extracted in step S801. i The total weight w of the four blocks extracted in step S802 i The final heart rate (HR) is obtained by performing a weighted summation, and the calculation formula is as follows:

[0049]

[0050] The final heart rate detection result is obtained by weighing the reliability of the heart rate detection in each block by the total weight of the blocks, resulting in a heart rate detection result with high accuracy and less impact from partial facial occlusion.

[0051] The present invention has the following advantages and effects compared with the prior art:

[0052] 1) This invention uses a block-based three-dimensional residual neural network for rPPG signal extraction. In scenarios where the face is partially occluded, useful heart rate information can be extracted from the unoccluded facial area, avoiding the loss of heart rate detection results due to occlusion.

[0053] 2) This invention proposes a block weight extraction method based on correlation coefficient. By calculating and statistically analyzing the Pearson correlation coefficient between the predicted signal and the supervision signal of each block in the training video sample, the weight of each block is obtained. This weight allows the heart rate detection results of blocks with higher accuracy to account for a larger proportion of the final heart rate detection, thereby improving the accuracy of heart rate detection.

[0054] 3) This invention proposes a cluster-based block weight extraction method, which assigns different weights to each block heart rate detection result by calculating the distance between the block frequency domain signal and the cluster center. This weight reflects the degree to which the heart rate detection result of each block is affected by facial occlusion, thereby improving the robustness of non-contact heart rate detection technology in scenarios with partial facial occlusion. Attached Figure Description

[0055] The accompanying drawings, which are included to provide a further understanding of the invention and form part of this application, illustrate exemplary embodiments of the invention and, together with their description, serve to explain the invention and do not constitute an undue limitation thereof. In the drawings:

[0056] Figure 1 This is a flowchart of a non-contact heart rate detection method for scenarios where the face is obscured, as disclosed in an embodiment of the present invention.

[0057] Figure 2 This is an example diagram of the training and testing process in an embodiment of the present invention. Detailed Implementation

[0058] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0059] Example 1

[0060] This embodiment discloses a non-contact heart rate detection method for scenarios where the face is obscured, such as... Figure 1 As shown, the specific steps are as follows:

[0061] S1. First, training videos were acquired. Thirty subjects sat quietly in front of a webcam, and one minute of facial video was recorded and uploaded to a hard disk recorder. Simultaneously, PPG signals were recorded using a standard finger-clip pulse oximeter. After acquisition, 300 consecutive frames were extracted from each training video. The face recognition function of the Dlib algorithm library was used to extract the face region from each frame. The extracted face image sequence was then scaled using bilinear interpolation to the same dimensions of 128 pixels in height and 128 pixels in width.

[0062] S2. Divide the processed image sequence into batches of three and input them into a block-based 3D residual neural network. Each input image sequence outputs four block rPPG signals after passing through the block-based 3D residual neural network. Calculate the Pearson correlation coefficient between the four output block rPPG signals and the PPG signals of the training video, and then sum them as the loss function, as shown in the following formula.

[0063]

[0064] Where Pearson() represents the calculation of the Pearson correlation coefficient between two signals, Si represents the block rPPG signal output by the three-dimensional convolutional layer Conv3d-43 of the i-th block signal extraction module in the block-based three-dimensional residual neural network, and S gt The training video uses PPG signals collected by a standard fingertip pulse oximeter. During training, the parameters of the block-based three-dimensional residual neural network are iteratively adjusted according to the loss function Loss until the difference in the absolute value of the loss function Loss between two adjacent iterations does not exceed 0.1, at which point the training ends.

[0065] S3. Record the Pearson correlation coefficient of each block calculated in step S2 for each training video. After training, sum the correlation coefficients recorded for each block, and then normalize the sums of the four blocks to obtain four block weights based on the correlation coefficients.

[0066] S4. For the block rPPG signal extracted in step S2, use Fast Fourier Transform to transform the rPPG signal into a block frequency domain signal; use K-means clustering to extract 8 cluster centers from the block frequency domain signal.

[0067] S5. Record a real video. The test subject wears a mask to simulate a scenario where the face is partially covered. At the same time, a 12-second facial video is recorded using a camera at a frame rate of 25fps. To compare the effectiveness of this method, the test subject also wears a standard finger-clip pulse oximeter to record the average heart rate value within 12 seconds, which is recorded as 85bpm. The recorded test video is then processed using the same method as in step S1 to extract the test subject's facial image sequence.

[0068] S6. Input the face image sequence extracted in step S5 into the trained block-based 3D residual neural network, and extract the depth feature map through the deep feature extraction module in the network. The depth feature map is divided into four blocks by dividing its height and width into two equal parts. The resulting feature map is as follows:

[0069] S7. Input the four block feature maps into the four block signal extraction modules of the trained block-based 3D residual neural network, and output four block rPPG signals. Use Fast Fourier Transform to transform the four block rPPG signals into four block frequency domain signals. Calculate the Euclidean distance between each block frequency domain signal and the eight cluster centers extracted in step S4, and calculate the extracted cluster-based block weights using the following formula:

[0070]

[0071] in, The cluster-based block weights represent the i-th block, where i = 1, 2, 3, 4; d i Let represent the Euclidean distance between the i-th block's frequency domain signal and the nearest cluster center among the K cluster centers. The four cluster-based block weights are calculated as follows:

[0072] S8. For the four frequency domain signals extracted in step S7, the amplitude values ​​between 0.8Hz and 3Hz are extracted, and the frequency corresponding to the largest amplitude value is selected as the detected heart rate of the block, resulting in four detected heart rates: hr1 = 87 bpm, hr2 = 88 bpm, hr3 = 60 bpm, and hr4 = 61 bpm. The block weights based on correlation coefficients and clustering are multiplied and normalized to obtain the total block weights: w1 = 0.407, w2 = 0.486, w3 = 0.039, and w4 = 0.068. The four detected heart rates are then weighted and summed with the total block weights to obtain the final detected heart rate HR. The calculation process is as follows:

[0073] HR=87×0.407+88×0.486+60×0.039+61×0.068=84.665bpm

[0074] The heart rate detected by this method was 84.665 bpm, which is only 0.335 bpm different from the heart rate recorded by the standard finger-clip pulse oximeter. The error is small, indicating that the heart rate detected by this method is highly accurate in scenarios where the face is partially obscured.

[0075] Example 2

[0076] This embodiment discloses a non-contact heart rate detection method for scenarios where the face is obscured. The specific steps are as follows:

[0077] S1. First, training videos were acquired. Fifty subjects sat quietly in front of webcams, and 30-second facial videos were recorded and uploaded to a hard disk recorder. Simultaneously, PPG signals were recorded using a standard finger-clip pulse oximeter. After acquisition, 100 consecutive frames were extracted from each training video. For each frame, the face recognition function of the Dlib algorithm library was used to extract the face region. The extracted face image sequence was then scaled using bilinear interpolation to the same dimensions of 64 pixels in height and 64 pixels in width.

[0078] S2. Divide the processed image sequence into batches of 5 sequences and input them into a block-based 3D residual neural network. Each input image sequence outputs 4 block rPPG signals after passing through the block-based 3D residual neural network. Calculate the Pearson correlation coefficient between the output 4 block rPPG signals and the PPG signals of the training video, and then sum them as the loss function, as shown in the following formula.

[0079]

[0080] Where Pearson() represents the calculation of the Pearson correlation coefficient between two signals, S i S represents the block rPPG signal output by the 3D convolutional layer Conv3d-43 of the i-th block signal extraction module in a block-based 3D residual neural network. gt The training video uses PPG signals collected by a standard fingertip pulse oximeter. During training, the parameters of the block-based three-dimensional residual neural network are iteratively adjusted according to the loss function Loss until the difference in the absolute value of the loss function Loss between two adjacent iterations does not exceed 0.1, at which point the training ends.

[0081] S3. Record the Pearson correlation coefficient of each block calculated in step S2 for each training video. After training, sum the correlation coefficients recorded for each block, and then normalize the sums of the four blocks to obtain four block weights based on the correlation coefficients.

[0082] S4. For the block rPPG signal extracted in step S2, use Fast Fourier Transform to transform the rPPG signal into a block frequency domain signal; use K-means clustering to extract 6 cluster centers from the block frequency domain signal.

[0083] S5. Record real video. The test subject wears a mask to simulate a scenario where the face is partially covered. At the same time, a camera is used to record a 4-second facial video at a frame rate of 25fps. To compare the effect of this method, the test subject wears a standard finger-clip pulse oximeter to record the average heart rate value within 4 seconds, which is recorded as 70 bpm. The recorded test video is processed using the same method as in step S1 to extract the test subject's facial image sequence.

[0084] S6. Input the face image sequence extracted in step S5 into the trained block-based 3D residual neural network, and extract the depth feature map through the deep feature extraction module in the network. The depth feature map is divided into four blocks by dividing its height and width into two equal parts. The resulting feature map is as follows:

[0085] S7. Input the four block feature maps into the four block signal extraction modules of the trained block-based 3D residual neural network, and output four block rPPG signals. Use Fast Fourier Transform to transform the four block rPPG signals into four block frequency domain signals. Calculate the Euclidean distance between each block frequency domain signal and the six cluster centers extracted in step S4, and calculate the extracted cluster-based block weights using the following formula:

[0086] in, The clustering-based block weights represent the i-th block, where i = 1, 2, 3, 4; d i Let represent the Euclidean distance between the i-th block's frequency domain signal and the nearest cluster center among the K cluster centers. The four cluster-based block weights are calculated as follows:

[0087] S8. For the four segmented frequency domain signals extracted in step S7, the amplitude values ​​with frequencies between 0.8Hz and 3Hz are extracted, and the frequency corresponding to the largest amplitude value is selected as the segmented heart rate. The four segmented heart rates are hr1 = 74 bpm, hr2 = 70 bpm, hr3 = 58 bpm, and hr4 = 65 bpm. The segment weights based on correlation coefficients and clustering are multiplied and normalized to obtain the total segment weights w1 = 0.385, w2 = 0.416, w3 = 0.088, and w4 = 0.109.

[0088] The final detected heart rate was calculated using four methods: no weights, block weights based only on correlation coefficients, block weights based only on clustering, and total block weights. The results are shown in the table below.

[0089] Table 1. Comparison of Heart Rate Measurement Results

[0090]

[0091]

[0092] The heart rate detected by this method using total weighted segmentation was 69.799 bpm, which is only 0.201 bpm different from the heart rate recorded by the standard finger-clip pulse oximeter (70 bpm). The error is smaller than that of the other three methods, indicating that the use of total weighted segmentation in this method can effectively reduce the heart rate detection error caused by partial obstruction of the face.

[0093] The above embodiments are preferred embodiments of the present invention, but the embodiments of the present invention are not limited to the above embodiments. Any changes, modifications, substitutions, combinations, or simplifications made without departing from the spirit and principle of the present invention shall be considered equivalent substitutions and shall be included within the protection scope of the present invention.

Claims

1. A non-contact heart rate detection method for scenarios where the face is obscured, characterized in that, The non-contact heart rate detection method includes the following steps: S1. Input multiple training videos and extract face image sequences through face recognition; S2. Train a block-based 3D residual neural network; the structure of the block-based 3D residual neural network is as follows: A deep feature extraction module is connected to four parallel block signal extraction modules; the structure of the deep feature extraction module is as follows: The layers connected from input to output are as follows: 3D convolutional layer Conv3d-1, average pooling layer AvgPool-2, 3D convolutional layer Conv3d-3, BN layer BatchNorm3d-4, ReLU layer ReLU-5, 3D convolutional layer Conv3d-6, BN layer BatchNorm3d-7, ReLU layer ReLU-8, 3D convolutional layer Conv3d-9, BN layer BatchNorm3d-10, ReLU layer ReLU-11, 3D convolutional layer Conv3d-12, BN layer BatchNorm3d-13, ReLU layer ReLU-14, average pooling layer AvgPool-15, 3D convolutional layer Conv3d-16, BN layer BatchNorm3d-17, ReLU layer ReLU-18, 3D convolutional layer Conv3d-19, BN layer BatchNorm3d-20, ReLU... U-layer ReLU-21, 3D convolutional layer Conv3d-22, BN layer BatchNorm3d-23, ReLU layer ReLU-24, 3D convolutional layer Conv3d-25, BN layer BatchNorm3d-26, ReLU layer ReLU-27, Average pooling layer AvgPool-28, 3D convolutional layer Conv3d-29, BN layer BatchNorm3d-30, ReLU layer ReLU-31, 3D convolutional layer Conv3d-32, BN layer BatchNorm3d-33, ReLU layer ReLU-34, 3D convolutional layer Conv3d-35, BN layer BatchNorm3d-36, ReLU layer ReLU-37, 3D convolutional layer Conv3d-38, BN layer BatchNorm3d-39, ReLU layer ReLU-40, Average pooling layer AvgPool-41; The signal extraction module has the following structure: The layers connected from the input layer to the output layer are: an adaptive average pooling layer (AdaptiveAvgPool3d-42) and a 3D convolutional layer (Conv3d-43). The process of step S2 is as follows: The face image sequence extracted in step S1 is input into the block-based three-dimensional residual neural network in batches, and the output is 4 block rPPG signals. The Pearson correlation coefficient is calculated between the output 4 block rPPG signals and the PPG signals collected by the training video through a standard finger clip pulse oximeter. The results are then accumulated as the loss function Loss, as shown in the following formula. Where Pearson() represents the Pearson correlation coefficient between two signals. This represents the block rPPG signal output by the 3D convolutional layer Conv3d-43 of the i-th block signal extraction module in a block-based 3D residual neural network. The training video uses PPG signals collected by a standard finger clip pulse oximeter. During training, the parameters of the block-based three-dimensional residual neural network are iteratively adjusted according to the loss function Loss until the difference between the absolute values ​​of the loss function Loss of two adjacent iterations does not exceed 0.1, at which point the training ends. S3. Extract block weights based on correlation coefficients; S4. Extract the cluster centers of the block frequency domain features; S5. Input a real video and extract a sequence of face images through face recognition; S6. Extract block depth features using a trained block-based 3D residual neural network; S7. Extract cluster-based block weights; the process of step S7 is as follows: S701. Divide the depth block feature map after dividing the 4 blocks. The signals are respectively input into the four block signal extraction modules of the trained block-based three-dimensional residual neural network, and the output is four block rPPG signals. S702. Use Fast Fourier Transform to transform the four block rPPG signals into four block frequency domain signals; S703. Calculate the Euclidean distance between the frequency domain signal of each block and the K cluster centers extracted in step S4, and calculate the block weights based on clustering using the following formula: ， in, Represents the clustering-based block weight of the i-th block. This represents the Euclidean distance between the i-th block frequency domain signal and the nearest center among the K cluster centers; S8. Extract the detected heart rate by segmentation, and sum the segmentation weights based on correlation coefficients and clustering to obtain the final detected heart rate; the process of step S8 is as follows: S801. For the four segmented frequency domain signals extracted in step S7, extract the amplitude values ​​with frequencies between 0.8Hz and 3Hz, and select the frequency corresponding to the largest amplitude value as the segmented heart rate detection. , ; S802. Multiply the correlation coefficient-based block weights and cluster-based block weights of the four blocks respectively, and then normalize them to obtain the total block weights. , ; S803, Detect heart rate using the four segments extracted in step S801. The total weight of the four blocks extracted in step S802 The final heart rate (HR) is obtained by performing a weighted summation, and the calculation formula is as follows: 。 2. The non-contact heart rate detection method for scenarios with facial occlusion as described in claim 1, characterized in that, The process of step S1 is as follows: S101. For each training video, including face videos and PPG signals collected by a standard fingertip pulse oximeter, extract the first T frames of each training video as training samples. S102. Perform face recognition on T video frames using the publicly available face recognition algorithm library Dlib. Obtain the face region coordinates of each video frame to obtain the face region and obtain a sequence of T face images. S103. Use bilinear interpolation to scale a sequence of T face images to the same specified dimension. S104. Repeat S101-S103 until all training videos have extracted T sequences of face images of the same specified dimension.

3. The non-contact heart rate detection method for scenarios with facial occlusion as described in claim 1, characterized in that, The process of step S3 is as follows: Record the Pearson correlation coefficients of each block calculated in step S2 for each training video. After training, sum the correlation coefficients recorded for each block, and then normalize the sums of the four blocks to obtain four block weights based on the correlation coefficients. , The calculation formula is as follows: Where N is the number of training videos. This represents the block rPPG signal output by the 3D convolutional layer Conv3d-43 of the ith block signal extraction module in the block-based 3D residual neural network for the j-th training video. This represents the PPG signal collected by a standard finger clip pulse oximeter for the j-th training video.

4. A non-contact heart rate detection method for scenarios with facial occlusion as described in claim 1, characterized in that, The process of step S4 is as follows: The segmented rPPG signal extracted in step S2 is transformed into a segmented frequency domain signal using Fast Fourier Transform; K-means clustering is then used to extract K cluster centers from the segmented frequency domain signal.

5. A non-contact heart rate detection method for scenarios with facial occlusion as described in claim 2, characterized in that, The process of step S5 is as follows: For the input real video, extract the T-frame face image sequence through steps S101-S103.

6. A non-contact heart rate detection method for scenarios with facial occlusion as described in claim 5, characterized in that, The process of step S6 is as follows: The face image sequence extracted in step S5 is input into a block-based 3D residual neural network, and the depth feature map is extracted by the depth feature extraction module in the 3D residual neural network. Where T represents the number of video frames, and C, H, and W represent the depth feature map respectively. The number of channels, feature map height, and feature map width; the depth feature map Divide the material into four equal parts based on both height and width. The resulting depth feature map is as follows: , .

Images