Method for estimating human heart rate based on radio frequency signals

Abstract

The method for estimating human heart rate based on radio frequency signals improves the real-time performance of heart rate prediction. Heart rate prediction is carried out based on short-time sample data of 1.2 s, which can reduce the time interval from radar data acquisition to heart rate estimation, and the real-time performance is greatly improved compared with traditional methods; it reduces the influence of personnel displacement on the prediction accuracy and improves the robustness of the heart rate prediction system; when performing Range-FFT transformation on the FMCW radar radio frequency data, this method uses the obtained personnel position information as a reference to judge whether the monitored person has a large amplitude of movement and discards the data with too large movement amplitude, which can reduce the prediction error and improve the robustness of the system; this method introduces the residual network ResNet50, which is beneficial to solving the problems of gradient explosion, gradient disappearance and degradation caused by CNN, thereby improving the training efficiency of the deep neural network and enhancing the robustness of the training process.

Description

Technical Field

[0001] The present invention belongs to the field of human vital sign monitoring, and particularly relates to a method for estimating human heart rate based on radio frequency signals. Background Art

[0002] Heart rate is an important indicator of human vital signs and also a reference basis for the diagnosis and treatment of many diseases. Traditional heart rate measurement generally uses contact devices, which have high accuracy, but are not convenient to use, have many restrictions on the movements and activities of patients, and there is also a certain risk of contact infection. Therefore, non-contact heart rate monitoring technology based on radio frequency signals has increasingly attracted the attention of researchers. Since FMCW radar has strong penetration ability, high range resolution, and low cost, it is an ideal radio frequency hardware device. Therefore, in recent years, more and more non-contact heart rate monitoring studies have been carried out based on FMCW radar and have become the mainstream and hotspots of research. Such as the reference document (Alizadeh M, Shaker G, De Almeida J C M, et al. Remote monitoring of human vital signs using mm-wave FMCW radar [J]. IEEE Access, 2019, 7: 54958 - 54968.).

[0003] The main technical challenge of non-contact heart rate monitoring is that the heart rate signal will become relatively weak under the strong interference of respiration and its harmonics, resulting in inaccurate heart rate estimation. The current processing methods mainly include DC offset, band-pass filtering, optimal frequency estimation, etc. These methods can filter out most of the fundamental frequency of respiration, but may still retain some harmonics of respiration, resulting in the extracted heart rate signal not being pure and affecting the subsequent heart rate estimation. Summary of the Invention

[0004] To solve the above problems, the present invention introduces a deep residual network to learn the signal data collected by FMCW radar, and proposes a non-contact heart rate estimation method based on radio frequency signals. This method is based on the deep residual network ResNet, and takes the radar phase data as the input of the residual network. Since the heartbeat signal is often submerged in the thoracic vibration signal caused by respiration and its harmonics, it is difficult for traditional signal processing methods to restore the heartbeat signal. Therefore, this method uses a deep learning network to address this problem. In addition, this method performs heart rate prediction based on short-time sample data, which can greatly reduce the observation time and improve the real-time performance of the system.

[0005] A method for estimating human heart rate based on radio frequency signals includes the following steps:

[0006] Step 1: Data acquisition;

[0007] Use an FMCW radar and an ECG analog electrocardiogram sensor to observe a target person; the FMCW radar emits a radio frequency signal in a specified frequency band; the radio frequency signal and the echo signal reflected by the human body are calculated by a mixer to obtain radar intermediate frequency (IF) signal data; the ECG sensor collects the real-time heart rate of the target person; the obtained IF signal data and real-time heart rate data are stored in a data file;

[0008] Step 2: Construct a data set using range Fourier transform;

[0009] Use range Fourier transform (Range-FFT) on the IF signal obtained in Step 1 to obtain the position information of the monitored person and the radar signal phase; extract the radar phase data according to the position of the person in the matrix, and associate the phase data with the ECG heart rate data at the same time, thus completing the creation of the sample data set;

[0010] Step 3: Construct and train a heart rate prediction network based on ResNet50;

[0011] Use ResNet50 as the main architecture of the heart rate prediction network, use the sample data set created in Step 2 as the input, and use the stochastic gradient descent optimizer to train the network; set its output result as a single value, that is, the predicted real-time heart rate; after multiple stochastic gradient descent trainings, select the network model parameters with an accuracy rate greater than 90% and save them;

[0012] Step 4: Apply the trained network model for real-time heart rate prediction;

[0013] Load the network model parameters saved in Step 3 and collect radar signal data in real time; when the data collection duration meets the conditions, input it into the trained network model for prediction calculation, and output the obtained result to the user; during the prediction process, use the position information of the person obtained by Range-FFT as reference information. If the change in the person's position exceeds the set threshold, discard the corresponding data, still use the predicted heart rate at the previous moment as the output, and prompt the user in time.

[0014] Advantages of the present invention:

[0015] (1) Improve the real-time performance of heart rate prediction. This method performs heart rate prediction based on short-time sample data of 1.2 s, which can reduce the time interval from radar data collection to heart rate estimation, and the real-time performance is greatly improved compared with traditional methods.

[0016] (2) It reduces the impact of personnel displacement on the prediction accuracy and improves the robustness of the heart rate prediction system. When performing Range-FFT transformation on the FMCW radar RF data, this method uses the obtained personnel position information as a reference to determine whether the monitored personnel have moved significantly, and discards the data with too large a movement amplitude, which can reduce the prediction error and improve the robustness of the system.

[0017] (3) Most of the existing methods for predicting heart rate using FMCW radar choose the convolutional neural network (CNN, Convolutional Neural Network) as the deep network architecture. The residual network introduced by this method helps to solve the problems of gradient explosion, gradient disappearance, and degradation brought by CNN, thereby improving the training efficiency of the deep neural network and enhancing the robustness of the training process. Description of the Drawings

[0018] Figure 1 It is a schematic diagram of the data acquisition operation in the embodiment of the present invention.

[0019] Figure 2 It is a flowchart for making the data set in the embodiment of the present invention.

[0020] Figure 3 It is an architecture diagram of the heart rate prediction network model in the embodiment of the present invention.

[0021] Figure 4 It is an architecture diagram of the residual block in the heart rate prediction network in the embodiment of the present invention. Detailed Embodiment

[0022] The technical solution of the present invention will be further described in detail below with reference to the accompanying drawings of the specification.

[0023] The method for estimating the human heart rate based on RF signals includes the following steps:

[0024] Step 1, data acquisition.

[0025] In this embodiment, two types of original data are collected in total, and the process is as Figure 1 shown. The first type of data is radar signal data, which is transmitted by an FMCW multi-antenna radar, reflected by the measured human body, and received by the multi-antenna. This reflected signal contains the minute vibrations of the human chest. The second type of data is ECG heart rate data, which is measured by connecting an ECG sensor to the human body to obtain the real-time heart rate data of the target person. The specific steps for collecting the two types of data are as follows.

[0026] (1) Radar signal data. The FMCW radar generates a chirp pulse, which is emitted through the transmitter. The time interval between two pulse transmissions is one radar frame. In this embodiment, the duration of the radar frame is set to 0.05 s. Assuming the radar transmitted signal is s(t), when the pulse signal reaches the target person, the human body acts as a reflector, reflects the pulse signal modulated by the chest vibration, and is received by the receiver. The received signal is denoted as r(t). The transmitted signal s(t) and the received signal r(t) are shown in formulas (1) and (2) respectively.

[0027] s(t) = A s exp(2πf c +πKt 2 )(1)

[0028] r(t) = A r exp[2πf c (t - t d ) + πK(t - t d ) 2 (2)

[0029] Where, f c and K are the initial frequency and the frequency growth slope of the transmitted signal, and t d is the signal round-trip time. A s and A r are the amplitudes of the transmitted signal and the received signal respectively.

[0030] The transmitted signal s(t) and the received signal r(t) are processed by a mixer to obtain the IF signal y(t) as shown in formula (3).

[0031] y(t) = A s A r exp(2πf c t d + 2πKt d t - πKt 2 d ) ≈ A s A r exp(2πf c t d + 2πKt d t)(3)

[0032] Subsequently, the IF signal y(t) is further processed by an analog-to-digital converter (ADC) to generate a digital signal of y(t). This digital signal is saved in the form of a two-dimensional matrix. The column width of the two-dimensional matrix is determined by the ADC sampling frequency, and the number of rows is determined by the number of radar frames (i.e., the number of pulse transmissions) within the observation time.

[0033] (2)ECG heart rate data. In this embodiment, an ECG sensor is connected to the human body to measure the heart rate of the target human body. Heart rate data is monitored by the ECG sensor over a period of time, and this heart rate data and the radar IF digital signal data are stored in a data file.

[0034] Step 2: Use distance Fourier transform to construct a data set.

[0035] The process of constructing the data set is as Figure 2 shown. Read the data file obtained in the previous step, and perform Range-FFT transformation on the radar data. The specific operations are as follows. Read the two-dimensional matrix digital signal. Each row of this matrix corresponds to the IF signal obtained by one pulse emission and can be expressed as: X = [x 1 , x 2 ,..., x i ,..., x m , where x i represents the i-th sampling value of the signal y(t) by the ADC, and m is the maximum number of samplings. Perform FFT on each row X of the matrix, and X can be decomposed into multiple frequency components. Since the amplitude of the signal reflected by the human body in the echo signal is the largest, among the multiple frequency components obtained after FFT processing, the peak frequency corresponds to the human body reflection signal. Thus, centered on the position of the peak frequency, extract the radar phase data. At the same time, read the ECG heart rate data and associate it with the radar phase data of 24 frames in the same time period. Since the duration of a single frame of the radar is 0.05 s, 24 frames correspond to the radar phase data captured within 1.2 s. The specific operation is to use a sliding window algorithm with a length of 24 to process the extracted radar phase data and encapsulate it with the ECG heart rate data in the corresponding time period of each window. Repeat the above operations for all radar signal data and heart rate data, and the sample data set is obtained. After completing the above operations, divide the sample data set into two parts, randomly select 20% of the data set to form the test set, and the remaining 80% as the training set.

[0036] Step 3: Construct and train a heart rate prediction network based on ResNet50.

[0037] The heart rate prediction network based on ResNet50 built in this embodiment is as Figure 3 shown. Structurally, this network can be divided into STAGE0 and STAGE1 to STAGE4.

[0038] The first module in STAGE0 includes convolution (CONV, Convolution), batch normalization (BN, BatchNormalization), and activation function (RELU, Linear RectificationFunction), and the second module is the max pooling (MAXPOOL) module.

[0039] STAGE1 to STAGE4 respectively contain 3, 4, 6, and 3 residual unit Bottlenecks. After passing through the AveragePool module and the Fully Connected module at the end of STAGE4, the output of the network is obtained.

[0040] The residual unit Bottleneck is divided into two categories, namely ConvBlock and Identity Block, and their structures are as Figure 4 shown. The input data of ConvBlock passes through 3 convolutional blocks on the left and a single convolutional block on the right respectively, adds the two, and then outputs after being processed by the RELU activation function. Compared with ConvBlock, IdentityBlock has one less convolutional module on the right. Its input data passes through 3 convolutional blocks on the left and then adds to itself, and then outputs after being processed by the RELU activation function. In the 3 convolutional blocks on the left of both types of residual units, the last convolutional block lacks the RELU activation function compared with the previous two.

[0041] The relevant parameter settings of ResNet50 are as follows. The number of input data channels in STAGE0 is set to 1, the output size of the average pooling layer is set to (1,1), and the number of classifications in the fully connected layer is set to 1. The constructed ResNet50 network is set to the train mode, the network optimizer selects the stochastic gradient descent method with momentum, and the error statistical criterion for network prediction selects the mean squared error.

[0042] After the network is built, the training set generated by segmentation in step 2 is passed to the heart rate prediction network for training. After multiple stochastic gradient descent trainings, the network model parameters with an accuracy greater than 90% are selected for saving.

[0043] Step 4, apply the trained network model for real-time heart rate prediction.

[0044] Connect the FMCW radar to the computer, capture the data sent back by the radar to the computer in real time. Whenever the observed radar phase data reaches 1.2 s, it is passed into the network model saved in step 3 for prediction calculation, and the obtained result is output to the user in real time. In addition, during the prediction process, large movements of personnel will cause large errors in radar data, and inputting it into the network will cause inaccurate results. To reduce such errors, in this embodiment, the personnel position information obtained by Range-FFT is used as reference information. If the displacement of personnel exceeds the threshold of 0.5 m, the corresponding data is discarded, and the predicted heart rate at the previous moment is still used as the output, and the user is prompted in time. In practical applications, this can improve the stability and anti-interference ability of the heart rate prediction system.

[0045] This embodiment proposes a method for estimating human heart rate based on radio frequency signals. To verify its performance, this method was compared with the FMCW radar heart rate prediction method in the reference (Alizadeh M, Shaker G, De Almeida JC M, et al. Remote monitoring of human vital signs using mm-wave FMCW radar[J]. IEEE Access, 2019, 7:54958-54968.) in a comparative experiment. The experimental method is as follows: On the same data set, this method performs heart rate prediction based on 1.2 s of sample data, and the comparative methods perform heart rate prediction based on 1.2 s, 2.4 s, 4.8 s, and 9.6 s of sample data respectively, and then compare their average heart rate prediction accuracies to verify the performance of this method in terms of accuracy and real-time performance.

[0046] The experimental results are shown in Table 1. It can be seen that as the observation time increases, the average accuracy of the comparative algorithm becomes higher and higher. However, when this method only uses 1.2 s of sample data for heart rate prediction, the accuracy also exceeds that of the comparative algorithm using 9.6 s of sample data.

[0047] Table 1

[0048]

[0049] In summary, the method for estimating human heart rate based on radio frequency signals designed in this embodiment significantly reduces the required observation time while maintaining the accuracy, and the real-time performance and accuracy of the algorithm have obvious advantages. This method is real and effective.

[0050] The above is only the preferred embodiment of the present invention, and the protection scope of the present invention is not limited to the above embodiment. Any equivalent modification or change made by those of ordinary skill in the art according to the content disclosed in the present invention shall be included in the protection scope recorded in the claims.

Claims

1. A method for estimating human heart rate based on radio frequency signals, characterized in that: It includes the following steps: Step 1: Data acquisition; Use an FMCW radar and an ECG analog electrocardiogram sensor to observe the target person; the FMCW radar emits radio frequency signals in a specified frequency band; the radio frequency signals and the echo signals reflected by the human body are calculated by a mixer to obtain radar intermediate frequency (IF) signal data; The ECG sensor collects the real-time heart rate of the target person; the obtained IF signal data and real-time heart rate data are stored in a data file; Step 2: Construct a data set using range Fourier transform; Perform range Fourier transform (Range-FFT) on the IF signals obtained in Step 1 to obtain the position information of the monitored person and the radar signal phase; extract the radar phase data according to the position of the person in the matrix. At the same time, read the ECG heart rate data and associate it with the 24-frame radar phase data in the same time period. The duration of a single frame of the radar is 0.05 s, thus completing the creation of the sample data set; Step 3: Construct and train a heart rate prediction network based on ResNet50; Use ResNet50 as the main architecture of the heart rate prediction network, take the sample data set created in Step 2 as the input, and use a stochastic gradient descent optimizer to train the network; Set its output result as a single value, that is, the predicted real-time heart rate; after multiple stochastic gradient descent trainings, select the network model parameters with an accuracy rate greater than 90% and save them; Step 4: Apply the trained network model for real-time heart rate prediction; Load the network model parameters saved in Step 3 and perform real-time acquisition of radar signal data; when the data acquisition duration meets the conditions, input the trained network model for prediction calculation, and output the obtained result to the user; During the prediction process, use the position information of the person obtained by Range-FFT as reference information. If the change in the person's position exceeds the set threshold, discard the corresponding data, still use the predicted heart rate at the previous moment as the output, and prompt the user in a timely manner.

2. The method for estimating human heart rate based on radio frequency signals according to claim 1, characterized in that: In step 1, for radar signal data, the FMCW radar generates a chirp pulse, which is transmitted by the transmitter; the time interval between two pulse transmissions is one radar frame. Let the radar transmitted signal be s ( t ). When the pulse signal reaches the target person, the human body acts as a reflector, reflecting the pulse signal modulated by the chest vibration, and the reflected signal is received by the receiver. Let the received signal be r ( t ). The transmitted signal s ( t ) and the received signal r ( t ) are processed by a mixer to obtain the IF signal y ( t ). Then, it is processed by an analog-to-digital converter ADC to generate the digital signal of y ( t ). The digital signal is saved in the form of a two-dimensional matrix; the column width of the two-dimensional matrix is determined by the ADC sampling frequency, and the number of rows is determined by the number of radar frames within the observation time.

3. The method for estimating human heart rate based on radio frequency signals according to claim 1, characterized in that: In Step 2, the Range-FFT transform is to perform a fast Fourier transform on each row of the two-dimensional matrix of radar data, and after the transform, obtain the position information of the monitored person in each frame and the radar signal phase.

4. The method for estimating human heart rate based on radio frequency signals according to claim 1, characterized in that: The specific operation of data association is to use a sliding window algorithm with a length of 24 to process the extracted radar phase data and encapsulate it with the ECG heart rate data in the corresponding time period of each window; repeat the above operation for all radar signal data and heart rate data, that is, obtain the sample data set; after completing the above operation, divide the sample data set into two parts, randomly select 20% of the data set to form the test set, and the remaining 80% as the training set.

5. The method for estimating human heart rate based on radio frequency signals according to claim 1, characterized in that: In step 3, the heart rate prediction network based on ResNet50 includes STAGE0, STAGE1, STAGE2, STAGE3, and STAGE4.

6. The method for estimating human heart rate based on radio frequency signals according to claim 5, wherein: STAGE0 includes two modules connected in sequence. The first module contains convolution (CONV), batch normalization (BN), and activation function (RELU), and the second module is a max pooling module (MAXPOOL).

7. The method for estimating human heart rate based on radio frequency signals according to claim 5, wherein: STAGE1 to STAGE4 respectively contain 3, 4, 6, and 3 bottleneck residual units; the output of the network is obtained after passing through an average pooling module (Average Pool) and a fully connected module (Fully Connected) at the end of STAGE4.

8. The method for estimating human heart rate based on radio frequency signals according to claim 7, wherein: The bottleneck residual units are divided into two categories: Conv Block and Identity Block; The input data of the Conv Block passes through 3 convolutional blocks and a single convolutional block respectively, the two are added together, and then output after being processed by the RELU activation function; The input data of the Identity Block passes through 3 convolutional blocks and is added to itself, and then output after being processed by the RELU activation function; In the 3 convolutional blocks of the two types of bottleneck residual units, the last convolutional block lacks the RELU activation function compared to the previous two blocks.

9. The method for estimating human heart rate based on radio frequency signals according to claim 5, wherein: The relevant parameters of ResNet50 are set as follows: the number of input data channels of STAGE0 is set to 1, the output size of the average pooling layer is set to (1, 1), and the number of classifications in the fully connected layer is set to 1; the constructed ResNet50 network is set to the train mode, the network optimizer selects the stochastic gradient descent method with momentum, and the error statistical standard for network prediction selects the mean squared error.

Images