Multi-person electrocardiogram reconstruction method based on millimeter wave radar

Abstract

The invention discloses a multi-person electrocardiogram reconstruction method based on a millimeter-wave radar, which is applied to the technical field of millimeter-wave radar detection, and aims at solving the problem that in the prior art, only simple heart rate estimation of a plurality of targets is considered, but the estimation of heartbeat signals is not involved. The method comprises the following steps: firstly, converting a radar phonocardiogram into a frequency domain signal; extracting a top-k frequency component and converting the top-k frequency component into a time domain signal; then, multi-head cross attention and multi-head self-attention mechanisms are used for carrying out self-correlation reconstruction on the signals; the obtained output is then input into a frequency Gaussian weighting module, and then a reconstructed ECG signal is output. Compared with traditional heartbeat information analysis of multi-target signals, more accurate electrocardiogram reconstruction information can be provided.

Description

Technical Field

[0001] The present invention belongs to the field of millimeter wave radar detection technology, and in particular relates to an electrocardiogram reconstruction technology based on millimeter wave radar. Background Art

[0002] Millimeter-wave radar vital sign detection technology primarily leverages millimeter-wave radar's sensitivity to minute target movements to detect, estimate, and separate minute chest movements caused by breathing and heartbeats, while also estimating the required signals. This technology has significant application value in medical testing, home health, and smart elderly care. Because breathing and heartbeats cause minute chest movements, these movements effectively reflect phase changes in millimeter-wave radar echoes. Therefore, electrocardiogram (ECG) reconstruction can be achieved by extracting, processing, and reconstructing the phase signal of the millimeter-wave radar echo. However, compared to simple single-person ECG reconstruction, multi-person ECG reconstruction is constrained by multiple factors and is affected by various environmental conditions. Multi-person ECG reconstruction can be categorized into scenarios such as ECG reconstruction for targets at different distances and ECG reconstruction for targets at the same distance. ECG reconstruction for targets at different distances can be achieved by selecting different distance thresholds for phase signal extraction. However, the challenge lies in reconstructing the ECGs of different targets at the same distance. Because multiple targets are in the same range unit, the phase changes caused by their breathing and heartbeats overlap. At the same time, because the breathing and heartbeats of multiple targets are in the same frequency range, the phase information between the echoes of different targets interferes with each other, making phase signal separation difficult. Therefore, it is very important to study how to separate the multiple overlapping signals and reconstruct multi-person ECG signals.

[0003] Many research institutions at home and abroad have carried out research on multi-target heartbeat signal separation and single-person ECG reconstruction. Fujian University proposed a multi-target heartbeat signal extraction method based on Health-VMD at different distances (Z. Xu, T. Ye, L. Chen, Y. Gao and Z. Chen, "Health-Radar: Noncontact Multitarget Heart Rate Variability Detection Using FMCW Radar," in IEEE Sensors Journal, vol. 25, no. 1, pp. 405-418, 1 Jan. 1, 2025, doi: 10.1109 / JSEN.2024.3494755.), which uses the phase separable characteristics of multiple targets at different radar distance units, selects the phase signals of different units for extraction, and finally uses the adaptive parameter VMD method to decompose the extracted phase signals to achieve the purpose of multi-target heartbeat signal extraction. The University of Maryland, Park Campus, USA, proposed an improved VMD signal separation method (F. Wang, X. Zeng, C. Wu, B. Wang and K. J. R. Liu, "mmHRV: Contactless Heart Rate Variability Monitoring Using Millimeter-Wave Radio," in IEEE Internet of Things Journal, vol. 8, no. 22, pp. 16623-16636, 15 Nov. 15, 2021, doi: 10.1109 / JIOT.2021.3075167.), which analyzes the case of multiple targets at the same distance and different angles. Angle estimation using MVDR makes the target separable in the angle dimension, and then the VMD method is used to decompose the signals obtained by angle separation to realize heartbeat signal estimation under the condition of multiple targets at the same distance and different angles. However, the above method often only considers the simple heart rate estimation of multiple targets, and does not perform more accurate heartbeat signal estimation, that is, the existing technology cannot realize multi-person ECG signal reconstruction estimation of multiple targets at the same distance and different angles. Therefore, it is of great value to study the field of human electrocardiogram reconstruction under the environment of multiple targets. SUMMARY

[0004] The technical scheme adopted by the present application is: a multi-person electrocardiogram reconstruction method based on millimeter wave radar, which can effectively reconstruct the electrocardiogram information of multiple targets at the same distance and different angles.

[0005] The technical scheme adopted by the present application is: a multi-person electrocardiogram reconstruction method based on millimeter wave radar, which can effectively reconstruct the electrocardiogram information of multiple targets at the same distance and different angles.

[0006] S1. Perform FFT on the original radar echo data in the fast time dimension to obtain the time-range image;

[0007] S2, performing static clutter filtering on the time-range image obtained in step S1;

[0008] S3, extracting the phase signal from the time-range image after filtering out static clutter;

[0009] S4, performing blind source signal separation processing on the phase signal extracted in step S3 to obtain a multi-component signal;

[0010] S5. Reconstruct the electrocardiogram based on the multi-component signal obtained in step S4.

[0011] Beneficial effects of the present invention: The present invention proposes a multi-person electrocardiogram reconstruction method based on millimeter-wave radar, which can effectively obtain the distance information of multiple targets and perform effective phase information extraction of multiple targets. At the same time, the method can effectively separate the superposition of multi-target phase information and effectively reconstruct the electrocardiogram signal from the separated phase signal. Compared with the traditional heartbeat information analysis of multi-target signals, the present invention can provide more accurate electrocardiogram reconstruction information. The present invention can effectively reconstruct the electrocardiogram information of multiple targets at the same distance but different angles, ensuring the practical effect of millimeter-wave radar in the field of wireless medical care, and providing a new direction for medical monitoring and disease diagnosis. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] Figure 1 Flowchart for multi-target ECG reconstruction.

[0013] Figure 2 Schematic diagram of the network structure for ECG reconstruction.

[0014] Figure 3 Schematic diagram of target data collection in a specific implementation manner.

[0015] Figure 4 It is the radar echo range image in a specific implementation manner.

[0016] Figure 5 This is a static clutter filtered range image in a specific implementation manner.

[0017] Figure 6 It is a time domain diagram of multi-target phase information in a specific implementation manner.

[0018] Figure 7 This is a Fast-ICA result diagram in a specific implementation method.

[0019] Figure 8 is the final result image in a specific implementation manner. DETAILED DESCRIPTION

[0020] To facilitate those skilled in the art to understand the technical content of the present invention, the present invention is further explained below with reference to the accompanying drawings.

[0021] For multi-target ECG reconstruction scenarios such as Figure 3 As shown, the targets are lying flat in the middle of three horizontally placed beds, the millimeter-wave radar is located 1.5 meters above the middle target, and the distance between the two beds is 1.1 meters.

[0022] like Figure 1 As shown, the method of the present invention comprises the following steps:

[0023] Step 1: Time-distance image acquisition

[0024] Obtain the original bin data file collected by the millimeter wave radar and parse it at the same time, that is, decode the binary bin file to get the original radar echo data. Then perform Fourier transform (FFT) and calculate the absolute value of the ADC sampling point dimension. Finally, the time distance image of multiple targets is obtained as follows Figure 4 Specifically:

[0025] The raw radar echo data contains target distance information. By performing an FFT on the raw radar echo data in the fast time dimension, a spectrum positively correlated with distance, known as the time-range profile, can be obtained. Based on the relationship between frequency and target distance, the intermediate frequency signal frequency is converted to target distance, thereby obtaining target distance information. Specifically, the intermediate frequency signal of a single raw radar echo can be represented as:

[0026]

[0027] Where f0 is the carrier frequency, S is the frequency modulation slope, v is the target speed, R is the target distance, and c is the speed of light. Since the change caused by speed in the distance dimension is much smaller than the change caused by distance, assuming the target speed is 0, the above formula can be rewritten as:

[0028]

[0029] From the above formula, we can know that x(t) is the frequency The trigonometric function of x(t) is the intermediate frequency signal. Therefore, after the FFT transformation, the target distance R is related to the intermediate frequency signal frequency. There is a positive correlation.

[0030] Step 2: Static clutter filtering

[0031] Radar echo signals contain not only the target's breathing and heartbeat information, but also a large amount of static object clutter. This clutter is primarily caused by the radar's environment. The chest cavity movements caused by breathing and heartbeat have a certain velocity. Therefore, static clutter can be suppressed in the Doppler dimension to better extract human targets. Clutter suppression is achieved using mean cancellation. The distance from a stationary target to the radar antenna is constant, and the time delay of each stationary target is also constant. Averaging all echoes yields the time delay of the stationary target in the entire radar environment. This mean is then subtracted from each radar echo to obtain the echo signal of a moving target.

[0032] In step 1, the time distance image is obtained by using FFT, and then the average of each frame of the time distance matrix is ​​calculated. The time distance image after static clutter filtering is obtained by subtracting the average from the time distance matrix. Most of the static objects in the time distance image are filtered out, while the chest of multiple targets is in motion and therefore will not be filtered out. The result after static clutter filtering is as follows: Figure 5 shown.

[0033] Step 3: Phase extraction

[0034] The variance of the time range image after the static clutter processing in step 2 is calculated along the time dimension, and the maximum variance is selected as the range unit of the moving target. Then the phase amplitude value of the range unit is extracted and unwrapped. The final result is as follows: Figure 6 The specific process is as follows:

[0035] Since there are only human targets in the entire environment, and the human targets are in a stationary state. Therefore, the Doppler information of the target in this distance unit should be in a large variance value. By calculating the variance value of each distance unit of the time range image and selecting the distance unit with the largest variance, the distance unit where the target is located can be obtained. Since the phase signal is phase-wrapped, the phase signal needs to be unwrapped. That is, for the phase amplitude value θ(t) of a signal, since the phase is a periodic quantity, its range is usually [-π,π]. Suppose we have two phase amplitude values ​​at time points t1 and t2, θ(t1) and θ(t2), respectively, and the phase amplitude value difference △θ(t1, t2) = θ(t2)-θ(t1). If this difference is greater than π, it means that there is a jump in the phase signal, that is, phase wrapping. The following formula can be used to eliminate the phase wrapping:

[0036] △θ(t1,t2)=θ(t2)-θ(t1)+2πk′

[0037] Wherein, k′ is an integer.

[0038] Step 4: Blind Source Signal Separation

[0039] The phase signal obtained after step 3 is a two-dimensional matrix composed of phase signals with M channels and N cycles. Blind source signal separation is performed based on the results obtained in step 3, and the final signal components are as follows: Figure 7 shown.

[0040] The processing process is as follows:

[0041] Because multiple targets are located in the same range unit, their phase signals are superimposed within the same range unit. Therefore, to obtain phase information for different targets, blind source signal separation (BSS) can be used to separate the phase signals and obtain preliminary phase signals for multiple targets. The method proposed in this paper employs a Fast-ICA-based method for BSS. Specifically, each target is treated as a signal source, assuming there are M of them. The phase signals from M channels and N periods are concatenated into a two-dimensional matrix X. This matrix is ​​then used for centering.

[0042]

[0043] Where y[i,n] is an element in X, i = 0, 1, ... M, n = 0, 1, ... N. y[i,n] represents the two-dimensional data matrix obtained after the centering process. Then, the two-dimensional data matrix after the centering process needs to be whitened. First, calculate the covariance matrix of the data matrix:

[0044]

[0045] The superscript T represents the matrix transpose. Performing eigenvalue decomposition on the above covariance matrix yields:

[0046] S=UΣU T

[0047] U is an orthogonal matrix, ∑ is a diagonal matrix. The data is decorrelated and projected onto the principal component axis to obtain the signal matrix:

[0048] X'=U T X

[0049] Then to U T The data on each row vector in X is scaled by the standard deviation of the eigenvalue:

[0050]

[0051] After the above steps, X pThe two-dimensional matrix is whitened. Then the non-Gaussianity is maximized, so a coefficient matrix w is needed to maximize the Gaussianity of the target matrix. Generally, a non-linear function g(·) is used to approximate the maximization of the negative entropy, and the iterative update rule is as follows:

[0052]

[0053] Then the new weight vector w + is normalized:

[0054]

[0055] Then orthogonalization is performed and convergence is achieved. In the case of multiple dimensions, it is necessary to ensure that the estimated components are orthogonal to each other, so the Gram-Schmidt process is usually used. The update rule is as follows:

[0056]

[0057] w i represents a weight vector w + in the normalized result w i + is a vector in the weight vector w + ;

[0058] w i and w i + is less than a certain threshold 1e-6, if less than, it is considered that an independent component is found. Then all weight vectors W are processed S = W T X p , and the estimated independent component is obtained.

[0059] Step 5: Electrocardiogram reconstruction

[0060] After obtaining the multi-component signal S separated by Fast-ICA, electrocardiogram signal reconstruction is needed. First, according to the frequency of the separated multi-component signal, the signal components outside the heart rate frequency range are removed. At this time, the remaining signal components are combined into the target number of signal components according to the Pearson correlation as input (Input) into the Multi-SiNet network. The neural network structure used in this network is as shown in Figure 2 First, the input signal is converted into a frequency domain signal using FFT, then the top-k frequency components in the frequency domain signal are selected and converted into time domain signals for combination. Then it is sent to multiple MVCB modules for processing, and then a single linear layer and a convolution layer, i.e. SiBL (linear layer and convolution layer) module, are used to realize the reconstruction output of the electrocardiogram signal.

[0061] The signal is input into the MVCB module. It first undergoes layer normalization (LayNorm) to normalize the signal. Then, the temporal multi-head self-attention mechanism (Time-MHSA) is used to estimate the signal's temporal attention to achieve periodic correlations and thus extract the signal's periodic features. The correlation here is specifically analyzed using the Pearson correlation between the two signal components.

[0062] Since the signal is actually sparse in the frequency domain, it can be assumed that the signal's frequency domain satisfies the superposition of multiple Gaussian distributions. Therefore, the features output by the multi-head self-attention mechanism are fed into a multi-layer perceptron (MLP) for processing, thereby obtaining the mean μ and variance σ of the Gaussian distribution in the frequency domain. The mean and variance output by the multi-layer perceptron are used to form a Gaussian window. This Gaussian window is used to weight the signal's time domain information, thereby suppressing the signal's frequency domain clutter and enhancing the desired signal frequency components. The inverse Fourier transform (IFFT) is then used to transform it into the time domain.

[0063] The multiple signal components obtained in step 4 are screened and combined, and then sent to the neural network for training and prediction. The loss function is MSELoss. A total of 400 rounds of training are performed, and AdamW is used as the optimizer with a learning rate of 1e-4. The final results are as follows: Figure 8 As shown, it can be seen that the method of the present invention can effectively realize the reconstruction of multiple ECGs.

[0064] Those skilled in the art will appreciate that the embodiments described herein are intended to aid the reader in understanding the principles of the present invention, and it should be understood that the scope of the present invention is not limited to such specific descriptions and embodiments. Various modifications and variations are readily apparent to those skilled in the art. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention are intended to be included within the scope of the claims.

Claims

1. A multi-person electrocardiogram reconstruction method based on millimeter wave radar, characterized in that: include: S1. Perform FFT on the original radar echo data in the fast time dimension to obtain the time-range image; S2, performing static clutter filtering on the time-range image obtained in step S1; S3, extracting the phase signal from the time-range image after filtering out static clutter; S4, performing blind source signal separation processing on the phase signal extracted in step S3 to obtain a multi-component signal; S5. Reconstruct the electrocardiogram based on the multi-component signal obtained in step S4.

2. The multi-person electrocardiogram reconstruction method based on millimeter wave radar according to claim 1 is characterized in that: Step S5 The following steps are included: S51, removing signal components outside the heartbeat frequency range of the multi-component signal obtained in step S4; S52, converting the signal obtained after processing in step S51 into a frequency domain signal; S53, respectively select the top-k frequency components of the frequency domain signal obtained in step S52 and convert them into a time domain signal combination; S54, using a multi-head self-attention mechanism to perform time dimension attention estimation on the time domain signal combination obtained in step S53 to achieve periodic correlation of the signal, thereby extracting the periodic characteristics of the radar phase signal; S55, sending the periodic features extracted in step S54 to a multi-layer perceptron for processing, thereby obtaining the mean and variance of the Gaussian distribution in the frequency domain dimension; S56. Use the mean and variance of the multilayer perceptron output to form a Gaussian window; S57, using the Gaussian window formed in step S56 to weight the periodic characteristic time domain information of the radar phase signal; S58. The result obtained in step S57 is passed through a single linear layer and a convolutional layer to reconstruct and output the electrocardiogram signal.

3. The multi-person electrocardiogram reconstruction method based on millimeter wave radar according to claim 2 is characterized in that: In step S3, if the calculated phase amplitude difference between any two phase signals is greater than π, the two phase signals are unwrapped.

4. The multi-person electrocardiogram reconstruction method based on millimeter wave radar according to claim 3 is characterized in that: The unwinding calculation formula is: △θ(t1,t2)=θ(t2)-θ(t1)+2πk′ Wherein, θ(t1) and θ(t2) represent two phase signals whose phase amplitude difference is greater than π, and k′ is an integer.

5. The multi-person electrocardiogram reconstruction method based on millimeter wave radar according to claim 4 is characterized in that: In step S4, a Fast-ICA-based method is used to perform blind source signal separation.

Images