Non-contact electrocardiogram monitoring method using millimeter-wave radar

Abstract

The present disclosure provides a method for non-contact electrocardiogram monitoring using a millimeter-wave radar, which includes step S1 of transmitting a millimeter-wave signal to a measurement target using a millimeter-wave radar and receiving an echo signal; step S2 of performing signal processing on the received echo signal to extract cardiac mechanical activity data hidden in the echo signal; step S3 of building an end-to-end network architecture for the extracted cardiac mechanical activity data and completing cross-domain mapping from cardiac mechanical activity to cardiac electrical activity; and step S4 of inputting the cardiac mechanical activity data extracted at the current time based on a deep learning network architecture that has mastered the cross-domain mapping of cardiac mechanical activity and cardiac electrical activity, and outputting the ECG measurement result at the current time, and finally completing non-contact electrocardiogram monitoring.

Description

Technical Field

[0001] The present disclosure relates to the field of intelligent sensing technology, and in particular, to a non-contact electrocardiogram monitoring method using a millimeter-wave radar.

Background Art

[0002] Electrocardiogram (ECG) is one of the most important biomedical signals currently for describing heart activities, and provides basic information for the diagnosis of heart diseases. As is clear from experimental evidence, the incidence rate and harm of heart diseases can be significantly reduced by continuous monitoring, diagnosis, control and prevention through analysis. The conventional electrocardiogram monitoring method detects minute potential changes generated at different parts of the body due to the electrical activities caused by myocardial contractions by electrodes attached to the human skin. Electrocardiogram monitoring by electrodes is widely used in clinical diagnosis and daily prevention. However, due to the requirement of human body contact in the measurement process, there are still many limitations in actual use. For example, due to the foreign body sensation caused by the long-term attachment of the electrodes to the skin, patients are subjectively very resistant to long-term continuous monitoring. There are many limitations such as difficulty in attaching the electrodes to the skin for some patients with burns, infectious diseases and babies.

[0003] Since the heart activity process is completed by the functional mechanical movement of the heart, the mechanical activity of the heart itself is also related to the changes in heart electrical activities. Theoretically, the heart mechanical activity and the heart electrical activity belong to different domain mappings of information from the same source. With the development of millimeter-wave radar, due to its increasingly accurate spatial sensing ability, the millimeter-wave radar enables non-contact and accurate monitoring of heart mechanical activities. However, the reflection of electromagnetic waves on the human body is very complex, easily interfered, and the mechanical movement amplitude caused by the heart activity itself is very weak (usually in the range of about 0.2 - 0.5 millimeters). Therefore, it is always buried in other body movements with larger amplitudes (such as breathing). The conventional methods mainly realize the monitoring of coarse-grained heart activities such as the estimation of the human heart rate. Therefore, the performance of the current methods is very limited, and it is difficult to realize the accurate measurement of heart mechanical activities.

SUMMARY OF THE INVENTION

[0004] The present disclosure provides a non-contact electrocardiogram monitoring method using a millimeter-wave radar, including steps S1 - S3. Step S1: Transmit a millimeter-wave signal to a measurement target using a millimeter-wave radar and receive an echo signal. Step S2: Perform signal processing on the received echo signal to extract cardiac mechanical activity data hidden in the echo signal. Step S3: Construct an end-to-end network architecture for the extracted cardiac mechanical activity data to complete the cross-domain mapping from cardiac mechanical activity to cardiac electrical activity.

[0005] According to an embodiment of the present disclosure, step S2 includes sub-steps S21 - S25. Sub-step S21: Construct a virtual antenna array plane based on the physical arrangement of millimeter-wave radar antennas, construct a phase shift vector based on the antenna pitch and signal bandwidth in the virtual antenna array plane, calculate spatial beamforming, and complete the spatial domain filtering of the radar echo signal. Sub-step S22: Extract the phase for all spatial position signals filtered in the spatial domain, and extract the micro-motion signal for the phase. Sub-step S23: Perform a cardiac micro-motion relevance evaluation based on periodic template matching for the micro-motion signal every period time T to find the spatial positions related to cardiac micro-motion. Sub-step S24: Perform threshold screening on the evaluated micro-motion signals, retain the micro-motion signals exceeding the threshold, and remove the remaining micro-motion signals. Sub-step S25: Extract cardiac mechanical activity data for the retained micro-motion signals to complete the measurement of cardiac mechanical activity using a millimeter-wave radar.

[0006] According to an embodiment of the present disclosure, a smoothed Lanczos difference filter based on minimum variance of the phase is used to extract the micro-motion signal.

[0007] According to an embodiment of the present disclosure, spatial domain aggregation filtering by K-means clustering is performed on the retained micro-motion signals to extract cardiac mechanical activity data.

[0008] According to an embodiment of the present disclosure, step S3 includes sub-steps S31 - S33. Sub-step S31: Extract the time-domain features of the cardiac micro-motion data using a convolutional neural network. Sub-step S32: Perform position encoding on the spatially sparse time-domain features, and extract the spatial-domain features of the cardiac micro-motion data using a Transformer module with a multi-head attention mechanism. Sub-step S33: Perform element-wise multiplication on the time-domain and spatial-domain features of the cardiac micro-motion to complete the fusion and extraction of the deep features of cardiac activity.

[0009] According to an embodiment of the present disclosure, step S3 further includes sub-step S34 of modeling cardiac activity as a time autoregressive model,

Number

[0010] According to an embodiment of the present disclosure, a sequence-to-sequence decoder model is constructed using a temporal convolutional network, and by utilizing the dilated convolution characteristics in the temporal convolutional network, long-term memory of high-sampling-rate radar data by the decoder model is realized.

[0011] According to an embodiment of the present disclosure, step S3 further includes sub-step S35 of learning the mapping relationship between different domains using a loss function based on the L2 distance between the predicted ECG result and the actual ECG measurement result at each time. After training with a large amount of data, the deep learning network architecture can calculate the cross-domain mapping relationship between cardiac mechanical activity and cardiac electrical activity. ​

[0012] According to an embodiment of the present disclosure, the method for non-contact electrocardiogram monitoring using millimeter-wave radar further includes step S4 of inputting cardiac mechanical activity data extracted at a current time based on a deep learning network architecture that has mastered cross-domain mapping of cardiac mechanical activity and cardiac electrical activity, outputting ECG measurement results at a current time, and finally completing non-contact electrocardiogram monitoring.

[0013] The millimeter-wave radar non-contact electrocardiogram monitoring method provided by the present disclosure can complete stable, effective, and accurate measurement of the human heart's mechanical activity, realize cross-domain information mapping from the heart's mechanical activity to the heart's electrical activity, and is non-contact during measurement, which is safer and more convenient. This alleviates the inconvenience and drawbacks of ECG monitoring in the prior art, which relies entirely on contact measurement methods, and alleviates the technical problem of the prior art non-contact cardiac measurement technology, which can only monitor the human heart's cardiac activity at a coarse level (e.g., heart rate, heartbeat interval, etc.). [Brief explanation of the drawings]

[0014]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

DETAILED DESCRIPTION OF THE INVENTION

[0015] The present disclosure provides a non-contact electrocardiogram monitoring method using a millimeter-wave radar. The combination of the millimeter-wave radar and deep learning constructs a bridge for non-contact electrocardiogram monitoring. The technical core is how to stably, effectively, and accurately measure the mechanical activities of the human heart using a millimeter-wave radar, and how to design a deep learning network structure for radar sensing data and cardiac activity characteristics, excavate deep features, realize the mapping of cross-domain information, and finally complete the process from the radar observation description of cardiac mechanical activities to the electrocardiogram description of cardiac electrical activities. The conventional methods have not designed and realized an end-to-end cross-domain learning architecture for cardiac activities with respect to the characteristics of millimeter-wave radar data and complex cardiac physiological states.

[0016] To make the objectives, technical solutions, and advantages of the present disclosure more clear, the present disclosure will be described in further detail below with reference to the drawings and specific embodiments.

[0017] In an embodiment of the present disclosure, a non-contact electrocardiogram monitoring method using a millimeter-wave radar is provided. Based on FIGS. 1 and 7, the non-contact electrocardiogram monitoring method using the millimeter-wave radar includes: Step S1 of transmitting a millimeter-wave signal to a measurement target using a millimeter-wave radar and receiving an echo signal; Step S2 of performing signal processing on the received echo signal and extracting cardiac mechanical activity data hidden in the echo signal; Step S3 includes constructing an end-to-end network architecture for the extracted cardiac mechanical activity data and completing cross-domain mapping from cardiac mechanical activity to cardiac electrical activity.

[0018] According to an embodiment of the present disclosure, the measurement target is a human or other animal.

[0019] In the embodiment of the present disclosure, step S2 includes sub-steps S21 to S25.

[0020] Substep S21: Construct a virtual antenna array plane based on the physical layout of the millimeter-wave radar antenna, construct a phase shift vector based on the antenna pitch and signal bandwidth in the virtual antenna array plane, calculate spatial beamforming, and complete spatial domain filtering of the radar echo signal. As a result, as shown in Figure 2, it can be seen that after spatial domain filtering is performed on the radar received signal, the signal reflections at different spatial positions are successfully separated.

[0021] Substep S22: Extract the phase for all spatial position signals filtered in the spatial domain, and extract the micro-motion signal for the phase. As a result, as shown in Figure 3, the phase information of the position near the thoracic cavity indicates that the main movement tendency at that time comes from human respiratory activity, which has a larger amplitude.

[0022] Sub-step S23: A cardiac microtremor relevance evaluation based on period template matching is performed on the microtremor signal for each period time T to find the spatial position of the cardiac microtremor.

[0023] Substep S24: Threshold screening is performed on the evaluated microtremor signal, and excess microtremor signals are retained and the remaining microtremor signals are removed. As shown in Figure 4, the excess microtremor signal has had large amplitude motion interference removed. By comparing the R wave peak and T wave peak of the electrocardiogram at the corresponding time, it can be seen that the microtremor signal has the same periodicity as cardiac activity.

[0024] Substep S25: Extract cardiac mechanical activity data from the retained microtremor signal to complete the measurement of cardiac mechanical activity by millimeter wave radar.

[0025] More specifically, a virtual antenna array plane is constructed based on the physical layout of the millimeter-wave radar antenna, and a phase shift vector is constructed based on the antenna pitch and signal bandwidth in the virtual antenna array plane, and spatial beamforming is calculated to complete spatial domain filtering of the radar echo signal. The signal S(x,y,z,t) at time t at spatial position (x,y,z) is given by:

number

[0026] where N is the number of receive antennas, M is the number of transmit antennas, and y n,m,t is the received signal at time t in the channel defined by the nth receive antenna and the mth transmit antenna, k represents the rate of change of the transmit frequency, λ represents the transmit signal wavelength, r(x,y,z,n,m) represents the round trip distance from the mth transmit antenna to the target location (x,y,z) and back to the nth receive antenna, and c is the speed of light.

[0027] The phase is extracted for all spatial position signals after spatial domain filtering, and the microtremor signal is extracted using a smoothed Lanczos difference filter based on minimum variance for the phase.

[0028] For each cycle time T, cardiac microtremor signals are evaluated for relevance using periodic template matching to find the spatial location of cardiac microtremors. Then, threshold screening is performed on the evaluated microtremor signals to retain signals exceeding the threshold and remove the remaining signals. Spatial domain cohesive filtering based on K-means clustering is then performed on the retained microtremor signals to extract cardiac mechanical activity data, ultimately completing the measurement of cardiac mechanical activity using millimeter-wave radar.

[0029] In an embodiment of the present disclosure, step S3 includes sub-steps S31 - S33.

[0030] Sub-step S31: Extract the time-domain features of the cardiac micro-motion data using a convolutional neural network.

[0031] Sub-step S32: Perform positional encoding on the spatially sparse time-domain features, and extract the spatial-domain features of the cardiac micro-motion data using a Transformer module with a multi-head attention mechanism.

[0032] Sub-step S33: Perform element-wise multiplication on the time-domain and spatial-domain features of the cardiac micro-motion to complete the fusion and extraction of the deep features of the cardiac activity.

[0033] More specifically, regarding the time and space characteristics in the cardiac mechanical activity data, first, extract the time-domain features of the cardiac micro-motion data using a convolutional neural network. Then, perform positional encoding on the spatially sparse time-domain features, and extract the spatial-domain features of the cardiac micro-motion data using a Transformer module with a multi-head attention mechanism. Subsequently, perform element-wise multiplication on the time-domain and spatial-domain features of the cardiac micro-motion to complete the fusion and extraction of the deep features of the cardiac activity.

[0034] According to an embodiment of the present disclosure, step S3 further includes sub-step S34 of modeling the cardiac activity as a time autoregressive model.

Number

[0035] The ECG measurement result x at each time t is the ECG measurement results (x1,..., x t-1 ) at the historical times and the deep feature h of the cardiac activity at the current time tIt is a conditional probability distribution. A sequence-to-sequence decoder model is constructed using a temporal convolutional network, and by utilizing the dilated convolution characteristics in the temporal convolutional network, long-term memory of high-sampling-rate radar data by the decoder model is realized.

[0036] According to an embodiment of the present disclosure, step S3 further includes a sub-step S35 of learning a mapping relationship between different domains using a loss function based on the L2 distance for the ECG prediction result and the actual ECG measurement result (contact measurement) at each time, and after training with a large amount of data, the deep learning network architecture can calculate the cross-domain mapping relationship between cardiac mechanical activity and cardiac electrical activity.

[0037] In an embodiment of the present disclosure, as shown in FIG. 1, the non-contact electrocardiogram monitoring method using a millimeter-wave radar further includes a step S4 of inputting the cardiac mechanical activity data extracted at the current time based on the deep learning network architecture of the cross-domain mapping between the trained cardiac mechanical activity and cardiac electrical activity, outputting the ECG measurement result at the current time, and finally completing non-contact electrocardiogram monitoring.

[0038] In an embodiment of the present disclosure, when verification is performed, when the measurement object maintains a lying position on the bed and the radar is 0.5 m away (as shown in FIG. 7), the performance of non-contact electrocardiogram monitoring by the algorithm provided in the present application is shown. In the experiment, the radar is in an operating state of 3 transmitters and 4 receivers, the start frequency of the set signal is 78 GHz, the bandwidth is 4 GHz, the chirp interval is 45 μs, the frame rate is 200 Hz, and the sampling points are 256. A total of 36 people were tested, and the radar and corresponding contact electrocardiogram monitoring data in 3 minutes were collected each time, for a total of about 810 minutes. The analysis of non-contact electrocardiogram monitoring performance is shown in FIGS. 5 and 6.

[0039] The embodiments of the present disclosure have been described in detail with reference to the drawings. Note that any implementation forms not illustrated or described in the attached drawings or the main text of the specification are in forms known to those skilled in the art, and detailed descriptions thereof are omitted. In addition, the definitions of the above elements and methods are not limited to the various specific structures, shapes, or forms mentioned in the embodiments, and those skilled in the art can easily modify or replace them.

[0040] From the above description, those skilled in the art should clearly recognize the non-contact electrocardiogram monitoring method using a millimeter-wave radar according to the present disclosure.

[0041] As described above, the present disclosure provides a non-contact electrocardiogram monitoring method using a millimeter-wave radar, which is a non-contact electrocardiogram monitoring method for measuring cardiac mechanical activity by a millimeter-wave radar and cross-domain deep learning of cardiac electrical activity. The combination of the millimeter-wave radar and deep learning constructs a bridge for non-contact electrocardiogram monitoring, uses the millimeter-wave radar to stably, effectively, and accurately measure human cardiac mechanical activity, designs a deep learning network structure for radar sensing data and cardiac activity characteristics, excavates deep features, realizes cross-domain information mapping, and finally completes from the radar observation description of cardiac mechanical activity to the electrocardiogram description of cardiac electrical activity.

[0042] The specific embodiments of the present disclosure described above do not limit the protection scope of the present disclosure. Any other appropriate changes or modifications made based on the technical idea of the present disclosure should be included in the protection scope of the claims of the present disclosure.

Claims

1. A non-contact electrocardiogram monitoring method using a millimeter-wave radar, which is executed by a processor, comprising: Step S1 of transmitting a millimeter-wave signal to a measurement target using a millimeter-wave radar and receiving an echo signal; Step S2 of performing signal processing on the received echo signal and extracting cardiac mechanical activity data hidden in the echo signal; Step S3 of constructing an end-to-end network architecture for the extracted cardiac mechanical activity data and completing cross-domain mapping from cardiac mechanical activity to cardiac electrical activity; Said step S2 includes: Sub-step S21 of constructing a virtual antenna array plane based on the physical arrangement of millimeter-wave radar antennas, constructing a phase shift vector based on the antenna pitch and signal bandwidth on the virtual antenna array plane, calculating spatial beamforming, and completing spatial domain filtering of the radar echo signal; Sub-step S22 of extracting the phase for all spatial position signals filtered in the spatial domain and extracting micro-motion signals for the phase; Sub-step S23 of performing cardiac micro-motion correlation evaluation based on periodic template matching on the micro-motion signals every period time T and finding the spatial position related to cardiac micro-motion; Sub-step S24 of performing threshold screening on the evaluated micro-motion signals, retaining the micro-motion signals exceeding the threshold, and removing the remaining micro-motion signals; Sub-step S25 of extracting cardiac mechanical activity data for the retained micro-motion signals and completing the measurement of cardiac mechanical activity by the millimeter-wave radar. A non-contact electrocardiogram monitoring method using a millimeter-wave radar.

2. Extracting micro-motion signals using a Lanczos difference filter smoothed based on minimum variance in phase. The non-contact electrocardiogram monitoring method using a millimeter-wave radar according to Claim 1.

3. Performing spatial domain aggregation filtering by K-means clustering on the retained micro-motion signals to extract cardiac mechanical activity data. The non-contact electrocardiogram monitoring method using a millimeter-wave radar according to Claim 1.

4. Said step S3 includes: Sub-step S31 of extracting time-domain features of cardiac micro-motion data using a convolutional neural network. Performing position encoding on spatially sparse time-domain features and extracting spatial-domain features of cardiac micro-motion data using a Transformer module with a multi-head attention mechanism, sub-step S32; Performing element-wise multiplication on the time-domain and spatial-domain features of cardiac micro-motion to complete the fusion and extraction of deep cardiac activity features, sub-step S33, including The non-contact electrocardiogram monitoring method using a millimeter-wave radar according to claim 1.

5. The step S3 includes: After the sub-step S33, further including a sub-step S34 of modeling cardiac activity as a time autoregressive model. 【Number 1】 Here, the ECG measurement result \(x\) at each time ｔ is the conditional probability distribution of the ECG measurement results (\(x\) １ , …, \(x\) ｔ－１ ) at the historical times and the deep cardiac activity feature \(h\) at the current time ｔ . The non-contact electrocardiogram monitoring method using a millimeter-wave radar according to claim 4.

6. Constructing a sequence-to-sequence decoder model using a time convolutional network and realizing long-term memory of high-sampling-rate radar data by the decoder model using the dilated convolution characteristics in the time convolutional network. The non-contact electrocardiogram monitoring method using a millimeter-wave radar according to claim 4.

7. The step S3 includes: After the sub-step S33, further including a sub-step S35 of learning the mapping relationship between different domains using a loss function based on the L2 distance between the ECG prediction results and the actual ECG measurement results at each time, and after training with a large amount of data, the deep learning network architecture can calculate the cross-domain mapping relationship between cardiac mechanical activity and cardiac electrical activity. The non-contact electrocardiogram monitoring method using a millimeter-wave radar according to claim 4.

8. After the step S3, further including a step S4 of inputting the cardiac mechanical activity data extracted at the current time based on the deep learning network architecture that has acquired the cross-domain mapping between cardiac mechanical activity and cardiac electrical activity, outputting the ECG measurement results at the current time, and finally completing non-contact electrocardiogram monitoring. The non-contact electrocardiogram monitoring method using a millimeter-wave radar according to claim 1.

Images