Cardiac motion monitoring method and device based on millimeter wave radar

Abstract

The present invention provides a method and device for monitoring cardiac motion based on millimeter wave radar, the method comprising: operation S1: transmitting a millimeter wave signal to act on a target; operation S2: capturing a reflected signal reflected from the target; operation S3: processing the reflected signal to determine the target chest area; operation S4: extracting the body surface mechanical motion signal of the chest area; operation S5: removing redundant signals in the radar scanning space to obtain a cardiac mechanical motion signal; operation S6: constructing a neural network model according to the cardiac mechanical motion signal; and operation S7: under the guidance of ECG mode, inputting the cardiac mechanical motion signal into the neural network model to obtain a classification result of the cardiac mechanical motion signal to achieve cardiac motion monitoring.

Description

Technical Field

[0001] The present invention relates to the field of radar and radio wave detection technology, and in particular to a cardiac motion monitoring method based on millimeter wave radar. Background Art

[0002] Seismocardiogram (SCG) is a record of the mechanical activity of the human heart. It captures fine-grained cardiovascular events, such as the opening and closing of heart valves and the contraction and relaxation of ventricles. Today's standard method for measuring SCG signals relies on an accelerometer that captures micro-vibrations of the chest wall, a process that typically requires the user to take off their shirt, lie supine and remain quasi-still, and use a chest strap to fix the accelerometer near the apex of the heart. Therefore, measuring SCG today is still invasive and inconvenient, and usually needs to be managed by professionals in a calibrated environment or a controlled environment. Therefore, a more convenient, effective and non-invasive method for monitoring cardiac motion signals is urgently needed. Summary of the invention

[0003] In view of this, in order to at least partially solve at least one of the above-mentioned technical problems, the present invention provides a method and device for monitoring cardiac motion based on millimeter wave radar.

[0004] In order to achieve the above object, the technical solution of the present invention is as follows:

[0005] According to an embodiment of one aspect of the present invention, a method for monitoring cardiac motion based on millimeter wave radar is provided, including operations S1-S7. Operation S1: transmitting a millimeter wave signal to act on a target; operation S2: capturing a reflected signal reflected from the target; operation S3: processing the reflected signal to determine the target chest region; operation S4: extracting the body surface mechanical motion signal of the chest region; operation S5: removing redundant signals in the radar scanning space to obtain a cardiac mechanical motion signal; operation S6: constructing a neural network model based on the cardiac mechanical motion signal; and operation S7: under the guidance of the ECG modality, inputting the cardiac mechanical motion signal into the neural network model to obtain a classification result of the cardiac mechanical motion signal to achieve cardiac motion monitoring.

[0006] According to an embodiment of the present invention, operation S3 includes: operation S31: mapping the propagation distance of the reflected signal at different positions of the chest cavity into the phase change; operation S32: obtaining spatial beamforming by performing coherent accumulation operation on the received signals in the virtual channel composed of different transmitting antennas and receiving antennas, and restoring the reflected signals at all positions of the entire chest cavity.

[0007] According to an embodiment of the present invention, operation S4 includes: operations S41-S45. Operation S41: add all reflected signals and compress them to the distance dimension: operation S42: perform first-order difference on the obtained signal, then search for peaks on the first-order difference signal to find the location of the peak signal; operation S43: select the location where the first peak appears as the candidate signal center point, and the signals of the distance units adjacent to the center point in the distance dimension are also selected as candidate signals; operation S44: add all reflected signals and compress them to the horizontal and pitch angle dimensions, and select the point with the maximum energy as the mechanical motion area of ​​the heart; and operation S45: select the adjacent signals of the point with the maximum energy in the horizontal and pitch angles as the mechanical motion signal.

[0008] According to an embodiment of the present invention, operation S5 includes: removing respiratory interference by performing second-order difference processing on the mechanical motion signal of the chest region to obtain a cardiac mechanical motion signal.

[0009] According to an embodiment of the present invention, in operation S6, a deep neural network based on comparative knowledge distillation is used.

[0010] According to an embodiment of the present invention, operation S7 includes: operation S71: constructing an ECG feature encoder and a radar modal feature encoder respectively; operation S72: creating an ECG classifier and a radar signal classifier respectively after the ECG feature encoder and the radar modal feature encoder; and operation S73: the features obtained by the ECG classifier through label supervision ECG contain information about heart movement, and the radar signal classifier is used to output the final classification result.

[0011] According to an embodiment of the present invention, in operation S7, the ECG signal is input into the ECG encoder, and the processed radar reflection signal is input into the radar encoder. The features obtained from a group of ECG signals and radar reflection signals collected simultaneously are regarded as a pair of positive sample pairs, and the features obtained from the ECG signals and radar reflection signals collected non-simultaneously are regarded as a negative sample pair.

[0012] According to an embodiment of the present invention, in operation S7, the contrast loss is calculated based on the positive and negative sample pairs, the clustering knowledge of the ECG modality is transferred to the radar modality, and the MSE loss function is used to constrain the radar features to align with the ECG features.

[0013] Another aspect of the embodiments of the present invention provides a cardiac motion monitoring device based on millimeter wave radar using the above monitoring method, including: a transmitting module, a receiving module, a signal processing module, and a neural network model unit. The transmitting module is used to transmit millimeter wave signals to act on the target; the receiving module is used to capture the reflected signal reflected from the target; the signal processing module is used to process the reflected signal to determine the target chest area, and extract the body surface mechanical motion signal of the chest area, remove the redundant signals in the radar scanning space to obtain the cardiac mechanical motion signal; the neural network model unit is used to process the input cardiac mechanical motion signal under the guidance of the ECG mode to obtain the signal in the cardiac mechanical motion signal that exceeds the set threshold, so as to realize cardiac motion monitoring.

[0014] According to an embodiment of the present invention, when a cardiac mechanical motion signal exceeding a set threshold is acquired, a corresponding cardiac motion signal classification result is output. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] The above and other objects, features and advantages of the present invention will become more apparent through the following description of the embodiments of the present invention with reference to the accompanying drawings, in which:

[0016] Figure 1 The figure is a flow chart of a method for monitoring cardiac motion based on millimeter wave radar according to an embodiment of the present invention.

[0017] Figure 2 The present invention is an attempt to provide a cardiac motion monitoring device based on millimeter wave radar according to an embodiment of the present invention.

[0018] Figure 3 It is a schematic diagram of the extracted body surface mechanical motion signals and cardiac electrical signals in the chest area according to an embodiment of the present invention. DETAILED DESCRIPTION

[0019] The present invention provides a method and device for monitoring cardiac motion based on millimeter wave radar, which can transmit and receive signals based on millimeter wave radar, filter noise and capture fine cardiac mechanical motion through body surface reflection signals, and use the deep neural network architecture of contrast knowledge distillation to accurately identify mechanical motion patterns related to cardiac motion, thereby achieving accurate cardiac motion classification. The proposed method can provide all-weather cardiac motion monitoring without the use of wearable devices.

[0020] In order to make the objectives, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with specific embodiments and with reference to the accompanying drawings.

[0021] In an embodiment of the present invention, a method for monitoring cardiac motion based on millimeter wave radar is provided, such as Figure 1 As shown, the monitoring method comprises:

[0022] Operation S1: transmitting a millimeter wave signal to act on a target;

[0023] Operation S2: capturing a reflected signal reflected from a target;

[0024] Operation S3: Processing the reflected signal to determine the target chest area;

[0025] Operation S4: extracting a body surface mechanical motion signal of the chest area;

[0026] Operation S5: removing redundant signals in the radar scanning space to obtain a cardiac mechanical motion signal;

[0027] Operation S6: constructing a neural network model according to the cardiac mechanical motion signal; and

[0028] Operation S7: under the guidance of the ECG modality, input the cardiac mechanical motion signal into the neural network model to obtain a classification result of the cardiac mechanical motion signal to achieve cardiac motion monitoring.

[0029] According to an embodiment of the present invention, operation S3 includes: operation S31: mapping the propagation distance of the reflected signal at different positions of the chest cavity into the phase change; operation S32: obtaining spatial beamforming by performing coherent accumulation operation on the received signals in the virtual channel composed of different transmitting antennas and receiving antennas, and restoring the reflected signals at all positions of the entire chest cavity.

[0030] According to an embodiment of the present invention, operation S4 includes: operation S41: adding all reflected signals and compressing them to the distance dimension; operation S42: performing a first-order difference on the obtained signal, and then searching for peaks on the first-order difference signal to find the position of the peak signal; operation S43: selecting the position where the first peak appears as the candidate signal center point, and the signals of the distance units adjacent to the center point in the distance dimension are also selected as candidate signals; operation S44: adding all reflected signals, compressing them to the horizontal and pitch angle dimensions, and selecting the point with the maximum energy as the mechanical motion area of ​​the heart; and operation S45: selecting the adjacent signals of the point with the maximum energy in the horizontal and pitch angles as the mechanical motion signal.

[0031] According to an embodiment of the present invention, operation S5 includes: removing respiratory interference by performing second-order difference processing on the mechanical motion signal of the chest region to obtain a cardiac mechanical motion signal.

[0032] According to an embodiment of the present invention, in operation S6, a deep neural network based on comparative knowledge distillation is used.

[0033] According to an embodiment of the present invention, operation S7 includes: operation S71: constructing an ECG feature encoder and a radar modal feature encoder respectively; operation S72: creating an ECG classifier and a radar signal classifier respectively after the ECG feature encoder and the radar modal feature encoder; and operation S73: the features obtained by the ECG classifier through label supervision ECG contain information about heart movement, and the radar signal classifier is used to output the final classification result.

[0034] According to an embodiment of the present invention, in operation S7, the ECG signal is input into the ECG encoder, and the processed radar reflection signal is input into the radar encoder. The features obtained from a group of ECG signals and radar reflection signals collected simultaneously are regarded as a pair of positive sample pairs, and the features obtained from the ECG signals and radar reflection signals collected non-simultaneously are regarded as a negative sample pair.

[0035] According to an embodiment of the present invention, in operation S7, the contrast loss is calculated based on the positive and negative sample pairs, the clustering knowledge of the ECG modality is transferred to the radar modality, and the MSE loss function is used to constrain the radar features to align with the ECG features.

[0036] The above monitoring method can filter noise and capture fine cardiac mechanical motion through body surface reflection signals. A deep neural network architecture based on contrastive knowledge distillation is used to accurately identify cardiac mechanical motion patterns, thereby achieving accurate cardiac motion classification. The proposed monitoring method can provide all-weather cardiac motion monitoring without the use of wearable devices, and output cardiac motion classification monitoring results.

[0037] More specifically, the monitoring method includes: transmitting millimeter wave signals by using a millimeter wave radar and capturing the signals reflected from the human body. The received phase signal is processed, the area where the human chest cavity is located is determined according to the algorithm, and the mechanical movement of the heart in the corresponding area is extracted. It is specifically completed by the following steps: using the two-dimensional array antenna and frequency-modulated continuous wave system of the millimeter wave radar, the propagation distance of the reflected signal at different positions of the chest cavity is mapped to the phase change, and finally the spatial beamforming is calculated by performing coherent accumulation operation on the received signals in the virtual channel composed of different transmitting antennas and receiving antennas, and the reflected signals at all positions of the entire chest are restored:

[0038] ;

[0039] Where N represents the number of virtual channels composed of different transmitting antennas and receiving antennas, y n,t represents the received signal of virtual channel n at time t, T represents the total time of receiving the signal, k represents the rate of change of the transmission frequency, λ represents the wavelength of the transmitted signal, r(x,y,z,n) represents the round-trip propagation distance of the reflected signal at the three-dimensional spatial position (x,y,z) to the virtual channel n, and c is the speed of light.

[0040] In order to remove redundant signals in the radar scanning space and accurately locate the chest position where the human heartbeat signal is the strongest, the present invention proposes a signal screening method, firstly, all signals are added and compressed to the distance dimension: the obtained signal is subjected to first-order difference, and then the first-order difference signal is peaked to find the location of the peak signal. Since the radar is placed directly opposite the human chest and there is no object blocking the radar and the human body, we select the location where the first peak appears as the center point of the candidate signal, and the distance units adjacent to the center point in the distance dimension (for example, the vertical direction) are also selected as candidate signals; all reflected signals are added again, compressed to the horizontal and pitch angle dimensions, and the point with the maximum energy is selected as the area of ​​the mechanical motion of the heart. The adjacent signals of the maximum energy point in the horizontal and pitch angles are selected as the final mechanical motion signal of the heart. In order to highlight the surface motion caused by the mechanical motion of the heart and remove respiratory interference, we finally perform second-order difference processing on the signal.

[0041] The extracted surface cardiac vibration signal is used to construct a deep neural network based on contrast knowledge distillation. Under the guidance of the ECG modality, the abnormal cardiac motion signal in the radar signal is extracted. Specifically, it can include: constructing feature encoders for ECG and radar modalities respectively. The ECG signal is input into the ECG encoder, and the processed radar signal is input into the radar encoder. The features obtained from a set of ECG and radar signals collected simultaneously are regarded as a pair of positive samples, and the features obtained from other non-simultaneous ECG and radar signals are regarded as negative sample pairs. The contrast loss is calculated based on the positive and negative sample pairs. The purpose of this step is to transfer the clustering knowledge of the ECG modality to the radar modality. At the same time, the MSE loss function is used to constrain the radar features to align with the ECG features. A classifier is created after the encoders of the ECG and radar modalities respectively. The features obtained by label supervision of ECG contain abnormal information, and the radar signal classifier is used to output the final classification result. The output probability distribution of the ECG classifier and the radar classifier is constrained by the KL divergence (Kullback-Leibler divergence). Since the proportion of abnormal cardiac mechanical motion samples in the data set is very small, there is a sample imbalance problem. During model training, a method of adjusting the classifier output logical value based on label frequency is used to alleviate the sample imbalance problem. Based on the deep neural network architecture that has learned the mapping between the cardiac mechanical motion on the human body surface and the classification relationship, the cardiac mechanical motion signal on the human chest surface is input and the classification result is output.

[0042] According to an embodiment of the present invention, a cardiac motion monitoring device based on millimeter wave radar is also provided. Figure 2 As shown, the monitoring device comprises:

[0043] A transmitting module is used to transmit millimeter wave signals to act on the target;

[0044] Receiving module: used to capture the reflected signal reflected from the target;

[0045] A signal processing module, used to process the reflected signal to determine the target chest area, extract the body surface mechanical motion signal of the chest area, remove the redundant signal in the radar scanning space to obtain the heart mechanical motion signal; and

[0046] The neural network model unit is used to process the input cardiac mechanical motion signal under the guidance of the ECG modality to obtain the classification result of the cardiac mechanical motion signal and realize cardiac motion monitoring. For example, when the cardiac mechanical motion signal exceeding the set threshold is obtained, the cardiac motion is judged to be abnormal and the abnormal classification result is output.

[0047] When conducting experimental verification, the target to be measured lies flat on the bed and maintains a quasi-static state. The millimeter wave radar is set at 0.5-0.6m above the human chest, and the target's cardiac movement is monitored by the monitoring device and monitoring method proposed in the present invention. In this embodiment of the present invention, the radar is in a 3-transmit 4-receive working state, and the starting frequency of the signal is set to 77GHz, the bandwidth is 4GHz, the frame rate is 100Hz, and the ECG device is 100Hz. In the experiment, the following F1 scores, sensitivity, and specificity are mainly used to evaluate the performance of cardiac motion monitoring. The F1 score expression is The sensitivity expression is , the specific expression is , where TP is the true positive sample, that is, the number of samples predicted by the classifier as positive samples and actually positive samples; FP is the false positive sample, that is, the number of samples predicted by the classifier as positive samples and actually negative samples; TN is the true negative sample, that is, the number of samples predicted by the classifier as negative samples and actually negative samples; FN is the false negative sample, that is, the number of samples predicted by the classifier as negative samples and actually positive samples. Figure 3 As shown, the monitoring method and monitoring device of the present invention can effectively extract the mechanical motion signal of the heart on the human body surface. For sample A, its cardiac electrical signal and mechanical motion signal are both normal. For sample B, its mechanical motion signal is abnormal, and the cardiac electrical signal also shows corresponding abnormalities (premature beats).

[0048] More experiments were conducted based on the cardiac motion monitoring method and device based on millimeter wave radar of the present invention, and a total of 22,464 10-second cardiac mechanical motion signal sequences were obtained from 1,415 test samples. The specific statistical data are shown in Table 1 below:

[0049] Table 1

[0050] Accuracy F1 score Sensitivity Specificity True negative example False Positive False Negative True example result 98.65% 91.32% 92.59% 99.15% 1296 11 8 100

[0051] As can be seen from Table 1 above, for these 1415 subjects in the clinical environment, the cardiac motion monitoring method and device based on millimeter wave radar of the present invention can show good classification performance, with an accuracy rate of up to 98.65%, and the accuracy evaluation index F1 score reflecting data imbalance is 91.32%, and both sensitivity and specificity reach a high level.

[0052] So far, the embodiments of the present invention have been described in detail in conjunction with the accompanying drawings. It should be noted that the implementation methods not shown or described in the drawings or the body of the specification are all forms known to ordinary technicians in the relevant technical field and are not described in detail. In addition, the above definitions of each element and method are not limited to the various specific structures, shapes or methods mentioned in the embodiments, and ordinary technicians in the field can simply change or replace them.

[0053] Based on the above description, those skilled in the art should have a clear understanding of the method and device for monitoring cardiac motion based on millimeter wave radar of the present invention.

[0054] In summary, the present invention provides a method and device for monitoring cardiac motion based on millimeter wave radar, which can realize non-contact real-time monitoring of cardiac mechanical motion and output classification results including normal and various abnormal conditions.

[0055] In addition, unless the steps are specifically described or must occur in sequence, the order of the above steps is not limited to the above list, and can be changed or rearranged according to the required design. And the above embodiments can be mixed and matched with each other or with other embodiments based on design and reliability considerations, that is, the technical features in different embodiments can be freely combined to form more embodiments.

[0056] The specific embodiments described above further illustrate the objectives, technical solutions and beneficial effects of the present invention in detail. It should be understood that the above description is only a specific embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A method for monitoring cardiac motion based on millimeter wave radar, characterized in that: include: Operation S1: transmitting a millimeter wave signal to act on a target; Operation S2: capturing a reflected signal reflected from a target; Operation S3: Processing the reflected signal to determine the target chest area; Operation S4: extracting a body surface mechanical motion signal of the chest area; Operation S5: removing redundant signals in the radar scanning space to obtain a cardiac mechanical motion signal; Operation S6: constructing a neural network model according to the cardiac mechanical motion signal; and Operation S7: under the guidance of the ECG modality, inputting the cardiac mechanical motion signal into the neural network model to obtain a classification result of the cardiac mechanical motion signal to achieve cardiac motion monitoring; Operation S7 includes: Operation S71: constructing an ECG feature encoder and a radar modality feature encoder respectively; Operation S72: creating an ECG classifier and a radar signal classifier respectively after the ECG feature encoder and the radar modality feature encoder; and Operation S73: the ECG classifier uses label supervision to obtain features of the ECG that contain information about heart movement, and the radar signal classifier is used to output a final classification result.

2. The cardiac motion monitoring method according to claim 1, characterized in that: Operation S3 includes: Operation S31: mapping the propagation distances of the reflected signals at different positions of the chest cavity into phase changes; Operation S32: spatial beamforming is obtained by performing coherent accumulation operation on the received signals in the virtual channels formed by different transmitting antennas and receiving antennas, so as to restore the reflected signals at all positions of the entire chest cavity.

3. The cardiac motion monitoring method according to claim 1, characterized in that: Operation S4 includes: Operation S41: adding all reflected signals and compressing them to the distance dimension; Operation S42: performing a first-order difference on the obtained signal, and then performing peak search on the first-order difference signal to find the position of the peak signal; Operation S43: selecting the position where the first peak appears as the candidate signal center point, and the signals of the distance units adjacent to the center point in the distance dimension are also selected as candidate signals; Operation S44: adding all reflected signals, compressing them into horizontal and elevation angle dimensions, and selecting the point with maximum energy as the cardiac mechanical motion area; and Operation S45: selecting adjacent signals of the maximum energy points at the horizontal and pitch angles as mechanical motion signals.

4. The cardiac motion monitoring method according to claim 3, characterized in that: Operation S5 includes: removing respiratory interference by performing second-order difference processing on the mechanical motion signal of the chest area to obtain a cardiac mechanical motion signal.

5. The cardiac motion monitoring method according to claim 1, characterized in that: In operation S6, a deep neural network based on contrastive knowledge distillation is used.

6. The cardiac motion monitoring method according to claim 1, characterized in that: In operation S7, the ECG signal is input into the ECG encoder, and the processed radar reflection signal is input into the radar encoder. The features obtained from a set of ECG signals and radar reflection signals collected simultaneously are regarded as a pair of positive samples, and the features obtained from ECG signals and radar reflection signals collected non-simultaneously are regarded as a negative sample pair.

7. The cardiac motion monitoring method according to claim 6, characterized in that: In operation S7, the contrast loss is calculated based on the positive and negative sample pairs, the clustering knowledge of the ECG modality is transferred to the radar modality, and the MSE loss function is used to constrain the radar features to align with the ECG features.

8. A cardiac motion monitoring device based on millimeter wave radar using the monitoring method of claim 1, characterized in that: include: A transmitting module is used to transmit millimeter wave signals to act on the target; Receiving module: used to capture the reflected signal reflected from the target; The signal processing module is used to process the reflected signal to determine the target chest area, extract the body surface mechanical motion signal of the chest area, and remove the redundant signals in the radar scanning space to obtain the heart mechanical motion signal; as well as The neural network model unit is used to process the input cardiac mechanical motion signal under the guidance of the ECG modality to obtain the cardiac mechanical motion signal classification result and realize cardiac motion monitoring.

9. The cardiac motion monitoring device according to claim 8, characterized in that: When a cardiac mechanical motion signal exceeding a set threshold is obtained, a corresponding cardiac motion signal classification result is output.

Images