Multi-target vital sign monitoring method based on millimeter wave radar

Abstract

A multi-target vital sign monitoring method based on a millimeter-wave radar comprises the following steps: sending out a linear frequency modulation electromagnetic wave signal through the millimeter-wave radar, and constructing a three-dimensional data matrix according to a reflected signal of the linear frequency modulation electromagnetic wave signal; fourier transform and data processing are carried out on different dimensions of the data matrix to obtain distance, speed and angle information of a target and make a three-dimensional point cloud data set; clustering the generated point cloud data to obtain related information of a plurality of targets, and generating a two-dimensional motion state scatter diagram based on the clustered information; inputting the scatter diagram into a motion state classifier network model for motion state recognition; signals of the target in the motion intermission period are extracted, and the respiration rate and the heartbeat rate of the target are monitored. The method has the characteristics of good privacy protection, low time delay, non-contact property and the like, and multi-target vital signs can be monitored.

Description

Technical Field

[0001] The present invention belongs to the technical field of millimeter wave radar vital signs monitoring, and in particular relates to a multi-target vital signs monitoring method based on millimeter wave radar. Background Art

[0002] During physical exercise, vital sign monitoring plays an important role in protecting life safety and evaluating exercise. Existing human vital sign monitoring methods can be roughly divided into vision-based monitoring, wearable device-based monitoring, and radio frequency-based monitoring. Compared with vision-based monitoring, radio frequency-based vital sign monitoring does not require light and provides good privacy protection. In addition, radio frequency-based monitoring is more suitable for people who are inconvenient to wear equipment or have sensitive skin than wearable device-based monitoring. Radio frequency-based monitoring methods are currently mainly divided into two types: WiFi and millimeter wave radar. However, the deployment of WiFi devices is complex and easily interfered by the surrounding environment, resulting in unstable performance of vital sign monitoring. Vital sign monitoring based on millimeter wave radar has great advantages, high measurement accuracy, good privacy protection while ensuring non-contact, and low cost, which is more suitable for vital sign monitoring in daily life.

[0003] Although existing FMCW-based methods have many advantages, they are mostly suitable for single-target monitoring with small movements or require the target to remain stationary. In real life, vital sign monitoring may face complex and unstable situations, such as multi-person aerobic exercise. In these cases, the signal intensity of human limb movement will be large, resulting in the signal of small chest fluctuations caused by breathing and heartbeat being covered. Therefore, studying the vital sign monitoring method of multiple targets in motion scenarios is of great significance in the field of vital sign detection. Summary of the invention

[0004] In order to overcome the shortcomings of the prior art, the present invention proposes a multi-target vital signs monitoring method based on millimeter-wave radar, which realizes the vital signs monitoring of multiple targets in a motion scene and has the characteristics of low latency and high accuracy.

[0005] The technical solution adopted by the present invention is as follows:

[0006] A multi-target vital sign monitoring method based on millimeter wave radar, characterized in that it includes the following steps:

[0007] S1. The millimeter-wave radar emits a linear frequency-modulated electromagnetic wave signal, which is reflected by the target and then passes through a mixer to generate an intermediate frequency signal. The data is stored according to the antenna transmission and reception sequence to obtain a three-dimensional data matrix.

[0008] S2, by performing Fourier transform and data processing on different dimensions of the stored data matrix, the distance, speed, and angle information of the target are obtained, and a three-dimensional point cloud data set is constructed using these three pieces of information;

[0009] S3, clustering the generated point cloud data to obtain relevant information of multiple targets, and extracting data of each target based on the obtained information; compressing the three-dimensional point cloud data into a two-dimensional space on a distance-angle plane to obtain a two-dimensional motion state scatter plot;

[0010] S4, inputting the generated two-dimensional motion state scatter diagram into the trained motion state classifier network model, and identifying the motion state of the target as motion or motion interval through the classifier network model;

[0011] S5. When it is identified that at least one target is in a motion intermission period, extract the original signal of the target in the motion intermission period, and enhance the signal of the target using a weight vector enhancement method;

[0012] S6. Extract the phase information of the intermediate frequency signal after signal enhancement, perform bandpass filtering on the signal after phase unwrapping processing to extract the respiratory signal and heartbeat signal; then use the variational mode decomposition algorithm to decompose the respiratory signal and heartbeat signal to obtain signal components, and estimate the respiratory rate and heartbeat rate based on the obtained signal components.

[0013] Further, in step S1, the millimeter wave radar is placed facing the area to be detected, and the relevant parameters of the transmitted frequency modulated continuous wave are set. The transmitted electromagnetic wave signal is reflected by the target and then generates an intermediate frequency signal through a mixer; the collected raw data is stored according to the order of antenna transmission and reception, and the data is stored in N adc ×N chirp ×N rx Sequential storage, where N rx is the number of receiving channels of the antenna, and N adc ×N chirp is a frame of data, N chirp is the number of signals in each frame, N adc The number of data samples in each chirp signal.

[0014] Furthermore, in step S2, the data in each chirp in the received data is preprocessed to remove the static component in the signal, the mean value of the signal is calculated for the data in each chirp, and the mean signal is subtracted from the original signal to obtain the signal after the static component is removed;

[0015] The fast time dimension of the three-dimensional data matrix with the static component removed is N adcThe data dimension is fast Fourier transformed to obtain the distance information of the target. The slow time dimension of the data matrix is ​​N chirp Perform a fast Fourier transform on the data dimension to obtain the target's speed information. Perform a short-time Fourier transform on the channel number dimension of the data matrix to obtain the target's angle information;

[0016] The obtained distance, speed, and angle information are used as coordinate axes to construct a three-dimensional point cloud.

[0017] Furthermore, in step S3, in order to better identify and distinguish the information of multiple targets in the point cloud, the DBSCAN clustering algorithm is used to cluster the data in the point cloud, and each target data is extracted based on the obtained information; the three-dimensional point cloud data is compressed on the distance-angle plane to obtain a two-dimensional plane scatter plot.

[0018] In step S4, the motion state classifier network model includes a ShuffleNetV2 neural network and a memory module.

[0019] In step S5, when at least one target in the monitoring scene is in a motion interval, we extract the original signal of the target area based on the obtained target information, and enhance the signal using a weight vector enhancement method based on the location of the target.

[0020] In step S6, phase information is extracted from the enhanced signal. After phase unwrapping, a mixed signal of the breathing and heartbeat signals is obtained. Then, bandpass filtering is performed according to the normal breathing and heartbeat rates to obtain rough breathing signals and heartbeat signals respectively. The breathing and heartbeat signals are decomposed using a variational mode decomposition algorithm, and the most relevant signal components are selected. The fine-grained breathing rate and heartbeat rate are obtained by performing fast Fourier transform on the selected signal components.

[0021] The beneficial effects of the present invention are mainly manifested in:

[0022] (1) It can monitor the vital signs of multiple people in sports scenarios, and has the characteristics of good privacy protection, low latency, and non-contact.

[0023] (2) The classification network designed based on the ShuffleNetV2 neural network and memory module can achieve higher motion state recognition accuracy.

[0024] (3) While monitoring the vital signs of the target during the exercise interval, it also provides relevant support for the evaluation of the target's exercise effect. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] Figure 1It is a flow chart of the multi-target vital signs monitoring method based on millimeter wave radar of the present invention;

[0026] Figure 2 It is a schematic diagram of the motion state classifier network structure of the present invention. DETAILED DESCRIPTION

[0027] The present invention will be further described below in conjunction with the accompanying drawings.

[0028] Reference Figure 1 and Figure 2 , a multi-target vital sign monitoring method based on millimeter wave radar, comprising the following steps:

[0029] S1. The millimeter-wave radar emits a linear frequency-modulated electromagnetic wave signal, which is reflected by the target and then passes through a mixer to generate an intermediate frequency signal. The data is stored according to the antenna transmission and reception sequence to obtain a three-dimensional data matrix.

[0030] The process of step S1 is as follows:

[0031] Place the millimeter wave radar facing the area to be detected, set the relevant parameters of the transmitted frequency modulated continuous wave, and the transmitted electromagnetic wave signal is reflected by the target and then passes through the mixer to generate an intermediate frequency signal;

[0032] The collected raw data is stored in the order of antenna transmission and reception, with N adc ×N chirp ×N rx Sequential storage, where N rx is the number of receiving channels of the antenna, and N adc ×N chirp is a frame of data, N chirp is the number of signals in each frame, N adc is the number of data sampling points in each chirp signal;

[0033] S2, by performing Fourier transform and data processing on different dimensions of the stored data matrix, the distance, speed, and angle information of the target are obtained, and a three-dimensional point cloud data set is constructed using these three pieces of information;

[0034] The process of step S2 is as follows:

[0035] The data in each chirp in the received data is preprocessed to remove the static component in the signal. The mean value of the signal is calculated for the data in each chirp, and the mean signal is subtracted from the original signal to obtain the signal after removing the static component. The formula is as follows:

[0036]

[0037] in, is the signal after the static component is eliminated, y(t) is the original signal, and n is the number of sampling points of the signal;

[0038] The fast time dimension of the three-dimensional data matrix with the static component removed is N adc The data dimension is fast Fourier transformed to obtain the distance information of the target, and the slow time dimension of the data matrix, that is, N chirp The data dimension is subjected to fast Fourier transform to obtain the target's speed information, and the channel number dimension of the data matrix is ​​subjected to short-time Fourier transform to obtain the target's angle information;

[0039] Different from the traditional three-dimensional space coordinate system, the present invention uses the obtained distance, speed and angle information as coordinate axes to construct a three-dimensional point cloud.

[0040] S3, clustering the generated point cloud data to obtain relevant information of multiple targets, and extracting data of each target based on the obtained information, compressing the three-dimensional point cloud data into a two-dimensional space on a distance-angle plane, and obtaining a two-dimensional motion state scatter plot;

[0041] The process of step S3 is as follows:

[0042] In order to better identify and distinguish the information of multiple targets in the point cloud, the DBSCAN clustering algorithm is used to cluster the data in the point cloud. The advantage of the DBSCAN algorithm is that it can automatically determine the number of clusters and is more effective in dealing with noise points. In addition, it can handle irregularly shaped clusters and is suitable for data with large density differences. By performing DBSCAN clustering on the point cloud data, the clustered results can be extracted, and each target data can be extracted based on the obtained information.

[0043] Compress the three-dimensional point cloud data onto the distance-angle plane to obtain a two-dimensional plane scatter plot, that is, project the entire point cloud onto the distance-angle plane;

[0044] S4, inputting the generated two-dimensional motion state scatter plot into the trained motion state classifier network model, and identifying the motion state of the target as motion or motion interval (non-motion) through the classifier network model;

[0045] The process of step S4 is as follows:

[0046] In this embodiment, the two-dimensional motion state scatter diagram is classified by the designed motion state classifier network model, and the motion state recognition task is converted into an image classification task; Figure 2As shown, the motion state classifier network model is constructed by a MobileNet-V2 neural network and a memory module. This embodiment adds a memory module behind the MobileNet-V2 neural network model to more accurately realize the recognition of the motion state.

[0047] Specifically, we first construct a two-dimensional scatter plot of the motion state, and then input the scatter plot into the neural network ShuffleNetV2 for motion state recognition. Then, we associate the current sample with the previous sample through the memory module to correct the state recognition errors caused by short time, and output the final correct state recognition result.

[0048] Among them, ShuffleNetV2 is an efficient convolutional neural network proposed by the Megvii research team in 2018, designed for mobile devices with limited computing resources. Its core idea is to improve feature expression capabilities while reducing the amount of computation by designing two modules: Channel Shuffle and Grouped Convolution. Channel shuffle promotes feature interaction by disrupting the order of channels, while grouped convolution reduces the amount of computation and data transmission costs. Through these designs, ShuffleNetV2 achieves faster speed and lower computing cost while maintaining high accuracy, making it suitable for edge computing and real-time applications.

[0049] Furthermore, after obtaining the recognition result from the ShuffleNetV2 neural network, it is input into the memory module. The specific process is as follows: Figure 2 As shown. We first define the output of the ShuffleNetV2 neural network as the initial value X n , the actual output of the algorithm is defined as the recognition value R n Among them, X n It is the classification result of the current sample after passing through the ShuffleNetV2 network, X n-1 and X n-2 They are the classification results of the two samples before the current sample. By judging and comparing with the classification values ​​of the first two samples, the correlation between the current sample and the previous samples is improved, which can effectively correct the recognition errors and improve the accuracy of motion state recognition.

[0050] S5. When it is identified that at least one target is in a motion intermission period, extract the original signal of the target in the motion intermission period, and enhance the signal of the target using a weight vector enhancement method;

[0051] The process of step S5 is as follows:

[0052] When at least one target is identified as being in a non-moving state, the signal is enhanced using a weight vector enhancement method according to the location of the target and the corresponding angle. Specifically, in this embodiment, the focus of the signal is achieved by controlling the phase and amplitude of the antenna array;

[0053] The angle between the received signal and the vertical direction of the antenna is regarded as θ, l is the distance between the receiving antennas, τ is the signal reception delay of adjacent antennas, and the value of τ is calculated based on the set relationship:

[0054]

[0055] In this embodiment, the signal of the first receiving antenna is used as a reference signal, which is recorded as x(t). Since there is a time delay τ between two adjacent antennas, a row vector a(θ) can be written as:

[0056]

[0057] Then, the signal of each antenna can be expressed as:

[0058] X(t)=a(θ)×x(t)

[0059] In order to maximize the signal gain at the target angle θ, each antenna needs to be phase compensated in a specified direction. Let the weight vector w = a(θ) to obtain the output signal y(t):

[0060] y(t)=w×X(t)

[0061] Furthermore, signal enhancement of the target angular position can be achieved.

[0062] S6, extracting the phase information of the intermediate frequency signal after signal enhancement, performing bandpass filtering on the signal after phase unwrapping processing to extract the respiratory signal and the heartbeat signal, and then using the variational mode decomposition algorithm to decompose the respiratory signal and the heartbeat signal to obtain signal components, and estimating the respiratory rate and the heartbeat rate according to the obtained signal components;

[0063] The process of step S6 is as follows:

[0064] Phase information is extracted from the enhanced signal, and after phase unwrapping, we can obtain a mixed signal of the breathing and heartbeat signals; the breathing frequency of a person at rest is about 0.2-0.34 Hz, but the breathing frequency during exercise can reach up to 0.67 Hz, the heartbeat frequency at rest is 1-1.67 Hz, and the maximum value that can be reached during exercise is 2.67 Hz. In this embodiment, the filter bandpass frequency ranges for extracting the breathing signal and the heartbeat signal are set to 0.1-0.9 Hz and 1.0-3.0 Hz, respectively, and rough breathing signals and heartbeat signals are obtained after bandpass filtering.

[0065] The variational mode decomposition algorithm is used to decompose the respiratory and heartbeat signals. The main process of the variational mode decomposition algorithm is to search for the optimal solution of the variational model in an iterative manner in the process of obtaining the components, so as to determine the frequency center and bandwidth of each component, and finally the frequency domain of the signal can be adaptively divided and the components can be separated. In this embodiment, the number of signal components is set to 3. The most relevant signal component is selected from the decomposed signal components, and the fine-grained respiratory rate and heartbeat rate are obtained by fast Fourier transforming the selected signal components.

[0066] The embodiments described in this specification are merely an enumeration of the implementation forms of the inventive concept, and the description is relatively specific and detailed, but it cannot be understood as limiting the scope of the invention patent. It should be pointed out that for ordinary technicians in this field, several modifications and improvements can be made without departing from the concept of this application, which all belong to the protection scope of this application. Therefore, the protection scope of the patent of this application shall be based on the attached claims.

Claims

1. A multi-target vital signs monitoring method based on millimeter wave radar, characterized in that: The method comprises the following steps: S1. The millimeter-wave radar emits a linear frequency-modulated electromagnetic wave signal, which is reflected by the target and then passes through a mixer to generate an intermediate frequency signal. The data is stored according to the antenna transmission and reception sequence to obtain a three-dimensional data matrix. S2, by performing Fourier transform and data processing on different dimensions of the stored data matrix, the distance, speed, and angle information of the target are obtained, and a three-dimensional point cloud data set is constructed using these three pieces of information; S3, clustering the generated point cloud data to obtain relevant information of multiple targets, and extracting data of each target based on the obtained information; compressing the three-dimensional point cloud data into a two-dimensional space on a distance-angle plane to obtain a two-dimensional motion state scatter plot; S4, inputting the generated two-dimensional motion state scatter diagram into the trained motion state classifier network model, and identifying the motion state of the target as motion or motion interval through the classifier network model; S5. When it is identified that at least one target is in a motion intermission period, extract the original signal of the target in the motion intermission period, and enhance the signal of the target using a weight vector enhancement method; S6. Extract the phase information of the intermediate frequency signal after signal enhancement, perform bandpass filtering on the signal after phase unwrapping processing to extract the respiratory signal and heartbeat signal; then use the variational mode decomposition algorithm to decompose the respiratory signal and heartbeat signal to obtain signal components, and estimate the respiratory rate and heartbeat rate based on the obtained signal components.

2. The multi-target vital signs monitoring method based on millimeter wave radar according to claim 1 is characterized in that: In step S1, the millimeter wave radar is placed facing the area to be detected, and the relevant parameters of the transmitted frequency modulated continuous wave are set. The transmitted electromagnetic wave signal is reflected by the target and then generates an intermediate frequency signal through a mixer; the collected raw data is stored according to the antenna transmission and reception order, and the data is stored in N adc ×N chirp ×N rx Sequential storage, where N rx is the number of receiving channels of the antenna, and N adc ×N chirp is a frame of data, N chirp is the number of signals in each frame, N adc The number of data samples in each chirp signal.

3. The multi-target vital signs monitoring method based on millimeter wave radar according to claim 2 is characterized in that: In step S2, the data in each chirp in the received data is preprocessed to remove the static component in the signal, the mean value of the signal is calculated for the data in each chirp, and the mean value signal is subtracted from the original signal to obtain the signal after the static component is removed; The fast time dimension of the three-dimensional data matrix with the static component removed is N adc The data dimension is fast Fourier transformed to obtain the distance information of the target; the slow time dimension of the data matrix, that is, N chirp The data dimension is subjected to fast Fourier transform to obtain the speed information of the target; the channel number dimension of the data matrix is ​​subjected to short-time Fourier transform to obtain the angle information of the target; The obtained distance, speed, and angle information are used as coordinate axes to construct a three-dimensional point cloud.

4. The multi-target vital signs monitoring method based on millimeter wave radar according to claim 1 is characterized in that: In step S3, the DBSCAN clustering algorithm is used to cluster the data in the point cloud. Each target data is extracted based on the obtained information; The three-dimensional point cloud data is compressed onto the distance-angle plane to obtain a two-dimensional plane scatter plot.

5. The multi-target vital signs monitoring method based on millimeter wave radar according to claim 1 is characterized in that: In step S4, the motion state classifier network model includes a ShuffleNetV2 neural network and a memory module.

6. The multi-target vital signs monitoring method based on millimeter wave radar according to claim 1 is characterized in that: In step S5, when at least one target in the monitoring scene is in a motion interval, the original signal of the target area is extracted according to the obtained target information, and the signal is enhanced using a weight vector enhancement method according to the location of the target.

7. The multi-target vital signs monitoring method based on millimeter wave radar according to claim 1 is characterized in that: In step S6, the phase information is extracted from the enhanced signal, and after phase unwrapping, a mixed signal of the breathing and heartbeat signals is obtained, and then a bandpass filter is performed according to the normal breathing and heartbeat rates to obtain a rough breathing signal and a rough heartbeat signal respectively; The variational mode decomposition algorithm is used to decompose the respiratory and heartbeat signals, and the most relevant signal components are selected. The fine-grained respiratory rate and heartbeat rate are obtained by performing fast Fourier transform on the selected signal components.

Images