A method for monitoring human vital signs in sports scenarios

Abstract

This invention discloses a method for monitoring human vital signs in motion scenarios. It receives radar reflection signals reflected from the human body, mixes these signals with the transmitted radar signal to obtain a raw intermediate frequency (IF) signal, performs a Fourier transform on the IF signal to construct feature spectrum data, and uses a sliding window sampling method to extract the feature spectrum according to a preset window length and step size. This extracted spectrum is then input into a trained classifier network model to identify motion or non-motion states, and the type of motion within the motion state. During non-motion states within the motion period, it monitors respiratory or heart rate. This invention utilizes millimeter-wave radar to monitor human vital signs, offering advantages such as low light dependence, good privacy protection, high monitoring sensitivity, and non-contact operation.

Description

Technical Field

[0001] This application belongs to the field of vital sign monitoring technology, specifically involving a method for monitoring human vital signs in sports scenarios. Background Technology

[0002] Vital sign monitoring plays a crucial role in assessing exercise effectiveness and ensuring the safety of athletes. Rehabilitation patients are prone to danger during exercise, and vital signs such as heart rate and respiration are important indicators for predicting such risks. Traditional vital sign monitoring mainly relies on wearable devices, which is cumbersome and inconvenient. Existing contactless monitoring methods, such as WiFi, are inferior in accuracy and sensitivity.

[0003] Radar technology offers significant advantages in vital sign monitoring, such as being wearable, unaffected by ambient light, and providing good privacy protection. However, existing radar-based vital sign monitoring methods often require the human body to be stationary or allow only slight movement, making them unsuitable for motion scenarios involving large limb movements. While motion state recognition can help select samples for vital sign monitoring, current research typically only includes posture recognition and trajectory tracking. The former lacks sufficient accuracy in recognizing continuous movements, while the latter lacks measurement of motion information, thus making it unsuitable for long-duration, multi-type motion scenarios.

[0004] Therefore, improving the ability to recognize motion states and thereby detect vital signs in motion scenarios has become an urgent problem to be solved. Summary of the Invention

[0005] The purpose of this application is to propose a method for monitoring human vital signs in motion scenarios, in order to overcome the shortcomings of the existing technologies, improve the feasibility of monitoring, and achieve high-precision, low-latency, and universal monitoring of vital signs in motion based on FMCW millimeter-wave radar.

[0006] To achieve the above objectives, the technical solution of this application is as follows:

[0007] A method for monitoring human vital signs in sports scenarios includes:

[0008] The radar reflected signal after being reflected by the human body is received, and the radar reflected signal is mixed with the radar transmitted signal to obtain the original intermediate frequency signal.

[0009] The original intermediate frequency signal is subjected to Fourier transform to extract the target's position and velocity information. The dominant velocity information of each position is arranged together to calculate a heat map and construct characteristic spectrum data.

[0010] The generated feature spectrograms are sampled using a sliding window method, and the feature spectrograms are truncated according to the preset window length and step size. The sampled spectrograms are then input into the trained classifier network model to identify whether the state is moving or not, and the type of movement in the moving state.

[0011] When the target is identified as being in a non-moving state, the phase information of the original intermediate frequency signal is extracted, and after unwinding, bandpass filtering is performed according to the different frequencies of heartbeat and respiration to obtain the respiration signal and heartbeat signal. Then, the variational mode decomposition algorithm is used to separate the bandpass filtered respiration signal and heartbeat signal into component signals.

[0012] The component signal with the highest correlation coefficient to the bandpass filtered respiratory or heartbeat signal is selected using the Pearson correlation coefficient as the corresponding respiratory or heartbeat signal. The frequency of the respiratory or heartbeat signal is calculated by Fourier transform, and finally the respiratory frequency and heartbeat frequency are obtained.

[0013] Furthermore, the step of performing a Fourier transform on the original intermediate frequency signal to extract the target's position and velocity information, arranging the dominant velocity information at each position together, calculating a heatmap, and constructing characteristic spectrum data includes:

[0014] The original intermediate frequency signal is divided into individual chirped signals according to the sampling frequency. A preset number of chirped signals are combined into a frame. A fast Fourier transform is performed on each chirped signal to obtain distance data.

[0015] Using the distance data corresponding to each chirp signal as a column, arrange the distance data corresponding to all chirp signals in a frame together to form a matrix, and perform a fast Fourier transform on each row of the matrix to obtain velocity data;

[0016] Within a frame, the velocity with the largest absolute velocity value is selected from each distance as the dominant velocity at that distance;

[0017] The dominant velocities at each distance in all frames within a preset time period are arranged in chronological order, and a heatmap is calculated to form a characteristic spectrum map.

[0018] Furthermore, the classifier network model includes residual neural networks and recurrent neural networks.

[0019] Furthermore, the component signal with the highest correlation coefficient to the bandpass-filtered respiratory signal or heartbeat signal is selected as the corresponding respiratory signal or heartbeat signal using the Pearson correlation coefficient. The correlation coefficient is calculated using the following formula:

[0020]

[0021] Where, ρ sign,imfThe correlation coefficient between the bandpass filtered respiratory or heartbeat signal *sign* and its component signal *imf* is represented by *cov(sign,imf)*, where *cov(sign,imf)* represents the covariance between the two signals, and σ is the coefficient of correlation between the two signals. sign σ imf These represent the standard deviations of the bandpass filtered respiratory or heartbeat signal and its component signals, respectively.

[0022] The proposed method for monitoring human vital signs in motion scenarios has the following advantages compared with existing technologies:

[0023] Monitoring human vital signs using millimeter-wave radar has advantages such as low light dependence, good privacy protection, high monitoring sensitivity, and non-contact operation.

[0024] This device monitors respiratory or heart rate during non-exercise periods, enabling vital sign monitoring in exercise scenarios. It boasts advantages such as strong anti-interference capabilities, wide applicability, and convenient and quick device deployment.

[0025] A classification network composed of residual neural networks and recurrent neural networks is used to improve the sensitivity and accuracy of motion state classification.

[0026] Monitoring based on frequency-modulated continuous waves from radar is safe and harmless to the human body. Attached Figure Description

[0027] Figure 1 This is a flowchart of the method for monitoring human vital signs in sports scenarios according to this application;

[0028] Figure 2 This is a schematic diagram illustrating the application scenario of this application;

[0029] Figure 3 This is a schematic diagram of the Fourier transform in an embodiment of this application;

[0030] Figure 4 This is a diagram showing the target distance detection results for this application;

[0031] Figure 5 This is a structural diagram of the classifier network model in an embodiment of this application;

[0032] Figure 6 This is a diagram showing the calculated original phase and the phase after unwinding in this invention;

[0033] Figure 7 This is a diagram showing the result of variational mode decomposition in this invention. Detailed Implementation

[0034] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0035] In one embodiment, such as Figure 1 As shown, a method for monitoring human vital signs in motion scenarios is provided, including:

[0036] Step S1: Receive the radar reflection signal reflected by the human body, and mix the radar reflection signal with the radar transmission signal to obtain the original intermediate frequency signal.

[0037] In application scenarios, millimeter-wave radar is placed in living rooms or other environments where people move around, with the radar's integrated antenna board facing directly at the front of the person. The millimeter-wave radar module contains three transmitting units and four receiving units, with a usable frequency of 60GHz-64GHz. The three transmitting antennas emit continuous wave signals that have undergone sawtooth linear frequency modulation, which are then received by the four receiving antennas after being reflected from the human body surface and the environment.

[0038] like Figure 2 As shown, the received radar reflection signal and the transmitted signal are then mixed, low-pass filtered, and converted from digital to analog to obtain the original intermediate frequency (IF) signal data, which includes breathing signals, heartbeat signals, and some interference signals. The original IF signal is represented as shown in formula (1):

[0039]

[0040] Where A represents the signal amplitude, f0 represents the initial frequency, τ represents the signal delay, and k represents the signal frequency rise slope. t represents the time variable, and j represents the imaginary part of the complex signal.

[0041] In one specific embodiment, different application scenarios have different performance requirements for the radar. By adjusting parameters such as the number of radar antennas, starting frequency, rise slope, chirp duration, sampling rate, and number of chirps per frame, the maximum detection range of the radar is set to 5.6m, the range resolution is 3.75cm, the maximum detectable speed is 10m / s, and the range resolution is 7.7cm / s. This setting basically meets the requirements for human motion scenarios. The obtained raw intermediate frequency signal contains the motion state information, vital sign information, and interference information of the person moving.

[0042] Step S2: Perform Fourier transform on the original intermediate frequency signal to extract the target's position and velocity information, arrange the dominant velocity information of each position together, calculate the heat map, and construct characteristic spectrum data.

[0043] In one specific embodiment, this application performs a Fourier transform on the original intermediate frequency signal to extract the target's position and velocity information, and arranges the dominant velocity information of each position together to calculate a heatmap and construct characteristic spectrum data, including:

[0044] Step S2.1: Divide the original intermediate frequency signal into individual chirped signals according to the sampling frequency, combine a preset number of chirped signals into a frame, and perform a fast Fourier transform on each chirped signal to obtain distance data.

[0045] Radar signals are not transmitted continuously, but in segments, each segment called a chirp. A fixed number of chirps constitute a frame, which serves as the observation window for radar processing. Since the raw data acquired is continuous, it needs to be segmented according to the signal sampling frequency. Specifically, the raw intermediate frequency (IF) signal is segmented into individual chirps based on the sampling frequency; that is, the sampling frequency f and the number of chirps N in each frame are first determined. f In the original data, every f data points represent the same chirp signal; every N... f The data in each chirp signal is data within the same frame. Chirps and frames are well-established technologies in this field and will not be elaborated upon here.

[0046] like Figure 3 As shown, the chirp signals within a frame are arranged as follows: Figure 3 The first matrix in the array, with each column containing a chirp signal, N f N represents the number of chirp signals within a frame. c This indicates the number of samples for a single chirp signal.

[0047] A fast Fourier transform is performed on each chirp signal data to obtain the signal frequency information, and then the target distance information is calculated. The distance calculation method is shown in formula (2):

[0048]

[0049] Where c represents the speed of light, f IF Indicates frequency.

[0050] The calculated distance curve is shown below. Figure 4 As shown, the signal strength is represented at different distances, with the peak of the curve indicating the location of the human body. For example, in... Figure 4 In this context, the distance from the main surface of the human body to the radar is 1-2m.

[0051] Step S2.2: Using the distance data corresponding to each chirp signal as a column, arrange the distance data corresponding to all chirp signals in a frame together to form a matrix, and perform a fast Fourier transform on each row of the matrix to obtain the velocity data.

[0052] like Figure 3 As shown, the distance data corresponding to a single chirp signal within a frame is treated as a column and arranged in sequence. Figure 3 The second matrix in the matrix. Then, a Fast Fourier Transform is performed on each row of this matrix to obtain velocity information at different distances, such as... Figure 3 The third matrix is ​​shown in the figure. The velocity calculation method is shown in formula (3):

[0053]

[0054] Where λ is the signal wavelength, ω is the phase difference between different chirps, and T c This is the chirp interval time.

[0055] Step S2.3: Within a frame, select the velocity with the largest absolute velocity value from each distance as the dominant velocity at that distance.

[0056] This step calculates the distance information and velocity information of each distance point within a frame. Then, it selects the velocity value with the largest absolute velocity value at each distance point as the dominant velocity at that distance point, thus forming the distance-dominant velocity information of a frame of data.

[0057] Step S2.4: Arrange the dominant velocities of each distance in all frames within the preset time period in chronological order, calculate the heat map, and form a characteristic spectrum map.

[0058] Within a preset time period, the distance-dominant velocity information of multiple frames within that period is calculated and arranged sequentially along the time axis to form a matrix. The distance-velocity information, arranged chronologically, becomes a two-dimensional matrix. The `heatmap` function in MATLAB can directly convert this numerical matrix into a heatmap, where larger numbers correspond to darker colors. This embodiment uses the `heatmap` function to calculate the heatmap of this matrix, obtaining the distance-dominant velocity characteristic spectrum of the motion within that time period.

[0059] Step S3: The generated feature spectrum is sampled using a sliding window method. The feature spectrum is truncated according to the preset window length and step size, and then input into the trained classifier network model to identify the motion or non-motion state, as well as the motion type in the motion state.

[0060] This embodiment uses a trained classifier network model to classify the feature spectrum, identifying motion or non-motion states, and the type of motion within the motion state. For example... Figure 5 As shown, the classifier network model includes a residual neural network and a recurrent neural network (RNN). The residual neural network consists of convolutional layers, pooling layers, multiple residual blocks, and fully connected layers; the recurrent neural network can employ a Long Short-Term Memory (LSTM) network. The residual neural network classifier has excellent image recognition performance, while the recurrent neural network can combine data from before and after a time series for prediction. Since the movements of a person are continuous, combining the two can give the classifier a better ability to identify motion states. Feature spectrum samples are first transformed in dimension through convolutional and pooling layers, then passed through multiple residual blocks, and finally through fully connected layers, outputting feature vectors that are fed into the LSTM for further analysis. The recurrent neural network can use feature information from before and after a time series point as the basis for classification at the current time point, improving classification accuracy and sensitivity. The feature vector of a sample output above, as well as the feature vectors output by samples before and after that sample at that time, are all input into the network model for comprehensive analysis. The SoftMax function is then used for classification, outputting the result of motion state recognition.

[0061] In practice, a dataset is first constructed, and a classifier network model is trained using the constructed dataset. After the classifier network model is trained, it is used for formal application.

[0062] The dataset is constructed by sampling the continuous feature spectrograms using a sliding window method to obtain individual sample data. The sample data is then input into the classifier network model for training. The training of the network model is a well-established technique in this field and will not be elaborated upon here.

[0063] Once the classifier network model is trained, in actual monitoring, it is only necessary to sample the continuous feature spectrum through a sliding window, and then input the sampled data into the trained classifier network model to obtain the classification result.

[0064] Specifically, in a residual block, a sample passes through a convolutional layer, a batch normalization layer, a ReLU activation layer, another convolutional layer, and a batch normalization layer, and is then added to the data passed from the direct mapping part to form the input of the next level residual block. In multiple residual blocks, the input of the (l+1)th residual block is determined by the following formula (4):

[0065]

[0066] Where, x (l+1) x is the output of the l-th residual block and the input of the (l+1)-th residual block; (l)is the input of the l-th residual block; L is the number of residual blocks; F(x) is the residual mapping function; W is the convolution operation.

[0067] Since the motion state studied in this application is divided into two categories: motion and non-motion, and the motion state is further divided into six different types of motion, there are a total of 7 results. Therefore, after passing through the fully connected layer, the network output is a feature vector with a length of 7.

[0068] In a recurrent neural network, the input of the l-th sample includes the output vector x of the (l-1)-th sample in the residual neural network. l-1 and the output vector x of the l-th sample in the residual neural network l After passing through the recurrent neural network L(x) l-1 ,x l After the SoftMax classification function, the output is the predicted value y for the motion state. l Specifically, when l equals 1, let x l-1 It is a random constant.

[0069] The specific process is shown in formula (5):

[0070] y l =Softmax(L(x) l ,x l-1 (5).

[0071] The classifier network model in this application classifies the recognition results into either a motion state or a non-motion state. When the recognition result is a motion state, it outputs the specific motion type; when the recognition result is a non-motion state, it begins to extract human vital signs information from the data. First, the phase information of the original intermediate frequency signal is calculated and unwrapped. Then, a bandpass filter is applied to extract the phase information containing respiratory and heartbeat signals respectively. Next, a variational mode decomposition algorithm is applied to decompose the signal into different component signals. Finally, Pearson correlation coefficient is applied to select the component signal that is closest to the original intermediate frequency signal as the respiratory and heartbeat signals.

[0072] In another embodiment, a method for monitoring human vital signs in motion scenarios further includes:

[0073] Step S4: When the target is identified as being in a non-moving state, the phase information of the original intermediate frequency signal is extracted, and after unwinding, bandpass filtering is performed according to the different frequencies of heartbeat and respiration to obtain the respiration signal and heartbeat signal. Then, the variational mode decomposition algorithm is used to separate the bandpass filtered respiration signal and heartbeat signal into component signals.

[0074] The vital signs of the human body are mainly the vibrations of the chest cavity surface caused by breathing and heartbeat. Since the amplitude of the chest cavity surface vibration is small, it cannot be reflected in the changes of the intermediate frequency signal frequency, but it can be reflected in the phase fluctuation. Therefore, it is necessary to first calculate the phase signal of the corresponding position based on the position of the target.

[0075] During exercise, due to significant body movements, limbs may obscure the chest cavity, the center of gravity may shift, and the chest area may even move out of radar detection range, making it difficult to accurately detect heartbeat and respiration. Therefore, this application monitors heartbeat and respiration during the non-motion phase, where monitoring is more effective, by determining the state of motion. In the non-motion phase, after unwinding, bandpass filtering is applied according to different frequencies of heartbeat and respiration. Then, a variational mode decomposition algorithm is used to separate the original signal into multiple component signals.

[0076] Specifically, the regular vibrations of the chest cavity caused by breathing and heartbeat are mainly reflected in the phase changes of the signal. Therefore, the phase information is first extracted from the original intermediate frequency signal. The method for calculating the phase information is shown in formula (6):

[0077]

[0078] Where I(y(t)) represents the imaginary part of the original intermediate frequency signal, and R(y(t)) represents the real part of the original intermediate frequency signal.

[0079] Phase information such as Figure 6 As shown in (a), the phase information obtained through the above calculation exhibits a certain degree of aliasing, requiring phase unwinding to process the phase signal. The unwinding operation addresses the issue of a phase jump of 2π at π (-π), which causes a cluttered phase curve and hinders further analysis. The solution is to shift the phase angle by incrementing it by an integer multiple of ±2π whenever the jump between consecutive phase angles is greater than or equal to π radians, until the jump is less than π.

[0080] Phase information after unwinding process, such as Figure 6 As shown in (b), the phase waveform can basically reflect the shape of the respiratory signal. In addition, there are small fluctuations at the marked points 1, 2, and 3, which are small displacements caused by the heartbeat.

[0081] Since the obtained phase contains both respiratory and heartbeat signals, different bandpass filters are needed to separate them. Based on the physiological characteristics of the human body during exercise, the bandpass frequency for respiratory signals is set to 0.2-1.2Hz, and the bandpass frequency for heartbeat signals is set to 1.0-3.0Hz; the bandpass gain of the filter is 0dB, and the stopband gain is -40dB. The Butterworth filter is used, and the order N is calculated as shown in formula (7):

[0082]

[0083] Among them, f start f is the bandpass start frequency. stop For the bandpass cutoff frequency, F s The sampling frequency.

[0084] Even after passing through a bandpass filter, the respiratory and heartbeat signals are still relatively coarse, containing some noise and interference. Therefore, variational mode decomposition algorithms are needed to extract more refined vital signs from the filtered respiratory and heartbeat signals.

[0085] The variational mode decomposition algorithm takes a bandpass-filtered respiratory signal or heartbeat signal as input and outputs multiple IMF component signals, the number of which can be set. The IMF component signal with the highest correlation coefficient to the original respiratory and heartbeat signals can be considered as the separated respiratory and heartbeat signals.

[0086] In calculating the IMF components, the variational mode decomposition algorithm treats the IMF components as signals with a center frequency and finite bandwidth. k (t), and continuously update u k The center frequency and bandwidth of (t) are used to adaptively decompose the signal frequency band according to the signal frequency domain characteristics, and finally the decomposed IMF component signal is obtained.

[0087] Step S5: Using the Pearson correlation coefficient, select the component signal with the largest correlation coefficient with the bandpass filtered respiratory signal or heartbeat signal as the corresponding respiratory signal or heartbeat signal. Calculate the frequency of the respiratory signal or heartbeat signal through Fourier transform to finally obtain the respiratory frequency and heartbeat frequency.

[0088] like Figure 7 As shown, the original waveform represents the bandpass-filtered respiratory or heartbeat signal, IMF1 and IMF2 represent the two separated component signals, and Res represents the residual signal. For the multiple IMF components obtained after decomposition, the Pearson correlation coefficient is used to compare their correlation coefficients with the bandpass-filtered respiratory or heartbeat signal, and the component signal with the largest correlation coefficient is taken as the corresponding respiratory or heartbeat signal.

[0089] The correlation coefficient calculation process is shown in formula (8):

[0090]

[0091] Where, ρ sign,imf The correlation coefficient between the bandpass filtered respiratory or heartbeat signal *sign* and the IMF component signal *imf* is represented by *cov(sign,imf)*, where *cov(sign,imf)* represents the covariance between the two. σsign σ imf These represent the standard deviations of the bandpass filtered respiratory or heartbeat signal and the IMF component, respectively.

[0092] Finally, Fourier transforms are performed on the separated respiratory and heart rate signals to calculate the frequency information, thus obtaining the respiratory rate and heart rate. In the application scenario of this application, for monitoring the vital signs of a target during a period of exercise, since the rest period is relatively short, the changes in respiratory and heart rate are not significant. Therefore, the vital sign signals during the rest period are directly used to characterize the characteristic signals of that exercise period. Furthermore, based on the exercise state before and after this period, the changes in vital signs under different exercise types can be determined, and the effects of different exercise types can be evaluated.

[0093] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the invention patent. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this patent application should be determined by the appended claims.

Claims

1. A method for monitoring human vital signs in sports scenarios, characterized in that, The method for monitoring human vital signs in sports scenarios includes: The radar reflected signal after being reflected by the human body is received, and the radar reflected signal is mixed with the radar transmitted signal to obtain the original intermediate frequency signal. The original intermediate frequency signal is subjected to Fourier transform to extract the target's position and velocity information. The dominant velocity information of each position is arranged together to calculate a heat map and construct a characteristic spectrum map. The generated feature spectrograms are sampled using a sliding window method, and the feature spectrograms are truncated according to the preset window length and step size. The sampled spectrograms are then input into the trained classifier network model to identify whether the state is moving or not, and the type of movement in the moving state. When the target is identified as being in a non-moving state, the phase information of the original intermediate frequency signal is extracted, and after unwinding, bandpass filtering is performed according to the different frequencies of heartbeat and respiration to obtain the respiration signal and heartbeat signal. Then, the variational mode decomposition algorithm is used to separate the bandpass filtered respiration signal and heartbeat signal into component signals. The component signal with the highest correlation coefficient to the bandpass filtered respiratory signal or heartbeat signal is selected using the Pearson correlation coefficient as the corresponding respiratory signal or heartbeat signal. The frequency of the respiratory signal or heartbeat signal is calculated by Fourier transform, and finally the respiratory frequency and heartbeat frequency are obtained. The process of performing a Fourier transform on the original intermediate frequency signal to extract the target's position and velocity information, arranging the dominant velocity information at each position together, calculating a heatmap, and constructing a feature spectrum map includes: The original intermediate frequency signal is divided into individual chirped signals according to the sampling frequency. A preset number of chirped signals are combined into a frame. A fast Fourier transform is performed on each chirped signal to obtain distance data. Using the distance data corresponding to each chirp signal as a column, arrange the distance data corresponding to all chirp signals in a frame together to form a matrix, and perform a fast Fourier transform on each row of the matrix to obtain velocity data; Within a frame, the velocity with the largest absolute velocity value is selected from each distance as the dominant velocity at that distance; The dominant velocities at each distance in all frames within a preset time period are arranged in chronological order, and a heatmap is calculated to form a characteristic spectrum map.

2. The method for monitoring human vital signs in motion scenarios according to claim 1, characterized in that, The classifier network model includes residual neural networks and recurrent neural networks.

3. The method for monitoring human vital signs in motion scenarios according to claim 1, characterized in that, The Pearson correlation coefficient is used to select the component signal with the highest correlation coefficient to the bandpass filtered respiratory signal or heartbeat signal as the corresponding respiratory signal or heartbeat signal. The correlation coefficient is calculated using the following formula: in, This represents the correlation coefficient between the bandpass filtered respiratory or heartbeat signal (sign) and its component signal (imf). Represents the covariance between the two. , These represent the standard deviations of the bandpass filtered respiratory or heartbeat signal and its component signals, respectively.

Images