Human sleeping posture recognition and vital signs monitoring method based on FMCW radar

Abstract

The present invention provides a method for human sleeping posture recognition and vital signs monitoring based on FMCW radar, including: obtaining a grayscale image corresponding to a distance-time matrix after two radars are spliced ​​together under each human sleeping posture; using the grayscale image data to train a sleeping posture recognition model; using the trained sleeping posture recognition model to classify the human body to be monitored for sleeping posture recognition; selecting one from two distance-time matrices according to the recognized sleeping posture result; using a chest displacement signal correlation algorithm to accurately locate multiple continuous distance gates where the human chest is located; using a GA-MVMD method to jointly decompose the chest displacement signals of multiple continuous distance gates occupied by the human body to obtain multiple IMF components; and reconstructing a respiratory signal and a heartbeat signal according to each IMF component. This method improves the accuracy of sleeping posture recognition and can achieve accurate measurement of human vital signs under different sleeping postures.

Description

Technical Field

[0001] The present invention relates to the field of radar human body perception technology, and in particular to a human body sleeping posture recognition and vital sign monitoring method based on FMCW radar. Background Art

[0002] With the continuous development of social economy, people's awareness of personal health issues continues to increase, and the demand for health monitoring systems has become more urgent. In this context, sleep quality assessment has become a topic of great concern. Identifying and monitoring sleeping postures is crucial for assessing the sleep quality of patients with obstructive sleep apnea (OSA). In addition, physiological parameters such as respiratory rate (RR), heart rate (HR), heart rate variability (HRV), and apnea status are also important indicators for assessing sleep quality. At present, sleep monitoring devices are mainly divided into two categories: 1) methods based on contact sensors; 2) methods based on non-contact sensors. Contact sensors include electrocardiogram (ECG), electroencephalogram (EEG), and pressure-sensitive bed sheet fabric sensors. However, contact sensors may cause sleep discomfort and have high hardware costs. In contrast, non-contact sensors, such as cameras, Wi-Fi sensors, and radar sensors, have more advantages. However, cameras have privacy issues and perform poorly in low-light environments; Wi-Fi sensors are susceptible to environmental interference, and their performance is usually limited by distance and signal attenuation. In recent years, frequency modulated continuous wave (FMCW) radar systems have been widely used in the detection of respiratory rate (RR), heart rate (HR), heart rate variability (HRV), apnea events and sleep posture recognition due to their high resolution, low power consumption and high system integration.

[0003] For the problem of static sleeping posture recognition, deep learning methods usually require a large amount of training data and may have poor generalization performance. In addition, traditional machine learning methods require time-consuming and laborious selection of predefined statistical features. In the sleeping scene, the accuracy of radar vital sign monitoring is highly dependent on the sleeping posture. Due to the limitation of the angle between the radar antenna direction and the human chest, the monitoring performance of the side-lying posture is relatively poor. In recent years, the variational mode decomposition (VMD) algorithm has been widely used in vital sign monitoring. However, VMD can only decompose a single sequence signal, which limits its application. To address this problem, multivariate variational mode decomposition (MVMD) considers the situation where the human chest occupies multiple distance units and has been successfully applied to the vital sign monitoring of drivers. However, the performance of MVMD is highly dependent on the selection of the number of modes and the penalty coefficient. The inappropriate selection of the number of modes K and the penalty coefficient α may lead to large reconstruction errors of vital sign signals.

[0004] In view of the above problems, a method for accurate and reliable sleep posture prediction and vital signs monitoring using FMCW radar is proposed, which has become an urgent problem to be solved. Summary of the invention

[0005] In view of this, the present invention provides a method for human sleeping posture recognition and vital signs monitoring based on FMCW radar to solve the problem of simultaneous human sleeping posture recognition and vital signs monitoring during sleep monitoring, and at the same time aims to improve the problems of difficult feature selection and low accuracy in the human static sleeping posture recognition method based on distance time matrix (RTM), as well as the problems of respiratory harmonic interference and difficult parameter selection in the vital signs monitoring method.

[0006] To achieve the above object, the present invention provides a method for human sleeping posture recognition and vital signs monitoring based on FMCW radar, comprising the following steps:

[0007] Step 1: respectively collect radar echo data of two FMCW radars in different human sleeping positions, and pre-process the acquired data to obtain a grayscale image corresponding to the distance-time matrix after the two FMCW radars are spliced ​​in each human sleeping position, wherein the two FMCW radars are a top radar and a side radar, and the top radar and the side radar are respectively fixed above and to the side of the bed where the human body is located;

[0008] Step 1 The specific steps are as follows:

[0009] (1) In an indoor environment, two FMCW radar systems are used to measure different human sleeping postures and obtain corresponding radar echo data;

[0010] (2) Let the intermediate frequency signal corresponding to the mth chirp and the nth sampling point of the i-th radar be expressed as Y i (m, n), i = 1, 2, m = 1, 2, ..., M, n = 1, 2, ..., N, where M is the number of fast time samples, that is, the number of data sampling points corresponding to each frequency modulation period, and N is the number of slow time samples, that is, the number of chirps corresponding to the data. i (m, n) are stacked column by column into an M×N dimensional matrix Y i ;

[0011] (3) For matrix Y i Perform a fast Fourier transform in the fast time dimension to obtain the distance-time matrix U i ;

[0012] (4) In the slow time dimension, the distance-time matrix U i Perform mean elimination processing to remove static clutter and obtain the distance-time matrix Q after removing clutter. i ;

[0013] (5) The distance-time matrix Q for removing clutter i The fast time dimension is cropped to obtain the region of interest. After cropping, the matrix generated by the two radars is defined as W i ;

[0014] (6) The matrix W generated after clipping i Splicing is performed along the slow time dimension to obtain a new splicing matrix Z, which is mapped into a grayscale image and adjusted to 224×224 pixels;

[0015] Step 2: Using the grayscale image obtained in step 1 as a sleep posture data set, and dividing the sleep posture data set into a training set, a validation set, and a test set. After that, using directional gradient histogram feature extraction and support vector machine and using the training set and the validation set to train and validate the sleep posture recognition model, and using the test set to test, to obtain a trained sleep posture recognition model;

[0016] Step 3: Use the method described in step 1 to collect and preprocess radar echo data of the monitored human body to obtain a grayscale image corresponding to the distance-time matrix after the two FMCW radars are spliced, and then input the grayscale image into the trained sleep posture recognition model to perform sleep posture recognition and classification;

[0017] Step 4: According to the results of the recognized sleeping posture, the two distance time matrices U i Select one to perform vital sign monitoring, wherein if the patient is identified as lying prone or supine, the respiratory rate and heart rate are estimated using the distance-time matrix obtained by the top radar, and if the patient is identified as lying on the side, the respiratory rate and heart rate are estimated using the distance-time matrix obtained by the side radar;

[0018] Step 5: Extract the selected distance-time matrix U i The phase signal of each range gate in the image is obtained and the corresponding chest displacement signal is obtained, and the chest displacement signal correlation algorithm is used to accurately locate the multiple continuous range gates where the human chest is located;

[0019] Step 6: jointly decompose the chest displacement signals of multiple continuous range gates occupied by the human body through the genetic algorithm-multivariate variational mode decomposition method to obtain multiple intrinsic mode function components;

[0020] Step 7: Calculate the percentage of breathing and heartbeat energy of each intrinsic mode function component and add the intrinsic mode function components that satisfy the condition of equation (13) to reconstruct the breathing signal s r (t) and heartbeat signal s h (t):

[0021]

[0022] Where, j = 1, 2, ..., K, E(j) is the jth intrinsic mode function component IMF j The frequency domain energy, E r (j) and E h (j) are the jth intrinsic mode function components IMF j Energy in the respiratory and heartbeat bands, δ r and δ h It is the threshold for judging the energy ratio of breathing and heartbeat signals.

[0023] Preferably, in step 2, the process of extracting the directional gradient histogram feature is as follows:

[0024] (1) Calculate the gradient: For each pixel in the grayscale image, calculate the gradient in the horizontal and vertical directions. The formula is as follows:

[0025] G x (x,y)=I(x+1,y)-I(x-1,y) (1)

[0026] G y (x,y)=I(x,y+1)-I(x,y-1) (2)

[0027] Where I(x, y) is the pixel value of the pixel point (x, y), and the gradient magnitude G(x, y) and gradient direction Θ(x, y) of each pixel point are defined as:

[0028]

[0029]

[0030] (2) Constructing cell directional gradient histogram: Divide the grayscale image into non-overlapping cells, each cell consists of 8×8 pixels, and construct a 9-bin unsigned (0°-180°) directional gradient histogram for each cell;

[0031] (3) Construct blocks and normalize: merge 2×2 cells into a block, normalize the directional gradient histogram of each block, and generate a feature vector of length 36;

[0032] (4) Integration of directional gradient histogram feature vectors: The size of the spliced ​​grayscale image is 224×224. The entire image is traversed with a step size of 8 to obtain a directional gradient histogram feature vector with a length of 26244.

[0033] Further preferably, in step 2, the support vector machine classifier realizes nonlinear classification by introducing a kernel function. The principle of the kernel function is to convert the input space into a high-dimensional feature space through a nonlinear transformation, thereby linearly separating all samples in the feature space. The decision function is expressed as:

[0034]

[0035] Among them, p d is the dth support vector, p is the input directional gradient histogram feature vector, D is the number of support vectors, y d is the category of the corresponding support vector, 1 represents the positive category, -1 represents the negative category, α d is the learning coefficient for each corresponding support vector, b is the learning bias, and β is the kernel function.

[0036] Further preferably, step 5 specifically comprises the following steps:

[0037] The slow-time signal at the n0th range gate is defined as U(n0, m). After arc tangent demodulation and dewrapping, the phase signal of the n0th range gate is obtained as follows:

[0038]

[0039] Among them, unwrap(·) is an unwrapping operation that ensures that the phase difference between two adjacent samples is not greater than π to obtain a smooth phase signal, and ∠(·) is a complex phase extraction operator based on the four-quadrant inverse tangent function. The unwrapping process is as follows:

[0040]

[0041] After obtaining the phase signal, the human chest displacement signal is expressed as:

[0042]

[0043] Where c is the speed of light, f c is the starting frequency of the FMCW radar transmitting signal;

[0044] The range gate with the largest average power and its nearby highly correlated range gates are retained for vital sign extraction, where z is assumed to be max is the chest displacement signal vector corresponding to the range gate with the largest average power, z a is the chest displacement signal vector corresponding to the ath adjacent range gate, and the Pearson correlation coefficient is calculated as:

[0045]

[0046] Among them, cov(z max , z a ) is the covariance between two vectors, and They are z max and z aWhen the Pearson correlation coefficient between the chest displacement signal vector corresponding to the range gate with the largest average power and the chest displacement signal vector corresponding to the adjacent range gate exceeds the preset threshold, the chest displacement signal vectors corresponding to these range gates will be selected for subsequent vital sign extraction.

[0047] More preferably, in step 6, z(t)=[z1(t), z2(t), ..., z L (t)] T represents the chest displacement signal vector of all L selected range gates. The multivariate variational mode decomposition algorithm aims to solve the problem of decomposing the chest displacement signal vector z(t) of multiple range gates into K band-limited signals The variational problem, where u k (t)=[u k，1 (t),u k，2 (t),…,u k，L (t)] T , 1≤k≤K, through the multivariate variational mode decomposition algorithm, the chest displacement signals of L range gates can be decomposed into L groups of intrinsic mode functions, each group of intrinsic mode functions consists of respiratory signals, heartbeat signals, respiratory harmonics and noise components.

[0048] Further preferably, in step 6, the final intrinsic mode function decomposition result u is obtained by using the empirical mean value k (t), 1≤k≤K:

[0049]

[0050] Further preferably, in step 6, the optimal number of modes K and penalty coefficient α are determined by a genetic algorithm-multivariate variational mode decomposition method. For genetic algorithm optimization, the maximum number of iterations, population size, crossover probability and mutation probability are set to 30, 10, 0.8 and 0.2 respectively; the maximum local inverse envelope entropy is selected as the fitness function of the genetic algorithm, the sequence u(m) is a discrete sequence of the intrinsic mode function after decomposition, and the maximum local inverse envelope entropy IEE of the intrinsic mode function is defined as:

[0051]

[0052] Among them, H m =((u(m)-u(m-1)) 2 +T s 2 ) 1 / 2 , T s Indicates the slow time sampling period.

[0053] Further preferably, in step 6, the steps of determining the optimal number of modes K and penalty coefficient α by using the genetic algorithm-multivariate variational mode decomposition method are as follows:

[0054] First, a population is randomly generated to start the multivariate variational mode decomposition parameter optimization process. Each individual in the population represents a combination of the number of modes K and the penalty coefficient α.

[0055] Then, multivariate variational mode decomposition is performed on each individual in the population, and the corresponding fitness function value is calculated;

[0056] Then, selection, crossover and mutation operations are performed to generate a new population. The selection operation is based on the fitness function value of the individual. The better the individual, the higher the probability of being selected in the next generation. The crossover operation randomly combines the parameters of the selected individuals to generate new individuals, while the mutation operation introduces random disturbances to maintain the diversity of the population. The multivariate variational mode decomposition parameter optimization process will continue until the preset maximum number of iterations is reached.

[0057] Finally, by analyzing the maximum local inverse envelope entropy of the decomposed intrinsic mode function, the optimal combination of the number of modes K and the penalty coefficient α can be determined.

[0058] Further preferably, in step 7, the breathing signal s r (t) and heartbeat signal s h (t)Reconstructed as:

[0059]

[0060] The human sleeping posture recognition and vital signs monitoring method based on FMCW radar provided by the present invention adopts two FMCW radar systems combined with directional gradient histogram (HOG) and support vector machine (SVM) classifiers to realize accurate recognition of sleeping posture. According to the recognized sleeping posture, the corresponding radar system is intelligently called to monitor vital signs. In addition, a multivariate variational mode decomposition (MVMD) signal decomposition method based on genetic algorithm (GA) is also used for accurate monitoring of vital signs during sleep. Through this innovative method, not only the accuracy of sleeping posture recognition is improved, but also it is ensured that during the vital signs monitoring process, the human chest cavity always faces the antenna of the radar system used, thereby realizing accurate measurement of human vital signs under different sleeping postures. BRIEF DESCRIPTION OF THE DRAWINGS

[0061] Figure 1 A flow chart of a method for human sleeping posture recognition and vital signs monitoring based on FMCW radar provided by the present invention;

[0062] Figure 2 Measured scene diagram for two radar configurations;

[0063] Figure 3 It is a flow chart of the distance time matrix preprocessing method;

[0064] Figure 4 It is the hyperparameter optimization curve for optimizing support vector machine;

[0065] Figure 5 It is the result graph of fitness function convergence;

[0066] Figure 6 To reconstruct the respiratory waveform;

[0067] Figure 7 To reconstruct the heartbeat waveform;

[0068] Figure 8 A comparison chart of the respiratory rate obtained by genetic algorithm-multivariate mode decomposition (GA-MVMD) and the respiratory rate obtained by the contact sensor;

[0069] Fig. 9 This is a comparison chart of the heart rate obtained by genetic algorithm-multivariate mode decomposition (GA-MVMD) and the heart rate obtained by the contact sensor. DETAILED DESCRIPTION

[0070] The specific implementation of the present invention is further described in detail below in conjunction with the accompanying drawings and examples. The following examples are used to illustrate the present invention, but are not intended to limit the scope of the present invention.

[0071] like Figure 1 As shown, this embodiment provides a method for human sleeping posture recognition and vital sign monitoring based on FMCW radar, comprising the following steps:

[0072] Step 1: Collect radar echo data of two FMCW radars (frequency modulated continuous wave radars) under different human sleeping positions respectively, and pre-process the acquired data to obtain the grayscale image corresponding to the distance-time matrix after the two FMCW radars are spliced ​​under different human sleeping positions, where Figure 2 As shown, the two FMCW radars are a top radar and a side radar, which are respectively fixed above and to the side of the bed where the human body is located, and the two radars are connected to the same computer for transmitting radio frequency data;

[0073] like Figure 3 As shown, the specific steps of step 1 are as follows:

[0074] (1) In an indoor environment, two FMCW radar systems are used to measure different human sleeping postures and obtain corresponding radar echo data;

[0075] (2) Let the intermediate frequency signal corresponding to the mth chirp and the nth sampling point of the i-th radar be expressed as Yi (m, n), i = 1, 2, m = 1, 2, ..., M, n = 1, 2, ..., N, where M is the number of fast time samples, that is, the number of data sampling points corresponding to each frequency modulation period, and N is the number of slow time samples, that is, the number of chirps corresponding to the data. i (m, n) are stacked column by column into an M×N dimensional matrix Y i ;

[0076] (3) For matrix Y i Perform a fast Fourier transform (FFT) in the fast time dimension to obtain the distance-time matrix U i ;

[0077] (4) In the slow time dimension, the distance-time matrix U i Perform mean elimination processing to remove static clutter and obtain the distance-time matrix Q after removing clutter. i ;

[0078] (5) The distance-time matrix Q for removing clutter i The fast time dimension is cropped to obtain the region of interest. After cropping, the matrix generated by the two radars is defined as W i ;

[0079] (6) The matrix W generated after clipping i Splicing is performed along the slow time dimension to obtain a new splicing matrix Z, which is mapped into a grayscale image and adjusted to 224×224 pixels;

[0080] Step 2: Use the grayscale image obtained in step 1 as a sleep posture data set, and divide the sleep posture data set into a training set, a validation set, and a test set. Then, use directional gradient histogram feature extraction and support vector machine to train and validate the sleep posture recognition model using the training set and the validation set, and use the test set to test, so as to obtain a trained sleep posture recognition model.

[0081] The grayscale image of the received radar signal contains a lot of edge information related to sleeping posture. Therefore, the directional gradient histogram feature (HOG feature) can be extracted from the grayscale image to classify different sleeping postures. The process of directional gradient histogram feature extraction is as follows:

[0082] (1) Calculate the gradient: For each pixel in the grayscale image, calculate the gradient in the horizontal and vertical directions. The formula is as follows:

[0083] G x (x,y)=I(x+1,y)-I(x-1,y) (1)

[0084] G y(x,y)=I(x,y+1)-I(x,y-1) (2)

[0085] Where I(x, y) is the pixel value of the pixel point (x, y), and the gradient magnitude G(x, y) and gradient direction Θ(x, y) of each pixel point are defined as:

[0086]

[0087] (2) Constructing cell directional gradient histogram: The grayscale image is divided into non-overlapping cells, each cell consists of 8×8 pixels, and a 9-bin unsigned (0°-180°) directional gradient histogram is constructed for each cell. This histogram can be represented by a feature vector of length 9;

[0088] (3) Construct blocks and normalize: merge 2×2 cells into a block, normalize the directional gradient histogram of each block, and generate a feature vector of length 36;

[0089] (4) Integration of directional gradient histogram feature vectors: The size of the spliced ​​grayscale image is 224×224. The entire image is traversed with a step size of 8 to obtain a directional gradient histogram feature vector with a length of 26244. The calculation method is as follows:

[0090]

[0091] Sleep posture recognition is performed using a support vector machine (SVM) classifier. The classifier implements nonlinear classification by introducing a kernel function. The kernel function works by converting the input space into a high-dimensional feature space through a nonlinear transformation, thereby performing linear separation on all samples in the feature space. The decision function is expressed as:

[0092]

[0093] Among them, p d is the dth support vector, p is the input HOG feature vector, D is the number of support vectors, y d is the category of the corresponding support vector (1 represents the positive category, -1 represents the negative category), α d is the learning coefficient for each corresponding support vector, b is the learning bias, and β is the kernel function;

[0094] In order to optimize the performance of the support vector machine classification model, the present invention uses the ClassificationLearner App in MATLAB for testing. For hyperparameter tuning, Bayesian optimization with a box constraint level of 0.001-1000 is used to iterate the classifier 30 times.

[0095] Step 3: Use the method described in step 1 to collect and preprocess radar echo data of the monitored human body to obtain a grayscale image corresponding to the distance-time matrix after the two FMCW radars are spliced, and then input the grayscale image into the trained sleep posture recognition model to perform sleep posture recognition and classification;

[0096] Step 4: According to the results of the recognized sleeping posture, the two distance time matrices U i Select one to monitor vital signs, wherein if the person is identified as lying prone or supine, the respiratory rate (RR) and heart rate (HR) are estimated using the distance-time matrix obtained by the top radar; if the person is identified as lying sideways, the respiratory rate (RR) and heart rate (HR) are estimated using the distance-time matrix obtained by the side radar, so that the radar antenna is always facing the human chest during vital sign monitoring;

[0097] Step 5: Extract the selected distance-time matrix U i The phase signal of each range gate in the image is obtained and the corresponding chest displacement signal is obtained, and the chest displacement signal correlation algorithm is used to accurately locate the multiple continuous range gates where the human chest is located;

[0098] The slow-time signal at the n0th range gate is defined as U(n0, m). After arc tangent demodulation and dewrapping, the phase signal of the n0th range gate is obtained as follows:

[0099]

[0100] Among them, unwrap(·) is an unwrapping operation that ensures that the phase difference between two adjacent samples is not greater than π to obtain a smooth phase signal, and ∠(·) is a complex phase extraction operator based on the four-quadrant inverse tangent function. The unwrapping process is as follows:

[0101]

[0102] After obtaining the phase signal, the human chest displacement signal can be expressed as:

[0103]

[0104] Where c is the speed of light, f c is the starting frequency of the FMCW radar transmitting signal. Considering the range resolution of the FMCW radar and the size of the human chest, the reflections from the human chest will occupy different range gates. Therefore, the important information contained in multiple range gates can be jointly optimized to improve the estimation accuracy. The present invention retains the range gate with the largest average power and its nearby highly correlated range gates for vital sign extraction. Assume z max is the chest displacement signal vector corresponding to the range gate with the largest average power, z ais the chest displacement signal vector corresponding to the ath adjacent range gate, and the Pearson correlation coefficient can be calculated as:

[0105]

[0106] Among them, cov(z max , z a ) is the covariance between two vectors, and They are z max and z a When the Pearson correlation coefficient between the chest displacement signal vector corresponding to the range gate with the largest average power and the chest displacement signal vector corresponding to the adjacent range gate exceeds the preset threshold of 0.8, these range gates will be selected for subsequent vital sign extraction.

[0107] Step 6: The chest displacement signals of multiple continuous range gates occupied by the human body are jointly decomposed by the genetic algorithm-multivariate variational mode decomposition (GA-MVMD) method to obtain multiple intrinsic mode function components (IMF components);

[0108] z(t)=[z1(t),z2(t),…,z L (t)] T Represents the chest displacement signal vector of all L selected range gates. The multivariate mode decomposition (MVMD) algorithm aims to solve the problem of decomposing the chest displacement signal vector z(t) of multiple range gates into K band-limited signals The variational problem, where u k (t)=[u k，1 (t)u k，2 (t),…,u k，L (t)] T (1≤k≤K). Through the multivariate variational mode decomposition algorithm, the chest displacement signals of L range gates can be decomposed into L groups of intrinsic mode functions, each of which is composed of respiratory signals, heartbeat signals, respiratory harmonics and noise components. In order to further reduce the impact of noise, the empirical mean can be used to obtain the final intrinsic mode function decomposition result u k (t)(1≤k≤K):

[0109]

[0110] In the multivariate variational modal decomposition algorithm, the selection of the number of modes K and the penalty coefficient α will greatly affect the accuracy of the vital signs monitoring results. The present invention determines the optimal number of modes K and the penalty coefficient α through the genetic algorithm-multivariate variational modal decomposition method. The genetic algorithm (GA) searches for the optimal value of the fitness function through operations such as selection, crossover and mutation. The selection of the fitness function is the key to obtaining the optimal result. For genetic algorithm optimization, the maximum number of iterations (maxgen), population size (sizepop), crossover probability (pcross) and mutation probability (pmutation) are set to 30, 10, 0.8 and 0.2 respectively. The present invention selects the maximum local inverse envelope entropy (IEE) as the fitness function of the genetic algorithm. When the maximum local inverse envelope entropy is used as the fitness function, the decomposition result of the signal tends to be sinusoidal. Considering that the sequence u(m) is a discrete sequence of the intrinsic modal function after decomposition, the maximum local inverse envelope entropy of the intrinsic modal function is defined as:

[0111]

[0112] Among them, H m =((u(m)-u(m-1)) 2 +T s 2 ) 1 / 2 , T s Represents the slow time sampling period. First, the multivariate variational mode decomposition parameter optimization process is started by randomly generating a population, and each individual in the population represents a set of mode numbers K and penalty coefficients α. Subsequently, multivariate variational mode decomposition is performed on each individual in the population, and the corresponding fitness function value is calculated. Then selection, crossover and mutation operations are performed to generate a new population. The selection operation is based on the fitness function value of the individual, and the better the individual, the higher the probability of being selected in the next generation. The crossover operation randomly combines the parameters of the selected individuals to generate new individuals, while the mutation operation introduces random perturbations to maintain the diversity of the population. The multivariate variational mode decomposition parameter optimization process will continue until the preset maximum number of iterations is reached. Finally, by analyzing the maximum local inverse envelope entropy (IEE) of the decomposed intrinsic mode function, the optimal combination of mode number K and penalty coefficient α can be determined.

[0113] Step 7: Reconstruction of life signals: After decomposing the chest displacement signals of multiple range gates using genetic algorithm-multivariate variational mode decomposition, multiple intrinsic mode function components are obtained. Usually, the respiratory frequency is between 0.1 and 0.5 Hz, and the heart rate is between 0.8 and 2.0 Hz. In order to obtain the energy distribution in the frequency domain, the percentage of respiratory and heart rate energy of each intrinsic mode function component is calculated as follows:

[0114]

[0115] Where, j = 1, 2, ..., K, E(j) is the jth intrinsic mode function component IMF j The frequency domain energy, E r (j) and E h (j) are the jth intrinsic mode function components IMF j Energy in the respiratory and heartbeat bands, δ r and δ h To determine the energy ratio of the breathing and heartbeat signals, the thresholds are set to 0.5. The intrinsic mode function components that satisfy the conditions of formula (13) are added together. The breathing signal s r (t) and heartbeat signal s h (t) can be refactored as:

[0116]

[0117] The following is a detailed description of the above-mentioned method for human sleeping posture recognition and vital signs monitoring based on FMCW radar using a specific embodiment:

[0118] In this embodiment, the top radar is suspended on a bracket 1.4 meters above the bed, while the side radar is placed on a chair 1.6 meters away from the bed. Both FMCW radars are connected to the same computer for transmitting RF data. Four different human sleeping positions were measured for 18 volunteers in an indoor environment, including (a) supine, (b) prone, (c) left side, and (d) right side. In order to ensure that the two FMCW radars do not interfere with each other when working at the same time, a frequency division multiplexing mode is adopted. In this mode, the starting frequency of the top radar is 77GHz and the starting frequency of the side radar is 79GHz. The remaining parameters of the two radar systems are consistent: the bandwidth is 2GHz, the fast time sampling frequency is 3.2MHz, the slow time sampling number is 6000, the slow time sampling period is 20ms, and the fast time sampling number is 128. The measurement time for each sleeping position is 120 seconds, and the measurement is repeated 10 times. A total of 720 sets of echo data were obtained for 18 volunteers and 4 sleeping positions.

[0119] The echo data of the two radars are preprocessed. The distance-time matrix of the two radars without clutter is obtained by fast Fourier transform (FFT) and mean cancellation in the fast time dimension. The distance-time matrix is ​​then clipped according to the distance between the sleeping person and the two radars. The clipping range of the top radar is 0.6 meters to 1.95 meters, and the clipping range of the side radar is 1.05 meters to 2.4 meters. The clipped distance-time matrix is ​​spliced ​​along the slow time dimension and mapped into a grayscale image of 224×224 pixels.

[0120] The training set and validation set contain a total of 600 (15 volunteers × 10 measurements × 4 sleeping positions = 600) grayscale images, and the test set contains a total of 120 (3 volunteers × 10 measurements × 4 sleeping positions = 120) grayscale images.

[0121] Usually, the grayscale image obtained by each radar contains a lot of edge information related to sleeping posture. Therefore, the directional gradient histogram features can be extracted from the spliced ​​grayscale images to classify different sleeping postures. The 224×224 pixel grayscale image is divided into non-overlapping cells, each cell consists of 8×8 pixels, and a 9-bin unsigned (0°-180°) directional gradient histogram is constructed for each cell. The 2×2 cells are merged into a block, and the length of the directional gradient histogram feature vector obtained by traversing the entire grayscale image with a step size of 8 is 26244.

[0122] The present invention uses a support vector machine classifier for sleep posture recognition. In order to optimize the performance of the support vector machine classification model, the Classification Learner App in MATLAB was used for testing. For hyperparameter tuning, Bayesian optimization with a box constraint level of 0.001-1000 was used to iterate the classifier 30 times. Figure 4 The hyperparameter optimization curve of the support vector machine is displayed. For the grayscale image of the two radars, the support vector machine classification model using the Gaussian kernel function and the one-to-many multi-classification method achieved the best performance, with an accuracy of 97.5% and a minimum error of 0.025. At this time, the parameter box constraint level is 784.0007, and the optimal model after optimization is saved. The test data is input into the optimized support vector machine classification model, and the test accuracy is 96.7%.

[0123] Different radar data are selected for vital sign monitoring according to the recognition results of sleeping posture. Taking the supine sleeping posture as an example, the intermediate frequency echo signal from the top radar is subjected to fast Fourier transform and mean elimination in the fast time dimension to generate a distance time matrix. Considering that the human target is an extended target, the FMCW radar has a high range resolution, and the human target will occupy multiple continuous range gates. The chest displacement signal correlation processing is used to determine the range gate occupied by the human subject. Finally, the chest displacement signals of the determined multiple range gates are jointly decomposed using the genetic algorithm-multivariate variational mode decomposition method.

[0124] The fitness function convergence result is as follows Figure 5 As shown. Figure 5It can be seen that the maximum local inverse envelope entropy converges to the maximum value when K = 5, α = 2817 at the 21st iteration, which can be considered as the optimal decomposition parameters of the multivariate variational mode decomposition algorithm. The multivariate variational mode decomposition algorithm with the optimal decomposition parameters K = 5, α = 2817 is used to decompose the chest displacement signal, and the breathing and heartbeat signals are reconstructed according to the energy percentage criterion, as shown in Figure 2. Figure 6 and Figure 7 shown.

[0125] The reconstructed respiratory signal and heartbeat signal are subjected to short-time Fourier transform (STFT) to obtain the corresponding respiratory rate (RR) and heart rate (HR) estimation results. When the duration is 120 seconds, a 10-second sliding window with an interval of 2 seconds is used to estimate the respiratory rate (RR) and heart rate (HR). The experimental results are shown in Figure 2. Figure 8 and Fig. 9 As shown. The mean absolute error (MAE) is used to quantify the estimation accuracy of the monitoring results of this algorithm, which is defined as follows:

[0126]

[0127] Among them, N c is the total number of windows, f s，r is the respiratory rate (RR) or heart rate (HR) estimated by the radar corresponding to the sth window, f s，c is the reference value of the respiratory rate (RR) or heart rate (HR) obtained by the contact sensor corresponding to the s-th window. It is calculated that the mean absolute error (MAE) of the genetic algorithm-multivariate variational mode decomposition (GA-MVMD) method proposed in the present invention in respiratory rate (RR) monitoring is 0.32 times / minute (bpm), and the mean absolute error (MAE) in heart rate (HR) monitoring is 2.10 times / minute (bpm).

[0128] The human sleeping posture recognition and vital signs monitoring method based on FMCW radar provided by the present invention can accurately recognize the sleeping posture of the human body by fusing the data of two radars. According to the recognition result of the sleeping posture, the radar data facing the human chest cavity is selected for vital signs monitoring. During the monitoring process, the chest cavity displacement signals of multiple range gates are jointly decomposed to obtain stable and reliable respiratory rate and heart rate monitoring results.

[0129] The human sleeping posture recognition and vital signs monitoring method based on FMCW radar provided by the present invention can accurately recognize the sleeping posture of the human body, and realize the vital signs monitoring of the human body in different sleeping postures, realizing the integration of the functions of sleeping posture recognition and vital signs monitoring. The recognition accuracy of the human sleeping posture can be improved by extracting the directional gradient histogram (HOG) feature combined with the support vector machine (SVM) to classify the grayscale image spliced ​​by two radars. In addition, calling the corresponding radar to monitor the human vital signs according to the recognized sleeping posture result can ensure that the human chest cavity is always facing the antenna of a certain radar system. The genetic algorithm-multivariate variational mode decomposition (GA-MVMD) method not only considers that the human chest cavity is an extensible target but also can obtain the optimal decomposition parameters, thereby improving the accuracy of monitoring the vital signs of the human body in various sleeping postures.

[0130] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the above embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope defined by the claims of the present invention.

Claims

1. A method for human sleeping posture recognition and vital signs monitoring based on FMCW radar, characterized in that: The following steps are involved: Step 1: respectively collect radar echo data of two FMCW radars in different human sleeping positions, and pre-process the acquired data to obtain a grayscale image corresponding to the distance-time matrix after the two FMCW radars are spliced ​​in each human sleeping position, wherein the two FMCW radars are a top radar and a side radar, and the top radar and the side radar are respectively fixed above and to the side of the bed where the human body is located; Step 1 The specific steps are as follows: (1) In an indoor environment, two FMCW radar systems are used to measure different human sleeping postures and obtain corresponding radar echo data; (2) Let the intermediate frequency signal corresponding to the mth chirp and the nth sampling point of the i-th radar be expressed as Y i (m, n), i = 1, 2, m = 1, 2, ..., M, n = 1, 2, ..., N, where M is the number of fast time samples, that is, the number of data sampling points corresponding to each frequency modulation period, and N is the number of slow time samples, that is, the number of chirps corresponding to the data. i (m, n) are stacked column by column into an M×N dimensional matrix Y i ; (3) For matrix Y i Perform a fast Fourier transform in the fast time dimension to obtain the distance-time matrix U i ; (4) In the slow time dimension, the distance-time matrix U i Perform mean elimination processing to remove static clutter and obtain the distance-time matrix Q after removing clutter. i ; (5) The distance-time matrix Q for removing clutter i The fast time dimension is cropped to obtain the region of interest. After cropping, the matrix generated by the two radars is defined as W i ; (6) splicing the cropped matrix W along the slow time dimension to obtain a new spliced ​​matrix Z, mapping the spliced ​​matrix into a grayscale image and adjusting it to 224×224 pixels; Step 2: Using the grayscale image obtained in step 1 as a sleep posture data set, and dividing the sleep posture data set into a training set, a validation set, and a test set. After that, using directional gradient histogram feature extraction and support vector machine and using the training set and the validation set to train and validate the sleep posture recognition model, and using the test set to test, to obtain a trained sleep posture recognition model; Step 3: Use the method described in step 1 to collect and preprocess radar echo data of the monitored human body to obtain a grayscale image corresponding to the distance-time matrix after the two FMCW radars are spliced, and then input the grayscale image into the trained sleep posture recognition model to perform sleep posture recognition and classification; Step 4: According to the results of the recognized sleeping posture, the two distance time matrices U i Select one to perform vital sign monitoring, wherein if the patient is identified as lying prone or supine, the respiratory rate and heart rate are estimated using the distance-time matrix obtained by the top radar, and if the patient is identified as lying on the side, the respiratory rate and heart rate are estimated using the distance-time matrix obtained by the side radar; Step 5: Extract the selected distance-time matrix U i The phase signal of each range gate in the image is obtained and the corresponding chest displacement signal is obtained, and the chest displacement signal correlation algorithm is used to accurately locate the multiple continuous range gates where the human chest is located; Step 6: jointly decompose the chest displacement signals of multiple continuous range gates occupied by the human body through the genetic algorithm-multivariate variational mode decomposition method to obtain multiple intrinsic mode function components; Step 7: Calculate the percentage of breathing and heartbeat energy of each intrinsic mode function component and add the intrinsic mode function components that satisfy the condition of equation (13) to reconstruct the breathing signal s r (t) and heartbeat signal s h (t): Where, j = 1, 2, ..., K, E(j) is the jth intrinsic mode function component IMF j The frequency domain energy, E r (j) and E h (j) are the jth intrinsic mode function components IMF j Energy in the respiratory and heartbeat bands, δ r and δ h It is the threshold for judging the energy ratio of breathing and heartbeat signals.

2. The method for human sleeping posture recognition and vital sign monitoring based on FMCW radar according to claim 1, characterized in that: In step 2, the process of oriented gradient histogram feature extraction is as follows: (1) Calculate the gradient: For each pixel in the grayscale image, calculate the gradient in the horizontal and vertical directions. The formula is as follows: G x (x,y)=I(x+1,y)-I(x-1,y) (1) G y (x,y)=I(x,y+1)-I(x,y-1) (2) Where I(x, y) is the pixel value of the pixel point (x, y), and the gradient magnitude G(x, y) and gradient direction Θ(x, y) of each pixel point are defined as: (2) Constructing cell directional gradient histogram: Divide the grayscale image into non-overlapping cells, each cell consists of 8×8 pixels, and construct a 9-bin unsigned (0°-180°) directional gradient histogram for each cell; (3) Construct blocks and normalize: merge 2×2 cells into a block, normalize the directional gradient histogram of each block, and generate a feature vector of length 36; (4) Integration of directional gradient histogram feature vectors: The size of the spliced ​​grayscale image is 224×224. The entire image is traversed with a step size of 8 to obtain a directional gradient histogram feature vector with a length of 26244.

3. The method for human sleeping posture recognition and vital sign monitoring based on FMCW radar according to claim 1, characterized in that: In step 2, the support vector machine classifier implements nonlinear classification by introducing the kernel function. The principle of the kernel function is to convert the input space into a high-dimensional feature space through nonlinear transformation, thereby linearly separating all samples in the feature space. The decision function is expressed as: Among them, p d is the dth support vector, p is the input directional gradient histogram feature vector, D is the number of support vectors, y d is the category of the corresponding support vector, 1 represents the positive category, -1 represents the negative category, α d is the learning coefficient for each corresponding support vector, b is the learning bias, and β is the kernel function.

4. The method for human sleeping posture recognition and vital sign monitoring based on FMCW radar according to claim 1, characterized in that: Step 5 specifically includes the following steps: The slow-time signal at the n0th range gate is defined as U(n0, m). After arc tangent demodulation and dewrapping, the phase signal of the n0th range gate is obtained as follows: Among them, unwrap(·) is an unwrapping operation that ensures that the phase difference between two adjacent samples is not greater than π to obtain a smooth phase signal, and ∠(·) is a complex phase extraction operator based on the four-quadrant inverse tangent function. The unwrapping process is as follows: After obtaining the phase signal, the human chest displacement signal is expressed as: Where c is the speed of light, f c is the starting frequency of the FMCW radar transmitting signal; The range gate with the largest average power and its nearby highly correlated range gates are retained for vital sign extraction, where z is assumed to be max is the chest displacement signal vector corresponding to the range gate with the largest average power, z a is the chest displacement signal vector corresponding to the ath adjacent range gate, and the Pearson correlation coefficient is calculated as: Among them, cov(z max , z a ) is the covariance between two vectors, and They are z max and z a When the Pearson correlation coefficient between the chest displacement signal vector corresponding to the range gate with the largest average power and the chest displacement signal vector corresponding to the adjacent range gate exceeds the preset threshold, the chest displacement signal vectors corresponding to these range gates will be selected for subsequent vital sign extraction.

5. The method for human sleeping posture recognition and vital sign monitoring based on FMCW radar according to claim 1, characterized in that: In step 6, z(t)=[z1(t),z2(t),…,z L (t)] T represents the chest displacement signal vector of all L selected range gates. The multivariate variational mode decomposition algorithm aims to solve the problem of decomposing the chest displacement signal vector z(t) of multiple range gates into K band-limited signals The variational problem, where u k (t)=[u k，1 (t)u k，2 (t),…,u k，L (t)] T , 1≤k≤K, through the multivariate variational mode decomposition algorithm, the chest displacement signals of L range gates can be decomposed into L groups of intrinsic mode functions, each group of intrinsic mode functions consists of respiratory signals, heartbeat signals, respiratory harmonics and noise components.

6. The method for human sleeping posture recognition and vital sign monitoring based on FMCW radar according to claim 5, characterized in that: In step 6, the empirical mean is used to obtain the final intrinsic mode function decomposition result u k (t), 1≤k≤K:

7. The method for human sleeping posture recognition and vital sign monitoring based on FMCW radar according to claim 6, characterized in that: In step 6, the optimal number of modes K and penalty coefficient α are determined by the genetic algorithm-multivariate variational mode decomposition method. For genetic algorithm optimization, the maximum number of iterations, population size, crossover probability and mutation probability are set to 30, 10, 0.8 and 0.2 respectively; the maximum local inverse envelope entropy is selected as the fitness function of the genetic algorithm. The sequence u(m) is the discrete sequence of the intrinsic mode function after decomposition. The maximum local inverse envelope entropy IEE of the intrinsic mode function is defined as: Among them, H m =((u(m)-u(m-1)) 2 +T s 2 ) 1 / 2 , T s Indicates the slow time sampling period.

8. The method for human sleeping posture recognition and vital sign monitoring based on FMCW radar according to claim 7, characterized in that: In step 6, the steps of determining the optimal number of modes K and penalty coefficient α by genetic algorithm-multivariate mode decomposition method are as follows: First, a population is randomly generated to start the multivariate variational mode decomposition parameter optimization process. Each individual in the population represents a combination of the number of modes K and the penalty coefficient α. Then, multivariate variational mode decomposition is performed on each individual in the population, and the corresponding fitness function value is calculated; Then, selection, crossover and mutation operations are performed to generate a new population. The selection operation is based on the fitness function value of the individual. The better the individual, the higher the probability of being selected in the next generation. The crossover operation randomly combines the parameters of the selected individuals to generate new individuals, while the mutation operation introduces random disturbances to maintain the diversity of the population. The multivariate variational mode decomposition parameter optimization process will continue until the preset maximum number of iterations is reached. Finally, by analyzing the maximum local inverse envelope entropy of the decomposed intrinsic mode function, the optimal combination of the number of modes K and the penalty coefficient α can be determined.

9. The method for human sleeping posture recognition and vital sign monitoring based on FMCW radar according to claim 1, characterized in that: In step 7, the breathing signal s r (t) and heartbeat signal s h (t)Reconstructed as:

Images