Sleep movement recognition method and system

Abstract

The present invention provides a sleep movement recognition method, including its implementation on a sleep movement recognition system that cooperates with two continuous-wave Doppler radars. The method comprises: the sleep movement recognition system periodically receives and preprocesses I and Q channel signals sampled by the two radars when a person's sleep movements occur; based on the preprocessed I and Q channel signals, the sleep movement recognition system constructs spectrograms of the two radars and stacks them into a mixed spectrogram; the sleep movement recognition system inputs the mixed spectrogram into a pre-trained sleep movement recognition neural network model to perform sleep movement recognition and output a recognition result. The present invention overcomes the problem of low movement recognition accuracy caused by the small signal transmission angle of existing single-channel radar methods.

Description

Technical Field

[0001] The present invention relates to the field of computer intelligent recognition technology, and in particular to a sleep movement recognition method and system. Background Art

[0002] Sleep is a crucial physiological activity for humans, and high-quality sleep is crucial for human health. In addition to breathing and heart rate, various body movements during sleep are also important indicators of sleep quality and sleep stages. Therefore, identifying sleep movements in daily life is crucial for assessing sleep quality and human health.

[0003] In recent years, with the rapid development of Internet of Things technology, some methods for detecting and identifying sleep movements in daily life have emerged. They can be divided into two categories: contact sensor-based methods and non-contact methods.

[0004] The contact sensor-based method detects sleep movements by sensing changes in angular velocity, acceleration, and pressure caused by body movements using gyroscopes, accelerometers, and pressure sensors. This type of method can effectively detect various sleep movements, but requires the sensor to be in close contact with the human body, which can easily cause discomfort and affect sleep quality.

[0005] Non-contact sleep movement detection methods usually use RGB color cameras, infrared cameras, depth cameras, and microphones as sensors to detect sleep movements, but these methods require collecting images or sounds during sleep, which inevitably leads to users' concerns about privacy.

[0006] In order to solve the above problems, some scholars have proposed a radar-based sleep movement detection method. This method identifies sleep movements by analyzing radar echoes modulated by human movements. It can effectively avoid the problems of contact-induced discomfort and infringement of personal privacy that exist in the above systems or methods. However, since radar is insensitive to movements perpendicular to the signal emission direction, the accuracy of a single radar in identifying certain sleep movements will be limited due to sensitivity problems caused by a small emission angle.

[0007] Therefore, it is necessary to propose a new sleep action recognition method that can overcome the problem of low action recognition accuracy caused by the small signal transmission angle of the existing single-channel radar method. Summary of the Invention

[0008] The technical problem to be solved by the embodiments of the present invention is to provide a method and system for sleep action recognition, which can overcome the problem of low action recognition accuracy caused by the small signal transmission angle of the existing single-channel radar method.

[0009] To solve the above technical problems, an embodiment of the present invention provides a sleep movement recognition method, which is implemented on a sleep movement recognition system that cooperates with two continuous wave Doppler radars. The method includes the following steps:

[0010] The sleep movement recognition system periodically receives the I and Q channel signals sampled by the two radars when the human body's sleep movement occurs and performs pre-processing;

[0011] The sleep action recognition system constructs spectrograms of the two radars based on the preprocessed I and Q channel signals of the two radars, and stacks them into a mixed spectrogram;

[0012] The sleep movement recognition system inputs the mixed spectrum into a pre-trained sleep movement recognition neural network model to perform sleep movement recognition and output a recognition result.

[0013] Among them, one of the two radars is an upper radar installed 1.5 meters above the human body and facing the human waist, and the other is a side radar installed 0.3 meters to the left side of the human body and 0.15 meters from the ground and facing the left side of the human waist.

[0014] The I and Q channel signals of the upper radar are expressed by the following formula (1); the I and Q channel signals of the side radar are expressed by the following formula (2):

[0015]

[0016]

[0017] Among them: I 上 (t), Q 上 (t) is the output of the upper radar I and Q channels; x 上 (t) is the body displacement relative to the upper radar; 上 is the carrier wavelength of the upper radar; θ 上 is a fixed initial phase; φ 上 are respectively the I channel DC offset, Q channel DC offset, I channel gain, Q channel gain, and I / Q channel phase difference of the upper radar, which can be estimated from the I and Q signals by an ellipse fitting algorithm based on least squares; φ 上 are all fixed parameter values; I 侧 (t), Q 侧 (t) is the output of the I and Q channels of the side radar; x 侧 (t) is the body displacement relative to the side radar; λ 侧 is the carrier wavelength of the side radar; θ侧 is a fixed initial phase; φ 侧 are respectively the I channel DC offset, Q channel DC offset, I channel gain, Q channel gain, and I / Q channel phase difference of the side radar, which can be estimated from the I and Q signals by an ellipse fitting algorithm based on least squares; φ 侧 These are fixed parameter values.

[0018] The above radar and the side radar are both pre-processed by performing the following steps, specifically including:

[0019] By using formulas (3) and (4), the Q channel signals of the upper radar and the side radar are corrected to eliminate the gain deviation and phase difference of each I and Q signal;

[0020]

[0021]

[0022] By using formulas (5)-(8), the I and Q signals of the upper radar and the side radar are respectively subjected to high-pass filtering with a cutoff frequency of 2 Hz to reduce the interference of breathing and heartbeat at 0.2-2 Hz;

[0023]

[0024]

[0025]

[0026]

[0027] Among them, a1=3.974, a2=-5.922, a3=3.922, a4=-0.974, b0=0.987, b1=-3.948, b2=5.922, b3=-3.948, b4=0.987, when k=1~4,

[0028] The complex signals of the upper radar and the side radar are constructed respectively by formulas (9) and (10);

[0029]

[0030]

[0031] The specific steps of constructing the spectrum diagrams of the two radars based on the preprocessed I and Q channel signals of the two radars include:

[0032] Get 4320 sampling points for 3.456 seconds corresponding to the complex signal S of the upper radar and the side radar 上 (k),S 上 (k+1),S 上 (k+2)…S 上 (k+4319) and S 侧 (k),S 侧 (k+1),S 侧 (k+2)…S 侧 (k+4319);

[0033] The complex signal S of the upper radar corresponding to the 256 sampling points 上 (k+32j),S 上 (k+32j+1),S 上 (k+32j+2)…S 上 (k+32j+255) and the complex signal S corresponding to the side radar 侧 (k+32j),S 侧 (k+32j+1),S 侧 (k+32j+2)…S 侧 (k+32j+255) are all Fourier transformed to generate a 256×128 two-dimensional array X 上 and X 侧 ; Among them: X 上 The jth column corresponds to S 上 (k+32j),S 上 (k+32j+1),S 上 (k+32j+2)…S 上 (k+32j+255) is the amplitude of each frequency after Fourier transform; X 侧 The jth column corresponds to S 侧 (k+32j),S 侧 (k+32j+1),S 侧 (k+32j+2)…S 侧 (k+32j+255) is the amplitude of each frequency after Fourier transform; j = 0-127, corresponding to 128 columns;

[0034] The 256×128 two-dimensional array X 上 and X 侧 Reduce the dimension to a 128×128 two-dimensional array M 上 and M 侧 Among them, M 上 [i][j]=(X 上[2i][j]+X 上 [2i+1][j]) / 2; M 侧 [i][j]=(X 侧 [2i][j]+X 侧 [2i+1][j]) / 2, i=0~127, j=0~127; M 上 and M 侧 Original spectrograms corresponding to the upper radar and the side radar respectively;

[0035] For a two-dimensional array M 上 and M 侧 All elements of are logarithmically operated to obtain the matrix and in: i=0~127,j=0~127;

[0036] Calculate the matrix separately and The average of all elements to get the corresponding average value and And the matrix and All elements in are binarized to obtain the corresponding matrix after binarization and in, and The binarized spectrograms corresponding to the upper radar and the side radar respectively;

[0037] According to the contour search algorithm, find and All contours in and After the values ​​of the points outside the maximum contour are all set to 0, the denoised spectrum of the upper radar and the side radar are formed respectively.

[0038] The mixed spectrum is formed by stacking the denoised spectrum of the upper radar and the side radar as two channels.

[0039] Wherein, the method further comprises:

[0040] A sleep movement recognition neural network model was predefined and trained using the mini-batch Adam optimization algorithm and historical mixed spectrograms labeled with sleep movement labels as the training dataset. The loss function between the output of the training sample after the neural network and the actual training sample output is as follows:

[0041]

[0042] i, k are index values; M1 is the number of training samples; K is the number of types of sleep movements; L is the cross entropy loss function; a i is the inactivated output of the i-th training sample in the softmax layer; z i is the classification vector of the i-th training sample in the training set, and the target classification z in the vector ik Equal to 1, other categories z ik =0; is the output of softmax, and the likelihood probability of the i-th training sample being predicted to belong to category k is calculated; the loss function E is optimized to minimize the extreme value to obtain the parameters of the neural network.

[0043] The sleep movement recognition neural network model includes four convolutional layers, four pooling layers and three fully connected layers.

[0044] An embodiment of the present invention further provides a sleep movement recognition system connected to two continuous wave Doppler radars, comprising:

[0045] A sleep signal receiving unit is used to periodically receive and pre-process the I and Q channel signals obtained by the two radars respectively sampling when the human body sleeps;

[0046] A spectrum diagram construction unit is used to construct spectrum diagrams of the two radars based on the preprocessed I and Q channel signals of the two radars, respectively, and stack them into a mixed spectrum diagram;

[0047] The sleep movement recognition unit is used to input the mixed spectrum graph into a pre-trained sleep movement recognition neural network model to perform sleep movement recognition and output a recognition result.

[0048] Among them, one of the two radars is an upper radar installed 1.5 meters above the human body and facing the human waist, and the other is a side radar installed 0.3 meters to the left side of the human body and 0.15 meters from the ground and facing the left side of the human waist.

[0049] The implementation of the embodiments of the present invention has the following beneficial effects:

[0050] The present invention utilizes a deep convolutional neural network to treat sleep action recognition based on a dual-channel continuous wave Doppler radar as a multi-value classification problem of images, thereby avoiding the problem of single-channel radar being insensitive to tangential motion and improving the accuracy of sleep action recognition, thereby overcoming the problem of low action recognition accuracy caused by a small signal transmission angle in existing single-channel radar methods. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, without paying any creative work, other drawings obtained based on these drawings still fall within the scope of the present invention.

[0052] Figure 1 A flowchart of a sleep movement recognition method provided by an embodiment of the present invention;

[0053] Figure 2 A schematic diagram of the installation of two continuous wave Doppler radars in a sleep movement recognition method provided by an embodiment of the present invention;

[0054] Figure 3 A structural diagram of a sleep movement recognition neural network model in a sleep movement recognition method provided by an embodiment of the present invention;

[0055] Figure 4 Figure 1 shows the original spectrogram, logarithmically transformed spectrogram, binarized spectrogram, and denoised spectrogram of the upper radar and side radar during a left turning motion in a sleep movement recognition method provided by an embodiment of the present invention. The original spectrogram, logarithmically transformed spectrogram, binarized spectrogram, and denoised spectrogram of the upper radar are (a), (c), (e), and (g), respectively; the original spectrogram, logarithmically transformed spectrogram, binarized spectrogram, and denoised spectrogram of the side radar are (b), (d), (f), and (h), respectively.

[0056] Figure 5 Spectrum diagram of ten typical sleep movements after denoising in a sleep movement recognition method provided by an embodiment of the present invention;

[0057] Figure 6 Schematic diagram of confusion matrices for ten typical sleep movement inputs in a sleep movement recognition method provided by an embodiment of the present invention, when the inputs are a mixed spectrum, an upper radar spectrum, and a side radar spectrum, respectively; (a), (b), and (c) are confusion matrices for the mixed spectrum, the upper radar spectrum, and the side radar spectrum, respectively;

[0058] Figure 7 This is a structural diagram of a sleep movement recognition system provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0059] In order to make the objectives, technical solutions and advantages of the present invention more clear, the present invention will be described in further detail below with reference to the accompanying drawings.

[0060] like Figure 1FIG. 1 shows a sleep movement recognition method proposed in an embodiment of the present invention, which is implemented on a sleep movement recognition system that cooperates with two continuous wave Doppler radars. The method includes the following steps:

[0061] Step S1, the sleep movement recognition system periodically receives the I and Q channel signals sampled by the two radars when a human body moves during sleep and performs pre-processing;

[0062] Step S2: The sleep action recognition system constructs spectrograms of the two radars based on the preprocessed I and Q channel signals of the two radars, respectively, and stacks them into a mixed spectrogram;

[0063] Step S3: The sleep movement recognition system inputs the mixed spectrum into a pre-trained sleep movement recognition neural network model to perform sleep movement recognition and output a recognition result.

[0064] The specific process is as follows: before step S1, Figure 2 As shown, two continuous wave Doppler radars are pre-set, one of which is installed 1.5 meters above the human body and facing the human waist, serving as an upper radar, and the other is installed 0.3 meters to the left of the human body and 0.15 meters from the ground and facing the left side of the human waist, serving as a side radar.

[0065] At the same time, in the sleep action recognition system, a sleep action recognition neural network model is predefined, and the mini-batch Adam optimization algorithm and the historical mixed spectrum graph marked with sleep action labels are used as the training data set for training. Its architecture is as follows: Figure 3 As shown in the figure, the loss function between the output of the training sample after the neural network and the actual training sample output is as follows:

[0066]

[0067] i, k are index values; M1 is the number of training samples; K is the number of types of sleep movements; L is the cross entropy loss function; a i is the inactivated output of the i-th training sample in the softmax layer; z i is the classification vector of the i-th training sample in the training set, and the target classification z in the vector ik Equal to 1, other categories z ik =0; is the output of softmax, and the likelihood probability of the i-th training sample being predicted to belong to category k is calculated; the loss function E is optimized to minimize the extreme value to obtain the parameters of the neural network.

[0068] It should be noted that the sleep movement recognition neural network model consists of four convolutional layers, four pooling layers, and three fully connected layers. The kernel size and number of each convolutional layer are shown in Table 1. Each pooling layer uses 2×2 max pooling. The number of input and output neurons in the fully connected layers is shown in Table 2. Each convolutional layer is followed by a ReLU activation function and a pooling layer. The FC1 and FC2 fully connected layers are followed by a ReLU activation function. The final fully connected layer, FC3, is followed by a softmax activation function.

[0069] Table 1

[0070] type Conv1 Conv2 Conv3 Conv4 Convolution kernel size 3×3 3×3 3×3 3×3 Number of convolution kernels 64 128 256 512

[0071] Table 2

[0072] type FC1 FC2 FC3 Number of input neurons 18432 1024 512 Number of output neurons 1024 512 K

[0073] The sleep movement classification neural network consists of four sets of convolutional layers + ReLU activation function + pooling layer, two sets of fully connected layers + ReLU activation function, and finally one set of fully connected layers + softmax activation function. The connection order of the entire neural network is:

[0074] Conv1-ReLU-Pool1-Conv2-ReLU-Pool2-Conv3-ReLU-Pool3-Conv4-ReLU-Pool4-FC1-ReLU-FC2-ReLU-FC3-softmax;

[0075] Among them, Conv1, Conv2, Conv3, and Conv4 are convolutional layers, FC1, FC2, and FC3 are fully connected layers, Conv1 has 2 input channels and 64 output channels; Conv2 has 64 input channels and 128 output channels; Conv3 has 128 input channels and 256 output channels; Conv4 has 256 input channels and 512 output channels; FC1 has 18432 input neurons and 1024 output neurons; FC2 has 1024 input neurons and 512 output neurons; FC3 has 512 input neurons and K output neurons, where K is the number of sleep behavior types; ReLU is a nonlinear activation function, ReLU(x) = max(0, x);

[0076] The specific operations of the entire neural network are as follows:

[0077] J(c)=f softmax (f FC3 (f ReLU (f FC2 (f ReLU (f FC1 (fPool4 (f Relu (f Conv4 (f Pool3 (f Relu (f Conv3 (f Pool2 (f Relu (f Conv2 (f Pool1 (f Relu (f Conv1 (c)))))))))))))))))

[0078] Where: c = (upper radar spectrum, side radar spectrum) is the normalized vector of the dual radar spectrum; f Relu Corresponding to ReLU activation function; f softmax Corresponding softmax activation function; f Conv1 -f Conv4 Corresponding to the convolution layer; f Pool1 -f Pool4 Corresponding to the pooling layer; f FC1 -f FC3 Corresponding to the fully connected layer.

[0079] It should be noted that the formation process of the historical mixed spectrum is to first construct two corresponding spectrum graphs from two long-term radar signals containing a large number of sleep movements by manually intercepting the signal segments when the sleep movement occurs. Then the two spectrum graphs of each movement are stacked as two channels and manually marked with the label of the sleep movement to which they belong.

[0080] In step S1, first, the two radars are sampled at a frequency of 1250 Hz. The I and Q channel signals of the upper radar at time t are expressed by the following formula (1); the I and Q channel signals of the side radar at time t are expressed by the following formula (2):

[0081]

[0082]

[0083] Among them: I 上 (t), Q 上 (t) is the output of the upper radar I and Q channels; x 上 (t) is the body displacement relative to the upper radar; λ 上 is the carrier wavelength of the upper radar; θ 上 is a fixed initial phase; φ 上They are the I channel DC offset, Q channel DC offset, I channel gain, Q channel gain, and I / Q channel phase difference of the upper radar, which can be estimated from the I and Q signals using the least squares-based ellipse fitting algorithm; φ 上 are all fixed parameter values; I 侧 (t), Q 侧 (t) is the output of the I and Q channels of the side radar; x 侧 (t) is the body displacement relative to the side radar; λ 侧 is the carrier wavelength of the side radar; θ 侧 is a fixed initial phase; φ 侧 They are the I channel DC offset, Q channel DC offset, I channel gain, Q channel gain, and I / Q channel phase difference of the side radar, which can be estimated from the I and Q signals using an ellipse fitting algorithm based on least squares. φ 侧 These are fixed parameter values.

[0084] Secondly, the following steps are performed for pre-processing of both the upper radar and the side radar, including:

[0085] (1.1) Using formulas (3) and (4), the Q channel signals of the upper radar and the side radar are corrected to eliminate the gain deviation and phase difference of each I and Q signal;

[0086]

[0087]

[0088] (1.2) Using formulas (5)-(8), the I and Q signals of the upper radar and the side radar are respectively subjected to high-pass filtering with a cutoff frequency of 2 Hz to reduce the interference of breathing and heartbeat at 0.2-2 Hz;

[0089]

[0090]

[0091]

[0092]

[0093] Among them, a1=3.974, a2=-5.922, a3=3.922, a4=-0.974, b0=0.987, b1=-3.948, b2=5.922, b3=-3.948, b4=0.987, when k=1~4,

[0094] (1.3) Using formulas (9) and (10), construct the complex signals of the upper radar and the side radar respectively;

[0095]

[0096]

[0097] In step S2, first, based on the pre-processed I and Q channel signals of the two radars, the specific steps of constructing the spectrograms of the two radars respectively include:

[0098] (2.1) Obtain the complex signal S corresponding to the upper radar and the side radar for a total of 4320 sampling points for 3.456 seconds 上 (k), S 上 (k+1), S 上 (k+2)…S 上 (k+4319) and S 侧 (k),S 侧 (k+1),S 侧 (k+2)…S 侧 (k+4319);

[0099] (2.2) The complex signal S of the upper radar corresponding to the 256 sampling points 上 (k+32j),S 上 (k+32j+1),S 上 (k+32j+2)…S 上 (k+32j+255) and the corresponding complex signal S of the side radar 侧 (k+32j),S 侧 (k+32j+1),S 侧 (k+32j+2)…S 侧 (k+32j+255) are all Fourier transformed to generate a 256×128 two-dimensional array X 上 and X 侧 ; Among them: X 上 The jth column corresponds to S 上 (k+32j),S 上 (k+32j+1),S 上 (k+32j+2)…S 上 (k+32j+255) is the amplitude of each frequency after Fourier transform; X 侧 The jth column corresponds to S 侧 (k+32j),S 侧 (k+32j+1),S 侧 (k+32j+2)…S 侧(k+32j+255) is the amplitude of each frequency after Fourier transform; j = 0-127, corresponding to 128 columns;

[0100] In one example, the Fourier transform of the complex signals of the overhead radar and the side radar can be performed in the following manner:

[0101] Fourier transform S(k), S(k+1)…S(k+255) to generate the first column of X

[0102] Fourier transform S(k+2), S(k+2)…S(k+256) to generate the second column of X

[0103] This process is deduced in this way until S(k+4064), S(k+4064)…S(k+4319) are Fourier transformed to generate the 128th column of X.

[0104] (2.3) The 256×128 two-dimensional array X 上 and X 侧 Reduce the dimension to a 128×128 two-dimensional array M 上 and M 侧 Among them, M 上 [i][j]=(X 上 [2i][j]+X 上 [2i+1][j]) / 2; M 侧 [i][j]=(X 侧 [2i][j]+X 侧 [2i+1][j]) / 2, i=0~127, j=0~127; M 上 and M 侧 The original spectrograms corresponding to the upper radar and the side radar respectively; in an example, the original spectrograms of the upper radar and the side radar when turning over to the left (from lying on the back to lying on the left side) correspond to the following Figure 4 The spectrograms shown in (a) and (b);

[0105] (2.4) For the two-dimensional array M 上 and M 侧 All elements of are logarithmically operated to obtain the matrix and in: i = 0 to 127, j = 0 to 127; In one example, the logarithmic transformed spectra of the original spectra of the upper radar and the side radar during the left turning action (from lying on the back to lying on the left side) correspond to the following spectra: Figure 4 The spectrograms shown in (c) and (d);

[0106] (2.5) Calculate the matrices separately and The average of all elements to get the corresponding average value and And the matrix and All elements in are binarized to obtain the corresponding matrix after binarization and in, and Corresponding to the spectrum diagrams after binarization of the upper radar and the side radar respectively; in one example, when turning over to the left (lying on the back to the left side), the spectrum diagrams after binarization of the upper radar and the side radar respectively correspond to Figure 4 Spectrum diagrams shown in (e) and (f);

[0107] (2.6) According to the contour search algorithm, find and All contours in and After the values ​​of the points outside the maximum contour are set to 0, the denoised spectra of the upper radar and the side radar are formed respectively; in an example, the denoised spectra of the upper radar and the side radar during the left turn (from lying on the back to the left side) correspond to the following: Figure 4 The spectra are shown in (g) and (h).

[0108] like Figure 5 The figure shows the denoised spectrograms of ten typical sleep movements, including (a) lying on the back to lying on the left side, (b) lying on the left side to lying on the back, (c) lying on the back to lying on the right side, (d) lying on the right side to lying on the back, (e) sitting up, (f) lying down, (g) bending the legs, (h) extending the legs, (i) waving the hands upward, and (j) waving the hands downward. In the two spectrograms for each movement, the left spectrogram is generated from the upper radar data, and the right spectrogram is generated from the side radar data.

[0109] Secondly, the denoised spectrograms of the upper radar and the side radar are stacked as two channels to form a mixed spectrogram.

[0110] In step S4, the mixed spectrum obtained in step S3 is input into the above-mentioned pre-trained sleep movement recognition neural network model to perform sleep movement recognition, and the recognition result is output.

[0111] The following is an experimental verification of the effect of the sleep movement recognition method provided by the embodiment of the present invention: at time T, two-way radar data from (T-3.456) to T (4320 groups of sampling points at 1250Hz in total) are used to construct a spectrum graph, and stack them into a mixed spectrum graph and input them into the sleep movement classification neural network. In actual applications, this step is repeated every 0.5 seconds using the latest 4320 groups of sampling data. The trained sleep movement classification neural network acts as a multi-value classifier to generate the likelihood probability that the mixed spectrum graph belongs to each sleep movement. Finally, the final spectrum graph classification result is calculated based on the likelihood probability output by the neural network. Table 3 shows the sleep movement classification neural network when the input is a mixed spectrum graph, an upper radar spectrum graph, and a side radar spectrum graph. Figure 3 The accuracy of sleep movement classification in this case is Figure 6 is the corresponding confusion matrix, Figure 6 (a) is the confusion matrix when the input is a mixed spectrum graph, Figure 6 (b) is the confusion matrix when the input is the radar spectrum above. Figure 6 (c) is the confusion matrix when the input is the side radar spectrum. From Table 3 and Figure 6 It can be seen that the embodiment of the present invention uses dual-path radar to identify sleeping movements, which can effectively avoid the problem of single-path radar being insensitive to movements perpendicular to the antenna transmission direction due to the small antenna transmission angle, thereby greatly improving the accuracy of sleeping movement recognition.

[0112] Table 3

[0113] enter Mixed spectrum diagram Radar spectrum above Side radar spectrum Classification accuracy 97.22％ 86.15％ 89.71％

[0114] The embodiment of the present invention uses Python 3.9 and the PyTorch 1.10.0 library to implement a sleep movement classification method based on a deep convolutional neural network. All experiments are performed on an NVIDIA GeForce RTX 2080Ti GPU and an Intel i5-10400F CPU machine.

[0115] like Figure 7 FIG. 1 shows a sleep movement recognition system for implementing the sleep movement recognition method according to an embodiment of the present invention, which is connected to two continuous wave Doppler radars and includes:

[0116] The sleep signal receiving unit 110 is used to periodically receive and pre-process the I and Q channel signals obtained by respectively sampling the two radars when the human body sleeps;

[0117] The spectrogram construction unit 120 is used to construct spectrograms of the two radars based on the preprocessed I and Q channel signals of the two radars, respectively, and stack them into a mixed spectrogram;

[0118] The sleep movement recognition unit 130 is configured to input the mixed spectrum into a pre-trained sleep movement recognition neural network model to perform sleep movement recognition and output a recognition result.

[0119] Among them, one of the two radars is an upper radar installed 1.5 meters above the human body and facing the human waist, and the other is a side radar installed 0.3 meters to the left side of the human body and 0.15 meters from the ground and facing the left side of the human waist.

[0120] The implementation of the embodiments of the present invention has the following beneficial effects:

[0121] The present invention utilizes a deep convolutional neural network to treat sleep action recognition based on a dual-channel continuous wave Doppler radar as a multi-value classification problem of images, thereby avoiding the problem of single-channel radar being insensitive to tangential motion and improving the accuracy of sleep action recognition, thereby overcoming the problem of low action recognition accuracy caused by a small signal transmission angle in existing single-channel radar methods.

[0122] It is worth noting that in the above system embodiment, the various units included are only divided according to functional logic, but are not limited to the above division, as long as the corresponding functions can be achieved; in addition, the specific names of the functional units are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of the present invention.

[0123] Those skilled in the art will understand that all or part of the steps in the above-mentioned embodiment method can be completed by instructing the relevant hardware through a program, and the program can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc.

[0124] The above disclosure is only a preferred embodiment of the present invention and certainly cannot be used to limit the scope of the present invention. Therefore, equivalent changes made according to the claims of the present invention are still within the scope of the present invention.

Claims

1. A sleep movement recognition method, characterized in that: The method is implemented on a sleep movement recognition system that cooperates with two continuous wave Doppler radars. The method includes the following steps: The sleep movement recognition system periodically receives the I and Q channel signals obtained by sampling the two radars when the human body's sleep movement occurs and performs preprocessing; The sleep action recognition system constructs spectrograms of the two radars based on the preprocessed I and Q channel signals of the two radars, and stacks them into a mixed spectrogram; The sleep movement recognition system inputs the mixed spectrum into a pre-trained sleep movement recognition neural network model to perform sleep movement recognition and output a recognition result; One of the two radars is an upper radar facing the human waist, and the other is a side radar facing the left side of the human waist. The I and Q channel signals of the upper radar are expressed by the following formula (1); the I and Q channel signals of the side radar are expressed by the following formula (2); （1）； （2）； in: 、 The outputs of the I and Q channels of the upper radar; is the body displacement relative to the above radar; is the carrier wavelength of the upper radar; is a fixed initial phase; 、 、 、 、 are the I channel DC offset, Q channel DC offset, I channel gain, Q channel gain, and I / Q channel phase difference of the upper radar, respectively, estimated from the I and Q signals by an ellipse fitting algorithm based on least squares; 、 All are fixed parameter values; 、 The output of the I and Q channels of the side radar; is the body displacement relative to the side radar; is the carrier wavelength of the side radar; is a fixed initial phase; 、 、 、 、 The I channel DC offset, Q channel DC offset, I channel gain, Q channel gain, and I / Q channel phase difference of the side radar are estimated from the I and Q signals by using an ellipse fitting algorithm based on least squares; 、 All are fixed parameter values; The following steps are performed on both the upper radar and the side radar for preprocessing, specifically including: By using formulas (3) and (4), the Q channel signals of the upper radar and the side radar are corrected to eliminate the gain deviation and phase difference of each I and Q signal; （3）； （4）。 2. The sleep movement recognition method according to claim 1, wherein: The upper radar is installed 1.5 meters above the human body, and the side radar is installed 0.3 meters to the left of the human body and 0.15 meters from the ground.

3. The sleep movement recognition method according to claim 2, wherein: By using formulas (5)-(8), the I and Q signals of the upper radar and the side radar are respectively subjected to high-pass filtering with a cutoff frequency of 2 Hz to reduce the interference of breathing and heartbeat of 0.2-2 Hz; （5）； （6）； （7）； （8）； in, , , , , , , , , ,when hour, , , , ; The complex signals of the upper radar and the side radar are constructed respectively by formulas (9) and (10); （9）； （10）。 4. The sleep movement recognition method according to claim 3, wherein: The specific steps of constructing the spectrograms of the two radars based on the preprocessed I and Q channel signals of the two radars include: Get 4320 sampling points for 3.456 seconds corresponding to the complex signals of the upper radar and the side radar and ; The complex signal of the upper radar corresponding to the 256 sampling points and the complex signal corresponding to the side radar Perform Fourier transform to generate a 256×128 two-dimensional array and ;in: No. j Column correspondence The amplitude of each frequency after Fourier transform; No. j Column correspondence The amplitude of each frequency after Fourier transform; j =0-127, corresponding to 128 columns; Convert a 256×128 two-dimensional array and Reduce the dimension to a 128×128 two-dimensional array and ;in, ; , ; and Original spectrograms corresponding to the upper radar and the side radar respectively; For a two-dimensional array and All elements of are logarithmically operated to obtain the matrix and ,in: , , ; Calculate the matrix separately and The average of all elements to get the corresponding average value and , and the matrix and All elements in are binarized to obtain the corresponding matrix after binarization and ;in, , ; and The binarized spectrograms corresponding to the upper radar and the side radar respectively; According to the contour search algorithm, find and All contours in and After the values ​​of the points outside the maximum contour are all set to 0, the denoised spectrum of the upper radar and the side radar are formed respectively.

5. The sleep movement recognition method according to claim 4, wherein: The mixed spectrum is formed by stacking the denoised spectrum of the upper radar and the side radar as two channels.

6. The sleep movement recognition method according to claim 1, wherein: The method further comprises: A sleep movement recognition neural network model was predefined and trained using the mini-batch Adam optimization algorithm and historical mixed spectrograms labeled with sleep movement labels as the training dataset. The loss function between the output of the training sample after the neural network and the actual training sample output is as follows: （11）； i, k is the index value; is the number of training samples; K is the number of types of sleep movements; L is the cross entropy loss function; For the i The inactive output of the softmax layer for each training sample; is the classification vector of the i-th training sample in the training set, and the target classification in the vector Equal to 1, other categories =0; is the output of softmax, and the i The training samples are predicted to belong to the classification k Likelihood probability; for the loss function E Optimize and find the minimum extreme value to obtain the parameters of the neural network.

7. The sleep movement recognition method according to claim 6, wherein: The sleep action recognition neural network model includes four convolutional layers, four pooling layers and three fully connected layers.

8. A sleep movement recognition system, characterized in that: It is connected to two continuous wave Doppler radars, including: The sleep signal receiving unit is used to periodically receive and pre-process the I and Q channel signals obtained by sampling the two radars when the human body sleeps; A spectrum diagram construction unit is used to construct spectrum diagrams of the two radars based on the preprocessed I and Q channel signals of the two radars, respectively, and stack them into a mixed spectrum diagram; A sleep movement recognition unit, configured to input the mixed spectrum into a pre-trained sleep movement recognition neural network model to perform sleep movement recognition and output a recognition result; One of the two radars is an upper radar facing the human waist, and the other is a side radar facing the left side of the human waist. The I and Q channel signals of the upper radar are expressed by the following formula (1); the I and Q channel signals of the side radar are expressed by the following formula (2); （1）； （2）； in: 、 The outputs of the I and Q channels of the upper radar; is the body displacement relative to the above radar; is the carrier wavelength of the upper radar; is a fixed initial phase; 、 、 、 、 are the I channel DC offset, Q channel DC offset, I channel gain, Q channel gain, and I / Q channel phase difference of the upper radar, respectively, estimated from the I and Q signals by an ellipse fitting algorithm based on least squares; 、 All are fixed parameter values; 、 The output of the I and Q channels of the side radar; is the body displacement relative to the side radar; is the carrier wavelength of the side radar; is a fixed initial phase; 、 、 、 、 The I channel DC offset, Q channel DC offset, I channel gain, Q channel gain, and I / Q channel phase difference of the side radar are estimated from the I and Q signals by using an ellipse fitting algorithm based on least squares; 、 All are fixed parameter values; The following steps are performed on both the upper radar and the side radar for preprocessing, specifically including: By using formulas (3) and (4), the Q channel signals of the upper radar and the side radar are corrected to eliminate the gain deviation and phase difference of each I and Q signal; （3）； （4）。 9. The sleep movement recognition system according to claim 8, wherein: One of the two radars is an upper radar installed 1.5 meters above the human body and facing the human waist, and the other is a side radar installed 0.3 meters to the left side of the human body and 0.15 meters from the ground and facing the left side of the human waist.

Images