Non-contact sleep stage classification method and device based on multi-task expert network

Abstract

The invention discloses a non-contact sleep stage classification method and device based on a multi-task expert network. The method comprises the following steps: carrying out signal data acquisition by utilizing IR-UWB radar equipment and positioning the position of a bed; detecting whether the detected individual is in a bed range, and if the detected individual is in the bed, extracting activity signal data of the detected individual; preprocessing the signal data of the tested individual; sleep stage classification is carried out on a public data set composed of healthy objects in combination with a feature encoder and a sequence encoder, and a pre-trained model is obtained in combination with a contrast learning technology; a multi-task learning module is introduced into a pre-training model to extract apnea information in a signal as auxiliary information, and the model is finely adjusted by using the preprocessed signal of a tested individual, so that sleep stage classification is performed on the tested individual, and parameters related to sleep are calculated. According to the invention, classification of human body all-night sleep stages is realized through a non-contact means, and prediction precision of human body sleep stage classification is improved.

Description

Technical Field

[0001] The present invention relates to the field of non-contact sensing and the field of deep learning of artificial intelligence, and in particular to a non-contact sleep stage classification method and device based on a multi-task expert network. Background Art

[0002] Sleep is an indispensable and important process in human life, occupying nearly one-third of a person's day. High-quality sleep not only helps the body recover, but also plays an important role in memory consolidation and learning ability improvement. In addition, sleep is closely related to the function of the immune system, cardiovascular health, and emotional stability. However, in modern society, many people face sleep problems such as chronic insomnia, narcolepsy, and severe sleep apnea syndrome, which seriously affect people's sleep quality. Poor sleep quality may be a cause of obesity, hypertension, diabetes, and some cardiovascular diseases. A night of high-quality sleep should include sufficient total sleep time and a reasonable distribution of sleep stages. Among them, according to their respective physiological functions and characteristics, the human body's full night sleep can be divided into five sleep stages: Wake, N1, N2, N3, and REM. Therefore, monitoring the sleep stage of the human body all night in a home environment is of great significance to human health. It can not only provide early warning of some potential sleep problems in the human body and remind users to seek medical treatment in time, but also be used to monitor the recovery of sleep within a period of time after medical treatment, so that doctors can pay attention to the patient's sleep condition at any time.

[0003] In hospitals, doctors use polysomnography (PSG) to monitor the human body's various physiological signals (including electroencephalogram, electrocardiogram, electromyogram, and electrooculogram, etc.) throughout the night, divide the whole night's sleep into multiple 30-second periods, and mark a sleep stage for each period according to the changes in the physiological signals in the corresponding period. However, using PSG for sleep monitoring requires professional medical staff to assist the individual being tested in wearing a variety of medical instruments, and medical experts are required to mark the measurement results, which is expensive and time-consuming. Therefore, PSG is currently limited to hospitals and is not suitable for long-term monitoring scenarios at home.

[0004] In recent years, research on wearable devices in the field of sleep stage classification has achieved certain results, making it a solution for home sleep monitoring. It monitors the activity status of the human body by wearing devices such as bracelets, watches, or chest and abdominal belts on the human body, thereby achieving sleep stage classification. However, the data source of wearable devices is single (for example, bracelets can only obtain pulse rate and body movement data), and the subjects are required to wear the device all night. Long-term and frequent wearing of the device will cause contact discomfort to the subjects, and there are also problems such as unusable measurement results due to improper wearing of the device. At the same time, existing research works mostly use public data sets for performance testing rather than conducting experiments in real sleep scenarios. Therefore, the practicality of these works needs to be studied.

[0005] Another solution for home sleep monitoring is sleep monitoring based on non-contact sensing technology. Non-contact sensing technology senses the human body by transmitting wireless signals (including RFID, Wi-Fi, etc.) to the human body and analyzing the echo signals. It has the advantages of low cost, non-intrusiveness and wide sensing range. However, the existing sleep monitoring work based on non-contact sensing has the following limitations. First, most existing works use CNN or recurrent neural network to model the temporal relationship in the sequence, but these methods cannot well capture the global dependency and long-distance dependency in the ultra-long signal sequence of the whole night; second, most of the existing works ignore the relationship between sleep stages and sleep disorders, and these sleep disorder information (such as sleep apnea events) has not been fully mined and utilized to assist in the classification of sleep stages. Therefore, the existing work can achieve good performance on healthy subjects, but the performance on patients with sleep disorders is not satisfactory. This is because there are a large number of abnormal sleep events in the whole night sleep of patients with sleep disorders, which disrupts the normal sleep stage transition pattern and makes their sleep structure more complicated; finally, most of the existing works are carried out in a controlled environment, and their performance is very sensitive to changes in the environment. Therefore, accurate full-night sleep monitoring using contactless sensing technology remains challenging. Summary of the invention

[0006] In order to solve the above technical problems, the present invention provides a non-contact sleep stage classification method and device based on a multi-task expert network, which encapsulates the currently commercially available IR-UWB radar chip and Raspberry Pi into a simple and easy-to-use small portable device, and uses it to perform non-contact and non-intrusive human sleep stage classification throughout the night. The device can be quickly deployed and used in a home environment, improves the prediction accuracy of human sleep stage classification, and can achieve good performance on healthy subjects and patients with different degrees of sleep apnea hypopnea syndrome, and can be used for the initial screening of potential sleep disorders and postoperative follow-up of patients with sleep disorders.

[0007] The technical solution of the present invention is:

[0008] A contactless sleep stage classification method based on a multi-task expert network comprises the following steps:

[0009] 1) Using radar equipment to collect signal data, and obtaining the position range of the bed according to the collected signal data;

[0010] 2) Performing human presence detection according to the position range of the bed. If the individual is on the bed, extracting the one-dimensional activity signal data of the individual and calculating the corresponding amplitude and phase sequence;

[0011] 3) Preprocessing the amplitude and phase sequences of the measured individual to remove noise from the signal;

[0012] 4) using a public dataset of healthy subjects to train a sleep stage classification model, and using contrastive learning techniques to enhance the robustness of the sleep stage classification model, thereby obtaining a pre-trained sleep stage classification model;

[0013] 5) using the preprocessed signal of the individual under test to fine-tune the feature encoder and the sequence encoder in the pre-trained sleep stage classification model, and using the output of the fine-tuned sequence encoder to perform multi-task learning, wherein the multi-task learning takes sleep stage classification as the main task and sleep apnea event recognition as the auxiliary task, and obtains a sleep stage classification model based on multi-task learning by jointly training the main task and the auxiliary task, and uses the trained sleep stage classification model based on multi-task learning to classify the sleep stage of the individual under test;

[0014] 6) Calculate sleep-related parameters based on the sleep stage classification results.

[0015] Further, step 1) places the IR-UWB radar device on the bedside table for signal data collection, and collects a two-dimensional complex signal matrix (one dimension represents the distance between the device and the reflector, and one dimension represents time). Use a 30-second sliding window to divide the two-dimensional matrix into multiple non-overlapping 30-second sub-matrices along the time dimension, and then eliminate the signal reflected by the static objects in the environment from each sub-matrix, and calculate the position with the maximum signal intensity in the sub-matrix, and extract the 30-second one-dimensional signal sequence on the coordinates of the position. Then, determine whether the signal sequence is a human breathing signal, and record the human body position corresponding to each human breathing signal. Finally, the position range of the bed is obtained by combining all the recorded position information and the width of the bed.

[0016] Furthermore, in step 2), on the 30-second sub-matrix, the observation range is limited to the position range corresponding to the bed, and human presence detection is performed on the signal data within the position range of the bed. If the individual being tested is in bed, the one-dimensional activity signal data of the individual being tested is extracted, and the corresponding amplitude and phase sequences are calculated.

[0017] Furthermore, step 3) uses algorithms including interpolation, correction and standardization to perform signal preprocessing on the amplitude and phase sequences of the measured individual respectively, so as to remove noise in the signal and improve the quality of the signal.

[0018] Furthermore, in step 4), on a public dataset consisting only of healthy subjects, a CNN-based feature encoder and a self-attention mechanism-based sequence encoder are combined to perform sleep stage classification, the sleep structure and characteristics of healthy subjects are modeled, and contrastive learning technology is used to enhance the robustness of the model, thereby obtaining a model pre-trained on this dataset.

[0019] Furthermore, in step 5), a multi-task learning module is introduced on the pre-trained model, and sleep stage classification is used as the main task, and sleep apnea event recognition is used as the auxiliary task. The performance of the sleep stage classification task is improved by jointly training two related sleep tasks. The model is fine-tuned using the pre-processed signal of the tested individual, so as to classify the sleep stage of the tested individual.

[0020] Furthermore, step 6) calculates various sleep-related parameters based on the results of the sleep stage classification, including total sleep time, proportion of time in each sleep stage, time in bed, sleep delay, sleep efficiency, total awakening time after falling asleep, and number of awakenings after falling asleep.

[0021] Preferably, in step 1), the IR-UWB radar device is placed on a bedside table and faces the direction of the measured individual, and the distance and angle to the measured individual are within the maximum detection range and maximum angle range of the radar chip.

[0022] Preferably, the window length (ie, 30 seconds) used in step 1) is the same as the duration of a sleep stage defined medically.

[0023] Preferably, the method for calculating the position with the largest signal strength in the 30-second matrix in step 1) is: the signal amplitude corresponding to the 30-second complex matrix is ​​obtained by taking the signal modulus, and superimposed along the time axis to obtain the 30-second total amplitude strength corresponding to each distance coordinate, and the distance coordinate with the largest total amplitude strength is the desired result.

[0024] Preferably, in step 1), the method for determining whether the signal sequence is a human breathing signal is as follows: when the signal sequence satisfies the following two conditions at the same time, the signal sequence can be considered to be a human breathing signal:

[0025] First, the signal phase sequence corresponding to the complex signal sequence is calculated by taking the phase angle, and the range of the phase sequence (i.e., the difference between the maximum and minimum values) is calculated. Combined with the wavelength of the signal, the maximum displacement of the signal sequence within 30 seconds is calculated, and this maximum displacement is less than a set threshold α;

[0026] Secondly, the autocorrelation function is used for the amplitude and phase sequences of the signal to obtain the corresponding autocorrelation curves, and the observation range is set within the range corresponding to the normal breathing frequency of the human body. One of the amplitude and phase sequences satisfies: there is a maximum value within the range and the maximum value is greater than a set threshold β.

[0027] Preferably, the method for detecting the presence of a human body in step 2) is: the total amplitude intensity of 30 seconds corresponding to each distance coordinate is obtained by the method in step 1), which can be expressed as a curve. The observation range is limited to the range corresponding to the bed. If there is a maximum value on the curve and the maximum value is greater than a set threshold value γ, it is determined that the human body is within the range of the bed.

[0028] Preferably, in step 3), the interpolation method is a fusion algorithm including spline interpolation method and polynomial interpolation method; the correction method needs to first use the ADF (augmented Dickey-Fuller) algorithm to determine whether the signal sequence is in a non-stationary state, and then perform a detrend operation on the non-stationary signal sequence; the standardization method is the Z-score method.

[0029] Preferably, in step 4), the specific steps of combining a CNN-based feature encoder and a self-attention mechanism-based sequence encoder to classify sleep stages and using contrastive learning technology to enhance the robustness of the model are:

[0030] a) Perform signal preprocessing on the amplitude and phase sequences of the signal data in the public data set, and input them into a CNN encoder to extract local features in the signal, and add the obtained results to obtain a fused local feature matrix;

[0031] b) Add the fused local feature matrix to a position encoding matrix representing the position of each element in the signal to inject position information, and then input the result into a sequence encoder (the encoder is a two-layer multi-head self-attention module) to obtain the sequence embedding matrix 1;

[0032] c) Input the same signal sequence into the CNN encoder and sequence encoder again. Due to the randomness of the Dropout function in the sequence encoder, a new sequence embedding matrix 2 is obtained. Matrix 1 and matrix 2 form a "positive sample pair". When other signal data in the same batch as the current input signal data are input into the CNN encoder and sequence encoder, the obtained matrix forms multiple "negative sample pairs" with matrix 1; by optimizing the infoNCE loss function, the distance between positive sample pairs is continuously minimized during the training process, and the distance between negative sample pairs is maximized;

[0033] d) The sequence embedding matrix 1 is input into a two-layer fully connected neural network for five sleep stage classification (Wake, N1, N2, N3 and REM).

[0034] Preferably, in step 5), a multi-task learning module is introduced on the pre-trained model, and the model is fine-tuned to classify the sleep stages of the tested individual. The specific steps are:

[0035] a) The preprocessed signal of the tested individual is input into the pre-trained model (i.e., CNN encoder and sequence encoder, but the last fully connected prediction layer is removed), and the contrastive learning technique is also used to fine-tune the model with a smaller learning rate.

[0036] b) Input the sequence embedding matrix output by the sequence encoder into a multi-task learning module, and take sleep stage classification as the main task and sleep apnea event recognition as the auxiliary task, and improve the performance of the sleep stage classification task by jointly training two related sleep tasks. Specifically, the multi-task learning module extracts the shared features and unique features of the two tasks from the input data through three expert clusters, and then outputs the prediction results of the two tasks, namely the sleep stage classification result and the sleep apnea event prediction result, by weighted summation, thereby achieving the sleep stage classification of the tested individual. During training, this module uses a large learning rate to update parameters.

[0037] Preferably, in step 6), sleep delay refers to the time difference between going to bed and falling asleep for the first time, and sleep efficiency is the ratio of total sleep time to bed time.

[0038] The present invention also provides a non-contact sleep stage classification device based on a multi-task expert network, which includes a portable signal acquisition device and an information processing device; the portable signal acquisition device includes a radar chip and a microcomputer; the radar chip transmits signals to the environment through a transmitting antenna, and uses a receiving antenna to receive echo signal data reflected by human bodies and objects in the environment, and then transmits it to the microcomputer. The microcomputer transmits the received signal data to the information processing device by wireless transmission, and the information processing device uses the above method to perform data processing.

[0039] Preferably, the radar chip is an IR-UWB radar chip, and the microcomputer is a Raspberry Pi.

[0040] The present invention discloses a non-contact sleep stage classification method and device based on a multi-task expert network, which encapsulates the currently commercially available IR-UWB radar chip and Raspberry Pi into a simple and easy-to-use small portable device, and uses it to perform non-contact and non-intrusive human sleep stage classification throughout the night. The device can be quickly deployed and used in a home environment, improves the prediction accuracy of human sleep stage classification, and can achieve good performance on healthy subjects and patients with different degrees of sleep apnea hypopnea syndrome, solving the problem that the existing sleep monitoring method is only applicable to healthy people, while the prediction performance is reduced on real patients. It can be used for the initial screening of potential sleep disorders and the follow-up of patients with sleep disorders after surgery, and is suitable for smart elderly care, smart home and other fields. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] Figure 1 is a flow chart of a non-contact sleep stage classification method based on a multi-task expert network in an embodiment of the present invention;

[0042] Figure 2 A schematic diagram of a process flow corresponding to the method provided by the present invention;

[0043] Figure 3 This is a flowchart of using a public data set to pre-train a sleep stage classification model in an embodiment of the present invention;

[0044] Figure 4 This is a flow chart of fine-tuning a sleep stage classification model using measured individual data in an embodiment of the present invention. DETAILED DESCRIPTION

[0045] In order to make the purpose, technical solutions and advantages of the present invention more clear, the present invention is further described in detail below through specific implementations and in conjunction with the accompanying drawings.

[0046] like Figure 1As shown, a non-contact sleep stage classification method based on a multi-task expert network provided by an embodiment of the present invention includes the following steps:

[0047] Step S1: Place the IR-UWB (pulse ultra-wideband) radar device on the bedside table to collect signal data. After segmenting the signal data and eliminating static reflections, extract the data at the position coordinates with the largest signal amplitude. Next, determine whether the extracted signal data is a human breathing signal, and record the position information corresponding to each human breathing signal. Finally, combine all the recorded position information and the width of the bed to obtain the position range of the bed.

[0048] Step S2: limit the observation range to the range corresponding to the bed, perform human presence detection on the signal data within the bed range, and if the individual under test is in bed, extract the activity signal data of the individual under test and calculate the corresponding amplitude and phase sequence.

[0049] Step S3: Use algorithms including interpolation, correction and standardization to perform signal preprocessing on the amplitude and phase sequences of the individual being measured, so as to remove noise from the signal and improve the quality of the signal.

[0050] Step S4: On a public dataset consisting only of healthy subjects, a CNN-based feature encoder and a self-attention mechanism-based sequence encoder are combined to perform sleep stage classification, model the sleep structure and characteristics of healthy subjects, and use contrastive learning technology to enhance the robustness of the model, thereby obtaining a model pre-trained on this dataset.

[0051] Step S5: Introduce a multi-task learning module on the pre-trained model, take sleep stage classification as the main task, take sleep apnea event recognition as the auxiliary task, and improve the performance of the sleep stage classification task by jointly training two related sleep tasks. Fine-tune the model using the pre-processed signal of the tested individual to classify the sleep stage of the tested individual.

[0052] Step S6: Based on the results of the sleep stage classification, various sleep-related parameters are calculated, including total sleep time, proportion of time in each sleep stage, time in bed, sleep delay, sleep efficiency, total awakening time after falling asleep, and number of awakenings after falling asleep.

[0053] Figure 2 The present invention provides a flow chart corresponding to the method.

[0054] In one embodiment, the above step S1 specifically includes:

[0055] Step S11: Place the IR-UWB radar device on the bedside table, facing the direction of the individual being measured, and the distance and angle to the individual being measured are within the maximum detection range and maximum angle range of the radar chip. The radar chip transmits signals to the environment through the transmitting antenna, and uses the receiving antenna to receive the echo signals reflected by the human body and objects in the environment, and then transmits them to the Raspberry Pi (a microcomputer) through the DuPont line, so as to store the received signal data on the Raspberry Pi. Then, the data is transmitted to the computer by wireless transmission through the Internet of Things card on the Raspberry Pi for subsequent data processing.

[0056] Step S12: Carrier demodulation is performed on the signal data collected by the IR-UWB radar chip, including in-phase / quadrature sampling and low-pass filtering, to obtain a baseband signal. The baseband signal is a two-dimensional complex matrix, one dimension represents the distance between the device and the reflector, and one dimension represents the time. Considering that the duration of a sleep stage defined medically is 30 seconds, a 30-second sliding window is used here to perform segmented operations on the signal data collected throughout the night. Specifically, a 30-second sliding window is used to divide the two-dimensional matrix into multiple non-overlapping 30-second sub-matrices along the time dimension, and then a background subtraction algorithm is performed on each sub-matrix to eliminate the interference caused by the signals reflected by static objects in the environment. The specific operation can be expressed as the following formulas (1)-(2):

[0057] S i =σS i-1 +(1-σ)U i (1)

[0058] U i =U i -S i (2)

[0059] Among them, U i is the data of the i-th row in the 30-second signal matrix, representing the received signals at each distance corresponding to time i, S i is the static reflection that needs to be eliminated, σ is a hyperparameter, and S0=U0.

[0060] Step S13: On the 30-second signal matrix after background subtraction, calculate the position with the maximum signal intensity in the matrix. Specifically, the signal amplitude corresponding to the 30-second complex matrix is ​​obtained by taking the signal modulus, and superimposed along the time axis to obtain the 30-second total amplitude intensity corresponding to each distance coordinate, and the distance coordinate with the maximum total amplitude intensity is the desired result. Finally, on the 30-second matrix before background subtraction, extract the 30-second one-dimensional signal sequence on the coordinate according to the obtained distance coordinate.

[0061] Step S14: Determine whether the extracted 30-second signal sequence is a human breathing signal. Specifically, when the signal sequence satisfies the following two conditions at the same time, it can be considered that the signal sequence is a human breathing signal:

[0062] First, the signal phase sequence corresponding to the complex signal sequence is calculated by taking the phase angle, and the range of the phase sequence (i.e., the difference between the maximum and minimum values) is calculated. Combined with the wavelength of the signal, the maximum displacement of the signal sequence within 30 seconds is calculated, and this maximum displacement is less than a set threshold α;

[0063] Secondly, the autocorrelation function is used for the amplitude and phase sequences of the signal to obtain the corresponding autocorrelation curves, and the observation range is set within the range corresponding to the normal breathing frequency of the human body. One of the amplitude and phase sequences satisfies: there is a maximum value within the range and the maximum value is greater than a set threshold β.

[0064] According to the method, the position information corresponding to each human respiratory signal is recorded. Finally, a set of position information corresponding to all respiratory signals is obtained.

[0065] Step S15: Calculate the median of the position set, set the median as the midpoint of the bed position, and combine it with the width information of the bed to obtain the specific position range of the bed.

[0066] In one embodiment, the above step S2 specifically includes:

[0067] Step S21: For the 30-second signal matrix after background subtraction, the signal amplitude corresponding to the 30-second complex matrix is ​​obtained by taking the signal modulus, and superimposed along the time axis to obtain the 30-second total amplitude intensity corresponding to each distance coordinate, which can be expressed as a curve. The observation range is limited to the position coordinate range corresponding to the bed. If there is a maximum value on the curve and the maximum value is greater than a set threshold γ, it is determined that the human body is within the bed range.

[0068] Step S22: If the human body is within the bed, the position coordinates of the largest maximum value on the curve are used as the position coordinates of the human body, and based on the coordinates, the human body's activity signal data (i.e., one-dimensional time series) is extracted from the 30-second complex signal matrix before background subtraction. Finally, the corresponding amplitude and phase sequence are calculated by taking the modulus and phase angle.

[0069] In one embodiment, the above step S3 specifically includes:

[0070] Step S31: Solve the frame loss problem in the signal collection process through the signal interpolation algorithm. Here, a fusion algorithm including spline interpolation and polynomial interpolation is proposed. Let n be the length of a signal segment with continuous frame loss, η1 and η2 are two pre-set length thresholds. The specific method of the algorithm is: when n<η1, use the spline interpolation method to interpolate the segment; when η1≤n<η2, use the polynomial interpolation method to interpolate the segment; when n≥η2, the segment has serious frame loss, and the 30-second signal sequence in which the segment is located is directly discarded.

[0071] Step S32: Solve the problem of signal waveform distortion through signal correction algorithm. First, use ADF algorithm to determine whether the signal sequence is in a non-stationary state. Second, perform detrend operation on the non-stationary signal sequence;

[0072] Step S33: Scale the numerical ranges of different signal sequences to the same scale through a signal standardization algorithm. Here, the Z-score method is used to standardize the data, and each signal sample point is subtracted from the mean of the 30-second signal sequence and then divided by the standard deviation of the 30-second signal sequence.

[0073] In one embodiment, the above step S4 specifically includes:

[0074] Step S41: Use a public data set as a data set for model pre-training. The public data set collects wireless signal data of healthy subjects sleeping all night in different bedrooms, and every 30 seconds of data is labeled with a sleep stage. The data set is divided using a ten-fold cross-validation method, and it is ensured that the data of the same subject only appears in one fold. Finally, the model with the best performance (F1 score) on the ten folds is selected as the final pre-training model.

[0075] Step S42: Calculate the amplitude and phase sequences of the signals in the public data set respectively, and perform a Z-score operation on each 30-second sequence. Then, considering that there is a certain timing relationship and transition pattern between different sleep stages, the N 30-second signal sequences are directly spliced ​​together in the timing dimension to obtain the spliced ​​amplitude long sequence samples and phase long sequence samples. Figure 3As shown in the figure, the amplitude long sequence samples and the phase long sequence samples are respectively input into a CNN encoder to extract local features in the long sequence samples. The specific structure of the CNN encoder is as follows: if the current layer is an odd layer, the layer consists of a one-dimensional convolution layer, a batch normalization layer and a leaky ReLU layer; if the current layer is an even layer, the layer consists of a one-dimensional convolution layer, a leaky ReLU layer and a maximum pooling layer. The last layer is an average pooling layer, which is used to ensure that the length of the output feature sequence is the same as the number of sleep stages corresponding to the long sequence samples. Finally, the results obtained after inputting the amplitude long sequence samples and the phase long sequence samples into the CNN encoder are added to obtain the fused feature sequence.

[0076] Step S43: First, add a position encoding matrix to the fused feature sequence. The initial value of the encoding is the position coordinate value of each element in the sequence, and the encoding will be continuously updated during the training process of the model. Then, the sequence data is input into a sequence encoder based on a multi-head self-attention mechanism to learn the long-distance and short-distance dependencies between different elements in the sequence data. The sequence encoder is a two-layer structure. In each layer, the input data will be copied into three copies (Q, K, V), where Q represents the query matrix (Query), K represents the key matrix (Key), and V represents the value matrix (Value), so as to perform a multi-head self-attention operation. The specific operation can be expressed as the following formulas (3)-(4):

[0077]

[0078] A=concat(Attention(Q 1 ,K 1 ,V 1 ),…,Attention(Q h ,K h ,V h )) (4)

[0079] Where d is the feature dimension of the input sequence, h is the number of attention heads, and Q i ,K i ,V iis the result of segmentation on the feature dimensions of Q, K, and V, and i∈[1,h], concat represents the matrix concatenation operation, and A represents the sequence data after attention enhancement. Then, the sequence data after attention enhancement and the original input data are added (Add) through residual connection and input into the layer normalization (Norm) layer. Next, the data is input into the fully connected neural network (MLPs), and then added again through residual connection and the data before input into the fully connected neural network, and finally input into the layer normalization layer to obtain the output result. Let the final output of the sequence encoder be the sequence embedding matrix 1.

[0080] Step S44: Use contrastive learning technology to enhance the quality and generalization of the learned sequence embedding matrix. Repeat steps S42 and S43, and input the same signal sequence into the CNN encoder and sequence encoder again. Due to the randomness of the Dropout function in the sequence encoder, a new sequence embedding matrix 2 is obtained. Matrix 1 and matrix 2 form a "positive sample pair". After other signal data in the same batch as the current input signal data are input into the CNN encoder and sequence encoder, the obtained matrix forms multiple "negative sample pairs" with matrix 1. By optimizing the infoNCE loss function, the distance between positive sample pairs is continuously minimized during the training process, and the distance between negative sample pairs is maximized. Here, the cosine similarity is used to measure the distance between two samples. The higher the similarity, the closer the distance, and the lower the similarity, the farther the distance.

[0081] Step S45: Input the sequence embedding matrix 1 into a two-layer fully connected neural network to perform five sleep stage classifications (Wake, N1, N2, N3 and REM).

[0082] In one embodiment, the above step S5 specifically includes:

[0083] Step S51: Divide the data obtained after data preprocessing in step S3 into a training set, a validation set, and a test set (8:1:1). The data in each set are independent of each other, and the data of the same subject will only appear in one set. In this data set, each 30-second data segment corresponds to a sleep stage label and a sleep apnea event label. Fine-tune the pre-trained model obtained in step S4 on the training set (see step S52 for details), select the model with the best performance (F1 score) based on the validation set, and perform a performance test on the test set.

[0084] Step S52: directly splice N 30-second signal sequences together in the time dimension to obtain spliced ​​amplitude long sequence samples and phase long sequence samples. Figure 4As shown, the amplitude long sequence samples and phase long sequence samples of the tested individual are respectively input into the pre-trained model in step S4 (i.e., CNN encoder and sequence encoder, but the last fully connected prediction layer is removed), and the contrastive learning technique is also used, but the parameters of the model are fine-tuned at a smaller learning rate.

[0085] Step S53: Input the sequence embedding matrix output by the sequence encoder into a multi-task learning module for multi-task learning. Considering that there is a certain connection between sleep stages and sleep apnea (i.e., specific apnea events will occur in specific sleep stages, and too many apnea events will cause sleep stage transitions, etc.), sleep stage classification is used as the main task, and sleep apnea event recognition is used as the auxiliary task. The performance of the sleep stage classification task is improved by jointly training two related sleep tasks. The specific operations in the multi-task learning module can be expressed as the following formulas (5)-(8):

[0086] E k+ =MLPs(H) (5)

[0087] E k,S =concat(E k ,E S ) (6)

[0088] G k =softmax(MLPs(H))E k,S (7)

[0089] y k =MLPs(G k ) (8)

[0090] Where H is the sequence embedding matrix output by the sequence encoder, k+∈{SS,SA,S}, corresponding to sleep stage SS (SS), sleep apnea (SA), and shared (S), k∈{SS,SA}, y k is the final prediction result (sleep stage or sleep apnea event), MLPs represents a fully connected neural network (i.e., the structure of an expert cluster), E k+ represents the features extracted from the expert cluster k+ (including the shared expert cluster S and the unique expert clusters SS and SA), E k represents the unique features extracted from the unique expert cluster k, E S represents the shared features extracted from the shared expert cluster S, E k,S Indicates that unique and shared features (E k and E S ) is the result of splicing, G kRepresents the final feature (sleep stage feature or sleep apnea feature) obtained by weighted summation. This multi-task learning module extracts the shared features and unique features of the two tasks (sleep stage classification and sleep apnea event recognition) from the input data through three expert clusters (SS, SA and S), and then outputs the fusion features corresponding to the two tasks respectively by weighted summation. Finally, it inputs it into a multi-layer fully connected neural network to obtain the prediction results of the corresponding tasks, namely the sleep stage classification results and the sleep apnea event prediction results, thereby realizing the sleep stage classification of the tested individual. This method provides effective features for sleep stage classification by introducing apnea information, thereby improving the performance of the sleep stage classification task. During training, this module uses a large learning rate to update the parameters of the model.

[0091] In one embodiment, the above step S6 calculates the following sleep-related parameters based on the whole-night sleep stage classification result obtained in step S5:

[0092] a. Total sleep duration: the sum of the duration of N1, N2, N3 and REM stages;

[0093] b. The proportion of time in each sleep stage: the ratio of each stage of N1, N2, N3 and REM to the total sleep time;

[0094] c. Bed rest time: the total time of being judged as "the individual being tested is in bed" during the human presence detection in step S2;

[0095] d. Sleep delay: the time difference between going to bed and falling asleep for the first time;

[0096] e. Sleep efficiency: the ratio between total sleep time and time in bed;

[0097] f. Total wakefulness time after falling asleep: the total time the human body is in the wake stage between the first falling asleep and the last awakening;

[0098] g. Number of awakenings after falling asleep: the total number of times the human body awakens (i.e., switches from other sleep stages to the Wake stage) between the first time falling asleep and the last time awakening.

[0099] In another embodiment of the present invention, a non-contact sleep stage classification device based on a multi-task expert network is provided, which includes a portable signal acquisition device and an information processing device; the portable signal acquisition device includes a radar chip and a microcomputer; the radar chip transmits a signal to the environment through a transmitting antenna, and uses a receiving antenna to receive echo signal data reflected by human bodies and objects in the environment, and then transmits it to the microcomputer, and the microcomputer transmits the received signal data to the information processing device by wireless transmission, and the information processing device uses the above method to process data. In this embodiment, the radar chip is an IR-UWB radar chip, and the microcomputer is a Raspberry Pi.

[0100] Finally, it is noted that the above embodiments are only used to describe the purpose of the present invention, rather than to limit the scope of the present invention. The scope of the present invention is defined by the appended claims. Various equivalent substitutions and modifications made without departing from the spirit and principle of the present invention should be included within the scope of the present invention.

Claims

1. A non-contact sleep stage classification method based on a multi-task expert network, characterized in that: The following steps are involved: The radar equipment is used to collect signal data, and the position range of the bed is obtained according to the collected signal data; Perform human presence detection according to the position range of the bed. If the individual is on the bed, extract the one-dimensional activity signal data of the individual and calculate the corresponding amplitude and phase sequence; The amplitude and phase sequences of the measured individuals are preprocessed to remove noise from the signals; The sleep stage classification model is trained using a public dataset of healthy subjects, and the contrastive learning technique is used to enhance the robustness of the sleep stage classification model to obtain a pre-trained sleep stage classification model. Using the preprocessed signal of the individual under test to fine-tune the feature encoder and the sequence encoder in the pre-trained sleep stage classification model, using the output of the fine-tuned sequence encoder to perform multi-task learning, wherein the multi-task learning takes sleep stage classification as the main task and sleep apnea event recognition as the auxiliary task, and obtains a sleep stage classification model based on multi-task learning by jointly training the main task and the auxiliary task, and uses the trained sleep stage classification model based on multi-task learning to classify the sleep stage of the individual under test; Sleep-related parameters were calculated based on the sleep stage classification results.

2. The method according to claim 1, characterized in that The method of collecting signal data using the IR-UWB radar device and obtaining the position range of the bed according to the collected signal data includes: The IR-UWB radar device is placed on the bedside table to collect signal data, and a two-dimensional complex signal matrix is ​​collected, in which one dimension represents the distance between the device and the reflector, and the other dimension represents the time; A 30-second sliding window is used to divide the two-dimensional complex signal matrix into multiple non-overlapping 30-second sub-matrices along the time dimension, and the signal reflected by static objects in the environment is eliminated from each sub-matrix. The position with the maximum signal intensity in the sub-matrix is ​​calculated, and a 30-second one-dimensional signal sequence at the coordinates of the position is extracted; Determine whether the one-dimensional signal sequence is a human breathing signal, record the human body position corresponding to each human breathing signal, and combine all recorded position information and the width of the bed to obtain the position range of the bed.

3. The method according to claim 2, characterized in that The method for calculating the position with the largest signal strength in the submatrix is: by taking the signal modulus, the signal amplitude corresponding to the 30-second complex matrix is ​​obtained, and superimposed along the time axis to obtain the 30-second total amplitude strength corresponding to each distance coordinate, and the distance coordinate with the largest total amplitude strength is the desired result.

4. The method according to claim 2, characterized in that: The determination of whether the one-dimensional signal sequence is a human breathing signal is that when the signal sequence satisfies the following two conditions at the same time, the signal sequence is considered to be a human breathing signal: The phase sequence corresponding to the complex signal sequence is calculated by taking the phase angle, and the range of the phase sequence is calculated. The maximum displacement of the signal sequence within 30 seconds is calculated in combination with the wavelength of the signal, and this maximum displacement is less than a set threshold α; The corresponding autocorrelation curves are obtained by using the autocorrelation function for the amplitude and phase sequences of the signal respectively. The observation range is set within the range corresponding to the normal breathing frequency of the human body. One of the amplitude and phase sequences satisfies: there is a maximum value within the range and the maximum value is greater than a set threshold β.

5. The method according to claim 1, characterized in that The human body presence detection includes: obtaining the 30-second total amplitude intensity corresponding to each distance coordinate, expressing it as a curve, limiting the observation range to the range corresponding to the bed, and if there is a maximum value on the curve and the maximum value is greater than a set threshold γ, it is determined that the human body is within the bed range.

6. The method according to claim 1, characterized in that The method uses a public dataset of healthy subjects to train a sleep stage classification model, and uses contrastive learning technology to enhance the robustness of the sleep stage classification model, including: The amplitude and phase sequences of the signal data in the public data set are preprocessed and respectively input into the CNN-based feature encoder to extract the local features in the signal. The obtained results are added to obtain the fused local feature matrix. The fused local feature matrix and the position encoding matrix representing the position of each element in the signal are added to inject the position information, and then the result is input into the sequence encoder to obtain the sequence embedding matrix 1; the sequence encoder includes a two-layer multi-head self-attention module; The same signal sequence is input into the CNN-based feature encoder and sequence encoder again, and a new sequence embedding matrix 2 is obtained by using the randomness of the Dropout function. The sequence embedding matrix 1 and the sequence embedding matrix 2 form a positive sample pair. The matrix obtained by inputting other signal data in the same batch as the current input signal data into the CNN-based feature encoder and sequence encoder forms multiple negative sample pairs with the sequence embedding matrix 1. By optimizing the infoNCE loss function, the distance between the positive sample pairs is continuously minimized during the training process, and the distance between the negative sample pairs is maximized. The sequence embedding matrix 1 is input into a two-layer fully connected neural network for five sleep stage classification.

7. The method according to claim 1, characterized in that The method of fine-tuning a feature encoder and a sequence encoder in a pre-trained sleep stage classification model using the pre-processed signal of the individual being tested, and performing multi-task learning using the output of the fine-tuned sequence encoder, comprises: The preprocessed signal of the tested individual is input into the pre-trained CNN-based feature encoder and sequence encoder, and the model is fine-tuned using contrastive learning technology and a small learning rate; The sequence embedding matrix output by the sequence encoder is input into the multi-task learning module, and sleep stage classification is used as the main task, and sleep apnea event recognition is used as the auxiliary task. The performance of the sleep stage classification task is improved by jointly training the two tasks; the multi-task learning module extracts the shared features and unique features of the two tasks from the input data through three expert clusters, and then outputs the prediction results of the two tasks, namely the sleep stage classification result and the sleep apnea event prediction result, through weighted summation, so as to realize the sleep stage classification of the individual under test; the three expert clusters are the sleep stage expert cluster, the sleep apnea expert cluster and the shared expert cluster.

8. The method according to claim 1, characterized in that: The sleep-related parameters include: total sleep duration, proportion of time in each sleep stage, bed rest time, sleep delay, sleep efficiency, total awakening time after falling asleep, and number of awakenings after falling asleep.

9. A non-contact sleep stage classification device based on a multi-task expert network, characterized in that: It comprises a portable signal acquisition device and an information processing device; the portable signal acquisition device comprises a radar chip and a microcomputer; the radar chip transmits signals to the environment via a transmitting antenna, and uses a receiving antenna to receive echo signal data reflected by human bodies and objects in the environment, and then transmits it to the microcomputer; the microcomputer transmits the received signal data to the information processing device via wireless transmission, and the information processing device uses the method described in any one of claims 1 to 8 to perform data processing.

10. The device according to claim 9, characterized in that The radar chip is an IR-UWB radar chip, and the microcomputer is a Raspberry Pi.

Images