Lightweight two-layer nested sleep staging method based on deep learning and wearable device

Abstract

The invention discloses a lightweight two-layer nested sleep staging method based on deep learning and wearable equipment. The lightweight two-layer nested sleep staging method comprises the steps that to-be-tested PSG signals including electroencephalogram signals and electro-oculogram signals are acquired, the to-be-tested PSG signals are preprocessed, and the electroencephalogram signals in the preprocessed to-be-tested PSG signals are filtered; inputting the filtered to-be-tested PSG signal into the trained first-stage neural network model to obtain a first predicted sleep period; the first predicted sleep period comprises at least one of a W stage, an NREM stage and an REM stage; if the NREM stage exists in the first predicted sleep period, inputting the filtered PSG signal to be tested corresponding to the NREM stage into a trained second-stage neural network model to obtain a second predicted sleep period; the second predicted sleep period comprises at least one of an N1 stage, an N2 stage and an N3 stage. The method is more suitable for being deployed on wearable edge hardware equipment and the like.

Description

Technical Field

[0001] The present invention belongs to the technical field of sleep monitoring of electroencephalogram signals, and specifically relates to a lightweight two-layer nested sleep staging method based on deep learning and a wearable device. Background Art

[0002] Sleep is essential to health. It is an essential physiological process and an important part of the body's recovery and memory consolidation. Insufficient and irregular sleep can cause various physiological and psychological problems and diseases, such as decreased immunity, memory loss, depression, anxiety, and emotional instability. A survey released by the World Health Organization shows that 27% of the world's population has sleep problems. Sleep quality assessment is an important early risk indicator that can effectively reduce the incidence of various diseases.

[0003] Sleep staging is an important method for assessing sleep quality. When people sleep, their brain activity shows a series of periodic changes. In the latest sleep staging standard, the American Academy of Sleep Medicine (AASM) standard, adult sleep is divided into five consecutive stages, namely wakefulness (Wake, W), non-rapid eye movement (NREM), and rapid eye movement (REM). The NREM stage can be divided into non-rapid eye movement stage 1 (N1), non-rapid eye movement stage 2 (N2), and non-rapid eye movement stage 3 (N3). When people have sleep disorders or related diseases, the normal rhythm will be disrupted, which may be manifested as changes in the proportion of each stage in the sleep cycle, or difficulty entering a certain stage of sleep. Therefore, sleep staging can effectively support the assessment of sleep quality, thereby providing a standard diagnostic method for sleep problems. Sleep staging is achieved by analyzing the recorded polysomnography (PSG), which contains different types of physiological signals, such as electroencephalogram (EEG), electrooculogram (EOG), electrocardiogram (ECG), electromyogram (EMG), oral and nasal airflow, and oxygen saturation. Sleep experts comprehensively analyze the physiological signals of each channel in PSG and calibrate the various stages of sleep according to the sleep staging standards.

[0004] Manually labeling sleep stages is time-consuming, labor-intensive, and inefficient, and the labeling results are subject to the subjective influence of experts. In order to solve these problems, many early studies used machine learning (ML) methods for sleep staging. First, various manual features need to be extracted from PSG, and then classified through ML algorithms. The effect of this method is very dependent on the selected manual features. With the development of deep learning (DL), many studies have begun to focus on sleep staging through DL algorithms. Compared with ML, DL does not rely on manual feature extraction. The model automatically learns the features of the original signal during the training phase, completes the end-to-end classification task, and has better performance. For example, Y. Dai et al. proposed a Transformer-based MultiChannelSleepNet model that uses EEG and EOG channels for sleep staging. The model uses the Transformer encoder to extract single-channel features and fuse multi-channel features. This method achieved an accuracy of 87.2% and a macro F1 score of 81.2 on the Sleep-EDF-20 dataset. Y.Lin et al. proposed a deep network based on multi-view fusion. The architecture includes a multi-scale local feature extractor (MSLFE) and a generalized relation modeling (GRM) module. The classification accuracy of this method on the Sleep-EDF dataset exceeds 84%. The model with the best performance has 1.485M parameters and 69.76MFLOPs of computation.

[0005] However, existing technologies all use a single deep network to directly perform 5-category sleep staging. In practical applications, there is often a need for rough classification, that is, merging N1, N2, and N3 into NREM stages. Although existing methods can also meet the needs, they consume more hardware resources in rough classification application scenarios, which is a huge challenge for wearable devices or embedded devices with limited resources. These sleep staging algorithms implemented by deep learning cannot run efficiently on low-power, low-storage devices, which limits their application in practical scenarios. Summary of the invention

[0006] In order to solve the above problems existing in the prior art, the present invention provides a lightweight two-layer nested sleep staging method based on deep learning. The technical problem to be solved by the present invention is achieved through the following technical solutions:

[0007] In a first aspect, an embodiment of the present invention provides a lightweight two-layer nested sleep staging method based on deep learning, comprising:

[0008] Acquire a PSG signal to be tested including an electroencephalogram signal and an electrooculogram signal, preprocess the PSG signal to be tested, and filter the electroencephalogram signal in the preprocessed PSG signal to be tested;

[0009] Inputting the filtered PSG signal to be tested into the trained first-stage neural network model to obtain a first predicted sleep period; the first predicted sleep period includes at least one of the W stage, the NREM stage, and the REM stage;

[0010] If the first predicted sleep period includes an NREM stage, the filtered PSG signal to be tested corresponding to the NREM stage is input into the trained second-stage neural network model to obtain a second predicted sleep period; the second predicted sleep period includes at least one of the N1 stage, the N2 stage, and the N3 stage; the second-stage neural network model has the same network structure as the first-stage neural network model.

[0011] In one embodiment of the present invention, preprocessing the PSG signal to be tested includes:

[0012] The PSG signal to be tested is trimmed to retain the awake segment of the PSG signal to be tested within a preset time threshold before entering the non-awake stage, and the awake segment of the PSG signal to be tested within a preset time threshold after finally exiting the sleep state and entering the awake stage.

[0013] In one embodiment of the present invention, the first-stage neural network model and the second-stage neural network model both include three CNN modules, one BiGRU module and one fully connected layer connected in sequence.

[0014] In one embodiment of the present invention, the network structures of the three CNN modules are the same; each CNN module includes a convolutional layer, a normalization layer, an activation layer and a pooling layer connected in sequence.

[0015] In one embodiment of the present invention, the sizes of the convolution kernels of the convolution layers in the three CNN modules increase sequentially.

[0016] In one embodiment of the present invention, the BiGRU module includes a BiGRU layer and a normalization layer connected in sequence.

[0017] In one embodiment of the present invention, a discard layer is further included between the last CNN module and the BiGRU module.

[0018] In one embodiment of the present invention, the process of training the first stage neural network model and the second stage neural network model includes:

[0019] Acquire a training PSG signal and a verification PSG signal including an electroencephalogram signal and an electrooculogram signal, preprocess the training PSG signal and the verification PSG signal respectively, and filter the electroencephalogram signals in the preprocessed training PSG signal and the verification PSG signal respectively; the training PSG signal and the verification PSG signal after filtering respectively include PSG signals of the W stage, the N1 stage, the N2 stage, the N3 stage and the REM stage;

[0020] The PSG signals of the N1 stage, N2 stage, and N3 stage in the filtered training PSG signal and the verification PSG signal are merged respectively as the PSG signals of the corresponding NREM stage;

[0021] The PSG signals of the W stage, NREM stage and REM stage in the filtered training PSG signals are input into the initial first-stage neural network model for training, and the trained first-stage neural network model is verified by using the PSG signals of the W stage, NREM stage and REM stage in the filtered verification PSG signals to obtain a trained first-stage neural network model;

[0022] The PSG signal of the NREM stage in the filtered training PSG signal is input into the initial second-stage neural network model for training, and the trained second-stage neural network model is verified using the PSG signal of the NREM stage in the filtered verification PSG signal to obtain a trained second-stage neural network model.

[0023] In one embodiment of the present invention, preprocessing the training PSG signal and the verification PSG signal respectively includes:

[0024] The training PSG signal is trimmed to retain the awake segment of the training PSG signal within a preset time threshold before entering the non-awake stage, and the awake segment of the training PSG signal within a preset time threshold after finally exiting the sleep state and entering the awake stage;

[0025] The verification PSG signal is trimmed to retain the awake segment of the verification PSG signal within a preset time threshold before entering the non-awake stage and the awake segment of the verification PSG signal within a preset time threshold after finally exiting the sleep state and entering the awake stage.

[0026] In a second aspect, an embodiment of the present invention provides a wearable device, wherein the wearable device integrates a first-stage neural network model and a second-stage neural network model, and executes the lightweight two-layer nested sleep staging method based on deep learning according to any one of the first aspects according to user needs; wherein,

[0027] If the user requirement is to output the three-stage sleep period, displaying the first predicted sleep period on the display interface of the wearable device, and storing the first predicted sleep period on the wearable device;

[0028] If the user requirement is to output five-stage sleep periods, the first predicted sleep period and the second predicted sleep period are displayed on the display interface of the wearable device.

[0029] Beneficial effects of the present invention:

[0030] The lightweight two-layer nested sleep staging method based on deep learning proposed in the present invention has the advantages of being more suitable for deployment on edge hardware devices such as wearables and more adaptable to actual needs. Specifically: an existing single-stage 5-classification task is converted into a two-stage 3-classification task. Compared with the neural network model using 5-classification, the neural network models of the two stages in the method proposed in the present invention are both 3-classification models. In terms of model complexity and parameter quantity, the overhead of the 3-classification model is much lower than that of the 5-classification model, which is very important for application on edge hardware devices such as wearables, because the computing power and storage space of edge devices are usually relatively low. Even if the method proposed in the present invention requires two 3-classification models, their scale is much smaller than that of a 5-classification model, and the network structure of the two 3-classification models is the same. In hardware implementation, the two stages reuse the same network structure, which further reduces the hardware overhead. The method proposed in the present invention can achieve classification performance comparable to that of the existing 5-classification model, but is much smaller in scale, realizing a lightweight sleep staging algorithm, which is more suitable for deployment on edge hardware devices such as wearables. Moreover, when actually applying the sleep staging algorithm, sometimes fine classification is not required, and only a rough classification into the three stages of W, NREM, and REM is required. When the method proposed in the present invention is deployed in hardware, the output of the first-stage neural network model can be directly used as the final result without starting the second-stage neural network model. This further reduces the amount of calculation, thereby reducing power consumption and calculation delay, and is more suitable for sleep monitoring equipment that requires real-time processing and feedback. The method proposed in the present invention can adjust whether to start the second-stage neural network model according to actual needs, and the application is more flexible.

[0031] The present invention will be further described in detail below with reference to the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] Figure 1 It is a flowchart of a lightweight two-layer nested sleep staging method based on deep learning provided by an embodiment of the present invention;

[0033] Figure 2 It is a schematic diagram of a framework implementation of a lightweight two-layer nested sleep staging method based on deep learning provided by an embodiment of the present invention;

[0034] Figure 3 is a schematic diagram of the structure of the first-stage neural network model provided by an embodiment of the present invention;

[0035] Figure 4 It is a structural diagram of a CNN module in a first-stage neural network model provided by an embodiment of the present invention;

[0036] Figure 5 Schematic diagram of the structure of the BiGRU module in the first-stage neural network model provided by an embodiment of the present invention;

[0037] Figure 6 Schematic diagram of the implementation of the BiGRU layer in the BiGRU module provided in an embodiment of the present invention;

[0038] Figure 7 It is a schematic diagram of the implementation of the GRU unit in the BiGRU layer provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0039] The present invention is further described in detail below with reference to specific embodiments, but the embodiments of the present invention are not limited thereto.

[0040] First, see Figure 1 The embodiment of the present invention provides a lightweight two-layer nested sleep staging method based on deep learning, which specifically includes the following steps:

[0041] S10, obtaining a PSG signal to be tested including an EEG signal and an EOG signal, preprocessing the PSG signal to be tested, and filtering the EEG signal in the preprocessed PSG signal to be tested.

[0042] The embodiment of the present invention uses the AASM standard (American Academy of Sleep Medicine, AASM) as the sleep staging standard. In sleep staging, the information of a single or multiple channels (multiple channels refer to physiological signals from different sources) in PSG is usually used to determine which sleep stage the human body is currently in. In view of the current disclosure of a large number of open source sleep data sets, the embodiment of the present invention uses open source data sets to test and verify the algorithm during the design phase. The original open source data set contains too many PSG signals in the awake state. Therefore, after obtaining the PSG signal to be tested from the original open source data set, it is necessary to crop the PSG signal to be tested to retain the awake segment of the preset time threshold before entering the non-awake stage in the PSG signal to be tested, and the awake segment of the preset time threshold after finally exiting the sleep state and entering the awake stage. More specifically:

[0043] The open source dataset was trimmed to retain only the first 30 seconds (the preset time threshold is 30 seconds) of the awake segment entering the non-awake stage (the "entering the non-awake stage" here refers to the moment of first entering the N1 stage from the W stage) in the PSG signals recorded throughout the night, and the 30 seconds of awake segment after the last exit of the sleep state to enter the wake stage (the "exit of the sleep state" here refers to the moment of entering the W stage from the last sleep stage REM) in the PSG signals recorded throughout the night.

[0044] Furthermore, the embodiment of the present invention needs to filter the EEG signal in the PSG signal to be tested. This is because: according to existing research, the EEG waves mainly used for sleep staging include delta waves, theta waves, alpha waves, beta waves, and sleep spindle waves, and the frequency range mainly covers below 35Hz. This frequency band contains most of the useful information, and because there is a lot of ultra-low frequency noise below 0.5Hz, the PSG signal to be tested is subjected to a bandpass filter process of 0.5Hz to 35Hz to remove useless high-frequency components and ultra-low frequency noise to improve classification performance.

[0045] Existing methods classify sleep into five categories (W, N1, N2, N3, REM) based on the AASM standard. However, in actual applications, N1, N2, and N3 are merged into one label NREM in many scenarios, so performing five classifications each time will waste a lot of hardware overhead. In the AASM sleep staging standard, the N1, N2, and N3 stages belong to the NREM stage and have similar characteristics. Therefore, a neural network model can be used to perform the first-stage rough classification of the input original PSG signal, which is divided into three stages: W, NREM, and REM. If the output result is W or REM, the classification task has been completed. If the output result is NREM, the original PSG signal corresponding to the NREM stage needs to be finely classified in the second stage, that is, the input of the second-stage neural network model is the original PSG signal corresponding to the NREM stage identified in the first-stage neural network model, that is, the PSG signal input to the second-stage neural network model has been identified as the NREM stage, so the function of the second-stage neural network model is to divide the input NREM stage corresponding to the PSG signal into three stages: N1, N2, and N3. Finally, through the first-stage coarse classification and the second-stage fine classification, the five-stage classification of W, N1, N2, N3, and REM can be achieved.

[0046] It can be seen that the present invention is aimed at the practical application needs of sleep staging, such as Figure 2As shown in the figure, the 5-classification task is converted into 2 3-classification tasks, and 2 network models with the same structure, namely the first-stage neural network model and the second-stage neural network model, are used to implement the classification tasks of the 2 stages respectively. In terms of hardware implementation, the same network structure is reused in the two stages, which reduces the hardware overhead. In addition, whether to start the second-stage model can be adjusted according to actual needs, which makes the application more flexible and reduces power consumption. The specific implementation includes the following steps S20 and S30:

[0047] S20, inputting the filtered PSG signal to be tested into the trained first-stage neural network model to obtain a first predicted sleep period; the first predicted sleep period includes at least one of the W stage, the NREM stage, and the REM stage.

[0048] Usually, the input used by the network model is a combination of several channel data in the PSG signal. The inventor found in the process of model testing that when the lightweight network model only uses EEG signals, its classification performance is poor, but after adding EOG signals, its classification performance is greatly improved. Therefore, the present invention uses a combination of EEG and EOG signals for classification tasks. In existing sleep staging studies, two forms of input are usually used: one is to directly use the original PSG signal as the network model input, and the other is to use the time-frequency diagram of the PSG signal as the model input. Relevant studies have shown that although the time-frequency diagram provides more information, due to its limited resolution, it cannot significantly improve the classification performance of the model, and because the acquisition of the time-frequency diagram also requires the transformation and calculation of the original PSG signal, the calculation complexity is increased, and the resource pressure of the edge device is increased. Therefore, the embodiment of the present invention uses the original PSG signal as the input of the model.

[0049] Furthermore, the first-stage neural network model of the embodiment of the present invention is a 3-classification model, and the input PSG signal is a PSG signal to be tested after preprocessing and filtering in sequence in S10. The PSG signal to be tested is input into the trained first-stage neural network model to obtain a first predicted sleep period, which includes at least one of the W stage, NREM stage, and REM stage.

[0050] The embodiment of the present invention provides a network structure of a first-stage neural network model as follows: Figure 3As shown in the figure, the network of the first-stage neural network model includes three CNN (Convolutional Neural Network) modules, one BiGRU (Bidirectional Gate Recurrent Unit, BiGRU) module and one fully connected layer. Among them, CNN is mainly used to extract the spatial features of PSG signals, and BiGRU is mainly used to capture the temporal dependency features of PSG signals; the fully connected layer maps the output of the last time step of BiGRU to three classification results. More specifically:

[0051] The network structures of the three CNN modules in the embodiment of the present invention are the same; each CNN module is as follows Figure 4 As shown in the figure, it includes a convolutional layer (CNN), a normalization layer (BN), an activation layer (ReLU) and a pooling layer (POOL) connected in sequence. The CNN module receives multi-channel PSG signals. The convolutional layers in the three CNN modules use convolution kernels of increasing sizes. They are connected in series to gradually extract the abstract features of each channel. For example, after three CNN modules, the number of output channels is 8, 16, and 32 respectively. Specifically:

[0052] The convolutional layer CNN implements the basic convolution operation to extract the spatial features of the input PSG signal. The calculation formula is:

[0053]

[0054] Formula (1) describes the calculation process of mapping an element in an input sample to the output through a one-dimensional convolutional layer, C in represents the total number of input channels of the convolutional layer CNN input feature map, K represents the size of the convolution kernel, in(c in ,n·S+kP) represents the cth feature map in the convolutional layer CNN input in The element value of the (n·S+kP)th position of the input channel, n represents the cth position in the output feature map of the convolutional layer CNN out The nth data position of the output channel, S represents the step size of the convolutional layer CNN, k represents the position of the convolution kernel, P represents the padding size, W(c out ,c in ,k) represents the cth out The cth convolution kernel in the output channel in The weight value of the k-th position of the input channel, b(c out ) indicates the c out The offset value of the output channel, c out The value range is 1 to C out , C outIndicates the total number of output channels of the convolutional layer CNN output feature map, out(c out ,n) represents the cth out The output value of the nth data position of the output channel.

[0055] The normalization layer BN applies standardization to each small batch of data in each layer, so that the data distribution remains relatively stable during the training process, thereby alleviating the problem of gradient disappearance and explosion. The introduction of the BN layer can make network training more effective and improve the training efficiency and generalization ability of the network. The batch normalization formula of the normalization layer is:

[0056]

[0057]

[0058]

[0059] Among them, x i represents the i-th sample in the mini-batch data, m represents the total amount of data in the mini-batch data, Represents x i The standardized result, μ B and are the mean and variance of the small batch data respectively. The calculation formulas are shown in equations (3) and (4). ε is a small positive number used to prevent division by zero errors. γ and β are learnable parameters used to restore the expressiveness of the data.

[0060] The activation layer ReLU is set after the normalization layer BN, which enables the network to express and learn more complex nonlinear relationships. The introduction of the activation layer ReLU can also control the output range of the network layer, making the training of the deep network more stable. Its expression is:

[0061] ReLU(x)=max(x,0) (5);

[0062] Among them, x represents the input feature map of the activation layer ReLU, and max is the maximum value operation. The ReLU function essentially retains the positive part of the input feature map and averages all negative parts to 0. It introduces nonlinearity and solves the problem of gradient disappearance. Its expression is very concise and easy to implement through circuits.

[0063] The pooling layer POOL prefers the maximum pooling layer, which performs downsampling, reduces the spatial size of the feature map, greatly reduces the amount of calculation of the subsequent network layer, improves the training and reasoning speed of the network model, and reduces the amount of data to alleviate the overfitting phenomenon to a certain extent. The calculation process is:

[0064] out 1 (c,n1 )=max[in(c,n 1 ·S 1 +k 1 )],k 1 =0,1,...,K 1 -1 (6);

[0065] Among them, in(c,n 1 ·S 1 +k 1 ) represents the nth channel in the cth channel of the pooling layer POOL input feature map 1 ·S 1 +k 1 The element value at position S 1 Indicates the step size of the pooling layer POOL, n 1 Indicates the nth channel in the cth channel of the pooling layer POOL output feature map 1 data locations, k 1 represents the position of the data in the pooling layer window, K 1 Indicates the pooling layer window size, out 1 (c,n 1 ) represents the nth channel in the cth channel of the pooling layer POOL output feature map 1 The output value of each data location.

[0066] Furthermore, the BiGRU module in the embodiment of the present invention includes a BiGRU layer and a normalization layer connected in sequence. The BiGRU module captures the temporal dependency relationship of the feature graph output by the CNN module from both positive and negative directions. Figure 5 As shown in the figure, it consists of a BiGRU layer and a BN layer. The BiGRU layer consists of two GRU units, which are implemented as follows Figure 6 As shown, the implementation of a single GRU unit is as follows Figure 7 As shown, the calculation formula is as follows:

[0067] z t =σ(W z ·[h t-1 ,x t ]+b z ) (7);

[0068] r t =σ(W r ·[h t-1 ,x t ]+b r ) (8);

[0069]

[0070] Among them, z trepresents the output of the update gate at the tth time step, r t represents the output of the reset gate at the tth time step, σ represents the Sigmoid activation function, tanh represents the Tanh activation function, and h t-1 represents the hidden layer state output at the t-1th time step, h t represents the hidden layer state output at the tth time step, x t represents the input feature map at time t, represents the candidate hidden layer state output at the tth time step, W z , W r , W h Represents the weight matrix of each gate, b z 、b r 、b h Represent the bias of each gate, and ⊙ represents the Hadamard product, which is the multiplication of the corresponding elements of the vector. Compared with GRU, BiGRU can capture the temporal dependency of the input feature map from both positive and negative directions, and can more comprehensively capture the feature information of each channel in PSG, thereby improving the accuracy of sleep staging. For example, the hidden layer size of the BiGRU layer is 32, and the number of output channels is 64.

[0071] Furthermore, the fully connected layer of the embodiment of the present invention maps the output of the last time step of the BiGRU module to three classification results, and the calculation formula of the fully connected layer is:

[0072] out = W·in + b (11);

[0073] Among them, W represents the weight matrix of the fully connected layer, which is obtained from the training stage, in represents the input feature vector of the fully connected layer, b represents the bias vector of the fully connected layer, and out represents the output vector of the fully connected layer.

[0074] Furthermore, in the embodiment of the present invention, a dropout layer is further included between the last CNN module and the BiGRU module. The dropout layer can prevent overfitting and improve the generalization of the model. For example, the parameter of the dropout layer can be set to 0.5.

[0075] S30. If the first predicted sleep period includes the NREM stage, the filtered PSG signal to be tested corresponding to the NREM stage is input into the trained second-stage neural network model to obtain the second predicted sleep period; the second predicted sleep period includes at least one of the N1 stage, the N2 stage, and the N3 stage; the second-stage neural network model has the same network structure as the first-stage neural network model, except that the network parameters are different, and the second-stage neural network model inputs the filtered PSG signal to be tested corresponding to the NREM stage, and outputs a second predicted sleep period including at least one of the N1 stage, the N2 stage, and the N3 stage. For details, see S20, which will not be repeated here.

[0076] Furthermore, the process of training the first-stage neural network model and the second-stage neural network model in the embodiment of the present invention includes:

[0077] A training PSG signal and a verification PSG signal including an electroencephalogram signal and an electrooculogram signal are obtained, and the training PSG signal and the verification PSG signal are preprocessed respectively, and the electroencephalogram signals in the preprocessed training PSG signal and the verification PSG signal are filtered respectively; the filtered training PSG signal and the verification PSG signal include PSG signals of the W stage, N1 stage, N2 stage, N3 stage and REM stage respectively; the PSG signals of the N1 stage, N2 stage and N3 stage in the filtered training PSG signal and the verification PSG signal are merged as the PSG signals corresponding to the NREM stage; The PSG signals of the W stage, NREM stage and REM stage are input into the initial first-stage neural network model for training, and the trained first-stage neural network model is verified by using the PSG signals of the W stage, NREM stage and REM stage in the verification PSG signal after filtering, so as to obtain a trained first-stage neural network model; the PSG signal of the NREM stage in the training PSG signal after filtering is input into the initial second-stage neural network model for training, and the trained second-stage neural network model is verified by using the PSG signal of the NREM stage in the verification PSG signal after filtering, so as to obtain a trained second-stage neural network model.

[0078] The embodiment of the present invention performs preprocessing and filtering on the training PSG signal and the verification PSG signal similar to the PSG signal to be tested. For example, the training PSG signal and the verification PSG signal are preprocessed separately, including: cropping the training PSG signal to retain the awake segment of the training PSG signal before entering the non-awake stage within a preset time threshold, and the awake segment of the preset time threshold after finally exiting the sleep state and entering the awake stage; cropping the verification PSG signal to retain the awake segment of the verification PSG signal before entering the non-awake stage within a preset time threshold, and the awake segment of the preset time threshold after finally exiting the sleep state and entering the awake stage. Perform bandpass filtering processing of 0.5Hz to 35Hz on the EEG signals in the training PSG signal and the verification PSG signal to remove useless high-frequency components and ultra-low-frequency noise.

[0079] The embodiment of the present invention verifies the proposed method based on the Pytorch platform. The process of training the first-stage neural network model and the second-stage neural network model is the same, except that the input data is different. The input data of the first-stage neural network model is the PSG signal including the three stages of W, NREM, and REM, and the input data of the second-stage neural network model is the PSG signal of the NREM stage. During the training process, the loss function uses the cross entropy loss function. In Pytorch, the cross entropy loss function already includes the Softmax activation layer, so there is no need to add the Softmax layer at the end in the network model architecture. The optimizer uses AdamW, which includes L2 regularization compared to Adam and has higher computational efficiency. The initial learning rate in the training stage is set to 0.0008, and the training process is monitored. When the network performance stagnates, the learning rate is actively reduced to further seek improvement. To prevent overfitting during training, the early stopping method was used to monitor the training process. The total number of training cycles was set to 100. After each cycle of training, it was evaluated on the validation set. When the loss on the validation set did not improve within 10 cycles, the training was stopped. The network model with the best performance on the validation set before stopping training was recorded as the trained first-stage neural network model and the second-stage neural network model.

[0080] In order to verify the effectiveness of the lightweight two-layer nested sleep staging method based on deep learning provided by the embodiment of the present invention, the following experiments were conducted for verification.

[0081] The present invention verifies the proposed method on the public datasets SleepEDF-20 and SleepEDF-78. The SleepEDF dataset was obtained from a study on the effects of age on sleep in healthy Caucasians from 1987 to 1991. SleepEDF-20 includes 20 subjects aged 25-34 years old. During two days and nights, two PSGs of about 20 hours each were recorded at the subjects' homes. The data of the second night of subject 13 was lost due to the failure of the cassette or laser disk, and a total of 39 PSG files were recorded. SleepEDF-78 includes 78 subjects aged 25-101 years old. The data of the first night of subjects 36 and 52, and the data of the second night of subject 13 were lost due to the failure of the cassette or laser disk, and a total of 153 PSG files were recorded. Both EEG and EOG signals are sampled at 100Hz.

[0082] According to the AASM standard, the N4 stage in the SleepEDF dataset is merged into the N3 stage, and the motion and unknown segments are deleted. The remaining data will be used for model evaluation. The SleepEDF dataset uses 30 seconds as the minimum time unit to annotate the sleep stages of PSG. Due to the large amount of awake state data in the SleepEDF dataset, only the first 30 seconds of entering the non-awake state and the first 30 seconds of awake segments after waking up are retained in each PSG file. Table 1 summarizes the distribution of each label in the processed datasets SleepEDF-20 and SleepEDF-78.

[0083] The two-stage progressive sleep staging method proposed in the present invention requires special processing of the data set when the two-stage neural network models are trained separately. The data set used for training and verifying the first-stage neural network model needs to merge the N1, N2, and N3 stages into the NREM stage, and the data set used for training and verifying the second-stage neural network model needs to delete the W and REM stages. Finally, all data and labels are retained when the trained two-stage network model is used for full-stage 5 classification verification.

[0084] For the two-stage progressive sleep staging method proposed in this invention, in order to facilitate comparison with the experimental results of other studies, the dataset SleepEDF-20 or SleepEDF-78 is first divided into 20 subsets, and one subset is designated as the validation set each time. The remaining 19 subsets are used as training sets to perform 3-classification training on the first-stage neural network model and the second-stage neural network model respectively, and then the validation set is used to perform an overall 5-classification validation on the two trained models, thereby obtaining an experimental result. The above process is repeated 20 times, and different subsets are designated as validation sets for experiments each time. Finally, the average of the 20 experimental results is taken to obtain the result of 20-fold cross validation.

[0085] During training, the two-stage network models are trained separately, and the three-classification test is performed in each stage. Finally, the two trained network models are used to complete the five-classification test. Table 2 shows the results of the two-stage network model separate test, and Table 3 shows the test results of the five-classification using the trained two-stage model.

[0086] For the evaluation indicators of the overall performance of the model, three indicators are usually used: accuracy, macro F1 score, and Kappa coefficient. These indicators are defined as:

[0087]

[0088] Among them, Acc. represents the accuracy, C represents the total number of label categories, TP j represents the true number of examples of the jth label, and N represents the total number of samples.

[0089]

[0090] Among them, MF1. represents the macro F1 score, F1 j Represents the F1 score of the jth label.

[0091]

[0092] Where k' represents the Kappa coefficient, p o Indicates the consistency between the model prediction results and the actual observation results, p e Indicates the agreement between the classifier's predictions and the actual observations due to random chance.

[0093] In terms of the single-stage 3-classification task, it can be seen from Table 2 that the accuracy of the first-stage neural network model on the two data sets reached more than 90%, the macro F1 score reached more than 87, and the Kappa coefficient reached more than 87; the accuracy of the second-stage neural network model on the two data sets reached more than 84%, the macro F1 score reached more than 80, and the Kappa coefficient reached more than 78.

[0094] When using the two-stage network model for 5-class classification, it can be seen from Table 3 that the accuracy reaches 90.81%, the macro F1 score reaches 87.44, and the Kappa coefficient reaches 87.54 on the SleepEDF-20 dataset; the accuracy reaches 84.35%, the macro F1 score reaches 80.01, and the Kappa coefficient reaches 78.71 on the SleepEDF-78 dataset.

[0095] From the experimental results, it can be seen that the single-stage model parameter amount of the sleep staging algorithm designed in the present invention is only 0.016M, the amount of computation is only 2.03MOPs, and a relatively high classification performance is maintained, which is very suitable for deployment on wearable devices with limited resources such as wearables.

[0096] Table 1. Distribution of SleepEDF dataset

[0097]

[0098] Table 2 Two-stage model test results

[0099]

[0100] Table 3 Full-stage model test results

[0101]

[0102] In summary, the lightweight two-layer nested sleep staging method based on deep learning proposed in the embodiment of the present invention has the advantages of being more suitable for deployment on edge hardware devices such as wearables and more adaptable to actual needs. Specifically: an existing single-stage 5-classification task is converted into a two-stage 3-classification task. Compared with the neural network model using 5-classification, the neural network models of the two stages in the method proposed in the present invention are both 3-classification models. In terms of model complexity and parameter quantity, the overhead of the 3-classification model is much lower than that of the 5-classification model, which is very important for applications on edge hardware devices such as wearables, because the computing power and storage space of edge devices are usually relatively low. Even if the method proposed in the present invention requires two 3-classification models, their scale is much smaller than that of a 5-classification model, and the network structure of the two 3-classification models is the same. In hardware implementation, the two stages reuse the same network structure, which further reduces the hardware overhead. The method proposed in the present invention can achieve classification performance comparable to that of the existing 5-classification model, but is much smaller in scale, thereby realizing a lightweight sleep staging algorithm that is more suitable for deployment on edge hardware devices such as wearables. Moreover, when actually applying the sleep staging algorithm, sometimes fine classification is not required, and only a rough classification into the three stages of W, NREM, and REM is required. When the method proposed in the present invention is deployed in hardware, the output of the first-stage neural network model can be directly used as the final result without starting the second-stage neural network model. This further reduces the amount of calculation, thereby reducing power consumption and calculation delay, and is more suitable for sleep monitoring equipment that requires real-time processing and feedback. The method proposed in the present invention can adjust whether to start the second-stage neural network model according to actual needs, and the application is more flexible.

[0103] In a second aspect, an embodiment of the present invention provides a wearable device, which integrates a first-stage neural network model and a second-stage neural network model, and executes any one of the lightweight two-layer nested sleep staging methods based on deep learning according to user needs in the first aspect; wherein,

[0104] If the user requirement is to output the three-stage sleep period, the first predicted sleep period is displayed on the display interface of the wearable device, and the first predicted sleep period is stored in the wearable device;

[0105] If the user's requirement is to output five-stage sleep periods, the first predicted sleep period and the second predicted sleep period are displayed on the display interface of the wearable device.

[0106] As for the wearable device embodiment of the second aspect, since it is basically similar to the method embodiment of the first aspect, the description is relatively simple, and the relevant parts can refer to the partial description of the method embodiment of the first aspect.

[0107] In the description of the present invention, it should be understood that the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of the features. In the description of the present invention, the meaning of "plurality" is two or more, unless otherwise clearly and specifically defined.

[0108] Although the present invention is described herein in conjunction with various embodiments, in the process of implementing the claimed invention, those skilled in the art may understand and implement other variations of the disclosed embodiments by viewing the specification and its drawings. In the specification, the word "comprising" does not exclude other components or steps, and "a" or "one" does not exclude multiple situations. Certain measures are recorded in different embodiments, but this does not mean that these measures cannot be combined to produce good results.

[0109] The above contents are further detailed descriptions of the present invention in combination with specific preferred embodiments, and it cannot be determined that the specific implementation of the present invention is limited to these descriptions. For ordinary technicians in the technical field to which the present invention belongs, several simple deductions or substitutions can be made without departing from the concept of the present invention, which should be regarded as falling within the scope of protection of the present invention.

Claims

1. A lightweight two-layer nested sleep staging method based on deep learning, characterized in that: The method comprises: Acquire a PSG signal to be tested including an electroencephalogram signal and an electrooculogram signal, preprocess the PSG signal to be tested, and filter the electroencephalogram signal in the preprocessed PSG signal to be tested; Inputting the filtered PSG signal to be tested into the trained first-stage neural network model to obtain a first predicted sleep period; the first predicted sleep period includes at least one of the W stage, the NREM stage, and the REM stage; If the first predicted sleep period includes an NREM stage, the filtered PSG signal to be tested corresponding to the NREM stage is input into the trained second-stage neural network model to obtain a second predicted sleep period; the second predicted sleep period includes at least one of the N1 stage, the N2 stage, and the N3 stage; the second-stage neural network model has the same network structure as the first-stage neural network model.

2. The light-weight two-layer nested sleep staging method based on deep learning according to claim 1, characterized in that: Preprocessing the PSG signal to be tested includes: The PSG signal to be tested is trimmed to retain the awake segment of the PSG signal to be tested within a preset time threshold before entering the non-awake stage, and the awake segment of the PSG signal to be tested within a preset time threshold after finally exiting the sleep state and entering the awake stage.

3. The light-weight two-layer nested sleep staging method based on deep learning according to claim 1, characterized in that: The networks of the first-stage neural network model and the second-stage neural network model both include three CNN modules, one BiGRU module and one fully connected layer connected in sequence.

4. The light-weight two-layer nested sleep staging method based on deep learning according to claim 3, characterized in that: The network structures of the three CNN modules are the same; each CNN module includes a convolutional layer, a normalization layer, an activation layer, and a pooling layer connected in sequence.

5. The light-weight two-layer nested sleep staging method based on deep learning according to claim 4, characterized in that: The sizes of the convolution kernels of the convolutional layers in the three CNN modules increase successively.

6. The light-weight two-layer nested sleep staging method based on deep learning according to claim 3, characterized in that: The BiGRU module includes a BiGRU layer and a normalization layer connected in sequence.

7. The light-weight two-layer nested sleep staging method based on deep learning according to claim 3, characterized in that: There is also a discard layer between the last CNN module and the BiGRU module.

8. The light-weight two-layer nested sleep staging method based on deep learning according to claim 1, characterized in that: The process of training the first-stage neural network model and the second-stage neural network model includes: Acquire a training PSG signal and a verification PSG signal including an electroencephalogram signal and an electrooculogram signal, preprocess the training PSG signal and the verification PSG signal respectively, and filter the electroencephalogram signals in the preprocessed training PSG signal and the verification PSG signal respectively; the training PSG signal and the verification PSG signal after filtering respectively include PSG signals of the W stage, the N1 stage, the N2 stage, the N3 stage and the REM stage; The PSG signals of the N1 stage, N2 stage, and N3 stage in the filtered training PSG signal and the verification PSG signal are merged respectively as the PSG signals of the corresponding NREM stage; The PSG signals of the W stage, NREM stage and REM stage in the filtered training PSG signals are input into the initial first-stage neural network model for training, and the trained first-stage neural network model is verified by using the PSG signals of the W stage, NREM stage and REM stage in the filtered verification PSG signals to obtain a trained first-stage neural network model; The PSG signal of the NREM stage in the filtered training PSG signal is input into the initial second-stage neural network model for training, and the trained second-stage neural network model is verified using the PSG signal of the NREM stage in the filtered verification PSG signal to obtain a trained second-stage neural network model.

9. The light-weight two-layer nested sleep staging method based on deep learning according to claim 8, characterized in that: Preprocessing the training PSG signal and the verification PSG signal respectively includes: The training PSG signal is trimmed to retain the awake segment of the training PSG signal within a preset time threshold before entering the non-awake stage, and the awake segment of the training PSG signal within a preset time threshold after finally exiting the sleep state and entering the awake stage; The verification PSG signal is trimmed to retain the awake segment of the verification PSG signal within a preset time threshold before entering the non-awake stage and the awake segment of the verification PSG signal within a preset time threshold after finally exiting the sleep state and entering the awake stage.

10. A wearable device, characterized in that: The wearable device integrates a first-stage neural network model and a second-stage neural network model, and executes the lightweight two-layer nested sleep staging method based on deep learning according to any one of claims 1 to 9 according to user needs; wherein, If the user requirement is to output the three-stage sleep period, displaying the first predicted sleep period on the display interface of the wearable device, and storing the first predicted sleep period on the wearable device; If the user's requirement is to output five-stage sleep periods, the first predicted sleep period and the second predicted sleep period are displayed on the display interface of the wearable device.

Images