An automatic sleep staging method for single-channel electroencephalogram based on feature reconstruction

Abstract

The present invention provides a single-channel electroencephalogram automatic sleep staging method based on feature reconstruction, including: collecting and annotating overnight single-channel electroencephalograms of sleep to obtain a dataset for training a model; preprocessing the dataset to perform data normalization and data augmentation; establishing and initializing the model, and using the preprocessed dataset to perform unsupervised pre-training and supervised training on the model; using the trained model to achieve the annotation of sleep stages. The present invention can accurately predict sleep staging only through single-channel electroencephalogram signals, reducing the discomfort of patients during medical treatment and the difficulty of signal acquisition.

Description

Technical Field

[0001] The present invention relates to the field of electroencephalogram classification using deep learning. Specifically, it relates to a method for automatic sleep staging of single-channel electroencephalograms based on feature reconstruction. Background Art

[0002] During the entire night's sleep process, the activities of the human brain, muscles, heart, and other parts are in a dynamic change process, from which the sleep state can be classified. According to the standards of the American Academy of Sleep Medicine, based on the polysomnogram composed of signals such as electroencephalogram, electrocardiogram, electrooculogram, and electromyogram, the sleep state every thirty seconds can be divided into five stages: wakefulness, rapid eye movement sleep, non-rapid eye movement sleep stage 1, non-rapid eye movement sleep stage 2, and non-rapid eye movement sleep stage 3.

[0003] Each sleep stage plays a necessary role in human health. Therefore, sleep staging is an important means for sleep quality assessment and detection of sleep-related diseases.

[0004] Sleep staging is mainly completed by sleep experts with many years of experience, but this approach has the following limitations:

[0005] (1) The entire night's sleep staging is extremely time-consuming. Sleep experts need to classify the signals of multiple channels every thirty seconds into a sleep stage. However, the sleep signals throughout the night are often as long as eight hours. Therefore, the entire process requires a large amount of time from sleep experts, and patients also need to wait a long time to obtain the results;

[0006] (2) Sleep staging is very difficult. Only sleep experts with many years of experience can perform sleep staging. And due to the influence of working hours and subjective experience, the sleep staging results given by two sleep experts often also show differences. Therefore, many researchers have begun to focus on how to achieve automatic sleep staging.

[0007] Some early researchers built automatic sleep staging systems based on pattern recognition. However, the patterns corresponding to each sleep stage are relatively complex and difficult to identify, which limits the performance of the system. Subsequently, researchers established automatic sleep staging models through machine learning. This type of method first requires researchers to manually extract various features from the polysomnogram every thirty seconds, and then send the features into machine learning models such as neural networks, support vector machines, and random forests for classification. This method has the following defects: (1) The classification performance is extremely dependent on how to select the manually extracted features, and there is no clear standard for feature selection, which only depends on the experience of the researcher; (2) The changes in sleep stages have certain transition rules, and the features extracted from a single thirty-second signal segment cannot accurately reflect the transition relationship between sleep stages.

[0008] With the improvement of computer computing power in recent years, deep learning has received increasing attention from researchers and achieved extremely excellent results in tasks such as computer vision and natural language processing. Therefore, many sleep-related researchers have begun to attempt to use deep learning to achieve automatic sleep staging. Different from machine learning that requires manual feature extraction, deep learning can directly use convolutional neural networks to automatically extract features, thus realizing an end-to-end automatic sleep staging system.

[0009] Sleep staging systems based on sleep experts, pattern recognition, and machine learning often need to measure overnight polysomnograms. However, overnight polysomnograms consist of multi-channel signals, require professional large-scale instruments, are extremely troublesome to measure, and patients have a poor physical experience. Since different sleep stages have sleep brain waves of different frequency bands, most deep learning-based automatic sleep staging models can complete sleep staging only relying on single-channel electroencephalograms. Single-channel electroencephalograms can be detected by small devices, are relatively simple to measure, and patients are more comfortable during the measurement process.

[0010] Different from computer vision tasks, there are the following technical problems in automatic sleep staging based on electroencephalograms: (1) The information density of electroencephalogram data is low. To avoid signal distortion, electroencephalograms are usually collected at a relatively high sampling rate (usually greater than 100Hz), so they have a high data dimension, resulting in a low information density. The information content per unit data dimension is much smaller than that of images. (2) There is a transition relationship between sleep stages. The sleep staging standard requires that sleep annotations should simultaneously pay attention to the corresponding electroencephalogram segments and their adjacent electroencephalogram segments, and the changes in sleep stages during the whole night have certain rules. (3) Electroencephalograms contain a large amount of noise. Electroencephalograms themselves are weak potential signals, and noise caused by body movement and measurement errors will inevitably be introduced during the measurement process. Automatic sleep staging models based on electroencephalograms must consider the above problems. Existing models can be classified into pure convolutional neural network models and hybrid models of convolutional and recurrent neural networks according to their structures.

[0011] A pure convolutional neural network model is a neural network that only contains convolutional layers, such as the U-Time model. Due to features such as translational invariance and locality of convolutional layers, they are often used to process images. Electroencephalogram signals, as one-dimensional data sequences, can be regarded as a special type of image. In the automatic sleep staging task, the model receives a sequence of continuous electroencephalogram segments of a certain length each time and outputs the corresponding sequence of sleep segments. As the number of convolutional layers increases, the receptive field of the model gradually expands. Therefore, it can extract features from adjacent continuous electroencephalogram segments, thus implicitly encoding the sleep transition rules. However, as mentioned above, the data dimension of 30-second electroencephalogram segments is relatively high. To extract the features of continuous sleep segments, a high requirement is placed on the receptive field of the model, resulting in a more complex model, slower operation, and a higher number of parameters.

[0012] The hybrid model of convolutional and recurrent neural networks introduces a recurrent neural network after the convolutional neural network, such as TinySleepNet. The introduction of the recurrent neural network enables the model to extract temporal relationships. Therefore, the convolutional neural network part in the model extracts features from a single EEG segment, and the subsequent recurrent neural network part learns the transition relationships between adjacent sleep segments. Although such models reduce the requirements for the receptive field, the relatively shallow model results in a high temporal dimension of the output of the convolutional neural network part, and the introduction of the recurrent neural network increases the difficulty of model training. Summary of the Invention

[0013] Aiming at the defects in the prior art, the purpose of the present invention is to provide a single-channel electroencephalogram automatic sleep staging method based on feature reconstruction.

[0014] According to one aspect of the present invention, there is provided a single-channel electroencephalogram automatic sleep staging method based on feature reconstruction, including:

[0015] Collect and label the single-channel electroencephalogram of the whole night's sleep to obtain a dataset for training the model;

[0016] Preprocess the dataset, perform data standardization and data augmentation;

[0017] Establish and initialize the model, and perform unsupervised pre-training and supervised training on the model using the preprocessed dataset;

[0018] Use the trained model to achieve the annotation of sleep stages.

[0019] Preferably, the collection and annotation of the electroencephalogram of the whole night's sleep to obtain a dataset for training the model includes: measuring the single-channel electroencephalogram of Fpz-Cz through a collection device, and removing the segments containing noise caused by the body movement of the subject and the segments of non-sleep stages.

[0020] Preferably, the data preprocessing includes:

[0021] Perform standardization on each whole-night sleep EEG record with a mean of 0 and a variance of 1 or a median of 0 and an interquartile range of 1;

[0022] Randomly select several 30-second sleep segments within each training round for translation and inversion to achieve data augmentation.

[0023] Preferably, the established model includes:

[0024] Convolutional reconstruction module, the convolutional reconstruction module uses a second convolutional neural network, and the second convolutional neural network includes a forward part and a backward part; the forward part extracts the features of the input image, and the backward part reconstructs and outputs the output features of the first layer of the forward part;

[0025] Global max and average pooling module, the global max and average pooling module reduces the dimensionality of the features output by the convolutional reconstruction module and extracts time-invariant features.

[0026] Preferably, the established model further includes:

[0027] Low-level feature extraction module, the low-level feature extraction module uses a first convolutional neural network to extract features and reduce the dimensionality of the input electroencephalogram to obtain low-level features as the input of the convolutional reconstruction module;

[0028] Recurrent neural network module, the recurrent neural network module uses a first recurrent neural network and receives the output of the recurrent neural network module for the previous electroencephalogram segment and the output of the global max and average pooling module as inputs, learns the transition relationship between sleep segments, and outputs the sleep staging result.

[0029] Preferably, the first convolutional neural network includes: 1 convolutional layer, 1 max pooling layer and 1 dropout layer;

[0030] Preferably, the second convolutional neural network includes: 3 forward convolutional layers, 2 backward convolutional layers and 1 max pooling layer;

[0031] Preferably, the global max and average pooling module includes: a global max pooling layer, a global average pooling layer, a connection layer and a dropout layer; the global max pooling layer and the global average pooling layer respectively perform global pooling on the output of the convolutional reconstruction module, and the connection layer connects the two obtained features together in the channel dimension.

[0032] Preferably, the recurrent neural network module includes: 1 long short-term memory network and 1 linear layer;

[0033] Preferably, the unsupervised pre-training is to pre-train the low-level feature extraction module and the convolutional reconstruction module, including:

[0034] The loss function for measuring the reconstruction quality of the convolutional reconstruction module is the mean square error loss, which is defined as:

[0035]

[0036] Where, represents the output of the first forward convolutional layer, Represents the output of the transposed convolutional layer;

[0037] The supervised training is to train the overall model using classification loss and reconstruction loss, including:

[0038] The loss function used incorporates a weighted cross-entropy classification loss L cls , and the total loss function is set to the classification loss and the weighted reconstruction loss, that is:

[0039] L total = L cls + α * L rec ;

[0040] Where α is the weight of the reconstruction loss, which is set to 1e-5.

[0041] Compared with the prior art, the present invention has the following beneficial effects:

[0042] The present invention can accurately predict sleep stages only through single-channel electroencephalogram signals, reducing the discomfort of patients during medical treatment and the difficulty of signal acquisition.

[0043] The present invention proposes a neural network model based on feature reconstruction with fewer parameters, which can better extract the features required for sleep staging from electroencephalogram signals and requires less training time to achieve sleep staging results similar to or even better than existing methods. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] By reading the following detailed description of the non-limiting embodiments with reference to the accompanying drawings, other features, objects, and advantages of the present invention will become more apparent:

[0045] Figure 1 FIG. is a flowchart of an automatic single-channel electroencephalogram sleep staging method based on feature reconstruction provided by the present invention.

[0046] Figure 2 FIG. is an overall architecture diagram of an automatic single-channel electroencephalogram sleep staging model based on feature reconstruction in a preferred embodiment provided by the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0047] The present invention will be described in detail below with reference to specific embodiments. The following embodiments will help those skilled in the art to further understand the present invention, but do not limit the present invention in any form. It should be noted that those of ordinary skill in the art can make several modifications and improvements without departing from the concept of the present invention. These all belong to the protection scope of the present invention.

[0048] Such as Figure 1As shown in the figure, it is a flowchart of a single-channel electroencephalogram automatic sleep staging method based on feature reconstruction provided by the present invention, including:

[0049] S1. Collect and label the single-channel electroencephalogram of the whole-night sleep to obtain a dataset for training the model;

[0050] S2. Preprocess the dataset, including data normalization and data augmentation;

[0051] S3. Establish and initialize the model, and perform unsupervised pre-training and supervised training on the model using the preprocessed dataset;

[0052] S4. Use the trained model to label the sleep stages.

[0053] The present invention provides a preferred embodiment to execute S1 to collect the single-channel electroencephalogram. The wearable device or electroencephalograph is used to collect the electroencephalogram signals of the Fpz-Cz channel of the subject throughout the night, and the sleep stages are labeled, thereby generating a dataset for model training. In addition, public datasets with sleep stage labels provided by websites such as Physionet can also be used. Some sleep segments introduce a large amount of noise due to the large movement of the subject's body, and such segments should be directly excluded from the dataset; some sleep segments that cannot confirm the sleep cycle should also be excluded.

[0054] The present invention provides a preferred embodiment to execute S2 to preprocess the dataset and normalize each whole-night sleep electroencephalogram signal to assist model training. As a preferred embodiment, the electroencephalogram signal of each night is translated and scaled to a mean of 0 and a variance of 1; or to a median of 0 and an interquartile range of 1, and this normalization method has better robustness.

[0055] In deep learning, the learning ability of the model is often positively correlated with the size of the data scale. Therefore, in this embodiment, some 30-second electroencephalogram segments are augmented by translation or inversion with an amplitude of 5% to 10% (i.e., 150 to 300 sample points at a sampling rate of 100 Hz), so as to generate more diverse electroencephalogram data and further expand the data volume.

[0056] In order to solve the defects existing in the prior art, enhance the training speed and improve the staging accuracy, the present invention provides a preferred embodiment to execute S3 to establish an overall model. In this embodiment, the overall model includes four modules, namely:

[0057] Low-level feature extraction module, convolutional reconstruction module, global maximum and average pooling module, and recurrent neural network module.

[0058] The low-level feature extraction module uses a first convolutional neural network to perform a primary feature extraction and dimensionality reduction on the input electroencephalogram, obtaining the features of an original signal.

[0059] The convolutional reconstruction module uses a first convolutional neural network and consists of a forward part and a backward part. The forward part further processes the features extracted by the low-level feature extraction module to obtain more advanced and abstract features; the backward part reconstructs the output features of the first layer of the forward part as much as possible.

[0060] The global max and average pooling module further processes the features output by the convolutional reconstruction module, extracting time-invariant features while reducing the data dimension.

[0061] The recurrent neural network module uses a first recurrent neural network and receives the output of the recurrent neural network module for the previous electroencephalogram segment and the output of the global max and average pooling module as inputs simultaneously, learning the transition relationship between sleep segments, and finally outputting the sleep staging result.

[0062] Further, as Figure 2 shown, it is the overall architecture diagram of the automatic sleep analysis model based on a deep neural network in this embodiment. As can be seen from the figure, among the established model parameters, there are:

[0063] The first convolutional neural network used in the low-level feature extraction module includes: 1 convolutional layer, 1 max pooling layer, and 1 dropout layer. Among them: the convolutional layer includes a convolutional operation layer and a ReLU activation function, the convolutional kernel size is 50, the convolutional stride is 6, and the number of output channels is 128; the max pooling layer size is 8, and the stride is 8; the dropout layer probability is 0.5.

[0064] The second convolutional neural network used in the convolutional reconstruction module includes: 3 forward convolutional layers, 2 backward convolutional layers, and 1 max pooling layer. Among them, the convolutional layer includes a convolutional operation layer and a ReLU activation function, the convolutional kernel size is 8, the convolutional stride is 1, and the number of output channels is 128; the max pooling layer size is 4, and the stride is 4.

[0065] The global max and average pooling module includes: a global max pooling layer, a global average pooling layer, a connection layer, and a dropout layer. Among them, the dropout layer probability is 0.5; the global max pooling layer and the global average pooling layer respectively perform global pooling on the output of the convolutional reconstruction module, and the connection layer connects the two obtained features together in the channel dimension.

[0066] The first recurrent neural network adopted by the recurrent neural network module includes: 1 layer of long short-term memory network and 1 layer of linear layer. Among them, the number of hidden layer states of the long short-term memory network is 128, the number of neurons in the linear layer is 5, and the activation function is the Softmax function. The long short-term memory network only accepts 1-dimensional data as input, and the output feature of the convolutional reconstruction module is where C represents the number of channels of the feature, and T represents the time dimension of the feature. In the case of no global maximum and average pooling module, to input this feature into the long short-term memory, the feature needs to be flattened into However, since the gates in the long short-term memory network are similar to the linear layer, this direct flattening of the feature increases the dimension of the feature, resulting in a huge increase in the number of model parameters. In addition, according to the criteria of sleep staging, the main discriminant method of sleep staging is the presence and proportion of brain waves corresponding to each sleep stage, rather than paying attention to its specific position in the corresponding EEG signal, and the direct feature flattening destroys this property. After adding the global maximum and average pooling module, the output feature is significantly reducing the feature dimension, and thus greatly reducing the number of model parameters.

[0067] The present invention provides a preferred implementation for unsupervised pre-training. Specifically, the low-level feature extraction module and the convolutional reconstruction module are pre-trained using the unsupervised reconstruction loss. An additional reverse feature reconstruction convolutional layer is proposed outside the forward convolutional neural network to help the feature extraction of the forward structure. This part aims to reconstruct the intermediate features in the forward process as well as possible. A large amount of noise is inevitably introduced during the acquisition process of electroencephalogram. Therefore, the features near the input part of the model contain more noise, so it is very difficult to reconstruct the input EEG signal or the output of the low-level feature extraction module, and it may even conflict with the forward feature extraction, resulting in a decline in the performance of the model. Further, the reverse convolutional layer takes the output of the forward convolutional layer as input, and its reconstruction target is the output of the first forward convolutional layer. The loss function for measuring the reconstruction quality is the mean square error loss, which is defined as:

[0068]

[0069] where, represents the output of the first forward convolutional layer, represents the output of the reverse convolutional layer.

[0070] Furthermore, before formally training the model using the sleep stage labels, the low-level feature extraction module and the convolutional reconstruction module are pre-trained for 10 rounds without using the sleep stage labels.

[0071] The present invention provides a preferred embodiment for performing supervised training, that is, the overall model is supervised to be trained using a classification loss and a reconstruction loss. Different from the loss function used in pre-training, the loss function used in the formal training adds a weighted cross-entropy classification loss L cls , considering the class imbalance problem in the dataset, such as the significantly less data of non-rapid eye movement sleep stage 1, the weights of wakefulness, rapid eye movement sleep, non-rapid eye movement sleep stage 1, non-rapid eye movement sleep stage 2, and non-rapid eye movement sleep stage 3 are 1, 1, 1.5, 1, and 1 respectively. The total loss function is set as the classification loss and the weighted reconstruction loss, that is:

[0072] L total = L cls + α * L rec

[0073] where α is the weight of the reconstruction loss, which is set to 1e-5.

[0074] Different from the fully convolutional neural network, the long short-term memory network needs to save the internal neuron states corresponding to each sleep stage as the input for the next sleep stage. Therefore, the EEG signals within each batch come from different records, and the EEG signals within each batch are length-aligned by zero-padding, so that the model can achieve batch processing of EEG records, accelerating the training process. At the same time, the loss of the padded part is not used for model optimization, and the loss calculation of the padded part is meaningless for model optimization. At the beginning of each batch, the internal neurons of the long short-term memory network are initialized to zero until all the EEG signals within the batch are input into the model. For example, in one epoch, for a dataset D with n overnight EEG signals n , with a batch size of B, the data for each batch is D B , where the longest EEG signal has L thirty-second EEG segments, and the remaining signals will be zero-padded to length L at the end, that is and a padding matrix is introduced where the values of each item correspond to the original signal or the padded signal here, the value at the padding position is 0, and the value at the original signal position is 1. Considering the limitations of video memory and memory, only a continuous segment of EEG signal can be input each time. If the length is l, the input for each batch at a time is padding matrix In the convolutional neural network part, it can be regarded as for processing, and after passing through the global max and average pooling modules, the feature is From this, the sleep stage classification results can be calculated one by one along the l direction, and the item-by-item loss can be obtained

[0075] Then the final classification loss is:

[0076]

[0077] Where ⊙ represents element-wise multiplication. This part is to enable the model to perform batch processing on EEG records. The significance of batch processing is to accelerate the training process.

[0078] The states of the long short-term memory network neurons in the last sleep stage are saved, which are used to calculate the first segment of the next sequence adjacent to the EEG segment sequence of length l this time. After the calculation of all batches is completed, the long short-term memory network neurons are initialized to zero. After the training of all batches is completed, this step is repeated for the next round of training. The setting of the total number of rounds should be negatively correlated with the amount of training data, that is, the more data, the fewer training rounds are required.

[0079] To better perform sleep stage annotation, the present invention provides a preferred embodiment to execute S4. In this embodiment, during the actual use after the model is trained, in order to quickly annotate multiple overnight EEG records for sleep stages, the same as the training process, the shorter EEG signals also need to be padded during the testing process. The padded EEG data is sent into the trained model, and after the data is sent into the model, the Softmax layer outputs a vector Where and are 1, representing the probabilities that the model assigns to a sleep segment belonging to wakefulness, rapid eye movement sleep, non-rapid eye movement sleep stage 1, non-rapid eye movement sleep stage 2, and non-rapid eye movement sleep stage 3 respectively. The prediction with the highest probability is selected as the staging result of the model for this segment.

[0080] This embodiment only needs to collect single-channel electroencephalogram signals for the training of the used model. Compared with the existing methods, the used model has fewer parameters and only requires less training time to achieve comparable or even better sleep staging performance for single-channel EEG signals.

[0081] The specific embodiments of the present invention have been described above. It should be understood that the present invention is not limited to the above specific embodiments, and those skilled in the art can make various deformations or modifications within the scope of the claims, which do not affect the essence of the present invention. The above preferred features can be combined arbitrarily without conflict.

Claims

1. An automatic sleep staging method for single-channel electroencephalogram based on feature reconstruction, characterized in that Including: Collecting and annotating single-channel electroencephalogram (EEG) of all-night sleep to obtain a dataset for training the model; Preprocessing the dataset, including data standardization and data augmentation; Building and initializing the model, and using the preprocessed dataset to perform unsupervised pre-training and supervised training on the model; Using the trained model to achieve the annotation of sleep stages; The collecting and annotating single-channel EEG of all-night sleep to obtain a dataset for training the model includes: measuring the single-channel EEG of Fpz-Cz through a collecting device, and removing the segments containing noise caused by the body movement of the subject and the segments of non-sleep stages; The preprocessing the dataset includes: Standardizing each piece of the single-channel EEG data of all-night sleep with a mean of 0 and a variance of 1 or a median of 0 and an interquartile range of 1; Randomly selecting several 30-second sleep segments within each training round; Translating and reversing each of the 30-second sleep segments one by one to achieve data augmentation; The building and initializing the model includes: A convolutional reconstruction module, the convolutional reconstruction module adopts a second convolutional neural network, and the second convolutional neural network includes a forward part and a backward part; the forward part extracts the features of the single-channel EEG of all-night sleep, and the backward part reconstructs and outputs the output features of the forward part; A global max and average pooling module, the global max and average pooling module reduces the dimension of the features output in the convolutional reconstruction module and extracts time-invariant features; The building and initializing the model further includes: A low-level feature extraction module, the low-level feature extraction module adopts a first convolutional neural network to extract and reduce the dimension of the input single-channel EEG of all-night sleep to obtain low-level features, which are used as the input of the convolutional reconstruction module; A recurrent neural network module, the recurrent neural network module adopts a first recurrent neural network, and simultaneously receives the output of the recurrent neural network module for the convolutional reconstruction module and the output of the global max and average pooling module as inputs, learns the transition relationship between sleep segments, and outputs the sleep staging result.

2. The automatic sleep staging method for single-channel electroencephalogram based on feature reconstruction according to claim 1, characterized in that, The first convolutional neural network includes: 1 convolutional layer, 1 max pooling layer and 1 dropout layer.

3. The automatic sleep staging method for single-channel electroencephalogram based on feature reconstruction according to claim 1, characterized in that, The second convolutional neural network includes: 3 forward convolutional layers, 2 backward convolutional layers and 1 max pooling layer.

4. The automatic sleep staging method for single-channel electroencephalogram based on feature reconstruction according to claim 1, wherein The global max and average pooling module includes: a global max pooling layer, a global average pooling layer, a connection layer and a dropout layer; the global max pooling layer and the global average pooling layer respectively perform global pooling on the output of the convolutional reconstruction module; the connection layer connects the two features obtained through the global max pooling layer and the global average pooling layer together in the dimension of a single channel.

5. A single-channel electroencephalogram automatic sleep staging method based on feature reconstruction according to claim 1, characterized in that, The recurrent neural network module includes: 1 long short-term memory network and 1 linear layer.

6. The automatic sleep staging method for single-channel electroencephalogram based on feature reconstruction according to claim 1, wherein The unsupervised pre-training is to pre-train the low-level feature extraction module and the convolutional reconstruction module, including: The loss function for measuring the reconstruction quality of the convolutional reconstruction module is the mean square error loss, which is defined as: Among them, represents the output of the first forward convolutional layer, represents the output of the transposed convolutional layer; The supervised training is to train the overall model using classification loss and reconstruction loss, including: The loss function used incorporates the weighted cross-entropy classification loss L cls , and the total loss function is set to the cross-entropy classification loss and the weighted mean squared error loss, i.e.: L total = L cls + α * L rec ; Among them, α is the weight of the mean squared error loss, which is set to 1e-5.

Images