Automatic sleep staging system based on deep neural network and brain-computer interface

Abstract

The present invention discloses an automatic sleep staging system based on a deep neural network and a brain-computer interface, comprising: a data acquisition module for collecting and recording the user's EEG signals; a data preprocessing module for processing EEG signals; a staging model training module, which uses a deep neural network to construct an automatic sleep staging model, and uses the constructed automatic sleep staging model to perform feature representation, feature enhancement, and feature capture on single-channel EEG signals, and obtains the optimal automatic sleep staging model through multiple iterations; an automatic staging module, which uses the optimal automatic sleep staging model to perform fully automatic sleep stage discrimination on preprocessed sleep EEG samples; and a feedback module, which is used to feed back the results processed by the automatic staging module, output them in real time, and write them into the whole night's sleep analysis report. The present invention provides an end-to-end automatic sleep staging framework, which overcomes the inefficiency and tediousness of traditional manual staging and improves the classification accuracy of human sleep stages.

Description

Technical Field

[0001] The present invention relates to the technical field of deep learning and biomedicine, and in particular to an automatic sleep staging system based on a deep neural network and a brain-computer interface. Background Art

[0002] Faced with increasing work and life pressures, more and more people are forced into fast-paced lifestyles. This is accompanied by the distress caused by sleep disorders and a sharp decline in sleep quality. In this era, research on sleep activity and improving sleep quality is of great significance. Sleep staging is a key area of ​​sleep activity research. Through sleep staging, an individual's sleep activity progression becomes clear at a glance. It also simplifies the calculation of sleep quality assessment indicators such as deep sleep duration, total sleep time, and sleep efficiency. Sleep staging is a valuable aid in sleep quality assessment. Summary of the Invention

[0003] The purpose of the present invention is to overcome the shortcomings and deficiencies of the existing technology and propose an automatic sleep staging system based on deep neural networks and brain-computer interfaces. The system provides an end-to-end automatic sleep staging framework, overcoming the inefficiency and tediousness of traditional manual staging, while improving the classification accuracy of human sleep stages.

[0004] To achieve the above objectives, the present invention provides a technical solution: an automatic sleep staging system based on a deep neural network and a brain-computer interface, comprising:

[0005] Data acquisition module, used to collect and record the user's EEG signals;

[0006] The data preprocessing module is used to process the EEG signals obtained by the data acquisition module, including window segmentation and data enhancement. It also preprocesses the data to reduce noise and dimensionality in the original data, and obtain 30-second data samples that comply with the American Academy of Sleep Medicine (AASM) standards.

[0007] The staging model training module uses a deep neural network to build an automatic sleep staging model. It then uses the constructed automatic sleep staging model to perform feature representation, feature enhancement, and feature capture on single-channel EEG signals, and obtains the optimal automatic sleep staging model through multiple iterations.

[0008] The automatic staging module uses the optimal automatic sleep staging model obtained through iteration to automatically identify the sleep stages of pre-processed sleep EEG samples;

[0009] The feedback module is used to transmit the results processed by the automatic staging module, output them in real time, and write them into the whole night's sleep analysis report.

[0010] Furthermore, the data acquisition module collects the user's EEG signals and is connected to a computer equipped with a data processing module via a serial port connection line and a parallel port connection line.

[0011] Furthermore, the data acquisition module uses a 32-channel EEG electrode cap and a signal amplifier to record scalp EEG signals and transmit the EEG signals to the data preprocessing module; the subjects need to wear the EEG electrode cap and then fall asleep, and the EEG signals of all channels are referenced to the right ear protrusion. Only the EEG signal of a single channel is used for training the automatic sleep staging model and subsequent calculation of the staging accuracy of the real-time sleep staging system; during the EEG acquisition process, the impedance values ​​of all electrodes are below 5KΩ to ensure good data quality, and the EEG signals are sampled at a frequency of 250Hz.

[0012] Furthermore, the data preprocessing module performs the following operations:

[0013] a. Following the American Academy of Sleep Medicine (AASM) standards, we divided each 30-second data sample in the dataset into five sleep stages: W, N1, N2, N3, and REM. We removed data samples that could not be identified according to the AASM standards because they did not belong to any sleep stage. We then only retained the W stage, which lasted half an hour before and after sleep.

[0014] b. Since the dataset is the raw EEG signal collected by the EEG electrode cap, in order to reduce the noise and retain the effective information in the signal, bandpass filtering is performed in the range of 0.5 to 25 Hz;

[0015] c. Since the original EEG signal is sampled at a frequency of 250 Hz, in order to reduce the dimension of the original EEG signal, it needs to be downsampled. The sampling frequency of the downsampled signal is 100 Hz, and then the filtered and downsampled data is normalized;

[0016] d. Fixedly select the EEG signal data of one channel in the dataset as the input signal of the automatic sleep staging model. Use a sliding window method to cut the EEG signal into multiple segments with a time window of 30 seconds. Use a fixed step size sliding window of less than 30 seconds to enhance the data. An EEG signal segment within the 30-second window constitutes a data sample.

[0017] Furthermore, the staging model training module performs the following operations:

[0018] 1) Build an automatic sleep staging model, which consists of three parts: a one-dimensional convolutional neural network structure as a sample feature extraction module, a ResNet structure in the convolutional neural network as a sample feature enhancement module, and a self-attention mechanism structure in the convolutional neural network as a sample feature capture module;

[0019] 2) The automatic sleep staging model is used to perform feature representation, feature enhancement, and feature extraction on single-channel EEG signals, as follows:

[0020] The initial input to the automatic sleep staging model is characterized by the sample feature extraction module. The operation of the sample feature extraction module consists of four groups of one-dimensional convolutional layers, batch normalization layers, and nonlinear activation layers. The nonlinear activation layer is completed by the LRelu function g(x):

[0021] g(x)=max(μx,x)

[0022] Where x is the output of the previous batch normalization layer, μ is the coefficient that ensures that the output of x is not 0 when it is in the negative interval. It is determined before training to ensure that when some input samples enter the left half of LRelu, there is still a small gradient that can be used for update;

[0023] The output of the sample feature extraction module is used as the input of the sample feature enhancement module. This module uses an average pooling layer with a convolution kernel size equal to the input length and a stride of 1 according to the American Academy of Sleep Medicine (AASM) standard to simulate and calculate the duration of the characteristic waveform to enhance the sample features.

[0024] The output of the sample feature enhancement module is used as the input of the sample feature capture module. The sample feature capture module adopts the self-attention mechanism structure to obtain an attention weight matrix, which is multiplied by the two-dimensional output of the sample feature enhancement module. Then, it passes through a global average pooling layer to output a weighted sequence, thereby using the self-attention mechanism structure to recognize the characteristic waveform within the sample;

[0025] The output of the sample feature capture module is input into a softmax layer with 5 neurons. The softmax layer calculates the classification probabilities of the 5 sleep stages corresponding to the input sample. The probability values ​​of all neurons are in the range of (0, 1), and their sum is 1. The sleep stage with the highest classification probability is used as the result of automatic sleep staging:

[0026]

[0027] Where m is the number of neurons in the softmax layer, i is the reference number of the neuron, and the exp function is an exponential function with e as the base. The output x of the sample feature capture module is iTransform it to an exponential function to ensure the non-negativity of the probability, and then divide the transformed result by the sum of the transformed results output by each neuron to ensure that the classification probability is between 0 and 1;

[0028] 3) To train the automatic sleep staging model, because human sleep structure is affected by age and development, the dataset is divided into n parts according to the age group of the subjects. Each part is used as a validation set, and the remaining parts are used as training sets.

[0029] 4) The training set is sequentially input into the automatic sleep staging model for training. After multiple iterations, the most appropriate parameters are selected to obtain n automatic sleep staging models containing the learned parameters, i.e., the optimal automatic sleep staging model.

[0030] Furthermore, the automatic staging module identifies different sleep characteristic waveforms based on single-channel EEG signals and performs automatic sleep staging, including the following steps:

[0031] 1) Use the data acquisition module to load the single-channel EEG signal, and use the data preprocessing module to process the EEG signal obtained by the data acquisition module;

[0032] 2) The EEG signal processed in step 1) is automatically classified into sleep stages using n automatic sleep staging models with learned parameters trained by the staging model training module;

[0033] 3) Obtaining n probability sequences from n automatic sleep staging models with learned parameters, and obtaining the final prediction result by summing the probability sequences.

[0034] Furthermore, the feedback module can output the sleep staging results of the current sleep stage at an adjustable time interval, and the lag time between the result and the actual state of the subject is small, and finally output a complete sleep report, including nighttime sleep time, deep sleep ratio, light sleep ratio, rapid eye movement ratio and number of awakenings.

[0035] Compared with the prior art, the present invention has the following advantages and beneficial effects:

[0036] 1. An end-to-end automatic sleep staging model was built using deep learning methods, which improved the efficiency of sleep stage classification in the auxiliary diagnosis process of sleep disorders.

[0037] 2. Sleep monitoring based on brain-computer interface is more accurate and effective than other sleep staging methods (such as blood oxygen and electrocardiogram signal recognition). The use of EEG signal recognition can more realistically reflect the user's sleep state.

[0038] 3. Combining medicine with computer engineering will help promote the development of integrated medicine and engineering.

[0039] 4. Real-time sleep stage judgment can be combined with sleep intervention measures to further regulate the user's sleep quality. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] Figure 1 Schematic diagram of the relationship between the various modules of the system of the present invention. DETAILED DESCRIPTION

[0041] The present invention will be described in further detail below with reference to the embodiments and drawings, but the embodiments of the present invention are not limited thereto.

[0042] This embodiment provides an automatic sleep staging system based on deep neural network and brain-computer interface, which is developed in Python and can be run on Windows devices. The relationship between the various modules of the system is as follows: Figure 1 As shown, it includes:

[0043] Data acquisition module, used to collect and record the user's EEG signals;

[0044] The data preprocessing module is used to process the EEG signals obtained by the data acquisition module, including window segmentation and data enhancement. It also preprocesses the data to reduce noise and dimensionality in the original data, and obtain 30-second data samples that comply with the American Academy of Sleep Medicine (AASM) standards.

[0045] The staging model training module uses a deep neural network to build an automatic sleep staging model. It then uses the constructed automatic sleep staging model to perform feature representation, feature enhancement, and feature capture on single-channel EEG signals, and obtains the optimal automatic sleep staging model through multiple iterations.

[0046] The automatic staging module uses the optimal automatic sleep staging model obtained through iteration to automatically identify the sleep stages of pre-processed sleep EEG samples;

[0047] The feedback module is used to transmit the results processed by the automatic staging module, output them in real time, and write them into the whole night's sleep analysis report.

[0048] Specifically, the data acquisition module collects the user's EEG signals and is connected to a computer equipped with a data processing module via a serial port connection line and a parallel port connection line.

[0049] Specifically, the data acquisition module uses a 32-channel EEG electrode cap and a signal amplifier to record scalp EEG signals and transmit them to a data preprocessing module. The subject is required to wear the EEG electrode cap and fall asleep. The EEG signals of all channels are referenced to the right ear protrusion. In this embodiment, only a single channel of EEG signals is used for training the automatic sleep staging model and subsequently calculating the staging accuracy of the real-time sleep staging system. During the EEG acquisition process, the impedance of all electrodes is kept below 5 kΩ to ensure good data quality, and the EEG signals are sampled at a frequency of 250 Hz.

[0050] Specifically, the data preprocessing module performs the following operations:

[0051] a. Following the American Academy of Sleep Medicine (AASM) standards, we divided each 30-second data sample in the dataset into five sleep stages: W, N1, N2, N3, and REM. We removed data samples that could not be identified according to the AASM standards because they did not belong to any sleep stage. We then only retained the W stage, which lasted half an hour before and after sleep.

[0052] b. Since the dataset is the raw EEG signal collected by the EEG electrode cap, in order to reduce the noise and retain the effective information in the signal, bandpass filtering is performed in the range of 0.5 to 25 Hz;

[0053] c. Since the original EEG signal is sampled at a frequency of 250 Hz, in order to reduce the dimension of the original EEG signal, it needs to be downsampled. The sampling frequency of the downsampled signal is 100 Hz, and then the filtered and downsampled data is normalized;

[0054] d. Fixedly select the EEG signal data of one channel in the dataset as the input signal of the automatic sleep staging model. Use a sliding window method to cut the EEG signal into multiple segments with a time window of 30 seconds. Use a fixed step size sliding window of less than 30 seconds to enhance the data. An EEG signal segment within the 30-second window constitutes a data sample.

[0055] Specifically, the staging model training module performs the following operations:

[0056] 1) Build an automatic sleep staging model, which consists of three parts: a one-dimensional convolutional neural network structure as a sample feature extraction module, a ResNet structure in the convolutional neural network as a sample feature enhancement module, and a self-attention mechanism structure in the convolutional neural network as a sample feature capture module;

[0057] 2) The automatic sleep staging model is used to perform feature representation, feature enhancement, and feature extraction on single-channel EEG signals, as follows:

[0058] The initial input to the automatic sleep staging model is characterized by the sample feature extraction module. The operation of the sample feature extraction module consists of four groups of one-dimensional convolutional layers, batch normalization layers, and nonlinear activation layers. The nonlinear activation layer is completed by the LRelu function g(x):

[0059] g(x)=max(μx,x)

[0060] Where x is the output of the previous batch normalization layer, and μ is a coefficient that ensures that the output of x is not 0 when it is in the negative interval. It is usually a very small value that is determined before training to ensure that when some input samples enter the left half of LRelu, there is still a small gradient that can be used for updating.

[0061] The output of the sample feature extraction module is used as the input of the sample feature enhancement module. This module uses an average pooling layer with a convolution kernel size equal to the input length and a stride of 1 according to the American Academy of Sleep Medicine (AASM) standard to simulate and calculate the duration of the characteristic waveform to enhance the sample features.

[0062] The output of the sample feature enhancement module is used as the input of the sample feature capture module. The sample feature capture module adopts the self-attention mechanism structure to obtain an attention weight matrix, which is multiplied by the two-dimensional output of the sample feature enhancement module. Then, it passes through a global average pooling layer to output a weighted sequence, thereby using the self-attention mechanism structure to recognize the characteristic waveform within the sample;

[0063] The output of the sample feature capture module is input into a softmax layer with 5 neurons. The softmax layer calculates the classification probabilities of the 5 sleep stages corresponding to the input sample. The probability values ​​of all neurons are in the range of (0, 1), and their sum is 1. The sleep stage with the highest classification probability is used as the result of automatic sleep staging:

[0064]

[0065] Where m is the number of neurons in the softmax layer, i is the reference number of the neuron, and the exp function is an exponential function with e as the base. The output x of the sample feature capture module is i Transform it to an exponential function to ensure the non-negativity of the probability, and then divide the transformed result by the sum of the transformed results output by each neuron to ensure that the classification probability is between 0 and 1;

[0066] 3) To train the automatic sleep staging model, because human sleep structure is affected by age and development, the dataset is divided into n parts according to the age group of the subjects. Each part is used as a validation set, and the remaining parts are used as training sets.

[0067] 4) The training set is sequentially input into the automatic sleep staging model for training. After multiple iterations, the most appropriate parameters are selected to obtain n automatic sleep staging models containing the learned parameters, i.e., the optimal automatic sleep staging model.

[0068] Specifically, the automatic staging module identifies different sleep characteristic waveforms based on single-channel EEG signals and performs automatic sleep staging, including the following steps:

[0069] 1) Use the data acquisition module to load the single-channel EEG signal, and use the data preprocessing module to process the EEG signal obtained by the data acquisition module;

[0070] 2) The EEG signal processed in step 1) is automatically classified into sleep stages using n automatic sleep staging models with learned parameters trained by the staging model training module;

[0071] 3) Obtaining n probability sequences from n automatic sleep staging models with learned parameters, and obtaining the final prediction result by summing the probability sequences.

[0072] Specifically, the feedback module can output the sleep staging results of the current sleep stage at an adjustable time interval, and the lag time between the result and the actual state of the subject is small, and finally output a complete sleep report, including indicators such as nighttime sleep time, deep sleep ratio, light sleep ratio, rapid eye movement ratio, and number of awakenings.

[0073] The embodiments described above are only preferred embodiments of the present invention and are not intended to limit the scope of implementation of the present invention. Therefore, any changes made based on the shape and principle of the present invention should be included in the scope of protection of the present invention.

Claims

1. An automatic sleep staging system based on deep neural network and brain-computer interface, characterized by: include: Data acquisition module, used to collect and record the user's EEG signals; The data preprocessing module is used to process the EEG signals obtained by the data acquisition module, including window segmentation and data enhancement. It also preprocesses the data to reduce noise and dimensionality in the original data, and obtain 30-second data samples that comply with the American Academy of Sleep Medicine (AASM) standards. The staging model training module uses a deep neural network to build an automatic sleep staging model. It then uses the constructed automatic sleep staging model to perform feature representation, feature enhancement, and feature capture on single-channel EEG signals, and obtains the optimal automatic sleep staging model through multiple iterations. The automatic staging module uses the optimal automatic sleep staging model obtained through iteration to automatically identify the sleep stages of pre-processed sleep EEG samples; The feedback module is used to transmit the results processed by the automatic staging module, output them in real time, and write them into the whole night's sleep analysis report; The staging model training module performs the following operations: 1) Build an automatic sleep staging model, which consists of three parts: a one-dimensional convolutional neural network structure as a sample feature extraction module, a ResNet structure in the convolutional neural network as a sample feature enhancement module, and a self-attention mechanism structure in the convolutional neural network as a sample feature capture module; 2) The automatic sleep staging model is used to perform feature representation, feature enhancement, and feature extraction on single-channel EEG signals, as follows: The initial input to the automatic sleep staging model is characterized by the sample feature extraction module. The operation of the sample feature extraction module consists of four groups of one-dimensional convolutional layers, batch normalization layers, and nonlinear activation layers. The nonlinear activation layer is completed by the LRelu function g(x): g(x)=max(μx,x) Where x is the output of the previous batch normalization layer, μ is the coefficient that ensures that the output of x is not 0 when it is in the negative interval. It is determined before training to ensure that when some input samples enter the left half of LRelu, there is still a small gradient that can be used for update; The output of the sample feature extraction module is used as the input of the sample feature enhancement module. This module uses an average pooling layer with a convolution kernel size equal to the input length and a stride of 1 according to the American Academy of Sleep Medicine (AASM) standard to simulate and calculate the duration of the characteristic waveform to enhance the sample features. The output of the sample feature enhancement module is used as the input of the sample feature capture module. The sample feature capture module adopts the self-attention mechanism structure to obtain an attention weight matrix, which is multiplied by the two-dimensional output of the sample feature enhancement module. Then, it passes through a global average pooling layer to output a weighted sequence, thereby using the self-attention mechanism structure to recognize the characteristic waveform within the sample; The output of the sample feature capture module is input into a softmax layer with 5 neurons. The softmax layer calculates the classification probabilities of the 5 sleep stages corresponding to the input sample. The probability values ​​of all neurons are in the range of (0, 1), and their sum is 1. The sleep stage with the highest classification probability is used as the result of automatic sleep staging: Where m is the number of neurons in the softmax layer, i is the reference number of the neuron, and the exp function is an exponential function with e as the base. The output x of the sample feature capture module is i Transform it to an exponential function to ensure the non-negativity of the probability, and then divide the transformed result by the sum of the transformed results output by each neuron to ensure that the classification probability is between 0 and 1; 3) To train the automatic sleep staging model, because human sleep structure is affected by age and development, the dataset is divided into n parts according to the age group of the subjects. Each part is used as a validation set, and the remaining parts are used as training sets. 4) sequentially inputting the training set into the automatic sleep staging model for training, selecting the most appropriate parameters after multiple iterations, and obtaining n automatic sleep staging models containing the learned parameters, i.e., the optimal automatic sleep staging model; The feedback module can output the sleep staging results of the current sleep stage at an adjustable time interval, and the lag time between the results and the actual state of the subject is small. Finally, it outputs a complete sleep report, including nighttime sleep time, deep sleep ratio, light sleep ratio, rapid eye movement ratio, and number of awakenings.

2. The automatic sleep staging system based on deep neural network and brain-computer interface according to claim 1 is characterized by: The data acquisition module collects the user's brain electrical signals and is connected to a computer equipped with a data processing module via a serial port connection line and a parallel port connection line.

3. The automatic sleep staging system based on deep neural network and brain-computer interface according to claim 2 is characterized by: The data acquisition module uses a 32-channel EEG electrode cap and a signal amplifier to record scalp EEG signals and transmit the EEG signals to the data preprocessing module; the subject needs to wear the EEG electrode cap and then fall asleep. The EEG signals of all channels are referenced to the right ear protrusion. Only the EEG signal of a single channel is used for training the automatic sleep staging model and subsequently calculating the staging accuracy of the real-time sleep staging system; during the EEG acquisition process, the impedance values ​​of all electrodes are all below 5KΩ to ensure good data quality, and the EEG signals are sampled at a frequency of 250Hz.

4. The automatic sleep staging system based on deep neural network and brain-computer interface according to claim 3 is characterized in that: The data preprocessing module performs the following operations: a. Following the American Academy of Sleep Medicine (AASM) standards, we divided each 30-second data sample in the dataset into five sleep stages: W, N1, N2, N3, and REM. We removed data samples that could not be identified according to the AASM standards because they did not belong to any sleep stage. We then only retained the W stage, which lasted half an hour before and after sleep. b. Since the dataset is the raw EEG signal collected by the EEG electrode cap, in order to reduce the noise and retain the effective information in the signal, a bandpass filter is performed in the range of 0.5 to 25 Hz; c. Since the original EEG signal is sampled at a frequency of 250 Hz, in order to reduce the dimension of the original EEG signal, it needs to be downsampled. The sampling frequency of the downsampled signal is 100 Hz, and then the filtered and downsampled data is normalized; d. Fixedly select the EEG signal data of one channel in the dataset as the input signal of the automatic sleep staging model. Use a sliding window method to cut the EEG signal into multiple segments with a time window of 30 seconds. Use a fixed step size sliding window of less than 30 seconds to enhance the data. An EEG signal segment within the 30-second window constitutes a data sample.

5. The automatic sleep staging system based on deep neural network and brain-computer interface according to claim 4 is characterized in that: The automatic staging module identifies different sleep characteristic waveforms based on single-channel EEG signals and performs automatic sleep staging, including the following steps: 1) Use the data acquisition module to load the single-channel EEG signal, and use the data preprocessing module to process the EEG signal obtained by the data acquisition module; 2) The EEG signal processed in step 1) is automatically classified into sleep stages using n automatic sleep staging models with learned parameters trained by the staging model training module; 3) Obtaining n probability sequences from n automatic sleep staging models with learned parameters, and obtaining the final prediction result by summing the probability sequences.

Images