A sleep state detection method, device and system based on an EEG acquisition headband

Abstract

The present invention provides a sleep state detection method, device and system based on an EEG acquisition headband. Among them, a sleep state detection method based on an EEG acquisition headband includes obtaining EEG brain signals of FP1 electrode and FP2 electrode in the 10-20 international standard lead system; processing the EEG brain signals to obtain 30s of EEG epoch signals; inputting the 30s of EEG epoch signals into a trained electroencephalogram staging attention mechanism model to obtain the sleep stage staging result output by the electroencephalogram staging attention mechanism model. The present invention provides a sleep state detection method that can collect and analyze EEG signals to obtain the sleep state of the user, and further provide a reference for the user to understand their own sleep health status.

Description

Technical Field

[0001] The present invention belongs to the field of sleep detection, and in particular, relates to a sleep state detection method, device and system based on an EEG acquisition headband. Background Art

[0002] Sleep is one of the most important physiological activities of human beings, and good sleep is beneficial to human health. However, with the progress of the times, the pressure on people in work, study and life has also increased, and more and more people suffer from sleep disorders. Sleep disorders can cause many problems, including fatigue, anxiety, depression and the risk of death, and are diseases with public harm. Sleep disorders are common diseases among sub-healthy people, manifested as poor sleep quality, insomnia, etc., and may induce various cardiovascular and cerebrovascular diseases. According to the survey data of the World Health Organization in 2019, there are more than 300 million people in China with sleep disorders, among which the insomnia incidence rate of adults is as high as 38.2%, and more than 60% of young people feel sleep-deprived. Research has found that cognitive impairment, memory impairment, Alzheimer's disease, cardiovascular diseases and some chronic diseases are all closely related to sleep quality. Electroencephalogram (EEG) records the activities of the cerebral cortex and reflects the activity state and thinking state of the human brain. EEG signal detection technology can reflect a person's sleep state in real time, so that people can understand their own sleep health status and make timely adjustments.

[0003] Compared with the traditional EEG acquisition instruments in medical institutions, the portable EEG acquisition system has an extremely small volume and mass under the condition of ensuring the acquisition accuracy and meeting the requirements of the acquisition speed, has low working condition requirements, greatly improves the portability, and has been widely used in brain-computer interface devices. Especially under the condition that the user needs to carry out daily activities, the portable EEG acquisition system can provide the conditions for the user to perform EEG detection, and provides convenience for some physiological states that need to continuously monitor the changes of EEG. However, EEG data is often large in quantity and difficult to intuitively find its characteristics. Therefore, it will be very difficult to perform manual analysis after the data is collected. Therefore, in recent years, with the development of computer technology, more and more scientific researchers prefer to use computers to analyze and process EEG data using some analysis algorithms. However, traditional EEG analysis methods either do not pay attention to cross-time analysis, or often use LSTM for analysis, resulting in a cumbersome training process. Moreover, the existing neural network models highly rely on hyperparameter tuning and have poor applicability. After changing to another tester, the model needs to be retrained and retuned, otherwise good detection effects cannot be obtained. Summary of the Invention

[0004] In view of this, the present invention aims to propose a sleep state detection method, device and system based on an EEG acquisition headband to solve the problem that the test effect of the neural network model is not good when testing different testers in the prior art.

[0005] To achieve the above object, the technical solution of the present invention is implemented as follows:

[0006] In a first aspect, an embodiment of the present invention provides a sleep state detection method based on an EEG acquisition headband, including:

[0007] Obtain EEG signals of FP1 electrode and FP2 electrode in the 10-20 international standard lead system;

[0008] Process the EEG signals to obtain 30s EEG epoch signals;

[0009] Input the 30s EEG epoch signals into the trained electroencephalogram staging attention mechanism model to obtain the approximate Hilbert transform of the 30s EEG epoch signals, that is, the EEG signal epoch;

[0010] Encode the EEG signal epoch using the encoder in the electroencephalogram staging attention mechanism model to obtain a vector mapping sequence; the encoder includes a first residual block, a second residual block, and a global average pooling layer connected in sequence;

[0011] Decode the vector mapping sequence using the decoder based on the multi-head attention mechanism in the electroencephalogram staging attention mechanism model; the decoder includes a multi-head attention sub-module, a first fully connected layer, and a second fully connected layer connected in sequence;

[0012] Obtain the sleep stage staging result output by the electroencephalogram staging attention mechanism model.

[0013] Further, the processing of the EEG signals to obtain 30s EEG epoch signals includes:

[0014] Process the EEG signals using a sixth-order Butterworth band-pass filter with a cut-off frequency of 0.1-50Hz;

[0015] Downsample the processed EEG signals from 500Hz to 100Hz, and then divide the complete EEG signals into time series segments of 30s each to obtain several 30s EEG epoch signals.

[0016] Further, the method for the approximate Hilbert transform of the 30s EEG epoch signals includes:

[0017] Input the 30s EEG epoch signal into the first convolutional layer. The size of the convolutional kernel in the first convolutional layer is Fs = N / 100, with 64 convolutional kernels, and the moving step size is Sd = N / 1000. Here, N is the number of sampling points contained in each 30s EEG epoch signal, that is, N = 30×100 = 3000.

[0018] Input the output of the first convolutional layer into two max-pooling layers respectively. Among them, the size of the output of the first max-pooling layer is N / 1000, and its output is the frequency feature of the 30s EEG epoch signal. The size of the output of the second max-pooling layer is π×N / 1000, and its output is the amplitude feature of the input 30s EEG epoch signal.

[0019] Sum the outputs of the two max-pooling layers point by point to obtain the approximate Hilbert transform of the 30s EEG epoch signal. Denote the input 30s EEG epoch signal as s(t), then the Hilbert transform is:

[0020] s a (t) = s(t) + i*H(s(t))

[0021] Among them, the real part contains frequency information, and the imaginary part contains amplitude information.

[0022] * represents the convolution operation.

[0023] Furthermore, the first residual block in the encoder includes:

[0024] The second convolutional layer, the number of convolutional kernels in the second convolutional layer is 128, the size of the convolutional kernel is 1×1, and the step size is 2.

[0025] The third convolutional layer, the number of convolutional kernels in the third convolutional layer is 64, the size of the convolutional kernel is 3×1, and the step size is 1.

[0026] The fourth convolutional layer, the number of convolutional kernels in the fourth convolutional layer is 128, the size of the convolutional kernel is 1×1, and the step size is 1.

[0027] Among them, the shortcut connection of the first residual block residual network uses an average pooling layer, and the size of the average pooling layer is 2, and the step size is 2, so as to realize the conversion of the 64-dimensional input into a 128-dimensional output; sum the output of the fourth convolutional layer and the output of the average pooling layer, and then after passing through the RELU function, obtain the output of the first residual block.

[0028] Furthermore, the second residual block in the encoder includes:

[0029] The fifth convolutional layer, the number of convolutional kernels in the fifth convolutional layer is 128, the size of the convolutional kernel is 1×1, and the step size is 2.

[0030] The sixth convolutional layer, the number of convolutional kernels in the sixth convolutional layer is 64, the size of the convolutional kernel is 3×1, and the stride is 1;

[0031] The seventh convolutional layer, the number of convolutional kernels in the seventh convolutional layer is 128, the size of the convolutional kernel is 1×1, and the stride is 1;

[0032] Among them, the shortcut connection of the second residual block residual network uses a direct connection; the output of the seventh convolutional layer and the direct connection are summed, and after passing through the RELU function, the output of the second residual block is obtained.

[0033] Furthermore, after encoding the EEG signal epoch using the encoder in the EEG staging attention mechanism model to obtain a vector mapping sequence, the method further includes:

[0034] Input the vector mapping sequence output by the global average pooling layer into the first DropOut layer to obtain the output of the encoder; among them, the DropOut probability of the first DropOut layer is 0.5.

[0035] Furthermore, the decoding of the vector mapping sequence using the decoder based on the multi-head attention mechanism in the EEG staging attention mechanism model includes:

[0036] After summing and normalizing the output of the encoder and the positional encoding PE, input it into the multi-head attention mechanism sub-module of the decoder; among them, the calculation method of the positional encoding PE is as follows:

[0037]

[0038] where p represents the position of the current EEG signal epoch in the input EEG signal epoch queue; d represents the number of EEG signal epochs included in the input queue input to the multi-head attention mechanism sub-module; dim represents the number of data output by the global average pooling layer;

[0039] Using the residual connection, sum and normalize the input and output of the decoder, and then input it into the second DropOut layer; among them, the DropOut probability of the second DropOut layer is 0.5;

[0040] Input the output of the second DropOut layer into the SoftMax layer, and the SoftMax layer outputs the sleep stage staging result.

[0041] Furthermore, after summing and normalizing the output of the encoder and the positional encoding PE and then inputting it into the multi-head attention mechanism sub-module of the decoder, the method further includes:

[0042] The output of the multi-head attention mechanism sub-module is input into the third DropOut layer, and the DropOut probability of the third DropOut layer is 0.8. The output of the multi-head attention mechanism sub-module is normalized through the processing of the third DropOut layer. Among them, the number of heads in the multi-head attention mechanism sub-module is 8, the size of each head is 64 dimensions, and the attention mechanism is executed in parallel for each head. The attention mechanism function is expressed as:

[0043]

[0044] where X is the matrix composed of the input sequence, and dim represents the data dimension; the in the expression corresponds to the scaling layer in the multi-head attention mechanism sub-module;

[0045] After inputting the output of the third DropOut layer into the first fully connected layer, the result is then activated by the RELU function;

[0046] The output of the first fully connected layer is input into the second fully connected layer, and the output result of the second fully connected layer is the output of the decoder. Among them, the normalized output of the multi-head attention mechanism is added to the outputs of the first fully connected layer and the second fully connected layer through skip connections.

[0047] In a second aspect, an embodiment of the present invention further provides a sleep state detection device based on an electroencephalogram acquisition headband, including:

[0048] An acquisition module, configured to acquire EEG brain electrical signals of FP1 electrode and FP2 electrode in the 10-20 international standard lead system;

[0049] A preprocessing module, configured to process the EEG brain electrical signals to obtain 30s EEG epoch signals;

[0050] A transformation module, configured to input the 30s EEG epoch signals into the trained electroencephalogram staging attention mechanism model to obtain an approximate Hilbert transform of the 30s EEG epoch signals, that is, the EEG signal epoch;

[0051] An encoding module, configured to encode the EEG signal epoch by using the encoder in the electroencephalogram staging attention mechanism model to obtain a vector mapping sequence;

[0052] A decoding module, configured to decode the vector mapping sequence by using the decoder based on the multi-head attention mechanism in the electroencephalogram staging attention mechanism model;

[0053] An output module, configured to obtain the sleep stage staging result output by the electroencephalogram staging attention mechanism model.

[0054] Thirdly, an embodiment of the present invention further provides a sleep state detection system based on an EEG acquisition headband, including:

[0055] The sleep state detection device based on the EEG acquisition headband provided in the above embodiment;

[0056] An EEG signal acquisition headband and an EEG signal analysis device.

[0057] Compared with the prior art, the sleep state detection method, device and system based on the EEG acquisition headband of the present invention have the following advantages:

[0058] The present invention provides a sleep state detection method capable of collecting and analyzing EEG signals to obtain the sleep state of the user, and further providing a reference for the user to understand their own sleep health state.

[0059] The present invention also provides a sleep state detection device and system based on a portable EEG acquisition headband, which can accurately collect, accurately identify and correctly classify EEG signals, and classify the sleep state of the user by identifying the EEG signals of the user for the user to refer to and understand their own sleep health state. BRIEF DESCRIPTION OF THE DRAWINGS

[0060] The drawings constituting a part of the present invention are used to provide a further understanding of the present invention. The schematic embodiments and descriptions thereof of the present invention are used to explain the present invention and do not constitute an improper limitation to the present invention. In the drawings:

[0061] Figure 1 is a schematic flow chart of the sleep state detection method based on the EEG acquisition headband in Embodiment 1 of the present invention;

[0062] Figure 2 is a schematic structural diagram of the EEG signal acquisition headband in the sleep state detection method based on the EEG acquisition headband provided in Embodiment 1 of the present invention;

[0063] Figure 3 is a schematic flow chart of the method for obtaining the 30s EEG epoch signal in the sleep state detection method based on the EEG acquisition headband provided in Embodiment 1 of the present invention;

[0064] Figure 4 is a schematic structural diagram of the neural network unit of the Hilbert transform in the sleep state detection method based on the EEG acquisition headband provided in Embodiment 1 of the present invention;

[0065] Figure 5 is a schematic structural diagram of the first residual block in the sleep state detection method based on the EEG acquisition headband provided in Embodiment 1 of the present invention;

[0066] Figure 6Schematic diagram of the second residual block in the sleep state detection method based on an EEG acquisition headband provided in the first embodiment of the present invention;

[0067] Figure 7 Flow chart of the sleep state detection method based on an EEG acquisition headband provided in the second embodiment of the present invention;

[0068] Figure 8 Schematic diagram of the encoder in the sleep state detection method based on an EEG acquisition headband provided in the second embodiment of the present invention;

[0069] Figure 9 Flow chart of the sleep state detection method based on an EEG acquisition headband provided in the third embodiment of the present invention;

[0070] Figure 10 Schematic diagram of the decoder in the sleep state detection method based on an EEG acquisition headband provided in the third embodiment of the present invention;

[0071] Figure 11 Schematic diagram of the sleep state detection device based on an EEG acquisition headband provided in the fourth embodiment of the present invention;

[0072] Figure 12 Schematic diagram of the sleep state detection system based on an EEG acquisition headband provided in the fourth embodiment of the present invention. Detailed implementation manners

[0073] It should be noted that, without conflict, the embodiments in the present invention and the features in the embodiments may be combined with each other.

[0074] In the description of the present invention, it should be understood that the orientation or positional relationship indicated by the terms "center", "longitudinal", "transverse", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inner", "outer", etc. are based on the orientation or positional relationship shown in the drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and thus should not be construed as a limitation to the present invention. In addition, the terms "first", "second", etc. are only used for descriptive purposes and cannot be understood as indicating or implying relative importance or implicitly specifying the number of the indicated technical features. Thus, the features defined with "first", "second", etc. may explicitly or implicitly include one or more of such features. In the description of the present invention, unless otherwise specified, the meaning of "a plurality" is two or more.

[0075] In the description of the present invention, it should be noted that unless otherwise clearly defined and limited, the terms "installation", "connection", and "coupling" should be understood in a broad sense. For example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be directly connected or indirectly connected through an intermediate medium, and it can be the communication inside two components. For those of ordinary skill in the art, the specific meanings of the above terms in the present invention can be understood through specific situations.

[0076] The present invention will be described in detail below with reference to the accompanying drawings and in conjunction with embodiments.

[0077] Embodiment 1

[0078] Figure 1 FIG. 10 is a schematic flowchart of a sleep state detection method based on an EEG acquisition headband provided in Embodiment 1 of the present invention. This embodiment is applicable to the sleep state detection using a portable EEG acquisition headband. This method can be executed by a sleep state detection device based on an EEG acquisition headband, and specifically includes the following steps:

[0079] S110. Obtain the EEG brain electrical signals of the FP1 electrode and the FP2 electrode in the 10 - 20 international standard lead system;

[0080] Exemplarily, a portable EEG signal acquisition headband can be used, and the electrode patches can be placed with reference to the 10 - 20 system electrode placement method to obtain the EEG brain electrical signals of two electrodes, namely FP1 and FP2, corresponding to the 10 - 20 system electrode placement method of the user.

[0081] Figure 2 FIG. 20 is a schematic structural diagram of an EEG signal acquisition headband in the sleep state detection method based on an EEG acquisition headband provided in Embodiment 1 of the present invention. Refer to Figure 2 , the EEG signal acquisition headband includes: an electrode patch for collecting EEG brain electrical signals and a connection device connected in sequence, a bioelectrical signal acquisition module for amplifying and converting brain electrical signals, an ESP32 processor for controlling the bioelectrical signal acquisition module, and a WIFI wireless transmission circuit for transmitting EEG signals. The EEG signal acquisition headband further includes a power supply circuit respectively connected to the bioelectrical signal acquisition module and the ESP32 processor, and a telescopic headband housing for carrying content chips and circuits. Among them, the electrode patch and the connection device collect the EEG signals in the prefrontal region of the user and are connected to the bioelectrical signal acquisition module through a flexible flat cable for the acquisition and transmission of bioelectrical signals.

[0082] The above bioelectric signal acquisition module is composed of a bioelectric signal acquisition chip integrating a high common-mode rejection ratio analog input module for receiving the voltage signal collected by the EEG cap, a low-noise programmable gain amplifier (PGA) for amplifying the brain voltage signal, and a high-resolution synchronous sampling analog-to-digital converter (ADC) for converting the analog signal into a digital signal.

[0083] The above ESP32 processor is used to control the acquisition mode and parameters of the bioelectric signal acquisition module and control the WiFi wireless transmission mode and transmission speed.

[0084] The input of the above WiFi wireless transmission circuit is the output of the bioelectric signal acquisition module, and it transmits the EEG signal data to the EEG signal analysis software.

[0085] The input voltage of the above power supply circuit is 5V. It is powered by a lithium battery and provides different working voltages required by the system through a voltage conversion module. It can also be directly powered by a USB data cable without using a lithium battery, but this will affect its portability.

[0086] The above retractable headband shell includes a post-auricular fixation device, a telescopic device, an elastic lining, a shell, and electrode openings. It can be worn on the human head and fixed behind the ears through the fixation device. For people with different head shapes, the tightness of the headband can be adjusted through the telescopic device. The shell is provided with four electrode openings in total for placing four electrodes to contact the human body, and the four electrodes are located on the forehead and behind the ears respectively.

[0087] S120. Process the EEG brain electrical signal to obtain a 30s EEG epoch signal;

[0088] First, it is necessary to preprocess the collected brain electrical signal. By intercepting a 30s EEG epoch signal, the data processing difficulty can be reduced and the algorithm analysis efficiency can be improved. Exemplarily, a sixth-order Butterworth band-pass filter with a cut-off frequency of 0.1 - 50Hz can be used to process the complete electroencephalogram vector to filter out the noise and improve the accuracy of subsequent signal processing. Then, downsample the collected EEG signal from 500Hz to 100Hz. Then divide the electroencephalogram vector into time series segments of 30s each, and a 30s EEG epoch signal can be obtained.

[0089] Figure 3 It is a flow schematic diagram of the method for obtaining a 30s EEG epoch signal in the sleep state detection method based on an EEG acquisition headband provided in the first embodiment of the present invention, as Figure 3As shown, the EEG signals can be processed first using a sixth-order Butterworth band-pass filter with a cut-off frequency of 0.1 - 50 Hz. Then, the processed EEG signals are downsampled from 500 Hz to 100 Hz, and then the complete EEG signals are segmented into time series segments of 30 s each, resulting in several 30 s EEG epoch signals.

[0090] S130: Input the 30 s EEG epoch signals into the trained electroencephalogram staging attention mechanism model to obtain the approximate Hilbert transform of the 30 s EEG epoch signals, that is, the EEG epoch signals.

[0091] The Hilbert transform is an important tool for signal analysis and is very useful in signal processing systems and communication systems. It has the following three main functions: First, it is used to construct an analytic signal so that the signal spectrum only contains positive frequency components, thereby reducing the signal sampling rate; second, it can be used to represent band-pass signals, thus providing a method for signal modulation in radio communication; third, it can be combined with other transforms and decompositions for spectral analysis of non-stationary signals.

[0092] Exemplarily, the 30 s EEG epoch can be first input into the first convolutional layer. The convolutional kernel size of this convolutional layer is Fs = N / 100, with 64 convolutional kernels, and the moving step size Sd = N / 1000. N is the number of sampling points contained in each EEG epoch, that is, N = 30 × 100 = 3000. Then, the outputs of the convolutional layer are respectively input into two max-pooling layers. The size of the first pooling layer is N / 1000, and its output is the frequency feature of the input EEG epoch. The size of the second output is π × N / 1000, and its output is the amplitude feature of the input EEG epoch. Finally, the outputs of the two pooling layers are summed point by point to obtain the approximate Hilbert transform of the input signal. Denote the input 30 s EEG epoch as s(t), then the Hilbert transform is:

[0093] s a (t) = s(t) + i * H(s(t))

[0094] Among them, the real part contains frequency information, and the imaginary part contains amplitude information;

[0095] Here, * represents the convolution operation.

[0096] Figure 4Schematic diagram of the neural network unit structure of Hilbert transform in the sleep state detection method based on an EEG acquisition headband provided in the first embodiment of the present invention; the neural network unit includes a first convolutional layer and two max pooling layers; the 30s EEG epoch signal is input into the first convolutional layer, and the convolutional kernel size of the first convolutional layer is Fs = N / 100, with 64 convolutional kernels, and the moving step size Sd = N / 1000; where N is the number of sampling points included in each 30s EEG epoch signal, that is, N = 30 × 100 = 3000; the output of the first convolutional layer is respectively input into the two max pooling layers; among them, the size of the output of the first max pooling layer is N / 1000, and its output is the frequency feature of the 30s EEG epoch signal, and the size of the output of the second max pooling layer is π × N / 1000, and its output is the amplitude feature of the input 30s EEG epoch signal.

[0097] S140. Use the encoder in the electroencephalogram staging attention mechanism model to encode the EEG epoch signal to obtain a vector mapping sequence; the encoder includes a first residual block, a second residual block, and a global average pooling layer connected in sequence.

[0098] Figure 5 Schematic diagram of the structure of the first residual block in the sleep state detection method based on an EEG acquisition headband provided in the first embodiment of the present invention; as Figure 5 shown, the first residual block includes: a second convolutional layer, the number of convolutional kernels of the second convolutional layer is 128, the convolutional kernel size is 1×1, and the step size is 2; a third convolutional layer, the number of convolutional kernels of the third convolutional layer is 64, the convolutional kernel size is 3×1, and the step size is 1; a fourth convolutional layer, the number of convolutional kernels of the fourth convolutional layer is 128, the convolutional kernel size is 1×1, and the step size is 1; the shortcut connection of the first residual block residual network uses an average pooling layer, and the size of the average pooling layer is 2 and the step size is 2 to realize the conversion of the 64-dimensional input into a 128-dimensional output; the output of the fourth convolutional layer and the output of the average pooling layer are summed, and then after passing through the RELU function, the output of the first residual block is obtained.

[0099] Figure 6 Schematic diagram of the structure of the second residual block in the sleep state detection method based on an EEG acquisition headband provided in the first embodiment of the present invention, as Figure 6As shown in the figure, the second residual block includes: a fifth convolutional layer with 128 convolutional kernels, a convolutional kernel size of 1×1, and a stride of 2; a sixth convolutional layer with 64 convolutional kernels, a convolutional kernel size of 3×1, and a stride of 1; a seventh convolutional layer with 128 convolutional kernels, a convolutional kernel size of 1×1, and a stride of 1; wherein, the shortcut connection of the second residual block residual network uses a direct connection; the output of the seventh convolutional layer and the direct connection are summed, and then after passing through the RELU function, the output of the second residual block is obtained.

[0100] Compared with the traditional CNN where the last layer is a fully connected layer with a very large number of parameters, which is prone to overfitting (such as Alexnet), in a CNN model, most of the parameters are occupied by the fully connected layer. Therefore, in this embodiment, global average pooling is proposed to replace the fully connected layer. Different from the traditional fully connected layer, we perform global average pooling on the entire image, so that each feature map can obtain an output. By using average pooling in this way, parameters can be omitted, greatly reducing the network parameters and avoiding overfitting. On the other hand, it has a characteristic that each feature map is equivalent to an output feature, and this feature represents the feature of our output class. Figure 1 Specifically, the advantages of global average pooling include: by strengthening the consistency between the feature map and the category, making the convolutional structure simpler; no parameter optimization is required, so this layer can avoid overfitting; it sums up the spatial information, so it is more stable to the spatial transformation of the input. Therefore, by adopting the global average pooling layer, the processing efficiency of the electroencephalogram staging attention mechanism model can be effectively improved.

[0101]

[0102] S150. Use the decoder based on the multi-head attention mechanism in the electroencephalogram staging attention mechanism model to decode the vector mapping sequence; the decoder includes a multi-head attention sub-module, a first fully connected layer, and a second fully connected layer connected in sequence;

[0103] The vector mapping sequence generated after encoding is decoded by the multi-head attention sub-module and two fully connected layers.

[0104] Exemplarily, the output of the encoder can be summed with the positional encoding PE and normalized, and then input into the multi-head attention mechanism sub-module of the decoder; wherein, the calculation method of the positional encoding PE is as follows:

[0105]

[0106]

[0107] ​Where p represents the position of the current EEG signal epoch in the input EEG signal epoch queue; d represents the number of EEG signal epochs included in the input queue input to the multi-head attention mechanism sub-module; dim represents the number of data outputs by the global average pooling layer; the position mapping of each EEG signal epoch can be directly calculated by the above equation without training, thereby accelerating the training speed and helping the algorithm learn transition rules in two directions.

[0108] Using residual connection, the input and output of the decoder are summed and normalized, and then input into the second DropOut layer; wherein, the DropOut probability of the second DropOut layer is 0.5;

[0109] The output of the second DropOut layer is input into the SoftMax layer, and the SoftMax layer outputs the sleep stage staging result.

[0110] Compared with the traditional method, the multi-head attention mechanism is introduced in this embodiment. Traditional EEG analysis methods either do not pay attention to cross-time analysis, that is, they do not pay attention to the influence of previous and subsequent epochs on the current epoch when analyzing the current epoch, or often use LSTM for analysis, but LSTM cannot perform parallel computing and will prolong the training process. Introducing the multi-head attention mechanism can perform parallel computing and shorten the training process while performing cross-time analysis.

[0111] S160. Obtain the sleep stage staging result output by the EEG staging attention mechanism model.

[0112] By collecting and analyzing EEG signals, the sleep state of the user can be obtained, and thus it can provide a reference for the user to understand their own sleep health status; the sleep state detection method based on the EEG acquisition headband provided in this embodiment can accurately collect, accurately identify and correctly classify EEG signals, classify the sleep state of the user by identifying the EEG signals of the user, for the user to refer to and understand their own sleep health status.

[0113] Embodiment 2

[0114] Figure 7 It is a schematic flowchart of the sleep state detection method based on the EEG acquisition headband provided in Embodiment 2 of the present invention; this embodiment is optimized based on the above embodiment. In this embodiment, after encoding the EEG signal epoch using the encoder in the EEG staging attention mechanism model to obtain a vector mapping sequence, the following steps are added: the vector mapping sequence output by the global average pooling layer is input into the first DropOut layer to obtain the output of the encoder; wherein, the DropOut probability of the first DropOut layer is 0.5.

[0115] Correspondingly, the sleep state detection method based on the EEG acquisition headband provided in this embodiment specifically includes:

[0116] S210. Obtain the EEG signals of the FP1 electrode and the FP2 electrode in the 10-20 international standard lead system;

[0117] S220. Process the EEG signals to obtain 30s of EEG epoch signals;

[0118] S230. Input the 30s of EEG epoch signals into the trained electroencephalogram staging attention mechanism model to obtain the approximate Hilbert transform of the 30s of EEG epoch signals, that is, the EEG signal epoch;

[0119] S240. Encode the EEG signal epoch using the encoder in the electroencephalogram staging attention mechanism model to obtain a vector mapping sequence; the encoder includes a first residual block, a second residual block, and a global average pooling layer connected in sequence;

[0120] S250. Input the vector mapping sequence output by the global average pooling layer into the first DropOut layer to obtain the output of the encoder; wherein, the DropOut probability of the first DropOut layer is 0.5;

[0121] Exemplarily, since 64 convolution kernels are used in the first convolutional layer in the above steps, the features input to the encoder have 64 feature maps; Figure 8 It is a schematic structural diagram of the encoder in the sleep state detection method based on the EEG acquisition headband provided in the second embodiment of the present invention, as Figure 8 shown, the encoder in the electroencephalogram staging attention mechanism model includes:

[0122] A second convolutional layer, the number of convolution kernels of the second convolutional layer is 128, the size of the convolution kernel is 1×1, and the stride is 2.

[0123] A third convolutional layer, the number of convolution kernels is 64, the size of the convolution kernel is 3×1, and the stride is 1.

[0124] A fourth convolutional layer, the fourth convolutional layer is introduced to obtain the percentage of signals in different frequency bands in an epoch. The number of convolution kernels of the fourth convolutional layer is 128, the size of the convolution kernel is 1×1, and the stride is 1.

[0125] The shortcut connection of the residual network. Since the input of the second convolutional layer is 64-dimensional and the output of the fourth convolutional layer is 128-dimensional, in order to perform dimension matching, an average pooling layer with a size of 2 and a stride of 2 is used in the shortcut connection to convert the 64-dimensional input into a 128-dimensional output.

[0126] Sum the output of the fourth convolutional layer and the output of the average pooling layer, and then pass through the RELU function to obtain the output of the first residual block.

[0127] The fifth convolutional layer, with the number of convolutional kernels being 128, the size of the convolutional kernels being 1×1, and the stride being 2.

[0128] The sixth convolutional layer, with the number of convolutional kernels being 64, the size of the convolutional kernels being 3×1, and the stride being 1.

[0129] The seventh convolutional layer, with the number of convolutional kernels being 128, the size of the convolutional kernels being 1×1, and the stride being 1.

[0130] The shortcut connection of the residual network. Since the dimensions of the input and output signals of the second residual block are both 128, a direct connection is used.

[0131] Sum the output of the seventh convolutional layer and the direct connection, and then pass through the RELU function to obtain the output of the second residual block.

[0132] Input the output of the second residual block into a global average pooling layer to obtain a vector mapping. Since the output of the second residual block is 128-dimensional data, 128 data are obtained after passing through the global average pooling layer.

[0133] The first DropOut layer, with a DropOut probability of 0.5. For each neural network unit, it is temporarily discarded from the network with a certain probability to alleviate the degree of overfitting.

[0134] By using the connection of two residual blocks and the subsequent global average pooling layer as an encoder, encode the 30s EEG signal epoch. Compared with traditional methods, this network structure overcomes the drawback that the network model highly depends on hyperparameter tuning. After changing to another tester, the model needs to be retrained and retuned, otherwise good results cannot be obtained. However, our network structure can adapt to different testers without retraining and retuning. After training the model with the data of one tester, good results can also be achieved when replacing the data of other testers. Exactly for this reason, this sleep state detection system based on a portable EEG acquisition headband does not require the user to perform a training process during use.

[0135] S260. Use the decoder based on the multi-head attention mechanism in the EEG staging attention mechanism model to decode the vector mapping sequence; the decoder includes a multi-head attention sub-module, a first fully connected layer, and a second fully connected layer connected in sequence;

[0136] S270. Obtain the sleep stage staging result output by the EEG staging attention mechanism model.

[0137] In this embodiment, by adding a first DropOut layer after the global average pooling layer, each neural network unit can be temporarily discarded from the network with a certain probability to alleviate the overfitting degree and improve the signal processing efficiency.

[0138] Embodiment III

[0139] Figure 9 It is a schematic flowchart of the sleep state detection method based on an EEG acquisition headband provided in Embodiment III of the present invention; this embodiment is optimized based on the above-mentioned embodiment. In this embodiment, after summing and normalizing the output of the encoder and the positional encoding PE, and then inputting it into the multi-head attention mechanism sub-module of the decoder, the following steps are added: inputting the output of the multi-head attention mechanism sub-module into the third DropOut layer, and the DropOut probability of the third DropOut layer is 0.8, and the output of the multi-head attention mechanism sub-module is normalized through the processing of the third DropOut layer; among them, the number of heads in the multi-head attention mechanism sub-module is 8, and the size of each head is 64 dimensions, and the attention mechanism is executed in parallel for each head; the attention mechanism function is expressed as:

[0140]

[0141] where X is the matrix composed of the input sequence, and dim represents the data dimension; in the expression corresponds to the scaling layer in the multi-head attention mechanism sub-module; after inputting the output of the third DropOut layer into the first fully connected layer, the result is activated by the RELU function; inputting the output of the first fully connected layer into the second fully connected layer, and the output result of the second fully connected layer is the output of the decoder; among them, the normalized output of the multi-head attention mechanism is added to the outputs of the first fully connected layer and the second fully connected layer through skip connection.

[0142] Correspondingly, the sleep state detection method based on an EEG acquisition headband provided in this embodiment specifically includes:

[0143] S301. Obtain the EEG signals of the FP1 electrode and the FP2 electrode in the 10-20 international standard lead system;

[0144] S302. Process the EEG signals to obtain 30s of EEG epoch signals;

[0145] S303. Input the 30s of EEG epoch signals into the trained electroencephalogram staging attention mechanism model to obtain the approximate Hilbert transform of the 30s of EEG epoch signals, that is, the EEG signal epoch;

[0146] S304. Encode the EEG signal epoch using the encoder in the EEG stage attention mechanism model to obtain a vector mapping sequence. The encoder includes a first residual block, a second residual block, and a global average pooling layer connected in sequence.

[0147] S305. After summing and normalizing the output of the encoder with the positional encoding PE, input it into the multi-head attention mechanism sub-module of the decoder. The calculation method of the positional encoding PE is as follows:

[0148]

[0149] where p represents the position of the current EEG signal epoch in the input EEG signal epoch queue; d represents the number of EEG signal epochs included in the input queue input to the multi-head attention mechanism sub-module; dim represents the number of data output by the global average pooling layer.

[0150] Exemplarily, d is 30, indicating that the input queue input to the multi-head attention mechanism contains 30 EEG signal epochs, dim is 128, which is the number of data output by the above global average pooling layer. The position mapping of each epoch can be directly calculated by the above equation without training, thus accelerating the training speed and helping the algorithm learn the transition rules from two directions.

[0151] S306. Input the output of the multi-head attention mechanism sub-module into the third DropOut layer. The DropOut probability of the third DropOut layer is 0.8, and the output of the multi-head attention mechanism sub-module is normalized through the processing of the third DropOut layer. Among them, the number of heads in the multi-head attention mechanism sub-module is 8, the size of each head is 64 dimensions, and the attention mechanism is executed in parallel for each head. The attention mechanism function is expressed as:

[0152]

[0153] where X is the matrix composed of the input sequence, and dim represents the data dimension; the in the expression corresponds to the scaling layer in the multi-head attention mechanism sub-module; then the outputs of each head in the multi-head attention mechanism sub-module are connected and the concatenated result is projected from a higher dimension (8×64 = 512) to the same dimension 128 of the vector mapping originally input to the decoder using a fully connected layer.

[0154] S307. After inputting the output of the third DropOut layer into the first fully connected layer, activate the result through the RELU function. Among them, the first fully connected layer is a fully connected layer composed of 2048 neurons.

[0155] S308. Input the output of the first fully connected layer into the second fully connected layer, and the output result of the second fully connected layer is the output of the decoder. Among them, the second fully connected layer can project the activation result back to 128 dimensions; the normalized output of the multi-head attention mechanism is added to the outputs of the first fully connected layer and the second fully connected layer through skip connections.

[0156] S309. Use residual connection to sum and normalize the input and output of the decoder, and then input it into the second DropOut layer. Among them, the DropOut probability of the second DropOut layer is 0.5. By temporarily discarding each neural network unit from the network with a certain probability, the degree of overfitting can be alleviated. Among them, by connecting the encoder output with the output residual of the previous step, the performance can be improved and the problem of gradient disappearance in the backpropagation process can be avoided.

[0157] S310. Input the output of the second DropOut layer into the SoftMax layer, and the SoftMax layer outputs the sleep stage staging result.

[0158] S311. Obtain the sleep stage staging result output by the electroencephalogram staging attention mechanism model.

[0159] Figure 10 It is a schematic structural diagram of the decoder in the sleep state detection method based on an electroencephalogram acquisition headband provided in Embodiment III of the present invention. As Figure 10 shown, the above decoder includes a multi-head attention mechanism sub-module, a third DropOut layer, a first fully connected layer, a second fully connected layer, a second DropOut layer, and a SoftMax layer connected in sequence.

[0160] In this embodiment, by introducing the multi-head attention mechanism, cross-time analysis can be performed while parallel computing can be achieved and the training process can be shortened, so as to improve the detection efficiency of the user's sleep state.

[0161] Embodiment IV

[0162] Figure 11 It is a schematic structural diagram of the sleep state detection device based on an electroencephalogram acquisition headband provided in Embodiment IV of the present invention. As Figure 11 shown, the device includes:

[0163] An acquisition module 410, configured to acquire EEG brain electrical signals of FP1 electrode and FP2 electrode in the 10-20 international standard lead system.

[0164] A preprocessing module 420, configured to process the EEG brain electrical signals to obtain 30s of EEG epoch signals.

[0165] A transformation module 430, configured to input the 30s EEG epoch signal into the trained electroencephalogram staging attention mechanism model to obtain an approximate Hilbert transform of the 30s EEG epoch signal, that is, the electroencephalogram signal epoch;

[0166] An encoding module 440, configured to encode the electroencephalogram signal epoch by using an encoder in the electroencephalogram staging attention mechanism model to obtain a vector mapping sequence;

[0167] A decoding module 450, configured to decode the vector mapping sequence by using a decoder based on a multi-head attention mechanism in the electroencephalogram staging attention mechanism model;

[0168] An output module 460, configured to obtain the sleep stage staging result output by the electroencephalogram staging attention mechanism model.

[0169] The sleep state detection device based on an electroencephalogram acquisition headband provided in this embodiment acquires the EEG electroencephalogram signals of the FP1 electrode and the FP2 electrode in the 10-20 international standard lead system, processes the EEG electroencephalogram signals to obtain a 30s EEG epoch signal; inputs the 30s EEG epoch signal into the trained electroencephalogram staging attention mechanism model to obtain an approximate Hilbert transform of the 30s EEG epoch signal, that is, the electroencephalogram signal epoch; then encodes the electroencephalogram signal epoch by using an encoder in the electroencephalogram staging attention mechanism model to obtain a vector mapping sequence, and decodes the vector mapping sequence by using a decoder based on a multi-head attention mechanism in the electroencephalogram staging attention mechanism model, and finally can output the sleep stage staging result for the user to refer to.

[0170] Embodiment 5

[0171] Figure 12 It is a schematic structural diagram of the sleep state detection system based on an electroencephalogram acquisition headband provided in Embodiment 4 of the present invention, as Figure 12 shown, the system includes:

[0172] The sleep state detection device based on an electroencephalogram acquisition headband provided in the above embodiment;

[0173] An EEG signal acquisition headband and an EEG signal analysis device

[0174] Note that the above is only the preferred embodiment of the present invention and the technical principles applied. Those skilled in the art will understand that the present invention is not limited to the specific embodiments described herein, and various obvious changes, re-adjustments and substitutions can be made by those skilled in the art without departing from the protection scope of the present invention. Therefore, although the present invention has been described in more detail through the above embodiments, the present invention is not limited to the above embodiments. Without departing from the concept of the present invention, more other equivalent embodiments can be included, and the scope of the present invention is determined by the scope of the appended claims.

Claims

1. A sleep state detection method based on an EEG acquisition headband, characterized in that, it includes: Obtain EEG brain signals of FP1 electrode and FP2 electrode in the 10-20 international standard lead system; Process the EEG brain signals to obtain 30s EEG epoch signals; Input the 30s EEG epoch signals into the trained electroencephalogram staging attention mechanism model to obtain the approximate Hilbert transform of the 30s EEG epoch signals, that is, the EEG signal epoch; the method for the approximate Hilbert transform of the 30s EEG epoch signals includes: inputting the 30s EEG epoch signals into the first convolutional layer, the convolutional kernel size of the first convolutional layer Fs = N / 100, 64 convolutional kernels, and the moving step size Sd = N / 1000; where N is the number of sampling points included in each 30s EEG epoch signal, that is, N = 30×100 = 3000; input the output of the first convolutional layer into two max-pooling layers respectively; among them, the output size of the first max-pooling layer is N / 1000, and the output is the frequency characteristic of the 30s EEG epoch signal, and the output size of the second max-pooling layer is π×N / 1000, and the output is the amplitude characteristic of the input 30s EEG epoch signal; sum the outputs of the two max-pooling layers point by point to obtain the approximate Hilbert transform of the 30s EEG epoch signals; where, denote the input 30s EEG epoch signal as s(t), then the Hilbert transform is: s a (t) = s(t) + i * H(s(t)) where the real part contains frequency information and the imaginary part contains amplitude information; * denotes the convolution operation; Use the encoder in the electroencephalogram staging attention mechanism model to encode the EEG signal epoch to obtain a vector mapping sequence; the encoder includes a first residual block, a second residual block, and a global average pooling layer connected in sequence; Use the decoder based on the multi-head attention mechanism in the electroencephalogram staging attention mechanism model to decode the vector mapping sequence; the decoder includes a multi-head attention sub-module, a first fully connected layer, and a second fully connected layer connected in sequence; Obtain the sleep stage staging result output by the electroencephalogram staging attention mechanism model.

2. The method according to claim 1, characterized in that, the processing of the EEG brain signals to obtain 30s EEG epoch signals includes: Process the EEG brain signals using a sixth-order Butterworth band-pass filter with a cut-off frequency of 0.1-50Hz; Downsample the processed EEG brain signals, from 500Hz to 100Hz, and then divide the complete EEG brain signals into time series segments of 30s each to obtain several 30s EEG epoch signals.

3. The method according to claim 1, characterized in that, the first residual block in the encoder includes: A second convolutional layer, the number of convolutional kernels of the second convolutional layer is 128, the convolutional kernel size is 1×1, and the step size is 2; The third convolutional layer, the number of convolutional kernels in the third convolutional layer is 64, the size of the convolutional kernel is 3×1, and the stride is 1; The fourth convolutional layer, the number of convolutional kernels in the fourth convolutional layer is 128, the size of the convolutional kernel is 1×1, and the stride is 1; Among them, the shortcut connection of the first residual block residual network uses an average pooling layer, the size of the average pooling layer is 2, and the stride is 2 to convert the 64-dimensional input into a 128-dimensional output; the output of the fourth convolutional layer and the output of the average pooling layer are summed, and then after passing through the RELU function, the output of the first residual block is obtained.

4. The method according to claim 1, characterized in that, the second residual block in the encoder includes: The fifth convolutional layer, the number of convolutional kernels in the fifth convolutional layer is 128, the size of the convolutional kernel is 1×1, and the stride is 2; The sixth convolutional layer, the number of convolutional kernels in the sixth convolutional layer is 64, the size of the convolutional kernel is 3×1, and the stride is 1; The seventh convolutional layer, the number of convolutional kernels in the seventh convolutional layer is 128, the size of the convolutional kernel is 1×1, and the stride is 1; Among them, the shortcut connection of the second residual block residual network uses a direct connection; the output of the seventh convolutional layer and the direct connection are summed, and then after passing through the RELU function, the output of the second residual block is obtained.

5. The method according to claim 1, characterized in that, After encoding the EEG signal epoch using the encoder in the EEG staging attention mechanism model to obtain a vector mapping sequence, the method further includes: Inputting the vector mapping sequence output by the global average pooling layer into the first DropOut layer to obtain the output of the encoder; among them, the DropOut probability of the first DropOut layer is 0.

5.

6. The method according to claim 1, characterized in that, The decoding of the vector mapping sequence using the decoder based on the multi-head attention mechanism in the EEG staging attention mechanism model includes: After summing and normalizing the output of the encoder and the positional encoding PE, it is then input into the multi-head attention mechanism sub-module of the decoder; among them, the calculation method of the positional encoding PE is as follows: where p represents the position of the current EEG signal epoch in the input EEG signal epoch queue; d represents the number of EEG signal epochs included in the input queue input into the multi-head attention mechanism sub-module; dim represents the number of data output by the global average pooling layer; Using residual connection to sum and normalize the input and output of the decoder and then input it into the second DropOut layer; among them, the DropOut probability of the second DropOut layer is 0.5; Inputting the output of the second DropOut layer into the SoftMax layer, and the SoftMax layer outputs the sleep stage staging result.

7. The method according to claim 6, characterized in that, After summing and normalizing the output of the encoder and the positional encoding PE and then inputting it into the multi-head attention mechanism sub-module of the decoder, the method further includes: The output of the multi-head attention mechanism sub-module is input into the third DropOut layer with a DropOut probability of 0.

8. The processing by the third DropOut layer normalizes the output of the multi-head attention mechanism sub-module. Among them, the number of heads in the multi-head attention mechanism sub-module is 8, the size of each head is 64 dimensions, and the attention mechanism is executed in parallel for each head. The attention mechanism function is expressed as: where X is a matrix composed of input sequences, and dim represents the data dimension; in the expression, corresponds to the scaling layer in the multi-head attention mechanism sub-module; After the output of the third DropOut layer is input into the first fully-connected layer, the result is then activated by the RELU function. The output of the first fully-connected layer is input into the second fully-connected layer, and the output result of the second fully-connected layer is the output of the decoder. Among them, the normalized output of the multi-head attention mechanism is added to the outputs of the first fully-connected layer and the second fully-connected layer through skip connections.

8. A sleep state detection device based on an EEG acquisition headband Characterized in that It includes: An acquisition module for acquiring EEG brain signals of FP1 electrode and FP2 electrode in the 10-20 international standard lead system. A preprocessing module for processing the EEG brain signals to obtain 30s EEG epoch signals. A transformation module for inputting the 30s EEG epoch signals into the trained electroencephalogram staging attention mechanism model to obtain the approximate Hilbert transform of the 30s EEG epoch signals, that is, the EEG signal epoch. The method for the approximate Hilbert transform of the 30s EEG epoch signals includes: inputting the 30s EEG epoch signals into the first convolutional layer, where the convolution kernel size of the first convolutional layer is Fs = N / 100, with 64 convolutional kernels and a moving step size of Sd = N / 1000; where N is the number of sampling points included in each 30s EEG epoch signal, that is, N = 30×100 = 3000; the output of the first convolutional layer is respectively input into two max-pooling layers; among them, the size of the output of the first max-pooling layer is N / 1000, and the output is the frequency feature of the 30s EEG epoch signal, and the size of the output of the second max-pooling layer is π×N / 1000, and the output is the amplitude feature of the input 30s EEG epoch signal; the outputs of the two max-pooling layers are summed point by point to obtain the approximate Hilbert transform of the 30s EEG epoch signals; where, denoting the input 30s EEG epoch signal as s(t), the Hilbert transform is: s a (t) = s(t) + i * H(s(t)) Among them, the real part contains frequency information and the imaginary part contains amplitude information. * indicates a convolution operation; An encoding module for encoding the EEG signal epoch using the encoder in the electroencephalogram staging attention mechanism model to obtain a vector mapping sequence. The encoder includes a first residual block, a second residual block, and a global average pooling layer connected in sequence. A decoding module, which is used to decode the vector mapping sequence by using a decoder based on a multi-head attention mechanism in the electroencephalogram staging attention mechanism model; the decoder includes a multi-head attention sub-module, a first fully connected layer, and a second fully connected layer connected in sequence; An output module, which is used to obtain the sleep stage staging result output by the electroencephalogram staging attention mechanism model.

9. A sleep state detection system based on an electroencephalogram acquisition headband, characterized in that it includes: the sleep state detection device based on an electroencephalogram acquisition headband according to claim 8; an EEG signal acquisition headband and an EEG signal analysis device.

Images