A single-channel electroencephalogram drowsiness detection method based on a lightweight neural network

Abstract

This invention relates to the field of signal analysis technology, and more particularly to a single-channel EEG sleepiness detection method based on a lightweight neural network. The method includes preprocessing the single-channel EEG signal, performing wavelet packet decomposition on the preprocessed EEG signal to convert it into a time-frequency matrix, generating a sleepiness detection dataset; constructing a new lightweight neural network based on the F-MBConv and MBConv modules in the EfficientNetv2 network, and adding an improved Ghost module between the two modules to achieve automatic extraction of EEG features; replacing the SE module in the original network with an attention mechanism ECA module to further reduce network weight; setting the model's training parameters, using cross-entropy as the loss function, and obtaining the optimal recognition model through training. This invention improves the recognition performance of subject sleepiness detection with lower complexity and computational cost by constructing a lightweight deep neural network.

Description

Technical Field

[0001] This invention relates to the field of signal analysis technology, and in particular to a single-channel EEG sleepiness detection method based on a lightweight neural network. Background Technology

[0002] Studies have shown that 10% to 30% of fatal traffic accidents can be attributed to driver fatigue. Because electroencephalogram (EEG) signals contain rich electrical information about human physiological activity, they are widely recognized as the most objective basis for fatigue detection. Researching the characteristics of EEG signals in awake and drowsy states can lay the foundation for the development of fatigue early warning devices and has significant research value.

[0003] To identify whether a driver is drowsy using electroencephalogram (EEG) signals, signal processing and feature representation of the EEG are first required, followed by modeling to identify the driver's drowsy state. Feature representation is crucial for EEG-based drowsiness detection, and commonly used methods include frequency domain methods, time-frequency analysis, and brain network-based analysis. However, traditional feature extraction methods require manual selection of appropriate features for the classifier, and feature selection depends on the skill level of the professional designer. Furthermore, individual differences among subjects also limit the effectiveness of drowsiness detection.

[0004] As a method of deep learning, convolutional neural networks simulate biological neural networks through weight sharing, greatly reducing the number of weights and complexity of the network model, providing better representation of features, and avoiding the manual feature selection process. However, existing models such as MobileNetv3, EfficientNetv2, and GhostNet still suffer from problems such as excessively large number of parameters and limited practical applications. Summary of the Invention

[0005] To address the shortcomings of existing algorithms, this invention significantly improves the performance of subject drowsiness detection by constructing a lightweight deep neural network.

[0006] The technical solution adopted in this invention is: a single-channel EEG sleepiness detection method based on a lightweight neural network, comprising the following steps:

[0007] Step 1: Preprocess the single-channel EEG signal;

[0008] Step 2: Perform wavelet packet decomposition on the preprocessed EEG signal to generate a time-frequency matrix, thereby obtaining the sleep detection dataset;

[0009] Furthermore, the wavelet packet decomposition uses the db10 wavelet.

[0010] Step 3: Construct a new lightweight neural network;

[0011] Furthermore, the new lightweight neural network includes: a first convolutional layer followed by two F-MBConv modules; a second F-MBConv module followed by two Ghost modules; a second Ghost module followed by two MBConv modules; a second MBConv module followed by a global average pooling layer, then a fully connected layer, and finally fed into a softmax classifier.

[0012] Furthermore, the first F-MBConv module's main branch uses a regular 3x3 convolutional kernel in its first convolutional layer, with 64 kernels and a stride of 1. This is followed by a batch normalization layer and a Swish activation function. A second 1x1 convolutional kernel is used, and subsequent layers only use batch normalization. Finally, a shortcut branch is used to add the output to the input, forming a residual structure to avoid gradient explosion or vanishing.

[0013] Furthermore, the first convolutional layer of the second F-MBConv module uses a regular convolutional kernel with a size of 3*3, a number of kernels of 64, and a stride of 2; followed by a batch normalization layer and a Swish activation function; then a second 1*1 convolutional kernel is used to adjust the number of output channels to 32, with no shortcut branches;

[0014] The formula for a standard convolution operation is as follows:

[0015]

[0016] Among them, F i (l) This represents the i-th feature map of the l-th layer and the feature map of the (l+1)-th layer. It can be obtained through the i-th feature map F of the l-th layer i (l) After performing convolution summation and adding the bias, we get b. j It is the bias term, K ij This represents the j-th convolution kernel in the i-th channel, "*" indicates the convolution operation, and σ is the activation function.

[0017] Furthermore, the first submodule of the main branch of the first Ghost module uses a regular convolutional kernel with a size of 1*1, 32 kernels, and a stride of 1; followed by a batch normalization layer and the Swish activation function; the second convolutional layer uses a depthwise separable convolutional layer with a size of 3*3; the output of the first submodule is obtained by concatenating the outputs of the two convolutional layers; using a batch normalization layer and the Swish activation function;

[0018] Then the ECA module was improved;

[0019] The expression for the depthwise separable convolution used is shown in Equation (2), and the expression for the pointwise convolution is shown in Equation (3):

[0020]

[0021]

[0022] Where W is the convolution kernel, y is the input feature map, i and j are the input feature map resolutions, k and l are the output feature map resolutions, and m is the number of channels. This indicates element-wise multiplication;

[0023] The number of convolution kernels in the pointwise convolution of the second submodule is halved compared to the number in the first module, resulting in 16 convolution kernels; the output channels are 32.

[0024] Use a shortcut branch to add the output of the second submodule to the original input.

[0025] Furthermore, the first submodule of the second Ghost module has 64 kernels in its pointwise convolutional layer, resulting in 128 concatenated feature map channels. A 3x3 depthwise separable convolutional layer with a stride of 2 is then added. The second submodule's pointwise convolutional layer has 32 kernels and 64 output channels. A 3x3 depthwise separable convolutional kernel with a stride of 2 and 64 pointwise convolutions are used on the shortcut branch to facilitate the formation of residual structures and avoid gradient explosion or vanishing. The Ghost module uses low-cost linear mapping operations to generate half of the features, effectively reducing the number of model parameters and computational cost.

[0026] Furthermore, the improved ECA module uses a one-dimensional convolutional layer with a kernel size of 3 to extract channel weights.

[0027] Furthermore, the first MBConv module first uses pointwise convolution to increase the channel dimension to 256 to avoid losing effective information. Then, it uses a 3*3 depthwise separable convolutional layer with a stride of 2 to halve the height and width of the feature map. Both convolutional layers use batch normalization layers and the Swish activation function. Then, an improved ECA module is used to extract channel weights. Finally, pointwise convolution is used to obtain an output feature map with a channel dimension of 96.

[0028] Furthermore, the second MBConv module is structurally similar to the first module, except that the first pointwise convolution has 384 channel outputs and the second pointwise convolution has 128 channel outputs. The two MBConv modules can enhance the network's ability to capture spatial information while reducing the number of network parameters.

[0029] The fully connected layer consists of 2 neurons, which is the final classification number.

[0030] Step 4: Set the training parameters of the model, use cross-entropy as the loss function, minimize the loss function through training, and compare the similarity between the predicted softmax output probability distribution and the target class probability distribution.

[0031] The beneficial effects of this invention are:

[0032] 1. Construct a lightweight network model that simulates biological neural networks through weight sharing in convolutional neural networks. This model has a better ability to express features and greatly reduces the number and complexity of network model parameters, thereby enhancing its practical application value.

[0033] 2. Integrating the ECA mechanism into the model not only provides global channel information but also has a lower computational cost compared to the SE module;

[0034] 3. Using publicly available EEG datasets, our proposed method achieved an accuracy of 96.5 ± 0.5% in sleep detection, which is superior to the control model method. Attached Figure Description

[0035] Figure 1 This is a flowchart of the single-channel EEG sleepiness detection method based on a lightweight neural network according to the present invention;

[0036] Figure 2 This is a diagram of the lightweight network structure of the present invention;

[0037] Figure 3 This is the first F-MBConv module diagram of the present invention;

[0038] Figure 4 This is the second F-MBConv module diagram of the present invention;

[0039] Figure 5 This is the first Ghost module diagram of the present invention;

[0040] Figure 6 This is a schematic diagram illustrating the working principle of the depth-separable convolutional layer and the point-by-point convolutional layer of the present invention;

[0041] Figure 7 This is a diagram of the ECA module of the present invention;

[0042] Figure 8 This is the second Ghost module diagram of the present invention;

[0043] Figure 9 This is the first MBConv module diagram;

[0044] Figure 10 This is the diagram for the second MBConv module. Detailed Implementation

[0045] The present invention will be further described below with reference to the accompanying drawings and embodiments. The drawings are simplified schematic diagrams, which only illustrate the basic structure of the present invention in a schematic manner, and therefore only show the components related to the present invention.

[0046] like Figure 1 As shown, a single-channel EEG sleepiness detection method based on a lightweight neural network includes the following steps:

[0047] Step 1: Preprocess the single-channel EEG signal;

[0048] Using Pz-Oz channel EEG data from 50 subjects in the Sleep-EDF dataset, a sleep detection experiment was conducted using the proposed method during the awake and sleep phases of the collected EEG data, while maintaining an equal sample size for both phases.

[0049] Step 2: Perform wavelet packet decomposition on the preprocessed EEG signals to convert them into a time-frequency matrix and generate a sleep detection dataset;

[0050] The wavelet packet decomposition method uses the db10 wavelet from the Daubechies wavelet basis series, decomposing it into 6 layers to obtain a time-frequency matrix. Similar to the storage format of grayscale images, the size of the obtained matrix is ​​adjusted to 64*64 to facilitate convolution operations.

[0051] The dataset, consisting of 50 individuals aged 20 to 60, was divided into training, validation, and test sets according to a sample size ratio of 8:1:1.

[0052] Step 3: Construct a new lightweight neural network;

[0053] Based on the F-MBConv and MBConv modules in the EfficientNetv2 network, and with the addition of an improved Ghost module between the two modules, a new lightweight neural network is constructed to achieve automatic extraction of EEG features. Furthermore, the attention mechanism ECA module is used to replace the SE module in the original network, further reducing the network's weight.

[0054] New lightweight neural network architectures such as Figure 2 As shown, the first convolutional layer is followed by two F-MBConv modules, the second F-MBConv module is followed by two Ghost modules, the second Ghost module is followed by two MBConv modules, the second MBConv module is followed by a global average pooling layer, followed by a fully connected layer, and finally fed into a softmax classifier to determine whether the subject is drowsy.

[0055] The overall network structure greatly reduces the number and complexity of network model parameters, and has a better ability to express features.

[0056] The first convolutional layer is a regular convolutional layer with a kernel size of 3*3 and 16 kernels. Batch normalization layers are used to improve model training speed and network generalization performance. A Swish activation function with non-linear factors is added to improve the neural network's expressive power over the model.

[0057] The formula for a standard convolution operation is as follows:

[0058]

[0059] Where F i (l) This represents the i-th feature map of the l-th layer and the feature map of the (l+1)-th layer. It can be obtained through the i-th feature map F of the l-th layer i (l) After performing convolution summation and adding the bias, we get b. j It is the bias term, K ij This represents the j-th convolution kernel in the i-th channel. The "*" in the formula represents the convolution operation, and σ is the activation function.

[0060] Because the number of features extracted is relatively small, the first three shallow network modules of this invention use ordinary convolution and the residual structure composed of them, which can utilize existing hardware devices to accelerate computation and reduce training time.

[0061] like Figure 3 As shown, in the first F-MBConv module, the first convolutional layer of the main branch uses a regular convolutional kernel with a size of 3*3, a number of kernels of 64, and a stride of 1. This is followed by a batch normalization layer and a Swish activation function. Immediately afterward, a second 1*1 convolutional kernel is used to adjust the number of output channels to be equal to the number of input channels. Only a batch normalization layer is used afterward. Finally, a shortcut branch is used to add the output result to the input, which is then used as the input for the next module.

[0062] like Figure 4 As shown, in the second F-MBConv module, the first convolutional layer uses a regular convolutional kernel with a size of 3*3, a number of kernels of 64, and a stride of 2. This is followed by a batch normalization layer and a Swish activation function. Then, a second 1*1 convolutional kernel is used to adjust the number of output channels to 32. There is no shortcut branch.

[0063] like Figure 5As shown, this is the first Ghost module. The first convolutional layer in the main branch of the Ghost module uses a 1*1 kernel with 32 kernels and a stride of 1. This is followed by a batch normalization layer and the Swish activation function. Then, a second 3*3 deep separable convolutional layer is used to obtain the same number and size of feature maps, which are then concatenated with the output of the first convolutional layer to form a feature map with a channel depth of 64. This is then improved through the ECA module.

[0064] like Figure 6 As shown, the expression for a depthwise separable convolutional layer is given by equation (2), and the expression for a pointwise convolution is given by equation (3).

[0065]

[0066]

[0067] Where W is the convolution kernel, y is the input feature map, i and j are the input feature map resolutions, k and l are the output feature map resolutions, and m is the number of channels. This indicates that corresponding elements are multiplied.

[0068] The second submodule is similar in structure to the first submodule, except that the number of convolution kernels is changed from 32 to 16, and the final output channel is 32. Finally, a shortcut branch is used to add the output to the original input of the module, which is then used as the input of the next module.

[0069] like Figure 7 As shown, to improve the ECA module, the original Ghost module's approach of using two fully connected layers to first reduce dimensionality and then increase it to extract weights from each channel's SE module is abandoned. Instead, only a single one-dimensional convolutional layer with a kernel size of 3 is used to extract channel weights. While avoiding information loss due to reducing channel dimensionality, the improved ECA module obtains cross-channel interaction information in an extremely lightweight manner, significantly reducing model complexity while maintaining good performance.

[0070] like Figure 8 As shown, the main difference between the second Ghost module and the first is that the first submodule has 64 1x1 convolutional kernels and 128 concatenated feature map channels; a 3x3 depthwise separable convolutional layer with a stride of 2 is added after the first submodule to halve the height and width of the feature map; the second submodule has 32 convolutional kernels and 64 output channels; on the shortcut branch, a 3x3 depthwise separable convolutional kernel with a stride of 2 and 64 1x1 convolutions are used to obtain a feature map with the same format as the main branch, and the two results are added together as the input to the next module. The Ghost module first uses pointwise convolution to obtain half of the features, and then uses linear transformation to obtain the remaining features, reducing the number of network model parameters and floating-point operations (FLOPs), thus achieving a lightweight model.

[0071] like Figure 9 The first MBConv module is shown. Its structure is as follows: First, a 1*1 convolutional kernel is used to enlarge the input from 64 channels to 256 channels, followed by a batch normalization layer and a Swish activation function. Then, a 3*3 depthwise separable convolution with a stride of 2 is used to halve the height and width of the feature map, followed by another batch normalization layer and a Swish activation function. Next, an improved ECA module is used, which uses a 3-dimensional convolutional layer to extract channel weights. Finally, a 1*1 dot product is used to obtain a 96-channel output feature map.

[0072] like Figure 10 The second MBConv module is shown. It has a similar structure to the first module, but the difference is that the first 1*1 convolutional kernel has 384 channel outputs, while the second 1*1 convolutional kernel has 128 channel outputs.

[0073] The fully connected layer consists of 2 neurons, which is the final classification number.

[0074] Step 4: Set the training parameters for the model;

[0075] Convolutional neural networks are trained to minimize the loss function L(W, b). The model uses cross-entropy as the loss function and compares the similarity between the predicted softmax output probability distribution and the target class probability distribution.

[0076] To minimize the loss function, this paper uses the Adam optimizer with a learning rate of 0.00001 and all other parameters using default values.

[0077] Deep learning networks are trained using EEG data training sets.

[0078] The batch size was set to 256 based on the computer's RAM size, and the training period was set to 100. Training was performed using forward propagation and backpropagation with minimized loss. The optimal recognition model weight parameters were obtained based on the accuracy of the validation set.

[0079] The test set is fed into the trained recognition model to detect drowsiness in the test subjects, providing a methodological basis for driver fatigue.

[0080] To demonstrate the superiority of this algorithm, it is compared with other algorithms using the same dataset, as shown in Table 1:

[0081] Table 1 Comparison of this invention with other algorithms

[0082] Algorithm Model Accuracy / % MobileNetv3-large 90.08 MobileNetv3-small 93.92 EfficientNetv2-s 92.00 GhostNet 92.86 ResNet-50 93.92 This invention 96.5

[0083] Based on the above-described preferred embodiments of the present invention, and through the foregoing description, those skilled in the art can make various changes and modifications without departing from the inventive concept. The technical scope of this invention is not limited to the contents of the specification, but must be determined according to the scope of the claims.

Claims

1. A single-channel EEG sleepiness detection method based on a lightweight neural network, characterized in that, Includes the following steps: Step 1: Preprocess the single-channel EEG signal; The EEG signals are from the waking and drowsy periods; Step 2: Perform wavelet packet decomposition on the preprocessed EEG signals to convert them into a time-frequency matrix and generate a sleep detection dataset; Step 3: Construct a lightweight neural network; The lightweight neural network consists of: a first convolutional layer followed by the first F-MBConv module and the second F-MBConv module; then the first Ghost module and the second Ghost module; then the first MBConv module and the second MBConv module followed by a global average pooling layer, then a fully connected layer, and finally fed into a softmax classifier. The first F-MBConv module's main branch uses a regular 3x3 kernel in its first convolutional layer with 64 kernels and a stride of 1; it also uses a batch normalization layer and the Swish activation function. The second convolutional layer uses a 1x1 kernel with a stride of 1; subsequent layers use only batch normalization layers; finally, a shortcut branch is used to add the output to the input. The first convolutional layer of the second F-MBConv module uses a regular convolutional kernel with a size of 3*3, 64 kernels, and a stride of 2; followed by a batch normalization layer and a Swish activation function; the second convolutional layer uses a 1*1 kernel to adjust the number of output channels to 32, with no shortcut branches; The first submodule of the first Ghost module's main branch uses pointwise convolution in its first convolutional layer, with the number of channels in the kernel halved based on the submodule's output channels. The second convolutional layer uses a 3x3 depthwise separable convolutional layer. The output of the first submodule is obtained by concatenating the outputs of the two convolutional layers. Batch normalization layers and the Swish activation function are used. Then, an improved ECA module is applied. The number of kernels in the second submodule is reduced from 32 to 16, and the output channels are 32. Finally, a shortcut branch is used to add the output to the module's original input, which is then used as the input to the next module. The main branch of the second Ghost module adds a 3*3 depthwise separable convolutional layer with a stride of 2 before the improved ECA module to halve the height and width of the feature map and reduce the amount of computation; on the shortcut branch, a 3*3 depthwise separable convolutional layer with a stride of 2 and a pointwise convolutional layer are used, both of which use batch normalization layers. The improved ECA module uses a one-dimensional convolutional layer with a kernel size of 3 to extract channel weights; Step 4: Set the training parameters of the model, minimize the loss function through training, use cross-entropy as the loss function, and compare the similarity between the predicted softmax output probability distribution and the target class probability distribution.

2. The single-channel EEG sleepiness detection method based on a lightweight neural network according to claim 1, characterized in that, Wavelet packet decomposition uses the db10 wavelet.

3. The single-channel EEG sleepiness detection method based on a lightweight neural network according to claim 1, characterized in that: The first MBconv module uses a pointwise convolutional layer in the first layer to increase the channel dimension. Then, it uses a 3*3 depthwise separable convolution with a stride of 2 to halve the height and width of the feature map. Both convolutional layers use batch normalization layers and the Swish activation function. Then, the channel weights are extracted by an improved ECA module with a channel attention mechanism. Finally, pointwise convolution is used to obtain the output feature map.

4. The single-channel EEG sleepiness detection method based on a lightweight neural network according to claim 1, characterized in that: The first pointwise convolution of the second MBConv module has a channel output of 384, and the second pointwise convolution has a channel output of 128.

Images