Electroencephalogram attention classification model and method and computer system

Abstract

The invention discloses an electroencephalogram attention classification model, an electroencephalogram attention classification method and a computer system, and solves the problems of low electroencephalogram signal recognition accuracy, insufficient feature extraction and difficulty in processing complex electroencephalogram signals in the prior art. The model comprises a multi-scale separable convolutional network which performs convolution operation on electroencephalogram signals of each input channel and extracts a low-level feature map; the Squeeze-and-Excitation module is used for carrying out inter-channel dynamic adjustment on the low-level feature map so as to generate a re-weighted feature map, and the Squeeze-and-Excitation module is used for carrying out inter-channel dynamic adjustment on the low-level feature map; the lightweight Transform is used for calculating attention weights between time steps corresponding to feature vectors in the re-weighted feature map from a time dimension based on an attention mechanism so as to capture a dependency relationship between the time steps; the MobileNet is used for extracting an advanced feature map from the feature map in which the dependency relationship between the time steps is determined; and the classification module maps the feature vectors in the advanced feature map to vectors of the category number through a full connection layer, and calculates the category probability through a Softmax function to obtain an attention classification result.

Description

Technical Field

[0001] The present invention relates to the technical field of electroencephalogram (EEG) signal processing, and in particular to an EEG attention classification model, method and computer system based on multi-scale feature extraction. Background Art

[0002] Electroencephalography (EEG) is a non-invasive technique that records brain electrical activity by placing electrodes on the scalp. EEG signals have high temporal resolution and are widely used in neuroscience research, brain-computer interface (BCI), medical diagnosis and other fields. Especially in attention analysis, EEG signals can reflect the dynamic changes of the brain under different attention states. Therefore, developing an efficient and accurate EEG signal processing and classification method is of great significance to improve the accuracy of attention state recognition.

[0003] Traditional EEG signal analysis methods mainly rely on manual feature extraction and classical machine learning algorithms. These methods usually include time domain, frequency domain and time-frequency domain feature extraction, such as power spectrum density, waveform amplitude, band energy, etc. However, manual feature extraction relies on expert knowledge, and the processing process is cumbersome and difficult to adapt to complex EEG data. With the development of deep learning technology, EEG signal classification methods based on convolutional neural networks (CNN) and recurrent neural networks (RNN) have gradually become a research hotspot. These methods can automatically extract effective features from data and avoid the limitations of manual feature extraction.

[0004] At present, many studies use deep learning models to process EEG signals, but these models still face some challenges. First, multi-scale feature extraction of EEG signals is a difficult point, because information in different frequency bands has different importance for the discrimination of attention state. Second, deep neural networks are prone to gradient vanishing problems when processing EEG signals, resulting in unstable training process. Third, the key features in EEG signals may be distributed on different channels, and how to effectively weight the features of each channel is a challenge. Finally, EEG signals have strong time dependence, and the model needs to be able to capture long-distance time dependencies. Summary of the invention

[0005] The purpose of the present invention is to propose an EEG attention classification model, method and computer system based on multi-scale feature extraction, aiming to solve the problems of low recognition accuracy, insufficient feature extraction and difficulty in processing complex EEG signals in the prior art, thereby achieving efficient and accurate classification of attention states.

[0006] According to a first aspect of the embodiment, an EEG attention classification model is proposed, comprising: a multi-scale separable convolutional network, which is configured to perform independent convolution operations on the EEG signal of each input channel to extract low-level feature maps; a Squeeze-and-Excitation module, which is configured to dynamically adjust the low-level feature maps between channels to generate re-weighted feature maps; a lightweight Transformer, which is configured to calculate the attention weights between each time step corresponding to each feature vector in the re-weighted feature map from the time dimension based on the attention mechanism to capture the dependency between time steps; a MobileNet, which is configured to extract a high-level feature map from the feature map that determines the dependency between time steps; and a classification module, which is configured to map the feature vector in the high-level feature map to a vector of the number of categories through a fully connected layer, and then calculate the category probability through a Softmax function to obtain an attention classification result.

[0007] In some examples, a multi-scale separable convolutional network includes a small-scale convolution kernel for capturing short-term features of EEG signals and a large-scale convolution kernel for capturing long-term features of EEG signals. The results of the convolution of the small-scale convolution kernel and the large-scale convolution kernel are batch normalized respectively, and then the normalized outputs are ReLU activated. The outputs after ReLU activation are combined from all channels, and a 1x1 point-by-point convolution operation is performed to fuse information from different channels. Then, the results of the 1x1 point-by-point convolution operation are also batch normalized and ReLU activated to obtain output feature maps of the small-scale convolution kernel and the large-scale convolution kernel. The output feature maps of the small-scale convolution kernel and the large-scale convolution kernel are concatenated to form a multi-scale feature representation, and the multi-scale features are further fused through parallel convolution operations. The results of the parallel convolution operations are then processed by batch normalization and ReLU activation functions to obtain low-level feature maps.

[0008] In some examples, the Squeeze-and-Excitation module performs global average pooling on each channel of the low-level feature map to generate a global feature description for each channel, maps the global feature description vector to a low-dimensional space using a fully connected layer, and then maps it back to the high-dimensional space to generate attention weights, and then multiplies the attention weights by the low-level feature map element-by-channel to generate a reweighted feature map.

[0009] In some examples, the lightweight Transformer linearly transforms the feature vector of the input reweighted feature map, maps it to a specific dimension, adds position information to each time step, calculates the attention weights between each time with added position information, and processes the feature vector corresponding to each time step through a two-layer fully connected network as follows: FFN(x)=ReLU(xW3+b1)W4+b2; where x is the feature vector corresponding to each time step, W3 and W4 are the weight matrices of the fully connected layer, and b1 and b2 are the bias vectors of the fully connected layer.

[0010] In some examples, multiple depthwise separable convolution blocks are used to perform convolution operations on each input channel of the feature map whose dependencies between time steps have been determined by the lightweight Transformer layer by layer to extract the high-level feature map, and the convolution output of the last layer of the depthwise separable convolution block is globally average pooled to obtain the feature vector of the high-level feature map.

[0011] According to the second aspect of the embodiment, an EEG attention classification method is proposed, including: collecting the EEG signal of the tester, inputting the EEG signal into the EEG attention classification model, and the EEG attention classification model outputs the attention classification result.

[0012] According to the third aspect of the embodiment, a computer system is proposed, comprising: a processor; a memory, comprising one or more computer program modules; wherein the one or more computer program modules are stored in the memory and configured to be executed by the processor, and the one or more computer program modules include instructions for implementing the EEG attention classification method.

[0013] According to a fourth aspect of the embodiment, a computer-readable storage medium is proposed for storing non-temporary computer-readable instructions, which can implement the EEG attention classification method when executed by a computer.

[0014] The beneficial effects of the present invention are: the present invention can extract rich feature information from EEG signals, capture features at different time scales and different spatial scales, and improve the accuracy of the classification model. The present invention combines low-level feature extraction, channel attention mechanism and high-level feature extraction, and can extract the features of EEG signals from multiple angles and multiple levels. In particular, the channel attention mechanism helps to highlight the features of key channels and enhance the feature representation capability. The EEG attention classification model / method of the present invention has a reasonable structure, and the modular design facilitates expansion and adjustment in practical applications, and can flexibly respond to different application scenarios and needs. The present invention applies deep learning to EEG signals, extracts and classifies EEG signals through artificial intelligence algorithms, further improves the accuracy of classification, and is easy to promote and apply to daily mental health monitoring and attention status assessment. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Figure 1 4 is a flowchart of an EEG attention classification method based on multi-scale feature extraction according to an embodiment of the present invention.

[0016] Figure 2 4 is an electrode distribution diagram for collecting EEG signals according to an embodiment of the present invention.

[0017] Figure 3 4 is a structural block diagram of an EEG attention classification model according to an embodiment of the present invention.

[0018] Figure 4 is a structural diagram of a multi-scale separable convolutional network according to an embodiment of the present invention.

[0019] Figure 5 is a structural diagram of MobileNet according to an embodiment of the present invention. DETAILED DESCRIPTION

[0020] Figure 1 A flowchart of an EEG attention classification method based on multi-scale feature extraction is shown. The method is described in detail below.

[0021] Step 1: Obtain the EEG signals of the subjects under different attention states.

[0022] The EEG signal of the subject is collected using an EEG acquisition device. The EEG acquisition device is a prior art and will not be introduced in the present invention. In order to obtain the best signal quality, the following parameters are set: sampling rate 1000 Hz, 32 channels, specifically 32 channels in the frontal lobe, parietal lobe, central lobe, occipital lobe and temporal lobe areas of the brain, including Fz, Fp1, Fp2, F3, F4, F7, F8, FC1, FC2, FC5, FC6, FT9, FT10, Cz, C3, C4, T7, T8, CP1, CP2, CP5, CP6, TP9, TP10, Pz, P3, P4, P7, P8, Oz, O1, O2, with Fz as the reference electrode and the ground electrode Gnd placed on the subject's forehead. Figure 2 These parameters can be adjusted by those skilled in the art according to actual applications.

[0023] Step 2: Preprocess the collected EEG signals to obtain preprocessed EEG data.

[0024] Step 21: Use a 0.5-30 Hz bandpass filter to filter the original EEG signal to remove some signals in the unused frequency range, as well as high-frequency EMG signals and low-frequency EKG signals, skin electroencephalogram signals and respiratory belt signals.

[0025] Step 22: Use independent component analysis (ICA) to remove the electrooculogram signal from the original signal.

[0026] The filtered EEG signal X filtered Decompose the mixed signal into independent components. The calculation formula is as follows:

[0027] S=WX filtered ;

[0028] Where S is the independent component matrix and W is the inverse matrix of the mixing matrix A found by the ICA algorithm.

[0029] Then the electrooculogram signal components are identified and removed from the independent component matrix S. The calculation formula is as follows:

[0030] S EEG =SS EOG ;

[0031] Where S EOG is the electrooculogram signal component, S EEG It is the remaining EEG component.

[0032] Finally, use the remaining independent components S EEG And the mixing matrix A reconstructs the EEG signal, and the calculation formula is as follows:

[0033] X clean =ASEEG ;

[0034] Step 23: segment the EEG signal for subsequent feature extraction and analysis.

[0035] Step 3: Input the preprocessed EEG signal into the EEG attention classification model based on multi-scale feature extraction, train the model, and obtain a trained classification model. Figure 3 The figure shows the processing process of EEG signals by the EEG attention classification model.

[0036] The EEG attention classification model includes a multi-scale separable convolutional network, a Squeeze-and-Excitation module, a lightweight Transformer and a MobileNet.

[0037] Figure 4 The processing process of EEG signals by multi-scale separable convolutional network is shown. The multi-scale separable convolutional network (one-dimensional depth-separable convolutional layers of different scales (1x3 and 1x5)) performs convolution operation on the input EEG signal. The small-scale convolution kernel (1x3) is used to capture the short-term features of the EEG signal, and the large-scale convolution kernel (1x5) is used to capture the long-term features of the EEG signal. An independent convolution operation is performed on each input channel to extract low-level feature maps. The size of the small-scale convolution kernel and the large-scale convolution kernel can be adjusted by technicians in this field according to the actual application. The 1x3 small-scale convolution kernel and the 1x5 large-scale convolution kernel here are just an example. Then batch normalization is applied to normalize the output to ensure that the features extracted from different channels have similar scales. The combination of one-dimensional convolutional layer and batch normalization enables the proposed network to explain multi-channel dynamics and capture multivariate interactions. The normalized output is then ReLU activated. The output after ReLU activation is combined from all channels and a 1x1 point-by-point convolution operation is performed to fuse the information of different channels. Batch normalization and ReLU activation are also performed afterwards. The output feature maps of convolution kernels of different scales are spliced ​​together to form the final multi-scale feature representation to enhance the expressiveness of the model. Multi-scale features are further integrated through parallel convolution operations (3x3 and 5x5 convolution kernels) to ensure that information is not lost in the deep network. The 3x3 convolution kernel and the 5x5 convolution kernel are just an example. Batch normalization and ReLU activation function processing are then performed to ensure the stability of information flow in the deep network. Through multi-scale convolution and residual blocks, subtle and global features in EEG signals can be effectively extracted, laying the foundation for subsequent advanced feature extraction and classification.

[0038] After extracting low-level features, the Squeeze-and-Excitation module is used to dynamically adjust the features between channels: global average pooling is performed on each channel of the extracted low-level feature map H to generate a global feature description for each channel. The calculation formula is as follows:

[0039]

[0040] Among them, S c represents the global feature description of channel c, T represents the total number of time steps, and H c,t is the value of the input feature map at channel c and time step t.

[0041] Use two fully connected layers to map the global feature description vector to a low-dimensional space, and then map it back to a high-dimensional space to generate attention weights. The calculation formula is as follows:

[0042] e=ReLU(W1S c );

[0043] a = Sigmoid(W2e);

[0044] where e∈R C / r is the intermediate feature vector (C is the number of channels, r is the scaling factor), W1∈R C / r×C is the weight matrix of the first fully connected layer, W2∈R C×c / r is the weight matrix of the second fully connected layer, a∈R C is the attention weight.

[0045] Then, the attention weights are multiplied element-wise with the low-level feature maps by channel to generate a re-weighted feature map. The calculation formula is as follows:

[0046] H reweighted =a·H;

[0047] Among them, H reweighted is the re-weighted feature map.

[0048] The time-dependent features in EEG signals are captured by lightweight Transformer: the re-weighted features are linearly transformed and mapped to a specific dimension. Position information is added to each time step to enhance the model's understanding of the sequence order. The calculation formula is as follows:

[0049]

[0050] Among them, PE is the position encoding, t is the position of the time step, i is the dimension index, and d is the feature dimension.

[0051] Use the multi-head attention mechanism to improve the expressiveness of the model, calculate the attention weights between each time step, and capture the dependencies between time steps. The calculation formula is as follows:

[0052]

[0053] Where Q, K, and V are query, key, and value matrices respectively, and d k The dimension of the key.

[0054] The feature vector corresponding to each time step is further processed through a two-layer fully connected network. The calculation formula is as follows:

[0055] FFN(x)=ReLU(xW3+b1)W4+b2;

[0056] Among them, x is the feature vector corresponding to each time step, W3 and W4 are the weight matrices of the fully connected layer, and b1 and b2 are the bias vectors of the fully connected layer.

[0057] Residual connections and layer normalization are then used to enhance training stability.

[0058] Figure 5 The structure diagram of MobileNet is shown. The deep separable convolution structure of MobileNet is used to extract high-level features: each input channel of the feature map that has been determined by the lightweight Transformer to determine the dependencies between time steps is convolved separately. The number of channels does not change, but the features of each channel are convolved in the spatial dimension. Each channel is linearly combined using a 1x1 convolution kernel to generate a new feature map. Each layer of convolution output is then batch normalized and ReLU activated. Multi-layer feature extraction is performed through 4 deep separable convolution blocks, each of which includes deep convolution, point-by-point convolution, batch normalization, and ReLU activation to extract high-level features layer by layer. The output of the last layer of convolution is globally average pooled to convert the feature map into a feature vector, reduce the feature dimension, and enhance the generalization ability of the model.

[0059] The high-level features are mapped to a vector of the number of categories through a fully connected layer, and then the category probability is calculated through the Softmax function to obtain the attention classification result.

[0060] Step 4: Collect the tester's EEG data, use the trained model to detect the tester's EEG signal, and classify the tester's attention state.

[0061] The EEG data of the test subject is collected by EEG acquisition equipment such as an EEG cap. It should be noted that 32 channels must also be used for data acquisition, and the sampling rate is set to 1000 Hz.

[0062] The trained model is used to detect the test subject's EEG signals and classify the test subject's attention state.

[0063] The present invention also provides an embodiment of a computer system. The computer system includes a processor and a memory. The memory is used to store non-temporary computer-readable instructions (e.g., one or more computer program modules). The processor is used to run the non-temporary computer-readable instructions, and when the non-temporary computer-readable instructions are run by the processor, one or more steps in the EEG attention classification method described above can be executed. The memory and the processor can be interconnected through a bus system and / or other forms of connection mechanisms.

[0064] For example, the processor may be a central processing unit (CPU), a graphics processing unit (GPU), or other forms of processing units with data processing capabilities and / or program execution capabilities. For example, the central processing unit (CPU) may be an X86 or ARM architecture, etc. The processor may be a general-purpose processor or a special-purpose processor, and may control other components in the computer to perform desired functions.

[0065] For example, the memory may include any combination of one or more computer program products, and the computer program product may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. Volatile memory may include, for example, random access memory (RAM) and / or cache memory (cache), etc. Non-volatile memory may include, for example, read-only memory (ROM), hard disk, erasable programmable read-only memory (EPROM), compact disc read-only memory (CD-ROM), USB memory, flash memory, etc. One or more computer program modules may be stored on the computer-readable storage medium, and the processor may run one or more computer program modules to implement various functions of the computer.

[0066] The present invention also provides a computer-readable storage medium for storing non-transitory computer-readable instructions. When the non-transitory computer-readable instructions are executed by a computer, one or more steps in the above-mentioned EEG attention classification method can be implemented. When the EEG attention classification model and method provided by the embodiment of the present invention are implemented in the form of software and sold or used as an independent product, they can be stored in a computer-readable storage medium. For relevant instructions on the storage medium, please refer to the corresponding description of the memory in the computer above, which will not be repeated here.

Claims

1. An EEG attention classification model, characterized in that: include: A multi-scale separable convolutional network, which is configured to perform independent convolution operations on the EEG signal of each input channel to extract low-level feature maps; The Squeeze-and-Excitation module is configured to dynamically adjust the low-level feature maps between channels to generate re-weighted feature maps; A lightweight Transformer, which is configured to calculate the attention weights between each time step corresponding to each feature vector in the re-weighted feature map from the time dimension based on the attention mechanism to capture the dependencies between time steps; MobileNet, which is configured to extract high-level feature maps from feature maps that determine dependencies between time steps; The classification module is configured to map the feature vector in the high-level feature map to a vector of the number of categories through a fully connected layer, and then calculate the category probability through the Softmax function to obtain the attention classification result.

2. The EEG attention classification model according to claim 1, characterized in that: The multi-scale separable convolutional network includes a small-scale convolution kernel for capturing the short-term characteristics of the EEG signal and a large-scale convolution kernel for capturing the long-term characteristics of the EEG signal. The results of the convolution of the small-scale convolution kernel and the large-scale convolution kernel are batch normalized respectively, and then the normalized output is ReLU activated. The outputs after ReLU activation are combined from all channels, and a 1x1 point-by-point convolution operation is performed to fuse the information of different channels. After that, the results of the 1x1 point-by-point convolution operation are also batch normalized and ReLU activated to obtain the output feature maps of the small-scale convolution kernel and the large-scale convolution kernel. The output feature maps of the small-scale convolution kernel and the large-scale convolution kernel are spliced ​​together to form a multi-scale feature representation, and then the multi-scale features are further fused through parallel convolution operations. The results of the parallel convolution operations are then processed by batch normalization and ReLU activation functions to obtain low-level feature maps.

3. The EEG attention classification model according to claim 2, characterized in that: The 3x3 convolution kernel and the 5x5 convolution kernel are used to perform parallel convolution operations to fuse multi-scale features.

4. The EEG attention classification model according to claim 1, characterized in that: The Squeeze-and-Excitation module performs global average pooling on each channel of the low-level feature map to generate a global feature description for each channel, uses a fully connected layer to map the global feature description vector to a low-dimensional space, and then maps it back to the high-dimensional space to generate attention weights, and then multiplies the attention weights by the low-level feature map element-by-element by channel to generate a reweighted feature map.

5. The EEG attention classification model according to claim 1, characterized in that: The lightweight Transformer performs a linear transformation on the feature vector of the input re-weighted feature map, maps it to a specific dimension, adds position information to each time step, calculates the attention weights between each time step with added position information, and processes the feature vector corresponding to each time step through a two-layer fully connected network as follows: FFN(x)=ReLU(xW3+b1)W4+b2; Among them, x is the feature vector corresponding to each time step, W3 and W4 are the weight matrices of the fully connected layer, and b1 and b2 are the bias vectors of the fully connected layer.

6. The EEG attention classification model according to claim 5, characterized in that: Improving lightweight Transformer training stability using residual connections and layer normalization.

7. The EEG attention classification model according to claim 1, characterized in that: Through multiple depth-wise separable convolution blocks, convolution operations are performed on each input channel of the feature map whose time steps have been determined by the lightweight Transformer, and the high-level feature map is extracted. The convolution output of the last layer of depth-wise separable convolution blocks is globally average pooled to obtain the feature vector of the high-level feature map.

8. A method for classifying EEG attention, characterized in that: include: Collect the EEG signal of the tester, input the EEG signal into the EEG attention classification model described in any one of claims 1-7, and the EEG attention classification model outputs the attention classification result.

9. A computer system, characterized in that: include: processor; a memory including one or more computer program modules; Wherein, the one or more computer program modules are stored in the memory and configured to be executed by the processor, and the one or more computer program modules include instructions for implementing the EEG attention classification method described in claim 8.

10. A computer-readable storage medium for storing non-transitory computer-readable instructions, characterized in that: When the non-transitory computer-readable instructions are executed by a computer, the EEG attention classification method of claim 8 can be implemented.

Images