Convolutional attention-based adaptive separation network for electroencephalogram artifact removal

Abstract

The invention discloses a convolutional attention-based adaptive separation network for removing electroencephalogram artifacts. The convolutional attention-based adaptive separation network comprises a coding module, a separation module and a decoding module which are connected in sequence, the coding module comprises an input module, three sub-modules A and three down-sampling modules, the three sub-modules A are connected in sequence, and each sub-module A is connected with one down-sampling module; the separation module comprises a sub-module B formed by combining a plurality of time sequence convolution modules, a PRelu activation function, a point convolution and a sigmod activation function; the decoding module comprises three sub-modules C, three up-sampling modules and an output module, the three sub-modules C are connected in sequence, one up-sampling module is connected in front of each sub-module C, and the output of the third sub-module C is connected with the output module. According to the method, the clean electroencephalogram signals can be efficiently extracted from the polluted original electroencephalogram, and the method is of great significance to automatic and real-time preprocessing of the electroencephalogram signals.

Description

Technical Field

[0001] The present invention belongs to the technical field of data processing, and in particular relates to an adaptive separation network based on convolutional attention for removing electroencephalogram artifacts. Background Art

[0002] Electroencephalogram (EEG) is an electrical signal that records the electrophysiological activity of the cerebral cortex through sensors on the scalp. It is the main method for non-invasively collecting brain signals. It is of great significance in the fields of cognitive psychology, brain-computer interface (BCI) and diagnosis of neurological diseases. However, EEG signals are weak and have a high temporal resolution. They are easily disturbed by the noise generated by experimental equipment and artifacts caused by physiological activities, thus generating various artifacts. The sources of these artifacts include measuring instruments and human activities. Among them, artifacts generated by measuring instruments can be avoided by precise recording systems and strict recording procedures. However, the generation of physiological artifacts is very complex and difficult to eliminate. It includes eye movement artifacts (EOG), electromyographic artifacts (EMG), electrocardiographic artifacts (ECG) and external artifacts (motion artifacts, vascular wave artifacts). Studies have shown that physiological artifacts can seriously affect the acquisition of the true characteristics of EEG signals and interfere with the subsequent analysis of EEG signals.

[0003] The biggest challenge in EEG artifact removal is to retain the potential neural activity information as much as possible while identifying and removing artifacts. Traditional artifact removal methods mainly include regression, filtering, blind source separation (BSS), wavelet transform (WT), empirical mode decomposition, and hybrid methods. However, most of these methods rely on previous analysis experience and are difficult to automatically remove EEG artifacts. These methods are only applicable to specific types of artifact removal, and the arrangement and selection of decomposed EEG components and artifact components have not been effectively solved. In addition, traditional methods have high computational complexity and are not suitable for online applications. EEG artifact removal methods require better time and precision efficiency. The use of deep learning methods to achieve effective automatic artifact removal is a hot topic in current research.

[0004] With the rapid development of deep learning, it has been widely used in tasks such as image denoising and speech denoising in recent years. At the same time, inspired by these tasks, it is also reflected in EEG artifact removal. Gao et al. (Gao T, Chen D, Tang Y, et al. EEG Reconstruction With a Dual-Scale CNN-LSTM Model for Deep Artifact Removal [J]. IEEE Journal of Biomedical and Health Informatics, 2022, 27 (3): 1283-1294.) proposed a dual-scale model combining convolution and LSTM, which is conducive to the extraction of spatiotemporal features of EEG signals. Enhance the denoising effect of the model. Sawangjai et al. (Sawangjai P, Trakulruangroj M, Boonnag C, et al. EEGANet: Removal of ocular artifacts from the EEG signal using generative adversarial networks [J]. IEEE Journal of Biomedical and Health Informatics, 2021, 26 (10): 4913-4924) proposed a gan model to remove ocular artifacts and adaptively generate pure EEG signals from noisy signals. Yi et al. (Yi P, Chen K, Ma Z, et al. EEGDnet: fusing non-local and local self-similarity for 1-D EEG signal denoising with 2-D transformer [J]. arXiv preprint arXiv: 2109.04235, 2021) designed a denoising model based on the transformer architecture, slicing the EEG signal as input, reducing the complexity of the model. Chen et al. (Chen J, Pi D, Jiang X, et al. Denosieformer: A Transformer based Approach for Single-Channel EEG Artifact Removal [J]. IEEE Transactions on Instrumentation and Measurement, 2023) designed a variational autoencoder model that combines U-Net and Transformer. Compared with advanced methods in recent years, it has better denoising effect.Zhang et al. (Zhang Z, Yu X, Rong X, et al. A Novel Multimodule Neural Network for EEG Denoising [J]. IEEE Access, 2022, 10: 49528-49541) designed a multi-module denoising model, passing the output of the previous module to the next module as input, and achieved good denoising effect. Yu et al. (Yu J, Li C, Lou K, et al. Embedding decomposition for artifacts removal in EEG signals [J]. Journal of Neural Engineering, 2022, 19 (2): 026052) proposed a method combining encoding, decoding and decomposition, multiplying the features obtained by encoding with the attenuation vector obtained by decomposition, and then decoding, which can generate pure EEG signals and artifacts.

[0005] In addition, in recent years, Transformer has been widely used in natural language processing, computer vision and other fields, and its attention mechanism is a key part of the internal structure design. The self-attention mechanism used by Transformer is a quadratic computational complexity, which brings high computational complexity, high memory usage, and low efficiency in predicting long sequence outputs. The Transformer model is good at capturing content-based global interactions, but it is not good at extracting local features. Therefore, there are many studies on optimizing its structure. Zhou et al. (Zhou H, Zhang S, Peng J, et al. Informer: Beyond efficient transformer for long sequence time-series forecasting [C] / / Proceedings of the AAAI conference on artificial intelligence. 2021, 35 (12): 11106-11115) designed the Informer structure, halved the output of each self-attention layer of the Transformer model to highlight the main attention and reduce the computational complexity of the self-attention mechanism. W. Yu et al. (W. Yu, M. Luo, P. Zhou, C. Si, Y. Zhou, X. Wang, J. Feng, and S. Yan, “Metaformer is actually what you need for vision,” in Proceedings of the IEEE / CVF conference on computer vision and pattern recognition, 2022, pp. 10 819-10 829.) designed the Metaformer model and verified that the multi-head self-attention module of the Transformer model can be replaced by spatial MLP or PoolFormer to achieve better results and lower computational complexity. Gulati et al. (Gulati A, Qin J, Chiu CC, et al. Conformer: Convolution-augmented transformer for speech recognition[J].arXiv preprint arXiv:2005.08100,2020) designed the Conformer model, which comprehensively considered the characteristics of the Transformer model being good at capturing global features and the CNN modeling effectively using local features, and combined the convolutional neural network and Transformer to model the local and global dependencies of sequence information, thereby achieving the best results.Liu et al. (Liu Z, Lin Y, Cao Y, et al. Swin transformer: Hierarchical vision transformer using shifted windows [C] / / Proceedings of the IEEE / CVF international conference on computer vision. 2021: 10012-10022) designed the Swin transformer model based on the ViT model and introduced a sliding window mechanism to enable the model to learn cross-window information. At the same time, through the downsampling layer, the model can save computational complexity and pay attention to global and local information. L. Chen et al. (L. Chen, X. Chu, X. Zhang, and J. Sun, "Simple baselines for image restoration," arXiv preprint arXiv: 2204.04676, 2022) improved the Transformer model structure, reducing computational complexity while ensuring the effect, and using a module combining convolution and channel attention instead of a multi-head self-attention module. The gate control unit (GLU) was introduced to replace the RELU activation and simplify it to improve the model effect.

[0006] It is worth noting that the U-Net structure has been shown to have a good effect on denoising, and combining it with the Transformer can further improve the denoising effect of the model (Z. Zhu, S. Zhang, L. Qiu, H. Wang and G. Luo, "Axis-based Transformer UNet for RGB remote sensing image denoising," in IEEE Signal Processing Letters, doi: 10.1109 / LSP.2024.3418717). Cao et al. (Cao, H. et al. (2023). Swin-Unet: Unet-Like Pure Transformer for Medical Image Segmentation. In: Karlinsky, L., Michaeli, T., Nishino, K. (eds) Computer Vision-ECCV 2022 Workshops. ECCV 2022) designed the Swin-Unet model for image denoising: replacing the convolution in the Unet architecture with Swin-Transformer, which uses sliding windows, relative position encoding and hierarchical structures. Extract global and local features. The model has better image denoising effect than advanced methods. Liu L et al. (Liu L, Guan H, Ma J, et al. A mask free neural network for monaural speech enhancement [J]. arXiv preprint arXiv: 2306.04286, 2023) designed the speech denoising model MFNet and used STDCT to obtain real two-dimensional data. They also replaced the ordinary convolution with the global and local feedforward module (GLFB) module based on the U-Net architecture and reduced the number of model parameters. This module has global and local modeling capabilities similar to transformers. The speech denoising effect is good and the computational complexity is low.

[0007] Through the overview, it is found that most of these denoising methods use the generation method of reconstructing EEG, and there are few methods for separating latent features. Most of the model structures use convolutional neural networks, and only a few methods combine convolution with sequence models such as LSTM and Transformer. Previous studies have shown that the combination of convolution and sequence models can well extract and fuse local and global features. However, most of the existing studies have insufficient ability to extract global and local coarse-grained and fine-grained features interactively, and the computational complexity of the model is too high. Summary of the invention

[0008] The purpose of the present invention is to overcome the shortcomings of the prior art and provide an adaptive separation network based on convolutional attention for EEG artifact removal. The separation module adaptively separates the discriminative features of EEG by constraining the potential features; the disadvantages of the Transformer itself are improved, and a novel convolutional attention module is designed to improve the interactive learning ability of coarse and fine granular features, while reducing the number of parameters and the amount of calculation. The present invention can efficiently extract clean EEG signals from contaminated original EEGs, which is of great significance for the automatic and real-time preprocessing of EEG signals.

[0009] The object of the present invention is achieved through the following technical scheme: an adaptive separation network based on convolutional attention for removing EEG artifacts, comprising a sequentially connected encoding module, a separation module and a decoding module; the input of the network is a single-channel contaminated EEG signal, and the output is a pure EEG signal;

[0010] (1) The encoding module includes an input module, three submodules A and three downsampling modules, the three submodules A are connected in sequence, and each submodule A is connected to a downsampling module;

[0011] The input module uses standard one-dimensional convolution to convert one-dimensional input data into two-dimensional data with two dimensions: sequence and channel, as shown in the following formula:

[0012] X=InputBlock(PE) (1)

[0013] PE represents the contaminated EEG signal, X represents the output matrix after the input module transformation; InputBlock represents the input module;

[0014] The encoding process of submodule A and downsampling module is expressed as:

[0015] E1 = CABS(X) (2)

[0016] E2 = CABS(DownSampling(E1)) (3)

[0017] E3 = CABS(DownSampling(E2)) (4)

[0018] M1 = DownSampling(E3) (5)

[0019] E1, E2, and E3 represent the encoded features output by the three submodules A, respectively. M1 represents the latent features. CABS represents the submodule A. DownSampling represents the downsampling module.

[0020] (2) The separation module includes: a submodule B composed of a plurality of sequential convolution modules connected in sequence, a PRelu activation function, a point convolution module, and a sigmod activation function, and the submodule B, the PRelu activation, the point convolution module, and the sigmod activation function are connected in sequence; the latent feature M1 passes through these modules to obtain the weight of the feature related to the EEG signal, and then the weight is multiplied by the latent feature M1 to obtain a new latent feature M2, thereby realizing the adaptive separation of the latent feature; the specific process is shown in the following formula:

[0021] F=PRelu(TCBS(M1)) (6)

[0022] W=Sigmoid(PointConv(F))) (7)

[0023] M2 = W*M1 (8)

[0024] W represents the weight of the features related to the EEG signal, TCBS represents submodule B, and PointConv represents point convolution;

[0025] (3) The decoding module includes three submodules C, three upsampling modules and an output module. The three submodules C are connected in sequence, each submodule C is connected to an upsampling module, and the output of the third submodule C is connected to the output module; wherein the input of the first submodule C is a combination of the output of the third submodule A and the first upsampling module, the input of the second submodule C is a combination of the output of the second submodule A and the second upsampling module, and the input of the third submodule C is a combination of the output of the first submodule A and the third upsampling module; the decoding process of the submodule C and the upsampling module is shown in the following formula:

[0026] M3 = UpSampling(M2) (9)

[0027] D1= UpSampling(CABS(M3+E3)) (10)

[0028] D2= UpSampling(CABS(D1+E2)) (11)

[0029] D3 = CABS(D2+E1) (12)

[0030] D1, D2, and D3 represent the decoding features in the decoding process, CABS represents the submodule C, and UpSampling represents the upsampling module;

[0031] The output module uses a standard one-dimensional convolution with a convolution kernel size of 3 to convert the two-dimensional data with two dimensions of sequence and channel into one-dimensional reconstructed EEG data to obtain a pure EEG signal, as shown in the following formula:

[0032] CE = OutBlock(D3) (13)

[0033] CE stands for reconstructed EEG and OutBlock stands for output module.

[0034] The beneficial effects of the present invention are as follows: based on the U-net architecture, the present invention adaptively separates EEG discriminative features by constraining potential features through a separation module. In addition, the present invention improves on the disadvantages of the Transformer itself and designs a novel convolutional attention module to improve the interactive learning ability of coarse and fine granular features, while reducing the number of parameters and the amount of calculation. Experimental results show that ASNet achieves good artifact removal effects in both semi-synthetic data sets and real data sets. In addition, ASNet has a lower computational complexity than other methods, indicating that this method can efficiently extract clean EEG signals from contaminated raw EEGs, which is of great significance for the automatic and real-time preprocessing of EEG signals. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] Figure 1 It is a schematic diagram of the structure of the adaptive separation network of the present invention;

[0036] Figure 2 It is a structural schematic diagram of submodule B in the adaptive separation network of the present invention;

[0037] Figure 3 Schematic diagram of the structure of the extended convolution module in the adaptive separation network of the present invention;

[0038] Figure 4 It is a schematic diagram of the structure of submodule A and submodule C of the present invention;

[0039] Figure 5 Schematic diagram of the structure of the channel attention module in submodule A and submodule C of the present invention;

[0040] Figure 6 Schematic diagram of the structure of the spatial attention module in the submodule A and submodule C of the present invention;

[0041] Figure 7 for Figure 4 RRMSE and CC comparison chart of EOG removal results;

[0042] Figure 8 Comparison chart of RRMSE and CC of EOG removal results under different signal-to-noise ratios;

[0043] Fig. 9 Five waveforms with Eog artifacts removed;

[0044] Fig.10RRMSE and CC comparison diagram of EMG removal results;

[0045] Fig.11 Comparison chart of RRMSE and CC of EMG removal results under different signal-to-noise ratios;

[0046] Fig.12 The following are five waveforms after removing EMG artifacts. DETAILED DESCRIPTION

[0047] The present invention proposes an adaptive separation network (ASNet) based on convolutional attention for removing EEG artifacts. The network mainly includes three main functional blocks, namely, an encoding module, a separation module and a decoding module. The encoding module includes an input module, three submodules composed of multiple convolutional attention modules and three downsampling modules, which abstract latent features from the input polluted EEG. The separation module includes a submodule composed of multiple temporal convolution modules, and the temporal convolution module is composed of a module composed of multiple dilated convolutions with gradually enlarged convolution kernel spacing. The weight of the correlation between the input latent feature and the EEG signal feature is obtained from the input latent feature, and then multiplied by the latent feature to realize adaptive separation of EEG discriminative features. The module has a gradually enlarging receptive field, so that the convolution module can capture global features. The decoding module includes an output module, three submodules composed of multiple convolutional attention modules and three upsampling modules, which decode the latent features into reconstructed EEG. Among them, the present invention also improves the disadvantages of Transformer itself, and improves the interactive learning ability of coarse and fine granularity features by designing a novel convolutional attention module, while reducing the amount of parameters and calculation. And it is used as the main part of the encoder and decoder in the U-net architecture, so that the features extracted and fused by the U-net architecture take into account the local dependency and global temporal dependency of EEG information.

[0048] The ASNet of the present invention is an end-to-end network that reconstructs clean EEG signals from original contaminated EEG signals by learning nonlinear transformation. The technical solution of the present invention is further described below in conjunction with the accompanying drawings.

[0049] like Figure 1 As shown, an adaptive separation network based on convolutional attention for removing EEG artifacts of the present invention comprises a sequentially connected encoding module, a separation module and a decoding module; the input of the network is a single-channel contaminated EEG signal, and the output is a pure EEG signal;

[0050] (1) The encoding module includes an input module, three submodules A and three downsampling modules, the three submodules A are connected in sequence, and each submodule A is connected to a downsampling module;

[0051] The input module uses standard one-dimensional convolution to convert one-dimensional input data into two-dimensional data with two dimensions: sequence and channel, as shown in the following formula:

[0052] X=InputBlock(PE) (1)

[0053] PE represents the contaminated EEG signal, X represents the output matrix after the input module transformation; InputBlock represents the input module; the shape of the data variable is: PE∈R 1xS (S represents the sequence length), X∈R CxS (C represents the channel dimension), R represents the real number domain. For input shape R S The one-dimensional EEG data is expanded to get a shape of R 1xS The contaminated EEG is then input into the input module for processing to obtain two-dimensional data X.

[0054] The encoding process of submodule A and downsampling module is expressed as:

[0055] E1 = CABS(X) (2)

[0056] E2 = CABS(DownSampling(E1)) (3)

[0057] E3 = CABS(DownSampling(E2)) (4)

[0058] M1 = DownSampling(E3) (5)

[0059] E1, E2, and E3 represent the encoded features output by the three submodules A, respectively. M1 represents the latent features. CABS represents the submodule A. DownSampling represents the downsampling module. The shape of the data variable is expressed as: X∈R CxS , E1∈R CxS , E2∈

[0060] R 2CxS / 2 , E3∈R 4CxS / 4 , M1∈R 8CxS / 8 As shown in formula (2), the feature matrix X is first passed into the submodule to obtain E1; as shown in formula (3), E1 is input into the downsampling module and then passed into the submodule for encoding to obtain E2; as shown in formula (4), E2 is input into the downsampling module and then passed into the submodule to obtain E3; as shown in formula (5), E3 is passed into the downsampling module to obtain the latent feature M1.

[0061] (2) The separation module includes: a submodule B composed of a plurality of sequential convolution modules connected in sequence, a PRelu activation function, a point convolution module, and a sigmod activation function, and the submodule B, the PRelu activation, the point convolution module, and the sigmod activation function are connected in sequence; the latent feature M1 passes through these modules to obtain the weight of the feature related to the EEG signal, and then the weight is multiplied by the latent feature M1 to obtain a new latent feature M2, thereby realizing the adaptive separation of the latent feature; the specific process is shown in the following formula:

[0062] F=PRelu(TCBS(M1)) (6)

[0063] W=Sigmoid(PointConv(F))) (7)

[0064] M2 = W*M1 (8)

[0065] W represents the weight of the features related to the EEG signal, TCBS represents the submodule B, PRelu represents the PRelu activation function, Sigmoid represents the Sigmoid activation function, and PointConv represents the point convolution. The shape of the data variable: M1∈R 8CxS / 8 , M2∈R 8CxS / 8 , F∈R 8CxS / 8 , W∈R 8CxS / 8 As shown in formula (6), the latent feature M1 is extracted by the submodule and activated by PRelu to obtain F; as shown in formula (7), the feature F is point-convolved and activated by Sigmoid to obtain the weight W; as shown in formula (8), the corresponding points of the weight matrix W and the latent feature matrix M1 are multiplied to obtain M2.

[0066] The submodule B includes multiple sequential convolution modules, which are composed of multiple extended convolution modules (DCB) that gradually expand the convolution kernel spacing. Figure 2 As shown in the figure, the distance between convolution kernels increases exponentially; the value of d used in the figure represents the distance between convolution kernels. The sequence dimension of the latent features designed by the model is 64, the one-dimensional convolution kernel size is 3, and the maximum convolution kernel distance is 2. i In the model, i is set to 4, that is, the maximum convolution kernel spacing is 16, to ensure that the convolution kernel fully extracts the context features and ensures that the extracted features have the long-distance time dependence of the EEG information.

[0067] like Figure 3 As shown in the figure, the dilated convolution module consists of depthwise separable convolution and pointwise convolution. The input is added to the output using residual connection. It is composed of pointwise convolution, layer specification, PRelu function, depthwise dilated convolution, and pointwise convolution. The input data of the dilated convolution module is combined with the output data of the pointwise convolution as the output of the dilated convolution module.

[0068] (3) The decoding module includes three submodules C, three upsampling modules and an output module. The three submodules C are connected in sequence, each submodule C is connected to an upsampling module, and the output of the third submodule C is connected to the output module; wherein the input of the first submodule C is a combination of the output of the third submodule A and the first upsampling module, the input of the second submodule C is a combination of the output of the second submodule A and the second upsampling module, and the input of the third submodule C is a combination of the output of the first submodule A and the third upsampling module; the decoding process of the submodule C and the upsampling module is shown in the following formula:

[0069] M3 = UpSampling(M2) (9)

[0070] D1= UpSampling(CABS(M3+E3)) (10)

[0071] D2= UpSampling(CABS(D1+E2)) (11)

[0072] D3 = CABS(D2+E1) (12)

[0073] D1, D2, and D3 represent the decoding features output during the decoding process, CABS represents the submodule C, and UpSampling represents the upsampling module; the shape of the data variable: M2∈R 8CxS / 8 , M3∈R 4CxS / 4 , D1∈R 2CxS / 2 , D2∈R CxS , D3∈R CxS As shown in formula (9), the latent feature M2 is first input into the upsampling module for processing to obtain M3; as shown in formula (10), M3 and E3 are added and input into the submodule and then into the upsampling module to obtain D1; as shown in formula (11), D1 and E2 are added and input into the submodule and then into the upsampling module to obtain D2; as shown in formula (12), D2 and E1 are added and input into the submodule to obtain D3.

[0074] The output module uses a standard one-dimensional convolution with a convolution kernel size of 3 to convert the two-dimensional data with two dimensions of sequence and channel into one-dimensional reconstructed EEG data to obtain a pure EEG signal, as shown in the following formula:

[0075] CE = OutBlock(D3) (13)

[0076] CE represents reconstructed EEG (Corrected EEG), OutBlock represents the output module. D3 represents the decoding feature in the decoding process, and the shape of the data variable is: D3∈R CxS , CE∈R 1xSD3 is passed to the output module to get CE, and then the shape is R 1xS The reconstructed EEG reduces the dimension to get the shape R S Reconstructed EEG.

[0077] The submodule A and submodule C have the same structure, both of which contain multiple convolutional attention modules connected in sequence. The design of the convolutional attention module is based on the Transformer architecture including a multi-head self-attention module and a feedforward network module. The self-attention mechanism used by the multi-head self-attention module has a quadratic computational complexity, making it unsuitable for mobile devices or resource-constrained devices. Insufficient extraction of local features. In order to solve this problem, Figure 4 As shown, the present invention uses a combination of depthwise separable convolution and attention module to replace the multi-head attention module, including sequentially connected layer specification A, pointwise convolution A, depthwise convolution, gate activation function A, channel attention module, spatial attention module, and pointwise convolution B. Two pointwise convolutions are used to replace the feedforward layer, which are connected after the combination of depthwise separable convolution and attention module, including sequentially connected layer specification B, pointwise convolution C, gate activation function B, and pointwise convolution D. The input of layer specification B and the input of layer specification A are combined with the output of pointwise convolution B, and the input of layer specification B and the output of pointwise convolution D are combined as the output of the submodule; the combination of depthwise separable convolution and attention module is used to replace the multi-head attention module, and two pointwise convolutions are used to replace the feedforward layer, which can ensure the effective extraction of global and local features while achieving model lightweight and reducing the amount of parameters and calculation.

[0078] Depthwise separable convolution consists of pointwise convolution and depthwise convolution. Pointwise convolution is used to extract channel features and aggregate all channels to increase or decrease feature dimensions. Depthwise convolution is used to extract spatial features, group convolutions on the feature dimension, and perform independent depthwise convolution on each channel. Compared with standard convolution, it has the same feature extraction effect, but with low computational complexity and low parameter count.

[0079] like Figure 5 As shown, the channel attention module of the present invention is composed of global average pooling (GAP), point convolution and Sigmoid activation function connected in sequence; global average pooling performs global average pooling on the input feature map to obtain one-dimensional channel information, the one-dimensional channel information is subjected to point convolution and Sigmoid activation function to obtain the channel attention weight, and then the channel attention weight is multiplied with the input feature map to learn the channel relationship between the convolution features as the output of the channel attention module.

[0080] like Figure 6As shown, the spatial attention module is used to learn the spatial relationship between convolutional features, including sequentially connected global maximum pooling (GMP), global average pooling (GAP), splicing layer, point convolution module and Sigmoid activation function; the spatial attention module performs global maximum pooling and global average pooling on the data output by the channel attention module, so that features with more useful information are concentrated, while features without information are filtered, and two feature maps representing different information are generated; the two feature maps are spliced ​​and merged through the splicing layer and then input into the point convolution module for feature fusion, and finally a weight map is generated through the Sigmoid operation, and then the weight map is multiplied with the input feature map of the spatial attention module as the output of the spatial attention module, so that the target area can be enhanced. The combination of the channel attention module and the spatial attention module can learn the channel and spatial relationship of the convolutional features. The integration of global and local features is achieved.

[0081] The technical effects of the present invention are further verified by specific examples below.

[0082] 1. Training and testing: The contaminated EEG signal is used as input and the reconstructed EEG signal is output to train the model in a supervised manner. The mean square error (MSE) between the output and the expected reconstructed clean EEG is used as the loss function. To train the network, the Adam optimization algorithm is used. The description of the model's hyperparameters is shown in Table 1(a), and the setting of the hyperparameters is shown in Table 1(b). After mixing the EEG data and artifacts of the semi-synthetic dataset, the training set, validation set, and test set are divided in a ratio of 8:1:1. After training and testing, the model can be used to remove eye artifacts and myoelectric artifacts in single-channel EEG. The effect of EEG denoising may affect subsequent EEG tasks, such as event-related potential (ERP) analysis and EEG source localization. The model will be tested on semi-synthetic datasets and real datasets.

[0083] Table 1(a)

[0084] Symbol Description N1 The number of convolutional attention modules contained in the submodule (CABS) N2 The number of temporal convolution modules contained in the submodule (TCBS) B Batch size S The length of the input EEG data K The size of the convolution kernel I Initial learning rate

[0085] Table 1(b)

[0086] N1 N2 B S K I 8 8 50 512 3 0.0005

[0087] 2. Dataset

[0088] (1) Semi-synthetic dataset: In EEGdenoiseNet (Zhang H, Zhao M, Wei C, et al. EEGdenoiseNet: a benchmark dataset for deep learning solutions of EEGdenoising [J]. Journal of Neural Engineering, 2021, 18 (5): 056057), a semi-synthetic dataset for EEG artifact removal was designed, which contains 4514 clean EEG segments, 3400 eye artifact segments, and 5598 muscle artifact segments. Each EEG signal segment has 512 samples (2s). The clean EEG signal segment is linearly mixed with the EOG or EMG artifact segment to generate contaminated EEG signals with different signal-to-noise ratios (Formula (3)) according to formula (2). Where x represents clean EEG, n represents artifact, and y represents contaminated EEG. The signal-to-noise ratio ranges from -7 to 2dB. A semi-synthetic dataset of contaminated EEG obtained by mixing clean EEG and artifact is used to train the model.

[0089] y=x+λ·n (14)

[0090]

[0091] (2) Real dataset: BCI Competition 2008 Graz Dataset A (Brunner C, Leeb R, Müller-Putz G, et al. BCI Competition 2008-Graz data set A [J]. Institute for knowledge discovery (laboratory of brain-computer interfaces), Graz University of Technology, 2008, 16: 1-6) covers the EEG data of 9 participants in an instruction-based BCI study, involving four motor imagery tasks: left hand, right hand, feet and tongue, divided into four categories. Each participant took two tests on two days, each with 6 rounds and 48 attempts per round, for a total of 288 times. Before the test, 5 minutes of EEG data were collected to evaluate the EOG effect, including eyes open, eyes closed and eye movement. The data were recorded through 22 10-20 system electrodes, with a sampling rate of 250Hz, a bandpass filter of 0.5-100Hz, and a 50Hz notch filter. Three EOG channels were recorded simultaneously to assist in removing artifacts. The data are checked by experts to remove bad experiments and stored in GDF format with detailed annotations for use in BCI performance evaluation.

[0092] (3) Evaluation indicators: Two quantitative performance indicators, relative root mean square error (RRMSE) and correlation coefficient (CC), were used to evaluate the model.

[0093] 1) RRMSE: RRMSE is a normalized expression that uses the actual value as the scale term in the RMSE calculation.

[0094] In the scenario of removing EEG signal artifacts, RRMSE can measure the deviation between the clean signal and the contaminated signal, and its calculation method is as follows:

[0095]

[0096] Where S = {s k |k=1,2,…,K} represents a clean EEG signal segment with a sequence length of K, is the correction segment corresponding to S. When the RRMSE value decreases, the performance of the artifact removal method improves.

[0097] 2) CC: CC can measure the similarity between clean and contaminated EEG segments, indicating the extent to which the corrected EEG segment contains valid information from the clean EEG segment. The calculation of CC can be defined as:

[0098]

[0099] Among them, Cov and Var represent the calculation of covariance and variance respectively. It is worth noting that the higher the CC, the richer the information contained in the corrected EEG segment.

[0100] 3. Experiments and Results

[0101] (1) Results of semi-synthetic EEG data

[0102] This experiment first tests the denoising performance of the model using a semi-synthetic EEG dataset and compares the results with EEGDNet, Denoiseformer, and DeepSeparator.

[0103] 1) EOG artifact removal: This experiment uses two indicators to quantify the average performance of EOG artifact removal for more than 5000 test samples, namely, relative root mean square error (RRMSE) and correlation coefficient (CC). As shown in Table 2, ASNet obtains the lowest RRMSE, the highest CC and the lowest standard deviation, which indicates that ASNet is superior to other deep learning methods in EOG artifact removal. In this embodiment, the RRMSE of the EOG removal result is as follows: Figure 7 As shown in (a), CC Figure 7 (b). Figure 7From the box plot in , the distribution of ASNet in RRMSE, and correlation coefficient is more concentrated than other models, which indicates that ASNet is more robust in EOG artifact removal. In order to further study the robustness of the model denoising, this experiment also tested the performance under various signal-to-noise ratio levels (-7 to 2dB in formula (3)). The RRMSE and CC of the EOG removal results under different signal-to-noise ratios are shown as follows: Figure 8 (a) and (b). In general, the performance of all four methods deteriorates as the SNR decreases, but ASNet does outperform the other methods at SNR levels with lower RRMSE and higher CC. The results show that ASNet is robust even in the presence of large EOG artifacts in the input. Fig. 9 Five waveforms after removing EOG artifacts are shown in Figure 1, including contaminated EEG signals, clean EEG signals, and reconstructed EEG signals. Comparing the waveforms of clean EEG signals and reconstructed EEG signals, they are basically the same, indicating that the ASNet model can effectively remove EOG artifacts and retain clean EEG signals.

[0104] Table 2 Comparison of eog removal results

[0105] Metric EEGDNet Denoiseformer DeepSeparator ASNet RRMSE 0.384 0.292 0.612 0.252 CC 0.912 0.951 0.769 0.966

[0106] 2) EMG Artifact Removal: The performance of EMG artifact removal is quantified as shown in Table 3. The results show that the ASNet model performs much better than other competing models in all two indicators, relative root mean square error (RRMSE) and correlation coefficient (CC). Fig.10 (a) and (b) are the RRMSE and CC diagrams of the EMG removal results, respectively. Fig.10 The box plots in also show the robust performance of ASNet. Fig.11 In (a) and (b), the RRMSE and CC of EMG removal results under different signal-to-noise ratios are compared. Fig.11 The performance of EMG artifact removal at different signal-to-noise ratio levels was quantitatively evaluated, and the results were slightly different from those of EOG, while ASNet was still the best model. Fig.12 Five waveforms with EMG artifacts removed are shown. Since EMG artifacts are more complex than EOG artifacts, the effect of EMG artifact removal is worse than that of EOG artifact removal. However, the waveforms of clean EEG signals and reconstructed EEG signals can still basically overlap, indicating that the ASNet model can effectively remove EMG artifacts and retain clean EEG signals.

[0107] Table 3 Comparison of emg removal results

[0108] Metric EEGDNet Denoiseformer DeepSeparator ASNet RRMSE 0.624 0.372 0.712 0.328 CC 0.691 0.894 0.634 0.915

[0109] (2) Results of real datasets: This experiment uses the BCI Competition 2008 Graz Dataset A for training and testing of the four-category motor imagery. Ang et al. (Ang, Kai & Chin, Zhang & Zhang, Haihong & Guan, Cuntai. (2008). Filter Bank Common Spatial Pattern (FBCSP) in brain-computer interface. Proceedings of the International Joint Conference on Neural Networks. 2390-2397.10.1109 / IJCNN.2008.4634130) proposed a novel filter bank common spatial pattern (FBCSP) method that can automatically select key spatiotemporal discriminative EEG features. Sherwani (Sherwani, Fahad & Shanta, Shahnoor & Ksm Kader Ibrahim, Babul Salam & Huq, Saif. (2016). Waveletbased feature extraction for classification of motor imagery signals. 360-364.10.1109 / IECBES.2016.7843474) and other EEG classification techniques that combine wavelet transform analysis and neural network (NN). Wavelet decomposition was used to decompose the recorded EEG signal into four levels. The data was filtered, useless channels were deleted, and segmented. Then a certain amount of noise was added, and the ASNet model was used for denoising and non-denoising for comparison. The motor imagery features of EEG were extracted using the common spatial pattern (FBCSP) method and wavelet transform, and then the Sequential model in the classifier model Keras was used for classification. Finally, the test was performed. As shown in Table 4, the effect of motor imagery four-classification was improved after denoising using the ASNet model, which means that the denoising effect of ASNet has improved the subsequent classification tasks of the real data set.

[0110] Table 4 Results of the motor imagery experiment

[0111] Accuracy Kappa Precision Recall Before Denoising 0.630 0.504 0.645 0.631 After Denoising 0.636 0.512 0.658 0.636

[0112] (3) Ablation experiment: This experiment conducted ablation experiments on the gated linear unit (GLU), channel attention and spatial attention (CA&SA) and separator. As shown in Table 5, removing the gated linear unit module has little impact on the ASNet model. The correlation coefficients of the removal effects of EOG artifacts and EMG artifacts decreased by 0.01 and 0.034 respectively. The gated linear unit has a certain improvement on the model. The impact of removing channel attention and spatial attention is greater than that of removing the gated linear unit module. The correlation coefficients of the removal effects of EOG artifacts and EMG artifacts decreased by 0.023 and 0.044 respectively, indicating that channel attention and spatial attention enhance feature extraction. The separator has the greatest impact on the model. After removing the separator module, the model performance is greatly reduced. The correlation coefficients of the removal effects of EOG artifacts and EMG artifacts decreased by 0.039 and 0.072 respectively. This shows the importance of separating latent features.

[0113] Table 5 Ablation experiment results

[0114] GLU CA&SA Separator EOG EMG √ √ 0.956 0.881 √ √ 0.943 0.871 √ √ 0.927 0.843

[0115] (4) Model parameters and inference time: The submodules (CABS) in the encoder and decoder contain multiple convolutional attention modules, and the separator uses a combination module (TCBS) of multiple temporal convolutional modules. The number of modules contained in CABS and TCBS affects the model effect. With the repeated superposition of small modules in the module, the model can be improved, but there is an upper limit. As shown in Table 6, when the number of modules is 8, the model achieves the best performance for EOG and EMG removal.

[0116] Table 6 Parameters experimental results

[0117] Number of modules EOG EMG 8 0.966 0.915 2 0.954 0.901 3 0.958 0.908 16 0.960 0.912

[0118] To measure the quality of EEG denoising models, in addition to the two indicators (relative root mean square error and correlation coefficient), the number of model parameters and inference time are also equally important, especially for portable EEG devices, which not only require effective denoising, but also real-time performance. As shown in Table 7, the ASNet model has lower parameters and less inference time than the Denoiseformer model with similar effects. The improvement of the Transformer model reduces the number of parameters and calculations.

[0119] Table 7 Comparison of parameter quantity and inference time

[0120] Method Parameter(million) Inference time(ms) EEGDNet 0.18 1.97 Denoiseformer 8.06 24.17 DeepSeparator 0.68 2.75 ASNet 5.48 4

[0121] In the present invention, a deep learning model ASNet is proposed, which can extract EEG signals from contaminated EEG data. Removing artifacts mixed in EEG signals is an important step in the subsequent EEG signal analysis. Compared with previous deep learning methods, ASNet can achieve satisfactory performance when processing single-channel EEG signal data containing electrooculographic artifacts or electromyographic artifacts. The powerful denoising effect of ASNet provides great potential for the application of EEG denoising. In addition, the performance of ASNet at different signal-to-noise ratio levels is reliable, which allows ASNet to be used in data in different noise environments. Once ASNet is applied after training, its denoising ability becomes fast and automatic. It can be further combined with artificial intelligence hardware to achieve real-time EEG denoising, which will greatly promote the application of brain-computer interface, ERP analysis and source localization based on EEG.

[0122] Compared with other deep learning models, another advantage of ASNet is its low number of parameters and computation. To address the computational complexity of the Transformer's multi-head self-attention module, the MobileNet module with an added attention mechanism is used to replace the multi-head attention module. The deep separable convolution used by the MobileNet module can significantly reduce the computational complexity while ensuring the feature extraction effect. This is conducive to lightweight modules and reduced parameter and computational complexity, as shown in Table 6.

[0123] Those skilled in the art will appreciate that the embodiments described herein are intended to help readers understand the principles of the present invention, and should be understood that the protection scope of the present invention is not limited to such specific statements and embodiments. Those skilled in the art can make various other specific variations and combinations that do not deviate from the essence of the present invention based on the technical revelations disclosed by the present invention, and these variations and combinations are still within the protection scope of the present invention.

Claims

1. An adaptive separation network based on convolutional attention for EEG artifact removal, characterized in that: It includes a sequentially connected encoding module, a separation module and a decoding module; the input of the network is a single-channel contaminated EEG signal, and the output is a pure EEG signal; (1) The encoding module includes an input module, three submodules A and three downsampling modules, the three submodules A are connected in sequence, and each submodule A is connected to a downsampling module; The input module uses standard one-dimensional convolution to convert one-dimensional input data into two-dimensional data with two dimensions: sequence and channel, as shown in the following formula: X=InputBlock(PE) (1) PE represents the contaminated EEG signal, X represents the output matrix after the input module transformation; InputBlock represents the input module; The encoding process of submodule A and downsampling module is expressed as: E1 = CABS(X) (2) E2 = CABS(DownSampling(E1)) (3) E3 = CABS(DownSampling(E2)) (4) M1 = DownSampling(E3) (5) E1, E2, and E3 represent the encoded features output by the three submodules A, respectively. M1 represents the latent features. CABS represents the submodule A. DownSampling represents the downsampling module. (2) The separation module includes: a submodule B composed of a plurality of sequential convolution modules connected in sequence, a PRelu activation function, a point convolution module, and a sigmod activation function, and the submodule B, the PRelu activation, the point convolution module, and the sigmod activation function are connected in sequence; the latent feature M1 passes through these modules to obtain the weight of the feature related to the EEG signal, and then the weight is multiplied by the latent feature M1 to obtain a new latent feature M2, thereby realizing the adaptive separation of the latent feature; the specific process is shown in the following formula: F=PRelu(TCBS(M1)) (6) W=Sigmoid(PointConv(F))) (7) M2 = W*M1 (8) W represents the weight of the features related to the EEG signal, TCBS represents submodule B, and PointConv represents point convolution; (3) The decoding module includes three submodules C, three upsampling modules and an output module. The three submodules C are connected in sequence, each submodule C is connected to an upsampling module, and the output of the third submodule C is connected to the output module; wherein the input of the first submodule C is a combination of the output of the third submodule A and the first upsampling module, the input of the second submodule C is a combination of the output of the second submodule A and the second upsampling module, and the input of the third submodule C is a combination of the output of the first submodule A and the third upsampling module; the decoding process of the submodule C and the upsampling module is shown in the following formula: M3 = UpSampling(M2) (9) D1= UpSampling(CABS(M3+E3)) (10) D2= UpSampling(CABS(D1+E2)) (11) D3 = CABS(D2+E1) (12) CABS represents submodule C, and UpSampling represents upsampling module; The output module uses a standard one-dimensional convolution with a convolution kernel size of 3 to convert the two-dimensional data with two dimensions of sequence and channel into one-dimensional reconstructed EEG data to obtain a pure EEG signal, as shown in the following formula: CE = OutBlock(D3) (13) CE stands for reconstructed EEG and OutBlock stands for output module.

2. The convolutional attention-based adaptive separation network for EEG artifact removal according to claim 1, characterized in that: The submodule A and submodule C have the same structure and both include multiple convolutional attention modules connected in sequence. The convolutional attention module includes layer specification A, point-by-point convolution A, depth convolution, gate activation function A, channel attention module, spatial attention module, point-by-point convolution B, layer specification B, point-by-point convolution C, gate activation function B, point-by-point convolution D connected in sequence. The input of layer specification B and the input of layer specification A are combined with the output of point-by-point convolution B, and the input of layer specification B and the output of point-by-point convolution D are combined as the output of the submodule.

3. The convolutional attention-based adaptive separation network for EEG artifact removal according to claim 2, characterized in that: The channel attention module consists of global average pooling, point convolution and Sigmoid activation function connected in sequence; global average pooling performs global average pooling on the input feature map to obtain one-dimensional channel information, the one-dimensional channel information is subjected to point convolution and Sigmoid activation function to obtain the channel attention weight, and then the channel attention weight is multiplied by the input feature map as the output of the channel attention module.

4. The convolutional attention-based adaptive separation network for EEG artifact removal according to claim 2, characterized in that: The spatial attention module is used to learn the spatial relationship between convolutional features, including sequentially connected global maximum pooling, global average pooling, splicing layer, point convolution module and Sigmoid activation function; the spatial attention module performs global maximum pooling and global average pooling on the data output by the channel attention module, respectively, to generate two feature maps representing different information; the two feature maps are spliced ​​and merged through the splicing layer and then input into the point convolution module for feature fusion, and finally a weight map is generated through a Sigmoid operation, and then the weight map is multiplied with the input feature map of the spatial attention module as the output of the spatial attention module.

5. The convolutional attention-based adaptive separation network for EEG artifact removal according to claim 1, characterized in that: The submodule B includes multiple temporal convolution modules, which are composed of a plurality of extended convolution modules that gradually expand the distance between convolution kernels, and the distance between convolution kernels increases exponentially; the extended convolution module is composed of point-by-point convolution, layer specification, PRelu function, depth-dilated convolution, and point-by-point convolution in sequence, and the input data of the extended convolution module is combined with the output data of the point-by-point convolution as the output of the extended convolution module.

Images